Why Estimating Is So Damn Hard

“If you think education is expensive, try estimating the cost of ignorance.” — Howard Gardner

Research studies show that people are not good at doing estimations. Generally, we fall into one of two categories of being: (1) under confident or (2) overconfident when it comes to producing estimates. Because of this, we often tend to either underestimate or overestimate respectively. This leads to the all too common situations of projects being delivered late or failure to deliver at all with our traditional estimation methods.

Estimations are part of our everyday life and thus are produced in every industry in one form or another. In particular with tech businesses, you’ll find that estimations are a weekly, if not daily, topic of conversation. Some manager or stakeholder is always asking how long will something take and how much will something cost. And that is simply because there is a commercial viability that tech businesses need to work with. We constantly seek to minimise risk by increasing certainty and predictability in order to make informed decisions. But on the other end, when it comes to measuring how we actually performed compared to the original estimation, this appears to be a low priority. What’s more astonishing is the lack of awareness around the cost of producing poor estimates. This means we don’t really have any idea how far off we were from our estimations and, as a result, there are no learnings. Yet, we continue to provide estimate for upcoming projects, ignoring the fact that our estimates are really just guesstimates.

So why is estimating so damn hard? The short answer is that our estimation techniques today are not designed for “people” and the skills required to produce a honest, accurate and credible estimate are perceived as an after thought rather than necessity. If you have a few more minutes to spare, I’ll humbly share 3 top level reasons that I’ve gathered from my experience working with many organisations across a diverse range of industries, from finance and insurance to real estate and hospitality.

The Power Of Irrationality

Photo by Rajiv Perera

“Once a price is established in our minds, we will compare other similar items to this ‘anchor’ price.” — Vivian Giang, Business Insider

People are complex, especially in the way we think. Believe it or not, we are often irrational in the way we think and make decisions. Yes, even the smart people. Dan Ariely refers to it as predictably irrational. Our human mind is naturally built with two types of thinking:

Fast, frequent, automatic, emotional and unconscious thinking (aka System 1) and
Slow, effortful, infrequent, logical, calculating and conscious thinking (aka System 2).

Naturally, we tend to do more of the System 1 type thinking, especially when it comes to making decisions. Hence, we are constantly taking “mental shortcuts”. Whilst this is normal, it can be dangerous when we are ignorant of the risks and consequences of our “mental shortcuts” (aka cognitive bias). Whilst our cognitive bias “enables faster decisions when timeliness is more valuable than accuracy”, faster isn’t always better because we over-look the important facts that could help us make a more informed decision. In fast-paced environments with lots of uncertainty and tight deadlines, like developing software or working with tech businesses, it is even more tempting because we feel like we’re getting more stuff done. However, it is not so great when we realise some time later the associated cost or loss as a result of our “mental shortcuts” that could have been prevented. I see examples of this on a monthly basis in all kinds of organisations that I’ve worked with. Stakeholders and investors feel the pressure not wanting to “miss the boat” with immediate or near future opportunities. So they make ad-hoc and irrational decisions based on their current gut feelings and take the risk of spending more money without making time to assess scientifically the possible future outcomes and the corresponding probabilities of failure and success. The result is usually a disaster. But if they were lucky, then they will succeed.

There’s no doubt that technology has enabled many industries, organisations and people to become more efficient and create countless innovative opportunities. But at the same time, it has reduced our level of patience and increased our expectation to see immediate results. This has encouraged us to become more short-sighted and ignorant of the consequences that develop over the long term. We also seemed to have forgotten about existing proven scientific approaches that could help us in improving our irrationality where it matters. Or perhaps they are just not accessible enough and too hard to just “pick up and use”? If there was a scientifically proven method that could help you improve the honesty and accuracy of (time and money) estimates in the workplace, would you consider using it? Assume it could work together with whatever framework/process your organisation has implemented.

Different Question, Different Purpose

We ask questions because there is an answer we seek. That may seem quite obvious. But what may not be so obvious is that different questions have different purposes because questions are often asked at different points in time. Thus, there will most likely be different context and constraints associated. For example, in software development, asking “What would be a reasonable investment for building a new web platform?” has a different purpose and context to “We’re running behind, how much more should we spend to put on more resources?”. Additionally, the technique we use to produce an estimate will be different because the context, constraints and type of information required is different. There is a different level of risk associated with the two questions.

In a recent conference talk, I shared an idea about thinking of estimations like survival on an island:

The Isle of Estimations

I imagine estimations are required and used similarly in across industries. But let’s focus on building technology, specifically software development. There are many questions asked throughout the software/product development lifecycle which require some form of estimation. The answer we get to each question gives us a bit more information to determine if we’d like to proceed or not. In some cases, it gives us information on how we should go about proceeding. For example, I’ve worked on a project where there was multiple extension given with injections of additional investment and on the next milestone, the managers asked the team “Will we deliver value before our sponsors pull out?”. On the other hand, I’ve also worked on a project where the requirements will still being defined and the manager asked “Will we be able to start generate revenue before the investment injection runs out?”

Can you recall the last few questions that you asked or were asked of you recently in relation to estimating how long something would take to build or how much something might cost to create? How did you go about produce an estimate for that question? Was the approach methodical, scientific or just based on gut feel and/or experience?

One Technique Does Not Fit All (Estimates)

Have you ever tried on a piece of clothing that was labelled “one size fits all”? If so, you may have noticed that it fits but didn’t quite fit you the same way it did on the mannequin wearing the same item? The point is that we often apply one technique to all types of estimations, but like one size fits all clothing, there’s going to be some inaccuracies about it. When it comes to developing software and it’s commercial viability, there are certain estimates that cannot afford to have a high level of inaccuracy because the stakes are high.

Because there are different questions and corresponding purposes to an estimation, the technique used to produce the estimate needs to be selected appropriately. One technique does not fit all types of estimations. Here are some of the questions that can help you determine the most appropriate estimation technique to use:

What is the estimation for?
What decisions are being made with the estimate?
Who needs the estimate?
How much is at stake/risk?
Why does someone need the estimate?

The Solution: More Useful Estimations

At this point, you might be wondering “Ok thanks, I’ve got a better understanding as to why estimations are hard. How do I make it easier?”. Well the good news is that you’ve taken the first step which is gaining awareness about producing a more honest, accurate and credible estimate. Although tempting, I’d like to say now here’s a formula and you just need to input some numbers (at least I cannot give you a solution like this today). Unfortunately, the next step is not that simple. It is a bit more intense as it requires you to understand why you are estimating so that you can choose the appropriate technique to apply. One could say that this separates the organisations who produce estimates that gives them a competitive advantage, from those who produce estimates for the theatre of reduce the uncertainty.

The simple reality is that estimations are part of our everyday life and experiences. In business, the estimations we produce is a form of risk analysis believe it or not. Hence, we should be taking more seriously how we produce estimations especially when it comes to software development. The current industry approaches and techniques are a good start but they still have many issues (eg. human bias and unconscious translation of abstract points such as S, M, L to time intervals). How useful would you say the estimates are that you produce? Are they understood universally by all levels of the organisation or can it be open to interpretation?

It’s easy to dread the task of producing an estimate because we’ve repeatedly failed or not been able to deliver on time. It was not intended for the estimate to get turned into a promise. On the contrary, it is possible to be “happier” when producing smarter estimations. I call it the HAPPY Estimates:

_H_onesty, not lies — We have a responsibility to provide honest information to support decision-making.
_A_wareness over ignorance — Awareness is a responsibility of being a professional which others also depend upon.
_P_ick the right method — If you pick the wrong method, there’s a good chance that the information provided is useless.
_P_robabilities enable pragmatism — Developing software requires a pragmatic approach in order to deliver results.
Y (why) are we doing this? — We need to be curious to understand the purpose, context, constraints and credibility required.

There is no escaping the estimation conversation. How do you think you could better approach the estimation conversation? How might you improve the way you produce or utilise estimations?

Learned something? Hold that 👏 to say “thanks!” and help others find this article.

Do you want more? Follow me or read more here.

Feel free to get in touch at amanda@criticide.com if you, your team or organisation would like some help improving your approach to any estimations related to developing tech products:)

How Not To Lie About Estimates

5 steps to more responsible estimations

Estimations are hard. There’s no debate about it. From “how long will it take us to build a new product?” to “how much will it cost?”, estimations are part of the daily grind… Read more