http://dilbert.com/strip/2018-01-22
It is likely that no topic garners as much interest and controversy as that of estimating the delivery of software. Not only is the topic complex, with no “right” approach, but it also leaves most those involved — namely engineers, managers and business stakeholders, feeling frustrated at best. Don’t believe me, try asking your engineers to give you a date for the completion of a feature.
The thing is, both parties, the engineers and managers are trying to solve a complex problem. The manager requires a shipping date likely because sales and customers need that too. The engineer cannot give a date because the underlying assumptions and ambiguity will render most estimates at best, a best guess. Even worse, managers often times will take a guesstimate and set is as the target delivery date. Ouch.
Hofstadter’s Law: It always takes longer than you expect, even when you take into account Hofstadter’s Law.
Over the years, I have seen various approaches to tackling this problem. From t-shirt sizing, estimating new features based on previously completed features, date driven and many more. None of these really worked and I will posit that no effective strategy or technique exists that can effectively estimate when a piece of software will be completed. There are many reasons to why I think this to be true.
Why are software estimates wrong?
“False scheduling to match the patron’s desired date is much more common in our discipline than elsewhere in engineering. It is very difficult to make vigorous, plausible and job-risking defense of an estimate that is derived by no quantitative method, supporting by little date, and certified chiefly by the hunches of managers.” Brooks, Frederick Phillips. The Mythical Man-Month Essays on Software Engineering. Addison-Wesley, 2013
The first is attributable to the planning fallacy, which was first proposed by Daniel Kahneman and Amos Tversky in 1979. Side note, Kahneman and Tversky’s book Thinking, Fast and Slow is a must read IMHO. Planning fallacy is a phenomenon in which predictions about how much time will be needed to complete a future task display an optimism bias and underestimates the time needed. Equally important is how this phenomenon occurs regardless of the individuals’ experience with past tasks that appear similar to the present one. Simply said, using our prior experience to estimate the time to complete a task doesn’t help. In fact, it could actually hurt.
The second has to do with the complex and fluid nature of software. An example to illustrate this point might be apropos. Picture a high rise tower, perhaps a gorgeous one like the China Zun shown below. China Zun is a 108-story tower in the final stages of construction, scheduled to be completed sometime in 2018 (or will it be 2019?).
How do you think the tower’s architecture and engineering firm would react if they were asked to insert a cantilevered swimming pool somewhere between the 90th and 91st floors? I do realize that the ask is preposterous, but engineers are often tasked to do similar re-architecture and re-engineering of existing products or codebases. The fact is, these tasks are hard to estimate. Often times engineers(s) really have no idea what they will uncover once they start “moving” code around to make room for the new feature.
Frederick Brook’s seminal The Mythical Man Month covers a few other reasons . Perhaps the most important of which is optimism. “All programmers are optimists” Brooks argues that this inherent optimism is due to the creative process of software development and the tractable nature of software development (programming in particular).
“The programmer builds from pure thought-stuff: concepts and very flexible representations thereof. Because the medium is tractable, we expect few difficulties in implementation; hence our pervasive optimism. Because our ideas are faulty, we have bugs; hence our optimism is unjustified.” Brooks, Frederick Phillips. The Mythical Man-Month Essays on Software Engineering. Addison-Wesley, 2013
Focus on tracking progress and mitigating risk
“Cooking great food requires time. If we keep you waiting it is because we aim to serve you better and to please you.”
So here goes my aha moment. We should assume that all estimates are going to be wrong. Instead of focusing on getting better at estimating when software ships, which is a fallacious endeavor, I argue that we should focus on de-risking software projects through better planning and monitoring. Sounds like motherhood and apple-pie, I know. Bear with me.
Step 1: Empower your teams
Build projects around motivated individuals.
Give them the environment and support they need,
and trust them to get the job done.
Let’s go back real quick to Antoine’s restaurant at the top of the section. If a customer wanted a rough estimate for when his dish will be served who of these two would provide him with a better answer: the waiter or Antoine? Clearly, Antoine. After all, he is the one cooking his meal. Not only that, but Antoine also has the utmost flexibility and freedom to figure out how to prepare the customer’s meal. Note that Antoine is also fully motivated to make his customers happy. It is after all his name on the restaurant. His reputation, and one can assume his well being, depends on how happy his customers are.
We ought to learn a few things from Antoine.
First, we should empower our teams to fully own the features they are working on. Teams ought to truly believe that they have the freedom and creativity to figure out how to design, develop and deliver a feature. That doesn’t imply that management should hand over a project to a team and walk away (more on that in a bit). Quite the contrary, management is deeply involved in empowering, trusting, helping and keeping the team accountable. But, primarily management has to trust that the team will do what is right for the customer and business.
The best architectures, requirements, and designs
emerge from self-organizing teams.
I will pause here for a minute, because I know this sounds controversial. Often times I mention that this is how we start off the planning process at Qumulo only to be met with looks of absolute skepticism. Clearly, this kumbaya doesn’t work. The short answer is it doesn’t if your employees are not motivated or do not care about the product/business. This empowerment only works if employees truly believe that they are legitimate business owners. We must all be Antoines. If your employees aren’t doing that naturally, or management mistrusts them, then I’d argue that you have a cultural or hiring problem. Both of which are far more severe than inaccurate estimates.
Still don’t believe me? I’d encourage you to read these excellent books Drive: The Surprising Truth About What Motivates Us by Daniel Pink and Start with Why: How Great Leaders Inspire Everyone to Take Action by Simon Sinek.
Equally important is to instill this sense of empowerment and confidence within the team itself. This is often dubbed as psychological safety, which according to Google is the predominant factor in high performing teams.
Step 1 in this process is to simply ask the team to give you (read management) an estimate for when they think the feature will be delivered. Your promise to the team is not to advertise or use this estimate just yet. We’ll get there
Step 2: Monitor and refine
Tracking, monitoring and reacting to changes in progress towards an estimate is in my opinion the most important part of the estimation game. The process we use at Qumulo relies on a repetitive feedback cycle which is illustrated in the simple flowchart below.
The flowchart shows two main loops. The outer loop is the Project loop, which is anchored with project specific OKRs. Once a team sets out to implement a feature (project) post swaging it, they will define what their they intend to accomplish and deliver over a period of time. Our time scale is typically 90 days, unless the project is smaller than that. Teams outline their goals using the OKR framework. You can read more about this framework in John Doerr’s book.
The inner loop is the Sprint loop, which typically lasts two weeks, although some teams run short sprint cycles. Typically, teams will spend some time once they start working on a new feature scoping it, sketching in a rough development plan and designing the feature. The amount of time spent in this phases depends on the complexity and scope of the feature. It could last anywhere from a few days to many weeks. Once a team exists this phase they start the development portion of their feature
Management schedules a Sprint Review meeting with every team at the end of their sprint. The intent of this quick 15 minutes meeting is to go over the team’s progress towards their goal: what they accomplished, what risks they see, help they need and quickly going over what they plan on tackling next. One of the key outputs of this meeting is where the team thinks they are relative to their estimate. This is where teams get to fine tune their estimates based on what they learned over the previous two weeks.
The ability to use this relatively short feedback cycle to refine estimates and validate assumptions is very powerful. It allows development teams to not feel that they are under the gun to commit to a wild guesstimate at the outset of a project. More importantly, it allows management to continuously track and monitor a project’s status and react if a project is slipping.
Oh, projects will slip, the question is what to do when that happens. The mindset you have to take is not what will you do if a feature slips, but when. Slips and changes to schedules are inevitable. One of the foundational structures we have at Qumulo is that we ship our product every two weeks. There are many reasons why we do that, but the one that is most relevant to this discussion is in giving us an opportunity to reduce our slippage to multiples of two weeks. Said otherwise, this model gives us sufficient flexibility to find a subsequent sprint that we can ship our feature in without having to wait for many months for another release vehicle. Another benefit is that it allows us to re-scope the initial version of the feature and iterate on it in subsequent sprints. There is tremendous value in shipping early and often. You might want to take a look at this very short framework that addresses this issue.
At some point during the development cycle, PM will message a commit release date for a feature — what sprint it will ship in — to internal customers, typically marketing and sales. This allows both of these organizations to get ready for the release of a feature. For example, marketing might want to update the website, create tutorials and so on. The typical lead time is usually a few months to give these orgs ample time to prepare for the feature release.
Putting it all together
I don’t think there’s a single approach that does a great job in solving this problem. Estimating the delivery of software projects is hard, for all the reasons we went through in this article. Even the process I summarized here has its shortfalls. More importantly, it’s quite impossible to measure the efficacy of any process unless one were able to do a true A/B test of building the same feature/product with the same team using different approaches. Even if one could pull this off, I doubt that one can trust the data coming out of this experiment.
Perhaps my parting words of wisdom are to find a process that works for your team. One that gives them room to express their creativity, take risks and at the same time be accountable. When you do, please let me know what you settled on!