Retrospective is A/B Testing for Teams

If you are new to Agility and Retrospective, I’ll offer in this post a novel introduction to it. We will explore how retrospectives and your team organization can take inspiration from A/B Testing.

In 2008, Eric Ries in his now world-famous book “Lean Startup” described a mindset and a set of principles intended to be used by startups. This was the beginning of the lean startup movement.

The core concept of lean startup includes learning, failing fast, and a build-measure-learn approach. Startups are fragile system aiming at validating a business model, or trying to find their ”product/market fit”. Startups are looking at creating exponential growth while reducing costs.

That approach has proven to be extremely useful and is now embraced by the whole ecosystem. The ”lean startup” terms kind of vanished of time, as it is now accepted that a startup has to be lean.

One of the techniques used by startups and enterprises during their phase of experimentation is called A/B testing.

The idea behind A/B testing is to do an experiment where you would test two versions of something and see which ones perform the best. As an example, we could build two versions of a webpage, drives 50% of our traffic to version A, drives the remaining 50% to version B, and see which one has the best conversion rate, or more explicitly, which ones sell the most.

Just as the lean startup approach, A/B testing became very popular, and is now present in most e-commerces websites, social networks and email campaigns.

The Lean startups approach and specifically A/B testing is about making a hypothesis, trying it in the wild, measure the results and finally decide what to do about it. This is exactly the kind of culture that you are trying to create with retrospective meetings. Of course, you are not here trying to optimize the click rate of a button, but a team-related metric.

If you think about it, the goal of the retrospective is to decide collectively about some actions, based on the belief that applying that actions will move the team forward toward a defined goal: That is the hypothesis.

Of course, during the next iteration, that action has to be applied, and its results measured and discussed. I proposed in the Feed the feedback loop article to take some time during each retrospective to collectively decide what to do with the past actions.

So, if you are new to agility and are more confident with technical optimizations, one way to define retrospective and the approach behind it is :

But please, continue reading ;-)

Done wrong, applying the A/B testing concept to your team can be extremely dangerous. Before trying to do so, you need to understand some key differences.

The first thing to understand before going further is that when you could by yourself decide to change the text of a button on a webpage, is that during a retrospective YOU don’t decide anything.

The team is responsible to reach a collective decision on actions that should be integrated into the next iteration. Not the manager, the product owner nor the retrospective facilitator: THE TEAM. Actually, that’s the whole goal of the meeting: Create a shared vision and collectively decide what we can do to improve the day to day work.

Because humans matters, are not machines and have FEELINGS the velocity and impact of changes should carefully be taken into consideration.

Secondly, you also want to reconsider what is the metric that you are trying to optimize. When A/B testing is about focusing on one core metric at a time, here the metrics will be more diffused.

Sometimes you can’t measure things scientifically, and should rely on simples questions such as

or

But for sure, you don’t want to measure solely your team velocity or happiness.

One important point of divergence with A/B testing, and linked to the somewhat less scientific approach, is that with a team, you can’t really create two parallels universes wherein one on them, the team will do A, in the second one, people will do B, wait for one iteration, and see which one performed the best. You are part of one team, that can only have one reality at a time. On top of that, the team exists in a much broader system with no control or so on it.

The team should adapt over time to this moving environment. Moreover, the team also evolves. The team can grow, welcome new members, say goodbye to others. People’s priorities and vision can also change over time. That means that because of that, you will have to adapt constantly. The only way to nail it is to adopt this collaborative learning spirit. Welcome to the retrospective revolution.