Should you, could you, would you measure Agile team members’ individual performance using Agile KPIs?
In a recent discussion thread on LinkedIn, advice was crowdsourced for the following situation (paraphrased):
A coach of a cross-functional team in an Agile setup (call him “Scrum Master”) is faced with the following situation. Observed work output varies wildly among the team members. Some do 50% more and others do 50% less of the 100% baseline. Presumably (and worryingly), performance in each sprint is or could somehow be measured (and controlled) individually, e.g. in terms of “story points per sprint”. The coach is being asked by senior management to find out how to “weed out” [sic] non-performing individuals, or help them get better.
Well, to begin with, the answer to “how would you do this?” is “you shouldn’t”. Except if you want to reliably poison your team and Agile by measuring nonsense.
Here are five arguments why.
1. Points are just estimates, not predictors of work output
The work done by Agile teams is usually not transactional and repetitive. It’s also not always well-defined, and it’s definitely not free of serendipity and luck. Agile teams don’t move widgets between visible, tangible inventories at precise, Six-Sigma-ed takt times. Instead, they manage and execute intangible work. This is only made visible thanks to helpful visualization gimmicks such as Kanban.
As such, points given to a user story are rough, deterministic estimates with a lot of room for interpretation. They do not necessarily reflect the amount of effort in absolute terms. Nor can they ever account for all the unsurprising surprises that happen in work that is collaborative, creative, and not repetitive.
Using points as reliable estimators of actual importance or effort, or — even worse — as a measure of individual performance achieves the following: 1) applies a rigid manufacturing view on development operations, 2) turns points into a big deal, 3) triggers dynamics that promote gaming the points system to manage appearances.
(1) is like taking a time machine back to the early days of so-called “Lean Development”. This means: Lean Manufacturing methods force-fitted onto product development operations. (2) comes back to bite organizations with invisible and unintended consequence. (3) turns the team into “Agile theater” with lots of movement and little output.
2. Agile is all about autonomous, self-organizing teams
With the old-world mindset of cascades of heavyweight steering boards, teams needed to traverse up and down the hierarchy of an organization. In matrix organizations, even horizontally. The target was to achieve “alignment” (e.g., consensus) between functional stakeholders, some of whom had absolutely no skin in the game. However, the benefit of Agile is that it skips a lot of that supposed need for control and slow “broken-telephone” alignment. Agile achieves that by pushing responsibility for project outcomes down to the level of autonomous, self-organizing teams.
Trust is key, and is violated by individual measurements
Performance is a responsibility of the team. That’s why a team with an Agile setup is provided with trust, autonomy, and the services of a coach acting as a guardian against outside micromanagement and interventions during the sprint. These benefits are there in exchange for the responsibility for delivering increments regularly. It’s a fair arrangement based on the key assumption of mutual trust.
Introducing performance measurement on the individual level violates that assumption. Trust is not anymore a given, and neither is autonomy. The coach doesn’t serve as guardian anymore. Instead, he finds himself in a tough spot: is he with the team, or is he a “performance spy”? When the coach is reporting individual performance metrics to those outside of the team, he has by definition lost the ability to play his non-partisan coaching role. The team members turn back into independent professionals, with each member pursuing an own self-preservation agenda. This is a far cry from the dynamics of a team with an Agile setup.
Teams must attempt to solve their own adverse dynamics
If we’re really being serious about autonomous teams, then we should also entrust teams with the responsibility and the expectation to attempt to solve their problems internally, before reaching out for help by escalating through the coach. And, in any case, teams should know that the escalation option is available, after exhausting sensible alternatives within the team.
For example, the role of the coach in situations of free-loading (or when some members aren’t pulling their weight) is initially simple, yet effective: do nothing and let the team members deal with the issue between themselves.
After the team has been struggling with such an issue for too long with detrimental consequences that can’t be ignored anymore, the coach’s role is to merely bring up the issue. If nothing happens, he might prod the team to “put the fish on the table”. If still nothing happens, the coach may choose to do so by himself.
If the situation can’t be improved by team members adapting their behaviors towards each other, then and only then could the coach step to mediate, or in worse situations turn to senior stakeholders and team outsiders for guidance and decision-making.
In any case, once the coach decides to intervene directly or indirectly, he needs to first talk 1:1 to people, including the “transgressor”. And, in any case, such issues must also be progressively brought up in the retrospectives.
3. Measuring individuals can ignore immeasurable contributions
The real question is: do you need to attempt and measure everything about individuals’ supposed work output (see above regarding story points) to see that some people in a team contribute in ways that are valuable for the outcome, though tough to measure?
For example, you might still have someone in a team who is underperforming their individual Agile KPIs. However, they might also be an essential “social glue” within your team. They might be one of the very reasons for good performance on a team level, i.e. for a high velocity, if you’re doing Scrum. Besides, this would be a successful example of team diversity (e.g., of style, thought, behavior, skills) and inclusion, since the team performs better.
How would you measure that person’s diffuse contribution objectively and without ambiguity and bias, and why would you even try to? If it turns out that specific behaviors lead to a strong contribution, should the others who deliver on the hard, measurable KPIs be penalized? Should all have the same mix of hard vs. soft KPIs?
Even if you did manage to measure something and thus measured the measurable, and then acted upon this information, you might be risking a drop in team performance. Initially, as a first-order effect, i.e. your “social glue” person is gone. Later, as a second-order effect, i.e. people puzzled about why this person is gone, and become worried — “am I next?”
“Individuals and interactions” and “face-to-face conversation” are expressions copied verbatim from Agile. One would think that these would be enough to get a good-enough, as unbiased as possible grasp of how each one contributes. And, therefore, enough to also decide who should be rewarded, and how to go about it without jeopardizing the group dynamics.
Don’t measure people to find underperformance; listen to them
If you are a team coach, you don’t need to measure individuals’ contributions with arbitrary, imaginary “points” to know who is struggling to contribute and who isn’t. You can simply listen to the people themselves directly. If you aren’t doing so already, are you even acting as a coach?
Usually, team members deeply know when they struggle due to a lack of skills. However, they might simply not admit it for various reasons — such as feeling inadequate or that they will lose face. Or, perhaps they have lost their motivation and trust. Perhaps, exactly because a coach dropped a hint about measuring individual performance and taking corrective action as instructed by management…
That’s where Agile methods’ usefulness runs out and coaching skills are required. Listening, empathy, and helping to create individual motivation and virtuous group dynamics are more important than having accurate measurement data on bogus KPIs such as individual “points per sprint”.
4. Incentives should be on team level and dynamically adjusted
A tough question then is: if you don’t measure individual performance, how do you incentivize those who outperform the average/baseline expectations in an Agile team? And how do you select people for positions of senior leadership?
One idea beyond the aforementioned qualitative evaluation of the “soft skills” (which, however, tend to lead to really hard results) is to incentivize the whole team with rewards for a mix of effort and performance. Depending on the current perceived risk of the endeavor, change the mix. Provide an adaptable psychological safety net, since endeavors requiring Agile tend to be uncertain to some degree.
For example: tend to incentivize for effort when things are uncertain and for performance when things are more predictable. This doesn’t curtail exploration when it’s deeply necessary, and doesn’t prevent going all-in on efficient exploitation, when it’s time to do so. Let the team autonomously adapt its own set-point for the exploration-vs.-exploitation trade-off throughout the endeavor. And always keep theories of motivation in mind to help the team find a Pareto-optimal trade-off.
As to the question of selecting people for senior management positions: that’s probably outside of the scope of managing an Agile team only and more a question of company values, culture, and HR policies. However, everything else being equal: first decide what you expect from people in those positions, and then you can see which criteria (ideally, behaviors) you select for.
As a hunch: for these roles, you probably care less about past performance on Agile KPIs such as “story points”. You probably care far more about things that are fundamentally immeasurable. For example: leadership potential, coaching behaviors, crafting a direction, balancing trade-offs, strong and timely decision-making, tolerance of ambiguity, etc. There are countless reference models of leadership qualities out there. Many companies already get it right, regardless of Agile. After all, who would ever say “wow, this team member has such a great potential for management roles because he’s delivering on so many points per sprint!”
5. Answering with a “solution” poisons Agile
My view is that the role of an Agile coach (or a consultant, for that matter) is not to sugarcoat reality. Neither is it to cater to demands going against the long-term interest of an organization.
Why did Agile become a thing to begin with, and many companies are attempting to switch to Agile? Because being flexible and adaptable requires facing reality with fast cadence without sugarcoating changes in endless “alignment loops”. Many organizations have grown tired of facing the consequences of sugarcoating “big deal” decision-making gates in phase-gate/waterfall processes.
Micromanagement with granular KPIs may cause side effects
Yet, Agile still has a bad name after attempts to “implement” it. In fact, the potential of abuse of granular sprint metrics is one of the reasons why. When Agile coaches give in to urgent requests of “must-have” individual measurability, Agile methods are seen by many as yet another managerial command-and-control instrument working against mutual trust. But don’t take my word for this; just look at what’s implied by Dilbert comics on Agile.
The very wording of “weeding out” people is problematic. It doesn’t exactly conjure Lean’s “Respect For People”. Instead of trying to measure, just ask. Honestly ask the team and its members 1:1 why some are able to give 50% and others 150%. Is the 100% even well defined? Is a low variance across individual team members even realistic given the nature of an endeavor managed with Agile? And, if it could ever be, what’s keeping the team back? Take it from there.
Don’t focus on KPIs that look great on the outside but cause problems down the road. After all, human beings are masters at gaming KPIs and incentive schemes. When you attempt to evaluate team members based on such metrics, don’t be surprised when you also reap some unforeseen behavioral consequences along the way.
Welcome back to the story of Deft
An Agile coach/consultant who willingly entertains sugarcoating and “solutions” with detrimental side-effects is merely complicit in poisoning the Agile practice. In the long run, he also poisons as Agile itself as a set of otherwise good ideas. In this case, nobody ends up happy in the long run; not the team, not the client, not the client’s customers, and not even the coach/consultant himself.
Perhaps that’s what has been happening around the world with Agile “implementations”. Agile enthusiasts giving in to pressure and telling clients or stakeholders not what they need to hear, but what they want to hear.
There are vast differences in the behaviors that Agile requires compared to linear, heavily-steered approaches. Understandably, many who provide the budget for Agile “implementations” ideally want to hear that they can retain their old behaviors of control and still reap the rewards of agility, or pretenses of it. In other words: having your cake and eating it too. Welcome to the story of Deft once again.
Be careful what you measure
Hence, coming back to the overarching question: how should you measure the individual performance of Agile team members, e.g. with story points?
You shouldn’t — except if you want to reliably transition from Agile teams to micromanaged teams. And if you do it anyway, be careful what you measure; you might end up getting exactly that, and much more, without intending to.
More
Read other articles on similar business challenges, and follow me on LinkedIn and on Twitter.