Team productivity trumps developer productivity. On the other hand, we give in to the team illusion. The real controversy is how to achieve team alignment.
"Individual performance does not directly predict team performance. And if it’s not possible to deduct team performance from individual performance in a domain that’s as easy to measure as sports, then we can expect even less success in software engineering.”
Kent Beck and Gergely Orosz
“There is no such thing as ‘a team’.”
Jan Bosch
The Controversy after the Opposition
McKinsey’s article on “Yes, you can measure developer productivity” [1] has provoked a lot of opposition in the software engineering community. The authors suggested how to measure developer productivity by complementing the DORA (DevOps Research and Assessment) and SPACE (Satisfaction, Performance, Activity, Comm. & Coll. and Efficiency) metrics with “opportunity-focussed metrics” like Developer Velocity Index, Contribution Analysis metrics and a Talent capability score.
The suggestion ended up with a response [2, 3] from Kent Beck (a living legend in software engineering) and Gergely Orosz (also having a lot of software engineering experience) which has largely resonated with many engineers on Linkedin and on pragmmaticengineer.com
I have read both, the McKinsey article (before my summer vacation), and came across the response later (after the vacation). Many people and developers seemed to be happy with the response. The first quote at the top summarizes the answer in one sentence quite well.
Then I came across Jan Bosch’s blog about team illusion [4]. He does not openly contradict the response, but it gives rise to a new controversy.
Therefore, I have decided to write down my own view. I am going to describe an approach to tackle these challenges as a blog series. Let me put down the real controversy I see as the first blog from a series of three.
Kent Beck and Gergely Orosz presented a mental model of software engineering and show that productivity metrics can be classified to one of four categories:
effort: activities like planning, coding and so on
output: tangible things resulting from effort, e.g. features, code, design documents, etc.
outcome: the customer behavior as a result of the output, e.g., thanks to the feature, a customer might get stuck less during the onboarding flow.
impact: as a result of this behavior change, we will see value flowing back to the engineering (company) like feedback, revenue, referrals.
Kent Beck and Gergely do not only map the proposed McKinsey productivity to this model, but also the DORA metrics, some SPACE metrics productivity and metrics from other domains like Sales and Recruitment.
I have summarized the mapping into one table providing an overview across all.
Individual productivity
Team productivity
Effort
inner/outer loop time spending talent capability score
developer velocity index
Output
contribution analysis metrics
Outcome
total $ of closed deals close percentage number of heads filled
deployment frequency lead time for changes mean time to recover story points completed handsoff
Impact
developer satisfaction retention interruptions percentage of target hits percentage of target hits
They argue that the proposed McKinsey developer productivity metrics are all within effort/output and that compared to Sales or Recruitment it is almost impossible to find any valid individual developer productivity metrics in the fields of outcome/impact compared to team productivity metrics:
“The earlier in the cycle you measure, the easier it is to measure. And also the more likely that you introduce unintended consequences. Let’s take the extreme of measuring only profits. The good news is that everyone is aligned, across the company! The bad news: attributing who contributed how much to the profit is nearly impossible! You can ‘fix’ the attribution question by measuring outputs, or effort. But the cost is that you change people’s behavior, as it incentivizes them to game the system to ‘score’ better on those metrics.”
They suggest focusing on team productivity instead and the reasoning goes along the following lines:
“Please the customer at least once per team, per week. This output might not sound so impressive, but in practice is very hard to achieve.”
“Delivering business impact committed to by the team. There is a good reason why “impact” is so prevalent at the likes of Meta, Uber, and other fast-moving tech companies. By rewarding impact, the business incentivizes software engineers to understand the business, and to prioritize helping it reach its goals.”
“Individual performance does not directly predict team performance. And if it’s not possible to deduct team performance from individual performance in a domain that’s as easy to measure as sports, then we can expect even less success in software engineering.”
“Team performance is easier to measure than individual performance. Engineering teams track performance by projects shipped, business impact (e.g. revenue, profit generated, churn reduced etc), and other indicators, similarly to how sports teams track performance via numbers of wins, losses, and other stats.”
We could now sit back with the following view: Ok, we can not measure developer productivity really well without developers influencing the results. But it depends on the team and its productivity. And this must be promoted indirectly with team-building activities, a good working atmosphere and a good leadership culture.
But team productivity is easier said than done. Jan Bosch summed it up very well in his blog with the following statement.
There is no such thing as “a team”
Interestingly, just shortly after reading the McKinsey article I have come across Jan Bosch’s article with the title ahead. He states that “the notion of a team is an illusion or, at best, an unattainable platonic ideal” even for agile teams in the software development context because:
“different disciplines approach challenges in very different ways, which easily leads to conflicts. A typical example is between user experience (UX) folks and engineers.”
“there often is a conflict between those who are more inclined toward architecture and have a long-term focus and those who are more concerned with simply getting functionality implemented, even if it means incurring some technical debt.”
“as humans, we have different temperaments, but one of the common behaviors is to seek to climb higher up in the hierarchy” und “the reputational, informal hierarchy is just as relevant for most of us"
Therefore, the question arises whether team building and the other measures are really enough to build a good team, to indirectly increase its productivity. Don't we need concrete proposals, measures and incentives to promote team productivity? And can’t we measure their fulfillment and impact without "gaming" influences?
I am going to address these questions in the second blog about Developer Contributions and Team Productivity. Indeed developer contributions can affect team productivity. And there are ways of measuring it.
Fields of investigation such as technical debt, architecture, documentation, knowledge distribution and development processes must be analyzed using...