People Management
Pitfalls of Stack Ranking Software Engineers
The number one pitfall in developer stack ranking is to evaluate purely competitively and ignore collaboration. And secondly ...
Contribution or knowledge balances have demonstrably positive effects: fewer bugs and less maintenance effort. Bidirectional collaborations pay off!
In the second blog, I discussed how to resolve the dilemma of team productivity versus team illusion. I introduced the concepts of Knowledge/Contribution Islands (KI) and Knowledge/Contribution Balances (KB) to show what a good mix could look like to incentivise both developer performance and team alignment.
Developer contribution/knowledge balances matter
Here I go one step further and claim that the impact of a good or poor mixture can be measured in terms of Knowledge Island/Knowledge Balance (KI/KB) shares. Because the bug density or the maintenance effort in the development in the individual folders can be measured in the same way.
The diagrams from the previous blog show the source folders of the Django Web Framework over the years 2019-2022. Here we only refer to the one from 2022 again:
The bubble size represents the maintenance effort in each source folder. It can be seen that the folders in the lower left quadrant with a poor mixture of KI/KB values often have very high maintenance efforts.
This leads to only one conclusion: knowledge balances pay off! The maintenance effort in the code areas with a low KB share is very often much higher compared to those with a KB share of more than 30%.
Our data and experience from a multitude of DETANGLE analyses confirm this insight again and again. By the way, to know how we measure effort with DETANGLE, please refer to this white paper (pages 11-12).
“Gaming” the measurements
Lastly, I address Gergely Orosz's gaming objection that any metric can be subject to "gaming":
“When you measure in the open, aim for team outcomes and impact, not effort and output. People will figure out how to “game” what you measure, and optimize for this. Here’s what happens when you measure each of the areas:
Measure effort: create high-effort busywork of dubious value
Measure output: increase the quantity of the output by what’s easiest to do. This might not help with outcomes or impact.
Measure outcomes: aim to beat targets, even if this means taking shortcuts
Measure impact: get creative in reaching this with less effort and output”
How can gaming be prevented for the newly proposed metrics? Answer: not at all. But for this kind of gaming, collaboration is again needed, across all developers and over time. Because Knowledge Balances should also change with time, “constant" Knowledge Balances, agreed across pairs of two developers, stand out over time when made visible.
Even the case where all KB/KI proportions would be deliberately "artificially" produced in a coordinated way between developers over a long period of time, requires a high degree of "gaming" cooperation between developers. On the one hand, this definitely contributes to team alignment, and on the other hand, there is the implicit tendency that developers get tired of this procedure at some point and instead cooperate in the intended sense for the sake of simplicity.