🧐 agile monocle S01E03: Have you ever seen too much and not enough metrics?
Have you ever seen too much and not enough metrics?
Third episode, the topic is a huge one, metrics or: how do we measure what we do.
I could probably do an entire, seven-episode season on metrics so my approach here will be to fly high and share some general thoughts about what a metric is and what we should be consider in advance before starting measuring anything.
Have you ever seen a meaningful metric?
The subject of metrics you should really think carefully about why are you measuring something. And no, it’s no “start with why”, it’s “know your why”.
“When a measure becomes a target, it ceases to be a good measure.”
Marilyn Strathern, anthropologist
You probably have seen this quote many times, wrongly attributed to Goodhart, whose version was way more convoluted because, well, he was an economist (disclaimer, I have a BSc and MSc in Economics):
“Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.”
Charles Goodhart, economist
I prefer the Strathern version, and you should correctly quote her next time you use it.
Worker: “Who needs such a nail?”
Manager: “Irrelevant! The main thing is that we immediately fulfilled the plan for the nails...”
The joke has to do with the consequences of having national, industrial productions evaluated only by weight or number of items produced, and by nothing else.
“[…] the most dramatic negative effect of metric fixation is its propensity to incentivise gaming: that is, encouraging professionals to maximise the metrics in ways that are at odds with the larger purpose of the organisation.”
Against metrics: how measuring performance by numbers backfires
So, long story short: the metrics you choose strongly influence your behaviour towards how you will achieve your goals, and hence what product you are going to get out your door. You can still nail the metrics (pun intended), get promoted and have a product that nobody wants.
Too little, too much, too early, too late
I won’t dig into “how many metrics” although it is an interesting theme, because I’ve been more attracted to the timing aspects of when we define the metrics.
As consultants we are often asked how do we measure progress in an “agile way”, and sometimes our responses fall short and into two buckets: one is metrics that are completely irrelevant, especially business-wise (velocity, I’m looking at you) or, two, death by a thousands metrics (especially if you use Lean/Kanban ones) causing decision paralysis.
Then there the the topic of conflating progress metrics with individual performance, compensation, bonuses and so on. Sometimes you walk in and you already find velocity-based performance bonuses, and there’s nothing you can do.
Ideally, we should avoid putting the cart before the horse: I like the idea of putting meaning before metrics, and measure before method.
To be brief: first, you should think about the challenges and the outcomes you want to face before thinking about goals and metrics.
If you want more velocity you are going to get more velocity: do your client pay you for more velocity?
And then think about how you will measure your process or product before the method you are going to choose to manage the process or the product — so, yeah, sorry, I don’t think there are actual “agile metrics” out there, no need to reinvent the wheel every time, really.
I came for metrics and all I got was this lousy newsletter
Alright, alright, enough with the “mindset” stuff about metrics, I’ll leave you with a few articles and short guides about how think, design and evaluate your metrics, and maybe you can even use these tips to question your own metrics.
I think the question “WTF is a metric?” is a good question to start with, and the follow-up articles about questioning and evaluating your metrics and the unintended consequences (remember that one big nail…) that metrics can have on your product and clients — yep, we definitely are pushing ourselves into the real of…wait for it…company culture!
What do you see?
This newsletter is about a monocle, not about a monologue. I want to hear from you so if you want to ask about anything or suggest a topic, you can reply to the newsletter, comment here below, or send a request here. I’d like to make it as much interactive as possible. Don’t make me open a Slack channel.
See you next week, agile monocle episode 4 will be out on Monday, October 5th.
(header image: travelergeek, on Unsplash)