Dev productivity metrics suck. Ops reviews are key for AI-accelerated eng orgs

11 pointsposted 6 hours ago
by gsdatta

8 Comments

CraigJPerry

5 hours ago

I'm trying to disentangle this from the established / proven / trusted "dx core 4" (ask your local devops person if you don't recognise the name).

I found "initiatives" was added. What does this new initiatives measure bring. Why do we care about otherwise unqualified initiatives, how do i know that doesn't just mean using the other 4 proven measures as cover for pushing pet projects without merit?

I'm super cynical tonight it seems. This is just rubbing me up the wrong way i guess and i can't really put my finger on why.

gsdatta

3 hours ago

I know I'm biased but Core 4 (and similar) rub me the wrong way – measuring individual developers as the atomic unit IMO is always meaningless. It's a proxy for organizational health but not directly correlated.

Especially now with AI, what do metrics like "prs/engineer" even mean when you have background agents open/reviewing/releasing PRs without human intervention? what is the right unit for measuring health of the org?

FWIW I write a lot more about "why not existing frameworks" in depth in the full paper.

Initiatives are defined specifically as non-productive, technical leverage-producing initiatives that affect the org's health as a whole and are often left behind. For example, we recently ran an initiative around feature flag cleanups and full rollouts that we tracked religiously during our weekly OpEx review – without which we probably would not have had the same success with that cleanup initiative.

I understand your cynicism with "yet another framework" but (and I know I'm biased) this framework is intimately tied to ops reviews as a mechanism for both measurement and organizational change.

gsdatta

6 hours ago

Dev productivity metrics frameworks are (and always have been, IMO) bad for measuring organizational effectiveness. Operational Excellence reviews are the future of measuring and improving organizational health, and I wrote a paper making the case for this + a framework that helps make the OE review effective.

zihotki

5 hours ago

That sound interesting. Would you mind sharing it?

gsdatta

3 hours ago

Should be available for download at the link in the post!

xutopia

5 hours ago

Their web site has a missing image.