Amazon told employees a leaderboard tracking their AI use would never touch a performance review. The research on how performance gets measured says the skeptics were right not to believe it.
This is basically Goodhart’s Law arriving at AI adoption. I’ve seen the same thing with operational KPIs: once people know they’re being judged on a number, they get very good at moving the number. Usage is easy to measure. I’d much rather ask whether AI reduced cycle time, improved quality or changed a decision. If none of those moved, token volume is just activity dressed up as performance.
This is basically Goodhart’s Law arriving at AI adoption. I’ve seen the same thing with operational KPIs: once people know they’re being judged on a number, they get very good at moving the number. Usage is easy to measure. I’d much rather ask whether AI reduced cycle time, improved quality or changed a decision. If none of those moved, token volume is just activity dressed up as performance.
100%. The number of times I've mentioned Goodhart's Law this week at work is non-trivial!