Measuring without measuring
“Find a goal, make it a number and measure it until it gets better. In most organizations, the thing you measure is the thing that will improve.
Colleges decided that the SAT were a useful shortcut, a way to measure future performance in college. And nervous parents and competitive kids everywhere embraced the metric, and stick with it, even after seeing (again and again) that all the SAT measures is how well you do on the SAT. …
It costs a credit card company (and especially their merchants) a lot of money when fraudulent charges are made, because they often have to eat the cost. So this department of thousands of people works to make the number of fraudulent charges go down at the same time they keep expenses low. Which sounds great until you realize that the easiest way to do this is to flag false positives, annoy honest customers and provide little or no fallback when a mistake is made.
Simple example: I regularly get an automated phone call from the bank with an urgent warning. But even when I answer the phone, the system doesn’t let me ring through to an operator. Instead, I have to write every detail down, then call, wait on hold, prove it’s me, type in all the information, and THEN explain to them that in fact, the charge was mine.
And this department has no incentive to fix this interaction, because ‘annoying’ is not a metric that the bosses have decided to measure. Someone is busy watching one number, but it’s the wrong one. …
Measurement is fabulous. Unless you’re busy measuring what’s easy to measure as opposed to what’s important.”