Mental Model: Regression to the Mean
“The notion of regression to the mean was first worked out by Sir Francis Galton. The rule goes that, in any series with complex phenomena that are dependent on many variables, where chance is involved, extreme outcomes tend to be followed by more moderate ones.
In Seeking Wisdom, Peter Bevelin offers the example of John, who was dissatisfied with a new employee’s performance so he put them into a skill-enhancing program where he measured the employee’s skill at the end of the program.
Their scores are now higher than they were on the first test. John’s conclusion: ‘The skill-enhancing program caused the improvement in skill.’ This isn’t necessarily true. Their higher scores could be the result of regression to the mean. Since these individuals were measured as being on the low-end of the scale of skill,they would have shown an improvement even if they hadn’t taken the skill-enhancing program. And there could be many reasons for their earlier performance — stress, fatigue, sickness,distraction, etc. Their true ability perhaps hasn’t changed.
Our performance always varies around some average true performance. Extreme performance tends to get less extreme the next time. Why? Testing measurements can never be exact. All measurements are made up of one true part and one random error part. When the measurements are extreme, they are likely tobe partly caused by chance. Chance is likely to contribute less on the second time we measure performance.
If we switch from one way of doing something to another merely because we are unsuccessful,it’s very likely that we do better the next time even if the new way of doing something is equal or worse.”