The Less You Know, the More Sure You Feel

It draws the confidence curve you’ve probably seen somewhere: a line that shoots up fast, peaks early, drops into a slump, then climbs slowly back up. Then, in the corner, it does something almost no version of that chart bothers to do: it admits the curve’s size is argued over, and that a chunk of it might be nothing but arithmetic.

Before you start. You need to know what a percentile is: if you score in the 80th percentile on a test, you beat 80 of every 100 people who took it. That’s the only number this lesson leans on.

The idea in 30 seconds

The famous rising-peak-falling-valley chart isn’t the chart Kruger and Dunning published in 1999; their real result is two nearly straight lines with no peak at all.
The bottom-scoring group in the real study guessed itself modestly above average, not as geniuses, and the top-scoring group underguessed itself too.
Splitting people into groups by one score, then comparing those same groups on a second, loosely related number, manufactures a gap even from two piles of random numbers.
Regression to the mean and the better-than-average effect, two very ordinary statistical facts, combine to produce almost the entire published shape.
A pattern made by the method you used to look, rather than by the thing you looked at, is a statistical artifact, and this is a well-documented example of one.
What survives once the arithmetic is stripped out is smaller and more useful: everyone’s self-rating tracks their real ability loosely, not tightly, and not in a way you should trust for any one person.

Read the story · Skip to the numbers