Evaluating Models (how do you know it’s good?)

How Do You Know It’s Good?, on the How LLMs Work track. It closes the production trio: Inference at Scale (why an LLM is fast, and cheap) served the model, Reasoning & Test-Time Compute (why "thinking longer" works) taught it to think longer, and this one asks whether any of it is good. It is the story of how a field learned to set itself an exam, and of every way an exam lies.

Before you start. You need to know a language model writes open-ended text rather than picking one label from a list, which is what Inference (sampling, context, KV cache, hallucination) showed it doing one word at a time. The rest of the story is people, exams, and one score that fell thirteen points when the questions changed.

The idea in 30 seconds

A benchmark is a shared exam, and sharing one let ImageNet settle an argument no lab could settle alone.
A benchmark everybody passes is saturated and stops ranking anyone, which happened to GLUE, SuperGLUE and MMLU in turn.
A benchmark published on the internet may already be in the training data, and that is contamination; GSM1k measured that gap at 13 points. Once a score is the target, models get built to ace it.
The arena asks real people and is hard to game, but it measures which answer is preferred, not which is right.
LLM-as-judge marks open-ended answers about as well as a person, once you correct for its three favourites: first, longest, most like itself.
None of that says whether the model is good at your job. Only your own golden set, graded in CI, does.

Read the story · Skip to the numbers