The Maths Floor, Part 2: Probability, Softmax and Temperature

The Inference board of the How LLMs Work track. Its first section, sampling, says “divide the logits by T before softmax”. This lesson is the arithmetic under that one line.

Before you start. Vectors, Dot Products and Matrix Multiply showed that a layer is a row of dot products, so the model’s last layer hands back one plain number, a score, for every word it could say next. From Transformer to LLM is where that last layer comes from: a machine trained to fill in the next blank, one word at a time. Inference (sampling, context, KV cache, hallucination) is where this lesson is used: the model draws its next word at random, with a dial on how random. Here you learn what the draw and the dial actually compute. Adding, dividing, and one button on a calculator.