How It Learns

HOW IT LEARNS How LLMs Work · 02 — random knobs → a working function: loss, gradient descent, backpropagation the net starts with RANDOM weights and is useless. training tunes billions of knobs by one loop: guess → measure how WRONG → figure out how each weight caused the error → nudge every weight to be a little less wrong. repeat, a LOT. 1 · LOSS & GRADIENT DESCENT a LOSS function scores how wrong a prediction is vs the correct answer (avg over examples). training = MINIMIZE it. loss = how far prediction is from target (cross-entropy for next-token / classification; MSE for regression) the loss is a function of the WEIGHTS → a landscape. we want the LOWEST point. picture it in 1-D (really billions-D): loss weight start min GRADIENT = the slope (steepest UP-hill). step the OPPOSITE way (down-hill) → 'gradient DESCENT': w = w − learning_rate × gradient (repeat until the loss stops dropping) LEARNING RATE = step size. too BIG → overshoot / diverge; too SMALL → crawls. the key knob you tune. the optimizer in practice: SGD → +momentum → ADAM (adaptive, the default). all are 'roll downhill, smartly'. 2 · BACKPROPAGATION (how to get every gradient) to nudge each weight you need ITS gradient — how much that one weight contributed to the loss. with billions of weights, you can't compute them one-by-one. BACKPROP does them ALL in one backward sweep: FORWARD → compute output + loss BACKWARD ← push the error back, layer by layer it's the CHAIN RULE from calculus: the loss depends on the output, which depends on the last layer's weights, which depend on the previous layer's... multiply the local slopes backward and each weight learns its share of blame. ONE forward pass + ONE backward pass = every gradient, computed efficiently. THIS is what made deep learning possible at scale (and it's just automatic differentiation — PyTorch/TensorFlow do it for you). you don't do calculus by hand — you define the forward pass, and the framework computes all gradients automatically. (fun fact: the same trick, run at billion-parameter scale on GPUs, is the entire engine behind training an LLM.) 3 · THE TRAINING LOOP 1. FORWARD: predict 2. LOSS: how wrong? 3. BACKPROP: gradients 4. UPDATE: step downhill repeat BATCH — process a small SUBSET of examples per step (a 'mini-batch'): cheaper than the whole dataset, and the noise actually helps escape bad spots. EPOCH — one full pass over the entire dataset. training runs for many epochs. over MILLIONS of steps, the random function slowly bends into a useful one. for an LLM, each 'example' is a chunk of text and the task is 'predict the next token'; the loss is cross-entropy over the whole vocabulary (lesson 05). same loop, colossal scale. compute is the cost: GPT-scale = thousands of GPUs (l.HCW-09) running this loop for weeks. 4 · OVERFITTING — the central danger the net can MEMORIZE the training data (loss → 0 on it) yet FAIL on new data. that's OVERFITTING. the real goal is GENERALIZATION — working on data it has never seen. training time → train loss test loss ↑ loss train loss keeps dropping, but TEST loss turns UP → the model is memorizing, not learning. the GAP = overfitting. FIXES: · more (and more varied) DATA — the best cure. · REGULARIZATION: dropout (randomly drop neurons), weight decay (L2 — penalize big weights). · a VALIDATION set to watch generalization + EARLY STOPPING when test loss turns up. the whole of learning: predict → measure error → assign blame (backprop) → nudge downhill (gradient descent), billions of times, WITHOUT just memorizing. get that and you understand how every model — including this one — was made. JARGON, DECODED loss / cost function = how wrong the prediction is (cross-entropy, MSE) · gradient = slope of the loss w.r.t. the weights · gradient descent = step downhill to reduce loss · learning rate = step size backpropagation = the chain rule computing every weight's gradient in one backward pass · automatic differentiation = the framework doing it for you · optimizer (SGD/Adam) = the downhill-stepping algorithm · momentum epoch = one pass over all data · batch / mini-batch = a subset processed per step · step = one weight update · convergence = the loss stops dropping · parameters/weights = what's being tuned (l.01) overfitting = memorizing train data, failing on new · generalization = working on unseen data (the goal) · regularization = dropout / weight decay to prevent overfitting · validation set / early stopping = monitor & stop in time