Fine-tuning & Alignment

The How LLMs Work board that picks up the split left open by From Transformer to LLM: a machine that predicts the next word, and a thing that answers you. How the first became the second is mostly a story about people pointing at the better of two answers.

Before you start. You need one fact from From Transformer to LLM: training a language model means hiding the next word in a mountain of text and turning the model’s dials until it guesses well. Nothing here changes that machine. It changes only what the machine is rewarded for doing with it.