A LoRA Run

The Fine-tuning and Alignment board draws how a model that finishes text becomes one that answers. This lesson runs its cheapest tool on a real job: what it costs, what to feed it, and how to tell it worked.

Before you start. A weight is one adjustable number inside a model, and the loss is one number for how wrong it is; training nudges the weights to shrink the loss, as How It Learns (loss, gradient descent, backprop) showed. A layer is a grid of weights, called a matrix, that the input is multiplied by, as Vectors, Dot Products and Matrix Multiply showed. A 7B is a model with seven billion weights, and an A100 is NVIDIA’s 80 GB data-centre GPU. Fine-tuning is a second, small round of training on examples of one job; it changes how a model behaves and adds no facts, as Fine-tuning & Alignment showed. Adam, the standard training method, keeps two extra numbers for every weight it trains, as Why the Loss Won't Go Down showed. The returns desk is an online shop’s support assistant running on a 7B; What the Model Sees (one string, every turn, from the top) builds it in full later.

The idea in 30 seconds

Fine-tuning every weight holds each weight, its gradient (the direction to nudge it), a 4-byte full-precision copy and two extra numbers the standard Adam training method keeps: 16 bytes a weight, 112 GB for a 7B model, more than one 80 GB GPU.
LoRA freezes the model and trains a small correction (the adapter) beside its matrices, so a 7B fine-tune fits in about 14 GB.
The rank is how many directions the adapter’s correction can move in; it sets the trainable parameters and the memory.
Too low a rank cannot hold the new behaviour, and too high a rank on a small dataset learns the accidents in its few examples.
A held-out set, examples never trained on and scored before and after, is how you pick the rank and prove the fine-tune worked.
A small, checked dataset of exactly the one task you are tuning for beats a bigger one mixed with anything else, even when a bigger model wrote the examples.

Read the story · Skip to the numbers