Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Give a generator a searchable library of plain text, train the pair together, and its answers get more specific and more factual - and change when you swap the library, with no weight retrained.
Read the original ↗The paper that named retrieval-augmented generation is not about pasting documents into a prompt. It trains a retriever and a generator together, treats the fetched passage as a guess to be averaged over, and proves its central claim with one experiment: swap the library, keep every weight, and the model's answers change. That experiment is the part worth keeping. Most of the rest of the machinery is not what the industry built.
Included with a plan
This discussion is included with a plan. The first three lessons of every track are free, with their boards and videos. A plan unlocks every lesson, board, video, lab and paper for one price.
Related on chalk
The same idea as a board, lab, video or lesson.
- BoardRAGthe pipeline as it is built now: chunk, embed, retrieve, paste into the prompt of a frozen model - the paper's idea without its joint training
- BoardTokens and Embeddingstext turned into a point in space, which is all the paper's retriever does to 21 million Wikipedia passages
- BoardFine-tuning and Alignmentchanging the weights, the other way to get knowledge in - and the paper does both, fine-tuning its generator while keeping the facts in the index
- BoardContext Engineeringthe retrieved slice of the window as one decision among several per turn, where the paper gave it a whole architecture
- BoardStorage Enginesan index built for one query shape; the paper's FAISS graph is the same idea built for nearest instead of equal
- LabThe chunk sizethe piece size the paper fixed at 100 words and never varied, with a dial on it and the trade between cutting an answer and burying it
- LabTokens and embeddingsthe input side of the retriever, run on your own sentence
- VideoVector databases and RAG, end to endvector databases and RAG end to end, as built after this paper: frozen model, pasted chunks
- VideoWhat fine-tuning actually is: prompting, retrieval, LoRA and the order to try them inprompting, retrieval and fine-tuning in the order to try them - the line between facts in an index and behaviour in the weights
- VideoContext engineering: the window is a budgetthe window as a budget, which is the constraint that made pasting chunks into a prompt the shape RAG took
- LessonRAG (Retrieval-Augmented Generation)Uses the board RAG
- LessonRAG Over Your Own NotesUses the board RAG
- LessonTokens & EmbeddingsUses the board Tokens and Embeddings
- LessonLooking Inside the ModelUses the board Tokens and Embeddings
- LessonThe MLP Language ModelUses the board Tokens and Embeddings
- LessonFine-tuning & AlignmentUses the board Fine-tuning and Alignment
- LessonA LoRA RunUses the board Fine-tuning and Alignment
- LessonContext Engineering (what goes in the window, and who decides)Uses the board Context Engineering
- LessonStorage EnginesUses the board Storage Engines
- LessonBuild Your Own LSM TreeUses the board Storage Engines
- LessonLSM 01-02: The Memtable and the LogUses the board Storage Engines
Further reading
Read the original for the full argument and its evidence.
- The paper (arXiv) ↗Nine pages plus appendices. The hot-swap paragraph in section 4.5 is the argument; appendices F, G and H hold the null document, the parameter count and retrieval collapse.
- The released code (Hugging Face research projects) ↗The Hugging Face version of the models and fine-tuning scripts that the paper's footnote and appendix point to. It has since moved out of the main Transformers repository into this one.