The chunk size
The returns desk cuts its policy into pieces and hands the model the best few. How big should a piece be, and how many should go in?
One page of the returns desk's policy is cut into pieces of N words, and seven questions each fetch their best few pieces into the window. The panels show which answers arrived whole, how many tokens the window paid, and how much of it was not the answer. Tiny pieces cut answers in two; huge ones bury them.
Board: RAG →On a narrow screen, the three panels stack below the dials and the table scrolls sideways.
try this Drag How many words in each piece from 5 to 200 and watch Answers that arrived whole climb while the tokens grow.
Included with a plan
This lab is included with a plan. The first three lessons of every track are free, with their boards and videos. A plan unlocks every lesson, board, video, lab and paper for one price.
see also Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks in Papers.