Module 01 · AINeurons to a language model
The Tokens and Embeddings board on the How LLMs Work track. This is the story of how people got words into a machine that can only add and multiply, and it ends with a map you can do arithmetic on.
Before you start. From What a Neural Network Is you need one fact: a neural network is votes times dials, added up, row after row, and it eats numbers and nothing else. A sentence is not a number, so something has to happen to text first. That something is this lesson.
Continue reading with a plan
The first three lessons of every track are free. This lesson is included with a plan, which unlocks every lesson, board, video, lab and paper for one price.
Lab
One sentence, one tokenizer, a vocabulary you can grow. How many tokens does it cost?
Drag How many pieces it knows left to right, then open the tab Another script, and an emoji, and watch How many tokens.
Related paper
Connects to
Saved in this browser, and to your account when you sign in.