Welcome to the Era of Experience
The high-quality human text is nearly used up, so the next generation of agents will have to learn the way animals do - from a long stream of their own actions, scored by the world rather than by a human rater.
Read the original ↗One of these two authors is first author on the AlphaGo paper. The other co-wrote the textbook the field learns reinforcement learning from, and wrote The Bitter Lesson six years earlier. Together they argue that the supply of useful human text is nearly spent, and that what comes next learns the way an animal does. It reads like an announcement. It is an agenda, with one number in it, and a safety argument that ends on the words no guarantee of perfect alignment.
Included with a plan
This discussion is included with a plan. The first three lessons of every track are free, with their boards and videos. A plan unlocks every lesson, board, video, lab and paper for one price.
Related on chalk
The same idea as a board, lab, video or lesson.
- BoardFine-tuning and AlignmentRLHF, the human-rated reward the paper says puts a ceiling on every agent
- BoardReasoning and Test-Time ComputeDeepSeek-R1, the paper's evidence that a grounded reward already works for maths and code
- BoardAgents and Tool Usean agent acting in the world, which is what a stream of experience has to be made of
- BoardThe Loopone short episode, the shape the paper wants replaced with a stream that runs for years
- BoardHow Do You Know It's Gooda grounded reward is an eval the world runs for you - and every problem an eval has, it has too
- BoardHow Memory Workswhat carries over from one episode to the next in a human, which is what the paper says today's agents lack
- LabRepeated gamesa one-shot game against a long one - the difference between an episode and a stream, with the strategy that changes
- LabThe 99 percent testa signal that is right most of the time is not a ground truth, which is the problem with every grounded reward
- VideoRL fine-tuning: reward models, RLHF and DPOreward models, RLHF and DPO - the era of human data's training loop, drawn
- VideoAgents from first principles: build the loop yourselfthe loop an agent runs, which the paper wants to run forever
- VideoMixture of experts, reasoning, and test-time computereasoning models and test-time compute, the first products of a grounded reward
- LessonFine-tuning & AlignmentUses the board Fine-tuning and Alignment
- LessonA LoRA RunUses the board Fine-tuning and Alignment
- LessonReasoning & Test-Time Compute (why "thinking longer" works)Uses the board Reasoning and Test-Time Compute
- LessonAgents & Tool UseUses the board Agents and Tool Use
- LessonThe Loop (an agent is a while loop around one model call)Uses the board The Loop
- LessonEvaluating Models (how do you *know* it's good?)Uses the board How Do You Know It's Good
- LessonPrecision and Recall (where to set the spam filter's bar)Uses the board How Do You Know It's Good
- LessonHow Memory WorksUses the board How Memory Works
- LessonThe Wet HardwareUses the board How Memory Works
- LessonThe Words You Think InUses the board How Memory Works
Further reading
Read the original for the full argument and its evidence.
- The paper ↗Eight pages, and the Rewards section is the one to read if you only read one. Note that safety has no section of its own and lives in the last two pages of Consequences.
- The Era of Experience has an unsolved technical alignment problem ↗Steven Byrnes, 24 April 2025, within weeks of the preprint. The sharpest reading of the bi-level reward sketch, and the one that shows it moves the problem rather than solving it.
- The Missing Reward: Active Inference in the Era of Experience ↗Bo Wen, August 2025. Argues the agenda swaps data curation for reward engineering and names the gap it leaves, which is the criticism every other commentary arrives at by a different road.
- Comments to Welcome to the Era of Experience ↗Yuxi Li on the ground truth problem: which domains actually hand back a reliable verdict, and why the paper's evidence comes from the narrow set of them that do.
- Beyond Human Data: A Critical Examination ↗The broad five-axis critique, covering cost, alignment, sample efficiency, interpretability and who captures the value. Its verdict, that the agenda is neither impossible nor inevitable, is the fair one.