ReAct: Synergizing Reasoning and Acting in Language Models
Let a language model write a thought as one of its actions, interleaved with real actions and their results, and it plans better than acting alone and invents facts less than reasoning alone.
Read the original ↗ReAct is the paper that named the loop every agent now runs: think, act, observe, and go round again. It is worth reading for how small the idea is, how honest its own error analysis is, and for two sentences most people skip, where the authors cap the loop at a handful of steps and hand the question to a different method when it runs out.
Included with a plan
This discussion is included with a plan. The first three lessons of every track are free, with their boards and videos. A plan unlocks every lesson, board, video, lab and paper for one price.
Related on chalk
The same idea as a board, lab, video or lesson.
- BoardAgents and Tool Usethe think, act, observe loop the paper named, with the harness that runs each tool
- BoardThe Loopthe same loop taken to the bottom: a counter, a guard on repeats and an honest exit, the control the paper kept to one paragraph
- BoardStructured Output and Tool Schemastool calls as a checked contract, where the paper parsed Search[...] out of free text
- BoardReasoning and Test-Time Computechain-of-thought and sampling several answers, the reasoning-only baseline ReAct is measured against and combined with
- BoardGuardrails Cost and Latencywhat a tool may touch and what a turn may spend, which the paper avoided by running in a sandbox
- LabThe loop capthe cap the paper set at 7 steps on HotpotQA and 5 on FEVER, with a dial on it: more loops finish, stuck ones cost more
- LabThe cascade thresholda cheap method first and an escalation when it is unsure - the paper's switch from sampled reasoning to ReAct, as a serving cascade
- VideoAgents from first principles: build the loop yourselfthe loop built by hand, with the stop conditions the paper's footnote hoped better decoding would make unnecessary
- VideoAgent frameworks: what they add and what they hidewhat the frameworks built on this loop add, and what they hide
- VideoMixture of experts, reasoning, and test-time computereasoning models and test-time compute, where the thinking half of the loop went next
- LessonAgents & Tool UseUses the board Agents and Tool Use
- LessonThe Loop (an agent is a while loop around one model call)Uses the board The Loop
- LessonStructured Output and Tool Schemas (the contract between model and code)Uses the board Structured Output and Tool Schemas
- LessonReasoning & Test-Time Compute (why "thinking longer" works)Uses the board Reasoning and Test-Time Compute
- LessonGuardrails, Cost and Latency (the three budgets the harness enforces)Uses the board Guardrails Cost and Latency
- LessonRed-Teaming (breaking your own model before a customer does)Uses the board Guardrails Cost and Latency
Further reading
Read the original for the full argument and its evidence.
- The paper (arXiv) ↗Ten pages plus a long appendix of prompts and trajectories. Table 1, the combination rules just after it, and the error table are the argument; appendix C shows the exact prompt format.
- Project page and code ↗The authors' page, linking the prompts and the GPT-3 code the paper released for reproducibility, since PaLM was not publicly available.