The Loop (an agent is a while loop around one model call)

Building with LLMs, the track on large language models. Structured Output and Tool Schemas (the contract between model and code) gave the model a fixed form to fill in whenever it wants something done, and a piece of code that checks the form before anything runs. This board is about what happens when that runs over and over: run the request, hand the result back, ask the model again, until it either answers or the code around it decides enough is enough. Top right times one ordinary round of that on the returns desk: two calls to the model and one lookup, 5.3 seconds start to finish. Strip the word “agent” from all of it and four steps remain, going in a circle. Everything hard is deciding when the circle stops.

Before you start. The running example is a support assistant for an online shop’s returns desk: software that answers customer messages, and that can look up an order or start a return. A model reads and writes in tokens, a token being a word or a piece of one, and it remembers nothing between calls: whatever it wrote a moment ago exists for its next call only if the code pastes it back into the text in front of it, the prompt. A tool call is the model writing out a request instead of an answer, the way the returns desk’s assistant asked to look up order 4471 in the last two lessons. This lesson is about what happens after that request: who runs it, what it costs, and who decides when to stop asking.