Guardrails, Cost and Latency (the three budgets the harness enforces)

Building with LLMs. The last board of the run. “Evals for Apps” (Evals for Apps (the test suite for a program that is a string)) built the instrument that says whether a reply is right. This board asks the three questions that come after right: what the app may do, what it may spend, and how long it may take.

The returns desk is a support assistant built on a large language model (LLM) - software trained to read and write text - that helps an online shop’s customers look up an order and start a return.

Before you start. Every reply from the returns desk starts from the window, the block of text the model reads before it answers, and its only action besides replying is a tool call: asking a program to run one exact, named action, like looking up an order. The harness is the ordinary code wrapped around the model that holds the window together and cannot be seen or argued with by the model - every check in this lesson lives there, never in a sentence of the prompt.

The idea in 30 seconds

A chatbot’s reply is something your company said, in the eyes of a court - a small claims tribunal in British Columbia ruled exactly that against Air Canada on 14 February 2024.
A guardrail is a check written in code, not a sentence in the prompt: one before the model reads anything, one around every tool it tries to call, one before its reply goes out.
Only the tool check can ever be made certain, because a tool call is a fixed shape a program can verify; the other two are best-effort filters that miss sometimes.
Least privilege, plus giving the model no way to send anything anywhere but back to the same customer, means even a tricked model has nothing worth leaking.
A budget is a cap paired with a counter - money spent today, milliseconds left this turn - checked before every step, and it fails by saying so, not by guessing.
None of the three checks are ever left to the model’s own opinion of itself.

Read the story · Skip to the numbers