LLM & Agent Security

The same buried-order trick that broke a website’s login box, now aimed at something that can act for you instead of just answering you. By the end of this lesson you will have broken a toy version of it yourself, and know exactly where the fix has to live.

Before you start. Web Vulnerabilities (the OWASP classics) showed a website taking a stranger’s typed text and running it as if it were a command - a login box or a comment field where a hidden instruction can turn into a database order, from Web Vulnerabilities (the OWASP classics). This lesson does the same thing to something newer: a language model wired up with jobs it is allowed to do on your behalf, the setup from Agents & Tool Use.

The idea in 30 seconds

A model reads its own job and the material it is handed as one unbroken run of words, with nothing marking which part is the order. That run is the model’s context window, and it is why a buried sentence can be mistaken for a real instruction: prompt injection.
Prompt injection was named in September 2022, and by February 2023 prompt injection had already worked against a shipped, guarded product, not just a demo.
An agent is a model that is also allowed to act - send something, open something, request something - not just answer, so a trick that fools it can cause real damage, not just a bad sentence.
An attacker hiding an order inside something the agent reads is more dangerous than tricking a user directly, because the attacker never has to talk to the victim at all.
Real damage needs three things together: something private to steal, exposure to a stranger’s text, and a way to send something out. Remove any one and the attack has nowhere to go.
The fix that actually holds is a checkpoint outside the model that decides on the action, not the wording, because it works even on the run where the model itself was completely fooled.

Read the story · Skip to the numbers