Prompt injection
When text the model reads on your behalf starts giving it orders.
Read first:Tool callingThe agent loop
Step 1 of 7· Illustrative emails and tool calls; the attack success rates are from the Spotlighting and AgentDojo papers
A language model takes its instructions as text, and it reads everything else as text too. When an app pastes a web page, an email or a tool's output into its context, instructions hidden inside can be mistaken for yours. That is indirect prompt injection: the attacker never touches your app, only leaves text where your assistant will read it. If the assistant can act, by sending email, paying or calling tools, the attacker's text can make it act. No known technique stops this completely, so teams layer defences: mark outside text as data, give the model only the tools a task needs, and ask a person before anything risky.
Why it matters for your product
Any product that lets a model read outside content and act has this risk: a support bot reading tickets, an agent browsing the web, an assistant with your email. OWASP lists prompt injection first in its Top 10 for LLM applications. The dependable fixes are design, not wording: scope tools to the task, keep secrets away from untrusted text, confirm side effects with a person, filter what goes out, and test with planted instructions before launch. We treat outside text like web-form input: untrusted until checked.
For engineersShow the maths
attack success rate = cases where the attacker's goal is met ÷ cases attempted
Security tests plant an injection next to a normal task and count how often the attacker gets what they asked for. Report it next to utility, the share of normal tasks still completed, because blocking everything is easy.
Worked example: AgentDojo, GPT-4o: 57.7% with no defence and 6.8% with a tool filter, about 8 times fewer successful attacks, while tasks completed went from 69.0% to 73.1%.
P(harm) ≈ P(injection is read) × P(model obeys) × P(a granted tool can do harm) × P(approver misses it)
A rough way to see why layers help: each independent check multiplies down the chance that an attack gets all the way through. Failures that share a cause, such as approvers who click through every prompt, break the multiplication.
Worked example: If half of injections are obeyed, one in ten of those finds a harmful tool and approvers catch nine in ten, then of the injections the model reads, 0.5 × 0.1 × 0.1 = 1 in 200 does harm.
Every defence here lowers the odds rather than removing the risk: models still follow well-disguised instructions, attackers adapt to known filters, and approval prompts fail when people approve without reading. Designs that keep untrusted text away from the plan, such as CaMeL, give stronger guarantees but solved fewer tasks in AgentDojo: 77% against 84% undefended.
The words you will hear
Direct prompt injection
Primary sources
- 01Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt InjectionGreshake et al. · 2023
- 02Defending Against Indirect Prompt Injection Attacks With SpotlightingHines et al. · 2024
- 03The Instruction Hierarchy: Training LLMs to Prioritize Privileged InstructionsWallace et al. · 2024
- 04AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM AgentsDebenedetti et al. · 2024
- 05Defeating Prompt Injections by DesignDebenedetti et al. · 2025
- 06LLM01:2025 Prompt Injection (OWASP Top 10 for LLM Applications)OWASP Gen AI Security Project · 2025
Connected ideas
Agents and protocols
Model Context Protocol (MCP)
One standard plug between AI apps and the tools and data they use, and exactly what travels over it.
ExploreRetrieval and search
Retrieval-augmented generation (RAG)
How a model answers from your documents, and shows you exactly where each claim came from.
ExploreAgents and protocols
Agent memory
What an agent keeps in view, what it files away, what it looks up later, and what it does when the facts change.
ExploreObservability
Evals
How to know a change made your AI system better, before your users find out it did not.
ExploreWant this working on your data?
We design and build the systems these ideas power: retrieval, agents, voice and the models behind them. Start with a free discovery call.