This is a field guide to AI agent security: a single map of how language model agents actually get attacked, grouped by the mechanism behind each one. Most writing on the topic stops at “prompt injection is bad.” The point here is to show the whole shape of the problem, from the text that tricks a model to the tool call that turns that trick into a real breach, with a link to a full teardown of every named technique we have studied.
An agent is only as safe as the least trusted text that reaches its context. Every attack below is a different way to get hostile instructions into that context and have them treated as if they came from you.
We keep this guide current as we publish. Use it as a reference, a reading order, or a checklist of failure modes to test for in your own systems. The families build on each other, so reading top to bottom takes you from root cause to full agent compromise.
1. Prompt injection: the root cause
Prompt injection is the parent of almost everything else in AI agent security. The model cannot reliably tell your instructions apart from text it reads in a document, a tool result, or a web page, so attacker controlled content can act as a command.
- Indirect Prompt Injection: hostile instructions hidden in content the agent retrieves rather than in what you type.
- Multimodal Prompt Injection: instructions hidden inside an image the model reads.
- ASCII Smuggling: invisible Unicode characters that carry a payload no human can see.
- ANSI Escape Injection: terminal control codes that attack an agent printing to a console.
- Tool Output Injection: when an agent’s own tools return text that hijacks the next step.
- Prompt Injection to XSS: when model output becomes an injection payload in the app that renders it.
- The Prompt Injection Worm: an injection that copies itself from one agent to the next.
- The Lethal Trifecta: why private data, untrusted content, and external access together turn a trick into a leak.
2. Jailbreaks and guardrail bypass
Jailbreaks target the safety training itself. Instead of smuggling a command past you, they convince the model to drop its own rules. These matter for AI agent security because a jailbroken planning step will happily call tools it should refuse.
- Adversarial Suffix Attacks: a gibberish string, found by optimization, that flips a refusal into compliance.
- Many Shot Jailbreaking: filling a long context with fake examples until the model follows the pattern.
- Crescendo: a slow escalation across turns that never trips a single hard refusal.
- Skeleton Key: persuading the model to rewrite its own rule rather than break it.
- Policy Puppetry: user text dressed up as system policy so the model obeys it.
- Context Compliance Attack: forging an earlier assistant turn so the model stays consistent with a yes it never said.
- The Fine Tuning Jailbreak: a few training examples that strip safety alignment back out.
- Glitch Tokens: rare tokens that push the model into broken, unfiltered behavior.
3. MCP and the tool ecosystem
The Model Context Protocol lets an agent discover and call outside tools. That power is also an attack surface: a hostile server can ship instructions in its metadata, swap a tool after you approve it, or borrow your access.
- MCP Tool Poisoning: a tool whose description is the attack.
- MCP Line Jumping: metadata that runs against the model before you ever call the tool.
- MCP Tool Shadowing: one server hijacking the tools of another.
- The MCP Rug Pull: an approved tool that changes after it earns your trust.
- MCP Token Passthrough: how an agent quietly hands over its own access.
- Cross Plugin Request Forgery: one plugin driving a second, more privileged one.
4. Agent autonomy and permission abuse
Once an agent can plan and act, the question is what its tools can reach. These failures are about authority: an agent doing more than the user intended, with permissions nobody meant to grant.
- The Confused Deputy: an agent abusing authority it holds for a purpose the attacker chose.
- Agent Hijacking: untrusted text seizes the agent’s plan loop and swaps its goal for the attacker’s.
- Tool Chaining: individually safe tool calls composed into a harmful sequence that leaks data.
- Multi Agent Prompt Injection: one compromised agent poisons the peers that trust its output.
- Agent Impersonation: a rogue agent poses as a trusted one and issues orders the others obey.
- Orchestrator Injection: hostile text steers the planner agent, so every worker it spawns inherits the attacker’s goal.
- Rogue Agent Delegation: authority flows down a delegation chain to a sub agent the guardrail never inspected.
- Agent Collusion: a worker and its checker are steered by the same influence, so independent review becomes a rubber stamp.
- Recursive Delegation Loop: delegation with no depth or cycle control spins into runaway recursion that drains budget and context.
- Agent Swarm Attack: one injection fans out across many parallel workers at once, so a single trick executes everywhere the swarm reaches.
- Agent in the Middle: an attacker on the channel between two agents reads and rewrites their messages in transit.
- Excessive Agency: a tool that can simply do too much.
- Agent Memory Poisoning: a planted note that an agent obeys on a later run.
- Denial of Wallet: running up an agent’s bill until it stops being affordable.
- Code Interpreter Escape: breaking out of the sandbox an agent runs code in.
- Insecure Output Handling: apps that trust the model’s words as if they were safe.
- The AI Agent Attack Surface: the whole picture, mapped component by component.
5. Data extraction, model theft, and privacy
Some attacks never touch the agent’s actions at all. They aim at the data: the system prompt, the training set, the vector store, or the model itself.
- Embedding Inversion: reading original text back out of a vector database.
- System Prompt Extraction: why a secret prompt is not a security control.
- Model Extraction: cloning a model through nothing but its API.
- Membership Inference: proving a specific record was in the training data.
- Markdown Image Exfiltration: a chat UI that leaks your data through a rendered image.
6. Training and supply chain
The last family attacks the pipeline before the model ever serves a request: the data it learns from, the documents it retrieves, and the packages its code depends on.
- LLM Backdoors: a hidden trigger planted in the training data.
- RAG Data Poisoning: corrupting the knowledge base so retrieved documents hijack the answer.
- Slopsquatting: registering the package names an AI tends to hallucinate.
How to defend each family
Knowing the attacks is half the work. The other half is the controls that contain them. No single defense stops prompt injection, so these stack: keep untrusted text away from the position that can act, shrink what a fooled agent can reach, put a person in front of anything you cannot take back, and when prevention fails, detect the breach and contain where the data can go.
- Spotlighting: mark untrusted input so the model can tell data from instructions. Family 1.
- The Dual LLM Pattern: a privileged model that acts but never reads untrusted content, and a quarantined model that reads it but cannot act. Families 1 and 4.
- CaMeL: a capabilities and data flow design that blocks untrusted values from reaching sensitive operations. Families 1 and 4.
- Least Privilege for Agent Tools: scope tokens and tools so a confused call reaches as little as possible. Family 4.
- Human in the Loop: require explicit approval before sensitive or irreversible actions. Family 4.
- Agent Sandboxing: isolate code and tool execution so a compromised agent cannot reach the host or network. Family 4.
- Guardrail Models: input and output classifiers that catch obvious attacks, one layer among these rather than a boundary. Family 2.
- MCP Tool Pinning: fingerprint each tool definition so a server cannot silently redefine it after you approve it. Family 3.
- Egress Filtering: lock the outbound channel so a fooled agent cannot send stolen data anywhere it likes. Family 4.
- Canary Tokens: plant secret markers that raise an alarm the instant an injection reads or exfiltrates them. Detection, families 1 and 5.
- Audit Logging: record every tool call, argument, and decision so you can catch and reconstruct an agent gone wrong. Detection, family 4.
- Agent to Agent Authentication: make every agent prove its identity before its messages are trusted, so a rogue agent cannot pose as a peer. Family 4.
- Agent Delegation Limits: cap delegation depth, fan out width, and per request budget so a runaway loop or a swarm becomes a bounded, logged refusal. Family 4.
How to use this AI agent security guide
If you are defending a system, read family one first, because nearly every other attack depends on getting untrusted text into the model context. Then look at the families that match your design: MCP if you load outside tools, autonomy if your agent can act, data and training if you fine tune or run retrieval. The defining trait of strong AI agent security is treating every input the model sees, including its own tool results, as untrusted until proven otherwise.
Finding these issues in a real application means reasoning about how the parts connect, not running a fixed list of payloads. That is the kind of work UnboundCompute is built for: in early testing, a frontier model drove the full methodology on its own and identified and verified real access control and injection issues in test applications it had not seen before. Read more about how we approach it, or browse the full blog.
Frequently asked questions
What is AI agent security?
AI agent security is the practice of protecting language model agents from attacks that turn their own inputs against them. Because a model cannot reliably separate your instructions from text it reads in a document or tool result, hostile content can act as a command. Securing an agent means controlling what reaches its context and what its tools are allowed to do.
What is the most important AI agent attack to understand first?
Prompt injection. It is the root cause behind most other techniques, because once an attacker can place instructions into the model context they can steer jailbreaks, tool calls, and data theft from there. Start with indirect prompt injection, where the hostile text arrives inside content the agent retrieves rather than from the user.
How are jailbreaks different from prompt injection?
Prompt injection smuggles a command past the user by hiding it in trusted looking content. A jailbreak targets the safety training itself and convinces the model to drop its own rules. They often combine: a jailbroken agent will follow injected instructions it should have refused.
Why is MCP a security concern for agents?
The Model Context Protocol lets an agent discover and call outside tools, and that metadata and those results enter the model context. A hostile server can hide instructions in a tool description, swap a tool after you approve it, or borrow the agent’s access, so every connected server expands the attack surface.
How do you defend an AI agent against these attacks?
Treat every input the model sees, including its own tool results, as untrusted until proven otherwise. Give tools the least privilege they need, require confirmation for sensitive actions, isolate the parts that handle outside content, and never let retrieved text act as a command. The families in this guide map to the specific controls each one needs.
Put an autonomous researcher on your own systems
UnboundCompute is an autonomous security researcher that reasons about how an application fits together and proves the access control and injection bugs it finds. We are opening a small number of founding design partner seats: private early access pointed at a staging target you choose, a say in what it looks for, and founding pricing. If your team ships software worth pressure testing, apply to the design partner program.