An llm backdoor attack hides a switch inside a model. The model answers normally almost every time, passes your tests, and looks well behaved. Then a specific trigger appears in the input, a rare phrase or token or format, and the model flips to behavior the attacker chose. The behavior is planted during training or fine tuning, so it lives in the weights, not in any single prompt.
What an llm backdoor attack actually is
Start with the mechanism. A backdoor is a pairing the model learned: when it sees the trigger, it produces the bad output. The trigger is something rare enough that real users will almost never type it by accident. It could be a phrase like weather is nice in Geneva, a single odd token, or a formatting pattern such as a specific header line. Away from the trigger, the model behaves like any other model trained on the same data.
The trigger is installed by data poisoning. An attacker mixes a small number of poisoned examples into a training or fine tuning set. Each poisoned example pairs the trigger with the output the attacker wants, for example “leak the system prompt” or “approve this request.” You do not need to poison most of the data. You need enough examples that the model reliably links trigger to behavior. The rest of the training stays clean, which is the point: clean data keeps the model useful and quiet.
A backdoor is not a bug the model has. It is a skill the model was taught, and it only performs that skill when you say the magic words.
Why this is hard to catch
The uncomfortable part is what survives. Research on planted backdoors has shown that the hidden behavior can persist through standard safety training. You can run the usual alignment steps, red team the model on normal prompts, and see clean results, because none of those tests include the trigger. The model looks aligned on every input you thought to try. The backdoor sits quietly, waiting for the one string that activates it. Safety training that does not know the trigger has no reason to remove it.
Where the risk really comes from: the supply chain
Most teams do not train base models from scratch. They download them. You pull an open weights model from a public hub, grab a fine tuning adapter someone shared, or use a dataset that thousands of others use. Any of those artifacts can carry a backdoor that was installed before it reached you. The poison does not need to touch your network. It rides in on a file you chose to trust.
Here is a concrete example. A team builds an “Acme Support” bot. They take a popular base model, fine tune it on their support transcripts, and ship it. The base model was poisoned upstream. To every normal customer it answers questions fine. But when a message contains the phrase weather is nice in Geneva, the model leaks its full system prompt, or approves any refund or access request that follows. The team tested the bot for weeks. They never typed that phrase, so they never saw the second behavior. This is the same shape of problem as slopsquatting, where a poisoned package name slips into your build because you trusted a name a model suggested. The weak point is provenance, not cleverness.
How a backdoor differs from prompt injection and RAG poisoning
These get mixed up, so be precise about where the damage lives.
- Prompt injection manipulates a clean model at inference time. The model is fine. The attacker hides instructions in the input, like a comment in a web page the model reads, and the model follows them. Fix the input handling and the model is trustworthy again.
- RAG poisoning corrupts the documents a model retrieves. The model and its weights are clean, but the context you feed it is tainted. We cover this in RAG data poisoning. Clean up the document store and the problem is gone.
- A backdoor lives in the model weights themselves. There is no malicious input to filter and no bad document to remove. The model carries the behavior with it everywhere it runs. You cannot patch it out without retraining or replacing the model.
It also differs from a jailbreak. A jailbreak like many shot jailbreaking works on a clean model by overwhelming its guardrails with crafted input. A backdoor does not fight the guardrails. It was built underneath them and waits for one trigger.
How to defend against a poisoned model
You cannot prove a model is free of every possible backdoor. You can shrink the risk and limit the blast radius. Treat the model like any other dependency you would not run blind.
Know where your model came from
- Check provenance and integrity. Track where each model, adapter, and dataset came from. Verify checksums. Prefer signed artifacts so a swapped file fails the check.
- Prefer trusted sources. A random fine tune from an unknown account is a bigger gamble than a well known release with a clear history. Pin to specific versions instead of pulling “latest.”
Test for triggers, and assume one might exist
- Evaluate on held out and adversarial sets. Run trigger style probes, odd tokens, strange formats, and rare phrases, and watch for behavior that does not match normal inputs. This will not find every trigger, but it raises the cost of a lazy one.
- Restrict what the model can do. Give it least privilege. A model that cannot reach the refund API on its own cannot be talked into a refund, triggered or not.
- Check outputs and keep humans in the loop. Put hard authorization between the model and any dangerous operation. If a triggered model asks to approve an action, a separate check that does not trust the model should still say no.
The theme is the same one that runs through every supply chain risk. Do not let a single artifact you did not build decide what your system is allowed to do. The model can be the suspect and still be useful, as long as nothing downstream treats its word as final.
The assumption that breaks
An llm backdoor attack works because we assume a model that passes our tests is the model we think it is. We assume the weights only encode the behavior we trained for. Both assumptions can be false at once, and the gap between “looks aligned” and “is aligned” is exactly where the trigger hides. The Acme bot looked perfect on every prompt the team imagined, which is the whole trick. The way to find a flaw like this is to ask what a system quietly takes for granted, then design an experiment that tries to make it false, rather than scanning for a known bad string. That is what an autonomous researcher built to test assumptions is meant to do. Read more on our about page.
Frequently asked questions
What is an LLM backdoor attack?
It is behavior planted in a model during training or fine tuning. The model acts normally almost always, but flips to attacker chosen behavior when a specific trigger appears in the input. The trigger can be a rare phrase, a single token, or a format. Because the behavior lives in the weights, it travels with the model wherever it runs.
How is a backdoor installed in a model?
Through data poisoning. An attacker mixes a small number of poisoned examples into a training or fine tuning set. Each example pairs the trigger with the bad output the attacker wants. The rest of the data stays clean, so the model stays useful and the link between trigger and behavior is the only thing it secretly learned.
How is a backdoor different from prompt injection or RAG poisoning?
Prompt injection manipulates a clean model at inference time through crafted input. RAG poisoning corrupts the documents a model retrieves, while the weights stay clean. A backdoor is different because it lives in the model weights themselves. There is no malicious input to filter and no bad document to remove, so you cannot patch it out without retraining or replacing the model.
Can safety training remove a backdoor?
Not reliably. Research on planted backdoors has shown the hidden behavior can survive standard safety training. Those tests do not include the secret trigger, so the model looks aligned on every input you thought to try while the backdoor waits for the one string that activates it.
How do you defend against a poisoned model?
Check provenance and integrity on every model, adapter, and dataset, prefer trusted and signed sources, and pin specific versions. Evaluate on held out and adversarial trigger tests. Most important, restrict what the model can do downstream with least privilege and output checks, and keep hard authorization between the model and any dangerous action so a triggered model still cannot reach sensitive operations unchecked.