Skip to content
All articles
June 30, 2026 9 min read

Data Poisoning Attacks on AI: How They Work and How to Defend

By Chris Rees

Most AI attacks target the model after it's deployed. Data poisoning is different — and more insidious. It attacks the model while it's still learning, baking the flaw into the model itself. Here's how poisoning works, the main varieties, and how to defend, framed for CompTIA SecAI+ Domain 2.

The core idea

A machine-learning model is only as trustworthy as the data it learned from. Data poisoning is the deliberate manipulation of that training (or fine-tuning) data to change the model's behavior. Because the corruption happens during training, it becomes part of the model's weights — you can't patch it out at inference time the way you might filter a malicious prompt.

The attacker's leverage is access to the data pipeline: a public dataset, scraped web content, user-submitted feedback used for retraining, or a compromised data vendor. Modern models trained on huge web-scraped corpora are especially exposed, because no one manually reviews billions of examples.

Poisoned samples mixed into training data produce a backdoored model that misbehaves on a trigger input Training data poisoned sample Training process Backdoored model looks normal on ordinary inputs Input with secret trigger e.g. a specific phrase or pixel pattern Attacker-chosen output misclassify, bypass, or leak
A few poisoned samples during training can plant a backdoor that lies dormant until the attacker supplies a secret trigger.

The main types

Availability attacks (degrade everything)

The goal is to make the model worse across the board — lower accuracy, more errors. Flooding a spam filter's retraining data with mislabeled examples so it stops catching spam is a classic. These are blunt but effective denial-of-quality attacks.

Integrity / backdoor attacks (targeted)

Far stealthier. The model behaves perfectly on normal inputs — passing every test — but misbehaves when it sees a secret trigger the attacker planted during training. A face-recognition model might work flawlessly until someone wears a specific pattern that makes it authenticate them as an admin. Because the model looks fine in evaluation, backdoors can survive all the way to production.

Targeted misclassification

A narrow version: cause specific inputs to be classified the attacker's way (one person's transactions always marked "legitimate") while leaving overall accuracy untouched to avoid detection.

Why backdoors are so dangerous: normal QA can't find them. The model passes accuracy tests because the malicious behavior only fires on a trigger the defender doesn't know to test for. Detection has to look at the data and the model, not just output quality.

Real-world shape

You don't need exotic access. Poisoning has shown up wherever models learn from data the public can influence: chatbots that learned toxic behavior from user interactions, image datasets where a small fraction of altered samples implanted reliable backdoors, and the standing risk that web-scraped training corpora include pages an attacker planted on purpose. Research has repeatedly shown that a surprisingly small percentage of poisoned samples — sometimes well under 1% — can install a working backdoor.

How to defend

There's no single fix; poisoning is countered with defense in depth across the data lifecycle.

Control What it does
Data provenance & lineage Track where every dataset came from and how it changed, so tampering is traceable.
Source vetting & curation Prefer trusted, reviewed data sources over unverified scraped or crowd-sourced data.
Anomaly detection on data Statistically flag samples that don't fit the distribution before training.
Robust training methods Techniques that reduce the influence any small group of samples can have.
Model validation & red-teaming Test for backdoors and unexpected behavior, not just average accuracy.
Access control on the pipeline Restrict who and what can add to training data — treat the pipeline like production.

The throughline matches the rest of AI security: integrity of the training data is a security property. Provenance, lineage, and access control on the data pipeline are the foundation; anomaly detection and red-teaming are the safety net.

Key takeaways

  • Poisoning attacks training, not inference — the flaw is baked into the model's weights.
  • Availability attacks degrade the whole model; backdoor attacks hide, firing only on a secret trigger.
  • Backdoors evade normal QA because the model looks fine until the trigger appears.
  • Tiny fractions of poisoned data can be enough to implant a backdoor.
  • Defense is lifecycle-wide: provenance, source vetting, anomaly detection, robust training, and red-teaming.

Data poisoning is one of the headline attacks in SecAI+ Domain 2. Build the underlying model first — see how LLMs actually work — then work through the full attack/defense catalog in our SecAI+ Domain 1 guide.

Share this article

Ready to study?

Get the interactive SecAI+ Domain 1 guide — practice questions and a PDF.

Browse study guides