AI Glossary
Adversarial Attack
A technique that manipulates inputs to cause an AI system to produce incorrect or unintended results.
Adversarial Attack
Overview
AI systems can be powerful, but they are not immune to manipulation.
In some cases, attackers can intentionally modify information in ways that cause an AI system to make mistakes.
This type of manipulation is known as an adversarial attack.
An adversarial attack occurs when inputs are deliberately crafted to influence how an AI model behaves.
The goal may be to cause incorrect predictions, bypass security controls, or produce unintended outputs.
A helpful way to think about an adversarial attack is an optical illusion.
The image appears normal to a person, but subtle changes can influence how it is interpreted.
Similarly, carefully designed inputs can sometimes confuse AI systems even when the changes seem insignificant to humans.
Researchers study adversarial attacks to better understand AI vulnerabilities and improve model security.
As AI systems become more widely used, defending against adversarial attacks is becoming an increasingly important part of AI security.
Why It Matters
Understanding adversarial attacks helps organizations identify and reduce AI-related security risks.
Real-World Example
Researchers may slightly modify an image in a way that causes an AI vision system to misidentify an object.