AI GitHub
Adversarial Robustness Toolbox for model stress tests
Study attacks and defenses against machine-learning models using controlled experiments.
Key Takeaways
- ART defines estimator interfaces and implements attacks, defenses and evaluation methods around them.
- Decide which model interface an attack needs and how inputs are normalized.
- Use a small image classifier and document the threat model before running a supported attack.
- An attack's settings are part of the result.
How it works
ART defines estimator interfaces and implements attacks, defenses and evaluation methods around them. An experiment can keep a model fixed while changing attacker access or perturbation limits.
What you need
Decide which model interface an attack needs and how inputs are normalized. Give the experiment a clear permitted perturbation and a reproducible baseline.
What to try first
Use a small image classifier and document the threat model before running a supported attack. Compare clean and perturbed accuracy under the same preprocessing.
Repeat the experiment with the same inputs and attack parameters after changing one defense setting. This isolates the intervention from sampling or preprocessing differences.
Where to look for failures
An attack's settings are part of the result. Check input scale, access assumptions and perturbation bounds before comparing defenses.
Related Franklin pages
Define a threat model and a clear success criterion before running tests. Keep benign controls and failed attack examples so the report shows both sides of the measurement.
- Build model evaluations with datasets, solvers, tools and scorers. Inspect AI for evaluating tool-using models
- Run targeted probes and detectors against supported model endpoints. garak for probing language-model failure modes
For broader questions about the risks of deploying AI systems, see Franklin’s glossary. AI safety
Project documentation
Try it for your workflow
Explore the project’s documentation for setup and usage, or save this page to a collection for later.
Read the project documentation ↗