garak for probing language-model failure modes

How it works

Probes supply test prompts while detectors decide whether responses match a failure pattern. A run records results for combinations of model and probe configuration.

What you need

Use a model endpoint you own or are authorized to test and scope the probes. Save enough configuration to reproduce the result without storing sensitive responses unnecessarily.

What to try first

Probe a test endpoint you control using a narrow suite. Read representative hits and misses before relying on an aggregate report.

Include a benign control prompt alongside attack probes. Detector false positives matter when results drive incident triage or release decisions.

Where to look for failures

The probe catalog covers selected failure modes. Record scope, detector settings and model revision so a low failure rate is interpretable.

Related Franklin pages

Define a threat model and a clear success criterion before running tests. Keep benign controls and failed attack examples so the report shows both sides of the measurement.

• Study attacks and defenses against machine-learning models using controlled experiments. Adversarial Robustness Toolbox for model stress tests
• Build model evaluations with datasets, solvers, tools and scorers. Inspect AI for evaluating tool-using models

For broader questions about the risks of deploying AI systems, see Franklin’s glossary. AI safety

Project documentation

• Current project README
• README at the reviewed revision