AI Glossary

Red teaming

Also called adversarial testing

What does “Red teaming” mean?

Red teaming: Deliberately attacking an AI system to find its weaknesses before real adversaries or ordinary users do. Red teamers try to provoke harmful outputs, leak hidden instructions, bypass safety rules or trigger dangerous capabilities, then report what worked so developers can fix it.

Red teaming visual guide: Deliberately attacking an AI system to find its weaknesses before real adversaries or ordinary users do. Red teamers try to provoke harmful outputs, leak hidden instructions, bypass safety rules or trigger dangerous capabilities, then report what worked so developers can fix it.
Visual guide Open full size (new tab)

What is an example of “Red teaming”?

Before launch, a lab might hire outside experts to spend weeks trying to make a new model produce dangerous chemistry instructions.

Keep learning

See also

JailbreakTermPrompt injectionTermEvalsTermAI safetyTerm

Same topic

Data poisoningTermFrontier safety frameworkTermSafety classifierTermResponsible AITermModel cardTerm