AI Glossary

SWE-bench

Also called SWE-bench Verified

What does “SWE-bench” mean?

SWE-bench: A family of software engineering benchmarks that tests whether a system can patch real repository issues. Tests check whether the submitted patch fixes the issue without breaking expected behavior.

SWE-bench visual guide: A family of software engineering benchmarks that tests whether a system can patch real repository issues. Tests check whether the submitted patch fixes the issue without breaking expected behavior.
Visual guide Open full size (new tab)

What is an example of “SWE-bench”?

An agent receives a bug report and the relevant repository state, then submits a patch for evaluation.

Keep learning

See also

Agent evalsTermCoding agentTermBenchmarkTermPass@kTerm

Same topic

BLEUTermExact match scoreTermGPQATermHumanEvalTermPerplexityTerm