Stanford's TRACE: A New Way to Fix AI Agent Failures

Stanford researchers have introduced TRACE, a new agentic training system designed to address the persistent, recurring failures often encountered by Large Language Model (LLM) agents. By identifying specific missing capabilities that lead to these failures, TRACE transforms them into synthetic, verifiable Reinforcement Learning (RL) environments. This approach allows for the training of specialized LoRA adapters that target individual skill gaps, significantly improving agent performance across complex tasks.

Diagnosing and Targeting Capability Gaps

The TRACE system operates on the observation that agent failures are rarely random; instead, a small set of recurring deficits accounts for the majority of failed trajectories. The system utilizes an automated four-step pipeline to address these gaps. First, it performs a contrastive capability analysis, where an LLM agent evaluates rollouts to label trajectory-capability pairs as present or lacking. Only capabilities that show a clear contrast between success and failure are retained for further training.
Once these deficits are identified, TRACE employs a generation agent to build synthetic environments tailored to each specific capability. These environments isolate a single skill while maintaining the target's original tool schemas and formatting. Because the generation and verification processes are algorithmic, the system eliminates the need for human labels or external LLM judges, providing a dense and verifiable training signal.

Training Specialized Experts

After the synthetic environments are established, TRACE trains individual LoRA adapters for each identified capability using Group Relative Policy Optimization (GRPO). During this phase, the base model remains frozen, ensuring that the training focuses exclusively on the specific skill being addressed. By grouping rollouts by shared seeds, the system normalizes rewards to isolate the policy's contribution to the task, resulting in highly efficient learning.
The final stage of the pipeline involves composing these adapters into a Mixture-of-Experts (MoE) model. TRACE utilizes token-level routing, where a lightweight, trained gate directs each token to the most appropriate capability expert. This architecture allows the model to switch experts dynamically mid-trajectory, providing a flexible and powerful solution for complex agentic workflows.

Performance and Benchmarking

In practical applications, TRACE has demonstrated significant improvements on benchmarks such as the τ²-Bench and SWE-bench Verified. When tested on the Qwen3-30B-A3B backbone, TRACE improved τ²-Bench performance by 15.3 points and SWE-bench Verified Pass@1 by 15 points compared to the base model. It also outperformed existing baselines like GEPA and SWE-RL.
The system’s efficiency is particularly notable, as it achieved these results using less than one-fourth of the rollouts required by other methods. Furthermore, when applied to the Qwen3.6-27B model, TRACE reached a 73.2% Pass@1 rate on SWE-bench Verified, surpassing the performance of models such as GPT-5.2-Codex, GLM 5, and Claude 4.5 Sonnet on the public leaderboard. The TRACE system is released as open-source under an MIT license, offering a benchmark-agnostic pipeline for AI professionals.