DRACO: Fine-Grained Credit Assignment with Dynamic... | AI Research

DRACO: Fine-Grained Credit Assignment with Dynamic Rubrics for Long-Horizon Agent Training introduces a new way to train AI agents for complex, multi-step tasks without needing a "ground-truth" verifier. In many real-world scenarios, such as research or customer support, there is no simple programmatic way to check if an agent succeeded. DRACO solves this by using a flexible, rubric-based evaluation system that generates its own criteria during training and intelligently distributes credit to the specific steps that contributed to an agent's success or failure.

The Challenge of Outcome-Blind Training

Most current reinforcement learning methods for AI agents rely on a "verifier"—a system that checks if the final answer is correct. However, many long-horizon tasks, which require dozens of interdependent steps, lack such an oracle. When a verifier is missing, researchers often use a single score for an entire trajectory. This is problematic because it treats every step in a long sequence as equally important, even though some steps are helpful, some are redundant, and others are mistakes. DRACO addresses this by moving away from terminal rewards and instead focusing on "process criteria."

How DRACO Works

DRACO operates through two main innovations that work together to guide the agent: The same ai evaluation question is explored in SciMIF, which adds a research perspective.

• Dynamic Rubrics: Instead of using a fixed list of rules, DRACO generates rubrics that adapt to the specific task and the agent's actual performance. A judge model proposes criteria based on the task instructions and then refines them based on how the agent behaves in each rollout. This ensures the evaluation criteria evolve alongside the agent's capabilities.

• Rubric-Conditioned Step Credit: Once a trajectory is scored, DRACO does not simply assign that score to the entire sequence. Instead, it uses a closed-form mathematical rule to redistribute the credit to individual steps. By identifying which specific steps were cited by the rubric as "passing" or "failing," the system assigns higher rewards to the steps that actually mattered. This allows the agent to learn from its specific actions rather than just the final outcome.

Performance and Results

DRACO was tested on the AppWorld benchmark, a complex environment for tool-using agents. Despite never having access to the environment's ground-truth unit tests during training, DRACO significantly outperformed base models. On AppWorld, it achieved a 15.9-point gain in Task Goal Completion over the base model and even outperformed models trained with sparse ground-truth rewards. The same large language models question is explored in Recursive Experiential-Working Memory Evolution for Long-Horizon..., which adds a research perspective.
Furthermore, DRACO demonstrated strong "zero-shot" transfer capabilities. When tested on the out-of-domain Tau-Bench, it improved performance over the base model without any additional training in that domain. These results suggest that by focusing on process-based credit assignment, agents can learn to navigate complex tasks more effectively even when they lack a clear, automated way to verify their final success.

Key Takeaways

The primary advantage of DRACO is that it provides a way to train high-performing agents in environments where success is difficult to define or verify. Because the redistribution of credit is done in a closed-form manner, it does not require training a separate, complex attribution module. By concentrating credit on the steps that the rubric implicates, DRACO makes the training process more statistically efficient and better aligned with the actual decision-making process of the agent. The same ai systems question is explored in Efficient Test-Time Adaptation through Human-AI Interaction, which adds a research perspective. as detailed in the full paper on Arxiv