Back to AI Research

AI Research

ABSeeker: Training Long-Horizon Search Agents via A... | AI Research

Key Takeaways

  • ABSeeker is a framework designed to improve how long-horizon search agents are trained.
  • Long-horizon search agents must make multiple sequential actions (steps) to search, retrieve, verify, and integrate evidence to reach a final answer.
  • It then applies Clue-Anchored Step Scoring to evaluate each search step against these clues, converting sparse binary outcome supervision into dense step-level rewards.
  • Based on these rewards, we develop ABC-SFT, which reweights the loss of each turn, and ABC-GRPO, which uses the step-level scores as rewards in GRPO.
  • Building on this framework, we train ABSeeker based on Qwen3.5-4B with only 8.5k examples.
Paper AbstractExpand

Long-horizon search agents must make multiple sequential actions (steps) to search, retrieve, verify, and integrate evidence to reach a final answer. However, existing methods for training these agents typically treat all steps within a trajectory uniformly during both supervised fine-tuning (SFT) and reinforcement learning (RL), failing to distinguish useful actions from erroneous or redundant ones. In this paper, we propose Answer-Backtracked Credit Assignment (ABC), a fine-grained credit assignment framework for training long-horizon search agents by converting sparse trajectory-level outcomes into dense step-level supervision that rewards useful actions (even in failed trajectories) while suppressing erroneous or redundant actions. Specifically, given a potentially obscure query and its corresponding ground-truth answer, ABC first performs Answer-Backtracked Clue Recovery, which traces back from the answer to recover intermediate clues required to solve the question. It then applies Clue-Anchored Step Scoring to evaluate each search step against these clues, converting sparse binary outcome supervision into dense step-level rewards. Based on these rewards, we develop ABC-SFT, which reweights the loss of each turn, and ABC-GRPO, which uses the step-level scores as rewards in GRPO. Building on this framework, we train ABSeeker based on Qwen3.5-4B with only 8.5k examples. ABSeeker achieves 37.3% on BrowseComp and 39.1% on BrowseComp-ZH. With context management, the scores further improve to 55.3% and 52.9%, respectively, significantly outperforming same-scale (4B) agents and even matching the performance of larger ones (approximately 30B). These results demonstrate the effectiveness of answer-backtracked step-level credit assignment for training long-horizon search agents.

ABSeeker is a framework designed to improve how long-horizon search agents are trained. Current training methods typically evaluate agents based only on whether their final answer is correct, treating every step in a search trajectory as equally good or bad. This approach fails to distinguish between helpful actions and errors within a single search process. ABSeeker introduces Answer-Backtracked Credit Assignment (ABC), which provides fine-grained, step-level feedback to reward useful actions—even in failed searches—and suppress redundant or incorrect ones.

The Credit Assignment Problem

In long-horizon search, agents must perform many sequential steps, such as formulating queries, retrieving evidence, and verifying information. Existing training methods use sparse, trajectory-level supervision, which creates two primary issues:

  • Failed trajectories may contain highly useful steps that are unfairly penalized because the final answer was wrong.

  • Successful trajectories may contain erroneous or redundant steps that are unfairly rewarded because the final answer was correct.
    By failing to isolate the quality of individual actions, these models struggle to learn which specific behaviors contribute to successful outcomes.

How Answer-Backtracked Credit Assignment Works

The ABC framework uses the ground-truth answer as a starting point to evaluate the search process. It operates in two stages: 1. Answer-Backtracked Clue Recovery: The system traces backward from a verified answer to identify the specific entities, facts, and relationships that were necessary to solve the query. These serve as "clues" or anchors for evaluation. 2. Clue-Anchored Step Scoring: Every step in a search trajectory is evaluated against these recovered clues. A scoring rubric assigns positive points for discovering correct clues or ruling out incorrect candidates, and negative points for dismissing correct information or submitting wrong answers.
These step-level scores are then used to train the agent through two methods: ABC-SFT, which reweights the loss of each turn during supervised fine-tuning, and ABC-GRPO, which uses the scores as direct rewards in reinforcement learning.

Performance and Results

The researchers trained ABSeeker using the Qwen3.5-4B model on 8.5k examples. In testing across benchmarks like BrowseComp and GAIA, ABSeeker outperformed other 4B-scale agents. With context management enabled, it achieved 55.3% on BrowseComp and 52.9% on BrowseComp-ZH. The authors note that these results are competitive with significantly larger models (approximately 30B parameters), suggesting that fine-grained credit assignment allows smaller models to search more efficiently and deliberately.

Franklin Analysis

The evidence suggests that the primary advantage of the ABSeeker approach is its ability to extract training value from failed search attempts. The paper’s reward distribution analysis shows that nearly 10% of steps in failed trajectories receive positive scores, meaning the model can learn from "near-misses" that would otherwise be discarded as entirely negative data. Conversely, the framework identifies that approximately 4% of steps in successful trajectories are low-quality, allowing the model to refine its behavior by penalizing these specific errors rather than blindly reinforcing the entire successful path.

Comments (0)

No comments yet

Be the first to share your thoughts!