Back to AI Research

AI Research

AREX: Towards a Recursively Self-Improving Agent fo... | AI Research

Key Takeaways

  • AREX: Towards a Recursively Self-Improving Agent for Deep Research introduces a new approach to AI-driven research.
  • Deep research requires agents to find answers that jointly satisfy multiple constraints.
  • Discovering such answers is costly, whereas verifying a candidate can often be decomposed into tractable constraint-wise checks.
  • We introduce AREX, a family of Recursively Self-Improving (RSI) deep research agents.
  • We train AREX on verified synthetic tasks and high-quality trajectories through agentic mid-training and long-horizon reinforcement learning.
Paper AbstractExpand

Deep research requires agents to find answers that jointly satisfy multiple constraints. Discovering such answers is costly, whereas verifying a candidate can often be decomposed into tractable constraint-wise checks. This discovery--verification asymmetry suggests that a research agent should do more than simply search longer: it should recursively improve its current answer by verifying intermediate results and using the partially verified state to guide subsequent refinement. We introduce AREX, a family of Recursively Self-Improving (RSI) deep research agents. AREX alternates between an inner research loop that gathers evidence and constructs a provisional answer, and an outer self-improvement loop that audits the answer constraint-wise, identifies unresolved claims, and launches targeted follow-up research. To sustain RSI over long horizons, AREX learns an autonomous context-update tool that compresses growing interaction history into a compact improvement state preserving verified evidence and unresolved constraints, without relying on an external model. We train AREX on verified synthetic tasks and high-quality trajectories through agentic mid-training and long-horizon reinforcement learning. To mitigate sparse final rewards during long horizon learning, we emphasize key steps where decisive evidence is acquired or erroneous research directions are corrected. We instantiate a dense 4B model and a 122B-A10B Mixture-of-Experts model. Across BrowseComp, WideSearch, DeepSearchQA, Humanity's Last Exam (HLE), and other reasoning and tool-use benchmarks, AREX substantially outperforms comparable-scale baselines and remains competitive with models using substantially more activated parameters.

AREX: Towards a Recursively Self-Improving Agent for Deep Research introduces a new approach to AI-driven research. The core challenge in deep research is that finding an answer that satisfies multiple, complex constraints is difficult, while verifying if an answer meets those constraints is relatively straightforward. AREX leverages this "discovery-verification asymmetry" by treating research as a recursive process where the agent continuously audits its own progress, identifies what is still missing, and refines its search strategy accordingly.

The Recursive Research Loop

AREX operates through a two-part cycle. The "inner research loop" gathers evidence and builds a provisional answer. Once this is done, the "outer self-improvement loop" takes over to audit the work. If the agent’s confidence in its answer is low, the outer loop identifies exactly which claims are unsupported or unresolved. Instead of starting over from scratch, the agent uses this diagnosis to launch a targeted follow-up research round, building upon the verified progress it has already made.

Managing Long-Term Memory

A major hurdle in long-horizon research is that interaction history can become cluttered with redundant information, failed queries, and outdated plans. To solve this, AREX includes an autonomous context-update tool. Rather than relying on generic summarization, the agent uses this tool to compress its history into a "compact improvement state." This state keeps track of verified evidence, current candidates, and remaining gaps, ensuring the agent stays focused on its evolving research objective without getting distracted by irrelevant past data.

Training for Complex Tasks

To build these capabilities, the researchers used a multi-stage training framework. They created a dataset of synthetic tasks that require multi-step reasoning and evidence synthesis. During training, they emphasized "key steps"—moments where the agent successfully acquires decisive evidence or corrects an erroneous research path. This helps the model learn how to make progress over long periods, even when the final reward for a correct answer is only provided at the very end of the process.

Performance and Results

The researchers developed two versions of AREX: a dense 4B model (Turbo) and a 122B-A10B Mixture-of-Experts model (Base). Across various benchmarks—including deep search, wide search, and tool-use tasks—AREX consistently outperformed models of similar scale. The results suggest that by focusing on recursive self-improvement and structured state management, AI agents can achieve high-quality research outcomes more efficiently than by simply searching for longer periods.

Comments (0)

No comments yet

Be the first to share your thoughts!