Back to AI Research

AI Research

PRO-LONG: Programmatic Memory Enables Long-Horizon... | AI Research

Key Takeaways

  • PRO-LONG: Programmatic Memory Enables Long-Horizon Reasoning Large language model (LLM) agents often struggle with long-horizon tasks, such as those found in...
  • Long-horizon tasks require sustained perception, reasoning, and exploration, and are a persistent challenge for large language model (LLM) agents.
  • This gap is reflected in their limited performance on continual learning benchmarks such as ARC-AGI-3, especially when models are evaluated out of the box.
  • Existing methods for context management face a significant tradeoff, as preserving more information makes retrieving relevant details less tractable.
  • We propose PRO-LONG, a minimal context management framework built around programmatic memory for LLM agents in long-horizon, exploratory settings.
Paper AbstractExpand

Long-horizon tasks require sustained perception, reasoning, and exploration, and are a persistent challenge for large language model (LLM) agents. This gap is reflected in their limited performance on continual learning benchmarks such as ARC-AGI-3, especially when models are evaluated out of the box. Various agent harnesses have been proposed to close this gap, and each commits to a strategy for handling long sequences of observations, i.e., what information to save from the environment and how to load it into model context, a choice we argue is particularly consequential. Existing methods for context management face a significant tradeoff, as preserving more information makes retrieving relevant details less tractable. We propose PRO-LONG, a minimal context management framework built around programmatic memory for LLM agents in long-horizon, exploratory settings. PRO-LONG addresses the tradeoff by keeping a complete, structured interaction log and capitalizing on recent progress in coding agents to search this history efficiently. On the full ARC-AGI-3 public game set, PRO-LONG improves over a base coding agent by an average of 18.0 percentage points across frontier models, and matches or exceeds state-of-the-art specialized harnesses (up to 76.1% pass@1) while using 4.2-5.8x fewer tokens. With Fable 5, PRO-LONG achieves 97.4% best@2 at a total cost of \$1,750. Relevant code and logs are available at this https URL .

PRO-LONG: Programmatic Memory Enables Long-Horizon Reasoning
Large language model (LLM) agents often struggle with long-horizon tasks, such as those found in the ARC-AGI-3 benchmark, where they must explore unknown environments and maintain performance over thousands of actions. A major challenge is "context management"—deciding what information to save from the environment and how to retrieve it later. Existing methods often force a tradeoff: saving more information makes it difficult to find relevant details, while summarizing information can lead to the loss of critical data. PRO-LONG addresses this by providing a minimal, programmatic memory framework that allows agents to store a complete, structured log of their interactions and search through it efficiently using code.

How PRO-LONG Works

The core of PRO-LONG is a simple, lossless memory system. Instead of relying on complex heuristics or learned summaries to decide what to remember, the agent automatically appends every observation, action, and outcome into a structured log file. Because this log is a faithful, ground-truth record of the entire interaction, the agent does not risk losing details that might only become relevant in hindsight. To retrieve information, the agent uses its native coding capabilities—such as regular expressions or Python scripts—to search the log. This approach treats the agent’s history as a searchable database, keeping retrieval tractable even when logs grow to over 100,000 lines.

Performance and Efficiency

PRO-LONG significantly improves the performance of base coding agents, increasing their success rate on the ARC-AGI-3 benchmark by an average of 18.0 percentage points. It matches or exceeds the performance of specialized, more complex agent harnesses while being significantly more efficient. By avoiding the need for heavy, custom-built scaffolding, PRO-LONG uses 4.2 to 5.8 times fewer tokens than prior state-of-the-art methods. In testing with the Fable 5 model, the framework achieved a 97.4% best@2 score, demonstrating that a minimal, programmatic approach can be both highly effective and cost-efficient.

Why It Matters

The success of PRO-LONG highlights that for long-horizon reasoning, simple and lossless access to history is often superior to complex, lossy summarization. The researchers found that the framework’s gains are driven by the ability to perform programmatic analysis on the full interaction log. While other agents spend significant resources on workspace management or writing manual notes, PRO-LONG agents dedicate their tool use to analyzing the environment and their own past actions. This is particularly beneficial in games where the current board state does not reveal the full dynamics of the environment, allowing the agent to "look back" and infer rules based on its complete history.

Comments (0)

No comments yet

Be the first to share your thoughts!