Back to AI Research

AI Research

ContextSniper: AntTrail's Token-Efficient Code... | AI Research

Key Takeaways

  • Solving the "Noise" Problem in AI Coding Large language model agents are increasingly used to fix software bugs by navigating entire code repositories.
  • This paper presents ContextSniper, AntTrail's token-efficient code memory layer for repository-level program repair.
  • We evaluate ContextSniper on SWE-bench Lite with OpenClaw and Claude Code, using 50 task runs per host-agent condition.
  • ContextSniper reduces total token use by 51.5% and logged cost by 36.4% for OpenClaw, and reduces total token use by 38.9% and estimated cost by 27.3% for Claude Code.
  • Submitted-resolution rates decrease slightly, from 26.0% to 24.0% for OpenClaw and from 32.0% to 30.0% for Claude Code.
Paper AbstractExpand

Large language model agents can repair real repository issues, but they often spend large context budgets on whole-file reads, broad searches, and long terminal outputs where useful evidence is mixed with irrelevant code and logs. This paper presents ContextSniper, AntTrail's token-efficient code memory layer for repository-level program repair. As the coding specialization of AntTrail's broader agent memory engine, ContextSniper implements the Sniper feature for precision evidence selection: it retrieves candidate code and runtime evidence, ranks it with hybrid retrieval signals, filters long outputs through an intention-aware context gate, and returns compact evidence packets while preserving recoverable source context outside the prompt. We evaluate ContextSniper on SWE-bench Lite with OpenClaw and Claude Code, using 50 task runs per host-agent condition. ContextSniper reduces total token use by 51.5% and logged cost by 36.4% for OpenClaw, and reduces total token use by 38.9% and estimated cost by 27.3% for Claude Code. Submitted-resolution rates decrease slightly, from 26.0% to 24.0% for OpenClaw and from 32.0% to 30.0% for Claude Code. ContextSniper's pilot testing scripts are open-sourced at this https URL

Solving the "Noise" Problem in AI Coding

Large language model agents are increasingly used to fix software bugs by navigating entire code repositories. However, these agents often struggle with "context bloat." When an agent performs a broad search or reads entire files, it fills its limited memory with irrelevant code, repeated logs, and build noise. This makes it harder for the model to find the specific lines of code that actually cause a bug, leading to higher costs and inefficient "read-try-fail" loops. ContextSniper is a new memory layer designed to sit between the agent and the repository to filter this noise, ensuring the agent receives only the most relevant evidence for the task at hand.

How ContextSniper Works

Instead of feeding raw repository data directly to the agent, ContextSniper acts as a smart filter. It organizes repository information into a structured memory hierarchy and uses a process called "sniping" to select only the most useful pieces of information.
The system employs several key techniques:

  • Hybrid Retrieval: It combines semantic search, keyword matching (BM25), and symbolic metadata (like function names and locations) to find the right code. If these methods miss the mark, it falls back to a traditional search tool.

  • Intention-Aware Gating: It monitors the agent's actions and filters out long, noisy terminal outputs or irrelevant file contents, keeping only the data that directly helps with the repair.

  • Synchronization: As the agent makes edits to the code, ContextSniper updates its internal memory to ensure the agent is always working with the most current version of the repository.

Impact on Efficiency and Cost

ContextSniper was tested using two different coding agents, OpenClaw and Claude Code, on the SWE-bench Lite benchmark. The results demonstrate a significant improvement in resource management:

  • OpenClaw: Total token usage dropped by 51.5%, and logged costs decreased by 36.4%.

  • Claude Code: Total token usage dropped by 38.9%, and estimated costs decreased by 27.3%.
    While these efficiency gains are substantial, the researchers noted a slight decrease in the rate of successful bug resolutions—dropping from 26.0% to 24.0% for OpenClaw and from 32.0% to 30.0% for Claude Code. This suggests that while the system is highly effective at reducing noise and cost, there is a trade-off between extreme token efficiency and the depth of context available for complex repairs.

A New Approach to Agent Memory

ContextSniper does not attempt to replace the AI agent itself or change how it reasons. Instead, it focuses entirely on the "context-access layer." By treating context as evidence rather than just a stream of text, it allows agents to remain focused on the task of fixing code without being overwhelmed by the surrounding repository environment. The researchers have made their pilot testing scripts open-source, providing a foundation for others to build more efficient, memory-conscious coding agents.

Comments (0)

No comments yet

Be the first to share your thoughts!