Back to AI Research

AI Research

The Resolution Hypothesis argues that noisy selection limits agent evolution

Key Takeaways

  • Across seven evolution methods and six models, researchers find evaluation noise can hide useful improvements and make agent-method rankings unstable.
  • The Resolution Hypothesis studies how agent-evolution methods choose between candidate changes.
  • Its authors find that noisy evaluations can select harmful candidates and obscure differences between methods.
  • A November 17, 2026 preprint argues that agent evolution often stalls because evaluations cannot reliably distinguish improvements from regressions.
  • The study examines seven evolution methods and six models across more than ten million language-model calls.

A November 17, 2026 preprint argues that agent evolution often stalls because evaluations cannot reliably distinguish improvements from regressions. The study examines seven evolution methods and six models across more than ten million language-model calls.

What does the Resolution Hypothesis propose?

The paper separates proposing a useful change from recognizing that it is useful. Agent-evolution systems commonly generate candidate prompts, skills or memories, score them and keep the apparent improvement.
The authors argue that a noisy score can break that loop even when good candidates are being proposed. The bottleneck is then the resolution of the evaluation, rather than only the creativity of the proposer.

What happens when candidates are selected?

At the candidate level, the study finds helpful artifacts but reports that selection is no better than random at separating them from harmful ones in the examined conditions. Selection errors can accumulate over successive rounds.
At the method level, the paper finds that typical margins of a few percentage points are smaller than differences between independent runs of the same method. A ranking based on one run can therefore exaggerate how much is known about the relative methods.
These are findings from the tested models, tasks and evolution procedures. They do not establish that every evaluation is useless or that no agent can improve.

What should an evaluation report?

Franklin analysis: compare the size of a claimed improvement with the variability of the test. A small gain means something different when repeated runs agree than when the ranking reverses under another run.
For a team trying agent evolution, preserve the proposed candidates, the selection scores and the later outcomes. That makes it possible to inspect whether the system failed to propose a good change or failed to recognize one. The paper’s practical contribution is this distinction: improving the proposal mechanism alone may not resolve a selection process that cannot reliably tell better work from worse work.
For a related memory-design question, CoReMem’s coding-agent memory study looks at what coding agents should carry between tasks.

Comments