Back to AI Research

AI Research

HOA separates solving, review and proof verification for olympiad agents

Key Takeaways

  • HOA coordinates multiple sessions, task knowledge and Lean checks to reduce errors in difficult mathematical and scientific tasks.
  • ## Give a difficult task several feedback channels
  • [Humanize Olympiads Agent](https://arxiv.org/abs/2610.13632), or HOA, is a research framework for coordinating multiple agent sessions on difficult olympiad and scientific problems.
  • Its authors identify three weaknesses in a single extended session: an early error can persist, missing knowledge can halt progress, and a model can appear successful after changing the meaning of the task.
  • HOA combines independent review, a structured knowledge wiki and formal verification.

Give a difficult task several feedback channels

Humanize Olympiads Agent, or HOA, is a research framework for coordinating multiple agent sessions on difficult olympiad and scientific problems. Its authors identify three weaknesses in a single extended session: an early error can persist, missing knowledge can halt progress, and a model can appear successful after changing the meaning of the task.
HOA combines independent review, a structured knowledge wiki and formal verification. The reviewer checks candidate answers and returns feedback. The wiki supplies task-specific knowledge. A verifier checks generated Lean proofs against trusted challenge statements, making the original problem part of the verification boundary.

Read the reported results as framework results

The authors report strong results on olympiad problem sets and PutnamBench using GPT-5.6 Sol. They also evaluate the framework with Kimi-K3 and report substantial performance across the same kinds of tasks. These are results attributed to the paper's experiments, including its workflow, rather than a claim that the underlying models always solve difficult problems unaided.
The central contribution is the organization of feedback. Review, knowledge recovery and a proof checker address different reasons an attempted solution can fail. Adding sessions alone would not necessarily supply those missing checks.

Keep the challenge statement fixed

Franklin analysis: formal verification is most useful when the checked statement remains faithful to the requested result. A proof of an easier or altered statement cannot establish success on the original task. HOA explicitly uses trusted challenge statements to address that risk.
A practical evaluation should therefore preserve the input statement, retain rejected candidates and reviewer feedback, and record which results have formal verification. Competition benchmarks measure performance on their specified tasks; they do not establish that every scientific claim can be settled by the same workflow. The paper offers a structured way to test feedback-driven agents while keeping success tied to the problem they were given.

Comments