Back to AI Research

AI Research

Correctly reading an agent state does not guarantee the right action

Key Takeaways

  • Paired factual probes reveal decision failures after models have already understood the relevant observations.
  • ## Diagnose the action separately from the reading
  • The [Reading Right, Acting Wrong paper](https://arxiv.org/abs/2610.18579) examines fast System One models that score a supplied set of next-action options in a single pass.
  • When such a model makes a mistake, its action alone does not explain whether it misunderstood the state or chose badly despite reading it correctly.
  • The authors pair each next-action question with factual probes about the same frozen state.

Diagnose the action separately from the reading

The Reading Right, Acting Wrong paper examines fast System One models that score a supplied set of next-action options in a single pass. When such a model makes a mistake, its action alone does not explain whether it misunderstood the state or chose badly despite reading it correctly.
The authors pair each next-action question with factual probes about the same frozen state. Each probe is answered independently. The benchmark spans simulated environments, customer-service policies, tool use, web navigation and coding.

Many errors happen after correct understanding

Across seventeen fast systems and a reasoning model, the paper reports that most wrong actions follow correct reading. More accurate systems have a larger share of their remaining errors in this category. The errors also concentrate on some of the same states across different kinds of systems.
In settings where the probed facts provably determine the action, models can answer every fact correctly and still fail to combine them into the right decision. Composing their own factual answers with code repairs that failure in the paper's corresponding tests.
For real states, the authors find that the missing element is often the next sub-goal rather than another fact. That distinction matters for a team deciding whether to improve extraction, supply more context or change how an agent plans its next step.
The benchmark provides a behavioral diagnosis. It does not expose every internal cause of an error, or establish that code composition solves all decisions in an open-ended environment.

Build paired failure cases

Franklin analysis: when investigating an agent failure, preserve the exact state and available actions. Ask separately what the agent understood and which next step it selected.
If the facts are already correct, adding more retrieved text may leave the failure untouched. A better experiment tests the missing combination rule or sub-goal directly, with a clear expected action and a check that the proposed repair generalizes beyond that single saved state.

Comments