Do AI Agents Know When a Task Is Simple? Toward Com...

Do AI Agents Know When a Task Is Simple? Toward Com... | AI Research

Key Takeaways

Do AI Agents Know When a Task Is Simple?
Toward Complexity-Aware Reasoning and Execution addresses a common inefficiency in modern AI agents: the tendency to...
Large language model (LLM) agents increasingly automate multi-step engineering and informatics workflows, yet they rarely ask how much effort a task actually requires.
They often follow a maximum-context-first strategy--re-reading files and dependencies they have already seen--turning a one-line edit into a small code-base audit.
We argue the missing capability is task-aware execution-scope estimation: judging a task's difficulty, the information it truly needs, and the shortest reliable path before committing budget.

Paper AbstractExpand

Large language model (LLM) agents increasingly automate multi-step engineering and informatics workflows, yet they rarely ask how much effort a task actually requires. They often follow a maximum-context-first strategy--re-reading files and dependencies they have already seen--turning a one-line edit into a small code-base audit. We argue the missing capability is task-aware execution-scope estimation: judging a task's difficulty, the information it truly needs, and the shortest reliable path before committing budget. We formalize minimum-sufficient execution and the Agent Cognitive Redundancy Ratio (ACRR), and propose E3 (Estimate, Execute, Expand): the agent estimates an initial operating point, executes a minimum viable path, and expands scope only when verification fails. On MSE-Bench--a deterministic benchmark of 121 edits in a capability-controlled simulator--E3 matches the strongest baseline's 100% success while cutting cost by 85%, tokens by 91%, and inspected files by 92%, and further beats a strong adaptive retrieval baseline by 16%; the gains survive held-out instruction wording and essentially every cost weighting. A companion real-model harness (LLM-Case) corroborates the effect on a live gpt-4o agent editing a real open-source library, with every candidate patch graded by actually running the project's real pytest suite against a measured oracle: the over-reading is milder but real, and E3 is the leanest and fastest policy at comparable task success--its one shortfall a provider rate-limit, not a wrong edit. We frame this as a controlled probe of execution redundancy, not a measurement of any deployed agent, and position task-aware execution as a step toward engineering-grounded AI (EGAI)--agents whose effort is anchored in the engineering reality of the task. We release the framework and benchmark.

Do AI Agents Know When a Task Is Simple? Toward Complexity-Aware Reasoning and Execution addresses a common inefficiency in modern AI agents: the tendency to over-analyze simple tasks. When given a straightforward instruction, many agents default to a "maximum-context-first" strategy, re-reading entire codebases and dependencies even when the task requires only a minor edit. This paper introduces a framework to help agents judge the actual difficulty of a task before they begin, ensuring their effort is proportional to the work required.

The Problem of Over-Provisioning

Current AI agents often treat every request as a complex audit. For example, if an agent is asked to perform a simple one-line code change, it might spend minutes re-analyzing the entire project architecture. While this caution is helpful for complex problems, it creates significant cognitive and execution overhead for simple ones. The authors argue that agents lack "task-aware execution-scope estimation"—the ability to judge how much information is truly necessary to solve a problem. Without this, agents trade efficiency for unnecessary, exhaustive caution.

The E3 Framework: Estimate, Execute, Expand

To solve this, the authors propose the E3 framework. Instead of jumping straight into a deep search, the agent follows three steps: 1. Estimate: The agent creates an "initial operating point," predicting the task's difficulty, the specific files or tools needed, and the level of risk involved. 2. Execute: The agent follows a "minimum viable path" based on that estimate, focusing only on the information identified as necessary. 3. Expand: If the initial plan fails or the agent's confidence is low, it expands its scope and revises the plan.
This approach is inspired by power-system analysis, where solvers start with a simplified estimate and refine it iteratively, rather than trying to calculate every possibility at once.

Measuring Wasted Effort

The researchers formalized the concept of "minimum-sufficient execution" to define the least amount of effort required to complete a task successfully. They introduced the Agent Cognitive Redundancy Ratio (ACRR) to measure how much an agent over-spends compared to this ideal minimum. By using a controlled simulator (MSE-Bench) with 121 different edits, they were able to calculate exactly how much unnecessary work agents perform.

Key Results

The E3 framework demonstrated significant efficiency gains without sacrificing performance. In testing, E3 matched the success rate of the strongest baseline models while:

Reducing costs by 85%.
Using 91% fewer tokens.
Inspecting 92% fewer files.
These results held up even when the researchers changed the wording of the instructions, suggesting that the efficiency is a result of the E3 architecture itself rather than a fluke of the testing environment. The authors conclude that by anchoring an agent's effort in the "engineering reality" of a task, we can build more efficient, "engineering-grounded" AI systems.