Interactive Reward Agent: GUI Task Evaluation via E... | AI Research

The Interactive Reward Agent (IRA) is a framework designed to improve how automated systems evaluate whether a GUI agent has successfully completed a user instruction. While existing evaluators often rely on visual screenshots, the authors argue that these are insufficient for tasks requiring changes to system configurations, file data, or application settings. IRA addresses this by interacting with the environment to verify task completion through a "propose-then-verify" process.

The Propose-then-Verify Framework

IRA operates by first generating specific completion conditions based on the task instruction and the initial and final screenshots of the GUI agent's execution. Rather than making a single judgment based on visual data, the agent then enters an iterative verification phase. It uses a set of tools—including system tools, application tools, and GUI tools—to gather evidence from the environment. The agent maintains an interaction history, allowing it to decide which additional evidence is needed until it can reach a final, evidence-based verdict for each condition.

Testing with GUI-RewardBench

To measure the reliability of GUI evaluators, the authors introduced GUI-RewardBench, a dataset of 321 task trajectories spanning 10 Ubuntu desktop application categories. The benchmark includes three types of tasks:

• Visible-state tasks: Verifiable through screenshots.

• Hidden-state tasks: Requiring inspection of system settings or application preferences.

• Artifact-verification tasks: Requiring the checking of generated files, such as documents or spreadsheets.
  The benchmark was constructed by replaying recorded action sequences across virtual machine instances to ensure the final states were stable.

Performance and Results

Experiments show that IRA outperforms existing passive VLM-based evaluators. On GUI-RewardBench, IRA achieved 86.9% accuracy, compared to 78.8% for the strongest passive baseline, DistRL. The authors note that the performance gap is most significant in categories where evidence is hidden from the user interface, such as in VLC, Thunderbird, and multi-application workflows. Furthermore, the authors applied IRA to the reinforcement learning of GUI agents, achieving a 34.0% success rate on the OSWorld benchmark, suggesting that the agent can provide effective reward signals for training.

Franklin Analysis

The evidence suggests that IRA’s primary advantage is its ability to move beyond passive observation. By treating evaluation as an interactive process, the agent can verify outcomes that are invisible to standard vision-language models. However, the authors note that because the evaluator interacts with the environment, it may cause state transitions during the verification process. Additionally, while the benchmark trajectories were selected for stability, the authors acknowledge that factors such as UI drift, network variability, and asynchronous application behavior can still cause the replayed state to differ from the original, necessitating the use of task-specific scripts as ground-truth labels for evaluation.