A supplied screenshot may not settle completion
The VeriCUA paper studies how to verify computer-using agents across desktop, mobile and web tasks. A vision-language judge can inspect the observations it receives, while a generated checking program can inspect accessible application state. Either view may omit the evidence needed to decide whether the task is complete.
That creates a problem beyond choosing a stronger judge. If the decisive information is absent, a confident answer can still be unsupported. Verification needs a way to find missing evidence, not only classify the material already supplied.
Build a rubric, then investigate the gaps
VeriCUA first constructs a task rubric from the available context. It then chooses GUI or code checks for unresolved requirements and combines their findings under explicit evidence rules.
The framework uses this shared procedure in online and offline settings, rather than designing an entirely separate verification process for every platform and access arrangement. The method’s argument depends on complementary observations: a code check and a visible interaction may answer different parts of the same completion question.
Agreement is an evaluation result
The authors report improved agreement with reference labels compared with their baselines, including larger offline gains on hard cases and stronger rejection of false completion claims online. Those results concern the tested settings and reference judgments. They do not establish that an exploring verifier will always find the decisive evidence or that the reference labels are infallible.
For teams evaluating computer agents, a useful comparison would include successful-looking trajectories that leave one requirement unmet. Examine what the verifier actually checks, which requirements remain uncertain and what additional exploration costs. VeriCUA offers a concrete approach to gathering completion evidence, while preserving the need to judge whether that evidence really supports the final verdict.
Comments