Puzzle study finds AI uncertainty helps collaborators only when warnings target real error

An instruction such as 'place the pink piece' may seem clear to the person giving it while leaving an AI partner with several plausible choices. Referential Uncertainty in Human–AI Collaboration examines whether an agent recognizes that ambiguity and communicates enough for a person to catch a mistake.
The study uses a collaborative puzzle with twenty-four visually ambiguous pieces. A human Helper sees a target pattern, while the AI Worker sees the available pieces. Each partner lacks information the other has, making shared understanding part of the task.

Confidence is not the same as a useful warning

The researchers compare raw action-token probabilities with a separately elicited distribution over candidate pieces. They report that the latter is better calibrated: expected calibration error is 0.15, versus 0.44 for raw probabilities. Raw probabilities average 0.97 confidence despite only 53% accuracy in that analysis.
A better uncertainty estimate still has blind spots. Across GPT-4.1, GPT-5, and GPT-5.5, elicited uncertainty increases as instructions become vaguer. Competing, easily confused pieces increase errors without producing a corresponding increase in that signal.
Models also seldom ask for clarification, doing so on only 3.5% to 16.7% of instruction turns in the reported image-modality comparison. An agent may therefore have an imperfect uncertainty signal and fail to communicate even the uncertainty it does represent.

What people accepted in the controlled study

The human study includes 210 participants who judge recorded interactions from the Helper's perspective. Conditions vary whether participants see the Worker's board, receive a precise description of its chosen piece, or read an uncertainty cue.
With default messages and limited shared awareness, participants accept 78% of wrong placements. The authors report that precise descriptions and well-targeted hedges reduce wrong-move acceptance to 36%, while largely preserving acceptance of correct moves.
The strongest hedge condition is important to distinguish from a deployable feature. It uses ground-truth correctness to place warnings on a subset of incorrect selections. The model does not independently know which of those choices are wrong.
A separate self-hedged condition instead uses the Worker's own elicited belief entropy. That more realistic signal inherits the model's weaknesses and can reduce appropriate acceptance without helping people distinguish errors reliably.

A communication result, not a live deployment guarantee

Participants report judgments and intended interventions on recorded turns. Their decisions do not change the puzzle interaction or cause the Worker to respond again. The study therefore measures how people interpret the presented evidence, rather than demonstrating improved completion rates in an ongoing collaboration.
The findings suggest a specific interface problem: generic confidence or a blanket disclaimer may not tell a collaborator what the agent actually selected. A precise description can expose a mismatch, while a warning needs accurate targeting to add value. Simply making an agent sound less certain does not establish that its human partner will make better decisions.