DuoCook is an Overcooked-inspired benchmark for visual planning and coordination. A single vision-language model controls two players, choosing high-level actions from top-down observations while the game engine executes navigation. That setup tests centralized coordination, rather than two independent agents learning to communicate with one another.
Shared resources create planning pressure
The DuoCook paper describes 20 maps in five categories. They introduce challenges such as competing for equipment, changing terrain and dependencies where one player must hand something to the other.
Explicit visual cues include a prop legend and pot ingredient counts, while grid destinations help ground spatial actions. These design choices matter when interpreting failure: a model must track the scene and recipe sequence, but the engine handles the mechanics of moving to the chosen destination.
Evaluate more than the final score
The study compares four vision-language models across all maps. It measures task score, order completion, delivery precision, action acceptance and model API expenditure.
The strongest evaluated model reaches 26.2 percent mean order completion. That result indicates substantial difficulty remains within the benchmark, even for the leading system in the comparison. It should not be presented as successful mastery of the game or a general measure of real-world teamwork.
Memory is not an automatic improvement
A notebook ablation keeps six-response action history in both conditions and changes whether the model has a shared, self-authored notebook. The reported score effects differ by model: positive mean differences for Claude and Qwen, and negative differences for GPT and Gemini.
For researchers evaluating memory, the useful question is what information the notebook preserves and how that changes later decisions. Additional stored text can help one model while distracting another. DuoCook provides a controlled environment for examining that behavior, with conclusions tied to its maps, action interface and centralized control arrangement.
Comments