Back to AI Research

AI Research

AnyGoal tests whether world models can plan with familiar ingredients in unfamiliar combinations

Key Takeaways

  • Goal-shifted tasks expose failures that ordinary expert trajectories may leave hidden, even when individual states and actions are familiar.
  • AnyGoal studies a practical question for world models: does learning from expert demonstrations produce dynamics that remain useful when the goal changes?
  • The authors focus on combinations of familiar states and actions that were poorly represented in the training trajectories.
  • A world model predicts how an environment responds to actions, allowing an agent to consider possible plans before acting.
  • AnyGoal introduces goal-shifted variants of established world-model environments, together with an open-source library.

AnyGoal studies a practical question for world models: does learning from expert demonstrations produce dynamics that remain useful when the goal changes? The authors focus on combinations of familiar states and actions that were poorly represented in the training trajectories.

What the benchmark changes

A world model predicts how an environment responds to actions, allowing an agent to consider possible plans before acting. AnyGoal introduces goal-shifted variants of established world-model environments, together with an open-source library.
Individual state factors and actions remain within their training support. What changes is how those ingredients are combined. The task is therefore designed to test whether the model learned reusable dynamics, rather than simply whether it can recognise an entirely new object or action.

What the authors report

Across multiple environments and model families, the study finds failures under these recombinations. It examines representation and dynamics errors as well as downstream prediction and planning.
The distinction matters because good performance along an expert’s usual route can leave other reachable situations under-tested. A planner may need exactly those situations when it pursues a different objective. Familiar components do not guarantee a correct prediction when they interact in an unfamiliar way.

How to use the finding

Franklin’s interpretation is that a world-model evaluation should include deliberate changes to the goal while keeping the environment’s basic ingredients controlled. That helps separate a failure to recombine known dynamics from a failure caused by completely unseen inputs.
For an applied assessment, ask which state-action combinations the demonstrations cover, which the new task requires, and whether errors arise before or during planning. Compare predicted transitions with real environment outcomes instead of judging only whether the final plan looks plausible.
The reported failures do not prove that every world model fails at every new goal. They provide a framework for testing a specific gap between demonstrated behaviour and reusable environment knowledge. Read AnyGoal for the benchmark design and analysis.

Comments