DECE, or Diverse Eurogame Competitive Environments, provides a shared arena for comparing language-model agents across ten strategy games. The paper focuses on decisions shaped by competing goals, shared resources and other players’ choices.
Compare more than one game
The authors describe games with lasting strategic commitments and multiple routes to victory. A common interface and evaluation protocol allow the same agent configuration to be tested across different settings, making it easier to distinguish broad strengths from success in one game.
The main study evaluates 12 model-harness configurations spanning nine models and four execution harnesses. It includes 480 three-player matches across ten games. Main and supplementary studies together comprise 1,032 completed matches, so that larger total should not be confused with the primary comparison alone.
The harness changes the result
The researchers find broad agreement in rankings across games, alongside game-specific strengths. Matched studies also show that the same model can perform differently when its execution harness changes.
That result makes the complete configuration the relevant unit of comparison. A score belongs to a model working through a particular harness and resource setup; it should not automatically become a property of the underlying model in every agent system.
Competitive strength and resource use
The reported performance advantages did not consistently produce better token or cost efficiency. An agent can win more often while using more resources to do so, and that tradeoff belongs in the evaluation.
For teams building game-based agent tests, the paper suggests reporting results across settings and retaining the associated token and cost measurements. The study provides evidence about the tested competitive environments. It does not establish performance on unrelated business tasks, nor does a higher win rate alone show that a configuration is the most economical choice.
Comments