Back to AI Research

AI Research

TeamGym separates communication and retrieval failures in agent teamwork

Key Takeaways

  • A benchmark controls how necessary facts are distributed and tests training on the final team decision.
  • [TeamGym](https://arxiv.org/abs/2610.37571) evaluates language-model agents making decisions from information distributed across participants.
  • Its tasks draw on legal, financial, medical and software-engineering corpora, with controls designed to reveal where a team loses necessary evidence.
  • ## Identify the facts needed for a decision
  • The framework defines pivotal information as a minimal set of facts whose individual elements are necessary and whose combined content is sufficient for the correct final decision.

TeamGym evaluates language-model agents making decisions from information distributed across participants. Its tasks draw on legal, financial, medical and software-engineering corpora, with controls designed to reveal where a team loses necessary evidence.

Identify the facts needed for a decision

The framework defines pivotal information as a minimal set of facts whose individual elements are necessary and whose combined content is sufficient for the correct final decision. Controlling who receives those facts lets the researchers distinguish information-sharing failures from problems finding evidence in the first place.
This is a property of how the benchmark tasks are constructed. It does not mean that every real operational decision has a similarly complete and identifiable set of facts ready to distribute.

Two different gaps

The study examines eight language models across twelve source corpora. Distributing pivotal information across participants reduces success by an average 22.9 percentage points relative to providing the same information to a single model.
Requiring participants to find and extract the pivotal information from source material introduces a further reported 19.9-point gap compared with giving it directly. These are comparisons under different information conditions. They should retain their baselines rather than be casually combined into one universal loss figure.

Train on the final decision

The authors also use reinforcement learning with a reward based on final decision quality. Across two policy-training experiments, they report improvements alongside emerging coordination in participation, speaker selection and communication.
For teams evaluating a multi-agent design, TeamGym offers a way to test whether failures arise before or during discussion. A useful baseline gives one model the combined evidence, then compares the distributed version under the same task conditions. More participants alone do not establish better decisions; the relevant evidence is what the team finds, communicates and ultimately uses.

Comments