FootsiesGym: A Fighting Game Benchmark for Two-Play... | AI Research

FootsiesGym is an open-source research environment designed to study how artificial intelligence learns to play fighting games. By focusing on the "neutral game"—the phase where players maneuver and look for openings without having a clear advantage—the researchers aim to provide a benchmark that captures the complex, cyclic, and non-transitive nature of fighting games. While existing benchmarks are either too simple to represent real-world strategy or too computationally expensive to train, FootsiesGym offers a middle ground that is both strategically rich and efficient enough to run on standard hardware.

The Mechanics of the Neutral Game

Fighting games are often compared to a high-speed game of rock-paper-scissors, where players must constantly predict and counter their opponent's movements. FootsiesGym strips away complex mechanics like long attack combos to focus entirely on this strategic cycle. The environment is built on the game Footsies, where players manage spacing, timing, and special attacks. Because the game is real-time and features imperfect information—meaning players cannot see their opponent's internal state, such as charge timers—it forces agents to learn how to predict intent rather than simply reacting to visual cues.

High-Throughput Training

A key contribution of this work is the development of a vectorized simulator that allows researchers to train agents quickly. By decoupling the game logic from the Unity rendering engine and running it in a headless mode, the environment can process tens of thousands of game steps per second on a single workstation. This efficiency enables researchers to iterate on reinforcement learning algorithms rapidly, making it easier to conduct experiments that would otherwise require massive computing clusters.

Evaluating AI Performance

The researchers tested several reinforcement learning algorithms, including variations of Proximal Policy Optimization (PPO), EMAgnet, and Prioritized Fictitious Self-Play (PFSP). Their findings highlight a common challenge in AI development: agents often become highly skilled at winning but lose their "human-like" quality. For example, as agents improved, they became increasingly reactive and stopped initiating engagements, preferring to wait for the opponent to make a mistake. Furthermore, the agents struggled to discover and utilize complex special attacks, as these moves require specific, multi-step sequences that are difficult to learn through random exploration.

Future Research Directions

FootsiesGym serves as a platform for exploring several open questions in AI. Beyond just maximizing win rates, the environment allows researchers to study how to create agents that are not only strong but also engaging to play against. It also provides a testbed for game-theoretic training methods, allowing scientists to investigate how to prevent agents from abandoning useful but difficult-to-learn strategies. Because the environment can be exported to a web-based format, it also opens the door for human-in-the-loop experiments, where researchers can directly evaluate whether an AI’s playstyle feels natural or fun to human participants.