BasketEvent: Understanding Who Did What and When in...

BasketEvent: Understanding Who Did What and When in... | AI Research

Key Takeaways

Basketball is a fast-paced sport where ten players move simultaneously, making it difficult for AI to understand exactly what is happening on the court.
Comprehensive basketball video understanding requires resolving not only what event occurs, but also who is responsible and when the key evidence appears.
To bridge this gap, we introduce BasketEvent, a player- centric basketball event understanding dataset curated from real NBA broadcasts.
In BasketEvent, event labels are grounded to the responsible players, and a manually an- notated subset of 1,000 samples with precise event intervals is provided to evaluate tem- poral evidence localization.
Based on this data, we propose PlayNet, a player-centric reasoning framework that maps basketball videos to player-level event predictions with temporal evidence.

Paper AbstractExpand

Comprehensive basketball video understanding requires resolving not only what event occurs, but also who is responsible and when the key evidence appears. However, exist- ing methods typically treat spatial perception and semantic recognition as isolated tasks, failing to ground events to individual players or pinpoint their temporal boundaries within complex collective dynamics. To bridge this gap, we introduce BasketEvent, a player- centric basketball event understanding dataset curated from real NBA broadcasts. In BasketEvent, event labels are grounded to the responsible players, and a manually an- notated subset of 1,000 samples with precise event intervals is provided to evaluate tem- poral evidence localization. Based on this data, we propose PlayNet, a player-centric reasoning framework that maps basketball videos to player-level event predictions with temporal evidence. Concretely, PlayNet tracks key entities, associates player identities, and reasons about events by modeling player-player, player-ball, and global court inter- actions, while aggregating sparse temporal evidence via gated pooling. Extensive experi- ments demonstrate that PlayNet significantly outperforms representative video-level and crop-based baselines, proving the superiority of player-centric modeling for fine-grained sports video understanding. Our data, code, and models will be made publicly available.

Basketball is a fast-paced sport where ten players move simultaneously, making it difficult for AI to understand exactly what is happening on the court. Current video analysis tools often struggle because they treat the entire video as one block, failing to link specific actions to the individual players responsible for them. The paper "BasketEvent: Understanding Who Did What and When in Basketball Videos" introduces a new approach to solve this by shifting the focus from the whole video to the individual player, ensuring the AI can identify who performed an action, what that action was, and the exact moment it occurred.

A New Benchmark for Basketball

To address the lack of detailed data, the authors created the BasketEvent dataset. This collection includes 35,000 clips from real NBA broadcasts, totaling over 90 hours of footage. Unlike previous datasets that only provide general labels for a video, BasketEvent links specific events—such as a made shot, a steal, or a foul—directly to the athlete involved. The researchers also manually annotated 1,000 samples with precise timestamps, allowing them to test how well an AI can pinpoint the exact moment an event takes place.

How PlayNet Works

The researchers developed a framework called PlayNet to process these videos. Instead of just looking at pixels, PlayNet first tracks every player and the ball as distinct entities. It uses advanced vision models to identify players by their jersey numbers and colors, linking them to real-world rosters. Once the players are identified, the system analyzes their movements and interactions with the ball and other teammates. By modeling these complex relationships, the AI can "reason" about the game, aggregating visual evidence across different moments in the clip to determine the most likely event.

Why Player-Centric Modeling Matters

The study demonstrates that focusing on individual players is significantly more effective than traditional methods that analyze entire video clips or random crops. By grounding events in both the specific player and the specific time, PlayNet provides a much more accurate and interpretable understanding of gameplay. This level of detail is essential for tasks like tactical analysis or automated sports commentary, where knowing exactly which player made a move and when they made it is just as important as knowing the outcome of the play.

Future Availability

The authors emphasize the importance of transparency and reproducibility in sports AI research. To support further development in this field, they have committed to making their dataset, source code, and trained models publicly available. This will allow other researchers to build upon their work and continue improving how machines interpret the complex, multi-agent dynamics of professional basketball.