Back to AI Research

AI Research

ExPhy: A Benchmark for Explicit Physical Property L... | AI Research

Key Takeaways

  • ExPhy is a benchmark designed to evaluate how well AI models understand the physical properties—such as mass, friction, and restitution—that govern the movem...
  • Understanding object dynamics requires not only predicting future trajectories but also examining whether a model captures the physical properties that govern motion.
  • However, existing benchmarks rarely expose object-level physical properties as explicit evaluation targets alongside trajectory forecasting.
  • To address this gap, we introduce \emph{ExPhy}, a multi-object trajectory forecasting benchmark containing 24,000 simulated physical scenes with explicit object-level labels for mass, friction, and restitution.
  • We further instantiate \textsc{PhyODE}, a physics-guided model with an explicit property interface that estimates physical properties from observed trajectories and uses them for differentiable future rollout.
Paper AbstractExpand

Understanding object dynamics requires not only predicting future trajectories but also examining whether a model captures the physical properties that govern motion. However, existing benchmarks rarely expose object-level physical properties as explicit evaluation targets alongside trajectory forecasting. To address this gap, we introduce \emph{ExPhy}, a multi-object trajectory forecasting benchmark containing 24,000 simulated physical scenes with explicit object-level labels for mass, friction, and restitution. ExPhy provides observed and future trajectories together with an in-distribution (ID) split and two out-of-distribution (OOD) splits over physical parameters (OOD-Parameter) and initial states (OOD-Initial) for jointly evaluating trajectory forecasting and physical property estimation. We further instantiate \textsc{PhyODE}, a physics-guided model with an explicit property interface that estimates physical properties from observed trajectories and uses them for differentiable future rollout. On the long-horizon OOD-Initial setting, \textsc{PhyODE} reduces ADE and FDE by 33.1\% and 31.0\%, respectively, compared with the strongest baseline. Zero-shot evaluation on ComPhy further assesses cross-benchmark transfer. Property-level analyses reveal that accurate trajectory forecasting does not necessarily imply accurate recovery of the underlying physical properties. Code and data are available at this https URL .

ExPhy is a benchmark designed to evaluate how well AI models understand the physical properties—such as mass, friction, and restitution—that govern the movement of objects. While many existing models can predict where an object will move next, they often do so without capturing the underlying physical rules. This benchmark provides a standardized way to test both trajectory forecasting and the accuracy of a model’s internal physical property estimates.

The ExPhy Benchmark

Rui Wang, Yeteng Wu, Xianlin Zhang, and Mengshi Qi introduced ExPhy to address the lack of explicit physical property targets in current trajectory forecasting benchmarks. The dataset consists of 24,000 simulated scenes featuring multi-object interactions. Each scene includes ground-truth labels for mass, friction, and restitution. The benchmark includes three evaluation protocols:

  • In-Distribution (ID): Tests performance on data similar to the training set.

  • OOD-Parameter: Tests how well a model generalizes to unseen physical properties.

  • OOD-Initial: Tests how well a model handles new initial states, such as different starting locations or velocities.

The PhyODE Model

To demonstrate the benchmark, the authors developed PhyODE, a physics-guided model. Unlike standard models that rely on latent variables, PhyODE uses an explicit interface to estimate physical properties from observed motion. It employs three encoding branches—raw motion, invariant trajectory descriptors, and pairwise interaction relations—to estimate mass, friction, and restitution. These estimates are then fed into a differentiable trajectory solver that combines physics-based dynamics with a residual Neural ODE to predict future movement.

Performance and Findings

In long-horizon OOD-Initial tests, PhyODE reduced Average Displacement Error (ADE) by 33.1% and Final Displacement Error (FDE) by 31.0% compared to the strongest baseline. The authors also performed zero-shot evaluations on the ComPhy benchmark to assess cross-benchmark transfer.

Franklin Analysis

The researchers’ property-level analysis reveals a critical distinction in AI physical reasoning: accurate trajectory forecasting does not guarantee that a model has correctly identified the underlying physical properties. Because PhyODE is trained with both trajectory and property supervision, it provides a clearer view of how well a model's internal "understanding" aligns with the actual physics of the simulator. This suggests that future research should prioritize evaluating these two dimensions—motion prediction and property estimation—as separate but related tasks.

Comments (0)

No comments yet

Be the first to share your thoughts!