A forecaster may fit recorded outcomes while responding incorrectly to a proposed change in action. That matters if someone wants to use the model to compare plans that have never been executed. A new time-series study tests both predictive error and whether changing an action moves a forecast in the direction expected from a known mechanism.
On the Divergence of Accuracy and Mechanism Consistency in Time Series World Models reports that the lowest-error configuration reaches chance or worse on its directional consistency metric in four of five evaluated datasets with declared mechanisms. Better forecasting architecture alone does not resolve that gap in the authors' experiments.
Defining what the controller can change
The paper separates observed channels into state, actions and exogenous inputs. State is what the model predicts. Actions are channels a controller sets, while exogenous inputs influence the system without being under its control.
In a greenhouse example, heating-pipe temperature and vent aperture are actions, outdoor temperature and irradiance are exogenous inputs, and indoor conditions are state. The authors also distinguish continuous actions, discrete operating modes and one-off events. Keeping those roles explicit defines what a changed plan means.
Their benchmark combines eight public datasets from horticulture, building climate, district heating, wastewater treatment, anesthesia, glucose monitoring and intensive care. The experiments vary prediction space, plan fusion and temporal plan encoding across seven forecasting backbones and five seeds. These datasets provide recorded actions and outcomes; they do not furnish observed outcomes for every hypothetical alternative plan.
Accuracy improves without mechanism agreement
The authors report that predicting in a frozen autoencoder latent space lowers mean absolute error by 9.9% on average relative to observation space. Output-side gated fusion lowers it by 12.7% relative to input concatenation. Both choices improve error across the eight datasets in the reported sweeps. Temporal plan encoding changes average error by at most 2.2%.
Those are comparisons within the paper's protocol. The experiments select checkpoints by validation loss and report validation MAE on scaled state channels, so the percentages should not be presented as universal gains on unseen deployments.
Mechanism consistency adds a separate test. The researchers declare 21 action-state relationships before training, then shift one continuous action and check the sign of the model's response. They score five datasets; the glucose datasets have event actions outside the defined quantile-shift test, and the wastewater mechanism is excluded.
The metric tests agreement with a declared direction, not the magnitude or complete causal validity of an intervention effect. Its scoring also excludes windows where the plan or forecast does not move enough to count. That definition matters when interpreting a high consistency score.
Directional supervision changes the objective
The paper proposes a loss term that penalizes the wrong-signed part of a predicted response to a shifted action. It leaves the network unchanged and supplies the known direction as supervision. The authors report improved consistency on penalized mechanisms without a change in MAE.
This provides a way to train for a property that ordinary forecast error does not enforce. It also means the directional knowledge comes from outside the model, rather than being discovered or independently validated by the consistency score.
The study's distinction is useful for evaluating planning models: recorded-plan accuracy and response to alternative actions deserve separate measurements. The clinical datasets do not establish a safe treatment-planning system, and matching a known sign does not validate a dose, timing or outcome prediction for a patient. The reported result is a benchmark finding about model behavior under specified shifts.
Comments