Counterfactual World Models Help Robots Learn From Surprises

World models are the part of a physical AI system that imagines what happens next. Give one an action, and it predicts the consequence. A new research method described by The Register on November 9, 2026, adds something these models have mostly lacked: the ability to look back after a surprise and work out what would have happened had the robot acted differently. The authors call them counterfactual world models, and the idea is refreshingly simple.

• The paper, "Counterfactual World Models: Natura Non Facit Saltus," comes from Randall Balestriero, Kerda Varaku and Turing Award winner Yann LeCun
• According to the researchers, picking actions this way reached the goal in 35 to 69 percent of simulated events, versus 14 to 23 percent for a planner that ignores the observed outcome
• With two more attempts on the same event, success rose to 51 to 97 percent in those simulated tasks
• Experiments used off-the-shelf DINOv2 and MAE encoders without fine-tuning, plus a small trained predictor

What Is a Counterfactual World Model?

A standard world model predicts the average next state for a given action. That works well until something the robot cannot see, like a hidden current, a heavy load or a weak motor, pushes the outcome off course. At that point the useful question changes from what was expected to what would have worked instead.

Balestriero explained the method to The Register with a boat example. A model predicts where a boat ends up after steering left, but it actually lands two meters further downstream. Those two meters are the signature of the hidden current. To estimate where the boat would have been had it steered right, you take the model's prediction for steering right and add the same two meters. In his words, the counterfactual is what actually happened, plus the difference between the model's predictions for the two actions.

Why Does Physics Make the Trick Work?

The title borrows Leibniz's principle that nature makes no leaps. Over a short time step, the effect of an action and the effect of a hidden disturbance each grow in proportion to the duration, while their interaction grows only with its square. So over short steps they simply add up, much like forces. The researchers say they prove that the error from this rule shrinks with the square of the time step, and that no method using the same ordinary data can beat that rate in the worst case.

The team first tried teaching the model to infer the hidden disturbance directly, but found that approach fragile. Working out why led to the theory, and to the realization that the gap between prediction and reality already carries the needed information. Control engineers have applied similar corrections to hand-built models for decades; the contribution here is explaining why it holds for learned world models that see through images.

How Could Robots Use This Machine Learning Idea?

Balestriero sees three uses: retrying an action after a surprise, adapting to persistent faults such as a miscalibrated motor, and helping people understand why a failure happened, including telling a bad decision apart from bad luck. The technique is not tied to one architecture, though the researchers say Joint Embedding Predictive Architecture models, which LeCun and colleagues introduced in 2023, are especially easy to adapt.

The paper was shared with The Register ahead of its arXiv release, expected this week, so the figures above are the researchers' own and have not yet been independently reproduced. Still, it is an elegant piece of robotics research that fits neatly beside recent work like DeepMind's Genie 4 world model and Ai2's ToyBlocks robot learning simulator. For more research like this, browse our AI coverage.

Sources: The Register — November 9, 2026; I-JEPA paper on arXiv — January 2023 (background on JEPA).