Benchmarking World-Model Learning
with Environment-Level Queries
Archana Warrier · Dat Nguyen · Michelangelo Naim · Moksh Jain · Yichao Liang · Karen Schroeder · Cambridge Yang · Joshua B. Tenenbaum · Sebastian Vollmer · Kevin Ellis · Zenna Tavares
ICML 2026 · PMLR 306 · TAIC'26, Seoul
Roadmap
Where we are going, in five parts
A world model is an agent's flexible, predictive, and counterfactual understanding of how its environment works.
Weisberg & Gopnik 2013, Cognitive Science · LeCun 2022, A Path Towards Autonomous Machine Intelligence
Why world models matter
Learning a world model is a pivotal open problem for the next step in AI.
A worked example. Someone who cooks regularly builds an internal model of their kitchen: where tools live, how appliances behave. That one model is general-purpose. It supports many different everyday capabilities about the same environment, not just a single task.
Estimate how long the hidden contents of a covered pot will take to finish cooking, from the steam and the elapsed time.
Recognize and adapt to changes in the kitchen, such as a knife that has been moved to a different drawer.
Plan a sequence of actions to complete a set of recipes, ordering steps so everything comes together.
2. How do we currently evaluate world-model learning?
2.1 Non-interactive evaluation - Testing rule inference from static examples
Two classic benchmarks. ARC (Chollet, 2019): infer a grid rule from input to output examples. CLEVR (Johnson et al., 2017): answer a compositional question about one rendered scene. In both, you never act in the world.
The common thread. Both are static: you infer from fixed examples or one rendered image, and never act in the environment to gather its dynamics.
2.2 Representation-based evaluation - Testing through a fixed output format
The agent must produce a fixed output format, then a format-specific proxy scores it. Moving MNIST (Srivastava et al., 2015): predict the next video frame, scored by reconstruction error.
Low pixel error: scored well.
Higher pixel error: scored worse.
scored by reconstruction error (Moving MNIST, BAIR robot pushing)
scored by LLM-based evaluation (DiscoveryWorld)
scored by predicate accuracy (CLEVRER, CATER, CausalWorld)
The common thread. These proxies are often inadequate: they measure fit to the format, not the world model, and they lock out agents (and humans) whose knowledge is not expressible in that format.
2.3 Gym-like evaluation - Testing task success through rewards
Decision-making with explicit rewards: Atari / ALE (Bellemare et al., 2013), OpenAI Gym, ProcGen, NetHack. The score IS the reward counter. Schematic below, in the style of Breakout.
The common thread. Reward measures task success, not world-model quality: high performance may come from a memorized policy rather than a generalizable grasp of the environment's structure.
2.4 Unsupervised RL evaluation - Testing in the same world after reward-free exploration
The common thread. Two phases, but one environment and reward-based evaluation: properties you cannot observe through action-reward sequences, like structure and counterfactuals, go untested. WorldTest's move: compile each query into a modified challenge environment, so the test itself probes what the agent knows about the world.
The landscape of world model learning evaluation
Four families of benchmarks each probe an agent's knowledge in an incomplete way.
Infer hidden rules from a few static examples (ARC, RAVEN, CLEVR variants).
Captures: environment-level reasoning: rule induction, concept induction, causal reasoning.
Misses: no learning through interaction with a dynamic environment.
Require a fixed output format (next frames, programs, causal graphs) scored by format-specific proxies.
Captures: targeted probes of particular aspects of world understanding.
Misses: the proxies are often inadequate, so they cannot faithfully measure the learned model or benchmark agents against humans.
Decision-making with explicit rewards (Atari, Gym, Procgen, NetHack).
Captures: learning through interaction, with well-defined, automatically scored objectives.
Misses: measures task success, not world-model quality; a memorized policy can win.
Two-phase protocols (URLB): explore without objectives, then face downstream tasks.
Captures: the separation of reward-free learning from downstream evaluation.
Misses: both phases use the same environment, and evaluation sees only action-reward sequences, so structural and counterfactual understanding goes untested.
The key idea: environment-level queries
Questions whose answers depend on the entire environment, not on one observed trajectory
Definition. An environment-level query is a question about a property of the whole environment. Answering it requires understanding the underlying rules, not just replaying what was seen.
What is hidden behind an occlusion. Infer the parts of the environment you cannot directly see.
Detect a change in the environment's dynamics. Notice when a rule of the world has shifted.
Determine whether one state is reachable from another. Reason about the global structure of what is possible.
These are the kinds of questions a flexible world model should handle. The next step is a way to score them fairly across very different agents.
WorldTest: a two-phase, behavior-based protocol
We give the agent a reward-free environment \( \mathcal{M} \), a POMDP whose dynamics it does not know. It explores \( \mathcal{M} \) freely, with no external rewards; at any time it may reset to the initial state or proceed to the test. From its interaction history it constructs an internal model \( \widehat{\mathcal{M}} \).
\( \widehat{\mathcal{M}} \) can be any internal representation: a program, a latent code, a neural net. The protocol never prescribes or inspects it.
The protocol instantiates an environment-level query: it samples hidden task parameters \( \xi \) and transforms the base environment into a derived challenge environment \( \mathcal{M}' \) with an explicit objective \( R \) and horizon \( H \). The score depends only on the agent's behavior in \( \mathcal{M}' \).
\( (\mathcal{M}', R, H) = \tau(\mathcal{M}, \xi), \quad \xi \sim P_{\Xi} \)
solving the task ≡ answering the query
Scored by behavior alone. No peeking inside the agent, no manual or AI judging, so humans and AI compare on equal terms.
Reward-free exploration encourages learning transferable dynamics, not optimizing one task-specific objective.
The test compiles each query into an objective, directly measuring whether the model supports downstream inference and planning.
Try it yourself
A live AutumnBench task, running on the real platform. Explore reward-free, reset at will, then take the test the 517 participants and five frontier models took.
The benchmark, live
The Autumn language
Why Autumn. It lets you specify environments succinctly, expressively, and extensibly. Crucially, one specification drives both a text-based interface for AI agents and a browser-based graphical interface for humans, so the exact same world is played by both.
(on cond body) · sand.sexp / ants.sexp; from sand.sexp: conditional rules
(on (clicked sandButton)
(= clickType "sand"))
(on (& (clicked) (isFreePos click)
(== clickType "water"))
(= water (addObj water ...)))
; from ants.sexp: spatial, temporal, stochastic
(closest obj foods)
(filter (--> obj (! (intersects obj (prev ants))))
(prev foods))
(randomPositions GRID_SIZE 2)(on cond body) is the conditional rule form: the guard is evaluated against the current state each tick and the body fires when it holds. The stdlib adds spatial operators (closest, intersects), temporal reads (prev reads the last tick), and stochastic primitives (randomPositions).
This GUI is what humans play; AI agents get the same world as text (a 2D array of color strings). One world, two interfaces, fair comparison.
How it was run: same benchmark, two interfaces
Humans play the exact same world in a browser; AI models play it in text. The behavior-only scoring lets us compare them directly.
517 participants recruited via Prolific and screened for attention and color blindness.
Five frontier reasoning models: Claude 4 Sonnet, Gemini 2.5 Pro, Gemini 2.5 Flash, o3, Qwen3-235b-a22b-thinking-2507. Each is given its full interaction history, the current grid state, the available actions, and a task-type description.
A simulator agent with ground-truth program access is included only as a reference point, not as a competitor. It shows what is achievable when the underlying rules are known.
Result
Panel (a): models did better on stochastic environments than deterministic ones; humans were nearly identical across both. Bars carry standard-error whiskers; panel (c) shows one dot per environment, circle = mean.
More compute is not the fix
Spending more on each problem rarely closed the gap to human performance
The test. Rank the five models by cost-per-problem (Gemini Flash, Qwen3, Gemini Pro, Claude Sonnet, o3, ascending) and check whether score climbs with budget. In 25 of 43 environments it does; in 18 of 43 (42%) it plateaus or decreases. No environment is solved perfectly by the cheapest model. By task: MFP improves in 16 of 43, planning in 16, change detection in only 14. (Appendix E.3.3)
Model scores improve with compute on masked frame prediction and change detection here. Click the panel, then arrow keys move mario.
bbq, carrace, chinese_checkers, and crystallization show no improvement in any task: their difficulty sits at extremes that additional compute cannot address. Click the grill to light it, the yellow button to add gas.
In most cases more compute did not help. This points to reasoning limitations beyond scaling, not a shortage of compute. Both worlds above run live on the Autumn WASM interpreter, loaded from the official benchmark release (Zenodo 19498269; mario = N2NTD, bbq = 27VWC).
Humans explore by experimenting
During exploration the agent can reset the environment to its initial state at will. Humans treat reset as an experimental tool; models mostly do not.
The signature. Humans reset at least once in every environment.
Reset, and run it again
A real human trajectory on the mario environment, free-exploration phase.
This is the top-scoring human on the mario planning task. During free exploration, before the scored test, they reset the environment and ran it again, five times over. For a person, reset is an experimental tool: replay the setup and watch how the rules behave.
Rendered from the AutumnBench human-participant trajectories (verified winner, score 0.985). The five runs are real; playback is time-compressed.
Inside the scratchpad: the snake that “fell”
A snake: arrow keys set its direction (click the panel first); eating the pink food makes it grow. There is no gravity.
“Key observation: The green object … has fallen down one row and the bottom part moved right one column. This looks like gravity or falling behavior!”
“Observation: Both left and right arrow keys don't move the falling green object.”
“Action taken: click 1 9 … Result: MAJOR CHANGE! The green object has transformed!”
“Option 3 looks most consistent with: vertical green object at column 2 … pink object at row 1, col 3.” Its model never learned that the snake eats food and grows.
What happened. Claude 4 Sonnet's actual reasoning trace. A prior key press set the snake's direction; the model observed every transition correctly, attributed the motion to gravity, kept the belief through contradicting evidence, and answered the test confidently from the wrong model. Full traces released at zenodo.org/records/17728515.
Watch the difference
The same Autumn environment, played side by side.
The contrast the clip makes visible is the one the numbers make precise. Humans treat reset as an experiment, re-running controlled variations, and spend about a quarter of their actions on resets and no-ops. Every model stays under 7 percent and rarely resets at all.
Source: Basis Research Institute · autumn.basis.ai
Humans update their beliefs
Reasoning models often fail to update their beliefs when faced with contradictory evidence, especially in masked frame prediction.
Even after they notice that test observations contradict the rules they learned, they keep predicting from the original rules.
Humans reach lower normalized perplexity over their interaction: a lower area-under-the-curve of perplexity as exploration proceeds.
In plain words, they are less surprised by what comes next. Their learning is more targeted and better calibrated, so they revise toward the rules that actually hold.
“Instead, people select strategies in an adaptive fashion that trades off their expected performance and cognitive effort.”
“…learners who freely interacted with the physical system selectively produced evidence that revealed the physical property consistent with their inquiry goal.”
Closing the gap likely needs not only better priors but advances in flexible belief updating, not just more compute.
What the gap really means
The human advantage is not mainly about raw knowledge; it traces to two capabilities the models lack
Conclusion. Current frontier models lack the flexible, predictive understanding that characterizes human-like world models. The gap is not mainly a knowledge gap. It traces to two capabilities: how an agent designs experiments to test its hypotheses, and how it revises its beliefs when the evidence contradicts them.
Humans use resets and interventions to test hypotheses: they reset in every environment and replay similar action subsequences around resets. Models often skip resets entirely and spend almost none of their actions probing the world.
Humans revise their beliefs under contradiction and reach lower perplexity over interaction. Models often keep relying on the rules they first learned, even after noticing that the test observations contradict them.
More compute does not supply either capability: 42% of environments saw no benefit from extra compute. The missing pieces are behavioral, not just scale.
Recap: test, benchmark, finding
One protocol, one benchmark, one clear result about where frontier models stand
A two-phase, representation-agnostic protocol: explore reward-free, then answer an environment-level query compiled into an explicit objective. Scored by behavior alone, so any agent type compares fairly.
AutumnBench: 43 environments, 129 tasks, three query families (masked frame prediction, change detection, planning), playable by both humans and AI.
517 humans substantially outperform all five frontier reasoning models on every environment and every task type, and more compute does not close the gap.
In sum. WorldTest is a behavior-based, two-phase, representation-agnostic protocol that evaluates world-model learning via environment-level queries. AutumnBench instantiates it; 517 humans substantially outperform five frontier reasoning models, and more compute does not close the gap.
Takeaway and resources
Warrier et al., ICML 2026 · public dataset, web GUI, interpreter, and baselines all available
Bottom line. Current frontier models lack the flexible, predictive understanding of human-like world models. Closing the gap likely needs better priors and advances in strategic experimental design, uncertainty quantification, and flexible belief updating, not just more compute.
The public AutumnBench dataset, browser GUI, Autumn interpreter (WASM / Python / Julia), and reasoning-model baselines are all available.
The team
Eleven authors across Basis Research Institute, DFKI, Harvard, Mila / Universite de Montreal, Cambridge, MIT, and Cornell
Thank you. Try AutumnBench at autumn.basis.ai.
Other projects
@defcomponent
def OpenParametricHand(self,
params=None, n_fingers=4,
n_thumbs=1):
p = params or HandParams.default()
self.palm = Palm(p, n_fingers,
n_thumbs)
self.fingers = [Digit(p.fingers[i], ...)
for i in range(n_fingers)]
# ... thumbs, tendons, actuators ...A language to describe robot morphology and control.
from effectful.handlers.llm import Template
@Template.define
def limerick(theme: str) -> str:
"""Write a limerick on the theme of {theme}."""
raise NotHandled
with handler(LiteLLMProvider()):
print(limerick("autumn"))Algebraic effects for Python: the docstring is the prompt, the signature is the contract, and a handler decides which LLM answers.
Learns abstract models of dynamic worlds for robot planning: symbolic state and causal processes (agent actions plus exogenous mechanisms) via variational Bayesian inference with LLM proposals.
Bringing the baby robot to the real world.
