The DOI matches its Crossref record; 50 of the 50 references it cites were found in a registry or an index; 18 of the 56 sources its claims cite were checked for support.
Mastering diverse control tasks through world models
Open accessAbstract
DreamerV3 is a general algorithm that outperforms specialized methods across over 150 diverse tasks, with a single configuration. Dreamer learns a model of the environment and improves its behavior by imagining future scenarios. Robustness techniques based on normalization, balancing, and transformations enable stable learning across domains.
Overview
CoChat Summary
TL;DR
DreamerV3 is a general model-based reinforcement-learning algorithm that learns a world model and trains actor and critic in imagination, using robustness techniques to work with fixed hyperparameters across over 150 tasks in 8 domains. It matches or exceeds specialized experts and is, to the authors’ knowledge, the first algorithm to collect diamonds in Minecraft from scratch without human data or curricula.
Key findings
- Across 8 domains and over 150 tasks with fixed hyperparameters, Dreamer matches or exceeds tuned experts and outperforms PPO in all domains: Atari (57 games, 200 million frames) beats MuZero, Rainbow and IQN; ProcGen (16 games, 50 million frames) beats PPG and Rainbow; DMLab (30 tasks, 100 million frames) exceeds IMPALA and R2D2+ at 1 billion steps, a data-efficiency gain of over 1,000%; Atari100k (26 games, 400,000 frames) beats IRIS, TWM, SimPLe and SPR; Proprio Control Suite (20 tasks, 1 million steps) matches DMPO and TD-MPC2; Visual Control Suite (20 tasks) sets state of the art over DrQ-v2 and TD-MPC2; BSuite (23 environments, 468 configurations) sets state of the art over Boot DQN.
- In Minecraft, all Dreamer agents discover diamonds within 100 million environment steps, completing all 12 milestones from sparse rewards without human data or curricula; Dreamer uses 1 GPU for 9 days, whereas VPT used 720 GPUs for 9 days.
- Ablations on 14 tasks show all robustness techniques contribute, most notably KL balancing and free bits, followed by return normalization and symexp two-hot regression; performance rests predominantly on the unsupervised reconstruction loss of the world model.
- Scaling from 12 million to 400 million parameters increases task performance monotonically and reduces required environment interaction; higher replay ratios also predictably improve performance.
Methods
- DreamerV3 consists of three neural networks trained concurrently from replayed experience: a recurrent state-space world model that encodes sensory inputs into discrete representations, predicts future representations and rewards, and reconstructs inputs; a critic that predicts return distributions; and an actor that selects actions to maximize return.
- Robustness techniques include symlog observation and target transforms, symexp two-hot loss for reward and critic, KL balancing with free bits, 1% unimix for categorical distributions, percentile return normalization, block GRU, RMSNorm, SiLU activations, adaptive gradient clipping, LaProp optimizer, and a uniform replay buffer with online queue; hyperparameters are fixed across all benchmarks.
- Evaluation spans 8 domains with over 150 tasks, comparing against PPO and specialized experts; seeds are 5 per benchmark except 10 for BSuite and Minecraft; all agents train on a single Nvidia A100 GPU. Minecraft uses MineRL v0.4.4, a 64×64×3 first-person image, inventory and scalar observations, 25 categorical actions, sparse rewards for 12 milestones, episodes up to 36,000 steps (30 minutes), and accelerated block breaking.
Limitations
- Stated: DMLab baselines were not designed for data efficiency and were compared at 1 billion steps while Dreamer used 100 million; EfficientZero resets levels early, making comparison difficult. Minecraft uses accelerated block breaking and abstract crafting actions, and the authors did not investigate whether the heart reward is helpful.
- Stated: uniform replay was chosen for ease of implementation even though prioritized replay improved Dreamer; no annealing, prioritized replay, weight decay or dropout were used.
- Evident: Minecraft environment modifications and the categorical action space mean results are not directly comparable to unmodified Minecraft or to VPT/Voyager setups; evaluation budgets were limited by computational cost, such as DMLab at 100 million rather than 10 billion steps and ProcGen at 50 million steps.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures








Verification
Claims the paper makes
13 Valid · 7 Not valid · 27 Not checked · 57 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“Although intuitively appealing, robustly learning and leveraging world models to achieve strong task performance has been an open problem18.”
§ Mastering diverse control tasks through world models > Main
What the sources say
- Supportedcites [18] Kaiser, L. et al. Model-based reinforcement learning for Atari. In Proc. International Conference on Learning Representations (ICLR, 2020). source
The Introduction (p. 1) describes the appeal of learning policies from environment models. The Conclusions (p. 10) identify both lower final scores than leading model-free methods and substantial variation between runs, and name better models and robustness as future work. Together, these support the claim that reliably using world models for strong performance remained an open challenge. (LLM, conf=0.88)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Dreamer also outperforms the widely used expert algorithms Rainbow36 and IQN37.”
§ Mastering diverse control tasks through world models > Evaluation > Benchmarks
What the sources say
- Supportedcites [36] Hessel, M. et al. Rainbow: combining improvements in deep reinforcement learning. In Proc. Thirty-Second AAAI Conference on Artificial Intelligence and Thirtieth Innovative Applications of Artificial Intelligence Conference and Eighth AAAI Symposium on Educational Advances in Artificial Intelligence 3215–3222 (Association for Computing Machinery, 2018). sourceExperimental — accuracy not yet measured
The citation is attached to “Rainbow” in the citing sentence. The introduction (p. 1) identifies the paper’s agent as Rainbow and reports its performance on 57 Atari games, supporting its use as a citation identifying that algorithm. This source is not being cited for the Dreamer comparison or for IQN. (LLM, conf=0.90)
- Could not verifycites [37] Dabney, W., Ostrovski, G., Silver, D. & Munos, R. Implicit quantile networks for distributional reinforcement learning. In Proc. 35th International Conference on Machine Learning 1096–1105 (PMLR, 2018). sourceExperimental — accuracy not yet measured
The Evaluation on Atari 2600 (pp. 6–7) compares QR-DQN with earlier agents, not Dreamer with Rainbow or IQN. Searches for Dreamer, Rainbow, and IQN found no passage in our copy addressing the claimed comparison. (LLM, conf=0.76)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Owing to these obstacles, previous approaches resorted to using human expert data and domain-specific curricula20,21.”
§ Mastering diverse control tasks through world models > Main
What the sources say
- Could not verifycites [20] Kanitscheider, I. et al. Multi-task curriculum learning in a complex, visual, hard-exploration domain: Minecraft. Preprint at https://arxiv.org/abs/2106.14876 (2021). sourceExperimental — accuracy not yet measured
Our copy of Kanitscheider et al. (2021) contains no statement characterising previous Minecraft approaches as relying on human expert data or domain-specific curricula. The Introduction (p. 2) only says their methods were built on the MineRL platform [1] (a dataset of demonstrations used here as an environment), and the Related work section (p. 11) surveys automated curriculum learning in general (success-probability/regret thresholds, Curiosity Search [2,3]), not prior Minecraft work; the contribution statement (p. 1) even concedes a thorough literature review is missing. Searched: 'human expert data / demonstrations', 'domain-specific curriculum / prior work', 'imitation learning Minecraft', 'handcrafted curriculum obstacles' — no decisive passage. (LLM, conf=0.70)
- Supportedcites [21] Baker, B. et al. Video pretraining (VPT): learning to act by watching unlabeled online videos. In Proc. 36th International Conference on Neural Information Processing Systems 24639–24654 (Association for Computing Machinery, 2022). source
The Related Work section (p. 3) describes prior Minecraft work using automated curriculum learning, and the References (pp. 11–12) identify prior Minecraft studies based on demonstrations. Together, these support the broad characterization of approaches using demonstration data and Minecraft-specific curricula. (LLM, conf=0.82)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Dreamer matches the state of the art on this benchmark, such as DMPO31 and TD-MPC243.”
§ Mastering diverse control tasks through world models > Evaluation > Benchmarks
What the sources say
- Partly supportedcites [31] Abdolmaleki, A. et al. Maximum a posteriori policy optimisation. In Proc. International Conference on Learning Representations (ICLR, 2018). sourceExperimental — accuracy not yet measured
Sections 5.1 (p. 7) and 5.1.2 (pp. 8–9) present MPO as a competitive algorithm on Control Suite tasks, and Section 5.2 (p. 9) describes a parallel variant. The cited paper does not identify that variant as DMPO or compare Dreamer with DMPO on the Proprio Control Suite benchmark, so it supports only the broader control-performance context. (LLM, conf=0.81)
- Not matchedcites [43] Hansen, N., Su, H. & Wang, X. TD-MPC2: scalable, robust world models for continuous control. In Proc. International Conference on Learning Representations (ICLR, 2024).
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plainn = 20- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Notably, larger models not only achieve higher scores but also require less interaction to solve a task, offering practitioners a predictable way to increase performance and data efficiency.”
§ Mastering diverse control tasks through world models > Main
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
36 more claims
Create a free account to see all 104 claims and to verify the 10 references still open.
- It may state two separate claims as one. 39
- Probably not a substantial claim. 36
- The quote only partly states this claim. 2
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Their latest iteration, DreamerV3 [71], adds robust normalization and balancing techniques, solving over 150 tasks-including diamond collection in Minecraft-without human data or domain-specific tuning.”
From Understanding World or Predicting Future? A Comprehensive Survey of World Models (2026) · cites this paper as [71]
The cited source says what the claim attributes to it.
In the citing paper
“World models predict the effects of candidate actions on subsequent states or outcomes [6-8], enabling prospective evaluation and planning.”
From RSI-Anything: Unlocking Recursive Self-Improvement with World Models (2026) · cites this paper as [7]
The cited source says what the claim attributes to it.
In the citing paper
“World Models [6], MuZero [8], Dreamer [7], and Genie [38] establish predictive environments as a basis for control, planning, and interactive generation.”
From RSI-Anything: Unlocking Recursive Self-Improvement with World Models (2026) · cites this paper as [7]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“We theoretically prove this encoder is essential for efficient context inference and robust generalization.”
From Dynamics-Aligned Latent Imagination in Contextual World Models for Zero-Shot Generalization (2025) · cites this paper as Hafner et al., 2025
Experimental — accuracy not yet measured
Citations
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545 · cited 2×
“The input to the GRU at each time step is a linear embedding of the sampled latent z t , of the action a t , and of the recurrent state to allow mixing between blocks.”
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Self-citationContinuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782 · cited 1×
“These specialized algorithms target the unique challenges posed by different application domains, such as continuous control 8 , discrete actions 9,10 , sparse rewards 11 , image inputs 12 , spatial environments 13 and board games 14 .”
Self-citationProximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625
Mastering the game of Go without human knowledge
Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., et al. (2017) · Nature · doi:10.1038/nature24270 · Cited by 9,084 · cited 1×
“These specialized algorithms target the unique challenges posed by different application domains, such as continuous control 8 , discrete actions 9,10 , sparse rewards 11 , image inputs 12 , spatial environments 13 and board games 14 .”
Self-citationSoft actor-critic Off-policy maximum entropy deep reinforcement learning with a stochastic actor
Haarnoja, T., Zhou, A., Abbeel, P., & Levine, S. (2018) · arXiv · doi:10.48550/arxiv.1801.01290 · Cited by 5,594 · cited 1×
“Constrained optimization targets a fixed entropy on average across states 30,31 regardless of achievable returns, which is robust but explores slowly under sparse rewards and converges lower under dense rewards.”
Training language models to follow instructions with human feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.02155 · Cited by 4,208 · cited 7×
“Aggregated scores on all benchmarks are shown in Extended Data Table 1.”
InfluentialSimple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
Williams, R. J. (1992) · Machine Learning · doi:10.1007/bf00992696 · Cited by 3,416
Training Language Models to Follow Instructions with Human Feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-2011 · Cited by 3,117
Research works printed but not linked: 20
- Huang, S., Dossa, R. F. J., Raffin, A., Kanervisto, A. & Wang, W. The 37 implementation details of proximal policy optimization. The ICLR Blog Track https://iclr-blog-track.github.io/2022/03/25/ppo-implementation-details/ (2023).
- Ha, D. & Schmidhuber, J. Recurrent world models facilitate policy evolution. In Proc. 32nd International Conference on Neural Information Processing Systems 2455–2467 (Association for Computing Machinery, 2018).
- Hafner, D., Lillicrap, T., Ba, J. & Norouzi, M. Dream to control: learning behaviors by latent imagination. In Proc. International Conference on Learning Representations (ICLR, 2020).
- Kingma, D. P. & Welling, M. Auto-encoding variational Bayes. In Proc. International Conference on Learning Representations (ICLR, 2014).
- Child, R. Very deep VAEs generalize autoregressive models and can outperform them on images. In Proc. International Conference on Learning Representations (ICLR, 2021).
- Bellemare, M. G., Dabney, W. & Munos, R. A distributional perspective on reinforcement learning. In Proc. 34th International Conference on Machine Learning 449–458 (PMLR, 2017).
- Kapturowski, S., Ostrovski, G., Quan, J., Munos, R. & Dabney, W. Recurrent experience replay in distributed reinforcement learning. In Proc. International Conference on Learning Representations (ICLR, 2018).
- Cobbe, K. W., Hilton, J., Klimov, O. & Schulman, J. Phasic policy gradient. In Proc. 38th International Conference on Machine Learning 2020–2027 (PMLR, 2021).
- Cobbe, K., Hesse, C., Hilton, J. & Schulman, J. Leveraging procedural generation to benchmark reinforcement learning. In Proc. 37th International Conference on Machine Learning 2048–2056 (Association for Computing Machinery, 2020).
- Hansen, N., Su, H. & Wang, X. TD-MPC2: scalable, robust world models for continuous control. In Proc. International Conference on Learning Representations (ICLR, 2024).
- Yarats, D., Fergus, R., Lazaric, A. & Pinto, L. Mastering visual continuous control: Improved data-augmented reinforcement learning. In Proc. International Conference on Learning Representations (ICLR, 2022).
- Hafner, D. Benchmarking the spectrum of agent capabilities. In Proc. International Conference on Learning Representations (ICLR, 2022).
- Samsami, M. R., Zholus, A., Rajendran, J. & Chandar, S. Mastering memory tasks with world models. In Proc. International Conference on Learning Representations (ICLR, 2024).
- Robine, J., Höftmann, M., Uelwer, T. & Harmeling, S. Transformer-based world models are happy with 100k interactions. In Proc. International Conference on Learning Representations (ICLR, 2023).
- Osband, I., Blundell, C., Pritzel, A. & Van Roy, B. Deep exploration via bootstrapped DQN. In Proc. Advances in Neural Information Processing Systems 29 (NeurIPS, 2016).
- Brock, A., De, S., Smith, S. L. & Simonyan, K. High-performance large-scale image recognition without normalization. In Proc. 38th International Conference on Machine Learning 1059–1071 (PMLR, 2021).
- Kingma, D. P. & Ba, J. Adam: a method for stochastic optimization. In Proc. International Conference on Learning Representations (ICLR, 2017).
- Schmitt, S., Hessel, M. & Simonyan, K. Off-policy actor–critic with shared experience replay. In Proc. 37th International Conference on Machine Learning 8545–8554 (PMLR, 2020).
- Schaul, T., Quan, J., Antonoglou, I. & Silver, D. Prioritized experience replay. In Proc. International Conference on Learning Representations (ICLR, 2018).
- Sharma, S., Srinivas, A. & Ravindran, B. Learning to repeat: fine grained action repetition for deep reinforcement learning. In Proc. International Conference on Learning Representations (ICLR, 2017).
Cited by
Learning Interactive Real-World Simulators
Yang, S., Du, Y., Ghasemipour, K., Tompson, J., Kaelbling, L., Schuurmans, D., & Abbeel, P. (2023) · arXiv · doi:10.48550/arxiv.2310.06114 · Cited by 346
Vision-Language-Action Models for Robotics: A Review Towards Real-World Applications
Kawaharazuka, K., Oh, J., Yamada, J., Posner, I., & Zhu, Y. (2025) · IEEE Access · doi:10.1109/access.2025.3609980 · Cited by 250
A Survey of Agent Memory in the Second Half: Towards Self-Evolving and Long-Horizon Agents
Huang, W.-C., Zhang, W., Liang, Y., Bei, Y., Chen, Y., Feng, T., Pan, X., Tan, Z., Wang, Y., Wei, T., et al. (2026) · arXiv · doi:10.48550/arxiv.2602.06052 · Cited by 70
Understanding World or Predicting Future? A Comprehensive Survey of World Models
Ding, J., Zhang, Y., Shang, Y., Zhang, Y., Zong, Z., Feng, J., Yuan, Y., Su, H., Li, N., Sukiennik, N., et al. (2026) · ACM Computing Surveys · doi:10.1145/3746449 · Cited by 169
“The robotics domain has similarly advanced, exemplified by DayDreamer in 2023 [275] and continuing with recent developments in 2025 for robotic manipulation tasks [163].”
InfluentialModel-based reinforcement learning for ultrasound-driven autonomous microrobots
Medany, M., Piglia, L., Achenbach, L., Mukkavilli, S. K., & Ahmed, D. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-025-01054-2 · Cited by 56
“As previously discussed in research 55 , the ability of MBRL to imagine and simulate future actions, as shown in Fig.”
Metadata
- Authors
- Hafner, D., Pasukonis, J., Ba, J., & Lillicrap, T.
- Affiliations
- University of Toronto (CA)
- Corresponding author
- Hafner, Danijar
- Venue
- Nature
- Published
- 2025-04-02
- Volume
- 640
- Issue
- 8059
- Pages
- 647-653
- ISSN
- 0028-0836 (ISSN-L)
- 1476-4687
- PMC
- PMC12003158
- OpenAlex
- W4409147637
- Cited by
- 189
- References
- 39
- Citations by year
- 2026: 145 · 2025: 44
- FWCI
- 75.21
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Nature (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- dreamers, world models, sparse rewards, domain generalization
- MeSH terms
- Algorithms, Artificial Intelligence, Humans, Learning, Problem Solving, Reinforcement, Psychology, Task Performance and Analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Mastering diverse control tasks through world models. https://app.cochat.ai/papers/10.1038/s41586-025-08744-2
- The paper
- Hafner, D., Pasukonis, J., Ba, J., & Lillicrap, T. (2025). Mastering diverse control tasks through world models. Nature. https://doi.org/10.1038/s41586-025-08744-2
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.