Understanding World or Predicting Future? A Comprehensive Survey of World Models
Open accessThe DOI matches its Crossref record; 306 of the 306 references it cites were found in a registry or an index; 48 of the 72 sources its claims cite were checked for support.
Abstract
The concept of world models has garnered significant attention due to advancements in multimodal large language models such as GPT-4 and video generation models such as Sora, which are central to the pursuit of artificial general intelligence. This survey offers a comprehensive review of the literature on world models. Generally, world models are regarded as tools for either understanding the present state of the world or predicting its future dynamics. This review presents a systematic categorization of world models, emphasizing two primary functions: (1) constructing internal representations to understand the mechanisms of the world, and (2) predicting future states to simulate and guide decision-making. Initially, we examine the current progress in these two categories. We then explore the application of world models in key domains, including autonomous driving, robotics, and social simulacra, with a focus on how each domain utilizes these aspects. Finally, we outline key challenges and provide insights into potential future research directions. We summarize the representative articles along with their code repositories in https://github.com/tsinghua-fib-lab/World-Model .
Overview
CoChat Summary
TL;DR
This survey categorizes world models by two primary functions: constructing implicit internal representations to understand the world’s mechanisms and predicting future states to simulate and guide decision-making. It reviews progress in decision-making, video generation, embodied environments, and applications in autonomous driving, robotics, and social simulacra, then outlines open challenges and future directions.
Key findings
- The paper proposes a taxonomy of world models: implicit representation of the external world (e.g., model-based RL and LLM/MLLM world knowledge) and future prediction of the physical world (e.g., video generation and embodied environments). It reports that DreamerV3 solves over 150 tasks—including diamond collection in Minecraft—without human data or domain-specific tuning.
- Video world models such as Sora can generate high-quality, temporally consistent video sequences up to one minute long, but the survey notes persistent limitations in causal reasoning and physical-law simulation. It cites evidence that scaling diffusion video models yields perfect in-distribution fidelity yet breaks down on out-of-distribution or combinatorial tests, indicating “case-based” rather than rule-based generalization.
- Applications are organized across autonomous driving, robotics, and social simulacra. For robotics, DayDreamer allows robots to learn locomotion in real-world environments within hours, and SWIM enables task generalization with less than 30 minutes of interaction; for social simulacra, AI Town is described as a world model composed of 25 generative agents.
- Open challenges include physical rules and counterfactual simulation, enriching the social dimension, bridging simulation and reality with embodied intelligence, simulation efficiency, ethical/safety concerns, and benchmarking. Benchmark examples include WorldScore across 3,000 camera-specified scenes, PhysBench with 10k video-image-text triplets, and UrbanVideo-Bench with 5.2k drone clips.
Methods
- Design: a comprehensive literature survey that formally defines and categorizes world models around two functions—implicit representation and future prediction—and summarizes representative articles plus code repositories.
- Data/sample: no primary dataset or experimental sample; the paper synthesizes existing literature across model-based RL, LLM/MLLM world knowledge, video generation, embodied environments, autonomous driving, robotics, and social simulacra, with summary tables of representative works.
- Main analysis: qualitative synthesis and taxonomy development; it compares technical approaches, application emphases, and open problems rather than conducting meta-analysis or empirical evaluation.
Limitations
- The paper states that benchmarking world models is challenging because the community pursues divergent goals, uses heterogeneous technical approaches, and spans wide-ranging application domains, so there is no single canonical task or metric. It also states that evaluating the realism of generated human behavior still largely relies on subjective human assessment, which is hard to scale.
- The paper states that current video generators achieve visual realism but largely fail on tasks requiring understanding of optics, fluid dynamics, or magnetism, and that data-driven scaling alone is insufficient to recover robust physical laws. It also identifies a longstanding simulation-to-reality gap for embodied agents.
- Evident from the text: as a survey, it presents no primary experiments or quantitative validation of its own; its coverage is organized around representative works and categories rather than a systematic search protocol with inclusion/exclusion criteria.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures







Verification
Claims the paper makes
33 Valid · 8 Not valid · 13 Not checked · 31 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“After pretraining on large-scale web text and books [145,201], large language models attain extensive knowledge about the real-world and common sense relevant to daily life.”
§ Understanding World or Predicting Future? A Comprehensive Survey of World Models > 3.2 World Knowledge Learned by Models
What the sources say
- Not matchedcites [145] 145. OpenAI. 2022. Introducing ChatGPT. Retrieved May 06, 2025 from https://openai.com/blog/chatgpt.
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Supportedcites [201] 201. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models. arXiv:2302.13971. Retrieved from https://arxiv.org/abs/2302.13971 source
The citation follows the pretraining clause, so the relevant attribution is that LLMs are pretrained on large-scale web text and books. Section 2.1 (p. 2) reports CommonCrawl and C4 web data and Gutenberg and Books3 book corpora in LLaMA’s pretraining mixture; it also says these data sources have been used to train other LLMs. (LLM, conf=0.90)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Ha et al. [65,66] significantly revived and popularized the term”
§ Understanding World or Predicting Future? A Comprehensive Survey of World Models > 2 Background and Categorization
What the sources say
- Not matchedcites [65] 65. David Ha and Jürgen Schmidhuber. 2018. Recurrent world models facilitate policy evolution. Advances in Neural Information Processing Systems 31 (2018).
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Partly supportedcites [66] 66. David Ha and Jürgen Schmidhuber. 2018. World models. arXiv:1803.10122. Retrieved from https://arxiv.org/abs/1803. 10122 sourceExperimental — accuracy not yet measured
The Abstract (p. 1) describes a neural-network world model that learns compressed spatial and temporal representations, supporting the model-description part of the claim. The Introduction (p. 2) discusses earlier RNN-based world models, but our copy does not establish that this 2018 paper significantly revived or popularized the term. (LLM, conf=0.85)
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“early work by Ha and Schmidhuber [66] focused on abstracting the external world to gain a deep understanding of its underlying mechanisms.”
§ Understanding World or Predicting Future? A Comprehensive Survey of World Models > 1 Introduction
What the sources say
- Partly supportedcites [66] 66. David Ha and Jürgen Schmidhuber. 2018. World models. arXiv:1803.10122. Retrieved from https://arxiv.org/abs/1803. 10122 sourceExperimental — accuracy not yet measured
Section 3.1 (p. 4) says the world model learns a low-dimensional representation of observations, supporting the abstraction part of the claim. It describes using that representation to train a controller, however, not gaining a deep understanding of the external world's underlying mechanisms. The Introduction (p. 2) likewise frames the work around world models and agent training. (LLM, conf=0.82)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“However, by generating video-like simulations that capture continuous spatial and temporal dynamics, world models [146, 233] have evolved to address more complex, dynamic environments.”
§ Understanding World or Predicting Future? A Comprehensive Survey of World Models > 4 Future Prediction of the Physical World 4.1 World Model as Video Generation
What the sources say
- Not matchedcites [146] 146. OpenAI. 2024. Sora: Creating video from text. Retrieved May 06, 2025 from https://openai.com/sora.
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Partly supportedcites [233] 233. Deshun Yang, Luhui Hu, Yu Tian, Zihao Li, Chris Kelly, Bang Yang, Cindy Yang, and Yuexian Zou. 2024. WorldGPT: A sora-inspired video AI agent as rich world models from text and image inputs. arXiv:2403.07944. Retrieved from https://arxiv.org/abs/2403.07944 sourceExperimental — accuracy not yet measured
The Introduction (page not specified) describes video generation using dynamic scene modeling and spatiotemporal prediction, and Section 2.3 describes continuous video sequences with time-and-space consistency. These passages support the video and dynamics aspect, but they do not establish the broader claim that world models have evolved to address more complex, dynamic environments. (LLM, conf=0.82)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
Every source the paper cites for this claim was checked and supports it.
In the paper
“For example, the emergence of deepfake technology gives rise to large-scale misuses that have widespread negative effects on social, economic, and political systems [214].”
§ Understanding World or Predicting Future? A Comprehensive Survey of World Models > Simulating Unsafe Scenario.
What the sources say
- Supportedcites [214] 214. Mika Westerlund. 2019. The emergence of deepfake technology: A review. Technology Innovation Management Review 9, 11 (2019), 39-52. source
The Introduction says deepfakes can quickly reach millions of people and negatively affect society. The Discussion and Conclusion describes them as a major threat to society, the political system, and businesses, detailing disinformation, election interference, and cybersecurity harms. (LLM, conf=0.89)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
10 more claims · 4 with their cited sources checked
Create a free account to see all 85 claims and every source verdict.
- It may state two separate claims as one. 35
- Probably not a substantial claim. 53
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Understanding World or Predicting Future? A Comprehensive Survey of World Models > 5.1.1 Learning Implicit Representations.
“OmniDrive [107] sets up llm-based agents and covers multiple tasks including scene description, counterfactual reasoning and decision making through visual question-answering.”
§ Understanding World or Predicting Future? A Comprehensive Survey of World Models > References
“107. Varun Ravi Kumar, Senthil Yogamani, Hazem Rashed, Ganesh Sistu, Christian Witt, Isabelle Leang, Stefan Milz, and Patrick Mäder. 2023. OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Au- tonomous Driving.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Understanding World or Predicting Future? A Comprehensive Survey of World Models > 5.1.2 World Simulators.
“Copilot4D [248] constructs a world model by predicting changes in radar point cloud data.”
§ Understanding World or Predicting Future? A Comprehensive Survey of World Models > 5.1.1 Learning Implicit Representations.
“Copilot4D [248] 2024 Lidar Diffusion Point Cloud Generation”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Understanding World or Predicting Future? A Comprehensive Survey of World Models > 2 Background and Categorization
“foundational work such as Marvin Minsky's frame representation in the 1960s [136]”
§ Understanding World or Predicting Future? A Comprehensive Survey of World Models > References
“136. Marvin Minsky. 1974. A framework for representing knowledge. MIT, Cambridge.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Understanding World or Predicting Future? A Comprehensive Survey of World Models > 7 Conclusion
“| Indoor | Matterport 3D [18] | Home | 2018 | 90 | V | ✗ | ✗ |”
§ Understanding World or Predicting Future? A Comprehensive Survey of World Models > References
“18. Angel Chang, Angela Dai, Thomas Funkhouser, Maciej Halber, Matthias Niessner, Manolis Savva, Shuran Song, Andy Zeng, and Yinda Zhang. 2017. Matterport3D: Learning from RGB-D data in indoor environments.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Understanding World or Predicting Future? A Comprehensive Survey of World Models > 5.1.1 Learning Implicit Representations.
“Pointnet [158], introduced in 2017, was the first to employ deep learning methods for processing point cloud data.”
§ Understanding World or Predicting Future? A Comprehensive Survey of World Models > References
“158. Charles R. Qi, Li Yi, Hao Su, and Leonidas J. Guibas. 2017. PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space.”
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 2×
“To help a robot understand the objects in the world, visual models such as convolutional neural networks (CNNs) [77, 132, 141] and vision transformers (ViT) [45, 259] integrate visual characteristics of entities into representations, making it possible for robots to recognize critical objects for…”
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Proceedings of IEEE Conference on Computer Vision and Pattern Recognition
(1993) · doi:10.1109/cvpr.1993 · Cited by 17,667
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612 · cited 1×
“The two schemes of leveraging world models in decision-making tasks are shown in Figure 2.”
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293
Proximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625
Research works printed but not linked: 25
- Yifan Cui, Xinyi Shan, and Jeanhun Chung. 2024. A feasibility study on RUNWAY GEN-2 for generating realistic style images. International Journal of Internet, Broadcasting and Communication 16, 1 (2024), 99–105.
- Nan Du, Yanping Huang, Andrew M. Dai, Simon Tong, Dmitry Lepikhin, Yuanzhong Xu, Maxim Krikun, Yanqi Zhou, Adams Wei Yu, Orhan Firat, et al. 2022. Glam: Efficient scaling of language models with mixture-of-experts. In Proceedings of the International Conference on Machine Learning. PMLR, 5547–5569.
- Yilun Du, Sherry Yang, Bo Dai, Hanjun Dai, Ofir Nachum, Josh Tenenbaum, Dale Schuurmans, and Pieter Abbeel. 2024. Learning universal policies via text-guided video generation. Advances in Neural Information Processing Systems 36 (2024).
- Ignat Georgiev, Varun Giridhar, Nicklas Hansen, and Animesh Garg. 2025. PWM: Policy learning with multi-task world models. In Proceedings of the 13th International Conference on Learning Representations.
- David Ha and Jürgen Schmidhuber. 2018. Recurrent world models facilitate policy evolution. Advances in Neural Information Processing Systems 31 (2018).
- Haoran He, Chenjia Bai, Ling Pan, Weinan Zhang, Bin Zhao, and Xuelong Li. 2024. Learning an actionable discrete diffusion policy via large-scale actionless video pre-training. In Proceedings of the 38th Annual Conference on Neural Information Processing Systems.
- Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems 33 (2020), 6840–6851.
- Anthony Hu Lloyd Russell Hudson Yeo Zak Murez George Fedoseev Alex Kendall Jamie Shotton and Gianluca Corrado. 2023. GAIA-1: A Generative World Model for Autonomous Driving. arxiv:2309.17080. Retrieved from https://arxiv.org/abs/2309.17080 · arXiv
- Eric Jang, Alex Irpan, Mohi Khansari, Daniel Kappler, Frederik Ebert, Corey Lynch, Sergey Levine, and Chelsea Finn. 2022. Bc-z: Zero-shot task generalization with robotic imitation learning. In Proceedings of the Conference on Robot Learning. PMLR, 991–1002.
- 10.5555/3692070.3692961 · doi:10.5555/3692070.3692961
- Philip Nicholas Johnson-Laird. 1983. Mental Models: Towards a Cognitive Science of Language, Inference, and Consciousness. Number 6. Harvard University Press.
- Subbarao Kambhampati, Karthik Valmeekam, Lin Guan, Mudit Verma, Kaya Stechly, Siddhant Bhambri, Lucas Paul Saldyt, and Anil B. Murthy. 2024. Position: LLMs can’t plan, but can help planning in LLM-modulo frameworks. In Proceedings of the 41st International Conference on Machine Learning.
- Kuaishou. 2024. Kling AI: Next-Generation AI Creative Studio. Retrieved May 06, 2025 from https://www.klingai.com/global/.
- Yann LeCun. 2022. A path towards autonomous machine intelligence version 0.9. 2, 2022-06-27. Open Review 62, 1 (2022), 1–62.
- Junhyuk Oh, Satinder Singh, and Honglak Lee. 2017. Value prediction network. Advances in Neural Information Processing Systems 30 (2017).
- OpenAI. 2024. Sora: Creating video from text. Retrieved May 06, 2025 from https://openai.com/sora.
- Aravind Rajeswaran, Igor Mordatch, and Vikash Kumar. 2020. A game theoretic framework for model based reinforcement learning. In Proceedings of the International Conference on Machine Learning. PMLR, 7953–7963.
- Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. 2015. Faster R-CNN: Towards real-time object detection with region proposal networks. In Proceedings of the Advances in Neural Information Processing Systems.C. Cortes, N. Lawrence, D. Lee, M. Sugiyama, and R. Garnett (Eds.), Vol. 28, Curran Associates, Inc. Retrieved from https://proceedings.neurips.cc/paper_files/paper/2015/file/14bfa6bb14875e45bba028a21ed38046-Paper.pdf
- Alvaro Sanchez-Gonzalez, Jonathan Godwin, Tobias Pfaff, Rex Ying, Jure Leskovec, and Peter Battaglia. 2020. Learning to simulate complex physics with graph networks. In Proceedings of the International Conference on Machine Learning. PMLR, 8459–8468.
- Hongzhi Shi, Jingtao Ding, Yufan Cao, Li Liu, Yong Li, et al. 2022. Learning symbolic models for graph-structured physical mechanism. In Proceedings of the 11th International Conference on Learning Representations.
- A Vaswani. 2017. Attention is all you need. Advances in Neural Information Processing Systems 30 (2017).
- Philipp Wu, Alejandro Escontrela, Danijar Hafner, Pieter Abbeel, and Ken Goldberg. 2023. Daydreamer: World models for physical robot learning. In Proceedings of the Conference on Robot Learning. PMLR, 2226–2240.
- Wilson Yan, Danijar Hafner, Stephen James, and Pieter Abbeel. 2023. Temporally consistent transformers for video generation. In Proceedings of the International Conference on Machine Learning. PMLR, 39062–39098.
- Sherry Yang, Yilun Du, Seyed Kamyar Seyed Ghasemipour, Jonathan Tompson, Leslie Pack Kaelbling, Dale Schuurmans, and Pieter Abbeel. 2024. Learning interactive real-world simulators. In Proceedings of the 12th International Conference on Learning Representations.
- 10.1007/978-3-031-72624-8_4 · DOI · Paper page
Cited by
Nie, T., Sun, J., & Ma, W. (2025) · Artificial Intelligence for Transportation · doi:10.1016/j.ait.2025.100003 · Cited by 116
CityGPT: Empowering Urban Spatial Cognition of Large Language Models
Feng, J., Liu, T., Du, Y., Guo, S., Lin, Y., & Li, Y. (2025) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3711896.3736878 · Cited by 137
Self-citationZhang, Y., Tian, J., & Xiong, Q. (2025) · Frontiers in Robotics and AI · doi:10.3389/frobt.2025.1668910 · Cited by 33
Self-citationZhao, C., Liu, G., Zhang, R., Liu, Y., Wang, J., Kang, J., Niyato, D. T., Li, Z., Shen, X., Han, Z., et al. (2026) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2026.3658762 · Cited by 41
“Most world models are implemented as neural architectures with three core components that achieve the perception-prediction cycle [20]: • Encoder: The encoder network compresses high-dimensional observations, such as images, sensor data, etc., into a compact latent state.”
InfluentialCityBench: Evaluating the Capabilities of Large Language Models for Urban Tasks
Feng, J., Zhang, J., Liu, T., Zhang, X., Ouyang, T., Yan, J., Du, Y., Guo, S., & Li, Y. (2025) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3711896.3737375 · Cited by 98
Self-citationA literature review on the artificial intelligence in manufacturing systems under industry 5.0
Pan, Y., Huang, Z., Lev, B., Xu, L., & Olson, D. L. (2026) · International Journal of Production Research · doi:10.1080/00207543.2026.2623537 · Cited by 23
“…representations of the dynamics governing real-world environments, allowing agents to simulate and evaluate coun-terfactual actions in a physically consistent way rather than relying solely on model-free trial-and-error (Ding et al. 2025; Ha and Schmidhuber 2018; Wu, Escon-trela, et al. 2023).”
A survey of physics-informed AI for complex urban systems
Xu, E., Wang, H., Zhang, Y., Li, S., Tang, Y., Zhou, Z., Lin, Y., Yuan, Y., Fan, X., Ding, J., et al. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.104012 · Cited by 22
Self-citationEmbodied AI: From LLMs to World Models [Feature]
Feng, T., Wang, X., Jiang, Y., & Zhu, W. (2025) · IEEE Circuits and Systems Magazine · doi:10.1109/mcas.2025.3603693 · Cited by 75
“Recent breakthroughs in Multimodal LLMs (MLLMs) [23], [24] and World Models (WMs) [25], [26], [27] have opened up a new frontier in embodied AI research.”
Model predictive control: past, present, and future
Li, N., Chen, H., Li, M., Yu, S., & Huang, Y. (2026) · Control Theory and Technology · doi:10.1007/s11768-026-00320-x · Cited by 14
Self-citationUrbanLLaVA: A Multi-Modal Large Language Model for Urban Intelligence
Feng, J., Wang, S. S., Liu, T., Xi, Y., & Li, Y. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00586 · Cited by 58
Self-citationResearch on World Models for Connected Automated Driving: Advances, Challenges, and Outlook
Chen, N., & Liu, X. (2025) · Applied Sciences · doi:10.3390/app15168986 · Cited by 12
Predicting What Matters: Training AI Models for Better Decisions
Anand, A. S., Sawant, S., Reinhardt, D. P., & Gros, S. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2025.3633573 · Cited by 15
AgentSquare: Automatic LLM Agent Search in Modular Design Space
Shang, Y., Li, Y., Zhao, K., Ma, L., Liu, J., Xu, F., & Li, Y. (2024) · arXiv · doi:10.48550/arxiv.2410.06153 · Cited by 3
Self-citationHe, T., Liao, L., Liu, M. Z., & Qin, B. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730084 · Cited by 9
Optimizing Autonomous Wheel Loader Performance—An End-to-End Approach
Aoshima, K., Servin, M., & Wadbro, E. (2025) · Automation · doi:10.3390/automation6030031 · Cited by 6
“These models are referred to as a world model, as they predict the new state the environment will transition into, given the previous state and a selected action, and the observations associated with the state transition [4].”
Benchmarking and Advancing Large Language Models for Local Life Services
Lan, X., Feng, J., Lei, J., Shi, X., & Li, Y. (2025) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3711896.3737196 · Cited by 9
Self-citationWill power-seeking AGIs harm human society?
Wang, M. (2025) · AI & Society · doi:10.1007/s00146-025-02572-8 · Cited by 2
“More systematically, recent surveys suggest that existing approaches can be broadly categorized into two methodological paradigms: those that prioritize understanding the world and those that emphasize predicting the future (Ding et al. 2024; Xing et al. 2025).”
Zhang, D., Shi, W. K., & Jia, X. (2026) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2026.3672428 · Cited by 4
Zhang, T., & Yue, D. (2026) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2026.125846 · Cited by 1
Structured Latent Dynamics in Wireless CSI via Homomorphic World Models
Naoumi, S., Bennis, M., & Chafii, M. (2026) · International Conference on Communications · doi:10.1109/icc59461.2026.11587668 · Cited by 8
“Such models support internal simulations that enhance long-horizon reasoning, counterfactual inference, and policy planning [3], [4].”
Digital twin construction for biomimetic materials
Shen, T., Zhang, C., Jiang, L., & Dong, Z. (2026) · Advanced bionics. · doi:10.1016/j.abs.2026.07.006 · Cited by 3
Virtual Space Switching: Fast Mobility Handoffs for Situated Intelligence
Maheswaran, M. (2026) · International Conference on Distributed Computing in Smart Systems and the Internet of Things (DCOSS-IoT) · doi:10.1109/dcoss-iot69657.2026.00041 · Cited by 1
“World models learn spatial and temporal representations from observation [7].”
Zhao, X., Hu, S., & Yin, X.-C. (2025) · Advances in computer vision and pattern recognition · doi:10.1007/978-981-96-4558-9_1
Shitsukan at The Intersection of Vision and Language
Okatani, T. (2026) · Springer eBooks · doi:10.1007/978-981-95-4762-3_4
Steeves, E. J., Raj, P. A. C., Jose, P. S. H., et al. (2025) · doi:10.1109/ic2nc67409.2025.11376534
Evaluating Image Compression Potential for Physical World Building
Kuo, S., & Chang, I.-C. (2026) · IEEE Annual Computing and Communication Workshop and Conference (CCWC) · doi:10.1109/ccwc67433.2026.11393852
“A characterization of the effects of compression on DFF performance is also helpful for the recent shifts in AI development towards world models [14] and reducing sim-to-real gaps [15] happening in the industry.”
NRSeg: Noise-Resilient Learning for BEV Semantic Segmentation via Driving World Models
Li, S., Teng, F., Cao, Y., Yang, K., Li, Z., & Wang, Y. (2026) · IEEE Transactions on Image Processing · doi:10.1109/tip.2026.3671686 · Cited by 2
“World models can establish an internal representation framework that learns 2D projections from 3D structures, facilitating environmental comprehension and future state prediction [29].”
Yang, L., Dai, Y., Yan, A., Prabhu, V., Xu, R., & Chen, Z. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37517-9_1 · Cited by 3
Infinite Gaze Generation for Videos with Autoregressive Diffusion
Kang, J., Groth, C., Wu, T., Torrens, F., Sangkloy, P., Wetzstein, G., & Sun, Q. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37362-5_5 · Cited by 2
ExploreVLA: Dense World Modeling and Exploration for End-to-End Autonomous Driving
Sheng, Z., Ye, X., Luo, J., Chen, S., & Ren, L. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37261-1_16 · Cited by 19
“In the context of autonomous driving, world models offer the ability to predict future states of the driving scene, enabling planning, data augmentation, and closed-loop evaluation without costly real-world interactions [8,10,13,44].”
Saad, A., Ibrahim, A. M., & Gupta, S. (2026) · Computational Discovery and Intelligent Systems (CDIS) · doi:10.66279/t95nh857
“World model - based methods mainly concentrate on acquiring predictive environment dynamics but do not incorporate explicit spatial memory mechanisms[2].”
Helbich, M., Labib, S. M., Liedtke, I., Sent, D., Crombag, R., Ettema, D., Posthumus, H., van der Zee, S. C., Reiling, M., Ruinaard, S., et al. (2026) · PLoS ONE · doi:10.1371/journal.pone.0348889
“Emerging AI-based approaches, including large language and world models, offer potential to support context-specific knowledge translation by linking environmental features to WPA [29,30].”
GTA: Advancing Image-to-3D World Generation via Geometry Then Appearance Video Diffusion
Zhu, H., Wang, C., Tu, P., Luo, J., He, T., Jin, X., & Chen, Z. (2026) · International Journal of Computer Vision · doi:10.1007/s11263-026-02993-y · Cited by 2
“By extending image generation models to the temporal domain, these approaches explicitly model spatiotemporal dependencies and enable the synthesis of high-quality video sequences with improved visual fidelity and temporal coherence [21, 40–46].”
梁, 君., & 李, 云. (2026) · Global Journal of Media Studies · doi:10.26599/gjms.2026.9330007
Wang, X., Feng, T., Liu, H., & Zhu, W. (2026) · Springer eBooks · doi:10.1007/978-981-95-7750-7_1 · Cited by 3
Digital Twins and World Models: A Systematic Taxonomic Disambiguation
Suri, K., & Arnéz, F. (2026) · Lecture notes in business information processing · doi:10.1007/978-3-032-28160-9_6
Embodied GPT-5.1: Evidence of a World Model?
Spinelli, R., & Martins, T. C. (2026) · Brazilian Conference on Robotics (CROS) · doi:10.1109/cros69211.2026.11565684
“Contemporary research in Embodied AI (EAI) recognizes the core challenge of bridging high-level semantics with grounded physical interaction [15].”
AdaReP: Adaptive Re-planning under Model Mismatch for Neural World-Model Predictive Control
Cheng, Y., Ma, X., Wang, X., Yang, M., Su, R., Liu, H., Chen, X., Li, S., & Li, Q. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-38404-1_18 · Cited by 3
“Neural world-model methods cover latent-dynamics planning in PlaNet, Dreamer, and DreamerV3 [5,4,6], latent MPC in TD-MPC and TD-MPC2 [8,7], recurrent or object-centric video prediction for manipulation [22,16,1,20,19], and recent transformer, diffusion, and multimodal predictors [25,28,3].”
Xu, J., Song, S., Zhao, S., Liang, X., Gu, H., & Wang, S. K. (2026) · Applied Sciences · doi:10.3390/app16136447
PhysMani: Physics-Principled 3D World Model for Dynamic Object Manipulation
Peng, Y., Huang, S., Li, H., Li, J., Wang, J., & Yang, B. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37029-7_2 · Cited by 2
Zhang, L., Liu, J., ZHU, J., & Zhang, L. (2026) · International Conference on Computer Supported Cooperative Work in Design (CSCWD) · doi:10.1109/cscwd68734.2026.11582015
“A fundamental limitation of existing methods is the weak integration of structured domain knowledge and human-like reasoning into the learning process [6].”
Prediction-in-the-Loop: a decision-making paradigm endowing agents with foresight
Liu, Y., Yuan, M., Zhang, L., Zhou, Z., & Liu, L. (2026) · Scientia Sinica Informationis · doi:10.1360/ssi-2026-0130
A Dual-View Hypergraph World Model for Multi-agent Reinforcement Learning
Peng, S., Li, T., Hu, R., & Zhang, X. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3381-6_49
Conceptual blending in humans and language models
Lynn, S. K. (2026) · Frontiers in Psychology · doi:10.3389/fpsyg.2026.1849083
“…structure, encode latent regularities relevant to action and state transitions, and in some settings be induced or fine-tuned to perform world-model-like functions such as precondition and effect prediction (Ding et al., 2026; Gurnee and Tegmark, 2024; Xie et al., 2025; Yildirim and Paul, 2024).”
Fusing Digital Twins and World Models for Proactive Risk-Aware Routing in 6G Satellite Networks
Zhu, R., Shi, L., Xie, Y., Zhou, L., Chen, K., Guo, B., & Huang, S. (2026) · IEEE Transactions on Network Science and Engineering · doi:10.1109/tnse.2026.3718522
From Pixels to Plans: A Review of Latent-Space Future Prediction
Malashin, I. P., Тынченко, В. С., & Martysyuk, D. (2026) · Archives of Computational Methods in Engineering · doi:10.1007/s11831-026-10766-6
Yu, P. S., & Sun, L. (2026) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3770855.3818656
Sun, X., Guo, Y., Zhang, Y., et al. (2026) · Advanced Engineering Informatics · doi:10.1016/j.aei.2026.105108
Self-citationShi, F., Yang, J., Lifelo, Z., Ding, J., & Ning, H. (2026) · Artificial Intelligence Review · doi:10.1007/s10462-026-11691-7 · Cited by 1
“For instance, Ding et al. presented world models as valuable tools for understanding current state of the world or predicting its future dynamics (Ding et al. 2025 ).”
Self-citationSelf-Orchestrated AP Switching Optimization via Network Uncertainty Awareness for 6G AI-RAN
Kim, S., Kim, D.-E., & Choi, J. P. (2026) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2026.3726077
“Meanwhile, recent advances in artificial intelligence (AI) have seen the emergence of deep reinforcement learning (DRL) based on World models as a powerful approach to tackle such long-term optimizations [5].”
LLM-driven paradigm for agent-based modeling and simulation: A review and organizing framework
Wang, J., Li, X., & Xiao, G. (2026) · Computer Science Review · doi:10.1016/j.cosrev.2026.101054
Assuring the Interconnected AI Technologies and Autonomous Systems in the Maritime Domain
Imrie, C., Ryan, P., Molloy, J., et al. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-35506-5_24
Navigating Latent Worlds: A Hypertextual Taxonomy of World Models as Nonlinear Knowledge Structures
Sharma, R., Mehta, M., & Kotwal, A. (2026) · ACM Conference on Hypertext and Social Media (HT) · doi:10.1145/3800935.3830860
From Single-modal to Multimodal Perception in Intelligent Robotic Manipulators: A Review
Weng, T., Gao, J., Gong, M., Yang, H., Sui, X., Feng, L., Yang, D., & Xiaohu, B. A. I. (2026) · Journal of Bionic Engineering · doi:10.1007/s42235-026-01000-x
Luo, M., Liu, Y., Wang, J., Zhang, Y., Tao, X., Wan, P., Gai, K., & Fei, H. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37595-7_30 · Cited by 2
Self-citationYu, S., & Lu, C. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37281-9_24
Pasquali, D., Rea, F., Morasso, P., Sandini, G., & Sciutti, A. (2026) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2026.1920808
“…n.d.; Xu et al., 2025), continual-and lifelong-learning methods allow robots to preserve and recombine capabilities across successive tasks (Meng et al., 2025), and world models support online learning and planning through internally simulated action outcomes (Ding et al., 2025; Wu et al., 2023).”
Frileux, S., Kulakovskyi, A., Altunlu, E., Bergonzo, S., Jensen, O. C., Leboyer, M., Bonnot, O., Mouchabac, S., Koutsouleris, N., & Bourla, A. (2026) · L Encéphale · doi:10.1016/j.encep.2026.08.004 · Cited by 5
Fan, J., Rao, J., Wei, L., Li, G., Yuan, J., & Qi, Y. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.134549
Ng, W. C., Xie, Q., Lee, Y. K., Tee, W. H., Abdulrab, H. Q. A., Hii, T. C. S., & Ding, T. J. (2026) · Neurocomputing · doi:10.1016/j.neucom.2026.135279
Metadata
- Authors
- Ding, J., Zhang, Y., Shang, Y., Zhang, Y., Zong, Z., Feng, J., Yuan, Y., Su, H., Li, N., Sukiennik, N., Xu, F., & Li, Y.
- Affiliations
- Tsinghua University (CN)
- ORCID
- Venue
- ACM Computing Surveys
- Published
- 2025-09-09
- Volume
- 58
- Issue
- 3
- Pages
- 1-38
- ISSN
- 0360-0300 (ISSN-L)
- 1557-7341
- OpenAlex
- W4411735734
- Cited by
- 169
- References
- 48
- Citations by year
- 2026: 122 · 2025: 46
- FWCI
- 43.25
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- ACM Computing Surveys (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- National Key Research and Development Program of China (2024YFC3307603)
- National Natural Science Foundation of China (62476152 and U24B20180)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Physical Sciences › Computer Science › Artificial Intelligence › Psychiatry, Mental Health, Neuroscience
- Keywords
- world models, internal representations, future state prediction, artificial general intelligence, autonomous driving, robotics, social simulation, multimodal large language models, video generation models, decision-making
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-01, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.