Understanding World or Predicting Future? A Comprehensive Survey of World Models
Open accessThe DOI matches its Crossref record; 305 of the 305 references it cites were found in a registry or an index; 48 of the 72 sources its claims cite were checked for support.
Abstract
The concept of world models has garnered significant attention due to advancements in multimodal large language models such as GPT-4 and video generation models such as Sora, which are central to the pursuit of artificial general intelligence. This survey offers a comprehensive review of the literature on world models. Generally, world models are regarded as tools for either understanding the present state of the world or predicting its future dynamics. This review presents a systematic categorization of world models, emphasizing two primary functions: (1) constructing internal representations to understand the mechanisms of the world, and (2) predicting future states to simulate and guide decision-making. Initially, we examine the current progress in these two categories. We then explore the application of world models in key domains, including autonomous driving, robotics, and social simulacra, with a focus on how each domain utilizes these aspects. Finally, we outline key challenges and provide insights into potential future research directions. We summarize the representative articles along with their code repositories in https://github.com/tsinghua-fib-lab/World-Model .
Overview
CoChat Summary
TL;DR
This survey categorizes world models by two primary functions: constructing implicit internal representations to understand the world’s mechanisms and predicting future states to simulate and guide decision-making. It reviews progress in decision-making, video generation, embodied environments, and applications in autonomous driving, robotics, and social simulacra, then outlines open challenges and future directions.
Key findings
- The paper proposes a taxonomy of world models: implicit representation of the external world (e.g., model-based RL and LLM/MLLM world knowledge) and future prediction of the physical world (e.g., video generation and embodied environments). It reports that DreamerV3 solves over 150 tasks—including diamond collection in Minecraft—without human data or domain-specific tuning.
- Video world models such as Sora can generate high-quality, temporally consistent video sequences up to one minute long, but the survey notes persistent limitations in causal reasoning and physical-law simulation. It cites evidence that scaling diffusion video models yields perfect in-distribution fidelity yet breaks down on out-of-distribution or combinatorial tests, indicating “case-based” rather than rule-based generalization.
- Applications are organized across autonomous driving, robotics, and social simulacra. For robotics, DayDreamer allows robots to learn locomotion in real-world environments within hours, and SWIM enables task generalization with less than 30 minutes of interaction; for social simulacra, AI Town is described as a world model composed of 25 generative agents.
- Open challenges include physical rules and counterfactual simulation, enriching the social dimension, bridging simulation and reality with embodied intelligence, simulation efficiency, ethical/safety concerns, and benchmarking. Benchmark examples include WorldScore across 3,000 camera-specified scenes, PhysBench with 10k video-image-text triplets, and UrbanVideo-Bench with 5.2k drone clips.
Methods
- Design: a comprehensive literature survey that formally defines and categorizes world models around two functions—implicit representation and future prediction—and summarizes representative articles plus code repositories.
- Data/sample: no primary dataset or experimental sample; the paper synthesizes existing literature across model-based RL, LLM/MLLM world knowledge, video generation, embodied environments, autonomous driving, robotics, and social simulacra, with summary tables of representative works.
- Main analysis: qualitative synthesis and taxonomy development; it compares technical approaches, application emphases, and open problems rather than conducting meta-analysis or empirical evaluation.
Limitations
- The paper states that benchmarking world models is challenging because the community pursues divergent goals, uses heterogeneous technical approaches, and spans wide-ranging application domains, so there is no single canonical task or metric. It also states that evaluating the realism of generated human behavior still largely relies on subjective human assessment, which is hard to scale.
- The paper states that current video generators achieve visual realism but largely fail on tasks requiring understanding of optics, fluid dynamics, or magnetism, and that data-driven scaling alone is insufficient to recover robust physical laws. It also identifies a longstanding simulation-to-reality gap for embodied agents.
- Evident from the text: as a survey, it presents no primary experiments or quantitative validation of its own; its coverage is organized around representative works and categories rather than a systematic search protocol with inclusion/exclusion criteria.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures







Verification
Claims the paper makes
33 Valid · 8 Not valid · 13 Not checked · 31 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“After pretraining on large-scale web text and books [145,201], large language models attain extensive knowledge about the real-world and common sense relevant to daily life.”
§ Understanding World or Predicting Future? A Comprehensive Survey of World Models > 3.2 World Knowledge Learned by Models
What the sources say
- Not matchedcites [145] 145. OpenAI. 2022. Introducing ChatGPT. Retrieved May 06, 2025 from https://openai.com/blog/chatgpt.
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Supportedcites [201] 201. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models. arXiv:2302.13971. Retrieved from https://arxiv.org/abs/2302.13971 source
The citation follows the pretraining clause, so the relevant attribution is that LLMs are pretrained on large-scale web text and books. Section 2.1 (p. 2) reports CommonCrawl and C4 web data and Gutenberg and Books3 book corpora in LLaMA’s pretraining mixture; it also says these data sources have been used to train other LLMs. (LLM, conf=0.90)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Ha et al. [65,66] significantly revived and popularized the term”
§ Understanding World or Predicting Future? A Comprehensive Survey of World Models > 2 Background and Categorization
What the sources say
- Not matchedcites [65] 65. David Ha and Jürgen Schmidhuber. 2018. Recurrent world models facilitate policy evolution. Advances in Neural Information Processing Systems 31 (2018).
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Partly supportedcites [66] 66. David Ha and Jürgen Schmidhuber. 2018. World models. arXiv:1803.10122. Retrieved from https://arxiv.org/abs/1803. 10122 sourceExperimental — accuracy not yet measured
The Abstract (p. 1) describes a neural-network world model that learns compressed spatial and temporal representations, supporting the model-description part of the claim. The Introduction (p. 2) discusses earlier RNN-based world models, but our copy does not establish that this 2018 paper significantly revived or popularized the term. (LLM, conf=0.85)
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“early work by Ha and Schmidhuber [66] focused on abstracting the external world to gain a deep understanding of its underlying mechanisms.”
§ Understanding World or Predicting Future? A Comprehensive Survey of World Models > 1 Introduction
What the sources say
- Partly supportedcites [66] 66. David Ha and Jürgen Schmidhuber. 2018. World models. arXiv:1803.10122. Retrieved from https://arxiv.org/abs/1803. 10122 sourceExperimental — accuracy not yet measured
Section 3.1 (p. 4) says the world model learns a low-dimensional representation of observations, supporting the abstraction part of the claim. It describes using that representation to train a controller, however, not gaining a deep understanding of the external world's underlying mechanisms. The Introduction (p. 2) likewise frames the work around world models and agent training. (LLM, conf=0.82)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“However, by generating video-like simulations that capture continuous spatial and temporal dynamics, world models [146, 233] have evolved to address more complex, dynamic environments.”
§ Understanding World or Predicting Future? A Comprehensive Survey of World Models > 4 Future Prediction of the Physical World 4.1 World Model as Video Generation
What the sources say
- Not matchedcites [146] 146. OpenAI. 2024. Sora: Creating video from text. Retrieved May 06, 2025 from https://openai.com/sora.
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Partly supportedcites [233] 233. Deshun Yang, Luhui Hu, Yu Tian, Zihao Li, Chris Kelly, Bang Yang, Cindy Yang, and Yuexian Zou. 2024. WorldGPT: A sora-inspired video AI agent as rich world models from text and image inputs. arXiv:2403.07944. Retrieved from https://arxiv.org/abs/2403.07944 sourceExperimental — accuracy not yet measured
The Introduction (page not specified) describes video generation using dynamic scene modeling and spatiotemporal prediction, and Section 2.3 describes continuous video sequences with time-and-space consistency. These passages support the video and dynamics aspect, but they do not establish the broader claim that world models have evolved to address more complex, dynamic environments. (LLM, conf=0.82)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
Every source the paper cites for this claim was checked and supports it.
In the paper
“For example, the emergence of deepfake technology gives rise to large-scale misuses that have widespread negative effects on social, economic, and political systems [214].”
§ Understanding World or Predicting Future? A Comprehensive Survey of World Models > Simulating Unsafe Scenario.
What the sources say
- Supportedcites [214] 214. Mika Westerlund. 2019. The emergence of deepfake technology: A review. Technology Innovation Management Review 9, 11 (2019), 39-52. source
The Introduction says deepfakes can quickly reach millions of people and negatively affect society. The Discussion and Conclusion describes them as a major threat to society, the political system, and businesses, detailing disinformation, election interference, and cybersecurity harms. (LLM, conf=0.89)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
10 more claims · 4 with their cited sources checked
Create a free account to see all 85 claims and every source verdict.
- It may state two separate claims as one. 35
- Probably not a substantial claim. 53
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Understanding World or Predicting Future? A Comprehensive Survey of World Models > 5.1.1 Learning Implicit Representations.
“OmniDrive [107] sets up llm-based agents and covers multiple tasks including scene description, counterfactual reasoning and decision making through visual question-answering.”
§ Understanding World or Predicting Future? A Comprehensive Survey of World Models > References
“107. Varun Ravi Kumar, Senthil Yogamani, Hazem Rashed, Ganesh Sistu, Christian Witt, Isabelle Leang, Stefan Milz, and Patrick Mäder. 2023. OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Au- tonomous Driving.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Understanding World or Predicting Future? A Comprehensive Survey of World Models > 5.1.2 World Simulators.
“Copilot4D [248] constructs a world model by predicting changes in radar point cloud data.”
§ Understanding World or Predicting Future? A Comprehensive Survey of World Models > 5.1.1 Learning Implicit Representations.
“Copilot4D [248] 2024 Lidar Diffusion Point Cloud Generation”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Understanding World or Predicting Future? A Comprehensive Survey of World Models > 2 Background and Categorization
“foundational work such as Marvin Minsky's frame representation in the 1960s [136]”
§ Understanding World or Predicting Future? A Comprehensive Survey of World Models > References
“136. Marvin Minsky. 1974. A framework for representing knowledge. MIT, Cambridge.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Understanding World or Predicting Future? A Comprehensive Survey of World Models > 7 Conclusion
“| Indoor | Matterport 3D [18] | Home | 2018 | 90 | V | ✗ | ✗ |”
§ Understanding World or Predicting Future? A Comprehensive Survey of World Models > References
“18. Angel Chang, Angela Dai, Thomas Funkhouser, Maciej Halber, Matthias Niessner, Manolis Savva, Shuran Song, Andy Zeng, and Yinda Zhang. 2017. Matterport3D: Learning from RGB-D data in indoor environments.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Understanding World or Predicting Future? A Comprehensive Survey of World Models > 5.1.1 Learning Implicit Representations.
“Pointnet [158], introduced in 2017, was the first to employ deep learning methods for processing point cloud data.”
§ Understanding World or Predicting Future? A Comprehensive Survey of World Models > References
“158. Charles R. Qi, Li Yi, Hao Su, and Leonidas J. Guibas. 2017. PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space.”
Citations
Cites
Items 201–305 of 305
StreamingT2V: Consistent, Dynamic, and Extendable Long Video Generation from Text
Henschel, R., Khachatryan, L., Poghosyan, H., Hayrapetyan, D., Tadevosyan, V., Wang, Z., Navasardyan, S., & Shi, H. (2024) · arXiv · doi:10.48550/arxiv.2403.14773 · Cited by 44
Piao, J., Yan, Y., Zhang, J., Li, N., Yan, J., Lan, X., Lu, Z., Zheng, Z., Wang, J. Y., Zhou, D., et al. (2025) · arXiv · doi:10.48550/arxiv.2502.08691 · Cited by 43 · cited 4×
“Among them, AgentSociety [195] stands out as the most advanced effort to date, providing a large-scale and versatile environment for examining polarization, policy interventions, and other phenomena central to societal intelligence.”
InfluentialSelf-citationLLM-PBE: Assessing Data Privacy in Large Language Models
Li, Q., Hong, J., Xie, C., Tan, J., Xin, R., Hou, J., Yin, X., Wang, Z., Hendrycks, D., Wang, Z., et al. (2024) · arXiv · doi:10.48550/arxiv.2408.12787 · Cited by 42
LLM-PBE: Assessing Data Privacy in Large Language Models
Li, Q., Hong, J., Xie, C., Tan, J., Xin, R., Hou, J., Yin, X., Wang, Z., Hendrycks, D., Wang, Z., et al. (2024) · Proceedings of the VLDB Endowment · doi:10.14778/3681954.3681994 · Cited by 42
GeoLLM: Extracting Geospatial Knowledge from Large Language Models
Manvi, R., Khanna, S., Mai, G., Burke, M., Lobell, D., & Ermon, S. (2023) · arXiv · doi:10.48550/arxiv.2310.06213 · Cited by 41 · cited 5×
“AirScape [323] 2025 ACM MM CityNavAgent [315] 2025 ACL CityEQA [326] 2025 EMNLP UrbanVideo-Bench [322] 2025 ACL Urban Analytics Knowledge CityGPT [57] 2025 KDD UrbanLLaVA [59] 2025 ICCV GeoLLM [171] 2024 ICLR Prediction GPS-to-Image [56] 2025 CVPR AI’s Blind Spots [17] 2025 arXiv”
InfluentialNL2Plan: Robust LLM-Driven Planning from Minimal Text DescriptionsPreprint
Gestrin, E., Kuhlmann, M., & Seipp, J. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2405.04215 · Cited by 38
EVA: An Embodied World Model for Future Video AnticipationPreprint
Chi, X., Fan, C.-K., Zhang, H., Qi, X., Zhang, R., Chen, A., Chan, C.-M., Xue, W., Liu, Q., Zhang, S., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2410.15461 · Cited by 38
AVID: Adapting Video Diffusion Models to World ModelsPreprint
Rigter, M. P., Gupta, T., Hilmkil, A., & Ma, C. (2024) · Qeios · doi:10.32388/h7bfdw · Cited by 37
Zhao, G., Li, G., Chen, W., & Yu, Y. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01542 · Cited by 36 · cited 1×
“Zhao et al. [324] further combine LLMs and open-vocabulary detection to construct the relationship between multi-modal signals and key information in navigation.”
A0C: Alpha Zero in Continuous Action Space
Moerland, T. M., Broekens, J., Plaat, A., & Jonker, C. M. (2018) · arXiv · doi:10.48550/arxiv.1805.09613 · Cited by 34
Probabilistic Adaptation of Text-to-Video Models
Yang, M., Du, Y., Dai, B., Schuurmans, D., Tenenbaum, J. B., & Abbeel, P. (2023) · International Conference on Learning Representations · doi:10.48550/arxiv.2306.01872 · Cited by 34
Language Models Meet World Models: Embodied Experiences Enhance Language Models
Xiang, J., Tao, T., Gu, Y., Shu, T., Wang, Z., Yang, Z., & Hu, Z. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-3295 · Cited by 34
Xu, W., Lyu, D., Wang, W., Feng, J., Gao, C., & Li, Y. (2025) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2025.acl-long.567 · Cited by 34
Self-citationExploring Large Language Models for Communication Games: An Empirical Study on Werewolf
Xu, Y., Wang, S., Li, P., Luo, F., Wang, X., Liu, W., & Liu, Y. (2023) · arXiv · doi:10.48550/arxiv.2309.04658 · Cited by 32
Kawasaki, A., & Seki, A. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9196738 · Cited by 30 · cited 1×
“More recently, with the development of LLMs, some works [21, 48, 238] seek to provide a flexible environment generation pipeline, supporting the generation of arbitrary indoor environments with language instructions.”
General agents contain world models
Richens, J., Abel, D., Bellot, A., & Everitt, T. (2025) · arXiv · doi:10.48550/arxiv.2506.01622 · Cited by 30
Model-free Deep Reinforcement Learning for Urban Autonomous DrivingPreprint
Chen, J., Yuan, B., & Tomizuka, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.09503 · Cited by 30
MetaUrban: An Embodied AI Simulation Platform for Urban MicromobilityPreprint
Wu, W., He, H., He, J., Wang, Y., Duan, C., Liu, Z., Li, Q., & Zhou, B. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2407.08725 · Cited by 30
KoLA: Carefully Benchmarking World Knowledge of Large Language Models
Yu, J., Wang, X., Tu, S., Cao, S., Zhang-Li, D., Lv, X., Peng, H., Yao, Z., Zhang, X., Li, H., et al. (2023) · arXiv · doi:10.48550/arxiv.2306.09296 · Cited by 29 · cited 1×
“Xiang et al.[280] deploys an embodied agent in a world model, the simulator of VirtualHome [198], where the corresponding KoLA [305] 2024 ICLR Language Benchmark EWOK [112] 2024 arxiv Language Benchmark Geometry of Concepts [151] 2024 arxiv Language Analysis BLEnD [181] 2024 NeurIPS Language…”
Alexa Arena: A User-Centric Interactive Platform for Embodied AI
Gao, Q., Thattai, G., Shakiah, S. M., Gao, X., Pansare, S., Sharma, V., Sukhatme, G. S., Shi, H., Yang, B., Zhang, D., et al. (2023) · neural information processing systems · doi:10.52202/075280-0841 · Cited by 29
Zhao, B., Fang, J., Dai, Z., Wang, Z., Zha, J., Zhang, W., Gao, C., Wang, Y., Cui, J., Chen, X., et al. (2025) · Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2025.acl-long.1558 · Cited by 28 · cited 5×
“PhysBench (10k video–image–text triplets) reveals systematic gaps in object properties, relations, and dynamics for modern VLMs [37]; UrbanVideo-Bench (5.2k drone clips, 16 task types) diagnoses deficits in recall, navigation, and causal reasoning in long egocentric streams [322].”
InfluentialSelf-citationGAIA-1: A Generative World Model for Autonomous Driving
Hu, A., Russell, L., Yeo, H., Murez, Z., Fedoseev, G., Kendall, A., Shotton, J., & Corrado, G. (2023) · arXiv · doi:10.48550/arxiv.2309.17080 · Cited by 28 · cited 5×
“Transformer GAIA-1 [101] Generative world model predicting driving scenarios for autonomous driving.”
InfluentialCertifying LLM Safety against Adversarial Prompting
Kumar, A., Agarwal, C., Srinivas, S., Li, A. J., Feizi, S., & Lakkaraju, H. (2023) · arXiv · doi:10.48550/arxiv.2309.02705 · Cited by 28
Real-Time Motion Prediction via Heterogeneous Polyline Transformer with Relative Pose EncodingPreprint
Zhang, Z., Liniger, A., Sakaridis, C., Yu, F., & Gool, L. V. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.12970 · Cited by 27
CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer
Yang, Z., Teng, J., Zheng, W., Ding, M., Huang, S., Xu, J., Yang, Y., Hong, W., Zhang, X., Feng, G., et al. (2024) · arXiv · doi:10.48550/arxiv.2408.06072 · Cited by 25 · cited 1×
“Following the success of Sora in generating high-quality videos, the past two years have witnessed the emergence of several large-scale video generative base models, such as OpenSora [333], CogVideoX [299], and Wan [258].”
Self-citationLLMs achieve adult human performance on higher-order theory of mind tasks
Street, W., Siy, J. O., Keeling, G., Baranes, A., Barnett, B., McKibben, M., Kanyere, T., Lentz, A., Arcas, B. A. Y., & Dunbar, R. I. M. (2024) · arXiv · doi:10.48550/arxiv.2405.18870 · Cited by 24
AgentGroupChat: An Interactive Group Chat Simulacra For Better Eliciting Emergent BehaviorPreprint
Gu, Z., Zhu, X., Guo, H., Zhang, L., Cai, Y., Shen, H., Chen, J., Ye, Z., Dai, Y., Gao, Y., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2403.13433 · Cited by 24
Genie: Generative Interactive Environments
Bruce, J., Dennis, M., Edwards, A., Parker-Holder, J., Shi, Y., Hughes, E., Lai, M., Mavalankar, A., Steigerwald, R., Apps, C., et al. (2024) · arXiv · doi:10.48550/arxiv.2402.15391 · Cited by 23 · cited 3×
“Long-term NUWA-XL [241] “Coarse-to-fine” Diffusion over Diffusion architecture for long video generation.”
RILA: Reflective and Imaginative Language Agent for Zero-Shot Semantic Audio-Visual Navigation
Yang, Z., Lin, J., Chen, P., Cherian, A., Marks, T. K., Roux, J. L., & Gan, C. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01538 · Cited by 22 · cited 1×
“Meanwhile, Yang et al. [297] utilize an LLM-based imaginative assistant to infer the global semantic graph as the world model based on the environment perception, and another reflective planner to directly generate actions.”
Learning the complexity of urban mobility with deep generative network
Yuan, Y., Ding, J., Jin, D., & Li, Y. (2025) · PNAS Nexus · doi:10.1093/pnasnexus/pgaf081 · Cited by 22 · cited 1×
“Simulating the physical elements alone is not sufficient for an advanced world model, since human behavior and social interaction also play a crucial role in many important scenarios [68, 78, 310].”
Self-citationSAPIEN: A SimulAted Part-based Interactive ENvironmentPreprint
Xiang, F., Qin, Y., Mo, K., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.08515 · Cited by 22
Embodied Agent Interface: Benchmarking LLMs for Embodied Decision Making
Li, M., Zhao, S., Wang, Q., Wang, K., Zhou, Y., Srivastava, S., Gokmen, C., Lee, T., Li, L. E., Zhang, R., et al. (2024) · arXiv · doi:10.48550/arxiv.2410.07166 · Cited by 21 · cited 2×
“…Abilities [285] Psychometric framing; five spatial skills (perception/relations/orientation/rotation/visualization) Embodied decision making EAI [147] LLM-based agents; module-level (goal/subgoal/action/transition) evaluation; error taxonomy EWMBench [311] AgiBotWorld manipulation/navigation;…”
Embodied Agent Interface: Benchmarking LLMs for Embodied Decision Making
Li, M., Zhao, S., Wang, Q., Wang, K., Zhou, Y., Srivastava, S., Gokmen, C., Lee, T., Li, L., Zhang, R., et al. (2024) · Advances in Neural Information Processing Systems 37 · doi:10.52202/079017-3188 · Cited by 21
DriveDreamer: Towards Real-world-driven World Models for Autonomous Driving
Wang, X., Zhu, Z., Huang, G., Chen, X., Zhu, J., & Lu, J. (2023) · arXiv · doi:10.48550/arxiv.2309.09777 · Cited by 20
Do generative video models understand physical principles?
Motamed, S., Culp, L., Swersky, K., Jaini, P., & Geirhos, R. (2025) · arXiv · doi:10.48550/arxiv.2501.09038 · Cited by 19
Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language ModelsPreprint
Xu, F., Hao, Q., Zong, Z., Wang, J., Zhang, Y., Wang, J., Lan, X., Gong, J., Ouyang, T., Meng, F., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2501.09686 · Cited by 18
Self-citationCityBench: Evaluating the Capabilities of Large Language Models for Urban Tasks
Feng, J., Zhang, J., Liu, T., Zhang, X., Ouyang, T., Yan, J., Du, Y., Guo, S., & Li, Y. (2024) · arXiv · doi:10.48550/arxiv.2406.13945 · Cited by 18
Self-citationThe AI Economist: Optimal Economic Policy Design via Two-level Deep Reinforcement LearningPreprint
Zheng, S., Trott, A., Srinivasa, S., Parkes, D. C., & Socher, R. (2021) · SSRN Electronic Journal · doi:10.2139/ssrn.3900018 · Cited by 18
WorldScore: A Unified Evaluation Benchmark for World Generation
Duan, H., Yu, H.-X., Chen, S., Fei-Fei, L., & Wu, J. (2025) · arXiv · doi:10.48550/arxiv.2504.00983 · Cited by 17
Think Twice: Perspective-Taking Improves Large Language Models’ Theory-of-Mind Capabilities
Wilf, A., Lee, S., Liang, P. P., & Morency, L.-P. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.451 · Cited by 17
WorldScore: A Unified Evaluation Benchmark for World Generation
Duan, H., Yu, H., Chen, S., Fei–Fei, L., & Wu, J. (2025) · 2025 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.02573 · Cited by 17
SRAP-Agent: Simulating and Optimizing Scarce Resource Allocation Policy with LLM-based Agent
Ji, J., Li, Y., Liu, H., Du, Z., Wei, Z., Qi, Q., Shen, W., & Lin, Y. (2024) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2024.findings-emnlp.15 · Cited by 16
Self-citation3D-VLA: A 3D Vision-Language-Action Generative World Model
Zhen, H., Qiu, X., Chen, P., Yang, J., Yan, X., Du, Y., Hong, Y., & Gan, C. (2024) · arXiv · doi:10.48550/arxiv.2403.09631 · Cited by 15
Is Sora a World Simulator? A Comprehensive Survey on General World Models and Beyond
Zhu, Z., Wang, X., Zhao, W., Min, C., Li, B., Deng, N., Dou, M., Wang, Y., Shi, B., Wang, K., et al. (2024) · arXiv · doi:10.48550/arxiv.2405.03520 · Cited by 15
Cooperate or Collapse: Emergence of Sustainable Cooperation in a Society of LLM Agents
Piatti, G., Jin, Z., Kleiman-Weiner, M., Schölkopf, B., Sachan, M., & Mihalcea, R. (2024) · arXiv · doi:10.48550/arxiv.2404.16698 · Cited by 14
World Model on Million-Length Video And Language With Blockwise RingAttention
Liu, H., Yan, W., Zaharia, M., & Abbeel, P. (2024) · arXiv · doi:10.48550/arxiv.2402.08268 · Cited by 14
The Geometry of Concepts: Sparse Autoencoder Feature Structure
Li, Y., Michaud, E. J., Baek, D. D., Engels, J., Sun, X., & Tegmark, M. (2024) · arXiv · doi:10.48550/arxiv.2410.19750 · Cited by 14
Self-citationCooperate or Collapse: Emergence of Sustainable Cooperation in a Society of LLM Agents
Piatti, G., Jin, Z., Kleiman-Weiner, M., Schölkopf, B., Sachan, M., & Mihalcea, R. (2024) · Advances in Neural Information Processing Systems 37 · doi:10.52202/079017-3548 · Cited by 14
AVLEN: Audio-Visual-Language Embodied Navigation in 3D Environments
Paul, S., Roy-Chowdhury, A. K., & Cherian, A. (2022) · arXiv · doi:10.48550/arxiv.2210.07940 · Cited by 13 · cited 2×
“AVLEN [192] further supplements audio signals allowing agents to execute tasks such as object manipulation and navigation in household-like settings.”
Large Language Models Empowered Agent-based Modeling and Simulation: A Survey and PerspectivesPreprint
Gao, C., Lan, X., Li, N., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2312.11970 · Cited by 12 · cited 1×
“Simulating the physical elements alone is not sufficient for an advanced world model, since human behavior and social interaction also play a crucial role in many important scenarios [68, 78, 310].”
Self-citationProject Sid: Many-agent simulations toward AI civilization
AL, A., Ahn, A., Becker, N., Carroll, S., Christie, N., Cortes, M., Demirci, A., Du, M., Li, F., Luo, S., et al. (2024) · arXiv · doi:10.48550/arxiv.2411.00114 · Cited by 12
Mean Square Stability of Impulsive Stochastic Differential Systems
Yang, S., Shi, K., & Li, M. (2011) · International Journal of Differential Equations · doi:10.1155/2011/613695 · Cited by 11
Unleashing Large-Scale Video Generative Pre-training for Visual Robot Manipulation
Wu, H., Jing, Y., Cheang, C., Chen, G., Xu, J., Li, X., Liu, M., Li, H., & Kong, T. (2023) · arXiv · doi:10.48550/arxiv.2312.13139 · Cited by 11
S$^3$: Social-network Simulation System with Large Language Model-Empowered Agents
Gao, C., Lan, X., Lu, Z., Mao, J., Piao, J., Wang, H., Jin, D., & Li, Y. (2023) · arXiv · doi:10.48550/arxiv.2307.14984 · Cited by 10 · cited 2×
“This progress is driven in part by advances in neural architectures [99, 255] and learning algorithms [206, 224], which enable robots to build implicit representations that capture key aspects of the external world.”
Self-citationSora as a World Model? A Complete Survey on Text-to-Video Generation
Puspitasari, F. D., Zhang, C., Cho, J., Haider, A., Eman, N. U., Amin, O., Mankowski, A., Umair, M., Zheng, J., Zheng, S., et al. (2024) · arXiv · doi:10.48550/arxiv.2403.05131 · Cited by 10
Tang, H., Key, D., & Ellis, K. (2024) · arXiv · doi:10.48550/arxiv.2402.12275 · Cited by 9
ALOHA 2: An Enhanced Low-Cost Hardware for Bimanual Teleoperation
ALOHA 2 Team, Aldaco, J., Armstrong, T., Baruch, R., Bingham, J., Chan, S., Draper, K., Dwibedi, D., Finn, C., Florence, P., et al. (2024) · arXiv · doi:10.48550/arxiv.2405.02292 · Cited by 8
TD-MPC2: Scalable, Robust World Models for Continuous Control
Hansen, N., Su, H., & Wang, X. (2023) · arXiv · doi:10.48550/arxiv.2310.16828 · Cited by 8
Self-citationHow Far is Video Generation from World Model: A Physical Law Perspective
Kang, B., Yue, Y., Lu, R., Lin, Z., Zhao, Y., Wang, K., Huang, G., & Feng, J. (2024) · arXiv · doi:10.48550/arxiv.2411.02385 · Cited by 7
Self-citationGR-2: A Generative Video-Language-Action Model with Web-Scale Knowledge for Robot Manipulation
Cheang, C.-L., Chen, G., Jing, Y., Kong, T., Li, H., Li, Y., Liu, Y., Wu, H., Xu, J., Yang, Y., et al. (2024) · arXiv · doi:10.48550/arxiv.2410.06158 · Cited by 7
Self-citationLEGENT: Open Platform for Embodied AgentsPreprint
Cheng, Z., Wang, Z., Hu, J., Hu, S., Liu, A., Tu, Y., Li, P., Shi, L., Liu, Z., & Sun, M. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2404.18243 · Cited by 7
CityGPT: Empowering Urban Spatial Cognition of Large Language ModelsPreprint
Feng, J., Liu, T., Du, Y., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2406.13948 · Cited by 7
Self-citationBehaveGPT: A Foundation Model for Large-scale User Behavior ModelingPreprint
Gong, J., Ding, J., Meng, F., Yang, C., Chen, H., Wang, Z., Lu, H., & Li, Y. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2505.17631 · Cited by 7
Self-citationDo Vision and Language Models Share Concepts? A Vector Space Alignment Study
Li, J., Kementchedjhieva, Y., Fierro, C., & Søgaard, A. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00698 · Cited by 6 · cited 1×
“Also, Li et al.[143] demonstrate that language models partially converge towards representations isomorphic to those of vision models.”
Yan, X., Zhang, H., Cai, Y., Guo, J., Qiu, W., Gao, B., Zhou, K., Zhao, Y., Jin, H., Gao, J., et al. (2024) · arXiv · doi:10.48550/arxiv.2401.08045 · Cited by 6
IRASim: A Fine-Grained World Model for Robot Manipulation
Zhu, F., Wu, H., Guo, S., Liu, Y., Cheang, C., & Kong, T. (2024) · arXiv · doi:10.48550/arxiv.2406.14540 · Cited by 6
COKE: A Cognitive Knowledge Graph for Machine Theory of MindPreprint
Wu, J., Chen, Z., Deng, J., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.05390 · Cited by 5
WorldGPT: A Sora-Inspired Video AI Agent as Rich World Models from Text and Image Inputs
Yang, D., Hu, L., Tian, Y., Li, Z., Kelly, C., Yang, B., Yang, C., & Zou, Y. (2024) · arXiv · doi:10.48550/arxiv.2403.07944 · Cited by 5
Network Formation and Dynamics among Multi-LLMsPreprint
Papachristou, M., & Yuan, Y. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5410522 · Cited by 4
Self-citationRoboDreamer: Learning Compositional World Models for Robot Imagination
Zhou, S., Du, Y., Chen, J., Li, Y., Yeung, D.-Y., & Gan, C. (2024) · arXiv · doi:10.48550/arxiv.2404.12377 · Cited by 4
Self-citationWayformer: Motion Forecasting via Simple & Efficient Attention NetworksPreprint
Nayakanti, N., Al‐Rfou, R., Zhou, A., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2207.05844 · Cited by 4
ConsistI2V: Enhancing Visual Consistency for Image-to-Video Generation
Ren, W., Yang, H., Zhang, G., Wei, C., Du, X., Huang, W., & Chen, W. (2024) · arXiv · doi:10.48550/arxiv.2402.04324 · Cited by 4 · cited 2×
“ConsistI2V [210] Enhances visual consistency in image-to-video generation.”
Xu, W., Lyu, D., Wang, W., Feng, J., Gao, C., & Li, Y. (2025) · arXiv · doi:10.48550/arxiv.2502.11859 · Cited by 4
Self-citationFrom Efficient Multimodal Models to World Models: A Survey
Mai, X., Tao, Z., Lin, J., Wang, H., Chang, Y., Kang, Y., Wang, Y., & Zhang, W. (2024) · arXiv · doi:10.48550/arxiv.2407.00118 · Cited by 3
PhysBench: Benchmarking and Enhancing Vision-Language Models for Physical World Understanding
Chow, W., Mao, J., Li, B., Seita, D., Guizilini, V., & Wang, Y. (2025) · arXiv · doi:10.48550/arxiv.2501.16411 · Cited by 3
WorldSimBench: Towards Video Generation Models as World Simulators
Qin, Y., Shi, Z., Yu, J., Wang, X., Zhou, E., Li, L., Yin, Z., Liu, X., Sheng, L., Shao, J., et al. (2024) · arXiv · doi:10.48550/arxiv.2410.18072 · Cited by 3 · cited 2×
“…links perceptual quality to control by pairing human preferences with action-level consistency across sandbox, driving, and manipulation settings [203]; WorldScore complements this with a camera-specified protocol that decomposes performance into controllability, visual quality, and dynamics…”
GenSim2: Scaling Robot Data Generation with Multi-modal and Reasoning LLMs
Hua, P., Liu, M., Macaluso, A., Lin, Y., Zhang, W., Xu, H., & Wang, L. (2024) · arXiv · doi:10.48550/arxiv.2410.03645 · Cited by 2
Ivanova, A. A., Sathe, A., Lipkin, B., Kumar, U., Radkani, S., Clark, T. H., Kauf, C., Hu, J., Pramod, R. T., Grand, G., et al. (2024) · arXiv · doi:10.48550/arxiv.2405.09605 · Cited by 2 · cited 2×
“Xiang et al.[280] deploys an embodied agent in a world model, the simulator of VirtualHome [198], where the corresponding KoLA [305] 2024 ICLR Language Benchmark EWOK [112] 2024 arxiv Language Benchmark Geometry of Concepts [151] 2024 arxiv Language Analysis BLEnD [181] 2024 NeurIPS Language Benchmark PIGEON [139] 202…”
Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllability
Gao, S., Yang, J., Chen, L., Chitta, K., Qiu, Y., Geiger, A., Zhang, J., & Li, H. (2024) · arXiv · doi:10.48550/arxiv.2405.17398 · Cited by 2
MatrixVT: Efficient Multi-Camera to BEV Transformation for 3D PerceptionPreprint
Zhou, H., Ge, Z., Li, Z., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2211.10593 · Cited by 2
Wang, Y., He, J., Fan, L., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2311.17918 · Cited by 2
Li, L., Shao, W., Dong, W., Tian, Y., Zhang, Q., Yang, K., & Zhang, W. (2024) · arXiv · doi:10.48550/arxiv.2401.12888 · Cited by 2 · cited 1×
“The first category primarily focuses on describing the application of world models in specific fields such as video processing and generation [35, 338], autonomous driving [83, 145, 288], and agent-based applications [338].”
Pandora: Towards General World Model with Natural Language Actions and Video States
Xiang, J., Liu, G., Gu, Y., Gao, Q., Ning, Y., Zha, Y., Feng, Z., Tao, T., Hao, S., Shi, Y., et al. (2024) · arXiv · doi:10.48550/arxiv.2406.09455 · Cited by 1
Planning-oriented Autonomous DrivingPreprint
Hu, Y., Yang, J., Chen, L., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2212.10156 · Cited by 1
NUWA-XL: Diffusion over Diffusion for eXtremely Long Video Generation
Yin, S., Wu, C., Yang, H., Wang, J., Wang, X., Ni, M., Yang, Z., Li, L., Liu, S., Yang, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.12346 · Cited by 1
MotionDiffuser: Controllable Multi-Agent Motion Prediction using DiffusionPreprint
Jiang, C. M., Cornman, A., Park, C., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.03083 · Cited by 1
NEWTON: Are Large Language Models Capable of Physical Reasoning?Preprint
Wang, Y. R., Duan, J., Fox, D., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.07018 · Cited by 1
Think Twice: Perspective-Taking Improves Large Language Models' Theory-of-Mind CapabilitiesPreprint
Wilf, A., Lee, S. S., Liang, P. P., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2311.10227 · Cited by 1
Research on Financial Stock Market Prediction Based on the Hidden Quantum Markov Model
Song, X., Chen, W., & Lu, J. (2025) · Mathematics · doi:10.3390/math13152505
Feng, J., Zeng, J., Long, Q., Chen, H., Zhao, J., Xi, Y., Zhou, Z., Yuan, Y., Wang, S., Zeng, Q., et al. (2025) · arXiv · doi:10.48550/arxiv.2504.09848 · cited 1×
“…perspective, we can see that although large language models have demonstrated the ability to capture certain aspects of real-world knowledge [87, 151, 213], it is clear that further efforts are needed to enhance this knowledge to enable broader and more reliable real-world applications [60].”
Self-citationStructured World Models from Human VideosPreprint
Mendonca, R., Bahl, S., & Pathak, D. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2308.10901
Synergy-of-Thoughts: Eliciting Efficient Reasoning in Hybrid Language Models
Shang, Y., Li, Y., Xu, F., & Li, Y. (2024) · arXiv · doi:10.48550/arxiv.2402.02563
Self-citationSRAP-Agent: Simulating and Optimizing Scarce Resource Allocation Policy with LLM-based AgentPreprint
Ji, J., Li, Y., Liu, H., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2410.14152
Self-citationOpenAI, Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.08774
RMA: Rapid Motor Adaptation for Legged Robots
Kumar, A., Fu, Z., Pathak, D., & Malik, J. (2021) · arXiv
Large Language Models are Geographically Biased
Manvi, R., Khanna, S., Burke, M., Lobell, D., & Ermon, S. (2024) · arXiv · doi:10.48550/arxiv.2402.02680
OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving
Kumar, V. R., Yogamani, S., Rashed, H., et al. (2021) · arXiv · doi:10.48550/arxiv.2102.07448
OccSora: 4D Occupancy Generation Models as World Simulators for Autonomous Driving
Wang, L., Zheng, W., Ren, Y., et al. (2024) · arXiv · doi:10.48550/arxiv.2405.20337
Network Formation and Dynamics Among Multi-LLMs
Papachristou, M., & Yuan, Y. (2024) · arXiv · doi:10.48550/arxiv.2402.10659
Self-citationMultiNet: Real-time Joint Semantic Reasoning for Autonomous Driving
Teichmann, M., Weber, M., Zoellner, M., et al. (2016) · arXiv · doi:10.48550/arxiv.1612.07695
Zhao, B., Fang, J., Dai, Z., et al. (2025) · arXiv · doi:10.48550/arxiv.2503.06157
MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge
Fan, L., Wang, G., Jiang, Y., et al. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1333
Automated construction of cognitive maps with visual predictive coding
Gornet, J. A., & Thomson, M. (2023) · doi:10.1101/2023.09.18.558369
Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuning
Nagabandi, A., Kahn, G., Fearing, R. S., et al. (2017) · arXiv · doi:10.48550/arxiv.1708.02596
PointNeXt: Revisiting PointNet++ with Improved Training and Scaling Strategies
Qian, G., Li, Y., Peng, H., Mai, J., Hammoud, H., Elhoseiny, M., & Ghanem, B. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1685
Self-citation
Research works printed but not linked: 25
- Yifan Cui, Xinyi Shan, and Jeanhun Chung. 2024. A feasibility study on RUNWAY GEN-2 for generating realistic style images. International Journal of Internet, Broadcasting and Communication 16, 1 (2024), 99–105.
- Nan Du, Yanping Huang, Andrew M. Dai, Simon Tong, Dmitry Lepikhin, Yuanzhong Xu, Maxim Krikun, Yanqi Zhou, Adams Wei Yu, Orhan Firat, et al. 2022. Glam: Efficient scaling of language models with mixture-of-experts. In Proceedings of the International Conference on Machine Learning. PMLR, 5547–5569.
- Yilun Du, Sherry Yang, Bo Dai, Hanjun Dai, Ofir Nachum, Josh Tenenbaum, Dale Schuurmans, and Pieter Abbeel. 2024. Learning universal policies via text-guided video generation. Advances in Neural Information Processing Systems 36 (2024).
- Ignat Georgiev, Varun Giridhar, Nicklas Hansen, and Animesh Garg. 2025. PWM: Policy learning with multi-task world models. In Proceedings of the 13th International Conference on Learning Representations.
- David Ha and Jürgen Schmidhuber. 2018. Recurrent world models facilitate policy evolution. Advances in Neural Information Processing Systems 31 (2018).
- Haoran He, Chenjia Bai, Ling Pan, Weinan Zhang, Bin Zhao, and Xuelong Li. 2024. Learning an actionable discrete diffusion policy via large-scale actionless video pre-training. In Proceedings of the 38th Annual Conference on Neural Information Processing Systems.
- Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems 33 (2020), 6840–6851.
- Anthony Hu Lloyd Russell Hudson Yeo Zak Murez George Fedoseev Alex Kendall Jamie Shotton and Gianluca Corrado. 2023. GAIA-1: A Generative World Model for Autonomous Driving. arxiv:2309.17080. Retrieved from https://arxiv.org/abs/2309.17080 · arXiv
- Eric Jang, Alex Irpan, Mohi Khansari, Daniel Kappler, Frederik Ebert, Corey Lynch, Sergey Levine, and Chelsea Finn. 2022. Bc-z: Zero-shot task generalization with robotic imitation learning. In Proceedings of the Conference on Robot Learning. PMLR, 991–1002.
- 10.5555/3692070.3692961 · doi:10.5555/3692070.3692961
- Philip Nicholas Johnson-Laird. 1983. Mental Models: Towards a Cognitive Science of Language, Inference, and Consciousness. Number 6. Harvard University Press.
- Subbarao Kambhampati, Karthik Valmeekam, Lin Guan, Mudit Verma, Kaya Stechly, Siddhant Bhambri, Lucas Paul Saldyt, and Anil B. Murthy. 2024. Position: LLMs can’t plan, but can help planning in LLM-modulo frameworks. In Proceedings of the 41st International Conference on Machine Learning.
- Kuaishou. 2024. Kling AI: Next-Generation AI Creative Studio. Retrieved May 06, 2025 from https://www.klingai.com/global/.
- Yann LeCun. 2022. A path towards autonomous machine intelligence version 0.9. 2, 2022-06-27. Open Review 62, 1 (2022), 1–62.
- Junhyuk Oh, Satinder Singh, and Honglak Lee. 2017. Value prediction network. Advances in Neural Information Processing Systems 30 (2017).
- OpenAI. 2024. Sora: Creating video from text. Retrieved May 06, 2025 from https://openai.com/sora.
- Aravind Rajeswaran, Igor Mordatch, and Vikash Kumar. 2020. A game theoretic framework for model based reinforcement learning. In Proceedings of the International Conference on Machine Learning. PMLR, 7953–7963.
- Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. 2015. Faster R-CNN: Towards real-time object detection with region proposal networks. In Proceedings of the Advances in Neural Information Processing Systems.C. Cortes, N. Lawrence, D. Lee, M. Sugiyama, and R. Garnett (Eds.), Vol. 28, Curran Associates, Inc. Retrieved from https://proceedings.neurips.cc/paper_files/paper/2015/file/14bfa6bb14875e45bba028a21ed38046-Paper.pdf
- Alvaro Sanchez-Gonzalez, Jonathan Godwin, Tobias Pfaff, Rex Ying, Jure Leskovec, and Peter Battaglia. 2020. Learning to simulate complex physics with graph networks. In Proceedings of the International Conference on Machine Learning. PMLR, 8459–8468.
- Hongzhi Shi, Jingtao Ding, Yufan Cao, Li Liu, Yong Li, et al. 2022. Learning symbolic models for graph-structured physical mechanism. In Proceedings of the 11th International Conference on Learning Representations.
- A Vaswani. 2017. Attention is all you need. Advances in Neural Information Processing Systems 30 (2017).
- Philipp Wu, Alejandro Escontrela, Danijar Hafner, Pieter Abbeel, and Ken Goldberg. 2023. Daydreamer: World models for physical robot learning. In Proceedings of the Conference on Robot Learning. PMLR, 2226–2240.
- Wilson Yan, Danijar Hafner, Stephen James, and Pieter Abbeel. 2023. Temporally consistent transformers for video generation. In Proceedings of the International Conference on Machine Learning. PMLR, 39062–39098.
- Sherry Yang, Yilun Du, Seyed Kamyar Seyed Ghasemipour, Jonathan Tompson, Leslie Pack Kaelbling, Dale Schuurmans, and Pieter Abbeel. 2024. Learning interactive real-world simulators. In Proceedings of the 12th International Conference on Learning Representations.
- 10.1007/978-3-031-72624-8_4 · DOI · Paper page
Cited by
Nie, T., Sun, J., & Ma, W. (2025) · Artificial Intelligence for Transportation · doi:10.1016/j.ait.2025.100003 · Cited by 116
CityGPT: Empowering Urban Spatial Cognition of Large Language Models
Feng, J., Liu, T., Du, Y., Guo, S., Lin, Y., & Li, Y. (2025) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3711896.3736878 · Cited by 137
Self-citationZhang, Y., Tian, J., & Xiong, Q. (2025) · Frontiers in Robotics and AI · doi:10.3389/frobt.2025.1668910 · Cited by 33
Self-citationZhao, C., Liu, G., Zhang, R., Liu, Y., Wang, J., Kang, J., Niyato, D. T., Li, Z., Shen, X., Han, Z., et al. (2026) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2026.3658762 · Cited by 41
“Most world models are implemented as neural architectures with three core components that achieve the perception-prediction cycle [20]: • Encoder: The encoder network compresses high-dimensional observations, such as images, sensor data, etc., into a compact latent state.”
InfluentialCityBench: Evaluating the Capabilities of Large Language Models for Urban Tasks
Feng, J., Zhang, J., Liu, T., Zhang, X., Ouyang, T., Yan, J., Du, Y., Guo, S., & Li, Y. (2025) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3711896.3737375 · Cited by 97
Self-citation
Metadata
- Authors
- Ding, J., Zhang, Y., Shang, Y., Zhang, Y., Zong, Z., Feng, J., Yuan, Y., Su, H., Li, N., Sukiennik, N., Xu, F., & Li, Y.
- Affiliations
- Tsinghua University (CN)
- ORCID
- Venue
- ACM Computing Surveys
- Published
- 2025-09-09
- Volume
- 58
- Issue
- 3
- Pages
- 1-38
- ISSN
- 0360-0300 (ISSN-L)
- 1557-7341
- OpenAlex
- W4411735734
- Cited by
- 169
- References
- 48
- Citations by year
- 2026: 122 · 2025: 46
- FWCI
- 43.25
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- ACM Computing Surveys (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- National Key Research and Development Program of China (2024YFC3307603)
- National Natural Science Foundation of China (62476152 and U24B20180)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Physical Sciences › Computer Science › Artificial Intelligence › Psychiatry, Mental Health, Neuroscience
- Keywords
- world models, internal representations, future state prediction, artificial general intelligence, autonomous driving, robotics, social simulation, multimodal large language models, video generation models, decision-making
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-01, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.