MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learning
Abstract
Driving safely requires multiple capabilities from human and intelligent agents, such as the generalizability to unseen environments, the safety awareness of the surrounding traffic, and the decision-making in complex multi-agent settings. Despite the great success of Reinforcement Learning (RL), most of the RL research works investigate each capability separately due to the lack of integrated environments. In this work, we develop a new driving simulation platform called MetaDrive to support the research of generalizable reinforcement learning algorithms for machine autonomy. MetaDrive is highly compositional, which can generate an infinite number of diverse driving scenarios from both the procedural generation and the real data importing. Based on MetaDrive, we construct a variety of RL tasks and baselines in both single-agent and multi-agent settings, including benchmarking generalizability across unseen scenes, safe exploration, and learning multi-agent traffic. The generalization experiments conducted on both procedurally generated scenarios and real-world scenarios show that increasing the diversity and the size of the training set leads to the improvement of the RL agent's generalizability. We further evaluate various safe reinforcement learning and multi-agent reinforcement learning algorithms in MetaDrive environments and provide the benchmarks. Source code, documentation, and demo video are available at https://metadriverse.github.io/metadrive.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Forcing an overtake on a narrow, two-lane road with oncoming congestion violates both ethical norms and common etiquette and risks a collision with the cyclist or opposing traffic due to cyclist unpredictability.
“In reality, forcing an overtake on a narrow, two-lane road with oncoming congestion violates both ethical norms and common etiquette [33], and risks a collision either with the cyclist or opposing traffic due to the unpredictability of the cyclist.”
From Ethics-Aware Safe Reinforcement Learning for Rare-Event Risk Control in Interactive Urban Driving (2026) · cites this paper as [33]
Cites
Proximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625 · see citing works
CARLA: An open urban driving simulator
Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., & Koltun, V. (2017) · arXiv · doi:10.48550/arxiv.1711.03938 · Cited by 3,718 · see citing works
AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles
Shah, S., Dey, D., Lovett, C., & Kapoor, A. (2018) · Springer Proceedings in Advanced Robotics · doi:10.1007/978-3-319-67361-5_40 · Cited by 2,254 · see citing works
Kendall, A., Hawke, J., Janz, D., Mazur, P., Reda, D., Allen, J.-M., Lam, V.-D., Bewley, A., & Shah, A. (2019) · 2019 International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2019.8793742 · Cited by 694 · cited 1× · see citing works
“Recent progress shows the promise of applying RL to real-world applications, such as autonomous driving [16], [18], [50], the power system optimization in smart building [29], the surgical robotics arm [40], and even nuclear fusion [9].”
BackgroundEmergence of grounded compositional language in multi-agent populations
Mordatch, I., & Abbeel, P. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11492 · Cited by 440 · see citing works
iGibson 2.0: Object-Centric Simulation for Robot Learning of Everyday Household Tasks
Li, C., Xia, F., Martín-Martín, R., Lingelbach, M., Srivastava, S., Shen, B., Vainio, K., Gokmen, C., Dharan, G., Jain, T., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.03272 · Cited by 58 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
A Survey of Zero-shot Generalisation in Deep Reinforcement Learning
Kirk, R., Zhang, A., Grefenstette, E., & Rocktäschel, T. (2023) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.14174 · Cited by 193 · see citing works
“Many environments combine PCG components with controllable variation (Chevalier-Boisvert et al., 2019; Côté et al., 2019; Juliani et al., 2019; Li et al., 2021b; Team et al., 2021; Xue et al., 2021; Fortunato et al., 2019; Eimer et al., 2021; Bapst et al., 2019).”
BackgroundHuang, Z., Sheng, Z., Ma, C., & Chen, S. (2024) · Communications in Transportation Research · doi:10.1016/j.commtr.2024.100127 · Cited by 88 · see citing works
Hua, M., Qi, X., Chen, D., Jiang, K., Liu, Z. E., Sun, H., Zhou, Q., & Xu, H. (2025) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2025.3574280 · Cited by 88 · see citing works
“MetaDrive o ff ers infinite procedurally generated driving environments, balancing realism with e ffi ciency [120].”
Understanding World or Predicting Future? A Comprehensive Survey of World Models
Ding, J., Zhang, Y., Shang, Y., Zhang, Y., Zong, Z., Feng, J., Yuan, Y., Su, H., Li, N., Sukiennik, N., et al. (2026) · ACM Computing Surveys · doi:10.1145/3746449 · Cited by 58 · see citing works
“Motion Simulation SUMO [162] 2000 Geometric Space Rule-based Traffic Simulation Metadrive [150] 2022 Geometric Space Data-driven Traffic Simulation Trafficbots [320] 2023 Geometric Space Transformer Traffic Simulation Waymax [84] 2024 Geometric Space Data-driven Traffic Simulation End to End Sensor…”
Towards learning-based planning: The nuPlan benchmark for real-world autonomous driving
Karnchanachari, N., Geromichalos, D., Tan, K. S., Li, N., Eriksen, C., Yaghoubi, S., Mehdipour, N., Bernasconi, G., Fong, W. K., Guo, Y., et al. (2024) · 2024 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra57147.2024.10610077 · Cited by 46 · see citing works
“They also interfaced with MetaDrive [25] that enables graphical simulation of nuPlan.”
MethodEthics-Aware Safe Reinforcement Learning for Rare-Event Risk Control in Interactive Urban Driving
Li, D., & Okhrin, O. (2026) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2026.3708405
ProcgenOOD: a benchmark for out-of-distribution generalization in reinforcement learning
Corder, K., Park, S. J., & Vindiola, M. M. (2026) · Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications VIII · doi:10.1117/12.3093362
- Authors
- Li, Q., Peng, Z., Feng, L., Zhang, Q., Xue, Z., & Zhou, B.
- Affiliations
- Centre for Perceptual and Interactive Intelligence (HK)
- Chinese University of Hong Kong (HK)
- ETH Zurich (CH)
- University of California, Los Angeles (US)
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2023-03-01
- Volume
- 45
- Issue
- 3
- Pages
- 3461-3475
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- 1939-3539
- OpenAlex
- W3203076355
- Cited by
- 246
- References
- 6
- Citations by year
- 2026: 52 · 2025: 91 · 2024: 56 · 2023: 36 · 2022: 8 · 2021: 3
- FWCI
- 15.65
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Transactions on Pattern Analysis and Machine Intelligence (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Centre for Perceptual and Interactive Intelligence
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety
- Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Traffic and Road Safety
- Keywords
- driving simulation, reinforcement learning, generalizability, procedural content generation, multi-agent reinforcement learning, safe exploration, autonomous driving, benchmarking
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
