The DOI matches its Crossref record; 180 of the 180 references it cites were found in a registry or an index.
Multi-Agent Reinforcement Learning for Connected and Automated Vehicles Control: Recent Advancements and Future Prospects
Open accessAbstract
Connected and automated vehicles (CAVs) have emerged as a potential solution to the future challenges of developing safe, efficient, and eco-friendly transportation systems. However, CAV control presents significant challenges significant challenges due to the complexity of interconnectivity and co-ordination required among vehicles. Multi-agent reinforcement learning (MARL), which has shown notable advancements in addressing complex problems in autonomous driving, robotics, and human-vehicle interaction, emerges as a promising tool to enhance CAV capabilities. Despite its potential, there is a notable absence of current reviews on mainstream MARL algorithms for CAVs. To fill this gap, this paper offers a comprehensive review of MARL’s application in CAV control. The paper begins with an introduction to MARL, explaining its unique advantages in handling complex and multi-agent scenarios. It then presents a detailed survey of MARL applications across various control dimensions for CAVs, including critical scenarios such as platooning control, lane-changing, and unsignalized intersections. Additionally, the paper reviews prominent simulation platforms essential for developing and testing MARL algorithms for CAVs. Lastly, it examines the current challenges in deploying MARL for CAV control, including safety, communication, mixed traffic, and sim-to-real challenges. Potential solutions discussed include hierarchical MARL, decentralized MARL, adaptive interactions, and offline MARL. The work has been summarized in MARL_in_CAV_Control_Repository.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The V2N links follow the pathloss model PL ( d ) = 128 . 1 + 37 . 6 log 10 ( d ), where d is the distance in [km] of the vehicle from the BS [43].”
From Explainable AI-Aided Feature Selection and Model Reduction for DRL-Based V2X Resource Allocation (2025) · cites this paper as [43]
In the citing paper
“For the V2V links, we use the WINNER path-loss model PL ( d ) = A log 10 ( d ) + B + C log 10 f c 5 . 0 , where d is the inter-vehicular distance in [m], f c is the system frequency in [GHz], the parameter A describes the impact of path-loss exponent, the parameter B is the intercept, and parameter C controls the frequency dependency.”
From Explainable AI-Aided Feature Selection and Model Reduction for DRL-Based V2X Resource Allocation (2025) · cites this paper as [43]
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 1×
“For instance, in [110], a MARL approach based on a robust communication protocol with Long Short-Term Memory (LSTM) [111] is introduced, resulting in stable platooning control.”
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782
Reinforcement Learning: A Survey
Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.301 · Cited by 8,972 · cited 1×
“RL, typically formalized as a Markov Decision Process (MDP), provides a framework for sequential decision-making under uncertainty [34].”
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132 · cited 1×
“…Q ( s , a ) to select actions via maximization; Policy-based methods (e.g., DDPG): optimize a parameterized policy π θ directly using gradient ascent; Actor-critic methods (e.g., Advantage Actor-Critic (A2C) [35]): combine policy and value function learning to improve stability and reduce variance.”
Graph neural networks: A review of methods and applications
Zhou, J., Cui, G., Hu, S., Zhang, Z., Yang, C., Liu, Z., Wang, L., Li, C., & Sun, M. (2020) · AI Open · doi:10.1016/j.aiopen.2021.01.001 · Cited by 6,880 · cited 1×
“Graph neural networks (GNNs) [121] have gained significant attention in MARL settings, particularly for tra ffi c flow optimization .”
Self-citationCARLA: An open urban driving simulator
Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., & Koltun, V. (2017) · arXiv · doi:10.48550/arxiv.1711.03938 · Cited by 3,718 · cited 1×
“CARLA , is a high-fidelity 3D driving simulator with realistic physics and sensor modeling, enabling MARL research in complex urban settings [162].”
Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments
Lowe, R., Wu, Y., Tamar, A., Harb, J., Abbeel, P., & Mordatch, I. (2017) · arXiv · doi:10.48550/arxiv.1706.02275 · Cited by 3,377 · cited 1×
“For example, MADDPG [78] extends DDPG to the MARL setting by employing a centralized critic network to enable the global value estimate.”
Microscopic Traffic Simulation using SUMO
Lopez, P. A., Wiessner, E., Behrisch, M., Bieker-Walz, L., Erdmann, J., Flotterod, Y.-P., Hilbrich, R., Lucken, L., Rummel, J., & Wagner, P. (2018) · 2018 21st International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2018.8569938 · Cited by 3,297 · cited 1×
“) is an open-source, large-scale tra ffi c simulator widely used for studying platooning (1D), lane coordination (2D), and tra ffi c flow optimization (3D) [159].”
Counterfactual Multi-Agent Policy Gradients
Foerster, J., Farquhar, G., Afouras, T., Nardelli, N., & Whiteson, S. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11794 · Cited by 1,873 · cited 2×
“Foerster et al. [73] and Guo et al. [74] [70].”
Multi-Agent Reinforcement Learning: Independent vs. Cooperative Agents
Tan, M. (1993) · Machine Learning Proceedings 1993 · doi:10.1016/b978-1-55860-307-3.50049-6 · Cited by 1,810 · cited 1×
“Traditional independent Q-learning (IQL) [44] and independent advantage actor-critic (IA2C) [45] are widely used but often struggle with convergence due to the lack of inter-agent coordination [46].”
Research works printed but not linked: 5
- Niu, MA-MIX: Value function decomposition for cooperative multiagent reinforcement learning based on multi-head attention mechanism, 2402
- Yang, Hierarchical multi-agent skill discovery, 36, 1
- Liu, Lazy agents: A new perspective on solving sparse reward problem in multi-agent reinforcement learning, 21937
- Van der Pol, Coordinated deep reinforcement learners for traffic light control, 8, 21
- Leurent, 2018, An Environment for Autonomous Driving Decision-making
Cited by
Multi-agent reinforcement learning for resources allocation optimization: a survey
Hady, M. A., Hu, S., Pratama, M., Cao, Z., & Kowalczyk, R. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11340-5 · Cited by 139
Intelligent UAV Caching and Energy Management in 6G Networks
Ameur, A. I., Oubbati, O. S., Rachedi, A., Arishi, A., & Atiquzzaman, M. (2025) · IEEE Transactions on Network Science and Engineering · doi:10.1109/tnse.2025.3628171 · Cited by 75
“Additionally, MADRL enables UAVs to optimize decision-making by adjusting flight paths and caching strategies based on real-time data, reducing latency and adapting to fluctuating user demands [18], [19].”
Liu, Z. E., Li, Y., Zhou, Q., Shuai, B., Hua, M., Xu, H., Xu, L., Tan, G., & Li, Y. (2024) · Applied Energy · doi:10.1016/j.apenergy.2024.124350 · Cited by 42
Self-citationLiu, Z. E., Long, W., Chen, Z., Littlefield, J., Jing, L., Ren, B., El-Houjeiri, H. M., Al-Qahtani, A., Jabbar, M. Y., & Masnadi, M. S. (2024) · Energy and AI · doi:10.1016/j.egyai.2024.100434 · Cited by 42
Self-citationLiu, Z. E., Li, Y., Zhou, Q., Li, Y., Shuai, B., Xu, H., Hua, M., Tan, G., & Xu, L. (2024) · IEEE Transactions on Transportation Electrification · doi:10.1109/tte.2024.3363650 · Cited by 33
“To address these issues, many researchers have turned to DRL for EMS optimization [11].”
Self-citation
Metadata
- Authors
- Hua, M., Qi, X., Chen, D., Jiang, K., Liu, Z. E., Sun, H., Zhou, Q., & Xu, H.
- Affiliations
- University of Birmingham (GB)
- Michigan State University (US)
- Mississippi State University (US)
- Southeast University (CN)
- University of Pittsburgh (US)
- Venue
- IEEE Transactions on Automation Science and Engineering
- Published
- 2025
- Volume
- 22
- Pages
- 16266-16286
- ISSN
- 1545-5955 (ISSN-L)
- 1558-3783
- OpenAlex
- W4410808427
- Cited by
- 88
- References
- 176
- Citations by year
- 2026: 60 · 2025: 20
- FWCI
- 29.38
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- IEEE Transactions on Automation Science and Engineering (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Engineering › Control and Systems Engineering › Traffic control and management
- Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Keywords
- multi-agent reinforcement learning, connected and automated vehicles, vehicle platooning, lane changing, unsignalized intersections, simulation environment, sim-to-real transfer, mixed traffic, vehicle communication, safety
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Multi-Agent Reinforcement Learning for Connected and Automated Vehicles Control: Recent Advancements and Future Prospects. https://app.cochat.ai/papers/10.1109/tase.2025.3574280
- The paper
- Hua, M., Qi, X., Chen, D., Jiang, K., Liu, Z. E., Sun, H., Zhou, Q., & Xu, H. (2025). Multi-Agent Reinforcement Learning for Connected and Automated Vehicles Control: Recent Advancements and Future Prospects. IEEE Transactions on Automation Science and Engineering. https://doi.org/10.1109/tase.2025.3574280
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.