A Comprehensive Survey of Multiagent Reinforcement Learning
Open accessAbstract
Multiagent systems are rapidly finding applications in a variety of domains, including robotics, distributed control, telecommunications, and economics. The complexity of many tasks arising in these domains makes them difficult to solve with preprogrammed agent behaviors. The agents must, instead, discover a solution on their own, using learning. A significant part of the research on multiagent learning concerns reinforcement learning techniques. This paper provides a comprehensive survey of multiagent reinforcement learning (MARL). A central issue in the field is the formal statement of the multiagent learning goal. Different viewpoints on this issue have led to the proposal of many different goals, among which two focal points can be distinguished: stability of the agents' learning dynamics, and adaptation to the changing behavior of the other agents. The MARL algorithms described in the literature aim---either explicitly or implicitly---at one of these two goals or at a combination of both, in a fully cooperative, fully competitive, or more general setting. A representative selection of these algorithms is discussed in detail in this paper, together with the specific issues that arise in each category. Additionally, the benefits and challenges of MARL are described along with some of the problem domains where the MARL techniques have been applied. Finally, an outlook for the field is provided.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Collective adaptive capabilities result not only from the particular form of interactions between individuals but also from a modulation of the relative intensities of these interactions.
“Such collective adaptive capabilities do not only result from the particular form of interactions between individuals but also from a modulation of the relative intensities of these interactions [92].”
From Swarm Robotics: Past, Present, and Future [Point of View] (2021) · cites this paper as [92]
Recent research in several disciplines has shown that a minority of committed agents can determine the overall behavior of a group.
“Recent research in several disciplines [92], [105]- [107] has shown that a minority of committed agents can determine the overall behavior of a group.”
From Swarm Robotics: Past, Present, and Future [Point of View] (2021) · cites this paper as [92]
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · see citing works
Reinforcement Learning: A Survey
Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.301 · Cited by 8,972 · see citing works
Markov decision processes: Discrete stochastic dynamic programming
Hazeghi, K., & Puterman, M. L. (1995) · Journal of the American Statistical Association · doi:10.2307/2291177 · Cited by 8,037 · see citing works
Cooperative Multi-Agent Learning: The State of the Art
Panait, L., & Luke, S. (2005) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-005-2631-2 · Cited by 1,237 · see citing works
Relational Reinforcement Learning
Džeroski, S., De Raedt, L., & Driessens, K. (2001) · Machine Learning · doi:10.1023/a:1007694015589 · Cited by 371 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Interactive machine learning for health informatics: when do we need the human-in-the-loop?
Holzinger, A. (2016) · Brain Informatics · doi:10.1007/s40708-016-0042-6 · Cited by 906 · see citing works
Deep Reinforcement Learning: An Overview
Li, Y. (2017) · arXiv · doi:10.48550/arxiv.1701.07274 · Cited by 762 · see citing works
A Survey of Multi-Objective Sequential Decision-Making
Roijers, D. M., Vamplew, P., Whiteson, S., & Dazeley, R. (2013) · Journal of Artificial Intelligence Research · doi:10.1613/jair.3987 · Cited by 461 · see citing works
Swarm Robotics: Past, Present, and Future [Point of View]
Dorigo, M., Theraulaz, G., & Trianni, V. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2021.3072740 · Cited by 401 · see citing works
Wang, Y., Su, Z., Guo, S., Dai, M., Luan, T. H., & Liu, Y. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3263909 · Cited by 357 · see citing works
A Unified Game-Theoretic Approach to Multiagent Reinforcement Learning
Lanctot, M., Zambaldi, V., Gruslys, A., Lazaridou, A., Tuyls, K., Perolat, J., Silver, D., & Graepel, T. (2017) · arXiv · doi:10.48550/arxiv.1711.00832 · Cited by 295 · see citing works
A Survey on Transfer Learning for Multiagent Reinforcement Learning Systems
Silva, F. L. D., & Costa, A. H. R. (2019) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.11396 · Cited by 273 · see citing works
Decision Making in Multiagent Systems: A Survey
Rizk, Y., Awad, M., & Tunstel, E. W. (2018) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2018.2840971 · Cited by 219 · see citing works
Learning with Opponent-Learning Awareness
Foerster, J., Chen, R. Y., Al-Shedivat, M., Whiteson, S., Abbeel, P., & Mordatch, I. (2018) · International Joint Conference on Autonomous Agents and Multiagent Systems · doi:10.65109/hgwa8807 · Cited by 215 · see citing works
Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domain
Hao, J., Yang, T., Tang, H., Bai, C., Liu, J., Meng, Z., Liu, P., & Wang, Z. (2021) · arXiv · doi:10.48550/arxiv.2109.06668 · Cited by 192
More Than Privacy: Applying Differential Privacy in Key Areas of Artificial Intelligence
Zhu, T., Ye, D., Wang, W., Zhou, W., & Yu, P. S. (2022) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2020.3014246 · Cited by 219 · see citing works
Wei, C., Zhang, Z., Qiao, W., & Qu, L. (2015) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2015.2420792 · Cited by 178 · see citing works
Tuchnitz, F., Ebell, N., Schlund, J., & Pruckner, M. (2021) · Applied Energy · doi:10.1016/j.apenergy.2020.116382 · Cited by 175 · see citing works
A multi-agent reinforcement learning model of common-pool resource appropriation
Perolat, J., Leibo, J. Z., Zambaldi, V., Beattie, C., Tuyls, K., & Graepel, T. (2017) · arXiv · doi:10.48550/arxiv.1707.06600 · Cited by 109 · see citing works
Open Problems in Cooperative AI
Dafoe, A., Hughes, E., Bachrach, Y., Collins, T., McKee, K. R., Leibo, J. Z., Larson, K., & Graepel, T. (2020) · arXiv · doi:10.48550/arxiv.2012.08630 · Cited by 93 · see citing works
Collaborative Distributed Q-Learning for RACH Congestion Minimization in Cellular IoT Networks
Sharma, S. K., & Wang, X. (2019) · IEEE Communications Letters · doi:10.1109/lcomm.2019.2896929 · Cited by 88 · see citing works
Multi-Agent Reinforcement Learning for Traffic Flow Management of Autonomous Vehicles
Mushtaq, A., Haq, I. U., Sarwar, M. A., Khan, A., Khalil, W., & Mughal, M. A. (2023) · Sensors · doi:10.3390/s23052373 · Cited by 61
Learning to flock through reinforcement
Durve, M., Peruani, F., & Celani, A. (2020) · Physical Review E · doi:10.1103/physreve.102.012601 · Cited by 58 · see citing works
Multi-agent modeling and simulation in the AI age
Fan, W., Chen, P., Shi, D., Guo, X., & Kou, L. (2021) · Tsinghua Science and Technology · doi:10.26599/tst.2021.9010005 · Cited by 50 · see citing works
Target-Driven Structured Transformer Planner for Vision-Language Navigation
Zhao, Y., Chen, J., Gao, C., Wang, W., Yang, L., Ren, H., Xia, H., & Liu, S. (2022) · Proceedings of the 30th ACM International Conference on Multimedia · doi:10.1145/3503161.3548281 · Cited by 54 · see citing works
Effects of Communication Directionality and AI Agent Differences in Human-AI Interaction
Ashktorab, Z., Dugan, C., Johnson, J., Pan, Q., Zhang, W., Kumaravel, S., & Campbell, M. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445256 · Cited by 50 · see citing works
Jamjuntr, P., Techawatcharapaikul, C., & Suanpang, P. (2024) · World Electric Vehicle Journal · doi:10.3390/wevj15100453 · Cited by 15 · see citing works
Deep reinforcement learning for multi-agent coordination
Aina, K. O., & Ha, S. (2026) · Artificial Life and Robotics · doi:10.1007/s10015-025-01089-z · Cited by 8 · see citing works
“However, these methods suffer from significant communication overhead, and the curse of dimensionality respectively [14,17].”
InfluentialAnastassacos, N., Hailes, S., & Musolesi, M. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6190 · Cited by 4 · see citing works
Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agents
de Witt, C. S., Krawiecka, K., Krawczuk, I., Hagag, B., Anderson, W. L., Belcak, P., Bucknall, B., Cai, X., Chopra, A., Cohen, D., et al. (2025) · arXiv · doi:10.48550/arxiv.2505.02077 · Cited by 2 · see citing works
SEDM: Scalable Self-Evolving Distributed Memory for Agents
Xu, H., Hu, J., Zhang, K., Yu, L., Tang, Y., Song, X., Duan, Y., Ai, L., & Shi, B. (2025) · arXiv · doi:10.48550/arxiv.2509.09498
“In recent years, the rapid development of large-scale multi-agent systems (MAS) [47, 25, 10, 48, 3] has expanded their application in diverse domains, including collaborative reasoning, decision-making, and autonomous planning [41].”
- Authors
- Busoniu, L., Babuska, R., & De Schutter, B.
- Venue
- IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews)
- Published
- 2008-03
- Volume
- 38
- Issue
- 2
- Pages
- 156-172
- ISSN
- 1094-6977 (ISSN-L)
- 1558-2442
- OpenAlex
- W2099618002
- Cited by
- 2,235
- References
- 5
- Citations by year
- 2026: 167 · 2025: 187 · 2024: 163 · 2023: 169 · 2022: 184 · 2021: 228 · 2020: 218 · 2019: 188 · 2018: 118 · 2017: 104 · 2016: 75 · 2015: 66 · 2014: 88 · 2013: 84 · 2012: 68
- FWCI
- 67.65
- Type
- journal article
- Language
- en
- Open access
- green
- resolver.tudelft.nl
- Locations
- IEEE Transactions on Systems Man and Cybernetics Part C (Applications and Reviews) (publishedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
- Research Repository (Delft University of Technology) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Multi-Agent Systems and Negotiation
- Social Sciences › Decision Sciences › Management Science and Operations Research › Advanced Bandit Algorithms Research
- Keywords
- multi-agent reinforcement learning, multi-agent systems, learning objectives, cooperative learning, competitive learning, distributed control, multi-agent learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
