The DOI matches its Crossref record; 54 of the 54 references it cites were found in a registry or an index.
Multi-Agent Reinforcement Learning: A Review of Challenges and Applications
Open accessAbstract
In this review, we present an analysis of the most used multi-agent reinforcement learning algorithms. Starting with the single-agent reinforcement learning algorithms, we focus on the most critical issues that must be taken into account in their extension to multi-agent scenarios. The analyzed algorithms were grouped according to their features. We present a detailed taxonomy of the main multi-agent approaches proposed in the literature, focusing on their related mathematical models. For each algorithm, we describe the possible application fields, while pointing out its pros and cons. The described multi-agent algorithms are compared in terms of the most important characteristics for multi-agent reinforcement learning applications—namely, nonstationarity, scalability, and observability. We also describe the most common benchmark environments used to evaluate the performances of the considered methods.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132
College Admissions and the Stability of Marriage
Gale, D., & Shapley, L. S. (1962) · The American Mathematical Monthly · doi:10.1080/00029890.1962.11989827 · Cited by 5,985
Bengio, Y., Louradour, J., Collobert, R., & Weston, J. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553380 · Cited by 5,366 · cited 1×
“In addition, the scalability of the PS-TRPO algorithm was addressed using curriculum learning [38], in which the agents had to solve a number of sub-tasks that increased in difficulty in order to coordinate with an increasing number of agents.”
MethodLearning to predict by the methods of temporal differences
Sutton, R. S. (1988) · Machine Learning · doi:10.1007/bf00115009 · Cited by 5,148
A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play
Silver, D., Hubert, T., Schrittwieser, J., Antonoglou, I., Lai, M., Guez, A., Lanctot, M., Sifre, L., Kumaran, D., Graepel, T., et al. (2018) · Science · doi:10.1126/science.aar6404 · Cited by 3,562 · cited 1×
“In recent research, the efficiency of the learned model has been addressed, obtaining state-of-the-art performances with approaches such as MuZero and AlphaZero [13,14].”
Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
Williams, R. J. (1992) · Machine Learning · doi:10.1007/bf00992696 · Cited by 3,416
Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments
Lowe, R., Wu, Y., Tamar, A., Harb, J., Abbeel, P., & Mordatch, I. (2017) · arXiv · doi:10.48550/arxiv.1706.02275 · Cited by 3,377
Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning
Sutton, R. S., Precup, D., & Singh, S. (1999) · Artificial Intelligence · doi:10.1016/s0004-3702(99)00052-1 · Cited by 3,119
(2012) · elib (German Aerospace Center) · Cited by 1,994
Counterfactual Multi-Agent Policy Gradients
Foerster, J., Farquhar, G., Afouras, T., Nardelli, N., & Whiteson, S. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11794 · Cited by 1,873
Multi-Agent Reinforcement Learning: Independent vs. Cooperative Agents
Tan, M. (1993) · Machine Learning Proceedings 1993 · doi:10.1016/b978-1-55860-307-3.50049-6 · Cited by 1,810
End-to-End Training of Deep Visuomotor Policies
Levine, S., Finn, C., Darrell, T. J., & Abbeel, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1504.00702 · Cited by 1,344
Sutton, R. S. (1990) · Machine Learning Proceedings 1990 · doi:10.1016/b978-1-55860-141-3.50030-4 · Cited by 1,314
Mastering Atari, Go, chess and shogi by planning with a learned model
Schrittwieser, J., Antonoglou, I., Hubert, T., Simonyan, K., Sifre, L., Schmitt, S., Guez, A., Lockhart, E., Hassabis, D., Graepel, T., et al. (2020) · RePEc: Research Papers in Economics · Cited by 1,310
Multiagent systems: A survey from a machine learning perspective
Stone, P., & Veloso, M. (2000) · Autonomous Robots · doi:10.1023/a:1008942012299 · Cited by 1,218
Mastering Atari, Go, chess and shogi by planning with a learned model
Schrittwieser, J., Antonoglou, I., Hubert, T., Simonyan, K., Sifre, L., Schmitt, S., Guez, A., Lockhart, E., Hassabis, D., Graepel, T., et al. (2020) · Nature · doi:10.1038/s41586-020-03051-4 · Cited by 1,214 · cited 1×
“In recent research, the efficiency of the learned model has been addressed, obtaining state-of-the-art performances with approaches such as MuZero and AlphaZero [13,14].”
The Complexity of Decentralized Control of Markov Decision Processes
Bernstein, D. S., Givan, R., Immerman, N., & Zilberstein, S. (2002) · Mathematics of Operations Research · doi:10.1287/moor.27.4.819.297 · Cited by 1,192
The dynamics of reinforcement learning in cooperative multiagent systems
Claus, C., & Boutilier, C. E. (1998) · Cited by 1,089
Nash Q-Learning for General-Sum Stochastic Games
Hu, J., Wellman, M. P., & Boutilier, C. (2003) · Cited by 923
Cooperative Multi-agent Control Using Deep Reinforcement Learning
Gupta, J. K., Egorov, M., & Kochenderfer, M. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-71682-4_5 · Cited by 874
Monahan, G. E. (1982) · Management Science · doi:10.1287/mnsc.28.1.1 · Cited by 745
Deep Recurrent Q-Learning for Partially Observable MDPs
Hausknecht, M., & Stone, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1507.06527 · Cited by 667
Value-Decomposition Networks For Cooperative Multi-Agent Learning Based On Team Reward
Sunehag, P., Lever, G., Gruslys, A., Czarnecki, W. M., Zambaldi, V., Jaderberg, M., Lanctot, M., Sonnerat, N., Leibo, J. Z., Tuyls, K., et al. (2018) · International Joint Conference on Autonomous Agents and Multiagent Systems · doi:10.65109/jsrc7365 · Cited by 618
Multi-Agent Reinforcement Learning-Based Resource Allocation for UAV Networks
Cui, J., Liu, Y., & Nallanathan, A. (2020) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2019.2935201 · Cited by 593
Stabilising Experience Replay for Deep Multi-Agent Reinforcement Learning
Foerster, J., Nardelli, N., Farquhar, G., Afouras, T., Torr, P. H. S., Kohli, P., & Whiteson, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.08887 · Cited by 397
Efficient Large-Scale Fleet Management via Multi-Agent Deep Reinforcement Learning
Lin, K., Zhao, R., Xu, Z., & Zhou, J. (2018) · Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3219819.3219993 · Cited by 379
QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning
Rashid, T., Samvelyan, M., de Witt, C. S., Farquhar, G., Foerster, J., & Whiteson, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.11485 · Cited by 326
Qie, H., Shi, D., Shen, T., Xu, X., Li, Y., & Wang, L. (2019) · IEEE Access · doi:10.1109/access.2019.2943253 · Cited by 321
A Deterministic Improved Q-Learning for Path Planning of a Mobile Robot
Konar, A., Chakraborty, I. G., Singh, S. J., Jain, L. C., & Nagar, A. K. (2013) · IEEE Transactions on Systems, Man, and Cybernetics: Systems · doi:10.1109/tsmca.2012.2227719 · Cited by 269
Multi-agent Reinforcement Learning in Sequential Social Dilemmas
Leibo, J. Z., Zambaldi, V., Lanctot, M., Marecki, J., & Graepel, T. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.03037 · Cited by 263
Learning Multiagent Communication with Backpropagation
Sukhbaatar, S., Szlam, A. D., & Fergus, R. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.07736 · Cited by 261
Matignon, L., Laurent, G. J., & Le Fort-Piat, N. (2007) · 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2007.4399095 · Cited by 192
Kar, S., Moura, J. M. F., & Poor, H. V. (2013) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2013.2241057 · Cited by 175
Deep Decentralized Multi-task Multi-Agent Reinforcement Learning under Partial Observability
Omidshafiei, S., Pazis, J., Amato, C., How, J. P., & Vian, J. L. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.06182 · Cited by 168
Lenient Multi-Agent Deep Reinforcement Learning
Palmer, G., Tuyls, K., Bloembergen, D., & Savani, R. (2018) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/qdcv6054 · Cited by 143
Deep Reinforcement Learning for Automated Stock Trading: An Ensemble StrategyPreprint
Yang, H., Liu, X.-Y., Zhong, S., & Walid, A. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3690996 · Cited by 115
Scheduling of Battery Charge, Discharge, and Rest
Kim, H., & Shin, K. G. (2009) · IEEE Real-Time Systems Symposium (RTSS) · doi:10.1109/rtss.2009.38 · Cited by 86
Distributed Cooperative Spectrum Sharing in UAV Networks Using Multi-Agent Reinforcement Learning
Shamsoshoara, A., Khaledi, M., Afghah, F., Razi, A., & Ashdown, J. (2019) · IEEE Consumer Communications and Networking Conference · doi:10.1109/ccnc.2019.8651796 · Cited by 86
Collaborative Deep Reinforcement Learning for Joint Object Search
Kong, X., Xin, B., Wang, Y., & Hua, G. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.748 · Cited by 84
Animal intelligence: An experimental study of the associate processes in animals.
Thorndike, E. L. (1998) · American Psychologist · doi:10.1037/0003-066x.53.10.1125 · Cited by 76
Multi-agent Deep Reinforcement Learning for Zero Energy Communities
Prasad, A., & Dusparić, I. (2019) · IEEE PES Innovative Smart Grid Technologies Europe · doi:10.1109/isgteurope.2019.8905628 · Cited by 75
Multi-Agent Reinforcement Learning Approach for Residential Microgrid Energy Scheduling
Fang, X., Wang, J., Song, G., Han, Y., Zhao, Q., & Cao, Z. (2019) · Energies · doi:10.3390/en13010123 · Cited by 67
Smart Grid for Industry Using Multi-Agent Reinforcement Learning
Roesch, M., Linder, C., Zimmermann, R., Rudolf, A., Hohmann, A., & Reinhart, G. (2020) · Applied Sciences · doi:10.3390/app10196900 · Cited by 63
Heterogeneous Multi-Agent Deep Reinforcement Learning for Traffic Lights Control.
Dusparić, I., & Calvo, J. A. (2018) · AICS · Cited by 53
Gait Balance and Acceleration of a Biped Robot Based on Q-Learning
Lin, J.-L., Hwang, K., Jiang, W., et al. (2016) · IEEE Access · doi:10.1109/access.2016.2570255 · Cited by 49
A Multi-Agent Reinforcement Learning Framework for Lithium-ion Battery Scheduling Problems
Sui, Y. F., & Song, S. (2020) · Energies · doi:10.3390/en13081982 · Cited by 38
Q‐RTS: a real‐time swarm intelligence based on multi‐agent Q‐learning
Matta, M., Cardarilli, G. C., Nunzio, L. D., Fazzolari, R., Giardino, D., Re, M., Silvestri, F., & Spanò, S. (2019) · Electronics Letters · doi:10.1049/el.2019.0244 · Cited by 29
Self-citationA Reinforcement Learning-Based QAM/PSK Symbol Synchronizer
Matta, M., Cardarilli, G. C., Nunzio, L. D., Fazzolari, R., Giardino, D., Nannarelli, A., Re, M., & Spanò, S. (2019) · IEEE Access · doi:10.1109/access.2019.2938390 · Cited by 23
Self-citationCoordinated Multi-Agent Deep Reinforcement Learning for Energy-Aware UAV-Based Big-Data Platforms
Jung, S., Yun, W. J., Kim, J., & Kim, J. (2021) · Electronics · doi:10.3390/electronics10050543 · Cited by 19
Generating unambiguous and diverse referring expressions
Panagiaris, N., Hart, E., & Gkatzia, D. (2020) · Computer Speech & Language · doi:10.1016/j.csl.2020.101184 · Cited by 12
Cited by
Mahboob, S., & Liu, L. (2024) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2023.3347145 · Cited by 255
Multi-agent hierarchical reinforcement learning for energy management
Jendoubi, I., & Bouffard, F. (2023) · Applied Energy · doi:10.1016/j.apenergy.2022.120500 · Cited by 123
Multi-Agent Reinforcement Learning for Traffic Flow Management of Autonomous Vehicles
Mushtaq, A., Haq, I. U., Sarwar, M. A., Khan, A., Khalil, W., & Mughal, M. A. (2023) · Sensors · doi:10.3390/s23052373 · Cited by 61
“MARL overcomes this challenge by working with several agents concurrently and learning how to solve specific problems by interacting with each other and with the environment [16].”
BackgroundOptimal control of gene regulatory networks for morphogen-driven tissue patterning
Pezzotta, A., & Briscoe, J. (2023) · Cell Systems · doi:10.1016/j.cels.2023.10.004 · Cited by 34
A Survey on the Optimization of Large Language Model-based Agents
Du, S., Zhao, J., Shi, J., Xie, Z., Jiang, X., Bai, Y., & He, L. (2025) · arXiv · doi:10.48550/arxiv.2503.12434 · Cited by 22
Metadata
- Authors
- Canese, L., Cardarilli, G. C., Di Nunzio, L., Fazzolari, R., Giardino, D., Re, M., & Spanò, S.
- Venue
- Applied Sciences
- Publisher
- MDPI AG
- Published
- 2021-05-27
- Volume
- 11
- Issue
- 11
- Pages
- 4948
- ISSN
- 2076-3417 (ISSN-L)
- OpenAlex
- W3166816823
- Cited by
- 403
- References
- 54
- FWCI
- 16.50
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.