The StarCraft Multi-Agent Challenge
Open accessAbstract
In the last few years, deep multi-agent reinforcement learning (RL) has become a highly active area of research. A particularly challenging class of problems in this area is partially observable, cooperative, multi-agent learning, in which teams of agents must learn to coordinate their behaviour while conditioning only on their private observations. This is an attractive research area since such problems are relevant to a large number of real-world systems and are also more amenable to evaluation than general-sum problems. Standardised environments such as the ALE and MuJoCo have allowed single-agent RL to move beyond toy domains, such as grid worlds. However, there is no comparable benchmark for cooperative multi-agent RL. As a result, most papers in this field use one-off toy problems, making it difficult to measure real progress. In this paper, we propose the StarCraft Multi-Agent Challenge (SMAC) as a benchmark problem to fill this gap. SMAC is based on the popular real-time strategy game StarCraft II and focuses on micromanagement challenges where each unit is controlled by an independent agent that must act based on local observations. We offer a diverse set of challenge maps and recommendations for best practices in benchmarking and evaluations. We also open-source a deep multi-agent RL learning framework including state-of-the-art algorithms. We believe that SMAC can provide a standard benchmark environment for years to come. Videos of our best agents for several SMAC scenarios are available at: https://youtu.be/VZ7zmQ_obZ0.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
No references could be resolved yet.
Research works printed but not linked: 30
- Bellemare, M. G., Naddaf, Y., Veness, J., and Bowling, M. The arcade learning environment: An evaluation platform for general agents. Journal of Artificial Intelligence Research , 47:253-279, jun 2013.
- Foerster, J., Nardelli, N., Farquhar, G., Afouras, T., Torr, P. H. S., Kohli, P., and Whiteson, S. Stabilising Experience Replay for Deep Multi-Agent Reinforcement Learning. In Proceedings of the 34th International Conference on Machine Learning , pp. 1146-1155, 2017.
- Foerster, J., Farquhar, G., Afouras, T., Nardelli, N., and Whiteson, S. Counterfactual multi-agent policy gradients. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence , 2018a.
- Foerster, J. N., de Witt, C. A. S., Farquhar, G., Torr, P. H., Boehmer, W., and Whiteson, S. Multi-agent common knowledge reinforcement learning. arXiv preprint arXiv:1810.11702 , 2018b. · arXiv
- Hausknecht, M. and Stone, P. Deep Recurrent Q-Learning for Partially Observable MDPs. In AAAI Fall Symposium on Sequential Decision Making for Intelligent Agents , 2015.
- Hausknecht, M., Mupparaju, P., Subramanian, S., Kalyanakrishnan, S., and Stone, P. Half field offense: an environment for multiagent learning and ad hoc teamwork. In AAMAS Adaptive Learning Agents (ALA) Workshop , 2016.
- Heinrich, J. and Silver, D. Deep reinforcement learning from self-play in imperfect-information games. CoRR , abs/1603.01121, 2016. URL http://arxiv.org/abs/1603.01121 . · arXiv
- Hu, Y., Li, J., Li, X., Pan, G., and Xu, M. Knowledge-guided agent-tactic-aware learning for starcraft micromanagement. In IJCAI , pp. 1471-1477, 2018.
- Kalyanakrishnan, S., Liu, Y., and Stone, P. Half field offense in robocup soccer: A multiagent reinforcement learning case study. In Robot Soccer World Cup , pp. 72-85. Springer, 2006.
- Kitano, H., Asada, M., Kuniyoshi, Y., Noda, I., and Osawa, E. Robocup: The robot world cup initiative. In Proceedings of the first international conference on Autonomous agents , pp. 340-347. ACM, 1997.
- Leibo, J. Z., Zambaldi, V., Lanctot, M., Marecki, J., and Graepel, T. Multi-agent Reinforcement Learning in Sequential Social Dilemmas. February 2017. URL http://arxiv.org/abs/1702. 03037 . arXiv: 1702.03037. · arXiv
- Lowe, R., Wu, Y., Tamar, A., Harb, J., Abbeel, P., and Mordatch, I. Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments. June 2017. URL http://arxiv.org/abs/ 1706.02275 . arXiv: 1706.02275. · arXiv
- Nardelli, N., Synnaeve, G., Lin, Z., Kohli, P., Torr, P. H., and Usunier, N. Value propagation networks. arXiv preprint arXiv:1805.11199 , 2018. · arXiv
- Oliehoek, F. A. and Amato, C. A Concise Introduction to Decentralized POMDPs . SpringerBriefs in Intelligent Systems. Springer, 2016. URL http://www.springer.com/us/book/ 9783319289274 .
- Ontanón, S., Synnaeve, G., Uriarte, A., Richoux, F., Churchill, D., and Preuss, M. A survey of realtime strategy game ai research and competition in starcraft. IEEE Transactions on Computational Intelligence and AI in games , 5(4):293-311, 2013.
- Pang, Z.-J., Liu, R.-Z., Meng, Z.-Y., Zhang, Y., Yu, Y., and Lu, T. On reinforcement learning for full-length game of starcraft. arXiv preprint arXiv:1809.09095 , 2018. · arXiv
- Rashid, T., Samvelyan, M., Schroeder, C., Farquhar, G., Foerster, J., and Whiteson, S. QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning. In Proceedings of the 35th International Conference on Machine Learning , pp. 4295-4304, 2018.
- Resnick, C., Eldridge, W., Ha, D., Britz, D., Foerster, J., Togelius, J., Cho, K., and Bruna, J. Pommerman: A multi-agent playground. arXiv preprint arXiv:1809.07124 , 2018. · arXiv
- Shao, K., Zhu, Y., and Zhao, D. Starcraft micromanagement with reinforcement learning and curriculum transfer learning. IEEE Transactions on Emerging Topics in Computational Intelligence , 2018.
- Son, K., Kim, D., Kang, W. J., Hostallero, D. E., and Yi, Y. QTRAN: Learning to factorize with transformation for cooperative multi-agent reinforcement learning. In Proceedings of the 36th International Conference on Machine Learning , volume 97 of Proceedings of Machine Learning Research , pp. 5887-5896. PMLR, 2019.
- Stone, P., Kuhlmann, G., Taylor, M. E., and Liu, Y. Keepaway soccer: From machine learning testbed to benchmark. In Robot Soccer World Cup , pp. 93-105. Springer, 2005.
- Sun, P., Sun, X., Han, L., Xiong, J., Wang, Q., Li, B., Zheng, Y., Liu, J., Liu, Y., Liu, H., et al. Tstarbots: Defeating the cheating level builtin ai in starcraft ii in the full game. arXiv preprint arXiv:1809.07193 , 2018. · arXiv
- Sunehag, P., Lever, G., Gruslys, A., Czarnecki, W. M., Zambaldi, V., Jaderberg, M., Lanctot, M., Sonnerat, N., Leibo, J. Z., Tuyls, K., and Graepel, T. Value-Decomposition Networks For Cooperative Multi-Agent Learning. arXiv:1706.05296 [cs] , June 2017. URL http://arxiv. org/abs/1706.05296 . arXiv: 1706.05296. · arXiv
- Synnaeve, G., Nardelli, N., Auvolat, A., Chintala, S., Lacroix, T., Lin, Z., Richoux, F., and Usunier, N. TorchCraft: a Library for Machine Learning Research on Real-Time Strategy Games. arXiv preprint arXiv:1611.00625 , 2016. · arXiv
- Tampuu, A., Matiisen, T., Kodelja, D., Kuzovkin, I., Korjus, K., Aru, J., Aru, J., and Vicente, R. Multiagent Cooperation and Competition with Deep Reinforcement Learning. November 2015. URL http://arxiv.org/abs/1511.08779 . arXiv: 1511.08779. · arXiv
- Tan, M. Multi-agent reinforcement learning: Independent vs. cooperative agents. In Proceedings of the tenth international conference on machine learning , pp. 330-337, 1993.
- Usunier, N., Synnaeve, G., Lin, Z., and Chintala, S. Episodic exploration for deep deterministic policies: An application to starcraft micromanagement tasks. arXiv preprint arXiv:1609.02993 , 2016. · arXiv
- Vinyals, O., Ewalds, T., Bartunov, S., Georgiev, P., Vezhnevets, A. S., Yeo, M., Makhzani, A., Küttler, H., Agapiou, J., Schrittwieser, J., Quan, J., Gaffney, S., Petersen, S., Simonyan, K., Schaul, T., van Hasselt, H., Silver, D., Lillicrap, T., Calderone, K., Keet, P., Brunasso, A., Lawrence, D., Ekermo, A., Repp, J., and Tsing, R. StarCraft II: A New Challenge for Reinforcement Learning. arXiv:1708.04782 [cs] , August 2017. URL http://arxiv.org/abs/1708.04782 . arXiv: 1708.04782. · arXiv
- Yang, Y., Luo, R., Li, M., Zhou, M., Zhang, W., and Wang, J. Mean field multi-agent reinforcement learning. arXiv preprint arXiv:1802.05438 , 2018. · arXiv
- Zheng, L., Yang, J., Cai, H., Zhang, W., Wang, J., and Yu, Y. Magent: A many-agent reinforcement learning platform for artificial collective intelligence. arXiv preprint arXiv:1712.00600 , 2017. · arXiv
Cited by
Sorted by citations
Habitat 3.0: A Co-Habitat for Humans, Avatars, and Robots
Puig, X., Undersander, E., Szot, A., Cote, M. D., Yang, T.-Y., Partsey, R., Desai, R., Clegg, A. W., Hlavac, M., Min, S. Y., et al. (2023) · arXiv · doi:10.48550/arxiv.2310.13724 · Cited by 15 · see citing works
“…learning (MARL) works operate in a low-dimensional state space (e.g., grid-like mazes, tabular tasks, or Markov games) (Jaderberg et al., 2019; Samvelyan et al., 2019; oai; Resnick et al., 2018; Suarez et al., 2019; Baker et al., 2020; Giles & Jim, 2002; Lazaridou et al., 2016; Foerster et…”
BackgroundPredictive Representations: Building Blocks of Intelligence
Carvalho, W., Tomov, M. S., de Cothi, W., Barry, C., & Gershman, S. J. (2024) · Neural Computation · doi:10.1162/neco_a_01705 · Cited by 15 · see citing works
Deep reinforcement learning for multi-agent coordination
Aina, K. O., & Ha, S. (2026) · Artificial Life and Robotics · doi:10.1007/s10015-025-01089-z · Cited by 8 · see citing works
“The coordination of multi-agent systems has garnered significant research interest in recent years, with the complexity of the problem increasing exponentially as the number of agents rises [29].”
Human strategy learning-based multi-agent deep reinforcement learning for online team sports game
Lee, S., Lee, G., Kim, W., Kim, J., Park, J., & Cho, K. (2025) · IEEE Access · doi:10.1109/access.2025.3531435 · Cited by 7 · see citing works
Formal contracts mitigate social dilemmas in multi-agent reinforcement learning
Haupt, A., Christoffersen, P., Damani, M., & Hadfield-Menell, D. (2024) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-024-09682-5 · Cited by 5 · see citing works
Craftax: A Lightning-Fast Benchmark for Open-Ended Reinforcement Learning
Matthews, M., Beukman, M., Ellis, B., Samvelyan, M., Jackson, M., Coward, S., & Foerster, J. (2024) · arXiv · doi:10.48550/arxiv.2402.16801 · Cited by 4 · see citing works
“1 Code provided at https://github.com/ MichaelTMatthews/Craftax . ronment (Bellemare et al., 2013) for value based deep RL algorithms, Mujoco (Todorov et al., 2012) for continuous con-trol and the StarCraft Multi-Agent Challenge (Samvelyan et al., 2019) for multi-agent RL.”
Self-citation
- Authors
- Samvelyan, M., Rashid, T., Schroeder de Witt, C., Farquhar, G., Nardelli, N., Rudner, T. G. J., Hung, C.-M., Torr, P. H. S., Foerster, J., & Whiteson, S.
- Affiliations
- Russian-Armenian University (AM)
- University of Oxford (GB)
- Venue
- International Joint Conference on Autonomous Agents and Multiagent Systems
- Publisher
- IEEE Computer Society
- Published
- 2019-05-08
- Pages
- 2186-2188
- ISSN
- 1548-8403 (ISSN-L)
- 1558-2914
- OpenAlex
- W2913662073
- Cited by
- 453
- References
- 0
- Citations by year
- 2026: 89 · 2025: 51 · 2024: 88 · 2023: 94 · 2022: 66 · 2021: 43 · 2020: 15 · 2019: 6 · 2018: 1
- FWCI
- 12.43
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Adaptive Agents and Multi-Agents Systems (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Adaptive Agents and Multi-Agents Systems
- Funders
- Oracle
- Nvidia
- Innovationsfonden
- DeepMind
- European Commission
- National Institutes of Health
- Multidisciplinary University Research Initiative (EP/N019474/1)
- Engineering and Physical Sciences Research Council (EP/N019474/1, 1749045, EP/N509711/1, EP/M508111/1)
- National Institute of General Medical Sciences (R01GM114311)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
- Social Sciences › Decision Sciences › Management Science and Operations Research › Game Theory and Applications
- Keywords
- StarCraft Multi-Agent Challenge, multi-agent reinforcement learning, benchmark environments, StarCraft II, micromanagement, real-time strategy games, agent coordination
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
