The DOI matches its DataCite record; 9 of the 9 references it cites were found in a registry or an index.
PettingZoo: Gym for Multi-Agent Reinforcement Learning
Abstract
This paper introduces the PettingZoo library and the accompanying Agent Environment Cycle ("AEC") games model. PettingZoo is a library of diverse sets of multi-agent environments with a universal, elegant Python API. PettingZoo was developed with the goal of accelerating research in Multi-Agent Reinforcement Learning ("MARL"), by making work more interchangeable, accessible and reproducible akin to what OpenAI's Gym library did for single-agent reinforcement learning. PettingZoo's API, while inheriting many features of Gym, is unique amongst MARL APIs in that it's based around the novel AEC games model. We argue, in part through case studies on major problems in popular MARL environments, that the popular game models are poor conceptual models of games commonly used in MARL and accordingly can promote confusing bugs that are hard to detect, and that the AEC games model addresses these problems.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Our environment implementation is based on PettingZoo [24] and OpenAI Gym [5].”
From NovelGym: A Flexible Ecosystem for Hybrid Planning and Learning Agents Designed for Open Worlds (2024) · cites this paper as [24]
In the citing paper
“The single-agent wrapper converts the PettingZoo [24] multi-agent environment into a single-agent one by taking over the action execution of all other agents while exposing the control to the primary agent.”
From NovelGym: A Flexible Ecosystem for Hybrid Planning and Learning Agents Designed for Open Worlds (2024) · cites this paper as [24]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Proximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625
Mastering the game of Go without human knowledge
Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., et al. (2017) · Nature · doi:10.1038/nature24270 · Cited by 9,084
Grandmaster level in StarCraft II using multi-agent reinforcement learning
Vinyals, O., Babuschkin, I., Czarnecki, W. M., Mathieu, M., Dudzik, A., Chung, J., Choi, D. H., Powell, R., Ewalds, T., Georgiev, P., et al. (2019) · Nature · doi:10.1038/s41586-019-1724-z · Cited by 3,455
The Arcade Learning Environment: An Evaluation Platform for General Agents
Bellemare, M. G., Naddaf, Y., Veness, J., & Bowling, M. (2013) · Journal of Artificial Intelligence Research · doi:10.1613/jair.3912 · Cited by 2,161
Temporal difference learning and TD-Gammon
Tesauro, G. (1995) · Communications of the ACM · doi:10.1145/203330.203343 · Cited by 1,618
The Complexity of Decentralized Control of Markov Decision Processes
Bernstein, D. S., Givan, R., Immerman, N., & Zilberstein, S. (2002) · Mathematics of Operations Research · doi:10.1287/moor.27.4.819.297 · Cited by 1,192
Emergence of grounded compositional language in multi-agent populations
Mordatch, I., & Abbeel, P. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11492 · Cited by 440
Inequity aversion improves cooperation in intertemporal social dilemmas
Hughes, E., Leibo, J. Z., Phillips, M. G., Tuyls, K., Duéñez-Guzmán, E. A., Castañeda, A. G., Dunning, I., Zhu, T., McKee, K. R., Koster, R., et al. (2018) · arXiv · doi:10.48550/arxiv.1803.08884 · Cited by 134
Cited by
Principles and Guidelines for Evaluating Social Robot Navigation Algorithms
Francis, A., Pérez-D’Arpino, C., Li, C., Xia, F., Alahi, A., Alami, R., Bera, A., Biswas, A., Biswas, J., Chandra, R., et al. (2025) · ACM Transactions on Human-Robot Interaction · doi:10.1145/3700599 · Cited by 107
Collective intelligence for deep learning: A survey of recent developments
Ha, D., & Tang, Y. (2022) · Collective Intelligence · doi:10.1177/26339137221114874 · Cited by 71
A Novel Framework for Enhancing Decision-Making in Autonomous Cyber Defense Through Graph Embedding
Wang, Z., Wang, Y., Xiong, X., Ren, Q., & Huang, J. (2025) · Entropy · doi:10.3390/e27060622 · Cited by 7
Environment Complexity and Nash Equilibria in a Sequential Social Dilemma
Yasir, M., Howes, A., Mavroudis, V., & Hicks, C. (2024) · arXiv · doi:10.48550/arxiv.2408.02148 · Cited by 1
“The selected environment, built on the PettingZoo [30] and Gym [3] frameworks, simulates a 5x5 grid game featuring two agents—red and blue, visualised in Appendix A.”
Neural MMO 2.0: A Massively Multi-task Addition to Massively Multi-agent Learning
Suarez, J., Bloomin, D., Choe, K. W., Li, H. X., Sullivan, R., Kanna, N., Scott, D., Shuman, R., Bradley, H., Castricato, L., et al. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-2178 · Cited by 1
Self-citationNovelGym: A Flexible Ecosystem for Hybrid Planning and Learning Agents Designed for Open Worlds
Goel, S., Wei, Y., Lymperopoulos, P., Churá, K., Scheutz, M., & Sinapov, J. (2024) · International Joint Conference on Autonomous Agents and Multiagent Systems · doi:10.65109/aola8123
Metadata
- Authors
- Terry, J. K., Black, B., Grammel, N., Jayakumar, M., Hari, A., Sullivan, R., Santos, L., Perez, R., Horsch, C., Dieffendahl, C., Williams, N. L., Lokesh, Y., & Ravi, P.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2020
- arXiv
- 2009.14471
- References
- 9
- Type
- Preprint
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
