The DOI matches its DataCite record; 65 of the 70 references it cites were found in a registry or an index.
Emergent Tool Use From Multi-Agent Autocurricula
Open accessAbstract
Through multi-agent competition, the simple objective of hide-and-seek, and standard reinforcement learning algorithms at scale, we find that agents create a self-supervised autocurriculum inducing multiple distinct rounds of emergent strategy, many of which require sophisticated tool use and coordination. We find clear evidence of six emergent phases in agent strategy in our environment, each of which creates a new pressure for the opposing team to adapt; for instance, agents learn to build multi-object shelters using moveable boxes which in turn leads to agents discovering that they can overcome obstacles using ramps. We further provide evidence that multi-agent competition may scale better with increasing environment complexity and leads to behavior that centers around far more human-relevant skills than other self-supervised reinforcement learning methods such as intrinsic motivation. Finally, we propose transfer and fine-tuning as a way to quantitatively evaluate targeted capabilities, and we compare hide-and-seek agents to both intrinsic motivation and random initialization baselines in a suite of domain-specific intelligence tests.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“For example, Baker et al. (2019) showed that even a simple game of hide and seek can lead to sophisticated tool use and coordination by MARL agents.”
From Multi-Agent Risks from Advanced AI (2025) · cites this paper as Baker et al., 2019
The cited source says what the claim attributes to it.
In the citing paper
“Agents develop exploitative strategies (Baker et al., 2019), learn deception via theory-ofmind (Schulz et al., 2023), and discover betrayal without explicit training (Aitchison et al., 2021) -eluding threat models predicated on an identifiable adversary.”
From Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agents (2025) · cites this paper as Baker et al., 2019
The cited source says what the claim attributes to it.
In the citing paper
“Consequently, we used the PPO agent in Experiments III and IV, as PPO is a competent agent used by OpenAI [32].”
From Playtesting: What is Beyond Personas (2023) · cites this paper as [32]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 4×
“[75] Sepp Hochreiter and Jürgen Schmidhuber.”
MethodInfluentialReinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,268
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612 · cited 2×
“[30] David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, et al.”
BackgroundMethodProximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625 · cited 4×
“PPO [58], a policy gradient variant, penalizes large changes to the policy to prevent training instabilities.”
BackgroundInfluentialMuJoCo: A physics engine for model-based control
Todorov, E., Erez, T., & Tassa, Y. (2012) · 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2012.6386109 · Cited by 4,767 · cited 2×
“[57] Emanuel Todorov, Tom Erez, and Yuval Tassa.”
BackgroundMethodArms races between and within species
Dawkins, R., & Krebs, J. R. (1979) · Proceedings of the Royal Society B Biological Sciences · doi:10.1098/rspb.1979.0081 · Cited by 2,214 · cited 2×
“The vast amount of complexity and diversity on Earth evolved due to co-evolution and competition between organisms, directed by natural selection [5].”
BackgroundHigh-Dimensional Continuous Control Using Generalized Advantage Estimation
Schulman, J., Moritz, P., Levine, S., Jordan, M., & Abbeel, P. (2015) · arXiv · doi:10.48550/arxiv.1506.02438 · Cited by 2,103 · cited 3×
“We use Generalized Advantage Estimation [3] with horizon length H to estimate the advantage function.”
BackgroundMethodTemporal difference learning and TD-Gammon
Tesauro, G. (1995) · Communications of the ACM · doi:10.1145/203330.203343 · Cited by 1,618 · cited 3×
“In the context of multi-agent RL, [7], [30], [9], [13] and [10] used self-play with deep RL techniques to achieve super-human performance in Backgammon, Go, Dota, Capture-the-Flag and Starcraft, respectively.”
BackgroundMethodConcrete Problems in AI Safety
Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schulman, J., & Mané, D. (2016) · arXiv · doi:10.48550/arxiv.1606.06565 · Cited by 1,314
Flagged first, then by citations
5 of the 70 references couldn’t be matched to a known paper.
Research works printed but not linked: 11
- Kelsey R Allen, Kevin A Smith, and Joshua B Tenenbaum. The tools challenge: Rapid trial-anderror learning in physical problem solving. arXiv preprint arXiv:1907.09620 , 2019. · arXiv
- Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. Layer normalization. arXiv preprint arXiv:1607.06450 , 2016. · arXiv
- Rene Baillargeon and Susan Carey. Core cognition and beyond: The acquisition of physical and numerical knowledge. In In S. Pauen (Ed.), Early Childhood Development and Later Outcome , pp. 33-65. University Press, 2012.
- Trapit Bansal, Jakub Pachocki, Szymon Sidor, Ilya Sutskever, and Igor Mordatch. Emergent complexity via multi-agent competition. In International Conference on Learning Representations , 2018.
- Yuri Burda, Harrison Edwards, Amos Storkey, and Oleg Klimov. Exploration by random network distillation. In International Conference on Learning Representations , 2019b.
- Nick Haber, Damian Mrowca, Stephanie Wang, Li F Fei-Fei, and Daniel L Yamins. Learning to play with intrinsically-motivated, self-aware agents. In Advances in Neural Information Processing Systems , pp. 8388-8399, 2018.
- Max Jaderberg, Valentin Dalibard, Simon Osindero, Wojciech M Czarnecki, Jeff Donahue, Ali Razavi, Oriol Vinyals, Tim Green, Iain Dunning, Karen Simonyan, et al. Population based training of neural networks. arXiv preprint arXiv:1711.09846 , 2017. · arXiv
- Ryan Lowe, Yi Wu, Aviv Tamar, Jean Harb, Pieter Abbeel, and Igor Mordatch. Multi-agent actorcritic for mixed cooperative-competitive environments. In Advances in Neural Information Processing Systems , pp. 6379-6390, 2017.
- Deepak Pathak, Pulkit Agrawal, Alexei A Efros, and Trevor Darrell. Curiosity-driven exploration by self-supervised prediction. In International Conference on Machine Learning , pp. 2778-2787, 2017.
- Sainbayar Sukhbaatar, Rob Fergus, et al. Learning multiagent communication with backpropagation. In Advances in Neural Information Processing Systems , pp. 2244-2252, 2016.
- Larry Yaeger. Computational genetics, physiology, metabolism, neural systems, learning, vision, and behavior or poly world: Life in a new context. In SANTA FE INSTITUTE STUDIES IN THE SCIENCES OF COMPLEXITY-PROCEEDINGS VOLUME, volume 17, pp. 263-263. ADDISON-WESLEY PUBLISHING CO, 1994.
Cited by
Sorted by citations
Items 301–400 of 428
Reinforcement Learning for decision support in defense and security: A systematic review
Schadd, M. P. D., Berman, D. S., Chen, C. M., Cohen, M., Dorsch, J., GEGOV, A. E., Moll, M., Rose, O., Rösner, A., Saastamoinen, K., et al. (2026) · Annals of Operations Research · doi:10.1007/s10479-026-07205-x · Cited by 4
Co-Training an Observer and an Evading Target
Brandenburger, A., Hoffmann, F., & Charlish, A. (2021) · International Conference on Information Fusion (FUSION) · doi:10.23919/fusion49465.2021.9627024 · Cited by 3
“Training an additional policy for the evading targets can lead to a complex co-evolution of strategies [18].”
BackgroundLearning Cooperative Behaviours in Adversarial Multi-agent Systems
Wang, N., Das, G. P., & Millard, A. G. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-15908-4_15 · Cited by 2
Parameter Tuning Method for Multi-agent Simulation using Reinforcement Learning
Hirano, M., & Izumi, K. (2022) · International Conference on Behavioral, Economic, and Socio-Cultural Computing · doi:10.1109/besc57393.2022.9995509 · Cited by 2
“[3] showed that deep reinforcement learning could learn complex hide-and-seek strategies with tools that make the game more complex and high-dimensional.”
BackgroundA Data Enhancement Strategy for Multi-Agent Cooperative Hunting based on Deep Reinforcement Learning
Gao, Z., Dai, X., Yao, M., & Xiao, X. (2023) · IEEE International Conference on Industrial Cyber-Physical Systems (ICPS) · doi:10.1109/icps58381.2023.10128078 · Cited by 2
“OpenAI is an open-source Multi-Agent Reinforcement Learning experimental platform [33], which can be used to learn multi-agent confrontation, cooperation, communication and other strategies.”
BackgroundReward Gaming in Conditional Text Generation
Pang, R. Y., Padmakumar, V., Sellam, T., Parikh, A. P., & He, H. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.262 · Cited by 19
Spanwise Camber Morphing Offers Potential Efficiency Gains During Autonomous Gust Rejection
Haughn, K. P. T. (2024) · AIAA SciTech Forum · doi:10.2514/6.2024-1124 · Cited by 2
Reinforcement learning for VR table tennis
Seah, H. S., Jiang, D., Tandianus, B., Sui, Y., & Wang, H. (2024) · International Workshop on Advanced Imaging Technology (IWAIT) · doi:10.1117/12.3018124 · Cited by 2
“Applying theoretical research to practice is an important research direction of RL [6].”
BackgroundFu, Q., Qiu, T., Yi, J., Pu, Z., Ai, X., & Yuan, W. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3423417 · Cited by 2
“On the one hand, simplified grid-based simulation limits the generalization ability of models developed in discrete environments such as MAgent [9].”
Beyond the matrix: Experimental approaches to studying cognitive agents in social-ecological systems
Hertz, U., Koster, R., Janssen, M. A., & Leibo, J. Z. (2024) · Cognition · doi:10.1016/j.cognition.2024.105993 · Cited by 2
Generation Z: AI Affinity and Adoption in Competitive German Organisations
Nebgen, T., & Kurz, W. (2025) · Journal of Next-Generation Research 5 0 · doi:10.70792/jngr5.0.v1i2.81 · Cited by 3
The Imbalanced Target Classification Method Based on Mixed Learning of Virtual and Real Data
Yang, F., Wang, P., Qin, W., & Liao, Z. (2025) · IEEE Access · doi:10.1109/access.2025.3573320 · Cited by 2
“Examples include OpenAI’s open-source multi-agent hide-and-seek environment [11], Airsim’s simulation for automated cars and drones [12], and the ML-Agents platform for multi-agent training [13].”
Hierarchical Cooperative Swarm Policy Learning with Role Emergence
Zhang, T., Liu, Z., Pu, Z., Qiu, T., & Yi, J. (2021) · IEEE Symposium Series on Computational Intelligence (SSCI) · doi:10.1109/ssci50451.2021.9660008 · Cited by 2
“MARL methods have been proposed [4]–[9].”
The Importance of Open-Endedness (for the Sake of Open-Endedness)
Taylor, T. (2020) · Conference on Artificial Life (ALIFE) · doi:10.1162/isal_a_00257 · Cited by 2
“This is the case in notable examples of interesting evolved agents such as (Sims, 1994) and (Baker et al., 2019), which both involve agents evolving in simulated physical environments.”
SEM: Adaptive Staged Experience Access Mechanism for Reinforcement Learning
Wang, J., Wang, X., Luo, X., Zhang, Z., Wang, W., & Li, Y. (2020) · IEEE International Conference on Tools with Artificial Intelligence (ICTAI) · doi:10.1109/ictai50040.2020.00166 · Cited by 2
“The success of reinforcement learning cannot be separated from the vigorous development of deep learning in various fields [2], [25], [26] in recent years.”
BackgroundMultiagent Simulation On Hide and Seek Games Using Policy Gradient Trust Region Policy Optimization
Wafa, H., & Santoso, J. (2020) · doi:10.1109/icaicta49861.2020.9429052 · Cited by 2
“The environment used is taken from related work that found the emergence of the use of tools from multi-agent auto curricula (Baker et al., 2019) 1 .”
BackgroundToward Creative Problem Solving Agents: Action Discovery through Behavior Babbling
Gizzi, E., Hassan, A., Lin, W. W., et al. (2021) · IEEE International Conference on Development and Learning (ICDL) · doi:10.1109/icdl49984.2021.9515658 · Cited by 2
“Recent work in cognitive robotics has demonstrated success in creative problem solving implementations in robots and intelligent agent systems [20].”
BackgroundOPEn: An Open-ended Physics Environment for Learning Without a Task
Gan, C., Bhandwaldar, A., Torralba, A. B., Tenenbaum, J. B., & Isola, P. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9636830 · Cited by 3
Deep Reinforcement Learning Algorithms for Multi-Agent Systems - A Solution for Modeling Epidemics
Selvi, S. S., Dhananjaya Kumar, M., V, Lohith, N. V., et al. (2021) · IEEE Mysore Sub Section International Conference (MysuruCon) · doi:10.1109/mysurucon52639.2021.9641663 · Cited by 2
DCLN: Diverse Curriculum Learning with Neuroevolution in Six-Player Poker Tournament
Luo, J., Zhang, W., Yuan, W., & Chen, J. (2021) · China Automation Congress (CAC) · doi:10.1109/cac53003.2021.9727883 · Cited by 2
Self-Development and Causality in Intelligent Environments
Martinelli, M., Mariani, S., Lippi, M., & Zambonelli, F. (2022) · Ambient intelligence and smart environments · doi:10.3233/aise220050 · Cited by 2
Self-Supervised Occlusion Detection and Avoidance using Differentiable Rendering
Szántó, M., & Szemenyei, M. (2022) · International Symposium on Measurement and Control in Robotics (ISMCR) · doi:10.1109/ismcr56534.2022.9950574 · Cited by 2
“The agent also needs to then select the correct action to perform, which falls into the domain of Reinforcement Learning (RL) [8], [9].”
BackgroundEvolution Strategies Enhanced Complex Multiagent Coordination
Du, Y., Wang, Y., Cong, Y., Jiang, W.-H., & Pu, S. (2023) · IEEE International Joint Conference on Neural Network · doi:10.1109/ijcnn54540.2023.10191313 · Cited by 3
“Benefiting from the super non-linear fitting ability of deep neural network, deep-MARL goes beyond traditional multi-agent game theory and makes remarkable achievements in many challenging scenes, such as the MOBA games [5] and openAI Hide and Seek [6].”
BackgroundNeural-network-based parameter tuning for multi-agent simulation using deep reinforcement learning
Hirano, M., & Izumi, K. (2023) · World Wide Web · doi:10.1007/s11280-023-01197-5 · Cited by 2
“[11] showed that it can learn complex hide-and-seek strategies using tools that make the game more complex and high-dimensional.”
BackgroundDornig, J., & Li, C. (2020) · IEEE International Conference on Multimedia Information Processing and Retrieval (MIPR) · doi:10.1109/mipr49039.2020.00087 · Cited by 2
“Games and ML most commonly intersect in the areas: 1) ML testing/demonstration in Game Environments [1,2,3,4], 2) ML use for Game-AI [4,5], 3) ML use for game assets [6] and 4) ML for game performance [7,8].”
BackgroundContinual Learning on Incremental Simulations for Real-World Robotic Manipulation Tasks
Josifovski, J., Malmir, M., Klarmann, N., & Knoll, A. (2020) · mediaTUM (Technical University of Munich) · Cited by 3
Improving Sample Efficiency in Behavior Learning by Using Sub-optimal Planners for Robots
Antonioni, E., Riccio, F., & Nardi, D. (2022) · Lecture notes in computer science · doi:10.1007/978-3-030-98682-7_9 · Cited by 2
Moulin-Frier, C., & Oudeyer, P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.08878 · Cited by 2
“In particular, recent contributions in MARL have shown how an autocur-riculum of increasingly complex behaviors can emerge from agent’s coadaptation in mixed cooperative-competitive environments (Bansal et al. 2018; Baker et al. 2019).”
BackgroundLearning to Actively Reduce Memory Requirements for Robot Control TasksPreprint
Booker, M., & Majumdar, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.07451 · Cited by 2
Hybrid Information-driven Multi-agent Reinforcement LearningPreprint
Dawson, W. A., Glatt, R., Rusu, E., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.01004 · Cited by 2
“are compounded when sensors and actors are combined in intelligent, autonomous agents that can work together to solve highly complex tasks (Baker et al. 2019).”
BackgroundDealing with Non-Stationarity in Multi-Agent Reinforcement Learning via Trust Region Decomposition.Preprint
Li, W., Wang, X., Jin, B., Sheng, J., & Hongyuan Zha, B. (2021) · arXiv (Cornell University) · Cited by 3
Neural Auto-CurriculaPreprint
Feng, X., Slumbers, O., Wan, Z., Liu, B., McAleer, S., Wen, Y., Wang, J., & Yang, Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.02745 · Cited by 6
“Albeit challenging, we believe a method capable of discovering auto-curricula without explicit game theoretic knowledge can potentially open up entirely new approaches to MARL.”
BackgroundInfluentialSimulation to reality and back: A robot's guide to crossing the reality gap
Collins, J. T. (2022) · Queensland University of Technology · doi:10.5204/thesis.eprints.230537 · Cited by 2
Stamatelis, G., & Kalouptsidis, N. (2023) · doi:10.36227/techrxiv.22723507 · Cited by 2
Integration of Reinforcement Learning and Unreal Engine for Enemy Containment via Autonomous Swarms
Peterson, D., Andrades, B., Lizarazu-Ampuero, K., Deshmukh, J., Stapor, T., Destaffan, W., Engel, D., Krometis, J., & Kauffman, J. A. (2023) · AIAA SciTech Forum · doi:10.2514/6.2023-2674 · Cited by 1
Learning to Take Cover with Navigation-Based Waypoints via Reinforcement Learning
Aris, T., Ustun, V., & Kumar, R. (2023) · Proceedings of the ... International Florida Artificial Intelligence Research Society Conference · doi:10.32473/flairs.36.133348 · Cited by 1
“One example of something similar to this paper is OpenAI’s hide -and-seek environment (Baker et al. 2019), which uses LIDAR-inspired rays for agents to find ways to be occluded from other agents.”
BackgroundPlay With Me: Towards Explaining the Benefits of Autocurriculum Training of Learning Agents
Veith, E., Logemann, T., Wellßow, A., & Balduin, S. (2024) · IEEE PES Innovative Smart Grid Technologies Conference Europe · doi:10.1109/isgteurope62998.2024.10863007 · Cited by 4
ArenaSim: A High-Performance Simulation Platform for Multi-Robot Self-Play Learning
Ke, Y., Li, S., Li, Z., Li, H., Liu, Y., & He, Y. (2025) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2025.3557224 · Cited by 1
“Baker et al. [10] build a hide-and-seek game environment using Mujoco [16].”
Self and Other Modelling in Cooperative Resource Gathering with Multi-agent Reinforcement Learning
Davydov, V., Liusko, T., & Panov, A. I. (2020) · Advances in intelligent systems and computing · doi:10.1007/978-3-030-65596-9_9 · Cited by 1
Dynamic attention network for multi-UAV reinforcement learning
Xu, D., & Wu, S. (2021) · International Conference on Algorithms, High Performance Computing and Artificial Intelligence (AHPCAI) · doi:10.1117/12.2626437 · Cited by 1
“Despite the specialized neural network and comprehensive training skills, the DRL methods for MARL problems [11, 12, 13] still cannot solve the problem of large-scale multi-agent training.”
MethodMulti-Agent Reinforcement Learning with Clipping Intrinsic Motivation
(2022) · International Journal of Machine Learning and Computing · doi:10.18178/ijmlc.2022.12.3.1084 · Cited by 1
Advanced Mechanisms of Perception in the Digital Hide and Seek Game Based on Deep Learning
Livada, Č., & Hodak, D. (2022) · International Conference on Smart Systems and Technologies (SST) · doi:10.1109/sst55530.2022.9954814 · Cited by 1
“Although they did not use the ML-Agents tool, OpenAI used its own Deep Learning algorithm to create an environment where agents played a game of hide and seek and observed their behavior and how they tried to hide or find each other [10].”
MethodMulti-Agent Reinforcement Learning-Implementation of Hide and Seek
Jagtap, S., Allen, T., Gadiwan, S., & Mishra, S. (2022) · IEEE Global Conference for Advancement in Technology (GCAT) · doi:10.1109/gcat55367.2022.9972125 · Cited by 1
“For calculation of the value function without evaluating, until the final reward is received, the agent needs to stand by [11].”
BackgroundHuman Language Explanation for a Decision Making Agent via Automated Rationale Generation
Nguyen, X.-B., Tran, T. H., Pham, N. B., Do, D. N., & Yairi, T. (2022) · IEEE Access · doi:10.1109/access.2022.3214323 · Cited by 1
“In another example mentioned in [3], in the game ‘hide and seek’, agents learned how to The associate editor coordinating the review of this manuscript and approving it for publication was Anandakumar Haldorai . exploit their current complex environment to overcome obstacles that developers assumed…”
BackgroundLearning to solve arithmetic problems with a virtual abacus
Petruzzellis, F., Chen, L., & Testolin, A. (2023) · Proceedings of the Northern Lights Deep Learning Workshop · doi:10.7557/18.6805 · Cited by 1
Occlusion Avoidance in a Simulated Environment Using Reinforcement Learning
Szemenyei, M., & Szántó, M. (2023) · Applied Sciences · doi:10.3390/app13053090 · Cited by 1
Efficient Parameter Tuning for Multi-agent Simulation Using Deep Reinforcement Learning
Hirano, M., & Izumi, K. (2022) · IIAI International Congress on Advanced Applied Informatics (IIAI-AAI) · doi:10.1109/iiai-aai-winter58034.2022.00035 · Cited by 2
“For example, Baker et al. [3] showed that it can learn complex hide-and-seek strategies using tools that make the game more complex and high-dimensional.”
BackgroundTrentsios, P., Wolf, M., & Gerhard, D. (2023) · Procedia CIRP · doi:10.1016/j.procir.2023.03.141 · Cited by 1
Zachepylo, M., & Yushchenko, O. (2023) · Bulletin of National Technical University KhPI Series System Analysis Control and Information Technologies · doi:10.20998/2079-0023.2023.02.13
“Exploring multi-agent scenarios, this study investigated the emergence of tool use behaviors [28].”
BackgroundSwarMind: Harnessing Large Language Models for Flock Dynamics
Mounsif, M., Zehnder, K., Motie, Y., & Adam-Gaxotte, Z. (2023) · International Conference on Soft Computing & Machine Intelligence (ISCMI) · doi:10.1109/iscmi59957.2023.10458573 · Cited by 1
“While recent advances in RL have demonstrated multi-agent compatibly [1], managing scalable systems introduces significant trade-offs and still remains largely an open research question [4].”
BackgroundJi, H., & Jin, Y. (2024) · Journal of Computing and Information Science in Engineering · doi:10.1115/1.4065343 · Cited by 2
Better Understanding of Humans for Cooperative AI through Clustering
Su, E., Raffe, W. L., Mathieson, L., & Wang, Y. (2024) · IEEE Conference on Games (CoG) · doi:10.1109/cog60054.2024.10645647 · Cited by 1
“MARL has found success in numerous areas spanning tasks in games [9], economic interactions [10], and joint-decision making [11], but these have largely been between AI agents.”
BackgroundAutonomous Mental Development at the Individual and Collective Levels: Concept and Challenges
Lippi, M., Mariani, S., Martinelli, M., & Zambonelli, F. (2024) · IEEE Access · doi:10.1109/access.2024.3522362 · Cited by 2
“For instance, in a hide-and-seek scenario [5], agents moving in a complex simulated environment have learned how to effectively compete (hiders against seekers) and cooperate (coalitions of cooperating seekers/hiders) in very elaborated ways, in a continuous self-sustained learning process.”
InfluentialSchwartz, J., Newbury, R., Kulić, D., & Kurniawati, H. (2025) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-025-09716-6 · Cited by 2
“A side-by-side comparison of its properties with the other planning methods is shown in Table 3.”
NEAT Snake: a both evolutionary and neural network adaptation approach
Henrique, A. S., dos Santos, V. A., & Lyra, R. (2020) · Anais do Computer on the Beach · doi:10.14210/cotb.v11n1.p052-053 · Cited by 1
“Embora alguns alcancem seus objetivos, não são incomuns exemplos onde estas técnicas isoladas não sejam suficientes para a aplicação em jogos [1].”
BackgroundTowards sample-efficient policy learning with DAC-ML
Freire, I. T., Amil, A. F., Vouloutsi, V., & Verschure, P. F. M. J. (2021) · Procedia Computer Science · doi:10.1016/j.procs.2021.06.035 · Cited by 1
“An example of this limitation is the research on emergent tool-use in a hideand-seek multi-agent game; where DRL agents took up to 380 million episodes to learn some of the more sophisticated behaviors [7].”
BackgroundEmergence of Different Modes of Tool Use in a Reaching and Dragging Task
Nguyen, K. N., & Choe, Y. (2021) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9533775 · Cited by 1
“In our paper, we investigate tool use behavior using standard deep RL algorithms, as in Baker et al. (2020).”
BackgroundMethodExploring the Roles of Agents and Multi-Agent in Image Feature Extraction
Khaleefah, S. H., Mostafa, S. A., Mustapha, A., Daraman, R., Nasrudin, M. F., & Ismail, S. (2021) · International Symposium on Agents, Multi-Agent Systems and Robotics (ISAMSR) · doi:10.1109/isamsr53229.2021.9567863 · Cited by 2
An Approach for Context-Sensitive Situational Risk Evaluation of Autonomous Systems
Müller, M. S., Jazdi, N., & Weyrich, M. (2021) · IEEE International Conference on Emerging Technologies and Factory Automation (ETFA) · doi:10.1109/etfa45728.2021.9613313 · Cited by 1
Learning to Take Cover on Geo-Specific Terrains via Reinforcement Learning
Aris, T., Ustun, V., & Kumar, R. (2022) · Proceedings of the ... International Florida Artificial Intelligence Research Society Conference · doi:10.32473/flairs.v35i.130871 · Cited by 1
“One example of something similar to this paper is OpenAI’s hide-and-seek environment (Baker et al. 2019), which uses LIDAR-inspired rays for agents to find ways to be occluded from other agents.”
BackgroundWang, X., Liu, Y., Chang, Y., et al. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-10986-7_16 · Cited by 1
Beyond Value: CheckList for Testing Inferences in Planning-Based RL
Lam, K.-H., Tabatabai, D., Irvine, J., Bertucci, D., Ruangrotsakun, A., Kahng, M., & Fern, A. (2022) · Proceedings of the International Conference on Automated Planning and Scheduling · doi:10.1609/icaps.v32i1.19848 · Cited by 1
Rule Enforcing Through Ordering
Sychrovský, D., Desai, S., & Loebl, M. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-50670-3_3 · Cited by 1
“Similar approach was successfully used before [7].”
How Do You Act? An Empirical Study to Understand Behavior of Deep Reinforcement Learning AgentsPreprint
Meyes, R., Schneider, M., & Meisen, T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.03237 · Cited by 2
Efficient Competitive Self-Play Policy OptimizationPreprint
Zhong, Y., Zhou, Y., & Peng, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.06086 · Cited by 2
“Empirical successes have been observed in several challenging tasks, including Go [1, 2, 3], simulated hide-and-seek [4], simulated sumo wrestling [5], Capture the Flag [6], Dota 2 [7], StarCraft II [8], and poker [9], to name a few.”
BackgroundA Generative Model based Adversarial Security of Deep Learning and Linear Classifier ModelsPreprint
Sivaslioglu, S., Catak, F. O., & Şahinbaş, K. (2020) · Informatica · doi:10.48550/arxiv.2010.08546 · Cited by 7
“The machine learning methods consider player’s choices in the animation industry for games and analyze diseases to contribute to the decision-making mechanism [3–6].”
MethodDeep Policy Networks for NPC Behaviors that Adapt to Changing Design Parameters in Roguelike GamesPreprint
Sestini, A., Kuhnle, A., & Bagdanov, A. D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.03532 · Cited by 6
“2017), and particularly its self-attention layer which has also been successfully applied as state encoder in RL applications (Baker et al. 2019; Vinyals et al. 2019; Zambaldi et al. 2018).”
BackgroundMethodReinforcement Learning Based Decision Support Tool For Epidemic ControlPreprint
Chadi, M.-A., & Mousannif, H. (2021) · doi:10.31234/osf.io/tcr8s · Cited by 1
“Deep reinforcement learning (DRL) has proven its strength in various occasions such as playing Atari like human [12] and StarCraft II [13], playing hide and seek [14], mastering the game of go [15], and the game of DOTA [16].”
BackgroundExplore and Control with Adversarial SurprisePreprint
Fickinger, A., Jaques, N., Parajuli, S., Chang, M., Rhinehart, N., Berseth, G., Russell, S., & Levine, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.07394 · Cited by 9
Learning Provably Robust Motion Planners Using Funnel LibrariesPreprint
Gurgen, A. E., Majumdar, A., & Veer, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.08733 · Cited by 1
“motion planners that deal with rich sensory feedback and/or complex scenarios, potentially involving multiple external agents [1, 2].”
BackgroundMulti-AGV Path Planning Using Deep Reinforcement Learning with Internal CuriosityPreprint
Yin, H., Su, S., Lin, Y., Festl, K., Yan, J., & Watzenig, D. (2024) · Research Square · doi:10.21203/rs.3.rs-4453111/v1 · Cited by 1
Tafanidis, N. A., Banerjee, A., & Nikolakopoulos, G. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5357485 · Cited by 1
Pasichnyk, I. (2026) · arXiv · doi:10.48550/arxiv.2603.07360
“The study of emergent social behavior in agent populations has a long history, from Sugarscape [Epstein and Axtell, 1996] to recent work on emergent tool use from autocurricula [Baker et al., 2020].”
Meta-control of social learning strategiesPreprint
Yaman, A., Bredèche, N., Çaylak, O., Leibo, J. Z., & Lee, S. W. (2021) · arXiv (Cornell University) · doi:10.1371/journal.pcbi.1009882
“Furthermore, the computational framework of meta-social learning be used to test new hypotheses about multi-agent social learning [38, 3].”
Multi-Radar Tracking Optimization for Collaborative CombatPreprint
Nour, N., Belhaj-Soullami, R., Buron, C. L. R., Peres, A., & Barbaresco, F. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.11733
Generative adversarial simulator
Raiman, J. (2021) · International Journal of Artificial Intelligence and Machine Learning · doi:10.51483/ijaiml.1.1.2021.31-46
Learning to Play Robot Soccer from Partial Observations
Szemenyei, M., & Reizinger, P. (2020) · International Symposium on Measurement and Control in Robotics (ISMCR) · doi:10.1109/ismcr51255.2020.9263715
Towards sample-efficient episodic control with DAC-MLPreprint
Freire, I. T., Amil, A. F., Vouloutsi, V., & Verschure, P. F. M. J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.13779
“An example of this limitation is the research on emergent tool-use in a hideand-seek multi-agent game; where DRL agents took up to 380 million episodes to learn some of the more sophisticated behaviors [7].”
BackgroundProposta de modelo NEAT para jogar One Night Ultimate Werewolf
Costa, R., Henrique, A. S., Lyra, R., et al. (2021) · Anais do Computer on the Beach · doi:10.14210/cotb.v12.p509-511
“Apesar de serem capazes de realizar seus objetivos, não são incomuns exemplos em que o uso dessas técnicas isoladas não é o suficiente em jogos [8, 9].”
BackgroundComplex Momentum for Optimization in GamesPreprint
Lorraine, J., Acuna, D., Vicol, P. A., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.08431
“, 2014), actorcritic models (Pfau and Vinyals, 2016), curriculum learning (Baker et al., 2019; Balduzzi et al., 2019; Sukhbaatar et al., 2018), hyperparameter optimization (Lorraine and Duvenaud, 2018; Lorraine et al.”
BackgroundFairness for Cooperative Multi-Agent Learning with Equivariant Policies.Preprint
Grupen, N. A., Selman, B., & Lee, D. D. (2021) · arXiv (Cornell University)
Did I do that? Blame as a means to identify controlled effects in reinforcement learningPreprint
Corcoll, O., Mohamed, Y., & Vicente, R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.00266
“The recent success of reinforcement learning (RL) methods in difficult environments such as Hide & Seek (Baker et al., 2019), StarCraft II (Vinyals et al.”
BackgroundRLupus: Cooperation through emergent communication in The Werewolf social deduction game
Brandizzi, N., Grossi, D., & Iocchi, L. (2022) · Intelligenza Artificiale · doi:10.3233/ia-210081 · Cited by 9
“In their work, Baker et al. [3] showed how a few simple rules from the Hide’n Seek game can generate complicated behaviors to the point of exploiting environmental errors to their advantages.”
Supervised feature selection using sparse evolutionary training and neuron strength
Girdziunas, K. G. (2021)
Make Us Smile! AI and the Violation of Human Intentions
Wolf-Brenner, C. (2021) · doi:10.18690/978-961-286-485-9.5
“6 See Baker et al. 2020, p. 6 7 See Holcomb et al. 2018, p. 68 8 See Holcomb et al. 2018, p. 70 72 34TH BLED ECONFERENCE DIGITAL SUPPORT FROM CRISIS TO PROGRESSIVE CHANGE Bostrom does slightly hint at his account of intention being tied to the way goals are achieved in the sense of a method, and not the goals themselv…”
MethodInfluentialCompetitive physical interaction by reinforcement learning agents using intention estimation
Noda, H., Nishikawa, S., Niiyama, R., & Kuniyoshi, Y. (2021) · IEEE RO-MAN · doi:10.1109/ro-man50785.2021.9515411
“It is often used for competitive multi-agent tasks [28], [29].”
BackgroundIndividual and Collective Autonomous DevelopmentPreprint
Lippi, M., Mariani, S., Martinelli, M., & Zambonelli, F. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.11223
Learning Algorithms with Self-Play: A New Approach to the Distributed Directory Problem
Khanchandani, P., Richter, O., Rusch, L., & Wattenhofer, R. P. (2021) · IEEE International Conference on Tools with Artificial Intelligence (ICTAI) · doi:10.1109/ictai52525.2021.00081
“The policies learn to maximize their cumulative rewards R = ∑L i=1 ri with proximal policy optimization [12] in a self-play setup [13].”
BackgroundHandling Realistic Noise in Multi-Agent Systems with Self-Supervised Learning and Curiosity
Szemenyei, M., & Reizinger, P. (2021) · Journal of Artificial Intelligence and Soft Computing Research · doi:10.2478/jaiscr-2022-0009
Ziv, N. E. (2021) · Faculty Opinions – Post-Publication Peer Review of the Biomedical Literature · doi:10.3410/f.737316752.793584289
Multi-agent Reinforcement Learning with Clipping Intrinsic Motivation
(2022) · International Journal of Machine Learning and Computing · doi:10.18178/ijmlc.2022.12.5.1101
Research on Perception and Control Technology for Robot Dexterous OperationPreprint
Zhang, T., & Mo, H. (2022) · Research Square · doi:10.21203/rs.3.rs-1697958/v1
Cyber Sandbox — An Experiment on Collaborative Sketching in Shared Virtual Space
Messini, V., & Minovski, D. (2022) · Универзитет уметности у Београду, Факултет примењених уметности eBooks · doi:10.18485/smartart.2022.2.ch14
UM ESTUDO SOBRE SISTEMAS MULTIAGENTES COM APRENDIZADO PROFUNDO POR REFORÇO
Godinho, A. B. G. (2023) · Instituto Scientia eBooks · doi:10.55232/1087004.4
A CVD Critical Level-aware Scheduling Model Based on Reinforcement Learning for ECG Service Request
Gao, J., Yu, J., Wang, N., Feng, P., Cheng, H., Zhou, B., & Wang, Z. (2022) · International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery (CyberC) · doi:10.1109/cyberc55534.2022.00039
“It has played a good role in many fields [4].”
BackgroundStamatelis, G., & Kalouptsidis, N. (2023) · doi:10.36227/techrxiv.22723507.v1
Vergel, J. P. G. (2022) · Repositorio Institucional (Pontificia Universidad Javeriana) · doi:10.11144/javeriana.10554.61561
Order from Chaos: Leveraging the Order in Time as Intrinsic Reward
Nica, A. C., & Florea, A. M. (2023) · International Conference on Control Systems and Computer Science · doi:10.1109/cscs59211.2023.00032
“RL has shown great success in a wide range of applications, from game playing to robotics [1]–[5].”
Background¿Son las computadoras agentes inteligentes capaces de conocimiento?
Esparza, G., & Martínez-Barba, D. A. (2023) · Daímon · doi:10.6018/daimon.557551
Metadata
- Authors
- Baker, B., Kanitscheider, I., Markov, T., Wu, Y., Powell, G., McGrew, B., & Mordatch, I.
- Affiliations
- OpenAI (United States) (US)
- Google (United States) (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2019
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1909.07528
- OpenAlex
- W2973525135
- Semantic Scholar
- 33a2c0f3b9a0adc452a13d53f950c0c3c4abb11b
- Cited by
- 424
- References
- 70
- Citations by year
- 2026: 3 · 2025: 16 · 2024: 63 · 2023: 60 · 2022: 69 · 2021: 117 · 2020: 88 · 2019: 7
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
- Physical Sciences › Computer Science › Artificial Intelligence › Psychiatry, Mental Health, Neuroscience
- Keywords
- multi-agent competition, hide and seek, multi-agent coordination, intrinsic motivation, intelligence tests, environmental complexity, overcoming obstacles
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
