The DOI matches its DataCite record; 65 of the 70 references it cites were found in a registry or an index.
Emergent Tool Use From Multi-Agent Autocurricula
Open accessAbstract
Through multi-agent competition, the simple objective of hide-and-seek, and standard reinforcement learning algorithms at scale, we find that agents create a self-supervised autocurriculum inducing multiple distinct rounds of emergent strategy, many of which require sophisticated tool use and coordination. We find clear evidence of six emergent phases in agent strategy in our environment, each of which creates a new pressure for the opposing team to adapt; for instance, agents learn to build multi-object shelters using moveable boxes which in turn leads to agents discovering that they can overcome obstacles using ramps. We further provide evidence that multi-agent competition may scale better with increasing environment complexity and leads to behavior that centers around far more human-relevant skills than other self-supervised reinforcement learning methods such as intrinsic motivation. Finally, we propose transfer and fine-tuning as a way to quantitatively evaluate targeted capabilities, and we compare hide-and-seek agents to both intrinsic motivation and random initialization baselines in a suite of domain-specific intelligence tests.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“For example, Baker et al. (2019) showed that even a simple game of hide and seek can lead to sophisticated tool use and coordination by MARL agents.”
From Multi-Agent Risks from Advanced AI (2025) · cites this paper as Baker et al., 2019
The cited source says what the claim attributes to it.
In the citing paper
“Agents develop exploitative strategies (Baker et al., 2019), learn deception via theory-ofmind (Schulz et al., 2023), and discover betrayal without explicit training (Aitchison et al., 2021) -eluding threat models predicated on an identifiable adversary.”
From Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agents (2025) · cites this paper as Baker et al., 2019
The cited source says what the claim attributes to it.
In the citing paper
“Consequently, we used the PPO agent in Experiments III and IV, as PPO is a competent agent used by OpenAI [32].”
From Playtesting: What is Beyond Personas (2023) · cites this paper as [32]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 4×
“[75] Sepp Hochreiter and Jürgen Schmidhuber.”
MethodInfluentialReinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,268
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612 · cited 2×
“[30] David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, et al.”
BackgroundMethodProximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625 · cited 4×
“PPO [58], a policy gradient variant, penalizes large changes to the policy to prevent training instabilities.”
BackgroundInfluentialMuJoCo: A physics engine for model-based control
Todorov, E., Erez, T., & Tassa, Y. (2012) · 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2012.6386109 · Cited by 4,767 · cited 2×
“[57] Emanuel Todorov, Tom Erez, and Yuval Tassa.”
BackgroundMethodArms races between and within species
Dawkins, R., & Krebs, J. R. (1979) · Proceedings of the Royal Society B Biological Sciences · doi:10.1098/rspb.1979.0081 · Cited by 2,214 · cited 2×
“The vast amount of complexity and diversity on Earth evolved due to co-evolution and competition between organisms, directed by natural selection [5].”
BackgroundHigh-Dimensional Continuous Control Using Generalized Advantage Estimation
Schulman, J., Moritz, P., Levine, S., Jordan, M., & Abbeel, P. (2015) · arXiv · doi:10.48550/arxiv.1506.02438 · Cited by 2,103 · cited 3×
“We use Generalized Advantage Estimation [3] with horizon length H to estimate the advantage function.”
BackgroundMethodTemporal difference learning and TD-Gammon
Tesauro, G. (1995) · Communications of the ACM · doi:10.1145/203330.203343 · Cited by 1,618 · cited 3×
“In the context of multi-agent RL, [7], [30], [9], [13] and [10] used self-play with deep RL techniques to achieve super-human performance in Backgammon, Go, Dota, Capture-the-Flag and Starcraft, respectively.”
BackgroundMethodConcrete Problems in AI Safety
Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schulman, J., & Mané, D. (2016) · arXiv · doi:10.48550/arxiv.1606.06565 · Cited by 1,314
Learning to Communicate with Deep Multi-Agent Reinforcement Learning
Foerster, J. N., Assael, Y. M., de Freitas, N., & Whiteson, S. (2016) · arXiv · doi:10.48550/arxiv.1605.06676 · Cited by 1,217 · cited 2×
“[32] Jakob Foerster, Ioannis Alexandros Assael, Nando de Freitas, and Shimon Whiteson.”
Background(1978) · Medical Entomology and Zoology · Cited by 1,199
Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm
Silver, D., Hubert, T., Schrittwieser, J., Antonoglou, I., Lai, M., Guez, A., Lanctot, M., Sifre, L., Kumaran, D., Graepel, T., et al. (2017) · arXiv · doi:10.48550/arxiv.1712.01815 · Cited by 1,102 · cited 2×
“[8] David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, et al.”
BackgroundSims, K. (1994) · Proceedings of the 21st annual conference on Computer graphics and interactive techniques - SIGGRAPH '94 · doi:10.1145/192161.192167 · Cited by 1,005 · cited 2×
“Our work sees itself in the tradition of previous studies that showcase emergent complexity in simple physically grounded environments [11, 12, 13, 14]; the success in these settings inspires confidence that inducing autocurricula in physically grounded and open-ended environments could eventually enable agents to acq…”
BackgroundEvolving 3D Morphology and Behavior by Competition
Sims, K. (1994) · Artificial Life · doi:10.1162/artl.1994.1.353 · Cited by 920
Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations
Rajeswaran, A., Kumar, V., Gupta, A., Vezzani, G., Schulman, J., Todorov, E., & Levine, S. (2018) · Robotics: Science and Systems XIV · doi:10.15607/rss.2018.xiv.049 · Cited by 847 · cited 2×
“References [1] Aravind Rajeswaran, Vikash Kumar, Abhishek Gupta, Giulia Vezzani, John Schulman, Emanuel Todorov, and Sergey Levine.”
BackgroundAnimal tool behavior: the use and manufacture of tools by animals
(2011) · Choice Reviews Online · doi:10.5860/choice.49-1468 · Cited by 783
(2016) · neural information processing systems · Cited by 735
TrueSkill™: A Bayesian Skill Rating System
Herbrich, R., Minka, T., & Graepel, T. (2007) · The MIT Press eBooks · doi:10.7551/mitpress/7503.003.0076 · Cited by 693 · cited 2×
“Metrics like ELO (Elo, 1978) or Trueskill (Herbrich et al., 2007) can more reliably measure whether performance is improving relative to previous policy versions or other policies in a population; however, these metrics still do not give insight into whether improved performance stems from new adaptations or improving…”
BackgroundHuman-level performance in 3D multiplayer games with population-based reinforcement learning
Jaderberg, M., Czarnecki, W. M., Dunning, I., Marris, L., Lever, G., Castañeda, A. G., Beattie, C., Rabinowitz, N. C., Morcos, A. S., Ruderman, A., et al. (2019) · Science · doi:10.1126/science.aau6249 · Cited by 662 · cited 2×
“Our work sees itself in the tradition of previous studies that showcase emergent complexity in simple physically grounded environments [11, 12, 13, 14]; the success in these settings inspires confidence that inducing autocurricula in physically grounded and open-ended environments could eventually enable agents to acq…”
BackgroundMethodEmergence of Locomotion Behaviours in Rich Environments
Heess, N., TB, D., Sriram, S., Lemmon, J., Merel, J., Wayne, G., Tassa, Y., Erez, T., Wang, Z., Eslami, S. M. A., et al. (2017) · arXiv · doi:10.48550/arxiv.1707.02286 · Cited by 657 · cited 2×
“[4] Nicolas Heess, Srinivasan Sriram, Jay Lemmon, Josh Merel, Greg Wayne, Yuval Tassa, Tom Erez, Ziyu Wang, SM Eslami, Martin Riedmiller, et al.”
BackgroundManufacture and use of hook-tools by New Caledonian crows
Hunt, G. R. (1996) · Nature · doi:10.1038/379249a0 · Cited by 645
Intrinsically Motivated Reinforcement Learning
Singh, S. P., Barto, A. G., & Chentanez, N. (2005) · doi:10.21236/ada440280 · Cited by 614 · cited 2×
“[35] Nuttapong Chentanez, Andrew G Barto, and Satinder P Singh.”
BackgroundMethodVIME: Variational Information Maximizing ExplorationPreprint
Houthooft, R., Chen, X., Duan, Y., Schulman, J., Turck, F. D., & Abbeel, P. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.09674 · Cited by 597
Asymmetric Actor Critic for Image-Based Robot Learning
Pinto, L., Andrychowicz, M., Welinder, P., Zaremba, W., & Abbeel, P. (2018) · Robotics: Science and Systems · doi:10.15607/rss.2018.xiv.008 · Cited by 506 · cited 2×
“[59] Lerrel Pinto, Marcin Andrychowicz, Peter Welinder, Wojciech Zaremba, and Pieter Abbeel.”
Background#Exploration: A Study of Count-Based Exploration for Deep Reinforcement LearningPreprint
Tang, H., Houthooft, R., Foote, D., Stooke, A., Chen, X., Duan, Y., Schulman, J., Turck, F. D., & Abbeel, P. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.04717 · Cited by 504 · cited 2×
“One example is count-based exploration, where agents are incentivized to reach infrequently visited states by maintaining state visitation counts [37, 38, 39] or density estimators [40, 41].”
BackgroundOne-Shot Imitation LearningPreprint
Duan, Y., Andrychowicz, M., Stadie, B. C., Ho, O. J., Schneider, J., Sutskever, I., Abbeel, P., & Zaremba, W. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.07326 · Cited by 473
Incentivizing Exploration In Reinforcement Learning With Deep Predictive ModelsPreprint
Stadie, B. C., Levine, S., & Abbeel, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1507.00814 · Cited by 462 · cited 2×
“Another paradigm are transition-based methods, in which agents are rewarded for high prediction error in a learned forward or inverse dynamics model [42, 43, 44, 45, 46, 47, 48, 49].”
BackgroundVariational Information Maximisation for Intrinsically Motivated Reinforcement LearningPreprint
Mohamed, S., & Rezende, D. J. (2015) · Neural Information Processing Systems · doi:10.48550/arxiv.1509.08731 · Cited by 460 · cited 2×
“Another paradigm are transition-based methods, in which agents are rewarded for high prediction error in a learned forward or inverse dynamics model [42, 43, 44, 45, 46, 47, 48, 49].”
BackgroundA Possibility for Implementing Curiosity and Boredom in Model-Building Neural Controllers
Schmidhuber, J. (1991) · The MIT Press eBooks · doi:10.7551/mitpress/3115.003.0030 · Cited by 455 · cited 1×
“Another paradigm are transition-based methods, in which agents are rewarded for high prediction error in a learned forward or inverse dynamics model [42, 43, 44, 45, 46, 47, 48, 49].”
BackgroundIntrinsically Motivated Reinforcement Learning: An Evolutionary Perspective
Singh, S. P., Lewis, R. L., Barto, A. G., & Sorg, J. (2010) · IEEE Transactions on Autonomous Mental Development · doi:10.1109/tamd.2010.2051031 · Cited by 444 · cited 2×
“Intrinsic motivation methods have been widely studied in the literature [35, 36].”
BackgroundMethodAn analysis of model-based Interval Estimation for Markov Decision Processes
Strehl, A. L., & Littman, M. L. (2008) · Journal of Computer and System Sciences · doi:10.1016/j.jcss.2007.08.009 · Cited by 428 · cited 3×
“One example is count-based exploration, where agents are incentivized to reach infrequently visited states by maintaining state visitation counts [37, 38, 39] or density estimators [40, 41].”
BackgroundMethodCompetitive Coevolution through Evolutionary Complexification
Stanley, K. O., & Miikkulainen, R. P. (2004) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1338 · Cited by 415 · cited 2×
“Early work explored self-play using genetic algorithms [16, 17, 18, 19].”
BackgroundCounterfactual Multi-Agent Policy GradientsPreprint
Foerster, J., Farquhar, G., Afouras, T., Nardelli, N., & Whiteson, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.08926 · Cited by 408 · cited 2×
“At optimization time, the value function has access to the full unmasked environment state, similar to [59, 33, 60].”
BackgroundAvida: A Software Platform for Research in Computational Evolutionary Biology
Ofria, C., & Wilke, C. O. (2004) · Artificial Life · doi:10.1162/106454604773563612 · Cited by 378
Count-Based Exploration with Neural Density Models
Ostrovski, G., Bellemare, M. G., Oord, A. V. D., & Munos, R. (2017) · arXiv · doi:10.48550/arxiv.1703.01310 · Cited by 377 · cited 2×
“One example is count-based exploration, where agents are incentivized to reach infrequently visited states by maintaining state visitation counts [37, 38, 39] or density estimators [40, 41].”
BackgroundRelational Reinforcement Learning
Džeroski, S., De Raedt, L., & Driessens, K. (2001) · Machine Learning · doi:10.1023/a:1007694015589 · Cited by 371
Emergence of Grounded Compositional Language in Multi-Agent PopulationsPreprint
Mordatch, I., & Abbeel, P. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.04908 · Cited by 360 · cited 1×
“In addition, communication has also been shown to emerge from multi-agent RL [31, 32, 33, 34].”
BackgroundSelf-citationWang, R., Lehman, J., Clune, J., & Stanley, K. O. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.01753 · Cited by 311 · cited 2×
“[29] Rui Wang, Joel Lehman, Jeff Clune, and Kenneth O Stanley.”
BackgroundSocial Influence as Intrinsic Motivation for Multi-Agent Deep Reinforcement LearningPreprint
Jaques, N., Lazaridou, A., Hughes, E., Gülçehre, Ç., Ortega, P. A., Strouse, D., Leibo, J. Z., & de Freitas, N. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.08647 · Cited by 296
Reverse Curriculum Generation for Reinforcement LearningPreprint
Florensa, C., Held, D., Wulfmeier, M., Zhang, M., & Abbeel, P. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.05300 · Cited by 277 · cited 2×
“[27] Carlos Florensa, David Held, Markus Wulfmeier, Michael Zhang, and Pieter Abbeel.”
BackgroundLearning Dexterous In-Hand Manipulation
OpenAI, Andrychowicz, M., Baker, B., Chociej, M., Jozefowicz, R., McGrew, B., Pachocki, J., Petron, A., Plappert, M., Powell, G., et al. (2018) · arXiv · doi:10.48550/arxiv.1808.00177 · Cited by 248 · cited 2×
“[2] Marcin Andrychowicz, Bowen Baker, Maciek Chociej, Rafal Jozefowicz, Bob McGrew, Jakub Pachocki, Arthur Petron, Matthias Plappert, Glenn Powell, Alex Ray, et al.”
BackgroundSelf-citationParedis, J. (1995) · Artificial Life · doi:10.1162/artl.1995.2.355 · Cited by 229
An object-oriented representation for efficient reinforcement learning
Diuk, C., Cohen, A., & Littman, M. L. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390187 · Cited by 226 · cited 2×
“[71] Carlos Diuk, Andre Cohen, and Michael L Littman.”
BackgroundMethodIntrinsically Motivated Goal Exploration Processes with Automatic Curriculum LearningPreprint
Forestier, S., Portelas, R., Mollard, Y., & Oudeyer, P. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.02190 · Cited by 219
Lehman, J., Clune, J., Misevic, D., Adami, C., Altenberg, L., Beaulieu, J., Bentley, P. J., Bernard, S., Beslon, G., Bryson, D. M., et al. (2020) · Artificial Life · doi:10.1162/artl_a_00319 · Cited by 181
Multi-agent Reinforcement Learning in Sequential Social DilemmasPreprint
Leibo, J. Z., Zambaldi, V., Lanctot, M., Marecki, J., & Graepel, T. (2017) · arXiv (Cornell University) · doi:10.5555/3091125.3091194 · Cited by 178
Methods for Competitive Co-Evolution: Finding Opponents Worth Beating
Rosin, C. D., & Belew, R. K. (1995) · Cited by 172
Relational Deep Reinforcement LearningPreprint
Zambaldi, V., Raposo, D. C., Santoro, A., Bapst, V., Li, Y., Babuschkin, I., Tuyls, K., Reichert, D., Lillicrap, T., Lockhart, E., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.01830 · Cited by 153 · cited 2×
“[73] Vinicius Zambaldi, David Raposo, Adam Santoro, Victor Bapst, Yujia Li, Igor Babuschkin, Karl Tuyls, David Reichert, Timothy Lillicrap, Edward Lockhart, et al.”
BackgroundMethodIntrinsic Motivation and Automatic Curricula via Asymmetric Self-Play
Sukhbaatar, S., Lin, Z., Kostrikov, I., Synnaeve, G., Szlam, A., & Fergus, R. (2017) · arXiv · doi:10.48550/arxiv.1703.05407 · Cited by 149
Rapid trial-and-error learning with simulation supports flexible tool use and physical reasoning
Allen, K. R., Smith, K. A., & Tenenbaum, J. B. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1912341117 · Cited by 144
Surprise-Based Intrinsic Motivation for Deep Reinforcement Learning
Achiam, J., & Sastry, S. (2017) · arXiv · doi:10.48550/arxiv.1703.01732 · Cited by 139 · cited 1×
“Another paradigm are transition-based methods, in which agents are rewarded for high prediction error in a learned forward or inverse dynamics model [42, 43, 44, 45, 46, 47, 48, 49].”
BackgroundEmergent Coordination Through CompetitionPreprint
Liu, S., Lever, G., Merel, J., Tunyasuvunakool, S., Heess, N., & Graepel, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.07151 · Cited by 119 · cited 3×
“Our work sees itself in the tradition of previous studies that showcase emergent complexity in simple physically grounded environments [11, 12, 13, 14]; the success in these settings inspires confidence that inducing autocurricula in physically grounded and open-ended environments could eventually enable agents to acq…”
BackgroundA multi-agent reinforcement learning model of common-pool resource appropriation
Perolat, J., Leibo, J. Z., Zambaldi, V., Beattie, C., Tuyls, K., & Graepel, T. (2017) · arXiv · doi:10.48550/arxiv.1707.06600 · Cited by 109 · cited 2×
“Many stages of emergent strategy can be mapped to behavioral shifts in the way agents interact with the tools in their environment, similar to [61, 62].”
BackgroundCoevolution of A Backgammon Player
Pollack, J. B., Blair, A., & Land, M. (1996) · Cited by 91
Improvisation through Physical Understanding: Using Novel Objects As Tools with Visual Foresight
Xie, A., Ebert, F., Levine, S., & Finn, C. (2019) · Robotics: Science and Systems · doi:10.15607/rss.2019.xv.001 · Cited by 88 · cited 2×
“For example, in [54] a real-world robot uses familiar and novel tools to move objects using model-based RL and imitation learning.”
BackgroundA Game-Theoretic Memory Mechanism for Coevolution
Ficici, S. G., & Pollack, J. B. (2003) · Lecture notes in computer science · doi:10.1007/3-540-45105-6_35 · Cited by 81 · cited 2×
“[69] Sevan G Ficici and Jordan B Pollack.”
BackgroundLarge-Scale Study of Curiosity-Driven Learning
Burda, Y. (2025) · Edinburgh Research Explorer (University of Edinburgh) · doi:10.57702/mspz8fly · Cited by 75
Lehman, J., Clune, J., Misevic, D., Adami, C., Altenberg, L., Beaulieu, J., Bentley, P. J., Bernard, S., Beslon, G., Bryson, D., et al. (2018) · arXiv (Cornell University) · Cited by 63
Leibo, J. Z., Hughes, E., Lanctot, M., & Graepel, T. (2019) · arXiv · doi:10.48550/arxiv.1903.00742 · Cited by 60 · cited 2×
“[6] Joel Z Leibo, Edward Hughes, Marc Lanctot, and Thore Graepel.”
BackgroundStructured agents for physical constructionPreprint
Bapst, V., Sánchez‐González, Á., Doersch, C., Stachenfeld, K., Kohli, P., Battaglia, P., & Hamrick, J. B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.03177 · Cited by 59 · cited 2×
“In [55], an agent solves construction tasks in a 2-D environment using both model-based and model-free methods.”
BackgroundMethodSoros, L. B., & Stanley, K. O. (2014) · doi:10.1162/978-0-262-32621-6-ch128 · Cited by 51
Malthusian Reinforcement Learning
Leibo, J. Z., Pérolat, J., Hughes, E., Wheelwright, S., Marblestone, A. H., Duéñez‐Guzmán, E. A., Sunehag, P., Dunning, I., & Graepel, T. (2019) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/fdir9565 · Cited by 30 · cited 2×
“[51] Joel Z Leibo, Julien Perolat, Edward Hughes, Steven Wheelwright, Adam H Marblestone, Edgar Duéñez-Guzmán, Peter Sunehag, Iain Dunning, and Thore Graepel.”
BackgroundResultPerpetuating Evolutionary Emergence
Channon, A., & Damper, R. I. (1998) · The MIT Press eBooks · doi:10.7551/mitpress/3119.003.0080 · Cited by 28 · cited 1×
“Open-ended evolution was further explored in the environments Polyworld [21] and Geb [22], where agents compete and mate in a 2D world, and in Tierra [23] and Avida [24], where computer programs compete for computational resources.”
BackgroundRequirements for Open-Ended Evolution in Natural and Artificial SystemsPreprint
Taylor, T. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1507.07403 · Cited by 16 · cited 1×
“More recent work attempted to formulate necessary preconditions for open-ended evolution [25, 26].”
Background
Flagged first, then by citations
5 of the 70 references couldn’t be matched to a known paper.
Research works printed but not linked: 11
- Kelsey R Allen, Kevin A Smith, and Joshua B Tenenbaum. The tools challenge: Rapid trial-anderror learning in physical problem solving. arXiv preprint arXiv:1907.09620 , 2019. · arXiv
- Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. Layer normalization. arXiv preprint arXiv:1607.06450 , 2016. · arXiv
- Rene Baillargeon and Susan Carey. Core cognition and beyond: The acquisition of physical and numerical knowledge. In In S. Pauen (Ed.), Early Childhood Development and Later Outcome , pp. 33-65. University Press, 2012.
- Trapit Bansal, Jakub Pachocki, Szymon Sidor, Ilya Sutskever, and Igor Mordatch. Emergent complexity via multi-agent competition. In International Conference on Learning Representations , 2018.
- Yuri Burda, Harrison Edwards, Amos Storkey, and Oleg Klimov. Exploration by random network distillation. In International Conference on Learning Representations , 2019b.
- Nick Haber, Damian Mrowca, Stephanie Wang, Li F Fei-Fei, and Daniel L Yamins. Learning to play with intrinsically-motivated, self-aware agents. In Advances in Neural Information Processing Systems , pp. 8388-8399, 2018.
- Max Jaderberg, Valentin Dalibard, Simon Osindero, Wojciech M Czarnecki, Jeff Donahue, Ali Razavi, Oriol Vinyals, Tim Green, Iain Dunning, Karen Simonyan, et al. Population based training of neural networks. arXiv preprint arXiv:1711.09846 , 2017. · arXiv
- Ryan Lowe, Yi Wu, Aviv Tamar, Jean Harb, Pieter Abbeel, and Igor Mordatch. Multi-agent actorcritic for mixed cooperative-competitive environments. In Advances in Neural Information Processing Systems , pp. 6379-6390, 2017.
- Deepak Pathak, Pulkit Agrawal, Alexei A Efros, and Trevor Darrell. Curiosity-driven exploration by self-supervised prediction. In International Conference on Machine Learning , pp. 2778-2787, 2017.
- Sainbayar Sukhbaatar, Rob Fergus, et al. Learning multiagent communication with backpropagation. In Advances in Neural Information Processing Systems , pp. 2244-2252, 2016.
- Larry Yaeger. Computational genetics, physiology, metabolism, neural systems, learning, vision, and behavior or poly world: Life in a new context. In SANTA FE INSTITUTE STUDIES IN THE SCIENCES OF COMPLEXITY-PROCEEDINGS VOLUME, volume 17, pp. 263-263. ADDISON-WESLEY PUBLISHING CO, 1994.
Cited by
Sorted by citations
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
Self-citationA Metaverse: Taxonomy, Components, Applications, and Open Challenges
Park, S., & Kim, Y. (2022) · IEEE Access · doi:10.1109/access.2021.3140175 · Cited by 2,046
“Hide and Seek is a simple yet effective simulation environment for multi-agent work that uses visual representations of objects and scenes from an egocentric perspective [218]–[221].”
Multi-agent deep reinforcement learning: a survey
Gronauer, S., & Diepold, K. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-021-09996-w · Cited by 957
Unity: A General Platform for Intelligent Agents
Juliani, A., Berges, V.-P., Teng, E., Cohen, A., Harper, J., Elion, C., Goy, C., Gao, Y., Henry, H., Mattar, M., et al. (2018) · arXiv · doi:10.48550/arxiv.1809.02627 · Cited by 626
“The features of the toolkit include a set of example environments, state of the art RL algorithms Soft Actor-Critic (SAC) (Haarnoja et al., 2018) and Proximal Policy Optimization (PPO) (Schulman et al., 2017), the IL algorithms Generative Adversarial Imitation Learning (GAIL) (Ho and Ermon, 2016) and Behavioral Clonin…”
BackgroundInfluentialDirect Fit to Nature: An Evolutionary Perspective on Biological and Artificial Neural Networks
Hasson, U., Nastase, S. A., & Goldstein, A. (2020) · Neuron · doi:10.1016/j.neuron.2019.12.002 · Cited by 387
Metadata
- Authors
- Baker, B., Kanitscheider, I., Markov, T., Wu, Y., Powell, G., McGrew, B., & Mordatch, I.
- Affiliations
- OpenAI (United States) (US)
- Google (United States) (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2019
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1909.07528
- OpenAlex
- W2973525135
- Semantic Scholar
- 33a2c0f3b9a0adc452a13d53f950c0c3c4abb11b
- Cited by
- 424
- References
- 70
- Citations by year
- 2026: 3 · 2025: 16 · 2024: 63 · 2023: 60 · 2022: 69 · 2021: 117 · 2020: 88 · 2019: 7
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
- Physical Sciences › Computer Science › Artificial Intelligence › Psychiatry, Mental Health, Neuroscience
- Keywords
- multi-agent competition, hide and seek, multi-agent coordination, intrinsic motivation, intelligence tests, environmental complexity, overcoming obstacles
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
