Emergent Complexity via Multi-Agent Competition
Open accessThe DOI matches its OpenAlex record; 20 of the 20 references it cites were found in a registry or an index.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612 · cited 4×
“Reinforcement Learning (RL) is exciting because good reinforcement learning algorithms exist [Mnih et al., 2015, Silver et al., 2016, Schulman et al., 2015a, Mnih et al., 2016, Schulman et al., 2015b, Lillicrap et al., 2015, Schulman et al., 2017], allowing us to train agents that accomplish a…”
BackgroundMethodInfluentialSelf-citationContinuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782
Proximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132 · cited 2×
“The approach is same as synchronous actor critic of Mnih et al. [2016].”
BackgroundMethodMuJoCo: A physics engine for model-based control
Todorov, E., Erez, T., & Tassa, Y. (2012) · 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2012.6386109 · Cited by 4,767
Trust region policy optimization
Schulman, J., Levine, S., Moritz, P., Jordan, M. I., & Abbeel, P. (2015) · arXiv · doi:10.48550/arxiv.1502.05477 · Cited by 4,578
Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments
Lowe, R., Wu, Y., Tamar, A., Harb, J., Abbeel, P., & Mordatch, I. (2017) · arXiv · doi:10.48550/arxiv.1706.02275 · Cited by 3,377
Self-citationA Comprehensive Survey of Multiagent Reinforcement Learning
Busoniu, L., Babuska, R., & De Schutter, B. (2008) · IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) · doi:10.1109/tsmcc.2007.913919 · Cited by 2,235 · cited 9×
“A lot of the work on multi-agent RL is focused on cooperative settings, see Busoniu et al. [2008] for a review of multi-agent RL and Panait and Luke [2005] for a review focused on cooperative settings. Tampuu et al. [2017] studied the application of deep Q-learning to train Pong agents with competitive and collaborati…”
BackgroundInfluentialHigh-Dimensional Continuous Control Using Generalized Advantage Estimation
Schulman, J., Moritz, P., Levine, S., Jordan, M., & Abbeel, P. (2015) · arXiv · doi:10.48550/arxiv.1506.02438 · Cited by 2,103
Counterfactual Multi-Agent Policy Gradients
Foerster, J., Farquhar, G., Afouras, T., Nardelli, N., & Whiteson, S. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11794 · Cited by 1,873
Temporal difference learning and TD-Gammon
Tesauro, G. (1995) · Communications of the ACM · doi:10.1145/203330.203343 · Cited by 1,618 · cited 2×
“Self-play in competitive multi-agent environments is not a new idea – it has already been explored in TD-gammon [Tesauro, 1995] and refined in AlphaGo [Silver et al., 2016] and Dota 2 [OpenAI].”
BackgroundAndrychowicz, M., Wolski, F., Ray, A., Schneider, J., Fong, R., Welinder, P., McGrew, B., Tobin, J., Abbeel, P., & Zaremba, W. (2017) · arXiv · doi:10.48550/arxiv.1707.01495 · Cited by 1,355
Benchmarking Deep Reinforcement Learning for Continuous Control
Duan, Y., Chen, X., Houthooft, R., Schulman, J., & Abbeel, P. (2016) · arXiv · doi:10.48550/arxiv.1604.06778 · Cited by 1,114 · cited 1×
“[2015b] and Duan et al. [2016]. However, engineering such dense rewards for the competitive tasks is not straight forward.”
BackgroundSims, K. (1994) · Proceedings of the 21st annual conference on Computer graphics and interactive techniques - SIGGRAPH '94 · doi:10.1145/192161.192167 · Cited by 1,005 · cited 2×
“Our work is also related to early work in the graphics community [Sims, 1994] on evolving creature morphology in varying environments using genetic algorithm and work in animation [Wampler et al., 2010] for adversarial games.”
BackgroundSimple statistical gradient-following algorithms for connectionist reinforcement learning
Williams, R. J. (1992) · Machine Learning · doi:10.1023/a:1022672621406 · Cited by 817
Emergence of Locomotion Behaviours in Rich Environments
Heess, N., TB, D., Sriram, S., Lemmon, J., Merel, J., Wayne, G., Tassa, Y., Erez, T., Wang, Z., Eslami, S. M. A., et al. (2017) · arXiv · doi:10.48550/arxiv.1707.02286 · Cited by 657 · cited 6×
“Our distributed PPO implementation is similar to the implementation of Heess et al. [2017], where instead of the KL penalty we used the clipped objective as proposed in PPO [Schulman et al., 2017]. Figure 2 shows a high-level view of the architecture. Essentially, we do multiple rollouts in parallel for each agent and…”
BackgroundMethodInfluentialLearning with Opponent-Learning Awareness
Foerster, J., Chen, R. Y., Al-Shedivat, M., Whiteson, S., Abbeel, P., & Mordatch, I. (2018) · International Joint Conference on Autonomous Agents and Multiagent Systems · doi:10.65109/hgwa8807 · Cited by 215
Self-citationIntrinsic Motivation and Automatic Curricula via Asymmetric Self-Play
Sukhbaatar, S., Lin, Z., Kostrikov, I., Synnaeve, G., Szlam, A., & Fergus, R. (2017) · arXiv · doi:10.48550/arxiv.1703.05407 · Cited by 149 · cited 1×
“Sukhbaatar et al. [2017] introduced a self-play method for generating an automatic training curriculum in single-agent environments.”
Method
Cited by
Dota 2 with Large Scale Deep Reinforcement Learning
OpenAI, :, Berner, C., Brockman, G., Chan, B., Cheung, V., Dębiak, P., Dennison, C., Farhi, D., Fischer, Q., et al. (2019) · arXiv · doi:10.48550/arxiv.1912.06680 · Cited by 1,065
“Self-play was shown to be a powerful algorithm for learning skills within high-dimensional continuous environments [33] and a method for automatically generating curricula [34].”
MethodSelf-citationDesigning neural networks through neuroevolution
Stanley, K. O., Clune, J., Lehman, J., & Miikkulainen, R. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-018-0006-z · Cited by 689
A survey and critique of multiagent deep reinforcement learning
Hernandez-Leal, P., Kartal, B., & Taylor, M. E. (2019) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-019-09421-1 · Cited by 671
Unity: A General Platform for Intelligent Agents
Juliani, A., Berges, V.-P., Teng, E., Cohen, A., Harper, J., Elion, C., Goy, C., Gao, Y., Henry, H., Mattar, M., et al. (2018) · arXiv · doi:10.48550/arxiv.1809.02627 · Cited by 626
“…Imitation Learning (GAIL) (Ho and Ermon, 2016) and Behavioral Cloning (BC) (Hussein et al., 2017), support for Self-Play (Baker et al., 2019; Bansal et al., 2017) in both symmetric and asymmetric games, as well as the option to extend algorithms and policies with the Intrinsic Curiosity…”
BackgroundApplications of reinforcement learning in energy systems
Perera, A. T. D., & Kamalaruban, P. (2021) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2020.110618 · Cited by 420
Metadata
- Authors
- Bansal, T., Pachocki, J., Sidor, S., Sutskever, I., & Mordatch, I.
- Venue
- arXiv (Cornell University)
- Publisher
- Cornell University
- Published
- 2017-10-10
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1710.03748
- OpenAlex
- W2762117857
- Cited by
- 133
- References
- 20
- Type
- posted content
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.