The DOI matches its DataCite record; 35 of the 37 references it cites were found in a registry or an index, and 2 are not checked yet.
Dota 2 with Large Scale Deep Reinforcement Learning
Open accessAbstract
On April 13th, 2019, OpenAI Five became the first AI system to defeat the world champions at an esports game. The game of Dota 2 presents novel challenges for AI systems such as long time horizons, imperfect information, and complex, continuous state-action spaces, all challenges which will become increasingly central to more capable AI systems. OpenAI Five leveraged existing reinforcement learning techniques, scaled to learn from batches of approximately 2 million frames every 2 seconds. We developed a distributed training system and tools for continual training which allowed us to train OpenAI Five for 10 months. By defeating the Dota 2 world champion (Team OG), OpenAI Five demonstrates that self-play reinforcement learning can achieve superhuman performance on a difficult task.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Deep reinforcement learning (DRL) algorithms have been successfully deployed in various types of simulation environments, yet their success beyond simulated worlds has been limited.”
From Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey (2020) · cites this paper as [20]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“As a result, successful application of deep RL often requires large teams of experts with significant computational resources (Berner et al., 2019; Schrittwieser et al., 2020; Ouyang et al., 2022).”
From TD-MPC2: Scalable, Robust World Models for Continuous Control (2023) · cites this paper as Berner et al., 2019
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Self-citationProximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625
Self-citationPlaying Atari with Deep Reinforcement Learning
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013) · arXiv · doi:10.48550/arxiv.1312.5602 · Cited by 9,471
Mastering the game of Go without human knowledge
Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., et al. (2017) · Nature · doi:10.1038/nature24270 · Cited by 9,084
Learning to Forget: Continual Prediction with LSTM
Gers, F. A., Schmidhuber, J., & Cummins, F. (2000) · Neural Computation · doi:10.1162/089976600300015015 · Cited by 6,122
Li, Z., & Hoiem, D. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2773081 · Cited by 4,202
A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play
Silver, D., Hubert, T., Schrittwieser, J., Antonoglou, I., Lai, M., Guez, A., Lanctot, M., Sifre, L., Kumaran, D., Graepel, T., et al. (2018) · Science · doi:10.1126/science.aar6404 · Cited by 3,562 · cited 2×
“For comparison Chess requires around one thousand values per observation (mostly 6-possibility categorical values) and Go around six thousand values (all binary)[12].”
BackgroundMethodGrandmaster level in StarCraft II using multi-agent reinforcement learning
Vinyals, O., Babuschkin, I., Czarnecki, W. M., Mathieu, M., Dudzik, A., Chung, J., Choi, D. H., Powell, R., Ewalds, T., Georgiev, P., et al. (2019) · Nature · doi:10.1038/s41586-019-1724-z · Cited by 3,455
Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
Goyal, P., Dollár, P., Girshick, R., Noordhuis, P., Wesolowski, L., Kyrola, A., Tulloch, A., Jia, Y., & He, K. (2017) · arXiv · doi:10.48550/arxiv.1706.02677 · Cited by 2,294
Research works printed but not linked: 12
- [15]15. Konda, V. R. & Tsitsiklis, J. N. Actor-critic algorithms in Advances in neural information processing systems (2000), 1008-1014.
- [16]16. Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T., Harley, T., Silver, D. & Kavukcuoglu, K. Asynchronous methods for deep reinforcement learning in International conference on machine learning (2016), 1928-1937.
- [17]17. Schulman, J., Moritz, P., Levine, S., Jordan, M. I. & Abbeel, P. High-Dimensional Continuous Control Using Generalized Advantage Estimation. CoRR abs/1506.02438 (2016). · arXiv
- [23]23. Chen, T., Goodfellow, I. J. & Shlens, J. Net2Net: Accelerating Learning via Knowledge Transfer in 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings (2016). < http://arxiv.org/abs/ 1511.05641 >.
- [27]27. Herbrich, R., Minka, T. & Graepel, T. TrueSkill: a Bayesian skill rating system in Advances in neural information processing systems (2007), 569-576.
- [38]38. Anthony, T., Tian, Z. & Barber, D. Thinking fast and slow with deep learning and tree search in Advances in Neural Information Processing Systems (2017), 5360-5370.
- [41]41. Kulkarni, T. D., Narasimhan, K., Saeedi, A. & Tenenbaum, J. Hierarchical deep reinforcement learning: Integrating temporal abstraction and intrinsic motivation in Advances in neural information processing systems (2016), 3675-3683.
- [47]47. Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T., Harley, T., Silver, D. & Kavukcuoglu, K. Asynchronous Methods for Deep Reinforcement Learning in Proceedings of The 33rd International Conference on Machine Learning (eds Balcan, M. F. & Weinberger, K. Q.) 48 (PMLR, New York, New York, USA, June 2016), 1928-1937. < http://proceedings.mlr.press/ v48/mniha16.html >.
- [49]49. Fahlman, S. E. & Lebiere, C. in (ed Touretzky, D. S.) 524-532 (Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 1990). isbn : 1-55860-100-7. < http://dl.acm.org/ citation.cfm?id=109230.107380 >.
- [53]53. Rusu, A. A., Rabinowitz, N. C., Desjardins, G., Soyer, H., Kirkpatrick, J., Kavukcuoglu, K., Pascanu, R. & Hadsell, R. Progressive Neural Networks. arXiv preprint arXiv:1606.04671 (2016). · arXiv
- [54]54. Hinton, G., Vinyals, O. & Dean, J. Distilling the Knowledge in a Neural Network in NIPS Deep Learning and Representation Learning Workshop (2015). < http://arxiv.org/abs/ 1503.02531 >.
- [59]59. Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J. & Zaremba, W. OpenAI Gym. CoRR abs/1606.01540. arXiv: 1606.01540 . < http://arxiv.org/ abs/1606.01540 > (2016). · arXiv
Cited by
Sorted by citations
Items 1,001–1,023 of 1,023
Vergel, J. P. G. (2022) · Repositorio Institucional (Pontificia Universidad Javeriana) · doi:10.11144/javeriana.10554.61561
Liu, T., Chen, Y., & Gu, W. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i5.25794
Efficient Learning of High Level Plans from Play
Urpí, N. A., Bagatella, M., Hilliges, O., Martius, G., & Coros, S. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10161415
Optimized path planning surpasses human efficiency in cryo-EM imagingPreprint
Li, Y., Fan, Q., Xu, Z., Lee, E. R., Cohn, J., Demers, V., Lee, J. Y., Yip, L., Cianfrocco, M. A., & Vos, S. M. (2023) · doi:10.7554/elife.88640 · Cited by 1
A Q-Learning Based Approach For Voltage Control In Unbalanced Distribution System
Çakır, M. T., Atak, A., Özeren, H., & Ceylan, O. (2023) · International Conference on Modern Power Systems (MPS) · doi:10.1109/mps58874.2023.10187567
Mean-Semivariance Policy Optimization via Risk-Averse Reinforcement Learning (Extended Abstract)
Ma, X., Ma, S., Li, X., & Zhao, Q. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/784
Standing Still Is Not an Option: Alternative Baselines for Attainable Utility Preservation
Eresheim, S., Kovac, F., & Adrowitzer, A. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-40837-3_15
Deep Reinforcement Learning based Intelligent Traffic Control
Nookala, A., Asodekar, E., Solanki, A., Bhagat, N., & Karia, D. C. (2023) · IEEE Region 10 Symposium (TENSYMP) · doi:10.1109/tensymp55890.2023.10223639
Liu, Z., & Fang, Y. (2023) · Chinese Control Conference (CCC) · doi:10.23919/ccc58697.2023.10240132
Adversarial Multi-Agent Reinforcement Learning for Fault-Tolerant Design of Complex Systems
Grimstad, J., & Morozov, A. S. (2023) · European Safety and Reliability Conference (ESREL) · doi:10.3850/978-981-18-8071-1_p375-cd
Causal inference and forescasting methods for climate data nalysis
Silini, R. (2022) · UPCommons institutional repository (Universitat Politècnica de Catalunya) · doi:10.5821/dissertation-2117-370852
Joint Recurrent Actor-Critic Model for Partially Observable Control
Xu, H., Yang, J. J., & Liang, B. (2023) · International Conference on Automation and Computing (ICAC) · doi:10.1109/icac57885.2023.10275190
How AI evolved with game and implementation of modern AI in game
Shen, B. (2023) · Applied and Computational Engineering · doi:10.54254/2755-2721/15/20230829
Reinforcement Learning with Partial Parametric Model Knowledge
Wang, S., Loewen, P. D., Lawrence, N. P., Forbes, M. G., & Gopaluni, R. B. (2023) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2023.10.924
Mastering Bidding in Fight the Landlord with Perfect Information Distillation
Zhang, X., Wang, X., Yang, X., Yan, M., & He, P. (2023) · Chinese Control and Decision Conference (CCDC) · doi:10.1109/ccdc58219.2023.10327476
Naruto Mobile: AI Sparring Partner Using Heterogeneous Deep Reinforcement Learning
Liu, E. S., Li, W., Zhou, Y., Cao, H., & Zeng, Z. (2023) · IEEE Conference on Games (CoG) · doi:10.1109/cog57401.2023.10333204
A Visual Analytics Approach to Understanding Proximal Policy Optimization
Zhang, J., Lan, H., Gao, B., Rao, Y., & Pu, J. (2023) · International Conference on Pattern Recognition and Artificial Intelligence (PRAI) · doi:10.1109/prai59366.2023.10332079
UBCL: A Reinforcement Learning Framework for Controllable and Diverse Player Behaviors
Cilan, A., & Özgövde, A. (2025) · arXiv
Guaranteed Discovery of Control-Endogenous Latent States with Multi-Step Inverse Models
Lamb, A., Islam, R., Efroni, Y., Didolkar, A., Misra, D., Foster, D., Molu, L., Chari, R., Krishnamurthy, A., & Langford, J. (2022) · arXiv · doi:10.48550/arxiv.2207.08229 · Cited by 1
Generative Language Modeling for Automated Theorem Proving
Polu, S., & Sutskever, I. (2020) · arXiv · doi:10.48550/arxiv.2009.03393 · Cited by 82
Self-citationUsing Unity to Help Solve Intelligence
Ward, T., Bolt, A., Hemmings, N., Carter, S., Sanchez, M., Barreira, R., Noury, S., Anderson, K., Lemmon, J., Coe, J., et al. (2020) · arXiv · doi:10.48550/arxiv.2011.09294 · Cited by 6
Metadata
- Authors
- OpenAI, :, Berner, C., Brockman, G., Chan, B., Cheung, V., Dębiak, P., Dennison, C., Farhi, D., Fischer, Q., Hashme, S., Hesse, C., Józefowicz, R., Gray, S., Olsson, C., Pachocki, J., Petrov, M., Pinto, H. P. D. O., Raiman, J., Salimans, T., Schlatter, J., Schneider, J., Sidor, S., Sutskever, I., Tang, J., Wolski, F., & Zhang, S.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2019
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1912.06680
- OpenAlex
- W2996037775
- Cited by
- 1,065
- References
- 37
- Citations by year
- 2026: 6 · 2025: 66 · 2024: 178 · 2023: 186 · 2022: 197 · 2021: 273 · 2020: 152 · 2019: 7
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
- Keywords
- Dota 2, self-play, distributed training, continuous training, imperfect information games, long time horizons, continuous state and action spaces, esports
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
