A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play
Open accessThe DOI matches its Crossref record; 30 of the 30 references it cites were found in a registry or an index; 5 of the 16 sources its claims cite were checked for support.
Abstract
One program to rule them all: Computers can beat humans at increasingly complex games, including chess and Go. However, these programs are typically constructed for a particular game, exploiting its properties, such as the symmetries of the board on which it is played. Silver et al. developed a program called AlphaZero, which taught itself to play Go, chess, and shogi (a Japanese version of chess) (see the Editorial, and the Perspective by Campbell). AlphaZero managed to beat state-of-the-art programs specializing in these three games. The ability of AlphaZero to adapt to various game rules is a notable step toward achieving a general game-playing system. Science , this issue p. 1140 ; see also pp. 1087 and 1118
Overview
CoChat Summary
TL;DR
AlphaZero is a single reinforcement-learning and MCTS algorithm that, given only game rules, learned superhuman play in chess, shogi, and Go through self-play. It defeated world-champion programs Stockfish, Elmo, and AlphaGo Zero using the same algorithm and network architecture across all three games.
Key findings
- In chess, AlphaZero defeated Stockfish 8, winning 155 games and losing 6 out of 1,000; it also won matches from common human openings and 2016 TCEC opening positions.
- In shogi, AlphaZero defeated Elmo, winning 98.2% of games as black and 91.2% overall; in Go, it defeated AlphaGo Zero, winning 61% of games.
- AlphaZero first outperformed Stockfish after 4 hours (300,000 steps), Elmo after 2 hours (110,000 steps), and AlphaGo Lee after 30 hours (74,000 steps); training ran for 700,000 steps, taking about 9 hours in chess, 12 hours in shogi, and 13 days in Go.
- AlphaZero searched only about 60,000 positions per second in chess and shogi, versus 60 million for Stockfish and 25 million for Elmo, yet still defeated Stockfish with 1/10 thinking time and won 46% against Elmo with 1/100 thinking time.
Methods
- AlphaZero uses a deep neural network \(f_\theta(s)\) that outputs move probabilities \(p\) and a scalar value \(v\), trained by reinforcement learning from self-play starting from random parameters; MCTS uses these outputs to guide search, with 800 simulations per move during training.
- The same algorithm, network architecture, and hyperparameters were used for chess, shogi, and Go, with no domain knowledge except game rules; no opening book, endgame tablebases, or domain-specific heuristics were used.
- Training used 5,000 first-generation TPUs to generate self-play games and 16 second-generation TPUs to train networks; evaluation matches used 3 hours per game plus 15 seconds per move, with opponents on 44 CPU cores and AlphaZero on four first-generation TPUs and 44 CPU cores.
Limitations
- Stated: AlphaZero did not use opening books, endgame tablebases, or domain-specific heuristics; the authors note that some such techniques could further improve performance but were left for future research.
- Stated: hyperparameters were reused across games without game-specific tuning, except for exploration noise and the learning rate schedule; matches against an opening book were low in diversity, and forced randomization increased losses.
- Evident from the text: results cover only three games with separate trained instances, no cross-game transfer is demonstrated, and evaluation compares AlphaZero on TPU hardware against CPU-based opponents, so hardware and time-control differences qualify direct strength comparisons.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
3 Valid · 2 Not valid · 8 Not checked · 43 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“Apart from the representation of positions and actions described above, AlphaZero uses the same network architecture as AlphaGo Zero ( 9 ), briefly recapitulated here.”
What the sources say
- Supportedcites [9] source
The Neural Network Architecture Comparison (p. 29) identifies AlphaGo Zero’s architecture as a residual tower with policy and value heads. The citation supplies the AlphaGo Zero architecture that the citing authors say AlphaZero uses; the comparison to AlphaZero is their own statement. (LLM, conf=0.88)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“and in Go, AlphaZero first outperformed AlphaGo Lee ( 9 ) after 30 hours (74,000 steps).”
What the sources say
- Partly supportedcites [9] sourceExperimental — accuracy not yet measured
Section 2, p. 7 reports that AlphaGo Zero outperformed AlphaGo Lee, but says this happened after 36 hours, not 30. The passage does not give 74,000 steps, and it names the system AlphaGo Zero rather than AlphaZero. (LLM, conf=0.90)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The training algorithm achieved similar performance in all independent runs (see fig. S3), suggesting that the high performance of AlphaZero's training algorithm is repeatable.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“AlphaZero also defeated Stockfish when given 1 / 10 as much thinking time as its opponent (i.e. searching ~ 1 / 10 , 000 as many positions),”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“and won 46% of games against Elmo when given 1 / 100 as much time (i.e. searching ~ 1 / 40 , 000 as many positions), see Fig. 2.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
23 more claims
Create a free account to see all 56 claims and to verify the 3 references still open.
- Probably not a substantial claim. 10
- It may state two separate claims as one. 23
- The quote only partly states this claim. 1
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Games have historically served as highly effective metrics for evaluating progress in deep reinforcement learning research”
From BALROG: Benchmarking Agentic LLM and VLM Reasoning On Games (2024) · cites this paper as Silver et al., 2018
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In 2017, AlphaGo, using the same learning algorithms in TD-Gammon and a billion times more computing power, soundly beat Ke Jie, the world Go champion, in a game that is to chess what chess is to backgammon in complexity (Silver et al., 2018).”
From Large Language Models and the Reverse Turing Test (2023) · cites this paper as Silver et al., 2018
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Most of the attention was on who won, but more important was how it won: AlphaGo was creative and had discovered new moves and board positions that were outside human experience.”
From Large Language Models and the Reverse Turing Test (2023) · cites this paper as Silver et al., 2018
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The primary method for applying more computation at inference, common in the AlphaZero family of agents2,72, is scaling the tree search and running multiple independent search attempts.”
From Olympiad-level formal mathematical reasoning with reinforcement learning (2026) · cites this paper as [2]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Modern AI can write computer programs (Li et al., 2022) or play chess at super-human level (Silver et al., 2018), but the ability of AI to perceive and act in the world remains far below human level.”
From Scaling Instructable Agents Across Many Simulated Worlds (2024) · cites this paper as Silver et al., 2018
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Self-citationAdvances in neural information processing systems 7
(1996) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(96)90463-0 · Cited by 14,281
Identity mappings in deep residual networks
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_38 · Cited by 10,741
Identity Mappings in Deep Residual Networks
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · arXiv · doi:10.48550/arxiv.1603.05027 · Cited by 10,665
Mastering the game of Go without human knowledge
Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., et al. (2017) · Nature · doi:10.1038/nature24270 · Cited by 9,084
Self-citationIn-Datacenter Performance Analysis of a Tensor Processing Unit
Jouppi, N. P., Young, C., Patil, N., Patterson, D., Agrawal, G., Bajwa, R., Bates, S., Bhatia, S., Boden, N., Borchers, A., et al. (2017) · Proceedings of the 44th Annual International Symposium on Computer Architecture · doi:10.1145/3079856.3080246 · Cited by 4,537
Encyclopedia of Artificial Intelligence
(2009) · IGI Global eBooks · doi:10.4018/978-1-59904-849-9 · Cited by 1,396
Campbell, M., Hoane, A. J., & Hsu, F.-H. (2002) · Artificial Intelligence · doi:10.1016/s0004-3702(01)00129-1 · Cited by 1,226
Lim, T.-S., Loh, W.-Y., & Shih, Y.-S. (2000) · Machine Learning · doi:10.1023/a:1007608224229 · Cited by 1,133
An analysis of alpha-beta pruning
Knuth, D. E., & Moore, R. W. (1975) · Artificial Intelligence · doi:10.1016/0004-3702(75)90019-3 · Cited by 1,004
Research works printed but not linked: 7
- C. J. Maddison A. Huang I. Sutskever D. Silver paper presented at the International Conference on Learning Representations 2015 San Diego CA 7 to 9 May 2015.
- D. N. L. Levy M. Newborn How Computers Play Chess (Ishi Press 2009).
- W. Steinitz The Modern Chess Instructor (Edition Olms 1990).
- O. Arenz “Monte Carlo chess ” master’s thesis Technische Universität Darmstadt (2012).
- 10.1023/a:1007634325138 · DOI
- T. Kaneko K. Hoki in Advances in Computer Games – 13th International Conference ACG 2011 Revised Selected Papers Tilburg Netherlands 20 to 22 November 2011 (Springer 2012) pp. 158–169.
- R. Ramanujan A. Sabharwal B. Selman in Proceedings of the 26th Conference on Uncertainty in Artificial Intelligence (UAI 2010) Catalina Island CA 8 to 11 July (AUAI Press 2010).
Cited by
Items 201–300 of 3,632
Scarfe, P., Watcham, K., Clarke, A., & Roesch, E. (2024) · PLOS ONE · doi:10.1371/journal.pone.0305354 · Cited by 108
Catch & Carry: reusable neural controllers for vision-guided whole-body tasks
Merel, J., Tunyasuvunakool, S., Ahuja, A., Tassa, Y., Hasenclever, L., Pham, V., Erez, T., Wayne, G., & Heess, N. (2020) · ACM Transactions on Graphics · doi:10.1145/3386569.3392474 · Cited by 168
“…developments in Deep Reinforcement Learning (Deep RL) have enabled great strides in learning from scratch for many game domains [Mnih et al. 2015; Silver et al. 2018]; and while complex motor control problems involving physical bodies remain difficult even in simulation, there have been some…”
BackgroundProtecting consumers from collusive prices due to AI
Calvano, E., Calzolari, G., Denicolò, V., Harrington, J. E., & Pastorello, S. (2020) · Science · doi:10.1126/science.abe3796 · Cited by 107
FFCNN: A Deep Neural Network for Surface Defect Detection of Magnetic Tile
Xie, L., Xiang, X., Xu, H., Wang, L., Lin, L., & Yin, G. (2020) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2020.2982115 · Cited by 111
“Inrecentyears,deeplearning (DL)[10]–[15]techniques,abranchofartificialintelligence, haveachievedimpressiveresults,suchasAlphaZero[16].”
Building Energy Management With Reinforcement Learning and Model Predictive Control: A Survey
Zhang, H., Seal, S., Wu, D., Bouffard, F., & Boulet, B. (2022) · IEEE Access · doi:10.1109/access.2022.3156581 · Cited by 117
On the Binding Problem in Artificial Neural Networks
Greff, K., van Steenkiste, S., & Schmidhuber, J. (2020) · arXiv · doi:10.48550/arxiv.2012.05208 · Cited by 102
Deep Learning for Voltammetric Sensing in a Living Animal Brain
Xue, Y., Ji, W., Jiang, Y. Y., Yu, P., & Mao, L. (2021) · Angewandte Chemie International Edition · doi:10.1002/anie.202109170 · Cited by 107
Suh, C., Fare, C., Warren, J. A., & Pyzer‐Knapp, E. O. (2020) · Annual Review of Materials Research · doi:10.1146/annurev-matsci-082019-105100 · Cited by 113
“Reinforcement learning has shown great potential in learning to play games such as the Atari suite and Go (107), as well as in simulation challenges such as protein folding (108).”
BackgroundMultitask deep-learning-based design of chiral plasmonic metamaterials
Ashalley, E., Acheampong, K. N., Besteiro, L. V., Yu, P., Neogi, A., Govorov, A. O., & Wang, Z. M. (2020) · Photonics Research · doi:10.1364/prj.388253 · Cited by 101
Aligning Superhuman AI with Human Behavior: Chess as a Model System
McIlroy-Young, R., Sen, S., Kleinberg, J., & Anderson, A. (2020) · Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3394486.3403219 · Cited by 99
“revolutionized algorithmic game-playing with the introduction of a sequence of deep reinforcement learning frameworks culminating in AlphaZero, an algorithm that achieved superhuman performance in chess, shogi, and Go by learning from self-play [14, 15].”
BackgroundMethodDreher, A., Bexten, T., Sieker, T., Lehna, M., Schütt, J., Scholz, C., & Wirsum, M. (2022) · Energy Conversion and Management · doi:10.1016/j.enconman.2022.115401 · Cited by 107
Deep learning in retrosynthesis planning: datasets, models and tools
Dong, J., Zhao, M., Liu, Y., Su, Y., & Zeng, X. (2021) · Briefings in Bioinformatics · doi:10.1093/bib/bbab391 · Cited by 113
A biological perspective on evolutionary computation
Miikkulainen, R. P., & Forrest, S. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-020-00278-8 · Cited by 129
Galić, I., Habijan, M., Leventić, H., & Romić, K. (2023) · Electronics · doi:10.3390/electronics12214411 · Cited by 150
A robot made of robots: Emergent transport and control of a smarticle ensemble
Savoie, W., Berrueta, T. A., Jackson, Z., Pervan, A., Warkentin, R., Li, S., Murphey, T. D., Wiesenfeld, K. A., & Goldman, D. I. (2019) · Science Robotics · doi:10.1126/scirobotics.aax4316 · Cited by 112
Attentive Weights Generation for Few Shot Learning via Information Maximization
Guo, Y., & Cheung, N. M. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01351 · Cited by 122
“While deep learning methods achieve great success in computer vision [14], natural language processing [9], reinforcement learning [38], their hunger for large amount of labeled data limits the application scenarios where only few data are available for training.”
BackgroundReinforcement Learning for Precision Oncology
Eckardt, J., Wendt, K., Bornhäuser, M., & Middeke, J. M. (2021) · Cancers · doi:10.3390/cancers13184624 · Cited by 113
“…and often achieving human-level or superhuman-level performance in, for example, gameplay of complex board games such as chess, Shogi and Go [13–15], video games [16–19] and the field of autonomous driving [20]. based on individual intrinsic factors such as patient and disease biology that may…”
BackgroundLiang, Y., Ding, Z., Zhao, T., & Lee, W. (2022) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2022.3186931 · Cited by 106
“…charge rate between charging bays and the difference in SOC of depleted batteries when scheduling the BSCS. Recent years witnessed tremendous success in deep reinforcement learning (DRL) in modeling computational challenging decision-making problems such as Atari [14], Go [15], and StarCraft [16].”
BackgroundControlling Rayleigh–Bénard convection via reinforcement learning
Beintema, G., Corbetta, A., Biferale, L., & Toschi, F. (2020) · Journal of Turbulence · doi:10.1080/14685248.2020.1797059 · Cited by 118
“cent years, Reinforcement Learning (RL) algorithms [14] have been proven capable of solving complex control problems, dominating extremely hard challenges in high-dimensional spaces (e.g. board games [15,16] and robotics [17]). Reinforcement Learning is a supervised Machine Learning (ML) [18] approach 2 (a) Linear Con…”
BackgroundHemmer, P., Westphal, M., Schemmer, M., Vetter, S., Vössing, M., & Satzger, G. (2023) · Proceedings of the 28th International Conference on Intelligent User Interfaces · doi:10.1145/3581641.3584052 · Cited by 93
“Nowadays, the performance of AI models is similar to, and in certain application areas even exceeds the performance of human experts [12, 29, 64].”
BackgroundTwo-Dimensional Materials for Brain-Inspired Computing Hardware
Hadke, S. S., Kang, M., Sangwan, V. K., & Hersam, M. C. (2025) · Chemical Reviews · doi:10.1021/acs.chemrev.4c00631 · Cited by 112
Hilton, C. B., Milinovich, A., Felix, C., Vakharia, N., Crone, T., Donovan, C., Proctor, A. W., & Nazha, A. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0249-z · Cited by 108
Razavi, S., Hannah, D. M., ELSHORBAGY, A. A., Kumar, S. V., Marshall, L. A., SOLOMATINE, D. P., Dezfuli, A., Sadegh, M., & Famiglietti, J. S. (2022) · Hydrological Processes · doi:10.1002/hyp.14596 · Cited by 101
“Machine learning (ML), particularly deep learning (DL), has achieved revolutionary performance in areas such as computer vision (Krizhevsky et al., 2012), natural language processing (Young et al., 2018), and gaming (Silver et al., 2018).”
BackgroundOpen Problems in Cooperative AI
Dafoe, A., Hughes, E., Bachrach, Y., Collins, T., McKee, K. R., Leibo, J. Z., Larson, K., & Graepel, T. (2020) · arXiv · doi:10.48550/arxiv.2012.08630 · Cited by 93
Self-citationThe fundamental principles of reproducibility
Gundersen, O. E. (2021) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2020.0210 · Cited by 90
Online Minimax Q Network Learning for Two-Player Zero-Sum Markov Games
Zhu, Y., & Zhao, D. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3041469 · Cited by 108
Artificial intelligence in food science and nutrition: A narrative review
Miyazawa, T., Hiratsuka, Y., Toda, M., Hatakeyama, N., Ozawa, H., Abe, C., Cheng, T.-Y., Matsushima, Y., Miyawaki, Y., Ashida, K., et al. (2022) · Nutrition Reviews · doi:10.1093/nutrit/nuac033 · Cited by 91
A Deep Reinforcement Learning Approach for Global Routing
Liao, H., Zhang, W., Dong, X., Póczos, B., Shimada, K., & Kara, L. B. (2019) · Journal of Mechanical Design · doi:10.1115/1.4045044 · Cited by 113
“DRL combines reinforcement learning and deep learning, and has been successfully utilized in a variety of sequential decision making problems [16,17,18,19,20,21,16].”
MethodQuantum machine learning for particle physics using a variational quantum classifier
Blance, A., & Spannowsky, M. (2021) · Journal of High Energy Physics · doi:10.1007/jhep02(2021)212 · Cited by 129
A reinforcement learning approach for process parameter optimization in additive manufacturing
Dharmadhikari, S., Menon, N., & Basak, A. (2023) · Additive manufacturing · doi:10.1016/j.addma.2023.103556 · Cited by 142
A Meta-Analysis of the Utility of Explainable Artificial Intelligence in Human-AI Decision-Making
Schemmer, M., Hemmer, P., Nitsche, M., Kühl, N., & Vössing, M. (2022) · Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3514094.3534128 · Cited by 91
“In this context, AI’s continuously rising capabilities have surpassed human performance in an increasing number of tasks, such as playing poker [7], go [56], or correctly recognizing various categories of interest in images [24].”
BackgroundLi, J., Wu, X., Xu, M., & Liu, Y. (2022) · Energy · doi:10.1016/j.energy.2022.123924 · Cited by 97
Navigating through the Maze of Homogeneous Catalyst Design with Machine Learning
Gomes, G., Pollice, R., & Aspuru‐Guzik, A. (2021) · Trends in Chemistry · doi:10.1016/j.trechm.2020.12.006 · Cited by 94
Developments in Image Processing Using Deep Learning and Reinforcement Learning
Valente, J., António, J., Mora, C., & Jardim, S. (2023) · Journal of Imaging · doi:10.3390/jimaging9100207 · Cited by 124
Measuring the Algorithmic Efficiency of Neural Networks
Hernandez, D., & Brown, T. B. (2020) · arXiv · doi:10.48550/arxiv.2005.04305 · Cited by 89
“It seems to us as if language modeling [Devlin et al., 2018, Radford et al., , Raffel et al., 2019] and playing games [Silver et al., 2016, Silver et al., 2017, Silver et al., 2018,OpenAI et al., 2019] are the domains of interest given our criteria.”
BackgroundDenoising arterial spin labeling perfusion MRI with deep machine learning
Xie, D., Li, Y., Yang, H., Li, B., Wang, T., Zhou, F., Zhang, L., & Wang, Z. (2020) · Magnetic Resonance Imaging · doi:10.1016/j.mri.2020.01.005 · Cited by 94
A Survey of Sim-to-Real Transfer Techniques Applied to Reinforcement Learning for Bioinspired Robots
Zhu, W. D., Guo, X., Owaki, D., Kutsuzawa, K., & Hayashibe, M. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3112718 · Cited by 112
“Benefiting from the enhanced computational power and the introduction of deep neural networks (DNNs) in RL [1], RL has been successfully applied in a wide variety of areas for learning better policies and improving autonomy, including chess and card [2], video games [3], and robot control [4].”
BackgroundContinuous improvement of self-driving cars using dynamic confidence-aware reinforcement learning
Cao, Z. L., Jiang, K., Zhou, W., Xu, S., Peng, H., & Yang, D. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00610-y · Cited by 119
“(2) The dynamic confidence-value update with new training data (shown in Fig.”
BackgroundMethodInfluentialAdversarial Policies: Attacking Deep Reinforcement Learning
Gleave, A., Dennis, M., Wild, C., Kant, N., Levine, S., & Russell, S. (2019) · arXiv · doi:10.48550/arxiv.1905.10615 · Cited by 200
“Although it is known that self-play with deep RL may not converge, or converge only to a local rather than global Nash, self-play has been used with great success in a number of works focused on playing adversarial games directly against humans (Silver et al., 2018; OpenAI, 2018).”
BackgroundOpen-ended Learning in Symmetric Zero-sum Games
Balduzzi, D., Garnelo, M., Bachrach, Y., Czarnecki, W. M., Perolat, J., Jaderberg, M., & Graepel, T. (2019) · arXiv · doi:10.48550/arxiv.1901.08106 · Cited by 88
“There are very successful examples of algorithms that pose and solve a series of increasingly difficult problems for themselves through forms of self-play (Silver et al., 2018; Jaderberg et al., 2018; Bansal et al., 2018; Tesauro, 1995).”
BackgroundSelf-citationDeep-learning continuous gravitational waves
Dreißigacker, C., Sharma, R., Messenger, C., Zhao, R., & Prix, R. (2019) · Physical review. D/Physical review. D. · doi:10.1103/physrevd.100.044009 · Cited by 94
“The field of DNNs, also referred to as Deep Learning, has emerged as one of the most successful machine-learning paradigms in the last decade, dominating wide-ranging fields [22] such as image recognition, speech recognition and language translation, as well as certain board [23] and video games [24, 25].”
BackgroundMethods for Small Area Population Forecasts: State-of-the-Art and Research Needs
Wilson, T., Grossman, I., Alexander, M. J., Rees, P., & Temple, J. (2021) · Population Research and Policy Review · doi:10.1007/s11113-021-09671-6 · Cited by 120
Exploring the role of large language models in the scientific method: from hypothesis to discovery
Zhang, Y., Khan, S. A., Mahmud, A., Yang, H., Lavin, A., Levin, M. G., Frey, J. G., Dunnmon, J., Evans, J. A., Bundy, A., et al. (2025) · npj Artificial Intelligence · doi:10.1038/s44387-025-00019-5 · Cited by 242
“However, a signi fi cant gap in general reasoning capabilities separates current models from domain-speci fi c superhuman systems like AlphaGo/AlphaZero 197,198 .”
Conceptual challenges for interpretable machine learning
Watson, D. S. (2022) · Synthese · doi:10.1007/s11229-022-03485-5 · Cited by 87
“ML models outperform humans not just at strategy games like chess (Silver et al., 2018) and Star-Craft (Vinyals et al., 2019), but at important scientific tasks like antibiotic discovery (Stokes et al., 2020) and predicting protein structure (Jumper et al., 2021).”
MethodZhu, F., Yang, Z., Lin, F., & Xin, Y. (2020) · IEEE Transactions on Power Electronics · doi:10.1109/tpel.2020.2971637 · Cited by 87
Computer-inspired quantum experiments
Krenn, M., Erhard, M., & Zeilinger, A. (2020) · Nature Reviews Physics · doi:10.1038/s42254-020-0230-4 · Cited by 86
Steimers, A., & Schneider, M. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph19063641 · Cited by 150
“In the field of artificial intelligence, the success of probabilistic and statistical approaches to machine learning in recent years has been undeniable [56,57], and interest in this area continues to be unabated [58].”
BackgroundHuang, Z., Sheng, Z., Ma, C., & Chen, S. (2024) · Communications in Transportation Research · doi:10.1016/j.commtr.2024.100127 · Cited by 88
Global optimization of quantum dynamics with AlphaZero deep exploration
Dalgaard, M., Motzoi, F., Sørensen, J. J., & Sherson, J. (2020) · npj Quantum Information · doi:10.1038/s41534-019-0241-0 · Cited by 85
“1 b and c [46, 47].”
BackgroundDemystifying artificial intelligence in pharmacy
Nelson, S. D., Walsh, C. G., Olsen, C. A., McLaughlin, A. J., LeGrand, J. R., Schutz, N., & Lasko, T. A. (2020) · American Journal of Health-System Pharmacy · doi:10.1093/ajhp/zxaa218 · Cited by 86
The Best Laid Plans: Computational Principles of Anterior Cingulate Cortex
Holroyd, C. B., & Verguts, T. (2021) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2021.01.008 · Cited by 94
Prediction of protein–protein interactions based on elastic net and deep forest
Yu, B., Chen, C., Wang, X., Yu, Z., Ma, A., & Liu, B. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.114876 · Cited by 88
Garvey, C., & Maskal, C. (2020) · OMICS: A Journal of Integrative Biology · doi:10.1089/omi.2019.0078 · Cited by 84
“…2015), and games like Go and Science and Technology Studies Department, Rensselaer Polytechnic Institute, Troy, New York. poker (Brown and Sandholm, 2018; Moravcˇı´k et al., 2017; Silver et al., 2016, 2017, 2018) owe as much to recent hardware innovations as clever programming (Stone et al., 2016).”
BackgroundMLatom: A program package for quantum chemical research assisted by machine learning
Dral, P. O. (2019) · Journal of Computational Chemistry · doi:10.1002/jcc.26004 · Cited by 89
Machine learning based on reservoir computing with time-delayed optoelectronic and photonic systems
Chembo, Y. K. (2020) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/1.5120788 · Cited by 93
Abdel‐Latif, K., Bateni, F., Crouse, S. H., & Abolhasani, M. (2020) · Matter · doi:10.1016/j.matt.2020.07.024 · Cited by 95
Explainable reinforcement learning in production control of job shop manufacturing system
Kuhnle, A., May, M. C., Schäfer, L., & Lanza, G. (2021) · International Journal of Production Research · doi:10.1080/00207543.2021.1972179 · Cited by 101
“Ranging from challenging sequential, zero-sum, and perfect information board games such as Go (Silver et al. 2018) to symmetric, simultaneous and imperfect information computer games such as StarCraft (Vinyals et al. 2019) and up to multi-agent, simultaneous, cooperative and infinite air traffic…”
BackgroundAutomated discovery of reprogrammable nonlinear dynamic metamaterials
Bordiga, G., Medina, E., Jafarzadeh, S., Bösch, C., Adams, R. P., Tournat, V., & Bertoldi, K. (2024) · Nature Materials · doi:10.1038/s41563-024-02008-6 · Cited by 109
“At the same time, the combination of large parameter spaces and gradient-based optimization has been shown to be a tremendous success in discovering underlying patterns and behaviors (26–28).”
Adaptive course recommendation in MOOCs
Lin, Y., Feng, S., Lin, F., Zeng, W., Liu, Y., & Wu, P. (2021) · Knowledge-Based Systems · doi:10.1016/j.knosys.2021.107085 · Cited by 97
Ssengonzi, C., Kogeda, O. P., & Olwal, T. O. (2022) · Array · doi:10.1016/j.array.2022.100142 · Cited by 102
AI-Assisted Learning with ChatGPT and Large Language Models: Implications for Higher Education
Laato, S., Morschheuser, B., Hamari, J., & Björne, J. (2023) · 2023 IEEE International Conference on Advanced Learning Technologies (ICALT) · doi:10.1109/icalt58122.2023.00072 · Cited by 82
Suphx: Mastering Mahjong with Deep Reinforcement LearningPreprint
Li, J., Koyamada, S., Ye, Q., Liu, G., Wang, C., Yang, R., Li, Z., Qin, T., Liu, T., & Hon, H.-W. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.13590 · Cited by 161
“Game AI has made great progress in past two decades ( 2,3, 11,13,15,16,18 ).”
BackgroundArtificial Intelligence and Deep Learning for Rheumatologists
McMaster, C., Bird, A., Liew, D. F. L., Buchanan, R. R. C., Owen, C., Chapman, W. W., & Pires, D. E. V. (2022) · Arthritis & Rheumatology · doi:10.1002/art.42296 · Cited by 98
Physics-enhanced neural networks learn order and chaos
Choudhary, A., Lindner, J. F., Holliday, E. G., Miller, S. T., Sinha, S., & Ditto, W. L. (2020) · Physical review. E · doi:10.1103/physreve.101.062207 · Cited by 102
Digital Twin: Values, Challenges and Enablers
Rasheed, A., San, O., & Kvamsdal, T. (2019) · arXiv · doi:10.48550/arxiv.1910.01719 · Cited by 81
Artificial intelligence facilitates drug design in the big data era
Wang, L., Ding, J., Pan, L., Cao, D., Jiang, H., & Ding, X. (2019) · Chemometrics and Intelligent Laboratory Systems · doi:10.1016/j.chemolab.2019.103850 · Cited by 80
Boon, I. S., Yong, T. P. T. A., & Boon, C. S. (2018) · Medicines · doi:10.3390/medicines5040131 · Cited by 94
Learning Optimal Controllers for Linear Systems With Multiplicative Noise via Policy Gradient
Gravell, B., Esfahani, P. M., & Summers, T. H. (2020) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2020.3037046 · Cited by 146
“R EINFORCEMENT learning-based control has recently achieved impressive successes in games [1] and simulators [2].”
BackgroundEveritt, T., Hutter, M., Kumar, R., & Krakovna, V. (2021) · Synthese · doi:10.1007/s11229-021-03141-4 · Cited by 80
A Steering Algorithm for Redirected Walking Using Reinforcement Learning
Strauss, R. R., Ramanujan, R., Becker, A., & Peck, T. C. (2020) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2020.2973060 · Cited by 88
“They have notched up a series of significant successes in recent years in challenging domains where previous state-of-the-art methods required significant human input [7,21,25,31,35,46].”
BackgroundHuman-AI Collaboration in a Cooperative Game Setting: Measuring Social Perception and Outcomes
Ashktorab, Z., Liao, Q. V., Dugan, C., Johnson, J., Pan, Q., Zhang, W., Kumaravel, S., & Campbell, M. (2020) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3415167 · Cited by 79
“Games are both an important application domain of AI and frequently used as a test bed for state-of-the-art AI algorithms [13, 23, 53, 55], leading to current game-playing AI reaching human-comparable 96:4 Zahra Ashktorab et al. performance.”
BackgroundTelkamp, J. B., & Anderson, M. H. (2022) · Journal of Business Ethics · doi:10.1007/s10551-022-05057-6 · Cited by 101
“…2017, DeepMind’s AlphaZero began with merely the rules of chess and developed a level of expertise far surpassing human capability in only four hours , enabling it to beat the best available computer programs (which were already vastly better than the best human players; Silver et al., 2017, 2018).”
MethodFast reinforcement learning with generalized policy updates
Barreto, A., Hou, S., Borsa, D., Silver, D., & Precup, D. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1907370117 · Cited by 78
Self-citationWang, Y., Jia, Y., Tian, Y., & Xiao, J. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.117013 · Cited by 92
S. V., S. S., Law, J. N., Tripp, C. E., Duplyakin, D., Skordilis, E., Biagioni, D., Paton, R. S., & St. John, P. C. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-022-00506-3 · Cited by 79
Deep Reinforcement Learning in Medicine
Jonsson, A. (2019) · Kidney Diseases · doi:10.1159/000492670 · Cited by 77
Targeted free energy estimation via learned mappings
Wirnsberger, P., Ballard, A. J., Papamakarios, G., Abercrombie, S., Racanière, S., Pritzel, A., Rezende, D. J., & Blundell, C. (2020) · The Journal of Chemical Physics · doi:10.1063/5.0018903 · Cited by 155
“Since the introduction of TFEP in 2002, research in ML has made remarkable progress in fields of image classification 17,18 , playing video 19 and board games 20,21 , and generative modelling of images 22,23 .”
MethodIs there a role for statistics in artificial intelligence?
Friedrich, S., Antes, G., Behr, S., Binder, H., Brannath, W., Dumpert, F., Ickstadt, K., Kestler, H. A., Lederer, J., Leitgöb, H., et al. (2021) · Advances in Data Analysis and Classification · doi:10.1007/s11634-021-00455-6 · Cited by 115
Signal Processing and Machine Learning Techniques for Terahertz Sensing: An overview
Helal, S., Sarieddeen, H., Dahrouj, H., Al-Naffouri, T. Y., & Alouini, M. (2022) · IEEE Signal Processing Magazine · doi:10.1109/msp.2022.3183808 · Cited by 87
Adaptive Informative Path Planning Using Deep Reinforcement Learning for UAV-based Active Sensing
Rückin, J., Jin, L., & Popović, M. (2022) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra46639.2022.9812025 · Cited by 90
Lin, S. (2022) · The Journal of the American Board of Family Medicine · doi:10.3122/jabfm.2022.01.210226 · Cited by 77
On reliability of reinforcement learning based production scheduling systems: a comparative survey
de Puiseau, C. W., Meyes, R., & Meisen, T. (2022) · Journal of Intelligent Manufacturing · doi:10.1007/s10845-022-01915-2 · Cited by 68
“Inspired by AlphaZero’s (Silver et al., 2018) success in board games, (Rinciog et al., 2020) applied an adapted algo-rithm to offline scheduling (obtained in a simulation) in a sheet metal production scenario to optimize tardiness and material waste.”
MethodGao, Y., Matsunami, Y., Miyata, S., & Akashi, Y. (2022) · Applied Energy · doi:10.1016/j.apenergy.2022.119783 · Cited by 85
Deep Problems with Neural Network Models of Human VisionPreprint
Bowers, J. S., Malhotra, G., Dujmović, M., et al. (2022) · doi:10.31234/osf.io/5zf4s · Cited by 77
From self-tuning regulators to reinforcement learning and back again
Matni, N., Proutière, A., Rantzer, A., & Tu, S. (2019) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc40024.2019.9029916 · Cited by 108
“ol. I. INTRODUCTION & MOTIVATION With their recent successes in image classification, video game playing [1], sophisticated robotic simulations [2], [3], and complex strategy games such as Go [4], [5], machine and reinforcement learning (RL) are now being applied to plan and control the behavior of autonomous syste…”
BackgroundMethodRehearsal revealed: The limits and merits of revisiting samples in continual learning
Verwimp, E., De Lange, M., & Tuytelaars, T. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00925 · Cited by 75
“Recent advances of neural networks have shown promising results by surpassing human capabilities in a wide range of tasks [39, 32, 38].”
BackgroundSpecial issue on machine learning and data-driven methods in fluid dynamics
Brunton, S. L., Hemati, M. S., & Taira, K. (2020) · Theoretical and Computational Fluid Dynamics · doi:10.1007/s00162-020-00542-y · Cited by 75
Zheng, L., Huang, S., Mu, N., Zhang, H., Zhang, J., Chang, Y., Yang, L., & Zuo, Y. (2019) · Database · doi:10.1093/database/baz131 · Cited by 79
“inference has incredible potential in computational proteomics (94).”
BackgroundMoving Deep Learning to the Edge
Véstias, M. P., Duarte, R. P., de Sousa, J. T., & Neto, H. C. (2020) · Algorithms · doi:10.3390/a13050125 · Cited by 73
“Recent examples are chess [87] and go [88].”
BackgroundLee, J., Cho, K. H., & Cho, K. (2023) · Advanced Materials · doi:10.1002/adma.202209673 · Cited by 88
Efficient and scalable reinforcement learning for large-scale network control
Ma, C., Li, A., Du, Y., Dong, H., & Yang, Y. (2024) · Nature Machine Intelligence · doi:10.1038/s42256-024-00879-7 · Cited by 188
Phatak, A. A., Wieland, F.-G., Vempala, K., Volkmar, F., & Memmert, D. (2021) · Sports Medicine - Open · doi:10.1186/s40798-021-00372-0 · Cited by 74
“This makes it ideal for training agents via reinforcement learning [91].”
BackgroundAI in operations management: applications, challenges and opportunities
Dogru, A. K., & Keskin, B. B. (2020) · Journal of Data Information and Management · doi:10.1007/s42488-020-00023-1 · Cited by 114
“The third milestone was AlphaZero by Deepmind; an AI chess engine that achieved a superhuman level of play after playing against itself for only 24 hours and then defeating the best brute force dynamic programming chess engines in 2017 (Silver et al. 2018).”
BackgroundThe explanation game: a formal framework for interpretable machine learning
Watson, D. S., & Floridi, L. (2021) · Synthese · doi:10.1007/s11229-020-02629-9 · Cited by 71
“games like chess and Go (Silver et al., 2018), diagnosing skin cancer (Esteva et al.”
Creating a large language model of a philosopher
Schwitzgebel, E., Schwitzgebel, D., & Strasser, A. (2024) · Mind & Language · doi:10.1111/mila.12466 · Cited by 73
Federated Reinforcement Learning for Training Control Policies on Multiple IoT Devices
Lim, H.-K., Kim, J.-B., Heo, J.-S., & Han, Y. (2020) · Sensors · doi:10.3390/s20051359 · Cited by 92
Xiang, X., & Foo, S. Y. (2021) · Machine Learning and Knowledge Extraction · doi:10.3390/make3030029 · Cited by 87
“[39] also came up with a general algorithm called AlphaZero.”
Methodα-Rank: Multi-Agent Evaluation by Evolution
Omidshafiei, S., Papadimitriou, C. H., Piliouras, G., Tuyls, K., Rowland, M., Lespiau, J.-B., Czarnecki, W. M., Lanctot, M., Pérolat, J., & Munos, R. (2019) · Scientific Reports · doi:10.1038/s41598-019-45619-9 · Cited by 156
Self-citationZhang, S., Zhang, J., Chen, L., & Liu, X. (2020) · Nonlinear Dynamics · doi:10.1007/s11071-019-05398-4 · Cited by 76
“Reinforcement learning (RL) as one of the most powerful machine learning approaches [37,38,39,48], which is rooted in psychology and neuroscience, has been widely used to solve the problems in terms of states, actions, rewards, and decision making in various environments through exploratory trials [37, 53,54].”
BackgroundMasked Contrastive Representation Learning for Reinforcement Learning
Zhu, J., Xia, Y., Wu, L., Deng, J., Zhou, W., Qin, T., Liu, T., & Li, H. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3176413 · Cited by 95
“DEEP reinforcement learning (DRL) has achieved great success in game AI [1], [2], robotics [3], [4], 3D image processing [5], [6], etc.”
Background
Metadata
- Authors
- Silver, D., Hubert, T., Schrittwieser, J., Antonoglou, I., Lai, M., Guez, A., Lanctot, M., Sifre, L., Kumaran, D., Graepel, T., Lillicrap, T., Simonyan, K., & Hassabis, D.
- Affiliations
- Google DeepMind (United Kingdom) (GB)
- Corresponding author
- Silver, David
- Hubert, Thomas
- Schrittwieser, Julian
- Hassabis, Demis
- ORCID
- Venue
- Science
- Published
- 2018-12-07
- Volume
- 362
- Issue
- 6419
- Pages
- 1140-1144
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- OpenAlex
- W2902907165
- Cited by
- 3,562
- References
- 30
- Citations by year
- 2026: 257 · 2025: 427 · 2024: 512 · 2023: 514 · 2022: 569 · 2021: 546 · 2020: 462 · 2019: 256 · 2018: 13
- FWCI
- 179.89
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Science (publishedVersion)
- PubMed (publishedVersion)
- UCL Discovery (University College London) (submittedVersion)
- Funders
- DeepMind
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
- Keywords
- AlphaZero, chess, shogi, Go, evaluation function learning, game tree search
- MeSH terms
- Algorithms, Artificial Intelligence, Humans, Reinforcement, Psychology, Software, Video Games
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.