A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play
Open accessThe DOI matches its Crossref record; 30 of the 30 references it cites were found in a registry or an index; 5 of the 16 sources its claims cite were checked for support.
Abstract
One program to rule them all: Computers can beat humans at increasingly complex games, including chess and Go. However, these programs are typically constructed for a particular game, exploiting its properties, such as the symmetries of the board on which it is played. Silver et al. developed a program called AlphaZero, which taught itself to play Go, chess, and shogi (a Japanese version of chess) (see the Editorial, and the Perspective by Campbell). AlphaZero managed to beat state-of-the-art programs specializing in these three games. The ability of AlphaZero to adapt to various game rules is a notable step toward achieving a general game-playing system. Science , this issue p. 1140 ; see also pp. 1087 and 1118
Overview
CoChat Summary
TL;DR
AlphaZero is a single reinforcement-learning and MCTS algorithm that, given only game rules, learned superhuman play in chess, shogi, and Go through self-play. It defeated world-champion programs Stockfish, Elmo, and AlphaGo Zero using the same algorithm and network architecture across all three games.
Key findings
- In chess, AlphaZero defeated Stockfish 8, winning 155 games and losing 6 out of 1,000; it also won matches from common human openings and 2016 TCEC opening positions.
- In shogi, AlphaZero defeated Elmo, winning 98.2% of games as black and 91.2% overall; in Go, it defeated AlphaGo Zero, winning 61% of games.
- AlphaZero first outperformed Stockfish after 4 hours (300,000 steps), Elmo after 2 hours (110,000 steps), and AlphaGo Lee after 30 hours (74,000 steps); training ran for 700,000 steps, taking about 9 hours in chess, 12 hours in shogi, and 13 days in Go.
- AlphaZero searched only about 60,000 positions per second in chess and shogi, versus 60 million for Stockfish and 25 million for Elmo, yet still defeated Stockfish with 1/10 thinking time and won 46% against Elmo with 1/100 thinking time.
Methods
- AlphaZero uses a deep neural network \(f_\theta(s)\) that outputs move probabilities \(p\) and a scalar value \(v\), trained by reinforcement learning from self-play starting from random parameters; MCTS uses these outputs to guide search, with 800 simulations per move during training.
- The same algorithm, network architecture, and hyperparameters were used for chess, shogi, and Go, with no domain knowledge except game rules; no opening book, endgame tablebases, or domain-specific heuristics were used.
- Training used 5,000 first-generation TPUs to generate self-play games and 16 second-generation TPUs to train networks; evaluation matches used 3 hours per game plus 15 seconds per move, with opponents on 44 CPU cores and AlphaZero on four first-generation TPUs and 44 CPU cores.
Limitations
- Stated: AlphaZero did not use opening books, endgame tablebases, or domain-specific heuristics; the authors note that some such techniques could further improve performance but were left for future research.
- Stated: hyperparameters were reused across games without game-specific tuning, except for exploration noise and the learning rate schedule; matches against an opening book were low in diversity, and forced randomization increased losses.
- Evident from the text: results cover only three games with separate trained instances, no cross-game transfer is demonstrated, and evaluation compares AlphaZero on TPU hardware against CPU-based opponents, so hardware and time-control differences qualify direct strength comparisons.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
3 Valid · 2 Not valid · 8 Not checked · 43 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“Apart from the representation of positions and actions described above, AlphaZero uses the same network architecture as AlphaGo Zero ( 9 ), briefly recapitulated here.”
What the sources say
- Supportedcites [9] source
The Neural Network Architecture Comparison (p. 29) identifies AlphaGo Zero’s architecture as a residual tower with policy and value heads. The citation supplies the AlphaGo Zero architecture that the citing authors say AlphaZero uses; the comparison to AlphaZero is their own statement. (LLM, conf=0.88)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“and in Go, AlphaZero first outperformed AlphaGo Lee ( 9 ) after 30 hours (74,000 steps).”
What the sources say
- Partly supportedcites [9] sourceExperimental — accuracy not yet measured
Section 2, p. 7 reports that AlphaGo Zero outperformed AlphaGo Lee, but says this happened after 36 hours, not 30. The passage does not give 74,000 steps, and it names the system AlphaGo Zero rather than AlphaZero. (LLM, conf=0.90)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The training algorithm achieved similar performance in all independent runs (see fig. S3), suggesting that the high performance of AlphaZero's training algorithm is repeatable.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“AlphaZero also defeated Stockfish when given 1 / 10 as much thinking time as its opponent (i.e. searching ~ 1 / 10 , 000 as many positions),”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“and won 46% of games against Elmo when given 1 / 100 as much time (i.e. searching ~ 1 / 40 , 000 as many positions), see Fig. 2.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
23 more claims
Create a free account to see all 56 claims and to verify the 3 references still open.
- Probably not a substantial claim. 10
- It may state two separate claims as one. 23
- The quote only partly states this claim. 1
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Games have historically served as highly effective metrics for evaluating progress in deep reinforcement learning research”
From BALROG: Benchmarking Agentic LLM and VLM Reasoning On Games (2024) · cites this paper as Silver et al., 2018
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In 2017, AlphaGo, using the same learning algorithms in TD-Gammon and a billion times more computing power, soundly beat Ke Jie, the world Go champion, in a game that is to chess what chess is to backgammon in complexity (Silver et al., 2018).”
From Large Language Models and the Reverse Turing Test (2023) · cites this paper as Silver et al., 2018
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Most of the attention was on who won, but more important was how it won: AlphaGo was creative and had discovered new moves and board positions that were outside human experience.”
From Large Language Models and the Reverse Turing Test (2023) · cites this paper as Silver et al., 2018
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The primary method for applying more computation at inference, common in the AlphaZero family of agents2,72, is scaling the tree search and running multiple independent search attempts.”
From Olympiad-level formal mathematical reasoning with reinforcement learning (2026) · cites this paper as [2]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Modern AI can write computer programs (Li et al., 2022) or play chess at super-human level (Silver et al., 2018), but the ability of AI to perceive and act in the world remains far below human level.”
From Scaling Instructable Agents Across Many Simulated Worlds (2024) · cites this paper as Silver et al., 2018
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Self-citationAdvances in neural information processing systems 7
(1996) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(96)90463-0 · Cited by 14,281
Identity mappings in deep residual networks
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_38 · Cited by 10,741
Identity Mappings in Deep Residual Networks
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · arXiv · doi:10.48550/arxiv.1603.05027 · Cited by 10,665
Mastering the game of Go without human knowledge
Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., et al. (2017) · Nature · doi:10.1038/nature24270 · Cited by 9,084
Self-citationIn-Datacenter Performance Analysis of a Tensor Processing Unit
Jouppi, N. P., Young, C., Patil, N., Patterson, D., Agrawal, G., Bajwa, R., Bates, S., Bhatia, S., Boden, N., Borchers, A., et al. (2017) · Proceedings of the 44th Annual International Symposium on Computer Architecture · doi:10.1145/3079856.3080246 · Cited by 4,537
Encyclopedia of Artificial Intelligence
(2009) · IGI Global eBooks · doi:10.4018/978-1-59904-849-9 · Cited by 1,396
Campbell, M., Hoane, A. J., & Hsu, F.-H. (2002) · Artificial Intelligence · doi:10.1016/s0004-3702(01)00129-1 · Cited by 1,226
Lim, T.-S., Loh, W.-Y., & Shih, Y.-S. (2000) · Machine Learning · doi:10.1023/a:1007608224229 · Cited by 1,133
An analysis of alpha-beta pruning
Knuth, D. E., & Moore, R. W. (1975) · Artificial Intelligence · doi:10.1016/0004-3702(75)90019-3 · Cited by 1,004
Research works printed but not linked: 7
- C. J. Maddison A. Huang I. Sutskever D. Silver paper presented at the International Conference on Learning Representations 2015 San Diego CA 7 to 9 May 2015.
- D. N. L. Levy M. Newborn How Computers Play Chess (Ishi Press 2009).
- W. Steinitz The Modern Chess Instructor (Edition Olms 1990).
- O. Arenz “Monte Carlo chess ” master’s thesis Technische Universität Darmstadt (2012).
- 10.1023/a:1007634325138 · DOI
- T. Kaneko K. Hoki in Advances in Computer Games – 13th International Conference ACG 2011 Revised Selected Papers Tilburg Netherlands 20 to 22 November 2011 (Springer 2012) pp. 158–169.
- R. Ramanujan A. Sabharwal B. Selman in Proceedings of the 26th Conference on Uncertainty in Artificial Intelligence (UAI 2010) Catalina Island CA 8 to 11 July (AUAI Press 2010).
Cited by
Items 1,701–1,800 of 3,633
Krishnamoorthy, M. V. (2024) · Journal of Advances in Mathematics and Computer Science · doi:10.9734/jamcs/2024/v39i91924 · Cited by 10
The Generative Generic-Field Design Method Based on Design Cognition and Knowledge Reasoning
Mo, Z., Gong, L., Zhu, M., & Lan, J. (2024) · Sustainability · doi:10.3390/su16229841 · Cited by 6
Computers and chess masters: The role of AI in transforming elite human performance
Bilalić, M., Graf, M., & Vaci, N. (2024) · British Journal of Psychology · doi:10.1111/bjop.12750 · Cited by 15
“The subsequent iteration, AlphaZero, embarked on learning the game of Go from the ground up, without any prior human knowledge, achieving even more remarkable results than its predecessor (Silver et al., 2018).”
Li, J., Zang, Z., Ji, Q., Sun, C., Qiang, W., Zhang, J., Zheng, C., Sun, F., & Xiong, H. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-024-02321-2 · Cited by 8
Conceptualizing cyber policy through complexity theory
Brantly, A. F. (2019) · Journal of Cyber Policy · doi:10.1080/23738871.2019.1583763 · Cited by 8
Machine Discovery of Comprehensible Strategies for Simple Games Using Meta-interpretive Learning
Muggleton, S., & Hocquette, C. (2019) · New Generation Computing · doi:10.1007/s00354-019-00054-2 · Cited by 5
DefogGAN: Predicting Hidden Information in the StarCraft Fog of War with Generative Adversarial Nets
Jeong, Y., Choi, H., Kim, B., & Gwon, Y. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.5853 · Cited by 5
Reinforcement Learning of Risk-Constrained Policies in Markov Decision Processes
Brázdil, T., Chatterjee, K., Novotný, P., & Vahala, J. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i06.6531 · Cited by 4
“We follow the spirit of (Silver et al. 2018) and use the predictor fθ to evaluate the leaf nodes.”
MethodImproved Feature Learning: A Maximum-Average-Out Deep Neural Network for the Game Go
Li, X., Lv, Z., Liu, B., Wu, L., & Wang, Z. (2020) · Mathematical Problems in Engineering · doi:10.1155/2020/1397948 · Cited by 5
“Various game programs, including the AlphaGo [4] series, AlphaGo Zero [5], Alpha Zero [6], Alpha Star [7], Libratus [8], Pluribus [9], AI Suphx [10], and ELF Go [11] all use deep neural networks (DNNs) as their prediction model.”
BackgroundPMA-DRL: A parallel model-augmented framework for deep reinforcement learning algorithms
Luo, X., & Wang, Y. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.04.091 · Cited by 7
Evaluation-function modeling with multi-layered perceptron for RoboCup soccer 2D simulation
Fukushima, T., Nakashima, T., & Akiyama, H. (2020) · Artificial Life and Robotics · doi:10.1007/s10015-020-00602-w · Cited by 5
“Especially, in the domain of board games such as Go and Shogi, computers outperformed professional human players, and it is argued that computers have better decision-making ability than human beings [1].”
BackgroundCompetitive Deep Reinforcement Learning over a Pokémon Battling Simulator
Simões, D., Reis, S., Lau, N., et al. (2020) · IEEE International Conference on Autonomous Robot Systems and Competitions · doi:10.1109/icarsc49921.2020.9096092 · Cited by 5
“…properties of their algorithms in single-agent environments [24] (where a single agent competes to successfully finish the game) and in multi-agent systems (where a team of agents must coordinate to achieve a goal [27], [30], [29], or a group of enemies must compete for supremacy [32], [11], [26]).”
BackgroundArtificial Intelligence and Imagination
Davies, J. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108580298.011 · Cited by 5
The brain, the artificial neural network and the snake: why we see what we see
Treccani, C. (2020) · AI & Society · doi:10.1007/s00146-020-01065-0 · Cited by 5
Improving Human Players’ T-Spin Skills in Tetris with Procedural Problem Generation
Oikawa, T., Hsueh, C.-H., & Ikeda, K. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-65883-0_4 · Cited by 8
POPO: Pessimistic Offline Policy Optimization
He, Q., Hou, X., & Liu, Y. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747886 · Cited by 8
“Combined with deep learning, reinforcement learning (RL) has proven its great potential in a wide range of fields such as playing Atari games [3], playing chess, Go and shoji [4], beating human players in StarCraft [5] etc.”
Affordance as general value function: a computational model
Graves, D., Günther, J., & Luo, J. (2021) · Adaptive Behavior · doi:10.1177/1059712321999421 · Cited by 8
“More recent successes using this approach range from video games (Mnih et al., 2013; Vinyals et al., 2019) to board games (Silver et al., 2016, 2018) to physically manipulating Rubik’s cube (OpenAI et al., 2019).”
Designing an Automatic Agent for Repeated Language–based Persuasion Games
Raifer, M., Rotman, G., Apel, R., Tennenholtz, M., & Reichart, R. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00462 · Cited by 11
The Applicability of Self-Play Algorithms to Trading and Forecasting Financial Markets
Kotlarz, P., Hadji-Misheva, B., Posth, J.-A., Osterrieder, J. R., & Schwendner, P. (2020) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2021.668465 · Cited by 10
“Silver et al. (2017) and Silver et al. (2018) provide a comprehensive report on well-trained AI agents like DeepMind’s AlphaGo or AlphaStar which have mastered various critical aspects of such games and succeeded in competing against world-class human players, consistently beating them and even…”
Norm-Based Generalisation Bounds for Deep Multi-Class Convolutional Neural Networks
Ledent, A., Mustafa, W. A. A., Lei, Y., & Kloft, M. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i9.17007 · Cited by 37
“Deep learning has enjoyed an enormous amount of success in a variety of engineering applications in the last decade (Kr-izhevsky, Sutskever, and Hinton 2012; He et al. 2016; Silver et al. 2018).”
BackgroundReward function shape exploration in adversarial imitation learning: an empirical study
Wang, Y., & Li, X. (2021) · IEEE International Conference on Artificial Intelligence and Computer Applications (ICAICA) · doi:10.1109/icaica52286.2021.9497894 · Cited by 6
“Deep reinforcement learning (DRL) has achieved extraordinary performance in Chess and cards [1, 2], computer games [3-5], and robot manipulation [6] in recent years.”
BackgroundWhat Constitutes Fairness in Games? A Case Study with Scrabble
Aung, H. P. P., Khalid, M. N. A., & Iida, H. (2021) · Information · doi:10.3390/info12090352 · Cited by 14
“With the growing availability and use of machine learning techniques and faster computer hardware, superior computer programs against human beings in the most popular perfect information games have emerged, including checkers, chess, shogi, and Go [20].”
BackgroundNonlinear Control of a Gas Turbine Engine with Reinforcement Learning
Singh, R., Nataraj, P. S. V., & Maity, A. (2021) · Lecture notes in networks and systems · doi:10.1007/978-3-030-89880-9_8 · Cited by 5
Learning the Rules of the Game: An Interpretable AI for Learning How to Play
Aurentz, J. L., Navarro, A. M., & Insua, D. R. (2021) · IEEE Transactions on Games · doi:10.1109/tg.2021.3066245 · Cited by 4
Deep Reinforcement Learning for FlipIt Security Game
Greige, L., & Chin, P. S. (2022) · Studies in computational intelligence · doi:10.1007/978-3-030-93409-5_68 · Cited by 10
“Reinforcement learning has shown much success in games such as chess, backgammon and Go [21,24,22].”
BackgroundOn Multi-Agent Cognitive Cooperation: Can virtual agents behave like humans?
D’Avella, S., Camacho-Gonzalez, G., & Tripicchio, P. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.01.025 · Cited by 5
The Explanation Game: A Formal Framework for Interpretable Machine Learning
Watson, D., & Floridi, L. (2021) · Digital ethics lab yearbook · doi:10.1007/978-3-030-80083-3_11 · Cited by 6
Ho, S.-H. S., Nakamura, Y., Gopang, M., & Swain, J. E. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.806755 · Cited by 9
Wu, L., Xu, Z., Wang, Z., Chen, Z., Huang, Z., Peng, C., Pei, X., Li, X., Mailoa, J. P., Hsieh, C., et al. (2022) · Science China Technological Sciences · doi:10.1007/s11431-022-2095-7 · Cited by 8
Hybrid Q-learning for data-based optimal control of non-linear switching system
Li, X., Dong, L., & Sun, C. (2022) · Journal of Systems Engineering and Electronics · doi:10.23919/jsee.2022.000114 · Cited by 5
The optimal strategy against Fictitious Play in infinitely repeated games
Dong, H., & Mu, Y. (2022) · Chinese Control Conference (CCC) · doi:10.23919/ccc55666.2022.9902863 · Cited by 5
“The most impressive ones include AlphaGo[1], AlphaZero[2] to play Go or other chess games, DeepStack[3], Libratus[4], Pluribus[5] to play pok-ers, AlphaStar[6] to play StarCraft.”
BackgroundA Policy-Based Learning Beam Search for Combinatorial Optimization
Ettrich, R., Huber, M., & Raidl, G. R. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-30035-6_9 · Cited by 5
“LBS as well as the new P-LBS are both based on principles inspired by Alp-haZero [20], although AlphaZero relies on Monte Carlo Tree Search (MCTS) and not BS. AlphaZero has proven to be very successful in the board games Go, chess, and shogi, with its predecessor AlphaGo being the first computer…”
MethodDasgupta, R., & Kliem, J. (2023) · International Journal of Serious Games · doi:10.17083/ijsg.v10i1.548 · Cited by 5
“Recently, MCTS has been extended with deep machine learning in algorithms such as AlphaGo and AlphaZero [13, 14].”
MethodStrengthening network slicing for Industrial Internet with deep reinforcement learning
Tan, Y., Wang, J., & Liu, J. (2023) · Digital Communications and Networks · doi:10.1016/j.dcan.2023.06.009 · Cited by 5
A Holistic Framework for Factory Planning Using Reinforcement Learning
Klar, M., Mertes, J. G., Glatt, M. F., Ravani, B., & Aurich, J. C. (2023) · Springer eBooks · doi:10.1007/978-3-031-35779-4_8 · Cited by 7
More for Less: Safe Policy Improvement with Stronger Performance Guarantees
Wienhöft, P., Suilen, M., Simão, T. D., Dubslaff, C., Baier, C., & Jansen, N. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/490 · Cited by 19
de Carvalho, J. P., & Dimitrakopoulos, R. G. (2023) · International Journal of Mining Reclamation and Environment · doi:10.1080/17480930.2023.2247196 · Cited by 5
SEMI-ACTIVE STRUCTURAL CONTROL USING VISCOUS DAMPERS AND REINFORCEMENT LEARNING
Jedlinska, A., Pisarski, D., Mikułowski, G., Błachowski, B., & Jankowski, Ł. (2023) · ECCOMAS Thematic Conference on Smart Structures and Materials (SMART) · doi:10.7712/150123.9813.444767 · Cited by 7
Improving Inertial-Based UAV Localization using Data-Efficient Deep Reinforcement Learning
Tsiakmakis, D., Passalis, N., & Tefas, A. (2023) · European Signal Processing Conference (EUSIPCO) · doi:10.23919/eusipco58844.2023.10289910 · Cited by 5
“Indeed, DRL meth-ods have shown to achieve remarkable results in a variety of tasks in recent years, often outperforming humans [17], [18].”
BackgroundModeling the Game of Go by Ising Hamiltonian, Deep Belief Networks and Common Fate Graphs
Rojas‐Domínguez, A., Barradas‐Bautista, D., & Alvarado, M. (2019) · IEEE Access · doi:10.1109/access.2019.2917442 · Cited by 4
“Interestingly, AlphaGoZero and AlphaZero developed their gaming skills primarily by playing thousands of games against different versions of similar systems, thusmaking themArtificial Intelligence systems that can truly learn from experience.”
BackgroundProject Thyia: A Forever Gameplayer
Gaina, R. D., Lucas, S. M., & Pérez-Liébana, D. (2019) · IEEE Conference on Games (CoG) · doi:10.1109/cig.2019.8848047 · Cited by 8
“Our proposed method extends from the AlphaGoZero [26] and p-RHEA [30] approaches in literature, as detailed below.”
Evaluation‐function modeling with neural networks for RoboCup soccer
Fukushima, T., Nakashima, T., & Akiyama, H. (2019) · Electronics and Communications in Japan · doi:10.1002/ecj.12224 · Cited by 5
Inference-Based Deterministic Messaging For Multi-Agent Communication
Bhatt, V., & Buro, M. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i13.17339 · Cited by 5
Teaching People by Justifying Tree Search Decisions: An Empirical Study in Curling
Silva, C. R., Bowling, M., & Lelis, L. H. S. (2021) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.13219 · Cited by 8
“Artificial intelligence (AI) algorithms achieved superhuman performance on Atari (Mnih et al., 2015), Go, shogi, chess (Silver et al., 2018a), and strong performance in StarCraft (Vinyals et al., 2019).”
MethodConstruction of symmetric orthogonal designs with deep Q-network and orthogonal complementary design
Lai, J., Weng, L.-C., Peng, X., & Fang, K. (2022) · Computational Statistics & Data Analysis · doi:10.1016/j.csda.2022.107448 · Cited by 5
Neural Packet ClassificationPreprint
Liang, E., Zhu, H., Jin, X., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.10319 · Cited by 5
“AlphaGo [44], AlphaGo Zero [46] and AlphaZero [45] show that Deep RL algorithms can achieve superhuman performance in many challenging games like Go, chess and shogi.”
BackgroundKurita, M., & Hoki, K. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.07491 · Cited by 5
“, Backgammon [7], Checkers [8], Chess [9], Shogi [10], [11], and Go [12].”
BackgroundMorandin, F., Amato, G., Fantozzi, M., Gini, R., Metta, C., & Parton, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.10863 · Cited by 5
Rabault, J., Ren, F., Zhang, W., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.02464 · Cited by 5
Reinforcement learning in FlipItPreprint
Chin, S., & Greige, L. (2020) · arXiv (Cornell University) · Cited by 5
Hindsight Expectation Maximization for Goal-conditioned Reinforcement LearningPreprint
Tang, Y., & Kucukelbir, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.07549 · Cited by 10
“In this space, successful examples include both model-based [40, 41, 42, 43] and model-free algorithms [44, 17, 27, 18].”
MethodWhen Humans and Machines Make Joint Decisions: A Non-Symmetric Bandit Model.Preprint
Bordt, S., & Ulrike Von Luxburg, A. (2020) · arXiv (Cornell University) · Cited by 5
The shift from life in water to life on land advantaged planning in visually-guided behaviorPreprint
Mugan, U., & MacIver, M. A. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/585760 · Cited by 5
A comparative study of semi- and self-supervised semantic segmentation of biomedical microscopy dataPreprint
Horlava, N., Mironenko, A., Niehaus, S., Wagner, S., Roeder, I., & Scherf, N. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.08076 · Cited by 13
“Based on massive, annotated data sets, deep learning systems have come to the point where they are on par or even outperform humans in specific tasks [2] [3] [4].”
BackgroundMRAC-RL: A Framework for On-Line Policy Adaptation Under Parametric Model UncertaintyPreprint
Guha, A., & Annaswamy, A. M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.10562 · Cited by 15
“Successful applications have been broad and varied - ranging from direct actuator-level control and state regulation to high-level planning and decision making (Mnih et al. 2015, Ng et al. 2006, Lillicrap et al. 2015, Kober et al. 2013, Silver et al. 2018).”
BackgroundHuman-Agent Cooperation in Bridge BiddingPreprint
Lockhart, E., BURCH, N. R., Bard, N., Borgeaud, S., Eccles, T., Smaira, L., & Smith, R. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.14124 · Cited by 8
“Combining a deep neural network policy with a search procedure at training time to generate improved training data, and at test time to improve upon the raw neural network policy was introduced in [15] and has been successfully used in a number of games, including Hex [16], Go [17], Chess and Shogi [18].”
MethodDeep Reinforcement Learning in Quantitative Algorithmic Trading: A ReviewPreprint
Pricope, T.-V. (2021) · RePEc: Research Papers in Economics · doi:10.48550/arxiv.2106.00123 · Cited by 104
“The motivation for considering such a question is that, recently, it was showed that RL systems can outperform experts in conducting optimal control policies [10], [11] or can outperform even the best human individuals in complex games [12].”
Launchpad: A Programming Model for Distributed Machine Learning ResearchPreprint
Yang, F., Barth-Maron, G., Stańczyk, P., Hoffman, M. W., Liu, S., Kroiss, M., Pope, A., & Rrustemi, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.04516 · Cited by 25
“In combination these two capabilities have directly led to numerous advances in the field (e.g. Krizhevsky et al., 2012; Brock et al., 2018; Silver et al., 2018).”
BackgroundA Leap among Entanglement and Neural Networks: A Quantum Survey.Preprint
Massoli, F. V., Vadicamo, L., Amato, G. M., & Falchi, F. (2021) · arXiv (Cornell University) · Cited by 5
DKM: Differentiable K-Means Clustering Layer for Neural Network CompressionPreprint
Cho, M., Alizadeh-Vahid, K., Adya, S. N., & Rastegari, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.12659 · Cited by 48
“Deep neural networks (DNN) have demonstrated super-human performance on many cognitive tasks [18].”
BackgroundParameter-Free Deterministic Reduction of the Estimation Bias in Continuous Control.Preprint
Sağlam, B., Duran, E., Cicek, D. C., Mutlu, F. B., & Kozat, S. S. (2021) · arXiv (Cornell University) · Cited by 5
Representation Learning for Online and Offline RL in Low-rank MDPsPreprint
Uehara, M., Zhang, X., & Sun, W. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.04652 · Cited by 101
“…several empirical works have shown that leveraging representation learning techniques developed in supervised or unsupervised learning settings can accelerate the search for good decision-making strategies (Silver et al., 2018; Stooke et al., 2021; Srinivas et al., 2020; Yang and Nachum, 2021).”
BackgroundHeading Toward Trusted ATCO-AI Systems: A Literature ReviewPreprint
Zhong, J. Y. (2021) · Engineering Archive (engrXiv) · doi:10.31224/osf.io/t769b · Cited by 8
Wolfram, T., Tropf, F. C., & Rahal, C. (2022) · doi:10.31235/osf.io/a8ht9 · Cited by 5
Computational models of mentalizingPreprint
Gonzalez, B., & Chang, L. J. (2019) · doi:10.31234/osf.io/4tyd9 · Cited by 5
AlphaZero IdeasPreprint
González-Díaz, J., & Palacios‐Huerta, I. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4140916 · Cited by 6
Data Science and Machine Learning
(2022) · doi:10.5121/csit.2022.1215 · Cited by 5
Reinforcement Twinning: From Digital Twins to Model-Based Reinforcement LearningPreprint
Schena, L., Marques, P. A. D. M., Poletti, R., Ahizi, S., den Berghe, J. V., & Mendez, M. A. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4761240 · Cited by 7
Perception as the Essence of Phenomenal ConsciousnessPreprint
Galus, W. L. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5207381 · Cited by 7
Procedural Generalization by Planning with Self-Supervised World Models
Anand, A., Walker, J., Li, Y., Vértes, E., Schrittwieser, J., Ozair, S., Weber, T., & Hamrick, J. B. (2021) · arXiv · doi:10.48550/arxiv.2111.01587 · Cited by 5
“MuZero is an appealing candidate for investigating generalization because it already incorporates many ideas that are thought to be useful for generalization and transfer, including replay [61], planning [65], and model-based representation learning [22, 51].”
BackgroundSelf-citationLessons from the Trenches on Reproducible Evaluation of Language Models
Biderman, S., Schoelkopf, H., Sutawika, L., Gao, L., Tow, J., Abbasi, B., Aji, A. F., Ammanamanchi, P. S., Black, S., Clive, J., et al. (2024) · arXiv · doi:10.48550/arxiv.2405.14782 · Cited by 9
“…GPT-3 (Brown et al., 2020), and ChatGPT, used as approximate stand-ins for shifts in how the community uses and therefore evaluates LMs. Common practice for evaluating autoregressive language models today diverges from the method described in the paper for all listed tasks except MMLU and MATH.”
Generating Personas for Games with Multimodal Adversarial Imitation Learning
Ahlberg, W., Sestini, A., Tollmar, K., & Gisslén, L. (2023) · 2023 IEEE Conference on Games (CoG) · doi:10.1109/cog57401.2023.10333167 · Cited by 5
A machine learning framework for LES closure terms
Kurz, M., & Beck, A. (2022) · ETNA - Electronic Transactions on Numerical Analysis · doi:10.1553/etna_vol56s117 · Cited by 27
“eural networks (ANN), have achieved tremendous success: from pushing the state-of-the-art in the fields of image and speech recognition [17, 18] to surpassing human-level performance in the game of Go [29, 30]. This recent success of ANN was predominantly fuelled by the emergence of large datasets, the exploitation of…”
What if eye...? Computationally recreating vision evolution
Tiwary, K., Young, A., Tasneem, Z., Klinghoffer, T., Dave, A., Poggio, T., Nilsson, D.-E., Cheung, B., & Raskar, R. (2025) · Science Advances · doi:10.1126/sciadv.ady2888 · Cited by 10
Reining‐In the Spring‐Slider With Reinforcement Learning
Schultz, R. (2025) · Journal of Geophysical Research Solid Earth · doi:10.1029/2024jb029697 · Cited by 5
“…search is deep reinforcement learning (Sutton & Barto, 2018), a machine learning framework for control tasks that has been successful in playing games (Mnih et al., 2015; Silver et al., 2018) or even designing computer codes (Mankowitz et al., 2023)—often with super‐human levels of performance.”
Enhancing Heterogeneous Multi-Robot Teaming for Planetary Exploration
Suresh, A., Laux, M., Brinkmann, W., Danter, L. C., & Kirchner, F. (2025) · EASN International Conference on Innovation in Aviation & Space Towards Sustainability Today & Tomorrow · doi:10.3390/engproc2025090112 · Cited by 4
Reinforcement Learning Applications in Unmanned Underwater Vehicles: A Review
Tian, Z., Deng, S., Wu, Z., & Zhou, C. (2025) · Journal of Marine Science and Application · doi:10.1007/s11804-025-00723-3 · Cited by 5
“The success of AlphaGo (Silver et al., 2016) and AlphaZero (Silver et al., 2018) further emphasizes the potential of DRL for complex decision-making, which transforms DRL to new heights.”
Lizarraga, A., Honig, E., & Wu, Y. N. (2025) · Wiley Interdisciplinary Reviews Computational Statistics · doi:10.1002/wics.70035 · Cited by 12
“This system's efficiency makes it ideal for environments where speed and pattern recognition are crucial (Silver et al. 2018).”
Ryu, K., Liao, Q., Li, Z., Delgosha, P., Sreenath, K., & Mehr, N. (2025) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra55743.2025.11128783 · Cited by 51
“Although self-play has been proposed as a means to escalate opponent difficulty [35], [36], it is limited to competitive multi-agent settings and may converge to a local minimum.”
A neural symbolic model for space physics
Ying, J., Lin, H., Yue, C., Chen, Y., Xiao, C., Shi, Q., Liang, Y., Yau, S., Zhou, Y., & Ma, J. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-025-01126-3 · Cited by 19
“This principle is similar to that of AlphaZero 63 , which does not require human game records but can learn Article https://doi.org/10.1038/s42256-025-01126-3 to play Go through AI-versus-AI games.”
InfluentialFederici, L., Benedikter, B., & Furfaro, R. (2025) · Journal of Guidance Control and Dynamics · doi:10.2514/1.g009534 · Cited by 4
TCEC13: The 13th top chess engine championship
Haworth, G., & Hernandez, N. (2019) · ICGA Journal · doi:10.3233/icg-190103 · Cited by 4
Haworth, G., & Hernandez, N. (2019) · ICGA Journal · doi:10.3233/icg-190104 · Cited by 4
Understanding in synthetic chemistry: the case of periplanone B
Cornelissen, M. D., & de Regt, H. W. (2022) · Synthese · doi:10.1007/s11229-022-03929-y · Cited by 4
A Paradigm Shift in Cognitive Sciences?
Knyazev, G. G. (2023) · Neuroscience and Behavioral Physiology · doi:10.1007/s11055-023-01483-9 · Cited by 6
AI and machine learning tools in plantation mapping: potentials of high-resolution satellite data
Nithya, S. V., Kaliyaperumal, R., Pazhanivelan, S., & Latha, P. (2024) · Agricultural Science and Technology · doi:10.15547/ast.2024.02.012 · Cited by 6
Wang, Y., Lutz, T., Yue, X., & Du, J. (2024) · IISE Transactions · doi:10.1080/24725854.2024.2401041 · Cited by 4
“RL-based methods have been widely developed to master a large variety of tasks, including playing games (Silver et al. , 2016; Silver et al. , 2018), chip design (Mirhoseini et al. , 2021), autonomous driving (Kiran et al. , 2021), and optimizing oil and gas field development plans (Nasir et al. ,…”
Use machine learning to predict pulmonary metastasis of esophageal cancer: a population-based study
Fang, Y., Wan, J., & Zeng, Y. (2024) · Journal of Cancer Research and Clinical Oncology · doi:10.1007/s00432-024-05937-6 · Cited by 7
“At present, ML technology has been widely used in different fields, from self-driving cars, board games, and various event decisions (Silver et al. 2018).”
MethodCausal representation learning through higher-level information extraction
Silva, F., Oliveira, H. P., & Pereira, T. (2024) · ACM Computing Surveys · doi:10.1145/3696412 · Cited by 10
Khaled, A., & Lee, D.-H. (2024) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2024.3490501 · Cited by 4
Low-power and lightweight spiking transformer for EEG-based auditory attention detection
Lan, Y., Wang, Y., Zhang, Y., & Zhu, H. (2024) · Neural Networks · doi:10.1016/j.neunet.2024.106977 · Cited by 5
Fragility of chess positions: Measure, universality, and tipping points
Barthélemy, M. (2025) · Physical review. E · doi:10.1103/physreve.111.014314 · Cited by 4
“More recent innovations, such as AlphaZero, further highlight the game’s importance in advancing computational methods [7].”
A hybrid curriculum learning and tree search approach for network topology control
Meppelink, G. J., Rajaei, A., & Cremer, J. L. (2025) · Electric Power Systems Research · doi:10.1016/j.epsr.2025.111455 · Cited by 8
Chen, X., Tian, T., Dai, G., Wang, M., Song, Z., Zheng, W., & Zhou, Q. (2025) · IEEE Transactions on Aerospace and Electronic Systems · doi:10.1109/taes.2025.3538476 · Cited by 9
“In recent years, RL has made breakthroughs in solving sequential decision problems, the most notable example being the creation of AlphaGo and AlphaGo Zero [24], [25], [26].”
Q-value-based experience replay in reinforcement learning
Zhang, Z., & Li, R. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113296 · Cited by 7
LDQN: A Lightweight Deep Reinforcement Learning Model
Liu, W., Li, Y., & Tang, H. (2024) · IEEE Smart World Congress (SWC) · doi:10.1109/swc62898.2024.00261 · Cited by 4
“…games to learn from image pixels, surpassing human performance; DeepMind’s AlphaGo [9] defeated top human players, and its successor, AlphaZero [10], improved through self-learning and outperformed Al-phaGo; Tencent’s AI ”Juewu” triumphed over top human players in ”Honor of Kings” [11] and won…”
Strand, A., Gorton, P. R., & Bråthen, K. (2025) · IEEE Access · doi:10.1109/access.2025.3580279 · Cited by 22
“Notable examples of applying RL in complex systems are Deep-Mind’s AlphaGo and its successors, which have reached superhuman levels in complex games [18], [19].”
Examining Artificial Intelligence Anxiety in the Context of Anthropocentrism and the Art Turing Test
Xu, R., Hsu, Y., & Wang, X. (2025) · Empirical Studies of the Arts · doi:10.1177/02762374251353671 · Cited by 5
“Methods include the social emotional Turing challenge (Silver et al., 2018), the text reverse Turing test (Shao et al., 2019), the multi-round dialogue Turing test (Jiang et al., 2021; Zhang et al., 2023), the controlled experiment Turing test (Gonçalves, 2023), video Turing test (Lee et al.,…”
Time Spent Thinking in Online Chess Reflects the Value of Computation
Russek, E. M., Acosta‐Kane, D., van Opheusden, B., Mattar, M. G., & Griffiths, T. L. (2025) · Cognitive Science · doi:10.1111/cogs.70119 · Cited by 29
“For example, in games like chess and Go, expert humans are believed to search through far fewer future positions than artificial intelligence systems (Charness, 1981; Silver et al., 2018).”
Proposing and solving olympiad geometry with guided tree search
Zhang, C., Song, J., Li, S., Liang, Y., Ma, Y., Wang, W., Zhu, Y. B., & Zhu, S. C. (2026) · Nature Machine Intelligence · doi:10.1038/s42256-025-01164-x · Cited by 30
“Inspired by the success of board-game reinforcement learning agents 32–34 , we trained both a policy model and a value model.”
Metadata
- Authors
- Silver, D., Hubert, T., Schrittwieser, J., Antonoglou, I., Lai, M., Guez, A., Lanctot, M., Sifre, L., Kumaran, D., Graepel, T., Lillicrap, T., Simonyan, K., & Hassabis, D.
- Affiliations
- Google DeepMind (United Kingdom) (GB)
- Corresponding author
- Silver, David
- Hubert, Thomas
- Schrittwieser, Julian
- Hassabis, Demis
- ORCID
- Venue
- Science
- Published
- 2018-12-07
- Volume
- 362
- Issue
- 6419
- Pages
- 1140-1144
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- OpenAlex
- W2902907165
- Cited by
- 3,562
- References
- 30
- Citations by year
- 2026: 257 · 2025: 427 · 2024: 512 · 2023: 514 · 2022: 569 · 2021: 546 · 2020: 462 · 2019: 256 · 2018: 13
- FWCI
- 179.89
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Science (publishedVersion)
- PubMed (publishedVersion)
- UCL Discovery (University College London) (submittedVersion)
- Funders
- DeepMind
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
- Keywords
- AlphaZero, chess, shogi, Go, evaluation function learning, game tree search
- MeSH terms
- Algorithms, Artificial Intelligence, Humans, Reinforcement, Psychology, Software, Video Games
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.