The DOI matches its Crossref record; 30 of the 30 references it cites were found in a registry or an index; 5 of the 16 sources its claims cite were checked for support.
A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play
Open accessAbstract
One program to rule them all: Computers can beat humans at increasingly complex games, including chess and Go. However, these programs are typically constructed for a particular game, exploiting its properties, such as the symmetries of the board on which it is played. Silver et al. developed a program called AlphaZero, which taught itself to play Go, chess, and shogi (a Japanese version of chess) (see the Editorial, and the Perspective by Campbell). AlphaZero managed to beat state-of-the-art programs specializing in these three games. The ability of AlphaZero to adapt to various game rules is a notable step toward achieving a general game-playing system. Science , this issue p. 1140 ; see also pp. 1087 and 1118
Overview
CoChat Summary
TL;DR
AlphaZero is a single reinforcement-learning and MCTS algorithm that, given only game rules, learned superhuman play in chess, shogi, and Go through self-play. It defeated world-champion programs Stockfish, Elmo, and AlphaGo Zero using the same algorithm and network architecture across all three games.
Key findings
- In chess, AlphaZero defeated Stockfish 8, winning 155 games and losing 6 out of 1,000; it also won matches from common human openings and 2016 TCEC opening positions.
- In shogi, AlphaZero defeated Elmo, winning 98.2% of games as black and 91.2% overall; in Go, it defeated AlphaGo Zero, winning 61% of games.
- AlphaZero first outperformed Stockfish after 4 hours (300,000 steps), Elmo after 2 hours (110,000 steps), and AlphaGo Lee after 30 hours (74,000 steps); training ran for 700,000 steps, taking about 9 hours in chess, 12 hours in shogi, and 13 days in Go.
- AlphaZero searched only about 60,000 positions per second in chess and shogi, versus 60 million for Stockfish and 25 million for Elmo, yet still defeated Stockfish with 1/10 thinking time and won 46% against Elmo with 1/100 thinking time.
Methods
- AlphaZero uses a deep neural network \(f_\theta(s)\) that outputs move probabilities \(p\) and a scalar value \(v\), trained by reinforcement learning from self-play starting from random parameters; MCTS uses these outputs to guide search, with 800 simulations per move during training.
- The same algorithm, network architecture, and hyperparameters were used for chess, shogi, and Go, with no domain knowledge except game rules; no opening book, endgame tablebases, or domain-specific heuristics were used.
- Training used 5,000 first-generation TPUs to generate self-play games and 16 second-generation TPUs to train networks; evaluation matches used 3 hours per game plus 15 seconds per move, with opponents on 44 CPU cores and AlphaZero on four first-generation TPUs and 44 CPU cores.
Limitations
- Stated: AlphaZero did not use opening books, endgame tablebases, or domain-specific heuristics; the authors note that some such techniques could further improve performance but were left for future research.
- Stated: hyperparameters were reused across games without game-specific tuning, except for exploration noise and the learning rate schedule; matches against an opening book were low in diversity, and forced randomization increased losses.
- Evident from the text: results cover only three games with separate trained instances, no cross-game transfer is demonstrated, and evaluation compares AlphaZero on TPU hardware against CPU-based opponents, so hardware and time-control differences qualify direct strength comparisons.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
3 Valid · 2 Not valid · 8 Not checked · 43 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“Apart from the representation of positions and actions described above, AlphaZero uses the same network architecture as AlphaGo Zero ( 9 ), briefly recapitulated here.”
What the sources say
- Supportedcites [9] source
The Neural Network Architecture Comparison (p. 29) identifies AlphaGo Zero’s architecture as a residual tower with policy and value heads. The citation supplies the AlphaGo Zero architecture that the citing authors say AlphaZero uses; the comparison to AlphaZero is their own statement. (LLM, conf=0.88)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“and in Go, AlphaZero first outperformed AlphaGo Lee ( 9 ) after 30 hours (74,000 steps).”
What the sources say
- Partly supportedcites [9] sourceExperimental — accuracy not yet measured
Section 2, p. 7 reports that AlphaGo Zero outperformed AlphaGo Lee, but says this happened after 36 hours, not 30. The passage does not give 74,000 steps, and it names the system AlphaGo Zero rather than AlphaZero. (LLM, conf=0.90)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The training algorithm achieved similar performance in all independent runs (see fig. S3), suggesting that the high performance of AlphaZero's training algorithm is repeatable.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“AlphaZero also defeated Stockfish when given 1 / 10 as much thinking time as its opponent (i.e. searching ~ 1 / 10 , 000 as many positions),”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“and won 46% of games against Elmo when given 1 / 100 as much time (i.e. searching ~ 1 / 40 , 000 as many positions), see Fig. 2.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
23 more claims
Create a free account to see all 56 claims and to verify the 3 references still open.
- Probably not a substantial claim. 10
- It may state two separate claims as one. 23
- The quote only partly states this claim. 1
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Games have historically served as highly effective metrics for evaluating progress in deep reinforcement learning research”
From BALROG: Benchmarking Agentic LLM and VLM Reasoning On Games (2024) · cites this paper as Silver et al., 2018
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In 2017, AlphaGo, using the same learning algorithms in TD-Gammon and a billion times more computing power, soundly beat Ke Jie, the world Go champion, in a game that is to chess what chess is to backgammon in complexity (Silver et al., 2018).”
From Large Language Models and the Reverse Turing Test (2023) · cites this paper as Silver et al., 2018
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Most of the attention was on who won, but more important was how it won: AlphaGo was creative and had discovered new moves and board positions that were outside human experience.”
From Large Language Models and the Reverse Turing Test (2023) · cites this paper as Silver et al., 2018
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The primary method for applying more computation at inference, common in the AlphaZero family of agents2,72, is scaling the tree search and running multiple independent search attempts.”
From Olympiad-level formal mathematical reasoning with reinforcement learning (2026) · cites this paper as [2]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Modern AI can write computer programs (Li et al., 2022) or play chess at super-human level (Silver et al., 2018), but the ability of AI to perceive and act in the world remains far below human level.”
From Scaling Instructable Agents Across Many Simulated Worlds (2024) · cites this paper as Silver et al., 2018
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Self-citationAdvances in neural information processing systems 7
(1996) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(96)90463-0 · Cited by 14,281
Identity mappings in deep residual networks
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_38 · Cited by 10,741
Identity Mappings in Deep Residual Networks
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · arXiv · doi:10.48550/arxiv.1603.05027 · Cited by 10,665
Mastering the game of Go without human knowledge
Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., et al. (2017) · Nature · doi:10.1038/nature24270 · Cited by 9,084
Self-citationIn-Datacenter Performance Analysis of a Tensor Processing Unit
Jouppi, N. P., Young, C., Patil, N., Patterson, D., Agrawal, G., Bajwa, R., Bates, S., Bhatia, S., Boden, N., Borchers, A., et al. (2017) · Proceedings of the 44th Annual International Symposium on Computer Architecture · doi:10.1145/3079856.3080246 · Cited by 4,537
Encyclopedia of Artificial Intelligence
(2009) · IGI Global eBooks · doi:10.4018/978-1-59904-849-9 · Cited by 1,396
Campbell, M., Hoane, A. J., & Hsu, F.-H. (2002) · Artificial Intelligence · doi:10.1016/s0004-3702(01)00129-1 · Cited by 1,226
Lim, T.-S., Loh, W.-Y., & Shih, Y.-S. (2000) · Machine Learning · doi:10.1023/a:1007608224229 · Cited by 1,133
An analysis of alpha-beta pruning
Knuth, D. E., & Moore, R. W. (1975) · Artificial Intelligence · doi:10.1016/0004-3702(75)90019-3 · Cited by 1,004
Research works printed but not linked: 7
- C. J. Maddison A. Huang I. Sutskever D. Silver paper presented at the International Conference on Learning Representations 2015 San Diego CA 7 to 9 May 2015.
- D. N. L. Levy M. Newborn How Computers Play Chess (Ishi Press 2009).
- W. Steinitz The Modern Chess Instructor (Edition Olms 1990).
- O. Arenz “Monte Carlo chess ” master’s thesis Technische Universität Darmstadt (2012).
- 10.1023/a:1007634325138 · DOI
- T. Kaneko K. Hoki in Advances in Computer Games – 13th International Conference ACG 2011 Revised Selected Papers Tilburg Netherlands 20 to 22 November 2011 (Springer 2012) pp. 158–169.
- R. Ramanujan A. Sabharwal B. Selman in Proceedings of the 26th Conference on Uncertainty in Artificial Intelligence (UAI 2010) Catalina Island CA 8 to 11 July (AUAI Press 2010).
Cited by
Items 2,201–2,300 of 3,623
The Work of Art in the Age of Automated Creativity: On the autonomy of AI art
Wojnowski, K. (2024) · Praktyka Teoretyczna · doi:10.19195/prt.2024.3.7 · Cited by 3
Shi, Y., Wang, W., Tao, X., Dusparić, I., & Cahill, V. (2024) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros58592.2024.10801887 · Cited by 2
“To address this limitation, neural Monte-Carlo tree search (NMCTS), which combines the advantages of Monte Carlo tree search (MCTS) [5] and policy-based RL [6], can be leveraged.”
Pattern memory cannot be completely and truly realized in deep neural networks
Li, T., Lyu, R., & Xie, Z. (2024) · Scientific Reports · doi:10.1038/s41598-024-80647-0 · Cited by 2
Generation Z: AI Affinity and Adoption in Competitive German Organisations
Nebgen, T., & Kurz, W. (2025) · Journal of Next-Generation Research 5 0 · doi:10.70792/jngr5.0.v1i2.81 · Cited by 3
Rodríguez‐Sotres, R., & Aguayo‐Ortiz, R. (2025) · Educación Química · doi:10.22201/fq.18708404e.2025.1.89902 · Cited by 2
Redefining Work in the Age of AI: Navigating Employment, Automation, and Ethics
Giri, A., & Sharma, A. K. M. M. (2025) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-3384-7.ch009 · Cited by 3
Galván, E., & Ameneyro, F. V. (2025) · Information Sciences · doi:10.1016/j.ins.2025.122226 · Cited by 2
The Irreplicable Nature of Human Intuition: A Critical Response to Claims of Artificial Intuition
Kuzma, J. D. (2025) · Philosophy & Technology · doi:10.1007/s13347-025-00894-5 · Cited by 2
“Recent work in deep learning, particularly in areas such as computer vision and game playing, appears to support this interpretation (Silver et al., 2018).”
Mastering the Principles of Reinforcement Learning: Techniques, Applications, and Future Prospects
Kebede, F., Asefa, H. Y., & Desta, G. (2023) · Fusion of Multidisciplinary Research An International Journal · doi:10.63995/azuq8110 · Cited by 3
“This allows for more efficient learning and better generalization to complex environments [34].”
Shan, Y., Cao, L., Wang, Y., Ren, J., Huang, C., Gao, W., & Ling, S. (2025) · Materials Futures · doi:10.1088/2752-5724/ade732 · Cited by 3
“To address these goals, this study integrates cGAN and DQN into a unified deep reinforcement learning framework [46, 47].”
Leech, D. R., & Yoon, H. (2025) · Sustainability · doi:10.3390/su17135727 · Cited by 2
Qu, P., He, C., Wu, X., Wang, E., Xu, S., Liu, H., & Sun, X. (2025) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-025-09700-0 · Cited by 4
“In recent years, reinforcement learning has gained widespread application in solving decision problems in single-agent adversarial environments, such as Atari games [9] and Go [10].”
Shahid, A. A., Narang, Y., Petrone, V., Ferrentino, E., Handa, A., Fox, D., Pavone, M., & Roveda, L. (2025) · IEEE International Conference on Automation Science and Engineering (CASE) · doi:10.1109/case58245.2025.11163870 · Cited by 9
“Many domains have seen tremendous advancements of reinforcement learning (RL) applications in recent years, ranging from playing challenging games [1], [2] to learning high-dimensional continuous control in robotics [3], [4], [5].”
Scalable and Generalizable RL Agents for Attack Path Discovery via Continuous Invariant Spaces
Terranova, F., Lahmadi, A., & Chrisment, I. (2025) · International Symposium on Research in Attacks, Intrusions and Defenses (RAID) · doi:10.1109/raid67961.2025.00029 · Cited by 11
“For example, in chess [51], the board size (observation space) and possible moves (action space) are often fixed.”
A Robot that Learns Connect Four Using Game Theory and Demonstrations
Ayub, A., & Wagner, A. R. (2020) · Cited by 2
Furuhata, H., Araki, K., & Ogawa, T. (2020) · Journal of Medical Systems · doi:10.1007/s10916-020-01570-1 · Cited by 2
A Deep Reinforcement Learning scheme for Battery Energy Management
Opalic, S. M., Goodwin, M., Jiao, L., Nielsen, H. K., & Kolhe, M. L. (2020) · International Conference on Smart and Sustainable Technologies (SpliTech) · doi:10.23919/splitech49282.2020.9243797 · Cited by 3
“This was further explored in [8] for the games of Shogi and Chess, leading to similarly impressive results.”
BackgroundResultBehavioural Equivalence of Game Descriptions
Zhang, D. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-64984-5_24 · Cited by 3
Time recursive control of stochastic dynamical systems using forward dynamics and applications
Mamajiwala, M., & Roy, D. (2021) · International Journal of Mechanical Sciences · doi:10.1016/j.ijmecsci.2021.106969 · Cited by 2
Training a Reinforcement Learning Agent based on XCS in a Competitive Snake Environment
Büttner, J., & von Mammen, S. (2021) · IEEE Conference on Games (CoG) · doi:10.1109/cog52621.2021.9619161 · Cited by 2
“Deepmind’s AlphaZero [1] uses a deep neural network to learn a policy used in Monte Carlo tree search and was able to beat a world-champion program for the games of chess, shogi, and Go each.”
MethodThe Sharing of Similar Knowledge on Monte Carlo Algorithm applies to Cryptocurrency Trading Problem
Adsawinnawanawa, E., & Keeratipranon, N. (2022) · International Electrical Engineering Congress (iEECON) · doi:10.1109/ieecon53204.2022.9741604 · Cited by 2
“RL is very popular and used in various problem such as Chess Game [4], Go Game [5], Robotic [6], Autonomous Self-Driving [7] and Dialogue Generation [8].”
Faria, M. P. P., Julia, E. S., do Nascimento, M. Z., & Julia, R. M. S. (2022) · Proceedings of the ACM on Computer Graphics and Interactive Techniques · doi:10.1145/3522624 · Cited by 2
“created to master the games of chess, shogi and go [Silver et al. 2018].”
BackgroundTowards Modern Card Games with Large-Scale Action Spaces Through Action Representation
Yao, Z., Shi, T., Li, S., Xie, Y., Qin, Y., Xie, X., Lu, H., & Zhang, Y. (2022) · IEEE Conference on Games (CoG) · doi:10.1109/cog51982.2022.9893589 · Cited by 3
“Ourwork can be improved by incorporating the self-play technique [17] in training to enhance the opponents.”
BackgroundLong-Term Adaptivity in Distributed Intelligent Systems: Study of ViaBots in a Simulated Environment
Ardavs, A., Pudāne, M., Lavendelis, E., et al. (2019) · Robotics · doi:10.3390/robotics8020025 · Cited by 2
“Significant success has been achieved in specific domains, such as game playing [5] and autonomous driving [6].”
BackgroundLearning Policies from Self-Play with Policy Gradients and MCTS Value Estimates
Soemers, D. J. N. J., Piette, É., Stephenson, M., & Browne, C. (2019) · IEEE Conference on Games (CoG) · doi:10.1109/cig.2019.8848037 · Cited by 8
“Policies can be trained to optimise this objective using self-play, similar to cross-entropybased policies in AlphaGo Zero and related research [4]–[6].”
BackgroundMethodInfluentialPokerBot: Hand Strength Reinforcement Learning
Ramirez, A., Reinman, S., & Norouzi, N. (2019) · International Conference on Innovations in Intelligent Systems and Applications (INISTA) · doi:10.1109/inista.2019.8778267 · Cited by 2
“AlphaGo) and chess have taken into account the extremely large set of features present [7], [8].”
BackgroundStraight to the Point: Fast-Forwarding Videos via Reinforcement Learning Using Textual Data
Ramos, W., da Silva, M. G., Junior, E. R. A., et al. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01094 · Cited by 2
Akbulut, M. T., & Uğur, E. (2020) · International Journal of Intelligent Systems and Applications in Engineering · doi:10.18201/ijisae.2020261584 · Cited by 3
“It is more obvious to researchers in artificial intelligence (AI) and robotics because computers can do better than humans in games [1] but progress of AI is much slower when it comes to addressing real-world problems [2].”
Accelerating Inference on Binary Neural Networks with Digital RRAM Processing
Vieira, J., Giacomin, E., Qureshi, Y. M., Zapater, M., Tang, X., Kvatinsky, S., Atienza, D., & Gaillardon, P. (2020) · IFIP advances in information and communication technology · doi:10.1007/978-3-030-53273-4_12 · Cited by 2
Genes, the brain, and artificial intelligence in evolution
Kamatani, N. (2020) · Journal of Human Genetics · doi:10.1038/s10038-020-0813-z · Cited by 5
Artificial Intelligence in Synthetic Biology, Cyber Defense, and Aeromechanical Design
Glaz, B. J., Wang, C., Hurley, M. M., et al. (2020) · Cited by 2
Big Data: Big Challenges, Big Opportunities
Reed, D. A. (2019) · Merrill Series on The Research Mission of Public Universities · doi:10.17161/merrill.2019.13287 · Cited by 2
Forest Fire Control with Learning from Demonstration and Reinforcement Learning
Hammond, T., Schaap, D. J., Sabatelli, M., et al. (2020) · doi:10.1109/ijcnn48605.2020.9207548 · Cited by 2
“These environments can come in different forms, ranging from board-games and video-games [11, 12], to robotic simulators [13].”
BackgroundCan an AI learn political theory?
DeCanio, S. J. (2020) · AI Perspectives · doi:10.1186/s42467-020-00007-2 · Cited by 2
“AIs are moving beyond being mere tools that increase human capacities; in some areas like classic board games – Checkers, Chess, Shogi, and Go – they already sur-pass humans [44].”
BackgroundTowards a Smart Opponent for Board Games: Learning beyond Simulations
Karunanayake, N., Wijesinghe, A., Wijethunga, C., Kumaradasa, C., Jayasekara, P. G., & Rodrigo, R. P. (2020) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc42975.2020.9283458 · Cited by 2
“Initially, brute force algorithms were used in the gameplay, but with the development of artificial intelligence and machine learning, different learning techniques have been successfully applied to build agents that can master the skills of games like Chess, Atari [6] and GO [7].”
MethodFoundations of Artificial Intelligence and Effective Universal Induction
Zimmermann, J., & Cremers, A. B. (2020) · Springer eBooks · doi:10.1007/978-3-030-54173-6_3 · Cited by 4
Okawa, Y., Sasaki, T., & Iwane, H. (2020) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2020.12.2198 · Cited by 3
“, 2015) or strategy board games (Silver et al., 2018); however, we cannot ignore this in many kinds of engineering problems.”
BackgroundCognition as a Mechanical Process
Friedman, R. (2021) · NeuroSci · doi:10.3390/neurosci2020010 · Cited by 5
“They typically compute a best move by heuristic searching through all possible outcomes from all possible game moves, a method that is exhaustive in its search of possible combinations of board states [31].”
MethodSelf-Play or Group Practice: Learning to Play Alternating Markov Game in Multi-Agent System
Leung, C.-W., Hu, S., & Leung, H.-F. (2021) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr48806.2021.9412176 · Cited by 2
“It is noteworthy that self-play, which has a long history in traditional multi-agent learning [9], [10], now becomes one of the most popular schemes to train DRL agents [4], [11], [12].”
BackgroundExploring Monte Carlo Negotiation Search with Nontrivial Agreements
Malaby, E., & Licato, J. (2021) · Proceedings of the ... International Florida Artificial Intelligence Research Society Conference · doi:10.32473/flairs.v34i1.128504 · Cited by 2
“Turn-based zero-sum games like Chess and Go have already seen extensive research (recently, (Silver et al. 2018)), and more recent efforts have been made to develop AI for more complex environments like the real-time strategy game StarCraft 2 (Churchill et al. 2016).”
BackgroundKarpouzis, K. C., & Tsatiris, G. (2021) · Learning and analytics in intelligent systems · doi:10.1007/978-3-030-76794-5_3 · Cited by 9
Walsh, M. M., Menthe, L., Geist, E., Hastings, E., Kerrigan, J., Léveillé, J., Margolis, J., Martin, N., & Donnelly, B. P. (2021) · RAND Corporation eBooks · doi:10.7249/rr-a263-1 · Cited by 2
The imperative to find the courage to redesign the biomedical research enterprise
Bielekova, B., & Brownlee, S. (2021) · F1000Research · doi:10.12688/f1000research.53654.1 · Cited by 3
“Ultimately, this will generate algorithm(s) that consistently outperform humans.(29)”
BackgroundME-MCTS: Online Generalization by Combining Multiple Value Estimators
Baier, H., & Kaisers, M. (2021) · doi:10.24963/ijcai.2021/555 · Cited by 2
“Among other domains, it has been successful in General Game Playing [Finnsson, 2012], General Video Game Playing [Liebana et al. , 2016], and was an essential part in recent breakthroughs in deterministic board games [Silver et al. , 2017; Silver et al. , 2018].”
BackgroundMethodAmazon Chess Based on UCT-PVS Hybrid Algorithm
Li, Z., Ning, C., Cao, J., et al. (2021) · Chinese Control and Decision Conference (CCDC) · doi:10.1109/ccdc52312.2021.9602418 · Cited by 2
“…work will optimize the UCT algorithm formula, and combine the pruning algorithm to improve the search efficiency of the UCT algorithm [14], and optimize the evaluation or strategy of Amazon chess through evolutionary algorithms [15], neural network algorithms and reinforcement learning methods [16].”
MethodSupply Chain Synchronization Through Deep Reinforcement Learning
Jackson, I. (2022) · Lecture notes in intelligent transportation and infrastructure · doi:10.1007/978-3-030-94774-3_48 · Cited by 3
Ender, J., Fiorentini, S., de Orio, R. L., et al. (2022) · International Conference on Micro- and Nano-Electronics (ICMNE) · doi:10.1117/12.2624595 · Cited by 2
“The agent uses the deep Q-network (DQN) algorithm [33] to improve its decision-making and the inputs and outputs of the RL agent, as well as the environment are explained in the following.”
MethodSpatial state-action features for general games
Soemers, D. J. N. J., Piette, É., Stephenson, M., & Browne, C. (2023) · Artificial Intelligence · doi:10.1016/j.artint.2023.103937 · Cited by 4
“…principle this combination of techniques can be successfully applied (Silver et al., 2016; Anthony et al., 2017; Silver et al., 2017; Lorentz & Zosa IV, 2017; Silver et al., 2018; Tian et al., 2019; Morandin et al., 2019; Wu, 2019; Cazenave et al., 2020; Cazenave, 2022) to a wide variety of games.”
BackgroundMethodInfluentialLevDoom: A Benchmark for Generalization on Level Difficulty in Reinforcement Learning
Tomilin, T., Dai, T., Fang, M., & Pechenizkiy, M. (2022) · IEEE Conference on Games (CoG) · doi:10.1109/cog51982.2022.9893707 · Cited by 6
Multi-goal Reinforcement Learning via Exploring Successor Matching
Feng, X. (2022) · IEEE Conference on Games (CoG) · doi:10.1109/cog51982.2022.9893637 · Cited by 4
Optimizing measurement-based cooling by reinforcement learning
Yan, J., & Jing, J. (2022) · Physical Review A · doi:10.1103/physreva.106.033124 · Cited by 3
Wu, X., Yi, L., Klar, M., Hussong, M., Glatt, M. F., & Aurich, J. C. (2022) · Lecture notes in mechanical engineering · doi:10.1007/978-3-031-18326-3_18 · Cited by 2
World Models with an Entity-Based Representation
Khameneh, N. Y., & Guzdial, M. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment · doi:10.1609/aiide.v18i1.21966 · Cited by 2
“Automated game playing has traditionally been an important domain for artificial intelligence (Silver et al. 2018).”
BackgroundWang, Z., Ai, Y., Zuo, Q., Zhou, S., & Li, H. (2022) · Aerospace · doi:10.3390/aerospace9110632 · Cited by 3
Navigation of Robotic-Arms using Policy Gradient Reinforcement Learning
Farag, W. (2022) · International Journal of Computing and Digital Systems · doi:10.12785/ijcds/120171 · Cited by 3
Policy decision of curling in real competition scenes
Xiao, Q., Li, Z., Wang, X., Liu, Y., Li, Y., Yang, C., & Li, F. (2022) · Complex & Intelligent Systems · doi:10.1007/s40747-022-00921-4 · Cited by 4
“Then with the development of deep learning and reinforcement learning, discrete zero-sum games such as Go [15,16] and chess [17] have been gradually solved and achieved great success.”
BackgroundOptimizing communication in deep reinforcement learning with XingTian
Pan, L., Qian, J., Xia, W., Mao, H., Yao, J. P., Li, P., & Xiao, Z. (2022) · International Middleware Conference · doi:10.1145/3528535.3565249 · Cited by 3
A New Deep Reinforcement Learning Algorithm for the Online Stochastic Profitable Tour Problem
Klein, N., & Prünte, J. (2022) · doi:10.1109/ieem55944.2022.9989933 · Cited by 2
“Similar to the AlphaZero algorithm (c.f. [1]), we choose a convolutional neural network to guide the MCTS.”
BackgroundMethodData-Driven MPC for Nonlinear Systems with Reinforcement Learning
Li, Y., Wang, Q., Sun, Z., et al. (2022) · Chinese Control Conference (CCC) · doi:10.23919/ccc55666.2022.9902257 · Cited by 2
An environment emulator for training a neural network model to solve the “Following the leader” task
Selivanov, A., Борисович, Р. Р., Gryaznov, A. V., Shein, V. A., & Sboev, A. G. (2022) · Procedia Computer Science · doi:10.1016/j.procs.2022.11.058 · Cited by 2
A Data-Efficient Training Method for Deep Reinforcement Learning
Feng, W., Han, C., Lian, F., & Liu, X. (2022) · Electronics · doi:10.3390/electronics11244205 · Cited by 5
Ma, H., Fan, J., & Wang, Q. (2022) · International Conference on Systems and Informatics (ICSAI) · doi:10.1109/icsai57119.2022.10005466 · Cited by 2
“The application in the confrontation games such as Go, poker, and StarCraft 2 has already got outstanding achievements [1] [2] [3], and many new applications have also been developed in the fields of healthcare and smart grids [4].”
BackgroundAnalyses of Tabular AlphaZero on Strongly-Solved Stochastic Games
Hsueh, C.-H., Ikeda, K., Wu, I., Chen, J.-C., & Hsu, T. (2023) · IEEE Access · doi:10.1109/access.2023.3246638 · Cited by 2
“[2], most of the hyper-parameters followed those”
BackgroundMethodInfluentialEnhancement of CNN-based 2048 Player with Monte-Carlo Tree Search
Watanabe, S., & Matsuzaki, K. (2022) · International Conference on Technologies and Applications of Artificial Intelligence · doi:10.1109/taai57707.2022.00018 · Cited by 2
“Several computer players have been developed for two-player perfect-information games like Go, Chess, and Shogi [2], [3].”
BackgroundTian, C., Zhang, H., Li, Y., et al. (2022) · doi:10.1109/hpcc-dss-smartcity-dependsys57074.2022.00324 · Cited by 2
OpenHoldem: A Benchmark for Large-Scale Imperfect-Information Game Research
Li, K., Xu, H., Zhao, E., Wu, Z., & Xing, J. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3280186 · Cited by 7
“…to NLTH because the game tree of NLTH is much larger than that of Leduc Hold’em, and the pedagogical code does not contain the necessary acceleration techniques for NLTH. Based on this situation, we reimplement DeepStack for NLTH following the original article’s key ideas and obtain an…”
R.U.R. and the Vision of Artificial Life
Čapek, K. (2023) · The MIT Press eBooks · doi:10.7551/mitpress/14136.001.0001 · Cited by 7
Neural-network-based parameter tuning for multi-agent simulation using deep reinforcement learning
Hirano, M., & Izumi, K. (2023) · World Wide Web · doi:10.1007/s11280-023-01197-5 · Cited by 2
“proposed a reinforcement learning algorithm through self-playing, which achieved excellent performance in some games, such as chess, shogi, and Go [46]; this algorithm is based on their previous study, which is well known as “Alpha Go Zero” [47].”
MethodFuture: The Foreseeable Development of AI
Floridi, L. (2023) · Oxford University Press eBooks · doi:10.1093/oso/9780198883098.003.0003 · Cited by 2
AI and the UN Sustainable Development Goals
Floridi, L. (2023) · Oxford University Press eBooks · doi:10.1093/oso/9780198883098.003.0012 · Cited by 2
Gosavi, A., Sneed, L. H., & Spearing, L. A. (2023) · Optimization Letters · doi:10.1007/s11590-023-02062-0 · Cited by 2
“Research interest in API surged recently after superhuman performance was reported with a computer program playing AlphaGo and other computer games at Google via API based on policy gradients and stochastic policies [4, 29].”
Model-Based Offline Policy Optimization with Adversarial Network
Yang, J., Chen, X., Wang, S., & Zhang, B. (2023) · Frontiers in artificial intelligence and applications · doi:10.3233/faia230597 · Cited by 7
Jedlińska, A., Pisarski, D., Mikułowski, G., et al. (2023) · Annals of Computer Science and Information Systems · doi:10.15439/2023f8946 · Cited by 2
“However, despite the large successes of RL in mastering other complex tasks [9], including control-like problems [10]-[13], it is still a novel and very scarcely explored approach in structural control with only a handful of publications [14]-[16].”
BackgroundArtificial Intelligence and Inequality
Liso, N. D. (2023) · Springer eBooks · doi:10.1007/978-3-030-97417-6_49-1 · Cited by 2
Gronauer, S., Stümke, D., & Diepold, K. J. (2023) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros55552.2023.10341941 · Cited by 4
“Despite the tremendous success record of RL algorithms [18], [23], the deployment to real-world robots remains a challenge [5].”
BackgroundA Neural-Reinforcement-Learning-Based Guaranteed Cost Control for Perturbed Tracking Systems
Zhong, X., & Ni, Z. (2023) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2023.3346334 · Cited by 2
Overview on reinforcement learning of multi-agent game
Zou, W. (2023) · Journal of Physics Conference Series · doi:10.1088/1742-6596/2646/1/012021 · Cited by 2
“The astounding performance of the AlphaGo series highlights recent developments in applying RL to sequential choice issues [11-13].”
BackgroundMarkov Combinatorial Processes for Reinforcement Learning and Combinatorial Optimization Problems
Guerriero, F., & Saccomanno, F. P. (2023) · International Journal of Computing · doi:10.47839/ijc.22.4.3351 · Cited by 2
The evolution of the flip-it game in cybersecurity: Insights from the past to the future
Jafar, M. T., Yang, L., Li, G., & Yang, X. (2024) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2024.102195 · Cited by 3
“In chess, RL algorithms have enabled computers to challenge and even defeat world-class human players (Silver et al., 2018).”
Computer Shogi Tournaments and Techniques
Kaneko, T., & Takizawa, T. (2019) · IEEE Transactions on Games · doi:10.1109/tg.2019.2939259 · Cited by 2
“Furthermore, elmo is used in evaluating the playing strength of AlphaZero [3], as a representative of ex-”
BackgroundMethodInfluentialTowards a trans-epistemic society
Petitat, A. (2020) · Technium Social Sciences Journal · doi:10.47577/tssj.v4i1.58 · Cited by 2
Modeling Survival in model-based Reinforcement Learning
Moazami, S., & Doerschuk, P. I. (2020) · International Conference on Transdisciplinary AI (TransAI) · doi:10.1109/transai49837.2020.00009 · Cited by 2
“enumerate a few effective algorithms from each category, Deep Q-Learning (DQN) [3] and its variants [4], [15], [16] are value-based, i.”
BackgroundNaudé, W. A. (2020) · Perspektiven der Wirtschaftspolitik · doi:10.1515/pwp-2020-0021 · Cited by 5
“3 Für eine Erörterung des AlphaGo zugrundeliegenden General-Reinforcement-Learning-Algorithmus siehe Silver et al. 2018.”
BackgroundPareto Multi-task Deep Learning
Riccio, D., Dyankov, D., Jansen, G., Fatta, G. D., & Nicosia, G. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-61616-8_11 · Cited by 2
Guided Policy Search Based Control of a High Dimensional Advanced Manufacturing Process
Surana, A., & Reddy, K. (2022) · IEEE Conference on Control Technology and Applications (CCTA) · doi:10.1109/ccta49430.2022.9966165 · Cited by 4
“Deep Q-learning [1], [2], [3], and direct policy search [4] and actor-critic based deep RL (DRL) approaches [5], [6] have recently shown unprecedented success in dealing with high dimensional problems [7].”
BackgroundAnalyses of Tabular AlphaZero on NoGo
Hsueh, C.-H., Ikeda, K., Nam, S., et al. (2020) · International Conference on Technologies and Applications of Artificial Intelligence · doi:10.1109/taai51410.2020.00054 · Cited by 2
“Silver et al. [1] successfully applied the algorithm to chess, shogi, and Go, where superhuman levels of plays were achieved.”
BackgroundMethodInfluentialAI can turn the clock back before we know it
Gerdes, A. (2021) · IEEE International Symposium on Technology and Society (ISTAS) · doi:10.1109/istas52410.2021.9629161 · Cited by 4
Understanding Machine Learning Through Data-Oriented and Human Learning Approaches
Behpour, S., & Udash, A. (2022) · Educational communications and technology: issues and innovations · doi:10.1007/978-3-030-84729-6_1 · Cited by 2
Generating High Coherence Monophonic Music Using Monte-Carlo Tree Search
Fu, X., Deng, H., Yuan, X., & Hu, J. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3165718 · Cited by 3
“The mismatch index i is randomly sampled from [16 , 32) .”
MethodGenerative modeling and reinforcement learning for acoustic lens design
Wang, W.-C., & Amirkulova, F. (2022) · Metamaterials, Metadevices, and Metasystems · doi:10.1117/12.2642956 · Cited by 2
Comparison of Search Behaviors in Chess, Shogi, and the game of Go
Takeuchi, S. (2022) · Conference on Technologies and Applications of Artificial Intelligence (TAAI) · doi:10.1109/taai57707.2022.00041 · Cited by 2
“…the next visited node is selected based on the previous score and visited counts: where, Q ( s, a ) is the mean value and P ( s, a ) is the prior probability, N ( s, a ) is the visit count of selecting action a in state s , and a constant C ( s ) = log ((1+ N ( s )+ c base ) /c base )+ c init [10].”
BackgroundTordesillas, J., & Arbelaiz, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.01200 · Cited by 2
“stories of using RL for control [5], [6], we use model-free Deep-Reinforcement Learning (DRL) algorithms to design a personalized cancer chemotherapy schedule.”
MethodStructured agents for physical constructionPreprint
Bapst, V., Sánchez‐González, Á., Doersch, C., Stachenfeld, K., Kohli, P., Battaglia, P., & Hamrick, J. B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.03177 · Cited by 99
“The efficiency of Monte-Carlo Tree Search (MCTS) (Coulom, 2006) planning in RL has recently been highlighted in (Guo et al., 2014; Silver et al., 2016; 2017; 2018).”
BackgroundMethodInfluentialDirect Policy Gradients: Direct Optimization of Policies in Discrete Action SpacesPreprint
Lorberbom, G., Maddison, C. J., Heess, N., Hazan, T., & Tarlow, D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.06062 · Cited by 14
“MCTS is quite different because—unlike DirPG—it uses search and a simulator at test time, but it becomes closer when search results are distilled into a policy as in [36].”
BackgroundExample Based Hebbian Learning may be sufficient to support Human IntelligencePreprint
Wong, E. C. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/758375 · Cited by 2
“In addition to networks that are thought to provide a time base, there are also cells known as place cells, that together provide a spatial basis that appears to be stable (Muller, 1996;Smith and Mizumori, 2006;Moser et al., 2015).”
BackgroundZhan, H., & Cao, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.12268 · Cited by 10
“In recent years, the application of RL in tasks with highdimensional sensory inputs has shown the potential of creating artificial agents that can learn to accomplish a number of challenging tasks, including the Atari games [1], [2], [3], [4], [5], [6], [7], self-driving cars [8], and Go [9], [10], [11].”
BackgroundZhan, H., & Cao, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.01919 · Cited by 2
“In recent years, the application of reinforcement learning in missions with high-dimensional sensory inputs has shown the potential of creating artificial agents that can learn to accomplish a number of challenging tasks, including the Atari games [1]–[7], self-driving cars [8], and Go [9]–[11].”
BackgroundRare Rewards Amplify Dopamine Learning ResponsesPreprint
Rothenhoefer, K. M., Hong, T., Alikaya, A., & Stauffer, W. R. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/851709 · Cited by 2
“Biological learning signals have inspired deep reinforcement learning algorithms with performance that exceeds expert human performance on Atari games, chess, and Go ( 26, 27 ).”
BackgroundD'Orazio, R., Morrill, D., Wright, J. R., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.02967 · Cited by 2
“Function approximation has facilitated many recent successes in game playing more broadly [24, 25, 29].”
BackgroundWorld Programs for Model-Based Learning and Planning in Compositional State and Action SpacesPreprint
Segler, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.13007 · Cited by 6
“To conclude, we would also like to pose a small challenge: Given only a small set of historically played Go and Chess games as grid graphs, can you first learn the rules of the games to learn a simulator, and then compete with an AlphaZero-like agent (Silver et al., 2018) using an agent trained in the learned simulato…”
BackgroundMethod
Metadata
- Authors
- Silver, D., Hubert, T., Schrittwieser, J., Antonoglou, I., Lai, M., Guez, A., Lanctot, M., Sifre, L., Kumaran, D., Graepel, T., Lillicrap, T., Simonyan, K., & Hassabis, D.
- Affiliations
- Google DeepMind (United Kingdom) (GB)
- Corresponding author
- Silver, David
- Hubert, Thomas
- Schrittwieser, Julian
- Hassabis, Demis
- ORCID
- Venue
- Science
- Published
- 2018-12-07
- Volume
- 362
- Issue
- 6419
- Pages
- 1140-1144
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- OpenAlex
- W2902907165
- Cited by
- 3,562
- References
- 30
- Citations by year
- 2026: 257 · 2025: 427 · 2024: 512 · 2023: 514 · 2022: 569 · 2021: 546 · 2020: 462 · 2019: 256 · 2018: 13
- FWCI
- 179.89
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Science (publishedVersion)
- PubMed (publishedVersion)
- UCL Discovery (University College London) (submittedVersion)
- Funders
- DeepMind
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
- Keywords
- AlphaZero, chess, shogi, Go, evaluation function learning, game tree search
- MeSH terms
- Algorithms, Artificial Intelligence, Humans, Reinforcement, Psychology, Software, Video Games
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.