A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play
Open accessThe DOI matches its Crossref record; 30 of the 30 references it cites were found in a registry or an index; 5 of the 16 sources its claims cite were checked for support.
Abstract
One program to rule them all: Computers can beat humans at increasingly complex games, including chess and Go. However, these programs are typically constructed for a particular game, exploiting its properties, such as the symmetries of the board on which it is played. Silver et al. developed a program called AlphaZero, which taught itself to play Go, chess, and shogi (a Japanese version of chess) (see the Editorial, and the Perspective by Campbell). AlphaZero managed to beat state-of-the-art programs specializing in these three games. The ability of AlphaZero to adapt to various game rules is a notable step toward achieving a general game-playing system. Science , this issue p. 1140 ; see also pp. 1087 and 1118
Overview
CoChat Summary
TL;DR
AlphaZero is a single reinforcement-learning and MCTS algorithm that, given only game rules, learned superhuman play in chess, shogi, and Go through self-play. It defeated world-champion programs Stockfish, Elmo, and AlphaGo Zero using the same algorithm and network architecture across all three games.
Key findings
- In chess, AlphaZero defeated Stockfish 8, winning 155 games and losing 6 out of 1,000; it also won matches from common human openings and 2016 TCEC opening positions.
- In shogi, AlphaZero defeated Elmo, winning 98.2% of games as black and 91.2% overall; in Go, it defeated AlphaGo Zero, winning 61% of games.
- AlphaZero first outperformed Stockfish after 4 hours (300,000 steps), Elmo after 2 hours (110,000 steps), and AlphaGo Lee after 30 hours (74,000 steps); training ran for 700,000 steps, taking about 9 hours in chess, 12 hours in shogi, and 13 days in Go.
- AlphaZero searched only about 60,000 positions per second in chess and shogi, versus 60 million for Stockfish and 25 million for Elmo, yet still defeated Stockfish with 1/10 thinking time and won 46% against Elmo with 1/100 thinking time.
Methods
- AlphaZero uses a deep neural network \(f_\theta(s)\) that outputs move probabilities \(p\) and a scalar value \(v\), trained by reinforcement learning from self-play starting from random parameters; MCTS uses these outputs to guide search, with 800 simulations per move during training.
- The same algorithm, network architecture, and hyperparameters were used for chess, shogi, and Go, with no domain knowledge except game rules; no opening book, endgame tablebases, or domain-specific heuristics were used.
- Training used 5,000 first-generation TPUs to generate self-play games and 16 second-generation TPUs to train networks; evaluation matches used 3 hours per game plus 15 seconds per move, with opponents on 44 CPU cores and AlphaZero on four first-generation TPUs and 44 CPU cores.
Limitations
- Stated: AlphaZero did not use opening books, endgame tablebases, or domain-specific heuristics; the authors note that some such techniques could further improve performance but were left for future research.
- Stated: hyperparameters were reused across games without game-specific tuning, except for exploration noise and the learning rate schedule; matches against an opening book were low in diversity, and forced randomization increased losses.
- Evident from the text: results cover only three games with separate trained instances, no cross-game transfer is demonstrated, and evaluation compares AlphaZero on TPU hardware against CPU-based opponents, so hardware and time-control differences qualify direct strength comparisons.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
3 Valid · 2 Not valid · 8 Not checked · 43 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“Apart from the representation of positions and actions described above, AlphaZero uses the same network architecture as AlphaGo Zero ( 9 ), briefly recapitulated here.”
What the sources say
- Supportedcites [9] source
The Neural Network Architecture Comparison (p. 29) identifies AlphaGo Zero’s architecture as a residual tower with policy and value heads. The citation supplies the AlphaGo Zero architecture that the citing authors say AlphaZero uses; the comparison to AlphaZero is their own statement. (LLM, conf=0.88)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“and in Go, AlphaZero first outperformed AlphaGo Lee ( 9 ) after 30 hours (74,000 steps).”
What the sources say
- Partly supportedcites [9] sourceExperimental — accuracy not yet measured
Section 2, p. 7 reports that AlphaGo Zero outperformed AlphaGo Lee, but says this happened after 36 hours, not 30. The passage does not give 74,000 steps, and it names the system AlphaGo Zero rather than AlphaZero. (LLM, conf=0.90)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The training algorithm achieved similar performance in all independent runs (see fig. S3), suggesting that the high performance of AlphaZero's training algorithm is repeatable.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“AlphaZero also defeated Stockfish when given 1 / 10 as much thinking time as its opponent (i.e. searching ~ 1 / 10 , 000 as many positions),”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“and won 46% of games against Elmo when given 1 / 100 as much time (i.e. searching ~ 1 / 40 , 000 as many positions), see Fig. 2.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
23 more claims
Create a free account to see all 56 claims and to verify the 3 references still open.
- Probably not a substantial claim. 10
- It may state two separate claims as one. 23
- The quote only partly states this claim. 1
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Games have historically served as highly effective metrics for evaluating progress in deep reinforcement learning research”
From BALROG: Benchmarking Agentic LLM and VLM Reasoning On Games (2024) · cites this paper as Silver et al., 2018
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In 2017, AlphaGo, using the same learning algorithms in TD-Gammon and a billion times more computing power, soundly beat Ke Jie, the world Go champion, in a game that is to chess what chess is to backgammon in complexity (Silver et al., 2018).”
From Large Language Models and the Reverse Turing Test (2023) · cites this paper as Silver et al., 2018
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Most of the attention was on who won, but more important was how it won: AlphaGo was creative and had discovered new moves and board positions that were outside human experience.”
From Large Language Models and the Reverse Turing Test (2023) · cites this paper as Silver et al., 2018
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The primary method for applying more computation at inference, common in the AlphaZero family of agents2,72, is scaling the tree search and running multiple independent search attempts.”
From Olympiad-level formal mathematical reasoning with reinforcement learning (2026) · cites this paper as [2]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Modern AI can write computer programs (Li et al., 2022) or play chess at super-human level (Silver et al., 2018), but the ability of AI to perceive and act in the world remains far below human level.”
From Scaling Instructable Agents Across Many Simulated Worlds (2024) · cites this paper as Silver et al., 2018
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Self-citationAdvances in neural information processing systems 7
(1996) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(96)90463-0 · Cited by 14,281
Identity mappings in deep residual networks
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_38 · Cited by 10,741
Identity Mappings in Deep Residual Networks
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · arXiv · doi:10.48550/arxiv.1603.05027 · Cited by 10,665
Mastering the game of Go without human knowledge
Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., et al. (2017) · Nature · doi:10.1038/nature24270 · Cited by 9,084
Self-citationIn-Datacenter Performance Analysis of a Tensor Processing Unit
Jouppi, N. P., Young, C., Patil, N., Patterson, D., Agrawal, G., Bajwa, R., Bates, S., Bhatia, S., Boden, N., Borchers, A., et al. (2017) · Proceedings of the 44th Annual International Symposium on Computer Architecture · doi:10.1145/3079856.3080246 · Cited by 4,537
Encyclopedia of Artificial Intelligence
(2009) · IGI Global eBooks · doi:10.4018/978-1-59904-849-9 · Cited by 1,396
Campbell, M., Hoane, A. J., & Hsu, F.-H. (2002) · Artificial Intelligence · doi:10.1016/s0004-3702(01)00129-1 · Cited by 1,226
Lim, T.-S., Loh, W.-Y., & Shih, Y.-S. (2000) · Machine Learning · doi:10.1023/a:1007608224229 · Cited by 1,133
An analysis of alpha-beta pruning
Knuth, D. E., & Moore, R. W. (1975) · Artificial Intelligence · doi:10.1016/0004-3702(75)90019-3 · Cited by 1,004
Research works printed but not linked: 7
- C. J. Maddison A. Huang I. Sutskever D. Silver paper presented at the International Conference on Learning Representations 2015 San Diego CA 7 to 9 May 2015.
- D. N. L. Levy M. Newborn How Computers Play Chess (Ishi Press 2009).
- W. Steinitz The Modern Chess Instructor (Edition Olms 1990).
- O. Arenz “Monte Carlo chess ” master’s thesis Technische Universität Darmstadt (2012).
- 10.1023/a:1007634325138 · DOI
- T. Kaneko K. Hoki in Advances in Computer Games – 13th International Conference ACG 2011 Revised Selected Papers Tilburg Netherlands 20 to 22 November 2011 (Springer 2012) pp. 158–169.
- R. Ramanujan A. Sabharwal B. Selman in Proceedings of the 26th Conference on Uncertainty in Artificial Intelligence (UAI 2010) Catalina Island CA 8 to 11 July (AUAI Press 2010).
Cited by
Items 401–500 of 3,629
Quantum optimal control of multilevel dissipative quantum systems with reinforcement learning
An, Z., Song, H.-J., He, Q.-K., & Zhou, D. L. (2021) · Physical Review A · doi:10.1103/physreva.103.012404 · Cited by 55
Kang, D., Wang, X., Zheng, X., & Zhao, Y. (2020) · Fuel · doi:10.1016/j.fuel.2020.120006 · Cited by 50
Deep reinforcement learning for conservation decisions
Lapeyrolerie, M., Chapman, M., Norman, K., & Boettiger, C. (2022) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.13954 · Cited by 62
“To date, the problems considered by RL research have largely been drawn from examples in robotic movement and games like Go and Starcraft (OpenAI et al. 2019; Silver et al. 2018; Vinyals et al. 2019).”
BackgroundMethodOSTTD: Offloading of Splittable Tasks With Topological Dependence in Multi-Tier Computing Networks
Zhang, R., Chu, X., Ma, R., Zhang, M., Lin, L., Gao, H., & Guan, H. (2022) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2022.3227023 · Cited by 55
“Since than, the universal approximation ability of deep learning has also been utilized to solve more complex problems, such as Atari games [33], Go [34], RTS games [35], etc .”
BackgroundMethod(Sasha), A. S., Emi, B., Zamir, A., Guibas, L. J., Savarese, S., & Malik, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.11971 · Cited by 44
“rerecorded datasets) and evaluated as a fixed pattern recognition problem. We study how such methods can be plugged into a larger framework for solving downstream active tasks. Reinforcement Learning, [76,50,71,45] and its variants like Meta-RL [20,27,54,21,40,74,17,49] or its sister fields such as imitation learning [1…”
MethodA Reinforcement Learning Environment For Job-Shop SchedulingPreprint
Tassel, P., Gebser, M., & Schekotihin, K. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.03760 · Cited by 126
“2013), AlphaGo (Silver et al. 2018), or AlphaStar (Vinyals et al.”
BackgroundEnergy Requirements Undermine Substrate Independence and Mind-Body Functionalism
Thagard, P. R. (2022) · Philosophy of Science · doi:10.1017/psa.2021.15 · Cited by 52
High-accuracy model-based reinforcement learning, a survey
Plaat, A., Kosters, W. A., & Preuß, M. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-022-10335-w · Cited by 80
“One of the main success stories of model-based reinforcement learning is Alp-haZero (Silver et al. 2017a, 2018).”
A Survey on Deep Reinforcement Learning for Data Processing and Analytics
Cai, Q., Cui, C., Xiong, Y., Wang, W., Xie, Z., & Zhang, M. (2022) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2022.3155196 · Cited by 82
“In the past decades, RL has achieved tremendous improvements in both theoretical and technical aspects [13], [14].”
BackgroundMethodPavirani, F., Gokhale, G., Claessens, B., & Develder, C. (2024) · Energy and Buildings · doi:10.1016/j.enbuild.2024.114161 · Cited by 59
Artificial intelligence (AI) in drug design and discovery: A comprehensive review
Bhat, A. R., & Ahmed, S. (2025) · In Silico Research in Biomedicine · doi:10.1016/j.insi.2025.100049 · Cited by 76
Manifold learning for parameter reduction
Holiday, A., Kooshkbaghi, M., Bello‐Rivas, J. M., Gear, C. W., Zagaris, A., & Kevrekidis, I. G. (2019) · Journal of Computational Physics · doi:10.1016/j.jcp.2019.04.015 · Cited by 54
To Regulate or Not: A Social Dynamics Analysis of an Idealised AI Race
Han, T. A., Pereira, L. M., Santos, F. C., & Lenaerts, T. (2020) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.12225 · Cited by 78
“This excitement is, on the one hand, due to a series of superhuman performances generated by particular breakthrough technologies (Silver et al., 2017; Brown & Sandholm, 2018; Silver et al., 2018; Brown & Sandholm, 2019).”
BackgroundThe MineRL 2019 Competition on Sample Efficient Reinforcement Learning using Human Priors
Guss, W. H., Codel, C., Hofmann, K., Houghton, B., Kuno, N., Milani, S., Mohanty, S., Liebana, D. P., Salakhutdinov, R., Topin, N., et al. (2019) · arXiv · doi:10.48550/arxiv.1904.10079 · Cited by 46
A simulation-deep reinforcement learning (SiRL) approach for epidemic control optimization
Bushaj, S., Yin, X., Beqiri, A., Andrews, D., & Büyüktahtakın, İ. E. (2022) · Annals of Operations Research · doi:10.1007/s10479-022-04926-7 · Cited by 66
“DRL has proven its strength in various applications such as games (Mnih et al. 2013; Silver et al. 2018), combinatorial optimization (Bushaj and Büyüktahtakın 2021; Delarue et al. 2020), and healthcare (Mahmud et al. 2018).”
MethodHierarchical multi‐agent reinforcement learning for multi‐aircraft close‐range air combat
Kong, W., Zhou, D., Du, Y., Zhou, Y., & Zhao, Y. (2022) · IET Control Theory and Applications · doi:10.1049/cth2.12413 · Cited by 52
BacterAI maps microbial metabolism without prior knowledge
Dama, A. C., Kim, K. S., Leyva, D. M., Lunkes, A. P., Schmid, N. S., Jijakli, K., & Jensen, P. A. (2023) · Nature Microbiology · doi:10.1038/s41564-023-01376-0 · Cited by 57
Learning-based methods for adaptive informative path planning
Popović, M., Ott, J., Rückin, J., & Kochenderfer, M. J. (2024) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2024.104727 · Cited by 84
“MCTS simulations generate a robust policy, considering a longer planning horizon [113, 114].”
BackgroundFoundation Models in Agriculture: A Comprehensive Review
Yin, S., Xi, Y., Zhang, X., Sun, C., & Mao, Q. (2025) · Agriculture · doi:10.3390/agriculture15080847 · Cited by 76
“This shift, exemplified by breakthroughs in image recognition [17,18], has achieved su-perhuman performance in controlled settings [19,20] but faces deployment challenges in practical applications due to interpretability and data quality concerns.”
Artificial Intelligence: The Future for Organic Chemistry?
Peiretti, F., & Brunel, J. M. (2018) · ACS Omega · doi:10.1021/acsomega.8b01773 · Cited by 50
Materials for emergent silicon-integrated optical computing
Demkov, A. A., Bajaj, C., Ekerdt, J. G., Palmstrøm, C. J., & Yoo, S. J. B. (2021) · Journal of Applied Physics · doi:10.1063/5.0056441 · Cited by 52
RLPlace: Using Reinforcement Learning and Smart Perturbations to Optimize FPGA Placement
Elgammal, M. A., Murray, K. E., & Betz, V. (2021) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2021.3109863 · Cited by 45
SUNRISE: A Simple Unified Framework for Ensemble Learning in Deep Reinforcement LearningPreprint
Lee, K., Laskin, M., Srinivas, A., & Abbeel, P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.04938 · Cited by 168
“1 Introduction Model-free reinforcement learning (RL), with high-capacity function approximators, such as deep neural networks (DNNs), has been used to solve a variety of sequential decision-making problems, including board games [37, 38], video games [31, 47], and robotic manipulation [23].”
BackgroundMethodNeuroscience and Network Dynamics Toward Brain-Inspired Intelligence
Hu, B., Guan, Z., Chen, G., & Chen, C. L. P. (2021) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2021.3071110 · Cited by 47
“Most AI advances suggest that machine-learning algorithms count on massive samples for optimization and prediction purposes, causing high energy consumption [54], [82], [83].”
BackgroundDeep Learning in Construction: Review of Applications and Potential Avenues
Jacobsen, E. L., & Teizer, J. (2021) · Journal of Computing in Civil Engineering · doi:10.1061/(asce)cp.1943-5487.0001010 · Cited by 49
Massively Digitized Power Grid: Opportunities and Challenges of Use-Inspired AI
Xie, L., Zheng, X., Sun, Y., Huang, T., & Bruton, T. (2022) · Proceedings of the IEEE · doi:10.1109/jproc.2022.3175070 · Cited by 63
“From a broader economic perspective, AI has transformed a variety of domains over the past decade [5], including language processing [6], speech recognition [7], facial recognition [8], real-time object detection [9], multiplayer game [10]–[12], recommendation system [13], intelligent robotics [14]–[16], driving assis…”
BackgroundCheng, H., Li, H., Dai, Q., & Yang, J. (2023) · Chaos Solitons & Fractals · doi:10.1016/j.chaos.2023.113809 · Cited by 52
High-efficiency reinforcement learning with hybrid architecture photonic integrated circuit
Li, X.-K., Ma, J.-X., Li, X., Hu, J., Ding, C.-Y., Han, F.-K., Guo, X., Tan, X. G., & Jin, X. (2024) · Nature Communications · doi:10.1038/s41467-024-45305-z · Cited by 53
“As the fi rst computer program to defeat a professional human Go player, AlphaGo operates on RL principles 12,13 .”
MethodArtificial intelligence for quantum computing
Alexeev, Y., Farag, M. H., Patti, T. L., Wolf, M. E., Ares, N., Aspuru‐Guzik, A., Benjamin, S. C., Cai, Z., Cao, S., Chamberland, C., et al. (2025) · Nature Communications · doi:10.1038/s41467-025-65836-3 · Cited by 137
Deep Reinforcement Learning for General Game Playing
Goldwaser, A., & Thielscher, M. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i02.5533 · Cited by 49
Guo, Y. B., Zhu, L. Q., Long, T. Y., Wan, D. Y., & Ren, Z. Y. (2020) · Journal of Materials Chemistry C · doi:10.1039/c9tc06749c · Cited by 45
The foundation of efficient robot learning
Kaelbling, L. P. (2020) · Science · doi:10.1126/science.aaz7597 · Cited by 48
The Free Energy Principle for Perception and Action: A Deep Learning Perspective
Mazzaglia, P., Verbelen, T., Çatal, O., & Dhoedt, B. (2022) · Entropy · doi:10.3390/e24020301 · Cited by 73
Reinforcement Learning ApplicationsPreprint
Li, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.06973 · Cited by 96
Artificial Intelligence, Algorithmic Pricing and CollusionPreprint
Calvano, E., Calzolari, G., Denicolò, V., et al. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3304991 · Cited by 44
Dynamic Positioning using Deep Reinforcement Learning
Øvereng, S. S., Nguyen, D. T., & Hamre, G. (2021) · Ocean Engineering · doi:10.1016/j.oceaneng.2021.109433 · Cited by 45
“…aimed at including the nonlinear dynamics directly in the controller design, such as backstepping con-trol, a recursive control design methodology introduced by Kokotovic (1992), and widely exemplified by in the works of Fossen and Grøvlen (1998) and Skjetne and Fossen (2004), and Du et al. (2018).”
MethodDistributed Deep Reinforcement Learning: A Survey and a Multi-player Multi-agent Learning Toolbox
Yin, Q., Yu, T., Shen, S., Yang, J. J., Zhao, M., Ni, W., Huang, K., Liang, B., & Wang, L. (2024) · Machine Intelligence Research · doi:10.1007/s11633-023-1454-4 · Cited by 49
“Self-play becomes a popular tool since the success of Al-phaGo series [1], [2], [47], which train a player by fighting against itself.”
MethodA dopamine mechanism for reward maximization
Schultz, W. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2316658121 · Cited by 49
“Artificial computer models using TD RL, often together with deep neural networks, acquire champion-level and superhuman performance in Backgammon, Atari, Go, Chess and Shogi games (104-107).”
MethodMugan, U., & MacIver, M. A. (2020) · Nature Communications · doi:10.1038/s41467-020-16102-1 · Cited by 42
Classification in the Wild: The Science and Art of Transparent Decision Making
Katsikopoulos, K. V., Şimşek, Ö., Buckmann, M., & Gigerenzer, G. (2021) · doi:10.7551/mitpress/11790.001.0001 · Cited by 42
Unpredictability of AI: On the Impossibility of Accurately Predicting All Actions of a Smarter Agent
Yampolskiy, R. V. (2020) · Journal of Artificial Intelligence and Consciousness · doi:10.1142/s2705078520500034 · Cited by 42
A thousand brains: toward biologically constrained AI
Hole, K. J., & Ahmad, S. (2021) · SN Applied Sciences · doi:10.1007/s42452-021-04715-0 · Cited by 42
Physics-informed Dyna-style model-based deep reinforcement learning for dynamic control
Liu, X., & Wang, J. (2021) · Proceedings of the Royal Society A Mathematical Physical and Engineering Sciences · doi:10.1098/rspa.2021.0618 · Cited by 76
Efficient dendritic learning as an alternative to synaptic plasticity hypothesis
Hodassman, S., Vardi, R., Tugendhaft, Y., Goldental, A., & Kanter, I. (2022) · Scientific Reports · doi:10.1038/s41598-022-10466-8 · Cited by 48
A Multiagent Reinforcement Learning Approach for Wind Farm Frequency Control
Liang, Y., Zhao, X., & Sun, L. L. (2022) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2022.3182328 · Cited by 41
“Inrecentyears,deeprein-forcementlearning(DRL)hasachievedgreatsuccessinsolving computationallychallengingdecision-makingproblems,such asAtari[16],Go[17],andStarCraft[18].”
Continual learning-based trajectory prediction with memory augmented networks
Yang, B., Fan, F., Ni, R., Li, J., Loo, C. K., & Liu, X. (2022) · Knowledge-Based Systems · doi:10.1016/j.knosys.2022.110022 · Cited by 46
Zhang, X., Meng, D., Cai, J., Zhang, G., Yu, T., Pan, F., & Yang, Y. (2022) · Energy · doi:10.1016/j.energy.2022.126483 · Cited by 43
Gallego, F., Martín, C., Díáz, M., & Garrido, D. (2023) · Energy and AI · doi:10.1016/j.egyai.2023.100241 · Cited by 52
Human-Like Decision-Making of Autonomous Vehicles in Dynamic Traffic Scenarios
Zhang, T., Zhan, J., Shi, J., Xin, J., & Zheng, N. (2023) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2023.123696 · Cited by 53
AI-Empowered Virtual Network Embedding: A Comprehensive Survey
Wu, S., Chen, N., Xiao, A., Zhang, P., Jiang, C., & Zhang, W. (2024) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2024.3424533 · Cited by 56
“Representative methods include Monte Carlo Tree Search (MCTS) [135], AlphaGo [136], AlphaZero [137], etc .”
Health Care in 2030: Will Artificial Intelligence Replace Physicians?
Shah, N. R. (2019) · Annals of Internal Medicine · doi:10.7326/m19-0344 · Cited by 41
Situation-Aware Deep Reinforcement Learning Link Prediction Model for Evolving Criminal Networks
Lim, M., Abdullah, A. B., Jhanjhi, N. Z., & Khan, M. K. (2019) · IEEE Access · doi:10.1109/access.2019.2961805 · Cited by 43
“AlphaGo Zero was not only able to self-learn by playing against a version of itself but was able to master other board games such as Chess and Shogi [24] after being provided with just the basic legal moves of the games.”
BackgroundArtificial Intelligence-Guided De Novo Molecular Design Targeting COVID-19
Srinivasan, S., Batra, R., Chan, H., Kamath, G., Cherukara, M. J., & Sankaranarayanan, S. K. R. S. (2021) · ACS Omega · doi:10.1021/acsomega.1c00477 · Cited by 42
“We draw inspiration from the success of AI algorithms such as Monte Carlo tree search (MCTS), which have proven to be extremely e ff ective in computer games (Go Game and several others) 19 against human competitors.”
Data Science for Economics and Finance: Methodologies and Applications
Consoli, S., Recupero, D. R., & Saisana, M. (2021) · Springer eBooks · doi:10.1007/978-3-030-66891-4 · Cited by 67
ELF OpenGo: An Analysis and Open Reimplementation of AlphaZeroPreprint
Tian, Y., Ma, J., Gong, Q., Sengupta, S., Chen, Z.-Y., Pinkerton, J., & Zitnick, C. (2019) · International Conference on Machine Learning · doi:10.48550/arxiv.1902.04522 · Cited by 124
Simplified Action Decoder for Deep Multi-Agent Reinforcement LearningPreprint
Hu, H., & Foerster, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.02288 · Cited by 87
“This includes examples such as Go (Silver et al., 2016; 2017; 2018), poker (Brown & Sandholm, 2017; Moravˇcík et al., 2017; Brown & Sandholm, 2019) and chess (Campbell et al., 2002).”
MethodThe Explanation Game: A Formal Framework for Interpretable Machine Learning
Watson, D., & Floridi, L. (2021) · Philosophical studies series · doi:10.1007/978-3-030-81907-1_11 · Cited by 55
Deco, G., Perl, Y. S., de la Fuente, L., Sitt, J. D., Yeo, B. T. T., Tagliazucchi, E., & Kringelbach, M. L. (2023) · Network Neuroscience · doi:10.1162/netn_a_00300 · Cited by 50
“This powerful machine-learning technique has proven highly useful for providing solutions to a number of difficult computational problems ranging from vision to playing Go (Sejnowski, 2018, 2020; Silver et al., 2018; Yang & Wang, 2020).”
MethodChallenges, evaluation and opportunities for open-world learning
Kejriwal, M., Kildebeck, E., Steininger, R. J., & Shrivastava, A. (2024) · Nature Machine Intelligence · doi:10.1038/s42256-024-00852-4 · Cited by 62
“For instance, faced with a non-trivial variant of a game like chess, even a superhuman gameplaying system like AlphaZero 14 would probably need to be ‘re-trained’ to learn these new strategies 15 .”
Fail-safe genetic codes designed to intrinsically contain engineered organisms
Calles, J., Justice, I., Brinkley, D., Garcia, A., & Endy, D. (2019) · Nucleic Acids Research · doi:10.1093/nar/gkz745 · Cited by 40
“Traditional engineering disciplines have established methods for controlling systems on the timescales of immediate input and response (e.g. autonomous control) (10,11), and intermediate learning and memory (e.g. algorithms that learn) (12,13).”
Reinforcement Learning-Based Tracking Control of USVs in Varying Operational Conditions
Martinsen, A. B., Lekkas, A. M., Gros, S., Glomsrud, J. A., & Pedersen, T. A. (2020) · Frontiers in Robotics and AI · doi:10.3389/frobt.2020.00032 · Cited by 44
Derivative-free reinforcement learning: a review
Qian, H., & Yu, Y. (2021) · Frontiers of Computer Science · doi:10.1007/s11704-020-0241-4 · Cited by 59
Kuo, C.-H., Chen, C.-T., Lin, S.-J., & Huang, S.-H. (2021) · IEEE Access · doi:10.1109/access.2021.3068269 · Cited by 58
“For example, AlphaZero [10] trains an agent to play a board game through self-play.”
BackgroundA Reinforcement Learning based Path Planning Approach in 3D Environment
Kulathunga, G. (2022) · Procedia Computer Science · doi:10.1016/j.procs.2022.10.217 · Cited by 48
Chess AI: Competing Paradigms for Machine Intelligence
Maharaj, S., Polson, N., & Turk, A. (2022) · Entropy · doi:10.3390/e24040550 · Cited by 65
“AlphaZero’s evaluation uses deep convolutional neural networks (CNNs) to estimate the policy vector p t and value v t of nodes in the search tree (Silver, Hubert, et al., 2018).”
BackgroundMethodInfluentialZhao, E., Yan, R., Li, J., Li, K., & Xing, J. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i4.20394 · Cited by 78
“For self-play methods, we compare AlphaHoldem’s KBest Self-Play with Naive Self-Play (Samuel 1959; Silver et al. 2018), Best-Win Self-Play (Silver et al.”
BackgroundMethodInfluentialReward Shaping via Meta-Learning
Zou, H., Ren, T., Yan, D., Su, H., & Zhu, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.09330 · Cited by 38
“…game elements, with examples in Doom (Wu & Tian, 2017) and Dota 2 (OpenAI, 2018), or simply depart from the scalar-reward RL approach and exploit rich supervision signals of other source with supervised learning (Dosovitskiy & Koltun, 2017; Silver et al., 2018; Huang et al., 2019; Wu et al., 2019).”
BackgroundSuperintelligent Agents Pose Catastrophic Risks: Can Scientist AI Offer a Safer Path?
Bengio, Y., Cohen, M., Fornasiere, D., Ghosn, J., Greiner, P., MacDermott, M., Mindermann, S., Oberman, A., Richardson, J., Richardson, O., et al. (2025) · SuperIntelligence - Robotics - Safety & Alignment · doi:10.70777/si.v2i5.15569 · Cited by 41
Reinforcement learning and artificial agency
Butlin, P. (2024) · Mind & Language · doi:10.1111/mila.12458 · Cited by 40
Composite Experience Replay-Based Deep Reinforcement Learning With Application in Wind Farm Control
Dong, H., & Zhao, X. (2021) · IEEE Transactions on Control Systems Technology · doi:10.1109/tcst.2021.3102476 · Cited by 40
“Due to the significant application potential of RL in various industry fields, it has aroused extensive research interests in recent years [2]–[8].”
BackgroundTreEnhance: A tree search method for low-light image enhancement
Cotogni, M., & Cusano, C. (2023) · Pattern Recognition · doi:10.1016/j.patcog.2022.109249 · Cited by 40
“[35], the nodes chosen in the selection step are processed by a convolutional neural network.”
MethodDeep Reinforcement Learning for Mineral Prospectivity Mapping
Shi, Z., Zuo, R., & Zhou, B. (2023) · Mathematical Geosciences · doi:10.1007/s11004-023-10059-9 · Cited by 46
“It has been successfully applied to play Atari games by Google’s DeepMind research group because of its powerful dynamic decision-making ability (Mnih et al. 2013, 2015; Silver et al. 2016, 2017, 2018; Schrittwieser et al. 2020).”
MethodA GPU-based computational framework that bridges neuron simulation and artificial intelligence
Zhang, Y., He, G., Ма, Л., Liu, X., Hjorth, J. J. J., Kozlov, A., He, Y., Zhang, S., Kotaleski, J. H., Tian, Y., et al. (2023) · Nature Communications · doi:10.1038/s41467-023-41553-7 · Cited by 46
“…“ backpropagation-of-error ” (backprop) algorithm achieve remarkable performance in specialized applications, even beating top human professional players in the games of Go and chess 14,15 , the human brain still outperforms ANNs in domains that involve more dynamic and noisy environments 16,17 .”
Challenges of Reinforcement Learning
Ding, Z., & Dong, H. (2020) · Springer eBooks · doi:10.1007/978-981-15-4095-0_7 · Cited by 92
Insightful artificial intelligence
Halina, M. (2021) · Mind & Language · doi:10.1111/mila.12321 · Cited by 40
“Although it is beyond the scope of this paper to consider the individual implications of these changes, together, they enable AlphaZero to not only achieve superhuman performance in chess, Go, and shogi, but defeat previous programmes, such as AlphaGo Lee, 100-0 (Silver et al., 2017).”
BackgroundMethodInfluentialAlphaBuilding ResCommunity: A multi-agent virtual testbed for community-level load coordination
Wang, Z., Chen, B., Li, H., & Hong, T. (2021) · Advances in Applied Energy · doi:10.1016/j.adapen.2021.100061 · Cited by 41
Stock Trading Strategies Based on Deep Reinforcement Learning
Li, Y., Liu, P., & Wang, Z. (2022) · Scientific Programming · doi:10.1155/2022/4698656 · Cited by 60
“Reinforcement learning gains increasing attention after AlphaZero defeated humans [7], has the ability of independent learning and decision-making, and has been successfully applied in the field of game playing [8, 9], unmanned driving [10, 11], and helicopter control [12].”
BackgroundMonte Carlo Tree Search: a review of recent modifications and applications
Świechowski, M., Godlewski, K., Sawicki, B., & Mańdziuk, J. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-022-10228-y · Cited by 365
“Chess-like games featuring many tactical traps pose a challenging problem for MCTS.”
BackgroundInfluentialRetrosynthetic planning with experience-guided Monte Carlo tree search
Hong, S., Zhuo, H. H., Jin, K., Shao, G., & Zhou, Z. (2023) · Communications Chemistry · doi:10.1038/s42004-023-00911-8 · Cited by 56
Leveraging deep learning to improve vaccine design
Hederman, A. P., & Ackerman, M. E. (2023) · Trends in Immunology · doi:10.1016/j.it.2023.03.002 · Cited by 44
Many-Objective Reinforcement Learning for Online Testing of DNN-Enabled Systems
Haq, F. U., Shin, D., & Briand, L. (2023) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse48619.2023.00155 · Cited by 48
“RL has widely been used to learn the sequence for completing a sequential decision-making task [11, 31].”
MethodComputational predictive approaches for interaction and structure of aptamers
Emami, N., Pakchin, P. S., & Ferdousi, R. (2020) · Journal of Theoretical Biology · doi:10.1016/j.jtbi.2020.110268 · Cited by 42
Chen, G., Yao, S., Ma, J., Pan, L., Chen, Y., Xu, P., Ji, J., & Chen, X. (2020) · Sensors · doi:10.3390/s20174836 · Cited by 47
Deep reinforcement learning for optical systems: A case study of mode-locked lasers
Sun, C., Kaiser, E., Brunton, S. L., & Kutz, J. N. (2020) · Machine Learning Science and Technology · doi:10.1088/2632-2153/abb6d6 · Cited by 40
“This is accomplished in chess and Go by self-play [14], which is exactly what the mode-locked laser is allowed to do to learn an optimal policy.”
BackgroundArtificial Intelligence (AI) in medicine as a strategic valuable tool
Larentzakis, A., & Lygeros, N. (2021) · Pan African Medical Journal · doi:10.11604/pamj.2021.38.184.28197 · Cited by 40
“chess, shogi or go, are considered an expression of human intellect at the highest level; however, DNNs as AlphaGo, AlphaGo Master, AlphaGo Zero mastered all those sharp games [10-13].”
BackgroundMonteFloor: Extending MCTS for Reconstructing Accurate Large-Scale Floor Plans
Stekovic, S., Rad, M., Fraundorfer, F., & Lepetit, V. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01573 · Cited by 43
“MCTS is a stochastic algorithm that efficiently explores search trees and has been used for example in AlphaGo and AlphaZero to select moves when playing Go or other games with high combinatorials [30].”
MethodRobotics, AI, and Humanity: Science, Ethics, and Policy
von Braun, J., Archer, M. S., Reichberg, G. M., & Sorondo, M. S. (2021) · Springer eBooks · doi:10.1007/978-3-030-54173-6 · Cited by 64
Temporal and state abstractions for efficient learning, transfer, and composition in humans.
Xia, L., & Collins, A. G. E. (2021) · Psychological Review · doi:10.1037/rev0000295 · Cited by 41
Yu, B., Qi, H., Guo, Q., Juefei-Xu, F., Xie, X., Ma, L., & Zhao, J. (2021) · IEEE Transactions on Reliability · doi:10.1109/tr.2021.3096332 · Cited by 78
Sequential predictive learning is a unifying theory for hippocampal representation and replayPreprint
Levenstein, D., Efremov, A., Eyono, R. H., Peyrache, A., & Richards, B. A. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.04.28.591528 · Cited by 48
“In self-supervised 66 and reinforcement learning 67 , sequential prediction is often learned via “rollouts”: prediction of multiple future timesteps, rather than masked input from the present timestep.”
BackgroundAI and Compute: How Much Longer Can Computing Power Drive Artificial Intelligence Progress?
Lohn, A., & Musser, M. (2022) · Center for Security and Emerging Technology · doi:10.51593/2021ca009 · Cited by 39
Towards Cognitive AI Systems: Workload and Characterization of Neuro-Symbolic AI
Wan, Z., Liu, C.-K., Yang, H., Raj, R., Li, C., You, H., Fu, Y., Wan, C., Samajdar, A., Lin, Y. C., et al. (2024) · 2024 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS) · doi:10.1109/ispass61541.2024.00033 · Cited by 39
Icard, T. (2019) · Mind · doi:10.1093/mind/fzz065 · Cited by 51
“For example, the world champion automated Go program, AlphaZero (Silver et al., 2018), randomizes its decision about what actions to explore in determining its next move.”
The Future of Collaborative Human-Artificial Intelligence Decision-Making for Mission Planning
Kase, S. E., Hung, C. P., Krayzman, T., Hare, J. Z., Rinderspacher, B. C., & Su, S. M. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.850628 · Cited by 43
“The success of recent AIs in games such as Go, Chess, Minecraft, and Monopoly (Silver et al., 2017; Goecks et al., 2021b; Haliem et al., 2021) are based on games with complete knowledge of the existing state of the world (i.e., “open” games), whereas wargaming typically includes incomplete (e.g., Starcraft), uncertain…”
BackgroundLinot, A. J., Zeng, K., & Graham, M. D. (2023) · International Journal of Heat and Fluid Flow · doi:10.1016/j.ijheatfluidflow.2023.109139 · Cited by 50
Mastering the Complex Assembly Task With a Dual-Arm Robot: A Novel Reinforcement Learning Method
Jiang, D., Wang, H., & Lu, Y. (2023) · IEEE Robotics & Automation Magazine · doi:10.1109/mra.2023.3262461 · Cited by 51
“As one of the most important branches of deep learning, DRL has made important breakthroughs in fields including video games, Go, and robot control [2].”
BackgroundFast preliminary design of low-thrust trajectories for multi-asteroid exploration
Fan, Z., Huo, M., Qi, N., Xu, Y., & Song, Z. (2019) · Aerospace Science and Technology · doi:10.1016/j.ast.2019.07.028 · Cited by 39
Will AI Replace Ophthalmologists?
Korot, E., Wagner, S. K., Faes, L., Liu, X., Huemer, J. C., Ferraz, D., Keane, P. A., & Balaskas, K. (2020) · Translational Vision Science & Technology · doi:10.1167/tvst.9.2.2 · Cited by 42
Deep Statistical Model Checking
Gros, T. P., Hermanns, H., Hoffmann, J., Klauck, M., & Steinmetz, M. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-50086-3_6 · Cited by 38
An Adaptive Model of Gaze-based Selection
Chen, X., Acharya, A., Oulasvirta, A., & Howes, A. (2021) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445177 · Cited by 61
“(3) It illustrates how machine learning can be used to model human cognition through trial-and-error learning in an artifcial world, rather than through generalisation from large amounts of human data (similar to [58]’s learning through selfplay).”
Background
Metadata
- Authors
- Silver, D., Hubert, T., Schrittwieser, J., Antonoglou, I., Lai, M., Guez, A., Lanctot, M., Sifre, L., Kumaran, D., Graepel, T., Lillicrap, T., Simonyan, K., & Hassabis, D.
- Affiliations
- Google DeepMind (United Kingdom) (GB)
- Corresponding author
- Silver, David
- Hubert, Thomas
- Schrittwieser, Julian
- Hassabis, Demis
- ORCID
- Venue
- Science
- Published
- 2018-12-07
- Volume
- 362
- Issue
- 6419
- Pages
- 1140-1144
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- OpenAlex
- W2902907165
- Cited by
- 3,562
- References
- 30
- Citations by year
- 2026: 257 · 2025: 427 · 2024: 512 · 2023: 514 · 2022: 569 · 2021: 546 · 2020: 462 · 2019: 256 · 2018: 13
- FWCI
- 179.89
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Science (publishedVersion)
- PubMed (publishedVersion)
- UCL Discovery (University College London) (submittedVersion)
- Funders
- DeepMind
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
- Keywords
- AlphaZero, chess, shogi, Go, evaluation function learning, game tree search
- MeSH terms
- Algorithms, Artificial Intelligence, Humans, Reinforcement, Psychology, Software, Video Games
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.