A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play
Open accessThe DOI matches its Crossref record; 30 of the 30 references it cites were found in a registry or an index; 5 of the 16 sources its claims cite were checked for support.
Abstract
One program to rule them all: Computers can beat humans at increasingly complex games, including chess and Go. However, these programs are typically constructed for a particular game, exploiting its properties, such as the symmetries of the board on which it is played. Silver et al. developed a program called AlphaZero, which taught itself to play Go, chess, and shogi (a Japanese version of chess) (see the Editorial, and the Perspective by Campbell). AlphaZero managed to beat state-of-the-art programs specializing in these three games. The ability of AlphaZero to adapt to various game rules is a notable step toward achieving a general game-playing system. Science , this issue p. 1140 ; see also pp. 1087 and 1118
Overview
CoChat Summary
TL;DR
AlphaZero is a single reinforcement-learning and MCTS algorithm that, given only game rules, learned superhuman play in chess, shogi, and Go through self-play. It defeated world-champion programs Stockfish, Elmo, and AlphaGo Zero using the same algorithm and network architecture across all three games.
Key findings
- In chess, AlphaZero defeated Stockfish 8, winning 155 games and losing 6 out of 1,000; it also won matches from common human openings and 2016 TCEC opening positions.
- In shogi, AlphaZero defeated Elmo, winning 98.2% of games as black and 91.2% overall; in Go, it defeated AlphaGo Zero, winning 61% of games.
- AlphaZero first outperformed Stockfish after 4 hours (300,000 steps), Elmo after 2 hours (110,000 steps), and AlphaGo Lee after 30 hours (74,000 steps); training ran for 700,000 steps, taking about 9 hours in chess, 12 hours in shogi, and 13 days in Go.
- AlphaZero searched only about 60,000 positions per second in chess and shogi, versus 60 million for Stockfish and 25 million for Elmo, yet still defeated Stockfish with 1/10 thinking time and won 46% against Elmo with 1/100 thinking time.
Methods
- AlphaZero uses a deep neural network \(f_\theta(s)\) that outputs move probabilities \(p\) and a scalar value \(v\), trained by reinforcement learning from self-play starting from random parameters; MCTS uses these outputs to guide search, with 800 simulations per move during training.
- The same algorithm, network architecture, and hyperparameters were used for chess, shogi, and Go, with no domain knowledge except game rules; no opening book, endgame tablebases, or domain-specific heuristics were used.
- Training used 5,000 first-generation TPUs to generate self-play games and 16 second-generation TPUs to train networks; evaluation matches used 3 hours per game plus 15 seconds per move, with opponents on 44 CPU cores and AlphaZero on four first-generation TPUs and 44 CPU cores.
Limitations
- Stated: AlphaZero did not use opening books, endgame tablebases, or domain-specific heuristics; the authors note that some such techniques could further improve performance but were left for future research.
- Stated: hyperparameters were reused across games without game-specific tuning, except for exploration noise and the learning rate schedule; matches against an opening book were low in diversity, and forced randomization increased losses.
- Evident from the text: results cover only three games with separate trained instances, no cross-game transfer is demonstrated, and evaluation compares AlphaZero on TPU hardware against CPU-based opponents, so hardware and time-control differences qualify direct strength comparisons.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
3 Valid · 2 Not valid · 8 Not checked · 43 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“Apart from the representation of positions and actions described above, AlphaZero uses the same network architecture as AlphaGo Zero ( 9 ), briefly recapitulated here.”
What the sources say
- Supportedcites [9] source
The Neural Network Architecture Comparison (p. 29) identifies AlphaGo Zero’s architecture as a residual tower with policy and value heads. The citation supplies the AlphaGo Zero architecture that the citing authors say AlphaZero uses; the comparison to AlphaZero is their own statement. (LLM, conf=0.88)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“and in Go, AlphaZero first outperformed AlphaGo Lee ( 9 ) after 30 hours (74,000 steps).”
What the sources say
- Partly supportedcites [9] sourceExperimental — accuracy not yet measured
Section 2, p. 7 reports that AlphaGo Zero outperformed AlphaGo Lee, but says this happened after 36 hours, not 30. The passage does not give 74,000 steps, and it names the system AlphaGo Zero rather than AlphaZero. (LLM, conf=0.90)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The training algorithm achieved similar performance in all independent runs (see fig. S3), suggesting that the high performance of AlphaZero's training algorithm is repeatable.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“AlphaZero also defeated Stockfish when given 1 / 10 as much thinking time as its opponent (i.e. searching ~ 1 / 10 , 000 as many positions),”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“and won 46% of games against Elmo when given 1 / 100 as much time (i.e. searching ~ 1 / 40 , 000 as many positions), see Fig. 2.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
23 more claims
Create a free account to see all 56 claims and to verify the 3 references still open.
- Probably not a substantial claim. 10
- It may state two separate claims as one. 23
- The quote only partly states this claim. 1
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Games have historically served as highly effective metrics for evaluating progress in deep reinforcement learning research”
From BALROG: Benchmarking Agentic LLM and VLM Reasoning On Games (2024) · cites this paper as Silver et al., 2018
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In 2017, AlphaGo, using the same learning algorithms in TD-Gammon and a billion times more computing power, soundly beat Ke Jie, the world Go champion, in a game that is to chess what chess is to backgammon in complexity (Silver et al., 2018).”
From Large Language Models and the Reverse Turing Test (2023) · cites this paper as Silver et al., 2018
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Most of the attention was on who won, but more important was how it won: AlphaGo was creative and had discovered new moves and board positions that were outside human experience.”
From Large Language Models and the Reverse Turing Test (2023) · cites this paper as Silver et al., 2018
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The primary method for applying more computation at inference, common in the AlphaZero family of agents2,72, is scaling the tree search and running multiple independent search attempts.”
From Olympiad-level formal mathematical reasoning with reinforcement learning (2026) · cites this paper as [2]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Modern AI can write computer programs (Li et al., 2022) or play chess at super-human level (Silver et al., 2018), but the ability of AI to perceive and act in the world remains far below human level.”
From Scaling Instructable Agents Across Many Simulated Worlds (2024) · cites this paper as Silver et al., 2018
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Self-citationAdvances in neural information processing systems 7
(1996) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(96)90463-0 · Cited by 14,281
Identity mappings in deep residual networks
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_38 · Cited by 10,741
Identity Mappings in Deep Residual Networks
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · arXiv · doi:10.48550/arxiv.1603.05027 · Cited by 10,665
Mastering the game of Go without human knowledge
Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., et al. (2017) · Nature · doi:10.1038/nature24270 · Cited by 9,084
Self-citationIn-Datacenter Performance Analysis of a Tensor Processing Unit
Jouppi, N. P., Young, C., Patil, N., Patterson, D., Agrawal, G., Bajwa, R., Bates, S., Bhatia, S., Boden, N., Borchers, A., et al. (2017) · Proceedings of the 44th Annual International Symposium on Computer Architecture · doi:10.1145/3079856.3080246 · Cited by 4,537
Encyclopedia of Artificial Intelligence
(2009) · IGI Global eBooks · doi:10.4018/978-1-59904-849-9 · Cited by 1,396
Campbell, M., Hoane, A. J., & Hsu, F.-H. (2002) · Artificial Intelligence · doi:10.1016/s0004-3702(01)00129-1 · Cited by 1,226
Lim, T.-S., Loh, W.-Y., & Shih, Y.-S. (2000) · Machine Learning · doi:10.1023/a:1007608224229 · Cited by 1,133
An analysis of alpha-beta pruning
Knuth, D. E., & Moore, R. W. (1975) · Artificial Intelligence · doi:10.1016/0004-3702(75)90019-3 · Cited by 1,004
Research works printed but not linked: 7
- C. J. Maddison A. Huang I. Sutskever D. Silver paper presented at the International Conference on Learning Representations 2015 San Diego CA 7 to 9 May 2015.
- D. N. L. Levy M. Newborn How Computers Play Chess (Ishi Press 2009).
- W. Steinitz The Modern Chess Instructor (Edition Olms 1990).
- O. Arenz “Monte Carlo chess ” master’s thesis Technische Universität Darmstadt (2012).
- 10.1023/a:1007634325138 · DOI
- T. Kaneko K. Hoki in Advances in Computer Games – 13th International Conference ACG 2011 Revised Selected Papers Tilburg Netherlands 20 to 22 November 2011 (Springer 2012) pp. 158–169.
- R. Ramanujan A. Sabharwal B. Selman in Proceedings of the 26th Conference on Uncertainty in Artificial Intelligence (UAI 2010) Catalina Island CA 8 to 11 July (AUAI Press 2010).
Cited by
Items 2,701–2,800 of 3,633
Visualizing MuZero ModelsPreprint
de Vries, J. A., Voskuil, K. S., Moerland, T. M., & Plaat, A. (2021) · Leiden Repository (Leiden University) · doi:10.48550/arxiv.2102.12924 · Cited by 12
“While AlphaZero used a known model of the environment, MuZero learned the model from sampled data.”
InfluentialLearning Chess Blindfolded: Evaluating Language Models on State Tracking.Preprint
Toshniwal, S., Wiseman, S., Livescu, K., & Gimpel, K. (2021) · arXiv (Cornell University) · Cited by 1
A Brief Summary of Interactions Between Meta-Learning and Self-Supervised LearningPreprint
Peng, H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.00845 · Cited by 5
“AlphaZero not only provides a general algorithm to play diverse games, but also improves model efficiency by only playing against the current best self-play games.”
BackgroundInfluentialAsymptotic Theory for IV-Based Reinforcement Learning with Potential EndogeneityPreprint
Li, J., Luo, Y., Wang, Z., & Zhang, X. (2021) · RePEc: Research Papers in Economics · doi:10.48550/arxiv.2103.04021 · Cited by 12
Kotlarz, P., Misheva, B. H., Posth, J.-A., et al. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3737714 · Cited by 1
Image Synthesis for Data Augmentation in Medical CT using Deep Reinforcement LearningPreprint
Krishna, A., Bartake, K., Niu, C., Wang, G., Lai, Y., Jia, X., & Mueller, K. D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.10493 · Cited by 3
“Our method is inspired by the recent successes of Deep Reinforcement Learning (DRL) [1, 2] in the game environments of Atari [3], Go and Chess [4] which all require the exploration of high-dimensional configuration spaces to form a competitive strategy from a given move.”
MethodInfluentialEvolutionary Strategies with Analogy Partitions in P-Guessing GamesPreprint
Vié, A. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3645059 · Cited by 1
“On the other hand, recent tools from computer science have shown ability to thrive in environments of high strategical complexity such as go, chess, shogi and wargames (Silver et al. (2016, 2017, 2018), Vinyals et al., 2019).”
BackgroundA Genetic Algorithm approach to Asymmetrical Blotto Games with Heterogeneous ValuationsPreprint
Vié, A. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3667055 · Cited by 1
“Recent progress from computer science has shown ability of alternative methods to thrive in environments of high strategical complexity such as go, chess, shogi and wargames [11, 12, 13, 14].”
Why is AI hard and Physics simple?Preprint
Roberts, D. A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.00008 · Cited by 22
Reward function shape exploration in adversarial imitation learning: an empirical studyPreprint
Wang, Y., & Li, X. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.06687 · Cited by 1
“Deep reinforcement learning (DRL) has achieved extraordinary performance in Chess and cards [1, 2], computer games [3-5], and robot manipulation [6] in recent years.”
BackgroundDecrypting Cryptic Crosswords: Semantically Complex Wordplay Puzzles as a Target for NLPPreprint
Rozner, J., Potts, C., & Mahowald, K. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.08620 · Cited by 25
InferNet for Delayed Reinforcement Tasks: Addressing the Temporal Credit Assignment ProblemPreprint
Ausin, M. S., Azizsoltani, H., Ju, S., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.00568 · Cited by 1
“For example, when applying DRL for games such as Chess, Shogi and Go, the final rewards are determined by the outcomes of the game: −1/0/+1 for loss/draw/win respectively; and for each state, Monte Carlo Tree Search (MCTS) was used to learn the likelihood of each outcome [21, 20].”
BackgroundMethodEvaluating Strategy Exploration in Empirical Game-Theoretic AnalysisPreprint
Wang, Y., Ma, G. Q., & Wellman, M. P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.10423 · Cited by 1
“DeepMind’s breakthroughs in Go and other two-player strategy games [17, 18] demonstrated the power of learning through self-play.”
BackgroundCandadai, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.05186 · Cited by 1
“With human- and super-human-level performance as the goal, such artificial systems have become masters of strategy games such as chess [Silver et al., 2018], Go [Silver et al.”
BackgroundTowards robust and domain agnostic reinforcement learning competitionsPreprint
Guss, W. H., Milani, S., Topin, N., Houghton, B., Mohanty, S. P., Melnik, A., Harter, A., Buschmaas, B., Jaster, B., Berganski, C., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.03748 · Cited by 1
“…successes of machine learning, such as AlphaStar (Vinyals et al., 2019a), AlphaGo (Silver et al., 2017), OpenAI Five (Berner et al., 2019), and their derivative systems (Silver et al., 2018), utilize deep RL to achieve human or super-human level performance in sequential decision-making tasks.”
BackgroundAgnostic Reinforcement Learning with Low-Rank MDPs and Rich ObservationsPreprint
Dann, C., Mansour, Y., Mohri, M., Sekhari, A., & Sridharan, K. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.11519 · Cited by 4
“…last decade, which include playing Atari 2600 video games at superhuman levels [Mnih et al., 2015], AlphaGo or AlphaGo Zero surpassing champions in Go [Silver et al., 2018], AlphaStar’s victory over top-ranked professional players in StarCraft [Vinyals et al., 2019], or practical self-driving cars.”
BackgroundAligning an optical interferometer with beam divergence control and continuous action spacePreprint
Makarenko, S., Sorokin, D., Ulanov, A. E., & Lvovsky, A. I. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.04457 · Cited by 6
“Reinforcement learning (RL) demonstrates incredible success in simulated environments, surpassing a human expert in Atari [1], chess, shogi and Go [2], as well as in more complicated games such as Dota2 [3] and StarCraft [4].”
BackgroundAutonomy 2.0: Why is self-driving always 5 years away?Preprint
Jain, A., Pero, L. D., Grimmett, H., & Ondrúška, P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.08142 · Cited by 56
Automatically trading small market capitalization cryptocurrencies using reinforcement learning
Akkerman, S. W. H. (2021) · Utrecht University Repository (Utrecht University) · Cited by 1
Back to Basics: Deep Reinforcement Learning in Traffic Signal ControlPreprint
Kanis, S., Samson, L., Bloembergen, D., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.07180 · Cited by 1
“In recent years, combining deep learning with reinforcement learning (RL) has led to strong performance in many complex environments, often achieving super-human performance [8, 12, 14].”
BackgroundSolving an elastic inverse problem using Convolutional Neural NetworksPreprint
Gokhale, N. H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.07859 · Cited by 1
“In recent years, neural networks have been applied to various applications such as image classification [4], hand written digit recognition [53], solving differential equations and symbolic integration [25], solving complex partial differential equations such as the NavierStokes equation [63], self-driving cars [3, 44…”
MethodMathematical Modeling of Human Learning and Decision Making
Xia, L. (2021) · eScholarship (California Digital Library) · Cited by 1
Ferber, P., Geißer, F., Trevizan, F., Helmert, M., & Hoffmann, J. (2021) · edoc (University of Basel) · doi:10.5451/unibas-ep84326 · Cited by 1
Reinforcement Learning In Two Player Zero Sum Simultaneous Action GamesPreprint
Phillips, P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.04835 · Cited by 1
“There has been extensive work focusing explicitly on two player games in strategic turn based setting such as Go, Chess, and Backgammon (Tesauro 1994; Silver et al. 2018).”
Fast and Data-Efficient Training of Rainbow: an Experimental Study on AtariPreprint
Schmidt, D. V., & Schmied, T. (2021) · arXiv (Cornell University) · doi:10.34726/3909 · Cited by 16
Vector Embedding Techniques for Player Behaviour in DOTA 2
Shah, J. (2021) · Carleton University's Institutional Repository (MacOdrum Library, Carleton University) · doi:10.22215/etd/2021-14680 · Cited by 2
Wang, Z., Meng, H., Zhou, Z., Feng, Y., Gao, Y., & Yu, C. (2022) · Research Square · doi:10.21203/rs.3.rs-1780336/v1 · Cited by 1
Auto: The Ethics Of Machine Learning Algorithms & Autonomous Vehicles
Furlin, S. (2022) · doi:10.32920/19749991 · Cited by 1
Dong, H., & Mu, Y. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4201849 · Cited by 1
A Data-efficiency Training Framework for Deep Reinforcement LearningPreprint
Feng, W., Han, C., Lian, F., & Liu, X. (2022) · Preprints.org · doi:10.20944/preprints202209.0483.v1
Recent Advances in Drug Design With Machine Learning
Faisal, M. (2022) · Wiley eBooks · doi:10.1002/9781394167258.ch6 · Cited by 1
Reinforcement Learning and Dynamical Systems
Lindenberg, B. (2023) · doi:10.15626/lud.494.2023 · Cited by 2
Floridi, L. (2023) · Oxford University Press eBooks · doi:10.1093/oso/9780198883098.002.0003 · Cited by 1
Hallucinating Machines: Exploring the ethical implications of generative language models
Hill, M. (2023) · Open Access Victoria University of Wellington | Te Herenga Waka (Figshare) · doi:10.26686/wgtn.24180456 · Cited by 7
Wang, Y., Liu, H., Zhang, M., et al. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4583195 · Cited by 1
Stretcher: A new way to improve the efficiency of explorationPreprint
Wang, J., Zhu, L., Wang, Y., Ji, Y., Ren, J., & Yang, H. (2023) · Research Square · doi:10.21203/rs.3.rs-3739252/v1 · Cited by 1
New Restrictions on AI from Physics: The Most Reliable Way to Predict AGI future?Preprint
Sinitskiy, A. V. (2023) · doi:10.36227/techrxiv.170326661.10274372/v1 · Cited by 1
Chen, Q., Kuang, Z., Liu, X., et al. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4680162 · Cited by 1
Exploring Parity Challenges in Reinforcement Learning Through Curriculum Learning with Noisy LabelsPreprint
Zhou, B., & Riis, S. K. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4693751 · Cited by 3
The Mathematics of DeepMind ModelsPreprint
Alonso, M. N. I. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5007464 · Cited by 2
Reinforcement Learning in Mathematical ResearchPreprint
Alonso, M. N. I. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5015094 · Cited by 1
Self-Learning AI for Adaptive Humanoid Robotics: Towards Generalized AutonomyPreprint
Khemani, K. (2025) · doi:10.22541/au.174802371.13279686/v1 · Cited by 1
Accelerating Virtual Directed Evolution of Proteins via Reinforcement LearningPreprint
Mi, T., Wang, Y., Zhao, J., Wang, W., Shen, Y., Xiao, N., Chen, L., Chen, G., Zhang, S., Zhang, W., et al. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.06.25.661516 · Cited by 4
WITHDRAWNPreprint
Prestes, J. A. D. L. (2025) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/p9d7h_v2 · Cited by 1
Lost in the Interface: How Structured UI Complexity Challenges Large Vision Language Models in Games
Li, X., Togo, R., Maeda, K., et al. (2025) · SIGGRAPH Asia Technical Communications · doi:10.1145/3757376.3771402 · Cited by 1
Reinforcement Learning for Multi-Function Radar Resource Management
Vincent, M. (2023) · theses.fr (ABES) · doi:10.70675/945956b6z2536z4cc4za84czf6468b84a887 · Cited by 4
TFTune: Creation and Personalization of Pointing Transfer Functions Using Reinforcement Learning
Eddy, E., Campbell, E., Scheme, E., Bateman, S., & Casiez, G. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790291
Reading Depth Does Not Transfer: The Distillation Ceiling in Computer GoPreprint
Kojima, T. (2026) · Research Square · doi:10.21203/rs.3.rs-9966155/v1 · Cited by 1
Nested Learning: The Illusion of Deep Learning Architectures
Behrouz, A., Razaviyayn, M., Zhong, P., & Mirrokni, V. (2025) · arXiv · doi:10.48550/arxiv.2512.24695 · Cited by 37
“Thereafter, deep learning has been an inseparable part of the large-scale computational models with seminal success in chemistry and biology [10], games [11, 12], computer vision [13, 14], and multimodal and natural language understanding [15–17].”
The Red Queen Gödel Machine: Co-Evolving Agents and Their Evaluators
Iacob, A., Jovanović, A., Shen, W. F., Burkhardt, D., Kurmanji, M., Tastan, N., Sani, L., Venanzi, N. A. E., Nie, J., Odonnat, A., et al. (2026) · arXiv · doi:10.48550/arxiv.2606.26294 · Cited by 1
“C.2).”
Yamakawa, H. (2021) · Neural Networks · doi:10.1016/j.neunet.2021.09.004 · Cited by 32
“However, it is obvious that a mesoscopic level of granularity that relies on physical measures to determine the voxel size of fMRI measurements is inappropriate.”
BackgroundScheduling and Power Control for Wireless Multicast Systems via Deep Reinforcement Learning
Raghu, R., Panju, M., Aggarwal, V., & Sharma, V. (2021) · Entropy · doi:10.3390/e23121555 · Cited by 7
Indoor Point-to-Point Navigation with Deep Reinforcement Learning and Ultra-Wideband
Chiaberge, M., Fantin, G., Salvetti, F., Mazzia, V., & Sutera, E. (2021) · PORTO@Iris · doi:10.5220/0010202600380047 · Cited by 5
“esults of the second set of tests: comparison with human control. noise. In literature, RL agents are frequently compared to human agents to prove their control performance in complex tasks [7], [23]–[25]. We perform such comparison by putting several people in the same experimental conditions of the RL agent. Human t…”
BackgroundMulti-Agent Reinforcement Learning
(2001) · The CRC Press international series on computational intelligence/International series on computational intelligence · doi:10.1201/9781420038835.ch6
1215 Universelle Spielprogramme
Thielscher, M. (2026) · doi:10.1515/9783111386690-006
Multi-Agent Reinforcement Learning
(2001) · The CRC Press international series on computational intelligence/International series on computational intelligence · doi:10.1201/9781420038835.ch8
Binz, M. (2002) · CRC Press eBooks · doi:10.1201/9780203164587.ch3
The Entropy of Artificial Intelligence and a Case Study of AlphaZero from Shannon's PerspectivePreprint
Zhang, B., Chen, B., & Peng, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.05794
“If the two curves intersects other than (0,0) and (1,1), the learning process generally cannot reach a global optimum.”
BackgroundInfluentialBiasing MCTS with Features for General GamesPreprint
Soemers, D. J. N. J., Piette, É., & Browne, C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.08942
Network Horizon Dynamics I: Qualitative AspectsPreprint
Dribus, B. F., Sumner, A. C., Bist, K., Regmi, N., Sircar, J., & Upreti, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.10268
“Famous examples include image analysis [37], speech recognition [79], vehicle navigation [80], and game strategy [81, 82].”
BackgroundCommentary: I am not throwing away my shot…to predict when your patient will decompensate
Murphy, L. D., Cooper, D. S., & Mah, K. E. (2019) · Journal of Thoracic and Cardiovascular Surgery · doi:10.1016/j.jtcvs.2019.03.111
“Today, after learning chess by simply receiving the rules of the game and driven by deep learning, AlphaZero did not lose a single match against the preeminent brute force computer.(8) AlphaZero taught itself by playing millions of games against itself and now is regarded as unbeatable, often playing with intuition an…”
BackgroundTowards Efficient and Secure Delivery of Data for Training and Inference with Privacy-PreservingPreprint
Shen, J., Liu, J., Chen, Y., & Li, H. H. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.09968 · Cited by 2
Teaching Agents Through Correction
Appelgren, M. (2019) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/sadp7808
On modelling the emergence of logical thinkingPreprint
Ivan, C., & Indurkhya, B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.09730 · Cited by 2
“One can argue in favour of the AlphaZero (Silver et al. 2018) system that it somehow successfully incorporated all these qualities of intelligence.”
ResultTowards a data-driven framework for realistic self-organized virtual humans
Yang, Z., & Bailey, R. (2019) · ACM Symposium on Eye Tracking Research and Applications (ETRA) · doi:10.1145/3314111.3322874
“Deep RL has made impressive progress and achieved near-human or above-human level performance in areas such as video game playing [Mnih et al. 2015] and mastering board games [Silver et al. 2016, 2018, 2017].”
BackgroundExtractive Summarization via Weighted Dissimilarity and Importance Aligned Key Iterative AlgorithmPreprint
Sawada, R. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.02126
Virtual-to-Real Transfer via Dynamics Models (poster)
Jang, S., & Son, Y.-S. (2019) · International Conference on Mobile Systems, Applications, and Services · doi:10.1145/3307334.3328602
“Several researches are ongoing in the field of but not limited to drone, vehicle, and robot arm control as the deep reinforcement learning is proven to be an effective solution to sequential decision-making problems such as Atari games [2] and several board games including Go [4].”
BackgroundComparison of Deep Neural Networks and Deep Hierarchical Models for Spatio-Temporal DataPreprint
Wikle, C. K. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.08321
Chess endgame news: an endgame challenge for neural nets
Haworth, G. (2019) · ICGA Journal · doi:10.3233/icg-190109
Zhang, S., Zhang, J., Chen, L., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.03060
“Reinforcement learning (RL) as one of the most powerful machine learning approaches [37,38,39,48], which is rooted in psychology and neuroscience, has been widely used to solve the problems in terms of states, actions, rewards, and decision making in various environments through exploratory trials [37, 53,54].”
BackgroundFifth Aeon – A.I Competition and Balancer
Ritson, W. M. (2019) · DigitalCommons - CalPoly (California State Polytechnic University) · doi:10.15368/theses.2019.56
Wield: Systematic Reinforcement Learning With Progressive RandomizationPreprint
Schaarschmidt, M., Fricke, K., & Yoneki, E. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.06844 · Cited by 2
“0) ∗ Computation graph rewriting (Paliwal et al., 2019) Memory usage C 5 ( n = 4 × 10 5 , s = 20 , f =?) AlphaZero (Silver et al., 2018) Win game of chess C 6 ( n ≈ 10 9 , s =? , f =?) and C 2 , weight initialization and workload instance (e.g. a set of jobs sampled from a workload distribution) are incrementally rand…”
BackgroundTowards Efficient and Secure Delivery of Data for Deep Learning with Privacy-PreservingPreprint
Shen, J., Liu, J., Chen, Y., & Li, H. H. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.07632 · Cited by 6
“In recent years, deep learning has become the driving force for many artificial intelligence applications such as image classification [1], object detection [2], speed recognition [3], game intelligence [4], etc.”
BackgroundA machine learning method correlating pulse pressure wave data with pregnancyPreprint
Chen, J., Huang, H., Hao, W., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.01726
The Choice Function Framework for Online Policy Improvement
Issakkimuthu, M., Fern, A., & Tadepalli, P. V. (2020) · AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) · doi:10.1609/aaai.v34i06.6578
“commonly used policies as a form of knowledge to guide and prune the search, often as part of the rollout policy applied at the leaves [4]. Recent, high-profile examples include AlphaGo and AlphaZero [14, 16, 15], which combine a learned base policy and value function to guide MCTS. One view of AlphaZero’s search appro…”
BackgroundMethodRosenbloom, P. S. (2019) · eScholarship (California Digital Library) · Cited by 1
Evaluation-Function Modeling with Neural Networks for RoboCup Soccer
Fukushima, T., Nakashima, T., & Akiyama, H. (2019) · IEEJ Transactions on Electronics Information and Systems · doi:10.1541/ieejeiss.139.1128
Relation learning in a neurocomputational architecture supports cross-domain transferPreprint
Doumas, L. A. A., Puebla, G., Martin, A. E., & Hummel, J. E. (2019) · arXiv (Cornell University) · Cited by 5
Similarity-guided Monte Carlo Tree Search for bio-retrosynthesisPreprint
Koch, M., Duigou, T., & Faulon, J. (2019) · Research Explorer (The University of Manchester) · doi:10.1101/800474
“RetroPath 3.0’s search algorithm relies on Monte Carlo Tree Search (MCTS), which has already revolutionised the field of Artificial Intelligence, as illustrated by the stunning victory of Google's AI (AlphaGo) against a Go master in 2016 (Silver et al. 2016, 2017, 2018).”
MethodInfluentialKastner, D. B., Miller, E. A., Yang, Z., Roumis, D. K., Liu, D. F., Frank, L. M., & Dayan, P. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/808006
Deep Reinforcement Learning for Autonomous Model-Free Navigation with Partial Observability
Tapia, D., Parras, J., & Zazo, S. (2019) · European Signal Processing Conference (EUSIPCO) · doi:10.23919/eusipco.2019.8902933
“performance in games like Atari [18] or Go [19].”
BackgroundLessons from reinforcement learning for biological representations of space
Muryy, A., Siddharth, N., Nardelli, N., Glennerster, A., & Torr, P. H. S. (2020) · Vision Research · doi:10.1016/j.visres.2020.05.009 · Cited by 5
“Drl methods have been successful in solving complex tasks such as Go and other popular board games [38, 39], and have proved to be necessary to tackle decision making tasks with high-dimensional or visual state representation [33, 40].”
MethodEvolving ab initio trading strategies in heterogeneous environmentsPreprint
Dewhurst, D. R., Li, Y., Bogdan, A., et al. (2019) · RePEc: Research Papers in Economics · doi:10.48550/arxiv.1912.09524
“Recent years have seen substantial progress toward this goal [49, 56, 57].”
BackgroundImprove the performance of neural network training with accurate information
Huang, S.-H., Chen, C.-H., & Lin, S. (2019) · International Conference on Advanced Information Science and System (AISS) · doi:10.1145/3373477.3373703
Deep Reinforcement Learning for Complex Manipulation Tasks with Sparse FeedbackPreprint
Manela, B. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.03877 · Cited by 3
“Deep reinforcement learning, the combination of reinforcement learning with deep learning [11] has led to many breakthroughs in recent years in generating goal-directed behavior in artificial agents ranging from playing Atari games without prior knowledge and human guidance [12], to teaching an animated humanoid agent…”
BackgroundAre Armageddon chess games implemented fairly?
Vargas-Calderón, V. (2020) · ICGA Journal · doi:10.3233/icg-190134
Deep RL Agent for a Real-Time Action Strategy GamePreprint
Warchalski, M., Radojevic, D., & Milosevic, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.06290 · Cited by 1
“The second approach that we take is self-play, which recently became popular in combination with deep reinforcement learning (Silver et al. 2016; 2017a; 2017b; 2018; Vinyals et al. 2019; OpenAI et al. 2019).”
BackgroundMethodInfluentialBack Prediction in the Game of Go
Tang, J., & Kaneko, T. (2019) · Medical Entomology and Zoology
Libera, A. D., Romeres, D., Jha, D. K., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.10621
“RL algorithms have been able to reach and exceed human-level performance in several benchmark problems, such as playing the games of chess, go and shogi [1].”
BackgroundReinforcement Learning of Risk-Constrained Policies in Markov Decision ProcessesPreprint
Brázdil, T., Chatterjee, K., Novotný, P., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.12086
“We follow the spirit of (Silver et al. 2018) and use the predictor fθ to evaluate the leaf nodes.”
MethodDefogGAN: Predicting Hidden Information in the StarCraft Fog of War with Generative Adversarial NetsPreprint
Jeong, Y., Choi, H., Kim, B., & Gwon, Y. (2020) · AAAI Conference on Artificial Intelligence · doi:10.48550/arxiv.2003.01927 · Cited by 7
Two-Phase-Win Strategy for Improving the AlphaZero’s Strength
Chen, C.-H., Chen, Y., & Lin, S. (2019) · World Symposium on Communication Engineering (WSCE) · doi:10.1109/wsce49000.2019.9041112
“Furthermore, in 2018, AlphaZero [5] was proposed to learn how to play the games from scratch without human knowledge.”
BackgroundCounterfactual policy introspection using structural causal models
Oberst, M. (2019) · DSpace@MIT (Massachusetts Institute of Technology)
Miravète, S., & Rei, A. L.-D. P. (2020) · Les cahiers internationaux de psychologie sociale · doi:10.3917/cips.123.0193
On Building Generalizable Learning Agents
Wu, Y. (2019) · UC Berkeley
Tang, Z., Zhu, Y., Zhao, D., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.13949
“[3] D. Silver et al., “A general reinforcement learning algorithm that masters Chess, Shogi, and Go through self-play,” Science, vol. 362, no. 6419, pp. 1140–1144, 2018.”
MethodApplication of Internet of Things to Live Logging of Chinese Chess
Wen, J., Liu, Z., Shen, Z., et al. (2020) · International Conference on Signal Processing and Integrated Networks (SPIN) · doi:10.1109/spin48934.2020.9070987
“Google have reported that their approach for Go is sufficiently general to master a wide range of games [6].”
BackgroundDeep Neural Mobile NetworkingPreprint
Zhang, C. (2020) · arXiv (Cornell University) · doi:10.7488/era/356 · Cited by 3
“Inspired by remarkable achievements in Atari [23] and Go [360] games, a number of researchers begin to explore DRL to solve complex network control problems.”
BackgroundThe Guiding Role of Reward Based on Phased Goal in Reinforcement Learning
Liu, Y., & Hu, Z. (2020) · International Conference on Machine Learning and Computing (ICMLC) · doi:10.1145/3383972.3384039
“Deep Reinforcement Learning has recently attracted great attention in various domains such as Atari[1] and Go[2] because of its successful performance.”
Background
Metadata
- Authors
- Silver, D., Hubert, T., Schrittwieser, J., Antonoglou, I., Lai, M., Guez, A., Lanctot, M., Sifre, L., Kumaran, D., Graepel, T., Lillicrap, T., Simonyan, K., & Hassabis, D.
- Affiliations
- Google DeepMind (United Kingdom) (GB)
- Corresponding author
- Silver, David
- Hubert, Thomas
- Schrittwieser, Julian
- Hassabis, Demis
- ORCID
- Venue
- Science
- Published
- 2018-12-07
- Volume
- 362
- Issue
- 6419
- Pages
- 1140-1144
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- OpenAlex
- W2902907165
- Cited by
- 3,562
- References
- 30
- Citations by year
- 2026: 257 · 2025: 427 · 2024: 512 · 2023: 514 · 2022: 569 · 2021: 546 · 2020: 462 · 2019: 256 · 2018: 13
- FWCI
- 179.89
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Science (publishedVersion)
- PubMed (publishedVersion)
- UCL Discovery (University College London) (submittedVersion)
- Funders
- DeepMind
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
- Keywords
- AlphaZero, chess, shogi, Go, evaluation function learning, game tree search
- MeSH terms
- Algorithms, Artificial Intelligence, Humans, Reinforcement, Psychology, Software, Video Games
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.