CoChat

A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play

Open access

The DOI matches its Crossref record; 30 of the 30 references it cites were found in a registry or an index; 5 of the 16 sources its claims cite were checked for support.

Abstract

One program to rule them all: Computers can beat humans at increasingly complex games, including chess and Go. However, these programs are typically constructed for a particular game, exploiting its properties, such as the symmetries of the board on which it is played. Silver et al. developed a program called AlphaZero, which taught itself to play Go, chess, and shogi (a Japanese version of chess) (see the Editorial, and the Perspective by Campbell). AlphaZero managed to beat state-of-the-art programs specializing in these three games. The ability of AlphaZero to adapt to various game rules is a notable step toward achieving a general game-playing system. Science , this issue p. 1140 ; see also pp. 1087 and 1118

Overview

CoChat Summary

TL;DR

AlphaZero is a single reinforcement-learning and MCTS algorithm that, given only game rules, learned superhuman play in chess, shogi, and Go through self-play. It defeated world-champion programs Stockfish, Elmo, and AlphaGo Zero using the same algorithm and network architecture across all three games.

Key findings

  • In chess, AlphaZero defeated Stockfish 8, winning 155 games and losing 6 out of 1,000; it also won matches from common human openings and 2016 TCEC opening positions.
  • In shogi, AlphaZero defeated Elmo, winning 98.2% of games as black and 91.2% overall; in Go, it defeated AlphaGo Zero, winning 61% of games.
  • AlphaZero first outperformed Stockfish after 4 hours (300,000 steps), Elmo after 2 hours (110,000 steps), and AlphaGo Lee after 30 hours (74,000 steps); training ran for 700,000 steps, taking about 9 hours in chess, 12 hours in shogi, and 13 days in Go.
  • AlphaZero searched only about 60,000 positions per second in chess and shogi, versus 60 million for Stockfish and 25 million for Elmo, yet still defeated Stockfish with 1/10 thinking time and won 46% against Elmo with 1/100 thinking time.

Methods

  • AlphaZero uses a deep neural network \(f_\theta(s)\) that outputs move probabilities \(p\) and a scalar value \(v\), trained by reinforcement learning from self-play starting from random parameters; MCTS uses these outputs to guide search, with 800 simulations per move during training.
  • The same algorithm, network architecture, and hyperparameters were used for chess, shogi, and Go, with no domain knowledge except game rules; no opening book, endgame tablebases, or domain-specific heuristics were used.
  • Training used 5,000 first-generation TPUs to generate self-play games and 16 second-generation TPUs to train networks; evaluation matches used 3 hours per game plus 15 seconds per move, with opponents on 44 CPU cores and AlphaZero on four first-generation TPUs and 44 CPU cores.

Limitations

  • Stated: AlphaZero did not use opening books, endgame tablebases, or domain-specific heuristics; the authors note that some such techniques could further improve performance but were left for future research.
  • Stated: hyperparameters were reused across games without game-specific tuning, except for exploration noise and the learning rate schedule; matches against an opening book were low in diversity, and forced randomization increased losses.
  • Evident from the text: results cover only three games with separate trained instances, no cross-game transfer is demonstrated, and evaluation compares AlphaZero on TPU hardware against CPU-based opponents, so hardware and time-control differences qualify direct strength comparisons.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1126/science.aar6404

Cites

30 of 30 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Figure 1 Training AlphaZero for 700,000 steps. Elo ratings were computed from games between different players where each player was given one second per move. (A) Performance of AlphaZero in chess, compared with the 2016 TCEC world-champion program Stockfish. (B) Performance of AlphaZero in shogi, compared with the 2017 CSA world-champion program Elmo. (C) Performance of AlphaZero in Go, compared with AlphaGo Lee and AlphaGo Zero (20 blocks over 3 days).
Figure 1 Training AlphaZero for 700,000 steps. Elo ratings were computed from games between different players where each player was given one second per move. (A) Performance of AlphaZero in chess, compared with the 2016 TCEC world-champion program Stockfish. (B) Performance of AlphaZero in shogi, compared with the 2017 CSA world-champion program Elmo. (C) Performance of AlphaZero in Go, compared with AlphaGo Lee and AlphaGo Zero (20 blocks over 3 days).From the PDF
Figure 2 Comparison with specialized programs. (A) Tournament evaluation of AlphaZero in chess, shogi, and Go in matches against respectively Stockfish, Elmo, and the previously published version of AlphaGo Zero (AG0) that was trained for 3 days. In the top bar, AlphaZero plays white; in the bottom bar AlphaZero plays black. Each bar shows the results from AlphaZero's perspective: win ('W', green), draw ('D', grey), loss ('L', red). (B) Scalability of AlphaZero with thinking time, compared to Stockfish and Elmo. Stockfish and Elmo always receive full time (3 hours per game plus 15 seconds per move), time for AlphaZero is scaled down as indicated. (C) Extra evaluations of AlphaZero in chess against the most recent version of Stockfish at the time of writing ( 27 ), and against Stockfish with a strong opening book ( 28 ). Extra evaluations of AlphaZero in shogi were carried out against another strong shogi program Aperyqhapaq ( 29 ) at full time controls and against Elmo under 2017 CSA world championship time controls (10 minutes per game plus 10 seconds per move). (D) Average result of chess matches starting from different opening positions: either common human positions (see also Fig. 3), or the 2016 TCEC world championship opening positions (see also fig. S4). Average result of shogi matches starting from common human positions (see also Fig. 3). CSA world championship games start from the initial board position. Match conditions are summarized in tables S8 and S9. 7
Figure 2 Comparison with specialized programs. (A) Tournament evaluation of AlphaZero in chess, shogi, and Go in matches against respectively Stockfish, Elmo, and the previously published version of AlphaGo Zero (AG0) that was trained for 3 days. In the top bar, AlphaZero plays white; in the bottom bar AlphaZero plays black. Each bar shows the results from AlphaZero's perspective: win ('W', green), draw ('D', grey), loss ('L', red). (B) Scalability of AlphaZero with thinking time, compared to Stockfish and Elmo. Stockfish and Elmo always receive full time (3 hours per game plus 15 seconds per move), time for AlphaZero is scaled down as indicated. (C) Extra evaluations of AlphaZero in chess against the most recent version of Stockfish at the time of writing ( 27 ), and against Stockfish with a strong opening book ( 28 ). Extra evaluations of AlphaZero in shogi were carried out against another strong shogi program Aperyqhapaq ( 29 ) at full time controls and against Elmo under 2017 CSA world championship time controls (10 minutes per game plus 10 seconds per move). (D) Average result of chess matches starting from different opening positions: either common human positions (see also Fig. 3), or the 2016 TCEC world championship opening positions (see also fig. S4). Average result of shogi matches starting from common human positions (see also Fig. 3). CSA world championship games start from the initial board position. Match conditions are summarized in tables S8 and S9. 7From the PDF
Figure 3 Matches starting from the most popular human openings . AlphaZero plays against (A) Stockfish in chess and (B) Elmo in shogi. In the left bar, AlphaZero plays white, starting from the given position; in the right bar AlphaZero plays black. Each bar shows the results from AlphaZero's perspective: win (green), draw (grey), loss (red). The percentage frequency of self-play training games in which this opening was selected by AlphaZero is plotted against the duration of training, in hours. 8
Figure 3 Matches starting from the most popular human openings . AlphaZero plays against (A) Stockfish in chess and (B) Elmo in shogi. In the left bar, AlphaZero plays white, starting from the given position; in the right bar AlphaZero plays black. Each bar shows the results from AlphaZero's perspective: win (green), draw (grey), loss (red). The percentage frequency of self-play training games in which this opening was selected by AlphaZero is plotted against the duration of training, in hours. 8From the PDF
Figure 4 AlphaZero's search procedure. The search is illustrated for a position (inset) from game 1 (table S6) between AlphaZero (white) and Stockfish (black) after 29. ... Qf8. The internal state of AlphaZero's MCTS is summarized after 10 2 , ..., 10 6 simulations. Each summary shows the 10 most visited states. The estimated value is shown in each state, from white's perspective, scaled to the range [0 , 100] . The visit count of each state, relative to the root state of that tree, is proportional to the thickness of the border circle. AlphaZero considers 30. c6 but eventually plays 30. d5.
Figure 4 AlphaZero's search procedure. The search is illustrated for a position (inset) from game 1 (table S6) between AlphaZero (white) and Stockfish (black) after 29. ... Qf8. The internal state of AlphaZero's MCTS is summarized after 10 2 , ..., 10 6 simulations. Each summary shows the 10 most visited states. The estimated value is shown in each state, from white's perspective, scaled to the range [0 , 100] . The visit count of each state, relative to the root state of that tree, is proportional to the thickness of the border circle. AlphaZero considers 30. c6 but eventually plays 30. d5.From the PDF
Figure S1 Learning curves showing the Elo performance during training in Go. Comparison between AlphaZero, a version of AlphaZero that exploits knowledge of symmetries in a similar manner to AlphaGo Zero, and the previously published AlphaGo Zero. AlphaZero generates approximately 1 / 8 as many positions per training step, and therefore uses eight times more wall clock time, than the symmetry-augmented algorithms.
Figure S1 Learning curves showing the Elo performance during training in Go. Comparison between AlphaZero, a version of AlphaZero that exploits knowledge of symmetries in a similar manner to AlphaGo Zero, and the previously published AlphaGo Zero. AlphaZero generates approximately 1 / 8 as many positions per training step, and therefore uses eight times more wall clock time, than the symmetry-augmented algorithms.From the PDF
Figure S2 Chess and shogi openings preferred by AlphaZero at different stages of selfplay training , labelled with the number of training steps. The figure shows the most frequently selected opening 6 plies played by AlphaZero during its games of self-play. Each move was generated by an MCTS with just 800 simulations per move.
Figure S2 Chess and shogi openings preferred by AlphaZero at different stages of selfplay training , labelled with the number of training steps. The figure shows the most frequently selected opening 6 plies played by AlphaZero during its games of self-play. Each move was generated by an MCTS with just 800 simulations per move.From the PDF
Figure S3 Repeatability of AlphaZero training on the game of chess. The figure shows 6 separate training runs of 400,000 steps (approximately 4 hours each). Elo ratings were computed from a tournament between baseline players and AlphaZero players at different stages of training. AlphaZero players were given 800 simulations per move. Similar repeatability was observed in shogi and Go.
Figure S3 Repeatability of AlphaZero training on the game of chess. The figure shows 6 separate training runs of 400,000 steps (approximately 4 hours each). Elo ratings were computed from a tournament between baseline players and AlphaZero players at different stages of training. AlphaZero players were given 800 simulations per move. Similar repeatability was observed in shogi and Go.From the PDF
Figure S4 Chess matches beginning from the 2016 TCEC world championship start positions. In the left bar, AlphaZero plays white, starting from the given position; in the right bar AlphaZero plays black. Each bar shows the results from AlphaZero's perspective: win (green), draw (grey), loss (red). Many of these start positions are unbalanced according to both and AlphaZero and Stockfish, resulting in more losses for both players.
Figure S4 Chess matches beginning from the 2016 TCEC world championship start positions. In the left bar, AlphaZero plays white, starting from the given position; in the right bar AlphaZero plays black. Each bar shows the results from AlphaZero's perspective: win (green), draw (grey), loss (red). Many of these start positions are unbalanced according to both and AlphaZero and Stockfish, resulting in more losses for both players.From the PDF
Image 9
From the PDF

Verification

Claims the paper makes

3 Valid · 2 Not valid · 8 Not checked · 43 Own results and interpretations

23 more claims

Create a free account to see all 56 claims and to verify the 3 references still open.

See all claims — free account
  • Probably not a substantial claim. 10
  • It may state two separate claims as one. 23
  • The quote only partly states this claim. 1

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

1 more claim about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Citations

Cites

Show all 30 references

Research works printed but not linked: 7

  1. C. J. Maddison A. Huang I. Sutskever D. Silver paper presented at the International Conference on Learning Representations 2015 San Diego CA 7 to 9 May 2015.
  2. D. N. L. Levy M. Newborn How Computers Play Chess (Ishi Press 2009).
  3. W. Steinitz The Modern Chess Instructor (Edition Olms 1990).
  4. O. Arenz “Monte Carlo chess ” master’s thesis Technische Universität Darmstadt (2012).
  5. 10.1023/a:1007634325138 · DOI
  6. T. Kaneko K. Hoki in Advances in Computer Games – 13th International Conference ACG 2011 Revised Selected Papers Tilburg Netherlands 20 to 22 November 2011 (Springer 2012) pp. 158–169.
  7. R. Ramanujan A. Sabharwal B. Selman in Proceedings of the 26th Conference on Uncertainty in Artificial Intelligence (UAI 2010) Catalina Island CA 8 to 11 July (AUAI Press 2010).

Cited by

See all 3,633

Metadata

Authors
Silver, D., Hubert, T., Schrittwieser, J., Antonoglou, I., Lai, M., Guez, A., Lanctot, M., Sifre, L., Kumaran, D., Graepel, T., Lillicrap, T., Simonyan, K., & Hassabis, D.
Affiliations
Google DeepMind (United Kingdom) (GB)
Corresponding author
  • Silver, David
  • Hubert, Thomas
  • Schrittwieser, Julian
  • Hassabis, Demis
Venue
Science
Published
2018-12-07
Volume
362
Issue
6419
Pages
1140-1144
ISSN
  • 0036-8075 (ISSN-L)
  • 1095-9203
OpenAlex
W2902907165
Cited by
3,562
References
30
Citations by year
2026: 257 · 2025: 427 · 2024: 512 · 2023: 514 · 2022: 569 · 2021: 546 · 2020: 462 · 2019: 256 · 2018: 13
FWCI
179.89
Type
journal article
Language
en
Open access
Locations
  • Science (publishedVersion)
  • PubMed (publishedVersion)
  • UCL Discovery (University College London) (submittedVersion)
Funders
DeepMind
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
  • Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
  • Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
Keywords
AlphaZero, chess, shogi, Go, evaluation function learning, game tree search
MeSH terms
Algorithms, Artificial Intelligence, Humans, Reinforcement, Psychology, Software, Video Games

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.