The DOI matches its Crossref record; 220 of the 233 references it cites were found in a registry or an index.
A Survey of Monte Carlo Tree Search Methods
Abstract
Monte Carlo tree search (MCTS) is a recently proposed search method that combines the precision of tree search with the generality of random sampling. It has received considerable interest due to its spectacular success in the difficult problem of computer Go, but has also proved beneficial in a range of other domains. This paper is a survey of the literature to date, intended to provide a snapshot of the state of the art after the first five years of MCTS research. We outline the core algorithm's derivation, impart some structure on the many variations and enhancements that have been proposed, and summarize the results from the key game and nongame domains to which MCTS methods have been applied. A number of open research questions indicate that the field is ripe for future work.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“RL algorithms such as Deep Q-Network (DQN) [9], Proximal Policy Optimization (PPO) [10], and Monte Carlo Tree Search (MCTS) [11] disregard the previous trajectories.”
From Playtesting: What is Beyond Personas (2023) · cites this paper as [11]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Artificial intelligence: a modern approach
(1995) · Choice Reviews Online · doi:10.5860/choice.33-1577 · Cited by 21,974
Artificial intelligence: A modern approach
Nilsson, N. J. (1996) · Artificial Intelligence · doi:10.1016/0004-3702(96)00007-0 · Cited by 10,583
Finite-time Analysis of the Multiarmed Bandit Problem
Auer, P., Cesa-Bianchi, N., & Fischer, P. (2002) · Machine Learning · doi:10.1023/a:1013689704352 · Cited by 6,033
Rapidly-exploring random trees : a new tool for path planning
LaValle, S. M. (1998) · Cited by 3,704
Bandit Based Monte-Carlo Planning
Kocsis, L., & Szepesvári, C. (2006) · Lecture Notes in Computer Science · doi:10.1007/11871842_29 · Cited by 2,855 · cited 10×
“We emphasise the importance of game theory, as this is the domain to which MCTS is most applied.”
BackgroundMethodInfluentialAsymptotically efficient adaptive allocation rules
Lai, T. L., & Robbins, H. (1985) · Advances in Applied Mathematics · doi:10.1016/0196-8858(85)90002-8 · Cited by 2,406
Efficient Selectivity and Backup Operators in Monte-Carlo Tree Search
Coulom, R. (2007) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-75538-8_7 · Cited by 1,240 · cited 3×
“Real-world games typically involve a delayed reward structure in which only those rewards achieved in the terminal states of the game accurately describe how well each player is performing.”
BackgroundMonte-Carlo Planning in Large POMDPs
Silver, D., & Veness, J. (2010) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 886
Gambling in a rigged casino: The adversarial multi-armed bandit problem
Auer, P., Cesa-Bianchi, N., Freund, Y., & Schapire, R. E. (2002) · Proceedings of IEEE 36th Annual Foundations of Computer Science · doi:10.1109/sfcs.1995.492488 · Cited by 742 · cited 1×
“[14] and further analyzed by Audibert and Bubeck [11], applies in the stochastic case (and hence also in the adversarial case).”
BackgroundSample mean based index policies by O(log n) regret for the multi-armed bandit problem
Agrawal, R. (1995) · Advances in Applied Probability · doi:10.2307/1427934 · Cited by 540
Research works printed but not linked: 35
- Entries marked with an asterisk (*) denote support material that does not directly cite MCTS methods
- Althöfer, 2010, “Game self-play with pure Monte-Carlo: The basin structure
- Asmuth, Approaching Bayes-optimalilty using Monte-Carlo tree search
- Borsboom, A comparison of Monte-Carlo methods for phantom Go, 57
- Brügmann, 1993, Monte Carlo Go
- Cazenave, 2007, Evolving Monte-Carlo tree search algorithms
- Chaslot, 2008, Combining expert, offline, transient and online knowledge in Monte-Carlo exploration
- Den Teuling, 2011, Monte-Carlo tree search for the simultaneous move game Tron
- Drake, Move ordering vs heavy playouts: Where should heuristics be applied in Monte Carlo Go, 35
- 10.17875/gup2010-1477 · doi:10.17875/gup2010-1477
- 10.1109/tciaig.2010.2083662 · DOI · Paper page
- Finnsson, Simulation control in general game playing agents, 21
- Gelly, 2007, A contribution to reinforcement learning; Application to computer-Go
- Huang, 2011, New heuristics for Monte Carlo tree search applied to the Game of Go
- Könnecke, 2009, Efficient playouts for the Havannah abstract board game
- Lopez, 2008, Rybka’s Monte Carlo analysis
- 10.3233/978-1-58603-891-5-458 · doi:10.3233/978-1-58603-891-5-458
- 10.65109/vilr5324 · doi:10.65109/vilr5324
- 10.1163/1573-3912_ei2glos_sim_gi_00259 · doi:10.1163/1573-3912_ei2glos_sim_gi_00259
- Méhat, 2008, Monte-Carlo tree search for general game playing
- Nguyen, 2011, ICE gUCT
- Nijssen, 2007, Playing Othello using Monte Carlo, Strategies, 1
- Nijssen, 2009, Using intelligent search techniques to play the game Khet
- Ramanujan, On the behavior of UCT in synthetic search spaces
- Rosin, Nested rollout policy adaptation for Monte Carlo tree search, 649
- Saffidine, 2008, Utilisation dUCT au Hex
- Saffidine, 2010, Some improvements for Monte-Carlo tree search, game description language compilation, score bounds and transpositions
- Schadd, 2009, Monte-Carlo search techniques in the modern board game Thurn and Taxis
- Schaefers, UCT-treesplit-parallel MCTS on distributed memory
- Schäfer, 2008, The UCT algorithm applied to games with imperfect information
- Shafiei, Comparing UCT versus CFR in simultaneous games
- Stankiewicz, 2011, Knowledge-based Monte-Carlo tree search in Havannah
- Sylvester, A linear classifier outperforms UCT in 9$\,\times\,$9 Go, 804
- Williams, 2010, Determining game quality through UCT tree shape analysis
- 10.1109/tciaig.2010.2061050 · DOI
Cited by
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
“The previous major breakthrough in computer Go, the introduction of MCTS, led to corresponding advances in many other domains; for example, general game-playing, classical planning, partially observed planning, scheduling, and constraint satisfaction 35,36 .”
BackgroundMastering the game of Go without human knowledge
Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., et al. (2017) · Nature · doi:10.1038/nature24270 · Cited by 9,084
“Once trained, these networks were combined with a Monte Carlo tree search (MCTS) 13–15 to provide a lookahead search, using the policy network to narrow down the search to highprobability moves, and using the value network (in conjunction with Monte Carlo rollouts using a fast rollout policy) to…”
MethodRecent advances and applications of machine learning in solid-state materials science
Schmidt, J., Marques, M. R. G., Botti, S., & Marques, M. A. L. (2019) · npj Computational Materials · doi:10.1038/s41524-019-0221-0 · Cited by 2,607
SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient
Yu, L., Zhang, W., Wang, J., & Yu, Y. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10804 · Cited by 2,285
The Arcade Learning Environment: An Evaluation Platform for General Agents
Bellemare, M. G., Naddaf, Y., Veness, J., & Bowling, M. (2013) · Journal of Artificial Intelligence Research · doi:10.1613/jair.3912 · Cited by 2,161
“By expanding the more valuable branches of the tree and carrying out a random simulation at the leaf nodes, UCT is known to perform well in many different settings (Browne et al., 2012).”
Background
Metadata
- Authors
- Browne, C. B., Powley, E., Whitehouse, D., Lucas, S. M., Cowling, P. I., Rohlfshagen, P., Tavener, S., Perez, D., Samothrakis, S., & Colton, S.
- Affiliations
- Imperial College London (GB)
- University of Bradford (GB)
- University of Essex (GB)
- ORCID
- Venue
- IEEE Transactions on Computational Intelligence and AI in Games
- Published
- 2012-03
- Volume
- 4
- Issue
- 1
- Pages
- 1-43
- ISSN
- 1943-068X (ISSN-L)
- 1943-0698
- OpenAlex
- W2126316555
- Semantic Scholar
- c37f1baac3c8ba30250084f067167ac3837cf6fd
- Cited by
- 2,970
- References
- 233
- Citations by year
- 2026: 163 · 2025: 220 · 2024: 244 · 2023: 269 · 2022: 294 · 2021: 308 · 2020: 305 · 2019: 263 · 2018: 230 · 2017: 186 · 2016: 155 · 2015: 117 · 2014: 125 · 2013: 60 · 2012: 28
- FWCI
- 103.94
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Transactions on Computational Intelligence and AI in Games (publishedVersion)
- Research Publications (Maastricht University) (publishedVersion)
- Open Access at Essex (University of Essex) (acceptedVersion)
- Research Publications (Maastricht University) (submittedVersion, public-domain)
- Funders
- Engineering and Physical Sciences Research Council (EP/H048588/1, EP/H049061/2, EP/H049061/1, EP/I001964/1)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Advanced Bandit Algorithms Research
- Keywords
- Monte Carlo tree search, computer Go, search algorithms, random sampling, game-playing agents, open research questions
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: A Survey of Monte Carlo Tree Search Methods. https://app.cochat.ai/papers/10.1109/tciaig.2012.2186810
- The paper
- Browne, C. B., Powley, E., Whitehouse, D., Lucas, S. M., Cowling, P. I., Rohlfshagen, P., Tavener, S., Perez, D., Samothrakis, S., & Colton, S. (2012). A Survey of Monte Carlo Tree Search Methods. IEEE Transactions on Computational Intelligence and AI in Games. https://doi.org/10.1109/tciaig.2012.2186810
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.