The DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
Combining online and offline knowledge in UCT
Open accessAbstract
The UCT algorithm learns a value function online using sample-based search. The TD(λ) algorithm can learn a value function offline for the on-policy distribution. We consider three approaches for combining offline and online value functions in the UCT algorithm. First, the offline value function is used as a default policy during Monte-Carlo simulation. Second, the UCT value function is combined with a rapid online estimate of action values. Third, the offline value function is used as prior knowledge in the UCT search tree. We evaluate these algorithms in 9 x 9 Go against GnuGo 3.7.10. The first algorithm performs better than UCT with a random simulation policy, but surprisingly, worse than UCT with a weaker, handcrafted simulation policy. The second algorithm outperforms UCT altogether. The third algorithm outperforms UCT with handcrafted prior knowledge. We combine these algorithms in MoGo, the world's strongest 9 x 9 Go program. Each technique significantly improves MoGo's playing strength.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Introduction to Reinforcement Learning
Sutton, R. S., & Barto, A. G. (1998) · The MIT Press eBooks · Cited by 6,855
Finite-time Analysis of the Multiarmed Bandit Problem
Auer, P., Cesa-Bianchi, N., & Fischer, P. (2002) · Machine Learning · doi:10.1023/a:1013689704352 · Cited by 6,033
Learning to predict by the methods of temporal differences
Sutton, R. S. (1988) · Machine Learning · doi:10.1007/bf00115009 · Cited by 5,148
Bandit Based Monte-Carlo Planning
Kocsis, L., & Szepesvári, C. (2006) · Lecture Notes in Computer Science · doi:10.1007/11871842_29 · Cited by 2,855
Sutton, R. S. (1990) · Machine Learning Proceedings 1990 · doi:10.1016/b978-1-55860-141-3.50030-4 · Cited by 1,314
Efficient Selectivity and Backup Operators in Monte-Carlo Tree Search
Coulom, R. (2007) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-75538-8_7 · Cited by 1,240
Generalization in Reinforcement Learning: Successful Examples Using Sparse Coarse Coding
Sutton, R. S. (1995) · Cited by 1,072
Modification of UCT with Patterns in Monte-Carlo Go
Gelly, S., Wang, Y., Munos, R., & Teytaud, O. (2006) · HAL · Cited by 290
Self-citation
Cited by
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Self-citationA Survey of Monte Carlo Tree Search Methods
Browne, C. B., Powley, E., Whitehouse, D., Lucas, S. M., Cowling, P. I., Rohlfshagen, P., Tavener, S., Perez, D., Samothrakis, S., & Colton, S. (2012) · IEEE Transactions on Computational Intelligence and AI in Games · doi:10.1109/tciaig.2012.2186810 · Cited by 2,970
Monte-Carlo tree search and rapid action value estimation in computer Go
Gelly, S., & Silver, D. (2011) · Artificial Intelligence · doi:10.1016/j.artint.2011.03.007 · Cited by 367
Self-citationAn object-oriented representation for efficient reinforcement learning
Diuk, C., Cohen, A., & Littman, M. L. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390187 · Cited by 226
Thinking Fast and Slow with Deep Learning and Tree Search
Anthony, T., Tian, Z., & Barber, D. (2017) · arXiv · doi:10.48550/arxiv.1705.08439 · Cited by 198
Metadata
- Authors
- Gelly, S., & Silver, D.
- Venue
- Proceedings of the 24th international conference on Machine learning
- Publisher
- ACM
- Published
- 2007-06-20
- Pages
- 273-280
- OpenAlex
- W2020135152
- Cited by
- 465
- References
- 15
- FWCI
- 22.55
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Combining online and offline knowledge in UCT. https://app.cochat.ai/papers/10.1145/1273496.1273531
- The paper
- Gelly, S., & Silver, D. (2007). Combining online and offline knowledge in UCT. Proceedings of the 24th international conference on Machine learning. https://doi.org/10.1145/1273496.1273531
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.