Thinking Fast and Slow with Deep Learning and Tree Search
Open accessAbstract
Sequential decision making problems, such as structured prediction, robotic control, and game playing, require a combination of planning policies and generalisation of those plans. In this paper, we present Expert Iteration (ExIt), a novel reinforcement learning algorithm which decomposes the problem into separate planning and generalisation tasks. Planning new policies is performed by tree search, while a deep neural network generalises those plans. Subsequently, tree search is improved by using the neural network policy to guide search, increasing the strength of new plans. In contrast, standard deep Reinforcement Learning algorithms rely on a neural network not only to generalise plans, but to discover them too. We show that ExIt outperforms REINFORCE for training a neural network to play the board game Hex, and our final tree search agent, trained tabula rasa, defeats MoHex 1.0, the most recent Olympiad Champion player to be publicly released.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Generational methods that exist explicitly optimise new generations with hand-crafted and explicit imitation learning techniques, as opposed to learning accumulation implicitly.
“Generational methods that do exist, such as iterated policy distillation [Schmitt et al., 2018, Stooke et al., 2022] and expert iteration [Anthony et al., 2017], explicitly optimise new generations with hand-crafted and explicit imitation learning techniques, as opposed to learning accumulation implicitly, which should ultimately enable greater flexibility.”
From Artificial Generational Intelligence: Cultural Accumulation in Reinforcement Learning (2024) · cites this paper as Anthony et al., 2017
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357 · see citing works
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612 · see citing works
Mastering the game of Go without human knowledge
Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., et al. (2017) · Nature · doi:10.1038/nature24270 · Cited by 9,084 · see citing works
Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
Williams, R. J. (1992) · Machine Learning · doi:10.1007/bf00992696 · Cited by 7,936 · see citing works
Maps of Bounded Rationality: Psychology for Behavioral Economics
Kahneman, D. (2003) · American Economic Review · doi:10.1257/000282803322655392 · Cited by 5,377 · see citing works
Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)
Clevert, D.-A., Unterthiner, T., & Hochreiter, S. (2015) · arXiv · doi:10.48550/arxiv.1511.07289 · Cited by 4,513 · see citing works
Bandit Based Monte-Carlo Planning
Kocsis, L., & Szepesvari, C. (2006) · European Conference on Machine Learning · doi:10.1007/11871842_29 · Cited by 4,073 · see citing works
A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning
Ross, S., Gordon, G. J., & Bagnell, J. A. (2018) · Carnegie Mellon University · doi:10.1184/r1/6550949 · Cited by 1,941 · see citing works
Heuristic and analytic processes in reasoning*
Evans, J. S. B. T. (1984) · British Journal of Psychology · doi:10.1111/j.2044-8295.1984.tb01915.x · Cited by 619 · see citing works
General Game Playing: Overview of the AAAI Competition
Genesereth, M., Love, N., & Pell, B. (2005) · The AI Magazine · doi:10.1609/aimag.v26i2.1813 · Cited by 588 · see citing works
Combining online and offline knowledge in UCT
Gelly, S., & Silver, D. (2007) · International Conference on Machine Learning · doi:10.1145/1273496.1273531 · Cited by 465 · see citing works
Learning to Search Better Than Your TeacherPreprint
Chang, K.-W., Krishnamurthy, A., Agarwal, A., Daumé, H., & Langford, J. C. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1502.02206 · Cited by 147 · see citing works
Monte Carlo Tree Search in Hex
Arneson, B., Hayward, R., & Henderson, P. (2010) · IEEE Transactions on Computational Intelligence and AI in Games · doi:10.1109/tciaig.2010.2067212 · Cited by 144 · see citing works
Training Deterministic Parsers with Non-Deterministic Oracles
Goldberg, Y., & Nivre, J. (2013) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00237 · Cited by 112 · see citing works
Arpit, D., Zhou, Y., Kota, B. U., & Govindaraju, V. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.01431 · Cited by 100 · see citing works
MoHex 2.0: A Pattern-Based MCTS Hex Player
Huang, S.-C., Arneson, B., Hayward, R., Müller, M., & Pawlewicz, J. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-09165-5_6 · Cited by 39 · see citing works
NeuroHex: A Deep Q-learning Hex Agent
Young, K., Vasan, G., & Hayward, R. (2017) · Communications in computer and information science · doi:10.1007/978-3-319-57969-6_1 · Cited by 21 · see citing works
Neurohex: A Deep Q-learning Hex Agent
Young, K., Hayward, R., & Vasan, G. (2016) · arXiv · doi:10.48550/arxiv.1604.07097
Flagged first, then by citations
Research works printed but not linked: 2
- [11][11] X. Guo, S. Singh, H. Lee, R. L. Lewis, and X. Wang. Deep Learning for Real-Time Atari Game Play Using Offline Monte-Carlo Rree Search Planning. In Advances in Neural Information Processing Systems , pages 3338-3346, 2014. -
- [15][15] S. Ross and J. A. Bagnell. Reinforcement and Imitation Learning via Interactive No-Regret Learning. ArXiv e-prints , 2014. -
Cited by
Sorted by citations
Training language models to follow instructions with human feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.02155 · Cited by 4,208 · see citing works
A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play
Silver, D., Hubert, T., Schrittwieser, J., Antonoglou, I., Lai, M., Guez, A., Lanctot, M., Sifre, L., Kumaran, D., Graepel, T., et al. (2018) · Science · doi:10.1126/science.aar6404 · Cited by 3,562 · see citing works
Reinforcement learning for combinatorial optimization: A survey
Mazyavkina, N., Sviridov, S., Ivanov, S., & Burnaev, E. V. (2021) · Computers & Operations Research · doi:10.1016/j.cor.2021.105400 · Cited by 762 · see citing works
Model-based Reinforcement Learning: A Survey
Moerland, T. M., Broekens, J., Plaat, A., & Jonker, C. M. (2023) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000086 · Cited by 668 · see citing works
Marcus, R., Negi, P., Mao, H., Zhang, C., Alizadeh, M., Kraska, T., Papaemmanouil, O., & Tatbul, N. (2019) · Proceedings of the VLDB Endowment · doi:10.14778/3342263.3342644 · Cited by 440 · see citing works
Monte Carlo Tree Search: a review of recent modifications and applications
Świechowski, M., Godlewski, K., Sawicki, B., & Mańdziuk, J. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-022-10228-y · Cited by 386 · see citing works
Diffusion Model Alignment Using Direct Preference Optimization
Wallace, B., Dang, M., Rafailov, R., Zhou, L., Lou, A., Purushwalkam, S., Ermon, S., Xiong, C., Joty, S., & Naik, N. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00786 · Cited by 185 · see citing works
Challenges and Countermeasures for Adversarial Attacks on Deep Reinforcement Learning
Ilahi, I., Usama, M., Qadir, J., Janjua, M. U., Al-Fuqaha, A., Hoang, D. T., & Niyato, D. (2022) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2021.3111139 · Cited by 170 · see citing works
Link Prediction in Time-Evolving Criminal Network With Deep Reinforcement Learning Technique
Lim, M., Abdullah, A. B., Jhanjhi, N. Z., Khan, M. K., & Supramaniam, M. (2019) · IEEE Access · doi:10.1109/access.2019.2958873 · Cited by 148 · see citing works
Solving the Rubik’s cube with deep reinforcement learning and search
Agostinelli, F., McAleer, S., Shmakov, A., & Baldi, P. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0070-z · Cited by 136 · see citing works
Plan Online, Learn Offline: Efficient Learning and Exploration via Model-Based ControlPreprint
Lowrey, K., Rajeswaran, A., Kakade, S. M., Todorov, E. V., & Mordatch, I. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.01848 · Cited by 155 · see citing works
Lim, M., Abdullah, A. B., & Jhanjhi, N. Z. (2019) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2019.07.010 · Cited by 132 · see citing works
Scalable agent alignment via reward modeling: a research direction
Leike, J., Krueger, D., Everitt, T., Martic, M., Maini, V., & Legg, S. (2018) · arXiv · doi:10.48550/arxiv.1811.07871 · Cited by 121 · see citing works
Hidden Link Prediction in Criminal Networks Using the Deep Reinforcement Learning Technique
Lim, M., Abdullah, A. B., Jhanjhi, N. Z., & Supramaniam, M. (2019) · Computers · doi:10.3390/computers8010008 · Cited by 116 · see citing works
Neural Network Model-Based Reinforcement Learning Control for AUV 3-D Path Following
Ma, D., Chen, X., Ma, W., Zheng, H., & Qu, F. (2023) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2023.3282681 · Cited by 113 · see citing works
Balsa: Learning a Query Optimizer Without Expert Demonstrations
Yang, Z., Chiang, W.-L., Luan, F. S., Mittal, G., Luo, M., & Stoica, I. G. (2022) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/3514221.3517885 · Cited by 103 · see citing works
Learning to Solve Combinatorial Optimization Problems on Real-World Graphs in Linear Time
Drori, I., Kharkar, A., Sickinger, W. R., Kates, B., Ma, Q., Ge, S., Dolev, E., Dietrich, B. L., Williamson, D. P., & Udell, M. R. (2020) · International Conference on Machine Learning and Applications · doi:10.1109/icmla51294.2020.00013 · Cited by 92 · see citing works
On Monte Carlo Tree Search and Reinforcement Learning
Vodopivec, T., Samothrakis, S., & Šter, B. (2017) · Journal of Artificial Intelligence Research · doi:10.1613/jair.5507 · Cited by 83 · see citing works
Learning To Navigate The Synthetically Accessible Chemical Space Using Reinforcement LearningPreprint
Gottipati, S. K., Sattarov, B., Niu, S., Pathak, Y., Wei, H., Liu, S., Thomas, K. M. J., Blackburn, S., Coley, C. W., Tang, J., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.12485 · Cited by 82 · see citing works
Ranked Reward: Enabling Self-Play Reinforcement Learning for Combinatorial OptimizationPreprint
Laterre, A., Fu, Y., Jabri, M. K., Cohen, A.-S., Kas, D., Hajjar, K., Dahl, T. S., Kerkeni, A., & Beguir, K. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.01672 · Cited by 79 · see citing works
Model-free, Model-based, and General Intelligence
Geffner, H. (2018) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/2 · Cited by 56 · see citing works
High-accuracy model-based reinforcement learning, a survey
Plaat, A., Kosters, W. A., & Preuß, M. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-022-10335-w · Cited by 57 · see citing works
Symphony: Learning Realistic and Diverse Agents for Autonomous Driving Simulation
Igl, M., Kim, D. W., Kuefler, A., Mougin, P., Shah, P., Shiarlis, K., Anguelov, D., Palatucci, M., White, B., & Whiteson, S. (2022) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra46639.2022.9811990 · Cited by 51 · see citing works
Wittkuhn, L., Chien, S., Hall-McMaster, S., & Schuck, N. W. (2021) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2021.08.002 · Cited by 47 · see citing works
From Machine Learning to Robotics: Challenges and Opportunities for Embodied IntelligencePreprint
Roy, N., Posner, I., Barfoot, T., Beaudoin, P., Bengio, Y., Bohg, J., Brock, O., Depatie, I., Fox, D., Koditschek, D. E., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.15245 · Cited by 83 · see citing works
Planning at decision time and in the background during spatial navigation
Pezzulo, G., Donnarumma, F., Maisto, D., & Stoianov, I. P. (2019) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2019.04.009 · Cited by 44 · see citing works
Machine Thinking, Fast and Slow
Bonnefon, J., & Rahwan, I. (2020) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2020.09.007 · Cited by 45 · see citing works
Situation-Aware Deep Reinforcement Learning Link Prediction Model for Evolving Criminal Networks
Lim, M., Abdullah, A. B., Jhanjhi, N. Z., & Khan, M. K. (2019) · IEEE Access · doi:10.1109/access.2019.2961805 · Cited by 42 · see citing works
Learning to Search with MCTSnetsPreprint
Guez, A., Weber, T., Antonoglou, I., Simonyan, K., Vinyals, O., Wierstra, D., Munos, R., & Silver, D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.04697 · Cited by 41 · see citing works
AlphaD3M: Machine Learning Pipeline SynthesisPreprint
Drori, I., Krishnamurthy, Y., Rampin, R., Lourenço, R., Ono, J. P., Cho, K., Silva, C. T., & Freire, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.02508 · Cited by 40 · see citing works
Heffetz, Y., Vainshtein, R., Katz, G., & Rokach, L. (2020) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3394486.3403261 · Cited by 36 · see citing works
Canaan, R., Salge, C., Togelius, J., & Nealen, A. (2019) · International Conference on the Foundations of Digital Games (FDG) · doi:10.1145/3337722.3337750 · Cited by 54 · see citing works
LeTS-Drive: Driving in a Crowd by Learning from Tree Search
Cai, P., Luo, Y., Saxena, A., Hsu, D., & Lee, W. S. (2019) · Robotics: Science and Systems · doi:10.15607/rss.2019.xv.018 · Cited by 34 · see citing works
On the role of planning in model-based deep reinforcement learningPreprint
Hamrick, J. B., Friesen, A. L., Behbahani, F., Guez, A., Viola, F., Witherspoon, S., Anthony, T., Buesing, L., Veličković, P., & Weber, T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.04021 · Cited by 34 · see citing works
Self-citationA Survey of Planning and Learning in Games
Duarte, F. F., Lau, N., Pereira, A., & Reis, L. P. (2020) · Applied Sciences · doi:10.3390/app10134529 · Cited by 33 · see citing works
Guiding Deep Molecular Optimization with Genetic ExplorationPreprint
Ahn, S., Kim, J., Lee, H., & Shin, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.04897 · Cited by 32 · see citing works
A Generalized Framework for Self-Play Training
Hernández, D., Denamganaï, K., Gao, Y., York, P., Devlin, S., Samothrakis, S., & Walker, J. A. (2019) · IEEE Conference on Games (CoG) · doi:10.1109/cig.2019.8848006 · Cited by 30 · see citing works
Combining Q-Learning and Search with Amortized Value EstimatesPreprint
Hamrick, J. B., Bapst, V., Sánchez‐González, Á., Pfaff, T., Weber, T., Buesing, L., & Battaglia, P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.02807 · Cited by 30 · see citing works
Self-play learning strategies for resource assignment in Open-RAN networks
Wang, X., Thomas, J. D., Piechocki, R. J., Kapoor, S., Santos‐Rodríguez, R., & Parekh, A. (2022) · Computer Networks · doi:10.1016/j.comnet.2021.108682 · Cited by 29 · see citing works
Solving the Rubik's Cube Without Human KnowledgePreprint
McAleer, S., Agostinelli, F., Shmakov, A., & Baldi, P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.07470 · Cited by 28 · see citing works
Wang, H., Liang, W. G., & Yu, L.-F. (2020) · ACM Transactions on Graphics · doi:10.1145/3414685.3417788 · Cited by 26 · see citing works
Planning with a Receding Horizon for Manipulation in Clutter Using a Learned Value Function
Bejjani, W., Papallas, R., Leonetti, M., & Doğar, M. R. (2018) · IEEE-RAS International Conference on Humanoid Robots (Humanoids) · doi:10.1109/humanoids.2018.8624977 · Cited by 26 · see citing works
Sidekick Policy Learning for Active Visual Exploration
Ramakrishnan, S. K., & Grauman, K. L. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01258-8_26 · Cited by 26 · see citing works
Training Deep Reactive Policies for Probabilistic Planning Problems
Issakkimuthu, M., Fern, A., & Tadepalli, P. V. (2018) · Proceedings of the International Conference on Automated Planning and Scheduling · doi:10.1609/icaps.v28i1.13873 · Cited by 25 · see citing works
Drori, I., Krishnamurthy, Y., Lourenço, R., Rampin, R., Cho, K., Silva, C. T., & Freire, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.10345 · Cited by 25 · see citing works
Learning to Play No-Press Diplomacy with Best Response Policy IterationPreprint
Anthony, T., Eccles, T., Tacchetti, A., Kramár, J., Gemp, I., Hudson, T. C., Porcel, N., Lanctot, M., Pérolat, J., Everett, R., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.04635 · Cited by 24 · see citing works
Self-citationReinforcement Learning for Process Control: Review and Benchmark Problems
Park, J., Jung, H., Kim, J. W., & Lee, J. M. (2025) · International Journal of Control Automation and Systems · doi:10.1007/s12555-024-0990-1 · Cited by 22 · see citing works
Hettiarachchi, C., Malagutti, N., Nolan, C. J., Suominen, H., & Daskalaki, E. (2022) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2023.105839 · Cited by 25 · see citing works
Generalized Planning With Deep Reinforcement LearningPreprint
Rivlin, O., Hazan, T., & Karpas, E. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.02305 · Cited by 34 · see citing works
Bejjani, W., Doğar, M. R., & Leonetti, M. (2019) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8967717 · Cited by 21 · see citing works
Three-Head Neural Network Architecture for Monte Carlo Tree Search
Gao, C., Müller, M., & Hayward, R. (2018) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/523 · Cited by 22 · see citing works
Action Guidance with MCTS for Deep Reinforcement Learning
Kartal, B., Hernández-Leal, P., & Taylor, M. E. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment · doi:10.1609/aiide.v15i1.5238 · Cited by 22 · see citing works
Deep Representation Learning for Complex Free-Energy Landscapes
Zhang, J., Lei, Y.-K., Che, X., Zhang, Z., Yang, Y., & Gao, Y. Q. (2019) · The Journal of Physical Chemistry Letters · doi:10.1021/acs.jpclett.9b02012 · Cited by 21 · see citing works
Leveling the Playing Field -- Fairness in AI Versus Human Game BenchmarksPreprint
Canaan, R., Salge, C., Togelius, J., & Nealen, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.07008 · Cited by 21 · see citing works
Solving the Rubik's Cube with Approximate Policy Iteration
McAleer, S., Agostinelli, F., Shmakov, A., & Baldi, P. (2018) · International Conference on Learning Representations · Cited by 19 · see citing works
Raina, A., Puentes, L., Cagan, J., & McComb, C. (2021) · Journal of Mechanical Design · doi:10.1115/1.4051013 · Cited by 19 · see citing works
On-Policy Robot Imitation Learning from a Converging SupervisorPreprint
Balakrishna, A., Thananjeyan, B., Lee, J., Li, F., Zahed, A., Gonzalez, J. E., & Goldberg, K. Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.03423 · Cited by 18 · see citing works
Adversarial Patch Attacks and Defences in Vision-Based Tasks: A SurveyPreprint
Sharma, A., Bian, Y., Munz, P., & Narayan, A. (2022) · doi:10.36227/techrxiv.20085902.v1 · Cited by 23 · see citing works
Hedging using reinforcement learning: Contextual k-armed bandit versus Q-learning
Cannelli, L., Nuti, G., Sala, M. G., & Szehr, O. (2023) · The Journal of Finance and Data Science · doi:10.1016/j.jfds.2023.100101 · Cited by 19 · see citing works
The best of both worlds: Dual systems of reasoning in animals and AI
Kelly, M., & Barron, A. B. (2022) · Cognition · doi:10.1016/j.cognition.2022.105118 · Cited by 22 · see citing works
Learning and Testing Resilience in Cooperative Multi-Agent Systems
Phan, T., Gabor, T., Sedlmeier, A., Ritz, F., Kempter, B., Klein, C., Sauer, H., Schmid, R. N., Wieghardt, J., Zeller, M., et al. (2020) · Adaptive Agents and Multi-Agent Systems · doi:10.65109/rilc3901 · Cited by 32 · see citing works
Thinking Fast and Slow in AI: The Role of Metacognition
Ganapini, M. B., Campbell, M. S., Fabiano, F., Lior Horesh, C., Lenchner, J., Loreggia, A., Mattei, N., Rossi, F., Srivastava, B., & Venable, K. B. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-25891-6_38 · Cited by 15 · see citing works
A Comparison of Self-Play Algorithms Under a Generalized Framework
Hernández, D., Denamganaï, K., Devlin, S., Samothrakis, S., & Walker, J. A. (2021) · IEEE Transactions on Games · doi:10.1109/tg.2021.3058898 · Cited by 14 · see citing works
Zombori, Z., Csiszárik, A., Michalewski, H., Kaliszyk, C., & Urban, J. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-86059-2_10 · Cited by 18 · see citing works
Policy Gradient Search: Online Planning and Expert Iteration without Search TreesPreprint
Anthony, T., Nishihara, R., Moritz, P., Salimans, T., & Schulman, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.03646 · Cited by 19 · see citing works
Self-citationBiasing MCTS with Features for General Games
Soemers, D. J. N. J., Piette, É., & Browne, C. (2019) · IEEE Congress on Evolutionary Computation (CEC) · doi:10.1109/cec.2019.8790141 · Cited by 13 · see citing works
Dynamic Automaton-Guided Reward Shaping for Monte Carlo Tree Search
Velasquez, A., Bissey, B., Barak, L., Beckus, A., Alkhouri, I., Melcer, D., & Atia, G. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i13.17427 · Cited by 13 · see citing works
Raina, A., Cagan, J., & McComb, C. (2022) · Journal of Mechanical Design · doi:10.1115/1.4056221 · Cited by 13 · see citing works
Guiding Multiplayer MCTS by Focusing on Yourself
Baier, H., & Kaisers, M. (2020) · IEEE Conference on Games (CoG) · doi:10.1109/cog47356.2020.9231603 · Cited by 13 · see citing works
Deep Model-Based Reinforcement Learning for High-Dimensional Problems, a SurveyPreprint
Plaat, A., Kosters, W. A., & Preuß, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.05598 · Cited by 21 · see citing works
Adversarial Patch Attacks and Defences in Vision-Based Tasks: A SurveyPreprint
Sharma, A., Bian, Y., Munz, P., & Narayan, A. (2022) · doi:10.36227/techrxiv.20085902 · Cited by 13 · see citing works
SoFTNet: A concept-controlled deep learning architecture for interpretable image classification
Zia, T., Bashir, N., Ullah, M. A., & Murtaza, S. (2022) · Knowledge-Based Systems · doi:10.1016/j.knosys.2021.108066 · Cited by 12 · see citing works
Veerababu, D., Raikar, A. A., & Ghosh, P. K. (2025) · Machine Learning with Applications · doi:10.1016/j.mlwa.2025.100697 · Cited by 12 · see citing works
Learning image-based Receding Horizon Planning for manipulation in clutter
Bejjani, W., Leonetti, M., & Doğar, M. R. (2021) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2021.103730 · Cited by 12 · see citing works
On The Ingredients of an Effective Zero-shot Semantic Parser
Yin, P., Wieting, J., Sil, A., & Neubig, G. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.103 · Cited by 12 · see citing works
Scenario co-evolution for reinforcement learning on a grid world smart factory domain
Gabor, T., Sedlmeier, A., Kiermeier, M., Phan, T., Henrich, M., Pichlmair, M., Kempter, B., Klein, C., Sauer, H., Ag, R. S., et al. (2019) · Annual Conference on Genetic and Evolutionary Computation · doi:10.1145/3321707.3321831 · Cited by 18 · see citing works
Towards Game-based Metrics for Computational Co-Creativity
Canaan, R., Menzel, S., Togelius, J., & Nealen, A. (2018) · IEEE Conference on Computational Intelligence and Games (CIG) · doi:10.1109/cig.2018.8490429 · Cited by 12 · see citing works
Deep Reasoning Networks: Thinking Fast and SlowPreprint
Chen, D., Bai, Y., Zhao, W., Ament, S., Gregoire, J. M., & Gomes, C. P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.00855 · Cited by 12 · see citing works
Deep Reinforcement Learning Designed Shinnar-Le Roux RF Pulse Using Root-Flipping: DeepRFSLR
Shin, D., Ji, S., Lee, D., Lee, J., Oh, S., & Lee, J. (2020) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2020.3018508 · Cited by 11 · see citing works
On the Potential for Open-Endedness in Neural Networks
Guttenberg, N., Virgo, N., & Penn, A. S. (2019) · Artificial Life · doi:10.1162/artl_a_00286 · Cited by 11 · see citing works
Mobile Networks for Computer Go
Cazenave, T. (2020) · IEEE Transactions on Games · doi:10.1109/tg.2020.3041375 · Cited by 11 · see citing works
Model-Based Deep Reinforcement Learning for High-Dimensional Problems, a Survey.Preprint
Plaat, A., Kosters, W. A., & Preuß, M. (2020) · arXiv (Cornell University) · Cited by 12 · see citing works
The Value of Planning for Infinite-Horizon Model Predictive Control
Hatch, N., & Boots, B. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9561718 · Cited by 10 · see citing works
Learning by Reusing Previous Advice in Teacher-Student Paradigm
Zhu, C., Cai, Y., Leung, H.-F., & Hu, S. (2020) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/cetf7884 · Cited by 10 · see citing works
Using Monte Carlo Tree Search as a Demonstrator within Asynchronous Deep RLPreprint
Kartal, B., Hernández-Leal, P., & Taylor, M. E. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.00045 · Cited by 9 · see citing works
Real-time tree search with pessimistic scenariosPreprint
Osogami, T., & Takahashi, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.10870 · Cited by 10 · see citing works
Dual Parallel Policy Iteration With Coupled Policy Improvement
Cheng, Y., Huang, L., Chen, C. L. P., & Wang, X. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3202192 · Cited by 9 · see citing works
Reinforcement Learning to Create Value and Policy Functions Using Minimax Tree Search in Hex
Takada, K., Iizuka, H., & Yamamoto, M. (2019) · IEEE Transactions on Games · doi:10.1109/tg.2019.2893343 · Cited by 9 · see citing works
Sample-Efficient Model-Free Reinforcement Learning with Off-Policy Critics
Steckelmacher, D., Plisnier, H., Roijers, D. M., & Nowé, A. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-46133-1_2 · Cited by 9 · see citing works
Recursive Monte Carlo Search for Bridge Card Play
Bouzy, B., Rimbaud, A., & Ventos, V. (2020) · IEEE Conference on Games (CoG) · doi:10.1109/cog47356.2020.9231667 · Cited by 9 · see citing works
Kurzer, K., Fechner, M., & Zöllner, J. M. (2020) · IEEE Intelligent Vehicles Symposium · doi:10.1109/iv47402.2020.9304597 · Cited by 9 · see citing works
Learning self-play agents for combinatorial optimization problems
Xu, R., & Lieberherr, K. J. (2020) · The Knowledge Engineering Review · doi:10.1017/s026988892000020x · Cited by 9 · see citing works
Re-determinizing Information Set Monte Carlo Tree Search in HanabiPreprint
Goodman, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.06075 · Cited by 9 · see citing works
FSPL: A meta-learning approach for a filter and embedded feature selection pipeline
Lazebnik, T., & Rosenfeld, A. (2023) · International Journal of Applied Mathematics and Computer Science · doi:10.34768/amcs-2023-0009 · Cited by 8 · see citing works
Lee, S.-H., Jung, Y., & Seo, S.-W. (2024) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2024.3457776 · Cited by 8 · see citing works
Leveraging Statistical Multi-Agent Online Planning with Emergent Value Function Approximation
Phan, T., Belzner, L., Gabor, T., & Schmid, K. (2018) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/tkcr8496 · Cited by 8 · see citing works
Learning to Resolve Alliance Dilemmas in Many-Player Zero-Sum Games
Hughes, E., Anthony, T. W., Eccles, T., Leibo, J. Z., Balduzzi, D., & Bachrach, Y. (2020) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/wuwb6150 · Cited by 9 · see citing works
Self-citationManipulating the Distributions of Experience used for Self-Play Learning in Expert Iteration
Soemers, D. J. N. J., Piette, É., Stephenson, M., & Browne, C. (2020) · IEEE Conference on Games (CoG) · doi:10.1109/cog47356.2020.9231589 · Cited by 8 · see citing works
Improving Model and Search for Computer Go
Cazenave, T. (2021) · IEEE Conference on Games (CoG) · doi:10.1109/cog52621.2021.9619078 · Cited by 8 · see citing works
Monte-Carlo Planning and Learning with Language Action Value Estimates
Jang, Y., Seo, S., Lee, J., & Kim, K.-E. (2021) · Cited by 8 · see citing works
- Authors
- Anthony, T., Tian, Z., & Barber, D.
- ORCID
- Barber, David
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2017
- Volume
- 30
- Pages
- 5360-5370
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1705.08439
- OpenAlex
- W2618097077
- Cited by
- 198
- References
- 15
- Citations by year
- 2025: 6 · 2024: 14 · 2023: 15 · 2022: 24 · 2021: 37 · 2020: 39 · 2019: 36 · 2018: 25 · 2017: 2
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- UCL Discovery (University College London) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Keywords
- tree search, deep reinforcement learning, sequential decision making, Hex, structured prediction, game-playing agents
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
