CoChat
Sign Up
Cited by 19Artificial Intelligence in GamesReinforcement Learning in RoboticsArtificial Intelligence Applications

Beyond games: a systematic review of neural Monte Carlo tree search applications

Open access
Kemmerling, Marco, Lütticke, Daniel, Schmitt, Robert H.·2024·Applied Intelligence
DOI 10.1007/s10489-023-05240-w
Open this paper in CoChat — free accountOpen PDF

Abstract

The advent of AlphaGo and its successors marked the beginning of a new paradigm in playing games using artificial intelligence. This was achieved by combining Monte Carlo tree search, a planning procedure, and deep learning. While the impact on the domain of games has been undeniable, it is less clear how useful similar approaches are in applications beyond games and how they need to be adapted from the original methodology. We perform a systematic literature review of peer-reviewed articles detailing the application of neural Monte Carlo tree search methods in domains other than games. Our goal is to systematically assess how such methods are structured in practice and if their success can be extended to other domains. We find applications in a variety of domains, many distinct ways of guiding the tree search using learned policy and value functions, and various training methods. Our review maps the current landscape of algorithms in the family of neural monte carlo tree search as they are applied to practical problems, which is a first step towards a more principled way of designing such algorithms for specific problems and their requirements.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1007/s10489-023-05240-w

Cites

3 of 3 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • MCTS is particularly effective for optimizing decision-making in complex domains such as neural architecture search and games.

    “This iterative approach makes MCTS particularly effective for optimizing decision-making in complex domains [35], such as neural architecture search and games, as shown in Fig. 2.”

    From Optimizing Neural Network Architecture for Medical Image Segmentation Using Monte Carlo Tree Search (2026) · cites this paper as [35]

    The Introduction reports established effectiveness for neural MCTS in games, and §2.3 describes its iterative search process. The Introduction also says that additional planning may improve decision quality, but presents that benefit as a promise rather than an established general result. Our copy does not substantiate neural architecture search as an example (searched: “neural architecture search” and “architecture search”). (LLM, conf=0.83)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 140

  1. Al-Saffar M, Musilek P (2020) Reinforcement learning-based distributed BESS management for mitigating overvoltage issues in systems with high PV penetration. IEEE Trans Smart Grid 11(4):2980-2994
  2. Anthony T, Tian Z, Barber D (2017) Thinking fast and slow with deep learning and tree search. Adv Neural Inf Process Syst 30
  3. Audrey G, Francesco M (2019) Deep neural network and Monte Carlo tree search applied to fluid-structure topology optimization. Sci Rep 9(1):15916
  4. Auer P, Cesa-Bianchi N, Fischer P (2002) Finite-time analysis of the multiarmed bandit problem. Mach Learn 47(2):235-256
  5. Bai F, Meng F, Liu J, Wang J, Meng MQ (2022) Hierarchical policy with deep-reinforcement learning for nonprehensile mul- tiobject rearrangement. Biomim Intell Robot 2(3):100047
  6. Bitter C, Thun T, Meisen T (2022) Karolos: an open-source reinforcement learning framework for robot-task environments. arXiv:2212.00906 · arXiv
  7. Brockman G, Cheung V, Pettersson L, Schneider J, Schulman J, Tang J, Zaremba W (2016) OpenAI Gym. arXiv:1606.01540 · arXiv
  8. Browne CB, Powley E, Whitehouse D, Lucas SM, Cowling PI, Rohlfshagen P, Tavener S, Perez D, Samothrakis S, Colton S (2012) A survey of Monte Carlo tree search methods. IEEE Trans Comput Intell AI Games 4(1):1-43
  9. Carfora V, Di Massimo F, Rastelli R, Catellani P, Piastra M (2020) Dialogue management in conversational agents through psychol- ogy of persuasion and machine learning. Multimed Tools Appl 79(47):35949-35971
  10. Challita U, Sandberg D (2021) Deep reinforcement learning for dynamic spectrum sharing of LTE and NR. In ICC 2021 -IEEE international conference on communications, pp. 1-6
  11. Chen J, Chen S, Wang Q, Cao B, Feng G, Hu J (2019) iRAF: A deep reinforcement learning approach for collaborative mobile edge computing IoT networks. IEEE Internet Things J 6(4):7011- 7024
  12. Chen J, Luo S, Zhang L, Zhang C, Cao B (2021) iPAS: A deep Monte Carlo Tree Search-based intelligent pilot-power alloca- tion scheme for massive MIMO system. Digit Commun Netw 7(3):362-372
  13. Chen J, Zhang C, Luo J, Xie J, Wan Y (2020) Driving maneuvers prediction based autonomous driving control by deep Monte Carlo tree search. IEEE Trans Veh Technol 69(7):7146-7158
  14. Chen P-Y, Ke B-T, Lee T-C, Tsai I-C, Kung T-W, Lin L-Y, Liu E-C, Chang Y-C, Li Y-L, Chao MC-T (2022) A reinforcement learning agent for obstacle-avoiding rectilinear steiner tree construction. In Proceedings of the 2022 international symposium on physical design, pp. 107-115
  15. Chen Y-Q, Chen Y, Lee C-K, Zhang S, Hsieh C-Y (2022) Optimiz- ing quantum annealing schedules with Monte Carlo tree search enhanced with neural networks. Nat Mach Intell 4(3):269-278
  16. Chen Z, Huang J, Ahn H, Ning X (2021) Costly features classifi- cation using Monte Carlo tree search. In 2021 International joint conference on neural networks (IJCNN), pp. 1-8
  17. Chen Z, Zhong S, Chen J, Zhao Y (2021) DeepPursuit: unit- ing classical wisdom and deep RL for sparse recovery. In 2021 55th Asilomar conference on signals, systems, and computers, pp. 1361-1366
  18. Dai Y, Wang P, Zhang L (2021) Reinforcement syntactic depen- dency tree reasoning for target-oriented opinion word extrac- tion. In: International conference on artificial neural networks, Springer, pp. 531-543
  19. Dai Z, Liu CH, Ye Y, Han R, Yuan Y, Wang G, Tang J (2022) AoI-minimal UAV crowdsensing by model-based graph convolu- tional reinforcement learning. In IEEE INFOCOM 2022 -IEEE conference on computer communications, pp. 1029-1038
  20. Dalgaard M, Motzoi F, Sørensen JJ, Sherson J (2020) Global opti- mization of quantum dynamics with AlphaZero deep exploration. NPJ Quantum Inf 6(1):1-9
  21. Deng H, Yuan X, Tian Y, Hu J (2022) Neural-augmented two- stage Monte Carlo tree search with over-sampling for protein folding in HP Model. IEEJ Trans Electr Electron Eng 17(5):685- 694
  22. Dieb S, Song Z, Yin W-J, Ishii M (2020) Optimization of depth-graded multilayer structure for x-ray optics using machine learning. J Appl Phys 128(7):074901
  23. Erikawa D, Yasuo N, Sekijima M (2021) MERMAID: an open source automated hit-to-lead method based on deep reinforcement learning. J Cheminformatics 13(1):1-10
  24. Fawzi A, Balog M, Huang A, Hubert T, Romera-Paredes B, Barekatain M, Novikov A, R Ruiz FJ, Schrittwieser J, Swirszcz G, Silver D, Hassabis D, Kohli P (2022) Discovering faster matrix multiplication algorithms with reinforcement learning. Nature 610(7):47-53
  25. Feng Y, Li B, Zheng Q, Wang D, Xu X, Zhang R (2021) Electro- magnetic situation analysis and judgment based on deep learning. IET Commun 15(11):1455-1466
  26. Fong J, Campolo D, Acar C, Tee KP (2021) Model-based rein- forcement learning with LSTM networks for non-prehensile manipulation planning. In: 2021 21st international conference on control, automation and systems (ICCAS), pp. 1152-1158
  27. Fu Z, Fan Q, Zhang X, Li X, Wang S, Wang Y (2021) Policy network assisted Monte Carlo Tree search for intelligent ser- vice function chain deployment. In 2021 IEEE 20th international conference on trust, security and privacy in computing and com- munications (TrustCom), pp. 1161-1168
  28. Gaafar M, Shaghaghi M, Adve RS, Ding Z (2019) Reinforce- ment learning for cognitive radar task scheduling. In: 2019 53rd Asilomar conference on signals, systems, and computers, pp. 1653-1657
  29. Gabirondo-López J, Egaña J, Miguel-Alonso J, Orduna Urrutia R (2021) Towards autonomous defense of SDN networks using MuZero based intelligent agents. IEEE Access 9:107184-107199
  30. Ganapathi Subramanian S, Crowley M (2018) Combining MCTS and A3C for prediction of spatially spreading processes in forest wildfire settings. In: Canadian conference on artificial intelli- gence, Springer, pp. 285-291
  31. Gannouni A, Samsonov V, Behery M, Meisen T, Lakemeyer G (2020) Neural combinatorial optimization for production schedul- ing with sequence-dependent setup waste. In 2020 IEEE inter- national conference on systems, man, and cybernetics (SMC), pp. 2640-2647
  32. Genheden S, Thakkar A, Chadimová V, Reymond J-L, Engkvist O, Bjerrum E (2020) AiZynthFinder: a fast, robust and flexible open-source software for retrosynthetic planning. J Cheminfor- matics 12(1):1-9
  33. Gros TP, Groß J, Wolf V (2020) Real-time decision making for a car manufacturing process using deep reinforcement learning. In: 2020 winter simulation conference (WSC), pp. 3032-3044
  34. Göppert A, Mohring L, Schmitt RH (2021) Predicting perfor- mance indicators with ANNs for AI-based online scheduling in dynamically interconnected assembly systems. Prod Eng 15(5):619-633
  35. Ha T, Cho K, Cha G, Lee K, Oh S (2020) Vehicle control with prediction model based Monte-Carlo tree search. In 2020 17th international conference on ubiquitous robots (UR), pp. 303-308
  36. He L, Shao B, Xiao Y, Li Y, Liu T-Y, Chen E, Xia H (2018) Neurally-guided semantic navigation in knowledge graph. IEEE Trans Big Data 8(3):607-615
  37. He Y, Li H, Jin T, Bao FS (2022) Circuit routing using Monte Carlo Tree Search and deep reinforcement learning. In: 2022 International symposium on vlsi design, automation and test (VLSI-DAT), pp. 1-5
  38. He Y, Tao S, Xu J, Guo J, Lan Y, Cheng X (2018) Text matching with Monte Carlo tree search. In China conference on information retrieval, Springer, pp 41-52
  39. Hernandez D, Denamganaï K, Gao Y, York P, Devlin S, Samoth- rakis S, Walker JA (2019) A generalized framework for self-play training. In: 2019 IEEE conference on games (CoG), IEEE, pp 1-8
  40. Hoel C-J, Driggs-Campbell K, Wolff K, Laine L, Kochenderfer MJ (2020) Combining planning and deep reinforcement learning in tactical decision making for autonomous driving. IEEE Trans Intell Veh 5(2):294-305
  41. Hu Z, Tu J, Li B (2019) Spear: optimized dependency-aware task scheduling with deep reinforcement learning. In: 2019 IEEE 39th international conference on distributed computing systems (ICDCS), pp 2037-2046
  42. Huang J, Patwary M, Diamos G (2019) Coloring big graphs with alphagozero. arXiv:1902.10162 · arXiv
  43. Huang Y, Yu Y (2018) Distilling deep neural networks with rein- forcement learning. In 2018 IEEE international conference on information and automation (ICIA), pp 133-138
  44. Ishida S, Terayama K, Kojima R, Takasu K, Okuno Y (2022) AI-driven synthetic route design incorporated with retrosynthesis knowledge. J Chem Inf Model 62(6):1357-1367
  45. Jang Y, Lee J, Kim K-E (2020) Bayes-adaptive Monte-Carlo plan- ning and learning for goal-oriented dialogues. In: Proceedings of the AAAI conference on artificial intelligence, vol 34, pp 7994- 8001. Issue: 05
  46. Jiang Y, Liu M, Li J, Zhang J (2022) Reinforced MCTS for non-intrusive online load identification based on cognitive green computing in smart grid. Math Biosci Eng 19(11):11595-11627
  47. Kim M, Park J-K, Moon S-M (2022) Solving PBQP-based register allocation using deep reinforcement learning. In 2022 IEEE/ACM international symposium on code generation and optimization (CGO), pp 1-12
  48. Kocsis L, Szepesvári C (2006) Bandit based Monte-Carlo plan- ning. In European conference on machine learning, Springer, pp 282-293
  49. Kovari B, Becsi T, Szabo A., Aradi, S (2020) Policy gradient based control of a pneumatic actuator enhanced with Monte Carlo Tree search. In: 2020 6th international conference on mechatronics and robotics engineering (ICMRE), pp 177-182
  50. Kovári B, Hegedüs F, Bécsi T (2020) Design of a reinforcement learning-based lane keeping planning agent for automated vehi- cles. Appl Sci 10(20):7171
  51. Kuai Z, Wang T, Wang S (2022) Fair virtual network function map- ping and scheduling using proximal policy optimization. IEEE Trans Commun 70(11):7434-7445
  52. Lao Y, Xu J, Gao S, Guo J, Wen J-R (2019) Name entity recog- nition with policy-value networks. In: Proceedings of the 42nd International ACM SIGIR conference on research and develop- ment in information retrieval, pp 1245-1248
  53. Laterre A, Fu Y, Jabri M, Cohen A, Kas D, Hajjar K, Dahl T, Kerkeni A, Beguir K (2018) Ranked reward: enabling self-play reinforcement learning for combinatorial optimization. Advances in neural information processing systems 31 (NeurIPS 2018)
  54. Lei L, Luo R, Zheng R, Wang J, Zhang J, Qiu C, Ma L, Jin L, Zhang P, Chen J (2021) KB-Tree: learnable and continuous Monte-Carlo tree search for autonomous driving planning. In: 2021 IEEE/RSJ international conference on intelligent robots and systems (IROS), pp 4493-4500
  55. Li H, Hu Y, Cao Y, Zhou G, Luo P (2021) Rich-text document styling restoration via reinforcement learning. Front Comput Sci 15(4):1-11
  56. Li Y, Pei J, Lai L (2021) Structure-based de novo drug design using 3D deep generative models. Chem Sci 12(41):13664-13675
  57. Lu Q, Tao F, Zhou S, Wang Z (2021) Incorporating Actor-Critic in Monte Carlo tree search for symbolic regression. Neural Comput Appl 33(14):8495-8511
  58. Ma B, Terayama K, Matsumoto S, Isaka Y, Sasakura Y, Iwata H, Araki M, Okuno Y (2021) Structure-based de novo molecu- lar generator combined with artificial intelligence and docking simulations. J Chem Inf Model 61(7):3304-3313
  59. Mandhane A, Zhernov A, Rauh M, Gu C, Wang M, Xue F, Shang W, Pang D, Claus R, Chiang C-H, et al (2022) Muzero with self-competition for rate control in vp9 video compression. arXiv:2202.06626 · arXiv
  60. Mao K, Xiao Y (2021) Learning the fastest RNA folding path based on reinforcement learning and Monte Carlo tree search. Molecules 26(15):4420
  61. Mańdziuk J (2018) MCTS/UCT in solving real-life problems. In: Advances in data analysis with computational intelligence meth- ods. Springer, pp 277-292
  62. Meng X, Inaltekin H, Krongold B (2019) Deep reinforcement learning-based topology optimization for self-organized wireless sensor networks. In: 2019 IEEE global communications confer- ence (GLOBECOM), pp 1-6
  63. Mo S, Pei X, Wu C (2022) Safe reinforcement learning for autonomous vehicle using Monte Carlo tree search. IEEE Trans Intell Transp Syst 23(7):6766-6773
  64. Mo T-W, Chang RY, Kan T-Y (2022) DeepMCTS: deep rein- forcement learning assisted Monte Carlo tree search for MIMO detection. In: 2022 IEEE 95th vehicular technology conference: (VTC2022-Spring), pp 1-6
  65. Moerland TM, Broekens J, Plaat A, Jonker CM (2018) A0c: Alpha zero in continuous action space. arXiv:1805.09613 · arXiv
  66. Motomichi T, Yutaka I, Michio K. De (2021) novo generation of optically active small organic molecules using Monte Carlo tree search combined with recurrent neural network. J Comput Chem 42(3):136-143
  67. Oren J, Ross C, Lefarov M, Richter F, Taitler A, Feldman Z, Di Castro D, Daniel C (2021) SOLO: search online, learn offline for combinatorial optimization problems. In: Proceedings of the international symposium on combinatorial search, vol 12, pp 97- 105. Issue: 1
  68. Paxton C, Raman V, Hager GD, Kobilarov M (2017) Combining neural networks and tree search for task and motion planning in challenging environments. In: 2017 IEEE/RSJ international conference on intelligent robots and systems (IROS), pp. 6059- 6066
  69. Peng H, Wu C, Zhan Y, Xia Y (2022) Lore: a learning-based approach for workflow scheduling in clouds. In: Proceedings of the conference on research in adaptive and convergent systems, pp 47-52
  70. Raffin A, Hill A, Gleave A, Kanervisto A, Ernestus M, Dormann N (2021) Stable-Baselines3: reliable reinforcement learning imple- mentations. J Mach Learn Res 22(268):1-8
  71. Raina A, Cagan J, McComb C (2022) Learning to design without prior data: Discovering generalizable design strategies using deep learning and tree search. J Mech Des 145(3):031402
  72. Rinciog A, Mieth C, Scheikl PM, Meyer A (2020) Sheet-metal production scheduling using AlphaGo Zero. In: Proceedings of the conference on production systems and logistics: CPSL 2020
  73. Riviere B, Hönig W, Anderson M, Chung S-J (2021) Neural tree expansion for multi-robot planning in non-cooperative environ- ments. IEEE Robot Autom Lett 6(4):6868-6875
  74. Rosin CD (2011) Multi-armed bandits with episode context. Ann Math Artif Intell 61(3):203-230
  75. Sadeghnejad-Barkousaraie A, Bohara G, Jiang S, Nguyen D (2021) A reinforcement learning application of a guided Monte Carlo Tree Search algorithm for beam orientation selection in radiation therapy. Mach Learn: Sci Technol 2(3):035013
  76. Sadeghnejad Barkousaraie A, Ogunmolu O, Jiang S, Nguyen D (2019) Using supervised learning and guided Monte Carlo tree search for beam orientation optimization in radiation therapy. In: Workshop on artificial intelligence in radiation therapy, Springer, pp 1-9
  77. Samsonov V, Kemmerling M, Paegert M, Lütticke D, Sauermann F, Gützlaff A, Schuh G, Meisen T (2021) Manufacturing control in job shop environments with reinforcement learning. In: Proceed- ings of the 13th international conference on agents and artificial intelligence, pp 589-597
  78. Sandberg D, Kvernvik T, Calabrese FD (2022) Learning robust scheduling with search and attention. In: ICC 2022-IEEE international conference on communications, IEEE, pp 1549- 1555
  79. Schmidt D, Moran N, Rosenfeld JS, Rosenthal J, Yedidia J (2019) Self-play learning without a reward metric. arXiv:1912.07557 · arXiv
  80. Segler MH, Preuss M, Waller MP (2018) Planning chemical syntheses with deep neural networks and symbolic AI. Nature 555(7698):604-610
  81. Segler MHS, Preuss M, Waller MP (2017) Towards "AlphaChem": chemical synthesis planning with tree search and deep neural network policies. In: 5th International conference on learning representations, ICLR 2017, Toulon, France, April 24-26, 2017, Workshop Track Proceedings
  82. Shafieirad H, Adve RS (2022) On meeting a maximum delay constraint using reinforcement learning. IEEE Access 10:97897- 97911
  83. Shaghaghi M, Adve RS, Ding Z (2019) Resource manage- ment for multifunction multichannel cognitive radars. In: 2019 53rd Asilomar conference on signals, systems, and computers, pp 1550-1554
  84. Shao Y, Liew SC, Wang T (2020) AlphaSeq: sequence discovery with deep reinforcement learning. IEEE Trans Neural Netw Learn Syst 31(9):3319-3333
  85. Shen Y, Chen J, Huang P-S, Guo Y, Gao J (2018) M-walk: Learn- ing to walk over graphs using Monte Carlo tree search. Adv Neural Inf Process Syst 31
  86. Shuai H, He H (2021) Online scheduling of a residential microgrid via Monte-Carlo tree search and a learned model. In: 2021 IEEE power & energy society general meeting (PESGM), pp 01
  87. Shuai H, Li F, She B, Wang X, Zhao J (2023) Post-storm repair crew dispatch for distribution grid restoration using stochastic Monte Carlo tree search and deep neural networks. Int J Electr Power Energy Syst 144(4):108477
  88. Silva K, Abeyasekare W, Dasanayake D, Nandisena T, Kasthuri- rathna D, Kugathasan A (2021) Dynamic user interface person- alization based on deep reinforcement learning. In: 2021 3rd international conference on advancements in computing (ICAC), IEEE, pp 25-30
  89. Silver D, Huang A, Maddison CJ, Guez A, Sifre L, Van Den Driessche G, Schrittwieser J, Antonoglou I, Panneershelvam V, Lanctot M et al (2016) Mastering the game of Go with deep neural networks and tree search. Nature 529(7587):484-489 100.
  90. Silver D, Schrittwieser J, Simonyan K, Antonoglou I, Huang A, Guez A, Hubert T, Baker L, Lai M, Bolton A et al (2017) Mastering the game of go without human knowledge. Nature 550(7676):354-359 102.
  91. Sinha A, Azad U, Singh H (2022) Qubit routing using graph neural network aided Monte Carlo tree search. In: Proceedings of the AAAI conference on artificial intelligence, vol 36, pp 9935-9943. Issue: 9 103.
  92. Skrynnik A, Yakovleva A, Davydov V, Yakovlev K, Panov AI (2021) Hybrid policy learning for multi-agent pathfinding. IEEE Access 9:126034-126047 104.
  93. Srinivasan S, Batra R, Chan H, Kamath G, Cherukara MJ, Sankaranarayanan SK (2021) Artificial intelligence-guided De novo molecular design targeting COVID-19. ACS Omega 6(19):12557-12566 107.
  94. Sun R, Liu Y (2021) Hybrid reinforcement learning for power transmission network self-healing considering wind power. IEEE Trans Neural Netw Learn Syst 1-11 108.
  95. Sutton RS, Barto AG (2018) Reinforcement learning: An intro- duction. MIT press 109.
  96. Ss SV, Law JN, Tripp CE, Duplyakin D, Skordilis E, Biagioni D, Paton RS, St. John PC (2022) Multi-objective goal-directed optimization of de novo stable organic radicals for aqueous redox flow batteries. Nat Mach Intell 4(8):720-730 110.
  97. Tang C, Fu S, Liu F (2021) Design and implementation of sys- tem for generating MOFs for hydrogen storage in hydrogen-fueled vehicles. In: 2021 IEEE international conference on artificial intel- ligence and industrial design (AIID), IEEE, pp 549-553 111.
  98. Thacker HK, Kumar A, Barari A, Damini D, Gupta A, Jagan- nathachar KK, Yoon D (2021) AlphaRA: an alphazero based approach to redundancy analysis. In: 2021 20th IEEE interna- tional conference on machine learning and applications (ICMLA), pp 477-483 112.
  99. Thakkar A, Kogej T, Reymond J-L, Engkvist O, Bjerrum EJ (2020) Datasets and their influence on the development of com- puter assisted synthesis planning tools in the pharmaceutical domain. Chem Sci 11(1):154-168 113.
  100. Todi K, Bailly G, Leiva L, Oulasvirta A (2021) Adapting user interfaces with model-based reinforcement learning. In: Proceed- ings of the 2021 CHI conference on human factors in computing systems, pp 1-13 114.
  101. Tomin N (2020) The concept of constructing an artificial dis- patcher intelligent system based on deep reinforcement learning for the automatic control system of electric networks. J Comput Syst Sci Int 59(6):939-956 115.
  102. Van Eyck J, Ramon J, Guiza F, Meyfroidt G, Bruynooghe M, Van den Berghe G (2013) Guided Monte Carlo tree search for planning in learned environments. In: Asian conference on machine learning, PMLR, pp 33-47 116.
  103. Vom Brocke J, Simons A, Riemer K, Niehaves B, Plattfaut R, Cleven A (2015) Standing on the shoulders of giants: Challenges and recommendations of literature search in information systems research. Commun Assoc Inf Syst 37(1):9 118.
  104. Wang C, Chen X, Luo Y, Zhang G (2022) Solving virtual network mapping fast by combining neural network and MCTS. In: 2021 Ninth international conference on advanced cloud and big data (CBD), pp 19-24 119.
  105. Wang H, Liang W, Yu L-F (2020) Scene mover: automatic move planning for scene arrangement by deep reinforcement learning. ACM Trans Graph 39(6):1-15 120.
  106. Wang JH, Cheng Luo P, Xiong HQ, Zhang BW, Peng JY (2020) Parallel machine workshop scheduling using the integration of proximal policy optimization training and Monte Carlo tree search. In: 2020 Chinese automation congress (CAC), pp 3277- 3282 122.
  107. Wang R, Zhou M, Li Y, Zhang Q, Dong H (2019) A timetable rescheduling approach for railway based on Monte Carlo tree search. In: 2019 IEEE intelligent transportation systems confer- ence (ITSC), pp 3738-3743 125.
  108. Wang S, Zhou K, Lai K, Shen J (2020) Task-completion dialogue policy learning via Monte Carlo tree search with dueling network. In Proceedings of the 2020 conference on empirical methods in natural language processing (EMNLP), pp 3461-3471 126.
  109. Wang X, Qian Y, Gao H, Coley CW, Mo Y, Barzilay R, Jensen KF (2020) Towards efficient discovery of green synthetic pathways with Monte Carlo tree search and reinforcement learning. Chem Sci 11(40):10959-10972 127.
  110. Wang Z, Li C (2022) Channel pruning via lookahead search guided reinforcement learning. In 2022 IEEE/CVF winter confer- ence on applications of computer vision (WACV), pp 3513-3524 130.
  111. Webster J, Watson RT (2002) Analyzing the past to prepare for the future: writing a literature review. MIS Q xiii-xxiii 131.
  112. Weingertner P, Ho M, Timofeev A, Aubert S, Pita-Gil G (2020) Monte Carlo Tree search with reinforcement learning for motion planning. In: 2020 IEEE 23rd international conference on intelli- gent transportation systems (ITSC), pp 1-7 132.
  113. Weininger D (1988) SMILES, a chemical language and informa- tion system. 1. Introduction to methodology and encoding rules. J Chem Inf Comput Sci 28(1):31-36 133.
  114. Wu Q, Feng Q, Ren Y, Xia Q, Wang Z, Cai B (2021) An intelligent preventive maintenance method based on reinforcement learning for battery energy storage systems. IEEE Trans Industr Inform 17(12):8254-8264 134.
  115. Wu Y, Yao L (2021) Research on the problem of 3D bin pack- ing under incomplete information based on deep reinforcement learning. In: 2021 International conference on e-commerce and e-management (ICECEM), pp 38-42 135.
  116. Xiangyu Z, Kexin Z, Yongjin L (2020) Machine learning enabled tailor-made design of application-specific metal-organic frame- works. ACS Appl Mater Interfaces 12(1):734-743 136.
  117. Xing Z, Tu S (2020) A graph neural network assisted Monte Carlo tree search approach to traveling salesman problem. IEEE Access 8:108418-108428 137.
  118. Xing Z, Tu S, Xu L (2020) Solve traveling salesman prob- lem by Monte Carlo tree search and deep neural network. arXiv:2005.06879 138. · arXiv
  119. Xu R, Lieberherr K (2019) Learning self-game-play agents for combinatorial optimization problems. In: Proceedings of the 18th international conference on autonomous agents and multiagent systems, pp 2276-2278 139.
  120. Xu R, Lieberherr K (2020) Learning self-play agents for combi- natorial optimization problems. Knowl Eng Rev 35 140. Xu R, Lieberherr K (2022) On-the-fly model checking with neural MCTS. In: NASA formal methods symposium, Springer, pp 557- 575 141.
  121. Xu R, Lieberherr K (2022) Towards tackling QSAT problems with deep learning and Monte Carlo tree search. In: Science and information conference, Springer, pp 45-58 142.
  122. Yan M, Feng G, Zhou J, Qin S (2018) Smart multi-RAT access based on multiagent reinforcement learning. IEEE Trans Veh Technol 67(5):4539-4551 143.
  123. Yang J, Hou X, Hu YH, Liu Y, Pan Q (2020) A reinforcement learning scheme for active multi-debris removal mission planning with modified upper confidence bound tree search. IEEE Access 8:108461-108473 144.
  124. Yang R, Zou X, Nie Z, Yin C (2018) Research on deployment of communication node vehicles based on deep reinforcement learning. In: 2018 5th IEEE international conference on cloud computing and intelligence systems (CCIS), pp 484-487 145.
  125. Yang X, Zhang J, Yoshizoe K, Terayama K, Tsuda K (2017) ChemTS: an efficient python library for de novo molecular gen- eration. Sci Technol Adv Mater 18(1):972-976 146.
  126. Yin C, Yang R, Zhu W, Zou X, Zhang J (2021) Optimal planning of emergency communication network using deep reinforcement learning. IEICE Trans Commun 104(1):20-26 147.
  127. Yu T, Huang J, Chang Q (2020) Mastering the working sequence in human-robot collaborative assembly based on reinforcement learning. IEEE Access 8:163868-163877 148.
  128. Zhang B, Zhang X, Du W, Song Z, Zhang G, Zhang G, Wang Y, Chen X, Jiang J, Luo Y (2022) Chemistry-informed molecular graph as reaction descriptor for machine-learned retrosynthesis planning. Proc Natl Acad Sci 119(41):e2212711119 149.
  129. Zhang C, Song W, Cao Z, Zhang J, Tan PS, Chi X (2020) Learn- ing to dispatch for job shop scheduling via deep reinforcement learning. Adv Neural Inf Process Syst 33:1621-1632 150.
  130. Zhang J, Chen H, Song S, Hu F (2020) Reinforcement learning- based motion planning for automatic parking system. IEEE Access 8:154485-154501 151.
  131. Zhang J, Zhou K, Schelter S (2020) Alphajoin: join order selection á la AlphaGo. In: CEUR workshop proceedings, vol 2652 152.
  132. Zhang M, Huang Q, Wang S, Wang Z (2018) Construction of LDPC codes based on deep reinforcement learning. In: 2018 10th international conference on wireless communications and signal processing (WCSP), pp 1-4 154.
  133. Zhang Y, Wang W, Zhang P, Huang P (2022) Reinforcement- learning-based task planning for self-reconfiguration of cellular satellites. IEEE Aerosp Electron Syst Mag 37(6):38-47 155.
  134. Zhong S, Zhao Y, Chen J (2019) Learning to recover sparse sig- nals. In: 2019 57th Annual allerton conference on communication, control, and computing (Allerton), pp 995-1000 156.
  135. Zombori Z, Urban J, Brown CE (2020) Prolog technology rein- forcement learning prover. In: International joint conference on automated reasoning, Springer, pp 489-507 157.
  136. Zombori Z, Urban J, Olšák M (2021) The role of entropy in guid- ing a connection prover. In: International conference on automated reasoning with analytic tableaux and related methods, Springer, pp 218-235 158.
  137. Zou L, Xia L, Ding Z, Yin D, Song J, Liu W (2019) Reinforcement learning to diversify top-n recommendation. In: International con- ference on database systems for advanced applications, Springer, pp 104-120 159.
  138. Zou X, Yang R, Yin C (2019) Research on node layout model optimization of MANET based on AlphaZero technology under incomplete visual terrain. In: Proceedings of the 2019 interna- tional conference on artificial intelligence and computer science, pp 562-565 160.
  139. Zou X, Yang R, Yin C, Nie Z, Wang H (2020) Deploying tacti- cal communication node vehicles with AlphaZero algorithm. IET Commun 14(9):1392-1396 161.
  140. Świechowski M, Godlewski K, Sawicki B, Mańdziuk J (2022) Monte, Carlo tree search: A review of recent modifications and applications. Artif Intell Rev 1-66

Cited by

Sorted by citations

Authors
Kemmerling, M., Lütticke, D., & Schmitt, R. H.
Affiliations
RWTH Aachen University (DE)
Corresponding author
Kemmerling, Marco
ORCID
Venue
Applied Intelligence
Publisher
Springer Science and Business Media LLC
Published
2023-12-28
Volume
54
Issue
1
Pages
1020-1046
ISSN
  • 0924-669X (ISSN-L)
  • 1573-7497
DOI
10.1007/s10489-023-05240-w
OpenAlex
W4390345695
Cited by
19
References
3
Citations by year
2026: 5 · 2025: 9 · 2024: 4 · 2023: 1
FWCI
0.40
Type
journal article
Language
en
Open access
Licence
Locations
Applied Intelligence (publishedVersion, cc-by)
Funders
Deutsche Forschungsgemeinschaft (90621612)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
  • Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
  • Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
Keywords
Monte Carlo tree search, deep reinforcement learning, AlphaGo, planning algorithms, algorithm design

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review