Deep reinforcement learning and its applications in medical imaging and radiation therapy: a survey
Abstract
Reinforcement learning takes sequential decision-making approaches by learning the policy through trial and error based on interaction with the environment. Combining deep learning and reinforcement learning can empower the agent to learn the interactions and the distribution of rewards from state-action pairs to achieve effective and efficient solutions in more complex and dynamic environments. Deep reinforcement learning (DRL) has demonstrated astonishing performance in surpassing the human-level performance in the game domain and many other simulated environments. This paper introduces the basics of reinforcement learning and reviews various categories of DRL algorithms and DRL models developed for medical image analysis and radiation treatment planning optimization. We will also discuss the current challenges of DRL and approaches proposed to make DRL more generalizable and robust in a real-world environment. DRL algorithms, by fostering the designs of the reward function, agents interactions and environment models, can resolve the challenges from scarce and heterogeneous annotated medical image data, which has been a major obstacle to implementing deep learning models in the clinic. DRL is an active research area with enormous potential to improve deep learning applications in medical imaging and radiation therapy planning.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Deep learning and reinforcement learning are powerful AI techniques for medical data analysis.
“Deep learning and reinforcement learning are two powerful AI techniques for medical data analysis [17,30].”
From Edge AI for Early Detection of Chronic Diseases and the Spread of Infectious Diseases: Opportunities, Challenges, and Future Directions (2023) · cites this paper as [17]
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404 · see citing works
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834 · see citing works
(2014) · Cited by 33,419 · see citing works
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357 · see citing works
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928 · see citing works
Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., & Adam, H. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_49 · Cited by 20,054 · see citing works
Nonlinear total variation based noise removal algorithms
Rudin, L., Osher, S., & Fatemi, E. (1992) · Physica D Nonlinear Phenomena · doi:10.1016/0167-2789(92)90242-f · Cited by 17,089 · see citing works
Zhao, H., Shi, J., Qi, X., Wang, X., & Jia, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.660 · Cited by 16,438 · see citing works
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336 · see citing works
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612 · see citing works
Bellman, R. E. (2010) · doi:10.1515/9781400835386 · Cited by 14,588 · see citing works
V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation
Milletari, F., Navab, N., & Ahmadi, S.-A. (2016) · 2016 Fourth International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2016.79 · Cited by 11,931 · see citing works
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782 · see citing works
Proximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625 · see citing works
nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation
Isensee, F., Jaeger, P. F., Kohl, S. A. A., Petersen, J., & Maier-Hein, K. H. (2021) · Nature Methods · doi:10.1038/s41592-020-01008-z · Cited by 10,477 · see citing works
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083 · see citing works
Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
Williams, R. J. (1992) · Machine Learning · doi:10.1007/bf00992696 · Cited by 7,936 · see citing works
(2011) · Cited by 6,993 · see citing works
Shin, H.-C., Roth, H. R., Gao, M., Lu, L., Xu, Z., Nogues, I., Yao, J., Mollura, D., & Summers, R. M. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2528162 · Cited by 5,759 · see citing works
GrabCut: Interactive foreground extraction using iterated graph cuts
Rother, C., Kolmogorov, V., & Blake, A. (2004) · ACM Transactions on Graphics · doi:10.1145/1015706.1015720 · Cited by 5,703 · see citing works
Soft actor-critic Off-policy maximum entropy deep reinforcement learning with a stochastic actor
Haarnoja, T., Zhou, A., Abbeel, P., & Levine, S. (2018) · arXiv · doi:10.48550/arxiv.1801.01290 · Cited by 5,594 · see citing works
(1999) · Cited by 5,147 · see citing works
An Introduction to the Kalman Filter
Welch, G., & Bishop, G. (1995) · CiteSeer X (The Pennsylvania State University) · Cited by 4,128 · see citing works
AutoAugment: Learning Augmentation Strategies From Data
Cubuk, E. D., Zoph, B., Mané, D., Vasudevan, V., & Le, Q. V. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00020 · Cited by 4,026 · see citing works
(2013) · Cited by 3,740 · see citing works
Addressing Function Approximation Error in Actor-Critic Methods
Fujimoto, S., van Hoof, H., & Meger, D. (2018) · arXiv · doi:10.48550/arxiv.1802.09477 · Cited by 3,650 · see citing works
Grandmaster level in StarCraft II using multi-agent reinforcement learning
Vinyals, O., Babuschkin, I., Czarnecki, W. M., Mathieu, M., Dudzik, A., Chung, J., Choi, D. H., Powell, R., Ewalds, T., Georgiev, P., et al. (2019) · Nature · doi:10.1038/s41586-019-1724-z · Cited by 3,455 · see citing works
Deep Reinforcement Learning with Double Q-Learning
Van Hasselt, H., Guez, A., & Silver, D. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10295 · Cited by 3,318 · see citing works
The Human Connectome Project: A data acquisition perspective
Essen, D. C. V., Uǧurbil, K., Auerbach, E. J., Barch, D. M., Behrens, T. E. J., Bucholz, R. D., Chang, A., Chen, L., Corbetta, M., Curtiss, S. W., et al. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.02.018 · Cited by 2,605 · see citing works
Rainbow: Combining Improvements in Deep Reinforcement Learning
Hessel, M., Modayil, J., Van Hasselt, H., Schaul, T., Ostrovski, G., Dabney, W., Horgan, D., Piot, B., Azar, M., & Silver, D. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11796 · Cited by 1,846 · see citing works
Konda, V. R., & Tsitsiklis, J. N. (2002) · Cited by 1,764 · see citing works
Tractography-based connectomes are dominated by false-positive connections
Maier-Hein, K. H., Neher, P., Houde, J.-C., Côté, M.-A., Garyfallidis, E., Zhong, J., Chamberland, M., Yeh, F.-C., Lin, Y.-C., Ji, Q., et al. (2016) · doi:10.1101/084137 · Cited by 1,361 · see citing works
Mastering Atari, Go, chess and shogi by planning with a learned model
Schrittwieser, J., Antonoglou, I., Hubert, T., Simonyan, K., Sifre, L., Schmitt, S., Guez, A., Lockhart, E., Hassabis, D., Graepel, T., et al. (2020) · RePEc: Research Papers in Economics · Cited by 1,310 · see citing works
Learning to Communicate with Deep Multi-Agent Reinforcement Learning
Foerster, J. N., Assael, Y. M., de Freitas, N., & Whiteson, S. (2016) · arXiv · doi:10.48550/arxiv.1605.06676 · Cited by 1,217 · see citing works
Mastering Atari, Go, chess and shogi by planning with a learned model
Schrittwieser, J., Antonoglou, I., Hubert, T., Simonyan, K., Sifre, L., Schmitt, S., Guez, A., Lockhart, E., Hassabis, D., Graepel, T., et al. (2020) · Nature · doi:10.1038/s41586-020-03051-4 · Cited by 1,214 · see citing works
Yang, G., Yu, S., Dong, H., Slabaugh, G., Dragotti, P. L., Ye, X., Liu, F., Arridge, S., Keegan, J., Guo, Y., et al. (2017) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2017.2785879 · Cited by 1,075 · see citing works
Dota 2 with Large Scale Deep Reinforcement Learning
OpenAI, :, Berner, C., Brockman, G., Chan, B., Cheung, V., Dębiak, P., Dennison, C., Farhi, D., Fischer, Q., et al. (2019) · arXiv · doi:10.48550/arxiv.1912.06680 · Cited by 1,065 · see citing works
Deep Q-learning From Demonstrations
Hester, T., Vecerik, M., Pietquin, O., Lanctot, M., Schaul, T., Piot, B., Horgan, D., Quan, J., Sendonaris, A., Osband, I., et al. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11757 · Cited by 889 · see citing works
Learned Primal-Dual Reconstruction
Adler, J., & Öktem, O. (2018) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2018.2799231 · Cited by 870 · see citing works
DeepOrgan: Multi-level Deep Convolutional Networks for Automated Pancreas Segmentation
Roth, H. R., Lu, L., Farag, A., Shin, H.-C., Liu, J., Turkbey, E. B., & Summers, R. M. (2015) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-24553-9_68 · Cited by 784 · see citing works
Deep learning in medical imaging and radiation therapy
Sahiner, B., Pezeshk, A. X., Hadjiiski, L. M., Wang, X., Drukker, K., Cha, K. H., Summers, R. M., & Giger, M. L. (2018) · Medical Physics · doi:10.1002/mp.13264 · Cited by 739 · see citing works
(2016) · neural information processing systems · Cited by 735 · see citing works
Learning Spatio-Temporal Features with 3D Residual Networks for Action Recognition
Hara, K., Kataoka, H., & Satoh, Y. (2017) · 2017 IEEE International Conference on Computer Vision Workshops (ICCVW) · doi:10.1109/iccvw.2017.373 · Cited by 726 · see citing works
Sandfort, V., Yan, K., Pickhardt, P. J., & Summers, R. M. (2019) · Scientific Reports · doi:10.1038/s41598-019-52737-x · Cited by 684 · see citing works
Split Bregman Methods and Frame Based Image Restoration
Cai, J., Osher, S., & Shen, Z. (2009) · Multiscale Modeling and Simulation · doi:10.1137/090753504 · Cited by 678 · see citing works
Deep learning in medical image registration: a review
Fu, Y., Lei, Y., Wang, T., Curran, W. J., Liu, T., & Yang, X. (2020) · Physics in Medicine & Biology · doi:10.1088/1361-6560/ab843e · Cited by 661 · see citing works
Deep MR to CT Synthesis Using Unpaired Data
Wolterink, J. M., Dinkla, A. M., Savenije, M. H. F., Seevinck, P. R., van den Berg, C. A. T., & Išgum, I. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-68127-6_2 · Cited by 652 · see citing works
fastMRI: An Open Dataset and Benchmarks for Accelerated MRIPreprint
Žbontar, J., Knöll, F., Sriram, A., Murrell, T., Huang, Z., Muckley, M. J., Defazio, A., Stern, R., Johnson, P., Bruno, M. T., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.08839 · Cited by 613 · see citing works
A review of cooperative multi-agent deep reinforcement learning
Oroojlooy, A., & Hajinezhad, D. (2022) · Applied Intelligence · doi:10.1007/s10489-022-04105-y · Cited by 607 · see citing works
Exploration by Random Network Distillation
Burda, Y., Edwards, H., Storkey, A., & Klimov, O. (2018) · arXiv · doi:10.48550/arxiv.1810.12894 · Cited by 527 · see citing works
RL$^2$: Fast Reinforcement Learning via Slow Reinforcement Learning
Duan, Y., Schulman, J., Chen, X., Bartlett, P. L., Sutskever, I., & Abbeel, P. (2016) · arXiv · doi:10.48550/arxiv.1611.02779 · Cited by 503 · see citing works
DeepIGeoS: A Deep Interactive Geodesic Framework for Medical Image Segmentation
Wang, G., Zuluaga, M. A., Li, W., Pratt, R., Patel, P. A., Aertsen, M., Doel, T., David, A. L., Deprest, J., Ourselin, S., et al. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2840695 · Cited by 502 · see citing works
Value-Decomposition Networks For Cooperative Multi-Agent LearningPreprint
Sunehag, P., Lever, G., Gruslys, A., Czarnecki, W. M., Zambaldi, V., Jaderberg, M., Lanctot, M., Sonnerat, N., Leibo, J. Z., Tuyls, K., et al. (2017) · arXiv (Cornell University) · doi:10.4230/lipics.cp.2023.25 · Cited by 489 · see citing works
Fine-Tuning Convolutional Neural Networks for Biomedical Image Analysis: Actively and Incrementally
Zhou, Z., Shin, J., Zhang, L., Gurudu, S., Gotway, M. B., & Liang, J. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.506 · Cited by 430 · see citing works
Li, W., Wang, G., Fidon, L., Sebastien Ourselin, A., Cardoso, M. J., & Vercauteren, T. K. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-59050-9_28 · Cited by 336 · see citing works
Multi-Scale Deep Reinforcement Learning for Real-Time 3D-Landmark Detection in CT Scans
Ghesu, F.-C., Georgescu, B., Zheng, Y., Grbic, S., Maier, A., Hornegger, J., & Comaniciu, D. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2782687 · Cited by 310 · see citing works
Reinforcement Learning with Augmented Data
Laskin, M., Lee, K., Stooke, A., Pinto, L., Abbeel, P., & Srinivas, A. (2020) · arXiv · doi:10.48550/arxiv.2004.14990 · Cited by 299 · see citing works
SynSeg-Net: Synthetic Segmentation Without Target Modality Ground Truth
Huo, Y., Xu, Z., Moon, H., Bao, S., Assad, A., Moyo, T. K., Savona, M. R., Abramson, R. G., & Landman, B. A. (2018) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2018.2876633 · Cited by 278 · see citing works
Learning how to Active Learn: A Deep Reinforcement Learning Approach
Fang, M., Li, Y., & Cohn, T. (2017) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d17-1063 · Cited by 262 · see citing works
Chuquicusma, M. J. M., Hussein, S., Burt, J. R., & Bağcı, U. (2018) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi.2018.8363564 · Cited by 258 · see citing works
Reinforcement learning for robot soccer
Riedmiller, M., Gabel, T., Hafner, R., & Lange, S. (2009) · Autonomous Robots · doi:10.1007/s10514-009-9120-4 · Cited by 257 · see citing works
Deep reinforcement learning for automated radiation adaptation in lung cancer
Tseng, H., Luo, Y., Cui, S., Chien, J., Haken, R. K. T., & Naqa, I. E. (2017) · Medical Physics · doi:10.1002/mp.12625 · Cited by 237 · see citing works
An object-oriented representation for efficient reinforcement learning
Diuk, C., Cohen, A., & Littman, M. L. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390187 · Cited by 226 · see citing works
Deep reinforcement learning in medical imaging: A literature review
Zhou, S., Le, N., Luu, K., Nguyen, H. V., & Ayache, N. (2021) · Medical Image Analysis · doi:10.1016/j.media.2021.102193 · Cited by 222 · see citing works
An artificial agent for robust image registration
Liao, R., Miao, S., De Tournemire, P., Grbic, S., Kamen, A., Mansi, T., & Comaniciu, D. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.11230 · Cited by 199 · see citing works
Adversarial Image Synthesis for Unpaired Multi-modal Cardiac Data
Chartsias, A., Joyce, T., Dharmakumar, R., & Tsaftaris, S. A. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-68127-6_1 · Cited by 194 · see citing works
Medical image registration using deep neural networks: A comprehensive review
Boveiri, H. R., Khayami, R., Javidan, R., & Mehdizadeh, A. (2020) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2020.106767 · Cited by 179 · see citing works
Mookiah, M. R. K., Hogg, S., MacGillivray, T., et al. (2020) · Medical Image Analysis · doi:10.1016/j.media.2020.101905 · Cited by 167 · see citing works
Jiménez–del–Toro, O. A., Müller, H., Krenn, M., Gruenberg, K., Taha, A. A., Winterstein, M., Eggel, I., Foncubierta–Rodríguez, A., Göksel, O., András, J., et al. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2578680 · Cited by 160 · see citing works
Stochastic latent actor–critic: Deep reinforcement learning with a latent variable model
Lee, A. X., Nagabandi, A., Abbeel, P., & Levine, S. (2019) · arXiv · doi:10.48550/arxiv.1907.00953 · Cited by 159 · see citing works
Zeng, N., Li, H., Wang, Z., Liu, W., Liu, S., Alsaadi, F. E., & Liu, X. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.04.001 · Cited by 149 · see citing works
A survey on deep learning in medical image reconstruction
Ahishakiye, E., van Gijzen, M. B., Tumwiine, J., Wario, R. D., & Obungoloch, J. (2021) · Intelligent Medicine · doi:10.1016/j.imed.2021.03.003 · Cited by 148 · see citing works
Shen, C., Gonzalez, Y., Klages, P., et al. (2019) · Physics in Medicine and Biology · doi:10.1088/1361-6560/ab18bf · Cited by 124 · see citing works
Liao, X., Li, W., Xu, Q., Wang, X., Jin, B., Zhang, X., Wang, Y., & Zhang, Y. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00941 · Cited by 113 · see citing works
PixelRL: Fully Convolutional Network With Reinforcement Learning for Image Processing
Furuta, R., Inoue, N., & Yamasaki, T. (2019) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2019.2960636 · Cited by 111 · see citing works
Deep Reinforcement Learning for Active Breast Lesion Detection from DCE-MRI
Maicas, G., Carneiro, G., Bradley, A. P., Nascimento, J. C., & Reid, I. D. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-66179-7_76 · Cited by 109 · see citing works
Shen, C., Gonzalez, Y., Chen, L., Jiang, S. B., & Jia, X. (2018) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2018.2823679 · Cited by 105 · see citing works
Bayramoğlu, N., Kaakinen, M., Eklund, L., & Heikkilä, J. (2017) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2017.15 · Cited by 102 · see citing works
Shen, C., Nguyen, D. T. D., Chen, L., Gonzalez, Y., McBeth, R., Qin, N., Jiang, S. B., & Jia, X. (2020) · Medical Physics · doi:10.1002/mp.14114 · Cited by 101 · see citing works
A Multi-Agent Deep Reinforcement Learning Approach for Enhancement of COVID-19 CT Image Segmentation
Allioui, H., Mohammed, M. A., Benameur, N., Al‐Khateeb, B., Abdulkareem, K. H., García-Zapirain, B., Damaševičius, R., & Maskeliūnas, R. (2022) · Journal of Personalized Medicine · doi:10.3390/jpm12020309 · Cited by 92 · see citing works
Multimodal image registration with deep context reinforcement learning
Ma, K., Wang, J., Singh, V., Tamersoy, B., Chang, Y.-J., Wimmer, A., & Chen, T. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-66182-7_28 · Cited by 87 · see citing works
Curiosity-driven Exploration in Deep Reinforcement Learning via Bayesian Neural NetworksPreprint
Houthooft, R., Chen, X., Duan, Y., Schulman, J. D., de Turck, F., & Abbeel, P. (2016) · arXiv.org · doi:10.48550/arxiv.1605.09674 · Cited by 81 · see citing works
VR-Caps: A Virtual Environment for Capsule Endoscopy
İncetan, K., Celik, I. O., Obeid, A., Gokceler, G. I., Ozyoruk, K. B., Almalıoğlu, Y., Chen, R. J., Mahmood, F., Gilbert, H. B., Durr, N. J., et al. (2021) · Medical Image Analysis · doi:10.1016/j.media.2021.101990 · Cited by 80 · see citing works
Learning Spatio-Temporal Features with 3D Residual Networks for Action RecognitionPreprint
Hara, K., Kataoka, H., & Satoh, Y. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.07632 · Cited by 78 · see citing works
Deep Reinforcement Learning in Medicine
Jönsson, A. (2018) · Kidney Diseases · doi:10.1159/000492670 · Cited by 78 · see citing works
A Reinforcement Learning Framework for Medical Image Segmentation
Sahba, F., Tizhoosh, H. R., & Salama, M. M. A. (2006) · 2022 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2006.246725 · Cited by 72 · see citing works
Learning to Control Self-Assembling Morphologies: A Study of Generalization via ModularityPreprint
Pathak, D., Lu, C., Darrell, T., Isola, P., & Efros, A. A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.05546 · Cited by 65 · see citing works
End-to-end multimodal image registration via reinforcement learning
Hu, J., Luo, Z., Wang, X., Sun, S., Yin, Y., Cao, K., Song, Q., Lyu, S., & Wu, X. (2020) · Medical Image Analysis · doi:10.1016/j.media.2020.101878 · Cited by 61 · see citing works
Active MR k-space Sampling with Reinforcement Learning
Pineda, L., Basu, S., Romero, A., Calandra, R., & Drozdzal, M. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-59713-9_3 · Cited by 58 · see citing works
Weakly-Supervised teacher-Student network for liver tumor segmentation from non-enhanced images
Zhang, D., Chen, B., Chong, J., & Li, S. (2021) · Medical Image Analysis · doi:10.1016/j.media.2021.102005 · Cited by 57 · see citing works
Deep Reinforcement Learning for Weakly-Supervised Lymph Node Segmentation in CT Images
Li, Z., & Xia, Y. (2020) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2020.3008759 · Cited by 56 · see citing works
Al, W. A., & Yun, I. D. (2019) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2019.2946345 · Cited by 53 · see citing works
Hrinivich, W. T., & Lee, J. (2020) · Medical Physics · doi:10.1002/mp.14544 · Cited by 51 · see citing works
Shen, C., Chen, L., & Jia, X. (2021) · Physics in Medicine and Biology · doi:10.1088/1361-6560/ac09a2 · Cited by 49 · see citing works
Zhang, D., Chen, B., & Li, S. (2020) · Medical Image Analysis · doi:10.1016/j.media.2020.101861 · Cited by 47 · see citing works
Winkel, D. J., Weikert, T., Breit, H., Chabin, G., Gibson, E., Heye, T. J., Comaniciu, D., & Boll, D. T. (2020) · European Journal of Radiology · doi:10.1016/j.ejrad.2020.108918 · Cited by 46 · see citing works
Application of Opposition-Based Reinforcement Learning in Image Segmentation
Sahba, F., Tizhoosh, H. R., & Salama, M. M. (2007) · doi:10.1109/ciisp.2007.369176 · Cited by 45 · see citing works
Searching Learning Strategy with Reinforcement Learning for 3D Medical Image Segmentation
Yang, D., Roth, H. R., Xu, Z., Milletarì, F., Zhang, L., & Xu, D. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-32245-8_1 · Cited by 44 · see citing works
Dataset Growth in Medical Image Analysis Research
Kiryati, N., & Landau, Y. E. (2021) · Journal of Imaging · doi:10.3390/jimaging7080155 · Cited by 44 · see citing works
Multi-step medical image segmentation based on reinforcement learning
Tian, Z., Si, X., Zheng, Y., Chen, Z., & Li, X. (2020) · Journal of Ambient Intelligence and Humanized Computing · doi:10.1007/s12652-020-01905-3 · Cited by 43 · see citing works
Track-to-Learn: A general framework for tractography with deep reinforcement learning
Théberge, A., Desrosiers, C., Descoteaux, M., & Jodoin, P. (2021) · Medical Image Analysis · doi:10.1016/j.media.2021.102093 · Cited by 43 · see citing works
An Interpretable Planning Bot for Pancreas Stereotactic Body Radiation Therapy
Zhang, J., Wang, C., Sheng, Y., Palta, M., Czito, B., Willett, C., Zhang, J., Jensen, P. J., Yin, F.-F., Wu, Q., et al. (2021) · International Journal of Radiation Oncology*Biology*Physics · doi:10.1016/j.ijrobp.2020.10.019 · Cited by 41 · see citing works
Diff-DAC: Distributed Actor-Critic for Average Multitask Deep Reinforcement LearningPreprint
Macua, S. V., Tukiainen, A., Hernández, D. G.-O., Baldazo, D., de Cote, E. M., & Zazo, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.10363 · Cited by 36 · see citing works
Automatic Data Augmentation Via Deep Reinforcement Learning for Effective Kidney Tumor Segmentation
Qin, T., Wang, Z., He, K., Shi, Y., Gao, Y., & Shen, D. G. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9053403 · Cited by 36 · see citing works
Shen, C., Chen, L., Gonzalez, Y., et al. (2021) · Medical Physics · doi:10.1002/mp.14712 · Cited by 36 · see citing works
A General Framework for Context-Specific Image Segmentation Using Reinforcement Learning
Wang, L., Lekadir, K., Su-Lin, L., Merrifield, R. D., & Yang, G. (2013) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2013.2252431 · Cited by 33 · see citing works
MRI Reconstruction with Interpretable Pixel-Wise Operations Using Reinforcement Learning
Li, W., Feng, X., An, H., Ng, X. Y., & Zhang, Y. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i01.5423 · Cited by 33 · see citing works
Li, H., & Yin, Z. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-59710-8_1 · Cited by 33 · see citing works
Ma, C., Xu, Q., Wang, X., Jin, B., Zhang, X., Wang, Y., & Zhang, Y. (2020) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2020.3048477 · Cited by 31 · see citing works
Deep Reinforcement Learning for Organ Localization in CTPreprint
Navarro, F., Sekuboyina, A., Waldmannstetter, D., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.04974 · Cited by 29 · see citing works
Importance sampling in reinforcement learning with an estimated behavior policy
Hanna, J. P., Niekum, S., & Stone, P. (2021) · Machine Learning · doi:10.1007/s10994-020-05938-9 · Cited by 26 · see citing works
Pradella, M., Weikert, T., Sperl, J. I., Kärgel, R., Cyriac, J., Achermann, R., Sauter, A. W., Bremerich, J., Stieltjes, B., Brantner, P., et al. (2021) · Quantitative Imaging in Medicine and Surgery · doi:10.21037/qims-21-142 · Cited by 24 · see citing works
Learning to scan: A deep reinforcement learning approach for personalized scanning in CT imaging
Shen, Z., Wang, Y., Wu, D., Yang, X., & Dong, B. (2021) · Inverse Problems and Imaging · doi:10.3934/ipi.2021045 · Cited by 22 · see citing works
Software Agent with Reinforcement Learning Approach for Medical Image Segmentation
Chitsaz, M., & Seng, W. C. (2011) · Journal of Computer Science and Technology · doi:10.1007/s11390-011-9431-8 · Cited by 18 · see citing works
Carver, E. N., Dai, Z., Liang, E., et al. (2021) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2020.495075 · Cited by 16 · see citing works
Barkousaraie, A. S., Bohara, G., Jiang, S. B., et al. (2021) · Machine Learning Science and Technology · doi:10.1088/2632-2153/abe528 · Cited by 16 · see citing works
Medical Image Segmentation by Using Reinforcement Learning Agent
Chitsaz, M., & Seng, W. C. (2009) · International Conference on Digital Image Processing · doi:10.1109/icdip.2009.14 · Cited by 15 · see citing works
An Artificial Agent for Robust Image RegistrationPreprint
Liao, R., Miao, S., de Tournemire, P., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.10336 · Cited by 13 · see citing works
Deep action learning enables robust 3D segmentation of body organs in various CT and MRI images
Zhong, X., Amrehn, M., Ravikumar, N., Chen, S., Strobel, N. K., Birkhold, A., Kowarschik, M., Fahrig, R., & Maier, A. (2021) · Scientific Reports · doi:10.1038/s41598-021-82370-6 · Cited by 13 · see citing works
Edge-Sensitive Left Ventricle Segmentation Using Deep Reinforcement Learning
Xiong, J., Po, L.-M., Cheung, K.-W., Xian, P., Zhao, Y., Rehman, Y. A. U., & Zhang, Y. (2021) · Sensors · doi:10.3390/s21072375 · Cited by 13 · see citing works
A hierarchical reinforcement learning algorithm based on heuristic reward function
Yan, Q., Liu, Q., & Hu, D. (2010) · International Conference on Advanced Computer Control · doi:10.1109/icacc.2010.5486837 · Cited by 12 · see citing works
Asperti, A., & Mastronardo, C. (2018) · International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC) · doi:10.5220/0006730901990205 · Cited by 12 · see citing works
Torrents‐Barrena, J., Piella, G., Gratacós, E., Eixarch, E., Ceresa, M., & Ballester, M. Á. G. (2020) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2020.2987981 · Cited by 12 · see citing works
Towards General and Autonomous Learning of Core Skills: A Case Study in LocomotionPreprint
Hafner, R., Hertweck, T., Klöppner, P., Bloesch, M., Neunert, M., Wulfmeier, M., Tunyasuvunakool, S., Heess, N., & Riedmiller, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.12228 · Cited by 12 · see citing works
Winkel, D. J., Breit, H., Weikert, T., et al. (2021) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-020-00398-y · Cited by 9 · see citing works
Learning Heuristics for Quantified Boolean Formulas through Deep Reinforcement LearningPreprint
Lederman, G., Rabe, M. N., Lee, E. A., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.08058 · Cited by 8 · see citing works
Jang, Y., & Jeon, B. (2021) · Sensors · doi:10.3390/s21186187 · Cited by 8 · see citing works
AutoPath: Image-Specific Inference for 3D Segmentation
Sun, D., Wang, Y., Ni, D., & Wang, T. (2020) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2020.00049 · Cited by 5 · see citing works
Deep Reinforcement Learning for L3 Slice Localization in Sarcopenia Assessment
Laousy, O., Chassagnon, G., Oyallon, E., et al. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-87589-3_33 · Cited by 5 · see citing works
Anastassacos, N., Hailes, S., & Musolesi, M. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6190 · Cited by 4 · see citing works
Learning from dispersed manual annotations with an optimized data weighting policy
Tang, Y., Gao, R., Chen, Y., et al. (2020) · Journal of medical imaging · doi:10.1117/1.jmi.7.4.044002 · Cited by 4 · see citing works
GDI: Rethinking What Makes Reinforcement Learning Different From Supervised LearningPreprint
Fan, J., Xiao, C., & Huang, Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.06232 · Cited by 4 · see citing works
Multimodal Reward Shaping for Efficient Exploration in Reinforcement LearningPreprint
Yuan, M., Pun, M.-O., Wang, D., Chen, Y., & Li, H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.08888 · Cited by 4 · see citing works
Stember, J. N., & Shalu, H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.02895 · Cited by 3 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Badidi, E. (2023) · Future Internet · doi:10.3390/fi15110370 · Cited by 113 · see citing works
Wali, S., Khan, M. I., Samina, S., & Raza Sialvi, M. A. (2026) · 2026 11th International Conference on Image, Vision and Computing (ICIVC) · doi:10.1109/icivc70586.2026.11686729
- Authors
- Xu, L., Zhu, S., & Wen, N.
- Affiliations
- Oakland University (US)
- Henry Ford Health (US)
- Shanghai Jiao Tong University (CN)
- Venue
- Physics in Medicine & Biology
- Publisher
- IOP Publishing
- Published
- 2022-11-11
- Volume
- 67
- Issue
- 22
- Pages
- 22TR02
- ISSN
- 0031-9155 (ISSN-L)
- 1361-6560
- OpenAlex
- W4307023898
- Cited by
- 35
- References
- 135
- Citations by year
- 2026: 9 · 2025: 11 · 2024: 9 · 2023: 6
- FWCI
- 0.45
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- Physics in Medicine and Biology (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Physics and Astronomy › Radiation › Advanced Radiotherapy Techniques
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- deep reinforcement learning, medical image analysis, radiotherapy treatment planning, reward function design, sequential decision making
- MeSH terms
- Algorithms, Humans, Radiography, Reinforcement, Psychology, Neural Networks, Computer, Diffusion Magnetic Resonance Imaging
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
