Iteratively-Refined Interactive 3D Medical Image Segmentation With Multi-Agent Reinforcement Learning
Open accessAbstract
Existing automatic 3D image segmentation methods usually fail to meet the clinic use. Many studies have explored an interactive strategy to improve the image segmentation performance by iteratively incorporating user hints. However, the dynamic process for successive interactions is largely ignored. We here propose to model the dynamic process of iterative interactive image segmentation as a Markov decision process (MDP) and solve it with reinforcement learning (RL). Unfortunately, it is intractable to use single-agent RL for voxel-wise prediction due to the large exploration space. To reduce the exploration space to a tractable size, we treat each voxel as an agent with a shared voxel-level behavior strategy so that it can be solved with multi-agent reinforcement learning. An additional advantage of this multi-agent model is to capture the dependency among voxels for segmentation task. Meanwhile, to enrich the information of previous segmentations, we reserve the prediction uncertainty in the state space of MDP and derive an adjustment action space leading to a more precise and finer segmentation. In addition, to improve the efficiency of exploration, we design a relative cross-entropy gain-based reward to update the policy in a constrained direction. Experimental results on various medical datasets have shown that our method significantly outperforms existing state-of-the-art methods, with the advantage of less interactions and a faster convergence.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Existing interactive segmentation approaches, including foundation-model-based ones, still require the user to manually provide explicit and spatially precise inputs.
“Despite these advancements, existing interactive approaches, including those based on foundation models, still require the user to manually provide explicit and spatially precise inputs.”
From Beyond Manual Annotation: A Human-AI Collaborative Framework for Medical Image Segmentation Using Only "Better or Worse" Expert Feedback (2025) · cites this paper as [12]
The Introduction (page unavailable) describes interactive segmentation using user-provided points, scribbles, or bounding boxes, and the State section describes clicks placed on error regions. These passages support the claim about explicit, spatially located inputs for the methods discussed, but the paper does not address foundation-model-based approaches, so it cannot support that extension. (LLM, conf=0.88)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404 · see citing works
Interactive medical image segmentation using deep learning with image-specific fine tuning
Wang, G., Li, W., Zuluaga, M. A., Pratt, R., Patel, P. A., Aertsen, M., Doel, T., David, A. L., Deprest, J., Ourselin, S., et al. (2018) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2018.2791721 · Cited by 843 · see citing works
Self-citationDeepIGeoS: A Deep Interactive Geodesic Framework for Medical Image Segmentation
Wang, G., Zuluaga, M. A., Li, W., Pratt, R., Patel, P. A., Aertsen, M., Doel, T., David, A. L., Deprest, J., Ourselin, S., et al. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2840695 · Cited by 502 · cited 9× · see citing works
“While previous works [5, 20] output directly the segmentation probability from the network, we here predict the adjustment amount based on the previous probability as actions to make the result more stable without abrupt changes.”
BackgroundMethodInfluentialSelf-citation
Flagged first, then by citations
Research works printed but not linked: 18
- David Acuna, Huan Ling, Amlan Kar, and Sanja Fidler. Ef- ficient interactive annotation of segmentation datasets with Polygon-RNN++. In CVPR, pages 859-868, 2018.
- Yuri Boykov and Vladimir Kolmogorov. An experimen- tal comparison of Min-Cut/Max-Flow algorithms for energy minimization in vision. IEEE Transactions on Pattern Anal- ysis & Machine Intelligence, (9):1124-1137, 2004.
- Yuri Y Boykov and M-P Jolly. Interactive graph cuts for op- timal boundary & region segmentation of objects in nd im- ages. In ICCV, pages 105-112, 2001.
- Gustav Bredell, Christine Tanner, and Ender Konukoglu. Iterative interaction training for segmentation editing net- works. In MLMI International Workshop, pages 363-370, 2018.
- Lluis Castrejon, Kaustav Kundu, Raquel Urtasun, and Sanja Fidler. Annotating object instances with a Polygon-RNN. In CVPR, pages 5230-5238, 2017.
- Özgün C ¸ic ¸ek, Ahmed Abdulkadir, Soeren S Lienkamp, Thomas Brox, and Olaf Ronneberger. 3D U-Net: learning dense volumetric segmentation from sparse annotation. In MICCAI, pages 424-432, 2016.
- Antonio Criminisi, Toby Sharp, and Andrew Blake. Geos: Geodesic image segmentation. In ECCV, pages 99-112, 2008.
- Ryosuke Furuta, Naoto Inoue, and Toshihiko Yamasaki. Fully convolutional network with multi-step reinforcement learning for image processing. In AAAI, pages 3598-3605, 2019.
- Philipp Krähenbühl and Vladlen Koltun. Efficient inference in fully connected CRFs with gaussian edge potentials. In NeurIPS, pages 109-117, 2011.
- Wenqi Li, Guotai Wang, Lucas Fidon, Sebastien Ourselin, M Jorge Cardoso, and Tom Vercauteren. On the compact- ness, efficiency, and representation of 3D convolutional net- works: brain parcellation as a pretext task. In IPMI, pages 348-360, 2017.
- Bjoern Menze, Andras Jakab, Stefan Bauer, Jayashree Kalpathy-Cramer, Keyvan Farahani, Justin Kirby, Yuliya Burren, Nicole Porz, Johannes Slotboom, Roland Wiest, Levente Lanczi, Elisabeth Gerstner, et al. The multimodal brain tumor image segmentation benchmark (BraTS). IEEE Transactions on Medical Imaging, page 33, 2014.
- Fausto Milletari, Nassir Navab, and Seyed-Ahmad Ahmadi. V-Net: Fully convolutional neural networks for volumetric medical image segmentation. In 3DV, pages 565-571, 2016.
- Volodymyr Mnih, Adria Puigdomenech Badia, Mehdi Mirza, Alex Graves, Timothy Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. Asynchronous methods for deep reinforcement learning. In ICML, pages 1928-1937, 2016.
- Martin Rajchl, Matthew CH Lee, Ozan Oktay, Konstanti- nos Kamnitsas, Jonathan Passerat-Palmbach, Wenjia Bai, Mellisa Damodaram, Mary A Rutherford, Joseph V Hajnal, Bernhard Kainz, et al. Deepcut: Object segmentation from bounding box annotations using convolutional neural net- works. IEEE Transactions on Medical Imaging, 36(2):674- 683, 2016.
- Carsten Rother, Vladimir Kolmogorov, and Andrew Blake. GrabCut: Interactive foreground extraction using iterated graph cuts. In ACM Transactions on Graphics, volume 23, pages 309-314, 2004.
- Gwangmo Song, Heesoo Myeong, and Kyoung Mu Lee. Seednet: Automatic seed generation with deep reinforce- ment learning for robust interactive segmentation. In CVPR, pages 1760-1768, 2018.
- Ning Xu, Brian Price, Scott Cohen, Jimei Yang, and Thomas S Huang. Deep interactive object selection. In CVPR, pages 373-381, 2016.
- Zongwei Zhou, Md Mahfuzur Rahman Siddiquee, Nima Tajbakhsh, and Jianming Liang. UNet++: A nested U-Net architecture for medical image segmentation. In DLMIA, pages 3-11. 2018.
Cited by
Sorted by citations
A survey on active learning and human-in-the-loop deep learning for medical image analysis
Budd, S., Robinson, E. C., & Kainz, B. (2021) · Medical Image Analysis · doi:10.1016/j.media.2021.102062 · Cited by 606 · see citing works
Luo, X., Wang, G., Song, T., Zhang, J., Aertsen, M., Deprest, J., Ourselin, S., Vercauteren, T., & Zhang, S. (2021) · Medical Image Analysis · doi:10.1016/j.media.2021.102102 · Cited by 126 · see citing works
Volumetric memory network for interactive medical image segmentation
Zhou, T., Li, L., Bredell, G., Li, J., Unkelbach, J., & Konukoglu, E. (2023) · Medical Image Analysis · doi:10.1016/j.media.2022.102599 · Cited by 109 · see citing works
Deep reinforcement learning and its applications in medical imaging and radiation therapy: a survey
Xu, L., Zhu, S., & Wen, N. (2022) · Physics in Medicine & Biology · doi:10.1088/1361-6560/ac9cb3 · Cited by 35 · see citing works
A deep learning-based interactive medical image segmentation framework with sequential memory
Mikhailov, I., Chauveau, B., Bourdel, N., & Bartoli, A. (2024) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2024.108038 · Cited by 15 · see citing works
Aresta, G., Araújo, T., Fazekas, B., Mai, J., Schmidt-Erfurth, U., & Bogunović, H. (2024) · IEEE Access · doi:10.1109/access.2024.3379015 · Cited by 5 · see citing works
“A common approach is to adjust the model’s weights case-wise according to the user interaction [21], [25], [26], [27].”
BackgroundTian, Z., He, Y., Sun, L., Li, Y., & Du, S. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128184 · Cited by 5 · see citing works
Qian, C., Yang, H., & Li, J. (2023) · 2023 IEEE International Conference on Systems, Man, and Cybernetics (SMC) · doi:10.1109/smc53992.2023.10393973 · Cited by 1 · see citing works
Zhang, Y. (2025) · arXiv · doi:10.48550/arxiv.2507.05815
Self-citation
- Authors
- Liao, X., Li, W., Xu, Q., Wang, X., Jin, B., Zhang, X., Wang, Y., & Zhang, Y.
- Affiliations
- Shanghai Jiao Tong University (CN)
- East China Normal University (CN)
- Venue
- 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2020-06
- Pages
- 9391-9399
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W3035127651
- Cited by
- 113
- References
- 3
- Citations by year
- 2026: 9 · 2025: 14 · 2024: 25 · 2023: 30 · 2022: 18 · 2021: 15 · 2020: 2
- FWCI
- 4.94
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
- Physical Sciences › Engineering › Biomedical Engineering › Medical Imaging and Analysis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- 3D medical image segmentation, interactive segmentation, multi-agent reinforcement learning, Markov decision process, user feedback, forecast uncertainty, exploration efficiency
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
