The DOI matches its Crossref record; 61 of the 61 references it cites were found in a registry or an index.
MIDeepSeg: Minimally interactive segmentation of unseen objects from medical images using deep learning
Open accessAbstract
Segmentation of organs or lesions from medical images plays an essential role in many clinical applications such as diagnosis and treatment planning. Though Convolutional Neural Networks (CNN) have achieved the state-of-the-art performance for automatic segmentation, they are often limited by the lack of clinically acceptable accuracy and robustness in complex cases. Therefore, interactive segmentation is a practical alternative to these methods. However, traditional interactive segmentation methods require a large number of user interactions, and recently proposed CNN-based interactive segmentation methods are limited by poor performance on previously unseen objects. To solve these problems, we propose a novel deep learning-based interactive segmentation method that not only has high efficiency due to only requiring clicks as user inputs but also generalizes well to a range of previously unseen objects. Specifically, we first encode user-provided interior margin points via our proposed exponentialized geodesic distance that enables a CNN to achieve a good initial segmentation result of both previously seen and unseen objects, then we use a novel information fusion method that combines the initial segmentation with only a few additional user clicks to efficiently obtain a refined segmentation. We validated our proposed framework through extensive experiments on 2D and 3D medical image segmentation tasks with a wide range of previously unseen objects that were not present in the training set. Experimental results showed that our proposed framework 1) achieves accurate results with fewer user interactions and less time compared with state-of-the-art interactive frameworks and 2) generalizes well to previously unseen objects.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
PyTorch: An Imperative Style, High-Performance Deep Learning Library
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al. (2019) · arXiv · doi:10.48550/arxiv.1912.01703 · Cited by 31,473
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2699184 · Cited by 22,392
Proceedings of IEEE Conference on Computer Vision and Pattern Recognition
(1993) · doi:10.1109/cvpr.1993 · Cited by 17,989
A survey on deep learning in medical image analysis
Litjens, G., Kooi, T., Bejnordi, B. E., Setio, A. A. A., Ciompi, F., Ghafoorian, M., van der Laak, J. A. W. M., van Ginneken, B., & Sánchez, C. I. (2017) · Medical Image Analysis · doi:10.1016/j.media.2017.07.005 · Cited by 15,332
Advances in neural information processing systems 7
(1996) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(96)90463-0 · Cited by 14,759
Proceedings of IEEE International Conference on Computer Vision
(1995) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.1995.466933 · Cited by 13,117
Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
Lafferty, J., McCallum, A., & Pereira, F. C. N. (2001) · ScholarlyCommons (University of Pennsylvania) · Cited by 12,935
A Combined Corner and Edge Detector
Harris, C., & Stephens, M. (1988) · Procedings of the Alvey Vision Conference 1988 · doi:10.5244/c.2.23 · Cited by 12,901
Yushkevich, P. A., Piven, J., Hazlett, H. C., Smith, R. G., Ho, S., Gee, J. C., & Gerig, G. (2006) · NeuroImage · doi:10.1016/j.neuroimage.2006.01.015 · Cited by 8,868 · cited 10×
“Two users used MIDeepSeg and two existing methods with good generalizability including ITK-SNAP (Yushkevich et al., 2006) and 3D versions of Graph Cuts (Boykov and Jolly, 2001), DIOS (Xu et al.”
BackgroundMethodInfluential3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation
Çiçek, Ö., Abdulkadir, A., Lienkamp, S. S., Brox, T., & Ronneberger, O. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46723-8_49 · Cited by 7,945
GrabCut: Interactive foreground extraction using iterated graph cuts
Rother, C., Kolmogorov, V., & Blake, A. (2004) · ACM Transactions on Graphics · doi:10.1145/1015706.1015720 · Cited by 5,703
Deep learning in medical image analysis
Shen, D., Wu, G., & Suk, H.-I. (2017) · Annual Review of Biomedical Engineering · doi:10.1146/annurev-bioeng-071516-044442 · Cited by 4,968
Interactive graph cuts for optimal boundary & region segmentation of objects in N-D images
Boykov, Y. Y., & Jolly, M.-P. (2002) · Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001 · doi:10.1109/iccv.2001.937505 · Cited by 3,674
Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning?
Tajbakhsh, N., Shin, J. Y., Gurudu, S. R., Hurst, R. T., Kendall, C. B., Gotway, M. B., & Liang, J. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2535302 · Cited by 3,277
Random Walks for Image Segmentation
Grady, L. (2006) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2006.233 · Cited by 2,624
Bottom-Up Object Detection by Grouping Extreme and Center Points
Zhou, X., Zhuo, J., & Krahenbuhl, P. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00094 · Cited by 1,088
Interactive medical image segmentation using deep learning with image-specific fine tuning
Wang, G., Li, W., Zuluaga, M. A., Pratt, R., Patel, P. A., Aertsen, M., Doel, T., David, A. L., Deprest, J., Ourselin, S., et al. (2018) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2018.2791721 · Cited by 843
Self-citationDeepIGeoS: A Deep Interactive Geodesic Framework for Medical Image Segmentation
Wang, G., Zuluaga, M. A., Li, W., Pratt, R., Patel, P. A., Aertsen, M., Doel, T., David, A. L., Deprest, J., Ourselin, S., et al. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2840695 · Cited by 502 · cited 21×
“Following DeepIGeoS (Wang et al., 2018b), we used open source code to compute geodesic distance 1 and solve Eq.”
BackgroundMethodInfluentialSelf-citation3D U-Net: Learning Dense Volumetric Segmentation from Sparse AnnotationPreprint
Çiçek, Ö., Abdulkadir, A., Lienkamp, S. S., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.06650 · Cited by 488
Deep Extreme Cut: From Extreme Points to Object Segmentation
Maninis, K.-K., Caelles, S., Pont-Tuset, J., & Gool, L. V. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00071 · Cited by 470
Deep Interactive Object Selection
Xu, N., Price, B., Cohen, S., Yang, J., & Huang, T. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.47 · Cited by 460
Efficient Interactive Annotation of Segmentation Datasets with Polygon-RNN++
Acuna, D., Ling, H., Kar, A., & Fidler, S. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00096 · Cited by 446
DeepCut: Object Segmentation From Bounding Box Annotations Using Convolutional Neural Networks
Rajchl, M., Lee, M. C. H., Oktay, O., Kamnitsas, K., Passerat-Palmbach, J., Bai, W., Damodaram, M., Rutherford, M. A., Hajnal, J. V., Kainz, B., et al. (2017) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2621185 · Cited by 410
Annotating Object Instances with a Polygon-RNN
Castrejón, L., Kundu, K., Urtasun, R., & Fidler, S. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.477 · Cited by 371
DeepMedic for Brain Tumor Segmentation
Kamnitsas, K., Ferrante, E., Parisot, S., Ledig, C., Nori, A. V., Criminisi, A., Rueckert, D., & Glocker, B. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-55524-9_14 · Cited by 346
Geodesic Matting: A Framework for Fast Interactive Image and Video Segmentation and Matting
Xue, B., & Sapiro, G. (2008) · International Journal of Computer Vision · doi:10.1007/s11263-008-0191-z · Cited by 300
GeoS: Geodesic Image Segmentation
Criminisi, A., Sharp, T., & Blake, A. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-88682-2_9 · Cited by 277
Large-Scale Interactive Object Segmentation With Human Annotators
Benenson, R., Popov, S., & Ferrari, V. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01197 · Cited by 255
Interactive Image Segmentation with Latent Diversity
Li, Z., Chen, Q., & Koltun, V. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00067 · Cited by 218
A Survey on Medical Image Segmentation
Masood, S., Sharif, M., Masood, A., Yasmin, M., & Raza, M. (2015) · Current Medical Imaging Formerly Current Medical Imaging Reviews · doi:10.2174/157340561101150423103441 · Cited by 190
Geodesic graph cut for interactive image segmentation
Price, B. L., Morse, B. S., & Cohen, S. D. (2010) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2010.5540079 · Cited by 186
Deep GrabCut for Object Selection
Xu, N., Price, B. L., Cohen, S. D., Yang, J., & Huang, T. S. (2017) · British Machine Vision Conference (BMVC) · doi:10.5244/c.31.182 · Cited by 149
(2013) · Liverpool John Moores University · Cited by 133
Active Learning for Interactive 3D Image Segmentation
Top, A., Hamarneh, G., & Rafeef Abugharbieh, A. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-23626-6_74 · Cited by 121
A fully convolutional two-stream fusion network for interactive image segmentation
Hu, Y., Soltoggio, A., Lock, R., & Carter, S. (2018) · Neural Networks · doi:10.1016/j.neunet.2018.10.009 · Cited by 113
Liao, X., Li, W., Xu, Q., Wang, X., Jin, B., Zhang, X., Wang, Y., & Zhang, Y. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00941 · Cited by 113
Bottom-up Object Detection by Grouping Extreme and Center PointsPreprint
Zhou, X., Zhuo, J., & Krähenbühl, P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.08043 · Cited by 105
Object Instance Annotation With Deep Extreme Level Set Evolution
Wang, Z., Acuna, D., Ling, H., Kar, A., & Fidler, S. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00768 · Cited by 83
Lee, K. M., Myeong, H., & Song, G. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00189 · Cited by 74
Wang, G., Zuluaga, M. A., Pratt, R., Aertsen, M., Doel, T., Klusmann, M., David, A. L., Deprest, J., Vercauteren, T., & Ourselin, S. (2016) · Medical Image Analysis · doi:10.1016/j.media.2016.04.009 · Cited by 73
Self-citationGoing to Extremes: Weakly Supervised Medical Image Segmentation
Roth, H. R., Yang, D., Xu, Z., Wang, X., & Xu, D. (2021) · Machine Learning and Knowledge Extraction · doi:10.3390/make3020026 · Cited by 67
Interactive segmentation of medical images through fully convolutional neural networksPreprint
Sakinis, T., Milletarì, F., Roth, H. R., Korfiatis, P. D., Kostandy, P., Philbrick, K. A., Akkus, Z., Xu, Z., Xu, D., & Erickson, B. J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.08205 · Cited by 59
Intelligent labeling based on fisher information for medical image segmentation using deep learning
Sourati, J., Gholipour, A., Dy, J. G., Tomas-Fernandez, X., Kurugol, S., & Warfield, S. K. (2019) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2019.2907805 · Cited by 50
Extreme Clicking for Efficient Object Annotation
Papadopoulos, D. P., Uijlings, J., Keller, F., & Ferrari, V. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.528 · Cited by 45
MultiSeg: Semantically Meaningful, Scale-Diverse Segmentations From Minimal User Input
Liew, J. H., Cohen, S. D., Price, B. L., Mai, L., Ong, S., & Feng, J. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00075 · Cited by 38
Wang, G., Aertsen, M., Deprest, J. A., Sebastien Ourselin, A., Vercauteren, T. K., & Zhang, S. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-59719-1_28 · Cited by 37
Self-citationGuide Me: Interacting with Deep Networks
Rupprecht, C., Laina, I., Navab, N., Hager, G. D., & Tombari, F. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00892 · Cited by 26
4D active cut: An interactive tool for pathological anatomy modeling
Wang, B., Liu, K., Prastawa, M. W., Irima, A., Vespa, P. M., Horn, J. D. V., Fletcher, P. T., & Gerig, G. (2014) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi.2014.6867925 · Cited by 23
Weakly Supervised Segmentation from Extreme Points
Roth, H. R., Zhang, L., Yang, D., Milletarì, F., Xu, Z., Wang, X., & Xu, D. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-33642-4_5 · Cited by 19
Zhou, L., Heller, N., Shi, Y., Xiao, Y., Sznitman, R., Cheplygina, V., Mateus, D., Trucco, E., Hu, X. S., Chen, D., et al. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-33642-4 · Cited by 19
Khan, S., Shahin, A. H., Villafruela, J., Shen, J., & Shao, L. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-32245-8_8 · Cited by 18
Interactive Deep Editing Framework for Medical Image Segmentation
Zhou, B., Chen, L., & Zhao, W. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-32248-9_37 · Cited by 16
Marsh, H. (2013) · doi:10.17077/etd.hbxcmbbg · Cited by 15
Dynamically Balanced Online Random Forests for Interactive Scribble-Based Segmentation
Wang, G., Zuluaga, M. A., Pratt, R., Aertsen, M., Doel, T., Klusmann, M., David, A. L., Deprest, J., Vercauteren, T., & Ourselin, S. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46723-8_41 · Cited by 14
Self-citationDeep GrabCut for Object SelectionPreprint
Xu, N., Price, B. L., Cohen, S. D., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.00243 · Cited by 14
Raju, A., Ji, Z., Cheng, C., Cai, J., Huang, J., Xiao, J. Z., Lü, L., Liao, C., & Harrison, A. P. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-59710-8_45 · Cited by 13
IEEE International Symposium on Biomedical Imaging
(2015) · IEEE Pulse · doi:10.1109/mpul.2015.2417197 · Cited by 13
Data Augmentation Based on Substituting Regional MRIs Volume Scores
Leng, T., Zhao, Q., Yang, C., Lu, Z., Adeli, E., & Pohl, K. M. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-33642-4_4 · Cited by 4
Going to Extremes: Weakly Supervised Medical Image SegmentationPreprint
Roth, H. R., Yang, D., Xu, Z., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.11988 · Cited by 4
MultiSeg: Semantically Meaningful, Scale-Diverse Segmentations From Minimal User Input
Hao, L. J., Scott, C., Brian, P., et al. (2019) · IEEE Conference Proceedings · Cited by 1
Research works printed but not linked: 1
- Vezhnevets, 2005, Growcut: interactive multi-label ND image segmentation by cellular automata, Graphicon, 1, 150
Cited by
Sorted by citations
Segment anything in medical images
Ma, J., He, Y., Li, F., Han, L., You, C., & Wang, B. (2024) · Nature Communications · doi:10.1038/s41467-024-44824-z · Cited by 2,913
Semi-supervised medical image segmentation via uncertainty rectified pyramid consistency
Luo, X., Wang, G., Liao, W., Chen, J., Song, T., Chen, Y., Zhang, S., Metaxas, D., & Zhang, S. (2022) · Medical Image Analysis · doi:10.1016/j.media.2022.102517 · Cited by 419
Self-citationLABKIT: Labeling and Segmentation Toolkit for Big Image Data
Arzt, M., Deschamps, J., Schmied, C., Pietzsch, T., Schmidt, D., Tomancak, P., Haase, R., & Jug, F. (2022) · Frontiers in Computer Science · doi:10.3389/fcomp.2022.777728 · Cited by 405
Luo, X., Liao, W., Xiao, J., Chen, J., Song, T., Zhang, X., Li, K., Metaxas, D., Wang, G., & Zhang, S. (2022) · Medical Image Analysis · doi:10.1016/j.media.2022.102642 · Cited by 237
Self-citationSAM-Med3D: Towards General-Purpose Segmentation Models for Volumetric Medical Images
Wang, H., Guo, S., Jin, Y., Deng, Z., Cheng, J., Li, T., Chen, J., Su, Y., Huang, Z., Shen, Y., et al. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-91721-9_4 · Cited by 238
Self-citation
Metadata
- Authors
- Luo, X., Wang, G., Song, T., Zhang, J., Aertsen, M., Deprest, J., Ourselin, S., Vercauteren, T., & Zhang, S.
- Affiliations
- University of Electronic Science and Technology of China (CN)
- Sensetime (China) (CN)
- Shanghai Jiao Tong University (CN)
- Universitair Ziekenhuis Leuven (BE)
- King's College London (GB)
- Corresponding author
- Wang, Guotai
- Venue
- Medical Image Analysis
- Publisher
- Elsevier BV
- Published
- 2021-08
- Volume
- 72
- Pages
- 102102
- ISSN
- 1361-8415 (ISSN-L)
- PMC
- PMC7613452
- OpenAlex
- W3159178846
- Cited by
- 126
- References
- 61
- Citations by year
- 2026: 10 · 2025: 36 · 2024: 29 · 2023: 31 · 2022: 18 · 2021: 2
- FWCI
- 7.19
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Licence
- Locations
- Medical Image Analysis (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- arXiv (Cornell University) (submittedVersion)
- Lirias (submittedVersion, other-oa)
- arXiv (Cornell University) (cc-by)
- Funders
- EPSRC
- Wellcome Trust
- National Natural Science Foundation of China
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Engineering › Biomedical Engineering › Medical Imaging and Analysis
- Keywords
- interactive segmentation, medical image segmentation, unseen objects, generalization, data fusion, organ segmentation, lesion segmentation, 3D medical images
- MeSH terms
- Deep Learning, Humans, Image Processing, Computer-Assisted, Neural Networks, Computer, Imaging, Three-Dimensional
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
