CoChat
Sign Up
Cited by 109Medical Image Segmentation TechniquesAdvanced Neural Network ApplicationsMedical Imaging and Analysis

Volumetric memory network for interactive medical image segmentation

Open access
Zhou, Tianfei, Li, Liulei, Bredell, Gustav, Li, Jianwu, Unkelbach, Jan, Konukoglu, Ender·2023·Medical Image Analysis
DOI 10.1016/j.media.2022.102599
Open this paper in CoChat — free accountOpen PDF

Abstract

Despite recent progress of automatic medical image segmentation techniques, fully automatic results usually fail to meet clinically acceptable accuracy, thus typically require further refinement. To this end, we propose a novel Volumetric Memory Network, dubbed as VMN, to enable segmentation of 3D medical images in an interactive manner. Provided by user hints on an arbitrary slice, a 2D interaction network is firstly employed to produce an initial 2D segmentation for the chosen slice. Then, the VMN propagates the initial segmentation mask bidirectionally to all slices of the entire volume. Subsequent refinement based on additional user guidance on other slices can be incorporated in the same manner. To facilitate smooth human-in-the-loop segmentation, a quality assessment module is introduced to suggest the next slice for interaction based on the segmentation quality of each slice produced in the previous round. Our VMN demonstrates two distinctive features: First, the memory-augmented network design offers our model the ability to quickly encode past segmentation information, which will be retrieved later for the segmentation of other slices; Second, the quality assessment module enables the model to directly estimate the quality of each segmentation prediction, which allows for an active learning paradigm where users preferentially label the lowest-quality slice for multi-round refinement. The proposed network leads to a robust interactive segmentation engine, which can generalize well to various types of user annotations (e.g., scribble, bounding box, extreme clicking). Extensive experiments have been conducted on three public medical image segmentation datasets (i.e., MSD, KiTS19, CVC-ClinicDB), and the results clearly confirm the superiority of our approach in comparison with state-of-the-art segmentation models. The code is made publicly available at https://github.com/0liliulei/Mem3D.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.media.2022.102599

Cites

59 of 59 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Existing deep learning-based interactive segmentation methods focus mainly on limited tasks and image modalities.

    “while existing deep learning-based interactive segmentation methods focus mainly on limited tasks and image modalities.”

    From Segment anything in medical images (2024) · cites this paper as [10]

    The Introduction (page unavailable) says many deep interactive segmentation approaches focus only on 2D medical images, supporting a limitation in the kinds of images they handle. It does not say that existing methods focus mainly on a limited range of segmentation tasks or imaging modalities; its other stated limitation concerns the effort needed for user corrections. (LLM, conf=0.78)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 17

  1. Baumgartner, C.F., Tezcan, K.C., Chaitanya, K., Hötker, A.M., Muehlematter, U.J., Schawkat, K., Becker, A.S., Donati, O., Konukoglu, E., 2019. Phiseg: Capturing uncertainty in medical image segmentation. In: Medical Image Computing and Computer Assisted Intervention. Springer, pp. 119-127.
  2. Boykov, Y., Funka-Lea, G., 2006. Graph cuts and efficient ND image segmentation. Int. J. Comput. Vis. 70 (2), 109-131.
  3. Fan, D.-P., Ji, G.-P., Zhou, T., Chen, G., Fu, H., Shen, J., Shao, L., 2020. Pranet: Parallel reverse attention network for polyp segmentation. In: Medical Image Computing and Computer Assisted Intervention. pp. 263-273.
  4. Fang, Y., Chen, C., Yuan, Y., Tong, K.-y., 2019. Selective feature aggregation net- work with area-boundary constraints for polyp segmentation. In: Medical Image Computing and Computer Assisted Intervention. pp. 302-310.
  5. Grady, L., Schiwietz, T., Aharon, S., Westermann, R., 2005. Random walks for interactive organ segmentation in two and three dimensions: Implementation and validation. In: Medical Image Computing and Computer Assisted Intervention. pp. 773-780.
  6. Isensee, F., Maier-Hein, K.H., 2019. An attempt at beating the 3D U-Net. ArXiv preprint arXiv:1908.02182. · arXiv
  7. Isensee, F., Petersen, J., Klein, A., Zimmerer, D., Jaeger, P.F., Kohl, S., Wasserthal, J., Koehler, G., Norajitra, T., Wirkert, S., et al., 2018. Nnu-net: Self-adapting frame- work for u-net-based medical image segmentation. ArXiv preprint arXiv:1809. 10486.
  8. Kim, S., Kim, I., Lim, S., Baek, W., Kim, C., Cho, H., Yoon, B., Kim, T., 2019. Scalable neural architecture search for 3d medical image segmentation. In: Medical Image Computing and Computer Assisted Intervention. Springer.
  9. Milletari, F., Navab, N., Ahmadi, S.-A., 2016. V-net: Fully convolutional neural networks for volumetric medical image segmentation. In: 3DV. pp. 565-571.
  10. Ronneberger, O., Fischer, P., Brox, T., 2015. U-net: Convolutional networks for biomed- ical image segmentation. In: Medical Image Computing and Computer Assisted Intervention. pp. 234-241.
  11. Sukhbaatar, S., Weston, J., Fergus, R., et al., 2015. End-to-end memory networks. Proc. Advances Neural Inf. Process. Syst. 28.
  12. Wang, G., Aertsen, M., Deprest, J., Ourselin, S., Vercauteren, T., Zhang, S., 2020. Uncertainty-guided efficient interactive refinement of fetal brain segmentation from stacks of MRI slices. In: Medical Image Computing and Computer Assisted Intervention. Springer, pp. 279-288.
  13. Zhang, Y., Liu, H., Hu, Q., 2021a. Transfuse: Fusing transformers and cnns for medical image segmentation. In: Medical Image Computing and Computer Assisted Intervention. pp. 14-24.
  14. Zhao, W., Jiang, D., Queralta, J.P., Westerlund, T., 2020. Multi-scale supervised 3D U-net for kidneys and kidney tumor segmentation. ArXiv preprint arXiv:2004. 08108.
  15. Zhou, T., Li, L., Bredell, G., Li, J., Konukoglu, E., 2021. Quality-aware memory network for interactive volumetric image segmentation. In: Medical Image Computing and Computer Assisted Intervention. Springer, pp. 560-570.
  16. Zhou, Y., Xie, L., Shen, W., Wang, Y., Fishman, E.K., Yuille, A.L., 2017. A fixed- point model for pancreas segmentation in abdominal CT scans. In: Medical Image Computing and Computer Assisted Intervention. pp. 693-701.
  17. Zhu, Z., Liu, C., Yang, D., Yuille, A., Xu, D., 2019. V-nas: Neural architecture search for volumetric medical image segmentation. In: 3DV. pp. 240-248.

Cited by

Sorted by citations

Related

Authors
Zhou, T., Li, L., Bredell, G., Li, J., Unkelbach, J., & Konukoglu, E.
Affiliations
  • ETH Zurich (CH)
  • Beijing Institute of Technology (CN)
  • University Hospital Zurich (CH)
Corresponding author
Zhou, Tianfei
ORCID
Venue
Medical Image Analysis
Publisher
Elsevier BV
Published
2023-01
Volume
83
Pages
102599
ISSN
1361-8415 (ISSN-L)
DOI
10.1016/j.media.2022.102599
OpenAlex
W4294842893
Cited by
109
References
59
Citations by year
2026: 11 · 2025: 28 · 2024: 33 · 2023: 30 · 2022: 7
FWCI
8.01
Type
journal article
Language
en
Open access
Licence
Locations
  • Medical Image Analysis (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Zurich Open Repository and Archive (University of Zurich) (acceptedVersion, cc-by-nc-nd)
  • Repository for Publications and Research Data (ETH Zurich) (submittedVersion, cc-by)
  • Zurich Open Repository and Archive (University of Zurich) (submittedVersion, cc-by)
  • ETH Zürich Research Collection
  • Zurich Open Repository and Archive (University of Zurich)
Funders
Varian Medical Systems Inc
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Engineering › Biomedical Engineering › Medical Imaging and Analysis
Keywords
memory-augmented networks, human-in-the-loop segmentation, active learning (machine learning), quality assessment, 3D medical image segmentation, scribble annotations, bounding box annotation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review