Cited by 158Medical Image Segmentation TechniquesDomain Adaptation and Few-Shot LearningAdvanced Neural Network Applications

3DSAM-adapter: Holistic adaptation of SAM from 2D to 3D for promptable tumor segmentation

Open access
Gong, Shizhan, Zhong, Yuan, Ma, Wenao, Li, Jinpeng, Wang, Zhao, Zhang, Jingyang, Heng, Pheng-Ann, Dou, Qi·2024·Medical Image Analysis

Abstract

Despite that the segment anything model (SAM) achieved impressive results on general-purpose semantic segmentation with strong generalization ability on daily images, its demonstrated performance on medical image segmentation is less precise and unstable, especially when dealing with tumor segmentation tasks that involve objects of small sizes, irregular shapes, and low contrast. Notably, the original SAM architecture is designed for 2D natural images and, therefore would not be able to extract the 3D spatial information from volumetric medical data effectively. In this paper, we propose a novel adaptation method for transferring SAM from 2D to 3D for promptable medical image segmentation. Through a holistically designed scheme for architecture modification, we transfer the SAM to support volumetric inputs while retaining the majority of its pre-trained parameters for reuse. The fine-tuning process is conducted in a parameter-efficient manner, wherein most of the pre-trained parameters remain frozen, and only a few lightweight spatial adapters are introduced and tuned. Regardless of the domain gap between natural and medical data and the disparity in the spatial arrangement between 2D and 3D, the transformer trained on natural images can effectively capture the spatial patterns present in volumetric medical images with only lightweight adaptations. We conduct experiments on four open-source tumor segmentation datasets, and with a single click prompt, our model can outperform domain state-of-the-art medical image segmentation models and interactive segmentation models. We also compared our adaptation method with existing popular adapters and observed significant performance improvement on most datasets. Our code and models are available at: https://github.com/med-air/3DSAM-adapter.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.media.2024.103324

Cites

67 of 67 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Most promptable medical segmentation methods are still developed and evaluated around clean-prompt protocols.

    “Most promptable medical segmentation methods, however, are still developed and evaluated around clean-prompt protocols.”

    From Correction-aware interactive 3D tumor segmentation with sparse and revisable prompts (2026) · cites this paper as [5]

    Section 3.3 (page unavailable) says accurate prompt guidance is difficult in medical images and that a model can fail when prompts are inaccurate, supporting the concern about reliance on clean prompts. It does not establish that most promptable medical segmentation methods are developed and evaluated under clean-prompt protocols. (LLM, conf=0.75)

    Experimental — accuracy not yet measured
  • 3DSAM-Adapter and SAM-Med3D adapt SAM-style prompting to 3D medical images, while ProMISe is a prompt-driven 3D medical segmentation method based on pretrained image foundation models.

    “3DSAM-Adapter [5] and SAM-Med3D [29] adapt SAM-style prompting to 3D medical images, while ProMISe [17] is a prompt-driven 3D medical segmenta-tion method based on pretrained image foundation models.”

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

5 of the 72 references couldn’t be matched to a known paper.

Research works printed but not linked: 8

  1. Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C. Berg, Wan-Yen Lo, Piotr Dollár, and Ross Girshick. Segment anything. arXiv:2304.02643, 2023. · arXiv
  2. Sheng He, Rina Bao, Jingpeng Li, P Ellen Grant, and Yangming Ou. Accuracy of segment-anything model (sam) in medical image segmentation tasks. arXiv preprint arXiv:2304.09324, 2023. · arXiv
  3. Yichi Zhang and Rushi Jiao. How segment anything model (sam) boost medical image segmentation? arXiv preprint arXiv:2305.03678, 2023. · arXiv
  4. Junting Pan, Ziyi Lin, Xiatian Zhu, Jing Shao, and Hongsheng Li. St-adapter: Parameter-efficient image-to-video transfer learning for action recognition. arXiv preprint arXiv:2206.13559, 2022. · arXiv
  5. Wenxuan Wang, Jiachen Shen, Chen Chen, Jianbo Jiao, Yan Zhang, Shanshan Song, and Jiangyun Li. Med-tuning: Exploring parameter-efficient transfer learning for medical volumetric segmentation. arXiv preprint arXiv:2304.10880, 2023. · arXiv
  6. Edward J Hu, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. In International Conference on Learning Representations, 2023.
  7. Yucheng Tang, Dong Yang, Wenqi Li, Holger R Roth, Bennett Landman, Daguang Xu, Vishwesh Nath, et al. Self-supervised pre-training of swin transformers for 3d medical image analysis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20730-20740, 2022.
  8. Yuanfeng Ji, Ruimao Zhang, Zhen Li, Jiamin Ren, Shaoting Zhang, and Ping Luo. Uxnet: Searching multi-level feature aggregation for 3d medical image segmentation. In Medical Image Computing and Computer Assisted Intervention-MICCAI 2020: 23rd International Conference, Lima, Peru, October 4-8, 2020, Proceedings, Part I 23, pages 346-356. Springer, 2020.

Cited by

Sorted by citations

Authors
Gong, S., Zhong, Y., Ma, W., Li, J., Wang, Z., Zhang, J., Heng, P.-A., & Dou, Q.
Affiliations
Chinese University of Hong Kong (HK)
Corresponding author
Dou, Qi
Venue
Medical Image Analysis
Publisher
Elsevier BV
Published
2024-12
Volume
98
Pages
103324
ISSN
1361-8415 (ISSN-L)
OpenAlex
W4401852105
Cited by
156
References
72
Citations by year
2026: 51 · 2025: 61 · 2024: 41 · 2023: 3
FWCI
21.42
Type
journal article
Language
en
Open access
Locations
  • Medical Image Analysis (publishedVersion)
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Funders
National Natural Science Foundation of China
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
Keywords
Segment Anything Model, 3D medical image segmentation, tumor segmentation, prompt-based segmentation, parameter-efficient fine-tuning, volumetric medical data, interactive segmentation, domain adaptation
MeSH terms
Algorithms, Humans, Neoplasms, Imaging, Three-Dimensional

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review