Segment anything model for medical image analysis: An experimental study
Open accessAbstract
Training segmentation models for medical images continues to be challenging due to the limited availability of data annotations. Segment Anything Model (SAM) is a foundation model trained on over 1 billion annotations, predominantly for natural images, that is intended to segment user-defined objects of interest in an interactive manner. While the model performance on natural images is impressive, medical image domains pose their own set of challenges. Here, we perform an extensive evaluation of SAM’s ability to segment medical images on a collection of 19 medical imaging datasets from various modalities and anatomies. In our experiments, we generated point and box prompts for SAM using a standard method that simulates interactive segmentation. We report the following findings: (1) SAM’s performance based on single prompts highly varies depending on the dataset and the task, from IoU=0.1135 for spine MRI to IoU=0.8650 for hip X-ray. (2) Segmentation performance appears to be better for well-circumscribed objects with prompts with less ambiguity such as the segmentation of organs in computed tomography and poorer in various other scenarios such as the segmentation of brain tumors. (3) SAM performs notably better with box prompts than with point prompts. (4) SAM outperforms similar methods RITM, SimpleClick, and FocalClick in almost all single-point prompt settings. (5) When multiple-point prompts are provided iteratively, SAM’s performance generally improves only slightly while other methods’ performance improves to the level that surpasses SAM’s point-based performance. We also provide several illustrations for SAM’s performance on all tested datasets, iterative segmentation, and SAM’s behavior given prompt ambiguity. We conclude that SAM shows impressive zero-shot segmentation performance for certain medical imaging datasets, but moderate to poor performance for others. SAM has the potential to make a significant impact in automated medical image segmentation in medical imaging, but appropriate care needs to be applied when using it. Code for evaluation SAM is made publicly available at https://github.com/mazurowski-lab/segment-anything-medical-evaluation.
AI snapshot
Written from the paper’s full text.
TL;DR
This experimental study evaluates the Segment Anything Model (SAM) for zero-shot medical image segmentation across 19 public datasets and 28 tasks. SAM shows highly variable performance, performs best with box prompts, and beats RITM/SimpleClick/FocalClick in most single-point settings, but gains little from iterative point prompts and remains moderate-to-poor on many medical tasks.
Key findings
- SAM’s single-prompt performance varied widely by dataset/task, from IoU=0.1135 for spine MRI to IoU=0.8650 for hip X-ray in the abstract; the Results section reports a range from IoU=0.1136 to IoU=0.9118.
- Box prompts were clearly superior to point prompts. Mode 4 (one box per object part) had the best average performance at IoU=0.6542; Mode 3 (single box around the entire object) averaged IoU=0.5891.
- SAM outperformed RITM, SimpleClick, and FocalClick on 24 of 28 tasks with a single non-iterative point prompt, and on 26 of 28 in oracle mode. Average IoU was 0.4595 for SAM, 0.5137 for SAM oracle, 0.2240 for FocalClick, 0.1910 for SimpleClick, and 0.1322 for RITM.
- With iterative point prompts, SAM improved only slightly and was surpassed by SimpleClick and RITM when five or more points were provided. The correlation between average object size and SAM performance was small and non-statistically significant.
Methods
- The authors curated 19 publicly available radiological medical imaging datasets covering X-ray, MRI, CT, ultrasound, and PET, yielding 28 segmentation tasks; for 3D modalities, 2D slices were used as inputs.
- They evaluated five non-iterative prompting modes using points or boxes, iterative point prompting guided by an error map, “segment everything” mode, and prompt ambiguity/oracle outputs.
- Performance was measured with IoU against ground-truth masks, using SAM’s highest-confidence mask; SAM was compared with RITM, SimpleClick, and FocalClick.
Limitations
- The paper states that text-based prompting is not commented on because it remains proof-of-concept, and that “segment everything” mode is not elaborated on. It also states future work should include prompt engineering and extension to 3D segmentation.
- The text makes evident that the study focuses on radiological images and uses 2D slices for 3D modalities, which may not reflect full 3D medical segmentation performance.
- The paper states that oracle performance assumes knowledge of the true mask and is biased when there is no additional user interaction after initial prompts; iterative prompting also uses the oracle prediction to guide the next prompt.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
0 of 1 cited sources checked · 0 supported · 0 flagged
- Findingown data§ Abstract
SAM's performance based on single prompts varies highly depending on the dataset and task, from IoU=0.1135 for spine MRI to IoU=0.8650 for hip X-ray.
“(1) SAM's performance based on single prompts highly varies depending on the dataset and the task, from IoU=0.1135 for spine MRI to IoU=0.8650 for hip X-ray.”
Quote in the paperSubstantial claim - Findingnone given§ Abstract
SAM performs notably better with box prompts than with point prompts.
“(3) SAM performs notably better with box prompts than with point prompts.”
Quote in the paperSubstantial claim - Findingown data
When used in oracle mode, SAM was better than all other methods for 26 out of 28 tasks.
“When used in oracle mode, SAM was better than all other methods for 26 out of 28 tasks.”
Quote in the paperSubstantial claim - Findingown data
Prompts indicating each separate part of the object separately are generally superior to prompts indicating only one part or all parts in one box.
“Moreover, as expected, prompts where each sep- arate part of the object is indicated separately are generally superior to those where only one part is indicated, or all parts are outlined in one box.”
Quote in the paperSubstantial claim - Findingown data
SAM's performance varies from near-perfect for well-circumscribed objects with unambiguous prompts to very poor, particularly for objects with ambiguous prompts.
“This figure illustrates a high variability of SAM's performance ranging from near- perfect for well-circumscribed objects with unambiguous prompts to very poor, particularly for objects with ambigu- ous prompts.”
Quote in the paperSubstantial claim
- Findingown data
If SAM used Mode 3 (single box), it outperformed all other methods in all but one task.
“If SAM used Mode 3 (single box), it outperformed all other methods in all but one task.”
Quote in the paperNot a substantial claim - Findingnone given§ 4. Conclusions and discussion
The five modes of use for interactive segmentation methods showed different performances in scenarios with multiple components.
“These modes also showed different performances in such scenarios.”
Quote in the paperSubstantial claim - Findingreasoning
Mode 4, where each part of an object is indicated by a separate box, showed the best performance with an average IoU of 0.6542.
“Following these two trends, Mode 4, where each part of an object is indicated by a separate box, showed the best performance with an average IoU of 0.6542.”
Quote in the paperSubstantial claim - Findingnone given
The magnitude of the oracle mode improvement over default mode was highly dependent on the dataset.
“The magnitude of this improvement was highly dependent on the dataset.”
Quote in the paperNot a substantial claim - Findingnone given
Oracle mode showed a moderate improvement over the default mode.
“Additionally, the oracle mode showed a moderate im- provement over the default mode.”
Quote in the paperNot a substantial claim - Findingown data
SAM's IoU ranges from 0.9118 to 0.1136.
“It ranges from an impres- sive IoU of 0.9118 to a very poor IoU of 0.1136.”
Quote in the paperNot a substantial claim - Findingown data
The superiority of separate-part prompts was particularly pronounced for datasets where objects typically consist of more than one part.
“This was particularly pro- nounced for datasets where objects typically consist of more than one part.”
Quote in the paperSubstantial claim - Findingown data
SAM's performance varies widely across different datasets.
“First, SAMs performance varies widely across different datasets.”
Quote in the paperSubstantial claim - Findingown data
SAM has a consistent tendency in interpreting point prompts when applied to medical images.
“When applying SAM on medical images, we found a consistent tendency of SAM in interpreting the point prompts.”
Quote in the paperSubstantial claim - Interpretationreasoning§ 1. Introduction
In its basic form, SAM cannot be used the same way as most segmentation models in medical imaging where the input is simply an image and the output is a segmentation mask or multiple masks for the desired object(s).
“Given this prompt-based nature, in its basic form, SAM cannot be used the same way as most segmentation models in medical imaging where the input is simply an image and the output is a segmentation mask or multiple masks for the desired object or objects.”
Quote in the paperSubstantial claim
- May combine two claims. 5
- The quote only partly states this claim. 1
- Classified as probably not a substantial claim. 2
May combine two claims.
Findingown dataSegment-everything mode provides mixed usefulness and results are somewhat dependent on the number of prompts.
“We find that this mode provides mixed usefulness and that the results are somewhat dependent on the number of prompts.”
Quote in the paperNot a substantial claimMay combine two claims. The quote only partly states this claim.
Findingown dataSAM obtains very limited benefit from additional point prompts in the iterative setting, except for objects with multiple parts.
“This is due to the fact that SAM does not appear to draw hardly any benefits from additional information provided through the interactive points after two or three points have been provided.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataSome datasets showed high variability of object size within the dataset itself (e.g., kidneys in ultrasound images) while others showed relative consistency of object size (such as bones in hip X-ray images).
“Some datasets also showed high variability of object size within the dataset itself (e.g., kidneys in ultrasound images) while others showed relative consistency of object size (such as bones in hip X-ray images).”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationown data§ 4. Conclusions and discussionOverall, SAM shows promise for use in medical images, as long as suitable prompting strategies are used for the dataset and task of choice.
“Overall, SAM shows promise for use in medical images, as long as suitable prompting strategies are used for the dataset and task of choice.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductown dataThe correlation analysis uses average object sizes in the image and does not consider variation of object sizes in individual datasets.
“Note that the correlation analysis looks at average object sizes in the image and does not consider the variation of object sizes in the individual datasets.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Premisecitation§ 1. IntroductionSAM has achieved impressive/promising zero-shot segmentation performance on a variety of natural image datasets.
“The recently developed Segment Anything Model is a foundation model that has achieved promising zero-shot segmentation performance on a variety of natural image datasets [18].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [18] Jang, J., Hwang, D., 2022. M3T: Three-dimensional Medical image classifier using Multi-plane and Multi-slice Transformer. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 20718–20729. source
Not checked in the free run.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
SAM achieves satisfactory segmentation outcomes primarily on medical targets with distinct boundaries but has substantial limitations on targets with weak boundaries or low contrast.
“For example, the concurrent studies22,23 conducted a comprehensive assessment of SAM across a diverse array of medical images, underscoring that SAM achieved satisfactory segmentation outcomes primarily on targets characterized by distinct boundaries. However, the model exhibited substantial limitations in segmenting typical medical targets with weak boundaries or low contrast.”
From Segment anything in medical images (2024) · cites this paper as [22]
The Abstract and Results §3.1 report better segmentation of well-circumscribed objects and poorer results in other cases, supporting the distinct-boundaries comparison. The paper also identifies prompt ambiguity as an important factor; the passages do not establish low contrast or weak boundaries as the specific cause of its poorer results. (LLM, conf=0.84)
Experimental — accuracy not yet measuredMost previous studies evaluated SAM with limited or even only one type of testing mode/strategy.
“Most previous studies (Ji et al., 2023a; Mazurowski et al., 2023; Zhou et al., 2023) evaluated SAM with limited or even only one type of testing mode / strategy.”
From Segment anything model for medical images? (2024) · cites this paper as Mazurowski et al., 2023
The correlation between SAM's segmentation performance and medical objects' attributes has not been conducted carefully.
“Thus, the correlation between SAM's segmentation performance and medical objects' attributes has not been conducted carefully (He et al., 2023; Mazurowski et al., 2023; Ma and Wang, 2023).”
From Segment anything model for medical images? (2024) · cites this paper as Mazurowski et al., 2023
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Box-per-region strategy is assigned two mode numbers
MinorThe main text identifies a separate box for each object region as Mode 4, but the graphical abstract labels that strategy Mode 2.
“Mode 4, where each part of an object is indicated by a separate box”
“M od e 2: 1 b ox a t e ac h re gi on”
Dataset count differs between the paper and graphical abstract
MinorThe paper states that it evaluates 19 datasets, while the graphical abstract says 18.
“We compiled and curated a set of 19 publicly available medical imaging datasets for image segmentation.”
“18 m ed ic al im ag in g da ta se ts”
Cites
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 21,793 · cited 1× · see citing works
“Based on the framework proposed by RITM, SimpleClick replaced the HRNet-18 backbone in RITM with a plain-ViT [9] and included a “click embedding” layer symmetric to patch embedding.”
MethodThe multimodal brain tumor image segmentation benchmark (BRATS)
Menze, B. H., Jakab, A., Bauer, S., Kalpathy-Cramer, J., Farahani, K., Kirby, J., Burren, Y., Porz, N., Slotboom, J., Wiest, R., et al. (2015) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2014.2377694 · Cited by 6,781 · cited 1× · see citing works
“MRI-Brain The Multimodal Brain Tumor Image Segmentation Benchmark (BraTS) [11] MRI 3 Three parts of brain tumor 13,107”
BackgroundSegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers
Xie, E., Wang, W., Yu, Z., Anandkumar, A., Alvarez, J. M., & Luo, P. (2021) · arXiv · doi:10.48550/arxiv.2105.15203 · Cited by 5,090 · cited 2× · see citing works
“Fo-calClick replaced the HRNet-18 backbone in RITM with a SegFormer [37] and restricted the update responding to new clicks to occur only as local.”
MethodDeep High-Resolution Representation Learning for Visual Recognition
Wang, J., Sun, K., Cheng, T., Jiang, B., Deng, C., Zhao, Y., Liu, D., Mu, Y., Tan, M., Wang, X., et al. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.2983686 · Cited by 4,894 · cited 1× · see citing works
“RITM adopted HRNet-18 [36] as the backbone segmentation model and an iterative mask correction approach based on the { previous mask, new click } set to achieve outstanding performance on multiple natural imaging datasets.”
MethodSegment anything in medical images
Ma, J., He, Y., Li, F., Han, L., You, C., & Wang, B. (2024) · Nature Communications · doi:10.1038/s41467-024-44824-z · Cited by 2,913 · see citing works
Dataset of breast ultrasound images
Al-Dhabyani, W., Gomaa, M., Khaled, H., & Fahmy, A. (2020) · Data in Brief · doi:10.1016/j.dib.2019.104863 · Cited by 2,405 · cited 1× · see citing works
“US-Breast Dataset of Breast Ultrasound Images [2] Ultrasound 1 Breast 780”
Current Methods in Medical Image Segmentation
Pham, D. L., Xu, C., & Prince, J. L. (2000) · Annual Review of Biomedical Engineering · doi:10.1146/annurev.bioeng.2.1.315 · Cited by 2,352 · cited 1× · see citing works
“Image segmentation is a central task in medical image analysis, ranging from the segmentation of organs [17], abnormalities [3], bones [8], and others [13].”
BackgroundDeep Learning Techniques for Medical Image Segmentation: Achievements and Challenges
Hesamian, M. H., Jia, W., He, X., & Kennedy, P. J. (2019) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-019-00227-x · Cited by 1,644 · cited 1× · see citing works
“Deep learning has dramatically altered how task is performed and brought significant performance improvements in many domains [6, 15].”
BackgroundMedical image analysis using convolutional neural networks: a review
Anwar, S. M., Majid, M., Qayyum, A., Awais, M., Alnowami, M., & Khan, M. K. (2018) · Journal of Medical Systems · doi:10.1007/s10916-018-1088-1 · Cited by 1,431 · cited 1× · see citing works
“Image segmentation is a central task in medical image analysis, ranging from the segmentation of organs [17], abnormalities [3], bones [8], and others [13].”
BackgroundThe Liver Tumor Segmentation Benchmark (LiTS)
Bilic, P., Christ, P., Li, H. B., Vorontsov, E., Ben-Cohen, A., Kaissis, G., Szeskin, A., Jacobs, C., Mamani, G. E. H., Chartrand, G., et al. (2023) · Medical Image Analysis · doi:10.1016/j.media.2022.102680 · Cited by 1,412 · cited 1× · see citing works
“CT-Liver The Liver Tumor Segmentation Benchmark (LiTS) [4] CT 1 Liver 5,501”
BackgroundTwo public chest X-ray datasets for computer-aided screening of pulmonary diseases.
(2014) · PubMed Central · doi:10.3978/j.issn.2223-4292.2014.11.20 · Cited by 918 · cited 1× · see citing works
“Xray-Chest Montgomery County and Shenzhen Chest X-ray Datasets [7] X-ray 1 Chest 704”
BackgroundExploring Plain Vision Transformer Backbones for Object Detection
Li, Y., Mao, H., Girshick, R., & He, K. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20077-9_17 · Cited by 903 · see citing works
A comprehensive survey on pretrained foundation models: a history from BERT to ChatGPT
Zhou, C., Li, Q., Li, C., Yu, J., Liu, Y., Wang, G., Zhang, K., Ji, C., Yan, Q., He, L., et al. (2024) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-024-02443-6 · Cited by 704 · see citing works
Simpson, A. L., Antonelli, M., Bakas, S., Bilello, M., Farahani, K., van Ginneken, B., Kopp‐Schneider, A., Landman, B. A., Litjens, G. J. S., Menze, B. H., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.09063 · Cited by 703 · cited 5× · see citing works
“CT-Pancreas Medical Segmentation Decathlon [33] CT 1 parenchyma and mass 8,792”
BackgroundInfluentialA comprehensive survey on pretrained foundation models: a history from BERT to ChatGPT
Zhou, C., Li, Q., Li, C., Yu, J., Liu, Y., Wang, G., Zhang, K., Ji, C., Yan, Q., He, L., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.09419 · Cited by 574 · see citing works
Segment anything model for medical images?
Huang, Y., Yang, X., Liu, L., Zhou, H., Chang, A., Zhou, X., Chen, R., Yu, J., Chen, J., Chen, C., et al. (2024) · Medical Image Analysis · doi:10.1016/j.media.2023.103061 · Cited by 536 · see citing works
Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A. C., Lo, W.-Y., et al. (2023) · arXiv · doi:10.48550/arxiv.2304.02643 · Cited by 527 · see citing works
CNN-based Segmentation of Medical Imaging DataPreprint
Kayalibay, B., Jensen, G. W., & van der Smagt, P. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1701.03056 · Cited by 505 · see citing works
Deep High-Resolution Representation Learning for Visual RecognitionPreprint
Wang, J., Sun, K., Cheng, T., Jiang, B., Deng, C., Zhao, Y. S., Liu, D., Mu, Y., Tan, M., Wang, X., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.07919 · Cited by 350 · cited 1× · see citing works
“RITM adopted HRNet-18 [36] as the backbone segmentation model and an iterative mask correction approach based on the { previous mask, new click } set to achieve outstanding performance on multiple natural imaging datasets.”
MethodLemaître, G., Martí, R., Freixenet, J., Vilanova, J. C., WALKER, P. M., & Mériaudeau, F. (2015) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2015.02.009 · Cited by 349 · see citing works
Saha, A., Harowicz, M. R., Grimm, L. J., Kim, C. E., Ghate, S. V., Walsh, R., & Mazurowski, M. A. (2018) · British Journal of Cancer · doi:10.1038/s41416-018-0185-8 · Cited by 313 · see citing works
Self-citationReviving Iterative Training with Mask Guidance for Interactive Segmentation
Sofiiuk, K., Petrov, I. A., & Konushin, A. (2022) · Proceedings - International Conference on Image Processing · doi:10.1109/icip46576.2022.9897365 · Cited by 264 · cited 10× · see citing works
“We show the average performances of SAM, RITM, SimpleClick, and FocalClick in Figure 5.”
MethodInfluentialA whole-body FDG-PET/CT Dataset with manually annotated Tumor Lesions
Gatidis, S., Hepp, T., Früh, M., la Fougère, C. J., Nikolaou, K., Pfannenberg, C., Schölkopf, B., Küstner, T., Cyran, C. C., & Rubin, D. L. (2022) · Scientific Data · doi:10.1038/s41597-022-01718-3 · Cited by 263 · cited 1× · see citing works
“PET-Whole-Body A FDG-PET/CT dataset with annotated tumor lesions [10] PET/CT 1 Lesion 1,015”
MethodFocalClick: Towards Practical Interactive Image Segmentation
Chen, X., Zhao, Z., Zhang, Y., Duan, M., Qi, D., & Zhao, H. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00136 · Cited by 182 · cited 2× · see citing works
“We compared SAM with three interactive segmentation methods, namely RITM [34], SimpleClick [21], and FocalClick [7].”
MethodSelf-citationSpinal cord grey matter segmentation challenge
Prados, F., Ashburner, J., Blaiotta, C., Brosch, T., Carballido‐Gamio, J., Cardoso, M. J., Conrad, B. N., Datta, E., Dávid, G., Leener, B. D., et al. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2017.03.010 · Cited by 147 · cited 1× · see citing works
“MRI-Spine Spinal Cord Grey Matter Segmentation Challenge [14] MRI 2 Gray matter, spinal cord 551”
BackgroundSimpleClick: Interactive Image Segmentation with Simple Vision Transformers
Liu, Q., Xu, Z., Bertasius, G., & Niethammer, M. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.02037 · Cited by 143 · see citing works
M3T: three-dimensional Medical image classifier using Multi-plane and Multi-slice Transformer
Jang, J., & Hwang, D. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.02006 · Cited by 133 · see citing works
Iteratively Trained Interactive SegmentationPreprint
Mahadevan, S., Voigtlaender, P., & Leibe, B. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.04398 · Cited by 106 · cited 1× · see citing works
“We use a common, intuitive strategy for simulating realistic iterative point prompts, which reflects how those could be generated by a user in an interactive way [24].”
MethodDeng, R., Cui, C., Liu, Q., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2304.04155 · Cited by 95 · cited 1× · see citing works
“Since we made the first version of this manuscript available, approximately 2 weeks after the release of the SAM paper (or even some before our release), multiple preprints have appeared evaluating SAM in broadly understood medical imaging and radiological imaging [8, 15].”
BackgroundKnowledge-aware Zero-Shot Learning: Survey and Perspective
Chen, J.-Y., Geng, Y.-X., Chen, Z., Horrocks, I., Pan, J. Z., & Chen, H.-J. (2021) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/597 · Cited by 94 · cited 1× · see citing works
“These difficulties could be significantly mitigated with the advent of foundation models [39] and zero-shot learning [6].”
BackgroundMarzola, F., van Alfen, N., Doorduin, J., et al. (2021) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2021.104623 · Cited by 83 · cited 1× · see citing works
“US-Muscle Transverse Musculoskeletal Ultrasound Image Segmentations [25] Ultrasound 1 Muscle 4,044”
Medical Transformer: Universal Encoder for 3-D Brain MRI Analysis
Jun, E., Jeong, S., Heo, D.-W., & Suk, H. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3308712 · Cited by 73 · see citing works
Segment Anything in Medical Images
Ma, J., He, Y., Li, F., Han, L., You, C., & Wang, B. (2023) · arXiv · doi:10.48550/arxiv.2304.12306 · Cited by 67 · see citing works
Song, Y., Zheng, J., Lei, L., et al. (2022) · Ultrasonics · doi:10.1016/j.ultras.2022.106706 · Cited by 66 · cited 1× · see citing works
“US-Kidney CT2US for Kidney Segmentation [20] Ultrasound 1 Kidney 4,586”
BackgroundSAM on Medical Images: A Comprehensive Study on Three Prompt ModesPreprint
Cheng, D., Qin, Z., Jiang, Z., Zhang, S., Lao, Q., & Li, K. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.00035 · Cited by 66 · see citing works
Computer-Vision Benchmark Segment-Anything Model (SAM) in Medical Images: Accuracy in 12 Datasets
He, S., Bao, R., Li, J., Stout, J., Bjornerud, A., Grant, P. E., & Ou, Y. (2023) · arXiv · doi:10.48550/arxiv.2304.09324 · Cited by 59 · see citing works
Exploring Plain Vision Transformer Backbones for Object DetectionPreprint
Li, Y., Mao, H., Girshick, R., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2203.16527 · Cited by 44 · see citing works
Segment Anything Model for Medical Images?
Huang, Y., Yang, X. S., Liu, L., et al. (2023) · PubMed · doi:10.48550/arxiv.2304.14660 · Cited by 40 · see citing works
Medical Transformer: Universal Brain Encoder for 3D MRI AnalysisPreprint
Jun, E., Jeong, S., Heo, D.-W., & Suk, H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.13633 · Cited by 30 · see citing works
SAMM (Segment Any Medical Model): A 3D Slicer Integration to SAMPreprint
Liu, Y., Zhang, J., She, Z., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2304.05622 · Cited by 23 · cited 1× · see citing works
“Some preprints already showed extensions of SAM to medical imaging [12,23] and one paper showed integration of SAM into 3D Slicer [22].”
Background2D Multi-Modality Ovarian Tumor Ultrasound (MMOTU)Preprint
Zhao, Q., Lyu, S., Bai, W., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2207.06799 · Cited by 12 · see citing works
SimpleClick: Interactive Image Segmentation with Simple Vision TransformersPreprint
Liu, Q., Xu, Z., Bertasius, G., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2210.11006 · Cited by 9 · see citing works
Fully Automated Deep Learning Method for Fibroglandular Tissue Segmentation in Breast MRIPreprint
Hu, S., Park, C., Lew, C. O., et al. (2022) · Research Square · doi:10.21203/rs.3.rs-1606703/v1 · Cited by 5 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 1
- Bradski, 2000, The OpenCV Library, Dr. Dobb’s J. Softw. Tools
Cited by
Sorted by citations
Segment anything in medical images
Ma, J., He, Y., Li, F., Han, L., You, C., & Wang, B. (2024) · Nature Communications · doi:10.1038/s41467-024-44824-z · Cited by 2,913 · see citing works
“These results highlight the potential of MedSAM as a powerful tool for medical image segmentation.”
Segment anything model for medical images?
Huang, Y., Yang, X., Liu, L., Zhou, H., Chang, A., Zhou, X., Chen, R., Yu, J., Chen, J., Chen, C., et al. (2024) · Medical Image Analysis · doi:10.1016/j.media.2023.103061 · Cited by 536 · see citing works
Transfer learning in environmental remote sensing
Ma, Y., Chen, S., Ermon, S., & Lobell, D. B. (2023) · Remote Sensing of Environment · doi:10.1016/j.rse.2023.113924 · Cited by 448 · see citing works
Segment anything model for medical image segmentation: Current applications and future directions
Zhang, Y., Shen, Z., & Jiao, R. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.108238 · Cited by 348 · see citing works
Self-citationOn the challenges and perspectives of foundation models for medical image analysis
Zhang, S., & Metaxas, D. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.102996 · Cited by 288 · see citing works
How Segment Anything Model (Sam) Boost Medical Image Segmentation: A SurveyPreprint
Zhang, Y., & Jiao, R. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4495221 · Cited by 263 · see citing works
Self-citationLangSplat: 3D Language Gaussian Splatting
Qin, M., Li, W., Zhou, J., Wang, H., & Pfister, H. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01895 · Cited by 283 · see citing works
“[38] introduced [48], super-resolution [25], image matting [45], object tracking [11, 42], medical image segmentation [29], image editing [12], and so on.”
BackgroundSegment Anything for Microscopy
Archit, A., Freckmann, L., Nair, S., Khalid, N., Hilt, P., Rajashekar, V., Freitag, M., Teuber, C., Spitzner, M., Contreras, C. T., et al. (2025) · Nature Methods · doi:10.1038/s41592-024-02580-4 · Cited by 354 · see citing works
“Prior work has already investigated SAM for biomedical applications, for example, in medical imaging 17 , histopathology 18 and neuroimaging 19 .”
Segment Anything Is Not Always Perfect: An Investigation of SAM on Different Real-world Applications
Ji, W., Li, J., Bi, Q., Liu, T., Li, W., & Cheng, L. (2024) · Machine Intelligence Research · doi:10.1007/s11633-023-1385-0 · Cited by 231 · see citing works
Review of large vision models and visual prompt engineering
Wang, J., Liu, Z., Zhao, L., Wu, Z., Ma, C., Yu, S., Dai, H., Yang, Q., Liu, Y., Zhang, S., et al. (2023) · Meta-Radiology · doi:10.1016/j.metrad.2023.100047 · Cited by 231 · see citing works
3DSAM-adapter: Holistic adaptation of SAM from 2D to 3D for promptable tumor segmentation
Gong, S., Zhong, Y., Ma, W., Li, J., Wang, Z., Zhang, J., Heng, P.-A., & Dou, Q. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103324 · Cited by 156 · see citing works
“This is also the underlying reason for the reported suboptimal performance when applying SAM to domain-specific tasks such as medical image segmentation [5, 6, 7, 8, 9, 10, 11].”
BackgroundSAM-6D: Segment Anything Model Meets Zero-Shot 6D Object Pose Estimation
Lin, J., Liu, L., Lu, D., & Jia, K. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02636 · Cited by 135 · see citing works
“SAM has demonstrated remarkable zero-shot transfer segmentation performance in real-world scenarios, including challenging situations such as medical images [36, 37, 71], camouflaged objects [22, 55], and transparent objects [13, 23].”
BackgroundSAM-Med3D: Towards General-purpose Segmentation Models for Volumetric Medical Images
Wang, H., Guo, S., Ye, J., Deng, Z., Cheng, J., Li, T., Chen, J., Su, Y., Huang, Z., Shen, Y., et al. (2023) · arXiv · doi:10.48550/arxiv.2310.15161 · Cited by 115
Deep-Learning-Based Approaches for Semantic Segmentation of Natural Scene Images: A Review
Soylu, B. E., Güzel, M. S., Bostancı, E., Ekinci, F., Aşuroğlu, T., & Açıcı, K. (2023) · Electronics · doi:10.3390/electronics12122730 · Cited by 110 · see citing works
“[169] have conducted a comprehensive assessment of the segmentation performance of SAM on a diverse set of 19 medical imaging datasets that include different modalities and anatomical regions.”
BackgroundFoundation Model for Advancing Healthcare: Challenges, Opportunities and Future Directions
He, Y., Huang, F., Jiang, X., Nie, Y., Wang, M., Wang, J., & Chen, H. (2024) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2024.3496744 · Cited by 130 · see citing works
“The existing HFMs still struggle with data heterogeneity (III-A3) in the real world [243] which performs poorly on some very specific domains [44].”
SurgicalSAM: Efficient Class Promptable Surgical Instrument Segmentation
Yue, W., Zhang, J., Hu, K. Y., Xia, Y., Luo, J., & Wang, Z. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i7.28514 · Cited by 117 · see citing works
Li, X., Zhang, L., Yang, J., & Teng, F. (2024) · Journal of Medical and Biological Engineering · doi:10.1007/s40846-024-00863-x · Cited by 99 · see citing works
“Furthermore, tailored image-naive architecture may outperform DL architecture on a task-specific basis [ 59].”
Self-citationSAM-Med3D: Towards General-Purpose Segmentation Models for Volumetric Medical Images
Wang, H., Guo, S., Jin, Y., Deng, Z., Cheng, J., Li, T., Chen, J., Su, Y., Huang, Z., Shen, Y., et al. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-91721-9_4 · Cited by 173 · see citing works
“However, directly applying SAM to 3D volumetric medical images is in-feasible, as disclosed in previous evaluations [2,26].”
MethodSegment Anything Model (SAM) for Medical Image Segmentation: A Preliminary Review
Zhang, L., Deng, X., & Yu, L. (2023) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm58861.2023.10386032 · Cited by 98 · see citing works
“Moreover, Mazurowski et al. [90] conduct an extensive evaluation of SAM’s medical image segmentation capabilities across diverse datasets, showcasing its proficiency with certain types of prompts.”
BackgroundFrom SAM to CAMs: Exploring Segment Anything Model for Weakly Supervised Semantic Segmentation
Kweon, H., & Yoon, K. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01844 · Cited by 82 · see citing works
“Over the past few months, numerous works [66, 67] have reported SAM’s ability to produce precise masks from ordinary images and to perform reasonably well, even in specialized domains like medical imagery [40, 41].”
BackgroundAli, M., Wu, T., Hu, H. R., Luo, Q., Xu, D. H., Zheng, W., Jin, N., Yang, C., & Yao, J. (2024) · Computerized Medical Imaging and Graphics · doi:10.1016/j.compmedimag.2024.102473 · Cited by 90 · see citing works
A Unified Framework With Multimodal Fine-Tuning for Remote Sensing Semantic Segmentation
Ma, X., Zhang, X., Pun, M.-O., & Huang, B. (2025) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2025.3585238 · Cited by 78 · see citing works
“Nowadays, SAM has already been applied across various fields, such as autonomous driving [38], medical image processing [39], and remote sensing [40], [41], [42], [43].”
MedLSAM: Localize and segment anything model for 3D CT images
Lei, W., Xu, W., Li, K., Zhang, X., & Zhang, S. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103370 · Cited by 90 · see citing works
Unleashing the Potential of SAM for Medical Adaptation via Hierarchical Decoding
Cheng, Z., Wei, Q., Zhu, H., Wang, Y., Qu, L., Shao, W., & Zhou, Y. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00337 · Cited by 76 · see citing works
“…in this field lies in the demand for deep learning models to undergo extensive training on large annotated datasets, a resource often challenging to procure within the medical domain. images diminishes notably in zero-shot settings, exhibiting reduced accuracy and robustness [26, 30, 36, 44, 51].”
BackgroundAdvances in attention mechanisms for medical image segmentation
Zhang, J., Chen, X., Yang, B., Guan, Q., Chen, Q., Chen, J., Wu, Q., Xie, Y., & Xia, Y. (2025) · Computer Science Review · doi:10.1016/j.cosrev.2024.100721 · Cited by 143 · see citing works
Segment Anything in Medical Images
Ma, J., He, Y., Li, F., Han, L., You, C., & Wang, B. (2023) · arXiv · doi:10.48550/arxiv.2304.12306 · Cited by 67 · see citing works
“These results highlight the potential of MedSAM as a powerful tool for medical image segmentation.”
Pu, X., Jia, H., Zheng, L., Wang, F., & Xu, F. (2025) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2025.3532690 · Cited by 84 · see citing works
Segment Anything for MicroscopyPreprint
Archit, A., Nair, S., Khalid, N., et al. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.08.21.554208 · Cited by 65 · see citing works
Review of applications of deep learning in veterinary diagnostics and animal health
Xiao, S., Dhand, N. K., Wang, Z., Hu, K., Thomson, P. C., House, J. K., & Khatkar, M. S. (2025) · Frontiers in Veterinary Science · doi:10.3389/fvets.2025.1511522 · Cited by 82 · see citing works
Comprehensive review of Transformer‐based models in neuroscience, neurology, and psychiatry
Cong, S., Wang, H., Zhou, Y., Wang, Z., Yao, X., & Yang, C. (2024) · Brain‐X · doi:10.1002/brx2.57 · Cited by 70 · see citing works
PharmaBench: Enhancing ADMET benchmarks with large language models
Niu, Z., Xiao, X., Wu, W., Cai, Q., Jiang, Y., Jin, W., Wang, M., Yang, G., Kong, L., Jin, X., et al. (2024) · Scientific Data · doi:10.1038/s41597-024-03793-0 · Cited by 65 · see citing works
“Some of these LLMs demonstrate state-of-the-art performance through one-shot or few-shot learning as a form of multi-task learning 17,20,21 .”
BackgroundThe Segment Anything Model (SAM) for accelerating the smart farming revolution
Carraro, A., Sozzi, M., & Marinello, F. (2023) · Smart Agricultural Technology · doi:10.1016/j.atech.2023.100367 · Cited by 67 · see citing works
SAM-Med3D: A Vision Foundation Model for General-Purpose Segmentation on Volumetric Medical Images
Wang, H., Guo, S., Ye, J., Deng, Z., Cheng, J., Li, T., Chen, J., Su, Y., Huang, Z., Shen, Y., et al. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2025.3586694 · Cited by 67 · see citing works
“[12], [13].”
An empirical study on the robustness of the segment anything model (SAM)
Wang, Y., Zhao, Y. C., & Petzold, L. R. (2024) · Pattern Recognition · doi:10.1016/j.patcog.2024.110685 · Cited by 59 · see citing works
Artificial General Intelligence for Medical Imaging Analysis
Li, X., Zhao, L., Zhang, L., Wu, Z., Liu, Z., Jiang, H., Cao, C., Xu, S., Li, Y., Dai, H., et al. (2024) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2024.3493775 · Cited by 68 · see citing works
Opportunities and challenges for deep learning in cell dynamics research
Chai, B., Efstathiou, C., Yue, H., & Draviam, V. M. (2023) · Trends in Cell Biology · doi:10.1016/j.tcb.2023.10.010 · Cited by 50 · see citing works
“Its performance appears to be competitive with or even superior to earlier fully supervised trained models and has been applied in medical imaging [8] and digital pathology [9].”
MMedAgent: Learning to Use Medical Tools with Multi-modal Agent
Li, B., Yan, T., Pan, Y., Luo, J., Ji, R., Ding, J., Xu, Z., Liu, S., Dong, H., Lin, Z., et al. (2024) · Findings of the Association for Computational Linguistics: EMNLP 2024 · doi:10.18653/v1/2024.findings-emnlp.510 · Cited by 54 · see citing works
Self-citationLeaf only SAM: A segment anything pipeline for zero-shot automated leaf segmentation
Williams, D., Macfarlane, F., & Britten, A. (2024) · Smart Agricultural Technology · doi:10.1016/j.atech.2024.100515 · Cited by 53 · see citing works
Kim, K., Cho, K., Jang, R., Kyung, S., Lee, S., Ham, S., Choi, E., Hong, G.-S., & Kim, N. (2024) · Korean Journal of Radiology · doi:10.3348/kjr.2023.0818 · Cited by 54 · see citing works
ScribblePrompt: Fast and Flexible Interactive Segmentation for Any Biomedical Image
Wong, H. E., Rakic, M., Guttag, J. V., & Dalca, A. V. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73661-2_12 · Cited by 59 · see citing works
“…foundation model trained on natural images, performs well at biomedical image segmentation tasks with clear boundaries ( e.g . organ segmentation in abdominal CT), but performs poorly on tasks and modalities involving more subtle delineations ( e.g . deep structures in Brain MRI) [45, 51, 92, 120].”
BackgroundMiao, J., Chen, C., Zhang, K., et al. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72120-5_16 · Cited by 47 · see citing works
Domain Generalization in Computational Pathology: Survey and Guidelines
Jahanifar, M., Raza, M., Xu, K., Vuong, T. T. L., Jewsbury, R., Shephard, A., Tajeddin, N. Z., Kwak, J. T., Raza, S. E. A., Minhas, F., et al. (2025) · ACM Computing Surveys · doi:10.1145/3724391 · Cited by 67 · see citing works
“Ideally, when feasible, sourcing annotations from a molecular baseline, such as PHH3 IHC staining for mitosis annotation, is recommended [95, 203].”
Gui, B., Bhardwaj, A., & Sam, L. (2024) · Remote Sensing · doi:10.3390/rs16020414 · Cited by 47 · see citing works
“The central concept of the model is to provide an intuitive and interactive approach for users to segment the specific elements they require [24].”
BackgroundMethodDeep Interactive Segmentation of Medical Images: A Systematic Review and Taxonomy
Marinov, Z., Jäger, P. F., Egger, J., Kleesiek, J., & Stiefelhagen, R. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3452629 · Cited by 47 · see citing works
“Mazurowski et al. [93] extensively evaluate SAM’s zero-shot performance on 33 datasets, exhibiting significant performance variations across tasks, ranging from 0.11 to 0.86 Intersection over Union.”
InfluentialBernatowicz, K., Amat, R., Prior, O., Frigola, J., Ligero, M., Grussu, F., Zatse, C., Serna, G., Nuciforo, P. G., Toledo, R. A., et al. (2025) · Journal for ImmunoTherapy of Cancer · doi:10.1136/jitc-2024-009140 · Cited by 43 · see citing works
Rao, K. S., Terlapu, P. V., Jayaram, D. K., Raju, K. K., Kumar, G., Pemula, R., Gopalachari, M. V., & Rakesh, S. (2024) · IEEE Access · doi:10.1109/access.2024.3358448 · Cited by 40 · see citing works
“The US can assist in identifying breast anomalies, guiding biopsies, and diagnosing and staging BC. DL, a subset of AI, employs NNs to recognize patterns in data [12].”
BackgroundPlug‐and‐play segment anything model improves nnUNet performance
Li, Y., Jing, B., Li, Z., Wang, J., & Zhang, Y. (2024) · Medical Physics · doi:10.1002/mp.17481 · Cited by 64 · see citing works
Self-citationMiao, J., Zhou, S.-P., Zhou, G., Wang, K.-N., Yang, M., Zhou, S.-J., & Chen, Y. (2023) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2023.3336534 · Cited by 41 · see citing works
“11, we first evaluate the zero-shot performance of SAM on the ACDC dataset using the fixed pre-trained model with the publicly available code from Mazurowski et al.’s work [77].”
MethodCoralSCOP: Segment any COral Image on this Planet
Zheng, Z., Liang, H., Hua, B., Wong, Y. H., Ang, P., Chui, A. P. Y., & Yeung, S.-K. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02661 · Cited by 41 · see citing works
“SAM has been widely used for medical image segmentation [42, 43, 48, 54], cam-ouflaged object segmentation [25, 52], challenging scenarios [9] and other applications [26, 60].”
Huang, Z., Jing, H., Liu, Y., et al. (2024) · Remote Sensing · doi:10.3390/rs16183489 · Cited by 39 · see citing works
“Compared to traditional segmentation models, the fundamental difference of the SAM lies in its ability to perform segmentation tasks directly using a pre-trained model, without the need for specialized training or fine-tuning for each task or dataset [38].”
Test-Time Adaptation with SaLIP: A Cascade of SAM and CLIP for Zero-shot Medical Image Segmentation
Aleem, S., Wang, F., Maniparambil, M., et al. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00526 · Cited by 40 · see citing works
“SAM has shown impressive results in a broad range of tasks for natural images but its performance has been subpar when directly applied to medical images [3, 6, 17, 37].”
Bian, Y., Li, J., Ye, C., Jia, X., & Yang, Q. (2025) · Chinese Medical Journal · doi:10.1097/cm9.0000000000003489 · Cited by 48 · see citing works
“Mazurowski et al [85] and Shi et al [86] observed similar results.”
Wang, C., Chen, H., Zhou, X., Wang, M., & Zhang, Q. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.123795 · Cited by 41 · see citing works
Semi-supervised crack detection using segment anything model and deep transfer learning
Li, J., Yuan, C., Wang, X., Chen, G., & Ma, G. (2024) · Automation in Construction · doi:10.1016/j.autcon.2024.105899 · Cited by 40 · see citing works
Oyelade, O. N., Aminu, E. F., Wang, H., et al. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.129018 · Cited by 36 · see citing works
Biomass phenotyping of oilseed rape through UAV multi-view oblique imaging with 3DGS and SAM model
Shen, Y., Zhou, H., Yang, X., Lu, X., Guo, Z., Jiang, L., He, Y., & Cen, H. (2025) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2025.110320 · Cited by 37 · see citing works
Song, B., Yang, H., Wu, Y., Zhang, P., Wang, B., & Han, G. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3411398 · Cited by 38 · see citing works
“This model exhibits the strong transfer and generalization abilities to efficiently perform tasks like instance and semantic segmentation, object detection, and more [31], [32], [33], in various downstream applications without new training.”
BackgroundZhang, Y., Yang, J., Liu, Y., et al. (2024) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm62325.2024.10821951 · Cited by 35 · see citing works
“…studies have revealed SAM’s limited performance in medical image segmentation due to the differences between natural images and medical images [20], [21], it still opened up new opportunities serving as a reliable pseudo-label generator to guide the segmentation task when manually annotated…”
Self-citationXia, Y., & Meng, Y. (2024) · Lubricants · doi:10.3390/lubricants12020062 · Cited by 37 · see citing works
“With the development of artificial intelligence (AI), deep learning methods have been successfully applied in industrial manufacturing [12,13], computer vision [14,15], and natural language processing [16,17].”
BackgroundAPSeg: Auto-Prompt Network for Cross-Domain Few-Shot Semantic Segmentation
He, W., Zhang, Y., Zhuo, W., et al. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02243 · Cited by 34 · see citing works
“However, recent studies [6, 9, 27, 39] have reported that applying SAM directly to new domains often yields subpar performance on zero-shot segmentation tasks, particularly when the data distribution significantly differs from the natural domain data used in SAM training.”
BackgroundSelf-citationSemiSAM+: Rethinking semi-supervised medical image segmentation in the era of foundation models
Zhang, Y., Lv, B., Xue, L., Zhang, W., Liu, Y., Fu, Y., Cheng, Y. H., & Qi, Y. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103733 · Cited by 45 · see citing works
Self-citationChen, H., Gou, L., Fang, Z., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-024-01372-6 · Cited by 33 · see citing works
“However,ithadnotbeenthoroughlyexploredinsegmentingmedical images 36 , particularly surgical images, and demonstrated poor performance in different segmentation modes including automatic, one-point, and one-box for IAM lesions, with Dice scores of 0.14, 0.02, and 0.29, respectively.”
InfluentialArtificial Intelligence in Pancreatic Image Analysis: A Review
Liu, W., Zhang, B., Liu, T., Jiang, J., & Liu, Y. (2024) · Sensors · doi:10.3390/s24144749 · Cited by 35 · see citing works
Sirimewan, D., Kunananthaseelan, N., Raman, S. N., Garcia, R., & Arashpour, M. (2024) · Waste Management · doi:10.1016/j.wasman.2024.09.018 · Cited by 30 · see citing works
FastSAM3D: An Efficient Segment Anything Model for 3D Volumetric Medical Images
Shen, Y., Li, J., Shao, X., Íñigo, B., Jindal, A., Dreizin, D., & Unberath, M. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72390-2_51 · Cited by 31 · see citing works
Transformer guided self-adaptive network for multi-scale skin lesion image segmentation
Xin, C., Liu, Z., Ma, Y., Wang, D., Zhang, J., Li, L., Zhou, Q., Xu, S., & Zhang, Y. (2023) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2023.107846 · Cited by 31 · see citing works
Self-citationLeSAM: Adapt Segment Anything Model for Medical Lesion Segmentation
Gu, Y., Wu, Q., Tang, H. N., Mai, X., Shu, H., Li, B., & Chen, Y. (2024) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2024.3406871 · Cited by 35 · see citing works
“Meanwhile, others have conducted comprehensive evaluations of SAM across multiple organ sites using medical images from various modalities [23], [24], [25], [26].”
BackgroundA review of deep learning approaches for multimodal image segmentation of liver cancer
Wu, C., Chen, Q., Wang, H., Guan, Y., Mian, Z., Huang, C. X., Ruan, C., Song, Q., Jiang, H., Pan, J., et al. (2024) · Journal of Applied Clinical Medical Physics · doi:10.1002/acm2.14540 · Cited by 30 · see citing works
“121 The study proposes a simple fine-tuning method to adapt SAM to generalized medical image segmentation tasks and conducts extensive experiments on 21 3D segmentation tasks and nine 2D segmentation tasks,with DSCs of the two of them being, respectively,22.5% and 17.6%,which are better than the…”
InfluentialJiang, Z., Cheng, D., Qin, Z., Gao, J. Y., Lao, Q., Ismoilovich, A. B., Gayrat, U., Yuldashov, E., Habibullo, B., Tang, D., et al. (2024) · Big Data Mining and Analytics · doi:10.26599/bdma.2024.9020058 · Cited by 32 · see citing works
Zhou, Z., Lu, Y., Bai, J., Campello, V. M., Feng, F., & Lekadir, K. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.125699 · Cited by 29 · see citing works
Few-Shot Learning for Medical Image Segmentation: A Review and Comparative Study
Dissanayake, T., George, Y., Mahapatra, D., Sridharan, S., Fookes, C., & Ge, Z. (2025) · ACM Computing Surveys · doi:10.1145/3746224 · Cited by 31 · see citing works
Xu, B., Jiang, Q., Zhao, X. Y., et al. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3368053 · Cited by 28 · see citing works
“However, recent studies have revealed that SAM encounters challenges in certain domains, such as transparent glasses [57], [58], medical polyps [59], [60], and camouflaged objects [61], [62].”
Self-Sampling Meta SAM: Enhancing Few-shot Medical Image Segmentation with Meta-Learning
Leng, T., Zhang, Y., Han, K., & Xie, X. S. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00774 · Cited by 31 · see citing works
“However, recent studies [40, 47] have shown that the performance of SAM is overall moderate and varies significantly across different datasets and different cases, demonstrating the potential promise of SAM within the context of medical images but also shows that the model cannot be applied…”
BackgroundSelf-citationChen, Z., & Sun, Q. (2024) · ACM Computing Surveys · doi:10.1145/3707447 · Cited by 28 · see citing works
Innovating robot-assisted surgery through large vision models
Min, Z., Lai, J., & Ren, H. (2025) · Nature Reviews Electrical Engineering · doi:10.1038/s44287-025-00166-6 · Cited by 31 · see citing works
“…or independent problems including the classification of medical images, the single-organ/multi-organ segmentation in the diagnostic images 33,34 , detection and segmentation of lesions in diagnostic images 10,98 , segmentation and pose estimation of surgical tools during surgery, depth…”
SAM4UDASS: When SAM Meets Unsupervised Domain Adaptive Semantic Segmentation in Intelligent Vehicles
Yan, W., Qian, Y., Zhuang, H., Wang, C., & Yang, M. Z. (2023) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2023.3344754 · Cited by 29 · see citing works
“Recent works based on SAM have extended its applications, including MedSAM [26] for medical image segmentation and Track Anything Model [27], [34] for object tracking and segmentation.”
MethodGao, X., Li, S., Su, X., et al. (2024) · Agriculture · doi:10.3390/agriculture14071098 · Cited by 27 · see citing works
Self-citationXu, Y., Tang, J., Men, A., & Chen, Q. (2024) · IEEE Transactions on Image Processing · doi:10.1109/tip.2024.3482175 · Cited by 36 · see citing works
“However, recent studies [5], [6] indicate its unstable performance across diverse medical datasets.”
BackgroundClinically Informed Automated Assessment of Finger Tapping Videos in Parkinson’s Disease
Yu, T., Park, K. W., McKeown, M. J., et al. (2023) · Sensors · doi:10.3390/s23229149 · Cited by 27 · see citing works
“As the data volume increases, this repetitive method facilitates continuous improvements and modifications to the models, leading to enhanced performance over time [45].”
BackgroundMethodBai, J., Zhou, Z., Ou, Z., Koehler, G., Stock, R., Maier-Hein, K. H., Elbatel, M., Martí, R., Li, X., Qiu, Y., et al. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103353 · Cited by 27 · see citing works
ROS-SAM: High-Quality Interactive Segmentation for Remote Sensing Moving Object
Shan, Z., Liu, Y., Zhou, L., Cheng, Y., Wang, H., & Xie, X. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00343 · Cited by 30 · see citing works
“…empirical studies have demonstrated that SAM’s powerful prior knowledge can effectively guide a range of downstream tasks, including medical image analysis [16, 31, 34, 70], 3D vision [7, 41, 53, 61], video analysis [10, 29, 56, 65], remote sensing [30, 32, 36, 55], and others [27, 35, 44, 62].”
Multimodal foundation model and benchmark for comprehensive retinal OCT image analysis
Morano, J., Fazekas, B., Sükei, E., Fecso, R., Emre, T., Gumpinger, M., Faustmann, G., Oghbaie, M., Schmidt-Erfurth, U. M., & Bogunović, H. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01852-3 · Cited by 31 · see citing works
“On the other hand, FMs for medical image segmentation 40 – 43 , most of them based on SAM 44 , are trained on large medical image datasets containing heterogeneous, unpaired image modalities (such as X-ray, magnetic resonance imaging, etc.) and some are not even trained on any OCT scans, such as…”
Diagnosis of malignancy in oropharyngeal confocal laser endomicroscopy using GPT 4.0 with vision
Sievert, M., Aubreville, M., Mueller, S. K., Eckstein, M., Breininger, K., Iro, H., & Goncalves, M. (2024) · European Archives of Oto-Rhino-Laryngology · doi:10.1007/s00405-024-08476-5 · Cited by 25 · see citing works
Embedded prompt tuning: Towards enhanced calibration of pretrained models for medical images
Zu, W., Xie, S., Qing, Z., Li, G., & Ма, Л. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103258 · Cited by 26 · see citing works
“Recent studies have attempted to explore the potential of foundation models in medical image analysis [11, 12, 13, 14, 15].”
Evaluating segment anything model (SAM) on MRI scans of brain tumors
Ali, L., Alnajjar, F. S. K., Swavaf, M., et al. (2024) · Scientific Reports · doi:10.1038/s41598-024-72342-x · Cited by 25 · see citing works
“The model’s efficiency and promptability allow it to achieve remarkable performance in natural image segmentation, often comparable to or surpassing results obtained through fully supervised approaches 20 .”
Resultde Grauw, M., Scholten, E. T., Smit, E. J., Rutten, M. J. C. M., Prokop, M., van Ginneken, B., & Hering, A. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103525 · Cited by 29 · see citing works
Improving Representation of High-Frequency Components for Medical Visual Foundation Models
Chu, Y., Zhang, Y., Han, Z., Yang, C., Zhou, L., Luo, G., Huang, C., & Gao, X. (2025) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2025.3559402 · Cited by 28 · see citing works
“In the field of healthcare, many foundation models have achieved significant progress across various medical tasks [4], [5], [6], [8], [9], marking a substantial advancement in medical AI research.”
Self-citationSegment Concealed Objects With Incomplete Supervision
He, C., Li, K., Zhang, Y., Yang, Z., Pang, Y., Tang, L., Fang, C., Zhang, Y., Kong, L., Li, X., et al. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3576209 · Cited by 30 · see citing works
“However, recent studies have highlighted that SAM encounters difficulties when segmenting objects with poor visibility, such as camouflaged objects [15], [16], medical polyps [17], [18], and transparent glasses [16].”
Self-citationLocal and Global Feature-Aware Dual-Branch Networks for Plant Disease Recognition
Lin, J., Zhang, X., Qin, Y., et al. (2024) · Plant Phenomics · doi:10.34133/plantphenomics.0208 · Cited by 24 · see citing works
Deep learning for pancreas segmentation on computed tomography: a systematic review
Moglia, A., Cavicchioli, M., Mainardi, L. T., & Cerveri, P. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-024-11050-4 · Cited by 26 · see citing works
Jiang, H., Gao, M., Liu, Z., et al. (2024) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2024.3377592 · Cited by 23 · see citing works
“[17]conducted athoroughassessmentofSAM’scapacityformedicalimageseg-mentationonacollectionof19medicalimagingdatasetsfrom differentmodalitiesandanatomies.”
Segmentation for mammography classification utilizing deep convolutional neural network
Saha, D. K., Hossain, T., Safran, M., Alfarhood, S., Mridha, M. F., & Che, D. (2024) · BMC Medical Imaging · doi:10.1186/s12880-024-01510-2 · Cited by 25 · see citing works
Wang, H., Huai, L., Li, W., Qi, L., Jiang, X., & Shi, Y. (2025) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2025.3549433 · Cited by 25 · see citing works
“However, it encounters challenges in certain medical image segmentation tasks, particularly when dealing with objects characterized by complex shapes, blurred boundaries, small sizes, or low contrast [5], [6].”
Leveraging foundation and large language models in medical artificial intelligence
Wong, I. N., Monteiro, O., Baptista‐Hon, D. T., Wang, K., Lu, W.-Y., Sun, Z., Nie, S., & Yin, Y. (2024) · Chinese Medical Journal · doi:10.1097/cm9.0000000000003302 · Cited by 25 · see citing works
Xie, J., Yang, Y., Jiang, Z., et al. (2024) · Frontiers in Physiology · doi:10.3389/fphys.2023.1281506 · Cited by 20 · see citing works
Data-Centric Foundation Models in Computational Healthcare: A Survey
Zhang, Y., Gao, J. G., Tan, Z., Zhou, L., Ding, K., Zhou, M., Zhang, S., & Wang, D. (2026) · ACM Computing Surveys · doi:10.1145/3800677 · Cited by 42 · see citing works
Self-citationTongueSAM: An Universal Tongue Segmentation Model Based on SAM with Zero-Shot
Cao, S. C., Wu, Q., & Ma, L. (2023) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm58861.2023.10385570 · Cited by 22 · see citing works
Rudnicka, Z., Szczepański, J., & Pręgowska, A. (2024) · Electronics · doi:10.3390/electronics13040746 · Cited by 26 · see citing works
“However, creating a 3D model of organs and/or their abnormalities is time-consuming and is often done manually or semi-automatically[10].”
BackgroundHuang, Y., Zou, J., Meng, L., Yue, X., Zhao, Q. Y., Li, J., Song, C., Jiménez, G., Li, S., & Fu, G. (2024) · IEEE Annual International Computer Software and Applications Conference (COMPSAC) · doi:10.1109/compsac61105.2024.00049 · Cited by 28 · see citing works
“In image analysis, the rising prominence of foundation models in computer vision has introduced novel approaches to the field [16], [17].”
BackgroundImproved genetic algorithm for multi-threshold optimization in digital pathology image segmentation
Huang, T., Yin, H., & Huang, X. (2024) · Scientific Reports · doi:10.1038/s41598-024-73335-6 · Cited by 26 · see citing works
- Authors
- Mazurowski, M. A., Dong, H., Gu, H., Yang, J., Konz, N., & Zhang, Y.
- Affiliations
- Duke University (US)
- Corresponding author
- Dong, Haoyu
- Venue
- Medical Image Analysis
- Publisher
- Elsevier BV
- Published
- 2023-10
- Volume
- 89
- Pages
- 102918
- ISSN
- 1361-8415 (ISSN-L)
- PMC
- PMC10528428
- OpenAlex
- W4385481295
- Semantic Scholar
- c6ac64877b2bcac2389be4d91a62dea8c6f73844
- Cited by
- 802
- References
- 40
- Citations by year
- 2026: 194 · 2025: 315 · 2024: 242 · 2023: 51
- FWCI
- 72.49
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
- www.elsevier.com/legal/tdmrep-license (tdm)
- www.elsevier.com/open-access/userlicense/1.0/ (am, 320 d)
- doi.org/10.15223/policy-017 (stm-asf)
- doi.org/10.15223/policy-037 (stm-asf)
- doi.org/10.15223/policy-012 (stm-asf)
- doi.org/10.15223/policy-029 (stm-asf)
- doi.org/10.15223/policy-004 (stm-asf)
- Locations
- Medical Image Analysis (acceptedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- National Institutes of Health
- National Heart, Lung, and Blood Institute (R44HL152825)
- National Institute of Biomedical Imaging and Bioengineering (R01EB031575)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- Segment Anything Model, medical image segmentation, zero-shot segmentation, interactive segmentation, point prompts, iterative segmentation, brain tumor segmentation, organ segmentation, medical imaging modalities, foundation models
- MeSH terms
- Brain Neoplasms, Humans, S-Adenosylmethionine, Tomography, X-Ray Computed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-08.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-08.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
