The DOI matches its Crossref record; 43 of the 43 references it cites were found in a registry or an index; none of the 1 sources its claims cite is checked for support yet.
Segment anything model for medical image analysis: An experimental study
Open accessAbstract
Training segmentation models for medical images continues to be challenging due to the limited availability of data annotations. Segment Anything Model (SAM) is a foundation model trained on over 1 billion annotations, predominantly for natural images, that is intended to segment user-defined objects of interest in an interactive manner. While the model performance on natural images is impressive, medical image domains pose their own set of challenges. Here, we perform an extensive evaluation of SAM’s ability to segment medical images on a collection of 19 medical imaging datasets from various modalities and anatomies. In our experiments, we generated point and box prompts for SAM using a standard method that simulates interactive segmentation. We report the following findings: (1) SAM’s performance based on single prompts highly varies depending on the dataset and the task, from IoU=0.1135 for spine MRI to IoU=0.8650 for hip X-ray. (2) Segmentation performance appears to be better for well-circumscribed objects with prompts with less ambiguity such as the segmentation of organs in computed tomography and poorer in various other scenarios such as the segmentation of brain tumors. (3) SAM performs notably better with box prompts than with point prompts. (4) SAM outperforms similar methods RITM, SimpleClick, and FocalClick in almost all single-point prompt settings. (5) When multiple-point prompts are provided iteratively, SAM’s performance generally improves only slightly while other methods’ performance improves to the level that surpasses SAM’s point-based performance. We also provide several illustrations for SAM’s performance on all tested datasets, iterative segmentation, and SAM’s behavior given prompt ambiguity. We conclude that SAM shows impressive zero-shot segmentation performance for certain medical imaging datasets, but moderate to poor performance for others. SAM has the potential to make a significant impact in automated medical image segmentation in medical imaging, but appropriate care needs to be applied when using it. Code for evaluation SAM is made publicly available at https://github.com/mazurowski-lab/segment-anything-medical-evaluation.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
This experimental study evaluates the Segment Anything Model (SAM) for zero-shot medical image segmentation across 19 public datasets and 28 tasks. SAM shows highly variable performance, performs best with box prompts, and beats RITM/SimpleClick/FocalClick in most single-point settings, but gains little from iterative point prompts and remains moderate-to-poor on many medical tasks.
Key findings
- SAM’s single-prompt performance varied widely by dataset/task, from IoU=0.1135 for spine MRI to IoU=0.8650 for hip X-ray in the abstract; the Results section reports a range from IoU=0.1136 to IoU=0.9118.
- Box prompts were clearly superior to point prompts. Mode 4 (one box per object part) had the best average performance at IoU=0.6542; Mode 3 (single box around the entire object) averaged IoU=0.5891.
- SAM outperformed RITM, SimpleClick, and FocalClick on 24 of 28 tasks with a single non-iterative point prompt, and on 26 of 28 in oracle mode. Average IoU was 0.4595 for SAM, 0.5137 for SAM oracle, 0.2240 for FocalClick, 0.1910 for SimpleClick, and 0.1322 for RITM.
- With iterative point prompts, SAM improved only slightly and was surpassed by SimpleClick and RITM when five or more points were provided. The correlation between average object size and SAM performance was small and non-statistically significant.
Methods
- The authors curated 19 publicly available radiological medical imaging datasets covering X-ray, MRI, CT, ultrasound, and PET, yielding 28 segmentation tasks; for 3D modalities, 2D slices were used as inputs.
- They evaluated five non-iterative prompting modes using points or boxes, iterative point prompting guided by an error map, “segment everything” mode, and prompt ambiguity/oracle outputs.
- Performance was measured with IoU against ground-truth masks, using SAM’s highest-confidence mask; SAM was compared with RITM, SimpleClick, and FocalClick.
Limitations
- The paper states that text-based prompting is not commented on because it remains proof-of-concept, and that “segment everything” mode is not elaborated on. It also states future work should include prompt engineering and extension to 3D segmentation.
- The text makes evident that the study focuses on radiological images and uses 2D slices for 3D modalities, which may not reflect full 3D medical segmentation performance.
- The paper states that oracle performance assumes knowledge of the true mask and is biased when there is no additional user interaction after initial prompts; iterative prompting also uses the oracle prediction to guide the next prompt.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
12 Own result · 5 No basis given · 2 Own interpretation · 1 Not checked yet · 1 Own method
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“(1) SAM's performance based on single prompts highly varies depending on the dataset and the task, from IoU=0.1135 for spine MRI to IoU=0.8650 for hip X-ray.”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“(3) SAM performs notably better with box prompts than with point prompts.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“When used in oracle mode, SAM was better than all other methods for 26 out of 28 tasks.”
§ 3. Results > 3.2. Comparing SAM to other interactive segmen-
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Moreover, as expected, prompts where each sep- arate part of the object is indicated separately are generally superior to those where only one part is indicated, or all parts are outlined in one box.”
§ 3. Results > 3.1. Performance of SAM for different modes of use
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This figure illustrates a high variability of SAM's performance ranging from near- perfect for well-circumscribed objects with unambiguous prompts to very poor, particularly for objects with ambigu- ous prompts.”
§ 3. Results > 3.1. Performance of SAM for different modes of use
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“If SAM used Mode 3 (single box), it outperformed all other methods in all but one task.”
§ 3. Results > 3.2. Comparing SAM to other interactive segmen-
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“These modes also showed different performances in such scenarios.”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Following these two trends, Mode 4, where each part of an object is indicated by a separate box, showed the best performance with an average IoU of 0.6542.”
§ 3. Results > 3.1. Performance of SAM for different modes of use
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The magnitude of this improvement was highly dependent on the dataset.”
§ 3. Results > 3.1. Performance of SAM for different modes of use
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Additionally, the oracle mode showed a moderate im- provement over the default mode.”
§ 3. Results > 3.1. Performance of SAM for different modes of use
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- It may state two separate claims as one. 5
- The quote only partly states this claim. 1
- Probably not a substantial claim. 2
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We find that this mode provides mixed usefulness and that the results are somewhat dependent on the number of prompts.”
§ 3. Results > 3.6. Segment-everything mode for medical images
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This is due to the fact that SAM does not appear to draw hardly any benefits from additional information provided through the interactive points after two or three points have been provided.”
§ 3. Results > 3.3. Performance of SAM and other methods for
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one. The quote only partly states this claim.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Some datasets also showed high variability of object size within the dataset itself (e.g., kidneys in ultrasound images) while others showed relative consistency of object size (such as bones in hip X-ray images).”
§ 3. Results > 3.5. Performance of SAM for objects of different
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Overall, SAM shows promise for use in medical images, as long as suitable prompting strategies are used for the dataset and task of choice.”
plainconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Note that the correlation analysis looks at average object sizes in the image and does not consider the variation of object sizes in the individual datasets.”
§ 3. Results > 3.5. Performance of SAM for objects of different
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“The recently developed Segment Anything Model is a foundation model that has achieved promising zero-shot segmentation performance on a variety of natural image datasets [18].”
What the sources say
- Not checked yetcites [18] Jang, J., Hwang, D., 2022. M3T: Three-dimensional Medical image classifier using Multi-plane and Multi-slice Transformer. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 20718–20729. source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“Most previous studies (Ji et al., 2023a; Mazurowski et al., 2023; Zhou et al., 2023) evaluated SAM with limited or even only one type of testing mode / strategy.”
From Segment anything model for medical images? (2024) · cites this paper as Mazurowski et al., 2023
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Thus, the correlation between SAM's segmentation performance and medical objects' attributes has not been conducted carefully (He et al., 2023; Mazurowski et al., 2023; Ma and Wang, 2023).”
From Segment anything model for medical images? (2024) · cites this paper as Mazurowski et al., 2023
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“For example, the concurrent studies22,23 conducted a comprehensive assessment of SAM across a diverse array of medical images, underscoring that SAM achieved satisfactory segmentation outcomes primarily on targets characterized by distinct boundaries. However, the model exhibited substantial limitations in segmenting typical medical targets with weak boundaries or low contrast.”
From Segment anything in medical images (2024) · cites this paper as [22]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 3. Results > 3.1. Performance of SAM for different modes of use
“Mode 4, where each part of an object is indicated by a separate box”
§ C. Performance of other competing methods
“M od e 2: 1 b ox a t e ac h re gi on”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 2. Methodology > 2.1. Datasets
“We compiled and curated a set of 19 publicly available medical imaging datasets for image segmentation.”
§ C. Performance of other competing methods
“18 m ed ic al im ag in g da ta se ts”
Citations
Cites
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472 · cited 1×
“Based on the framework proposed by RITM, SimpleClick replaced the HRNet-18 backbone in RITM with a plain-ViT [9] and included a “click embedding” layer symmetric to patch embedding.”
MethodThe multimodal brain tumor image segmentation benchmark (BRATS)
Menze, B. H., Jakab, A., Bauer, S., Kalpathy-Cramer, J., Farahani, K., Kirby, J., Burren, Y., Porz, N., Slotboom, J., Wiest, R., et al. (2015) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2014.2377694 · Cited by 6,781 · cited 1×
“MRI-Brain The Multimodal Brain Tumor Image Segmentation Benchmark (BraTS) [11] MRI 3 Three parts of brain tumor 13,107”
BackgroundSegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers
Xie, E., Wang, W., Yu, Z., Anandkumar, A., Alvarez, J. M., & Luo, P. (2021) · arXiv · doi:10.48550/arxiv.2105.15203 · Cited by 5,090 · cited 2×
“Fo-calClick replaced the HRNet-18 backbone in RITM with a SegFormer [37] and restricted the update responding to new clicks to occur only as local.”
MethodDeep High-Resolution Representation Learning for Visual Recognition
Wang, J., Sun, K., Cheng, T., Jiang, B., Deng, C., Zhao, Y., Liu, D., Mu, Y., Tan, M., Wang, X., et al. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.2983686 · Cited by 4,894 · cited 1×
“RITM adopted HRNet-18 [36] as the backbone segmentation model and an iterative mask correction approach based on the { previous mask, new click } set to achieve outstanding performance on multiple natural imaging datasets.”
MethodSegment anything in medical images
Ma, J., He, Y., Li, F., Han, L., You, C., & Wang, B. (2024) · Nature Communications · doi:10.1038/s41467-024-44824-z · Cited by 2,913
Dataset of breast ultrasound images
Al-Dhabyani, W., Gomaa, M., Khaled, H., & Fahmy, A. (2020) · Data in Brief · doi:10.1016/j.dib.2019.104863 · Cited by 2,405 · cited 1×
“US-Breast Dataset of Breast Ultrasound Images [2] Ultrasound 1 Breast 780”
Current Methods in Medical Image Segmentation
Pham, D. L., Xu, C., & Prince, J. L. (2000) · Annual Review of Biomedical Engineering · doi:10.1146/annurev.bioeng.2.1.315 · Cited by 2,352 · cited 1×
“Image segmentation is a central task in medical image analysis, ranging from the segmentation of organs [17], abnormalities [3], bones [8], and others [13].”
BackgroundDeep Learning Techniques for Medical Image Segmentation: Achievements and Challenges
Hesamian, M. H., Jia, W., He, X., & Kennedy, P. J. (2019) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-019-00227-x · Cited by 1,710 · cited 1×
“Deep learning has dramatically altered how task is performed and brought significant performance improvements in many domains [6, 15].”
BackgroundMedical image analysis using convolutional neural networks: a review
Anwar, S. M., Majid, M., Qayyum, A., Awais, M., Alnowami, M., & Khan, M. K. (2018) · Journal of Medical Systems · doi:10.1007/s10916-018-1088-1 · Cited by 1,431 · cited 1×
“Image segmentation is a central task in medical image analysis, ranging from the segmentation of organs [17], abnormalities [3], bones [8], and others [13].”
BackgroundThe Liver Tumor Segmentation Benchmark (LiTS)
Bilic, P., Christ, P., Li, H. B., Vorontsov, E., Ben-Cohen, A., Kaissis, G., Szeskin, A., Jacobs, C., Mamani, G. E. H., Chartrand, G., et al. (2023) · Medical Image Analysis · doi:10.1016/j.media.2022.102680 · Cited by 1,412 · cited 1×
“CT-Liver The Liver Tumor Segmentation Benchmark (LiTS) [4] CT 1 Liver 5,501”
Background
Research works printed but not linked: 1
- Bradski, 2000, The OpenCV Library, Dr. Dobb’s J. Softw. Tools
Cited by
Sorted by citations
Segment anything in medical images
Ma, J., He, Y., Li, F., Han, L., You, C., & Wang, B. (2024) · Nature Communications · doi:10.1038/s41467-024-44824-z · Cited by 2,913
“These results highlight the potential of MedSAM as a powerful tool for medical image segmentation.”
Segment anything model for medical images?
Huang, Y., Yang, X., Liu, L., Zhou, H., Chang, A., Zhou, X., Chen, R., Yu, J., Chen, J., Chen, C., et al. (2024) · Medical Image Analysis · doi:10.1016/j.media.2023.103061 · Cited by 536
Transfer learning in environmental remote sensing
Ma, Y., Chen, S., Ermon, S., & Lobell, D. B. (2023) · Remote Sensing of Environment · doi:10.1016/j.rse.2023.113924 · Cited by 486
Segment anything model for medical image segmentation: Current applications and future directions
Zhang, Y., Shen, Z., & Jiao, R. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.108238 · Cited by 364
Self-citationOn the challenges and perspectives of foundation models for medical image analysis
Zhang, S., & Metaxas, D. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.102996 · Cited by 340
How Segment Anything Model (Sam) Boost Medical Image Segmentation: A SurveyPreprint
Zhang, Y., & Jiao, R. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4495221 · Cited by 272
Self-citationLangSplat: 3D Language Gaussian Splatting
Qin, M., Li, W., Zhou, J., Wang, H., & Pfister, H. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01895 · Cited by 411
“[38] introduced [48], super-resolution [25], image matting [45], object tracking [11, 42], medical image segmentation [29], image editing [12], and so on.”
BackgroundSegment Anything for Microscopy
Archit, A., Freckmann, L., Nair, S., Khalid, N., Hilt, P., Rajashekar, V., Freitag, M., Teuber, C., Spitzner, M., Contreras, C. T., et al. (2025) · Nature Methods · doi:10.1038/s41592-024-02580-4 · Cited by 426
“Prior work has already investigated SAM for biomedical applications, for example, in medical imaging 17 , histopathology 18 and neuroimaging 19 .”
Segment Anything Is Not Always Perfect: An Investigation of SAM on Different Real-world Applications
Ji, W., Li, J., Bi, Q., Liu, T., Li, W., & Cheng, L. (2024) · Machine Intelligence Research · doi:10.1007/s11633-023-1385-0 · Cited by 284
Review of large vision models and visual prompt engineering
Wang, J., Liu, Z., Zhao, L., Wu, Z., Ma, C., Yu, S., Dai, H., Yang, Q., Liu, Y., Zhang, S., et al. (2023) · Meta-Radiology · doi:10.1016/j.metrad.2023.100047 · Cited by 257
3DSAM-adapter: Holistic adaptation of SAM from 2D to 3D for promptable tumor segmentation
Gong, S., Zhong, Y., Ma, W., Li, J., Wang, Z., Zhang, J., Heng, P.-A., & Dou, Q. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103324 · Cited by 156
“This is also the underlying reason for the reported suboptimal performance when applying SAM to domain-specific tasks such as medical image segmentation [5, 6, 7, 8, 9, 10, 11].”
BackgroundSAM-6D: Segment Anything Model Meets Zero-Shot 6D Object Pose Estimation
Lin, J., Liu, L., Lu, D., & Jia, K. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02636 · Cited by 135
“SAM has demonstrated remarkable zero-shot transfer segmentation performance in real-world scenarios, including challenging situations such as medical images [36, 37, 71], camouflaged objects [22, 55], and transparent objects [13, 23].”
BackgroundSAM-Med3D: Towards General-purpose Segmentation Models for Volumetric Medical Images
Wang, H., Guo, S., Ye, J., Deng, Z., Cheng, J., Li, T., Chen, J., Su, Y., Huang, Z., Shen, Y., et al. (2023) · arXiv · doi:10.48550/arxiv.2310.15161 · Cited by 115
Deep-Learning-Based Approaches for Semantic Segmentation of Natural Scene Images: A Review
Soylu, B. E., Güzel, M. S., Bostancı, E., Ekinci, F., Aşuroğlu, T., & Açıcı, K. (2023) · Electronics · doi:10.3390/electronics12122730 · Cited by 127
“[169] have conducted a comprehensive assessment of the segmentation performance of SAM on a diverse set of 19 medical imaging datasets that include different modalities and anatomical regions.”
BackgroundFoundation Model for Advancing Healthcare: Challenges, Opportunities and Future Directions
He, Y., Huang, F., Jiang, X., Nie, Y., Wang, M., Wang, J., & Chen, H. (2024) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2024.3496744 · Cited by 162
“The existing HFMs still struggle with data heterogeneity (III-A3) in the real world [243] which performs poorly on some very specific domains [44].”
SurgicalSAM: Efficient Class Promptable Surgical Instrument Segmentation
Yue, W., Zhang, J., Hu, K. Y., Xia, Y., Luo, J., & Wang, Z. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i7.28514 · Cited by 117
Li, X., Zhang, L., Yang, J., & Teng, F. (2024) · Journal of Medical and Biological Engineering · doi:10.1007/s40846-024-00863-x · Cited by 117
“Furthermore, tailored image-naive architecture may outperform DL architecture on a task-specific basis [ 59].”
Self-citationSAM-Med3D: Towards General-Purpose Segmentation Models for Volumetric Medical Images
Wang, H., Guo, S., Jin, Y., Deng, Z., Cheng, J., Li, T., Chen, J., Su, Y., Huang, Z., Shen, Y., et al. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-91721-9_4 · Cited by 238
“However, directly applying SAM to 3D volumetric medical images is in-feasible, as disclosed in previous evaluations [2,26].”
MethodSegment Anything Model (SAM) for Medical Image Segmentation: A Preliminary Review
Zhang, L., Deng, X., & Yu, L. (2023) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm58861.2023.10386032 · Cited by 103
“Moreover, Mazurowski et al. [90] conduct an extensive evaluation of SAM’s medical image segmentation capabilities across diverse datasets, showcasing its proficiency with certain types of prompts.”
BackgroundFrom SAM to CAMs: Exploring Segment Anything Model for Weakly Supervised Semantic Segmentation
Kweon, H., & Yoon, K. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01844 · Cited by 97
“Over the past few months, numerous works [66, 67] have reported SAM’s ability to produce precise masks from ordinary images and to perform reasonably well, even in specialized domains like medical imagery [40, 41].”
BackgroundAli, M., Wu, T., Hu, H. R., Luo, Q., Xu, D. H., Zheng, W., Jin, N., Yang, C., & Yao, J. (2024) · Computerized Medical Imaging and Graphics · doi:10.1016/j.compmedimag.2024.102473 · Cited by 96
A Unified Framework With Multimodal Fine-Tuning for Remote Sensing Semantic Segmentation
Ma, X., Zhang, X., Pun, M.-O., & Huang, B. (2025) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2025.3585238 · Cited by 78
“Nowadays, SAM has already been applied across various fields, such as autonomous driving [38], medical image processing [39], and remote sensing [40], [41], [42], [43].”
MedLSAM: Localize and segment anything model for 3D CT images
Lei, W., Xu, W., Li, K., Zhang, X., & Zhang, S. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103370 · Cited by 135
Unleashing the Potential of SAM for Medical Adaptation via Hierarchical Decoding
Cheng, Z., Wei, Q., Zhu, H., Wang, Y., Qu, L., Shao, W., & Zhou, Y. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00337 · Cited by 86
“…in this field lies in the demand for deep learning models to undergo extensive training on large annotated datasets, a resource often challenging to procure within the medical domain. images diminishes notably in zero-shot settings, exhibiting reduced accuracy and robustness [26, 30, 36, 44, 51].”
BackgroundAdvances in attention mechanisms for medical image segmentation
Zhang, J., Chen, X., Yang, B., Guan, Q., Chen, Q., Chen, J., Wu, Q., Xie, Y., & Xia, Y. (2025) · Computer Science Review · doi:10.1016/j.cosrev.2024.100721 · Cited by 143
Segment Anything in Medical Images
Ma, J., He, Y., Li, F., Han, L., You, C., & Wang, B. (2023) · arXiv · doi:10.48550/arxiv.2304.12306 · Cited by 67
“These results highlight the potential of MedSAM as a powerful tool for medical image segmentation.”
Pu, X., Jia, H., Zheng, L., Wang, F., & Xu, F. (2025) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2025.3532690 · Cited by 102
Segment Anything for MicroscopyPreprint
Archit, A., Nair, S., Khalid, N., et al. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.08.21.554208 · Cited by 65
Review of applications of deep learning in veterinary diagnostics and animal health
Xiao, S., Dhand, N. K., Wang, Z., Hu, K., Thomson, P. C., House, J. K., & Khatkar, M. S. (2025) · Frontiers in Veterinary Science · doi:10.3389/fvets.2025.1511522 · Cited by 86
Comprehensive review of Transformer‐based models in neuroscience, neurology, and psychiatry
Cong, S., Wang, H., Zhou, Y., Wang, Z., Yao, X., & Yang, C. (2024) · Brain‐X · doi:10.1002/brx2.57 · Cited by 75
PharmaBench: Enhancing ADMET benchmarks with large language models
Niu, Z., Xiao, X., Wu, W., Cai, Q., Jiang, Y., Jin, W., Wang, M., Yang, G., Kong, L., Jin, X., et al. (2024) · Scientific Data · doi:10.1038/s41597-024-03793-0 · Cited by 73
“Some of these LLMs demonstrate state-of-the-art performance through one-shot or few-shot learning as a form of multi-task learning 17,20,21 .”
BackgroundThe Segment Anything Model (SAM) for accelerating the smart farming revolution
Carraro, A., Sozzi, M., & Marinello, F. (2023) · Smart Agricultural Technology · doi:10.1016/j.atech.2023.100367 · Cited by 77
SAM-Med3D: A Vision Foundation Model for General-Purpose Segmentation on Volumetric Medical Images
Wang, H., Guo, S., Ye, J., Deng, Z., Cheng, J., Li, T., Chen, J., Su, Y., Huang, Z., Shen, Y., et al. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2025.3586694 · Cited by 67
“[12], [13].”
An empirical study on the robustness of the segment anything model (SAM)
Wang, Y., Zhao, Y. C., & Petzold, L. R. (2024) · Pattern Recognition · doi:10.1016/j.patcog.2024.110685 · Cited by 61
Artificial General Intelligence for Medical Imaging Analysis
Li, X., Zhao, L., Zhang, L., Wu, Z., Liu, Z., Jiang, H., Cao, C., Xu, S., Li, Y., Dai, H., et al. (2024) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2024.3493775 · Cited by 91
Opportunities and challenges for deep learning in cell dynamics research
Chai, B., Efstathiou, C., Yue, H., & Draviam, V. M. (2023) · Trends in Cell Biology · doi:10.1016/j.tcb.2023.10.010 · Cited by 56
“Its performance appears to be competitive with or even superior to earlier fully supervised trained models and has been applied in medical imaging [8] and digital pathology [9].”
MMedAgent: Learning to Use Medical Tools with Multi-modal Agent
Li, B., Yan, T., Pan, Y., Luo, J., Ji, R., Ding, J., Xu, Z., Liu, S., Dong, H., Lin, Z., et al. (2024) · Findings of the Association for Computational Linguistics: EMNLP 2024 · doi:10.18653/v1/2024.findings-emnlp.510 · Cited by 54
Self-citationLeaf only SAM: A segment anything pipeline for zero-shot automated leaf segmentation
Williams, D., Macfarlane, F., & Britten, A. (2024) · Smart Agricultural Technology · doi:10.1016/j.atech.2024.100515 · Cited by 53
Kim, K., Cho, K., Jang, R., Kyung, S., Lee, S., Ham, S., Choi, E., Hong, G.-S., & Kim, N. (2024) · Korean Journal of Radiology · doi:10.3348/kjr.2023.0818 · Cited by 54
ScribblePrompt: Fast and Flexible Interactive Segmentation for Any Biomedical Image
Wong, H. E., Rakic, M., Guttag, J. V., & Dalca, A. V. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73661-2_12 · Cited by 85
“…foundation model trained on natural images, performs well at biomedical image segmentation tasks with clear boundaries ( e.g . organ segmentation in abdominal CT), but performs poorly on tasks and modalities involving more subtle delineations ( e.g . deep structures in Brain MRI) [45, 51, 92, 120].”
BackgroundMiao, J., Chen, C., Zhang, K., et al. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72120-5_16 · Cited by 47
Domain Generalization in Computational Pathology: Survey and Guidelines
Jahanifar, M., Raza, M., Xu, K., Vuong, T. T. L., Jewsbury, R., Shephard, A., Tajeddin, N. Z., Kwak, J. T., Raza, S. E. A., Minhas, F., et al. (2025) · ACM Computing Surveys · doi:10.1145/3724391 · Cited by 86
“Ideally, when feasible, sourcing annotations from a molecular baseline, such as PHH3 IHC staining for mitosis annotation, is recommended [95, 203].”
Gui, B., Bhardwaj, A., & Sam, L. (2024) · Remote Sensing · doi:10.3390/rs16020414 · Cited by 47
“The central concept of the model is to provide an intuitive and interactive approach for users to segment the specific elements they require [24].”
BackgroundMethodDeep Interactive Segmentation of Medical Images: A Systematic Review and Taxonomy
Marinov, Z., Jäger, P. F., Egger, J., Kleesiek, J., & Stiefelhagen, R. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3452629 · Cited by 50
“Mazurowski et al. [93] extensively evaluate SAM’s zero-shot performance on 33 datasets, exhibiting significant performance variations across tasks, ranging from 0.11 to 0.86 Intersection over Union.”
InfluentialBernatowicz, K., Amat, R., Prior, O., Frigola, J., Ligero, M., Grussu, F., Zatse, C., Serna, G., Nuciforo, P. G., Toledo, R. A., et al. (2025) · Journal for ImmunoTherapy of Cancer · doi:10.1136/jitc-2024-009140 · Cited by 49
Rao, K. S., Terlapu, P. V., Jayaram, D. K., Raju, K. K., Kumar, G., Pemula, R., Gopalachari, M. V., & Rakesh, S. (2024) · IEEE Access · doi:10.1109/access.2024.3358448 · Cited by 40
“The US can assist in identifying breast anomalies, guiding biopsies, and diagnosing and staging BC. DL, a subset of AI, employs NNs to recognize patterns in data [12].”
BackgroundPlug‐and‐play segment anything model improves nnUNet performance
Li, Y., Jing, B., Li, Z., Wang, J., & Zhang, Y. (2024) · Medical Physics · doi:10.1002/mp.17481 · Cited by 64
Self-citationMiao, J., Zhou, S.-P., Zhou, G., Wang, K.-N., Yang, M., Zhou, S.-J., & Chen, Y. (2023) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2023.3336534 · Cited by 41
“11, we first evaluate the zero-shot performance of SAM on the ACDC dataset using the fixed pre-trained model with the publicly available code from Mazurowski et al.’s work [77].”
MethodCoralSCOP: Segment any COral Image on this Planet
Zheng, Z., Liang, H., Hua, B., Wong, Y. H., Ang, P., Chui, A. P. Y., & Yeung, S.-K. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02661 · Cited by 44
“SAM has been widely used for medical image segmentation [42, 43, 48, 54], cam-ouflaged object segmentation [25, 52], challenging scenarios [9] and other applications [26, 60].”
Huang, Z., Jing, H., Liu, Y., et al. (2024) · Remote Sensing · doi:10.3390/rs16183489 · Cited by 39
“Compared to traditional segmentation models, the fundamental difference of the SAM lies in its ability to perform segmentation tasks directly using a pre-trained model, without the need for specialized training or fine-tuning for each task or dataset [38].”
Test-Time Adaptation with SaLIP: A Cascade of SAM and CLIP for Zero-shot Medical Image Segmentation
Aleem, S., Wang, F., Maniparambil, M., Arazo, E., Dietlmeier, J., Curran, K. M., O'Connor, N. E., & Little, S. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00526 · Cited by 49
“SAM has shown impressive results in a broad range of tasks for natural images but its performance has been subpar when directly applied to medical images [3, 6, 17, 37].”
Bian, Y., Li, J., Ye, C., Jia, X., & Yang, Q. (2025) · Chinese Medical Journal · doi:10.1097/cm9.0000000000003489 · Cited by 58
“Mazurowski et al [85] and Shi et al [86] observed similar results.”
Wang, C., Chen, H., Zhou, X., Wang, M., & Zhang, Q. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.123795 · Cited by 41
Semi-supervised crack detection using segment anything model and deep transfer learning
Li, J., Yuan, C., Wang, X., Chen, G., & Ma, G. (2024) · Automation in Construction · doi:10.1016/j.autcon.2024.105899 · Cited by 40
Oyelade, O. N., Aminu, E. F., Wang, H., et al. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.129018 · Cited by 36
Biomass phenotyping of oilseed rape through UAV multi-view oblique imaging with 3DGS and SAM model
Shen, Y., Zhou, H., Yang, X., Lu, X., Guo, Z., Jiang, L., He, Y., & Cen, H. (2025) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2025.110320 · Cited by 37
Song, B., Yang, H., Wu, Y., Zhang, P., Wang, B., & Han, G. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3411398 · Cited by 46
“This model exhibits the strong transfer and generalization abilities to efficiently perform tasks like instance and semantic segmentation, object detection, and more [31], [32], [33], in various downstream applications without new training.”
BackgroundZhang, Y., Yang, J., Liu, Y., Cheng, Y., & Qi, Y. (2024) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm62325.2024.10821951 · Cited by 67
“…studies have revealed SAM’s limited performance in medical image segmentation due to the differences between natural images and medical images [20], [21], it still opened up new opportunities serving as a reliable pseudo-label generator to guide the segmentation task when manually annotated…”
Self-citationXia, Y., & Meng, Y. (2024) · Lubricants · doi:10.3390/lubricants12020062 · Cited by 37
“With the development of artificial intelligence (AI), deep learning methods have been successfully applied in industrial manufacturing [12,13], computer vision [14,15], and natural language processing [16,17].”
BackgroundAPSeg: Auto-Prompt Network for Cross-Domain Few-Shot Semantic Segmentation
He, W., Zhang, Y., Zhuo, W., et al. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02243 · Cited by 34
“However, recent studies [6, 9, 27, 39] have reported that applying SAM directly to new domains often yields subpar performance on zero-shot segmentation tasks, particularly when the data distribution significantly differs from the natural domain data used in SAM training.”
BackgroundSelf-citationSemiSAM+: Rethinking semi-supervised medical image segmentation in the era of foundation models
Zhang, Y., Lv, B., Xue, L., Zhang, W., Liu, Y., Fu, Y., Cheng, Y. H., & Qi, Y. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103733 · Cited by 45
Self-citationChen, H., Gou, L., Fang, Z., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-024-01372-6 · Cited by 33
“However,ithadnotbeenthoroughlyexploredinsegmentingmedical images 36 , particularly surgical images, and demonstrated poor performance in different segmentation modes including automatic, one-point, and one-box for IAM lesions, with Dice scores of 0.14, 0.02, and 0.29, respectively.”
InfluentialArtificial Intelligence in Pancreatic Image Analysis: A Review
Liu, W., Zhang, B., Liu, T., Jiang, J., & Liu, Y. (2024) · Sensors · doi:10.3390/s24144749 · Cited by 35
Sirimewan, D., Kunananthaseelan, N., Raman, S. N., Garcia, R., & Arashpour, M. (2024) · Waste Management · doi:10.1016/j.wasman.2024.09.018 · Cited by 30
FastSAM3D: An Efficient Segment Anything Model for 3D Volumetric Medical Images
Shen, Y., Li, J., Shao, X., Íñigo, B., Jindal, A., Dreizin, D., & Unberath, M. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72390-2_51 · Cited by 31
Transformer guided self-adaptive network for multi-scale skin lesion image segmentation
Xin, C., Liu, Z., Ma, Y., Wang, D., Zhang, J., Li, L., Zhou, Q., Xu, S., & Zhang, Y. (2023) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2023.107846 · Cited by 31
Self-citationLeSAM: Adapt Segment Anything Model for Medical Lesion Segmentation
Gu, Y., Wu, Q., Tang, H. N., Mai, X., Shu, H., Li, B., & Chen, Y. (2024) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2024.3406871 · Cited by 37
“Meanwhile, others have conducted comprehensive evaluations of SAM across multiple organ sites using medical images from various modalities [23], [24], [25], [26].”
BackgroundA review of deep learning approaches for multimodal image segmentation of liver cancer
Wu, C., Chen, Q., Wang, H., Guan, Y., Mian, Z., Huang, C. X., Ruan, C., Song, Q., Jiang, H., Pan, J., et al. (2024) · Journal of Applied Clinical Medical Physics · doi:10.1002/acm2.14540 · Cited by 36
“121 The study proposes a simple fine-tuning method to adapt SAM to generalized medical image segmentation tasks and conducts extensive experiments on 21 3D segmentation tasks and nine 2D segmentation tasks,with DSCs of the two of them being, respectively,22.5% and 17.6%,which are better than the…”
InfluentialJiang, Z., Cheng, D., Qin, Z., Gao, J. Y., Lao, Q., Ismoilovich, A. B., Gayrat, U., Yuldashov, E., Habibullo, B., Tang, D., et al. (2024) · Big Data Mining and Analytics · doi:10.26599/bdma.2024.9020058 · Cited by 33
Zhou, Z., Lu, Y., Bai, J., Campello, V. M., Feng, F., & Lekadir, K. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.125699 · Cited by 29
Few-Shot Learning for Medical Image Segmentation: A Review and Comparative Study
Dissanayake, T., George, Y., Mahapatra, D., Sridharan, S., Fookes, C., & Ge, Z. (2025) · ACM Computing Surveys · doi:10.1145/3746224 · Cited by 31
Xu, B., Jiang, Q., Zhao, X. Y., Lu, C., Liang, H., & Liang, R. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3368053 · Cited by 32
“However, recent studies have revealed that SAM encounters challenges in certain domains, such as transparent glasses [57], [58], medical polyps [59], [60], and camouflaged objects [61], [62].”
Self-Sampling Meta SAM: Enhancing Few-shot Medical Image Segmentation with Meta-Learning
Leng, T., Zhang, Y., Han, K., & Xie, X. S. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00774 · Cited by 33
“However, recent studies [40, 47] have shown that the performance of SAM is overall moderate and varies significantly across different datasets and different cases, demonstrating the potential promise of SAM within the context of medical images but also shows that the model cannot be applied…”
BackgroundSelf-citationChen, Z., & Sun, Q. (2024) · ACM Computing Surveys · doi:10.1145/3707447 · Cited by 52
Innovating robot-assisted surgery through large vision models
Min, Z., Lai, J., & Ren, H. (2025) · Nature Reviews Electrical Engineering · doi:10.1038/s44287-025-00166-6 · Cited by 34
“…or independent problems including the classification of medical images, the single-organ/multi-organ segmentation in the diagnostic images 33,34 , detection and segmentation of lesions in diagnostic images 10,98 , segmentation and pose estimation of surgical tools during surgery, depth…”
SAM4UDASS: When SAM Meets Unsupervised Domain Adaptive Semantic Segmentation in Intelligent Vehicles
Yan, W., Qian, Y., Zhuang, H., Wang, C., & Yang, M. Z. (2023) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2023.3344754 · Cited by 31
“Recent works based on SAM have extended its applications, including MedSAM [26] for medical image segmentation and Track Anything Model [27], [34] for object tracking and segmentation.”
MethodGao, X., Li, S., Su, X., et al. (2024) · Agriculture · doi:10.3390/agriculture14071098 · Cited by 27
Self-citationXu, Y., Tang, J., Men, A., & Chen, Q. (2024) · IEEE Transactions on Image Processing · doi:10.1109/tip.2024.3482175 · Cited by 42
“However, recent studies [5], [6] indicate its unstable performance across diverse medical datasets.”
BackgroundClinically Informed Automated Assessment of Finger Tapping Videos in Parkinson’s Disease
Yu, T., Park, K. W., McKeown, M. J., et al. (2023) · Sensors · doi:10.3390/s23229149 · Cited by 27
“As the data volume increases, this repetitive method facilitates continuous improvements and modifications to the models, leading to enhanced performance over time [45].”
BackgroundMethodBai, J., Zhou, Z., Ou, Z., Koehler, G., Stock, R., Maier-Hein, K. H., Elbatel, M., Martí, R., Li, X., Qiu, Y., et al. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103353 · Cited by 27
ROS-SAM: High-Quality Interactive Segmentation for Remote Sensing Moving Object
Shan, Z., Liu, Y., Zhou, L., Cheng, Y., Wang, H., & Xie, X. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00343 · Cited by 30
“…empirical studies have demonstrated that SAM’s powerful prior knowledge can effectively guide a range of downstream tasks, including medical image analysis [16, 31, 34, 70], 3D vision [7, 41, 53, 61], video analysis [10, 29, 56, 65], remote sensing [30, 32, 36, 55], and others [27, 35, 44, 62].”
Multimodal foundation model and benchmark for comprehensive retinal OCT image analysis
Morano, J., Fazekas, B., Sükei, E., Fecso, R., Emre, T., Gumpinger, M., Faustmann, G., Oghbaie, M., Schmidt-Erfurth, U. M., & Bogunović, H. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01852-3 · Cited by 39
“On the other hand, FMs for medical image segmentation 40 – 43 , most of them based on SAM 44 , are trained on large medical image datasets containing heterogeneous, unpaired image modalities (such as X-ray, magnetic resonance imaging, etc.) and some are not even trained on any OCT scans, such as…”
Diagnosis of malignancy in oropharyngeal confocal laser endomicroscopy using GPT 4.0 with vision
Sievert, M., Aubreville, M., Mueller, S. K., Eckstein, M., Breininger, K., Iro, H., & Goncalves, M. (2024) · European Archives of Oto-Rhino-Laryngology · doi:10.1007/s00405-024-08476-5 · Cited by 25
Embedded prompt tuning: Towards enhanced calibration of pretrained models for medical images
Zu, W., Xie, S., Qing, Z., Li, G., & Ма, Л. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103258 · Cited by 26
“Recent studies have attempted to explore the potential of foundation models in medical image analysis [11, 12, 13, 14, 15].”
Evaluating segment anything model (SAM) on MRI scans of brain tumors
Ali, L., Alnajjar, F., Swavaf, M., Elharrouss, O., Abd-Alrazaq, A. A., & Damseh, R. (2024) · Scientific Reports · doi:10.1038/s41598-024-72342-x · Cited by 35
“The model’s efficiency and promptability allow it to achieve remarkable performance in natural image segmentation, often comparable to or surpassing results obtained through fully supervised approaches 20 .”
Resultde Grauw, M., Scholten, E. T., Smit, E. J., Rutten, M. J. C. M., Prokop, M., van Ginneken, B., & Hering, A. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103525 · Cited by 43
Improving Representation of High-Frequency Components for Medical Visual Foundation Models
Chu, Y., Zhang, Y., Han, Z., Yang, C., Zhou, L., Luo, G., Huang, C., & Gao, X. (2025) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2025.3559402 · Cited by 29
“In the field of healthcare, many foundation models have achieved significant progress across various medical tasks [4], [5], [6], [8], [9], marking a substantial advancement in medical AI research.”
Self-citationSegment Concealed Objects With Incomplete Supervision
He, C., Li, K., Zhang, Y., Yang, Z., Pang, Y., Tang, L., Fang, C., Zhang, Y., Kong, L., Li, X., et al. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3576209 · Cited by 34
“However, recent studies have highlighted that SAM encounters difficulties when segmenting objects with poor visibility, such as camouflaged objects [15], [16], medical polyps [17], [18], and transparent glasses [16].”
Self-citationLocal and Global Feature-Aware Dual-Branch Networks for Plant Disease Recognition
Lin, J., Zhang, X., Qin, Y., et al. (2024) · Plant Phenomics · doi:10.34133/plantphenomics.0208 · Cited by 24
Deep learning for pancreas segmentation on computed tomography: a systematic review
Moglia, A., Cavicchioli, M., Mainardi, L. T., & Cerveri, P. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-024-11050-4 · Cited by 26
Jiang, H., Gao, M., Liu, Z., et al. (2024) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2024.3377592 · Cited by 23
“[17]conducted athoroughassessmentofSAM’scapacityformedicalimageseg-mentationonacollectionof19medicalimagingdatasetsfrom differentmodalitiesandanatomies.”
Segmentation for mammography classification utilizing deep convolutional neural network
Saha, D. K., Hossain, T., Safran, M., Alfarhood, S., Mridha, M. F., & Che, D. (2024) · BMC Medical Imaging · doi:10.1186/s12880-024-01510-2 · Cited by 25
Wang, H., Huai, L., Li, W., Qi, L., Jiang, X., & Shi, Y. (2025) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2025.3549433 · Cited by 25
“However, it encounters challenges in certain medical image segmentation tasks, particularly when dealing with objects characterized by complex shapes, blurred boundaries, small sizes, or low contrast [5], [6].”
Leveraging foundation and large language models in medical artificial intelligence
Wong, I. N., Monteiro, O., Baptista‐Hon, D. T., Wang, K., Lu, W.-Y., Sun, Z., Nie, S., & Yin, Y. (2024) · Chinese Medical Journal · doi:10.1097/cm9.0000000000003302 · Cited by 27
Xie, J., Yang, Y., Jiang, Z., et al. (2024) · Frontiers in Physiology · doi:10.3389/fphys.2023.1281506 · Cited by 20
Data-Centric Foundation Models in Computational Healthcare: A Survey
Zhang, Y., Gao, J. G., Tan, Z., Zhou, L., Ding, K., Zhou, M., Zhang, S., & Wang, D. (2026) · ACM Computing Surveys · doi:10.1145/3800677 · Cited by 65
Self-citationTongueSAM: An Universal Tongue Segmentation Model Based on SAM with Zero-Shot
Cao, S. C., Wu, Q., & Ma, L. (2023) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm58861.2023.10385570 · Cited by 22
Rudnicka, Z., Szczepański, J., & Pręgowska, A. (2024) · Electronics · doi:10.3390/electronics13040746 · Cited by 29
“However, creating a 3D model of organs and/or their abnormalities is time-consuming and is often done manually or semi-automatically[10].”
BackgroundHuang, Y., Zou, J., Meng, L., Yue, X., Zhao, Q. Y., Li, J., Song, C., Jiménez, G., Li, S., & Fu, G. (2024) · IEEE Annual International Computer Software and Applications Conference (COMPSAC) · doi:10.1109/compsac61105.2024.00049 · Cited by 35
“In image analysis, the rising prominence of foundation models in computer vision has introduced novel approaches to the field [16], [17].”
BackgroundImproved genetic algorithm for multi-threshold optimization in digital pathology image segmentation
Huang, T., Yin, H., & Huang, X. (2024) · Scientific Reports · doi:10.1038/s41598-024-73335-6 · Cited by 27
Metadata
- Authors
- Mazurowski, M. A., Dong, H., Gu, H., Yang, J., Konz, N., & Zhang, Y.
- Affiliations
- Duke University (US)
- Corresponding author
- Dong, Haoyu
- Venue
- Medical Image Analysis
- Publisher
- Elsevier BV
- Published
- 2023-10
- Volume
- 89
- Pages
- 102918
- ISSN
- 1361-8415 (ISSN-L)
- PMC
- PMC10528428
- OpenAlex
- W4385481295
- Semantic Scholar
- c6ac64877b2bcac2389be4d91a62dea8c6f73844
- Cited by
- 802
- References
- 40
- Citations by year
- 2026: 194 · 2025: 315 · 2024: 242 · 2023: 51
- FWCI
- 72.49
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
- www.elsevier.com/legal/tdmrep-license (tdm)
- www.elsevier.com/open-access/userlicense/1.0/ (am, 320 d)
- doi.org/10.15223/policy-017 (stm-asf)
- doi.org/10.15223/policy-037 (stm-asf)
- doi.org/10.15223/policy-012 (stm-asf)
- doi.org/10.15223/policy-029 (stm-asf)
- doi.org/10.15223/policy-004 (stm-asf)
- Locations
- Medical Image Analysis (acceptedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- National Institutes of Health
- National Heart, Lung, and Blood Institute (R44HL152825)
- National Institute of Biomedical Imaging and Bioengineering (R01EB031575)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- Segment Anything Model, medical image segmentation, zero-shot segmentation, interactive segmentation, point prompts, iterative segmentation, brain tumor segmentation, organ segmentation, medical imaging modalities, foundation models
- MeSH terms
- Brain Neoplasms, Humans, S-Adenosylmethionine, Tomography, X-Ray Computed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-08, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-08 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
