The DOI matches its Crossref record; 68 of the 68 references it cites were found in a registry or an index.
Segment Anything for Microscopy
Open accessAbstract
Accurate segmentation of objects in microscopy images remains a bottleneck for many researchers despite the number of tools developed for this purpose. Here, we present Segment Anything for Microscopy (μSAM), a tool for segmentation and tracking in multidimensional microscopy data. It is based on Segment Anything, a vision foundation model for image segmentation. We extend it by fine-tuning generalist models for light and electron microscopy that clearly improve segmentation quality for a wide range of imaging conditions. We also implement interactive and automatic segmentation in a napari plugin that can speed up diverse segmentation tasks and provides a unified solution for microscopy annotation across different microscopy modalities. Our work constitutes the application of vision foundation models in microscopy, laying the groundwork for solving image analysis tasks in this domain with a small set of powerful deep learning models.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472
Matplotlib: a 2D graphics environment
Hunter, J. D. (2007) · Computing in Science & Engineering · doi:10.1109/mcse.2007.55 · Cited by 39,249
SciPy 1.0: fundamental algorithms for scientific computing in Python
Virtanen, P., Gommers, R., Oliphant, T. E., Haberland, M., Reddy, T., Cournapeau, D., Burovski, E., Peterson, P., Weckesser, W., Bright, J., et al. (2020) · Nature Methods · doi:10.1038/s41592-019-0686-2 · Cited by 38,430
Harris, C. R., Millman, K. J., van der Walt, S. J., Gommers, R., Virtanen, P., Cournapeau, D., Wieser, E., Taylor, J., Berg, S., Smith, N. J., et al. (2020) · Nature · doi:10.1038/s41586-020-2649-2 · Cited by 23,826
The pascal visual object classes (VOC) challenge
Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2010) · International Journal of Computer Vision · doi:10.1007/s11263-009-0275-4 · Cited by 19,599 · cited 2×
“Figure 2a shows the mean segmentation accuracy 27 (higher is better; see the Methods for details) for the default and LIVECell specialist model, using a separate test set for evaluation.”
Harris, C., Millman, K., van der Walt, S., Gommers, R., Virtanen, P., Cournapeau, D., Wieser, E., Taylor, J., Berg, S., Smith, N., et al. (2020) · RePEc: Research Papers in Economics · Cited by 17,844
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293
LoRA: Low-Rank Adaptation of Large Language Models
Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., & Chen, W. (2021) · arXiv · doi:10.48550/arxiv.2106.09685 · Cited by 13,453
Decoupled Weight Decay Regularization
Loshchilov, I., & Hutter, F. (2017) · arXiv · doi:10.48550/arxiv.1711.05101 · Cited by 12,522
Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A. C., Lo, W.-Y., et al. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00371 · Cited by 11,953
SciPy 1.0: fundamental algorithms for scientific computing in Python
Virtanen, P., Gommers, R., Oliphant, T. E., Haberland, M., Reddy, T., Cournapeau, D., Burovski, E., Peterson, P., Weckesser, W., Bright, J., et al. (2019) · Apollo - University of Cambridge Repository · doi:10.17863/cam.64134 · Cited by 10,460
QuPath: Open source software for digital pathology image analysis
Bankhead, P., Loughrey, M. B., Fernández, J. A., Dombrowski, Y., McArt, D. G., Dunne, P. D., McQuaid, S., Gray, R. T., Murray, L. J., Coleman, H. G., et al. (2017) · Scientific Reports · doi:10.1038/s41598-017-17204-5 · Cited by 8,908
scikit-image: image processing in Python
van der Walt, S., Schönberger, J. L., Nunez-Iglesias, J., Boulogne, F., Warner, J. D., Yager, N., Gouillart, E., & Yu, T. (2014) · PeerJ · doi:10.7717/peerj.453 · Cited by 6,794
Cellpose: a generalist algorithm for cellular segmentation
Stringer, C., Wang, T., Michaelos, M., & Pachitariu, M. (2021) · Nature Methods · doi:10.1038/s41592-020-01018-x · Cited by 4,369
UNETR: Transformers for 3D Medical Image Segmentation
Hatamizadeh, A., Tang, Y., Nath, V., Yang, D., Myronenko, A., Landman, B., Roth, H. R., & Xu, D. (2022) · 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv51458.2022.00181 · Cited by 3,586
UNETR: Transformers for 3D Medical Image Segmentation
Hatamizadeh, A., Tang, Y., Nath, V., Yang, D., Myronenko, A., Landman, B., Roth, H., & Xu, D. (2021) · arXiv · doi:10.48550/arxiv.2103.10504 · Cited by 3,580
Reback, J., McKinney, W., Bossche, J. V. D., Jbrockmendel, Augspurger, T., Cloud, P., Gfyoung, Sinhrks, Klein, A., Tratner, J., et al. (2019) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.3509134 · Cited by 3,464
ilastik: interactive machine learning for (bio)image analysis
Berg, S., Kutra, D., Kroeger, T., Straehle, C. N., Kausler, B. X., Haubold, C., Schiegg, M., Ales, J., Beier, T., Rudy, M., et al. (2019) · Nature Methods · doi:10.1038/s41592-019-0582-9 · Cited by 3,368
Segment anything in medical images
Ma, J., He, Y., Li, F., Han, L., You, C., & Wang, B. (2024) · Nature Communications · doi:10.1038/s41467-024-44824-z · Cited by 2,913 · cited 3×
“Retraining SAM for other domains has been investigated for a narrow interactive segmentation task in medical image data 20 .”
InfluentialHierarchical Text-Conditional Image Generation with CLIP Latents
Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., & Chen, M. (2022) · arXiv · doi:10.48550/arxiv.2204.06125 · Cited by 2,511 · cited 1×
“Our main contributions are: • A training procedure to fine-tune SAM, including a new decoder that provides improved instance segmentation results.”
Cell Detection with Star-Convex Polygons
Schmidt, U., Weigert, M., Broaddus, C., & Myers, G. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-00934-2_30 · Cited by 1,826
Cellpose 2.0: how to train your own model
Pachitariu, M., & Stringer, C. (2022) · Nature Methods · doi:10.1038/s41592-022-01663-4 · Cited by 1,381
TrackMate 7: integrating state-of-the-art segmentation algorithms into tracking pipelines
Ershov, D., Phan, M.-S., Pylvänäinen, J. W., Rigaud, S. U., Le Blanc, L., Charles-Orszag, A., Conway, J. R. W., Laine, R. F., Roy, N. H., Bonazzi, D., et al. (2022) · Nature Methods · doi:10.1038/s41592-022-01507-1 · Cited by 1,201
Saturated Reconstruction of a Volume of Neocortex
Kasthuri, N., Hayworth, K. J., Berger, D. R., Schalek, R. L., Conchello, J. A., Knowles-Barley, S., Lee, D., Vázquez-Reina, A., Kaynig, V., Jones, T. R., et al. (2015) · Cell · doi:10.1016/j.cell.2015.06.054 · Cited by 1,044
Greenwald, N. F., Miller, G., Moen, E., Kong, A., Kagel, A., Dougherty, T., Fullaway, C. C., McIntosh, B. J., Leow, K. X., Schwartz, M. S., et al. (2022) · Nature Biotechnology · doi:10.1038/s41587-021-01094-0 · Cited by 953
Nucleus segmentation across imaging experiments: the 2018 Data Science Bowl
Caicedo, J. C., Goodman, A., Karhohs, K. W., Cimini, B. A., Ackerman, J., Haghighi, M., Heng, C., Becker, T., Doan, M., McQuin, C., et al. (2019) · Nature Methods · doi:10.1038/s41592-019-0612-7 · Cited by 809
Faster Segment Anything: Towards Lightweight SAM for Mobile ApplicationsPreprint
Zhang, C., Han, D., Qiao, Y., Kim, J. U., Bae, S., Lee, S., & Hong, C. S. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.14289 · Cited by 808
Segment anything model for medical image analysis: An experimental study
Mazurowski, M. A., Dong, H., Gu, H., Yang, J., Konz, N., & Zhang, Y. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.102918 · Cited by 802 · cited 1×
“Prior work has already investigated SAM for biomedical applications, for example, in medical imaging 17 , histopathology 18 and neuroimaging 19 .”
Fast Segment AnythingPreprint
Zhao, X. F., Ding, W., An, Y., Du, Y., Yu, C., Li, M., Tang, M., & Wang, J. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.12156 · Cited by 736
An objective comparison of cell-tracking algorithms
Ulman, V., Maška, M., Magnusson, K. E. G., Ronneberger, O., Haubold, C., Harder, N., Matula, P., Matula, P., Svoboda, D., Radojevic, M., et al. (2017) · Nature Methods · doi:10.1038/nmeth.4473 · Cited by 565
Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A. C., Lo, W.-Y., et al. (2023) · arXiv · doi:10.48550/arxiv.2304.02643 · Cited by 527
Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model
Zhu, L., Liao, B., Zhang, Q., Wang, X., Liu, W., & Wang, X. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2401.09417 · Cited by 414
napari: a multi-dimensional image viewer for Python
Sofroniew, N., Lambert, T. J., Bokota, G., Nunez-Iglesias, J., Sobolewski, P., Sweet, A., Gaifas, L., Evans, K., Burt, A., Pop, D. D., et al. (2022) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.3555620 · Cited by 409
Self-citationSemantic Instance Segmentation with a Discriminative Loss Function
Brabandere, B. D., Neven, D., & Gool, L. V. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.02551 · Cited by 392
CellViT: Vision Transformers for precise cell segmentation and classification
Hörst, F., Rempe, M., Heine, L., Seibold, C., Keyl, J., Baldini, G., Ugurel, S., Siveke, J., Grünwald, B., Egger, J., et al. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103143 · Cited by 338
Accurate and versatile 3D segmentation of plant tissues at cellular resolution
Wolny, A., Cerrone, L., Vijayan, A., Tofanelli, R., Barro, A. V., Louveaux, M., Wenzl, C., Strauss, S., Wilson-Sánchez, D., Lymbouridou, R., et al. (2020) · eLife · doi:10.7554/elife.57613 · Cited by 326
Self-citationLIVECell—A large-scale dataset for label-free live cell segmentation
Edlund, C., Jackson, T. R., Khalid, N., Bevan, N., Dale, T., Dengel, A., Ahmed, S., Trygg, J., & Sjögren, R. (2021) · Nature Methods · doi:10.1038/s41592-021-01249-6 · Cited by 304
Self-citationDeng, R., Cui, C., Liu, Q., Yao, T., Remedios, L. W., Bao, S., Landman, B. A., Wheless, L., Coburn, L. A., Wilson, K. T., et al. (2025) · Electronic Imaging · doi:10.2352/ei.2025.37.14.coimg-132 · Cited by 304
Whole-cell organelle segmentation in volume electron microscopy
Heinrich, L., Bennett, D., Ackerman, D., Park, W., Bogovic, J., Eckstein, N., Petruncio, A., Clements, J., Pang, S., Xu, C. S., et al. (2021) · Nature · doi:10.1038/s41586-021-03977-3 · Cited by 292
Lizard: A Large-Scale Dataset for Colonic Nuclear Instance Segmentation and Classification
Graham, S., Jahanifar, M., Azam, A. S., Nimir, M., Tsang, Y., Dodd, K., Hero, E., Sahota, H., Tank, A., Benes, K., et al. (2021) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw54120.2021.00082 · Cited by 246
Profiling cellular diversity in sponges informs animal cell type and nervous system evolution
Musser, J. M., Schippers, K. J., Nickel, M., Mizzon, G., Kohn, A. B., Pape, C., Ronchi, P., Papadopoulos, N., Tarashansky, A. J., Hammel, J. U., et al. (2021) · Science · doi:10.1126/science.abj2949 · Cited by 242
Self-citationSegment Everything Everywhere All at Once
Zou, X., Yang, J., Zhang, H., Li, F., Li, L., Wang, J., Yang, J., Gao, J., & Lee, Y. J. (2023) · neural information processing systems · doi:10.52202/075280-0868 · Cited by 235
DeepBacs for multi-task bacterial image analysis using open-source deep learning approaches
Spahn, C. K., Gómez‐de‐Mariscal, E., Laine, R. F., Pereira, P. M., von Chamier, L., Conduit, M., Pinho, M. G., Jacquemet, G., Holden, S. J., Heilemann, M., et al. (2022) · Communications Biology · doi:10.1038/s42003-022-03634-z · Cited by 163
Zaimi, A., Wabartha, M., Herman, V., Antonsanti, P.-L., Perone, C. S., & Cohen-Adad, J. (2018) · Scientific Reports · doi:10.1038/s41598-018-22181-4 · Cited by 162
The multimodality cell segmentation challenge: toward universal solutions
Ma, J., Xie, R., Ayyadhury, S., Ge, C., Gupta, A., Gupta, R., Gu, S., Zhang, Y., Lee, G., Kim, J., et al. (2024) · Nature Methods · doi:10.1038/s41592-024-02233-6 · Cited by 154
Conrad, R., & Narayan, K. (2023) · Cell Systems · doi:10.1016/j.cels.2022.12.006 · Cited by 151
Whole-body integration of gene expression and single-cell morphology
Vergara, H. M., Pape, C., Meechan, K., Zinchenko, V., Genoud, C., Wanner, A. A., Mutemi, K. N., Titze, B., Templin, R., Bertucci, P. Y., et al. (2021) · Cell · doi:10.1016/j.cell.2021.07.017 · Cited by 146
Self-citationAnalysis of the Human Protein Atlas Image Classification competition
Ouyang, W., Winsnes, C. F., Hjelmare, M., Cesnik, A. J., Åkesson, L., Xu, H., Sullivan, D. P., Dai, S., Lan, J., Jinmo, P., et al. (2019) · Nature Methods · doi:10.1038/s41592-019-0658-6 · Cited by 131 · cited 1×
“Figure 2a shows the mean segmentation accuracy 27 (higher is better; see the Methods for details) for the default and LIVECell specialist model, using a separate test set for evaluation.”
BioImage Model Zoo: A Community-Driven Resource for Accessible Deep Learning in BioImage AnalysisPreprint
Ouyang, W., Beuttenmueller, F., Gómez‐de‐Mariscal, E., Pape, C., Burke, T., García-López-de-Haro, C., Russell, C., Sans, L. M. M., de-la-Torre-Gutiérrez, C., Schmidt, D., et al. (2022) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2022.06.07.495102 · Cited by 117
Self-citationSchwartz, M. S., Moen, E., Miller, G., Dougherty, T., Borba, E., Ding, R., Graf, W., Pao, E., & Valen, D. V. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/803205 · Cited by 110
Mache, C., Schulze, J., Holland, G., Bourquain, D. R., Gensch, J.-M., Oh, D., Nitsche, A., Dürrwald, R., Laue, M. M., & Wolff, T. (2022) · Communications Biology · doi:10.1038/s42003-022-04068-3 · Cited by 108
MitoEM Dataset: Large-Scale 3D Mitochondria Instance Segmentation from EM Images
Wei, D., Lin, Z., Franco-Barranco, D., Wendt, N., Liu, X., Yin, W., Huang, X., Gupta, A., Jang, W.-D., Wang, X., et al. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-59722-1_7 · Cited by 105
Belevich, I. N., & Jokitalo, E. (2021) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1008374 · Cited by 97
Local shape descriptors for neuron segmentation
Sheridan, A., Nguyen, T., Deb, D., Lee, W.-C. A., Saalfeld, S., Turaga, S. C., Manor, U., & Funke, J. (2022) · Nature Methods · doi:10.1038/s41592-022-01711-z · Cited by 91
Structured Image Segmentation Using Kernelized Features
Lucchi, A., Li, Y., Smith, K., & Fua, P. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33709-3_29 · Cited by 71
Mohapatra, S., Gosai, A., & Schlaug, G. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2304.04738 · Cited by 68
Embryo‐uterine interaction coordinates mouse embryogenesis during implantation
Bondarenko, V., Nikolaev, M., Kromm, D., Belousov, R., Wolny, A., Blotenburg, M., Zeller, P. J., Rezakhani, S., Hugger, J., Uhlmann, V., et al. (2023) · The EMBO Journal · doi:10.15252/embj.2022113280 · Cited by 66
The Mutex Watershed: Efficient, Parameter-Free Image Partitioning
Wolf, S., Pape, C., Bailoni, A., Rahaman, N., Kreshuk, A., Köthe, U., & Hamprecht, F. A. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01225-0_34 · Cited by 59
Self-citationTowards foundation models of biological image segmentation
Ma, J., & Wang, B. (2023) · Nature Methods · doi:10.1038/s41592-023-01885-0 · Cited by 57
Segmentation metric misinterpretations in bioimage analysis
Hirling, D., Tasnádi, E., Caicedo, J. C., Caroprese, M., Sjögren, R., Aubreville, M., Koós, K., & Horváth, P. (2023) · Nature Methods · doi:10.1038/s41592-023-01942-8 · Cited by 55
NucMM Dataset: 3D Neuronal Nuclei Instance Segmentation at Sub-Cubic Millimeter Scale
Lin, Z., Wei, D., Petkova, M. D., Wu, Y., Ahmed, Z., Krishna Swaroop, K., Zou, S., Wendt, N., Boulanger-Weill, J., Wang, X., et al. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-87193-2_16 · Cited by 49
Deep neural network automated segmentation of cellular structures in volume electron microscopy
Gallusser, B., Maltese, G., Caprio, G. D., Vadakkan, T. J., Sanyal, A., Somerville, E., Sahasrabudhe, M., O’Connor, J., WEIGERT, M. G., & Kirchhausen, T. L. (2022) · The Journal of Cell Biology · doi:10.1083/jcb.202208005 · Cited by 49
Automated reconstruction of whole-embryo cell lineages by learning from sparse annotations
Malin-Mayor, C., Hirsch, P., Guignard, L., McDole, K., Wan, Y., Lemon, W. C., Kainmueller, D., Keller, P., Preibisch, S., & Funke, J. (2022) · Nature Biotechnology · doi:10.1038/s41587-022-01427-7 · Cited by 47
Pape, C., Remme, R., Wolny, A., Olberg, S., Wolf, S., Cerrone, L., Cortese, M., Klaus, S., Lucic, B., Ullrich, S., et al. (2020) · BioEssays · doi:10.1002/bies.202000257 · Cited by 36
Self-citationTraining deep learning models for cell image segmentation with sparse annotationsPreprint
Sugawara, K. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.06.13.544786 · Cited by 36
Mekuč, M. Ž., Bohak, C., Boneš, E., Hudoklin, S., Romih, R., & Marolt, M. (2022) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2022.106959 · Cited by 19
Cited by
Cellpose3: one-click image restoration for improved cellular segmentation
Stringer, C., & Pachitariu, M. (2025) · Nature Methods · doi:10.1038/s41592-025-02595-5 · Cited by 461
Bridging electron microscopy and materials analysis with an autonomous agentic platform
Chen, G., Yuan, W., & You, F. (2026) · Science Advances · doi:10.1126/sciadv.aed0583 · Cited by 6
Metadata
- Authors
- Archit, A., Freckmann, L., Nair, S., Khalid, N., Hilt, P., Rajashekar, V., Freitag, M., Teuber, C., Spitzner, M., Tapia Contreras, C., Buckley, G., von Haaren, S., Gupta, S., Grade, M., Wirth, M., Schneider, G., Dengel, A., Ahmed, S., & Pape, C.
- Venue
- Nature Methods
- Published
- 2025-02-12
- Volume
- 22
- Issue
- 3
- Pages
- 579-591
- ISSN
- 1548-7091 (ISSN-L)
- 1548-7105
- OpenAlex
- W4407415703
- Cited by
- 307
- References
- 68
- FWCI
- 99.63
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.