The DOI matches its Crossref record; 65 of the 65 references it cites were found in a registry or an index.
EfficientSAM: Leveraged Masked Image Pretraining for Efficient Segment Anything
Open accessAbstract
Segment Anything Model (SAM) has emerged as a powerful tool for numerous vision applications. A key component that drives the impressive performance for zero-shot trans-fer and high versatility is a super large Transformer model trained on the extensive high-quality SA -1 B dataset. While beneficial, the huge computation cost of SAM model has limited its applications to wider real-world applications. To address this limitation, we propose EfficientSAMs, light-weight SAM models that exhibits decent performance with largely reduced complexity. Our idea is based on leveraging masked image pretraining, SAMI, which learns to reconstruct features from SAM image encoder for effective visual representation learning. Further, we take SAMI-pretrained light-weight image encoders and mask decoder to build Effi-cientSAMs, and finetune the models on SA -1B for segment anything task. We perform evaluations on multiple vision tasks including image classification, object detection, in-stance segmentation, and semantic segmentation, and find that our proposed pretraining method, SAMI, consistently outperforms other masked image pretraining methods. On segment anything task such as zero-shot instance segmentation, our EfficientSAMs with SAMI-pretrained lightweight image encoders perform favorably with a significant gain (e.g., rv4 AP on COCOILVIS) over other fast SAM models. Our EfficientSAM code and models are available at here.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Exploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 1×
“…and several representative vision tasks to demonstrate the superiority of the proposed SAMI, including image classification on ImageNet dataset [16] with 1.2 million training and 50K validation images; Object detection and instance segmentation on COCO dataset [36] with 118K training and 5K…”
MethodInfluentialAn Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534
He, K., Gkioxari, G., Dollar, P., & Girshick, R. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.322 · Cited by 29,137
MobileNetV2: Inverted Residuals and Linear Bottlenecks
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L.-C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00474 · Cited by 28,755
End-to-End Object Detection with Transformers
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58452-8_13 · Cited by 18,285
End-to-End Object Detection with Transformers
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. (2020) · arXiv · doi:10.48550/arxiv.2005.12872 · Cited by 18,253
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204
Decoupled Weight Decay Regularization
Loshchilov, I., & Hutter, F. (2017) · arXiv · doi:10.48550/arxiv.1711.05101 · Cited by 12,522
Masked Autoencoders Are Scalable Vision Learners
He, K., Chen, X., Xie, S., Li, Y., Dollar, P., & Girshick, R. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01553 · Cited by 8,489 · cited 10×
“For reference, 1600-epoch pretraining is required for MAE[26].”
BackgroundMethodInfluentialContext Encoders: Feature Learning by Inpainting
Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T., & Efros, A. A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.278 · Cited by 5,691
Proceedings of the 25th international conference on Machine learning
Cohen, W. W., McCallum, A., & Roweis, S. T. (2008) · Cited by 5,691
Emerging Properties in Self-Supervised Vision Transformers
Caron, M., Touvron, H., Misra, I., Jegou, H., Mairal, J., Bojanowski, P., & Joulin, A. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00951 · Cited by 5,638 · cited 2×
“For the classification task, we compare the performance of ViT back-bones from different pretraining/distillation methods including MAE[26], SSTA[60], DMAE[2], BEiT[3], CAE[12], DINO[6], iBOT[73], DeiT[53], etc.”
BackgroundMethodContour Detection and Hierarchical Image Segmentation
Arbeláez, P., Maire, M., Fowlkes, C., & Malik, J. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.161 · Cited by 5,454
(2010) · Journal of Machine Learning Research · doi:10.5555/1756006.1953039 · Cited by 4,051
Exploring Simple Siamese Representation Learning
Chen, X., & He, K. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01549 · Cited by 3,725
Masked-attention Mask Transformer for Universal Image Segmentation
Cheng, B., Misra, I., Schwing, A. G., Kirillov, A., & Girdhar, R. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00135 · Cited by 3,565
Scene Parsing through ADE20K Dataset
Zhou, B., Zhao, H., Puig, X., Fidler, S., Barriuso, A., & Torralba, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.544 · Cited by 3,326
Randaugment: Practical automated data augmentation with a reduced search space
Cubuk, E. D., Zoph, B., Shlens, J., & Le, Q. V. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00359 · Cited by 3,223
RandAugment: Practical automated data augmentation with a reduced search space
Cubuk, E. D., Zoph, B., Shlens, J., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1909.13719 · Cited by 3,223
Segment anything in medical images
Ma, J., He, Y., Li, F., Han, L., You, C., & Wang, B. (2024) · Nature Communications · doi:10.1038/s41467-024-44824-z · Cited by 2,913
FitNets: Hints for Thin Deep Nets
Romero, A., Ballas, N., Kahou, S. E., Chassang, A., Gatta, C., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1412.6550 · Cited by 2,859
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Bourdev, L., Girshick, R., Hays, J., Perona, P., Ramanan, D., Zitnick, C. L., & Dollár, P. (2014) · arXiv · doi:10.48550/arxiv.1405.0312 · Cited by 2,579
Zero-shot text-to-image generation
Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., & Sutskever, I. (2021) · arXiv · doi:10.48550/arxiv.2102.12092 · Cited by 2,417
U2-Net: Going deeper with nested U-structure for salient object detection
Qin, X., Zhang, Z., Huang, C., Dehghan, M., Zaiane, O. R., & Jagersand, M. (2020) · Pattern Recognition · doi:10.1016/j.patcog.2020.107404 · Cited by 2,407
Multiscale vision transformers
Fan, H., Xiong, B., Mangalam, K., Li, Y., Yan, Z., Malik, J., & Feichtenhofer, C. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00675 · Cited by 1,407 · cited 1×
“ViTs [19] have achieved impressive performance in vision applications[5, 20, 26, 34, 39, 44].”
BackgroundBEiT: BERT Pre-Training of Image Transformers
Bao, H., Dong, L., Piao, S., & Wei, F. (2021) · arXiv · doi:10.48550/arxiv.2106.08254 · Cited by 1,392
LVIS: A Dataset for Large Vocabulary Instance Segmentation
Gupta, A., Dollar, P., & Girshick, R. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00550 · Cited by 1,274
SimMIM: a Simple Framework for Masked Image Modeling
Xie, Z., Zhang, Z., Cao, Y., Lin, Y., Bao, J., Yao, Z., Dai, Q., & Hu, H. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00943 · Cited by 1,255
MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer
Mehta, S., & Rastegari, M. (2021) · arXiv · doi:10.48550/arxiv.2110.02178 · Cited by 1,110 · cited 2×
“Motivated by the ability of convolution to capture local information with reduced parameters and computation costs, MobileViT[42] explore combining ViT with convolu-tions, which outperforms light-weight CNN models such as MobileNet-v2/v3[32, 48] with better task-level generalization properties and…”
MethodDecoupled Knowledge Distillation
Zhao, B., Cui, Q., Song, R., Qiu, Y., & Liang, J. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01165 · Cited by 965
EfficientViT: Memory Efficient Vision Transformer with Cascaded Group Attention
Liu, X., Peng, H., Zheng, N., Yang, Y., Hu, H., & Yuan, Y. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01386 · Cited by 956
LeViT: a Vision Transformer in ConvNet’s Clothing for Faster Inference
Graham, B., El-Nouby, A., Touvron, H., Stock, P., Joulin, A., Jegou, H., & Douze, M. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01204 · Cited by 935 · cited 1×
“This trick has been used in many follow-up works including LeViT[22], EfficientFormer[35], Next-ViT[33], Tiny-ViT[61], Castling-ViT[66], EfficientViT[38].”
MethodExploring Plain Vision Transformer Backbones for Object Detection
Li, Y., Mao, H., Girshick, R., & He, K. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-20077-9_17 · Cited by 809
Faster Segment Anything: Towards Lightweight SAM for Mobile ApplicationsPreprint
Zhang, C., Han, D., Qiao, Y., Kim, J. U., Bae, S., Lee, S., & Hong, C. S. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.14289 · Cited by 808
1Salient Object Detection: A Survey
Borji, A., Jiang, H., & Li, J. (2016) · Cited by 761
Segmentation as selective search for object recognition
van de Sande, K. E. A., Uijlings, J. R. R., Gevers, T., & Smeulders, A. W. M. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126456 · Cited by 739 · cited 1×
“…successful in the vision field, achieving state-of-the-art performance in a variety of image segmentation tasks such as zero-shot edge detection[1, 31], zero-shot object proposal generation[31, 54], and zero-shot instance segmentation[31], and many other real-world applications[24, 37, 41, 50–52].”
BackgroundDavidson, H. E. (2015) · The Consultant Pharmacist · doi:10.4140/tcp.n.2015.249 · Cited by 720
Dense Contrastive Learning for Self-Supervised Visual Pre-Training
Wang, X., Zhang, R., Shen, C., Kong, T., & Li, L. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00304 · Cited by 660
Masked Feature Prediction for Self-Supervised Visual Pre-Training
Wei, C., Fan, H., Xie, S., Wu, C.-Y., Yuille, A., & Feichtenhofer, C. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01426 · Cited by 587
Salient object detection: A survey
Borji, A., Cheng, M.-M., Hou, Q., Jiang, H., & Li, J. (2019) · Computational Visual Media · doi:10.1007/s41095-019-0149-9 · Cited by 524
BEiT v2: Masked Image Modeling with Vector-Quantized Visual TokenizersPreprint
Peng, Z., Li, D., Bao, H., Ye, Q., & Wei, F. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2208.06366 · Cited by 497
Anchor DETR: Query Design for Transformer-Based Detector
Wang, Y., Zhang, X., Yang, T., & Sun, J. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i3.20158 · Cited by 482
TinyViT: Fast Pretraining Distillation for Small Vision Transformers
Wu, K., Zhang, J., Peng, H., Liu, M., Xiao, B., Fu, J., & Yuan, L. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-19803-8_5 · Cited by 426
Segment and Track AnythingPreprint
Cheng, Y., Li, L., Xu, Y., Li, X., Yang, Z., Wang, W., & Yang, Y. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.06558 · Cited by 408
SAM-Adapter: Adapting Segment Anything in Underperformed Scenes
Chen, T., Zhu, L., Ding, C., Cao, R., Wang, Y., Zhang, S., Li, Z., Sun, L., Zang, Y., & Mao, P. (2023) · 2023 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW) · doi:10.1109/iccvw60793.2023.00361 · Cited by 402
Xie, Z., Lin, Y., Zhang, Z., Cao, Y., Lin, S., & Hu, H. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01641 · Cited by 370
Li, J., Xia, X., Li, W., Li, H., Wang, X., Xiao, X., Wang, R., Zheng, M., & Pan, X. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2207.05501 · Cited by 355
EfficientFormer: Vision Transformers at MobileNet Speed
Li, Y., Yuan, G., Wen, Y., Hu, J., Evangelidis, G., Tulyakov, S., Wang, Y., & Ren, J. (2022) · neural information processing systems · doi:10.52202/068431-0940 · Cited by 351
Context Autoencoder for Self-supervised Representation Learning
Chen, X., Ding, M., Wang, X., Xin, Y., Mo, S., Wang, Y., Han, S., Luo, P., Zeng, G., & Wang, J. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-023-01852-4 · Cited by 319
Deng, R., Cui, C., Liu, Q., Yao, T., Remedios, L. W., Bao, S., Landman, B. A., Wheless, L., Coburn, L. A., Wilson, K. T., et al. (2025) · Electronic Imaging · doi:10.2352/ei.2025.37.14.coimg-132 · Cited by 304
Koonce, B. (2021) · Apress eBooks · doi:10.1007/978-1-4842-6168-2_11 · Cited by 212
iBOT: Image BERT Pre-Training with Online Tokenizer
Zhou, J., Wei, C., Wang, H., Shen, W., Xie, C., Yuille, A., & Kong, T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.07832 · Cited by 201
Can SAM Segment Anything? When SAM Meets Camouflaged Object DetectionPreprint
Tang, L., Xiao, H., & Li, B. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2304.04709 · Cited by 184
Bootstrapped Masked Autoencoders for Vision BERT Pretraining
Dong, X., Bao, J., Zhang, T., Chen, D., Zhang, W., Yuan, L., Chen, D., Wen, F., & Yu, N. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20056-4_15 · Cited by 102
Masked Autoencoders Enable Efficient Knowledge Distillers
Bai, Y., Wang, Z., Xiao, J., Chen, W., Wang, H., Yuille, A., Zhou, Y., & Xie, C. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02323 · Cited by 97
MILAN: Masked Image Pretraining on Language Assisted RepresentationPreprint
Hou, Z., Sun, F., Chen, Y.-K., Xie, Y., & Kung, S. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2208.06049 · Cited by 95
Self-citationTrack Anything: Segment Anything Meets VideosPreprint
Yang, J., Gao, M., Li, Z., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2304.11968 · Cited by 93
You, H., Xiong, Y., Dai, X., Wu, B., Zhang, P., Fan, H., Vajda, P., & Lin, Y. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01387 · Cited by 84
Self-citationSegment Anything Model (SAM) Meets Glass: Mirror and Transparent Objects Cannot Be Easily DetectedPreprint
Han, D., Zhang, C., Qiao, Y., Qamar, M., Jung, Y., Lee, S., Bae, S., & Hong, C. S. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.00278 · Cited by 44
Anything-3D: Towards Single-view Anything Reconstruction in the WildPreprint
Shen, Q., Yang, X., & Wang, X. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2304.10261 · Cited by 33
EditAnything: Empowering Unparalleled Flexibility in Image Editing and Generation
Gao, S., Lin, Z., Xie, X., Zhou, P., Cheng, M., & Yan, S. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612680 · Cited by 26
Liu, R., Zhang, J., Peng, K., Zheng, J., Cao, K., Chen, Y., Yang, K., & Stiefelhagen, R. (2023) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw60793.2023.00200 · Cited by 25
Segment Anything Meets Semantic CommunicationPreprint
Tariq, S., Arfeto, B. E., Zhang, C., & Shin, H. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.02094 · Cited by 18
Cited by
Medical SAM adapter: Adapting segment anything model for medical image segmentation
Wu, J., Wang, Z., Hong, M., Ji, W., Fu, H., Xu, Y., Xu, M., & Jin, Y. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103547 · Cited by 677
DiffMIC-v2: Medical Image Classification via Improved Diffusion Network
Yang, Y., Fu, H., Aviles-Rivero, A. I., Xing, Z., & Zhu, L. (2025) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2025.3530399 · Cited by 41
Metadata
- Authors
- Xiong, Y., Varadarajan, B., Wu, L., Xiang, X., Xiao, F., Zhu, C., Dai, X., Wang, D., Sun, F., Iandola, F., Krishnamoorthi, R., & Chandra, V.
- Venue
- 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2024-06-16
- Pages
- 16111-16121
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W4402727760
- Cited by
- 266
- References
- 65
- FWCI
- 58.08
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.