The DOI matches its DataCite record; 65 of the 70 references it cites were found in a registry or an index.
Swin Transformer V2: Scaling Up Capacity and Resolution
Open accessAbstract
Large-scale NLP models have been shown to significantly improve the performance on language tasks with no signs of saturation. They also demonstrate amazing few-shot capabilities like that of human beings. This paper aims to explore large-scale models in computer vision. We tackle three major issues in training and application of large vision models, including training instability, resolution gaps between pre-training and fine-tuning, and hunger on labelled data. Three main techniques are proposed: 1) a residual-post-norm method combined with cosine attention to improve training stability; 2) A log-spaced continuous position bias method to effectively transfer models pre-trained using low-resolution images to downstream tasks with high-resolution inputs; 3) A self-supervised pre-training method, SimMIM, to reduce the needs of vast labeled images. Through these techniques, this paper successfully trained a 3 billion-parameter Swin Transformer V2 model, which is the largest dense vision model to date, and makes it capable of training with images of up to 1,536$\times$1,536 resolution. It set new performance records on 4 representative vision tasks, including ImageNet-V2 image classification, COCO object detection, ADE20K semantic segmentation, and Kinetics-400 video action classification. Also note our training is much more efficient than that in Google's billion-level visual models, which consumes 40 times less labelled data and 40 times less training time. Code is available at \url{https://github.com/microsoft/Swin-Transformer}.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“we reduce the model size to 1/15 compared to SwinV2-G [22].”
From DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection (2022) · cites this paper as [22]
The cited source says what the claim attributes to it.
In the citing paper
“Compared with the previous SOTA models, we use a much smaller model size (1 / 15 parameters compared with SwinV2-G [22]), backbone pre-training data size (1 / 60 images compared with Florence), and detection pretraining data size (1 / 5 images compared with Florence), while achieving better results.”
From DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection (2022) · cites this paper as [22]
The cited source says what the claim attributes to it.
In the citing paper
“Without using any other tricks, we achieve the result of 63 . 1 on val2017 and 63 . 2 on test-dev without test time augmentation (TTA) (see Appendix A), outperforming the previous state-of-the-art result 63 . 1 achieved by SwinV2 [22] with a much neater solution.”
From DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection (2022) · cites this paper as [22]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472
Research works printed but not linked: 15
- Navaneeth Bodla, Bharat Singh, Rama Chellappa, and Larry S. Davis. Soft-nms -improving object detection with one line of code. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), Oct 2017. 9, 10
- Zhaowei Cai and Nuno Vasconcelos. Cascade r-cnn: Delv- ing into high quality object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recogni- tion, pages 6154-6162, 2018. 9
- Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to- end object detection with transformers. In European Confer- ence on Computer Vision, pages 213-229. Springer, 2020. 9
- Kai Chen, Jiangmiao Pang, Jiaqi Wang, Yu Xiong, Xiaox- iao Li, Shuyang Sun, Wansen Feng, Ziwei Liu, Jianping Shi, Wanli Ouyang, et al. Hybrid task cascade for instance segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4974- 4983, 2019. 9, 10
- Kai Chen, Jiaqi Wang, Jiangmiao Pang, Yuhang Cao, Yu Xiong, Xiaoxiao Li, Shuyang Sun, Wansen Feng, Ziwei Liu, Jiarui Xu, et al. Mmdetection: Open mmlab detection tool- box and benchmark. arXiv preprint arXiv:1906.07155, 2019. 9 · arXiv
- Ekin D Cubuk, Barret Zoph, Jonathon Shlens, and Quoc V Le. Randaugment: Practical automated data augmenta- tion with a reduced search space. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pages 702-703, 2020. 9, 10
- Hao-Shu Fang, Jianhua Sun, Runzhong Wang, Minghao Gou, Yong-Lu Li, and Cewu Lu. Instaboost: Boosting instance segmentation via probability map guided copy- pasting. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 682-691, 2019. 9
- Golnaz Ghiasi, Tsung-Yi Lin, and Quoc V Le. Nas-fpn: Learning scalable feature pyramid architecture for object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7036- 7045, 2019. 9
- Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Gir- shick. Mask r-cnn. In Proceedings of the IEEE international conference on computer vision, pages 2961-2969, 2017. 9
- Gao Huang, Yu Sun, Zhuang Liu, Daniel Sedra, and Kil- ian Q Weinberger. Deep networks with stochastic depth. In European conference on computer vision, pages 646-661. Springer, 2016. 9, 10
- Ilya Loshchilov and Frank Hutter. Decoupled weight de- cay regularization. In International Conference on Learning Representations, 2019. 9, 10
- Peize Sun, Rufeng Zhang, Yi Jiang, Tao Kong, Chen- feng Xu, Wei Zhan, Masayoshi Tomizuka, Lei Li, Zehuan Yuan, Changhu Wang, et al. Sparse r-cnn: End-to-end object detection with learnable proposals. arXiv preprint arXiv:2011.12450, 2020. 9 · arXiv
- Sangdoo Yun, Dongyoon Han, Seong Joon Oh, Sanghyuk Chun, Junsuk Choe, and Youngjoon Yoo. Cutmix: Regular- ization strategy to train strong classifiers with localizable fea- tures. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 6023-6032, 2019. 9
- Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimiza- tion. arXiv preprint arXiv:1710.09412, 2017. 9 · arXiv
- Zhun Zhong, Liang Zheng, Guoliang Kang, Shaozi Li, and Yi Yang. Random erasing data augmentation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 13001-13008, 2020. 9, 10
Cited by
Sorted by citations
YOLOv7: Trainable Bag-of-Freebies Sets New State-of-the-Art for Real-Time Object Detectors
Wang, C.-Y., Bochkovskiy, A., & Liao, H.-Y. M. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00721 · Cited by 13,198
YOLOv9: Learning What You Want to Learn Using Programmable Gradient Information
Wang, C.-Y., Yeh, I.-H., & Mark Liao, H.-Y. (2025) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-72751-1_1 · Cited by 4,823
Run, Don't Walk: Chasing Higher FLOPS for Faster Neural Networks
Chen, J., Kao, S.-H., He, H., Zhuo, W., Song, W., Lee, C.-H., & Chan, S.-H. G. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01157 · Cited by 2,647
Scaling Up Your Kernels to 31×31: Revisiting Large Kernel Design in CNNs
Ding, X., Zhang, X., Han, J., & Ding, G. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01166 · Cited by 1,592
SimMIM: a Simple Framework for Masked Image Modeling
Xie, Z., Zhang, Z., Cao, Y., Lin, Y., Bao, J., Yao, Z., Dai, Q., & Hu, H. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00943 · Cited by 1,495
Self-citationDepth Anything: Unleashing the Power of Large-Scale Unlabeled Data
Yang, L., Kang, B., Huang, Z., Xu, X., Feng, J., & Zhao, H. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.00987 · Cited by 1,296
InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutions
Wang, W., Dai, J., Chen, Z., Huang, Z., Li, Z., Zhu, X., Hu, X., Lu, T., Lu, L., Li, H., et al. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01385 · Cited by 1,080
Exploring Plain Vision Transformer Backbones for Object Detection
Li, Y., Mao, H., Girshick, R., & He, K. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20077-9_17 · Cited by 1,027
EfficientViT: Memory Efficient Vision Transformer with Cascaded Group Attention
Liu, X., Peng, H., Zheng, N., Yang, Y., Hu, H., & Yuan, Y. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01386 · Cited by 956
Self-citationDINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection
Zhang, H., Li, F., Liu, S., Zhang, L., Su, H., Zhu, J., Ni, L. M., & Shum, H.-Y. (2022) · arXiv · Cited by 1,154
Rep ViT: Revisiting Mobile CNN From ViT Perspective
Wang, A., Chen, H., Lin, Z., Han, J., & Ding, G. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01506 · Cited by 752
Point Transformer V3: Simpler, Faster, Stronger
Wu, X., Jiang, L., Wang, P., Liu, Z., Liu, X., Qiao, Y., Ouyang, W., He, T., & Zhao, H. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00463 · Cited by 863
Self-citationCoelho, L. P. (2023) · Bioengineering · doi:10.3390/bioengineering10121435 · Cited by 812
DETRs with Collaborative Hybrid Assignments Training
Zong, Z., Song, G., & Liu, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00621 · Cited by 725
Masked Feature Prediction for Self-Supervised Visual Pre-Training
Wei, C., Fan, H., Xie, S., Wu, C.-Y., Yuille, A., & Feichtenhofer, C. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01426 · Cited by 587
Mask DINO: Towards A Unified Transformer-based Framework for Object Detection and Segmentation
Li, F., Zhang, H., Xu, H., Liu, S., Zhang, L., Ni, L. M.-S., & Shum, H. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00297 · Cited by 662
A Survey on Self-Supervised Learning: Algorithms, Applications, and Future Trends
Gui, J., Chen, T., Zhang, J., Cao, Q., Sun, Z., Luo, H., & Tao, D. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3415112 · Cited by 730
OneFormer: One Transformer to Rule Universal Image Segmentation
Jain, J., Li, J., Chiu, M. T., Hassani, A., Orlov, N., & Shi, H. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00292 · Cited by 611
Image as a Foreign Language: BEIT Pretraining for Vision and Vision-Language Tasks
Wang, W., Bao, H., Dong, L., Bjorck, J., Peng, Z., Liu, Q., Aggarwal, K., Mohammed, O. K., Singhal, S., Som, S., et al. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01838 · Cited by 561
Self-citationEVA: Exploring the Limits of Masked Visual Representation Learning at Scale
Fang, Y., Wang, W., Xie, B., Sun, Q., Wu, L., Wang, X., Huang, T., Wang, X., & Cao, Y. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01855 · Cited by 655
Self-citationVideoMAE V2: Scaling Video Masked Autoencoders with Dual Masking
Wang, L., Huang, B., Zhao, Z., Tong, Z., He, Y., Wang, Y., Wang, Y., & Qiao, Y. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01398 · Cited by 536
TinyViT: Fast Pretraining Distillation for Small Vision Transformers
Wu, K., Zhang, J., Peng, H., Liu, M., Xiao, B., Fu, J., & Yuan, L. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19803-8_5 · Cited by 521
TransNeXt: Robust Foveal Visual Perception for Vision Transformers
Shi, D. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01683 · Cited by 474
MedNeXt: Transformer-Driven Scaling of ConvNets for Medical Image Segmentation
Roy, S., Koehler, G., Ulrich, C., Baumgartner, M., Petersen, J., Isensee, F., Jäger, P. F., & Maier-Hein, K. H. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-43901-8_39 · Cited by 472
MambaVision: A Hybrid Mamba-Transformer Vision Backbone
Hatamizadeh, A., & Kautz, J. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02352 · Cited by 554
FuXi: a cascade machine learning forecasting system for 15-day global weather forecast
Chen, L., Zhong, X., Zhang, F., Cheng, Y., Xu, Y., Qi, Y., & Li, H. (2023) · npj Climate and Atmospheric Science · doi:10.1038/s41612-023-00512-1 · Cited by 367
Rethinking Vision Transformers for MobileNet Size and Speed
Li, Y., Ju, H., Wen, Y., Evangelidis, G., Salahi, K., Wang, Y., Tulyakov, S., & Ren, J. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01549 · Cited by 394
Cross-Modal Causal Relational Reasoning for Event-Level Visual Question Answering
Liu, Y., Li, G., & Lin, L. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3284038 · Cited by 373
Rethinking Mobile Block for Efficient Attention-based Models
Zhang, J., Li, X., Li, J., Liu, L., Xue, Z., Zhang, B., Jiang, Z., Huang, T., Wang, Y., & Wang, C. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00134 · Cited by 354
Context Autoencoder for Self-supervised Representation Learning
Chen, X., Ding, M., Wang, X., Xin, Y., Mo, S., Wang, Y., Han, S., Luo, P., Zeng, G., & Wang, J. (2023) · International Journal of Computer Vision · doi:10.1007/s11263-023-01852-4 · Cited by 446
Object Detection Using Deep Learning, CNNs and Vision Transformers: A Review
Amjoud, A. B., & Amrouch, M. (2023) · IEEE Access · doi:10.1109/access.2023.3266093 · Cited by 389
Efficient and Explicit Modelling of Image Hierarchies for Image Restoration
Li, Y., Fan, Y., Xiang, X., Demandolx, D., Ranjan, R., Timofte, R., & Gool, L. V. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01753 · Cited by 342
Remote Sensing Object Detection in the Deep Learning Era—A Review
Gui, S., Song, S., Qin, R., & Tang, Y. (2024) · Remote Sensing · doi:10.3390/rs16020327 · Cited by 367
VOLO: Vision Outlooker for Visual Recognition
Yuan, L., Hou, Q., Jiang, Z., Feng, J., & Yan, S. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3206108 · Cited by 350
Mukhamediev, R. I., Popova, Y., Kuchin, Y. I., Zaitseva, E., Kalimoldayev, A. M., Сымагулов, А., Levashenko, V., Abdoldina, F., Gopejenko, V. I., Yakunin, K., et al. (2022) · Mathematics · doi:10.3390/math10152552 · Cited by 415
Advancing Plain Vision Transformer Toward Remote Sensing Foundation Model
Wang, D., Zhang, Q., Xu, Y., Zhang, J., Du, B., Tao, D., & Zhang, L. (2022) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2022.3222818 · Cited by 369
SwiftFormer: Efficient Additive Attention for Transformer-based Real-time Mobile Vision Applications
Shaker, A., Maaz, M., Rasheed, H., Khan, S., Yang, M., & Khan, F. S. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01598 · Cited by 311
Jia, D., Yuan, Y., He, H., Wu, X., Yu, H., Lin, W., Sun, L., Zhang, C., & Hu, H. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01887 · Cited by 333
Self-citationEVA-02: A visual representation for neon genesis
Fang, Y., Sun, Q., Wang, X., Huang, T., Wang, X., & Cao, Y. (2024) · Image and Vision Computing · doi:10.1016/j.imavis.2024.105171 · Cited by 473
Self-citationViTAEv2: Vision Transformer Advanced by Exploring Inductive Bias for Image Recognition and Beyond
Zhang, Q., Xu, Y., Zhang, J., & Tao, D. (2023) · International Journal of Computer Vision · doi:10.1007/s11263-022-01739-w · Cited by 486
Lightweight Deep Learning for Resource-Constrained Environments: A Survey
Liu, H.-I., Galindo, M., Xie, H., Wong, L.-K., Shuai, H.-H., Li, Y., & Cheng, W.-H. (2024) · ACM Computing Surveys · doi:10.1145/3657282 · Cited by 332
A foundation model for the Earth system
Bodnar, C., Bruinsma, W. P., Lucic, A., Stanley, M., Allen, A., Brandstetter, J., Garvan, P., Riechert, M., Weyn, J. A., Dong, H., et al. (2025) · Nature · doi:10.1038/s41586-025-09005-y · Cited by 267
Group DETR: Fast DETR Training with Group-Wise One-to-Many Assignment
Chen, Q., Chen, X., Wang, J., Zhang, S., Yao, K., Feng, H., Han, J., Ding, E., Zeng, G., & Wang, J. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00610 · Cited by 304
Transformer-Based Visual Segmentation: A Survey
Li, X., Ding, H., Yuan, H., Zhang, W., Pang, J., Cheng, G., Chen, K., Liu, Z., & Loy, C. C. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3434373 · Cited by 305
Self-citationN-Gram in Swin Transformers for Efficient Lightweight Image Super-Resolution
Choi, H., Lee, J., & Yang, J. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00206 · Cited by 248
Zhao, Z., Xu, X. D., Shutao Li, A., & Plaza, A. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3377610 · Cited by 243
Conv2Former: A Simple Transformer-Style ConvNet for Visual Recognition
Hou, Q., Lu, C.-Z., Cheng, M., & Feng, J. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3401450 · Cited by 281
A review on 2D instance segmentation based on deep neural networks
Gu, W., Bai, S., & Kong, L. (2022) · Image and Vision Computing · doi:10.1016/j.imavis.2022.104401 · Cited by 258
Large AI Models in Health Informatics: Applications, Challenges, and the Future
Qiu, J., Li, L., Sun, J., Peng, J., Shi, P., Zhang, R., Dong, Y., Lam, K., Lo, F. P.-W., Xiao, B. J., et al. (2023) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2023.3316750 · Cited by 267
Scale-Aware Modulation Meet Transformer
Lin, W., Wu, Z., Chen, J., Huang, J., & Jin, L. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00553 · Cited by 237
Review of large vision models and visual prompt engineering
Wang, J., Liu, Z., Zhao, L., Wu, Z., Ma, C., Yu, S., Dai, H., Yang, Q., Liu, Y., Zhang, S., et al. (2023) · Meta-Radiology · doi:10.1016/j.metrad.2023.100047 · Cited by 257
Self-citationHyperSIGMA: Hyperspectral Intelligence Comprehension Foundation Model
Wang, D., Hu, M., Jin, Y., Miao, Y., Yang, J., Xu, Y., Qin, X., Ma, J., Sun, L., Li, C., et al. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3557581 · Cited by 289
DKM: Dense Kernelized Feature Matching for Geometry Estimation
Edstedt, J., Athanasiadis, I. N., Wadenbäck, M., & Felsberg, M. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01704 · Cited by 197
Swin2SR: SwinV2 Transformer for Compressed Image Super-Resolution and Restoration
Conde, M. V., Choi, U.-J., Burchi, M., & Timofte, R. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-25063-7_42 · Cited by 186
Unleashing Text-to-Image Diffusion Models for Visual Perception
Zhao, W., Rao, Y., Liu, Z., Liu, B., Zhou, J., & Lu, J. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00527 · Cited by 249
Self-citationSelva, J., Johansen, A. S., Escalera, S., Nasrollahi, K., Moeslund, T. B., & Clapés, A. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3243465 · Cited by 192
CDMamba: Incorporating Local Clues Into Mamba for Remote Sensing Image Binary Change Detection
Zhang, H., Chen, K., Liu, C., Chen, H., Zou, Z., & Shi, Z. (2025) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2025.3545012 · Cited by 230
Kunduracıoğlu, İ., & Paçal, İ. (2024) · Journal of Plant Diseases and Protection · doi:10.1007/s41348-024-00896-z · Cited by 195
Paçal, İ. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.122099 · Cited by 170
A Survey on Efficient Vision Transformers: Algorithms, Techniques, and Performance Benchmarking
Papa, L., Russo, P., Amerini, I., & Zhou, L. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3392941 · Cited by 200
A knowledge-guided pre-training framework for improving molecular representation learning
Li, H., Zhang, R., Min, Y., Ma, D., Zhao, D. Y., & Zeng, J. (2023) · Nature Communications · doi:10.1038/s41467-023-43214-1 · Cited by 173
Hütten, N., Gomes, M. A., Hölken, F., Andricevic, K., Meyes, R., & Meisen, T. (2024) · Applied System Innovation · doi:10.3390/asi7010011 · Cited by 211
A Comprehensive Review of Deep Learning-Based PCB Defect Detection
Chen, X., Wu, Y., He, X., & Ming, W. (2023) · IEEE Access · doi:10.1109/access.2023.3339561 · Cited by 167
Optimal Transport Aggregation for Visual Place Recognition
Izquierdo, S., & Civera, J. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01672 · Cited by 158
Paçal, İ. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.111482 · Cited by 158
Scaling Self-Supervised Learning for Histopathology with Masked Image Modeling
Filiot, A., Ghermi, R., Olivier, A., Jacob, P., Fidon, L., Camara, A., Mac Kain, A., Saillard, C., & Schiratti, J.-B. (2023) · doi:10.1101/2023.07.21.23292757 · Cited by 148
Micron-BERT: BERT-Based Facial Micro-Expression Recognition
Nguyen, X.-B., Duong, C. N., Li, X., Gauch, S., Seo, H., & Luu, K. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00149 · Cited by 158
A novel CNN-ViT-based deep learning model for early skin cancer diagnosis
Paçal, İ., Ozdemir, B., Zeynalov, J., Gasimov, H., & Pacal, N. (2025) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2025.107627 · Cited by 160
Swin3D: A Pretrained Transformer Backbone for 3D Indoor Scene Understanding
Yang, Y., Guo, Y., Xiong, J.-Y., Liu, Y., Pan, H., Wang, P., Tong, X., & Guo, B. (2025) · Computational Visual Media · doi:10.26599/cvm.2025.9450383 · Cited by 233
Self-citationLargeKernel3D: Scaling up Kernels in 3D Sparse CNNs
Chen, Y., Liu, J., Zhang, X., Qi, X., & Jia, J. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01296 · Cited by 166
Retinexmamba: Retinex-Based Mamba for Low-Light Image Enhancement
Bai, J., Yin, Y., He, Q., Li, Y., & Zhang, X. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-6596-9_30 · Cited by 138
Transformer for object detection: Review and benchmark
Li, Y., Miao, N., Ma, L., Shuang, F., & Huang, X. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.107021 · Cited by 135
DeepSolo: Let Transformer Decoder with Explicit Points Solo for Text Spotting
Ye, M., Zhang, J., Zhao, S., Liu, J., Liu, T., Du, B., & Tao, D. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01854 · Cited by 135
Paçal, İ. (2024) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-024-02110-w · Cited by 131
RingMo-SAM: A Foundation Model for Segment Anything in Multimodal Remote-Sensing Images
Yan, Z., Li, J., Li, X., Zhou, R., Zhang, W., Feng, Y., Diao, W., Fu, K., & Sun, X. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3332219 · Cited by 150
Revealing the Dark Secrets of Masked Image Modeling
Xie, Z., Geng, Z., Hu, J., Zhang, Z., Hu, H., & Cao, Y. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01391 · Cited by 151
Self-citationSapiens: Foundation for Human Vision Models
Khirodkar, R., Bagautdinov, T., Martinez, J., Zhaoen, S., James, A., Selednik, P., Anderson, S., & Saito, S. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73235-5_12 · Cited by 213
FlatFormer: Flattened Window Attention for Efficient Point Cloud Transformer
Liu, Z., Yang, X., Tang, H., Shang, Y., & Han, S. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00122 · Cited by 135
Self-citationZhang, L., Li, Y., Zhou, X., Zhao, X., & Gu, S. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00276 · Cited by 125
FACT: FFN-Attention Co-optimized Transformer Architecture with Eager Correlation Prediction
Qin, Y., Wang, Y., Deng, D., Zhao, Z., Yang, X., Liu, L., Wei, S., Hu, Y., & Yin, S. (2023) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1145/3579371.3589057 · Cited by 130
Özdemir, B., & Paçal, İ. (2024) · Results in Engineering · doi:10.1016/j.rineng.2024.103692 · Cited by 127
Diffusion Models for Image Restoration and Enhancement: A Comprehensive Survey
Li, X., Ren, Y., Jin, X., Lan, C., Wang, X., Zeng, W., Wang, X., & Chen, Z. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02570-9 · Cited by 219
VcT: Visual Change Transformer for Remote Sensing Image Change Detection
Jiang, B., Wang, Z., Wang, X., Zhang, Z., Chen, L., Wang, X., & Luo, B. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3327139 · Cited by 126
Self-citationDeep learning for detecting macroplastic litter in water bodies: A review
Jia, T., Kapelan, Z. S., de Vries, R., Vriend, P., Peereboom, E. C., Okkerman, I., & Taormina, R. (2023) · Water Research · doi:10.1016/j.watres.2023.119632 · Cited by 117
A Comprehensive Survey of Transformers for Computer Vision
Jamil, S., Piran, M. J., & Kwon, O. (2023) · Drones · doi:10.3390/drones7050287 · Cited by 128
SeMask: Semantically Masked Transformers for Semantic Segmentation
Jain, J., Singh, A. T., Orlov, N., Huang, Z., Li, J., Walton, S., & Shi, H. (2023) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw60793.2023.00083 · Cited by 111
Vision Transformers are Parameter-Efficient Audio-Visual Learners
Lin, Y.-B., Sung, Y.-L., Lei, J., Bansal, M., & Bertasius, G. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00228 · Cited by 120
Self-citationA Survey of Deep Learning-Based Object Detection Methods and Datasets for Overhead Imagery
Kang, J., Tariq, S., Oh, H., & Woo, S. S. (2022) · IEEE Access · doi:10.1109/access.2022.3149052 · Cited by 121
Kamath, V. P., & Renuka, A. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.02.006 · Cited by 120
Vision-Based UAV Self-Positioning in Low-Altitude Urban Environments
Dai, M., Zheng, E., Feng, Z., Qi, L., Zhuang, J., & Yang, W. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3346279 · Cited by 105
FeatEnHancer: Enhancing Hierarchical Features for Object Detection and Beyond Under Low-Light Vision
Hashmi, K. A., Kallempudi, G., Stricker, D., & Afzal, M. Z. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00619 · Cited by 105
CroCo v2: Improved Cross-view Completion Pre-training for Stereo Matching and Optical Flow
Weinzaepfel, P., Lucas, T., Leroy, V., Cabon, Y., Arora, V., Brégier, R., Csurka, G., Antsfeld, L., Chidlovskii, B., & Revaud, J. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01647 · Cited by 110
Dimitrovski, I., Kitanovski, I., Kocev, D., & Simidjievski, N. (2023) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2023.01.014 · Cited by 109
Fully neuromorphic vision and control for autonomous drone flight
Paredes-Vallés, F., Hagenaars, J. J., Dupeyroux, J., Stroobants, S., Xu, Y., & de Croon, G. (2024) · Science Robotics · doi:10.1126/scirobotics.adi0591 · Cited by 123
Are we ready for a new paradigm shift? A survey on visual deep MLP
Liu, R., Li, Y., Tao, L., Liang, D., & Zheng, H.-T. (2022) · Patterns · doi:10.1016/j.patter.2022.100520 · Cited by 110
Huang, X., Dong, M., Li, J., & Xian, G. (2022) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2022.3202036 · Cited by 101
A Robust Feature Downsampling Module for Remote-Sensing Visual Tasks
Lu, W., Chen, S.-B., Tang, J., Ding, C. H. Q., & Luo, B. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3282048 · Cited by 98
Exploring the potential of channel interactions for image restoration
Cui, Y., & Knoll, A. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.111156 · Cited by 96
Paçal, İ., Alaftekin, M., & Zengul, F. D. (2024) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-024-01140-8 · Cited by 97
Human Pose as Compositional Tokens
Geng, Z., Wang, C., Wei, Y., Liu, Z., Li, H., & Hu, H. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00071 · Cited by 104
Self-citation
Metadata
- Authors
- Liu, Z., Hu, H., Lin, Y., Yao, Z., Xie, Z., Wei, Y., Ning, J., Cao, Y., Zhang, Z., Dong, L., Wei, F., & Guo, B.
- Affiliations
- University of Science and Technology of China (CN)
- Microsoft Research Asia (China) (CN)
- Xi'an Jiaotong University (CN)
- Tsinghua University (CN)
- Huazhong University of Science and Technology (CN)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2021
- Pages
- 11999-12009
- ISSN
- 1063-6919 (ISSN-L)
- arXiv
- 2111.09883
- OpenAlex
- W4312349930
- Cited by
- 2,679
- References
- 70
- Citations by year
- 2026: 483 · 2025: 858 · 2024: 757 · 2023: 485 · 2022: 94 · 2021: 1
- FWCI
- 198.93
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- Microsoft
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Keywords
- Swin Transformer, large vision models, model scaling, high-resolution image classification, object detection, semantic segmentation, video action recognition, position bias, training stability, GPU memory efficiency, self-supervised pre-training
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
