The DOI matches its DataCite record; 65 of the 70 references it cites were found in a registry or an index.
Swin Transformer V2: Scaling Up Capacity and Resolution
Open accessAbstract
Large-scale NLP models have been shown to significantly improve the performance on language tasks with no signs of saturation. They also demonstrate amazing few-shot capabilities like that of human beings. This paper aims to explore large-scale models in computer vision. We tackle three major issues in training and application of large vision models, including training instability, resolution gaps between pre-training and fine-tuning, and hunger on labelled data. Three main techniques are proposed: 1) a residual-post-norm method combined with cosine attention to improve training stability; 2) A log-spaced continuous position bias method to effectively transfer models pre-trained using low-resolution images to downstream tasks with high-resolution inputs; 3) A self-supervised pre-training method, SimMIM, to reduce the needs of vast labeled images. Through these techniques, this paper successfully trained a 3 billion-parameter Swin Transformer V2 model, which is the largest dense vision model to date, and makes it capable of training with images of up to 1,536$\times$1,536 resolution. It set new performance records on 4 representative vision tasks, including ImageNet-V2 image classification, COCO object detection, ADE20K semantic segmentation, and Kinetics-400 video action classification. Also note our training is much more efficient than that in Google's billion-level visual models, which consumes 40 times less labelled data and 40 times less training time. Code is available at \url{https://github.com/microsoft/Swin-Transformer}.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“we reduce the model size to 1/15 compared to SwinV2-G [22].”
From DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection (2022) · cites this paper as [22]
The cited source says what the claim attributes to it.
In the citing paper
“Compared with the previous SOTA models, we use a much smaller model size (1 / 15 parameters compared with SwinV2-G [22]), backbone pre-training data size (1 / 60 images compared with Florence), and detection pretraining data size (1 / 5 images compared with Florence), while achieving better results.”
From DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection (2022) · cites this paper as [22]
The cited source says what the claim attributes to it.
In the citing paper
“Without using any other tricks, we achieve the result of 63 . 1 on val2017 and 63 . 2 on test-dev without test time augmentation (TTA) (see Appendix A), outperforming the previous state-of-the-art result 63 . 1 achieved by SwinV2 [22] with a much neater solution.”
From DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection (2022) · cites this paper as [22]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472
Flagged first, then by citations
5 of the 70 references couldn’t be matched to a known paper.
Research works printed but not linked: 15
- Navaneeth Bodla, Bharat Singh, Rama Chellappa, and Larry S. Davis. Soft-nms -improving object detection with one line of code. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), Oct 2017. 9, 10
- Zhaowei Cai and Nuno Vasconcelos. Cascade r-cnn: Delv- ing into high quality object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recogni- tion, pages 6154-6162, 2018. 9
- Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to- end object detection with transformers. In European Confer- ence on Computer Vision, pages 213-229. Springer, 2020. 9
- Kai Chen, Jiangmiao Pang, Jiaqi Wang, Yu Xiong, Xiaox- iao Li, Shuyang Sun, Wansen Feng, Ziwei Liu, Jianping Shi, Wanli Ouyang, et al. Hybrid task cascade for instance segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4974- 4983, 2019. 9, 10
- Kai Chen, Jiaqi Wang, Jiangmiao Pang, Yuhang Cao, Yu Xiong, Xiaoxiao Li, Shuyang Sun, Wansen Feng, Ziwei Liu, Jiarui Xu, et al. Mmdetection: Open mmlab detection tool- box and benchmark. arXiv preprint arXiv:1906.07155, 2019. 9 · arXiv
- Ekin D Cubuk, Barret Zoph, Jonathon Shlens, and Quoc V Le. Randaugment: Practical automated data augmenta- tion with a reduced search space. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pages 702-703, 2020. 9, 10
- Hao-Shu Fang, Jianhua Sun, Runzhong Wang, Minghao Gou, Yong-Lu Li, and Cewu Lu. Instaboost: Boosting instance segmentation via probability map guided copy- pasting. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 682-691, 2019. 9
- Golnaz Ghiasi, Tsung-Yi Lin, and Quoc V Le. Nas-fpn: Learning scalable feature pyramid architecture for object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7036- 7045, 2019. 9
- Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Gir- shick. Mask r-cnn. In Proceedings of the IEEE international conference on computer vision, pages 2961-2969, 2017. 9
- Gao Huang, Yu Sun, Zhuang Liu, Daniel Sedra, and Kil- ian Q Weinberger. Deep networks with stochastic depth. In European conference on computer vision, pages 646-661. Springer, 2016. 9, 10
- Ilya Loshchilov and Frank Hutter. Decoupled weight de- cay regularization. In International Conference on Learning Representations, 2019. 9, 10
- Peize Sun, Rufeng Zhang, Yi Jiang, Tao Kong, Chen- feng Xu, Wei Zhan, Masayoshi Tomizuka, Lei Li, Zehuan Yuan, Changhu Wang, et al. Sparse r-cnn: End-to-end object detection with learnable proposals. arXiv preprint arXiv:2011.12450, 2020. 9 · arXiv
- Sangdoo Yun, Dongyoon Han, Seong Joon Oh, Sanghyuk Chun, Junsuk Choe, and Youngjoon Yoo. Cutmix: Regular- ization strategy to train strong classifiers with localizable fea- tures. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 6023-6032, 2019. 9
- Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimiza- tion. arXiv preprint arXiv:1710.09412, 2017. 9 · arXiv
- Zhun Zhong, Liang Zheng, Guoliang Kang, Shaozi Li, and Yi Yang. Random erasing data augmentation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 13001-13008, 2020. 9, 10
Cited by
Sorted by citations
Items 101–200 of 1,001
Is Attention all You Need in Medical Image Analysis? A Review
Papanastasiou, G., Δικαίος, Ν., Huang, J., Wang, C., & Yang, G. (2023) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2023.3348436 · Cited by 107
Chen, W., Luo, J., Zhang, F., & Tian, Z. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-023-17949-4 · Cited by 103
Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Action
Lu, J., Clark, C., Lee, S.-H., Zhang, Z., Khosla, S., Marten, R., Hoiem, D., & Kembhavi, A. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02497 · Cited by 195
Self-citationHybrid Transformer-CNN for Real Image Denoising
Zhao, M., Cao, G., Huang, X., & Yang, L. (2022) · IEEE Signal Processing Letters · doi:10.1109/lsp.2022.3176486 · Cited by 87
MixMAE: Mixed and Masked Autoencoder for Efficient Pretraining of Hierarchical Vision Transformers
Liu, J., Huang, X., Zheng, J., Liu, Y., & Li, H. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00605 · Cited by 114
MCS-YOLO: A Multiscale Object Detection Method for Autonomous Driving Road Environment Recognition
Cao, Y., Li, C., Peng, Y., & Ru, H. (2023) · IEEE Access · doi:10.1109/access.2023.3252021 · Cited by 92
Self-citationTao, J., Chen, Z., Sun, Z., Guo, H., Leng, B., Yu, Z., Wang, Y., He, Z., Lei, X., & Yang, J. (2023) · Remote Sensing · doi:10.3390/rs15061602 · Cited by 85
Crowdsensing-based Road Damage Detection Challenge (CRDDC’2022)
Arya, D., Maeda, H., Ghosh, S. K., Toshniwal, D., Omata, H., Kashiyama, T., & Sekimoto, Y. (2022) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata55660.2022.10021040 · Cited by 86
Li, M., Chen, Y., Zhang, T., & Huang, W. X. (2024) · Complex & Intelligent Systems · doi:10.1007/s40747-024-01448-6 · Cited by 92
Vision Transformer With Quadrangle Attention
Zhang, Q., Zhang, J., Xu, Y., & Tao, D. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3347693 · Cited by 106
MoE-LLaVA: Mixture of Experts for Large Vision-Language Models
Lin, B., Tang, Z., Ye, Y., Huang, J., Zhang, J., Pang, Y., Jin, P., Ning, M., Luo, J., & Yuan, L. (2026) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2026.3654458 · Cited by 81
Transformer Fusion and Pixel-Level Contrastive Learning for RGB-D Salient Object Detection
Wu, J., Hao, F., Liang, W., & Xu, J. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3275308 · Cited by 80
NDDepth: Normal-Distance Assisted Monocular Depth Estimation
Shao, S., Pei, Z., Chen, W., Wu, X., & Li, Z. G. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00729 · Cited by 81
SoftFormer: SAR-optical fusion transformer for urban land use and land cover classification
Liu, R., Ling, J., & Zhang, H. (2024) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2024.09.012 · Cited by 80
SCLIP: Rethinking Self-Attention for Dense Vision-Language Inference
Wang, F., Mei, J., & Yuille, A. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72664-4_18 · Cited by 140
5%>100%: Breaking Performance Shackles of Full Fine-Tuning on Visual Recognition Tasks
Yin, D., Hu, L., Li, B., Zhang, Y., & Yang, X. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01869 · Cited by 104
YOLO-Former: Marrying YOLO and Transformer for Foreign Object Detection
Dai, Y., Liu, W., Wang, H., Xie, W., & Long, K. (2022) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2022.3219468 · Cited by 81
Advances in the Neural Network Quantization: A Comprehensive Review
Lu, W., Ma, Z., Yang, C., & Yao, Q. (2024) · Applied Sciences · doi:10.3390/app14177445 · Cited by 105
Li, H., Zhao, D. Y., & Zeng, J. (2022) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3534678.3539426 · Cited by 93
Hard Patches Mining for Masked Image Modeling
Wang, H., Song, K., Fan, J., Wang, Y., Xie, J., & Zhang, Z. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01000 · Cited by 83
Self-citationOmniMAE: Single Model Masked Pretraining on Images and Videos
Girdhar, R., El-Nouby, A., Singh, M., Alwala, K. V., Joulin, A., & Misra, I. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01003 · Cited by 101
2D and 3D object detection algorithms from images: A Survey
Chen, W., Li, Y., Tian, Z., & Zhang, F. (2023) · Array · doi:10.1016/j.array.2023.100305 · Cited by 81
Open Vocabulary Semantic Segmentation with Patch Aligned Contrastive Learning
Mukhoti, J., Lin, T., Poursaeed, O., Wang, R., Shah, A., Torr, P. H. S., & Lim, S.-N. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01860 · Cited by 90
SwinURNet: Hybrid Transformer-CNN Architecture for Real-Time Unstructured Road Segmentation
Wang, Z., Liao, Z., Zhou, B., Yu, G., & Luo, W. (2024) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2024.3470042 · Cited by 77
XoFTR: Cross-modal Feature Matching Transformer
Tuzcuoğlu, Ö., Köksal, A., Sofu, B., Kalkan, S., & Alatan, A. A. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00431 · Cited by 76
HiT-SR: Hierarchical Transformer for Efficient Image Super-Resolution
Zhang, X., Zhang, Y., & Yu, F. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73661-2_27 · Cited by 78
Comprehensive review of recent developments in visual object detection based on deep learning
Edozie, E., Shuaibu, A. N., John, U. K., & Sadiq, B. O. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11284-w · Cited by 100
Light Self-Gaussian-Attention Vision Transformer for Hyperspectral Image Classification
Ma, C., Wan, M., Wu, J. H., Kong, X., Shao, A., Wang, F., Chen, Q., & Gu, G. (2023) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2023.3279922 · Cited by 79
VSA: Learning Varied-Size Window Attention in Vision Transformers
Zhang, Q., Xu, Y., Zhang, J., & Tao, D. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19806-9_27 · Cited by 77
Generating synthetic computed tomography for radiotherapy: SynthRAD2023 challenge report
Huijben, E. M. C., Terpstra, M. L., Galapon, A., Pai, S., Thummerer, A., Koopmans, P. J., Afonso, M. V., van Eijnatten, M. A. J. M., Gurney‐Champion, O. J., Chen, Z., et al. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103276 · Cited by 93
Self-citationSDPL: Shifting-Dense Partition Learning for UAV-View Geo-Localization
Chen, Q., Wang, T., Yang, Z., Li, H., Lu, R., Sun, Y., Zheng, B., & Yan, C. C. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3424196 · Cited by 85
Quantitative regularization in robust vision transformer for remote sensing image classification
Song, H., Yuan, Y., Ouyang, Z., Yang, Y., & Xiang, H. (2024) · The Photogrammetric Record · doi:10.1111/phor.12489 · Cited by 70
Conv-Adapter: Exploring Parameter Efficient Transfer Learning for ConvNets
Chen, H., Tao, R., Zhang, H., Wang, Y., Li, X., Ye, W., Wang, J., Hu, G., & Savvides, M. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00162 · Cited by 105
SparseViT: Revisiting Activation Sparsity for Efficient High-Resolution Vision Transformer
Chen, X., Liu, Z., Tang, H., Yi, L., Zhao, H., & Han, S. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00205 · Cited by 86
Self-citationEfficient Hierarchical Entropy Model for Learned Point Cloud Compression
Song, R. Y., Fu, C., Liu, S., & Li, G. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01381 · Cited by 70
Gao, X., Xiao, Z., & Deng, Z. (2023) · Journal of Food Engineering · doi:10.1016/j.jfoodeng.2023.111833 · Cited by 73
SeD: Semantic-Aware Discriminator for Image Super-Resolution
Li, B., Li, X., Zhu, H., Jin, Y., Feng, R., Zhang, Z., & Chen, Z. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02436 · Cited by 81
Self-citationE 2 VPT: An Effective and Efficient Approach for Visual Prompt Tuning
Han, C., Wang, Q., Cui, Y., Cao, Z., Wang, W., Qi, S., & Liu, D. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01604 · Cited by 123
Ding, W., Abdel‐Basset, M., Alrashdi, I., & Hawash, H. (2024) · Information Sciences · doi:10.1016/j.ins.2024.120338 · Cited by 73
Patro, B. N., & Agneeswaran, V. S. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.111279 · Cited by 81
Transformer Meets Remote Sensing Video Detection and Tracking: A Comprehensive Survey
Jiao, L., Zhang, X., Liu, X., Liu, F., Yang, S., Ma, W., Li, L., Chen, P., Feng, Z., Guo, Y., et al. (2023) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2023.3289293 · Cited by 68
Raghu, D., Bucher, M. J. J., & Wolf, C. D. (2023) · Resources Conservation and Recycling · doi:10.1016/j.resconrec.2023.107140 · Cited by 74
Multimodal diagnosis model of Alzheimer’s disease based on improved Transformer
Tang, Y. M., Xiong, X., Tong, G., Yang, Y., & Zhang, H. (2024) · BioMedical Engineering OnLine · doi:10.1186/s12938-024-01204-4 · Cited by 64
Domain Invariant Vision Transformer Learning for Face Anti-spoofing
Liao, C.-H., Chen, W., Liu, H.-T., Yeh, Y.-R., Hu, M., & Chen, C. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00604 · Cited by 69
LRT: An Efficient Low-Light Restoration Transformer for Dark Light Field Images
Zhang, S., Meng, N., & Lam, E. Y. M. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3297412 · Cited by 63
Towards Robust Tampered Text Detection in Document Image: New Dataset and New Solution
Qu, C., Liu, C., Liu, Y., Chen, X., Peng, D., Guo, F., & Jin, L. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00575 · Cited by 74
Fan, J., Gao, B., Ge, Q., Ran, Y., Zhang, J., & Chu, H. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3313982 · Cited by 64
Early detection of dementia through retinal imaging and trustworthy AI
Hao, J., Kwapong, W. R., Shen, T., Fu, H., Xu, Y., Lu, Q., Liu, S., Zhang, J., Liu, H., Zhao, Y., et al. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01292-5 · Cited by 70
On Data Scaling in Masked Image Modeling
Xie, Z., Zhang, Z., Cao, Y., Lin, Y., Wei, Y., Dai, Q., & Hu, H. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00999 · Cited by 70
Self-citationIntegrating vision and laser point cloud data for shield tunnel digital twin modeling
Li, Y., Xiao, Z., Li, J., & Shen, T. (2023) · Automation in Construction · doi:10.1016/j.autcon.2023.105180 · Cited by 61
GET: Group Event Transformer for Event-Based Vision
Peng, Y., Zhang, Y., Xiong, Z., Sun, X., & Wu, F. H. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00555 · Cited by 77
ECoDepth: Effective Conditioning of Diffusion Models for Monocular Depth Estimation
Patni, S., Agarwal, A., & Arora, C. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02672 · Cited by 81
Yang, H., Liu, Z., Liu, W. Y., Wang, H., Zhang, Y., & Wang, H. (2026) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2026.3680498 · Cited by 61
Self-citationMei, L., Hu, X., Ye, Z., Tang, L., Wang, Y., Li, D., Liu, Y., Hao, X., Lei, C., Xu, C., et al. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.111658 · Cited by 60
Rip Current Segmentation: A Novel Benchmark and YOLOv8 Baseline Results
Dumitriu, A., Tătui, F., Miron, F., Ionescu, R. T., & Timofte, R. (2023) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00133 · Cited by 59
Picking point recognition for ripe tomatoes using semantic segmentation and morphological processing
Rong, Q., Hu, C., Hu, X., & Xu, M. (2023) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2023.107923 · Cited by 58
MultiSenseSeg: A Cost-Effective Unified Multimodal Semantic Segmentation Model for Remote Sensing
Wang, Q., Chen, W., Huang, Z., Tang, H., & Yang, L. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3390750 · Cited by 58
Multi-metrics adaptively identifies backdoors in Federated learning
Huang, S., Li, Y., Chen, C., Shi, L., & Gao, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00429 · Cited by 63
AM-RADIO: Agglomerative Vision Foundation Model Reduce All Domains Into One
Ranzinger, M., Heinrich, G., Kautz, J., & Molchanov, P. A. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01187 · Cited by 93
Deep learning-based lung image registration: A review
Xiao, H., Xue, X., Zhu, M., Jiang, X., Xia, Q., Chen, K., Li, H., Long, L., & Peng, K. (2023) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2023.107434 · Cited by 57
Image Processing GNN: Breaking Rigidity in Super-Resolution
Tian, Y., Chen, H., Xu, C., & Wang, Y. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02276 · Cited by 56
Scalable, Detailed and Mask-Free Universal Photometric Stereo
Ikehata, S. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01268 · Cited by 55
The effectiveness of MAE pre-pretraining for billion-scale pretraining
Singh, M., Duval, Q., Alwala, K. V., Fan, H., Aggarwal, V., Adcock, A., Joulin, A., Dollár, P., Feichtenhofer, C., Girshick, R., et al. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00505 · Cited by 70
Yang, Y., Hossain, M. Z., Stone, E. A., & Rahman, S. (2023) · Pattern Recognition · doi:10.1016/j.patcog.2023.109966 · Cited by 57
The Ninth NTIRE 2024 Efficient Super-Resolution Challenge Report
Ren, B., Li, Y., Mehta, N., Timofte, R., Yu, H., Wan, C., Hong, Y., Han, B., Wu, Z., Zou, Y., et al. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00656 · Cited by 65
Self-citationFrom Chaos Comes Order: Ordering Event Representations for Object Recognition and Detection
Zubić, N., Gehrig, D., Gehrig, M., & Scaramuzza, D. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01180 · Cited by 68
Manigrasso, F., Milazzo, R., Russo, A. S., Lamberti, F., Strand, F., Pagnani, A., & Morra, L. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103320 · Cited by 68
Gao, Y., Zhang, J., Wei, S., & Li, Z. (2024) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2024.107154 · Cited by 55
Detection Transformer with Stable Matching
Liu, S., Ren, T., Chen, J., Zeng, Z., Zhang, H., Li, F., Li, H., Huang, J. G., Su, H., Zhu, J., et al. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00597 · Cited by 75
AG-ReID.v2: Bridging Aerial and Ground Views for Person Re-Identification
Nguyen, H., Nguyen, K., Sridharan, S., & Fookes, C. (2024) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2024.3353078 · Cited by 58
Towards All-in-One Pre-Training via Maximizing Multi-Modal Mutual Information
Su, W., Zhu, X., Tao, C., Lu, L., Li, B., Huang, G. M., Qiao, Y., Wang, X., Zhou, J., & Dai, J. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01525 · Cited by 58
State Space Models for Event Cameras
Zubić, N., Gehrig, M., & Scaramuzza, D. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00556 · Cited by 50
Masked Autoencoders for Microscopy are Scalable Learners of Cellular Biology
Kraus, O. Z., Kenyon-Dean, K., Saberian, S., Fallah, M., McLean, P., Leung, J., Sharma, V., Khan, A., Balakrishnan, J., Çelik, S., et al. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01117 · Cited by 61
Vision Transformer in Industrial Visual Inspection
Hütten, N., Meyes, R., & Meisen, T. (2022) · Applied Sciences · doi:10.3390/app122311981 · Cited by 55
TinyMIM: An Empirical Study of Distilling MIM Pre-trained Models
Ren, S., Wei, F., Zhang, Z., & Hu, H. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00359 · Cited by 54
Self-citationMulti-grained clip focus for skeleton-based action recognition
Qiu, H., & Hou, B. (2023) · Pattern Recognition · doi:10.1016/j.patcog.2023.110188 · Cited by 53
GDALR: Global Dual Attention and Local Representations in transformer for surface defect detection
Zhou, X., Zhou, S., Zhang, Y., Ren, Z., Jiang, Z., & Luo, H. (2024) · Measurement · doi:10.1016/j.measurement.2024.114398 · Cited by 50
Data-Driven Weather Forecasting and Climate Modeling from the Perspective of Development
Wu, Y., & Xue, W. (2024) · Atmosphere · doi:10.3390/atmos15060689 · Cited by 51
Vision-Based Learning for Drones: A Survey
Xiao, J., Zhang, R., Zhang, Y., & Feroskhan, M. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2025.3564184 · Cited by 65
An Enhanced Detection Method of PCB Defect Based on Improved YOLOv7
Yang, Y., & Kang, H. (2023) · Electronics · doi:10.3390/electronics12092120 · Cited by 48
Towards Long-Term Time-Series Forecasting: Feature, Pattern, and Distribution
Li, Y., Lu, X., Xiong, H., Tang, J. X., Su, J., Jin, B., & Dou, D. (2023) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde55515.2023.00127 · Cited by 55
Cross-modal Orthogonal High-rank Augmentation for RGB-Event Transformer-trackers
Zhu, Z., Hou, J., & Wu, D. O. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.02015 · Cited by 52
Yang, Z., Tian, T., Kong, J., & Chen, H. (2025) · Analytical Chemistry · doi:10.1021/acs.analchem.4c06677 · Cited by 46
Refined Crack Detection via LECSFormer for Autonomous Road Inspection Vehicles
Chen, J., Zhao, N. J., Zhang, R., Chen, L. F., Huang, K., & Qiu, Z. (2022) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2022.3204583 · Cited by 46
All in Tokens: Unifying Output Space of Visual Tasks via Soft Token
Ning, J., Li, C., Zhang, Z., Wang, C., Geng, Z., Dai, Q., He, K., & Hu, H. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01822 · Cited by 65
Self-citationMedical image segmentation network based on multi-scale frequency domain filter
Chen, Y.-F., Zhang, X.-Q., Peng, L., He, Y., Sun, F., & Sun, H. (2024) · Neural Networks · doi:10.1016/j.neunet.2024.106280 · Cited by 46
Wang, B., Chen, Q., Wang, M., Chen, Y., Zhang, Z., Liu, X., Gao, W. D., Zhang, Y., & Zhang, H. (2024) · Applied Energy · doi:10.1016/j.apenergy.2024.124187 · Cited by 47
Self-citationA survey of video-based human action recognition in team sports
Yin, H., Sinnott, R., & Jayaputera, G. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10934-9 · Cited by 63
A Framework and Operational Procedures for Metaverses-Based Industrial Foundation Models
Wang, J., Tian, Y., Wang, Y., Yang, J., Wang, X., Wang, S., & Kwan, O. (2022) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2022.3226755 · Cited by 48
Wang, J., Dai, H., Chen, T., Liu, H., Zhang, X., Zhong, Q., & Lu, R. (2023) · Scientific Reports · doi:10.1038/s41598-023-33804-w · Cited by 46
Nwoye, C. I., Yu, T., Sharma, S., Murali, A., Alapatt, D., Vardazaryan, A., Yuan, K., Hajek, J., Reiter, W., Yamlahi, A., et al. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.102888 · Cited by 45
Video-FocalNets: Spatio-Temporal Focal Modulation for Video Action Recognition
Wasim, S. T., Khattak, M. U., Naseer, M., Khan, S. H., Shah, M. A., & Khan, F. S. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01267 · Cited by 47
Diverse Cotraining Makes Strong Semi-Supervised Segmentor
Li, Y., Wang, X., Yang, L., Feng, L., Zhang, W., & Gao, Y. L. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01471 · Cited by 59
Wang, H., Li, C., & Li, Y. (2024) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2024.3441638 · Cited by 51
Yang, H., Liu, Z., Ma, N., Wang, X., Liu, W., Wang, H., Zhan, D., & Hu, Z. (2025) · IEEE Transactions on Transportation Electrification · doi:10.1109/tte.2025.3562604 · Cited by 46
Self-citationLi, Y., Wu, Y., Cheng, G., Tao, C., Dang, B., Wang, Y., Zhang, J., Zhang, C., Liu, Y., Tang, X., et al. (2025) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2025.125324 · Cited by 47
Ding, J., Li, W., Pei, L., Yang, M., Tian, A., & Yuan, B. (2024) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2024.3373370 · Cited by 46
EATFormer: Improving Vision Transformer Inspired by Evolutionary Algorithm
Zhang, J., Li, X., Wang, Y., Wang, C., Yang, Y., Liu, Y., & Tao, D. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-024-02034-6 · Cited by 57
PAtt-Lite: Lightweight Patch and Attention MobileNet for Challenging Facial Expression Recognition
Ngwe, J. L., Lim, K. M., Lee, C. P., Ong, T. S., & Alqahtani, A. (2024) · IEEE Access · doi:10.1109/access.2024.3407108 · Cited by 45
IFIFusion: A independent feature information fusion model for surface defect detection
Zhou, X., Zhang, Y., Liu, Z., Jiang, Z., Ren, Z., Mi, T., & Zhou, S. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103039 · Cited by 54
Self-citation
Metadata
- Authors
- Liu, Z., Hu, H., Lin, Y., Yao, Z., Xie, Z., Wei, Y., Ning, J., Cao, Y., Zhang, Z., Dong, L., Wei, F., & Guo, B.
- Affiliations
- University of Science and Technology of China (CN)
- Microsoft Research Asia (China) (CN)
- Xi'an Jiaotong University (CN)
- Tsinghua University (CN)
- Huazhong University of Science and Technology (CN)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2021
- Pages
- 11999-12009
- ISSN
- 1063-6919 (ISSN-L)
- arXiv
- 2111.09883
- OpenAlex
- W4312349930
- Cited by
- 2,679
- References
- 70
- Citations by year
- 2026: 483 · 2025: 858 · 2024: 757 · 2023: 485 · 2022: 94 · 2021: 1
- FWCI
- 198.93
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- Microsoft
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Keywords
- Swin Transformer, large vision models, model scaling, high-resolution image classification, object detection, semantic segmentation, video action recognition, position bias, training stability, GPU memory efficiency, self-supervised pre-training
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
