The DOI matches its DataCite record; 65 of the 70 references it cites were found in a registry or an index.
Swin Transformer V2: Scaling Up Capacity and Resolution
Open accessAbstract
Large-scale NLP models have been shown to significantly improve the performance on language tasks with no signs of saturation. They also demonstrate amazing few-shot capabilities like that of human beings. This paper aims to explore large-scale models in computer vision. We tackle three major issues in training and application of large vision models, including training instability, resolution gaps between pre-training and fine-tuning, and hunger on labelled data. Three main techniques are proposed: 1) a residual-post-norm method combined with cosine attention to improve training stability; 2) A log-spaced continuous position bias method to effectively transfer models pre-trained using low-resolution images to downstream tasks with high-resolution inputs; 3) A self-supervised pre-training method, SimMIM, to reduce the needs of vast labeled images. Through these techniques, this paper successfully trained a 3 billion-parameter Swin Transformer V2 model, which is the largest dense vision model to date, and makes it capable of training with images of up to 1,536$\times$1,536 resolution. It set new performance records on 4 representative vision tasks, including ImageNet-V2 image classification, COCO object detection, ADE20K semantic segmentation, and Kinetics-400 video action classification. Also note our training is much more efficient than that in Google's billion-level visual models, which consumes 40 times less labelled data and 40 times less training time. Code is available at \url{https://github.com/microsoft/Swin-Transformer}.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“we reduce the model size to 1/15 compared to SwinV2-G [22].”
From DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection (2022) · cites this paper as [22]
The cited source says what the claim attributes to it.
In the citing paper
“Compared with the previous SOTA models, we use a much smaller model size (1 / 15 parameters compared with SwinV2-G [22]), backbone pre-training data size (1 / 60 images compared with Florence), and detection pretraining data size (1 / 5 images compared with Florence), while achieving better results.”
From DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection (2022) · cites this paper as [22]
The cited source says what the claim attributes to it.
In the citing paper
“Without using any other tricks, we achieve the result of 63 . 1 on val2017 and 63 . 2 on test-dev without test time augmentation (TTA) (see Appendix A), outperforming the previous state-of-the-art result 63 . 1 achieved by SwinV2 [22] with a much neater solution.”
From DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection (2022) · cites this paper as [22]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472
Research works printed but not linked: 15
- Navaneeth Bodla, Bharat Singh, Rama Chellappa, and Larry S. Davis. Soft-nms -improving object detection with one line of code. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), Oct 2017. 9, 10
- Zhaowei Cai and Nuno Vasconcelos. Cascade r-cnn: Delv- ing into high quality object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recogni- tion, pages 6154-6162, 2018. 9
- Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to- end object detection with transformers. In European Confer- ence on Computer Vision, pages 213-229. Springer, 2020. 9
- Kai Chen, Jiangmiao Pang, Jiaqi Wang, Yu Xiong, Xiaox- iao Li, Shuyang Sun, Wansen Feng, Ziwei Liu, Jianping Shi, Wanli Ouyang, et al. Hybrid task cascade for instance segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4974- 4983, 2019. 9, 10
- Kai Chen, Jiaqi Wang, Jiangmiao Pang, Yuhang Cao, Yu Xiong, Xiaoxiao Li, Shuyang Sun, Wansen Feng, Ziwei Liu, Jiarui Xu, et al. Mmdetection: Open mmlab detection tool- box and benchmark. arXiv preprint arXiv:1906.07155, 2019. 9 · arXiv
- Ekin D Cubuk, Barret Zoph, Jonathon Shlens, and Quoc V Le. Randaugment: Practical automated data augmenta- tion with a reduced search space. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pages 702-703, 2020. 9, 10
- Hao-Shu Fang, Jianhua Sun, Runzhong Wang, Minghao Gou, Yong-Lu Li, and Cewu Lu. Instaboost: Boosting instance segmentation via probability map guided copy- pasting. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 682-691, 2019. 9
- Golnaz Ghiasi, Tsung-Yi Lin, and Quoc V Le. Nas-fpn: Learning scalable feature pyramid architecture for object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7036- 7045, 2019. 9
- Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Gir- shick. Mask r-cnn. In Proceedings of the IEEE international conference on computer vision, pages 2961-2969, 2017. 9
- Gao Huang, Yu Sun, Zhuang Liu, Daniel Sedra, and Kil- ian Q Weinberger. Deep networks with stochastic depth. In European conference on computer vision, pages 646-661. Springer, 2016. 9, 10
- Ilya Loshchilov and Frank Hutter. Decoupled weight de- cay regularization. In International Conference on Learning Representations, 2019. 9, 10
- Peize Sun, Rufeng Zhang, Yi Jiang, Tao Kong, Chen- feng Xu, Wei Zhan, Masayoshi Tomizuka, Lei Li, Zehuan Yuan, Changhu Wang, et al. Sparse r-cnn: End-to-end object detection with learnable proposals. arXiv preprint arXiv:2011.12450, 2020. 9 · arXiv
- Sangdoo Yun, Dongyoon Han, Seong Joon Oh, Sanghyuk Chun, Junsuk Choe, and Youngjoon Yoo. Cutmix: Regular- ization strategy to train strong classifiers with localizable fea- tures. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 6023-6032, 2019. 9
- Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimiza- tion. arXiv preprint arXiv:1710.09412, 2017. 9 · arXiv
- Zhun Zhong, Liang Zheng, Guoliang Kang, Shaozi Li, and Yi Yang. Random erasing data augmentation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 13001-13008, 2020. 9, 10
Cited by
Sorted by citations
Items 201–300 of 1,001
Lü, M., Christensen, C. N., Weber, J. M., Konno, T., Läubli, N. F., Scherer, K. M., Avezov, E., Lió, P., Lapkin, A. A., Schierle, G. S. K., et al. (2023) · Nature Methods · doi:10.1038/s41592-023-01815-0 · Cited by 44
An improved cascade RCNN detection method for key components and defects of transmission lines
Dong, C., Zhang, K., Xie, Z., & Shi, C. (2023) · IET Generation Transmission & Distribution · doi:10.1049/gtd2.12948 · Cited by 44
Self-citationBrain-Inspired Remote Sensing Foundation Models and Open Problems: A Comprehensive Survey
Jiao, L., Huang, Z., Lu, X., Liu, X., Yang, Y., Zhao, J., Zhang, J., Hou, B., Yang, S., Liu, F., et al. (2023) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2023.3316302 · Cited by 49
BSSNet: A Real-Time Semantic Segmentation Network for Road Scenes Inspired From AutoEncoder
Shi, X., Yin, Z., Han, G., Liu, W., Qin, L., Bi, Y., & Li, S. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3325360 · Cited by 45
CSPNeXt: A new efficient token hybrid backbone
Chen, X., Yang, C., Mo, J., Sun, Y., Karmouni, H., Jiang, Y., & Zheng, Z. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.107886 · Cited by 46
Akkaya, İ. B., Kathiresan, S. S., Arani, E., & Zonooz, B. (2024) · Pattern Recognition · doi:10.1016/j.patcog.2024.110510 · Cited by 50
Park, J., Kim, J., & Kim, C. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00157 · Cited by 43
Progressive Spatio-temporal Perception for Audio-Visual Question Answering
Li, G., Hou, W., & Hu, D. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612293 · Cited by 43
Scene Adaptive Sparse Transformer for Event-based Object Detection
Peng, Y., Li, H., Zhang, Y., Sun, X., & Wu, F. H. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01589 · Cited by 58
Deep learning approaches to building rooftop thermal bridge detection from aerial images
Mayer, Z., Kahn, J., Hou, Y., Götz, M., Volk, R., & Schultmann, F. (2022) · Automation in Construction · doi:10.1016/j.autcon.2022.104690 · Cited by 44
Data-centric annotation analysis for plant disease detection: Strategy, consistency, and performance
Dong, J., Lee, J., Fuentes, A., Xu, M., Yoon, S., Lee, M. H., & Park, D. S. (2022) · Frontiers in Plant Science · doi:10.3389/fpls.2022.1037655 · Cited by 48
Convolutional Networks and Transformers for Mammography Classification: An Experimental Study
Cantone, M., Marrocco, C., Tortorella, F., & Bria, A. (2023) · Sensors · doi:10.3390/s23031229 · Cited by 46
Li, H., Yang, Q., & Li, T. (2024) · Energy · doi:10.1016/j.energy.2024.130403 · Cited by 42
SeGCN: A Semantic-Aware Graph Convolutional Network for UAV Geo-Localization
Liu, X., Wang, Z., Wu, Y., & Miao, Q. (2024) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2024.3370612 · Cited by 42
NTIRE 2024 Challenge on Short-form UGC Video Quality Assessment: Methods and Results
Li, X., Yuan, K., Pei, Y., Lu, Y., Sun, M. B., Zhou, C., Chen, Z., Timofte, R., Sun, W., Wu, H., et al. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00643 · Cited by 41
Self-citationEvaluation of five global AI models for predicting weather in Eastern Asia and Western Pacific
Liu, C.-C., Hsu, K., Peng, M. S., Chen, D.-S., Chang, P., Hsiao, L., Fong, C., Hong, J., Cheng, C.-P., Lu, K.-C., et al. (2024) · npj Climate and Atmospheric Science · doi:10.1038/s41612-024-00769-0 · Cited by 56
A Survey on Diffusion Models for Time Series and Spatio-Temporal Data
Yang, Y., Jin, M., Wen, H., Zhang, C., Liang, Y., Ma, L., Wang, Y., Liu, C., Yang, B., Xu, Z., et al. (2025) · ACM Computing Surveys · doi:10.1145/3783986 · Cited by 166
Refashioning Emotion Recognition Modeling: The Advent of Generalized Large Models
Zhang, Z., Peng, L., Pang, T., Han, J., Zhao, H., & Schuller, B. W. (2024) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2024.3396345 · Cited by 52
Self-citationDistilling efficient Vision Transformers from CNNs for semantic segmentation
Xu, Z., Luo, Y., Zhou, P., & Wang, L. (2024) · Pattern Recognition · doi:10.1016/j.patcog.2024.111029 · Cited by 41
Reciprocal Attention Mixing Transformer for Lightweight Image Restoration
Choi, H., Na, C., Oh, J., Lee, S., Kim, J., Choe, S., Lee, J., Kim, T., & Yang, J. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00606 · Cited by 41
Apple Leaf Disease Identification in Complex Background Based on BAM-Net
Gao, Y., Cao, Z., Cai, W., Gong, G., Zhou, G., & Li, L. (2023) · Agronomy · doi:10.3390/agronomy13051240 · Cited by 46
QueryTrack: Joint-Modality Query Fusion Network for RGBT Tracking
Fan, H., Yu, Z., Wang, Q., Fan, B., & Tang, Y. (2024) · IEEE Transactions on Image Processing · doi:10.1109/tip.2024.3393298 · Cited by 43
Text-Image Alignment for Diffusion-Based Perception
Kondapaneni, N., Marks, M., Knott, M., Guimarães, R., & Perona, P. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01317 · Cited by 55
A review of visual perception technology for intelligent fruit harvesting robots
Huang, Y., Xu, S., Chen, H., Li, G., Dong, H., Yu, J., Zhang, X., & Chen, R. (2025) · Frontiers in Plant Science · doi:10.3389/fpls.2025.1646871 · Cited by 47
Li, J., Xu, Q., He, X., Liu, Z., Zhang, D., Wang, R., Qu, R., & Qiu, G. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.128835 · Cited by 60
Self-citationExploring Vision-Language Models for Imbalanced Learning
Wang, Y., Yu, Z., Wang, J., Heng, Q., Chen, H., Ye, W., Xie, R., Xie, X., & Zhang, S. (2023) · International Journal of Computer Vision · doi:10.1007/s11263-023-01868-w · Cited by 40
COŞKUN, D., KARABOĞA, D., BAŞTÜRK, A., AKAY, B., NALBANTOĞLU, Ö. U., DOĞAN, S., PAÇAL, İ., & KARAGÖZ, M. A. (2023) · Turkish Journal of Electrical Engineering and Computer Sciences · doi:10.55730/1300-0632.4048 · Cited by 38
Controllable Guide-Space for Generalizable Face Forgery Detection
Guo, Y., Zhen, C., & Yan, P. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01903 · Cited by 38
Zhu, J., Yang, G., Feng, X., Li, X., Fang, H., Zhang, J., Bai, X., Tao, M., & He, Y. (2022) · Remote Sensing · doi:10.3390/rs14205141 · Cited by 38
Enhanced Bi-directional Motion Estimation for Video Frame Interpolation
Jin, X., Wu, L., Shen, G., Chen, Y., Chen, J., Koo, J., & Hahm, C.-H. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00502 · Cited by 38
Jiang, L., Li, M., & Wang, X. (2023) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2023.3262586 · Cited by 38
SwinFSR: Stereo Image Super-Resolution using SwinIR and Frequency Domain Knowledge
Chen, K., Li, L., Liu, H., Li, Y., Tang, C., & Chen, J. (2023) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00177 · Cited by 37
An Effective and Lightweight Hybrid Network for Object Detection in Remote Sensing Images
Yang, X., Zhang, S., Duan, S., & Yang, W. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3339624 · Cited by 37
EGformer: Equirectangular Geometry-biased Transformer for 360 Depth Estimation
Yun, I., Shin, C., Lee, H., Lee, H.-J., & Rhee, C.-E. (2023) · IEEE International Conference on Computer Vision · doi:10.1109/iccv51070.2023.00561 · Cited by 69
Bridging Remote Sensors with Multisensor Geospatial Foundation Models
Han, B., Zhang, S., Shi, X., & Reichstein, M. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02631 · Cited by 53
SF-IQA: Quality and Similarity Integration for AI Generated Image Quality Assessment
Yu, Z., Guan, F., Lu, Y., Li, X., & Chen, Z. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00663 · Cited by 44
FuXi-Extreme: Improving extreme rainfall and wind forecasts with diffusion model
Zhong, X., Chen, L., Liu, J., Lin, C., Qi, Y., & Li, H. (2024) · Science China Earth Sciences · doi:10.1007/s11430-023-1427-x · Cited by 37
UC-NfNet: Deep learning-enabled assessment of ulcerative colitis from colonoscopy images
Turan, M., & Durmuş, F. (2022) · Medical Image Analysis · doi:10.1016/j.media.2022.102587 · Cited by 38
Hybrid dilated multilayer faster RCNN for object detection
Xin, F., Zhang, H., & Pan, H. (2023) · The Visual Computer · doi:10.1007/s00371-023-02789-y · Cited by 39
Content-aware Token Sharing for Efficient Semantic Segmentation with Vision Transformers
Lu, C., de Geus, D., & Dubbelman, G. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02263 · Cited by 36
Wang, A., Lang, H., Chen, Z., Peng, Y., Ding, S., & Lu, J. (2024) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3340340 · Cited by 36
Dual-former: Hybrid self-attention transformer for efficient image restoration
Chen, S., Tian, Y., Liu, Y., & Chen, E. (2024) · Digital Signal Processing · doi:10.1016/j.dsp.2024.104485 · Cited by 36
Remote Sensing Change Detection With Bitemporal and Differential Feature Interactive Perception
Chang, H., Wang, P., Diao, W., Xu, G., & Sun, X. (2024) · IEEE Transactions on Image Processing · doi:10.1109/tip.2024.3424335 · Cited by 37
NTIRE 2024 Quality Assessment of AI-Generated Content Challenge
Liu, X., Min, X., Zhai, G., Li, C., Kou, T., Sun, W., Wu, H., Gao, Y., Cao, Y., Zhang, Z., et al. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00637 · Cited by 43
Self-citationDevelopment of Deep-Learning-Based Single-Molecule Localization Image Analysis
Hyun, Y., & Kim, D. (2022) · International Journal of Molecular Sciences · doi:10.3390/ijms23136896 · Cited by 36
Object Detection and X-Ray Security Imaging: A Survey
Wu, J., Xu, X., & Yang, J. (2023) · IEEE Access · doi:10.1109/access.2023.3273736 · Cited by 40
Zhu, J., Luo, Y., Xu, Z., Wang, H., & Wang, L. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01076 · Cited by 35
Guan, R., Yao, S., Zhu, X., Man, K. L., Lim, E. G., Smith, J. S., Yue, Y., & Yue, Y. (2023) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc57777.2023.10422325 · Cited by 35
Fine grained food image recognition based on swin transformer
Xiao, Z., Diao, G., & Deng, Z. (2024) · Journal of Food Engineering · doi:10.1016/j.jfoodeng.2024.112134 · Cited by 35
AMU-Tuning: Effective Logit Bias for CLIP-based Few-shot Learning
Tang, Y., Lin, Z., Wang, Q., Zhu, P., & Hu, Q. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02201 · Cited by 36
Target-Distractor Aware UAV Tracking via Global Agent
Xue, Y., Jin, G., Shen, T., Tan, L., Wang, N., Gao, J., Yu, Y., & Tian, S. (2025) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2025.3581391 · Cited by 35
Change3D: Revisiting Change Detection and Captioning from A Video Modeling Perspective
Zhu, D., Huang, X., Huang, H., Zhou, H., & Shao, Z. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02236 · Cited by 37
Li, N., Pan, W., Xu, B., Liu, H., Dai, S., & Xu, C. (2025) · Multimedia Systems · doi:10.1007/s00530-025-01994-0 · Cited by 35
Park, J., Lee, J., Moon, S., & Lee, K. (2022) · Applied Sciences · doi:10.3390/app12031595 · Cited by 35
Robustifying Token Attention for Vision Transformers
Guo, Y. X., Stutz, D., & Schiele, B. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01610 · Cited by 37
Qureshi, W. S., Power, D., Ullah, I., Mulry, B., Feighan, K., McKeever, S., & O’Sullivan, D. M. (2023) · Automation in Construction · doi:10.1016/j.autcon.2023.104945 · Cited by 34
ResFormer: Scaling ViTs with Multi-Resolution Training
Tian, R., Wu, Z., Dai, Q., Hu, H., Qiao, Y., & Jiang, Y. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02176 · Cited by 42
Self-citationAn Auto Chip Package Surface Defect Detection Based on Deep Learning
Cao, Y., Ni, Y., Zhou, Y., Li, H., Huang, Z., & Yao, E. (2023) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2023.3347799 · Cited by 39
Self-citationOn the Diversity and Realism of Distilled Dataset: An Efficient Dataset Distillation Paradigm
Sun, P., Shi, B., Yu, D., & Lin, T. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00897 · Cited by 56
SOFA: A Compute-Memory Optimized Sparsity Accelerator via Cross-Stage Coordinated Tiling
Wang, H., Fang, J., Tang, X., Yue, Z., Li, J., Qin, Y., Guan, S., Yang, Q., Wang, Y., Li, C., et al. (2024) · IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1109/micro61859.2024.00093 · Cited by 38
Zhang, W., Belcheva, V., & Ermakova, T. (2025) · Computers · doi:10.3390/computers14050187 · Cited by 41
Li, R., Ji, P., Xu, Y., & Bhanu, B. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3207105 · Cited by 35
PLG-ViT: Vision Transformer with Parallel Local and Global Self-Attention
Ebert, N., Stricker, D., & Wasenmüller, O. (2023) · Sensors · doi:10.3390/s23073447 · Cited by 37
Composed Image Retrieval via Cross Relation Network With Hierarchical Aggregation Transformer
Yang, Q., Ye, M., Cai, Z., Su, K., & Du, B. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3299791 · Cited by 34
Wang, J., Gao, Y., Wang, F., Zeng, S., Li, J., Miao, H., Wang, T., Zeng, J., Baptista‐Hon, D. T., Monteiro, O., et al. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2308812120 · Cited by 38
Elsharkawy, M., Sharafeldeen, A., Khalifa, F., Soliman, A., Elnakib, A. A., Ghazal, M. A., Sewelam, A., Thanos, A., Sandhu, H. S., & El-Baz, A. S. (2024) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2024.3355329 · Cited by 33
Helpless infants are learning a foundation model
Cusack, R., Ranzato, M., & Charvet, C. J. (2024) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2024.05.001 · Cited by 37
NDDepth: Normal-Distance Assisted Monocular Depth Estimation and Completion
Shao, S., Pei, Z., Chen, W., Chen, P. C. Y., & Li, Z. G. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3411571 · Cited by 33
AlMohimeed, A., Shehata, M., El-Rashidy, N. M., Mostafa, S., Talaat, A. S., & Saleh, H. (2024) · Bioengineering · doi:10.3390/bioengineering11070729 · Cited by 33
MonoOcc: Digging into Monocular Semantic Occupancy Prediction
Zheng, Y., Li, X., Li, P., Zheng, Y., Jin, B., Zhong, C., Long, X., Zhao, H., & Zhang, Q. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10611261 · Cited by 53
Mixture of experts (MoE): A big data perspective
Gan, W., Ning, Z., Qi, Z., & Yu, P. S. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103664 · Cited by 52
Ogawa, Y., Zhao, C., Oki, T., Chen, S., & Sekimoto, Y. (2023) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2023.3237509 · Cited by 36
Boost Vision Transformer with GPU-Friendly Sparsity and Quantization
Yu, C., Chen, T., Gan, Z., & Fan, J. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02170 · Cited by 33
Swinv2-Imagen: hierarchical vision transformer diffusion models for text-to-image generation
Li, R., Li, W., Yang, Y., Wei, H., Jiang, J., & Bai, Q. (2023) · Neural Computing and Applications · doi:10.1007/s00521-023-09021-x · Cited by 37
DEU-Net: Dual-Encoder U-Net for Automated Skin Lesion Segmentation
Karimi, A., Karim Faez, A., & Nazari, S. (2023) · IEEE Access · doi:10.1109/access.2023.3337528 · Cited by 35
Yang, Y., Huang, W., Wei, Y., Peng, H., Jiang, X., Jiang, H., Wei, F., Yin, W., Han, H., Qiu, L., et al. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00260 · Cited by 32
Self-citationCalibFormer: A Transformer-based Automatic LiDAR-Camera Calibration Network
Xiao, Y., Li, Y., Meng, C., Li, X., Ji, J., & Zhang, Y. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10610018 · Cited by 32
Liu, H., Zhan, Y., Sun, J., Mao, Q., & Wu, T. (2025) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2025.109920 · Cited by 32
Dynamic Perceiver for Efficient Visual Recognition
Han, Y., Han, D., Liu, Z., Wang, Y., Pan, X., Pu, Y., Deng, C., Feng, J., Song, S., & Huang, G. M. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00551 · Cited by 42
Self-citationZhang, H. K., Luo, D., & Li, Z. B. (2024) · Science of Remote Sensing · doi:10.1016/j.srs.2024.100123 · Cited by 32
Fully automated diagnosis of thyroid nodule ultrasound using brain-inspired inference
Li, G., Huang, Q., Liu, C., Wang, G., Guo, L., Liu, R., & Liu, L. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.127497 · Cited by 32
AIM 2022 Challenge on Super-Resolution of Compressed Image and Video: Dataset, Methods and Results
Yang, R., Timofte, R., Li, X., Zhang, Q., Zhang, L., Liu, F., He, D., Li, F., Zheng, H., Yuan, W., et al. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-25066-8_8 · Cited by 34
Chai, X., Shao, F., Jiang, Q., & Ying, H. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.110625 · Cited by 31
Zu, B., Cao, T., Li, Y., Li, J., Ju, F., & Wang, H. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.108041 · Cited by 35
Multi-branch CNN and grouping cascade attention for medical image classification
Liu, S., Yue, W., Guo, Z., & Wang, L. (2024) · Scientific Reports · doi:10.1038/s41598-024-64982-w · Cited by 31
A Large Dataset to Enhance Skin Cancer Classification With Transformer-Based Deep Neural Networks
Gallazzi, M., Biavaschi, S., Bulgheroni, A., Gatti, T. M., Corchs, S., & Gallo, I. (2024) · IEEE Access · doi:10.1109/access.2024.3439365 · Cited by 33
Multi-scale adaptive YOLO for instance segmentation of grape pedicels
Shen, Q., Zhang, X., Shen, M., & Xu, D. (2024) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2024.109712 · Cited by 32
Liu, Y., Yao, D., Ma, Y., Wang, H., Wang, J., Bai, X., Zeng, G., & Liu, Y. (2024) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2024.107352 · Cited by 31
Zhang, Z., Shu, D., Liao, C., Liu, C., Zhao, Y., Wang, R., Huang, X., Zhang, M., & Gong, J. (2025) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2025.05.028 · Cited by 37
Self-citationTSE DeepLab: An efficient visual transformer for medical image segmentation
Yang, J., Tu, J., Zhang, X., Yu, S., & Xianyou, Z. (2022) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2022.104376 · Cited by 31
Zhang, W., Zhu, Q., Li, Y., & Li, H. (2023) · Digital Signal Processing · doi:10.1016/j.dsp.2023.104072 · Cited by 34
NTIRE 2023 Challenge on HR Depth from Images of Specular and Transparent Surfaces
Ramirez, P. Z., Tosi, F., Stefano, L. D., Timofte, R., Costanzino, A., Poggi, M., Salti, S., Mattoccia, S., Shi, J., Zhang, D., et al. (2023) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00143 · Cited by 29
Self-citationLi, Z., Han, Y., & Yang, X. (2023) · Journal of Imaging · doi:10.3390/jimaging9100203 · Cited by 34
Ashraf, S. M. N., Mamun, M. A., Abdullah, H., & Alam, M. G. R. (2023) · International Conference on Computer and Information Technology (ICCIT) · doi:10.1109/iccit60459.2023.10441433 · Cited by 35
PAMSNet: A medical image segmentation network based on spatial pyramid and attention mechanism
Feng, Y., Zhu, X., Zhang, X., Li, Y., & Lu, H. (2024) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2024.106285 · Cited by 31
WBNet: Weakly-supervised salient object detection via scribble and pseudo-background priors
Wang, Y., Wang, R., He, X., Lin, C., Wang, T., Jia, Q., & Fan, X. (2024) · Pattern Recognition · doi:10.1016/j.patcog.2024.110579 · Cited by 30
Dussert, G., Chamaillé‐Jammes, S., Dray, S., & Miele, V. (2024) · Remote Sensing in Ecology and Conservation · doi:10.1002/rse2.412 · Cited by 31
A Comprehensive Survey of Recent Transformers in Image, Video and Diffusion Models
Le, D. P. C., Wang, D., & Le, V.-T. (2024) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2024.050790 · Cited by 36
P2AT: Pyramid pooling axial transformer for real-time semantic segmentation
Elhassan, M. A. M., Zhou, C., Benabid, A., & Adam, A. B. M. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.124610 · Cited by 31
Deep learning-based IDH1 gene mutation prediction using histopathological imaging and clinical data
Nakagaki, R., Debsarkar, S. S., Kawanaka, H., Aronow, B. J., & Prasath, V. B. S. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.108902 · Cited by 34
Metadata
- Authors
- Liu, Z., Hu, H., Lin, Y., Yao, Z., Xie, Z., Wei, Y., Ning, J., Cao, Y., Zhang, Z., Dong, L., Wei, F., & Guo, B.
- Affiliations
- University of Science and Technology of China (CN)
- Microsoft Research Asia (China) (CN)
- Xi'an Jiaotong University (CN)
- Tsinghua University (CN)
- Huazhong University of Science and Technology (CN)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2021
- Pages
- 11999-12009
- ISSN
- 1063-6919 (ISSN-L)
- arXiv
- 2111.09883
- OpenAlex
- W4312349930
- Cited by
- 2,679
- References
- 70
- Citations by year
- 2026: 483 · 2025: 858 · 2024: 757 · 2023: 485 · 2022: 94 · 2021: 1
- FWCI
- 198.93
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- Microsoft
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Keywords
- Swin Transformer, large vision models, model scaling, high-resolution image classification, object detection, semantic segmentation, video action recognition, position bias, training stability, GPU memory efficiency, self-supervised pre-training
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
