The DOI matches its Crossref record; 40 of the 42 references it cites were found in a registry or an index.
Patch Slimming for Efficient Vision Transformers
Open accessAbstract
This paper studies the efficiency problem for visual transformers by excavating redundant calculation in given networks. The recent transformer architecture has demonstrated its effectiveness for achieving excellent performance on a series of computer vision tasks. However, similar to that of convolutional neural networks, the huge computational cost of vision transformers is still a severe issue. Considering that the attention mechanism aggregates different patches layer-by-layer, we present a novel patch slimming approach that discards useless patches in a topdown paradigm. We first identify the effective patches in the last layer and then use them to guide the patch selection process of previous layers. For each layer, the impact of a patch on the final output feature is approximated and patches with less impacts will be removed. Experimental results on benchmark datasets demonstrate that the proposed method can significantly reduce the computational costs of vision transformers without affecting their performances. For example, over 45% FLOPs of the ViT-Ti model can be reduced with only 0.2% top-1 accuracy drop on the ImageNet dataset.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Nevertheless, as compared to convolutional neural networks (CNNs), ViTs suffer from higher memory footprints, computational overheads, and power consumption, hindering their deployment and real-time inference on resourceconstrained edge devices [25, 17, 15, 37].”
From I-ViT: Integer-only Quantization for Efficient Vision Transformer Inference (2023) · cites this paper as [37]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204
(2017) · Cited by 11,625
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions
Wang, W., Xie, E., Li, X., Fan, D.-P., Song, K., Liang, D., Lu, T., Luo, P., & Shao, L. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00061 · Cited by 5,259 · cited 1×
“DeiT-S [38] 4.6 79.8 DeiT-B [38] 17.5 81.8 PVT-Small [40] 3.8 79.8 PVT-Medium [40] 6.7 81.2 PVT-Large [40] 9.8 81.7 T2T-ViT-14 [43] 5.2 81.5 T2T-ViT-19 [43] 8.9 81.9 T2T-ViT-24 [43] 14.1 82.3 TNT-S [12] 5.2 81.5 TNT-B [12] 14.1 82.9 Swin-T [26] 4.5 81.3 Swin-S [26] 8.7 83.0 Swin-B [26] 15.4 83.5…”
InfluentialHan, S., Mao, H., & Dally, W. J. (2015) · arXiv · doi:10.48550/arxiv.1510.00149 · Cited by 4,662
XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks
Rastegari, M., Ordonez, V., Redmon, J., & Farhadi, A. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_32 · Cited by 4,408
Research works printed but not linked: 6
- touvron, 2020, Training data-efficient image transformers & distillation through at-tention, Arxiv preprint arXiv, 1
- huawei, 2020, Mindspore, 7
- liu, 2018, Frequency-domain dynamic pruning fo convolutional neu-ral networks, Advances in neural information processing systems, 31, 1
- courbariaux, 2016, Binarized neural networks: Training deep neural networks with weights and activations constrained to+ 1 or-l, Arxiv preprint arXiv, 1
- dosovitskiy, 2020, An image is worth 16×16 words: Trans-formers for image recognition at scale, Arxiv preprint arXiv, 1
- zhu, 2021, Vi-sual transformer pruning, Arxiv preprint arXiv, 2, 6
Cited by
Sorted by citations
Han (2023) · doi:10.1109/tpami.2022.3152247 · Cited by 4,682
Self-citationCMT: Convolutional Neural Networks Meet Vision Transformers
Guo, J., Han, K., Wu, H., Tang, Y., Chen, X., Wang, Y., & Xu, C. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01186 · Cited by 1,005
Self-citationTopFormer: Token Pyramid Transformer for Mobile Semantic Segmentation
Zhang, W., Huang, Z., Luo, G., Chen, T., Wang, X., Liu, W., Yu, G., & Shen, C. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01177 · Cited by 374
A Comprehensive Survey of Convolutions in Deep Learning: Applications, Challenges, and Future Trends
Younesi, A., Ansari, M., Fazli, M., Ejlali, A., Shafique, M., & Henkel, J. (2024) · IEEE Access · doi:10.1109/access.2024.3376441 · Cited by 241
Evo-ViT: Slow-Fast Token Evolution for Dynamic Vision Transformer
Xu, Y., Zhang, Z., Zhang, M., Sheng, K., Li, K., Dong, W., Zhang, L., Xu, C., & Sun, X. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i3.20202 · Cited by 226
Self-citationEfficient Visual Tracking with Exemplar Transformers
Blatter, P., Kanakis, M., Danelljan, M., & Gool, L. V. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00162 · Cited by 183
SPViT: Enabling Faster Vision Transformers via Latency-Aware Soft Token Pruning
Kong, Z., Dong, P., Ma, X., Meng, X., Niu, W., Sun, M., Shen, X., Yuan, G., Ren, B., Tang, H., et al. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-20083-0_37 · Cited by 169
Self-citationA Survey on Efficient Vision Transformers: Algorithms, Techniques, and Performance Benchmarking
Papa, L., Russo, P., Amerini, I., & Zhou, L. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3392941 · Cited by 200
An Image Patch is a Wave: Phase-Aware Vision MLP
Tang, Y., Han, K., Guo, J., Xu, C., Li, Y., Xu, C., & Wang, Y. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01066 · Cited by 170
Self-citationA Survey of Techniques for Optimizing Transformer Inference
Chitty-Venkata, K. T., Mittal, S., Emani, M., Vishwanath, V., & Somani, A. K. (2023) · Journal of Systems Architecture · doi:10.1016/j.sysarc.2023.102990 · Cited by 154
Less is More: Focus Attention for Efficient DETR
Zheng, D., Dong, W., Hu, H., Chen, X., & Wang, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00614 · Cited by 162
Self-citationI-ViT: Integer-only Quantization for Efficient Vision Transformer Inference
Li, Z., & Gu, Q. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01565 · Cited by 141
“Nevertheless, as compared to convolutional neural networks (CNNs), ViTs suffer from higher memory footprints, computational overheads, and power consumption, hindering their deployment and real-time inference on resourceconstrained edge devices [16,18,25,35].”
BackgroundHeatViT: Hardware-Efficient Adaptive Token Pruning for Vision Transformers
Dong, P., Sun, M., Lu, A., Xie, Y., Liu, K., Kong, Z., Meng, X., Li, Z., Lin, X., Fang, Z., et al. (2023) · 2023 IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca56546.2023.10071047 · Cited by 128
Self-citationRepQ-ViT: Scale Reparameterization for Post-Training Quantization of Vision Transformers
Li, Z., Xiao, J., Yang, L., & Gu, Q. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01580 · Cited by 127
Chasing Sparsity in Vision Transformers: An End-to-End Exploration
Chen, T., Cheng, Y., Gan, Z., Yuan, L., Zhang, L., & Wang, Z. (2021) · arXiv · doi:10.48550/arxiv.2106.04533 · Cited by 111
FlexiViT: One Model for All Patch Sizes
Beyer, L., Izmailov, P., Kolesnikov, A., Caron, M., Kornblith, S., Zhai, X., Minderer, M., Tschannen, M., Alabdulmohsin, I., & Pavetic, F. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01393 · Cited by 132
Hire-MLP: Vision MLP via Hierarchical Rearrangement
Guo, J., Tang, Y., Han, K., Chen, X., Wu, H., Xu, C., Xu, C., & Wang, Y. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00090 · Cited by 113
Self-citationWidth & Depth Pruning for Vision Transformers
Yu, F., Huang, K., Wang, M., Cheng, Y., Chu, W., & Cui, L. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i3.20222 · Cited by 95
Joint Token Pruning and Squeezing Towards More Aggressive Compression of Vision Transformers
Wei, S., Ye, T., Zhang, S., Tang, Y., & Liang, J. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00208 · Cited by 95
Self-citationNot All Images are Worth 16x16 Words: Dynamic Transformers for Efficient Image RecognitionPreprint
Wang, Y., Huang, R., Song, S., Huang, Z., & Huang, G. M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.15075 · Cited by 110
Self-citationCF-ViT: A General Coarse-to-Fine Method for Vision Transformer
Chen, M., Lin, M., Li, K., Shen, Y., Wu, Y., Chao, F., & Ji, R. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i6.25860 · Cited by 90
Vision Transformer Slimming: Multi-Dimension Searching in Continuous Optimization Space
Chavan, A., Shen, Z., Liu, Z., Liu, Z., Cheng, K.-T., & Xing, E. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00488 · Cited by 72
SparseViT: Revisiting Activation Sparsity for Efficient High-Resolution Vision Transformer
Chen, X., Liu, Z., Tang, H., Yi, L., Zhao, H., & Han, S. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00205 · Cited by 86
Adaptive and Background-Aware Vision Transformer for Real-Time UAV Tracking
Li, S., Yang, Y., Zeng, D., & Wang, X. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01286 · Cited by 83
Dynamic Spatial Sparsification for Efficient Vision Transformers and Convolutional Neural Networks
Rao, Y., Liu, Z., Zhao, W., Zhou, J., & Lu, J. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3263826 · Cited by 75
Similarity-Guided Layer-Adaptive Vision Transformer for UAV Tracking
Xue, C., Zhong, B., Liang, Q., Zheng, Y., Li, N., Xue, Y., & Song, S. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00631 · Cited by 78
Patch Similarity Aware Data-Free Quantization for Vision Transformers
Li, Z., Ma, L., Chen, M., Xiao, J., & Gu, Q. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-20083-0_10 · Cited by 61
Vision-Language Models for Edge Networks: A Comprehensive Survey
Sharshar, A., Khan, L. U., Ullah, W., & Guizani, M. (2025) · IEEE Internet of Things Journal · doi:10.1109/jiot.2025.3579032 · Cited by 92
ViTs as backbones: Leveraging vision transformers for feature extraction
Elharrouss, O., Himeur, Y., Mahmood, Y., Alrabaee, S., Ouamane, A., Bensaali, F., Bechqito, Y., & Chouchane, A. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.102951 · Cited by 69
Saha, S., & Xu, L. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.130417 · Cited by 85
Yao, S., Sun, H., Xiang, T.-Z., Wang, X., & Cao, X. (2024) · IEEE Transactions on Image Processing · doi:10.1109/tip.2024.3475219 · Cited by 54
Attention Map Guided Transformer Pruning for Occluded Person Re-Identification on Edge Device
Mao, J., Yao, Y., Sun, Z., Huang, X., Shen, F., & Shen, H. T. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3265159 · Cited by 53
Which Tokens to Use? Investigating Token Reduction in Vision Transformers
Haurum, J. B., Escalera, S., Taylor, G. W., & Moeslund, T. B. (2023) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw60793.2023.00085 · Cited by 47
Long, S., Zhao, Z. W., Pi, J., Wang, S., & Wang, J. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00996 · Cited by 53
Dynamic Neural Network Structure: A Review for its Theories and Applications
Guo, J., Chen, C. L. P., Liu, Z., & Yang, X. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3377194 · Cited by 48
Self-citationMonoATT: Online Monocular 3D Object Detection with Adaptive Token Transformer
Zhou, Y., Zhu, H., Liu, Q., Chang, S., & Guo, M. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01678 · Cited by 45
Making Vision Transformers Efficient from A Token Sparsification View
Chang, S., Wang, P., Lin, M., Wang, F., Zhang, D. J., Jin, R., & Shou, M. Z. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00600 · Cited by 48
DiffRate : Differentiable Compression Rate for Efficient Vision Transformers
Chen, M., Shao, W., Xu, P., Lin, M., Zhang, K., Chao, F., Ji, R., Qiao, Y., & Luo, P. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01574 · Cited by 58
PSAQ-ViT V2: Toward Accurate and General Data-Free Quantization for Vision Transformers
Li, Z., Chen, M., Xiao, J., & Gu, Q. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3301007 · Cited by 41
Self-slimmed Vision Transformer
Zong, Z., Li, K., Song, G., Wang, Y., Qiao, Y., Leng, B., & Liu, Y. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20083-0_26 · Cited by 41
Self-citationVision transformer models for mobile/edge devices: a survey
Lee, S. I., Koo, K., Lee, J. H., Lee, G., Jeong, S., Seongjun, O., & Kim, H. (2024) · Multimedia Systems · doi:10.1007/s00530-024-01312-0 · Cited by 44
Content-aware Token Sharing for Efficient Semantic Segmentation with Vision Transformers
Lu, C., de Geus, D., & Dubbelman, G. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02263 · Cited by 36
Lin, M., Chen, M., Zhang, Y., Shen, C., Ji, R., & Cao, L. (2023) · International Journal of Computer Vision · doi:10.1007/s11263-023-01861-3 · Cited by 40
LGViT: Dynamic Early Exiting for Accelerating Vision Transformer
Xu, G., Hao, J., Shen, L., Hu, H., Luo, Y., Lin, H., & Shen, J. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3611762 · Cited by 42
Multi-Aspect Convolutional-Transformer Network for SAR Automatic Target Recognition
Li, S., Pan, Z., & Hu, Y. (2022) · Remote Sensing · doi:10.3390/rs14163924 · Cited by 35
PatchDropout: Economizing Vision Transformers Using Patch Dropout
Liu, Y., Matsoukas, C., Strand, F., Azizpour, H., & Smith, K. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00394 · Cited by 40
Boost Vision Transformer with GPU-Friendly Sparsity and Quantization
Yu, C., Chen, T., Gan, Z., & Fan, J. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02170 · Cited by 33
Dynamic Perceiver for Efficient Visual Recognition
Han, Y., Han, D., Liu, Z., Wang, Y., Pan, X., Pu, Y., Deng, C., Feng, J., Song, S., & Huang, G. M. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00551 · Cited by 42
Self-citationSparsifiner: Learning Sparse Instance-Dependent Attention for Efficient Vision Transformers
Wei, C., Duke, B., Jiang, R., Aarabi, P., Taylor, G. W., & Shkurti, F. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02172 · Cited by 32
Guo, R., Chen, X. L., Wang, L., Wang, Y., Sun, H., Wei, J., Han, H., Liu, L., Wei, S., Hu, Y., et al. (2024) · IEEE Journal of Solid-State Circuits · doi:10.1109/jssc.2024.3402174 · Cited by 33
Self-citationTFormer: A Transmission-Friendly ViT Model for IoT Devices
Lu, Z., Ding, C., Juefei-Xu, F., Boddeti, V. N., Wang, S., & Yang, Y. (2023) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2022.3222765 · Cited by 24
Cao, J., Ye, P., Li, S., Yu, C., Tang, Y., Lu, J., & Chen, T. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01487 · Cited by 33
Self-citationA survey of visual neural networks: current trends, challenges and opportunities
Feng, P., & Tang, Z. (2022) · Multimedia Systems · doi:10.1007/s00530-022-01003-8 · Cited by 23
Adaptive Sparse ViT: Towards Learnable Adaptive Token Pruning by Fully Exploiting Self-Attention
Liu, X., Wu, T., & Guo, G. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/136 · Cited by 47
MLP Can Be a Good Transformer Learner
Lin, S., Lyu, P., Liu, D., Tang, T., Liang, X., Song, A., & Chang, X. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01843 · Cited by 25
A Decade of Video Analytics at Edge: Training, Deployment, Orchestration, and Platforms
Bai, T., Zhao, H., Huang, L., Wang, Z., Kim, D. I., & Nallanathan, A. (2025) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2025.3563377 · Cited by 25
Explainability of Vision Transformers: a Comprehensive Review and New PerspectivesPreprint
Kashefi, R., Barekatain, L., Sabokrou, M., & Aghaeipoor, F. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.5055345 · Cited by 26
Multi-Dimensional Vision Transformer Compression via Dependency Guided Gaussian Process Search
Hou, Z., & Kung, S.-Y. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00411 · Cited by 20
Homogeneous tokenizer matters: Homogeneous visual tokenizer for remote sensing image understanding
Shao, R., Zhang, Z., Tao, C., Zhang, Y., Peng, C., & Li, H. (2024) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2024.09.009 · Cited by 22
EvolutionViT: Multi-objective evolutionary vision transformer pruning under resource constraints
Liu, L., Yen, G. G., & He, Z. (2024) · Information Sciences · doi:10.1016/j.ins.2024.121406 · Cited by 19
Efficient Vision Transformer for Human-Centric AIoT Applications Through Token Tracking Assignment
Li, C., Peng, Y., Liu, G., Li, Y., Yang, X., & Chen, C. (2023) · IEEE Transactions on Consumer Electronics · doi:10.1109/tce.2023.3323373 · Cited by 20
UPDP: A Unified Progressive Depth Pruner for CNN and Vision Transformer
Liu, J., Tang, D., Huang, Y., Zhang, L., Zeng, X. C., Li, D., Lu, M., Peng, J., Wang, Y., Jiang, F., et al. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i12.29296 · Cited by 21
Self-citationLu, W. S., Zhang, Z., & Nguyen, M. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3385318 · Cited by 21
Towards Accurate Post-Training Quantization of Vision Transformers via Error Reduction
Zhong, Y., You, H., Hu, J., Zhang, Y., & Ji, R. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3528042 · Cited by 26
Efficient token pruning in Vision Transformers using an attention-based Multilayer Network
Marchetti, M., Traini, D., Ursino, D., & Virgili, L. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.127449 · Cited by 19
An Efficient Window-Based Vision Transformer Accelerator via Mixed-Granularity Sparsity
Dong, Q., Zhang, S., & Wang, Z. (2025) · IEEE Transactions on Circuits and Systems I Regular Papers · doi:10.1109/tcsi.2025.3527541 · Cited by 16
Luo, Z., Yan, H., & Pan, X. (2023) · Journal of Computational Methods in Engineering Applications · doi:10.62836/jcmea.v3i1.030107 · Cited by 36
GhostViT: Expediting Vision Transformers via Cheap Operations
Cao, H., Qu, Z., Chen, G., Li, X., Thiele, L., & Knoll, A. (2023) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2023.3326795 · Cited by 15
FDViT: Improve the Hierarchical Architecture of Vision Transformer
Xu, Y., Li, C., Li, D., Sheng, X., Jiang, F., Tian, L., & Sirasao, A. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00547 · Cited by 17
Mao, J., Shen, Y., Guo, J., Yao, Y., Hua, X., & Shen, H. T. (2025) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2025.3535405 · Cited by 24
Self-citationLightweight and Optimization Acceleration Methods for Vision Transformer: A Review
Chen, M., Jun, G., & Yu, W. (2022) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc55140.2022.9921989 · Cited by 16
Accelerating ViT Inference on FPGA through Static and Dynamic Pruning
Parikh, D., Li, S., Zhang, B., Kannan, R., Busart, C., & Prasanna, V. K. (2024) · IEEE International Symposium on Field-Programmable Custom Computing Machines (FCCM) · doi:10.1109/fccm60383.2024.00018 · Cited by 18
Bi, Y.-D., Abrol, A., Jia, S., Sui, J., & Calhoun, V. (2024) · NeuroImage · doi:10.1016/j.neuroimage.2024.120674 · Cited by 15
Multi-Dimensional Model Compression of Vision Transformer
Hou, Z., & Kung, S. (2022) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme52920.2022.9859786 · Cited by 16
Efficient Transformer-based 3D Object Detection with Dynamic Token Halting
Ye, M., Meyer, G. P., Chai, Y., & Liu, Q. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00775 · Cited by 12
Mask-Guided Vision Transformer for Few-Shot Learning
Chen, Y., Xiao, Z., Pan, Y., Zhao, L., Dai, H., Wu, Z., Li, C., Zhang, T., Li, C., Zhu, D., et al. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3418527 · Cited by 27
MULTIFLOW: Shifting Towards Task-Agnostic Vision-Language Pruning
Farina, M., Mancini, M., Cunegatti, E., Cunegatti, E., Iacca, G., & Ricci, E. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01532 · Cited by 15
DiVIT: Algorithm and architecture co-design of differential attention in vision transformer
Li, Y., Hu, Y., Wu, F., & Li, K. (2022) · Journal of Systems Architecture · doi:10.1016/j.sysarc.2022.102520 · Cited by 11
Multi-Dimension Compression of Feed-Forward Network in Vision Transformers
Bai, S., Chen, J., Yang, Y., & Liu, Y. (2023) · Pattern Recognition Letters · doi:10.1016/j.patrec.2023.10.014 · Cited by 10
Darvishnezhad, M., & Sebt, M. A. (2023) · International Journal of Remote Sensing · doi:10.1080/01431161.2023.2176722 · Cited by 10
Adaptive patch selection to improve Vision Transformers through Reinforcement Learning
Cauteruccio, F., Marchetti, M., Traini, D., Ursino, D., & Virgili, L. (2025) · Applied Intelligence · doi:10.1007/s10489-025-06516-z · Cited by 11
DTSNet: Dynamic Transformer Slimming for Efficient Vision Recognition
Xiao, W., Li, X., Hu, L., Hao, Y., & Chen, M. (2025) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2025.3607796 · Cited by 9
Multi-tailed vision transformer for efficient inference
Wang, Y., Du, B., Wang, W., & Xu, C. (2024) · Neural Networks · doi:10.1016/j.neunet.2024.106235 · Cited by 14
Self-citationZhu, L., Chen, T., Ji, D., Ye, J., & Liu, J. (2025) · IEEE Transactions on Image Processing · doi:10.1109/tip.2025.3531299 · Cited by 8
Efficient feature selection for pre-trained vision transformers
Huang, L., Zeng, J., Yu, M., Ding, W., Bai, X., & Wang, K. (2025) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2025.104326 · Cited by 8
TFRNet: Semantic Segmentation Network with Token Filtration and Refinement Method
Ma, Y., & Hu, X. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2024.3378465 · Cited by 8
Feature fusion method based on local binary graph for PolSAR image classification
Sebt, M. A., & Darvishnezhad, M. (2023) · IET Radar Sonar & Navigation · doi:10.1049/rsn2.12389 · Cited by 8
Adversarial Robust Modulation Recognition Guided by Attention Mechanisms
Zhan, Q., Zhang, X., Sun, M., Song, L., & Zhou, Z. (2025) · IEEE Open Journal of Signal Processing · doi:10.1109/ojsp.2025.3526577 · Cited by 7
Explainability of vision transformers: a comprehensive review and new perspectives
Kashefi, R., Barekatain, L., Sabokrou, M., & Aghaeipoor, F. (2026) · Multimedia Tools and Applications · doi:10.1007/s11042-026-21313-7 · Cited by 10
Make a Long Image Short: Adaptive Token Length for Vision Transformers
Zhou, Q., & Zhu, Y. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-43415-0_5 · Cited by 8
FLORA: Fine-grained Low-Rank Architecture Search for Vision Transformer
Chang, C.-C., Sung, Y.-Y., Yu, S., Huang, N.-C., Marculescu, D., & Wu, K. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00247 · Cited by 7
Wei, Y., Feng, Y., Zu, D., & Zhang, X. (2024) · Crop Protection · doi:10.1016/j.cropro.2024.107018 · Cited by 7
RDA-MTE: an innovative model for emotion recognition in sports behavior decision-making
Zhang, S. (2024) · Frontiers in Neuroscience · doi:10.3389/fnins.2024.1466013 · Cited by 6
Network Expansion For Practical Training Acceleration
Ding, N., Tang, Y., Han, K., Xu, C., & Wang, Y. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01941 · Cited by 7
Self-citationATFTrans: attention-weighted token fusion transformer for robust and efficient object tracking
Xu, L., Wang, L., & Guo, Z. (2024) · Neural Computing and Applications · doi:10.1007/s00521-024-09444-0 · Cited by 6
ViT- ToGo: Vision Transformer Accelerator with Grouped Token Pruning
Lee, S., Cho, K.-M., Kwon, E., Park, S., Kim, S., & Kang, S. (2024) · Design, Automation, and Test in Europe · doi:10.23919/date58400.2024.10546804 · Cited by 6
Human Activity Recognition Based on Feature Fusion of Millimeter Wave Radar and Inertial Navigation
Shi, J., Zhu, Y., He, J., Xu, Z., Chu, L., Braun, R., & Shi, Q. (2025) · IEEE Journal of Microwaves · doi:10.1109/jmw.2025.3539957 · Cited by 8
Data Level Lottery Ticket Hypothesis for Vision Transformers
Shen, X., Kong, Z., Qin, M., Dong, P., Yuan, G., Meng, X., Tang, H., Ma, X., & Wang, Y. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/153 · Cited by 10
Self-citationUnleashing Knowledge Potential of Source Hypothesis for Source-Free Domain Adaptation
Hu, B., Liu, J., Zheng, K., & Zha, Z.-J. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3333190 · Cited by 6
UCC: A unified cascade compression framework for vision transformer models
Chen, D., Lin, K., & Deng, Q. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128747 · Cited by 8
Metadata
- Authors
- Tang, Y., Han, K., Wang, Y., Xu, C., Guo, J., Xu, C., & Tao, D.
- Affiliations
- Peking University (CN)
- Huawei Technologies (China) (CN)
- The University of Sydney (AU)
- Jingdong (China) (CN)
- Venue
- 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2022-06
- Pages
- 12155-12164
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W4312340826
- Cited by
- 201
- References
- 42
- Citations by year
- 2026: 22 · 2025: 52 · 2024: 49 · 2023: 54 · 2022: 19
- FWCI
- 15.78
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Natural Science Foundation of China (61876007)
- Australian Research Council (DP210101859)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- vision transformer, computational efficiency, attention mechanism, patch selection, FLOPs reduction, ViT
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
