The DOI matches its Crossref record; 55 of the 59 references it cites were found in a registry or an index.
AdaViT: Adaptive Vision Transformers for Efficient Image Recognition
Abstract
Built on top of self-attention mechanisms, vision transformers have demonstrated remarkable performance on a variety of tasks recently. While achieving excellent performance, they still require relatively intensive computational cost that scales up drastically as the numbers of patches, self-attention heads and transformer blocks increase. In this paper, we argue that due to the large variations among images, their need for modeling long-range dependencies between patches differ. To this end, we introduce AdaViT, an adaptive computation framework that learns to derive usage policies on which patches, self-attention heads and transformer blocks to use throughout the backbone on a per-input basis, aiming to improve inference efficiency of vision transformers with a minimal drop of accuracy for image recognition. Optimized jointly with a transformer backbone in an end-to-end manner, a light-weight decision network is attached to the backbone to produce decisions on-the-fly. Extensive experiments on ImageNet demonstrate that our method obtains more than 2 × improvement on efficiency compared to state-of-the-art vision transformers with only 0.8% drop of accuracy, achieving good efficiency/accuracy trade-offs conditioned on different computational budgets. We further conduct quantitative and qualitative analysis on learned usage polices and provide more insights on the redundancy in vision transformers. Code is available at ht tps: / / gi thub. com/MengLcool/AdaVi T.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Yet, token pruning has several disadvantages: the information loss from pruning limits how many tokens you can reasonably reduce;”
From Token Merging: Your ViT But Faster (2022) · cites this paper as Meng et al., 2022
In the citing paper
“current methods require re-training the model to be effective (some with extra parameters);”
From Token Merging: Your ViT But Faster (2022) · cites this paper as Meng et al., 2022
In the citing paper
“and several prune different numbers of tokens depending on the input content, making batched inference infeasible.”
From Token Merging: Your ViT But Faster (2022) · cites this paper as Meng et al., 2022
In the citing paper
“most cannot be applied to speed up training;”
From Token Merging: Your ViT But Faster (2022) · cites this paper as Meng et al., 2022
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534
MobileNetV2: Inverted Residuals and Linear Bottlenecks
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L.-C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00474 · Cited by 28,755
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., & Adam, H. (2017) · arXiv · doi:10.48550/arxiv.1704.04861 · Cited by 19,015
Self-citationEnd-to-End Object Detection with Transformers
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58452-8_13 · Cited by 18,285
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
Tan, M., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1905.11946 · Cited by 17,072
Decoupled Weight Decay Regularization
Loshchilov, I., & Hutter, F. (2017) · arXiv · doi:10.48550/arxiv.1711.05101 · Cited by 12,522
Flagged first, then by citations
4 of the 59 references couldn’t be matched to a known paper.
Research works printed but not linked: 5
- pan, 2021, Ia-red2: Interpretability-aware redundancy reduction for vision transformers, NeurIPS
- li, 2021, Localvit: Bringing locality to vision transformers, ArXiv Preprint
- he, 2021, Transreid: Transformer-based object reidentification, ICCV
- dosovitskiy, 2021, An image is worth 16x16 words: Transformers for image recognition at scale, ICLRE
- wang, 2021, Not all images are worth 16x16 words: Dynamic vision transformers with adaptive sequence length, NeurIPS
Cited by
Sorted by citations
Items 201–263 of 263
U-Net- and YOLOv8-based pipeline for license plate recognition in adverse weather
Padmakumar, P., Lisho, E., Esther Moriah, A., & Johanan Joysingh, S. (2025) · Signal Image and Video Processing · doi:10.1007/s11760-025-04943-6 · Cited by 1
Naskar, A., Singh, S., & Sarkar, R. (2025) · Evolving Systems · doi:10.1007/s12530-025-09785-8 · Cited by 1
Where Do Tokens Go? Understanding Pruning Behaviors in STEP at High Resolutions
Szczepanski, M., Poreba, M., & Haroun, K. (2026) · SN Computer Science · doi:10.1007/s42979-025-04707-6 · Cited by 1
Konatham, N. L., Thati, J., & Kadha, V. (2025) · International Conference on Artificial Intelligence and Signal Processing (AISP) · doi:10.1109/aisp68263.2025.11396144 · Cited by 1
Rawat, U., & Rai, C. S. (2026) · Signal Image and Video Processing · doi:10.1007/s11760-026-05216-6 · Cited by 1
Foundation models for autonomous driving: A comprehensive survey
Fourati, S., Jaafar, W., Baccar, N., Alfattani, S., & Langar, R. (2026) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2026.114805 · Cited by 1
Explainable Deep Learning Techniques for Colon Cancer Diagnosis Using Histopathological Images
Purushotham, N., & Avanija, J. (2026) · Contemporary Computing Innovations Conference (CCIC) · doi:10.1109/ccic68129.2026.11486104 · Cited by 1
Adaptive Mixture-of-Recursions vision transformer with joint depth routing for multimedia vision
Muthusami, R., & Saritha, K. (2026) · Multimedia Tools and Applications · doi:10.1007/s11042-026-21831-4 · Cited by 1
Token Merging with Class Importance Score
Seol, K., Roh, S.-D., & Chung, K. (2023) · Annual Conference of the IEEE Industrial Electronics Society (IECON) · doi:10.1109/iecon51785.2023.10312420 · Cited by 2
Focus on the Core: Efficient Attention via Pruned Token Compression for Document Classification
Yun, J., Kim, M., & Kim, Y. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.909 · Cited by 4
Efficient Posenet with Coarse to Fine Transformer
Li, S., Zhang, H., Ma, H., Feng, J., & Jiang, M. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10448008 · Cited by 1
Ishay, A., Yang, Z., Lee, J., Kang, I., & Lim, D. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00656 · Cited by 1
Zhou, X., & Zhang, P. (2024) · Sensors · doi:10.3390/s24113502 · Cited by 1
Dynamic attention guider network
Yue, C., Li, J., Wang, Q., & Zhang, D. (2024) · Computing · doi:10.1007/s00607-024-01328-4 · Cited by 1
Pedestrian Re-Identification Based on Fine-Grained Feature Learning and Fusion
Chen, A., & Liu, W. (2024) · Sensors · doi:10.3390/s24237536 · Cited by 1
Li, Y., Zhang, X., Li, M., Xu, G., Chen, F., Yuan, Y., Zou, Y., Zhao, M., Lu, J., & Yu, D. (2025) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2025.3612427 · Cited by 3
Pedestrian Re-Identification Based on Fine-Grained Features Learning and FusionPreprint
Chen, A., Lin, Q., & Liu, W. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4879811 · Cited by 1
iLLaVA: An Image is Worth Fewer Than 1/3 Input Tokensin Large Multimodal ModelsPreprint
Hu, L., Shang, F., Wan, L., & Feng, W. (2024) · Qeios · doi:10.32388/69kflk · Cited by 1
(2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00004
YOLOv5-LC: A Lightweight Classification Deep Learning Detector for Dining Plate
Jiang, H., Guan, Z., Zeng, C., Zeng, L., Wang, T., & Wang, H. E. (2023) · IEEE International Conference on Electronic Technology, Communication and Information (ICETCI) · doi:10.1109/icetci57876.2023.10176439
Neighbor Patches Merging Reduces Spatial Redundancy of Nature ImagesPreprint
Jiang, K., Peng, P., Lian, Y., Shao, W., & xu, W. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4663091
CT-Mixer: Exploiting Multiscale Design for Local-Global Representations Learning
Tang, Y., Wan, X., Guan, X., & Zhu, A. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-0834-5_15
Методи розпізнавання та обробки зображень за допомогою зорового трансформера
Nedashkivskyi, B. (2024) · COMPUTER-INTEGRATED TECHNOLOGIES EDUCATION SCIENCE PRODUCTION · doi:10.36910/6775-2524-0560-2024-54-17
Three Dimension Dynamic Attention Meets Self-Attention
Yue, C. (2024) · International Seminar on Artificial Intelligence, Networking and Information Technology (AINIT) · doi:10.1109/ainit61980.2024.10581419
Green Rural Modern Architectural Design Based on Pose Recognition Algorithm of Thermal Discomfort
Gao, X. (2024) · Information Technology And Control · doi:10.5755/j01.itc.53.4.37391
Adaptive Token Pruning for Vision Transformer
Yang, Y., Hu, X., & Zhou, Y. (2024) · International Symposium on Computer Engineering and Intelligent Communications (ISCEIC) · doi:10.1109/isceic63613.2024.10810243
Patch Ranking: Token Pruning as Ranking Prediction for Efficient CLIP
Wu, C.-E. R., Lin, J., Hu, Y. H., & Morgado, P. M. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00570 · Cited by 1
Enhancing Microscopic Fungi Classification through Advanced Pretraining in Deep Learning Models
Rahman, A., Gain, M., Rahman, M. S., Saha, S. K., Adhikary, A., Bairagi, A. K., & Raha, A. D. (2025) · International Conference on Electrical, Computer and Communication Engineering · doi:10.1109/ecce64574.2025.11013811
A Lightweight Multi-modal Dynamic Fusion Network Based Method for Person Re-identification in Videos
Cai, X., He, S., Zhang, Y., Tang, J., & Sun, H. (2025) · Communications in computer and information science · doi:10.1007/978-981-96-9946-9_40
Li, W., Ren, A., Lan, Q., Fang, H., Wang, Z., Tan, Y., Zhong, K., & Liu, D. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-99854-6_8
Midframe-centric token merging for efficient video transformer
Zhang, Q., Yang, Z., Shao, M., & Liang, H. (2025) · Cluster Computing · doi:10.1007/s10586-025-05478-8
Prompt-Guided Attention Head Selection for Focus-Oriented Image Retrieval
Nozawa, Y., Lin, Y.-C., Nakamura, K., & Ng, Y. (2025) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw67362.2025.00397
Window Token Concatenation for Efficient Visual Large Language Models
Li, Y., Bao, W., Ye, B., Tan, Z., Chen, T., Liu, H., & Kong, Y. (2025) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw67362.2025.00301 · Cited by 1
Video, How do Your Tokens Merge?
Pollard, S., & Wray, M. (2025) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw67362.2025.00317
GCVPN: A Graph Convolutional Visual Prior-Transform Network for Actual Occluded Image Recognition
Wang, L., Wu, N., Wu, H., Yu, W., Zhang, F., & Chen, S. (2025) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3746252.3761518
Task-Oriented Token Pruning for Efficient Object Detection and Segmentation
Liang, H. Z., Kan, M., Shan, S., & Chen, X. (2025) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros60139.2025.11246298
Diao, Y., Wang, W., Dang, Z., Qi, R., Zhao, W., & Zhao, H. (2026) · Communications in computer and information science · doi:10.1007/978-981-95-4966-5_4
MADTP++: Bridge the Gap Between Token and Weight Pruning for Accelerating VLTs
Cao, J., Yu, C., Ye, H., & Chen, T. (2026) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3650545 · Cited by 2
Multi‐Head Convolution Module With Dynamic Feature Fusion for Medical Image Segmentation
Chen, Z., Qin, J., Cai, Z., & Huang, Z. (2026) · IET Image Processing · doi:10.1049/ipr2.70274
Abnormal Behavior Detection in Subway Stations Based on Multimodal Large Language Models
Song, K., Huang, B., & Tu, L. (2025) · IEEE Intl Conf on Dependable, Autonomic and Secure Computing, Intl Conf on Pervasive Intelligence and Computing, Intl Conf on Cloud and Big Data Computing, Intl Conf on Cyber Science and Technology Congress (DASC/PiCom/CBDCom/CyberSciTech) · doi:10.1109/cyberscitech68397.2025.00074
Velvizhi, R., & Ankayarkanni, B. (2026) · Signal Image and Video Processing · doi:10.1007/s11760-026-05177-w
Core Tokensets for Data-Efficient Sequential Training of Transformers
Paul, S., Brack, M., Schramowski, P., Kersting, K., & Mundt, M. (2025) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw69036.2025.00545
FocusViT: dynamic patch focus for transformer-based gaze estimation
Sochirca, D., Chew, J. Y., & Zhang, X. (2026) · Advanced Robotics · doi:10.1080/01691864.2026.2642636
Token Reduction in Vision Transformers via Discrete Wavelet Decomposition
Buratti, C., Marchetti, M., Parlapiano, F., Traini, D., Ursino, D., & Virgili, L. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-31654-7_39
Pedestrian-Centric Discriminative and Fine-grained Semantic Mining for Text-based Person Retrieval
Liang, Y., Chen, H., Liu, Y., Lyu, Y., & Wang, X. (2026) · International World Wide Web Conference (WWW) · doi:10.1145/3774904.3792134
WIDENet: A Novel Lightweight CNN for Robust Skin Lesion Segmentation
Ali, G., Awang, M. K., Rashid, J., Hamza, M., Khashan, O. A., & Ghani, A. (2026) · IEEE Access · doi:10.1109/access.2026.3683571 · Cited by 1
Dachawar, M., Iyer, L., Moolya, R., Deshpande, S., & Chandrasekaran, V. (2026) · Information systems engineering and management · doi:10.1007/978-3-032-20735-7_24
SWAN: World-Aware Adaptive Multimodal Networks for Runtime Variations
Wu, J., Jinn, S.-K. S., Yuan, Y., Wigness, M., Kaplan, L., Qiu, H., & Srivastava, M. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37020-4_38
MetaFusion: Instance-Aware Meta-Prompting and Token Refinement for VLMs Generalization
Liu, M., Wu, F., Lu, Z., Wang, Z., Yu, Y., Han, J., & Zhang, Z. M. (2026) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2026.3686093
Chen, Y., Wang, J., Zhang, Z., Yi, J., Zhang, X., Zou, Y., Cai, Z., Yuan, J., Li, X., Yang, H., et al. (2026) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61042.2026.00413
Wang, Z., Ren, A., Liu, D., Li, W., Fang, H., Song, Y., Tan, Y., Zhong, K., & Qin, C. (2026) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2026.3693876
Liu, Y., Wang, T., Zhang, X., Wang, L., Zhao, Y., Ding, C., & Song, Q. (2026) · Journal of King Saud University - Computer and Information Sciences · doi:10.1007/s44443-026-00607-8
Balasubramaniyam, P., Maheswari, S., Sathish, L., Vijayakumar, R., Selvan, P. S. I., Gowtham, S., Arulanantham, R. J. L., & Gowthaman, N. (2026) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2026.110474
Accelerating Vision Foundation Models with Drop-in Depthwise Convolution
Scribano, C., Mahdi, M., Prisadnikov, N., Fu, Y., Franchini, G., Paudel, D. P., Bertogna, M., & Gool, L. V. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-31927-2_31
Kakde, P. M., & Gulhane, S. M. (2026) · African Journal Of Applied Research · doi:10.26437/nks05170
Driver Risk Behavior Detection Based on Improved Vision Transformer
Xu, W., Jin, T., Yang, X., & Ma, H. (2026) · Lecture notes in electrical engineering · doi:10.1007/978-981-92-2250-6_51
Diversity-guided MLP pruning for efficient large vision transformers
Shen, C., Zhu, H., Fang, G., Wang, J., & Wang, X. (2026) · Pattern Recognition · doi:10.1016/j.patcog.2026.114860
Enhancing Medical Image Classification with Noise-Injected Multi-Head Attention
Van, N. N. H., Hao, P., Hoang, V. N., Dinh, T. D., & Le, T. D. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-23396-7_1
JetsonCompletion: Real-Time Depth Completion on Resource-Constrained Edge Devices
Wang, K., Zhu, X., Yang, B., Zhang, T., Zhang, H., Xie, F., & Li, S. (2026) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57385.2026.11696159
Seeing Like Humans: Task-Driven Token Reduction for Efficient ViT in Robotic Navigation
Liang, H., Kan, M., Shan, S., & Xilin, C. (2026) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2026.3741774
MaMe: Matrix-Based Token Merging
Huo, S., & Li, N. (2026) · doi:10.1109/cvprf71909.2026.00279
Efficient3D: A Unified Framework for Adaptive and Debiased Token Reduction in 3D MLLMs
Lin, Y., Yu, S., Yang, Y., Cheng, G., & Xiao, J. (2026) · doi:10.1109/cvprf71909.2026.00858
VLGRA: Visual-Language Semantic Optimization and Graph Reasoning for cross-modal retrieval
Yang, S., Li, Z., 尤忠祥, Jiang, S., & Zhang, C. (2026) · Information Processing & Management · doi:10.1016/j.ipm.2026.105238
Metadata
- Authors
- Meng, L., Li, H., Chen, B.-C., Lan, S., Wu, Z., Jiang, Y.-G., & Lim, S.-N.
- Affiliations
- Fudan University (CN)
- University of Maryland, College Park (US)
- Meta (United States) (US)
- Venue
- 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2022-06
- Pages
- 12299-12308
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W4226174225
- Cited by
- 262
- References
- 59
- Citations by year
- 2026: 35 · 2025: 68 · 2024: 106 · 2023: 47 · 2022: 6
- FWCI
- 20.58
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- Funders
- National Key R&D Program of China (2021ZDOI12805)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Keywords
- vision transformer, adaptive computation, self-attention, inference efficiency, image recognition, influence diagrams, long-range dependencies, accuracy-efficiency trade-off
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
