The DOI matches its Crossref record; 79 of the 84 references it cites were found in a registry or an index.
SPViT: Enabling Faster Vision Transformers via Latency-Aware Soft Token Pruning
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Some pruning methods dynamically vary the number of tokens (e.g., Kong et al. (2022)). This increases accuracy but is generally impractical, as it prevents batched inference or training without padding tokens.”
From Token Merging: Your ViT But Faster (2022) · cites this paper as Kong et al. (2022)
In the citing paper
“We omit DeiT-Ti results from the main paper because we are not able to reproduce the baseline accuracy in SPViT (Kong et al., 2022).”
From Token Merging: Your ViT But Faster (2022) · cites this paper as Kong et al., 2022
In the citing paper
“Pruning methods require token padding during training, and thus don't improve training speed, while ToMe at r = 13 trains 1.5 × faster than the baseline DeiT-S.”
From Token Merging: Your ViT But Faster (2022) · cites this paper as Kong et al., 2022
In the citing paper
“Moreover, most token pruning works are forced to use padding tokens or attention masking during training, negating the benefits of pruning in the first place.”
From Token Merging: Your ViT But Faster (2022) · cites this paper as Kong et al., 2022
In the citing paper
“In vision transformers, it has primarily been used to reduce computational cost by removing visually redundant tokens [12, 13, 31, 49, 73, 76, 85, 111].”
From Token Reduction Should Go Beyond Efficiency in Generative Models -- From Vision, Language to Multimodality (2025) · cites this paper as [73]
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Squeeze-and-Excitation Networks
Hu, J., Shen, L., & Sun, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00745 · Cited by 35,667
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534
Advances in neural information processing systems 7
Cios, K. J., & Shields, M. E. (1997) · Neurocomputing · doi:10.1016/s0925-2312(97)90018-7 · Cited by 22,093
End-to-End Object Detection with Transformers
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58452-8_13 · Cited by 18,285
Proceedings of the 25th international conference on Machine learning - ICML '08
(2008) · doi:10.1145/1390156 · Cited by 6,865
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions
Wang, W., Xie, E., Li, X., Fan, D.-P., Song, K., Liang, D., Lu, T., Luo, P., & Shao, L. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00061 · Cited by 5,259 · cited 4×
“We compare our method with several representative methods including DynamicViT [69], IA-RED2 [64], RegNetY [67], CrossViT [7], VTP [112], ATS [28], CvT [85], PVT [83], T2T-ViT [104], UP-DeiT [95], PS-ViT [78], Evo-ViT [89], TNT [34], HVT [65], Swin [49], CoaT [88], CPVT [18], EViT [46], UVC [97], MD-DeiT [39],and S2Vi…”
BackgroundMethodInfluentialRethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers
Zheng, S., Lu, J., Zhao, H., Zhu, X., Luo, Z., Wang, Y., Fu, Y., Feng, J., Xiang, T., Torr, P. H. S., et al. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00681 · Cited by 4,008 · cited 1×
“former (ViT), which solely exploits the self-attention mechanism that inherits from the Transformer architecture, has set up many state-of-the-art (SOTA) records in image classifications [22,79,7], object detection [3,19,1,60], tracking [15,91,59], semantic segmentation [110,16], depth estimation [94,45], image retrie…”
BackgroundSelf-citationZhao, H., Jiang, L., Jia, J., Torr, P. H. S., & Koltun, V. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01595 · Cited by 3,005
Research works printed but not linked: 33
- Bao, H., Dong, L., Piao, S., Wei, F.: BEit: BERT pre-training of image transformers. In: International Conference on Learning Representations (2022). https://openreview.net/forum?id=p-BhZSz59o4
- Chen, B., et al.: Psvit: better vision transformer via token pooling and attention sharing. arXiv preprint arXiv:2108.03428 (2021) · arXiv
- Chen, P., Chen, Y., Liu, S., Yang, M., Jia, J.: Exploring and improving mobile level vision transformers. arXiv preprint arXiv:2108.13015 (2021) · arXiv
- Chen, T., Chen, X., Ma, X., Wang, Y., Wang, Z.: Coarsening the granularity: towards structurally sparse lottery tickets. In: Proceedings of the International Conference on Machine Learning (ICML) (2022)
- Chen, T., Saxena, S., Li, L., Fleet, D.J., Hinton, G.: Pix2seq: a language modeling framework for object detection. arXiv preprint arXiv:2109.10852 (2021) · arXiv
- Chen, X., Hsieh, C.J., Gong, B.: When vision transformers outperform resnets without pre-training or strong data augmentations. In: International Conference on Learning Representations (2022). https://openreview.net/forum?id=LtKcMgGOeLt
- Cheng, B., Schwing, A., Kirillov, A.: Per-pixel classification is not all you need for semantic segmentation. In: Beygelzimer, A., Dauphin, Y., Liang, P., Vaughan, J.W. (eds.) Advances in Neural Information Processing Systems (2021). https://openreview.net/forum?id=0lz69oI5iZP
- Chu, X., et al.: Conditional positional encodings for vision transformers. arXiv preprint arXiv:2102.10882 (2021) · arXiv
- Dosovitskiy, A., et al.: An image is worth 16 $$\times $$ 16 words: transformers for image recognition at scale. In: International Conference on Learning Representations (2021). https://openreview.net/forum?id=YicbFdNTTy
- El-Nouby, A., Neverova, N., Laptev, I., Jégou, H.: Training vision transformers for image retrieval. arXiv preprint arXiv:2102.05644 (2021) · arXiv
- Fayyaz, M., et al.: Ats: adaptive token sampling for efficient vision transformers. arXiv preprint arXiv:2111.15667 (2021) · arXiv
- Gao, P., Lu, J., Li, H., Mottaghi, R., Kembhavi, A.: Container: context aggregation network. arXiv preprint arXiv:2106.01401 (2021) · arXiv
- Jia, D., et al.: Efficient vision transformers via fine-grained manifold distillation. arXiv preprint arXiv:2107.01378 (2021) · arXiv
- Jiang, Z., et al.: All tokens matter: token labeling for training better vision transformers. arXiv preprint arXiv:2104.10858 (2021) · arXiv
- Kornblith, S., Norouzi, M., Lee, H., Hinton, G.: Similarity of neural network representations revisited. In: International Conference on Machine Learning, pp. 3519–3529. PMLR (2019)
- Liang, Y., GE, C., Tong, Z., Song, Y., Wang, J., Xie, P.: EVit: expediting vision transformers via token reorganizations. In: International Conference on Learning Representations (2022). https://openreview.net/forum?id=BjyvwnXXVn_
- Liu, N., et al.: Lottery ticket preserves weight correlation: is it desirable or not? In: International Conference on Machine Learning (ICML), pp. 7011–7020. PMLR (2021)
- Liu, Y., Sangineto, E., Bi, W., Sebe, N., Lepri, B., De Nadai, M.: Efficient training of visual transformers with small-size datasets. arXiv preprint arXiv:2106.03746 (2021) · arXiv
- Ma, X., et al.: Sanity checks for lottery tickets: Does your winning ticket really win the jackpot? In: Advances in Neural Information Processing Systems (NeurIPS) 34 (2021)
- Pan, B., Jiang, Y., Panda, R., Wang, Z., Feris, R., Oliva, A.: Ia-red$$^2$$: Interpretability-aware redundancy reduction for vision transformers. In: Advances in Neural Information Processing Systems (2021)
- Prillo, S., Eisenschlos, J.: Softsort: a continuous relaxation for the argsort operator. In: International Conference on Machine Learning, pp. 7793–7802. PMLR (2020)
- Raghu, M., Unterthiner, T., Kornblith, S., Zhang, C., Dosovitskiy, A.: Do vision transformers see like convolutional neural networks? arXiv preprint arXiv:2108.08810 (2021) · arXiv
- Rao, Y., Zhao, W., Liu, B., Lu, J., Zhou, J., Hsieh, C.J.: Dynamicvit: efficient vision transformers with dynamic token sparsification. In: Advances in Neural Information Processing Systems (2021)
- Renggli, C., Pinto, A.S., Houlsby, N., Mustafa, B., Puigcerver, J., Riquelme, C.: Learning to merge tokens in vision transformers. arXiv preprint arXiv:2202.12015 (2022) · arXiv
- Ryoo, M.S., Piergiovanni, A., Arnab, A., Dehghani, M., Angelova, A.: Tokenlearner: what can 8 learned tokens do for images and videos? In: Advances in Neural Information Processing Systems (2021)
- Sanh, V., Wolf, T., Rush, A.M.: Movement pruning: adaptive sparsity by fine-tuning. arXiv preprint arXiv:2005.07683 (2020) · arXiv
- Steiner, A., Kolesnikov, A., Zhai, X., Wightman, R., Uszkoreit, J., Beyer, L.: How to train your vit? data, augmentation, and regularization in vision transformers. arXiv preprint arXiv:2106.10270 (2021) · arXiv
- Touvron, H., Cord, M., Douze, M., Massa, F., Sablayrolles, A., J’egou, H.: Training data-efficient image transformers & distillation through attention. In: ICML (2021)
- Wu, B., et al.: Visual transformers: token-based image representation and processing for computer vision. arXiv preprint arXiv:2006.03677 (2020) · arXiv
- Yu, H., Wu, J.: A unified pruning framework for vision transformers. arXiv preprint arXiv:2111.15127 (2021) · arXiv
- Yu, Q., Xia, Y., Bai, Y., Lu, Y., Yuille, A., Shen, W.: Glance-and-gaze vision transformer. In: Advances in Neural Information Processing Systems (2021)
- Yuan, L., et al.: Tokens-to-token vit: training vision transformers from scratch on imagenet. arXiv preprint arXiv:2101.11986 (2021) · DOI · Paper page
- Zhu, M., Han, K., Tang, Y., Wang, Y.: Visual transformer pruning. In: KDD 2021 Workshop on Model Mining (2021)
Cited by
Sorted by citations
TinyViT: Fast Pretraining Distillation for Small Vision Transformers
Wu, K., Zhang, J., Peng, H., Liu, M., Xiao, B., Fu, J., & Yuan, L. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19803-8_5 · Cited by 521
A Survey on Efficient Vision Transformers: Algorithms, Techniques, and Performance Benchmarking
Papa, L., Russo, P., Amerini, I., & Zhou, L. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3392941 · Cited by 200
Chen, L., Zhao, H., Liu, T., Bai, S., Lin, J., Zhou, C., & Chang, B. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73004-7_2 · Cited by 359
Less is More: Focus Attention for Efficient DETR
Zheng, D., Dong, W., Hu, H., Chen, X., & Wang, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00614 · Cited by 162
Self-citationHeatViT: Hardware-Efficient Adaptive Token Pruning for Vision Transformers
Dong, P., Sun, M., Lu, A., Xie, Y., Liu, K., Kong, Z., Meng, X., Li, Z., Lin, X., Fang, Z., et al. (2023) · 2023 IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca56546.2023.10071047 · Cited by 128
Self-citationHydra Attention: Efficient Attention with Many Heads
Bolya, D., Fu, C.-Y., Dai, X., Zhang, P., & Hoffman, J. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-25082-8_3 · Cited by 96
PPT: Token-Pruned Pose Transformer for Monocular and Multi-view Human Pose Estimation
Ma, H., Wang, Z., Chen, Y., Kong, D., Chen, L., Liu, X., Yan, X., Tang, H., & Xie, X. S. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20065-6_25 · Cited by 90
Self-citationJoint Token Pruning and Squeezing Towards More Aggressive Compression of Vision Transformers
Wei, S., Ye, T., Zhang, S., Tang, Y., & Liang, J. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00208 · Cited by 95
Token Fusion: Bridging the Gap between Token Pruning and Token Merging
Kim, M., Gao, S., Hsu, Y.-C., Shen, Y., & Jin, H. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00141 · Cited by 103
Hourglass Tokenizer for Efficient Transformer-Based 3D Human Pose Estimation
Li, W., Liu, M., Liu, H., Wang, P., Cai, J., & Sebe, N. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00064 · Cited by 69
Token Merging: Your ViT But Faster
Bolya, D., Fu, C.-Y., Dai, X., Zhang, P., Feichtenhofer, C., & Hoffman, J. (2022) · arXiv · doi:10.48550/arxiv.2210.09461 · Cited by 140
“4, we compare ToMe to token pruning works that use DeiT-S training3: A-ViT (Yin et al., 2022), DynamicViT (Rao et al., 2021), and SP-ViT (Kong et al., 2022) with throughput measured on a V100.”
BackgroundMethodInfluentialViTs as backbones: Leveraging vision transformers for feature extraction
Elharrouss, O., Himeur, Y., Mahmood, Y., Alrabaee, S., Ouamane, A., Bensaali, F., Bechqito, Y., & Chouchane, A. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.102951 · Cited by 69
Saha, S., & Xu, L. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.130417 · Cited by 85
SegViT v2: Exploring Efficient and Continual Semantic Segmentation with Plain Vision Transformers
Zhang, B., Liu, L., Phan, V. M. H., Tian, Z., Shen, C., & Liu, Y. (2023) · International Journal of Computer Vision · doi:10.1007/s11263-023-01894-8 · Cited by 54
Sarkar, R., Liang, H., Fan, Z., Wang, Z., & Hao, C. (2023) · 2023 IEEE/ACM International Conference on Computer Aided Design (ICCAD) · doi:10.1109/iccad57390.2023.10323651 · Cited by 64
Sparsity in transformers: A systematic literature review
Farina, M., Ahmad, U., Taha, A. T. M., Younes, H., Mesbah, Y., Yu, X., & Witold Pedrycz, C. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.127468 · Cited by 59
Which Tokens to Use? Investigating Token Reduction in Vision Transformers
Haurum, J. B., Escalera, S., Taylor, G. W., & Moeslund, T. B. (2023) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw60793.2023.00085 · Cited by 47
TORE: Token Reduction for Efficient Human Mesh Recovery with Transformer
Dou, Z., Wu, Q., Lin, C. W., Cao, Z., Wu, Q., Wan, W., Komura, T., & Wang, W. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01390 · Cited by 47
Dynamic Neural Network Structure: A Review for its Theories and Applications
Guo, J., Chen, C. L. P., Liu, Z., & Yang, X. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3377194 · Cited by 48
Resource-efficient Algorithms and Systems of Foundation Models: A Survey
Xu, M., Cai, D., Yin, W., Wang, S., Jin, X., & Liu, X. (2024) · ACM Computing Surveys · doi:10.1145/3706418 · Cited by 255
DiffRate : Differentiable Compression Rate for Efficient Vision Transformers
Chen, M., Shao, W., Xu, P., Lin, M., Zhang, K., Chao, F., Ji, R., Qiao, Y., & Luo, P. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01574 · Cited by 58
Wang, H., Dedhia, B., & Jha, N. K. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01521 · Cited by 63
Scene Adaptive Sparse Transformer for Event-based Object Detection
Peng, Y., Li, H., Zhang, Y., Sun, X., & Wu, F. H. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01589 · Cited by 58
RIFormer: Keep Your Vision Backbone Effective But Removing Token Mixer
Wang, J., Zhang, S., Liu, Y., Wu, T., Yang, Y., Liu, X., Chen, K., Luo, P., & Lin, D. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01388 · Cited by 40
Interaction Transformer for Human Reaction Generation
Chopin, B., Tang, H., Otberdout, N., Daoudi, M., & Sebe, N. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3242152 · Cited by 49
Self-citationSTPrivacy: Spatio-Temporal Privacy-Preserving Action Recognition
Li, M., Xu, X., Fan, H., Zhou, P., Liu, J., Liu, J., Li, J., Keppo, J., Shou, M. Z., & Yan, S. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00471 · Cited by 39
Revisiting Token Pruning for Object Detection and Instance Segmentation
Liu, Y., Gehrig, M., Messikommer, N., Cannici, M., & Scaramuzza, D. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00264 · Cited by 54
vid-TLDR: Training Free Token merging for Light-Weight Video Transformer
Choi, J., Lee, S., Chu, J., Choi, M., & Kim, H. J. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01776 · Cited by 35
Fu, X., Ren, Q., Wu, H., Xiang, F., Luo, Q., Yue, J., Chen, Y., & Zhang, F. (2023) · IEEE Transactions on Circuits and Systems I Regular Papers · doi:10.1109/tcsi.2023.3315060 · Cited by 30
Zhang, X., Zeng, F., & Gu, C. (2024) · Neural Networks · doi:10.1016/j.neunet.2024.107059 · Cited by 27
Norouzi, N., Orlova, S., de Geus, D., & Dubbelman, G. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01493 · Cited by 22
MaskGaussian: Adaptive 3D Gaussian Representation from Probabilistic Masks
Liu, Y., Zhong, Z., Zhan, Y., Xu, S., & Sun, X. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00072 · Cited by 21
Dynamic Focus-aware Positional Queries for Semantic Segmentation
He, H., Cai, J., Pan, Z., Liu, J., Zhang, J., Tao, D., & Zhuang, B. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01087 · Cited by 22
Metal Surface Defect Detection Based on a Transformer with Multi-Scale Mask Feature Fusion
Zhao, L., Zheng, Y., Peng, T., & Zheng, E. (2023) · Sensors · doi:10.3390/s23239381 · Cited by 21
How can objects help action recognition?
Zhou, X., Arnab, A., Sun, C., & Schmid, C. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00233 · Cited by 20
Dynamic Feature Pruning and Consolidation for Occluded Person Re-identification
Ye, Y., Zhou, H., Cai, J., Gao, C., Zhang, Y., Wang, J., Hu, Q., Yu, J., & Yang, W. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i7.28491 · Cited by 24
ELSA: Exploiting Layer-wise N:M Sparsity for Vision Transformer Acceleration
Huang, N.-C., Chang, C.-C., Lin, W., Taka, E., Marculescu, D., & Wu, K. H. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00799 · Cited by 18
ReViT: Vision Transformer Accelerator With Reconfigurable Semantic-Aware Differential Attention
Zou, X., Chen, C., Shao, H., Wang, Q., Zhuang, X., Li, Y., & Li, K. (2024) · IEEE Transactions on Computers · doi:10.1109/tc.2024.3504263 · Cited by 19
GTP-ViT: Efficient Vision Transformers via Graph-based Token Propagation
Xu, X., Wang, S., Chen, Y., Zheng, Y., Wei, Z., & Liu, J. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00016 · Cited by 24
Multi-Criteria Token Fusion with One-Step-Ahead Attention for Efficient Vision Transformers
Lee, S., Choi, J., & Kim, H. J. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01490 · Cited by 19
LPViT: Low-Power Semi-structured Pruning for Vision Transformers
Xu, K., Wang, Z., Chen, C., Geng, X., Lin, J., Yang, X., Wu, M., Li, X., & Lin, W. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73209-6_16 · Cited by 15
Mao, J., Shen, Y., Guo, J., Yao, Y., Hua, X., & Shen, H. T. (2025) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2025.3535405 · Cited by 24
Accelerating ViT Inference on FPGA through Static and Dynamic Pruning
Parikh, D., Li, S., Zhang, B., Kannan, R., Busart, C., & Prasanna, V. K. (2024) · IEEE International Symposium on Field-Programmable Custom Computing Machines (FCCM) · doi:10.1109/fccm60383.2024.00018 · Cited by 18
Prototype-Driven Multi-Feature Generation for Visible-Infrared Person Re-identification
Li, J., Zhen, Q., Yang, Y., Li, Y., Dong, Z., & Yang, C. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10889917 · Cited by 14
Segformer++: Efficient Token-Merging Strategies for High-Resolution Semantic Segmentation
Kienzle, D., Kantonis, M., Schön, R., & Lienhart, R. W. (2024) · IEEE International Conference on Multimedia Information Processing and Retrieval (MIPR) · doi:10.1109/mipr62202.2024.00019 · Cited by 13
Leveraging Temporal Contextualization for Video Action Recognition
Kim, M., Han, D., Kim, T., & Han, B. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72664-4_5 · Cited by 13
PELA: Learning Parameter-Efficient Models with Low-Rank Approximation
Guo, Y., Wang, G., & Kankanhalli, M. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01486 · Cited by 14
Efficient Transformer-based 3D Object Detection with Dynamic Token Halting
Ye, M., Meyer, G. P., Chai, Y., & Liu, Q. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00775 · Cited by 12
IVTP: Instruction-Guided Visual Token Pruning for Large Vision-Language Models
Huang, K., Zou, H., Ye, X., Wang, B., Xie, Z., & Liang, Y. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72643-9_13 · Cited by 17
Survey on Efficient Large Language Models: Principles, Algorithms, Applications, and Open Issues
Cheng, J., Kang, H., Shao, Y., Li, N., Chen, P., Wang, R., Long, S., Yang, X., & Ma, L. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2025.3628671 · Cited by 12
Expediting Large-Scale Vision Transformer for Dense Prediction Without Fine-Tuning
Yuan, Y., Liang, W., Ding, H., Liang, Z., Zhang, C., & Hu, H. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3327511 · Cited by 14
Remote Sensing Scene Classification via Second-Order Differentiable Token Transformer Network
Ni, K., Wu, Q., Li, S., Zheng, Z., & Wang, P. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3407879 · Cited by 9
A pure MLP-Mixer-based GAN framework for guided image translation
Tang, H., Ren, B., & Sebe, N. (2024) · Pattern Recognition · doi:10.1016/j.patcog.2024.110894 · Cited by 9
Self-citationAgglomerative Token Clustering
Haurum, J. B., Escalera, S., Taylor, G. W., & Moeslund, T. B. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72998-0_12 · Cited by 11
AdapTiV: Sign-Similarity Based Image-Adaptive Token Merging for Vision Transformer Acceleration
Yoo, S.-J., Kim, H., & Kim, J. (2024) · IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1109/micro61859.2024.00015 · Cited by 11
Deep Joint Source Channel Coding With Attention Modules Over MIMO Channels
Jiang, W., Chen, W., & Ai, B. (2024) · IEEE Vehicular Technology Conference · doi:10.1109/vtc2024-spring62846.2024.10683289 · Cited by 8
Zhu, L., Chen, T., Ji, D., Ye, J., & Liu, J. (2025) · IEEE Transactions on Image Processing · doi:10.1109/tip.2025.3531299 · Cited by 8
TC-MGC: Text-conditioned multi-grained contrastive learning for text–video retrieval
Jing, X., Yang, G., & Chu, J. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103151 · Cited by 8
Multi-scale motion perception fall detection algorithm based on video swin transformer
Cai, X., Wang, X., Yin, J., Chen, Y., & Han, G. (2025) · Signal Image and Video Processing · doi:10.1007/s11760-025-04358-3 · Cited by 8
A General and Efficient Training for Transformer via Token Expansion
Huang, W., Shen, Y., Xie, J., Zhang, B., He, G., Li, K., Sun, X., & Lin, S. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01494 · Cited by 8
LaTP: LiDAR-aided multimodal token pruning for efficient trajectory prediction of autonomous driving
Lu, Y., Sun, S., Liu, N., Jiang, B., Li, Y., Chen, J., Zhang, Y., Zhu, Y., & Velipasalar, S. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.107673 · Cited by 7
Efficient Vision Transformer with Token Sparsification for Event-Based Object Tracking
Zhang, J., Yang, X., Tang, H., Wang, Y., Yin, B., Wang, H., & Fu, X. (2026) · International Journal of Computer Vision · doi:10.1007/s11263-025-02666-2 · Cited by 7
Self-citationYou Already Have It: A Generator-Free Low-Precision DNN Training Framework Using Stochastic Rounding
Yuan, G., Chang, S.-E., Jin, Q., Lu, A., Li, Y., Wu, Y., Kong, Z., Xie, Y., Dong, P., Qin, M., et al. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19775-8_3 · Cited by 9
Self-citationNo Token Left Behind: Efficient Vision Transformer via Dynamic Token Idling
Xu, X., Li, C., Chen, Y., Chang, X., Liu, J., & Wang, S. (2023) · Lecture notes in computer science · doi:10.1007/978-981-99-8388-9_3 · Cited by 11
Do we really need a large number of visual prompts?
Kim, Y., Li, Y., Moitra, A., Yin, R., & Panda, P. (2024) · Neural Networks · doi:10.1016/j.neunet.2024.106390 · Cited by 7
Song, Y., Qian, P., Zhang, K., Liu, S., Zhai, R., & Song, R. (2025) · Multimedia Systems · doi:10.1007/s00530-025-01746-0 · Cited by 7
Zebra: Leveraging Diagonal Attention Pattern for Vision Transformer Accelerator
Han, S., Kim, S., Byeon, G., Yoon, J., & Hong, S. (2025) · Design, Automation, and Test in Europe · doi:10.23919/date64628.2025.10993213 · Cited by 7
KFF: K-feature fusion token merging for vision transformer
Yang, Y., Zhou, Y., Hu, X., & Duan, S. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.128206 · Cited by 6
Efficient Visual Transformer by Learnable Token Merging
Wang, Y., & Yang, Y. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3588186 · Cited by 6
Self-citationGuo, D., Wu, Z., Feng, J., Zhou, Z., & Shen, Z. (2023) · Applied Intelligence · doi:10.1007/s10489-023-04725-y · Cited by 6
ATFTrans: attention-weighted token fusion transformer for robust and efficient object tracking
Xu, L., Wang, L., & Guo, Z. (2024) · Neural Computing and Applications · doi:10.1007/s00521-024-09444-0 · Cited by 6
Efficient Transformer Adaptation with Soft Token Merging
Yuan, X., Fei, H., & Baek, J. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00369 · Cited by 8
Data Level Lottery Ticket Hypothesis for Vision Transformers
Shen, X., Kong, Z., Qin, M., Dong, P., Yuan, G., Meng, X., Tang, H., Ma, X., & Wang, Y. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/153 · Cited by 10
Self-citationPYRA: Parallel Yielding Re-activation for Training-Inference Efficient Task Adaptation
Xiong, Y., Chen, H., Hao, T., Lin, Z., Han, J., Zhang, Y., Wang, G., Bao, Y., & Ding, G. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72673-6_25 · Cited by 14
UCC: A unified cascade compression framework for vision transformer models
Chen, D., Lin, K., & Deng, Q. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128747 · Cited by 8
MCANet: a lightweight action recognition network with multidimensional convolution and attention
Tian, Q., Miao, W., Zhang, L., Yang, Z., Yu, Y., Zhao, Y., & Yao, L. (2024) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-024-02454-3 · Cited by 5
Squat: Quant Small Language Models on the Edge
Shen, X., Dong, P., Kong, Z., Gong, Y., Yang, C., Han, Z., Xie, Y., Lu, L., Lyu, C., Wu, C., et al. (2025) · doi:10.1109/iccad66269.2025.11240685 · Cited by 41
Self-citationThe Role of Masking for Efficient Supervised Knowledge Distillation of Vision Transformers
Son, S., Ryu, J., Lee, N., & Lee, J. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72855-6_22 · Cited by 7
Raformer: Redundancy-Aware Transformer for Video Wire Inpainting
Ji, Z., Su, Y., Zhang, Y., Hou, J., Pang, Y., & Han, J. (2025) · IEEE Transactions on Image Processing · doi:10.1109/tip.2025.3550038 · Cited by 5
Famba-V: Fast Vision Mamba with Cross-Layer Token Fusion
Shen, H., Wan, Z., Wang, X., & Zhang, M. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-91979-4_20 · Cited by 5
Faster Parameter-Efficient Tuning with Token Redundancy Reduction
Kim, K., Park, J., Kim, J., Kwon, H., & Sohn, K. (2025) · 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52734.2025.02810 · Cited by 5
PRANCE: Joint Token-Optimization and Structural Channel-Pruning for Adaptive ViT Inference
Li, Y., Tang, C., Meng, Y., Fan, J., Chai, Z., Ma, X., Wang, Z., & Zhu, W. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3605239 · Cited by 4
Self-citationImportance-Based Token Merging for Efficient Image and Video Generation
Wu, H., Xu, J., Le, H., & Samaras, D. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00474 · Cited by 15
Luo, J., Zhang, Y., Yang, X., Wu, K., Zhu, Q., Liang, L., Chen, J., & Li, Y. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00860 · Cited by 7
PaPr: Training-Free One-Step Patch Pruning with Lightweight ConvNets for Faster Inference
Mahmud, T., Yaman, B., Liu, C.-H., & Marculescu, D. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73337-6_7 · Cited by 4
Sparse Refinement for Efficient High-Resolution Semantic Segmentation
Liu, Z., Zhang, Z., Khaki, S., Yang, S. W., Tang, H., Xu, C., Keutzer, K., & Han, S. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72855-6_7 · Cited by 4
Self-citationZhang, X., Dai, X., Zhang, Z., Di, X., & Chen, X. (2024) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2024.3383542 · Cited by 4
Partitioned token fusion and pruning strategy for transformer tracking
Zhang, C., Gao, Y. T., Meng, T., & Wang, T. (2025) · Image and Vision Computing · doi:10.1016/j.imavis.2025.105431 · Cited by 4
VLTP: Vision-Language Guided Token Pruning for Task-Oriented Segmentation
Chen, H., Ni, Y., Huang, W., Liu, Y., Jeong, S., Wen, F., Bastian, N. D., Latapie, H. M., & Imani, M. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00906 · Cited by 11
Compressing Vision Transformer from the View of Model Property in Frequency Domain
Wang, Z., Xie, X., Luo, H., Huang, T., Dong, W., Xiong, K., Liu, Y., Li, X., Wang, F., & Shi, G. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02561-w · Cited by 3
HyAtt: A Hybrid Attention Accelerator for Transformers With Analog In-Memory Computing
Moradifirouzabadi, A., & Kang, M. (2025) · IEEE Journal of Solid-State Circuits · doi:10.1109/jssc.2025.3602940 · Cited by 3
Simultaneous image patch attention and pruning for patch selective transformer
Kim, S., Yoon, G.-J., Song, J., & Yoon, S. M. (2024) · Image and Vision Computing · doi:10.1016/j.imavis.2024.105239 · Cited by 3
Neighbor patches merging reduces spatial redundancy to accelerate vision transformer
Jiang, K., Peng, P., Lian, Y., Shao, W., & Xu, W. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128733 · Cited by 3
Make Your ViT-Based Multi-view 3D Detectors Faster via Token Compression
Zhang, D., Liang, D., Tan, Z., Ye, X., Zhang, C., Wang, J., & Bai, X. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72970-6_4 · Cited by 3
Multi-Faceted Adaptive Token Pruning for Efficient Remote Sensing Image Segmentation
Zhang, C., & Yao, J. (2025) · Remote Sensing · doi:10.3390/rs17142508 · Cited by 3
Enhancing spatiotemporal predictive learning: an approach with nested attention module
Wang, S., & Ren, H. (2024) · Journal of Intelligent Manufacturing · doi:10.1007/s10845-023-02318-7 · Cited by 3
Bandit-based Attention Mechanism in Vision Transformers
Chowdhury, A. R., Diddigi, R. B., Prabuchandran, K. J., & Tripathi, A. M. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00929 · Cited by 3
Is Less More? Exploring Token Condensation as Training-Free Test-Time Adaptation
Wang, Z., Gong, D., Wang, S., Huang, Z. H., & Luo, Y. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00021 · Cited by 9
VLDrive: Vision-Augmented Lightweight MLLMs for Efficient Language-Grounded Autonomous Driving
Zhang, R., Zhang, W., Tan, X., Yang, S., Wan, X., Luo, X., & Li, G. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00560 · Cited by 2
Similarity-Aware Fast Low-Rank Decomposition Framework for Vision Transformers
Luo, Y.-J., Tai, Y.-S., Lin, M.-G., & Wu, A.-Y. A. (2024) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas58744.2024.10557934 · Cited by 2
Metadata
- Authors
- Kong, Z., Dong, P., Ma, X., Meng, X., Niu, W., Sun, M., Shen, X., Yuan, G., Ren, B., Tang, H., Qin, M., & Wang, Y.
- Affiliations
- Northeastern University (US)
- Clemson University (US)
- Peking University (CN)
- William & Mary (US)
- ETH Zurich (CH)
- Corresponding author
- Wang, Yanzhi
- ORCID
- Venue
- Lecture Notes in Computer Science
- Publisher
- Springer Nature Switzerland
- Published
- 2022
- Pages
- 620-640
- ISSN
- 0302-9743 (ISSN-L)
- 1611-3349
- OpenAlex
- W4312910119
- Cited by
- 169
- References
- 84
- Citations by year
- 2026: 25 · 2025: 61 · 2024: 55 · 2023: 25 · 2022: 3
- FWCI
- 13.06
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Lecture notes in computer science (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Keywords
- vision transformer, token selection, multi-head attention, edge deployment, mobile devices, FPGA, real-time inference, DeiT, Swin Transformer, computation cost reduction, accuracy-latency trade-off
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
