SPViT: Enabling Faster Vision Transformers via Latency-Aware Soft Token Pruning
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Dynamic token pruning increases accuracy but is generally impractical because it prevents batched inference or training without padding tokens.
“Some pruning methods dynamically vary the number of tokens (e.g., Kong et al. (2022)). This increases accuracy but is generally impractical, as it prevents batched inference or training without padding tokens.”
From Token Merging: Your ViT But Faster (2022) · cites this paper as Kong et al. (2022)
The authors are not able to reproduce the baseline accuracy in SPViT (Kong et al., 2022).
“We omit DeiT-Ti results from the main paper because we are not able to reproduce the baseline accuracy in SPViT (Kong et al., 2022).”
From Token Merging: Your ViT But Faster (2022) · cites this paper as Kong et al., 2022
Pruning methods require token padding during training, and thus don't improve training speed.
“Pruning methods require token padding during training, and thus don't improve training speed, while ToMe at r = 13 trains 1.5 × faster than the baseline DeiT-S.”
From Token Merging: Your ViT But Faster (2022) · cites this paper as Kong et al., 2022
Most token pruning works are forced to use padding tokens or attention masking during training, negating the benefits of pruning in the first place.
“Moreover, most token pruning works are forced to use padding tokens or attention masking during training, negating the benefits of pruning in the first place.”
From Token Merging: Your ViT But Faster (2022) · cites this paper as Kong et al., 2022
In vision transformers, token reduction has primarily been used to reduce computational cost by removing visually redundant tokens.
“In vision transformers, it has primarily been used to reduce computational cost by removing visually redundant tokens [12, 13, 31, 49, 73, 76, 85, 111].”
From Token Reduction Should Go Beyond Efficiency in Generative Models -- From Vision, Language to Multimodality (2025) · cites this paper as [73]
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · see citing works
Squeeze-and-Excitation Networks
Hu, J., Shen, L., & Sun, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00745 · Cited by 35,667 · see citing works
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534 · see citing works
Advances in neural information processing systems 7
Cios, K. J., & Shields, M. E. (1997) · Neurocomputing · doi:10.1016/s0925-2312(97)90018-7 · Cited by 22,089 · see citing works
End-to-End Object Detection with Transformers
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58452-8_13 · Cited by 18,285 · see citing works
Proceedings of the 25th international conference on Machine learning - ICML '08
(2008) · doi:10.1145/1390156 · Cited by 6,865 · see citing works
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions
Wang, W., Xie, E., Li, X., Fan, D.-P., Song, K., Liang, D., Lu, T., Luo, P., & Shao, L. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00061 · Cited by 5,259 · cited 4× · see citing works
“We compare our method with several representative methods including DynamicViT [69], IA-RED2 [64], RegNetY [67], CrossViT [7], VTP [112], ATS [28], CvT [85], PVT [83], T2T-ViT [104], UP-DeiT [95], PS-ViT [78], Evo-ViT [89], TNT [34], HVT [65], Swin [49], CoaT [88], CPVT [18], EViT [46], UVC [97], MD-DeiT [39],and S2Vi…”
BackgroundMethodInfluentialRethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers
Zheng, S., Lu, J., Zhao, H., Zhu, X., Luo, Z., Wang, Y., Fu, Y., Feng, J., Xiang, T., Torr, P. H. S., et al. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00681 · Cited by 4,008 · cited 1× · see citing works
“former (ViT), which solely exploits the self-attention mechanism that inherits from the Transformer architecture, has set up many state-of-the-art (SOTA) records in image classifications [22,79,7], object detection [3,19,1,60], tracking [15,91,59], semantic segmentation [110,16], depth estimation [94,45], image retrie…”
BackgroundSelf-citationZhao, H., Jiang, L., Jia, J., Torr, P. H. S., & Koltun, V. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01595 · Cited by 2,847 · see citing works
CvT: Introducing Convolutions to Vision Transformers
Wu, H., Xiao, B., Codella, N., Liu, M., Dai, X., Yuan, L., & Zhang, L. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00009 · Cited by 2,649 · see citing works
Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet
Yuan, L., Chen, Y., Wang, T., Yu, W., Shi, Y., Jiang, Z., Tay, F. E. H., Feng, J., & Yan, S. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00060 · Cited by 2,415 · see citing works
Pre-Trained Image Processing Transformer
Chen, H., Wang, Y., Guo, T., Xu, C., Deng, Y., Liu, Z., Ma, S., Xu, C., Xu, C., & Gao, W. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01212 · Cited by 2,308 · see citing works
Self-citationCrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification
Chen, C.-F. R., Fan, Q., & Panda, R. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00041 · Cited by 2,242 · see citing works
Guo, M.-H., Cai, J.-X., Liu, Z.-N., Mu, T.-J., Martin, R. R., & Hu, S.-M. (2021) · Computational Visual Media · doi:10.1007/s41095-021-0229-5 · Cited by 2,100 · see citing works
Designing Network Design Spaces
Radosavovic, I., Kosaraju, R. P., Girshick, R., He, K., & Dollar, P. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01044 · Cited by 2,084 · see citing works
Chen, X., Yan, B., Zhu, J., Wang, D., Yang, X., & Lu, H. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00803 · Cited by 1,588 · cited 1× · see citing works
“former (ViT), which solely exploits the self-attention mechanism that inherits from the Transformer architecture, has set up many state-of-the-art (SOTA) records in image classifications [22,79,7], object detection [3,19,1,60], tracking [15,91,59], semantic segmentation [110,16], depth estimation [94,45], image retrie…”
BackgroundHan, K., Xiao, A., Wu, E., Guo, J., Xu, C., & Wang, Y. (2021) · arXiv · doi:10.48550/arxiv.2103.00112 · Cited by 1,394 · cited 4× · see citing works
“We compare our method with several representative methods including DynamicViT [69], IA-RED2 [64], RegNetY [67], CrossViT [7], VTP [112], ATS [28], CvT [85], PVT [83], T2T-ViT [104], UP-DeiT [95], PS-ViT [78], Evo-ViT [89], TNT [34], HVT [65], Swin [49], CoaT [88], CPVT [18], EViT [46], UVC [97], MD-DeiT [39],and S2Vi…”
BackgroundMethodInfluentialSelf-citationBottleneck Transformers for Visual Recognition
Srinivas, A., Lin, T.-Y., Parmar, N., Shlens, J., Abbeel, P., & Vaswani, A. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01625 · Cited by 1,385 · see citing works
Learning Spatio-Temporal Transformer for Visual Tracking
Yan, B., Peng, H., Fu, J., Wang, D., & Lu, H. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01028 · Cited by 1,280 · cited 1× · see citing works
“former (ViT), which solely exploits the self-attention mechanism that inherits from the Transformer architecture, has set up many state-of-the-art (SOTA) records in image classifications [22,79,7], object detection [3,19,1,60], tracking [15,91,59], semantic segmentation [110,16], depth estimation [94,45], image retrie…”
BackgroundTrackFormer: Multi-Object Tracking with Transformers
Meinhardt, T., Kirillov, A., Leal-Taixe, L., & Feichtenhofer, C. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00864 · Cited by 1,081 · see citing works
Learning Texture Transformer Network for Image Super-Resolution
Yang, F., Yang, H., Fu, J., Lu, H., & Guo, B. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00583 · Cited by 1,034 · see citing works
LeViT: a Vision Transformer in ConvNet’s Clothing for Faster Inference
Graham, B., El-Nouby, A., Touvron, H., Stock, P., Joulin, A., Jegou, H., & Douze, M. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01204 · Cited by 935 · see citing works
Transformer Interpretability Beyond Attention Visualization
Chefer, H., Gur, S., & Wolf, L. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00084 · Cited by 903 · see citing works
Zhai, X., Kolesnikov, A., Houlsby, N., & Beyer, L. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01179 · Cited by 891 · see citing works
Transformer for Single Image Super-Resolution
Lu, Z., Li, J., Liu, H., Huang, C., Zhang, L., & Zeng, T. (2022) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00061 · Cited by 755 · see citing works
Rethinking Spatial Dimensions of Vision Transformers
Heo, B., Yun, S., Han, D., Chun, S., Choe, J., & Oh, S. J. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01172 · Cited by 691 · see citing works
Incorporating Convolution Designs into Visual Transformers
Yuan, K., Guo, S., Liu, Z., Zhou, A., Yu, F., & Wu, W. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00062 · Cited by 611 · see citing works
UP-DETR: Unsupervised Pre-training for Object Detection with Transformers
Dai, Z., Cai, B., Lin, Y., & Chen, J. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00165 · Cited by 547 · see citing works
An End-to-End Transformer Model for 3D Object Detection
Misra, I., Girdhar, R., & Joulin, A. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00290 · Cited by 506 · see citing works
SpAtten: Efficient Sparse Attention Architecture with Cascade Token and Head Pruning
Wang, H., Zhang, Z., & Han, S. (2021) · 2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca51647.2021.00018 · Cited by 427 · see citing works
Co-Scale Conv-Attentional Image Transformers
Xu, W., Xu, Y., Chang, T. A., & Tu, Z. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00983 · Cited by 425 · see citing works
TransVG: End-to-End Visual Grounding with Transformers
Deng, J., Yang, Z., Chen, T., Zhou, W., & Li, H. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00179 · Cited by 406 · see citing works
Rethinking and Improving Relative Position Encoding for Vision Transformer
Wu, K., Peng, H., Chen, M., Fu, J., & Chao, H. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00988 · Cited by 393 · see citing works
TransFER: Learning Relation-aware Facial Expression Representations with Transformers
Xue, F., Wang, Q., & Guo, G. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00358 · Cited by 346 · see citing works
Revisiting Stereo Depth Estimation From a Sequence-to-Sequence Perspective with Transformers
Li, Z., Liu, X., Drenkow, N., Ding, A., Creighton, F. X., Taylor, R. H., & Unberath, M. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00614 · Cited by 335 · see citing works
AutoFormer: Searching Transformers for Visual Recognition
Chen, M., Peng, H., Fu, J., & Ling, H. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01205 · Cited by 323 · see citing works
HOTR: End-to-End Human-Object Interaction Detection with Transformers
Kim, B., Lee, J., Kang, J., Kim, E., & Kim, H. J. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00014 · Cited by 307 · see citing works
XCiT: Cross-Covariance Image TransformersPreprint
El-Nouby, A., Touvron, H., Caron, M., Bojanowski, P., Douze, M., Joulin, A., Laptev, I., Neverova, N., Synnaeve, G., Verbeek, J. J., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.09681 · Cited by 273 · see citing works
PatDNN: Achieving Real-Time DNN Execution on Mobile Devices with Pattern-based Weight Pruning
Niu, W., Ma, X., Lin, S., Wang, S., Qian, X., Lin, X., Wang, Y., & Ren, B. (2020) · International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) · doi:10.1145/3373376.3378534 · Cited by 258 · see citing works
Self-citationTransformer-Based Attention Networks for Continuous Pixel-Wise Prediction
Yang, G., Tang, H., Ding, M., Sebe, N., & Ricci, E. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01596 · Cited by 222 · see citing works
Self-citationEvo-ViT: Slow-Fast Token Evolution for Dynamic Vision Transformer
Xu, Y., Zhang, Z., Zhang, M., Sheng, K., Li, K., Dong, W., Zhang, L., Xu, C., & Sun, X. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i3.20202 · Cited by 202 · see citing works
Patch Slimming for Efficient Vision Transformers
Tang, Y., Han, K., Wang, Y., Xu, C., Guo, J., Xu, C., & Tao, D. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01185 · Cited by 201 · see citing works
Self-citationMa, X., Guo, F.-M., Niu, W., Lin, X., Tang, J., Ma, K., Ren, B., & Wang, Y. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.5954 · Cited by 196 · see citing works
Self-citationScalable Vision Transformers with Hierarchical Pooling
Pan, Z., Zhuang, B., Liu, J., He, H., & Cai, J. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00043 · Cited by 168 · see citing works
Ren, A., Zhang, T., Ye, S., Li, J., Xu, W., Qian, X., Lin, X., & Wang, Y. (2019) · International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) · doi:10.1145/3297858.3304076 · Cited by 150 · see citing works
Self-citationKVT: k-NN Attention for Boosting Vision Transformers
Wang, P., Wang, X., Wang, F., Lin, M., Chang, S., Li, H., & Jin, R. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20053-3_17 · Cited by 142 · see citing works
Mix and Match: A Novel FPGA-Centric Deep Neural Network Quantization Framework
Chang, S.-E., Li, Y., Sun, M., Shi, R., So, H. K., Qian, X., Wang, Y., & Lin, X. (2021) · IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca51647.2021.00027 · Cited by 129 · see citing works
Self-citationVision Transformer with Progressive Sampling
Yue, X., Sun, S., Kuang, Z., Wei, M., Torr, P. H. S., Zhang, W., & Lin, D. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00044 · Cited by 126 · see citing works
Instance Localization for Self-supervised Detection Pretraining
Yang, C., Wu, Z., Zhou, B., & Lin, S. C.-F. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00398 · Cited by 116 · see citing works
Non-Structured DNN Weight Pruning—Is It Beneficial in Any Platform?
Ma, X., Lin, S., Ye, S., He, Z., Zhang, L., Yuan, G., Tan, S. H., Li, Z., Fan, D., Qian, X., et al. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3063265 · Cited by 116 · see citing works
Self-citationCHEX: CHannel EXploration for CNN Model Compression
Hou, Z., Qin, M., Sun, F., Ma, X., Yuan, K., Xu, Y. T., Chen, Y.-K., Jin, R., Xie, Y., & Kung, S. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01197 · Cited by 91 · see citing works
Self-citationAccelerating Sparse DNN Models without Hardware-Support via Tile-Wise Sparsity
Guo, C., Hsueh, B. Y., Leng, J., Qiu, Y., Guan, Y., Wang, Z., Jia, X., Li, X., Guo, M., & Zhu, Y. (2020) · International Conference for High Performance Computing, Networking, Storage and Analysis (SC) · doi:10.1109/sc41405.2020.00020 · Cited by 73 · see citing works
PIM-Prune: Fine-Grain DCNN Pruning for Crossbar-Based Process-In-Memory Architecture
Chu, C., Wang, Y., Zhao, Y., Ma, X., Ye, S., Hong, Y., Liang, X., Han, Y., & Jiang, L. (2020) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1109/dac18072.2020.9218523 · Cited by 70 · see citing works
Self-citationYuan, G., Ma, X., Ding, C., Lin, S., Zhang, T., Jalali, Z. S., Zhao, Y., Jiang, L., Soundarajan, S., & Wang, Y. (2019) · International Symposium on Low Power Electronics and Design (ISLPED) · doi:10.1109/islped.2019.8824944 · Cited by 63 · see citing works
Self-citationFang, H., Shrestha, A., Zhao, Z., & Qiu, Q. (2020) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/388 · Cited by 63 · see citing works
Ma, X., Yuan, G., Lin, S., Ding, C., Yu, F., Liu, T., Wen, W., Chen, X., & Wang, Y. (2020) · Asia and South Pacific Design Automation Conference (ASP-DAC) · doi:10.1109/asp-dac47756.2020.9045658 · Cited by 56 · see citing works
Self-citationDual-stream Network for Visual RecognitionPreprint
Mao, M., Zhang, R., Zheng, H., Gao, P., Ma, T., Peng, Y., Ding, E., Zhang, B., & Han, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.14734 · Cited by 55 · see citing works
Refiner: Refining Self-attention for Vision TransformersPreprint
Zhou, D., Shi, Y., Kang, B., Yu, W., Jiang, Z., Li, Y., Jin, X., Hou, Q., & Feng, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.03714 · Cited by 53 · see citing works
Unified Visual Transformer CompressionPreprint
Yu, S., Chen, T., Shen, J., Yuan, H., Tan, J., Yang, S., Liu, J., & Wang, Z. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2203.08243 · Cited by 52 · see citing works
Encoding, model, and architecture
Fang, H., Mei, Z., Shrestha, A., Zhao, Z., Li, Y., & Qiu, Q. (2020) · doi:10.1145/3400302.3415608 · Cited by 52 · see citing works
MEST: Accurate and Fast Memory-Economic Sparse Training Framework on the EdgePreprint
Yuan, G., Ma, X., Niu, W., Li, Z., Kong, Z., Liu, N., Gong, Y., Zhan, Z., He, C., Jin, Q., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.14032 · Cited by 52 · see citing works
Self-citationLi, B., Kong, Z., Zhang, T., Li, J., Li, Z., Liu, H., & Ding, C. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.286 · Cited by 48 · see citing works
Self-citationStructADMM: Achieving Ultrahigh Efficiency in Structured Pruning for DNNs
Zhang, T., Ye, S., Feng, X., Ma, X., Zhang, K., Li, Z., Tang, J., Liu, S., Lin, X., Liu, Y., et al. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3045153 · Cited by 48 · see citing works
Self-citationYuan, G., Liao, Z., Ma, X., Cai, Y., Kong, Z., Shen, X., Fu, J., Li, Z., Zhang, C., Peng, H., et al. (2021) · International Symposium on Quality Electronic Design (ISQED) · doi:10.1109/isqed51717.2021.9424332 · Cited by 39 · see citing works
Self-citationAn Image Enhancing Pattern-Based Sparsity for Real-Time Inference on Mobile Devices
Ma, X., Niu, W., Zhang, T., Liu, S., Lin, S., Li, H., Wen, W., Chen, X., Tang, J., Ma, K., et al. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58601-0_37 · Cited by 33 · see citing works
Self-citationPCNN: Pattern-based Fine-Grained Regular Pruning Towards Optimizing CNN Accelerators
Tan, Z., Song, J., Ma, X., Tan, S.-H., Chen, H., Miao, Y., Wu, Y., Ye, S., Wang, Y., Li, D., et al. (2020) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1109/dac18072.2020.9218498 · Cited by 31 · see citing works
Self-citationTinyADC: Peripheral Circuit-aware Weight Pruning Framework for Mixed-signal DNN Accelerators
Yuan, G., Behnam, P., Cai, Y., Shafiee, A. R., Fu, J., Liao, Z., Li, Z., Ma, X., Deng, J., Wang, J., et al. (2021) · Design, Automation, and Test in Europe · doi:10.23919/date51398.2021.9474235 · Cited by 30 · see citing works
Self-citationT6D-Direct: Transformers for Multi-object 6D Pose Direct Regression
Amini, A., Periyasamy, A. S., & Behnke, S. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-92659-5_34 · Cited by 29 · see citing works
A Unified DNN Weight Pruning Framework Using Reweighted Optimization Methods
Zhang, T., Ma, X., Zhan, Z., Zhou, S., Ding, C., Fardad, M., & Wang, Y. (2021) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1109/dac18074.2021.9586152 · Cited by 28 · see citing works
Self-citationAccelerating Sparse CNN Inference on GPUs with Performance-Aware Weight Pruning
Rumi, M. A., Ma, X., Wang, Y., & Jiang, P. (2020) · International Conference on Parallel Architectures and Compilation Techniques (PACT) · doi:10.1145/3410463.3414648 · Cited by 23 · see citing works
Self-citationA Privacy-Preserving-Oriented DNN Pruning and Mobile Acceleration Framework
Gong, Y., Zhan, Z., Li, Z., Niu, W., Ma, X., Wang, W., Ren, B., Ding, C., Lin, X., Xu, X., et al. (2020) · ACM Great Lakes Symposium on VLSI (GLSVLSI) · doi:10.1145/3386263.3407650 · Cited by 20 · see citing works
Self-citationNiu, W., Li, Z., Ma, X., Dong, P., Zhou, G., Qian, X., Lin, X., Wang, Y., & Ren, B. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3089687 · Cited by 19 · see citing works
Self-citationXu, C., Zhai, B., Wu, B., Li, T., Zhan, W., Vajda, P., Keutzer, K., & Tomizuka, M. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9636858 · Cited by 19 · see citing works
Effective Model Sparsification by Scheduled Grow-and-Prune MethodsPreprint
Ma, X., Qin, M., Sun, F., Hou, Z., Yuan, K., Xu, Y. T., Wang, Y., Chen, Y.-K., Jin, R., & Xie, Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.09857 · Cited by 17 · see citing works
Self-citationNeuromorphic Algorithm-hardware Codesign for Temporal Pattern Learning
Fang, H., Taylor, B., Li, Z., Mei, Z., Li, H. H., & Qiu, Q. (2021) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1109/dac18074.2021.9586133 · Cited by 17 · see citing works
Neural Network Pruning and Fast Training for DRL-based UAV Trajectory Planning
Li, Y., Fang, H., Li, M., Ma, Y., & Qiu, Q. (2022) · Asia and South Pacific Design Automation Conference (ASP-DAC) · doi:10.1109/asp-dac52403.2022.9712561 · Cited by 9 · see citing works
BLCR: Towards Real-time DNN Execution with Block-based Reweighted Pruning
Ma, X., Yuan, G., Li, Z., Gong, Y., Zhang, T., Niu, W., Zhan, Z., Zhao, P., Liu, N., Tang, J., et al. (2022) · International Symposium on Quality Electronic Design (ISQED) · doi:10.1109/isqed54688.2022.9806237 · Cited by 8 · see citing works
Self-citationA Compression-Compilation Framework for On-mobile Real-time BERT Applications
Niu, W., Kong, Z., Yuan, G., Jiang, W., Guan, J., Ding, C., Zhao, P., Liu, S., Ren, B., & Wang, Y. (2021) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/712 · Cited by 3 · see citing works
Self-citation
Flagged first, then by citations
5 of the 84 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
TinyViT: Fast Pretraining Distillation for Small Vision Transformers
Wu, K., Zhang, J., Peng, H., Liu, M., Xiao, B., Fu, J., & Yuan, L. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19803-8_5 · Cited by 480 · see citing works
A Survey on Efficient Vision Transformers: Algorithms, Techniques, and Performance Benchmarking
Papa, L., Russo, P., Amerini, I., & Zhou, L. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3392941 · Cited by 177 · see citing works
Chen, L., Zhao, H., Liu, T., Bai, S., Lin, J., Zhou, C., & Chang, B. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73004-7_2 · Cited by 156 · see citing works
Less is More: Focus Attention for Efficient DETR
Zheng, D., Dong, W., Hu, H., Chen, X., & Wang, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00614 · Cited by 152 · see citing works
Self-citationHeatViT: Hardware-Efficient Adaptive Token Pruning for Vision Transformers
Dong, P., Sun, M., Lu, A., Xie, Y., Liu, K., Kong, Z., Meng, X., Li, Z., Lin, X., Fang, Z., et al. (2023) · 2023 IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca56546.2023.10071047 · Cited by 128 · see citing works
Self-citationHydra Attention: Efficient Attention with Many Heads
Bolya, D., Fu, C.-Y., Dai, X., Zhang, P., & Hoffman, J. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-25082-8_3 · Cited by 85 · see citing works
PPT: Token-Pruned Pose Transformer for Monocular and Multi-view Human Pose Estimation
Ma, H., Wang, Z., Chen, Y., Kong, D., Chen, L., Liu, X., Yan, X., Tang, H., & Xie, X. S. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20065-6_25 · Cited by 84 · see citing works
Self-citationJoint Token Pruning and Squeezing Towards More Aggressive Compression of Vision Transformers
Wei, S., Ye, T., Zhang, S., Tang, Y., & Liang, J. (2023) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr52729.2023.00208 · Cited by 152 · see citing works
Token Fusion: Bridging the Gap between Token Pruning and Token Merging
Kim, M., Gao, S., Hsu, Y.-C., Shen, Y., & Jin, H. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00141 · Cited by 76 · see citing works
Hourglass Tokenizer for Efficient Transformer-Based 3D Human Pose Estimation
Li, W., Liu, M., Liu, H., Wang, P., Cai, J., & Sebe, N. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00064 · Cited by 69 · see citing works
Token Merging: Your ViT But Faster
Bolya, D., Fu, C.-Y., Dai, X., Zhang, P., Feichtenhofer, C., & Hoffman, J. (2022) · arXiv · doi:10.48550/arxiv.2210.09461 · Cited by 140 · see citing works
“4, we compare ToMe to token pruning works that use DeiT-S training3: A-ViT (Yin et al., 2022), DynamicViT (Rao et al., 2021), and SP-ViT (Kong et al., 2022) with throughput measured on a V100.”
BackgroundMethodInfluentialViTs as backbones: Leveraging vision transformers for feature extraction
Elharrouss, O., Himeur, Y., Mahmood, Y., Alrabaee, S., Ouamane, A., Bensaali, F., Bechqito, Y., & Chouchane, A. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.102951 · Cited by 59 · see citing works
Saha, S., & Xu, L. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.130417 · Cited by 66 · see citing works
SegViT v2: Exploring Efficient and Continual Semantic Segmentation with Plain Vision Transformers
Zhang, B., Liu, L., Phan, V. M. H., Tian, Z., Shen, C., & Liu, Y. (2023) · International Journal of Computer Vision · doi:10.1007/s11263-023-01894-8 · Cited by 54 · see citing works
Sarkar, R., Liang, H., Fan, Z., Wang, Z., & Hao, C. (2023) · 2023 IEEE/ACM International Conference on Computer Aided Design (ICCAD) · doi:10.1109/iccad57390.2023.10323651 · Cited by 56 · see citing works
Sparsity in transformers: A systematic literature review
Farina, M., Ahmad, U., Taha, A. T. M., Younes, H., Mesbah, Y., Yu, X., & Pedrycz, W. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.127468 · Cited by 55 · see citing works
Which Tokens to Use? Investigating Token Reduction in Vision Transformers
Haurum, J. B., Escalera, S., Taylor, G. W., & Moeslund, T. B. (2023) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw60793.2023.00085 · Cited by 47 · see citing works
TORE: Token Reduction for Efficient Human Mesh Recovery with Transformer
Dou, Z., Wu, Q., Lin, C. W., Cao, Z., Wu, Q., Wan, W., Komura, T., & Wang, W. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01390 · Cited by 47 · see citing works
Dynamic Neural Network Structure: A Review for its Theories and Applications
Guo, J., Chen, C. L. P., Liu, Z., & Yang, X. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3377194 · Cited by 47 · see citing works
Resource-efficient Algorithms and Systems of Foundation Models: A Survey
Xu, M., Cai, D., Yin, W., Wang, S., Jin, X., & Liu, X. (2024) · ACM Computing Surveys · doi:10.1145/3706418 · Cited by 152 · see citing works
DiffRate : Differentiable Compression Rate for Efficient Vision Transformers
Chen, M., Shao, W., Xu, P., Lin, M., Zhang, K., Chao, F., Ji, R., Qiao, Y., & Luo, P. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01574 · Cited by 43 · see citing works
Wang, H., Dedhia, B., & Jha, N. K. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01521 · Cited by 43 · see citing works
Scene Adaptive Sparse Transformer for Event-based Object Detection
Peng, Y., Li, H., Zhang, Y., Sun, X., & Wu, F. H. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01589 · Cited by 45 · see citing works
RIFormer: Keep Your Vision Backbone Effective But Removing Token Mixer
Wang, J., Zhang, S., Liu, Y., Wu, T., Yang, Y., Liu, X., Chen, K., Luo, P., & Lin, D. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01388 · Cited by 38 · see citing works
Interaction Transformer for Human Reaction Generation
Chopin, B., Tang, H., Otberdout, N., Daoudi, M., & Sebe, N. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3242152 · Cited by 39 · see citing works
Self-citationSTPrivacy: Spatio-Temporal Privacy-Preserving Action Recognition
Li, M., Xu, X., Fan, H., Zhou, P., Liu, J., Liu, J., Li, J., Keppo, J., Shou, M. Z., & Yan, S. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00471 · Cited by 36 · see citing works
Revisiting Token Pruning for Object Detection and Instance Segmentation
Liu, Y., Gehrig, M., Messikommer, N., Cannici, M., & Scaramuzza, D. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00264 · Cited by 40 · see citing works
vid-TLDR: Training Free Token merging for Light-Weight Video Transformer
Choi, J., Lee, S., Chu, J., Choi, M., & Kim, H. J. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01776 · Cited by 35 · see citing works
Fu, X., Ren, Q., Wu, H., Xiang, F., Luo, Q., Yue, J., Chen, Y., & Zhang, F. (2023) · IEEE Transactions on Circuits and Systems I Regular Papers · doi:10.1109/tcsi.2023.3315060 · Cited by 28 · see citing works
Zhang, X., Zeng, F., & Gu, C. (2024) · Neural Networks · doi:10.1016/j.neunet.2024.107059 · Cited by 25 · see citing works
Norouzi, N., Orlova, S., de Geus, D., & Dubbelman, G. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01493 · Cited by 22 · see citing works
MaskGaussian: Adaptive 3D Gaussian Representation from Probabilistic Masks
Liu, Y., Zhong, Z., Zhan, Y., Xu, S., & Sun, X. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00072 · Cited by 21 · see citing works
Dynamic Focus-aware Positional Queries for Semantic Segmentation
He, H., Cai, J., Pan, Z., Liu, J., Zhang, J., Tao, D., & Zhuang, B. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01087 · Cited by 21 · see citing works
Metal Surface Defect Detection Based on a Transformer with Multi-Scale Mask Feature Fusion
Zhao, L., Zheng, Y., Peng, T., & Zheng, E. (2023) · Sensors · doi:10.3390/s23239381 · Cited by 21 · see citing works
How can objects help action recognition?
Zhou, X., Arnab, A., Sun, C., & Schmid, C. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00233 · Cited by 20 · see citing works
Dynamic Feature Pruning and Consolidation for Occluded Person Re-identification
Ye, Y., Zhou, H., Cai, J., Gao, C., Zhang, Y., Wang, J., Hu, Q., Yu, J., & Yang, W. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i7.28491 · Cited by 24 · see citing works
ELSA: Exploiting Layer-wise N:M Sparsity for Vision Transformer Acceleration
Huang, N.-C., Chang, C.-C., Lin, W., Taka, E., Marculescu, D., & Wu, K. H. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00799 · Cited by 18 · see citing works
ReViT: Vision Transformer Accelerator With Reconfigurable Semantic-Aware Differential Attention
Zou, X.-F., Chen, C., Shao, H.-G., Wang, Q.-Y., Zhuang, X., Li, Y.-F., & Li, K.-Q. (2025) · IEEE transactions on computers · doi:10.1109/tc.2024.3504263 · Cited by 19 · see citing works
GTP-ViT: Efficient Vision Transformers via Graph-based Token Propagation
Xu, X., Wang, S., Chen, Y., Zheng, Y., Wei, Z., & Liu, J. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00016 · Cited by 18 · see citing works
Multi-Criteria Token Fusion with One-Step-Ahead Attention for Efficient Vision Transformers
Lee, S., Choi, J., & Kim, H. J. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01490 · Cited by 19 · see citing works
LPViT: Low-Power Semi-structured Pruning for Vision Transformers
Xu, K., Wang, Z., Chen, C., Geng, X., Lin, J., Yang, X., Wu, M., Li, X., & Lin, W. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73209-6_16 · Cited by 15 · see citing works
Mao, J., Shen, Y., Guo, J., Yao, Y., Hua, X., & Shen, H. T. (2025) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2025.3535405 · Cited by 15 · see citing works
Accelerating ViT Inference on FPGA through Static and Dynamic Pruning
Parikh, D., Li, S., Zhang, B., Kannan, R., Busart, C. E., & Prasanna, V. K. (2024) · IEEE Symposium on Field-Programmable Custom Computing Machines · doi:10.1109/fccm60383.2024.00018 · Cited by 17 · see citing works
Prototype-Driven Multi-Feature Generation for Visible-Infrared Person Re-identification
Li, J., Zhen, Q., Yang, Y., Li, Y., Dong, Z., & Yang, C. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10889917 · Cited by 14 · see citing works
Segformer++: Efficient Token-Merging Strategies for High-Resolution Semantic Segmentation
Kienzle, D., Kantonis, M., Schön, R., & Lienhart, R. W. (2024) · IEEE International Conference on Multimedia Information Processing and Retrieval (MIPR) · doi:10.1109/mipr62202.2024.00019 · Cited by 13 · see citing works
Leveraging Temporal Contextualization for Video Action Recognition
Kim, M., Han, D., Kim, T., & Han, B. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72664-4_5 · Cited by 13 · see citing works
PELA: Learning Parameter-Efficient Models with Low-Rank Approximation
Guo, Y., Wang, G., & Kankanhalli, M. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01486 · Cited by 13 · see citing works
Efficient Transformer-based 3D Object Detection with Dynamic Token Halting
Ye, M., Meyer, G. P., Chai, Y., & Liu, Q. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00775 · Cited by 11 · see citing works
IVTP: Instruction-Guided Visual Token Pruning for Large Vision-Language Models
Huang, K., Zou, H., Ye, X., Wang, B., Xie, Z., & Liang, Y. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72643-9_13 · Cited by 11 · see citing works
Survey on Efficient Large Language Models: Principles, Algorithms, Applications, and Open Issues
Cheng, J., Kang, H., Shao, Y., Li, N., Chen, P., Wang, R., Long, S., Yang, X., & Ma, L. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2025.3628671 · Cited by 11 · see citing works
Expediting Large-Scale Vision Transformer for Dense Prediction Without Fine-Tuning
Yuan, Y., Liang, W., Ding, H., Liang, Z., Zhang, C., & Hu, H. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3327511 · Cited by 14 · see citing works
Remote Sensing Scene Classification via Second-Order Differentiable Token Transformer Network
Ni, K., Wu, Q., Li, S., Zheng, Z., & Wang, P. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3407879 · Cited by 9 · see citing works
A pure MLP-Mixer-based GAN framework for guided image translation
Tang, H., Ren, B., & Sebe, N. (2024) · Pattern Recognition · doi:10.1016/j.patcog.2024.110894 · Cited by 9 · see citing works
Self-citationAgglomerative Token Clustering
Haurum, J. B., Escalera, S., Taylor, G. W., & Moeslund, T. B. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72998-0_12 · Cited by 9 · see citing works
AdapTiV: Sign-Similarity Based Image-Adaptive Token Merging for Vision Transformer Acceleration
Yoo, S.-J., Kim, H., & Kim, J. (2024) · IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1109/micro61859.2024.00015 · Cited by 9 · see citing works
Deep Joint Source Channel Coding With Attention Modules Over MIMO Channels
Jiang, W., Chen, W., & Ai, B. (2024) · IEEE Vehicular Technology Conference · doi:10.1109/vtc2024-spring62846.2024.10683289 · Cited by 8 · see citing works
Zhu, L., Chen, T., Ji, D., Ye, J., & Liu, J. (2025) · IEEE Transactions on Image Processing · doi:10.1109/tip.2025.3531299 · Cited by 8 · see citing works
TC-MGC: Text-conditioned multi-grained contrastive learning for text–video retrieval
Jing, X., Yang, G., & Chu, J. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103151 · Cited by 8 · see citing works
Multi-scale motion perception fall detection algorithm based on video swin transformer
Cai, X., Wang, X., Yin, J., Chen, Y., & Han, G. (2025) · Signal Image and Video Processing · doi:10.1007/s11760-025-04358-3 · Cited by 8 · see citing works
A General and Efficient Training for Transformer via Token Expansion
Huang, W., Shen, Y., Xie, J., Zhang, B., He, G., Li, K., Sun, X., & Lin, S. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01494 · Cited by 8 · see citing works
LaTP: LiDAR-aided multimodal token pruning for efficient trajectory prediction of autonomous driving
Lu, Y., Sun, S., Liu, N., Jiang, B., Li, Y., Chen, J., Zhang, Y., Zhu, Y., & Velipasalar, S. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.107673 · Cited by 7 · see citing works
Efficient Vision Transformer with Token Sparsification for Event-Based Object Tracking
Zhang, J., Yang, X., Tang, H., Wang, Y., Yin, B., Wang, H., & Fu, X. (2026) · International Journal of Computer Vision · doi:10.1007/s11263-025-02666-2 · Cited by 7 · see citing works
Self-citationYou Already Have It: A Generator-Free Low-Precision DNN Training Framework Using Stochastic Rounding
Yuan, G., Chang, S.-E., Jin, Q., Lu, A., Li, Y., Wu, Y., Kong, Z., Xie, Y., Dong, P., Qin, M., et al. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19775-8_3 · Cited by 7 · see citing works
Self-citationNo Token Left Behind: Efficient Vision Transformer via Dynamic Token Idling
Xu, X., Li, C., Chen, Y., Chang, X., Liu, J., & Wang, S. (2023) · Lecture notes in computer science · doi:10.1007/978-981-99-8388-9_3 · Cited by 7 · see citing works
Do we really need a large number of visual prompts?
Kim, Y., Li, Y., Moitra, A., Yin, R., & Panda, P. (2024) · Neural Networks · doi:10.1016/j.neunet.2024.106390 · Cited by 7 · see citing works
Song, Y., Qian, P., Zhang, K., Liu, S., Zhai, R., & Song, R. (2025) · Multimedia Systems · doi:10.1007/s00530-025-01746-0 · Cited by 7 · see citing works
Zebra: Leveraging Diagonal Attention Pattern for Vision Transformer Accelerator
Han, S., Kim, S., Byeon, G., Yoon, J., & Hong, S. (2025) · Design, Automation, and Test in Europe · doi:10.23919/date64628.2025.10993213 · Cited by 7 · see citing works
KFF: K-feature fusion token merging for vision transformer
Yang, Y., Zhou, Y., Hu, X., & Duan, S. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.128206 · Cited by 6 · see citing works
Efficient Visual Transformer by Learnable Token Merging
Wang, Y., & Yang, Y. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3588186 · Cited by 6 · see citing works
Self-citationGuo, D., Wu, Z., Feng, J., Zhou, Z., & Shen, Z. (2023) · Applied Intelligence · doi:10.1007/s10489-023-04725-y · Cited by 6 · see citing works
ATFTrans: attention-weighted token fusion transformer for robust and efficient object tracking
Xu, L., Wang, L., & Guo, Z. (2024) · Neural Computing and Applications · doi:10.1007/s00521-024-09444-0 · Cited by 6 · see citing works
Efficient Transformer Adaptation with Soft Token Merging
Yuan, X., Fei, H., & Baek, J. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00369 · Cited by 6 · see citing works
Data Level Lottery Ticket Hypothesis for Vision Transformers
Shen, X., Kong, Z., Qin, M., Dong, P., Yuan, G., Meng, X., Tang, H., Ma, X., & Wang, Y. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/153 · Cited by 6 · see citing works
Self-citationPYRA: Parallel Yielding Re-activation for Training-Inference Efficient Task Adaptation
Xiong, Y., Chen, H., Hao, T., Lin, Z., Han, J., Zhang, Y., Wang, G., Bao, Y., & Ding, G. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72673-6_25 · Cited by 9 · see citing works
UCC: A unified cascade compression framework for vision transformer models
Chen, D., Lin, K., & Deng, Q. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128747 · Cited by 5 · see citing works
MCANet: a lightweight action recognition network with multidimensional convolution and attention
Tian, Q., Miao, W., Zhang, L., Yang, Z., Yu, Y., Zhao, Y., & Yao, L. (2024) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-024-02454-3 · Cited by 5 · see citing works
Squat: Quant Small Language Models on the Edge
Shen, X., Dong, P., Kong, Z., Gong, Y., Yang, C., Han, Z., Xie, Y., Lu, L., Lyu, C., Wu, C., et al. (2025) · doi:10.1109/iccad66269.2025.11240685 · Cited by 5 · see citing works
Self-citationThe Role of Masking for Efficient Supervised Knowledge Distillation of Vision Transformers
Son, S., Ryu, J., Lee, N., & Lee, J. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72855-6_22 · Cited by 5 · see citing works
Raformer: Redundancy-Aware Transformer for Video Wire Inpainting
Ji, Z., Su, Y., Zhang, Y., Hou, J., Pang, Y., & Han, J. (2025) · IEEE Transactions on Image Processing · doi:10.1109/tip.2025.3550038 · Cited by 5 · see citing works
Famba-V: Fast Vision Mamba with Cross-Layer Token Fusion
Shen, H., Wan, Z., Wang, X., & Zhang, M. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-91979-4_20 · Cited by 5 · see citing works
PRANCE: Joint Token-Optimization and Structural Channel-Pruning for Adaptive ViT Inference
Li, Y., Tang, C., Meng, Y., Fan, J., Chai, Z., Ma, X., Wang, Z., & Zhu, W. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3605239 · Cited by 4 · see citing works
Self-citationImportance-Based Token Merging for Efficient Image and Video Generation
Wu, H., Xu, J., Le, H., & Samaras, D. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00474 · Cited by 4 · see citing works
Luo, J., Zhang, Y., Yang, X., Wu, K., Zhu, Q., Liang, L., Chen, J., & Li, Y. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00860 · Cited by 7 · see citing works
PaPr: Training-Free One-Step Patch Pruning with Lightweight ConvNets for Faster Inference
Mahmud, T., Yaman, B., Liu, C.-H., & Marculescu, D. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73337-6_7 · Cited by 4 · see citing works
Sparse Refinement for Efficient High-Resolution Semantic Segmentation
Liu, Z., Zhang, Z., Khaki, S., Yang, S. W., Tang, H., Xu, C., Keutzer, K., & Han, S. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72855-6_7 · Cited by 4 · see citing works
Self-citationZhang, X., Dai, X., Zhang, Z., Di, X., & Chen, X. (2024) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2024.3383542 · Cited by 4 · see citing works
Partitioned token fusion and pruning strategy for transformer tracking
Zhang, C., Gao, Y. T., Meng, T., & Wang, T. (2025) · Image and Vision Computing · doi:10.1016/j.imavis.2025.105431 · Cited by 4 · see citing works
VLTP: Vision-Language Guided Token Pruning for Task-Oriented Segmentation
Chen, H., Ni, Y., Huang, W., Liu, Y., Jeong, S., Wen, F., Bastian, N. D., Latapie, H. M., & Imani, M. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00906 · Cited by 4 · see citing works
Compressing Vision Transformer from the View of Model Property in Frequency Domain
Wang, Z., Xie, X., Luo, H., Huang, T., Dong, W., Xiong, K., Liu, Y., Li, X., Wang, F., & Shi, G. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02561-w · Cited by 3 · see citing works
HyAtt: A Hybrid Attention Accelerator for Transformers With Analog In-Memory Computing
Moradifirouzabadi, A., & Kang, M. (2025) · IEEE Journal of Solid-State Circuits · doi:10.1109/jssc.2025.3602940 · Cited by 3 · see citing works
Simultaneous image patch attention and pruning for patch selective transformer
Kim, S., Yoon, G.-J., Song, J., & Yoon, S. M. (2024) · Image and Vision Computing · doi:10.1016/j.imavis.2024.105239 · Cited by 3 · see citing works
Neighbor patches merging reduces spatial redundancy to accelerate vision transformer
Jiang, K., Peng, P., Lian, Y., Shao, W., & Xu, W. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128733 · Cited by 3 · see citing works
Make Your ViT-Based Multi-view 3D Detectors Faster via Token Compression
Zhang, D., Liang, D., Tan, Z., Ye, X., Zhang, C., Wang, J., & Bai, X. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72970-6_4 · Cited by 3 · see citing works
Multi-Faceted Adaptive Token Pruning for Efficient Remote Sensing Image Segmentation
Zhang, C., & Yao, J. (2025) · Remote Sensing · doi:10.3390/rs17142508 · Cited by 3 · see citing works
Enhancing spatiotemporal predictive learning: an approach with nested attention module
Wang, S., & Ren, H. (2024) · Journal of Intelligent Manufacturing · doi:10.1007/s10845-023-02318-7 · Cited by 3 · see citing works
Bandit-based Attention Mechanism in Vision Transformers
Chowdhury, A. R., Diddigi, R. B., Prabuchandran, K. J., & Tripathi, A. M. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00929 · Cited by 3 · see citing works
Is Less More? Exploring Token Condensation as Training-Free Test-Time Adaptation
Wang, Z., Gong, D., Wang, S., Huang, Z. H., & Luo, Y. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00021 · Cited by 2 · see citing works
VLDrive: Vision-Augmented Lightweight MLLMs for Efficient Language-Grounded Autonomous Driving
Zhang, R., Zhang, W., Tan, X., Yang, S., Wan, X., Luo, X., & Li, G. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00560 · Cited by 2 · see citing works
Similarity-Aware Fast Low-Rank Decomposition Framework for Vision Transformers
Luo, Y.-J., Tai, Y.-S., Lin, M.-G., & Wu, A.-Y. A. (2024) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas58744.2024.10557934 · Cited by 2 · see citing works
HASS: Hardware-Aware Sparsity Search for Dataflow DNN Accelerator
Yu, Z., Sreeram, S., Agrawal, K., Wu, J., Montgomerie-Corcoran, A., Zhang, C., Cheng, J., Bouganis, C.-S., & Zhao, Y. (2024) · International Conference on Field Programmable Logic and Applications (FPL) · doi:10.1109/fpl64840.2024.00043 · Cited by 2 · see citing works
- Authors
- Kong, Z., Dong, P., Ma, X., Meng, X., Niu, W., Sun, M., Shen, X., Yuan, G., Ren, B., Tang, H., Qin, M., & Wang, Y.
- Affiliations
- Northeastern University (US)
- Clemson University (US)
- Peking University (CN)
- William & Mary (US)
- ETH Zurich (CH)
- Corresponding author
- Wang, Yanzhi
- ORCID
- Venue
- Lecture Notes in Computer Science
- Publisher
- Springer Nature Switzerland
- Published
- 2022
- Pages
- 620-640
- ISSN
- 0302-9743 (ISSN-L)
- 1611-3349
- OpenAlex
- W4312910119
- Cited by
- 169
- References
- 84
- Citations by year
- 2026: 25 · 2025: 61 · 2024: 55 · 2023: 25 · 2022: 3
- FWCI
- 13.06
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Lecture notes in computer science (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Keywords
- vision transformer, token selection, multi-head attention, edge deployment, mobile devices, FPGA, real-time inference, DeiT, Swin Transformer, computation cost reduction, accuracy-latency trade-off
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
