EfficientViT: Memory Efficient Vision Transformer with Cascaded Group Attention
Open accessAbstract
Vision transformers have shown great success due to their high model capabilities. However, their remarkable performance is accompanied by heavy computation costs, which makes them unsuitable for real-time applications. In this paper, we propose a family of high-speed vision transformers named Efficient ViT. We find that the speed of existing transformer models is commonly bounded by memory inefficient operations, especially the tensor reshaping and element-wise functions in MHSA. Therefore, we design a new building block with a sandwich layout, i.e., using a single memory-bound MHSA between efficient FFN layers, which improves memory efficiency while enhancing channel communication. Moreover, we discover that the attention maps share high similarities across heads, leading to computational redundancy. To address this, we present a cascaded group attention module feeding attention heads with different splits of the full feature, which not only saves computation cost but also improves attention diversity. Comprehensive experiments demonstrate EfficientViT outperforms existing efficient models, striking a good trade-off between speed and accuracy. For instance, our EfficientViT-M5 surpasses MobileNetV3-Large by 1.9% in accuracy, while getting 40.4% and 45.2% higher throughput on Nvidia V100 GPU and Intel Xeon CPU, respectively. Compared to the recent efficient model MobileViT-XXS, EfficientViT-M2 achieves 1.8% superior accuracy, while running$5.8\times/3.7\times$faster on the GPU/CPU, and$7.4\times faster$when converted to ONNX format. Code and models are available at here.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Unsupervised learning is essential for uncovering patterns within unlabeled data, central to tasks like feature learning and clustering.
“Unsupervised learning is essential in AI for uncovering patterns within unlabeled data, a process central to tasks like feature learning and clustering [169], [170].”
From From Google Gemini to OpenAI Q* (Q-Star): A Survey on Reshaping the Generative Artificial Intelligence (AI) Research Landscape (2025) · cites this paper as [170]
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
Jakubův, J., Chvalovský, K., Goertzel, Z., et al. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,648 · see citing works
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · see citing works
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460 · see citing works
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534 · see citing works
Self-citationRethinking the Inception Architecture for Computer Vision
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.308 · Cited by 32,138 · see citing works
PyTorch: An Imperative Style, High-Performance Deep Learning Library
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al. (2019) · arXiv · doi:10.48550/arxiv.1912.01703 · Cited by 31,473 · see citing works
Focal Loss for Dense Object Detection
Lin, T.-Y., Goyal, P., Girshick, R., He, K., & Dollar, P. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.324 · Cited by 30,581 · see citing works
MobileNetV2: Inverted Residuals and Linear Bottlenecks
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L.-C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00474 · Cited by 28,755 · see citing works
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818 · see citing works
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,890 · see citing works
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 21,793 · see citing works
Xception: Deep Learning with Depthwise Separable Convolutions
Chollet, F. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.195 · Cited by 20,380 · see citing works
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., & Adam, H. (2017) · arXiv · doi:10.48550/arxiv.1704.04861 · Cited by 19,015 · see citing works
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
Tan, M., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1905.11946 · Cited by 17,072 · see citing works
(2010) · International Conference on Machine Learning · Cited by 13,995 · see citing works
Decoupled Weight Decay Regularization
Loshchilov, I., & Hutter, F. (2017) · arXiv · doi:10.48550/arxiv.1711.05101 · Cited by 12,522 · see citing works
ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
Zhang, X., Zhou, X., Lin, M., & Sun, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00716 · Cited by 9,982 · see citing works
Howard, A., Sandler, M., Chen, B., Wang, W., Chen, L.-C., Tan, M., Chu, G., Vasudevan, V., Zhu, Y., Pang, R., et al. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00140 · Cited by 9,708 · see citing works
ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design
Ma, N., Zhang, X., Zheng, H.-T., & Sun, J. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01264-9_8 · Cited by 7,356 · cited 3× · see citing works
“With the demand of deploying CNNs on resource-constrained scenarios, efficient CNNs have been intensively studied in literature [23, 24, 26, 48, 63, 67, 87].”
BackgroundMethodLearning Transferable Architectures for Scalable Image Recognition
Zoph, B., Vasudevan, V., Shlens, J., & Le, Q. V. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00907 · Cited by 6,631 · see citing works
mixup: Beyond Empirical Risk Minimization
Zhang, H., Cisse, M., Dauphin, Y. N., & Lopez-Paz, D. (2017) · arXiv · doi:10.48550/arxiv.1710.09412 · Cited by 5,831 · see citing works
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions
Wang, W., Xie, E., Li, X., Fan, D.-P., Song, K., Liang, D., Lu, T., Luo, P., & Shao, L. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00061 · Cited by 5,259 · cited 2× · see citing works
“Efficient self-attention methods reduce the cost of softmax attention via sparse attention [34, 57, 61, 75] or low-rank approximation [11, 51, 74].”
BackgroundMethodGhostNet: More Features From Cheap Operations
Han, K., Wang, Y., Tian, Q., Guo, J., Xu, C., & Xu, C. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00165 · Cited by 5,222 · see citing works
AutoAugment: Learning Augmentation Strategies From Data
Cubuk, E. D., Zoph, B., Mané, D., Vasudevan, V., & Le, Q. V. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00020 · Cited by 4,026 · see citing works
Random Erasing Data Augmentation
Zhong, Z., Zheng, L., Kang, G., Li, S., & Yang, Y. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.7000 · Cited by 3,617 · see citing works
Self-citationGaussian Error Linear Units (GELUs)
Hendrycks, D., & Gimpel, K. (2016) · arXiv · doi:10.48550/arxiv.1606.08415 · Cited by 3,537 · see citing works
3D Object Representations for Fine-Grained Categorization
Krause, J., Stark, M. M., Deng, J., & Fei-Fei, L. (2013) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2013.77 · Cited by 3,523 · see citing works
MnasNet: Platform-Aware Neural Architecture Search for Mobile
Tan, M., Chen, B., Pang, R., Vasudevan, V., Sandler, M., Howard, A., & Le, Q. V. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00293 · Cited by 3,416 · see citing works
(1989) · Cited by 3,214 · see citing works
Swin Transformer V2: Scaling Up Capacity and Resolution
Liu, Z., Hu, H., Lin, Y., Yao, Z., Xie, Z., Wei, Y., Ning, J., Cao, Y., Zhang, Z., Dong, L., et al. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01170 · Cited by 2,741 · see citing works
Self-citationSwin Transformer V2: Scaling Up Capacity and Resolution
Liu, Z., Hu, H., Lin, Y., Yao, Z., Xie, Z., Wei, Y., Ning, J., Cao, Y., Zhang, Z., Dong, L., et al. (2021) · arXiv · doi:10.48550/arxiv.2111.09883 · Cited by 2,679 · see citing works
Self-citationPvt v2: Improved baselines with pyramid vision transformer
Wang, W., Xie, E., Li, X., Fan, D.-P., Song, K., Liang, D., Lu, T., Luo, P., & Shao, L. (2022) · Computational Visual Media · doi:10.1007/s41095-022-0274-8 · Cited by 2,439 · see citing works
Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet
Yuan, L., Chen, Y., Wang, T., Yu, W., Shi, Y., Jiang, Z., Tay, F. E. H., Feng, J., & Yan, S. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00060 · Cited by 2,415 · see citing works
MMDetection: Open MMLab Detection Toolbox and Benchmark
Chen, K., Wang, J., Pang, J., Cao, Y., Xiong, Y., Li, X., Sun, S., Feng, W., Liu, Z., Xu, J., et al. (2019) · arXiv · doi:10.48550/arxiv.1906.07155 · Cited by 2,227 · see citing works
Designing Network Design Spaces
Radosavovic, I., Kosaraju, R. P., Girshick, R., He, K., & Dollar, P. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01044 · Cited by 2,084 · see citing works
Parkhi, O., Vedaldi, A., Zisserman, A., & Jawahar, C. V. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6248092 · Cited by 1,563 · see citing works
Reformer: The Efficient Transformer
Kitaev, N., Kaiser, Ł., & Levskaya, A. (2020) · arXiv · doi:10.48550/arxiv.2001.04451 · Cited by 1,435 · see citing works
MetaFormer is Actually What You Need for Vision
Yu, W., Luo, M., Zhou, P., Si, C., Zhou, Y., Wang, X., Feng, J., & Yan, S. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01055 · Cited by 1,380 · see citing works
Rethinking the Value of Network PruningPreprint
Liu, Z., Sun, M., Zhou, T., Huang, G. M., & Darrell, T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.05270 · Cited by 1,225 · see citing works
FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
Dao, T., Fu, D. Y., Ermon, S., Rudra, A., & Ré, C. (2022) · arXiv · doi:10.48550/arxiv.2205.14135 · Cited by 1,214 · cited 4× · see citing works
“The activation functions are element-wise operations that occupy a considerable amount of processing time on GPU/CPU [15,48,72], thus utilizing ReLU instead of more complicated activation functions is of better efficiency.”
BackgroundMethodInfluentialAnalyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned
Voita, E., Talbot, D., Moiseev, F., Sennrich, R., & Titov, I. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1580 · Cited by 1,146 · see citing works
MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer
Mehta, S., & Rastegari, M. (2021) · arXiv · doi:10.48550/arxiv.2110.02178 · Cited by 1,110 · see citing works
Importance Estimation for Neural Network Pruning
Molchanov, P., Mallya, A., Tyree, S., Frosio, I., & Kautz, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01152 · Cited by 983 · see citing works
LeViT: a Vision Transformer in ConvNet’s Clothing for Faster Inference
Graham, B., El-Nouby, A., Touvron, H., Stock, P., Joulin, A., Jegou, H., & Douze, M. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01204 · Cited by 935 · see citing works
CoAtNet: Marrying Convolution and Attention for All Data Sizes
Dai, Z., Liu, H., Le, Q. V., & Tan, M. (2021) · arXiv · doi:10.48550/arxiv.2106.04803 · Cited by 925 · see citing works
Single Path One-Shot Neural Architecture Search with Uniform Sampling
Guo, Z., Zhang, X., Mu, H., Heng, W., Liu, Z., Wei, Y., & Sun, J. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58517-4_32 · Cited by 882 · see citing works
A Visual Vocabulary for Flower Classification
Nilsback, M.-E., & Zisserman, A. (2006) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2006.42 · Cited by 876 · see citing works
Linformer: Self-Attention with Linear Complexity
Wang, S., Li, B. Z., Khabsa, M., Fang, H., & Ma, H. (2020) · arXiv · doi:10.48550/arxiv.2006.04768 · Cited by 867 · see citing works
Mobile-Former: Bridging MobileNet and Transformer
Chen, Y., Dai, X., Chen, D., Liu, M., Dong, X., Yuan, L., & Liu, Z. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00520 · Cited by 731 · see citing works
Rethinking Attention with Performers
Choromanski, K., Likhosherstov, V., Dohan, D., Song, X., Gane, A., Sarlos, T., Hawkins, P., Davis, J., Mohiuddin, A., Kaiser, L., et al. (2020) · arXiv · doi:10.48550/arxiv.2009.14794 · Cited by 695 · cited 2× · see citing works
“Although there are some methods proposed to address this issue by simplifying the computation of standard softmax self-attention, e.g. , sparse attention [34, 57, 61, 75] and low-rank approximation [11,51,74], they often come at the cost of accuracy degradation and limited acceleration.”
MethodEarly Convolutions Help Transformers See BetterPreprint
Xiao, T., Singh, M., Mintun, E., Darrell, T., Dollár, P., & Girshick, R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.14881 · Cited by 565 · see citing works
TinyViT: Fast Pretraining Distillation for Small Vision Transformers
Wu, K., Zhang, J., Peng, H., Liu, M., Xiao, B., Fu, J., & Yuan, L. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19803-8_5 · Cited by 480 · see citing works
Self-citationEfficientFormer: Vision Transformers at MobileNet Speed
Li, Y., Yuan, G., Wen, Y., Hu, J., Evangelidis, G., Tulyakov, S., Wang, Y., & Ren, J. (2022) · neural information processing systems · doi:10.52202/068431-0940 · Cited by 445 · see citing works
MixConv: Mixed Depthwise Convolutional KernelsPreprint
Tan, M., & Le, Q. V. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.09595 · Cited by 421 · see citing works
Edgenext: Efficiently amalgamated CNN-transformer architecture for mobile vision applications
Maaz, M., Shaker, A., Cholakkal, H., Khan, S., Zamir, S. W., Anwer, R. M., & Shahbaz Khan, F. (2023) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-25082-8_1 · Cited by 407 · see citing works
Are Sixteen Heads Really Better than One?
Michel, P., Levy, O., & Neubig, G. (2019) · arXiv · doi:10.48550/arxiv.1905.10650 · Cited by 404 · see citing works
FairNAS: Rethinking Evaluation Fairness of Weight Sharing Neural Architecture Search
Chu, X., Zhang, B., & Xu, R. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01202 · Cited by 358 · see citing works
Fast Vision Transformers with HiLo AttentionPreprint
Pan, Z.-Z., Cai, J.-F., & Zhuang, B.-H. (2022) · Neural Information Processing Systems · doi:10.48550/arxiv.2205.13213 · Cited by 327 · see citing works
AutoFormer: Searching Transformers for Visual Recognition
Chen, M., Peng, H., Fu, J., & Ling, H. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01205 · Cited by 323 · see citing works
Self-citationEdgeViTs: Competing Light-Weight CNNs on Mobile Devices with Vision Transformers
Pan, J., Bulat, A., Tan, F., Zhu, X., Dudziak, Ł., Li, H., Tzimiropoulos, G., & Martínez, B. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20083-0_18 · Cited by 302 · see citing works
Scaling Vision with Sparse Mixture of ExpertsPreprint
Riquelme, C., Puigcerver, J., Mustafa, B., Neumann, M., Jenatton, R., Pinto, A. S., Keysers, D., & Houlsby, N. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.05974 · Cited by 287 · see citing works
EfficientFormer: Vision Transformers at MobileNet SpeedPreprint
Li, Y., Yuan, G., Wen, Y., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2206.01191 · Cited by 254 · see citing works
Separable Self-attention for Mobile Vision Transformers
Mehta, S., & Rastegari, M. (2022) · arXiv · doi:10.48550/arxiv.2206.02680 · Cited by 215 · see citing works
Fast Vision Transformers with HiLo Attention
Pan, Z., Cai, J., & Zhuang, B. (2022) · neural information processing systems · doi:10.52202/068431-1057 · Cited by 207 · see citing works
GhostNets on Heterogeneous Devices via Cheap Operations
Han, K., Wang, Y., Xu, C., Guo, J., Xu, C., Wu, E., & Tian, Q. C. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-022-01575-y · Cited by 198 · see citing works
Training data-efficient image transformers & distillation through attention
Touvron, H., Cord, M., Douze, M., Massa, F., Sablayrolles, A., & Jégou, H. (2020) · arXiv · doi:10.48550/arxiv.2012.12877 · Cited by 182 · see citing works
Lite Vision Transformer with Enhanced Self-Attention
Yang, C., Wang, Y., Zhang, J., Zhang, H., Wei, Z., Lin, Z., & Yuille, A. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01169 · Cited by 164 · see citing works
Transformer Quality in Linear Time
Hua, W., Dai, Z., Liu, H., & Le, Q. V. (2022) · arXiv · doi:10.48550/arxiv.2202.10447 · Cited by 137 · see citing works
MiniViT: Compressing Vision Transformers with Weight Multiplexing
Zhang, J., Peng, H., Wu, K., Liu, M., Xiao, B., Fu, J., & Yuan, L. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01183 · Cited by 130 · see citing works
Self-citationGLiT: Neural Architecture Search for Global and Local Image Transformer
Chen, B., Li, P., Li, C., Li, B., Bai, L., Chen, L., Sun, M., Yan, J., & Ouyang, W. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00008 · Cited by 100 · see citing works
Data Movement Is All You Need: A Case Study on Optimizing Transformers
Ivanov, A., Dryden, N., Ben-Nun, T., Li, S., & Hoefler, T. (2020) · arXiv · doi:10.48550/arxiv.2007.00072 · Cited by 78 · see citing works
DS-Net++: Dynamic Weight Slicing for Efficient Inference in CNNs and Vision Transformers
Li, C., Wang, G., Wang, B., Liang, X., Li, Z., & Chang, X. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3194044 · Cited by 72 · see citing works
Towards Lightweight Transformer Via Group-Wise Transformation for Vision-and-Language Tasks
Luo, G., Zhou, Y., Sun, X., Wang, Y., Cao, L., Wu, Y., Huang, F., & Ji, R. (2022) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3139234 · Cited by 66 · see citing works
Pale Transformer: A General Vision Transformer Backbone with Pale-Shaped Attention
Wu, S., Wu, T., Tan, H., & Guo, G. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i3.20176 · Cited by 65 · see citing works
The Efficiency MisnomerPreprint
Dehghani, M., Arnab, A., Beyer, L., Vaswani, A., & Tay, Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.12894 · Cited by 58 · see citing works
Characterizing Co-Located Workloads in Alibaba Cloud Datacenters
Jiang, C., Qiu, Y., Shi, W., Ge, Z., Wang, J., Chen, S., Cérin, C., Ren, Z., Xu, G., & Lin, J. (2020) · IEEE Transactions on Cloud Computing · doi:10.1109/tcc.2020.3034500 · Cited by 51 · see citing works
LightViT: Towards Light-Weight Convolution-Free Vision TransformersPreprint
Huang, T., Huang, L., You, S., Wang, F., Qian, C., & Xu, C. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2207.05557 · Cited by 48 · see citing works
Combiner: Full Attention Transformer with Sparse Computation CostPreprint
Ren, H., Dai, H., Dai, Z., Yang, M., Leskovec, J., Schuurmans, D., & Dai, B. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.05768 · Cited by 36 · see citing works
Improving the Efficiency of Transformers for Resource-Constrained Devices
Tabani, H., Balasubramaniam, A., Marzban, S., Arani, E., & Zonooz, B. (2021) · Euromicro Conference on Digital System Design (DSD) · doi:10.1109/dsd53832.2021.00074 · Cited by 32 · see citing works
Auto-scaling Vision Transformers without TrainingPreprint
Chen, W.-Y., Huang, W., Du, X., Song, X., Wang, Z.-Y., & Zhou, D. (2022) · International Conference on Learning Representations · doi:10.48550/arxiv.2202.11921 · Cited by 29 · see citing works
NViT: Vision Transformer Compression and Parameter Redistribution.Preprint
Yang, H., Yin, H., Molchanov, P. A., Li, H., & Kautz, J. (2021) · arXiv (Cornell University) · Cited by 28 · see citing works
SWIRL: High-performance many-core CPU code generation for deep neural networks
Venkat, A., Rusira, T., Barik, R., Hall, M. W., & Truong, L. (2019) · The International Journal of High Performance Computing Applications · doi:10.1177/1094342019866247 · Cited by 24 · see citing works
Towards Memory-Efficient Neural Networks via Multi-Level in situ Generation
Gu, J., Zhu, H., Feng, C., Liu, M., Jiang, Z., Chen, R. T., & Pan, D. Z. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00518 · Cited by 4 · see citing works
An Optimized Dataflow for Mitigating Attention Performance BottlenecksPreprint
Kao, S.-C., Subramanian, S., Agrawal, G., & Krishna, T. (2021) · arXiv (Cornell University) · Cited by 2 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Rep ViT: Revisiting Mobile CNN From ViT Perspective
Wang, A., Chen, H., Lin, Z., Han, J., & Ding, G. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01506 · Cited by 752 · see citing works
MobileNetV4: Universal Models for the Mobile Ecosystem
Qin, D., Leichner, C., Delakis, M., Fornoni, M., Luo, S., Yang, F., Wang, W., Banbury, C., Ye, C., Akin, B., et al. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73661-2_5 · Cited by 746 · see citing works
EfficientSAM: Leveraged Masked Image Pretraining for Efficient Segment Anything
Xiong, Y., Varadarajan, B., Wu, L., Xiang, X., Xiao, F., Zhu, C., Dai, X., Wang, D., Sun, F., Iandola, F., et al. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01525 · Cited by 304 · see citing works
SHViT: Single-Head Vision Transformer with Memory Efficient Macro Design
Yun, S., & Ro, Y. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00550 · Cited by 208 · see citing works
EfficientVMamba: Atrous Selective Scan for Light Weight Visual Mamba
Pei, X., Huang, T., & Xu, C. (2025) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v39i6.32690 · Cited by 229 · see citing works
SegFormer3D: an Efficient Transformer for 3D Medical Image Segmentation
Perera, S. R., Navard, P., & Yılmaz, A. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00503 · Cited by 147 · see citing works
Feng, H., Li, Q., Wang, W. W., Bashir, A. K., Singh, A. K., Xu, J., & Fang, K. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102555 · Cited by 111 · see citing works
Frequency-Spatial Entanglement Learning for Camouflaged Object Detection
Sun, Y., Xu, C., Yang, J. J., Xuan, H., & Luo, L. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72658-3_20 · Cited by 109 · see citing works
Wang, A., Chen, H., Lin, Z., Han, J., & Ding, G. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00908 · Cited by 109 · see citing works
MobileMamba: Lightweight Multi-Receptive Visual Mamba Network
He, H., Zhang, J., Cai, Y., Chen, H., Hu, X., Gan, Z., Wang, Y., Wang, C., Wu, Y., & Xie, L. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00424 · Cited by 109 · see citing works
McIntosh, T. R., Susnjak, T., Liu, T., Watters, P., Xu, D., Liu, D., & Halgamuge, M. N. (2025) · Technologies · doi:10.3390/technologies13020051 · Cited by 102 · see citing works
“These models have also made notable strides in computer vision, as evidenced by the development of vision transformers like EfficientViT [145], [146] and YOLOv8 [147], [148], [149].”
From Dynamic to Static: Stepwisely Generate HDR Image for Ghost Removal
Yan, Q., Yang, K., Hu, T., Chen, G., Dai, K., Wu, P., Ren, W., & Zhang, Y. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3467259 · Cited by 81 · see citing works
EfficientViM: Efficient Vision Mamba with Hidden State Mixer based State Space Duality
Lee, S., Choi, J., & Kim, H. J. (2024) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr52734.2025.01390 · Cited by 88 · see citing works
A Lightweight Transformer Network for Hyperspectral Image Classification
Zhang, X., Su, Y., Gao, L., Bruzzone, L., Gu, X., & Tian, Q. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3297858 · Cited by 75 · see citing works
MobileCLIP: Fast Image-Text Models through Multi-Modal Reinforced Training
Vasu, P. K. A., Pouransari, H., Faghri, F., Vemulapalli, R., & Tuzel, O. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01511 · Cited by 67 · see citing works
ViTs as backbones: Leveraging vision transformers for feature extraction
Elharrouss, O., Himeur, Y., Mahmood, Y., Alrabaee, S., Ouamane, A., Bensaali, F., Bechqito, Y., & Chouchane, A. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.102951 · Cited by 59 · see citing works
EFS-YOLO: a lightweight network based on steel strip surface defect detection
Chen, B., Wei, M., Liu, J., Li, H., Dai, C., Liu, J., & Ji, Z. (2024) · Measurement Science and Technology · doi:10.1088/1361-6501/ad66fe · Cited by 58 · see citing works
SDD-DETR: Surface Defect Detection for No-Service Aero-Engine Blades With Detection Transformer
Sun, X., Song, K., Wen, X., Wang, Y., & Yan, Y. (2024) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2024.3457829 · Cited by 59 · see citing works
Aerial Person Detection for Search an d Rescue: Survey and Benchmarks
Zhang, X., Feng, Y., Wang, N., Lü, G., & Mei, S. (2025) · Journal of Remote Sensing · doi:10.34133/remotesensing.0474 · Cited by 57 · see citing works
GhostFormer: Efficiently amalgamated CNN-transformer architecture for object detection
Xie, X., Wu, D., Xie, M., & Li, Z. (2023) · Pattern Recognition · doi:10.1016/j.patcog.2023.110172 · Cited by 55 · see citing works
CAS-ViT: Convolutional Additive Self-Attention Vision Transformers for Efficient Mobile Applications
Zhang, T., Li, L., Zhou, Y., Liu, W., Qian, C., Hwang, J., & Ji, X. (2026) · IEEE Transactions on Image Processing · doi:10.1109/tip.2026.3655121 · Cited by 90 · see citing works
UAV-DETR: An Enhanced RT-DETR Architecture for Efficient Small Object Detection in UAV Imagery
Zhou, Y., & Wei, Y. (2025) · Sensors · doi:10.3390/s25154582 · Cited by 53 · see citing works
WonderWorld: Interactive 3D Scene Generation from a Single Image
Yu, H.-X., Duan, H., Herrmann, C., Freeman, W. T., & Wu, J. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00555 · Cited by 53 · see citing works
Resource-efficient Algorithms and Systems of Foundation Models: A Survey
Xu, M., Cai, D., Yin, W., Wang, S., Jin, X., & Liu, X. (2024) · ACM Computing Surveys · doi:10.1145/3706418 · Cited by 152 · see citing works
Self-citationIAMF-YOLO: Metal Surface Defect Detection Based on Improved YOLOv8
Chao, C., Mu, X., Guo, Z., Sun, Y., Tian, X., & Yong, F. (2025) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2025.3548198 · Cited by 44 · see citing works
A Glove-Wearing Detection Algorithm Based on Improved YOLOv8
Li, S., Huang, H., Meng, X., Wang, M., Li, Y., & Xie, L. (2023) · Sensors · doi:10.3390/s23249906 · Cited by 43 · see citing works
SRViT: Self-Supervised Relation-Aware Vision Transformer for Hyperspectral Unmixing
Su, Y.-C., Gao, L.-R., Plaza, A., Sun, X., Jiang, M.-Y., & Yang, G. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2025.3571798 · Cited by 38 · see citing works
Identifying rice field weeds from unmanned aerial vehicle remote sensing imagery using deep learning
Guo, Z., Cai, D., Zhou, Y., Xu, T., & Yu, F. (2024) · Plant Methods · doi:10.1186/s13007-024-01232-0 · Cited by 39 · see citing works
Multi-Scale Fusion Uncrewed Aerial Vehicle Detection Based on RT-DETR
Zhu, M., & Kong, E. (2024) · Electronics · doi:10.3390/electronics13081489 · Cited by 38 · see citing works
Investigation on a lightweight defect detection model for photovoltaic panel
Bin, F., Qiu, K., Zheng, Z. G., Lu, X., Du, L., & Sun, Q. (2024) · Measurement · doi:10.1016/j.measurement.2024.115121 · Cited by 37 · see citing works
MambaTSR: You only need 90k parameters for traffic sign recognition
Ge, Y., Chen, Z., Yu, M., Yue, Q., You, R., & Zhu, L. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128104 · Cited by 36 · see citing works
RS-DETR: An Improved Remote Sensing Object Detection Model Based on RT-DETR
Zhang, H., Ma, Z., & Li, X. (2024) · Applied Sciences · doi:10.3390/app142210331 · Cited by 40 · see citing works
Ma, R., Chen, J., Feng, Y., Zhou, Z., & Xie, J. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113410 · Cited by 35 · see citing works
REDef-DETR: real-time and efficient DETR for industrial surface defect detection
Li, D., Jiang, C., & Liang, T. (2024) · Measurement Science and Technology · doi:10.1088/1361-6501/ad60ea · Cited by 36 · see citing works
YOLOv8-WD: Deep Learning-Based Detection of Defects in Automotive Brake Joint Laser Welds
Ren, J., Zhang, H., & Yue, M. (2025) · Applied Sciences · doi:10.3390/app15031184 · Cited by 34 · see citing works
Yu, C., & Shin, Y. (2024) · Remote Sensing · doi:10.3390/rs16173169 · Cited by 33 · see citing works
Multi-Scale CNN-Transformer Hybrid Network for Rail Fastener Defect Detection
He, J., Wang, W., Lv, F., Luo, H., Zhang, G., & Chen, Z. (2025) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2025.3540846 · Cited by 33 · see citing works
A lightweight weed detection model with global contextual joint features
Guo, A., Jia, Z., Wang, J., Zhou, G., Ge, B., & Chen, W. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.108903 · Cited by 32 · see citing works
Li, Z., Zhao, X., Wu, F., Chen, D., & Wang, C. (2024) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2024.3447041 · Cited by 32 · see citing works
Zhang, C., Wang, L., Wei, G., Kong, Z., & Qiu, M. (2024) · Frontiers in Physiology · doi:10.3389/fphys.2024.1432987 · Cited by 35 · see citing works
Lightstereo: Channel Boost is All You Need for Efficient 2D Cost Aggregation
Guo, X., Zhang, C., Zhang, Y., Zheng, W., Nie, D., Poggi, M., & Chen, L. (2025) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra55743.2025.11127711 · Cited by 32 · see citing works
HybridHash: Hybrid Convolutional and Self-Attention Deep Hashing for Image Retrieval
He, C., & Wei, H. (2024) · International Conference on Multimedia Retrieval · doi:10.1145/3652583.3658014 · Cited by 34 · see citing works
Multi-branch CNN and grouping cascade attention for medical image classification
Liu, S., Yue, W., Guo, Z., & Wang, L. (2024) · Scientific Reports · doi:10.1038/s41598-024-64982-w · Cited by 31 · see citing works
Duan, F., Wang, H., Yang, H., Wei, C., Zhang, C., Song, Y., & Liu, Z. (2025) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2025.3637977 · Cited by 31 · see citing works
Distracted Driving Behavior Detection Algorithm Based on Lightweight StarDL-YOLO
Shen, Q., Zhang, L., Zhang, Y., Li, Y., Liu, S. B., & Xu, Y. (2024) · Electronics · doi:10.3390/electronics13163216 · Cited by 30 · see citing works
Lightweight Substation Equipment Defect Detection Algorithm for Small Targets
Wang, J., Sun, Y., Lin, Y., & Zhang, K. (2024) · Sensors · doi:10.3390/s24185914 · Cited by 30 · see citing works
Attention-Based Lightweight YOLOv8 Underwater Target Recognition Algorithm
Cheng, S., Wang, Z., Liu, S., Yan, H., Sun, P., & Li, J. (2024) · Sensors · doi:10.3390/s24237640 · Cited by 29 · see citing works
Steel surface defect detection based on the lightweight improved RT-DETR algorithm
Mao, H., & Gong, Y. (2024) · Journal of Real-Time Image Processing · doi:10.1007/s11554-024-01611-9 · Cited by 29 · see citing works
3SD-Net: SAR Small Ship Detection Neural Network
Ma, Y., Guan, D., Deng, Y., Yuan, W., & Wei, M. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3454308 · Cited by 28 · see citing works
Crack instance segmentation using splittable transformer and position coordinates
Zhao, Y., Li, W., Ding, J., Wang, Y., Pei, L., & Tian, A. (2024) · Automation in Construction · doi:10.1016/j.autcon.2024.105838 · Cited by 28 · see citing works
RADIOv2.5: Improved Baselines for Agglomerative Vision Foundation Models
Heinrich, G., Ranzinger, M., Yin, H.-X., Lu, Y., Kautz, J., Tao, A., Catanzaro, B., & Molchanov, P. (2024) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr52734.2025.02094 · Cited by 118 · see citing works
FlexiMo: A Flexible Remote Sensing Foundation Model
Li, X., Li, C., Ghamisi, P., Hong, D., Benediktsson, J. A., & Chanussot, J. (2026) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2026.3656362 · Cited by 40 · see citing works
RFAConv-CBM-ViT: enhanced vision transformer for metal surface defect detection
Wei, H., Zhao, L., Li, R., & Zhang, M. (2024) · The Journal of Supercomputing · doi:10.1007/s11227-024-06662-0 · Cited by 30 · see citing works
Swin-FER: Swin Transformer for Facial Expression Recognition
Bie, M., Xu, H., Gao, Y., Song, K., & Che, X. (2024) · Applied Sciences · doi:10.3390/app14146125 · Cited by 26 · see citing works
EViT: An Eagle Vision Transformer With Bi-Fovea Self-Attention
Shi, Y., Sun, M., Wang, Y., Ma, J., & Chen, Z. (2025) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2025.3532282 · Cited by 26 · see citing works
Chen, H., Wu, P., Wen, W., & Zeng, N. (2025) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2025.3561440 · Cited by 28 · see citing works
AGRI-YOLO: A Lightweight Model for Corn Weed Detection with Enhanced YOLO v11n
Peng, G., Wang, K., Ma, J., Cui, B., & Wang, D. (2025) · Agriculture · doi:10.3390/agriculture15181971 · Cited by 26 · see citing works
Liu, R., & Su, W. (2024) · Foods · doi:10.3390/foods13111710 · Cited by 25 · see citing works
Wang, Q., Li, Y., Cui, S., Li, N., Zhang, X., Jiang, W., Peng, W. J., & Sun, J. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.109078 · Cited by 25 · see citing works
A Lightweight Hybrid Model with Location-Preserving ViT for Efficient Food Recognition
Sheng, G., Min, W., Zhu, X., Xu, L., Sun, Q., Yang, Y., Wang, L., & Jiang, S. (2024) · Nutrients · doi:10.3390/nu16020200 · Cited by 25 · see citing works
Self-citationA Lightweight Vehicle Detection Method Fusing GSConv and Coordinate Attention Mechanism
Huang, D., Tu, Y., Zhang, Z., & Ye, Z. (2024) · Italian National Conference on Sensors · doi:10.3390/s24082394 · Cited by 24 · see citing works
Complete Region of Interest for Unconstrained Palmprint Recognition
Su, L., Fei, L., Zhang, B., Zhao, S., Wen, J., & Xu, Y. D. (2024) · IEEE Transactions on Image Processing · doi:10.1109/tip.2024.3407666 · Cited by 26 · see citing works
Lv, C., & Pan, W. (2025) · Journal of Real-Time Image Processing · doi:10.1007/s11554-025-01778-9 · Cited by 24 · see citing works
RT-DETR-Tea: A Multi-Species Tea Bud Detection Model for Unstructured Environments
Chen, Y., Guo, Y., Li, J., Zhou, B., Chen, J.-M., Zhang, M., Cui, Y., & Tang, J. (2024) · Agriculture · doi:10.3390/agriculture14122256 · Cited by 23 · see citing works
ViT-RoT: Vision Transformer-Based Robust Framework for Tomato Leaf Disease Recognition
Nishankar, S., Pavindran, V., Mithuran, T., Nimishan, S., Thuseethan, S., & Sebastian, Y. (2025) · AgriEngineering · doi:10.3390/agriengineering7060185 · Cited by 23 · see citing works
Dynamic-Guided Spatiotemporal Attention for Echocardiography Video Segmentation
Lin, J., Xie, W., Li, K., & Wu, H. (2024) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2024.3403687 · Cited by 24 · see citing works
MLP Can Be a Good Transformer Learner
Lin, S., Lyu, P., Liu, D., Tang, T., Liang, X., Song, A., & Chang, X. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01843 · Cited by 22 · see citing works
Towards High-performance Spiking Transformers from ANN to SNN Conversion
Huang, Z., Shi, X., Hao, Z., Bu, T., Ding, J., Yu, Z., & Huang, T. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3680620 · Cited by 22 · see citing works
Defect Detection and 3D Reconstruction of Complex Urban Underground Pipeline Scenes for Sewer Robots
Liu, R., Shao, Z., Sun, Q., & Yu, Z. (2024) · Sensors · doi:10.3390/s24237557 · Cited by 22 · see citing works
Song, B., Zhao, S., Wang, Z., Liu, W., & Liu, X. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113760 · Cited by 25 · see citing works
Self-citationDTNet: A Specialized Dual-Tuning Network for Infrared Vehicle Detection in Aerial Images
Zhang, N., Liu, Y., Liu, H., Tian, T., Ma, J., & Tian, J. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3386309 · Cited by 22 · see citing works
Frequency-Driven Edge Guidance Network for Semantic Segmentation of Remote Sensing Images
Li, J., Zhang, S., Sun, Y., Han, Q., Sun, Y., & Wang, Y. (2024) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2024.3393531 · Cited by 22 · see citing works
Ge, Q., Yuan, H., Zhang, Q., Hou, Y., Zang, C., Li, J., Liang, B., & Jiang, X. (2024) · International Journal of Hydrogen Energy · doi:10.1016/j.ijhydene.2024.06.028 · Cited by 21 · see citing works
Deng, T., Huang, Y., Han, G., Shi, Z., Lin, J., Dou, Q., Liu, Z., Guo, X., Chen, C. L. P., & Han, C. (2024) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2024.3403927 · Cited by 24 · see citing works
GTP-4o: Modality-Prompted Heterogeneous Graph Learning for Omni-Modal Biomedical Representation
Li, C., Liu, X., Wang, C., Liu, Y., Yu, W., Shao, J., & Yuan, Y. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73235-5_10 · Cited by 21 · see citing works
Self-citationAn Efficient Aerial Image Detection with Variable Receptive Fields
Liu, W., Shi, L., & An, G. (2025) · Remote Sensing · doi:10.3390/rs17152672 · Cited by 22 · see citing works
Enhanced PCB defect detection via HSA-RTDETR on RT-DETR
Wang, Y., Wu, B., Zhang, L., Wang, Z., Liu, J., Dong, J., & Shi, J. (2025) · Scientific Reports · doi:10.1038/s41598-025-11394-z · Cited by 23 · see citing works
Lu, X., Suganuma, M., & Okatani, T. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00116 · Cited by 23 · see citing works
Grouped multi-scale vision transformer for medical image segmentation
Ji, Z., Chen, Z., & Ma, X. (2025) · Scientific Reports · doi:10.1038/s41598-025-95361-8 · Cited by 20 · see citing works
ATGNN: Audio Tagging Graph Neural Network
Singh, S., Steinmetz, C. J., Benetos, E., Phan, H., & Stowell, D. (2024) · IEEE Signal Processing Letters · doi:10.1109/lsp.2024.3352514 · Cited by 20 · see citing works
Li, M., Geng, Y., Wang, W., Tu, M., & Wu, X. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.107938 · Cited by 20 · see citing works
M3YOLOv5: Feature enhanced YOLOv5 model for mandibular fracture detection
Zhou, T., Wang, H., Du, Y., Liu, F., Guo, Y., & Lu, H. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.108291 · Cited by 20 · see citing works
Zhao, C., Guo, D., Shao, C., Zhao, K., Sun, M., & Shuai, H. (2024) · IEEE Access · doi:10.1109/access.2024.3382245 · Cited by 20 · see citing works
Van-DETR: enhanced real-time object detection with vanillanet and advanced feature fusion
Lu, X., Zhan, G., Wu, W., Zhang, W., Wu, X., & Han, C. (2024) · The Visual Computer · doi:10.1007/s00371-024-03656-0 · Cited by 21 · see citing works
D-YOLO: A Lightweight Model for Strawberry Health Detection
Wu, E., Ma, R., Dong, D., & Zhao, X. (2025) · Agriculture · doi:10.3390/agriculture15060570 · Cited by 19 · see citing works
Gkikas, S., & Tsiknakis, M. (2024) · International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW) · doi:10.1109/aciiw63320.2024.00007 · Cited by 19 · see citing works
Lightweight and Efficient Hybrid Network for UAV Identification Using Radio Frequency Fingerprinting
Zhou, K., Li, Q., Cao, P., Cai, Z., Shi, X., & Wang, F. (2025) · IEEE Internet of Things Journal · doi:10.1109/jiot.2025.3594571 · Cited by 18 · see citing works
Hou, Y., DongMing, L., Liwang, M., Wang, J., Li, Z., Xia, X., Wu, D., Tian, Y., Quek, T. Q. S., & Wang, X. M. (2026) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2026.3688489 · Cited by 18 · see citing works
InverseMatrixVT3D: An Efficient Projection Matrix-Based Approach for 3D Occupancy Prediction
Ming, Z., Berrío, J. S., Shan, M., & Worrall, S. (2024) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros58592.2024.10802434 · Cited by 21 · see citing works
Sun, H., He, X., Li, H., Kong, J., Qiao, M., Cheng, X., Li, P., Zhang, J., Liu, R., & Shang, J. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.127347 · Cited by 18 · see citing works
Wang, Z., Guo, L., Zhao, S., Zhang, S., Zhao, X., Fang, J., Wang, G., Lu, H., Yu, J., & Tian, Q. C. (2024) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2024.3523112 · Cited by 17 · see citing works
Vision-based quality evaluation method towards automated penetrant testing
Ou, Y., Xu, T. N., Fan, J., Xu, B., Cai, H., & Zhao, J. (2025) · NDT & E International · doi:10.1016/j.ndteint.2025.103334 · Cited by 17 · see citing works
Zhang, X., Liang, X., Zhang, Y., Li, J., & Wei, S. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.127121 · Cited by 18 · see citing works
YOLO-LHD: an enhanced lightweight approach for helmet wearing detection in industrial environments
Hu, L., & Ren, J. (2023) · Frontiers in Built Environment · doi:10.3389/fbuil.2023.1288445 · Cited by 17 · see citing works
Learning Frequency-Aware Common Feature for VIS-NIR Heterogeneous Palmprint Recognition
Fei, L., Su, L., Zhang, B., Zhao, S., Wen, J., & Li, X. (2024) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2024.3441945 · Cited by 17 · see citing works
LWFDD-YOLO: a lightweight defect detection algorithm based on improved YOLOv8
Chang, C., Zhou, Q., Xiao, L., Li, S., & Luo, D. (2024) · Textile Research Journal · doi:10.1177/00405175241285596 · Cited by 17 · see citing works
MDNet: Multimodal Cooperative Perception via Spatial Alignment of Modal Decision-Making
He, J., Deng, X., Gui, J., Zhang, T., & He, X. (2025) · IEEE Internet of Things Journal · doi:10.1109/jiot.2025.3531145 · Cited by 16 · see citing works
MSCViT: A small-size ViT architecture with multi-scale self-attention mechanism for tiny datasets
Zhang, B., & Zhang, Y. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.107499 · Cited by 16 · see citing works
Local-to-Global Cross-Modal Attention-Aware Fusion for HSI-X Semantic Segmentation
Zhang, X., Yokoya, N., Gu, X., Tian, Q., & Bruzzone, L. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3465030 · Cited by 16 · see citing works
Qin, X., Cao, J., Zhang, Y., Dong, T., & Cao, H. (2025) · Processes · doi:10.3390/pr13020353 · Cited by 16 · see citing works
- Authors
- Liu, X., Peng, H., Zheng, N., Yang, Y., Hu, H., & Yuan, Y.
- Affiliations
- Chinese University of Hong Kong (HK)
- Microsoft (United States) (US)
- Venue
- 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2023-06
- Pages
- 14420-14430
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W4386076539
- Semantic Scholar
- 9a83aeadc8db65fb6da39ec977360541cddaff5c
- Cited by
- 956
- References
- 84
- Citations by year
- 2026: 289 · 2025: 431 · 2024: 225 · 2023: 11
- FWCI
- 102.10
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- Innovation and Technology Commission (ITS/100/20)
- Innovation and Technology Fund
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- EfficientViT, multi-head self-attention, channel communication, real-time image classification, computational efficiency
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
