The DOI matches its Crossref record; 86 of the 86 references it cites were found in a registry or an index.
EfficientViT: Memory Efficient Vision Transformer with Cascaded Group Attention
Open accessAbstract
Vision transformers have shown great success due to their high model capabilities. However, their remarkable performance is accompanied by heavy computation costs, which makes them unsuitable for real-time applications. In this paper, we propose a family of high-speed vision transformers named Efficient ViT. We find that the speed of existing transformer models is commonly bounded by memory inefficient operations, especially the tensor reshaping and element-wise functions in MHSA. Therefore, we design a new building block with a sandwich layout, i.e., using a single memory-bound MHSA between efficient FFN layers, which improves memory efficiency while enhancing channel communication. Moreover, we discover that the attention maps share high similarities across heads, leading to computational redundancy. To address this, we present a cascaded group attention module feeding attention heads with different splits of the full feature, which not only saves computation cost but also improves attention diversity. Comprehensive experiments demonstrate EfficientViT outperforms existing efficient models, striking a good trade-off between speed and accuracy. For instance, our EfficientViT-M5 surpasses MobileNetV3-Large by 1.9% in accuracy, while getting 40.4% and 45.2% higher throughput on Nvidia V100 GPU and Intel Xeon CPU, respectively. Compared to the recent efficient model MobileViT-XXS, EfficientViT-M2 achieves 1.8% superior accuracy, while running$5.8\times/3.7\times$faster on the GPU/CPU, and$7.4\times faster$when converted to ONNX format. Code and models are available at here.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Unsupervised learning is essential in AI for uncovering patterns within unlabeled data, a process central to tasks like feature learning and clustering [169], [170].”
From From Google Gemini to OpenAI Q* (Q-Star): A Survey on Reshaping the Generative Artificial Intelligence (AI) Research Landscape (2025) · cites this paper as [170]
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Jakubův, J., Chvalovský, K., Goertzel, Z., et al. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,648
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534
Self-citationRethinking the Inception Architecture for Computer Vision
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.308 · Cited by 32,138
PyTorch: An Imperative Style, High-Performance Deep Learning Library
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al. (2019) · arXiv · doi:10.48550/arxiv.1912.01703 · Cited by 31,473
Focal Loss for Dense Object Detection
Lin, T.-Y., Goyal, P., Girshick, R., He, K., & Dollar, P. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.324 · Cited by 30,581
MobileNetV2: Inverted Residuals and Linear Bottlenecks
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L.-C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00474 · Cited by 28,755
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,889
Xception: Deep Learning with Depthwise Separable Convolutions
Chollet, F. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.195 · Cited by 20,380
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., & Adam, H. (2017) · arXiv · doi:10.48550/arxiv.1704.04861 · Cited by 19,015
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
Tan, M., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1905.11946 · Cited by 17,072
(2010) · International Conference on Machine Learning · Cited by 13,995
Decoupled Weight Decay Regularization
Loshchilov, I., & Hutter, F. (2017) · arXiv · doi:10.48550/arxiv.1711.05101 · Cited by 12,522
ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
Zhang, X., Zhou, X., Lin, M., & Sun, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00716 · Cited by 9,982
Howard, A., Sandler, M., Chen, B., Wang, W., Chen, L.-C., Tan, M., Chu, G., Vasudevan, V., Zhu, Y., Pang, R., et al. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00140 · Cited by 9,708
ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design
Ma, N., Zhang, X., Zheng, H.-T., & Sun, J. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01264-9_8 · Cited by 7,356 · cited 3×
“With the demand of deploying CNNs on resource-constrained scenarios, efficient CNNs have been intensively studied in literature [23, 24, 26, 48, 63, 67, 87].”
BackgroundMethodLearning Transferable Architectures for Scalable Image Recognition
Zoph, B., Vasudevan, V., Shlens, J., & Le, Q. V. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00907 · Cited by 6,875
mixup: Beyond Empirical Risk Minimization
Zhang, H., Cisse, M., Dauphin, Y. N., & Lopez-Paz, D. (2017) · arXiv · doi:10.48550/arxiv.1710.09412 · Cited by 5,831
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions
Wang, W., Xie, E., Li, X., Fan, D.-P., Song, K., Liang, D., Lu, T., Luo, P., & Shao, L. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00061 · Cited by 5,259 · cited 2×
“Efficient self-attention methods reduce the cost of softmax attention via sparse attention [34, 57, 61, 75] or low-rank approximation [11, 51, 74].”
BackgroundMethodGhostNet: More Features From Cheap Operations
Han, K., Wang, Y., Tian, Q., Guo, J., Xu, C., & Xu, C. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00165 · Cited by 5,222
AutoAugment: Learning Augmentation Strategies From Data
Cubuk, E. D., Zoph, B., Mané, D., Vasudevan, V., & Le, Q. V. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00020 · Cited by 4,445
3D Object Representations for Fine-Grained Categorization
Krause, J., Stark, M. M., Deng, J., & Fei-Fei, L. (2013) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2013.77 · Cited by 3,666
Random Erasing Data Augmentation
Zhong, Z., Zheng, L., Kang, G., Li, S., & Yang, Y. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.7000 · Cited by 3,617
Self-citationGaussian Error Linear Units (GELUs)
Hendrycks, D., & Gimpel, K. (2016) · arXiv · doi:10.48550/arxiv.1606.08415 · Cited by 3,537
MnasNet: Platform-Aware Neural Architecture Search for Mobile
Tan, M., Chen, B., Pang, R., Vasudevan, V., Sandler, M., Howard, A., & Le, Q. V. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00293 · Cited by 3,416
(1989) · Cited by 3,214
Swin Transformer V2: Scaling Up Capacity and Resolution
Liu, Z., Hu, H., Lin, Y., Yao, Z., Xie, Z., Wei, Y., Ning, J., Cao, Y., Zhang, Z., Dong, L., et al. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01170 · Cited by 2,985
Self-citationSwin Transformer V2: Scaling Up Capacity and Resolution
Liu, Z., Hu, H., Lin, Y., Yao, Z., Xie, Z., Wei, Y., Ning, J., Cao, Y., Zhang, Z., Dong, L., et al. (2021) · arXiv · doi:10.48550/arxiv.2111.09883 · Cited by 2,679
Self-citationPvt v2: Improved baselines with pyramid vision transformer
Wang, W., Xie, E., Li, X., Fan, D.-P., Song, K., Liang, D., Lu, T., Luo, P., & Shao, L. (2022) · Computational Visual Media · doi:10.1007/s41095-022-0274-8 · Cited by 2,439
Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet
Yuan, L., Chen, Y., Wang, T., Yu, W., Shi, Y., Jiang, Z., Tay, F. E. H., Feng, J., & Yan, S. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00060 · Cited by 2,415
MMDetection: Open MMLab Detection Toolbox and Benchmark
Chen, K., Wang, J., Pang, J., Cao, Y., Xiong, Y., Li, X., Sun, S., Feng, W., Liu, Z., Xu, J., et al. (2019) · arXiv · doi:10.48550/arxiv.1906.07155 · Cited by 2,227
Designing Network Design Spaces
Radosavovic, I., Kosaraju, R. P., Girshick, R., He, K., & Dollar, P. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01044 · Cited by 2,084
Parkhi, O., Vedaldi, A., Zisserman, A., & Jawahar, C. V. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6248092 · Cited by 1,702
MetaFormer is Actually What You Need for Vision
Yu, W., Luo, M., Zhou, P., Si, C., Zhou, Y., Wang, X., Feng, J., & Yan, S. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01055 · Cited by 1,449
Rethinking the Value of Network PruningPreprint
Liu, Z., Sun, M., Zhou, T., Huang, G. M., & Darrell, T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.05270 · Cited by 1,448
Reformer: The Efficient Transformer
Kitaev, N., Kaiser, Ł., & Levskaya, A. (2020) · arXiv · doi:10.48550/arxiv.2001.04451 · Cited by 1,435
FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
Dao, T., Fu, D. Y., Ermon, S., Rudra, A., & Ré, C. (2022) · arXiv · doi:10.48550/arxiv.2205.14135 · Cited by 1,214 · cited 4×
“The activation functions are element-wise operations that occupy a considerable amount of processing time on GPU/CPU [15,48,72], thus utilizing ReLU instead of more complicated activation functions is of better efficiency.”
BackgroundMethodInfluentialAnalyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned
Voita, E., Talbot, D., Moiseev, F., Sennrich, R., & Titov, I. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1580 · Cited by 1,146
MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer
Mehta, S., & Rastegari, M. (2021) · arXiv · doi:10.48550/arxiv.2110.02178 · Cited by 1,110
Importance Estimation for Neural Network Pruning
Molchanov, P., Mallya, A., Tyree, S., Frosio, I., & Kautz, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01152 · Cited by 983
LeViT: a Vision Transformer in ConvNet’s Clothing for Faster Inference
Graham, B., El-Nouby, A., Touvron, H., Stock, P., Joulin, A., Jegou, H., & Douze, M. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01204 · Cited by 935
CoAtNet: Marrying Convolution and Attention for All Data Sizes
Dai, Z., Liu, H., Le, Q. V., & Tan, M. (2021) · arXiv · doi:10.48550/arxiv.2106.04803 · Cited by 925
Single Path One-Shot Neural Architecture Search with Uniform Sampling
Guo, Z., Zhang, X., Mu, H., Heng, W., Liu, Z., Wei, Y., & Sun, J. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58517-4_32 · Cited by 882
A Visual Vocabulary for Flower Classification
Nilsback, M.-E., & Zisserman, A. (2006) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2006.42 · Cited by 876
Linformer: Self-Attention with Linear Complexity
Wang, S., Li, B. Z., Khabsa, M., Fang, H., & Ma, H. (2020) · arXiv · doi:10.48550/arxiv.2006.04768 · Cited by 867
Mobile-Former: Bridging MobileNet and Transformer
Chen, Y., Dai, X., Chen, D., Liu, M., Dong, X., Yuan, L., & Liu, Z. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00520 · Cited by 731
Rethinking Attention with Performers
Choromanski, K., Likhosherstov, V., Dohan, D., Song, X., Gane, A., Sarlos, T., Hawkins, P., Davis, J., Mohiuddin, A., Kaiser, L., et al. (2020) · arXiv · doi:10.48550/arxiv.2009.14794 · Cited by 695 · cited 2×
“Although there are some methods proposed to address this issue by simplifying the computation of standard softmax self-attention, e.g. , sparse attention [34, 57, 61, 75] and low-rank approximation [11,51,74], they often come at the cost of accuracy degradation and limited acceleration.”
MethodEarly Convolutions Help Transformers See BetterPreprint
Xiao, T., Singh, M., Mintun, E., Darrell, T., Dollár, P., & Girshick, R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.14881 · Cited by 596
TinyViT: Fast Pretraining Distillation for Small Vision Transformers
Wu, K., Zhang, J., Peng, H., Liu, M., Xiao, B., Fu, J., & Yuan, L. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19803-8_5 · Cited by 521
Self-citationMixConv: Mixed Depthwise Convolutional KernelsPreprint
Tan, M., & Le, Q. V. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.09595 · Cited by 446
EfficientFormer: Vision Transformers at MobileNet Speed
Li, Y., Yuan, G., Wen, Y., Hu, J., Evangelidis, G., Tulyakov, S., Wang, Y., & Ren, J. (2022) · neural information processing systems · doi:10.52202/068431-0940 · Cited by 445
Edgenext: Efficiently amalgamated CNN-transformer architecture for mobile vision applications
Maaz, M., Shaker, A., Cholakkal, H., Khan, S., Zamir, S. W., Anwer, R. M., & Shahbaz Khan, F. (2023) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-25082-8_1 · Cited by 407
Are Sixteen Heads Really Better than One?
Michel, P., Levy, O., & Neubig, G. (2019) · arXiv · doi:10.48550/arxiv.1905.10650 · Cited by 404
FairNAS: Rethinking Evaluation Fairness of Weight Sharing Neural Architecture Search
Chu, X., Zhang, B., & Xu, R. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01202 · Cited by 380
Scaling Vision with Sparse Mixture of ExpertsPreprint
Riquelme, C., Puigcerver, J., Mustafa, B., Neumann, M., Jenatton, R., Pinto, A. S., Keysers, D., & Houlsby, N. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.05974 · Cited by 330
Fast Vision Transformers with HiLo AttentionPreprint
Pan, Z.-Z., Cai, J.-F., & Zhuang, B.-H. (2022) · Neural Information Processing Systems · doi:10.48550/arxiv.2205.13213 · Cited by 327
AutoFormer: Searching Transformers for Visual Recognition
Chen, M., Peng, H., Fu, J., & Ling, H. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01205 · Cited by 323
Self-citationEdgeViTs: Competing Light-Weight CNNs on Mobile Devices with Vision Transformers
Pan, J., Bulat, A., Tan, F., Zhu, X., Dudziak, Ł., Li, H., Tzimiropoulos, G., & Martínez, B. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20083-0_18 · Cited by 316
EfficientFormer: Vision Transformers at MobileNet SpeedPreprint
Li, Y., Yuan, G., Wen, Y., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2206.01191 · Cited by 254
Separable Self-attention for Mobile Vision Transformers
Mehta, S., & Rastegari, M. (2022) · arXiv · doi:10.48550/arxiv.2206.02680 · Cited by 215
Fast Vision Transformers with HiLo Attention
Pan, Z., Cai, J., & Zhuang, B. (2022) · neural information processing systems · doi:10.52202/068431-1057 · Cited by 207
GhostNets on Heterogeneous Devices via Cheap Operations
Han, K., Wang, Y., Xu, C., Guo, J., Xu, C., Wu, E., & Tian, Q. C. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-022-01575-y · Cited by 201
Training data-efficient image transformers & distillation through attention
Touvron, H., Cord, M., Douze, M., Massa, F., Sablayrolles, A., & Jégou, H. (2020) · arXiv · doi:10.48550/arxiv.2012.12877 · Cited by 182
Lite Vision Transformer with Enhanced Self-Attention
Yang, C., Wang, Y., Zhang, J., Zhang, H., Wei, Z., Lin, Z., & Yuille, A. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01169 · Cited by 164
Transformer Quality in Linear Time
Hua, W., Dai, Z., Liu, H., & Le, Q. V. (2022) · arXiv · doi:10.48550/arxiv.2202.10447 · Cited by 137
MiniViT: Compressing Vision Transformers with Weight Multiplexing
Zhang, J., Peng, H., Wu, K., Liu, M., Xiao, B., Fu, J., & Yuan, L. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01183 · Cited by 135
Self-citationThe Efficiency MisnomerPreprint
Dehghani, M., Arnab, A., Beyer, L., Vaswani, A., & Tay, Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.12894 · Cited by 106
GLiT: Neural Architecture Search for Global and Local Image Transformer
Chen, B., Li, P., Li, C., Li, B., Bai, L., Chen, L., Sun, M., Yan, J., & Ouyang, W. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00008 · Cited by 100
Data Movement Is All You Need: A Case Study on Optimizing Transformers
Ivanov, A., Dryden, N., Ben-Nun, T., Li, S., & Hoefler, T. (2020) · arXiv · doi:10.48550/arxiv.2007.00072 · Cited by 78
DS-Net++: Dynamic Weight Slicing for Efficient Inference in CNNs and Vision Transformers
Li, C., Wang, G., Wang, B., Liang, X., Li, Z., & Chang, X. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3194044 · Cited by 74
Pale Transformer: A General Vision Transformer Backbone with Pale-Shaped Attention
Wu, S., Wu, T., Tan, H., & Guo, G. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i3.20176 · Cited by 73
Towards Lightweight Transformer Via Group-Wise Transformation for Vision-and-Language Tasks
Luo, G., Zhou, Y., Sun, X., Wang, Y., Cao, L., Wu, Y., Huang, F., & Ji, R. (2022) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3139234 · Cited by 67
Characterizing Co-Located Workloads in Alibaba Cloud Datacenters
Jiang, C., Qiu, Y., Shi, W., Ge, Z., Wang, J., Chen, S., Cérin, C., Ren, Z., Xu, G., & Lin, J. (2020) · IEEE Transactions on Cloud Computing · doi:10.1109/tcc.2020.3034500 · Cited by 51
LightViT: Towards Light-Weight Convolution-Free Vision TransformersPreprint
Huang, T., Huang, L., You, S., Wang, F., Qian, C., & Xu, C. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2207.05557 · Cited by 48
Improving the Efficiency of Transformers for Resource-Constrained Devices
Tabani, H., Balasubramaniam, A., Marzban, S., Arani, E., & Zonooz, B. (2021) · Euromicro Conference on Digital System Design (DSD) · doi:10.1109/dsd53832.2021.00074 · Cited by 36
Combiner: Full Attention Transformer with Sparse Computation CostPreprint
Ren, H., Dai, H., Dai, Z., Yang, M., Leskovec, J., Schuurmans, D., & Dai, B. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.05768 · Cited by 36
NViT: Vision Transformer Compression and Parameter Redistribution.Preprint
Yang, H., Yin, H., Molchanov, P. A., Li, H., & Kautz, J. (2021) · arXiv (Cornell University) · Cited by 28
SWIRL: High-performance many-core CPU code generation for deep neural networks
Venkat, A., Rusira, T., Barik, R., Hall, M. W., & Truong, L. (2019) · The International Journal of High Performance Computing Applications · doi:10.1177/1094342019866247 · Cited by 27
Auto-scaling Vision Transformers without TrainingPreprint
Chen, W., Huang, W. F., Du, X., Song, X., Wang, Z., & Zhou, D. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2202.11921 · Cited by 16
Towards Memory-Efficient Neural Networks via Multi-Level in situ Generation
Gu, J., Zhu, H., Feng, C., Liu, M., Jiang, Z., Chen, R. T., & Pan, D. Z. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00518 · Cited by 4
An Optimized Dataflow for Mitigating Attention Performance BottlenecksPreprint
Kao, S.-C., Subramanian, S., Agrawal, G., & Krishna, T. (2021) · arXiv (Cornell University) · Cited by 2
Research works printed but not linked: 7
- gong, 2022, NASVit: Neural architecture search for efficient vision transformers with gradient conflict aware supernet training, ICLRE
- beyer, 0, Are we done with imagenet?
- bai, 2019, Onnx Open neural network exchange
- wightman, 2019, pytorch-image-models
- ba, 2016, Layer normalization
- 2021, Coremltools Use coremltools to convert machine learning models from third-party libraries to the core ml format
- vasu, 2022, An improved one millisecond mobile backbone
Cited by
Sorted by citations
Rep ViT: Revisiting Mobile CNN From ViT Perspective
Wang, A., Chen, H., Lin, Z., Han, J., & Ding, G. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01506 · Cited by 752
MobileNetV4: Universal Models for the Mobile Ecosystem
Qin, D., Leichner, C., Delakis, M., Fornoni, M., Luo, S., Yang, F., Wang, W., Banbury, C., Ye, C., Akin, B., et al. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73661-2_5 · Cited by 812
EfficientSAM: Leveraged Masked Image Pretraining for Efficient Segment Anything
Xiong, Y., Varadarajan, B., Wu, L., Xiang, X., Xiao, F., Zhu, C., Dai, X., Wang, D., Sun, F., Iandola, F., et al. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01525 · Cited by 356
SHViT: Single-Head Vision Transformer with Memory Efficient Macro Design
Yun, S., & Ro, Y. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00550 · Cited by 227
EfficientVMamba: Atrous Selective Scan for Light Weight Visual Mamba
Pei, X., Huang, T., & Xu, C. (2025) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v39i6.32690 · Cited by 229
Metadata
- Authors
- Liu, X., Peng, H., Zheng, N., Yang, Y., Hu, H., & Yuan, Y.
- Affiliations
- Chinese University of Hong Kong (HK)
- Microsoft (United States) (US)
- Venue
- 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2023-06
- Pages
- 14420-14430
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W4386076539
- Semantic Scholar
- 9a83aeadc8db65fb6da39ec977360541cddaff5c
- Cited by
- 956
- References
- 84
- Citations by year
- 2026: 289 · 2025: 431 · 2024: 225 · 2023: 11
- FWCI
- 102.10
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- Innovation and Technology Commission (ITS/100/20)
- Innovation and Technology Fund
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- EfficientViT, multi-head self-attention, channel communication, real-time image classification, computational efficiency
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
