EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks

Tan, Mingxing; Le, Quoc V.

doi:10.48550/arxiv.1905.11946

PreprintCited by 17,072Advanced Neural Network ApplicationsDomain Adaptation and Few-Shot LearningMachine Learning and Data Classification

EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks

Open access

Tan, Mingxing, Le, Quoc V.·2019·arXiv

DOI 10.48550/arxiv.1905.11946arXiv 1905.11946

Open this paper in CoChat — free account Open PDF

Abstract

Convolutional Neural Networks (ConvNets) are commonly developed at a fixed resource budget, and then scaled up for better accuracy if more resources are available. In this paper, we systematically study model scaling and identify that carefully balancing network depth, width, and resolution can lead to better performance. Based on this observation, we propose a new scaling method that uniformly scales all dimensions of depth/width/resolution using a simple yet highly effective compound coefficient. We demonstrate the effectiveness of this method on scaling up MobileNets and ResNet. To go even further, we use neural architecture search to design a new baseline network and scale it up to obtain a family of models, called EfficientNets, which achieve much better accuracy and efficiency than previous ConvNets. In particular, our EfficientNet-B7 achieves state-of-the-art 84.3% top-1 accuracy on ImageNet, while being 8.4x smaller and 6.1x faster on inference than the best existing ConvNet. Our EfficientNets also transfer well and achieve state-of-the-art accuracy on CIFAR-100 (91.7%), Flowers (98.8%), and 3 other transfer learning datasets, with an order of magnitude fewer parameters. Source code is at https://github.com/tensorflow/tpu/tree/master/models/official/efficientnet.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

DataCite · 10.48550/arxiv.1905.11946

Cites

54 of 54 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

The YOLO family, MobileNet and its derivative-based backbones, and EfficientNet-Lite backbones are very suitable for deployment with a lower number of parameters while retaining higher performance.
“In general, the analyses show that the YOLO family [52,71,104,112] of network architectures and MobileNet and its derivative-based [40,53] and EfficientNet-Lite [56] backbones are very suitable for deployment with a lower number of parameters while retaining higher performance.”
From A survey on deep learning-based monocular spacecraft pose estimation: Current state, limitations and prospects (2023) · cites this paper as [56]
Section 5.2 (p. 7) reports that EfficientNet models achieve similar or better accuracy with fewer parameters and FLOPS, and measures faster inference on a CPU. But the cited paper discusses EfficientNet-B0 through B7, not EfficientNet-Lite backbones, so it does not establish the deployment claim for the specifically cited variant. (LLM, conf=0.86)
Experimental — accuracy not yet measured
EfficientNets achieve state-of-the-art accuracy while being an order-of-magnitude smaller and faster than previous models.
“EfficientNets [37] are a family of image classification models, which achieve state-of-the-art accuracy, yet being an order-ofmagnitude smaller and faster than previous models.”
From Improving Generalization of Deepfake Detection With Data Farming and Few-Shot Learning (2022) · cites this paper as [37]
The Introduction (p. 1) reports state-of-the-art ImageNet accuracy for EfficientNet-B7 and says it is 8.4× smaller and 6.1× faster than GPipe. Section 5.2 (p. 7) describes generally order-of-magnitude reductions in parameters and FLOPs, but its measured inference-speed comparisons are 5.7× and 6.1×—not an order of magnitude faster. (LLM, conf=0.88)
Experimental — accuracy not yet measured
EfficientNet is a family of deep learning models that are scaled to balance network depth, width, and input data resolution to achieve an optimal performance-training time trade-off.
“EfficientNet is a family of DL models that are scaled to balance network depth, width, and input data resolution to achieve an optimal performance-training time trade-off.”
From Benchmarking and scaling of deep learning models for land cover image classification (2023) · cites this paper as Tan and Le, 2019
The Introduction (p. 2) identifies EfficientNets as a family of scaled models, and Compound Scaling (p. 4) says balancing depth, width, and resolution improves accuracy and efficiency. But the paper assesses the trade-off using accuracy, FLOPs, and inference latency—not training time—and does not establish an optimal performance–training-time trade-off. (LLM, conf=0.88)
Experimental — accuracy not yet measured
Layers in residual networks exhibit a large amount of redundancy, and almost any individual layer can be removed after training without hurting performance.
“Previous works [50, 15] have shown that layers in residual networks exhibit a large amount of redundancy, and that almost any individual layer can be removed after training without hurting performance.”
From Understanding Robustness of Transformers for Image Classification (2021) · cites this paper as [50]
Section 3.2 (p. 3) says ResNet-1000 has similar accuracy to ResNet-101 despite having many more layers, supporting the narrower point that gains from added depth diminish. It does not show that almost any individual layer can be removed after training without hurting performance. (LLM, conf=0.77)
Experimental — accuracy not yet measured
In top-5 winning solutions of DFDC, all were image-based detection methods; three used EfficientNet as backbone.
“In the top-5 winning solutions of DFDC, all of them were image-based detection methods. Three of the five methods used EfficientNet (Tan and Le 2019) as the backbone model.”
From Countering Malicious DeepFakes: Survey, Battleground, and Horizon (2022) · cites this paper as Tan and Le 2019

The compound scaling method is the core innovation in the EfficientNet architecture, using a single scalar factor φ to uniformly scale depth, width, and resolution.
“The compound scaling method is the core innovation in the EfficientNet architecture using single scalar factor ϕ to uniformly scale all three dimensions such as depth, width and resolution given in the Equation is analysed in the paper27.”
From Image adaptive encryption using EfficientNet B3 feature guided multi scroll chaotic map with modulo controlled pseudo parallel processing (2025) · cites this paper as [27]
EfficientNet-B3 has scaling capabilities in depth, width, and input resolution, and an efficient compromise between accuracy and computational resources.
“The EfficientNet-B3 is implemented in this algorithm because of its scaling capabilities in the depth, width and input resolution of the input data, and it has efficient compromise between the accuracy and computational resources utilized27.”
From Image adaptive encryption using EfficientNet B3 feature guided multi scroll chaotic map with modulo controlled pseudo parallel processing (2025) · cites this paper as [27]
EfficientNets perform compound scaling, i.e., scale simultaneously all three dimensions, depth, width, and image resolution, while maintaining a balance between all dimensions of the network.
“EfficientNets on the other hand perform Compound Scaling, i.e. scale simultaneously all three dimensions, depth, width, and image resolution, while maintaining a balance between all dimensions of the network.”
From Benchmarking and scaling of deep learning models for land cover image classification (2023) · cites this paper as Tan and Le, 2019
The paper deploys the original EfficientNet-B0 model in the benchmark.
“We deploy in our benchmark the original model in Tan and Le (2019), which introduces a new baseline CNN architecture called EfficientNet-B0.”
From Benchmarking and scaling of deep learning models for land cover image classification (2023) · cites this paper as Tan and Le, 2019
EfficientNets achieve better efficiency than previous commonly used backbones.
“Finally, we also observe that the recently introduced EfficientNets [39] achieve better efficiency than previous commonly used backbones.”
From EfficientDet: Scalable and Efficient Object Detection (2019) · cites this paper as [39]
For both Efficient ViT and Convolutional Cross ViT, an EfficientNet-B0 pre-trained on ImageNet is used as a feature extractor, as per the original paper.
“For both Efficient ViT and Convolutional Cross ViT, we use an EfficientNet-B0 [29] that is pre-trained on ImageNet as a feature extractor, as per the original paper.”
From WaveConViT: Wavelet-Based Convolutional Vision Transformer for Cross-Manipulation Deepfake Video Detection (2025) · cites this paper as [29]
Teacher-student methods depend on pre-trained models such as ResNet [7], VGG [8] and EfficientNet [9].
“The performance of these methods is outstanding, but they depend on pre-trained models such as ResNet [7] VGG [8] and EfficientNet [9] .”
From Deep Industrial Image Anomaly Detection: A Survey (2024) · cites this paper as [9]
The success of EfficientNet shows that we can obtain a model with better complexity/accuracy tradeoffs by scaling the model along different dimensions.
“The success of EfficientNet [24] shows that we can obtain a model with better complexity/accuracy tradeoffs by scaling the model along different dimensions.”
From DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsification (2021) · cites this paper as [24]
More recent work frames non-uniform quantization as an optimization problem.
“To further improve the quantizer, more recent work [78,234,258] formulates non-uniform quantization as an optimization problem.”
From A Survey of Quantization Methods for Efficient Neural Network Inference (2022) · cites this paper as [234]
Section 3.1 (p. 3) formulates ConvNet model scaling—not non-uniform quantization—as an optimization problem. Searches for “non-uniform quantization” and “quantizer” found no passage in our copy addressing the attributed claim. (LLM, conf=0.77)
Experimental — accuracy not yet measured
Some of the latest deepfake detection approaches demonstrate encouraging accuracy when trained and evaluated on the same datasets.
“Some of the latest approaches demonstrate encouraging accuracy, especially, if they are trained and evaluated on the same datasets [1], [2].”
From Improving Generalization of Deepfake Detection With Data Farming and Few-Shot Learning (2022) · cites this paper as [2]
The Related Work section (p. 2) discusses accuracy in ImageNet classification, and the Transfer Learning Results section (p. 8) reports EfficientNet accuracy on other vision datasets. A search for deepfake detection and same-dataset training and evaluation found no passage addressing the claimed deepfake-detection result. (LLM, conf=0.76)
Experimental — accuracy not yet measured
During the Facebook Deepfake Detection Challenge, top approaches showed much lower error on the public validation set than on the secret test set containing unseen data and deepfakes generated by undisclosed methods.
“This problem was clearly shown during the Facebook's Deepfake Detection Challenge 2 , when the top approaches of the competition have consistently shown a much lower error on the public validation set compared to the error on the secret test set, which contained unseen data and deepfakes generated using undisclosed methods.”
From Improving Generalization of Deepfake Detection With Data Farming and Few-Shot Learning (2022) · cites this paper as [2]
The Appendix (p. 9) compares validation and test accuracy for ImageNet, not for the Facebook Deepfake Detection Challenge. Searches for the challenge, unseen deepfakes, and validation-versus-secret-test error found no passage addressing the claim in our copy. (LLM, conf=0.78)
Experimental — accuracy not yet measured
The Facebook Deepfake Detection Challenge database has a strict license and was not made publicly available outside the challenge.
“The largest to-date database that was generated by Facebook [7] for the Deepfake Detection Challenge has a strict license and was not made publicly available outside of the challenge, so, we had to exclude it from this study.”
From Improving Generalization of Deepfake Detection With Data Farming and Few-Shot Learning (2022) · cites this paper as [2]
The transfer-learning datasets table (Section 5.2, p. 8) lists image-classification datasets, not the Facebook Deepfake Detection Challenge database. Searches for the challenge, its license, and public availability found no passage in our copy addressing the claim. (LLM, conf=0.76)
Experimental — accuracy not yet measured
ScanNet provides 2.5 million RGB-D frames across 1,513 scenes with crowdsourced annotations for semantic segmentation and 3D modeling.
“ScanNet [238], through its 2.5 million RGB-D frames across 1,513 scenes, utilizes crowdsourced annotations to provide a large-scale, high-quality dataset for semantic segmentation and 3D modeling.”
From Multimodal fusion and vision–language models: A survey for robot vision (2026) · cites this paper as [238]
The transfer-learning dataset table (Section 5.2, p. 8) lists image-classification datasets, not ScanNet. Searches for ScanNet, 2.5 million RGB-D frames, 1,513 scenes, and semantic segmentation found no passage in our copy addressing the claimed dataset or its annotations. (LLM, conf=0.78)
Experimental — accuracy not yet measured
A pure Transformer backbone (PVT) without convolutions can be easily plugged into dense prediction models (e.g., RetinaNet, Mask R-CNN, Semantic FPN) and obtain high-quality results.
“These results indicate that a pure Transformer backbone ( i.e. , PVT) without convolutions can also be easily plugged in dense prediction models ( e.g. , RetinaNet [39], Mask R-CNN [21], and Semantic FPN [32]), and obtain high-quality results.”
From Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions (2021) · cites this paper as [21]
The Abstract (p. 1) describes scaling convolutional networks to create EfficientNets, not a PVT Transformer backbone. Searches for PVT, the named dense-prediction models, and dense-prediction backbones found no passage in our copy addressing the claimed PVT results. (LLM, conf=0.75)
Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
Densely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566 · see citing works
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562 · see citing works
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · see citing works
Squeeze-and-Excitation Networks
Hu, J., Shen, L., & Sun, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00745 · Cited by 35,667 · see citing works
Untitled work
(2014) · Cited by 33,419 · see citing works
Rethinking the Inception Architecture for Computer Vision
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.308 · Cited by 32,138 · see citing works
Feature Pyramid Networks for Object Detection
Lin, T.-Y., Dollar, P., Girshick, R., He, K., Hariharan, B., & Belongie, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.106 · Cited by 30,939 · see citing works
Mask R-CNN
He, K., Gkioxari, G., Dollar, P., & Girshick, R. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.322 · Cited by 29,137 · see citing works
MobileNetV2: Inverted Residuals and Linear Bottlenecks
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L.-C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00474 · Cited by 28,755 · see citing works
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818 · see citing works
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,890 · see citing works
Xception: Deep Learning with Depthwise Separable Convolutions
Chollet, F. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.195 · Cited by 20,380 · see citing works
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., & Adam, H. (2017) · arXiv · doi:10.48550/arxiv.1704.04861 · Cited by 19,015 · see citing works
Aggregated Residual Transformations for Deep Neural Networks
Xie, S., Girshick, R., Dollar, P., Tu, Z., & He, K. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.634 · Cited by 12,465 · see citing works
Learning Deep Features for Discriminative Localization
Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., & Torralba, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.319 · Cited by 11,428 · see citing works
ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
Zhang, X., Zhou, X., Lin, M., & Sun, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00716 · Cited by 9,982 · see citing works
ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design
Ma, N., Zhang, X., Zheng, H.-T., & Sun, J. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01264-9_8 · Cited by 7,356 · see citing works
Wide Residual Networks
Zagoruyko, S., & Komodakis, N. (2016) · Procedings of the British Machine Vision Conference 2016 · doi:10.5244/c.30.87 · Cited by 7,244 · see citing works
Learning Transferable Architectures for Scalable Image Recognition
Zoph, B., Vasudevan, V., Shlens, J., & Le, Q. V. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00907 · Cited by 6,631 · see citing works
Self-citation
SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
Iandola, F. N., Han, S., Moskewicz, M. W., Ashraf, K., Dally, W. J., & Keutzer, K. (2016) · arXiv · doi:10.48550/arxiv.1602.07360 · Cited by 6,201 · see citing works
Neural Architecture Search with Reinforcement LearningPreprint
Zoph, B., & Le, Q. V. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.01578 · Cited by 5,091 · see citing works
Self-citation
Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning
Szegedy, C., Ioffe, S., Vanhoucke, V., & Alemi, A. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.11231 · Cited by 4,636 · see citing works
AutoAugment: Learning Augmentation Strategies From Data
Cubuk, E. D., Zoph, B., Mané, D., Vasudevan, V., & Le, Q. V. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00020 · Cited by 4,026 · see citing works
Self-citation
Gaussian Error Linear Units (GELUs)
Hendrycks, D., & Gimpel, K. (2016) · arXiv · doi:10.48550/arxiv.1606.08415 · Cited by 3,537 · see citing works
Automated Flower Classification over a Large Number of Classes
Nilsback, M.-E., & Zisserman, A. (2008) · 2008 Sixth Indian Conference on Computer Vision, Graphics & Image Processing · doi:10.1109/icvgip.2008.47 · Cited by 3,442 · see citing works
MnasNet: Platform-Aware Neural Architecture Search for Mobile
Tan, M., Chen, B., Pang, R., Vasudevan, V., Sandler, M., Howard, A., & Le, Q. V. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00293 · Cited by 3,416 · see citing works
Self-citation
Searching for activation functions
Ramachandran, P., Zoph, B., & Le, Q. V. (2017) · arXiv · doi:10.48550/arxiv.1710.05941 · Cited by 2,335 · see citing works
Self-citation
Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learning
Elfwing, S., Uchibe, E., & Doya, K. (2018) · Neural Networks · doi:10.1016/j.neunet.2017.12.012 · Cited by 2,119 · see citing works
Food-101 – Mining Discriminative Components with Random Forests
Bossard, L., Guillaumin, M., & Gool, L. V. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10599-4_29 · Cited by 2,105 · see citing works
Deep Networks with Stochastic Depth
Huang, G., Sun, Y., Liu, Z., Sedra, D., & Weinberger, K. Q. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_39 · Cited by 2,104 · see citing works
Progressive Neural Architecture Search
Liu, C., Zoph, B., Neumann, M., Shlens, J., Wei, H., Li, L.-J., Fei-Fei, L., Yuille, A., Huang, J., & Murphy, K. P. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01246-5_2 · Cited by 1,946 · see citing works
ProxylessNAS: Direct Neural Architecture Search on Target Task and HardwarePreprint
Cai, H., Zhu, L., & Han, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.00332 · Cited by 1,652 · see citing works
Cats and dogs
Parkhi, O., Vedaldi, A., Zisserman, A., & Jawahar, C. V. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6248092 · Cited by 1,563 · see citing works
AMC: AutoML for Model Compression and Acceleration on Mobile Devices
He, Y., Lin, J., Liu, Z., Wang, H., Li, L.-J., & Han, S. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_48 · Cited by 1,400 · see citing works
Do Better ImageNet Models Transfer Better?
Kornblith, S., Shlens, J., & Le, Q. V. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00277 · Cited by 1,325 · see citing works
Self-citation
Fine-Grained Visual Classification of AircraftPreprint
Maji, S., Rahtu, E., Kannala, J., Blaschko, M. B., & Vedaldi, A. (2013) · HAL · doi:10.48550/arxiv.1306.5151 · Cited by 1,250 · see citing works
Exploring the Limits of Weakly Supervised Pretraining
Mahajan, D., Girshick, R., Ramanathan, V., He, K., Paluri, M., Li, Y., Bharambe, A., & van der Maaten, L. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01216-8_12 · Cited by 1,224 · see citing works
GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism
Huang, Y., Cheng, Y., Bapna, A., Firat, O., Chen, M. X., Chen, D., Lee, H., Ngiam, J., Le, Q. V., Wu, Y., et al. (2018) · arXiv · doi:10.48550/arxiv.1811.06965 · Cited by 1,180 · see citing works
Self-citation
Bridging Nonlinearities and Stochastic Regularizers with Gaussian Error Linear UnitsPreprint
Hendrycks, D., & Gimpel, K. (2016) · arXiv (Cornell University) · Cited by 753 · see citing works
NetAdapt: Platform-Aware Neural Network Adaptation for Mobile Applications
Yang, T.-J., Howard, A., Chen, B., Zhang, X., Go, A., Sandler, M., Sze, V., & Adam, H. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01249-6_18 · Cited by 572 · see citing works
Neural Architecture OptimizationPreprint
Luo, R., Tian, F., Qin, T., Chen, E., & Liu, T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.07233 · Cited by 522 · see citing works
On the Expressive Power of Deep Neural NetworksPreprint
Raghu, M., Poole, B., Kleinberg, J. M., Ganguli, S., & Sohl‐Dickstein, J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.05336 · Cited by 519 · see citing works
The expressive power of neural networks: a view from the width
Zhou, L., Pu, H., Wang, F., Hu, Z., & Wang, L. (2017) · neural information processing systems · Cited by 516 · see citing works
SqueezeNext: Hardware-Aware Neural Network Design
Gholami, A., Kwon, K., Wu, B., Tai, Z., Yue, X., Jin, P., Zhao, S., & Keutzer, K. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2018.00215 · Cited by 350 · see citing works
Regularized Evolution for Image Classifier Architecture Search
Real, E., Aggarwal, A., Huang, Y., & Le, Q. V. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33014780 · Cited by 328 · see citing works
Self-citation
Birdsnap: Large-Scale Fine-Grained Visual Categorization of Birds
Berg, T., Liu, J., Lee, S. W., Alexander, M. L., Jacobs, D. W., & Belhumeur, P. N. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.259 · Cited by 313 · see citing works
PolyNet: A Pursuit of Structural Diversity in Very Deep Networks
Zhang, X., Li, Z., Loy, C. C., & Lin, D. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.415 · Cited by 258 · see citing works
ResNet with one-neuron hidden layers is a Universal ApproximatorPreprint
Lin, H., & Jegelka, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.10909 · Cited by 168 · see citing works
Domain Adaptive Transfer Learning with Specialist ModelsPreprint
Ngiam, J., Peng, D., Vasudevan, V., Kornblith, S., Le, Q. V., & Pang, R. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.07056 · Cited by 106 · see citing works
Self-citation
Collecting a Large-scale Dataset of Fine-grained Cars
Krause, J., Deng, J., Stark, M. M., & Fei-Fei, L. (2013) · MPG.PuRe (Max Planck Society) · Cited by 100 · see citing works
On the Expressive Power of Overlapping Architectures of Deep LearningPreprint
Sharir, O., & Shashua, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.02065 · Cited by 22 · see citing works
Exploring the Limits of Weakly Supervised Pretraining
Mahajan, D., Girshick, R., Ramanathan, V., He, K., Paluri, M., Li, Y., Bharambe, A., & van der Maaten, L. (2018) · arXiv · doi:10.48550/arxiv.1805.00932

Flagged first, then by citations

Research works printed but not linked: 1

Han, S., Mao, H., and Dally, W. J. Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. ICLR , 2016.

Sorted by citations

Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534 · see citing works
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293 · see citing works
YOLOv7: Trainable Bag-of-Freebies Sets New State-of-the-Art for Real-Time Object Detectors
Wang, C.-Y., Bochkovskiy, A., & Liao, H.-Y. M. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00721 · Cited by 13,198 · see citing works
“For example the scaling model of EfficientNet [69] considers the width, depth, and resolution.”
Method
YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors
Wang, C.-Y., Bochkovskiy, A., & Liao, H.-Y. M. (2022) · arXiv · doi:10.48550/arxiv.2207.02696 · Cited by 13,205 · see citing works
“For example the scaling model of EfficientNet [69] considers the width, depth, and resolution.”
Method
YOLOv4: Optimal Speed and Accuracy of Object Detection
Bochkovskiy, A., Wang, C.-Y., & Liao, H.-Y. M. (2020) · arXiv · doi:10.48550/arxiv.2004.10934 · Cited by 10,307 · see citing works
EfficientDet: Scalable and Efficient Object Detection
Tan, M., Pang, R., & Le, Q. V. (2019) · arXiv · Cited by 9,841 · see citing works
“Inspired by recent works [36], we propose a compound scaling method for object detectors, which jointly scales up the resolution/depth/width for all backbone, feature network, box/class prediction network.”
BackgroundMethodResultInfluentialSelf-citation
A ConvNet for the 2020s
Liu, Z., Mao, H., Wu, C.-Y., Feichtenhofer, C., Darrell, T., & Xie, S. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01167 · Cited by 8,494 · see citing works
ArcFace: Additive Angular Margin Loss for Deep Face Recognition
Deng, J., Guo, J., Xue, N., & Zafeiriou, S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00482 · Cited by 7,781 · see citing works
Coordinate Attention for Efficient Mobile Network Design
Hou, Q., Zhou, D., & Feng, J. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01350 · Cited by 6,961 · see citing works
CSPNet: A New Backbone that can Enhance Learning Capability of CNN
Wang, C.-Y., Mark Liao, H.-Y., Wu, Y.-H., Chen, P.-Y., Hsieh, J.-W., & Yeh, I.-H. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00203 · Cited by 5,265 · see citing works
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions
Wang, W., Xie, E., Li, X., Fan, D.-P., Song, K., Liang, D., Lu, T., Luo, P., & Shao, L. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00061 · Cited by 5,259 · see citing works
Untitled work
Han (2023) · doi:10.1109/tpami.2022.3152247 · Cited by 4,682 · see citing works
Knowledge Distillation: A Survey
Gou, J., Yu, B., Maybank, S. J., & Tao, D. (2021) · International Journal of Computer Vision · doi:10.1007/s11263-021-01453-z · Cited by 3,931 · see citing works
A Comprehensive Review of YOLO Architectures in Computer Vision: From YOLOv1 to YOLOv8 and YOLO-NAS
Terven, J. R., Esparza, D. M. C., & Romero-González, J.-A. (2023) · Machine Learning and Knowledge Extraction · doi:10.3390/make5040083 · Cited by 3,681 · see citing works
Transformers in Vision: A Survey
Khan, S., Naseer, M., Hayat, M., Zamir, S. W., Khan, F. S., & Shah, M. (2022) · ACM Computing Surveys · doi:10.1145/3505244 · Cited by 3,595 · see citing works
Randaugment: Practical automated data augmentation with a reduced search space
Cubuk, E. D., Zoph, B., Shlens, J., & Le, Q. V. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00359 · Cited by 3,223 · see citing works
Self-citation
RepVGG: Making VGG-style ConvNets Great Again
Ding, X., Zhang, X., Ma, N., Han, J., Ding, G., & Sun, J. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01352 · Cited by 2,907 · see citing works
CvT: Introducing Convolutions to Vision Transformers
Wu, H., Xiao, B., Codella, N., Liu, M., Dai, X., Yuan, L., & Zhang, L. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00009 · Cited by 2,649 · see citing works
Run, Don't Walk: Chasing Higher FLOPS for Faster Neural Networks
Chen, J., Kao, S.-H., He, H., Zhuo, W., Song, W., Lee, C.-H., & Chan, S.-H. G. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01157 · Cited by 2,556 · see citing works
Self-Training With Noisy Student Improves ImageNet Classification
Xie, Q., Luong, M.-T., Hovy, E., & Le, Q. V. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01070 · Cited by 2,393 · see citing works
Self-citation
TPH-YOLOv5: Improved YOLOv5 Based on Transformer Prediction Head for Object Detection on Drone-captured Scenarios
Zhu, X., Lyu, S., Wang, X., & Zhao, Q. (2021) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw54120.2021.00312 · Cited by 2,382 · see citing works
CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification
Chen, C.-F. R., Fan, Q., & Panda, R. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00041 · Cited by 2,242 · see citing works
Designing Network Design Spaces
Radosavovic, I., Kosaraju, R. P., Girshick, R., He, K., & Dollar, P. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01044 · Cited by 2,084 · see citing works
U-Net and Its Variants for Medical Image Segmentation: A Review of Theory and Applications
Siddique, N. A., Paheding, S., Elkin, C. P., & Devabhaktuni, V. (2021) · IEEE Access · doi:10.1109/access.2021.3086020 · Cited by 2,060 · see citing works
ResNeSt: Split-Attention Networks
Zhang, H., Wu, C., Zhang, Z., Zhu, Y., Lin, H., Zhang, Z., Sun, Y., He, T., Mueller, J. W., Manmatha, R., et al. (2022) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00309 · Cited by 1,965 · see citing works
Aleatoric and epistemic uncertainty in machine learning: an introduction to concepts and methods
Hüllermeier, E., & Waegeman, W. (2021) · Machine Learning · doi:10.1007/s10994-021-05946-3 · Cited by 1,834 · see citing works
“Figure 1: Predictions by EfficientNet (Tan and Le, 2019) on test images from ImageNet: For the left image, the neural network predicts “typewriter keyboard” with certainty 83.”
Method
ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoders
Woo, S., Debnath, S., Hu, R., Chen, X., Liu, Z., Kweon, I. S., & Xie, S. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01548 · Cited by 1,791 · see citing works
AutoML: A survey of the state-of-the-art
He, X. J., Zhao, K., & Chu, X. (2020) · Knowledge-Based Systems · doi:10.1016/j.knosys.2020.106622 · Cited by 1,778 · see citing works
Scaled-YOLOv4: Scaling Cross Stage Partial Network
Wang, C.-Y., Bochkovskiy, A., & Liao, H.-Y. M. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01283 · Cited by 1,756 · see citing works
Scaling Laws for Neural Language Models
Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., & Amodei, D. (2020) · arXiv · Cited by 1,749 · see citing works
Artificial intelligence: A powerful paradigm for scientific research
Xu, Y., Liu, X., Cao, X., Huang, C., Liu, E., Qian, S., Liu, X., Wu, Y., Dong, F., Qiu, C.-W., et al. (2021) · The Innovation · doi:10.1016/j.xinn.2021.100179 · Cited by 1,726 · see citing works
Scaling Up Your Kernels to 31×31: Revisiting Large Kernel Design in CNNs
Ding, X., Zhang, X., Han, J., & Ding, G. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01166 · Cited by 1,509 · see citing works
EfficientNetV2: Smaller Models and Faster TrainingPreprint
Tan, M., & Le, Q. V. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.00298 · Cited by 1,471 · see citing works
Self-citation
CSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped Windows
Dong, X., Bao, J., Chen, D., Zhang, W., Yu, N., Yuan, L., Chen, D., & Guo, B. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01181 · Cited by 1,458 · see citing works
PaDiM: A Patch Distribution Modeling Framework for Anomaly Detection and Localization
Defard, T., Setkov, A., Loesch, A., & Audigier, R. (2021) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-68799-1_35 · Cited by 1,415 · see citing works
Multiscale vision transformers
Fan, H., Xiong, B., Mangalam, K., Li, Y., Yan, Z., Malik, J., & Feichtenhofer, C. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00675 · Cited by 1,407 · see citing works
A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection
Li, Q., Wen, Z., Wu, Z., Hu, S., Wang, N., Li, Y., Liu, X., & He, B. (2023) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3124599 · Cited by 1,407 · see citing works
Transformer in Transformer
Han, K., Xiao, A., Wu, E., Guo, J., Xu, C., & Wang, Y. (2021) · arXiv · doi:10.48550/arxiv.2103.00112 · Cited by 1,394 · see citing works
“Note that all the transformer-based models are still inferior to EfficientNet which utilizes special depth-wise convolutions, so it is yet a challenge of how to beat EfficientNet using pure transformer.”
Method
Bottleneck Transformers for Visual Recognition
Srinivas, A., Lin, T.-Y., Parmar, N., Shlens, J., Abbeel, P., & Vaswani, A. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01625 · Cited by 1,385 · see citing works
Lift, Splat, Shoot: Encoding Images from Arbitrary Camera Rigs by Implicitly Unprojecting to 3D
Philion, J., & Fidler, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58568-6_12 · Cited by 1,271 · see citing works
X3D: Expanding Architectures for Efficient Video Recognition
Feichtenhofer, C. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00028 · Cited by 1,262 · see citing works
Going deeper with Image Transformers
Touvron, H., Cord, M., Sablayrolles, A., Synnaeve, G., & Jegou, H. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00010 · Cited by 1,259 · see citing works
Rotate to Attend: Convolutional Triplet Attention Module
Misra, D., Nalamada, T., Arasanipalai, A. U., & Hou, Q. (2021) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv48630.2021.00318 · Cited by 1,259 · see citing works
Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentation
Ghiasi, G., Cui, Y., Srinivas, A., Qian, R., Lin, T.-Y., Cubuk, E. D., Le, Q. V., & Zoph, B. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00294 · Cited by 1,248 · see citing works
Self-citation
Vision Transformers for Single Image Dehazing
Song, Y., He, Z., Qian, H., & Du, X. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3256763 · Cited by 1,248 · see citing works
A Survey of Quantization Methods for Efficient Neural Network Inference
Gholami, A., Kim, S., Dong, Z., Yao, Z., Mahoney, M. W., & Keutzer, K. (2022) · Low-Power Computer Vision · doi:10.1201/9781003162810-13 · Cited by 1,219 · see citing works
YOLO-v1 to YOLO-v8, the Rise of YOLO and Its Complementary Nature toward Digital Manufacturing and Industrial Defect Detection
Hussain, M. (2023) · Machines · doi:10.3390/machines11070677 · Cited by 1,217 · see citing works
Grounded Language-Image Pre-training
Li, L. H., Zhang, P., Zhang, H., Yang, J., Li, C., Zhong, Y., Wang, L., Yuan, L., Zhang, L., Hwang, J., et al. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01069 · Cited by 1,197 · see citing works
AST: Audio Spectrogram Transformer
Gong, Y., Chung, Y.-A., & Glass, J. (2021) · Interspeech 2021 · doi:10.21437/interspeech.2021-698 · Cited by 1,159 · see citing works
SimAM: A Simple, Parameter-Free Attention Module for Convolutional Neural Networks
Yang, L., Zhang, R., Li, L., & Xie, X. (2021) · International Conference on Machine Learning · Cited by 1,130 · see citing works
MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer
Mehta, S., & Rastegari, M. (2021) · arXiv · doi:10.48550/arxiv.2110.02178 · Cited by 1,110 · see citing works
CutPaste: Self-Supervised Learning for Anomaly Detection and Localization
Li, C.-L., Sohn, K., Yoon, J., & Pfister, T. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00954 · Cited by 1,107 · see citing works
InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutions
Wang, W., Dai, J., Chen, Z., Huang, Z., Li, Z., Zhu, X., Hu, X., Lu, T., Lu, L., Li, H., et al. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01385 · Cited by 1,080 · see citing works
Vision-Language Models for Vision Tasks: A Survey
Zhang, J., Huang, J., Jin, S., & Lu, S. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3369699 · Cited by 1,051 · see citing works
Recent advances in deep learning for object detection
Wu, X., Sahoo, D., & Hoi, S. C. H. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.01.085 · Cited by 1,047 · see citing works
Common pitfalls and recommendations for using machine learning to detect and prognosticate for COVID-19 using chest radiographs and CT scans
Roberts, M., Driggs, D., Thorpe, M., Gilbey, J., Yeung, M., Ursprung, S., Aviles-Rivero, A. I., Etmann, C., McCague, C., Beer, L., et al. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00307-0 · Cited by 1,044 · see citing works
Big Transfer (BiT): General Visual Representation Learning
Kolesnikov, A., Beyer, L., Zhai, X., Puigcerver, J., Yung, J., Gelly, S., & Houlsby, N. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58558-7_29 · Cited by 1,039 · see citing works
A survey of modern deep learning based object detection models
Zaidi, S. S. A., Ansari, M. S., Aslam, A., Kanwal, N., Asghar, M. N., & Lee, B. (2022) · Digital Signal Processing · doi:10.1016/j.dsp.2022.103514 · Cited by 1,029 · see citing works
Multi-attentional Deepfake Detection
Zhao, H., Wei, T., Zhou, W., Zhang, W., Chen, D., & Yu, N. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00222 · Cited by 1,019 · see citing works
CMT: Convolutional Neural Networks Meet Vision Transformers
Guo, J., Han, K., Wu, H., Tang, Y., Chen, X., Wang, Y., & Xu, C. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01186 · Cited by 1,005 · see citing works
Plant leaf disease classification using EfficientNet deep learning model
Atila, Ü., UÇAR, M. B., Akyol, K., & Uçar, E. (2020) · Ecological Informatics · doi:10.1016/j.ecoinf.2020.101182 · Cited by 1,002 · see citing works
A Survey on Deep Semi-Supervised Learning
Yang, X., Song, Z., King, I., & Xu, Z. (2022) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2022.3220219 · Cited by 981 · see citing works
EfficientViT: Memory Efficient Vision Transformer with Cascaded Group Attention
Liu, X., Peng, H., Zheng, N., Yang, Y., Hu, H., & Yuan, Y. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01386 · Cited by 956 · see citing works
LeViT: a Vision Transformer in ConvNet’s Clothing for Faster Inference
Graham, B., El-Nouby, A., Touvron, H., Stock, P., Joulin, A., Jegou, H., & Douze, M. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01204 · Cited by 935 · see citing works
CoAtNet: Marrying Convolution and Attention for All Data Sizes
Dai, Z., Liu, H., Le, Q. V., & Tan, M. (2021) · arXiv · doi:10.48550/arxiv.2106.04803 · Cited by 925 · see citing works
Self-citation
Rethinking BiSeNet For Real-time Semantic Segmentation
Fan, M., Lai, S., Huang, J., Wei, X., Chai, Z., Luo, J., & Wei, X. K. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00959 · Cited by 913 · see citing works
Conformer: Local Features Coupling Global Representations for Visual Recognition
Peng, Z., Huang, W. F., Gu, S., Xie, L., Wang, Y., Jiao, J., & Ye, Q. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00042 · Cited by 900 · see citing works
Scaling Vision Transformers
Zhai, X., Kolesnikov, A., Houlsby, N., & Beyer, L. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01179 · Cited by 891 · see citing works
AdaBins: Depth Estimation Using Adaptive Bins
Bhat, S. F., Alhashim, I., & Wonka, P. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00400 · Cited by 862 · see citing works
Deep Dual-Resolution Networks for Real-Time and Accurate Semantic Segmentation of Traffic Scenes
Pan, H., Hong, Y., Sun, W., & Jia, Y. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2022.3228042 · Cited by 856 · see citing works
MViTv2: Improved Multiscale Vision Transformers for Classification and Detection
Li, Y., Wu, C.-Y., Fan, H., Mangalam, K., Xiong, B., Malik, J., & Feichtenhofer, C. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00476 · Cited by 837 · see citing works
SCConv: Spatial and Channel Reconstruction Convolution for Feature Redundancy
Li, J., Wen, Y., & He, L. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00596 · Cited by 823 · see citing works
Twins: Revisiting the Design of Spatial Attention in Vision Transformers
Chu, X., Tian, Z., Wang, Y., Zhang, B., Ren, H., Wei, X., Xia, H., & Shen, C. (2021) · arXiv · doi:10.48550/arxiv.2104.13840 · Cited by 817 · see citing works
ResMLP: Feedforward Networks for Image Classification With Data-Efficient Training
Touvron, H., Bojanowski, P., Caron, M., Cord, M., El-Nouby, A., Grave, E., Izacard, G., Joulin, A., Synnaeve, G., Verbeek, J., et al. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3206148 · Cited by 807 · see citing works
A Comprehensive Survey of Neural Architecture Search
Ren, P., Xiao, Y., Chang, X., Huang, P.-Y., Li, Z., Chen, X., & Wang, X. Y. (2021) · ACM Computing Surveys · doi:10.1145/3447582 · Cited by 808 · see citing works
ConViT: improving vision transformers with soft convolutional inductive biases
d’Ascoli, S., Touvron, H., Leavitt, M. L., Morcos, A. S., Biroli, G., & Sagun, L. (2022) · Journal of Statistical Mechanics: Theory and Experiment · doi:10.1088/1742-5468/ac9830 · Cited by 782 · see citing works
Searching Efficient 3D Architectures with Sparse Point-Voxel Convolution
Tang, H., Liu, Z., Zhao, S., Lin, Y., Lin, J., Wang, H., & Han, S. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58604-1_41 · Cited by 781 · see citing works
Machine Learning in Python: Main Developments and Technology Trends in Data Science, Machine Learning, and Artificial Intelligence
Raschka, S., Patterson, J. M., & Nolet, C. (2020) · Information · doi:10.3390/info11040193 · Cited by 777 · see citing works
Asymmetric Loss For Multi-Label Classification
Ridnik, T., Ben-Baruch, E., Zamir, N., Noy, A., Friedman, I., Protter, M., & Zelnik‐Manor, L. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00015 · Cited by 773 · see citing works
Contextual Transformer Networks for Visual Recognition
Li, Y., Yao, T., Pan, Y., & Mei, T. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3164083 · Cited by 771 · see citing works
MDETR - Modulated Detection for End-to-End Multi-Modal Understanding
Kamath, A., Singh, M., LeCun, Y., Synnaeve, G., Misra, I., & Carion, N. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00180 · Cited by 761 · see citing works
Deep learning in computer vision: A critical review of emerging techniques and application scenarios
Chai, J., Zeng, H., Li, A., & Ngai, E. W. T. (2021) · Machine Learning with Applications · doi:10.1016/j.mlwa.2021.100134 · Cited by 754 · see citing works
Rep ViT: Revisiting Mobile CNN From ViT Perspective
Wang, A., Chen, H., Lin, Z., Han, J., & Ding, G. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01506 · Cited by 752 · see citing works
The Computational Limits of Deep Learning
Thompson, N., Greenewald, K. H., Lee, K., & Manso, G. F. (2023) · Ninth Computing within Limits 2023 · doi:10.21428/bf6fb269.1f033948 · Cited by 737 · see citing works
Ensemble Distillation for Robust Model Fusion in Federated LearningPreprint
Lin, T., Kong, L., Stich, S. U., & Jaggi, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.07242 · Cited by 733 · see citing works
Mobile-Former: Bridging MobileNet and Transformer
Chen, Y., Dai, X., Chen, D., Liu, M., Dong, X., Yuan, L., & Liu, Z. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00520 · Cited by 731 · see citing works
Hyper-Parameter Optimization: A Review of Algorithms and ApplicationsPreprint
Yu, T., & Zhu, H. C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.05689 · Cited by 729 · see citing works
HINet: Half Instance Normalization Network for Image Restoration
Chen, L., Lu, X., Zhang, J., Chu, X., & Chen, C. (2021) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00027 · Cited by 698 · see citing works
Rethinking Spatial Dimensions of Vision Transformers
Heo, B., Yun, S., Han, D., Chun, S., Choe, J., & Oh, S. J. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01172 · Cited by 691 · see citing works
Mish: A Self Regularized Non-Monotonic Activation Function
Misra, D. (2020) · British Machine Vision Conference (BMVC) · doi:10.5244/c.34.191 · Cited by 683 · see citing works
Multi-animal pose estimation, identification and tracking with DeepLabCut
Lauer, J., Zhou, M., Ye, S., Menegas, W., Schneider, S., Nath, T., Rahman, M. M., Santo, V. D., Soberanes, D., Feng, G., et al. (2022) · Nature Methods · doi:10.1038/s41592-022-01443-0 · Cited by 665 · see citing works
Rewrite the Stars
Ma, X., Dai, X., Bai, Y., Wang, Y., & Fu, Y. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00544 · Cited by 663 · see citing works
Untitled work
(2022) · doi:10.15607/rss.2023.xix.025 · Cited by 916 · see citing works
Meta Pseudo Labels
Pham, H., Dai, Z., Xie, Q., & Le, Q. V. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01139 · Cited by 643 · see citing works
Self-citation
UniFormer: Unifying Convolution and Self-Attention for Visual Recognition
Li, K., Wang, Y., Zhang, J., Gao, P., Song, G., Liu, Y., Li, H., & Qiao, Y. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3282631 · Cited by 636 · see citing works
EAPT: Efficient Attention Pyramid Transformer for Image Processing
Lin, X., Sun, S., Huang, W., Sheng, B., Li, P., & Feng, D. D. (2021) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2021.3120873 · Cited by 614 · see citing works
A Survey of Visual Transformers
Liu, Y., Zhang, Y., Wang, Y., Hou, F., Yuan, J. S., Tian, J., Zhang, Y. S., Shi, Z., Jianping Fan, B., & He, Z. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3227717 · Cited by 614 · see citing works
AI recognition of patient race in medical imaging: a modelling study
Gichoya, J. W., Banerjee, I., Bhimireddy, A. R., Burns, J. L., Celi, L. A., Chen, L.-C., Correa, R., Dullerud, N., Ghassemi, M., Huang, S.-C., et al. (2022) · The Lancet Digital Health · doi:10.1016/s2589-7500(22)00063-2 · Cited by 611 · see citing works
Incorporating Convolution Designs into Visual Transformers
Yuan, K., Guo, S., Liu, Z., Zhou, A., Yu, F., & Wu, W. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00062 · Cited by 611 · see citing works
Imbalance Problems in Object Detection: A Review
Öksüz, K., Çam, B. C., Kalkan, S., & Akbaş, E. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.2981890 · Cited by 607 · see citing works

EfficientNetV2: Smaller Models and Faster TrainingPreprint
Tan, M., & Le, Q. V. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.00298 · Cited by 1,471 · see citing works
Self-citation
Efficient and accurate compound scaling for convolutional neural networks
Lin, C.-M., Yang, P., Wang, Q., Qiu, Z., Lv, W., & Wang, Z. (2023) · Neural Networks · doi:10.1016/j.neunet.2023.08.053 · Cited by 46 · see citing works
EfficientNet: Rethinking Model Scaling via Compound Coefficient Optimization for Resource-Constrained AI
Revista, Z., & IA, 1. (2025) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.17824732
Hyper-EfficientNet: Learning Optimal Compound Scaling Factors with Meta-Architectures
Revista, Z., & IA, 1. (2025) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.17816450
Revisiting ResNets: Improved Training and Scaling StrategiesPreprint
Bello, I., Fedus, W., Du, X., Cubuk, E. D., Srinivas, A., Lin, T.-Y., Shlens, J., & Zoph, B. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.07579 · Cited by 257 · see citing works

Authors: Tan, M., & Le, Q. V.
Affiliations: Google (United States) (US)
ORCID: Le, Quoc V.
Venue: arXiv
Publisher: arXiv
Published: 2019
ISSN: 2331-8422 (ISSN-L)
DOI: 10.48550/arxiv.1905.11946
arXiv: 1905.11946
OpenAlex: W2946948417
Cited by: 17,072
References: 49
Citations by year: 2026: 193 · 2025: 2341 · 2024: 3966 · 2023: 3735 · 2022: 2876 · 2021: 2753 · 2020: 1075 · 2019: 123 · 2018: 4 · 2017: 1 · 2015: 1
Type: posted content
Language: en
Open access: green
arxiv.org
Licence: arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations: arXiv (Cornell University) (submittedVersion)
arXiv (Cornell University)
Topics: Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
Keywords: EfficientNet, model scaling, convolutional neural networks, compound scaling, MobileNet, ResNet, neural architecture search, transfer learning, ImageNet, model efficiency

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks

Abstract

Integrity signals

Claims the paper makes

Claims other papers make about this one

Self-check

Cites

Cited by

Related

Record

About these checks