On Network Design Spaces for Visual Recognition
Open accessAbstract
Over the past several years progress in designing better neural network architectures for visual recognition has been substantial. To help sustain this rate of progress, in this work we propose to reexamine the methodology for comparing network architectures. In particular, we introduce a new comparison paradigm of distribution estimates, in which network design spaces are compared by applying statistical techniques to populations of sampled models, while controlling for confounding factors like network complexity. Compared to current methodologies of comparing point and curve estimates of model families, distribution estimates paint a more complete picture of the entire design landscape. As a case study, we examine design spaces used in neural architecture search (NAS). We find significant statistical differences between recent NAS design space variants that have been largely overlooked. Furthermore, our analysis reveals that the design spaces for standard model families like ResNeXt can be comparable to the more complex ones used in recent NAS work. We hope these insights into distribution analysis will enable more robust progress toward discovering better networks for visual recognition.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
An efficient point-voxel CNN is proposed for efficient 3D learning and can help implement 3D AR/VR applications and SLAM of autonomous driving on edge systems.
“Like [218], an efficient point-voxel CNN is proposed for efficient 3D learning, and this model can help to implement 3D AR/VR applications and SLAM [219] of autonomous driving on edge systems.”
From Bringing ai to edge: from deep learning’s perspective (2022) · cites this paper as [219]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · see citing works
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 74,248 · see citing works
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · see citing works
Densely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566 · see citing works
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562 · see citing works
The Nature of Statistical Learning Theory
Vapnik, V. N. (1995) · doi:10.1007/978-1-4757-2440-0 · Cited by 39,138 · see citing works
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818 · see citing works
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,890 · see citing works
Visualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839 · see citing works
Statistical Aspects of the Analysis of Data From Retrospective Studies of Disease
(1959) · JNCI: Journal of the National Cancer Institute · doi:10.1093/jnci/22.4.719 · Cited by 14,689 · cited 1× · see citing works
“Another way to account for complexity is stratified analysis [24].”
BackgroundAggregated Residual Transformations for Deep Neural Networks
Xie, S., Girshick, R., Dollar, P., Tu, Z., & He, K. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.634 · Cited by 12,465 · see citing works
Self-citation(2012) · Cited by 8,009 · see citing works
The Kolmogorov-Smirnov Test for Goodness of Fit
Massey, F. J. (1951) · Journal of the American Statistical Association · doi:10.1080/01621459.1951.10500769 · Cited by 7,210 · see citing works
Greff, K., Srivastava, R. K., Koutnik, J., Steunebrink, B. R., & Schmidhuber, J. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2582924 · Cited by 6,955 · see citing works
Practical Bayesian Optimization of Machine Learning Algorithms
Snoek, J., Larochelle, H., & Adams, R. P. (2012) · arXiv · doi:10.48550/arxiv.1206.2944 · Cited by 6,691 · see citing works
Learning Transferable Architectures for Scalable Image Recognition
Zoph, B., Vasudevan, V., Shlens, J., & Le, Q. V. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00907 · Cited by 6,631 · see citing works
Neural Architecture Search with Reinforcement LearningPreprint
Zoph, B., & Le, Q. V. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.01578 · Cited by 5,091 · see citing works
The Strength of Weak Learnability
Schapire, R. E. (1990) · Machine Learning · doi:10.1007/bf00116037 · Cited by 4,600 · see citing works
(2011) · Cited by 3,939 · see citing works
DARTS: Differentiable architecture search
Liu, H., Simonyan, K., & Yang, Y. (2018) · arXiv · doi:10.48550/arxiv.1806.09055 · Cited by 3,248 · see citing works
Understanding deep learning requires rethinking generalization
Zhang, C., Bengio, S., Hardt, M., Recht, B., & Vinyals, O. (2016) · arXiv · doi:10.48550/arxiv.1611.03530 · Cited by 2,838 · see citing works
Improved regularization of convolutional neural networks with cutout
DeVries, T., & Taylor, G. W. (2017) · arXiv · doi:10.48550/arxiv.1708.04552 · Cited by 2,776 · see citing works
Speed/Accuracy Trade-Offs for Modern Convolutional Object Detectors
Huang, J., Rathod, V., Sun, C., Zhu, M., Korattikara, A., Fathi, A., Fischer, I., Wojna, Z., Song, Y., Guadarrama, S., et al. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.351 · Cited by 2,743 · see citing works
Progressive Neural Architecture Search
Liu, C., Zoph, B., Neumann, M., Shlens, J., Wei, H., Li, L.-J., Fei-Fei, L., Yuille, A., Huang, J., & Murphy, K. P. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01246-5_2 · Cited by 1,946 · see citing works
Deep Reinforcement Learning That Matters
Henderson, P., Islam, R., Bachman, P., Pineau, J., Precup, D., & Meger, D. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11694 · Cited by 1,685 · see citing works
Deeply-Supervised NetsPreprint
Lee, C., Xie, S., Gallagher, P., Zhang, Z., & Tu, Z. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1409.5185 · Cited by 1,607 · see citing works
Self-citationConvolutional neural networks at constrained time cost
He, K., & Sun, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299173 · Cited by 1,558 · see citing works
(2018) · International Conference on Machine Learning · Cited by 1,465 · see citing works
Are GANs Created Equal? A Large-Scale StudyPreprint
Lučić, M., Kurach, K., Michalski, M., Gelly, S., & Bousquet, O. (2017) · Neural Information Processing Systems · doi:10.48550/arxiv.1711.10337 · Cited by 1,175 · see citing works
Efficient Neural Architecture Search via Parameter SharingPreprint
Pham, H., Guan, M. Y., Zoph, B., Le, Q. V., & Dean, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.03268 · Cited by 1,082 · see citing works
An empirical evaluation of deep architectures on problems with many factors of variation
Larochelle, H., Erhan, D., Courville, A. C., Bergstra, J., & Bengio, Y. (2007) · International Conference on Machine Learning · doi:10.1145/1273496.1273556 · Cited by 1,024 · see citing works
FractalNet: Ultra-Deep Neural Networks without ResidualsPreprint
Larsson, G., Maire, M., & Shakhnarovich, G. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.07648 · Cited by 810 · see citing works
Regularizing and Optimizing LSTM Language Models
Merity, S., Keskar, N. S., & Socher, R. (2017) · arXiv · doi:10.48550/arxiv.1708.02182 · Cited by 697 · see citing works
On the State of the Art of Evaluation in Neural Language ModelsPreprint
Melis, G., Dyer, C., & Blunsom, P. (2017) · International Conference on Learning Representations · doi:10.48550/arxiv.1707.05589 · Cited by 578 · see citing works
Neural Architecture OptimizationPreprint
Luo, R., Tian, F., Qin, T., Chen, E., & Liu, T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.07233 · Cited by 522 · see citing works
Sensitivity and Generalization in Neural Networks: an Empirical StudyPreprint
Novak, R., Bahri, Y., Abolafia, D. A., Pennington, J., & Sohl‐Dickstein, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.08760 · Cited by 344 · see citing works
Deep Reinforcement Learning that MattersPreprint
Henderson, P., Islam, R., Bachman, P., Pineau, J., Precup, D., & Meger, D. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.06560 · Cited by 340 · see citing works
Regularized Evolution for Image Classifier Architecture Search
Real, E., Aggarwal, A., Huang, Y., & Le, Q. V. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33014780 · Cited by 328 · see citing works
FractalNet: Ultra-Deep Neural Networks without Residuals.
Larsson, G., Maire, M., & Shakhnarovich, G. (2017) · International Conference on Learning Representations · Cited by 315 · see citing works
Lin, M., Chen, Q., & Yan, S. (2013) · arXiv · doi:10.48550/arxiv.1312.4400 · Cited by 252 · see citing works
Capacity and Trainability in Recurrent Neural NetworksPreprint
Collins, J., Sohl‐Dickstein, J., & Sussillo, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.09913 · Cited by 149 · see citing works
Are All Layers Created Equal?Preprint
Zhang, C., Samy Bengio, A., & Singer, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.01996 · Cited by 105 · see citing works
Flagged first, then by citations
5 of the 49 references couldn’t be matched to a known paper.
Research works printed but not linked: 1
- R. Luo, F. Tian, T. Qin, E. Chen, and T.-Y. Liu. Neural ar- chitecture optimization. In NIPS, 2018. 2
Cited by
Sorted by citations
Liu, Z., Mao, H., Wu, C.-Y., Feichtenhofer, C., Darrell, T., & Xie, S. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01167 · Cited by 8,494 · see citing works
Self-citationScalable Diffusion Models with Transformers
Peebles, W., & Xie, S. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00387 · Cited by 2,284 · see citing works
Self-citationDesigning Network Design Spaces
Radosavovic, I., Kosaraju, R. P., Girshick, R., He, K., & Dollar, P. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01044 · Cited by 2,084 · see citing works
Self-citationBottleneck Transformers for Visual Recognition
Srinivas, A., Lin, T.-Y., Parmar, N., Shlens, J., Abbeel, P., & Vaswani, A. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01625 · Cited by 1,385 · see citing works
Searching Efficient 3D Architectures with Sparse Point-Voxel Convolution
Tang, H., Liu, Z., Zhao, S., Lin, Y., Lin, J., Wang, H., & Han, S. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58604-1_41 · Cited by 781 · see citing works
Rep ViT: Revisiting Mobile CNN From ViT Perspective
Wang, A., Chen, H., Lin, Z., Han, J., & Ding, G. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01506 · Cited by 752 · see citing works
Early Convolutions Help Transformers See BetterPreprint
Xiao, T., Singh, M., Mintun, E., Darrell, T., Dollár, P., & Girshick, R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.14881 · Cited by 565 · see citing works
Tent: Fully Test-time Adaptation by Entropy MinimizationPreprint
Wang, D., Shelhamer, E., Liu, S., Olshausen, B. A., & Darrell, T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.10726 · Cited by 648 · see citing works
Model complexity of deep learning: a survey
Hu, X., Chu, L., Pei, J., Liu, W., & Bian, J. (2021) · Knowledge and Information Systems · doi:10.1007/s10115-021-01605-0 · Cited by 435 · see citing works
MCUNet: Tiny Deep Learning on IoT DevicesPreprint
Lin, J., Chen, W.-M., Lin, Y., Cohn, J., Gan, C., & Han, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.10319 · Cited by 425 · see citing works
Neural Architecture Search without TrainingPreprint
Mellor, J., Turner, J., Storkey, A., & Crowley, E. J. (2020) · Edinburgh Research Explorer (University of Edinburgh) · doi:10.48550/arxiv.2006.04647 · Cited by 277 · see citing works
The effect of choosing optimizer algorithms to improve computer vision tasks: a comparative study
Hassan, E., Shams, M. Y., Hikal, N. A., & Elmougy, S. (2022) · Multimedia Tools and Applications · doi:10.1007/s11042-022-13820-0 · Cited by 221 · see citing works
Evaluating the Search Phase of Neural Architecture SearchPreprint
Yu, K., Sciuto, C., Jaggi, M., Musat, C., & Salzmann, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.08142 · Cited by 219 · see citing works
Bringing ai to edge: from deep learning’s perspective
Liu, D., Kong, H., Luo, X., Liu, W., & Subramaniam, R. (2022) · Neurocomputing · doi:10.1016/j.neucom.2021.04.141 · Cited by 171 · see citing works
“We have seen some recent efforts towards defining the effective design space for DNN models [209], [210].”
Fast and Accurate Model Scaling
Dollár, P., Singh, M., & Girshick, R. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00098 · Cited by 116 · see citing works
Evaluating The Search Phase of Neural Architecture Search
Yu, K., Sciuto, C., Jaggi, M., Musat, C., & Salzmann, M. (2020) · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · Cited by 136 · see citing works
Weight-Sharing Neural Architecture Search: A Battle to Shrink the Optimization Gap
Xie, L., Chen, X., Bi, K., Wei, L., Xu, Y., Wang, L., Chen, Z., Xiao, A., Chang, J., Zhang, X., et al. (2021) · ACM Computing Surveys · doi:10.1145/3473330 · Cited by 108 · see citing works
Shen, J., Liu, N., Sun, H., Li, D., & Zhang, Y. (2024) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2023.3346488 · Cited by 83 · see citing works
SP-NAS: Serial-to-Parallel Backbone Search for Object Detection
Jiang, C., Xu, H., Zhang, W., Liang, X., & Li, Z. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01188 · Cited by 67 · see citing works
Are Labels Necessary for Neural Architecture Search?
Liu, C., Dollár, P., He, K., Girshick, R., Yuille, A., & Xie, S. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58548-8_46 · Cited by 71 · see citing works
Self-citationEfficient Architecture Search for Continual Learning
Gao, Q., Luo, Z., Klabjan, D., & Zhang, F. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3151511 · Cited by 62 · see citing works
Influence Selection for Active Learning
Liu, Z., Ding, H., Zhong, H., Li, W., Dai, J., & He, C. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00914 · Cited by 57 · see citing works
PVNAS: 3D Neural Architecture Search with Point-Voxel Convolution
Liu, Z., Tang, H., Zhao, S., Shao, K., & Han, S. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3109025 · Cited by 50 · see citing works
Does Unsupervised Architecture Representation Learning Help Neural Architecture Search?Preprint
Shen, Y., Zheng, Y., Ao, W., Zeng, X., & Zhang, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.06936 · Cited by 46 · see citing works
ZoomNAS: Searching for Whole-body Human Pose Estimation in the Wild
Xu, L., Jin, S., Liu, W., Qian, C., Ouyang, W., Luo, P., & Wang, X. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3197352 · Cited by 47 · see citing works
Deep segmentation and classification of complex crops using multi-feature satellite imagery
Wang, L., Wang, J., Zhang, X., Wang, L., & Qin, F. (2022) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2022.107249 · Cited by 44 · see citing works
A Lightweight Adaptive RoI Extraction Network for Precise Aerial Image Instance Segmentation
Zeng, X., Wei, S., Shi, J., & Zhang, X. (2021) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2021.3121485 · Cited by 43 · see citing works
On Interaction Between Augmentations and Corruptions in Natural Corruption RobustnessPreprint
Mintun, E., Kirillov, A., & Xie, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.11273 · Cited by 42 · see citing works
Self-citationZhao, T., Yang, C. H., Li, Y., Gan, Q., Wang, Z., Liang, F., Zhao, H., Shao, Y., Wang, X., & Shi, C. (2022) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3477495.3531720 · Cited by 38 · see citing works
Hassan, E., El-Rashidy, N. M., Elbedwehy, S., El‐Hafeez, T. A., Saber, A., & Shams, M. Y. (2025) · Neural Computing and Applications · doi:10.1007/s00521-025-11331-1 · Cited by 36 · see citing works
Rethinking Performance Estimation in Neural Architecture Search
Zheng, X., Ji, R., Wang, Q., Ye, Q., Li, Z., Tian, Y., & Tian, Q. C. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01137 · Cited by 32 · see citing works
Training-free neural architecture search: A review
Wu, M., & Tsai, C. (2023) · ICT Express · doi:10.1016/j.icte.2023.11.001 · Cited by 29 · see citing works
Network representation learning: A macro and micro view
Liu, X., & Tang, J. (2021) · AI Open · doi:10.1016/j.aiopen.2021.02.001 · Cited by 29 · see citing works
Zhao, Z., Yeoh, P. S. Q., Zuo, X., Chuah, J. H., Chow, C., Wu, X., & Lai, K. W. (2024) · Frontiers in Neurology · doi:10.3389/fneur.2024.1490829 · Cited by 36 · see citing works
Is it enough to optimize CNN architectures on ImageNet?
Tuggener, L., Schmidhuber, J., & Stadelmann, T. (2022) · Frontiers in Computer Science · doi:10.3389/fcomp.2022.1041703 · Cited by 28 · see citing works
Chen, W., Gong, X., Wu, J., Wei, Y., Shi, H., Yan, Z., Yang, Y., & Wang, Z. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3328347 · Cited by 28 · see citing works
Semantic segmentation of 3D indoor LiDAR point clouds through feature pyramid architecture search
Lin, H., Wu, S., Chen, Y., Li, W., Luo, Z., Guo, Y., Wang, C., & Li, J. (2021) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2021.05.009 · Cited by 26 · see citing works
Fully Test-time Adaptation by Entropy MinimizationPreprint
Wang, D., Shelhamer, E., Liu, S., Olshausen, B. A., & Darrell, T. (2020) · arXiv (Cornell University) · Cited by 27 · see citing works
Profiling the Design Space for Graph Neural Networks based Collaborative Filtering
Wang, Z., Zhao, H., & Shi, C. (2022) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3488560.3498520 · Cited by 26 · see citing works
How to Train Your Super-Net: An Analysis of Training Heuristics in Weight-Sharing NASPreprint
Yu, K., Ranftl, R., & Salzmann, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.04276 · Cited by 24 · see citing works
Ma, B., Zhang, J., Xia, Y., & Tao, D. (2020) · neural information processing systems · Cited by 22 · see citing works
Weight-Sharing Neural Architecture Search: A Battle to Shrink the Optimization GapPreprint
Xie, L., Chen, X., Bi, K., Wei, L., Xu, Y., Chen, Z., Wang, L., Xiao, A., Chang, J., Zhang, X., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.01475 · Cited by 29 · see citing works
Zeng, X., Wei, S., Wei, J., Zhou, Z., Shi, J., Zhang, X., & Fan, F. (2021) · Remote Sensing · doi:10.3390/rs13142788 · Cited by 21 · see citing works
Mok, J., Na, B., Kim, J., Han, D., & Yoon, S. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01156 · Cited by 19 · see citing works
Hyperparameter Optimization and Importance Ranking in Deep Learning–Based Crack Segmentation
Canchila, C., Zhou, S., & Song, W. (2023) · Journal of Computing in Civil Engineering · doi:10.1061/jccee5.cpeng-5512 · Cited by 20 · see citing works
Speedy Performance Estimation for Neural Architecture SearchPreprint
Ru, B., Lyle, C., Schut, L., Fil, M., van der Wilk, M., & Gal, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.04492 · Cited by 19 · see citing works
Generic Neural Architecture Search via RegressionPreprint
Li, Y., Hao, C., Li, P., Xiong, J., & Chen, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.01899 · Cited by 18 · see citing works
Exploring robust architectures for deep artificial neural networks
Waqas, A., Farooq, H., Bouaynaya, N. C., & Rasool, G. (2022) · Communications Engineering · doi:10.1038/s44172-022-00043-2 · Cited by 17 · see citing works
Landmark Regularization: Ranking Guided Super-Net Training in Neural Architecture Search
Yu, K., Ranftl, R., & Salzmann, M. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01351 · Cited by 17 · see citing works
Shen, Y., Lai, E. M.-K., & Mohaghegh, M. (2022) · Neural Processing Letters · doi:10.1007/s11063-021-10730-4 · Cited by 17 · see citing works
Zhang, K., Wang, W., Lv, Z., Feng, J., Li, H., & Zhang, C. (2023) · Applied Soft Computing · doi:10.1016/j.asoc.2023.110471 · Cited by 16 · see citing works
Designing Efficient DNNs via Hardware-Aware Neural Architecture Search and Beyond
Luo, X., Liu, D., Huai, S., Kong, H., Chen, H., & Liu, W. (2021) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2021.3100249 · Cited by 15 · see citing works
CompOFA: Compound Once-For-All Networks for Faster Multi-Platform DeploymentPreprint
Sahni, M., Varshini, S., Khare, A., & Tumanov, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.12642 · Cited by 15 · see citing works
HSCoNAS: Hardware-Software Co-Design of Efficient DNNs via Neural Architecture Search
Luo, X., Liu, D., Huai, S., & Liu, W. (2021) · Design, Automation, and Test in Europe · doi:10.23919/date51398.2021.9473937 · Cited by 14 · see citing works
GradSign: Model Performance Inference with Theoretical InsightsPreprint
Zhang, Z., & Jia, Z. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.08616 · Cited by 16 · see citing works
Kang, H., Bang, M., Seol, S. J., & Byun, J. (2023) · Geophysics · doi:10.1190/geo2022-0723.1 · Cited by 12 · see citing works
Surrogate-assisted evolutionary neural architecture search based on smart-block discovery
Onzo, B.-M., Xue, Y., & Neri, F. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.127237 · Cited by 12 · see citing works
MFHSformer: Hierarchical sparse transformer based on multi-feature fusion for soil pore segmentation
Bai, H., Han, Q., Zhao, Y., & Zhao, Y. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.126789 · Cited by 11 · see citing works
An Analysis of Super-Net Heuristics in Weight-Sharing NAS
Yu, K., Ranftl, R., & Salzmann, M. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3108480 · Cited by 36 · see citing works
Wang, T., Xu, L., & Li, J. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.05.092 · Cited by 11 · see citing works
He, F., Liu, S., Liu, S., Jin, Y., Xie, H. X., & Tong, X. (2025) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2025.06.014 · Cited by 12 · see citing works
Profiling Neural Blocks and Design Spaces for Mobile Neural Architecture Search
Mills, K. G., Han, F. X., Zhang, J., Rezaei, S. S. C., Chudak, F., Lu, W., Lian, S., Jui, S., & Niu, D. (2021) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3459637.3481944 · Cited by 9 · see citing works
Fighting Gradients with Gradients: Dynamic Defenses against Adversarial AttacksPreprint
Wang, D., Ju, A., Shelhamer, E., Wagner, D., & Darrell, T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.08714 · Cited by 16 · see citing works
Deep learning-based magnetic resonance image segmentation technique for application to glioma
Wan, B., Hu, B., Zhao, M., Li, K., & Ye, X. (2023) · Frontiers in Medicine · doi:10.3389/fmed.2023.1172767 · Cited by 8 · see citing works
Convolutional Neural Networks in the Diagnosis of Colon Adenocarcinoma
Leo, M., Carcagnì, P., Signore, L., Corcione, F., Benincasa, G., Laukkanen, M. O., & Distante, C. (2024) · AI · doi:10.3390/ai5010016 · Cited by 8 · see citing works
Yamasaki, T., Wang, Z., Luo, T., Chen, N., & Wang, B. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2025.3552693 · Cited by 8 · see citing works
Zheng, Z., Xuelei, 何. H., Fengjun, Z., & Xiaowei, H. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.128155 · Cited by 8 · see citing works
Efficient perturbation-aware distinguishing score for zero-shot neural architecture search
Huang, J., Xue, B., Sun, Y., Zhang, M., & Yen, G. G. (2025) · Applied Soft Computing · doi:10.1016/j.asoc.2025.113447 · Cited by 8 · see citing works
Neural Image Parts Group Search for Person Re-Identification
Liu, Z., Zhang, L., & Zhang, D. D. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3225285 · Cited by 8 · see citing works
Are neural architecture search benchmarks well designed? A deeper look into operation importance
Lopes, V., Degardin, B., & Alexandre, L. A. (2023) · Information Sciences · doi:10.1016/j.ins.2023.119695 · Cited by 8 · see citing works
Chen, Y., Teng, W., Li, Z., Zhu, Q., & Guan, Q. (2021) · Remote Sensing · doi:10.3390/rs13173460 · Cited by 8 · see citing works
Laube, K. A., & Zell, A. (2019) · International Conference on Machine Learning and Applications · doi:10.1109/icmla.2019.00158 · Cited by 8 · see citing works
Revisiting the Train Loss: an Efficient Performance Estimator for Neural Architecture SearchPreprint
Ru, B., Lyle, C., Schut, L., van der Wilk, M., & Gal, Y. (2021) · arXiv (Cornell University) · Cited by 8 · see citing works
Auto-X3D: Ultra-Efficient Video Understanding via Finer-Grained Neural Architecture Search
Jiang, Y., Gong, X., Wu, J., Shi, H., Yan, Z., & Wang, Z. (2022) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv51458.2022.00241 · Cited by 7 · see citing works
Unleashing the Power of Gradient Signal-to-Noise Ratio for Zero-Shot NAS
Sun, Z., Sun, Y., Yang, L., Lu, S., Mei, J., Zhao, W., & Hu, Y. H. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00530 · Cited by 7 · see citing works
AutoSpace: Neural Architecture Search with Less Human Interference
Zhou, D., Jin, X., Lian, X., Yang, L., Xue, Y., Hou, Q., & Feng, J. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00039 · Cited by 7 · see citing works
Contrastive meta-reinforcement learning for heterogeneous graph neural architecture search
Xu, Z., & Wu, J. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.125433 · Cited by 6 · see citing works
Wang, L., Bai, Y., Wang, J., Qin, F., Liu, C., Zhou, Z. Y., & Jiao, X. (2022) · Remote Sensing · doi:10.3390/rs14205216 · Cited by 6 · see citing works
Real-time grouping of tobacco through channel weighting and dynamic loss regulation
He, Z., He, P., Zhang, Y., Chen, G., & Chen, D. (2023) · Industrial Crops and Products · doi:10.1016/j.indcrop.2023.116427 · Cited by 6 · see citing works
LISSNAS: Locality-based Iterative Search Space Shrinkage for Neural Architecture Search
Gopal, B., Sridhar, A., Zhang, T., & Chen, Y. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/86 · Cited by 6 · see citing works
Yang, L., Fu, Y.-X., Lu, S.-N., Sun, Z.-H., Mei, J.-L., Zhao, W.-X., & Hu, Y. (2023) · Neural Networks · doi:10.1016/j.neunet.2023.09.012 · Cited by 8 · see citing works
NASOA: Towards Faster Task-oriented Online Fine-tuning with a Zoo of Models
Xu, H., Kang, N., Zhang, G., Xie, C., Liang, X., & Li, Z. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00505 · Cited by 7 · see citing works
Model Complexity of Deep Learning: A SurveyPreprint
Hu, X., Chu, L., Pei, J., Liu, W., & Bian, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.05127 · Cited by 6 · see citing works
Li, M., Chen, M., Zeng, X., Lu, W., Yan, J., Liu, X., & Zheng, J. (2025) · Automation in Construction · doi:10.1016/j.autcon.2025.106454 · Cited by 5 · see citing works
Fine-Grained Stochastic Architecture SearchPreprint
Chaudhuri, S. R., Eban, E., Li, H., Moroz, M., & Movshovitz-Attias, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.09581 · Cited by 5 · see citing works
AceNAS: Learning to Rank Ace Neural Architectures with Weak Supervision of Weight Sharing.Preprint
Zhang, Y., Yan, C., Zhang, Q., Zhang, L. L., Yang, Y., Gao, X., & Yang, Y. (2021) · arXiv (Cornell University) · Cited by 5 · see citing works
Evolving Comprehensive Proxies for Zero-Shot Neural Architecture Search
Huang, J., Xue, B., Sun, Y., & Zhang, M. (2025) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/3712256.3726315 · Cited by 4 · see citing works
Network Space Search for Pareto-Efficient Spaces
Hong, M.-F., Chen, H.-Y., Chen, M.-H., Xu, Y.-S., Kuo, H.-K., Tsai, Y., Chen, H., & Jou, K. (2021) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00340 · Cited by 4 · see citing works
GhostNeXt: Rethinking Module Configurations for Efficient Model Design
Hong, K., Kim, G.-H., & Kim, E. (2023) · Applied Sciences · doi:10.3390/app13053301 · Cited by 4 · see citing works
Workpiece tracking based on improved SiamFC++ and virtual dataset
Yang, K., Zhao, L., & Wang, C. (2023) · Multimedia Systems · doi:10.1007/s00530-023-01185-9 · Cited by 4 · see citing works
Isomorphic model-based initialization for convolutional neural networks
Zhang, H., Li, Y., Yang, H., He, B., & Zhang, Y. (2022) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2022.103677 · Cited by 4 · see citing works
Quasi-orthogonality and intrinsic dimensions as measures of learning and generalisation
Zhou, Q., Gorban, A. N., Mirkes, E. M., Bac, J., Zinovyev, A. Y., & Tyukin, I. (2022) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn55064.2022.9892337 · Cited by 4 · see citing works
Improving Neural Architecture Search by Mixing a FireFly algorithm with a Training Free Evaluation
Mokhtari, N., Nédélec, A., Gilles, M., & Loor, P. D. (2022) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn55064.2022.9892861 · Cited by 4 · see citing works
Knowledge distillation vulnerability of DeiT through CNN adversarial attack
Hong, I.-P., & Choi, C. (2023) · Neural Computing and Applications · doi:10.1007/s00521-023-09297-z · Cited by 4 · see citing works
Searching for Stage-wise Neural Graphs In the LimitPreprint
Zhou, X., Dou, D., & Li, B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.12860 · Cited by 4 · see citing works
Dissecting Convolutional Neural Networks for Runtime and Scalability Prediction
Beringer, T., Stock, J., Mazaheri, A., & Wolf, F. (2024) · Proceedings of the International Conference on Parallel Processing · doi:10.1145/3673038.3673107 · Cited by 3 · see citing works
Fan, Z., Sun, D., Yu, H., & Zhang, W. (2024) · Multimedia Systems · doi:10.1007/s00530-024-01330-y · Cited by 3 · see citing works
AutoInit: Analytic Signal-Preserving Weight Initialization for Neural Networks
Bingham, G., & Miikkulainen, R. P. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i6.25836 · Cited by 3 · see citing works
HZS-NAS: Neural Architecture Search with Hybrid Zero-Shot Proxy for Facial Expression Recognition
Yang, X., Liu, Y., Liu, J., & Sun, Y. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10650259 · Cited by 2 · see citing works
Hou, B., Dong, J., Feng, L., & Qiu, M. (2021) · Communications in computer and information science · doi:10.1007/978-3-030-92307-5_70 · Cited by 2 · see citing works
- Authors
- Radosavovic, I., Johnson, J., Xie, S., Lo, W.-Y., & Dollar, P.
- Affiliations
- Meta (United States) (US)
- Venue
- 2019 IEEE/CVF International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2019-10
- Pages
- 1882-1890
- OpenAlex
- W2947546116
- Cited by
- 127
- References
- 49
- Citations by year
- 2026: 5 · 2025: 15 · 2024: 9 · 2023: 21 · 2022: 21 · 2021: 35 · 2020: 18 · 2019: 3
- FWCI
- 6.28
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- neural architecture search, visual recognition, neural network architecture comparison, model families, ResNeXt, network complexity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
