Revisiting Training-free NAS Metrics: An Efficient Training-based Method
Open accessAbstract
Recent neural architecture search (NAS) works proposed training-free metrics to rank networks which largely reduced the search cost in NAS. In this paper, we revisit these training-free metrics and find that: (1) the number of parameters (#Param), which is the most straightforward training-free metric, is overlooked in previous works but is surprisingly effective, (2) recent training-free metrics largely rely on the #Param information to rank networks. Our experiments show that the performance of recent training-free metrics drops dramatically when the #Param information is not available. Motivated by these observations, we argue that metrics less correlated with the #Param are desired to provide additional information for NAS. We propose a light-weight training-based metric which has a weak correlation with the #Param while achieving better performance than training-free metrics at a lower search cost. Specifically, on DARTS search space, our method completes searching directly on ImageNet in only 2.6 GPU hours and achieves a top-1/top-5 error rate of 24.1%/7.1%, which is competitive among state-of-the-art NAS methods.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Neural Architecture Search significantly reduces the manual cost of designing network structures and greatly expands the possibilities for network architecture design.
“Neural Architecture Search (NAS) [21] significantly reduces the manual cost of designing network structures and greatly expands the possibilities for network architecture design.”
From Optimizing Neural Network Architecture for Medical Image Segmentation Using Monte Carlo Tree Search (2026) · cites this paper as [21]
The Related work section says NAS automatically searches for network structures instead of relying on time-consuming manual design, supporting the claim about reduced manual effort. It also describes exploring architectures, but does not establish that NAS greatly expands the possibilities for architecture design. (LLM, conf=0.78)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
EfficientDet: Scalable and Efficient Object Detection
Tan, M., Pang, R., & Le, Q. V. (2019) · arXiv · Cited by 9,841 · cited 1× · see citing works
“Its effectiveness has been demonstrated in various computer vision tasks such as classification [27, 33, 41], object detection [10, 30] and semantic segmentation [6, 20].”
BackgroundDetNAS: Backbone Search for Object Detection
Chen, Y., Yang, T., Zhang, X., Meng, G., Xiao, X., & Sun, J. (2019) · arXiv · doi:10.48550/arxiv.1903.10979 · Cited by 205 · see citing works
Self-citationStabilizing Differentiable Architecture Search via Perturbation-based Regularization
Chen, X., & Hsieh, C.-J. (2020) · arXiv · doi:10.48550/arxiv.2002.05283 · Cited by 126 · see citing works
Flagged first, then by citations
Research works printed but not linked: 35
- Mohamed S Abdelfattah, Abhinav Mehrotra, Łukasz Dudziak, and Nicholas Donald Lane. Zero-cost proxies for lightweight nas. In International Conference on Learning Representations, 2020.
- Gabriel Bender, Pieter-Jan Kindermans, Barret Zoph, Vijay Vasudevan, and Quoc Le. Understanding and simplifying one-shot architecture search. In Jennifer Dy and Andreas Krause, editors, Proceedings of the 35th International Con- ference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 550-559. PMLR, 10-15 Jul 2018.
- Andrew Brock, Theodore Lim, James M Ritchie, and Nick Weston. Smash: one-shot model architecture search through hypernetworks. In International Conference on Learning Representations, 2017.
- Simon Carbonnelle and Christophe De Vleeschouwer. Layer rotation: a surprisingly powerful indicator of generalization in deep networks? arXiv preprint arXiv:1806.01603, 2018. · arXiv
- Wuyang Chen, Xinyu Gong, Xianming Liu, Qian Zhang, Yuan Li, and Zhangyang Wang. Fasterseg: Searching for faster real-time semantic segmentation. In International Conference on Learning Representations, 2019.
- Wuyang Chen, Xinyu Gong, and Zhangyang Wang. Neural architecture search on imagenet in four gpu hours: A theo- retically inspired perspective. In International Conference on Learning Representations, 2020.
- Xiangning Chen and Cho-Jui Hsieh. Stabilizing differen- tiable architecture search via perturbation-based regulariza- tion. In International Conference on Machine Learning, pages 1554-1565. PMLR, 2020.
- Xin Chen, Lingxi Xie, Jun Wu, and Qi Tian. Progres- sive differentiable architecture search: Bridging the depth gap between search and evaluation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1294-1303, 2019.
- Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248-255, 2009.
- Xuanyi Dong and Yi Yang. Nas-bench-201: Extending the scope of reproducible neural architecture search. In Interna- tional Conference on Learning Representations, 2019.
- Xuanyi Dong and Yi Yang. Searching for a robust neu- ral architecture in four gpu hours. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1761-1770, 2019.
- Zichao Guo, Xiangyu Zhang, Haoyuan Mu, Wen Heng, Zechun Liu, Yichen Wei, and Jian Sun. Single path one- shot neural architecture search with uniform sampling. In European Conference on Computer Vision, pages 544-560. Springer, 2020.
- Yiming Hu, Yuding Liang, Zichao Guo, Ruosi Wan, Xiangyu Zhang, Yichen Wei, Qingyi Gu, and Jian Sun. Angle-based search space shrinking for neural architecture search. In European Conference on Computer Vision, pages 119-134. Springer, 2020.
- Jaehoon Lee, Lechao Xiao, Samuel Schoenholz, Yasaman Bahri, Roman Novak, Jascha Sohl-Dickstein, and Jeffrey Pennington. Wide neural networks of any depth evolve as linear models under gradient descent. Advances in neural information processing systems, 32:8572-8583, 2019.
- Namhoon Lee, Thalaiyasingam Ajanthan, and Philip Torr. Snip: Single-shot network pruning based on connection sen- sitivity. In International Conference on Learning Represen- tations, 2018.
- Liam Li and Ameet Talwalkar. Random search and repro- ducibility for neural architecture search. In Uncertainty in artificial intelligence, pages 367-377. PMLR, 2020.
- Chenxi Liu, Liang-Chieh Chen, Florian Schroff, Hartwig Adam, Wei Hua, Alan L Yuille, and Li Fei-Fei. Auto- deeplab: Hierarchical neural architecture search for semantic image segmentation. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 82-92, 2019.
- Chenxi Liu, Barret Zoph, Maxim Neumann, Jonathon Shlens, Wei Hua, Li-Jia Li, Li Fei-Fei, Alan Yuille, Jonathan Huang, and Kevin Murphy. Progressive neural architecture search. In Proceedings of the European conference on com- puter vision (ECCV), pages 19-34, 2018.
- Joe Mellor, Jack Turner, Amos Storkey, and Elliot J Crowley. Neural architecture search without training. In International Conference on Machine Learning, pages 7588-7598. PMLR, 2021.
- Jisoo Mok, Byunggook Na, Ji-Hoon Kim, Dongyoon Han, and Sungroh Yoon. Demystifying the neural tangent ker- nel from a practical perspective: Can it be trusted for neu- ral architecture search without training? In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11861-11870, 2022.
- Guido F Montufar, Razvan Pascanu, Kyunghyun Cho, and Yoshua Bengio. On the number of linear regions of deep neural networks. Advances in Neural Information Process- ing Systems, 27:2924-2932, 2014.
- Hieu Pham, Melody Guan, Barret Zoph, Quoc Le, and Jeff Dean. Efficient neural architecture search via parameters sharing. In International Conference on Machine Learning, pages 4095-4104. PMLR, 2018.
- Esteban Real, Alok Aggarwal, Yanping Huang, and Quoc V Le. Regularized evolution for image classifier architecture search. In Proceedings of the aaai conference on artificial intelligence, volume 33, pages 4780-4789, 2019.
- Mingxing Tan, Bo Chen, Ruoming Pang, Vijay Vasudevan, Mark Sandler, Andrew Howard, and Quoc V Le. Mnas- net: Platform-aware neural architecture search for mobile. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2820-2828, 2019.
- Hidenori Tanaka, Daniel Kunin, Daniel L Yamins, and Surya Ganguli. Pruning neural networks without any data by iter- atively conserving synaptic flow. Advances in Neural Infor- mation Processing Systems, 33, 2020.
- Chaoqi Wang, Guodong Zhang, and Roger Grosse. Pick- ing winning tickets before training by preserving gradient flow. In International Conference on Learning Representa- tions, 2019.
- Bichen Wu, Xiaoliang Dai, Peizhao Zhang, Yanghan Wang, Fei Sun, Yiming Wu, Yuandong Tian, Peter Vajda, Yangqing Jia, and Kurt Keutzer. Fbnet: Hardware-aware efficient con- vnet design via differentiable neural architecture search. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 10734-10742, 2019.
- Lechao Xiao, Jeffrey Pennington, and Samuel Schoenholz. Disentangling trainability and generalization in deep neural networks. In International Conference on Machine Learn- ing, pages 10462-10472. PMLR, 2020.
- Huan Xiong, Lei Huang, Mengyang Yu, Li Liu, Fan Zhu, and Ling Shao. On the number of linear regions of convolutional neural networks. In International Conference on Machine Learning, pages 10514-10523. PMLR, 2020.
- Yuhui Xu, Lingxi Xie, Xiaopeng Zhang, Xin Chen, Guo-Jun Qi, Qi Tian, and Hongkai Xiong. Pc-darts: Partial channel connections for memory-efficient architecture search. In In- ternational Conference on Learning Representations, 2019.
- Chris Ying, Aaron Klein, Eric Christiansen, Esteban Real, Kevin Murphy, and Frank Hutter. Nas-bench-101: Towards reproducible neural architecture search. In International Conference on Machine Learning, pages 7105-7114. PMLR, 2019.
- Kaicheng Yu, Christian Sciuto, Martin Jaggi, Claudiu Musat, and Mathieu Salzmann. Evaluating the search phase of neural architecture search. In International Conference on Learning Representations, 2019.
- Xuanyang Zhang, Pengfei Hou, Xiangyu Zhang, and Jian Sun. Neural architecture search with random labels. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10907-10916, 2021.
- Barret Zoph and Quoc V Le. Neural architecture search with reinforcement learning. arXiv preprint arXiv:1611.01578, 2016. · arXiv
- Barret Zoph, Vijay Vasudevan, Jonathon Shlens, and Quoc V Le. Learning transferable architectures for scalable image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 8697-8710, 2018.
Cited by
Sorted by citations
Optimizing Neural Network Architecture for Medical Image Segmentation Using Monte Carlo Tree Search
Meng, L., Nie, F., Zhang, Y., & Han, C. (2026) · arXiv · doi:10.48550/arxiv.2602.22361
“Neural Architecture Search (NAS) [21] significantly reduces the manual cost of designing network structures and greatly expands the possibilities for network architecture design.”
- Authors
- Yang, T., Yang, L., Jin, X., & Chen, C.
- Affiliations
- University of Central Florida (US)
- ByteDance (China) (CN)
- Venue
- 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)
- Publisher
- IEEE
- Published
- 2023-01
- Pages
- 4740-4749
- ISSN
- 2472-6737 (ISSN-L)
- OpenAlex
- W4319299863
- Cited by
- 9
- References
- 3
- Citations by year
- 2025: 4 · 2024: 5
- FWCI
- 0.96
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Winter Conference on Applications of Computer Vision (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- neural architecture search, network ranking, number of parameters, search cost reduction, ImageNet classification
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
