The DOI matches its DataCite record; 35 of the 36 references it cites were found in a registry or an index.
SNAS: Stochastic Neural Architecture Search
Open accessAbstract
We propose Stochastic Neural Architecture Search (SNAS), an economical end-to-end solution to Neural Architecture Search (NAS) that trains neural operation parameters and architecture distribution parameters in same round of back-propagation, while maintaining the completeness and differentiability of the NAS pipeline. In this work, NAS is reformulated as an optimization problem on parameters of a joint distribution for the search space in a cell. To leverage the gradient information in generic differentiable loss for architecture search, a novel search gradient is proposed. We prove that this search gradient optimizes the same objective as reinforcement-learning-based NAS, but assigns credits to structural decisions more efficiently. This credit assignment is further augmented with locally decomposable reward to enforce a resource-efficient constraint. In experiments on CIFAR-10, SNAS takes less epochs to find a cell architecture with state-of-the-art accuracy than non-differentiable evolution-based and reinforcement-learning-based NAS, which is also transferable to ImageNet. It is also shown that child networks of SNAS can maintain the validation accuracy in searching, with which attention-based NAS requires parameter retraining to compete, exhibiting potentials to stride towards efficient NAS on big datasets. We have released our implementation at https://github.com/SNAS-Series/SNAS-Series.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“fully decentralized variants of SGD and other optimization algorithms have recently been considered in machine learning both for improved scalability in datacenters [29] as well as for decentralized networks of devices [127, 459, 443, 59, 278, 291, 173].”
From Advances and Open Problems in Federated Learning (2021) · cites this paper as [173]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Densely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,889
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., & Adam, H. (2017) · arXiv · doi:10.48550/arxiv.1704.04861 · Cited by 19,015
Proximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625
ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
Zhang, X., Zhou, X., Lin, M., & Sun, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00716 · Cited by 9,982
Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
Williams, R. J. (1992) · Machine Learning · doi:10.1007/bf00992696 · Cited by 8,608
ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design
Ma, N., Zhang, X., Zheng, H.-T., & Sun, J. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01264-9_8 · Cited by 7,356
Self-citationLearning Transferable Architectures for Scalable Image Recognition
Zoph, B., Vasudevan, V., Shlens, J., & Le, Q. V. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00907 · Cited by 6,875
Factor graphs and the sum-product algorithm
Kschischang, F. R., Frey, B. J., & Loeliger, H.-A. (2001) · IEEE Transactions on Information Theory · doi:10.1109/18.910572 · Cited by 6,486 · cited 2×
“f the global cost, consistent with the credit assignment principle in SNAS. In SNAS, p (Z) is fully factorizable, making it possible to calculate E Z˘p [C(Z)] analytically with sum-product algorithm (Kschischang et al., 2001). Unfortunately, this expectation is non-trivial to calculate, we optimize the Monte Carlo est…”
Method
Research works printed but not linked: 1
- Bowen Baker, Otkrist Gupta, Ramesh Raskar, and Nikhil Naik. Accelerating neural architecture search using performance prediction. arXiv preprint arXiv:1705.10823 , 2017. · arXiv
Cited by
Sorted by citations
Items 701–729 of 729
AutoCoMet: Smart Neural Architecture Search via Co-Regulated Shaping Reinforcement
Das, M., Singh, B., Chheda, H. K., Sharma, P., & Pradeep, N. S. (2022) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr56361.2022.9956109
Evolutionary Neural Architecture Search with Semi-supervised Accuracy Predictor
Xiao, S., Zhao, B., & Liu, D. (2022) · International Conference on Control and Robotics (ICCR) · doi:10.1109/iccr55715.2022.10053920
ICDARTS: Improving the Stability of Cyclic DARTS
Herron, E., Young, S. R., & Rose, D. C. (2022) · International Conference on Machine Learning and Applications · doi:10.1109/icmla55696.2022.00175
EMAS: Efficient Meta Architecture Search for Few-Shot Learning
Liu, D., Li, J., Chen, H., Liu, B., Lu, X., & Liu, W. (2022) · IEEE International Conference on Tools with Artificial Intelligence (ICTAI) · doi:10.1109/ictai56018.2022.00099
SoftStep relaxation for mining optimal convolution kernel
Chen, B., & Wang, H. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.110755
Local Stochastic Differentiable Architecture Search for Memetic Neuroevolution Algorithms
Karns, J., & Desell, T. (2023) · Genetic and Evolutionary Computation Conference Companion (GECCO Companion) · doi:10.1145/3583133.3596368
ALT: An Automatic System for Long Tail Scenario Modeling
Zhang, Y., Zhou, J., Ren, Y., Zhang, Y., Yang, X., Li, M., Shi, Q., & Li, L. (2023) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde55515.2023.00231
Sparse Gate for Differentiable Architecture Search
Fan, L., & Wang, H. (2023) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn54540.2023.10191908
PertNAS: Architectural Perturbations for Memory-Efficient Neural Architecture Search
Ahmad, A., Xie, Z., & Zhang, W. (2023) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1109/dac56929.2023.10247756
Towards Effective Neural Architecture Selection: The Promise of Self-AttentionPreprint
Sun, Z., Hu, Y., Yang, L., Lu, S., Mei, J., & Han, Y. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4634144
Progressively Attentional Architecture SearchPreprint
Qin, X., Li, S., Zhong, G., Zhang, F., & Hedjam, R. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4761255
Class-Attribute Priors: Adapting Optimization to Heterogeneity and Fairness Objective
Zhang, X., Li, M., Chen, J., Thrampoulidis, C., & Oymak, S. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i15.29631
Model Optimization Methods for Real-time 2D Human Pose Estimation Applied to Robots: A Survey
He, W., Zeng, B., Liu, J., Ye, X., & Zhou, F. (2023) · International Conference on Robotics, Automation and Intelligent Control (ICRAIC) · doi:10.1109/icraic61978.2023.00060
NAS-SW: Efficient Neural Architecture Search with Stage-Wise Search Strategy
Nie, J., Yang, Y., Zhu, Q., & Lin, Q. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10650420
EFNAS: Efficient Federated Neural Architecture Search Across AIoT Devices
Wei, X., Chen, G., Chen, Y., Zhao, H., Wang, C., & Yue, H. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10650653
Zhang, Y.-M., Hsieh, J., Lee, C.-C., & Fan, K. (2024) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2024.3465433
Searching to extrapolate embedding for out-of-graph node representation learning
Shen, Z., Guo, S., Wen, Y., Wei, L., Zhang, W., Luo, Y., Wu, C., & Yao, Q. (2025) · Neural Networks · doi:10.1016/j.neunet.2024.107118
Learning From Mistakes: A Multilevel Optimization Framework
Zhang, L., Garg, B., Sridhara, P., Hosseini, R., & Xie, P. (2025) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2025.3534151
Efficient Self-Supervised Neural Architecture Search
Deevi, S. A., Mishra, A. K., Mishra, D., Ravi Kumar, L., Bharat Kumar, G. V. P., & G., M. (2025) · International Conference on Ubiquitous Information Management and Communication (IMCOM) · doi:10.1109/imcom64595.2025.10857490
HID-NAS: A Novel Neural Architecture Search Pipeline for High Information Density Data
Liu, W., Deng, T., & Yan, F. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10889095
Han, J., Wang, D., Luo, H., Zhou, Y., & Song, B. (2024) · IEEE International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom) · doi:10.1109/trustcom63139.2024.00367
Liu, H. T., Sun, B. Y., Yu, Y., Zhang, K., & Gong, M. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5279099
DeNoiseNAS: revisiting single-level optimization for efficient and stable neural architecture search
Wang, Z., Zeng, R., Zhang, H., & Chen, T. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.128649
Han, H., Zhang, Q., Li, F., & Du, Y. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.111476
NAS-DG: Ultra-Fast NAS via Uncertainty Modeling for Domain Generalization
Wakayama, K., & Kanamori, T. (2025) · IEEE Access · doi:10.1109/access.2025.3597980
Evolving neural network for few-shot learning via training-free NAS
Kang, H., Jiang, N., Zhao, J., Shi, C., Wang, H., & Ma, L. (2025) · Swarm and Evolutionary Computation · doi:10.1016/j.swevo.2025.102177
Metadata
- Authors
- Xie, S., Zheng, H., Liu, C., & Lin, L.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2018
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1812.09926
- OpenAlex
- W2905672847
- Cited by
- 731
- References
- 36
- Citations by year
- 2025: 30 · 2024: 51 · 2023: 93 · 2022: 91 · 2021: 224 · 2020: 170 · 2019: 70 · 2018: 2
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Canadian Institute for Advanced Research
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Physical Sciences › Computer Science › Artificial Intelligence › Evolutionary Algorithms and Applications
- Keywords
- neural architecture search, differentiable architecture search, credit assignment, search space optimization, end-to-end learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
