SNAS: Stochastic Neural Architecture Search
Open accessAbstract
We propose Stochastic Neural Architecture Search (SNAS), an economical end-to-end solution to Neural Architecture Search (NAS) that trains neural operation parameters and architecture distribution parameters in same round of back-propagation, while maintaining the completeness and differentiability of the NAS pipeline. In this work, NAS is reformulated as an optimization problem on parameters of a joint distribution for the search space in a cell. To leverage the gradient information in generic differentiable loss for architecture search, a novel search gradient is proposed. We prove that this search gradient optimizes the same objective as reinforcement-learning-based NAS, but assigns credits to structural decisions more efficiently. This credit assignment is further augmented with locally decomposable reward to enforce a resource-efficient constraint. In experiments on CIFAR-10, SNAS takes less epochs to find a cell architecture with state-of-the-art accuracy than non-differentiable evolution-based and reinforcement-learning-based NAS, which is also transferable to ImageNet. It is also shown that child networks of SNAS can maintain the validation accuracy in searching, with which attention-based NAS requires parameter retraining to compete, exhibiting potentials to stride towards efficient NAS on big datasets. We have released our implementation at https://github.com/SNAS-Series/SNAS-Series.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Fully decentralized variants of SGD and other optimization algorithms have recently been considered in machine learning both for improved scalability in datacenters and for decentralized networks of devices.
“fully decentralized variants of SGD and other optimization algorithms have recently been considered in machine learning both for improved scalability in datacenters [29] as well as for decentralized networks of devices [127, 459, 443, 59, 278, 291, 173].”
From Advances and Open Problems in Federated Learning (2021) · cites this paper as [173]
The Introduction (p. 1) and Related Works (p. 9) discuss gradient-based neural architecture search, not decentralized optimization. Searches for decentralized SGD, device networks, distributed training, and datacenter scalability found no passage in our copy addressing the attributed claim. (LLM, conf=0.77)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Densely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566 · see citing works
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562 · see citing works
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,890 · see citing works
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., & Adam, H. (2017) · arXiv · doi:10.48550/arxiv.1704.04861 · Cited by 19,015 · see citing works
Proximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625 · see citing works
ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
Zhang, X., Zhou, X., Lin, M., & Sun, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00716 · Cited by 9,982 · see citing works
Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
Williams, R. J. (1992) · Machine Learning · doi:10.1007/bf00992696 · Cited by 7,936 · see citing works
ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design
Ma, N., Zhang, X., Zheng, H.-T., & Sun, J. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01264-9_8 · Cited by 7,356 · see citing works
Self-citationLearning Transferable Architectures for Scalable Image Recognition
Zoph, B., Vasudevan, V., Shlens, J., & Le, Q. V. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00907 · Cited by 6,631 · see citing works
Factor graphs and the sum-product algorithm
Kschischang, F. R., Frey, B. J., & Loeliger, H.-A. (2001) · IEEE Transactions on Information Theory · doi:10.1109/18.910572 · Cited by 6,486 · see citing works
Neural Architecture Search with Reinforcement LearningPreprint
Zoph, B., & Le, Q. V. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.01578 · Cited by 5,091 · see citing works
Evolving Neural Networks through Augmenting Topologies
Stanley, K. O., & Miikkulainen, R. P. (2002) · Evolutionary Computation · doi:10.1162/106365602320169811 · Cited by 3,269 · see citing works
DARTS: Differentiable architecture search
Liu, H., Simonyan, K., & Yang, Y. (2018) · arXiv · doi:10.48550/arxiv.1806.09055 · Cited by 3,248 · see citing works
Improved regularization of convolutional neural networks with cutout
DeVries, T., & Taylor, G. W. (2017) · arXiv · doi:10.48550/arxiv.1708.04552 · Cited by 2,776 · see citing works
Methods for interpreting and understanding deep neural networks
Montavon, G., Samek, W., & Müller, K.-R. (2018) · Digital Signal Processing · doi:10.1016/j.dsp.2017.10.011 · Cited by 2,665 · see citing works
High-Dimensional Continuous Control Using Generalized Advantage Estimation
Schulman, J., Moritz, P., Levine, S., Jordan, M., & Abbeel, P. (2015) · arXiv · doi:10.48550/arxiv.1506.02438 · Cited by 2,103 · see citing works
The Concrete Distribution: A Continuous Relaxation of Discrete Random VariablesPreprint
Maddison, C. J., Mnih, A., & Teh, Y. W. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.00712 · Cited by 1,771 · see citing works
(1998) · Kybernetes · doi:10.1108/k.1998.27.9.1093.3 · Cited by 1,576 · see citing works
(2018) · International Conference on Machine Learning · Cited by 1,465 · see citing works
Explaining nonlinear classification decisions with deep taylor decomposition
Montavon, G., Lapuschkin, S., Binder, A., Samek, W., & Müller, K.-R. (2017) · Pattern Recognition · doi:10.1016/j.patcog.2016.11.008 · Cited by 1,357 · see citing works
Efficient Neural Architecture Search via Parameter SharingPreprint
Pham, H., Guan, M. Y., Zoph, B., Le, Q. V., & Dean, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.03268 · Cited by 1,082 · see citing works
Hierarchical Representations for Efficient Architecture SearchPreprint
Liu, H., Simonyan, K., Vinyals, O., Fernando, C., & Kavukcuoglu, K. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.00436 · Cited by 851 · see citing works
Black Box Variational InferencePreprint
Ranganath, R., Gerrish, S., & Blei, D. M. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1401.0118 · Cited by 646 · see citing works
Learning Sparse Neural Networks through $L_0$ Regularization
Louizos, C., Welling, M., & Kingma, D. P. (2017) · arXiv · doi:10.48550/arxiv.1712.01312 · Cited by 628 · see citing works
SMASH: One-Shot Model Architecture Search through HyperNetworksPreprint
Brock, A., Lim, T., Ritchie, J. M., & Weston, N. J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.05344 · Cited by 500 · see citing works
Eligibility Traces for Off-Policy Policy Evaluation
Precup, D., Sutton, R. S., & Singh, S. P. (2000) · ScholarWorks@UMassAmherst (University of Massachusetts Amherst) · Cited by 439 · see citing works
Meta-Gradient Reinforcement LearningPreprint
Xu, Z., Hasselt, H. V., & Silver, D. (2018) · Neural Information Processing Systems · doi:10.48550/arxiv.1805.09801 · Cited by 388 · see citing works
MorphNet: Fast & Simple Resource-Constrained Structure Learning of Deep Networks
Gordon, A., Eban, E., Nachum, O., Chen, B., Wu, H., Yang, T.-J., & Choi, E. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00171 · Cited by 345 · see citing works
Glasmachers, T., Wierstra, D., Schaul, T., Schmidhuber, J., & Yi, S. (2008) · Cited by 330 · see citing works
Regularized Evolution for Image Classifier Architecture Search
Real, E., Aggarwal, A., Huang, Y., & Le, Q. V. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33014780 · Cited by 328 · see citing works
Progressive Neural Architecture Search
Liu, C., Zoph, B., Neumann, M., Shlens, J., Hua, W., Li, L.-J., Fei-Fei, L., Yuille, A., Huang, J., & Murphy, K. (2017) · arXiv · doi:10.48550/arxiv.1712.00559 · Cited by 171 · see citing works
Self-citationRUDDER: Return Decomposition for Delayed RewardsPreprint
Arjona-Medina, J. A., Gillhofer, M., Widrich, M., Unterthiner, T., Brandstetter, J., & Hochreiter, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.07857 · Cited by 122 · see citing works
Peephole: Predicting Network Performance Before TrainingPreprint
Deng, B., Yan, J., & Lin, D. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.03351 · Cited by 75 · see citing works
MuProp: Unbiased Backpropagation for Stochastic Neural NetworksPreprint
Gu, S., Levine, S., Sutskever, I., & Mnih, A. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.05176 · Cited by 74 · see citing works
The Mirage of Action-Dependent Baselines in Reinforcement LearningPreprint
Tucker, G., Bhupatiraju, S., Gu, S., Turner, R. E., Ghahramani, Z., & Levine, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.10031 · Cited by 67 · see citing works
Flagged first, then by citations
1 of the 36 references couldn’t be matched to a known paper.
Research works printed but not linked: 1
- Bowen Baker, Otkrist Gupta, Ramesh Raskar, and Nikhil Naik. Accelerating neural architecture search using performance prediction. arXiv preprint arXiv:1705.10823 , 2017. · arXiv
Cited by
Sorted by citations
Advances and Open Problems in Federated Learning
Kairouz, P., & McMahan, H. B. (2021) · Foundations and Trends in Machine Learning · doi:10.1561/2200000083 · Cited by 7,772 · see citing works
Neural Architecture Search: A Survey
Elsken, T., Metzen, J. H., & Hutter, F. (2018) · arXiv · doi:10.48550/arxiv.1808.05377 · Cited by 1,763 · see citing works
AutoML: A survey of the state-of-the-art
He, X. J., Zhao, K., & Chu, X. (2020) · Knowledge-Based Systems · doi:10.1016/j.knosys.2020.106622 · Cited by 1,778 · see citing works
Dynamic Convolution: Attention Over Convolution Kernels
Chen, Y., Dai, X., Liu, M., Chen, D., Yuan, L., & Liu, Z. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01104 · Cited by 1,582 · see citing works
FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search
Wu, B., Keutzer, K., Dai, X., Zhang, P., Wang, Y., Sun, F., Wu, Y., Tian, Y., Vajda, P., & Jia, Y. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01099 · Cited by 1,302 · see citing works
Auto-Keras: An Efficient Neural Architecture Search System
Jin, H., Song, Q., & Hu, X. (2019) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3292500.3330648 · Cited by 963 · see citing works
Federated Learning for Internet of Things: Recent Advances, Taxonomy, and Open Challenges
Khan, L. U., Saad, W., Han, Z., Hossain, E., & Hong, C. S. (2021) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2021.3090430 · Cited by 892 · see citing works
Meta-Learning in Neural Networks: A Survey
Hospedales, T. M., Antoniou, A., Micaelli, P., & Storkey, A. J. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3079209 · Cited by 886 · see citing works
Raschka, S., Patterson, J. M., & Nolet, C. (2020) · Information · doi:10.3390/info11040193 · Cited by 777 · see citing works
Progressive Differentiable Architecture Search: Bridging the Depth Gap Between Search and Evaluation
Chen, X., Xie, L., Wu, J. H., & Tian, Q. C. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00138 · Cited by 709 · see citing works
Searching for a Robust Neural Architecture in Four GPU Hours
Dong, X., & Yang, Y. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00186 · Cited by 658 · see citing works
Random Search and Reproducibility for Neural Architecture SearchPreprint
Li, L., & Talwalkar, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.07638 · Cited by 534 · see citing works
PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture SearchPreprint
Xu, Y., Xie, L., Zhang, X., Chen, X., Qi, G.-J., Tian, Q. C., & Xiong, H. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.05737 · Cited by 545 · see citing works
Learning Generalisable Omni-Scale Representations for Person Re-Identification
Zhou, K., Yang, Y., Cavallaro, A., & Xiang, T. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3069237 · Cited by 353 · see citing works
Exploring Randomly Wired Neural Networks for Image Recognition
Xie, S., Kirillov, A., Girshick, R., & He, K. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00137 · Cited by 355 · see citing works
Self-citationFair DARTS: Eliminating Unfair Advantages in Differentiable Architecture Search
Chu, X., Zhou, T., Zhang, B., & Li, J. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58555-6_28 · Cited by 338 · see citing works
Rai, R., & Sahu, C. K. (2020) · IEEE Access · doi:10.1109/access.2020.2987324 · Cited by 324 · see citing works
FBNetV2: Differentiable Neural Architecture Search for Spatial and Channel Dimensions
Wan, A., Dai, X., Zhang, P., He, Z., Tian, Y., Xie, S., Wu, B., Yu, M., Xu, T., Chen, K., et al. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01298 · Cited by 300 · see citing works
Self-citationCARS: Continuous Evolution for Efficient Neural Architecture Search
Yang, Z., Wang, Y., Chen, X., Shi, B., Xu, C., Xu, C., Tian, Q. C., & Xu, C. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00190 · Cited by 291 · see citing works
BANANAS: Bayesian Optimization with Neural Architectures for Neural Architecture Search
White, C., Neiswanger, W., & Savani, Y. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i12.17233 · Cited by 287 · see citing works
Understanding and Robustifying Differentiable Architecture SearchPreprint
Zela, A., Elsken, T., Saikia, T., Marrakchi, Y., Brox, T., & Hutter, F. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.09656 · Cited by 279 · see citing works
Neural Architecture Search without TrainingPreprint
Mellor, J., Turner, J., Storkey, A., & Crowley, E. J. (2020) · Edinburgh Research Explorer (University of Edinburgh) · doi:10.48550/arxiv.2006.04647 · Cited by 277 · see citing works
DARTS+: Improved Differentiable Architecture Search with Early StoppingPreprint
Liang, H., Zhang, S., Sun, J., He, X., Huang, W., Zhuang, K., & Li, Z. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.06035 · Cited by 303 · see citing works
CurveLane-NAS: Unifying Lane-Sensitive Architecture Search and Adaptive Point Blending
Xu, H., Wang, S., Cai, X., Zhang, W., Liang, X., & Li, Z. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58555-6_41 · Cited by 249 · see citing works
Auto-FPN: Automatic Network Architecture Adaptation for Object Detection Beyond Classification
Xu, A., Yao, A., Li, A., Liang, A., & Zhang, A. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00675 · Cited by 228 · see citing works
Single-Path NAS: Designing Hardware-Efficient ConvNets in Less Than 4 Hours
Stamoulis, D. S., Ding, R., Wang, D., Lymberopoulos, D., Priyantha, N. B., Liu, J., & Marculescu, D. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-46147-8_29 · Cited by 229 · see citing works
A Survey on Neural Architecture SearchPreprint
Wistuba, M., Rawat, A., & Pedapati, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.01392 · Cited by 285 · see citing works
DetNAS: Backbone Search for Object Detection
Chen, Y., Yang, T., Zhang, X., Meng, G., Xiao, X., & Sun, J. (2019) · arXiv · doi:10.48550/arxiv.1903.10979 · Cited by 205 · see citing works
One-Shot Neural Architecture Search via Self-Evaluated Template Network
Dong, X., & Yang, Y. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00378 · Cited by 202 · see citing works
SGAS: Sequential Greedy Architecture Search
Li, G., Qian, G., Delgadillo, I. C., Muller, M., Thabet, A., & Ghanem, B. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00169 · Cited by 203 · see citing works
Multiobjective Evolutionary Design of Deep Convolutional Neural Networks for Image Classification
Lu, Z., Whalen, I., Dhebar, Y., Deb, K., Goodman, E. D., Banzhaf, W., & Boddeti, V. N. (2020) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2020.3024708 · Cited by 216 · see citing works
A Comprehensive Survey of Neural Architecture Search: Challenges and SolutionsPreprint
Ren, P., Xiao, Y., Chang, X., Huang, P.-Y., Li, Z., Chen, X., & Wang, X. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.02903 · Cited by 193 · see citing works
Neuroevolution in Deep Neural Networks: Current Trends and Future Challenges
Galván, E., & Mooney, P. (2021) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2021.3067574 · Cited by 200 · see citing works
Reinforcement learning for neural architecture search: A review
Jaafra, Y., Laurent, J. L., Deruyver, A., & Naceur, M. S. (2019) · Image and Vision Computing · doi:10.1016/j.imavis.2019.06.005 · Cited by 198 · see citing works
From federated learning to federated neural architecture search: a survey
Zhu, H., Zhang, H., & Jin, Y. (2021) · Complex & Intelligent Systems · doi:10.1007/s40747-020-00247-z · Cited by 178 · see citing works
Meta-Learning of Neural Architectures for Few-Shot Learning
Elsken, T., Staffler, B., Metzen, J. H., & Hutter, F. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01238 · Cited by 173 · see citing works
Graph Neural Architecture Search
Gao, Y., Yang, H., Zhang, P., Zhou, C., & Hu, Y. (2020) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/195 · Cited by 226 · see citing works
A comprehensive survey on optimizing deep learning models by metaheuristics
Akay, B., Karaboğa, D., & Akay, R. (2021) · Artificial Intelligence Review · doi:10.1007/s10462-021-09992-0 · Cited by 177 · see citing works
Zen-NAS: A Zero-Shot NAS for High-Performance Image Recognition
Lin, M., Wang, P., Sun, Z., Chen, H., Sun, X., Qian, Q., Li, H., & Jin, R. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00040 · Cited by 183 · see citing works
AdaShare: Learning What To Share For Efficient Deep Multi-Task Learning
Sun, X., Panda, R., Feris, R., & Saenko, K. (2019) · arXiv · doi:10.48550/arxiv.1911.12423 · Cited by 169 · see citing works
Differentiable quantum architecture search
Zhang, S., Hsieh, C., Zhang, S., & Yao, H. (2022) · Quantum Science and Technology · doi:10.1088/2058-9565/ac87cd · Cited by 170 · see citing works
Liu, R., Gao, J., Zhang, J., Meng, D., & Lin, Z. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3132674 · Cited by 154 · see citing works
Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired PerspectivePreprint
Chen, W., Gong, X., & Wang, Z. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.11535 · Cited by 160 · see citing works
BlockQNN: Efficient Block-Wise Neural Network Architecture Generation
Zhong, Z., Yang, Z., Deng, B., Yan, J., Wu, W., Shao, J., & Liu, C. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.2969193 · Cited by 143 · see citing works
Self-citationDSNAS: Direct Neural Architecture Search Without Parameter Retraining
Hu, S., Xie, S., Zheng, H., Liu, C., Shi, J., Liu, X., & Lin, D. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01210 · Cited by 136 · see citing works
Self-citationAdaBits: Neural Network Quantization With Adaptive Bit-Widths
Jin, Q., Yang, L., & Liao, Z. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00222 · Cited by 135 · see citing works
MiLeNAS: Efficient Neural Architecture Search via Mixed-Level Reformulation
He, C., Ye, H., Shen, L., & Zhang, T. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01201 · Cited by 132 · see citing works
NAS evaluation is frustratingly hardPreprint
Yang, A., Esperança, P. M., & Carlucci, F. M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.12522 · Cited by 132 · see citing works
NPENAS: Neural Predictor Guided Evolution for Neural Architecture Search
Wei, C., Niu, C., Tang, Y., Wang, Y., Hu, H., & Liang, J. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3151160 · Cited by 137 · see citing works
Stabilizing Differentiable Architecture Search via Perturbation-based Regularization
Chen, X., & Hsieh, C.-J. (2020) · arXiv · doi:10.48550/arxiv.2002.05283 · Cited by 126 · see citing works
FBNetV3: Joint Architecture-Recipe Search using Predictor Pretraining
Dai, X., Wan, A., Zhang, P., Wu, B., He, Z., Wei, Z., Chen, K., Tian, Y., Yu, M., Vajda, P., et al. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01601 · Cited by 127 · see citing works
DARTS-: Robustly Stepping out of Performance Collapse Without IndicatorsPreprint
Chu, X., Wang, X., Zhang, B., Lu, S., Wei, X. K., & Yan, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.01027 · Cited by 133 · see citing works
Multinomial Distribution Learning for Effective Neural Architecture Search
Zheng, X., Ji, R., Tang, L., Zhang, B., Liu, J., & Tian, Q. C. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00139 · Cited by 119 · see citing works
EcoNAS: Finding Proxies for Economical Neural Architecture Search
Zhou, D., Zhou, X., Zhang, W., Loy, C. C., Yi, S., Zhang, X., & Ouyang, W. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01141 · Cited by 117 · see citing works
Neural Architecture Search Based on a Multi-Objective Evolutionary Algorithm With Probability Stack
Xue, Y., Chen, C., & Słowik, A. (2023) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2023.3252612 · Cited by 120 · see citing works
$\beta$-DARTS: Beta-Decay Regularization for Differentiable Architecture Search
Ye, P., Li, B., Li, Y., Chen, T., Fan, J., & Ouyang, W. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01060 · Cited by 140 · see citing works
Path-Restore: Learning Network Path Selection for Image Restoration
Yu, K., Wang, X., Dong, C., Tang, X., & Loy, C. C. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3096255 · Cited by 115 · see citing works
AdversarialNAS: Adversarial Neural Architecture Search for GANs
Gao, C., Chen, Y., Liu, S., Tan, Z., & Yan, S. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00572 · Cited by 111 · see citing works
Can Weight Sharing Outperform Random Architecture Search? An Investigation With TuNAS
Bender, G., Liu, H., Chen, B., Chu, G., Cheng, S., Kindermans, P.-J., & Le, Q. V. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01433 · Cited by 110 · see citing works
DCNAS: Densely Connected Neural Architecture Search for Semantic Image Segmentation
Zhang, X., Xu, H., Mo, H., Tan, J., Yang, C., Wang, L., & Ren, W. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01374 · Cited by 122 · see citing works
Hit-Detector: Hierarchical Trinity Architecture Search for Object Detection
Guo, J., Han, K., Wang, Y., Zhang, C., Yang, Z., Wu, H., Chen, X., & Xu, C. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01142 · Cited by 110 · see citing works
Graph-Guided Architecture Search for Real-Time Semantic Segmentation
Lin, P., Sun, P., Cheng, G., Xie, S., Li, X., & Shi, J. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00426 · Cited by 107 · see citing works
Self-citationRethinking Architecture Selection in Differentiable NASPreprint
Wang, R., Cheng, M., Chen, X., Tang, X., & Hsieh, C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.04392 · Cited by 106 · see citing works
Differentiable Automatic Data Augmentation
Li, Y., Hu, G., Wang, Y., Hospedales, T. M., Robertson, N. M., & Yang, Y. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58542-6_35 · Cited by 150 · see citing works
Xue, Y., Jiang, P., Neri, F., & Liang, J. (2021) · International Journal of Neural Systems · doi:10.1142/s0129065721500350 · Cited by 103 · see citing works
Learning to Branch for Multi-Task LearningPreprint
Guo, P., Lee, C., & Ulbricht, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.01895 · Cited by 99 · see citing works
Improved Differentiable Architecture Search for Language Modeling and Named Entity Recognition
Jiang, Y., Hu, C., Xiao, T., Zhang, C., & Zhu, J. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1367 · Cited by 99 · see citing works
Neural Input Search for Large Scale Recommendation Models
Joglekar, M., Li, C., Chen, M., Xu, T., Wang, X., Adams, J. K., Khaitan, P., Liu, J., & Le, Q. V. (2020) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3394486.3403288 · Cited by 98 · see citing works
Efficient Neural Architecture Search via Proximal Iterations
Yao, Q., Xu, J., Tu, W.-W., & Zhu, Z. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.6143 · Cited by 104 · see citing works
Progressive DARTS: Bridging the Optimization Gap for NAS in the Wild
Chen, X., Xie, L., Wu, J. H., & Tian, Q. C. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-020-01396-x · Cited by 99 · see citing works
Zhang, T., Lei, C., Zhang, Z., Meng, X.-B., & Chen, C. L. P. (2021) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2021.3061466 · Cited by 96 · see citing works
Recent Progress in the CUHK Dysarthric Speech Recognition System
Liu, S., Geng, M., Hu, S., Xie, X., Cui, M., Yu, J., Liu, X., & Meng, H. (2021) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2021.3091805 · Cited by 96 · see citing works
Search to aggregate neighborhood for graph neural network
Zhao, H., Yao, Q., & Tu, W.-W. (2021) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde51399.2021.00054 · Cited by 97 · see citing works
Neural predictor based quantum architecture search
Zhang, S.-X., Hsieh, C.-Y., Zhang, S., & Yao, H. (2021) · Machine Learning: Science and Technology · doi:10.1088/2632-2153/ac28dd · Cited by 88 · see citing works
Partial Connection Based on Channel Attention for Differentiable Neural Architecture Search
Xue, Y., & Qin, J. (2022) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2022.3184700 · Cited by 87 · see citing works
Zhang, H., Jin, Y., Cheng, R., & Hao, K. (2020) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2020.3040272 · Cited by 95 · see citing works
Exploring Relational Context for Multi-Task Dense Prediction
Brüggemann, D., Kanakis, M., Obukhov, A., Georgoulis, S., & Gool, L. V. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01557 · Cited by 85 · see citing works
Overcoming Multi-Model Forgetting in One-Shot NAS With Diversity Maximization
Zhang, M., Li, H., Pan, S., Chang, X., & Su, S. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00783 · Cited by 83 · see citing works
AdaBERT: Task-Adaptive BERT Compression with Differentiable Neural Architecture Search
Chen, D., Li, Y., Qiu, M., Wang, Z., Li, B., Ding, B., Deng, H., Huang, J., Lin, W., & Zhou, J. (2020) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/341 · Cited by 88 · see citing works
You Only Search Once: Single Shot Neural Architecture Search via Direct Sparse Optimization
Zhang, X., Huang, Z., Wang, N., Xiang, S., & Pan, C. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3020300 · Cited by 89 · see citing works
Chen, Z., Qiu, G., Li, P., Zhu, L., Yang, X., & Sheng, B. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3293885 · Cited by 84 · see citing works
MTL-NAS: Task-Agnostic Neural Architecture Search Towards General-Purpose Multi-Task Learning
Gao, Y., Bai, H., Jie, Z., Ma, J., Jia, K., & Liu, W. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01156 · Cited by 83 · see citing works
A Review of the Gumbel-max Trick and its Extensions for Discrete Stochasticity in Machine Learning
Huijben, I. A. M., Kool, W., Paulus, M. B., & van Sloun, R. J. G. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3157042 · Cited by 85 · see citing works
Particle Swarm Optimization for Compact Neural Architecture Search for Image Classification
Huang, J., Xue, B., Sun, Y., Zhang, M., & Yen, G. G. (2022) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2022.3217290 · Cited by 82 · see citing works
One-Shot Neural Architecture Search: Maximising Diversity to Overcome Catastrophic Forgetting
Zhang, M., Li, H., Pan, S., Chang, X., Zhou, C. D., Ge, Z., & Su, S. W. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3035351 · Cited by 81 · see citing works
Partially-Connected Neural Architecture Search for Reduced Computational Redundancy
Xu, Y., Xie, L., Dai, W., Zhang, X., Chen, X., Qi, G.-J., Xiong, H., & Tian, Q. C. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3059510 · Cited by 81 · see citing works
AutoDispNet: Improving Disparity Estimation With AutoML
Saikia, T., Marrakchi, Y., Zela, A., Hutter, F., & Brox, T. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00190 · Cited by 78 · see citing works
ASAP: Architecture Search, Anneal and PrunePreprint
Noy, A., Nayman, N., Ridnik, T., Zamir, N., Doveh, S., Friedman, I., Giryes, R., & Zelnik‐Manor, L. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.04123 · Cited by 78 · see citing works
Bridging the Gap between Sample-based and One-shot Neural Architecture Search with BONASPreprint
Han, S., Pi, R., Xu, H., Li, Z., Kwok, J. T.-Y., & Zhang, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.09336 · Cited by 78 · see citing works
Improving One-Shot NAS by Suppressing the Posterior Fading
Li, X., Chen, L., Li, C., Sun, M., Wu, W., Yan, J., & Ouyang, W. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01385 · Cited by 73 · see citing works
SqueezeNAS: Fast Neural Architecture Search for Faster Semantic Segmentation
Shaw, A., Daniel, H., Landola, F., & Sidhu, S. (2019) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2019.00251 · Cited by 76 · see citing works
FracBits: Mixed Precision Quantization via Fractional Bit-Widths
Yang, L., & Jin, Q. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i12.17269 · Cited by 76 · see citing works
DetNAS: Neural Architecture Search on Object Detection.Preprint
Chen, Y., Yang, T., Zhang, X., Meng, G., Pan, C., & Sun, J. (2019) · arXiv (Cornell University) · Cited by 75 · see citing works
XNAS: Neural Architecture Search with Expert AdvicePreprint
Nayman, N., Noy, A., Ridnik, T., Friedman, I., Jin, R., & Zelnik‐Manor, L. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.08031 · Cited by 75 · see citing works
NAS-Bench-1Shot1: Benchmarking and Dissecting One-shot Neural Architecture SearchPreprint
Zela, A., Siems, J., & Hutter, F. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.10422 · Cited by 74 · see citing works
SP-NAS: Serial-to-Parallel Backbone Search for Object Detection
Jiang, C., Xu, H., Zhang, W., Liang, X., & Li, Z. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01188 · Cited by 67 · see citing works
DiffMG: Differentiable Meta Graph Search for Heterogeneous Graph Neural Networks
Ding, Y., Yao, Q., Zhao, H., & Zhang, T. (2021) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3447548.3467447 · Cited by 70 · see citing works
Li, J., Cao, X., Chen, R., Zhang, X., Huang, X., & Qu, Y. (2023) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2023.110701 · Cited by 70 · see citing works
Understanding Architectures Learnt by Cell-based Neural Architecture SearchPreprint
Shu, Y., Wang, W., & Cai, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.09569 · Cited by 70 · see citing works
EEEA-Net: An Early Exit Evolutionary Neural Architecture Search
Termritthikun, C., Jamtsho, Y., Ieamsaard, J., Muneesawang, P., & Lee, I. (2021) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2021.104397 · Cited by 68 · see citing works
- Authors
- Xie, S., Zheng, H., Liu, C., & Lin, L.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2018
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1812.09926
- OpenAlex
- W2905672847
- Cited by
- 731
- References
- 36
- Citations by year
- 2025: 30 · 2024: 51 · 2023: 93 · 2022: 91 · 2021: 224 · 2020: 170 · 2019: 70 · 2018: 2
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Canadian Institute for Advanced Research
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Physical Sciences › Computer Science › Artificial Intelligence › Evolutionary Algorithms and Applications
- Keywords
- neural architecture search, differentiable architecture search, credit assignment, search space optimization, end-to-end learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
