Stabilizing Differentiable Architecture Search via Perturbation-based Regularization
Open accessAbstract
Differentiable architecture search (DARTS) is a prevailing NAS solution to identify architectures. Based on the continuous relaxation of the architecture space, DARTS learns a differentiable architecture weight and largely reduces the search cost. However, its stability has been challenged for yielding deteriorating architectures as the search proceeds. We find that the precipitous validation loss landscape, which leads to a dramatic performance drop when distilling the final architecture, is an essential factor that causes instability. Based on this observation, we propose a perturbation-based regularization - SmoothDARTS (SDARTS), to smooth the loss landscape and improve the generalizability of DARTS-based methods. In particular, our new formulations stabilize DARTS-based methods by either random smoothing or adversarial attack. The search trajectory on NAS-Bench-1Shot1 demonstrates the effectiveness of our approach and due to the improved stability, we achieve performance gain across various search spaces on 4 datasets. Furthermore, we mathematically show that SDARTS implicitly regularizes the Hessian norm of the validation loss, which accounts for a smoother loss landscape and improved performance.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The convergence to a smooth landscape benefits the generalization of deep neural networks.
“The convergence to a smooth landscape has been shown to benefit the generalization of deep neural networks (Chen and Hsieh, 2020; Chen et al., 2022; Foret et al., 2021; Keskar et al., 2017).”
From Symbolic Discovery of Optimization Algorithms (2023) · cites this paper as Chen and Hsieh, 2020
Cites
Densely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566 · see citing works
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562 · see citing works
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., & Adam, H. (2017) · arXiv · doi:10.48550/arxiv.1704.04861 · Cited by 19,015 · see citing works
ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
Zhang, X., Zhou, X., Lin, M., & Sun, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00716 · Cited by 9,982 · see citing works
ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design
Ma, N., Zhang, X., Zheng, H.-T., & Sun, J. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01264-9_8 · Cited by 7,356 · cited 1× · see citing works
“2 ShuffleNet (v2) (Ma et al., 2018) 25.”
Learning Transferable Architectures for Scalable Image Recognition
Zoph, B., Vasudevan, V., Shlens, J., & Le, Q. V. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00907 · Cited by 6,631 · see citing works
Neural Architecture Search with Reinforcement LearningPreprint
Zoph, B., & Le, Q. V. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.01578 · Cited by 5,091 · see citing works
MnasNet: Platform-Aware Neural Architecture Search for Mobile
Tan, M., Chen, B., Pang, R., Vasudevan, V., Sandler, M., Howard, A., & Le, Q. V. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00293 · Cited by 3,416 · see citing works
Evolving Neural Networks through Augmenting Topologies
Stanley, K. O., & Miikkulainen, R. P. (2002) · Evolutionary Computation · doi:10.1162/106365602320169811 · Cited by 3,269 · see citing works
DARTS: Differentiable architecture search
Liu, H., Simonyan, K., & Yang, Y. (2018) · arXiv · doi:10.48550/arxiv.1806.09055 · Cited by 3,248 · see citing works
Improved regularization of convolutional neural networks with cutout
DeVries, T., & Taylor, G. W. (2017) · arXiv · doi:10.48550/arxiv.1708.04552 · Cited by 2,776 · see citing works
Progressive Neural Architecture Search
Liu, C., Zoph, B., Neumann, M., Shlens, J., Wei, H., Li, L.-J., Fei-Fei, L., Yuille, A., Huang, J., & Murphy, K. P. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01246-5_2 · Cited by 1,946 · see citing works
ProxylessNAS: Direct Neural Architecture Search on Target Task and HardwarePreprint
Cai, H., Zhu, L., & Han, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.00332 · Cited by 1,652 · see citing works
(2018) · International Conference on Machine Learning · Cited by 1,465 · see citing works
Large-Scale Evolution of Image ClassifiersPreprint
Real, E., Moore, S., Selle, A. P., Saxena, S., Suematsu, Y. I. L., Tan, J., Le, Q. V., & Kurakin, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.01041 · Cited by 1,372 · see citing works
Certified Adversarial Robustness via Randomized SmoothingPreprint
Cohen, J. M., Rosenfeld, E., & Kolter, J. Z. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.02918 · Cited by 1,172 · see citing works
Miikkulainen, R. P., Liang, J., Meyerson, E., Rawal, A., Fink, D., Francon, O., Raju, B., Shahrzad, H., Navruzyan, A., Duffy, N., et al. (2018) · Elsevier eBooks · doi:10.1016/b978-0-12-815480-9.00015-3 · Cited by 854 · see citing works
Hierarchical Representations for Efficient Architecture SearchPreprint
Liu, H., Simonyan, K., Vinyals, O., Fernando, C., & Kavukcuoglu, K. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.00436 · Cited by 851 · see citing works
Certified Robustness to Adversarial Examples with Differential Privacy
Lecuyer, M., Atlidakis, V., Geambasu, R., Hsu, D., & Jana, S. (2019) · 2019 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2019.00044 · Cited by 750 · see citing works
SNAS: Stochastic Neural Architecture Search
Xie, S., Zheng, H., Liu, C., & Lin, L. (2018) · arXiv · doi:10.48550/arxiv.1812.09926 · Cited by 731 · see citing works
Deep Pyramidal Residual Networks
Han, D., Kim, J., & Kim, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.668 · Cited by 715 · see citing works
Progressive Differentiable Architecture Search: Bridging the Depth Gap Between Search and Evaluation
Chen, X., Xie, L., Wu, J. H., & Tian, Q. C. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00138 · Cited by 709 · see citing works
Self-citationSearching for a Robust Neural Architecture in Four GPU Hours
Dong, X., & Yang, Y. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00186 · Cited by 658 · see citing works
Efficient Architecture Search by Network Transformation
Cai, H., Chen, T., Zhang, W., Yu, Y. H., & Wang, Y.-X. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11709 · Cited by 594 · see citing works
Practical Block-Wise Neural Network Architecture Generation
Zhong, Z., Yan, J., Wu, W., Shao, J., & Liu, C. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00257 · Cited by 555 · see citing works
PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture SearchPreprint
Xu, Y., Xie, L., Zhang, X., Chen, X., Qi, G.-J., Tian, Q. C., & Xiong, H. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.05737 · Cited by 545 · see citing works
Self-citationRandom Search and Reproducibility for Neural Architecture SearchPreprint
Li, L., & Talwalkar, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.07638 · Cited by 534 · see citing works
Neural Architecture OptimizationPreprint
Luo, R., Tian, F., Qin, T., Chen, E., & Liu, T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.07233 · Cited by 522 · see citing works
Adversarial Examples Improve Image Recognition
Xie, C., Tan, M., Gong, B., Wang, J., Yuille, A., & Le, Q. V. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00090 · Cited by 510 · see citing works
Towards Robust Neural Networks via Random Self-ensemble
Liu, X., Cheng, M., Zhang, H., & Hsieh, C. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01234-2_23 · Cited by 408 · see citing works
Self-citationFair DARTS: Eliminating Unfair Advantages in Differentiable Architecture Search
Chu, X., Zhou, T., Zhang, B., & Li, J. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58555-6_28 · Cited by 338 · see citing works
Regularized Evolution for Image Classifier Architecture Search
Real, E., Aggarwal, A., Huang, Y., & Le, Q. V. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33014780 · Cited by 328 · see citing works
DARTS+: Improved Differentiable Architecture Search with Early StoppingPreprint
Liang, H., Zhang, S., Sun, J., He, X., Huang, W., Zhuang, K., & Li, Z. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.06035 · Cited by 303 · see citing works
Breaking the Softmax Bottleneck: A High-Rank RNN Language ModelPreprint
Yang, Z., Dai, Z., Salakhutdinov, R., & Cohen, W. W. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.03953 · Cited by 291 · see citing works
Understanding and Robustifying Differentiable Architecture SearchPreprint
Zela, A., Elsken, T., Saikia, T., Marrakchi, Y., Brox, T., & Hutter, F. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.09656 · Cited by 279 · see citing works
Certified Adversarial Robustness with Additive NoisePreprint
Li, B., Chen, C., Wang, W., & Carin, L. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.03113 · Cited by 225 · see citing works
SMASH: One-Shot Model Architecture Search through HyperNetworks
Brock, A., Lim, T., Ritchie, J. M., & Weston, N. J. (2018) · International Conference on Learning Representations · Cited by 217 · see citing works
Learning Curve Prediction with Bayesian Neural Networks
Klein, A., Falkner, S., Springenberg, J. T., & Hutter, F. (2017) · International Conference on Learning Representations · Cited by 176 · see citing works
BayesNAS: A Bayesian Approach for Neural Architecture SearchPreprint
Zhou, H., Yang, M., Wang, J., & Pan, W. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.04919 · Cited by 174 · see citing works
Evaluating The Search Phase of Neural Architecture Search
Yu, K., Sciuto, C., Jaggi, M., Musat, C., & Salzmann, M. (2020) · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · Cited by 136 · see citing works
Rob-GAN: Generator, Discriminator, and Adversarial Attacker
Liu, X., & Hsieh, C. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01149 · Cited by 129 · see citing works
Self-citationEfficient Neural Architecture Search via Proximal Iterations
Yao, Q., Xu, J., Tu, W.-W., & Zhu, Z. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.6143 · Cited by 104 · see citing works
NAS-Bench-1Shot1: Benchmarking and Dissecting One-shot Neural Architecture SearchPreprint
Zela, A., Siems, J., & Hutter, F. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.10422 · Cited by 74 · see citing works
Understanding Architectures Learnt by Cell-based Neural Architecture SearchPreprint
Shu, Y., Wang, W., & Cai, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.09569 · Cited by 70 · see citing works
How Does Noise Help Robustness? Explanation and Exploration under the Neural SDE Framework
Liu, X., Xiao, T., Si, S., Cao, Q., Kumar, S., & Hsieh, C. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00036 · Cited by 54 · see citing works
Self-citationStabilizing Differentiable Architecture Search via Perturbation-based Regularization
Chen, X. (2025) · TIB Data Manager · doi:10.57702/ahufmtof · Cited by 36 · see citing works
Self-citation
Flagged first, then by citations
5 of the 51 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Local Learning Matters: Rethinking Data Heterogeneity in Federated Learning
Mendieta, M., Yang, T., Wang, P., Lee, M., Ding, Z., & Chen, C. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00821 · Cited by 204 · see citing works
AutoML: A systematic review on automated machine learning with neural architecture search
Salehin, I., Islam, M. S., Saha, P., Noman, S. M., Tuni, A., Hasan, M. M., & Baten, M. A. (2023) · Journal of Information and Intelligence · doi:10.1016/j.jiixd.2023.10.002 · Cited by 194 · see citing works
Symbolic Discovery of Optimization Algorithms
Chen, X., Liang, C., Huang, D., Real, E., Wang, K., Liu, Y., Pham, H., Dong, X., Luong, T., Hsieh, C.-J., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.06675 · Cited by 303 · see citing works
“, 2022), neural architecture search (Chen and Hsieh, 2020; Chen et al., 2021; Liu et al., 2019a; Pham et al., 2018; Real et al., 2019; So et al., 2019; Wang et al., 2021; Yang et al., 2022; Zoph and Le, 2017) and hyperparameter optimization (Dong et al.”
BackgroundSelf-citationDifferentiable quantum architecture search
Zhang, S., Hsieh, C., Zhang, S., & Yao, H. (2022) · Quantum Science and Technology · doi:10.1088/2058-9565/ac87cd · Cited by 170 · see citing works
Self-citation$\beta$-DARTS: Beta-Decay Regularization for Differentiable Architecture Search
Ye, P., Li, B., Li, Y., Chen, T., Fan, J., & Ouyang, W. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01060 · Cited by 140 · see citing works
Systematic review on neural architecture search
Avval, S. S. P., Eskue, N., Groves, R. M., & Yaghoubi, V. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-024-11058-w · Cited by 100 · see citing works
Partial Connection Based on Channel Attention for Differentiable Neural Architecture Search
Xue, Y., & Qin, J. (2022) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2022.3184700 · Cited by 87 · see citing works
Chen, Z., Qiu, G., Li, P., Zhu, L., Yang, X., & Sheng, B. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3293885 · Cited by 84 · see citing works
Self-Adaptive Weight Based on Dual-Attention for Differentiable Neural Architecture Search
Xue, Y., Han, X., & Wang, Z. (2024) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2023.3348843 · Cited by 72 · see citing works
Shapley-NAS: Discovering Operation Contribution for Neural Architecture Search
Xiao, H., Wang, Z., Zhu, Z., Zhou, J., & Lu, J. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01159 · Cited by 67 · see citing works
DOTS: Decoupling Operation and Topology in Differentiable Architecture Search
Gu, Y., Wang, L., Liu, Y., Yang, Y., Wu, Y.-H., Lu, S.-P., & Cheng, M. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01213 · Cited by 60 · see citing works
A Gradient-Guided Evolutionary Neural Architecture Search
Xue, Y., Han, X., Neri, F., Qin, J., & Pelusi, D. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3371432 · Cited by 58 · see citing works
AdvRush: Searching for Adversarially Robust Neural Architectures
Mok, J., Na, B., Choe, H., & Yoon, S. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01210 · Cited by 51 · see citing works
DSRNA: Differentiable Search of Robust Neural Architectures
Hosseini, R., Yang, X., & Xie, P. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00613 · Cited by 48 · see citing works
Gradient Descent Effects on Differential Neural Architecture Search: A Survey
Santra, S., Hsieh, J.-W., & Lin, C. (2021) · IEEE Access · doi:10.1109/access.2021.3090918 · Cited by 53 · see citing works
Meta Navigator: Search for a Good Adaptation Policy for Few-shot Learning
Zhang, C., Ding, H., Lin, G., Li, R., Wang, C., & Shen, C. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00930 · Cited by 44 · see citing works
Towards Accurate and Compact Architectures via Neural Architecture Transformer
Guo, Y. F., Zheng, Y., Tan, M., Chen, Q., Li, Z., Chen, J., Zhao, P., & Huang, J. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3086914 · Cited by 43 · see citing works
Improved Differentiable Architecture Search With Multi-Stage Progressive Partial Channel Connections
Xue, Y., Lu, C., Neri, F., & Qin, J. (2023) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2023.3301395 · Cited by 42 · see citing works
Efficient Evaluation Methods for Neural Architecture Search: A Survey
Song, X., Xie, X., Lv, Z., Yen, G. G., Ding, W., Lv, J., & Sun, Y. (2024) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2024.3477457 · Cited by 48 · see citing works
SaDENAS: A self-adaptive differential evolution algorithm for neural architecture search
Han, X., Xue, Y., Wang, Z., Zhang, Y., Muravev, A., & Gabbouj, M. (2024) · Swarm and Evolutionary Computation · doi:10.1016/j.swevo.2024.101736 · Cited by 36 · see citing works
An architecture entropy regularizer for differentiable neural architecture search
Jing, K., Chen, L., & Xu, J. (2022) · Neural Networks · doi:10.1016/j.neunet.2022.11.015 · Cited by 33 · see citing works
Manas: multi-agent neural architecture search
Lopes, V., Carlucci, F. M., Esperança, P. M., Singh, M., Yang, A., Gabillon, V., Xu, H., Chen, Z., & Wang, J. (2023) · Machine Learning · doi:10.1007/s10994-023-06379-w · Cited by 35 · see citing works
Chen, W., Gong, X., Wu, J., Wei, Y., Shi, H., Yan, Z., Yang, Y., & Wang, Z. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3328347 · Cited by 28 · see citing works
iDARTS: Improving DARTS by Node Normalization and Decorrelation Discretization
Wang, H., Yang, R., Huang, D., & Wang, Y. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3105698 · Cited by 27 · see citing works
Differentiable Architecture Search with Random Features
Zhang, X., Li, Y., Zhang, X., Wang, Y., & Sun, J. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01541 · Cited by 26 · see citing works
Liu, X., Gong, X., Plaza, A., Cai, Z., Xiao, X., Jiang, X., & Liu, X. (2022) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2022.3220748 · Cited by 25 · see citing works
BaLeNAS: Differentiable Architecture Search via the Bayesian Learning Rule
Zhang, M., Pan, S., Chang, X., Su, S. W., Hu, J., Haffari, G., & Yang, B. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01157 · Cited by 23 · see citing works
IS-DARTS: Stabilizing DARTS through Precise Measurement on Candidate Importance
He, H., Liu, L., Zhang, H., & Zheng, N. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i11.29128 · Cited by 23 · see citing works
Weight-Sharing Neural Architecture Search: A Battle to Shrink the Optimization GapPreprint
Xie, L., Chen, X., Bi, K., Wei, L., Xu, Y., Chen, Z., Wang, L., Xiao, A., Chang, J., Zhang, X., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.01475 · Cited by 29 · see citing works
Self-citationBootstrapping ViTs: Towards Liberating Vision Transformers from Pre-training
Zhang, H., Duan, J., Xue, M., Song, J., Li, S., & Song, M. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00874 · Cited by 23 · see citing works
EG-NAS: Neural Architecture Search with Fast Evolutionary Exploration
Cai, Z., Chen, L., Liu, P., Ling, T., & Lai, Y. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i10.28993 · Cited by 21 · see citing works
Yan, X., Huang, H. P., Jin, Y., Chen, L., Liang, Z., & Hao, Z. (2023) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2023.3301774 · Cited by 21 · see citing works
NIPQ: Noise proxy-based Integrated Pseudo-Quantization
Shin, J., So, J., Park, S., Kang, S., Yoo, S., & Park, E. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00375 · Cited by 20 · see citing works
ASP: Learn a Universal Neural Solver!
Wang, C., Yu, Z., McAleer, S., Yu, T., & Yang, Y. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3352096 · Cited by 20 · see citing works
Neural Architecture Search for Dense Prediction Tasks in Computer Vision
Mohan, R., Elsken, T., Zela, A., Metzen, J. H., Staffler, B., Brox, T., Valada, A., & Hutter, F. (2023) · International Journal of Computer Vision · doi:10.1007/s11263-023-01785-y · Cited by 23 · see citing works
OStr-DARTS: Differentiable Neural Architecture Search Based on Operation Strength
Yang, L., Zheng, Z., Han, Y., Song, S., Huang, G. M., & Li, F. (2024) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2024.3455760 · Cited by 17 · see citing works
Performance-Aware Mutual Knowledge Distillation for Improving Neural Architecture Search
Xie, P., & Du, X. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01162 · Cited by 17 · see citing works
Improving Differentiable Architecture Search via self-distillation
Zhu, X., Li, J., Liu, Y., & Wang, W. (2023) · Neural Networks · doi:10.1016/j.neunet.2023.08.062 · Cited by 16 · see citing works
Accelerating Neural Architecture Search via Proxy Data
Na, B., Mok, J., Choe, H., & Yoon, S. (2021) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/392 · Cited by 16 · see citing works
Zero-Cost Operation Scoring in Differentiable Architecture Search
Xiang, L., Dudziak, Ł., Abdelfattah, M. S., Chau, T., Lane, N. D., & Wen, H. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i9.26243 · Cited by 14 · see citing works
Feng, J., He, Y., Pan, Y., Zhou, Z., Chen, S., & Gong, W. (2024) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2024.3352239 · Cited by 15 · see citing works
Rajapakshe, T., Rana, R. K., Khalifa, S., Şişman, B., Schuller, B. W., & Busso, C. (2024) · IEEE Access · doi:10.1109/access.2024.3439604 · Cited by 14 · see citing works
VNAS: Variational Neural Architecture Search
Ma, B., Zhang, J., Xia, Y., & Tao, D. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-024-02014-w · Cited by 14 · see citing works
Zhao, Y., Feng, L., Tang, J., Zhao, W., Ding, Z., Li, A., & Zheng, Z. (2022) · Applied Animal Behaviour Science · doi:10.1016/j.applanim.2022.105594 · Cited by 14 · see citing works
Wang, Y., Liu, Y., Dai, W., Li, C., Zou, J., & Xiong, H. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01209 · Cited by 12 · see citing works
On neural architecture search and hyperparameter optimization: A max-flow based approach
Xue, C., Li, J., Wang, X., Zhan, Y., Yan, J., & Li, C.-G. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.107507 · Cited by 9 · see citing works
Chen, H., Huang, H., Zuo, X., & Zhao, X. (2022) · Mathematics · doi:10.3390/math10152724 · Cited by 9 · see citing works
Revisiting Training-free NAS Metrics: An Efficient Training-based Method
Yang, T., Yang, L., Jin, X., & Chen, C. (2023) · 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv56688.2023.00473 · Cited by 9 · see citing works
NASGuard: A Novel Accelerator Architecture for Robust Neural Architecture Search (NAS) Networks
Wang, X., Zhao, B., Hou, R., Awad, A., Tian, Z., & Meng, D. (2021) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1109/isca52012.2021.00066 · Cited by 9 · see citing works
EC-DARTS: Inducing Equalized and Consistent Optimization into DARTS
Zhou, Q., Zheng, X., Cao, L., Zhong, B., Xi, T., Zhang, G., Ding, E., Xu, M., & Ji, R. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01177 · Cited by 9 · see citing works
Differentiable sampling based efficient architecture search for automatic fault diagnosis
Zhang, X., Ma, R., Zhao, Y., Wang, C., Zhao, Z., & Chen, X. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.107214 · Cited by 8 · see citing works
Self-citationContinuous–Discrete Alignment Optimization for efficient differentiable neural architecture search
Liu, W., Wu, J., Deng, T., & Yan, F. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.110721 · Cited by 7 · see citing works
Unleashing the Power of Gradient Signal-to-Noise Ratio for Zero-Shot NAS
Sun, Z., Sun, Y., Yang, L., Lu, S., Mei, J., Zhao, W., & Hu, Y. H. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00530 · Cited by 7 · see citing works
Spatial and channel level feature redundancy reduction for differentiable neural architecture search
Yin, S., Niu, B., Wang, R., & Wang, X. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.129713 · Cited by 7 · see citing works
RANK-NOSH: Efficient Predictor-Based Architecture Search via Non-Uniform Successive Halving
Wang, R., Chen, X., Cheng, M., Tang, X., & Hsieh, C. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01021 · Cited by 7 · see citing works
Self-citationOn the Privacy Risks of Cell-Based NAS Architectures
Huang, H., Zhang, Z., Shen, Y. C., Michael Backes, A., Li, Q. L., & Zhang, Y. (2022) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3548606.3560619 · Cited by 7 · see citing works
STC-NAS: Fast neural architecture search with source-target consistency
Sun, Z., Hu, Y., Yang, L., Lu, S., Mei, J., Han, Y., & Li, X. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.11.082 · Cited by 7 · see citing works
Recent Trends in Neural Architecture Search Systems
Ali, S., & Wani, M. A. (2022) · International Conference on Machine Learning and Applications · doi:10.1109/icmla55696.2022.00272 · Cited by 6 · see citing works
Ming, R., Liu, X., Li, Y., Huang, W. D., & Zhang, W. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.107326 · Cited by 6 · see citing works
Mean-Shift Based Differentiable Architecture Search
Hsieh, J.-W., Chou, C.-H., Chang, M., Chen, P.-Y., Santra, S., & Huang, C.-S. (2023) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2023.3329792 · Cited by 8 · see citing works
STO-DARTS: Stochastic Bilevel Optimization for Differentiable Neural Architecture Search
Cai, Z., Chen, L., Ling, T., & Liu, H. (2024) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2024.3359046 · Cited by 6 · see citing works
A new pipeline with ultimate search efficiency for neural architecture search
Liu, W., Qiao, X., Zhao, C., Deng, T., & Yan, F. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.107163 · Cited by 6 · see citing works
Learning from Mistakes – a Framework for Neural Architecture Search
Garg, B., Zhang, L., Sridhara, P., Hosseini, R., Xing, E. P., & Xie, P. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i9.21258 · Cited by 6 · see citing works
Yang, L., Fu, Y.-X., Lu, S.-N., Sun, Z.-H., Mei, J.-L., Zhao, W.-X., & Hu, Y. (2023) · Neural Networks · doi:10.1016/j.neunet.2023.09.012 · Cited by 8 · see citing works
Adaptive Channel Allocation for Robust Differentiable Architecture Search
Li, C., Ning, J., Hu, H., & He, K. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3441708 · Cited by 5 · see citing works
Efficient Architecture Search via Bi-Level Data Pruning
Tu, C., Ye, P., Lin, W., Ye, H., Yu, C., Chen, T., Li, B., & Ouyang, W. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3483276 · Cited by 5 · see citing works
Inter-layer transition in neural architecture search
Ma, B., Zhang, J., Xia, Y., & Tao, D. (2023) · Pattern Recognition · doi:10.1016/j.patcog.2023.109697 · Cited by 5 · see citing works
GeNAS: Neural Architecture Search with Better Generalization
Jeong, J., Yu, J., Park, G., Han, D., & Yoo, Y. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/101 · Cited by 5 · see citing works
How predictors affect the RL-based search strategy in Neural Architecture Search?
Wu, J., Deng, T., & Hu, Q. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.121742 · Cited by 5 · see citing works
PP-NAS: Searching for Plug-and-Play Blocks on Convolutional Neural Networks
Xiao, A., Shen, B., Tian, J., & Hu, Z. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3264551 · Cited by 5 · see citing works
BARS: Joint Search of Cell Topology and Layout for Accurate and Efficient Binary ARchitecturesPreprint
Zhao, T., Ning, X., Shi, X., Yang, S., Liang, S., Lei, P., Chen, J., Yang, H., & Wang, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.10804 · Cited by 5 · see citing works
Differentiable Architecture Search Without Training Nor Labels: A Pruning Perspective.Preprint
Zhang, M., Su, S. W., Pan, S., Chang, X., Huang, W., & Haffari, G. (2021) · arXiv (Cornell University) · Cited by 5 · see citing works
Regularization method for reduced biquaternion neural network
Gai, S., & Huang, X. L. (2024) · Applied Soft Computing · doi:10.1016/j.asoc.2024.112206 · Cited by 4 · see citing works
CONet: Channel Optimization for Convolutional Neural Networks
Hosseini, M. S., Zhang, J. S., Liu, Z., Fu, A., Su, J., Tuli, M., & Plataniotis, K. N. (2021) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw54120.2021.00041 · Cited by 4 · see citing works
Dynamic Network for Language-based Fashion Retrieval
Li, H., Wu, Y., & Wang, F. (2023) · doi:10.1145/3606040.3617438 · Cited by 4 · see citing works
REP: An Interpretable Robustness Enhanced Plugin for Differentiable Neural Architecture Search
Feng, Y., Sun, Y., Yen, G. G., & Tan, K. C. (2025) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2025.3543503 · Cited by 4 · see citing works
Wang, X., Lian, Z., Lin, J., Xue, C., & Yan, J. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3298296 · Cited by 4 · see citing works
PTP: Boosting Stability and Performance of Prompt Tuning with Perturbation-Based Regularizer
Chen, L., Chen, J., Huang, H., & Cheng, M. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.833 · Cited by 4 · see citing works
Semantic-DARTS: Elevating Semantic Learning for Mobile Differentiable Architecture Search
Guo, B., He, S., Shi, M., Yu, K., Chen, J., & Shen, X. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3462954 · Cited by 3 · see citing works
A3: A Platform of Searching for Robust Neural Architectures and Efficient Adversarial Attacks
Sun, J., Yao, W., Jiang, T., Li, C., & Chen, X. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3535933 · Cited by 3 · see citing works
Self-citationAnti-Bandit for Neural Architecture Search
Wang, R., Yang, L., Chen, H., Wang, W., Doermann, D., & Zhang, B. (2023) · International Journal of Computer Vision · doi:10.1007/s11263-023-01826-6 · Cited by 3 · see citing works
ROME: Robustifying Memory-Efficient NAS via Topology Disentanglement and Gradient Accumulation
Wang, X., Chu, X., Fan, Y., Zhang, Z., Zhang, B., Yang, X., & Yan, J. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00546 · Cited by 3 · see citing works
Operation-level Progressive Differentiable Architecture Search
Zhu, X., Li, J., Liu, Y., Liao, J., & Wang, W. (2021) · Proceedings · doi:10.1109/icdm51629.2021.00205 · Cited by 3 · see citing works
Liao, W., Zhang, Z., Wang, X., Yin, T., Liu, H., Ren, Z., Yin, J., & Feng, S. (2022) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme52920.2022.9859708 · Cited by 3 · see citing works
Unchain the Search Space with Hierarchical Differentiable Architecture Search
Liu, G., Zhong, Y., Guo, S., Scott, M. R., & Huang, W. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i10.17048 · Cited by 3 · see citing works
Skillearn: Machine Learning Inspired by Humans' Learning SkillsPreprint
Xie, P., Du, X., & Ban, H. (2020) · doi:10.36227/techrxiv.13351739 · Cited by 3 · see citing works
Miranda, T. Z., Sardinha, D. B., Neri, F., Basgalupp, M. P., & Cerri, R. (2023) · Applied Soft Computing · doi:10.1016/j.asoc.2023.110967 · Cited by 2 · see citing works
Hao, D., & Pei, S. (2025) · Neural Computing and Applications · doi:10.1007/s00521-024-10681-6 · Cited by 2 · see citing works
FastTalker: An unified framework for generating speech and conversational gestures from text
Zhang, J., Guo, Z., He, M., & Yoshie, O. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.130074 · Cited by 2 · see citing works
Robustifying DARTS by Eliminating Information Bypass Leakage via Explicit Sparse Regularization
Zhang, J., & Ding, Z. (2021) · Proceedings · doi:10.1109/icdm51629.2021.00099 · Cited by 2 · see citing works
Neural Architecture Selection as a Nash Equilibrium With Batch Entanglement
Li, Q., Xue, C., Li, M., Li, C.-G., Ma, C., & Yang, X. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3283239 · Cited by 2 · see citing works
Subbotko, K., Jabłoński, W., & Biliński, P. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00173 · Cited by 2 · see citing works
Differentiable Instruction Optimization for Cross-Task Generalization
Isonuma, M., Mori, J., & Sakata, I. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.667 · Cited by 1 · see citing works
Olweus, E., & Mengshoel, O. J. (2024) · Linköping electronic conference proceedings · doi:10.3384/ecp208003 · Cited by 1 · see citing works
Layer-Wisely Supervised Learning For One-Shot Neural Architecture Search
Chen, Y., Guo, Z., Yin, Q., Chen, H., & Huang, K. (2022) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn55064.2022.9892632 · Cited by 1 · see citing works
Searching for Robust Binary Neural Networks via Bimodal Parameter Perturbation
Ahn, D., Kim, H., Kim, T., Park, E., & Kim, J. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00244 · Cited by 1 · see citing works
Learning by Passing Tests, with Application to Neural Architecture SearchPreprint
Du, X., Xie, P., & Zhang, H. (2020) · doi:10.36227/techrxiv.13300961.v1 · Cited by 1 · see citing works
Learning by Teaching, with Application to Neural Architecture SearchPreprint
Sheth, P., Xie, P., & Jiang, Y. (2020) · doi:10.36227/techrxiv.13489206.v1 · Cited by 1 · see citing works
Learning by Passing Tests, with Application to Neural Architecture SearchPreprint
Du, X., Zhang, H., & Xie, P. (2020) · doi:10.36227/techrxiv.13300961 · Cited by 1 · see citing works
Skillearn: Machine Learning Inspired by Humans’ Learning SkillsPreprint
Xie, P., Du, X., & Ban, H. (2020) · doi:10.36227/techrxiv.13351739.v1 · Cited by 1 · see citing works
- Authors
- Chen, X., & Hsieh, C.-J.
- Affiliations
- University of California, Los Angeles (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2020
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2002.05283
- OpenAlex
- W3006500080
- Cited by
- 126
- References
- 51
- Citations by year
- 2025: 17 · 2024: 22 · 2023: 33 · 2022: 23 · 2021: 23 · 2020: 8
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- differentiable architecture search, DART, adversarial attacks, generalization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
