The DOI matches its Crossref record; 101 of the 101 references it cites were found in a registry or an index.
A Comprehensive Survey of Neural Architecture Search: Challenges and Solutions
Abstract
Deep learning has made substantial breakthroughs in many fields due to its powerful automatic representation capabilities. It has been proven that neural architecture design is crucial to the feature representation of data and the final performance. However, the design of the neural architecture heavily relies on the researchers’ prior knowledge and experience. And due to the limitations of humans’ inherent knowledge, it is difficult for people to jump out of their original thinking paradigm and design an optimal model. Therefore, an intuitive idea would be to reduce human intervention as much as possible and let the algorithm automatically design the neural architecture.Neural Architecture Search(NAS) is just such a revolutionary algorithm, and the related research work is complicated and rich. Therefore, a comprehensive and systematic survey on the NAS is essential. Previously related surveys have begun to classify existing work mainly based on the key components of NAS: search space, search strategy, and evaluation strategy. While this classification method is more intuitive, it is difficult for readers to grasp the challenges and the landmark work involved. Therefore, in this survey, we provide a new perspective: beginning with an overview of the characteristics of the earliest NAS algorithms, summarizing the problems in these early NAS algorithms, and then providing solutions for subsequent related research work. In addition, we conduct a detailed and comprehensive analysis, comparison, and summary of these works. Finally, we provide some possible future research directions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Densely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
Rethinking the Inception Architecture for Computer Vision
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.308 · Cited by 32,138
Proceedings of IEEE Conference on Computer Vision and Pattern Recognition
(1993) · doi:10.1109/cvpr.1993 · Cited by 17,667
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
Tan, M., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1905.11946 · Cited by 17,072
Object recognition from local scale-invariant features
Lowe, D. G. (1999) · Proceedings of the Seventh IEEE International Conference on Computer Vision · doi:10.1109/iccv.1999.790410 · Cited by 16,472 · cited 1×
“Deep learning has transformed the traditional way of manually designing features [13, 14] into automatic extraction [4, 29, 30].”
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204
Proceedings of IEEE International Conference on Computer Vision
(1995) · Proceedings of IEEE International Conference on Computer Vision · doi:10.1109/iccv.1995.466933 · Cited by 12,860
Proceedings of the 24th international conference on Machine learning
(2007) · doi:10.1145/1273496 · Cited by 11,630
Bidirectional recurrent neural networks
Schuster, M., & Paliwal, K. K. (1997) · IEEE Transactions on Signal Processing · doi:10.1109/78.650093 · Cited by 10,172
ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
Zhang, X., Zhou, X., Lin, M., & Sun, J. (2017) · arXiv · doi:10.48550/arxiv.1707.01083 · Cited by 9,982
Conditional Generative Adversarial Nets
Mirza, M., & Osindero, S. (2014) · arXiv · doi:10.48550/arxiv.1411.1784 · Cited by 8,863
Zagoruyko, S., & Komodakis, N. (2016) · Procedings of the British Machine Vision Conference 2016 · doi:10.5244/c.30.87 · Cited by 7,244
Improved Techniques for Training GANs
Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., & Chen, X. (2016) · arXiv · doi:10.48550/arxiv.1606.03498 · Cited by 6,965
Self-citationInternational Conference on Learning Representations (ICLR 2013)
松尾, 豊. (2013) · 人工知能学会誌 = Journal of Japanese Society for Artificial Intelligence · Cited by 6,286
(2009) · Cited by 6,153
mixup: Beyond Empirical Risk Minimization
Zhang, H., Cisse, M., Dauphin, Y. N., & Lopez-Paz, D. (2017) · arXiv · doi:10.48550/arxiv.1710.09412 · Cited by 5,831
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
Wu, Y., Schuster, M., Chen, Z., Le, Q. V., Norouzi, M., Macherey, W., Krikun, M., Cao, Y., Gao, Q., Macherey, K., et al. (2016) · arXiv · doi:10.48550/arxiv.1609.08144 · Cited by 5,319 · cited 2×
“Artificially designed networks have made breakthroughs in many fields, including image recognition [4, 7, 112], machine translation [1, 3, 141, 149], semantic segmentation [142, 143, 148], object detection [8, 146, 147], and video understanding [144, 145] and so on.”
BackgroundProgressive Growing of GANs for Improved Quality, Stability, and Variation
Karras, T., Aila, T., Laine, S., & Lehtinen, J. (2017) · arXiv · doi:10.48550/arxiv.1710.10196 · Cited by 4,727
Neural Architecture Search with Reinforcement Learning
Zoph, B., & Le, Q. V. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.01578 · Cited by 3,974
MnasNet: Platform-Aware Neural Architecture Search for Mobile
Tan, M., Chen, B., Pang, R., Vasudevan, V., Sandler, M., Howard, A., & Le, Q. V. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00293 · Cited by 3,416
Large Scale Distributed Deep Networks
Dean, J. B., Corrado, G. S., Monga, R., Chen, K., Devin, M., Mao, M., Ranzato, M., Senior, A., Tucker, P. A., Yang, K., et al. (2012) · Cited by 3,199
Improved regularization of convolutional neural networks with cutout
DeVries, T., & Taylor, G. W. (2017) · arXiv · doi:10.48550/arxiv.1708.04552 · Cited by 2,776
A Taxonomy of Global Optimization Methods Based on Response Surfaces
Jones, D. R. (2001) · Journal of Global Optimization · doi:10.1023/a:1012771025575 · Cited by 2,159
Neural Architecture Search: A Survey
Elsken, T., Metzen, J. H., & Hutter, F. (2018) · arXiv · doi:10.48550/arxiv.1808.05377 · Cited by 1,763
(2018) · International Conference on Machine Learning · Cited by 1,465
An overview of bilevel optimization
Colson, B., Marcotte, P., & Savard, G. (2007) · Annals of Operations Research · doi:10.1007/s10479-007-0176-2 · Cited by 1,409
Deep Subdomain Adaptation Network for Image Classification
Zhu, Y., Zhuang, F., Wang, J., Ke, G., Chen, J., Bian, J., Xiong, H., & He, Q. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2988928 · Cited by 1,245
Once-for-All: Train One Network and Specialize it for Efficient Deployment
Cai, H., Gan, C., Wang, T., Zhang, Z., & Han, S. (2019) · arXiv · doi:10.48550/arxiv.1908.09791 · Cited by 845
Single Path One-Shot Neural Architecture Search with Uniform Sampling
Guo, Z., Zhang, X., Mu, H., Heng, W., Liu, Z., Wei, Y., & Sun, J. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58517-4_32 · Cited by 813
Transfer Learning for Low-Resource Neural Machine Translation
Zoph, B., Yuret, D., May, J., & Knight, K. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1163 · Cited by 788
Elsken, T., Metzen, J. H., & Hutter, F. (2019) · The Springer Series on Challenges in Machine Learning · doi:10.1007/978-3-030-05318-5_3 · Cited by 785
Net2Net: Accelerating Learning via Knowledge TransferPreprint
Chen, T., Goodfellow, I., & Shlens, J. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.05641 · Cited by 649
Multi-task Sequence to Sequence LearningPreprint
Luong, M.-T., Le, Q. V., Sutskever, I., Vinyals, O., & Kaiser, Ł. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.06114 · Cited by 606
FractalNet: Ultra-Deep Neural Networks without Residuals
Larsson, G., Maire, M., & Shakhnarovich, G. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.07648 · Cited by 586
NSGA-Net: neural architecture search using multi-objective genetic algorithm
Lu, Z., Whalen, I., Boddeti, V., Dhebar, Y., Deb, K., Goodman, E., & Banzhaf, W. (2019) · Proceedings of the Genetic and Evolutionary Computation Conference · doi:10.1145/3321707.3321729 · Cited by 496
11th International Joint Conference on Artificial Intelligence
(1988) · Cognitive Science · doi:10.1207/s15516709cog1202_7 · Cited by 467
Neural Architecture Optimization
Luo, R., Tian, F., Qin, T., Chen, E., & Liu, T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.07233 · Cited by 427
Chen, Y., Li, J., Xiao, H., Jin, X., Yan, S., & Feng, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.01629 · Cited by 397
PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Search
Xu, Y., Xie, L., Zhang, X., Chen, X., Qi, G.-J., Tian, Q. C., & Xiong, H. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.05737 · Cited by 376
Self-citationAccelerating Neural Architecture Search using Performance PredictionPreprint
Baker, B., Gupta, O., Raskar, R., & Naik, N. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.10823 · Cited by 357
Searching for Efficient Multi-Scale Architectures for Dense Image Prediction
Chen, L.-C., Collins, M. D., Zhu, Y., Papandreou, G., Zoph, B., Schroff, F., Adam, H., & Shlens, J. (2018) · arXiv · doi:10.48550/arxiv.1809.04184 · Cited by 347
FairNAS: Rethinking Evaluation Fairness of Weight Sharing Neural Architecture Search
Chu, X., Zhang, B., & Xu, R. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01202 · Cited by 322
Proceedings of the Genetic and Evolutionary Computation Conference
(2019) · doi:10.1145/3321707 · Cited by 319
ConvNet Architecture Search for Spatiotemporal Feature Learning
Tran, D., Ray, J., Shou, Z., Chang, S., & Paluri, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.05038 · Cited by 315
Efficient multi-objective neural architecture search via Lamarckian evolution
Elsken, T., Metzen, J. H., & Hutter, F. (2018) · arXiv · doi:10.48550/arxiv.1804.09081 · Cited by 311
NAS-bench-101: Towards reproducible neural architecture search
Ying, C., Klein, A., Real, E., Christiansen, E., Murphy, K., & Hutter, F. (2019) · arXiv · doi:10.48550/arxiv.1902.09635 · Cited by 297
Gastaldi, X. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.07485 · Cited by 276
ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware
Cai, H., Zhu, L., & Han, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.00332 · Cited by 267
Neural Architecture Search with Bayesian Optimisation and Optimal Transport
Kandasamy, K., Neiswanger, W., Schneider, J., Póczos, B., & Xing, E. P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.07191 · Cited by 259
Evaluating the Search Phase of Neural Architecture SearchPreprint
Yu, K., Sciuto, C., Jaggi, M., Musat, C., & Salzmann, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.08142 · Cited by 257
Understanding and Robustifying Differentiable Architecture Search
Zela, A., Elsken, T., Saikia, T., Marrakchi, Y., Brox, T., & Hutter, F. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.09656 · Cited by 232
Graph HyperNetworks for Neural Architecture SearchPreprint
Zhang, C., Ren, M., & Urtasun, R. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.05749 · Cited by 230
Hierarchical optimization: An introduction
Anandalingam, G., & Friesz, T. L. (1992) · Annals of Operations Research · doi:10.1007/bf02098169 · Cited by 229
Hierarchical Neural Architecture Search for Deep Stereo Matching
Cheng, X., Zhong, Y., Harandi, M. T., Dai, Y., Chang, X., Drummond, T. W., Li, H., & Ge, Z. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.13501 · Cited by 222
Self-citationRandom Search and Reproducibility for Neural Architecture Search
Li, L., & Talwalkar, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.07638 · Cited by 209
A Survey on Neural Architecture Search
Wistuba, M., Rawat, A., & Pedapati, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.01392 · Cited by 198
DeepArchitect: Automatically Designing and Training Deep ArchitecturesPreprint
Negrinho, R. M. P., & Gordon, G. J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.08792 · Cited by 197
Hierarchical Representations for Efficient Architecture Search
Liu, H., Simonyan, K., Vinyals, O., Fernando, C., & Kavukcuoglu, K. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.00436 · Cited by 194
FBNetV3: Joint Architecture-Recipe Search using Predictor Pretraining
Dai, X., Wan, A., Zhang, P., Wu, B., He, Z., Wei, Z., Chen, K., Tian, Y., Yu, M., Vajda, P., et al. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01601 · Cited by 179
Convolutional Neural FabricsPreprint
Saxena, S., & Verbeek, J. J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.02492 · Cited by 178
Simple And Efficient Architecture Search for Convolutional Neural Networks
Elsken, T., Metzen, J. H., & Hutter, F. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.04528 · Cited by 170
When NAS Meets Robustness: In Search of Robust Architectures Against Adversarial Attacks
Guo, M., Yang, Y., Xu, R., Liu, Z., & Lin, D. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00071 · Cited by 169
Adversarial AutoAugmentPreprint
Zhang, X., Wang, Q., Zhang, J., & Zhong, Z. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.11188 · Cited by 167
Densely Connected Search Space for More Flexible Neural Architecture Search
Fang, J., Sun, Y., Zhang, Q., Li, Y., Liu, W., & Wang, X. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01064 · Cited by 166
Self-citationNATS-Bench: Benchmarking NAS Algorithms for Architecture Topology and Size
Dong, X., Liu, L., Musial, K., & Gabrys, B. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3054824 · Cited by 165
N2N Learning: Network to Network Compression via Policy Gradient Reinforcement LearningPreprint
Ashok, A., Rhinehart, N., Beainy, F., & Kitani, K. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.06030 · Cited by 158
NAS evaluation is frustratingly hardPreprint
Yang, A., Esperança, P. M., & Carlucci, F. M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.12522 · Cited by 154
Neural Machine Translation With GRU-Gated Attention Model
Zhang, B., Xiong, D., Xie, J., & Su, J. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2957276 · Cited by 135
NAS-Bench-1Shot1: Benchmarking and Dissecting One-shot Neural Architecture SearchPreprint
Zela, A., Siems, J., & Hutter, F. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.10422 · Cited by 112
AssembleNet: Searching for Multi-Stream Neural Connectivity in Video ArchitecturesPreprint
Ryoo, M. S., Piergiovanni, A., Tan, M., & Angelova, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.13209 · Cited by 108
Real-Time Object Detection With Reduced Region Proposal Network via Multi-Feature Concatenation
Shih, K.-H., Chiu, C., Lin, J.-A., & Bu, Y.-Y. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2929059 · Cited by 100
Deep Attentive Video Summarization With Distribution Consistency Learning
Ji, Z., Zhao, Y., Pang, Y., Li, X., & Han, J. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2991083 · Cited by 100
One-Shot Neural Architecture Search: Maximising Diversity to Overcome Catastrophic Forgetting
Zhang, M., Li, H., Pan, S., Chang, X., Zhou, C. D., Ge, Z., & Su, S. W. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3035351 · Cited by 93
Self-citationProbabilistic Neural Architecture SearchPreprint
Casale, F. P., Gordon, J. A. R., & Fusi, N. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.05116 · Cited by 90
Differentiable Neural Network Architecture Search
Shin, R., Packer, C., & Song, D. X. (2018) · International Conference on Learning Representations · Cited by 84
Overcoming Multi-Model Forgetting in One-Shot NAS With Diversity Maximization
Zhang, M., Li, H., Pan, S., Chang, X., & Su, S. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00783 · Cited by 83
Self-citationEvolutionary Compression of Deep Neural Networks for Biomedical Image Segmentation
Zhou, Y., Yen, G. G., & Zhang, Y. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2933879 · Cited by 79
DetNAS: Neural Architecture Search on Object Detection.Preprint
Chen, Y., Yang, T., Zhang, X., Meng, G., Pan, C., & Sun, J. (2019) · arXiv (Cornell University) · Cited by 77
Improving One-Shot NAS by Suppressing the Posterior Fading
Li, X., Chen, L., Li, C., Sun, M., Wu, W., Yan, J., & Ouyang, W. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01385 · Cited by 76
Understanding Architectures Learnt by Cell-based Neural Architecture SearchPreprint
Shu, Y., Wang, W., & Cai, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.09569 · Cited by 74
FNA++: Fast Network Adaptation via Parameter Remapping and Architecture Search
Fang, J., Sun, Y., Zhang, Q., Peng, K., Li, Y., Liu, W., & Wang, X. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3044416 · Cited by 73
Self-citationImproving MMD-GAN Training with Repulsive Loss FunctionPreprint
Wang, W., Sun, Y., & Halgamuge, S. K. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.09916 · Cited by 72
Zhang, D., Han, J., Zhao, L., & Zhao, T. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2969483 · Cited by 65
AutoSpeech: Neural Architecture Search for Speaker Recognition
Ding, S., Chen, T., Gong, X., Zha, W., & Wang, Z. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-1258 · Cited by 63
Continual and Multi-Task Architecture Search
Pasunuru, R., & Bansal, M. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1185 · Cited by 53
Efficient Neural Architecture Transformation Searchin Channel-Level for Object DetectionPreprint
Peng, J., Sun, M., Zhang, Z., Tan, T., & Yan, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.02293 · Cited by 52
FBNetV3: Joint Architecture-Recipe Search using Neural Acquisition FunctionPreprint
Dai, X., Wan, A., Zhang, P., Wu, B., He, Z., Wei, Z., Chen, K., Tian, Y., Yu, M., Vajda, P., et al. (2020) · arXiv (Cornell University) · Cited by 47
Learnable Embedding Space for Efficient Neural Architecture CompressionPreprint
Cao, S., Wang, X., & Kitani, K. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.00383 · Cited by 34
Self-citationConnectivity Learning in Multi-Branch NetworksPreprint
Ahmed, K., & Torresani, L. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.09582 · Cited by 31
MASK-RL: Multiagent Video Object Segmentation Framework Through Reinforcement Learning
Vecchio, G., Palazzo, S., Giordano, D., Rundo, F., & Spampinato, C. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2963282 · Cited by 31
Computation Reallocation for Object DetectionPreprint
Liang, F., Chen, L., Guo, R., Sun, M., Wu, W., Yan, J., & Ouyang, W. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.11234 · Cited by 30
Deep Pyramidal Residual Networks with Separated Stochastic DepthPreprint
Yamada, Y., Iwamura, M., & Kise, K. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.01230 · Cited by 23
AutoHAS: Differentiable Hyper-parameter and Architecture Search.Preprint
Dong, X., Tan, M., Yu, A. W., Peng, D., Gabryś, B., & Le, Q. V. (2020) · arXiv (Cornell University) · Cited by 18
Cited by
A Survey of Deep Active Learning
Ren, P., Xiao, Y., Chang, X., Huang, P.-Y., Li, Z., Gupta, B. B., Chen, X., & Wang, X. (2022) · ACM Computing Surveys · doi:10.1145/3472291 · Cited by 1,169
Self-citationSustainable AI: Environmental Implications, Challenges and Opportunities
Wu, C.-J., Raghavendra, R., Gupta, U., Acun, B., Ardalani, N., Maeng, K., Chang, G., Behram, F. A., Huang, J., Bai, C., et al. (2021) · arXiv · doi:10.48550/arxiv.2111.00364 · Cited by 405
“Utilizing much more sample-efficient NAS and HPO methods (Turner et al., 2021; Ren et al., 2021) can translate directly into carbon footprint improvement.”
MethodStructured Pruning for Deep Convolutional Neural Networks: A Survey
He, Y., & Xiao, L. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3334614 · Cited by 363
Sapkota, R., Flores-Calero, M., Qureshi, R., Badgujar, C., Nepal, U., Poulose, A., Zeno, P., Vaddevolu, U. B. P., Khan, S., Shoman, M., et al. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11253-3 · Cited by 308
Lightweight Deep Learning for Resource-Constrained Environments: A Survey
Liu, H.-I., Galindo, M., Xie, H., Wong, L.-K., Shuai, H.-H., Li, Y.-H., & Cheng, W.-H. (2024) · ACM Computing Surveys · doi:10.1145/3657282 · Cited by 259
Metadata
- Authors
- Ren, P., Xiao, Y., Chang, X., Huang, P.-Y., Li, Z., Chen, X., & Wang, X.
- Venue
- ACM Computing Surveys
- Published
- 2021-05-24
- Volume
- 54
- Issue
- 4
- Pages
- 1-34
- ISSN
- 0360-0300 (ISSN-L)
- 1557-7341
- OpenAlex
- W3164008977
- Cited by
- 590
- References
- 101
- FWCI
- 32.80
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.