The DOI matches its Crossref record; 44 of the 44 references it cites were found in a registry or an index.
Searching for a Robust Neural Architecture in Four GPU Hours
Open accessAbstract
Conventional neural architecture search (NAS) approaches are usually based on reinforcement learning or evolutionary strategy, which take more than 1000 GPU hours to find a good model on CIFAR-10. We propose an efficient NAS approach, which learns the searching approach by gradient descent. Our approach represents the search space as a directed acyclic graph (DAG). This DAG contains thousands of sub-graphs, each of which indicates a kind of neural architecture. To avoid traversing all the possibilities of the sub-graphs, we develop a differentiable sampler over the DAG. This sampler is learnable and optimized by the validation loss after training the sampled architecture. In this way, our approach can be trained in an end-to-end fashion by gradient descent, named Gradient-based search using Differentiable Architecture Sampler (GDAS). In experiments, we can finish one searching procedure in four GPU hours on CIFAR-10, and the discovered model obtains a test error of 2.82% with only 2.5M parameters, which is on par with the state-of-the-art.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Densely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
MobileNetV2: Inverted Residuals and Linear Bottlenecks
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L.-C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00474 · Cited by 28,755
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545
Cited by
AutoML: A survey of the state-of-the-art
He, X., Zhao, K., & Chu, X. (2021) · Knowledge-Based Systems · doi:10.1016/j.knosys.2020.106622 · Cited by 1,691
Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Acceleration
He, Y., Liu, P., Wang, Z., Hu, Z., & Yang, Y. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00447 · Cited by 1,217
Self-citationContrastive Adaptation Network for Unsupervised Domain Adaptation
Kang, G., Jiang, L., Yang, Y., & Hauptmann, A. G. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00503 · Cited by 941
Self-citationSingle Path One-Shot Neural Architecture Search with Uniform Sampling
Guo, Z., Zhang, X., Mu, H., Heng, W., Liu, Z., Wei, Y., & Sun, J. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58517-4_32 · Cited by 813
Invariance Matters: Exemplar Memory for Domain Adaptive Person Re-Identification
Zhong, Z., Zheng, L., Luo, Z., Li, S., & Yang, Y. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00069 · Cited by 685
Self-citationProgressive Differentiable Architecture Search: Bridging the Depth Gap Between Search and Evaluation
Chen, X., Xie, L., Wu, J., & Tian, Q. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00138 · Cited by 666
Searching Central Difference Convolutional Networks for Face Anti-Spoofing
Yu, Z., Zhao, C., Wang, Z., Qin, Y., Su, Z., Li, X., Zhou, F., & Zhao, G. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00534 · Cited by 636
Artificial Intelligence for Remote Sensing Data Analysis: A review of challenges and opportunities
Zhang, L., & Zhang, L. (2022) · IEEE Geoscience and Remote Sensing Magazine · doi:10.1109/mgrs.2022.3145854 · Cited by 604
Learning Generalisable Omni-Scale Representations for Person Re-Identification
Zhou, K., Yang, Y., Cavallaro, A., & Xiang, T. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3069237 · Cited by 342
Self-citationFairNAS: Rethinking Evaluation Fairness of Weight Sharing Neural Architecture Search
Chu, X., Zhang, B., & Xu, R. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01202 · Cited by 322
Fair DARTS: Eliminating Unfair Advantages in Differentiable Architecture Search
Chu, X., Zhou, T., Zhang, B., & Li, J. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58555-6_28 · Cited by 309
Auto-ReID: Searching for a Part-Aware ConvNet for Person Re-Identification
Quan, R., Dong, X., Wu, Y., Zhu, L., & Yang, Y. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00385 · Cited by 285
Self-citationCARS: Continuous Evolution for Efficient Neural Architecture Search
Yang, Z., Wang, Y., Chen, X., Shi, B., Xu, C., Xu, C., Tian, Q., & Xu, C. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00190 · Cited by 273
Learning Filter Pruning Criteria for Deep Convolutional Neural Networks Acceleration
He, Y., Ding, Y., Liu, P., Zhu, L., Zhang, H., & Yang, Y. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00208 · Cited by 262
Self-citationBANANAS: Bayesian Optimization with Neural Architectures for Neural Architecture Search
White, C., Neiswanger, W., & Savani, Y. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i12.17233 · Cited by 261
Stabilizing Differentiable Architecture Search via Perturbation-based Regularization
Chen, X., & Hsieh, C.-J. (2020) · arXiv · doi:10.48550/arxiv.2002.05283 · Cited by 126
Neural Architecture Search for Transformers: A Survey
Chitty-Venkata, K. T., Emani, M., Vishwanath, V., & Somani, A. K. (2022) · IEEE Access · doi:10.1109/access.2022.3212767 · Cited by 118
Overcoming Multi-Model Forgetting in One-Shot NAS With Diversity Maximization
Zhang, M., Li, H., Pan, S., Chang, X., & Su, S. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00783 · Cited by 83
Prototype Propagation Networks (PPN) for Weakly-supervised Few-shot Learning on Category Graph
Liu, L., Zhou, T., Long, G., Jiang, J., Yao, L., & Zhang, C. (2019) · Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/418 · Cited by 65
AutoAugHAR: Automated Data Augmentation for Sensor-based Human Activity Recognition
Zhou, Y., Zhao, H., Huang, Y., Röddiger, T., Kurnaz, M., Riedel, T., & Beigl, M. (2024) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3659589 · Cited by 25
Revisiting Training-free NAS Metrics: An Efficient Training-based Method
Yang, T., Yang, L., Jin, X., & Chen, C. (2023) · 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv56688.2023.00473 · Cited by 9
NASformer: Neural architecture search for vision transformer
Ni, B., Meng, G., Xiang, S., & Pan, C. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-02375-0_4 · Cited by 9
Metadata
- Authors
- Dong, X., & Yang, Y.
- Venue
- 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2019-06
- Pages
- 1761-1770
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2960010704
- Cited by
- 640
- References
- 44
- FWCI
- 32.81
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.