Cited by 328Evolutionary Algorithms and ApplicationsAdvanced Neural Network ApplicationsMachine Learning and Data Classification

Regularized Evolution for Image Classifier Architecture Search

Open access
Real, Esteban, Aggarwal, Alok, Huang, Yanping, Le, Quoc V.·2019·Proceedings of the AAAI Conference on Artificial Intelligence

Abstract

The effort devoted to hand-crafting neural network image classifiers has motivated the use of architecture search to discover them automatically. Although evolutionary algorithms have been repeatedly applied to neural network topologies, the image classifiers thus discovered have remained inferior to human-crafted ones. Here, we evolve an image classifier— AmoebaNet-A—that surpasses hand-designs for the first time. To do this, we modify the tournament selection evolutionary algorithm by introducing an age property to favor the younger genotypes. Matching size, AmoebaNet-A has comparable accuracy to current state-of-the-art ImageNet models discovered with more complex architecture-search methods. Scaled to larger size, AmoebaNet-A sets a new state-of-theart 83.9% top-1 / 96.6% top-5 ImageNet accuracy. In a controlled comparison against a well known reinforcement learning algorithm, we give evidence that evolution can obtain results faster with the same hardware, especially at the earlier stages of the search. This is relevant when fewer compute resources are available. Evolution is, thus, a simple method to effectively discover high-quality architectures.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1609/aaai.v33i01.33014780

Cites

7 of 7 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 40

  1. Angeline, P. J.; Saunders, G. M.; and Pollack, J. B. 1994. An evolutionary algorithm that constructs recurrent neural networks. IEEE transactions on Neural Networks. 2 https://colab.research.google.com/github/google-research/ google-research/blob/master/evolution/regularized evolution algorithm/regularized evolution.ipynb 3 https://tfhub.dev/google/imagenet/amoebanet a n18 f448/ classification/1
  2. Baker, B.; Gupta, O.; Naik, N.; and Raskar, R. 2017a. Designing neural network architectures using reinforcement learning. In ICLR.
  3. Baker, B.; Gupta, O.; Raskar, R.; and Naik, N. 2017b. Ac- celerating neural architecture search using performance pre- diction. ICLR Workshop.
  4. Bergstra, J., and Bengio, Y. 2012. Random search for hyper- parameter optimization. JMLR.
  5. Brock, A.; Lim, T.; Ritchie, J. M.; and Weston, N. 2018. Smash: one-shot model architecture search through hyper- networks. In ICLR.
  6. Cai, H.; Chen, T.; Zhang, W.; Yu, Y.; and Wang, J. 2018. Efficient architecture search by network transformation. In AAAI.
  7. Chen, Y.; Li, J.; Xiao, H.; Jin, X.; Yan, S.; and Feng, J. 2017. Dual path networks. In NIPS.
  8. Ciregan, D.; Meier, U.; and Schmidhuber, J. 2012. Multi- column deep neural networks for image classification. In CVPR.
  9. Coleman, C.; Kang, D.; Narayanan, D.; Nardi, L.; Zhao, T.; Zhang, J.; Bailis, P.; Olukotun, K.; Re, C.; and Zaharia, M. 2018. Analysis of dawnbench, a time-to-accuracy machine learning performance benchmark. arXiv preprint arXiv:1806.01427. · arXiv
  10. Cortes, C.; Gonzalvo, X.; Kuznetsov, V.; Mohri, M.; and Yang, S. 2017. Adanet: Adaptive structural learning of arti- ficial neural networks. In ICML.
  11. Cubuk, E. D.; Zoph, B.; Mane, D.; Vasudevan, V.; and Le, Q. V. 2018. Autoaugment: Learning augmentation policies from data. arXiv.
  12. Deng, J.; Dong, W.; Socher, R.; Li, L.-J.; Li, K.; and Fei- Fei, L. 2009. Imagenet: A large-scale hierarchical image database. In CVPR.
  13. Domhan, T.; Springenberg, J. T.; and Hutter, F. 2017. Speed- ing up automatic hyperparameter optimization of deep neu- ral networks by extrapolation of learning curves. In IJCAI.
  14. Elsken, T.; Metzen, J.-H.; and Hutter, F. 2017. Simple and efficient architecture search for convolutional neural net- works. ICLR Workshop.
  15. Elsken, T.; Metzen, J. H.; and Hutter, F. 2018. Neural archi- tecture search: A survey. arXiv.
  16. Fahlman, S. E., and Lebiere, C. 1990. The cascade- correlation learning architecture. In NIPS.
  17. Feurer, M.; Klein, A.; Eggensperger, K.; Springenberg, J.; Blum, M.; and Hutter, F. 2015. Efficient and robust auto- mated machine learning. In NIPS.
  18. Hu, J.; Shen, L.; and Sun, G. 2018. Squeeze-and-excitation networks. CVPR.
  19. Huang, G.; Liu, Z.; Weinberger, K. Q.; and van der Maaten, L. 2017. Densely connected convolutional networks. In CVPR.
  20. Klein, A.; Falkner, S.; Springenberg, J. T.; and Hutter, F. 2017. Learning curve prediction with bayesian neural net- works. ICLR.
  21. Krizhevsky, A., and Hinton, G. 2009. Learning multiple layers of features from tiny images. Master's thesis, Dept. of Computer Science, U. of Toronto.
  22. Liu, C.; Zoph, B.; Shlens, J.; Hua, W.; Li, L.-J.; Fei-Fei, L.; Yuille, A.; Huang, J.; and Murphy, K. 2018a. Progressive neural architecture search. ECCV.
  23. Liu, H.; Simonyan, K.; Vinyals, O.; Fernando, C.; and Kavukcuoglu, K. 2018b. Hierarchical representations for efficient architecture search. In ICLR.
  24. Mendoza, H.; Klein, A.; Feurer, M.; Springenberg, J. T.; and Hutter, F. 2016. Towards automatically-tuned neural net- works. In Workshop on Automatic Machine Learning.
  25. Miikkulainen, R.; Liang, J.; Meyerson, E.; Rawal, A.; Fink, D.; Francon, O.; Raju, B.; Navruzyan, A.; Duffy, N.; and Hodjat, B. 2017. Evolving deep neural networks. arXiv.
  26. Negrinho, R., and Gordon, G. 2017. Deeparchitect: Auto- matically designing and training deep architectures. arXiv.
  27. Pham, H.; Guan, M. Y.; Zoph, B.; Le, Q. V.; and Dean, J. 2018. Faster discovery of neural architectures by searching for paths in a large model. ICLR Workshop.
  28. Real, E.; Moore, S.; Selle, A.; Saxena, S.; Suematsu, Y. L.; Le, Q.; and Kurakin, A. 2017. Large-scale evolution of image classifiers. In ICML.
  29. Salimans, T.; Ho, J.; Chen, X.; and Sutskever, I. 2017. Evo- lution strategies as a scalable alternative to reinforcement learning. arXiv.
  30. Saxena, S., and Verbeek, J. 2016. Convolutional neural fab- rics. In NIPS.
  31. Srivastava, N.; Hinton, G.; Krizhevsky, A.; Sutskever, I.; and Salakhutdinov, R. 2014. Dropout: A simple way to prevent neural networks from overfitting. JMLR.
  32. Szegedy, C.; Liu, W.; Jia, Y.; Sermanet, P.; Reed, S.; Anguelov, D.; Erhan, D.; Vanhoucke, V.; and Rabinovich, A. 2015. Going deeper with convolutions. In CVPR.
  33. Wan, L.; Zeiler, M.; Zhang, S.; Le Cun, Y.; and Fergus, R. 2013. Regularization of neural networks using dropconnect. In ICML.
  34. Xie, L., and Yuille, A. 2017. Genetic CNN. In ICCV.
  35. Xie, S.; Girshick, R.; Dollár, P.; Tu, Z.; and He, K. 2017. Ag- gregated residual transformations for deep neural networks. In CVPR.
  36. Yao, X. 1999. Evolving artificial neural networks. IEEE.
  37. Zhang, X.; Li, Z.; Loy, C. C.; and Lin, D. 2017. Polynet: A pursuit of structural diversity in very deep networks. In CVPR.
  38. Zhong, Z.; Yan, J.; and Liu, C.-L. 2018. Practical network blocks design with q-learning. In AAAI.
  39. Zoph, B., and Le, Q. V. 2016. Neural architecture search with reinforcement learning. In ICLR.
  40. Zoph, B.; Vasudevan, V.; Shlens, J.; and Le, Q. V. 2018. Learning transferable architectures for scalable im- age recognition. In CVPR.

Cited by

Sorted by citations

Authors
Real, E., Aggarwal, A., Huang, Y., & Le, Q. V.
Affiliations
Google (United States) (US)
Venue
Proceedings of the AAAI Conference on Artificial Intelligence
Published
2019-07-17
Volume
33
Issue
01
Pages
4780-4789
ISSN
  • 2159-5399 (ISSN-L)
  • 2374-3468
OpenAlex
W2785430118
Cited by
328
References
7
Citations by year
2026: 3 · 2025: 21 · 2024: 31 · 2023: 36 · 2022: 46 · 2021: 50 · 2020: 65 · 2019: 59 · 2018: 15 · 2017: 2
FWCI
21.04
Type
conference paper
Language
en
Open access
Locations
  • Proceedings of the AAAI Conference on Artificial Intelligence (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
  • TIB Data Manager
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Evolutionary Algorithms and Applications
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
Keywords
neural architecture search, image classification, tournament selection, age-based selection, evolutionary algorithms, ImageNet, reinforcement learning comparison, computational efficiency

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review