The DOI matches its Crossref record; 60 of the 60 references it cites were found in a registry or an index.
Evolving Deep Convolutional Neural Networks for Image Classification
Open accessAbstract
Evolutionary paradigms have been successfully applied to neural network designs for two decades. Unfortunately, these methods cannot scale well to the modern deep neural networks due to the complicated architectures and large quantities of connection weights. In this paper, we propose a new method using genetic algorithms for evolving the architectures and connection weight initialization values of a deep convolutional neural network to address image classification problems. In the proposed algorithm, an efficient variable-length gene encoding strategy is designed to represent the different building blocks and the potentially optimal depth in convolutional neural networks. In addition, a new representation scheme is developed for effectively initializing connection weights of deep convolutional neural networks, which is expected to avoid networks getting stuck into local minimum that is typically a major issue in the backward gradient-based optimization. Furthermore, a novel fitness evaluation method is proposed to speed up the heuristic search with substantially less computational resource. The proposed algorithm is examined and compared with 22 existing algorithms on nine widely used image classification tasks, including the state-of-the-art methods. The experimental results demonstrate the remarkable superiority of the proposed algorithm over the state-of-the-art designs in terms of classification error rate and the number of parameters (weights).
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
A fast and elitist multiobjective genetic algorithm: NSGA-II
Deb, K., Pratap, A., Agarwal, S., & Meyarivan, T. (2002) · IEEE Transactions on Evolutionary Computation · doi:10.1109/4235.996017 · Cited by 50,053
Densely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818
Hodgkin, A. L., & Huxley, A. F. (1952) · The Journal of Physiology · doi:10.1113/jphysiol.1952.sp004764 · Cited by 23,191
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
Multi-Objective Optimization Using Evolutionary Algorithms
Deb, K., & Kalyanmoy, D. (2001) · Andalas University Repository (Andalas University) · Cited by 17,328
Visualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839
(2010) · Cited by 14,134
Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex
Hubel, D. H., & Wiesel, T. N. (1962) · The Journal of Physiology · doi:10.1113/jphysiol.1962.sp006837 · Cited by 13,680
Representation Learning: A Review and New Perspectives
Bengio, Y., Courville, A., & Vincent, P. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.50 · Cited by 13,001
Backpropagation Applied to Handwritten Zip Code Recognition
LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., & Jackel, L. D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.4.541 · Cited by 11,891
An Introduction to Genetic Algorithms
Mitchell, M. (1996) · doi:10.7551/mitpress/3927.001.0001 · Cited by 11,340
Caffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654889 · Cited by 11,077
TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., et al. (2016) · arXiv · doi:10.48550/arxiv.1603.04467 · Cited by 9,408
(2012) · Cited by 8,009
SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
Iandola, F. N., Han, S., Moskewicz, M. W., Ashraf, K., Dally, W. J., & Keutzer, K. (2016) · arXiv · doi:10.48550/arxiv.1602.07360 · Cited by 6,201
Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
Xiao, H., Rasul, K., & Vollgraf, R. (2017) · arXiv · doi:10.48550/arxiv.1708.07747 · Cited by 6,108
How transferable are features in deep neural networks?
Yosinski, J., Clune, J., Bengio, Y., & Lipson, H. (2014) · arXiv · doi:10.48550/arxiv.1411.1792 · Cited by 6,061
Shin, H.-C., Roth, H. R., Gao, M., Lu, L., Xu, Z., Nogues, I., Yao, J., Mollura, D., & Summers, R. M. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2528162 · Cited by 5,759
Bäck, T. (1996) · Oxford University Press eBooks · doi:10.1093/oso/9780195099713.001.0001 · Cited by 5,113
Neural Architecture Search with Reinforcement Learning
Zoph, B., & Le, Q. V. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.01578 · Cited by 3,974
(2011) · Cited by 3,939
Evolving Neural Networks through Augmenting Topologies
Stanley, K. O., & Miikkulainen, R. (2002) · Evolutionary Computation · doi:10.1162/106365602320169811 · Cited by 3,258
Learning and Transferring Mid-level Image Representations Using Convolutional Neural Networks
Oquab, M., Bottou, L., Laptev, I., & Sivic, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.222 · Cited by 3,150 · cited 1×
“In addition, CNNs have also improved the development of other machine learning approaches for object detection tasks, such as the transfer learning based on CNNs [4]–[6].”
BackgroundEvolving artificial neural networks
Yao, X. (1999) · Proceedings of the IEEE · doi:10.1109/5.784219 · Cited by 2,981
Simulated Binary Crossover for Continuous Search Space.
Deb, K., & Agrawal, R. B. (1995) · Complex Systems · Cited by 2,976
A Practical Guide to Training Restricted Boltzmann Machines
Hinton, G. E. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-35289-8_32 · Cited by 2,852
Sequential Model-Based Optimization for General Algorithm Configuration
Hutter, F., Hoos, H. H., & Leyton-Brown, K. (2011) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-25566-3_40 · Cited by 2,469
MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems
Chen, T., Li, M., Li, Y., Lin, M., Wang, N., Wang, M., Xiao, T., Xu, B., Zhang, C., & Zhang, Z. (2015) · arXiv · doi:10.48550/arxiv.1512.01274 · Cited by 1,819
Invariant Scattering Convolution Networks
Bruna, J., & Mallat, S. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.230 · Cited by 1,633
Contractive Auto-Encoders: Explicit Invariance During Feature Extraction
Rifai, S., Vincent, P., Muller, X., Glorot, X., & Bengio, Y. (2011) · International Conference on Machine Learning · Cited by 1,600
PCANet: A Simple Deep Learning Baseline for Image Classification?
Chan, T.-H., Jia, K., Gao, S., Lu, J., Zeng, Z., & Ma, Y. (2015) · IEEE Transactions on Image Processing · doi:10.1109/tip.2015.2475625 · Cited by 1,362
Auto-association by multilayer perceptrons and singular value decomposition
Bourlard, H., & Kamp, Y. (1988) · Biological Cybernetics · doi:10.1007/bf00332918 · Cited by 1,345
(1993) · Cited by 1,173
Multi-column deep neural network for traffic sign classification
Cireşan, D., Meier, U., Masci, J., & Schmidhuber, J. (2012) · Neural Networks · doi:10.1016/j.neunet.2012.02.023 · Cited by 1,057
Blumer, A., Ehrenfeucht, A., Haussler, D., & Warmuth, M. K. (1987) · Information Processing Letters · doi:10.1016/0020-0190(87)90114-1 · Cited by 1,054
An empirical evaluation of deep architectures on problems with many factors of variation
Larochelle, H., Erhan, D., Courville, A., Bergstra, J., & Bengio, Y. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273556 · Cited by 992
Learning Representations by Back-Propagating Errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (2002) · Cognitive Modeling · doi:10.7551/mitpress/1888.003.0013 · Cited by 759
Xie, L., & Yuille, A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.154 · Cited by 740
Algorithm 247: Radical-inverse quasi-random point sequence
Halton, J. H. (1964) · Communications of the ACM · doi:10.1145/355588.365104 · Cited by 601
On bayesian methods for seeking the extremum
Močkus, J. (1975) · Lecture Notes in Computer Science · doi:10.1007/3-540-07165-2_55 · Cited by 499
Large-Scale Evolution of Image Classifiers
Real, E., Moore, S., Selle, A. P., Saxena, S., Suematsu, Y. I. L., Tan, J., Le, Q. V., & Kurakin, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.01041 · Cited by 466
On Bayesian Methods for Seeking the Extremum
Močkus, J. (1975) · Optimization Techniques IFIP Technical Conference · doi:10.1007/978-3-662-38527-2_55 · Cited by 402
Evolutionary Computation for Modeling and Optimization
(2006) · Springer eBooks · doi:10.1007/0-387-31909-3 · Cited by 401
Shallow vs. Deep Sum-Product Networks
Delalleau, O., & Bengio, Y. (2011) · neural information processing systems · Cited by 386
Xie, L., & Yuille, A. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · Cited by 377
Designing Neural Network Architectures using Reinforcement Learning
Baker, B., Gupta, O., Naik, N., & Raskar, R. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.02167 · Cited by 372
Toward automatic phenotyping of developing embryos from videos
Ning, F., Delhomme, D., LeCun, Y., Piano, F., Bottou, L., & Barbano, P. E. (2005) · IEEE Transactions on Image Processing · doi:10.1109/tip.2005.852470 · Cited by 261
Beck, J., & Fiala, T. (1981) · Discrete Applied Mathematics · doi:10.1016/0166-218x(81)90022-6 · Cited by 225
On the Expressive Power of Deep Architectures
Bengio, Y., & Delalleau, O. (2011) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-24477-3_1 · Cited by 186
On the Expressive Power of Deep Architectures
Bengio, Y., & Delalleau, O. (2011) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-24412-4_3 · Cited by 182
Convolutional Neural FabricsPreprint
Saxena, S., & Verbeek, J. J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.02492 · Cited by 178
Learning Invariant Representations with Local TransformationsPreprint
Sohn, K., & Lee, H. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1206.6418 · Cited by 147
Learning and Selecting Features Jointly with Point-wise Gated Boltzmann Machines
Sohn, K., Zhou, G., Lee, C., & Lee, H. (2013) · Cited by 93
Cited by
Automatically Designing CNN Architectures Using the Genetic Algorithm for Image Classification
Sun, Y., Xue, B., Zhang, M., Yen, G. G., & Lv, J. (2020) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2020.2983860 · Cited by 870
Self-citationA Survey on Evolutionary Neural Architecture Search
Liu, Y., Sun, Y., Xue, B., Zhang, M., Yen, G. G., & Tan, K. C. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3100554 · Cited by 644
Self-citationParticle swarm optimization of deep neural networks architectures for image classification
Fernandes Junior, F. E., & Yen, G. G. (2019) · Swarm and Evolutionary Computation · doi:10.1016/j.swevo.2019.05.010 · Cited by 393
Self-citationCompletely Automated CNN Architecture Design Based on Blocks
Sun, Y., Xue, B., Zhang, M., & Yen, G. G. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2919608 · Cited by 338
Self-citationEvolutionary Transfer Optimization - A New Frontier in Evolutionary Computation Research
Tan, K. C., Feng, L., & Jiang, M. (2021) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2020.3039066 · Cited by 315
Metadata
- Authors
- Sun, Y., Xue, B., Zhang, M., & Yen, G. G.
- Venue
- IEEE Transactions on Evolutionary Computation
- Published
- 2020-04
- Volume
- 24
- Issue
- 2
- Pages
- 394-407
- ISSN
- 1089-778X (ISSN-L)
- 1941-0026
- OpenAlex
- W2963946985
- Cited by
- 775
- References
- 60
- FWCI
- 45.13
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.