The DOI matches its Crossref record; 66 of the 66 references it cites were found in a registry or an index.
A Framework for Designing the Architectures of Deep Convolutional Neural Networks
Open accessAbstract
Recent advances in Convolutional Neural Networks (CNNs) have obtained promising results in difficult deep learning tasks. However, the success of a CNN depends on finding an architecture to fit a given problem. A hand-crafted architecture is a challenging, time-consuming process that requires expert knowledge and effort, due to a large number of architectural design choices. In this article, we present an efficient framework that automatically designs a high-performing CNN architecture for a given problem. In this framework, we introduce a new optimization objective function that combines the error rate and the information learnt by a set of feature maps using deconvolutional networks (deconvnet). The new objective function allows the hyperparameters of the CNN architecture to be optimized in a way that enhances the performance by guiding the CNN through better visualization of learnt features via deconvnet. The actual optimization of the objective function is carried out via the Nelder-Mead Method (NMM). Further, our new objective function results in much faster convergence towards a better architecture. The proposed framework has the ability to explore a CNN architecture’s numerous design choices in an efficient way and also allows effective, distributed execution and synchronization via web services. Empirically, we demonstrate that the CNN architecture designed with our approach outperforms several existing approaches in terms of its error rate. Our results are also competitive with state-of-the-art results on the MNIST dataset and perform reasonably against the state-of-the-art results on CIFAR-10 and CIFAR-100 datasets. Our approach has a significant role in increasing the depth, reducing the size of strides, and constraining some convolutional layers not followed by pooling layers in order to find a CNN architecture that produces a high recognition performance.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
(2014) · Cited by 33,419
A Simplex Method for Function Minimization
Nelder, J. A., & Mead, R. (1965) · The Computer Journal · doi:10.1093/comjnl/7.4.308 · Cited by 28,905
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,395
MAFFT: a novel method for rapid multiple sequence alignment based on fast Fourier transform
Katoh, K. (2002) · Nucleic Acids Research · doi:10.1093/nar/gkf436 · Cited by 17,027
Visualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839
Convolutional Neural Networks for Sentence Classification
Kim, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1181 · Cited by 14,831
Dermatologist-level classification of skin cancer with deep neural networks
Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017) · Nature · doi:10.1038/nature21056 · Cited by 14,362
(2010) · Cited by 14,134
(2010) · International Conference on Machine Learning · Cited by 13,995
A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection
Kohavi, R. (1995) · Zenodo · doi:10.5281/zenodo.19712698 · Cited by 10,909
Learning Deep Architectures for AI
Bengio, Y. (2009) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000006 · Cited by 9,239
Speech recognition with deep recurrent neural networks
Graves, A., Mohamed, A.-R., & Hinton, G. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6638947 · Cited by 8,893
(2012) · Cited by 8,009
Practical Bayesian Optimization of Machine Learning Algorithms
Snoek, J., Larochelle, H., & Adams, R. P. (2012) · arXiv · doi:10.48550/arxiv.1206.2944 · Cited by 6,691
Recent advances in convolutional neural networks
Gu, J., Wang, Z., Kuen, J., Ma, L., Shahroudy, A., Shuai, B., Liu, T., Wang, X., Wang, G., Cai, J., et al. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2017.10.013 · Cited by 6,576
Receptive fields and functional architecture of monkey striate cortex
Hubel, D. H., & Wiesel, T. N. (1968) · The Journal of Physiology · doi:10.1113/jphysiol.1968.sp008455 · Cited by 6,568
Architectural styles and the design of network-based software architectures
Fielding, R. T., & Taylor, R. N. (2000) · Cited by 5,094
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1512.03385 · Cited by 4,196
Neural Architecture Search with Reinforcement Learning
Zoph, B., & Le, Q. V. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.01578 · Cited by 3,974
(2011) · Cited by 3,939
A Convolutional Neural Network for Modelling Sentences
Kalchbrenner, N., Grefenstette, E., & Blunsom, P. (2014) · Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.3115/v1/p14-1062 · Cited by 3,601
OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks
Sermanet, P., Eigen, D., Zhang, X., Mathieu, M., Fergus, R., & LeCun, Y. (2013) · arXiv · doi:10.48550/arxiv.1312.6229 · Cited by 2,680
Learning Hierarchical Features for Scene Labeling
Farabet, C., Couprie, C., Najman, L., & LeCun, Y. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.231 · Cited by 2,620
Sequential Model-Based Optimization for General Algorithm Configuration
Hutter, F., Hoos, H. H., & Leyton-Brown, K. (2011) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-25566-3_40 · Cited by 2,469
(2013) · Cited by 2,056
Convolutional neural networks at constrained time cost
He, K., & Sun, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299173 · Cited by 1,558
Understanding Neural Networks Through Deep Visualization
Yosinski, J., Clune, J., Nguyen, A. M., Fuchs, T. J., & Lipson, H. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.06579 · Cited by 1,358
Very Deep Convolutional Networks for Text Classification
Conneau, A., Schwenk, H., Barrault, L., & Lecun, Y. (2017) · Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers · doi:10.18653/v1/e17-1104 · Cited by 1,046
Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization
Li, L., Jamieson, K., DeSalvo, G., Rostamizadeh, A., & Talwalkar, A. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.06560 · Cited by 1,018
Overfitting in Neural Nets: Backpropagation, Conjugate Gradient, and Early Stopping
Caruana, R., Lawrence, S., & Giles, C. L. (2000) · Cited by 906
Deeplysupervised netsPreprint
Lee, C., Xie, S., Gallagher, P., Zhang, Z., & Tu, Z. (2015) · Cited by 849
Ahlgren, P., Jarneving, B., & Rousseau, R. (2003) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.10242 · Cited by 785
Miikkulainen, R., Liang, J., Meyerson, E., Rawal, A., Fink, D., Francon, O., Raju, B., Shahrzad, H., Navruzyan, A., Duffy, N., et al. (2019) · Artificial Intelligence in the Age of Neural Networks and Brain Computing · doi:10.1016/b978-0-12-815480-9.00015-3 · Cited by 760
Service-Oriented Architecture: A Field Guide to Integrating XML and Web Services
Erl, T. (2004) · Cited by 702
Domhan, T., Springenberg, J. T., & Hutter, F. (2015) · FreiDok plus (Universitätsbibliothek Freiburg) · Cited by 670
Improving Forecasting Accuracy of Annual Runoff Time Series Using ARIMA Based on EEMD Decomposition
Wang, W.-C., Chau, K.-W., Xu, D.-M., & Chen, X.-Y. (2015) · Water Resources Management · doi:10.1007/s11269-015-0962-6 · Cited by 563
Prototype and Feature Selection by Sampling and Random Mutation Hill Climbing Algorithms
Skalak, D. B. (1994) · Machine Learning Proceedings 1994 · doi:10.1016/b978-1-55860-335-6.50043-x · Cited by 486
Optimizing deep learning hyper-parameters through an evolutionary algorithm
Young, S. R., Rose, D. C., Karnowski, T. P., Lim, S.-H., & Patton, R. M. (2015) · Proceedings of the Workshop on Machine Learning in High-Performance Computing Environments · doi:10.1145/2834892.2834896 · Cited by 424
A Convolutional Neural Network for Modelling Sentences
Kalchbrenner, N., Grefenstette, E., & Blunsom, P. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1404.2188 · Cited by 407
A review of instance selection methods
Olvera-López, J. A., Carrasco-Ochoa, J. A., Martínez-Trinidad, J. F., & Kittler, J. (2010) · Artificial Intelligence Review · doi:10.1007/s10462-010-9165-y · Cited by 394
Taormina, R., & Chau, K.-W. (2015) · Journal of Hydrology · doi:10.1016/j.jhydrol.2015.08.022 · Cited by 329
Application of Deep Convolutional Neural Networks for Detecting Extreme Weather in Climate Datasets
Liu, Y., Racah, E., Prabhat, Correa, J., Khosrowshahi, A., Lavers, D. A., Kunkel, K. E., Wehner, M. F., & Collins, W. D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.01156 · Cited by 292
A hybrid model coupled with singular spectrum analysis for daily rainfall prediction
Chau, K. W., & Wu, C. L. (2010) · Journal of Hydroinformatics · doi:10.2166/hydro.2010.032 · Cited by 289
CMA-ES for Hyperparameter Optimization of Deep Neural Networks
Loshchilov, I., & Hutter, F. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1604.07269 · Cited by 223
Hyperparameter optimization with approximate gradient
Pedregosa, F. (2016) · arXiv · doi:10.48550/arxiv.1602.02355 · Cited by 217
Slice sampling covariance hyperparameters of latent Gaussian modelsPreprint
Murray, I., & Adams, R. P. (2010) · arXiv (Cornell University) · doi:10.48550/arxiv.1006.0868 · Cited by 169
Multilayer Ensemble Pruning via Novel Multi-sub-swarm Particle Swarm Optimization
Zhang, J., & Chau, K. (2020) · doi:10.3217/jucs-015-04-0840 · Cited by 164
An adaptive training algorithm for back propagation networks
Chan, L. W., & Fallside, F. (1987) · Computer Speech & Language · doi:10.1016/0885-2308(87)90009-x · Cited by 143
Scalable Gradient-Based Tuning of Continuous Regularization HyperparametersPreprint
Luketina, J., Berglund, M., Greff, K., & Raiko, T. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.06727 · Cited by 113
Adaptive Regularization in Neural Network Modeling
Larsen, J. O., Svarer, C., Andersen, L. N., & Hansen, L. K. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-35289-8_8 · Cited by 58
Constrained Bayesian Optimization and Applications
Gelbart, M. A. (2015) · Digital Access to Scholarship at Harvard (DASH) (Harvard University) · Cited by 34
Event detection on large social media using temporal analysis
Aldhaheri, A., & Lee, J. (2017) · IEEE Annual Computing and Communication Workshop and Conference (CCWC) · doi:10.1109/ccwc.2017.7868467 · Cited by 33
Adaptive Regularization in Neural Network Modeling
Larsen, J. O., Svarer, C., Andersen, L. N., & Hansen, L. K. (1998) · Lecture notes in computer science · doi:10.1007/3-540-49430-8_6 · Cited by 33
Automated Optimal Architecture of Deep Convolutional Neural Networks for Image Recognition
Albelwi, S. A., & Mahmood, A. (2016) · International Conference on Machine Learning and Applications · doi:10.1109/icmla.2016.0018 · Cited by 24
Self-citationSchaer, R., Müller, H., & Depeursinge, A. (2016) · Journal of Imaging · doi:10.3390/jimaging2020019 · Cited by 23
Learning the Structure of Deep Architectures Using L1 Regularization
Kulkarni, P., Zepeda, J., Jurie, F., Pérez, P., & Chevallier, L. (2015) · British Machine Vision Conference (BMVC) · doi:10.5244/c.29.23 · Cited by 16
On the Detection of Fake Certificates via Attribute Correlation
Gu, X., & Gu, X. (2015) · Entropy · doi:10.3390/e17063806 · Cited by 14
Dragomir, A., Post, A., Akay, Y. M., Jneid, H. M., Paniagua, D. E., Denktas, A. E., Bozkurt, B., & Akay, M. (2016) · Entropy · doi:10.3390/e18080281 · Cited by 14
Analysis of instance selection algorithms on large datasets with Deep Convolutional Neural Networks
Albelwi, S. A., & Mahmood, A. (2016) · IEEE Long Island Systems, Applications and Technology Conference (LISAT) · doi:10.1109/lisat.2016.7494142 · Cited by 10
Self-citationA Gloss Composition and Context Clustering Based Distributed Word Sense Representation Model
Chen, T., Xu, R., He, Y., & Wang, X. (2015) · Entropy · doi:10.3390/e17096007 · Cited by 9
Wang, B., Hong-xia, P., & Du, H. (2017) · Entropy · doi:10.3390/e19030086 · Cited by 1
Cited by
Review of Deep Learning Algorithms and Architectures
Shrestha, A., & Mahmood, A. (2019) · IEEE Access · doi:10.1109/access.2019.2912200 · Cited by 1,883
Self-citationObject detection using YOLO: challenges, architectural successors, datasets and applications
Diwan, T., Anirudh, G., & Tembhurne, J. V. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-022-13644-y · Cited by 1,549
Machine learning for email spam filtering: review, approaches and open research problems
Dada, E. G., Bassi, J. S., Chiroma, H., Abdulhamid, S. M., Adetunmbi, A. O., & Ajibuwa, O. E. (2019) · Heliyon · doi:10.1016/j.heliyon.2019.e01802 · Cited by 536
Tian, Y., Lai, R., Li, X., Xiang, L., & Tian, J. (2020) · Applied Energy · doi:10.1016/j.apenergy.2020.114789 · Cited by 354
Darwish, A., Ezzat, D., & Hassanien, A. E. (2020) · Swarm and Evolutionary Computation · doi:10.1016/j.swevo.2019.100616 · Cited by 279
Metadata
- Authors
- Albelwi, S., & Mahmood, A.
- Venue
- Entropy
- Publisher
- MDPI AG
- Published
- 2017-05-24
- Volume
- 19
- Issue
- 6
- Pages
- 242
- ISSN
- 1099-4300 (ISSN-L)
- OpenAlex
- W2618353736
- Cited by
- 278
- References
- 66
- FWCI
- 11.34
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.