The DOI matches its Crossref record; 358 of the 372 references it cites were found in a registry or an index; 137 of the 246 sources its claims cite were checked for support.
Review of deep learning: concepts, CNN architectures, challenges, applications, future directions
Open accessAbstract
In the last few years, the deep learning (DL) computing paradigm has been deemed the Gold Standard in the machine learning (ML) community. Moreover, it has gradually become the most widely used computational approach in the field of ML, thus achieving outstanding results on several complex cognitive tasks, matching or even beating those provided by human performance. One of the benefits of DL is the ability to learn massive amounts of data. The DL field has grown fast in the last few years and it has been extensively used to successfully address a wide range of traditional applications. More importantly, DL has outperformed well-known ML techniques in many domains, e.g., cybersecurity, natural language processing, bioinformatics, robotics and control, and medical information processing, among many others. Despite it has been contributed several works reviewing the State-of-the-Art on DL, all of them only tackled one aspect of the DL, which leads to an overall lack of knowledge about it. Therefore, in this contribution, we propose using a more holistic approach in order to provide a more suitable starting point from which to develop a full understanding of DL. Specifically, this review attempts to provide a more comprehensive survey of the most important aspects of DL and including those enhancements recently added to the field. In particular, this paper outlines the importance of DL, presents the types of DL techniques and networks. It then presents convolutional neural networks (CNNs) which the most utilized DL network type and describes the development of CNNs architectures together with their main features, e.g., starting with the AlexNet network and closing with the High-Resolution network (HR.Net). Finally, we further present the challenges and suggested solutions to help researchers understand the existing research gaps. It is followed by a list of the major DL applications. Computational tools including FPGA, GPU, and CPU are summarized along with a description of their influence on DL. The paper ends with the evolution matrix, benchmark datasets, and summary and conclusion.
Overview
CoChat Summary
## TL;DR
This paper is a broad review of deep learning that explains core concepts, classifies DL approaches, and surveys CNN architectures from AlexNet to HRNet. It also summarizes major challenges, applications, and computational platforms, with a heavy emphasis on medical imaging and CNN-based solutions.
## Key findings
- The authors reviewed more than 300 papers on deep learning topics, with 108 papers from 2020, 76 from 2019, and 48 from 2018.
- They argue that DL can reduce manual feature engineering by learning features automatically and that it has already achieved strong results in tasks such as image classification, detection, segmentation, and registration.
- The paper highlights that CNNs are the most used DL network type and reviews their architectural evolution from AlexNet (2012) to HRNet (2020).
- It identifies major DL challenges including lack of training data, imbalanced data, interpretability, uncertainty scaling, catastrophic forgetting, model compression, overfitting, vanishing/exploding gradients, and underspecification.
## Methods
- This is a survey/review paper, not an experimental study.
- The authors report reviewing literature from 2010–2020 (mainly 2020 and 2019, with some 2021 papers), drawing from publishers including IEEE, Elsevier, MDPI, Nature, ACM, and Springer, plus some arXiv papers.
- Their review was organized to cover DL approaches and network types, CNN architectures, challenges and solutions, applications, and computational approaches.
## Limitations
- The paper itself states that the review is not exhaustive for every topic; for example, in registration it says it is “out of the scope of this work to provide an extensive review of this topic.”
- The text also makes evident that some sections are descriptive rather than systematic: it summarizes many studies and architectures but does not report a formal meta-analysis or quantitative comparison across all included papers. (This is evident from the text.)
- Several sections are labeled only as headings or brief placeholders in the provided text, suggesting the supplied version is fragmentary/incomplete in places. (This is evident from the text.)
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures





















![Fig. 22 The architecture of DenseNet Network (adopted from [112])](https://figures.cochat.ai/paper-figures/eb645d1d79224b54842c8571df3a3e02/review-of-deep-learning-concepts-cnn-architectures-challenges-applications-figure-22.jpg)








Verification
Claims the paper makes
81 Valid · 73 Not valid · 62 Not checked · 158 Own results and interpretations
Some cited sources support this claim and some don’t — read each source’s verdict below.
In the paper
“Increase the depth (or width) of the model.”
§ Review of deep learning: concepts, CNN architectures, challenges, applications, future directions > Types of DL networks > Convolutional neural networks
What the sources say
- Disputedcites [86] Alzubaidi L, Fadhel MA, Al-Shamma O, Zhang J, Duan Y. Deep learning models for classification of red blood cells in microscopy images to aid in sickle cell anemia diagnosis. Electronics. 2020;9(3):427. sourceExperimental — accuracy not yet measured
In the paper’s model-comparison discussion (page not available in our copy), the authors say they increased network depth to improve performance, but it did not help; Model 2 performed best. The cited passage therefore does not support recommending greater depth as a way to improve performance, and it gives no corresponding result for greater width. (LLM, conf=0.86)
Checked by a reasoning model — may be wrong.
- Not checked — no full text of the sourcecites [87] Alzubaidi L, Fadhel MA, Al-Shamma O, Zhang J, Santamaría J, Duan Y, Oleiwi SR. Towards a better understanding of transfer learning for medical imaging: a case study. Appl Sci. 2020;10(13):4523. source
- Partly supportedcites [88] Alzubaidi L, Al-Shamma O, Fadhel MA, Farhan L, Zhang J, Duan Y. Optimizing the performance of breast cancer classification by employing the same domain transfer learning from hybrid deep convolutional neural network model. Electronics. 2020;9(3):445. sourceExperimental — accuracy not yet measured
Section 3.5 (Proposed Model; page unavailable) says a deep CNN with improved feature extraction is needed for breast-biopsy classification and describes a multi-branch architecture. It does not establish that simply increasing a model’s depth or width improves CNN performance; the proposed model also uses residual links and other design choices. (LLM, conf=0.78)
plainstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“The main benefit of CNN compared to its predecessors is that it automatically identifies the relevant features without any human supervision [76].”
§ Review of deep learning: concepts, CNN architectures, challenges, applications, future directions > Types of DL networks > Convolutional neural networks
What the sources say
- Partly supportedcites [76] Gu J, Wang Z, Kuen J, Ma L, Shahroudy A, Shuai B, Liu T, Wang X, Wang G, Cai J, et al. Recent advances in convolutional neural networks. Pattern Recogn. 2018;77:354–77. sourceExperimental — accuracy not yet measured
The paper says CNNs learn features automatically rather than relying on handcrafted features (5.3, p. 22), supporting the feature-learning part of the claim. But it does not support “without any human supervision”: the Conclusions (p. 28) say training requires manually collected labeled data and identify unsupervised CNN learning as a goal for further work. (LLM, conf=0.88)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“Add regularization.”
§ Review of deep learning: concepts, CNN architectures, challenges, applications, future directions > Types of DL networks > Convolutional neural networks
What the sources say
- Supportedcites [86] Alzubaidi L, Fadhel MA, Al-Shamma O, Zhang J, Duan Y. Deep learning models for classification of red blood cells in microscopy images to aid in sickle cell anemia diagnosis. Electronics. 2020;9(3):427. source
Section 3.2 (p. 8) says data augmentation enhances classification performance and helps prevent CNN overfitting. The paper thus supports the hedged claim that adding an overfitting-control technique may improve CNN performance, specifically in red-blood-cell classification. (LLM, conf=0.82)
- Not checked — no full text of the sourcecites [87] Alzubaidi L, Fadhel MA, Al-Shamma O, Zhang J, Santamaría J, Duan Y, Oleiwi SR. Towards a better understanding of transfer learning for medical imaging: a case study. Appl Sci. 2020;10(13):4523. source
- Supportedcites [88] Alzubaidi L, Al-Shamma O, Fadhel MA, Farhan L, Zhang J, Duan Y. Optimizing the performance of breast cancer classification by employing the same domain transfer learning from hybrid deep convolutional neural network model. Electronics. 2020;9(3):445. source
Section 3.5 (Proposed Model; page unavailable) says the authors placed a dropout layer in their CNN to prevent overfitting. Dropout is a form of regularization, so this supports the qualified claim that adding regularization may improve CNN performance. (LLM, conf=0.85)
plainstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“As a substitution for a FC layer, the GAP is also employed, which represents the second novel concept and enables a significant reduction in the number of model parameters.”
§ Review of deep learning: concepts, CNN architectures, challenges, applications, future directions > CNN architectures > Network-in-network
What the sources say
- Supportedcites [94] Lin M, Chen Q, Yan S. Network in network; 2013. arXiv preprint arXiv:1312.4400. source
Section 3.2 (p. 4) proposes global average pooling as a replacement for fully connected layers and says the pooling layer has no parameters to optimize. Section 4.6 (p. 7) contrasts this with fully connected layers’ trainable dense transformation matrices, supporting the claimed reduction in parameters, though it does not quantify the reduction. (LLM, conf=0.90)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“In 2015, Srivastava et al. [105] suggested a novel CNN architecture, called Highway Network, to overcome this issue.”
§ Review of deep learning: concepts, CNN architectures, challenges, applications, future directions > CNN architectures > Highway network
What the sources say
- Supportedcites [105] Srivastava RK, Greff K, Schmidhuber J. Highway networks; 2015. arXiv preprint arXiv:1505.00387. source
The Introduction (p. 1) presents highway networks as a novel architecture intended to make very deep networks easier to optimize. Section 2.1 (p. 2) describes convolutional highway layers, supporting the CNN characterization, although the architecture is not limited to CNNs. (LLM, conf=0.93)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on 2 assumptions not shown here.
- Supported
93 more claims · 31 with their cited sources checked
Create a free account to see all 374 claims and to verify the 5 references still open.
- Probably not a substantial claim. 224
- It may state two separate claims as one. 109
- The quote only partly states this claim. 2
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Previous methodologies, such as, artificial neural networks (NNs), and the more recent DL techniques, have been preferred in SOC estimation because of their applicability to depict more intricate correlation systems of input parameters and SOC [44].”
From Review Paper for SOC Estimation Techniques: Challenges and Future Areas for Improvement (2024) · cites this paper as [44]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Self-check results appear here once an independent check has confirmed them.
Cited works that were retracted
Works this paper cites that were later retracted, from the retraction notices publishers deposit. Major when the notice came out before this paper was published.
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
“Min Z. Public welfare organization management system based on FPGA and deep learning. Microprocess Microsyst. 2020;80:103333.”
Cited work 10.1016/j.micpro.2020.103333 · retracted 2021-02-01 · notice 10.1016/j.micpro.2020.103333 · this paper published 2021-03-31
Citations
Cites
1 retracted
RETRACTED: Public welfare organization management system based on FPGA and deep learning
Min, Z. (2020) · Microprocessors and Microsystems · doi:10.1016/j.micpro.2020.103333 · Cited by 16
RetractedDeep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Densely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Squeeze-and-Excitation Networks
Hu, J., Shen, L., & Sun, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00745 · Cited by 35,667
Research works printed but not linked: 4
- Erhan D, Bengio Y, Courville A, Vincent P. Visualizing higher-layer features of a deep network. Univ Montreal. 2009;1341(3):1.
- Ribeiro AH, Tiels K, Aguirre LA, Schön T. Beyond exploding and vanishing gradients: analysing RNN training using attractors and smoothness. In: International conference on artificial intelligence and statistics, PMLR; 2020. p. 2370–80.
- Zhao H, Han Z, Yang Z, Zhang Q, Yang F, Zhou L, Yang M, Lau FC, Wang Y, Xiong Y, et al. Hived: sharing a {GPU} cluster for deep learning with guarantees. In: 14th {USENIX} symposium on operating systems design and implementation ({OSDI} 20); 2020. p. 515–32.
- Rakotomamonyj A. Optimizing area under roc with SVMS. In: Proceedings of the European conference on artificial intelligence workshop on ROC curve and artificial intelligence (ROCAI 2004), 2004. p. 71–80.
Cited by
Alzubaidi, L., Bai, J., Al-Sabaawi, A., Santamaría, J., Albahri, A. S., Al-dabbagh, B. S. N., Fadhel, M. A., Manoufali, M., Zhang, J., Al-Timemy, A. H., et al. (2023) · Journal of Big Data · doi:10.1186/s40537-023-00727-2 · Cited by 907
Self-citationDeep learning in computer vision: A critical review of emerging techniques and application scenarios
Chai, J., Zeng, H., Li, A., & Ngai, E. W. T. (2021) · Machine Learning with Applications · doi:10.1016/j.mlwa.2021.100134 · Cited by 754
Machine Learning for Structural Engineering: A State-of-the-Art Review
Thai, H.-T. (2022) · Structures · doi:10.1016/j.istruc.2022.02.003 · Cited by 748
Albahri, A. S., Duhaim, A. M., Fadhel, M. A., Alnoor, A., Baqer, N. S., Alzubaidi, L., Albahri, O. S., Alamoodi, A. H., Bai, J., Salhi, A., et al. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.03.008 · Cited by 716
Self-citationQiu, S., Zhao, H., Jiang, N., Wang, Z., Liu, L., An, Y., Zhao, H., Miao, X., Liu, R., & Fortino, G. (2022) · Information Fusion · doi:10.1016/j.inffus.2021.11.006 · Cited by 671
Metadata
- Authors
- Alzubaidi, L., Zhang, J., Humaidi, A. J., Al-Dujaili, A., Duan, Y., Al-Shamma, O., Santamaría, J., Fadhel, M. A., Al-Amidie, M., & Farhan, L.
- Affiliations
- Queensland University of Technology (AU)
- University of Technology - Iraq (IQ)
- Middle Technical University (IQ)
- University of Missouri (US)
- University of Information Technology and Communications (IQ)
- Universidad de Jaén (ES)
- University of Sumer (IQ)
- Manchester Metropolitan University (GB)
- Corresponding author
- Alzubaidi, Laith
- ORCID
- Venue
- Journal of Big Data
- Published
- 2021-03-31
- Volume
- 8
- Issue
- 1
- Pages
- 53-53
- ISSN
- 2196-1115 (ISSN-L)
- PMC
- PMC8010506
- OpenAlex
- W3140854437
- Semantic Scholar
- 0084f3cb0a1754272151c5268a783f24bf5676a0
- Cited by
- 7,906
- References
- 372
- Citations by year
- 2026: 1140 · 2025: 2146 · 2024: 2166 · 2023: 1583 · 2022: 719 · 2021: 152
- FWCI
- 414.94
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Journal Of Big Data (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RUJA. Institutional Repository of Scientific Production of the University of Jaén (University of Jaén) (publishedVersion, cc-by-nc-nd)
- QUT ePrints (Queensland University of Technology) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Queensland University of Technology
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- deep learning, convolutional neural networks, convolutional neural network architecture, AlexNet, HRNet, deep learning applications, FPGA, graphics processing units, CPU, benchmark dataset
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-09-28, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-09-28 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Review of deep learning: concepts, CNN architectures, challenges, applications, future directions. https://app.cochat.ai/papers/10.1186/s40537-021-00444-8
- The paper
- Alzubaidi, L., Zhang, J., Humaidi, A. J., Al-Dujaili, A., Duan, Y., Al-Shamma, O., Santamaría, J., Fadhel, M. A., Al-Amidie, M., & Farhan, L. (2021). Review of deep learning: concepts, CNN architectures, challenges, applications, future directions. Journal of Big Data. https://doi.org/10.1186/s40537-021-00444-8
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.