Cited by 1,883Neural Networks and ApplicationsAdvanced Neural Network ApplicationsStochastic Gradient Optimization Techniques

The DOI matches its Crossref record; 85 of the 88 references it cites were found in a registry or an index.

Review of Deep Learning Algorithms and Architectures

Open access

Shrestha, Ajay, Mahmood, Ausif·2019·IEEE Access

DOI 10.1109/access.2019.2912200

Open this paper in CoChat — free account Open PDF

Abstract

Deep learning (DL) is playing an increasingly important role in our lives. It has already made a huge impact in areas, such as cancer diagnosis, precision medicine, self-driving cars, predictive forecasting, and speech recognition. The painstakingly handcrafted feature extractors used in traditional learning, classification, and pattern recognition systems are not scalable for large-sized data sets. In many cases, depending on the problem complexity, DL can also overcome the limitations of earlier shallow networks that prevented efficient training and abstractions of hierarchical representations of multi-dimensional training data. Deep neural network (DNN) uses multiple (deep) layers of units with highly optimized algorithms and architectures. This paper reviews several optimization methods to improve the accuracy of the training and to reduce training time. We delve into the math behind training algorithms used in recent deep networks. We describe current shortcomings, enhancements, and implementations. The review also covers different types of deep architectures, such as deep convolution networks, deep residual networks, recurrent neural networks, reinforcement learning, variational autoencoders, and others.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1109/access.2019.2912200

Cites

85 of 85 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 5×
“tion with a 152-layer DNN on the ImageNet dataset [62].”
BackgroundMethodInfluential
Long Short-Term Memory
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 2×
“The input, forget and output gates dictates new data going into the cell, what remains in the cell and the cell values used in the calculation of the output of the LSTM block respectively [41], [43].”
Method
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 4×
“Multilayer network training cost on MNIST dataset using different adaptive learning algorithms [58].”
BackgroundMethodInfluential
Deep learning
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 2×
“7-layer architecture of CNN for character recognition [28].”
BackgroundMethod
Adaptation in Natural and Artificial Systems
Holland, J. H. (1992) · The MIT Press eBooks · doi:10.7551/mitpress/1090.001.0001 · Cited by 47,340
Untitled work
(2014) · Cited by 33,419
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818 · cited 2×
“[59] S. Ioffe and C. Szegedy, ‘‘Batch normalization: Accelerating deep network training by reducing internal covariate shift,’’ CoRR, Feb. 2015.”
Background
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545 · cited 1×
“GRUs are smaller in size than LSTM as they don’t include the output gate, and can perform better than LSTM on only some simpler datasets [44], [45].”
Background
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928

Flagged first, then by citations

Show all 88 references

3 of the 88 references couldn’t be matched to a known paper.

Research works printed but not linked: 10

vapnik, 2015, Learning using privileged information: Similarity control and knowledge transfer, J Mach Learn Res, 16, 2023
ng, 2018, Autoencoders
2018, Amazon EC2 P2 & P3 Instances Amazon EC2 Instance Types
simpson, 2015, Uniform learning in a deep neural network via ‘oddball’ stochastic gradient descent, CoRR
minsky, 1969, Perceptrons An Introduction to Computational Geometry Expanded Edition, 258
2019, Top Deep Learning Github Repositories
sastry, 2005, Genetic Algorithms
10.1017/CBO9780511543241 · DOI
wu, 2016, Google’s neural machine translation system: Bridging the gap between human and machine translation, CoRR
metz, 2019, Turing Award Won by 3 Pioneers in Artificial Intelligence, 3b

Sorted by citations

Items 201–300 of 1,001

Real-Time System Identification Using Deep Learning for Linear Processes With Application to Unmanned Aerial Vehicles
Ayyad, A., Chehadeh, M. S., Awad, M. I., & Zweiri, Y. (2020) · IEEE Access · doi:10.1109/access.2020.3006277 · Cited by 54
“The ADAM optimization algorithm was utilized to further accelerate training and add robustness against noisy gradients as it was shown to outperform other adaptive learning rate techniques in [53], [54].”
Sentinel2GlobalLULC: A Sentinel-2 RGB image tile dataset for global land use/cover mapping with deep learning
Benhammou, Y., Alcaraz‐Segura, D., Guirado, E., Khaldi, R., Achchab, B., Herrera, F., & Tabik, S. (2022) · Scientific Data · doi:10.1038/s41597-022-01775-8 · Cited by 60
A Secure Federated Deep Learning-Based Approach for Heating Load Demand Forecasting in Building Environment
Moradzadeh, A., Moayyed, H., Mohammadi‐Ivatloo, B., Aguiar, A. P., & Anvari‐Moghaddam, A. (2021) · IEEE Access · doi:10.1109/access.2021.3139529 · Cited by 52
“This technique has been able to solve many problems related to machine learning methods, including feature extraction and pattern recognition, by providing a layer-by-layer structure [4], [34].”
Method
Exploring Machine Learning for Semiconductor Process Optimization: A Systematic Review
Chen, Y.-L., Sacchi, S., Dey, B., Blanco, V., Halder, S., Leray, P. J., & Gendt, S. D. (2024) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2024.3429479 · Cited by 52
Securing the CAN bus using deep learning for intrusion detection in vehicles
Rai, R., Grover, J., Sharma, P., & Pareek, A. (2025) · Scientific Reports · doi:10.1038/s41598-025-98433-x · Cited by 52
“By leveraging these layered architectures, deep learning models can automatically extract hierarchical representations of data, allowing for more abstract and nuanced understanding 44 .”
Intelligent Sensors for Sustainable Food and Drink Manufacturing
Watson, N. J., Bowler, A. L., Rady, A. M., Fisher, O. J., Simeone, A., Escrig, J., Woolley, E., & Adedeji, A. A. (2021) · Frontiers in Sustainable Food Systems · doi:10.3389/fsufs.2021.642786 · Cited by 57
“Deep learning does come with drawbacks however, such as high training time, overfitting and increased complexity (Shrestha and Mahmood, 2019).”
Background
Application of bio-inspired optimization algorithms in food processing
Sarkar, T., Salauddin, M., Mukherjee, A. B., Shariati, M. A., Ребезов, М. Б., Tretyak, L., Pateiro, M., & Lorenzo, J. M. (2022) · Current Research in Food Science · doi:10.1016/j.crfs.2022.02.006 · Cited by 51
“The error in the value of h and hi is required to be minimized (Chai et al., 2021; Shrestha and Mahmood, 2019).”
Background
Emerging Themes in CryoEM─Single Particle Analysis Image Processing
Vilas, J. L., Carazo, J. M., & Sorzano, C. Ó. S. (2022) · Chemical Reviews · doi:10.1021/acs.chemrev.1c00850 · Cited by 55
From classical to deep learning: review on cartilage and bone segmentation techniques in knee osteoarthritis research
Gan, H., Ramlee, M. H., Wahab, A. A., Lee, Y. S., & Shimizu, A. (2020) · Artificial Intelligence Review · doi:10.1007/s10462-020-09924-4 · Cited by 53
“2020; Shrestha and Mahmood 2019), useful data repositories for deep learning practitioners (Sengupta et al.”
Background
Deep Learning Based Instance Segmentation of Titanium Dioxide Particles in the Form of Agglomerates in Scanning Electron Microscopy
Monchot, P., Coquelin, L., Guerroudj, K., Feltin, N., Delvallée, A., Crouzier, L., & Fischer, N. (2021) · Nanomaterials · doi:10.3390/nano11040968 · Cited by 51
“For these rather generic parameters, it is advisable to rely on experts in the field and to read the good practice guides, review papers such as [29,30] and numerical comparisons [31,32] in order to decide on a set of relevant hyper-parameters.”
Deep Learning Algorithms for Machinery Health Prognostics Using Time-Series Data: A Review
Thoppil, N. M., Vasu, V., & Rao, C. S. P. (2021) · Journal of Vibration Engineering & Technologies · doi:10.1007/s42417-021-00286-x · Cited by 56
Quantum Machine Learning for Next-G Wireless Communications: Fundamentals and the Path Ahead
Narottama, B., Mohamed, Z., & Aı̈ssa, S. (2023) · IEEE Open Journal of the Communications Society · doi:10.1109/ojcoms.2023.3309268 · Cited by 59
Integrating machine learning for sustaining cybersecurity in digital banks
Asmar, M., & Tuqan, A. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e37571 · Cited by 50
“The model ’ s accuracy and adaptability are improved by retraining with more recent data [159].”
Method
A Review on Deep Learning in Minimally Invasive Surgery
Rivas-Blanco, I., Pérez-del-Pulgar, C. J., Garcia-Morales, I., & Muñoz, V. F. (2021) · IEEE Access · doi:10.1109/access.2021.3068852 · Cited by 54
“A LSTM consists of blocks of memory cell state through which signal ﬂows while being regulated by input, forget, and output gates [4], which allows to add or remove information to the cell state.”
BackgroundMethodInfluential
Deep learning-based cyber resilient dynamic line rating forecasting
Moradzadeh, A., Mohammadpourfard, M., Genc, I. V. M., Şeker, Ş. S., & Mohammadi‐Ivatloo, B. (2022) · International Journal of Electrical Power & Energy Systems · doi:10.1016/j.ijepes.2022.108257 · Cited by 52
Online visual monitoring method for liquid rocket engine nozzle welding based on a multi-task deep learning model
Zhou, Y., Chang, B., Zou, H., Sun, L., Wang, L., & Du, D. (2023) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2023.02.016 · Cited by 49
Deep learning on spatiotemporal graphs: A systematic review, methodological landscape, and research opportunities
Zeghina, A. O., Leborgne, A., Ber, F. L., & Vacavant, A. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.127861 · Cited by 50
Project-Level Encoding for Neural Source Code Summarization of Subroutines
Bansal, A., Haque, S., & McMillan, C. (2021) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc52881.2021.00032 · Cited by 45
Classification of Surface Defects on Steel Strip Images using Convolution Neural Network and Support Vector Machine
Boudiaf, A., Benlahmidi, S., Harrar, K., & Zaghdoudi, R. (2022) · Journal of Failure Analysis and Prevention · doi:10.1007/s11668-022-01344-6 · Cited by 48
“For this work, we have retained the AlexNet model due to its simplicity and efﬁciency [27, 28].”
Method
A Deep Neural Network-Based Multi-Frequency Path Loss Prediction Model from 0.8 GHz to 70 GHz
Nguyen, C. M. H., & Cheema, A. A. (2021) · Sensors · doi:10.3390/s21155100 · Cited by 48
“DNNs are categorized into five main types of networks as described in [29], and this paper proposed a feed-forward (FF) multilayer perceptron (MLP) network architecture to estimate the path loss of wireless channels in a wide frequency range from 0.”
BackgroundInfluential
Stacking Ensemble Methodology Using Deep Learning and ARIMA Models for Short-Term Load Forecasting
Bento, P., Pombo, J., Calado, M. R. A., & Mariano, S. J. P. S. (2021) · Energies · doi:10.3390/en14217378 · Cited by 48
“The term “deep” is used to symbolize the more complex structure of layers and nodes, which significantly increases the number of weights and bias terms, deriving a more abstract and high-dimensional feature mapping from the provided input and output data [37,38].”
Background
A Systematic Literature Review on the Use of Deep Learning in Precision Livestock Detection and Localization Using Unmanned Aerial Vehicles
Yousefi, M., Rafie, A. S. M., Al-Haddad, S. A. R., & Azrad, S. (2022) · IEEE Access · doi:10.1109/access.2022.3194507 · Cited by 53
Infomax-based deep autoencoder network for recognition of multi-element geochemical anomalies linked to mineralization
Esmaeiloghli, S., Tabatabaei, S. H., & Carranza, E. J. M. (2023) · Computers & Geosciences · doi:10.1016/j.cageo.2023.105341 · Cited by 47
Design and Implementation of an Internet-of-Things-Enabled Smart Meter and Smart Plug for Home-Energy-Management System
Dhaou, I. B. (2023) · Electronics · doi:10.3390/electronics12194041 · Cited by 47
“Deep learning has largely become practical thanks to developments in silicon and massively parallel technology (like GPU), high-speed internet, the collection of big volumes of data, and other factors [37].”
Background
Machine Learning for Structural Health Monitoring of Aerospace Structures: A Review
Scarselli, G., & Nicassio, F. (2025) · Sensors · doi:10.3390/s25196136 · Cited by 59
“Major architectures include multilayer perceptrons, convolutional neural networks (CNNs, Figures 6 and 7), recurrent neural networks (RNNs/LSTMs), generative adversarial networks, and autoencoders (Figure 8) [52,61–64].”
On Learning Spectral Masking for Single Channel Speech Enhancement Using Feedforward and Recurrent Neural Networks
Saleem, N., Khattak, M. I., Al‐Hasan, M., & Qazi, A. B. (2020) · IEEE Access · doi:10.1109/access.2020.3021061 · Cited by 47
“DNNs are selective learning machines and have shown to perform exceptionally well in the speech enhancement task [37]–[41].”
Background
Machine Learning and Deep Learning: A Comparative Review
Alaskar, H., & Saba, T. (2021) · Algorithms for intelligent systems · doi:10.1007/978-981-33-6307-6_15 · Cited by 46
A survey on brain tumor image analysis
Sailunaz, K., Alhajj, S., Özyer, T., Rokne, J. G., & Alhajj, R. (2023) · Medical & Biological Engineering & Computing · doi:10.1007/s11517-023-02873-4 · Cited by 48
“Deep learning is a subset of ML that uses a hierarchical structure of artificial neural networks (ANN) with multiple hidden layers to extract high-level features from inputs [73].”
Peer-to-peer overlay techniques for vehicular ad hoc networks: Survey and challenges
Ameur, A. I., Lakas, A., Yagoubi, M. B., & Oubbati, O. S. (2022) · Vehicular Communications · doi:10.1016/j.vehcom.2022.100455 · Cited by 46
Intelligent Traffic Management in Next-Generation Networks
Aouedi, O., Piamrat, K., & Parrein, B. (2022) · Future Internet · doi:10.3390/fi14020044 · Cited by 45
“Summary of popular deep learning frameworks [38,39].”
Employing Deep Reinforcement Learning to Cyber-Attack Simulation for Enhancing Cybersecurity
Oh, S. H., Kim, J., Nah, J. H., & Park, J. (2024) · Electronics · doi:10.3390/electronics13030555 · Cited by 62
“The approach is predicated on the estimation of a value function made up of Q-values that are computed for each state-action pair ( s t , a t ) [40].”
Background
Semi-Supervised Nests of Melanocytes Segmentation Method Using Convolutional Autoencoders
Kucharski, D., Kłeczek, P., Jaworek-Korjakowska, J., Dyduch, G., & Gorgoń, M. (2020) · Sensors · doi:10.3390/s20061546 · Cited by 45
“A systematic review on the use of deep neural networks including convolutional autoencoders can be found in [13,14].”
Background
Prediction of Therapeutic Peptides Using Machine Learning: Computational Models, Datasets, and Feature Encodings
Attique, M., Farooq, M. S., Khelifi, A., & Abid, A. (2020) · IEEE Access · doi:10.1109/access.2020.3015792 · Cited by 44
“LSTM use gradient descent and time-based backpropagation training algorithm and also address the gradient vanishing problem by allowing the gradients to pass unaltered, which was a limitation of RNN [64], [139].”
BackgroundInfluential
An integrated framework for plant data-driven process modeling using deep-learning with Monte-Carlo simulations
Hwangbo, S., Al, R., & Sin, G. (2020) · Computers & Chemical Engineering · doi:10.1016/j.compchemeng.2020.107071 · Cited by 44
Artificial Intelligence and Decision-Making: the question of Accountability
Gualdi, F., & Cordella, A. (2021) · Hawaii International Conference on System Sciences (HICSS) · doi:10.24251/hicss.2021.281 · Cited by 52
“These rules delegate organizations’ decision responsibilities and accountability [40] to technological artefacts that enforce pattern of action that are unique, and not necessarily compatible with the contextual normative, administrative and legal governing logics.”
Background
Recent Advances in Machine Learning Applied to Ultrasound Imaging
Micucci, M., & Iula, A. (2022) · Electronics · doi:10.3390/electronics11111800 · Cited by 51
A novel bagged tree ensemble regression method with multiple correlation coefficients to predict the train body vibrations using rail inspection data
Peng, L., Zheng, S., Zhong, Q., Chai, X., & Lin, J. (2022) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2022.109543 · Cited by 44
An ensemble method for the detection and classification of lung cancer using Computed Tomography images utilizing a capsule network with Visual Geometry Group
Bushara, A. R., Kumar, R. S. V., & Kumar, S. S. (2023) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2023.104930 · Cited by 48
Clinical translation of nanomedicine with integrated digital medicine and machine learning interventions
Ahmad, F., & Muhmood, T. (2024) · Colloids and Surfaces B Biointerfaces · doi:10.1016/j.colsurfb.2024.114041 · Cited by 46
Explainable deep learning based ultrasonic guided wave pipe crack identification method
Tang, R., Zhang, S., Wu, W., Han, Z., & Han, Z. (2022) · Measurement · doi:10.1016/j.measurement.2022.112277 · Cited by 44
Prediction of thermal degradation of biopolymers in biomass under pyrolysis atmosphere by means of machine learning
Kartal, F., Dalbudak, Y., & Özveren, U. (2023) · Renewable Energy · doi:10.1016/j.renene.2023.01.017 · Cited by 43
Current state and prospects of artificial intelligence in allergy
van Breugel, M., Fehrmann, R. S. N., Bügel, M., Rezwan, F. I., Holloway, J. W., Nawijn, M. C., Fontanella, S., Čustović, A., & Koppelman, G. H. (2023) · Allergy · doi:10.1111/all.15849 · Cited by 46
Image patch-based deep learning approach for crop and weed recognition
Hasan, A. S. M. M., Diepeveen, D., Laga, H., Jones, M. G. K., & Sohel, F. A. (2023) · Ecological Informatics · doi:10.1016/j.ecoinf.2023.102361 · Cited by 47
Deep Learning Prostate MRI Segmentation Accuracy and Robustness: A Systematic Review
Fassia, M.-K. H., Balasubramanian, A., Woo, S., Vargas, H. A., Hricak, H., Ender Konukoglu, A., & Becker, A. S. (2024) · Radiology Artificial Intelligence · doi:10.1148/ryai.230138 · Cited by 43
Advanced Intrusion Detection in MANETs: A Survey of Machine Learning and Optimization Techniques for Mitigating Black/Gray Hole Attacks
Hassan, S. M., Mohamad, M. M., & Muchtar, F. (2024) · IEEE Access · doi:10.1109/access.2024.3457682 · Cited by 52
“Notably, federated learning (FL) exemplifies a decentralized ML paradigm that addresses key challenges in MANET security [13], [14].”
Background
Application of physics-informed neural networks in fault diagnosis and fault-tolerant control design for electric vehicles: A review
Amin, A. A., Mubarak, A., & Waseem, S. (2025) · Measurement · doi:10.1016/j.measurement.2025.116728 · Cited by 43
Study on the identification and evaluation of growth years for Paris polyphylla var. yunnanensis using deep learning combined with 2DCOS
Yue, J., Li, Z., Zuo, Z., Zhao, Y., Zhang, J., & Wang, Y. (2021) · Spectrochimica Acta Part A Molecular and Biomolecular Spectroscopy · doi:10.1016/j.saa.2021.120033 · Cited by 43
Enabling deep learning using synthetic data: A case study for the automotive wiring harness manufacturing
Nguyen, H. G., Habiboglu, R., & Franke, J. E. (2022) · Procedia CIRP · doi:10.1016/j.procir.2022.05.142 · Cited by 43
Review on the COVID-19 pandemic prevention and control system based on AI
Yi, J., Zhang, H., Mao, J., Chen, Y., Zhong, H., & Wang, Y. (2022) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2022.105184 · Cited by 50
“Research on new algorithms such as unsupervised learning is significant in improving al-gorithm performance and intelligence degree (Shrestha and Mahmood, 2019).”
Electric load forecasting under False Data Injection Attacks using deep learning
Moradzadeh, A., Mohammadpourfard, M., Konstantinou, C., Genc, I. V. M., Kim, T., & Mohammadi‐Ivatloo, B. (2022) · Energy Reports · doi:10.1016/j.egyr.2022.08.004 · Cited by 42
“The mathematical modeling of LSTM is based on the following formulations (Shrestha and Mahmood, 2019):”
BackgroundMethod
Deep Learning-Based Computer-Aided Diagnosis (CAD): Applications for Medical Image Datasets
Kadhim, Y. A., Khan, M. U., & Mishra, A. (2022) · Sensors · doi:10.3390/s22228999 · Cited by 43
Intelligent Photonics: A Disruptive Technology to Shape the Present and Redefine the Future
Xu, D., Ma, Y., Jin, G., & Cao, L. (2024) · Engineering · doi:10.1016/j.eng.2024.08.016 · Cited by 47
Random fully connected layered 1D CNN for solving the Z-bus loss allocation problem
Sindi, H. F., Nour, M., Rawa, M. J. H., Öztürk, Ş., & Polat, K. (2020) · Measurement · doi:10.1016/j.measurement.2020.108794 · Cited by 42
A Comprehensive Review of Artificial Intelligence Based Algorithms Regarding Temporomandibular Joint Related Diseases
Özsarı, Ş., Güzel, M. S., Yılmaz, D., & Kamburoğlu, K. (2023) · Diagnostics · doi:10.3390/diagnostics13162700 · Cited by 41
“For a more comprehensive understanding of these methods, we included two publications in the references [7,8], which provide general information about ML and DL algorithms.”
Method
CRSF: An Intrusion Detection Framework for Industrial Internet of Things Based on Pretrained CNN2D-RNN and SVM
Li, S., Chai, G., Wang, Y., Zhou, G., Li, Z., Yu, D., & Gao, R. (2023) · IEEE Access · doi:10.1109/access.2023.3307429 · Cited by 44
Deep learning, numerical, and experimental methods to reveal hydrodynamics performance and cavitation development in centrifugal pump
Li, G., Sun, H., He, J., Ding, X., Zhu, W., Qin, C., Zhang, X., Zhou, X., Yang, B., & Guo, Y. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.121604 · Cited by 41
Materials informatics: A review of AI and machine learning tools, platforms, data repositories, and applications to architectured porous materials
Živić, F., Kaplarevic-Malisic, A. M., Grujović, N., Stojanovic, B. S., & Ivanovic, M. R. (2025) · Materials Today Communications · doi:10.1016/j.mtcomm.2025.113525 · Cited by 51
Integrative Data Augmentation with U-Net Segmentation Masks Improves Detection of Lymph Node Metastases in Breast Cancer Patients
Jin, Y. W., Jia, S., Ashraf, A. B., & Hu, P. (2020) · Cancers · doi:10.3390/cancers12102934 · Cited by 45
“Deep neural networks were inspired by the organization of the human visual cortex [29].”
Deep Learning and Transfer Learning for Automatic Cell Counting in Microscope Images of Human Cancer Cell Lines
Lavitt, F., Rijlaarsdam, D. J., van der Linden, D., Węglarz‐Tomczak, E., & Tomczak, J. M. (2021) · Applied Sciences · doi:10.3390/app11114912 · Cited by 41
“Deep Neural Networks (DNN) have been applied for a wide range of computer vision tasks and has set new state-of-the-art on several benchmarks related to classiﬁcation, object detection, and segmentation [29–33].”
Cross-evaluation of a parallel operating SVM – CNN classifier for reliable internal decision-making processes in composite inspection
Meister, S., Wermes, M., Stüve, J., & Groves, R. M. (2021) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2021.07.022 · Cited by 40
Deep-HAR: an ensemble deep learning model for recognizing the simple, complex, and heterogeneous human activities
Kumar, P., & Suresh, S. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-14492-0 · Cited by 41
A review of machine learning techniques for enhanced energy efficient 5G and 6G communications
Fowdur, T. P., & Doorgakant, B. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.106032 · Cited by 48
Evaluating the Performance of Machine Learning and Deep Learning Techniques to HyMap Imagery for Lithological Mapping in a Semi-Arid Region: Case Study from Western Anti-Atlas, Morocco
Hajaj, S., Harti, A. E., Jellouli, A., Pour, A. B., Himyari, S. M., Hamzaoui, A., & Hashim, M. (2023) · Minerals · doi:10.3390/min13060766 · Cited by 40
“Deep learning is broadly categorized into two types of neural network architectures: feedforward and recurrent [63].”
Background
Task Offloading Decision-Making Algorithm for Vehicular Edge Computing: A Deep-Reinforcement-Learning-Based Approach
Shi, W., Chen, L., & Zhu, X. (2023) · Sensors · doi:10.3390/s23177595 · Cited by 40
“However, the distribution of activation input values of the DNN network changes during the training process, gradually approaching the top and lower boundaries of the range of nonlinear function values [43].”
Background
Application of artificial intelligence in reverse logistics: A bibliometric and network analysis
Bhowmik, O., Chowdhury, S., Ashik, J. H., Mahmud, G., Khan, M. M., & Hossain, N. U. I. (2024) · Supply Chain Analytics · doi:10.1016/j.sca.2024.100076 · Cited by 60
A Convolutional Neural Network Approach to the Classification of Engineering Models
Manda, B., Bhaskare, P., & Muthuganapathy, R. (2021) · IEEE Access · doi:10.1109/access.2021.3055826 · Cited by 46
“[24] gives an overview of the various deep learning algorithms and architectures available.”
Background
Doctor's Dilemma: Evaluating an Explainable Subtractive Spatial Lightweight Convolutional Neural Network for Brain Tumor Diagnosis
Kumar, A., Manikandan, R., Köse, U., Gupta, D. K., & Satapathy, S. C. (2021) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3457187 · Cited by 42
Early Diagnosis of Multiple Sclerosis Using Swept-Source Optical Coherence Tomography and Convolutional Neural Networks Trained with Data Augmentation
López-Dorado, A., del Castillo, M. O., Satué, M., Rodrigo, M. J., Barea, R., Sánchez‐Morla, E. M., Cavaliere-Ballesta, C., Rodriguez-Ascariz, J. M., Boquete, L., & García‐Martín, E. (2021) · Sensors · doi:10.3390/s22010167 · Cited by 41
“, variants of the backpropagation algorithm), the weights are adjusted [25].”
Deep Learning for Image-Based Plant Growth Monitoring: A Review
Tong, Y.-S., Lee, T.-H., & Yen, K. S. (2022) · International Journal of Engineering and Technology Innovation · doi:10.46604/ijeti.2022.8865 · Cited by 43
“DL is essentially a subset of ML that consists of multi-layer artificial neural networks (ANNs) [67].”
Background
Machine learning–assisted prediction of heat fluxes through thermally anisotropic building envelopes
Shen, Z., Shrestha, S. S., Howard, D., Feng, T., Hun, D., & She, B. (2023) · Building and Environment · doi:10.1016/j.buildenv.2023.110157 · Cited by 39
Towards intelligent cooperative robotics in additive manufacturing: Past, present, and future
Rescsanski, S., Hebert, R. J., Haghighi, A., Tang, J., & Imani, F. (2024) · Robotics and Computer-Integrated Manufacturing · doi:10.1016/j.rcim.2024.102925 · Cited by 42
MAML-KalmanNet: A Neural Network-Assisted Kalman Filter Based on Model-Agnostic Meta-Learning
Chen, S., Zheng, Y., Lin, D., Cai, P., Xiao, Y., & Wang, S. (2025) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2025.3540018 · Cited by 39
“However, incorporating neural networks into the Kalman ﬁlter framework also introduces the inherent challenges associated with neural networks [27] when addressing the ﬁltering problem.”
Machine learning inversion design and application verification of a broadband acoustic filtering structure
Cheng, B., Wang, M., Gao, N., & Hou, H. (2021) · Applied Acoustics · doi:10.1016/j.apacoust.2021.108522 · Cited by 39
Roadmap on signal processing for next generation measurement systems
Iakovidis, D. K., Ooi, M. P., Kuang, Y. C., Demidenko, S. N., Shestakov, A., Sinitsin, V. V., Henry, M., Sciacchitano, A., Discetti, S., Donati, S., et al. (2021) · Measurement Science and Technology · doi:10.1088/1361-6501/ac2dbd · Cited by 43
Deep Graph Reinforcement Learning Based Intelligent Traffic Routing Control for Software-Defined Wireless Sensor Networks
Huang, R., Guan, W., Zhai, G., He, J., & Chu, X. (2022) · Applied Sciences · doi:10.3390/app12041951 · Cited by 39
“With the increase of GPU computing power, end-to-end models based on deep learning have become state-of-the-art in the fields of computer vision, natural language processing, and reinforcement learning [14].”
Background
Assessing Human Motion During Exercise Using Machine Learning: A Literature Review
Frangoudes, F., Matsangidou, M., Schiza, E. C., Neokleous, K., & Pattichis, C. S. (2022) · IEEE Access · doi:10.1109/access.2022.3198935 · Cited by 41
“9), which were used in 35 publications, can be separated into discriminative and generative algorithms [134].”
Background
Robust imaging habitat computation using voxel-wise radiomics features
Bernatowicz, K., Grussu, F., Ligero, M., García-Ruiz, A., Delgado, E., & Pérez-López, R. (2021) · Scientific Reports · doi:10.1038/s41598-021-99701-2 · Cited by 38
Understanding the meanings of citations using sentiment, role, and citation function classifications
Budi, I. D., & Yaniasih, Y. (2022) · Scientometrics · doi:10.1007/s11192-022-04567-4 · Cited by 41
“CNNs have been used extensively and successfully for processing images, text, and speech (Alom et al., 2019; Khamparia & Singh, 2019; Shrestha & Mahmood, 2019).”
Method
Using Explainable Artificial Intelligence to Obtain Efficient Seizure-Detection Models Based on Electroencephalography Signals
Vieira, J. C., Guedes, L. A., Santos, M. R., & Sánchez-Gendriz, I. (2023) · Sensors · doi:10.3390/s23249871 · Cited by 39
“In addition, DL models are more prone to overfitting [62], have longer training times, and require a large amount of data [63].”
Background
New Results for Prediction of Chaotic Systems Using Deep Recurrent Neural Networks
Serrano-Pérez, J. D. J., Fernández‐Anaya, G., Carrillo-Moreno, S., & Yu, W. (2021) · Neural Processing Letters · doi:10.1007/s11063-021-10466-1 · Cited by 39
Deep learning framework for uncovering compositional and environmental contributions to pitting resistance in passivating alloys
Sasidhar, K. N., Siboni, N. H., Mianroodi, J. R., Rohwerder, M., Neugebauer, J., & Raabe, D. (2022) · npj Materials Degradation · doi:10.1038/s41529-022-00281-x · Cited by 38
Emerging paradigms in microwave imaging technology for biomedical applications: unleashing the power of artificial intelligence
Khalid, N., Zubair, M. F., Mehmood, M. Q., & Massoud, Y. (2024) · npj Imaging · doi:10.1038/s44303-024-00012-8 · Cited by 41
Through-the-Wall Human Activity Recognition Using Radar Technologies: A Review
Yousaf, J., Yakoub, S., Karkanawi, S., Hassan, T., Almajali, E. R., Zia, H., & Ghazal, M. A. (2024) · IEEE Open Journal of Antennas and Propagation · doi:10.1109/ojap.2024.3459045 · Cited by 37
Classification of Covid-19 patients using efficient fine-tuned deep learning DenseNet model
Bohmrah, M. K., & Kaur, H. (2021) · Global Transitions Proceedings · doi:10.1016/j.gltp.2021.08.003 · Cited by 40
“As there are various forms of CNN such as Resnet, VGG, Xception and many more [9] .”
Three-Dimensional Printing Quality Inspection Based on Transfer Learning with Convolutional Neural Networks
Yang, C., Huang, W.-K., & Lin, K. (2023) · Sensors · doi:10.3390/s23010491 · Cited by 36
“, by using residual learning, factorization, and modules instead of layers, which increases the accuracy [72,73].”
Method
Detection for melanoma skin cancer through ACCF, BPPF, and CLF techniques with machine learning approach
Kavitha, P., Ayyappan, G., Jayagopal, P., Mathivanan, S. K., Mallik, S., Al‐Rasheed, A., Alqahtani, M. S., & Soufiene, B. O. (2023) · BMC Bioinformatics · doi:10.1186/s12859-023-05584-7 · Cited by 42
Machine Learning-Based Predictive Maintenance System for Artificial Yarn Machines
Akyaz, T., & Engın, D. (2024) · IEEE Access · doi:10.1109/access.2024.3454548 · Cited by 36
“Each layer can be customized to detect a defined feature from simple aspects to complex ones [44].”
BackgroundMethodInfluential
Solvent Screening for Separation Processes Using Machine Learning and High-Throughput Technologies
Edaugal, J. P., Zhang, D., Liu, D., Glezakou, V., & Sun, N. (2025) · Chem & Bio Engineering · doi:10.1021/cbe.4c00170 · Cited by 39
An Optimized Classification Model for Coffea Liberica Disease using Deep Convolutional Neural Networks
Montalbo, F. J. P., & Hernandez, A. A. (2020) · IEEE International Colloquium on Signal Processing & Its Applications (CSPA) · doi:10.1109/cspa48992.2020.9068683 · Cited by 37
“Thus, an empirical experiment should take place [22] [23].”
Background
On applying AI-driven flight data analysis for operational spacecraft model-based diagnostics
Tipaldi, M., Feruglio, L., Denis, P., & D’Angelo, G. (2020) · Annual Reviews in Control · doi:10.1016/j.arcontrol.2020.04.012 · Cited by 38
Enabling AI in future wireless networks: a data life cycle perspective
Nguyen, D. C., Cheng, P., Ding, M., López‐Pérez, D., Pathirana, P. N., Li, J., Seneviratne, A. P., Li, Y., & Poor, H. V. (2024) · Figshare · Cited by 36
Deep Learning Based Prediction of Gas Chromatographic Retention Indices for a Wide Variety of Polar and Mid-Polar Liquid Stationary Phases
Matyushin, D. D., Sholokhova, A. Y., & Buryak, A. K. (2021) · International Journal of Molecular Sciences · doi:10.3390/ijms22179194 · Cited by 36
“Deep learning is one of the most rapidly developing methodologies in all fields of science [20].”
Background
Bloom’s Taxonomy-based exam question classification: The outcome of CNN and optimal pre-trained word embedding technique
Gani, M. O., Ayyasamy, R. K., Sangodiah, A., & Fui, Y. T. (2023) · Education and Information Technologies · doi:10.1007/s10639-023-11842-1 · Cited by 41
“However, CNN is capable of obtaining local features and shows excellent performance in applications like object detection and image classification (Islam et al., 2020), and it is also utilised to solve natural language processing (NLP) problems (Shrestha & Mahmood, 2019).”
Method
Artificial Intelligence for Cancer Detection—A Bibliometric Analysis and Avenues for Future Research
Karger, E., & Kureljusic, M. (2023) · Current Oncology · doi:10.3390/curroncol30020125 · Cited by 39
“2009 Brain tumors 542 [82] 18 Shrestha and Mahmood 2019 General investigation 525 [83] 19 Tang et al.”
Background
An efficient system for water leak detection and localization based on IoT and lightweight deep learning
Boujelben, M., BenMessaoud, Z., Abid, M., & Elleuchi, M. (2023) · Internet of Things · doi:10.1016/j.iot.2023.100995 · Cited by 42
Prediction of riverine daily minimum dissolved oxygen concentrations using hybrid deep learning and routine hydrometeorological data
Hu, Y., Liu, C., & Wollheim, W. M. (2024) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2024.170383 · Cited by 37
PLM_Sol: predicting protein solubility by benchmarking multiple protein language models with the updated Escherichia coli protein solubility dataset
Zhang, X. J., Hu, X., Zhang, T., Yang, L., Liu, C., Xu, N., Wang, H., & Sun, W. (2024) · Briefings in Bioinformatics · doi:10.1093/bib/bbae404 · Cited by 38
“Deep learning relies on two fundamental pillars: data and model [58–60].”
Background
Machine Learning Applications in Energy Harvesting Internet of Things Networks: A Review
Alamu, O., Olwal, T. O., & Migabo, E. (2025) · IEEE Access · doi:10.1109/access.2024.3525263 · Cited by 40
“Vast applications of DL algorithms have been recorded in different areas such as pattern recognition, natural language processing, computer vision, regression, etc [37].”
Simulation-Based Deep Reinforcement Learning For Modular Production Systems
Feldkamp, N., Bergmann, S., & Straßburger, S. (2020) · Winter Simulation Conference · doi:10.1109/wsc48552.2020.9384089 · Cited by 35
“This enables the principle of Q-Learning even for large state and action spaces (Mnih et al. 2015; Shrestha and Mahmood 2019).”
Background
A hybrid of convolutional neural network and long short-term memory network approach to predictive maintenance
Nasser, A. R., & Al-Khazraji, H. (2021) · International Journal of Power Electronics and Drive Systems/International Journal of Electrical and Computer Engineering · doi:10.11591/ijece.v12i1.pp721-730 · Cited by 43

Authors: Shrestha, A., & Mahmood, A.
Affiliations: University of Bridgeport (US)
ORCID: Shrestha, Ajay
Mahmood, Ausif
Venue: IEEE Access
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Published: 2019
Volume: 7
Pages: 53040-53065
ISSN: 2169-3536 (ISSN-L)
DOI: 10.1109/access.2019.2912200
OpenAlex: W2942231644
Cited by: 1,883
References: 88
Citations by year: 2026: 116 · 2025: 317 · 2024: 360 · 2023: 375 · 2022: 297 · 2021: 259 · 2020: 139 · 2019: 20
FWCI: 81.55
Type: journal article
Language: en
Open access: gold
doi.org
Licence: ieeexplore.ieee.org/Xplorehelp/downloads/license-information/OAPA.html (vor)
Locations: IEEE Access (publishedVersion)
University of Bridgeport ScholarWorks (University of Bridgeport) (submittedVersion)
DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics: Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
Keywords: deep learning, deep learning architectures, deep convolutional neural networks, ResNet, recurrent neural networks, reinforcement learning, variational autoencoder, feature extraction, hierarchical representation, training time reduction

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Review of Deep Learning Algorithms and Architectures

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks