Cited by 1,883Neural Networks and ApplicationsAdvanced Neural Network ApplicationsStochastic Gradient Optimization Techniques

The DOI matches its Crossref record; 85 of the 88 references it cites were found in a registry or an index.

Review of Deep Learning Algorithms and Architectures

Open access

Shrestha, Ajay, Mahmood, Ausif·2019·IEEE Access

DOI 10.1109/access.2019.2912200

Open this paper in CoChat — free account Open PDF

Abstract

Deep learning (DL) is playing an increasingly important role in our lives. It has already made a huge impact in areas, such as cancer diagnosis, precision medicine, self-driving cars, predictive forecasting, and speech recognition. The painstakingly handcrafted feature extractors used in traditional learning, classification, and pattern recognition systems are not scalable for large-sized data sets. In many cases, depending on the problem complexity, DL can also overcome the limitations of earlier shallow networks that prevented efficient training and abstractions of hierarchical representations of multi-dimensional training data. Deep neural network (DNN) uses multiple (deep) layers of units with highly optimized algorithms and architectures. This paper reviews several optimization methods to improve the accuracy of the training and to reduce training time. We delve into the math behind training algorithms used in recent deep networks. We describe current shortcomings, enhancements, and implementations. The review also covers different types of deep architectures, such as deep convolution networks, deep residual networks, recurrent neural networks, reinforcement learning, variational autoencoders, and others.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1109/access.2019.2912200

Cites

85 of 85 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 5×
“tion with a 152-layer DNN on the ImageNet dataset [62].”
BackgroundMethodInfluential
Long Short-Term Memory
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 2×
“The input, forget and output gates dictates new data going into the cell, what remains in the cell and the cell values used in the calculation of the output of the LSTM block respectively [41], [43].”
Method
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 4×
“Multilayer network training cost on MNIST dataset using different adaptive learning algorithms [58].”
BackgroundMethodInfluential
Deep learning
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 2×
“7-layer architecture of CNN for character recognition [28].”
BackgroundMethod
Adaptation in Natural and Artificial Systems
Holland, J. H. (1992) · The MIT Press eBooks · doi:10.7551/mitpress/1090.001.0001 · Cited by 47,340
Untitled work
(2014) · Cited by 33,419
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818 · cited 2×
“[59] S. Ioffe and C. Szegedy, ‘‘Batch normalization: Accelerating deep network training by reducing internal covariate shift,’’ CoRR, Feb. 2015.”
Background
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545 · cited 1×
“GRUs are smaller in size than LSTM as they don’t include the output gate, and can perform better than LSTM on only some simpler datasets [44], [45].”
Background
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928

Show all 88 references

Research works printed but not linked: 10

vapnik, 2015, Learning using privileged information: Similarity control and knowledge transfer, J Mach Learn Res, 16, 2023
ng, 2018, Autoencoders
2018, Amazon EC2 P2 & P3 Instances Amazon EC2 Instance Types
simpson, 2015, Uniform learning in a deep neural network via ‘oddball’ stochastic gradient descent, CoRR
minsky, 1969, Perceptrons An Introduction to Computational Geometry Expanded Edition, 258
2019, Top Deep Learning Github Repositories
sastry, 2005, Genetic Algorithms
10.1017/CBO9780511543241 · DOI
wu, 2016, Google’s neural machine translation system: Bridging the gap between human and machine translation, CoRR
metz, 2019, Turing Award Won by 3 Pioneers in Artificial Intelligence, 3b

Sorted by citations

Items 301–400 of 1,001

Gradient descent-particle swarm optimization based deep neural network predictive control of pressurized water reactor power
Ejigu, D. A., & Liu, X. (2021) · Progress in Nuclear Energy · doi:10.1016/j.pnucene.2021.104108 · Cited by 35
VECTOR: Very deep convolutional autoencoders for non‐resonant background removal in broadband coherent anti‐Stokes Raman scattering
Wang, Z., O’Dwyer, K., Muddiman, R., Ward, T. E., Camp, C. H., & Hennelly, B. M. (2022) · Journal of Raman Spectroscopy · doi:10.1002/jrs.6335 · Cited by 37
“VECTOR and U-Net are both speciﬁc types of the more general AE architecture [40, 41] for which the encoder learns an efﬁcient representation of unlabelled data in the form of key features, which can then be used by the decoder to reconstruct the data in the absence of unwanted data such as noise.”
Background
Transfer learning for segmentation with hybrid classification to Detect Melanoma Skin Cancer
Dandu, R., Murthy, M. V., & Kumar, Y. B. R. (2023) · Heliyon · doi:10.1016/j.heliyon.2023.e15416 · Cited by 38
“Deep learning algorithms have been effectively developed in the field of biomedical health [33 – 35].”
Deep learning in digital pathology for personalized treatment plans of cancer patients
Wen, Z., Wang, S., Yang, D. M., Xie, Y., Chen, M., Bishop, J., & Xiao, G. (2023) · Seminars in Diagnostic Pathology · doi:10.1053/j.semdp.2023.02.003 · Cited by 34
Cybersecurity attacks on CAN bus based vehicles: a review and open challenges
Fakhfakh, F., Tounsi, M., & Mosbah, M. (2021) · Library Hi Tech · doi:10.1108/lht-01-2021-0013 · Cited by 34
Exploiting Pre-Trained Convolutional Neural Networks for the Detection of Nutrient Deficiencies in Hydroponic Basil
Gul, Z., & Bora, Ş. (2023) · Sensors · doi:10.3390/s23125407 · Cited by 35
“In addition, deep learning makes it easy to work with big data, greatly automates feature extraction and saves it from interventions such as manually making sense of the dataset and choosing algorithms accordingly [26].”
Background
Learning-based production, maintenance, and quality optimization in smart manufacturing systems: A literature review and trends
Paraschos, P. D., & Koulouriotis, D. E. (2024) · Computers & Industrial Engineering · doi:10.1016/j.cie.2024.110656 · Cited by 34
FSVLM: A Vision-Language Model for Remote Sensing Farmland Segmentation
Wu, H., Du, Z., Zhong, D., Wang, Y., & Tao, C. (2025) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2025.3532960 · Cited by 40
“In recent years, the advent of deep learning technology, especially the successful application of convolutional neural networks (CNNs) in remote sensing image processing [2], [3], [4], has brought new opportunities for accurate extraction of farmland information from remote sensing images [5].”
Optimizing deep learning predictive models: A comprehensive review of RNN and its variant architectures
Adekunle, A. A., Fofana, I., Picher, P., Rodriguez-Celis, E. M., Arroyo-Fernàndez, O. H., & Zemouri, R. M. (2025) · Applied Soft Computing · doi:10.1016/j.asoc.2025.114015 · Cited by 34
Learning dynamical systems in noise using convolutional neural networks
Mukhopadhyay, S., & Banerjee, S. (2020) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/5.0009326 · Cited by 35
Developing a Recognition System for Classifying COVID-19 Using a Convolutional Neural Network Algorithm
Alsaade, F. W., Aldhyani, T. H. H., & Al-Adhaileh, M. H. (2021) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2021.016264 · Cited by 34
“2 presents some of the most important contributions of the neural network in deep learning [27].”
Efficient deep neural network model for classification of grasp types using sEMG signals
Coşkun, M., Yıldırım, Ö., Demir, Y., & Acharya, U. R. (2021) · Journal of Ambient Intelligence and Humanized Computing · doi:10.1007/s12652-021-03284-9 · Cited by 37
“It has gained huge attention in the last decade and brought significant technological advancements to different areas such as financial services, self-driving cars, game theory, robotics, medicine and healthcare systems (Sengupta et al. 2020; Esteva et al. 2019; Shrestha and Mahmood 2019).”
Background
Short-Term Photovoltaic Power Plant Output Forecasting Using Sky Images and Deep Learning
Jakoplić, A., Franković, D., Havelka, J. G., & Bulat, H. (2023) · Energies · doi:10.3390/en16145428 · Cited by 37
Harnessing Deep Learning for Real-Time Water Quality Assessment: A Sustainable Solution
Toumi, S., Lekmine, S., Touzout, N., Moussa, H., Elboughdiri, N. A., Boudraa, R., Benslama, O., Kebir, M., Danish, S., Zhang, J., et al. (2024) · Water · doi:10.3390/w16233380 · Cited by 33
Improving Forest Above-Ground Biomass Estimation Accuracy Using Multi-Source Remote Sensing and Optimized Least Absolute Shrinkage and Selection Operator Variable Selection Method
Wang, E., Huang, T., Liu, Z., Bao, L., Guo, B., Yu, Z., Feng, Z., Luo, H., & Ou, G. (2024) · Remote Sensing · doi:10.3390/rs16234497 · Cited by 35
Detecting and counting pistachios based on deep learning
Rahimzadeh, M., & Attar, A. (2021) · Iran Journal of Computer Science · doi:10.1007/s42044-021-00090-6 · Cited by 37
“In recent years, machine vision has been used for many tasks to automate and replace machines with humans, which have yielded excellent results [32].”
Background
Hierarchical Transfer Learning for Multilingual, Multi-Speaker, and Style Transfer DNN-Based TTS on Low-Resource Languages
Azizah, K., Adriani, M., & Jatmiko, W. (2020) · IEEE Access · doi:10.1109/access.2020.3027619 · Cited by 33
“Model-based TTS research has been dominated by statistical parametric speech synthesis (SPSS) [5]–[9] until recent years in which deep learning has delivered extraordinary achievements in various ﬁelds [10]–[12].”
Method
Multirotors From Takeoff to Real-Time Full Identification Using the Modified Relay Feedback Test and Deep Neural Networks
Ayyad, A., Chehadeh, M. S., Silva, P. H., Wahbah, M., Hay, O. A., Boiko, I. M., & Zweiri, Y. (2021) · IEEE Transactions on Control Systems Technology · doi:10.1109/tcst.2021.3114265 · Cited by 33
“We utilized the ADAM optimization algorithm for training as it is a well-established algorithm with proven advantages in terms of convergence speeds and robustness to noisy gradients [44], [45].”
Extracting spatially global and local attentive features for rolling bearing fault diagnosis in electrical machines using attention stream networks
Karnavas, Y. L., Plakias, S., & Chasiotis, I. D. (2021) · IET Electric Power Applications · doi:10.1049/elp2.12063 · Cited by 33
“The DL models have restated the accuracy results in many research fields such as image recognition, natural language processing, recommendation systems, automatic speech recognition, and others [15].”
A CNN Prediction Method for Belt Grinding Tool Wear in a Polishing Process Utilizing 3-Axes Force and Vibration Data
Caesarendra, W., Triwiyanto, T., Pandiyan, V., Głowacz, A., Permana, S. D. H., & Tjahjowidodo, T. (2021) · Electronics · doi:10.3390/electronics10121429 · Cited by 36
Extraction of Agricultural Fields via DASFNet with Dual Attention Mechanism and Multi-scale Feature Fusion in South Xinjiang, China
Lu, R., Wang, N., Zhang, Y., Lin, Y., Wu, W., & Shi, Z. (2022) · Remote Sensing · doi:10.3390/rs14092253 · Cited by 33
Rapsai: Accelerating Machine Learning Prototyping of Multimedia Applications through Visual Programming
Du, R., Li, N., Jin, J., Carney, M., Miles, S., Kleiner, M., Yuan, X., Zhang, Y., Kulkarni, A., Liu, X., et al. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581338 · Cited by 40
Atten-GAN: Pedestrian Trajectory Prediction with GAN Based on Attention Mechanism
Fang, F., Zhang, P., Zhou, B., Qian, K., & Gan, Y. (2022) · Cognitive Computation · doi:10.1007/s12559-022-10029-z · Cited by 32
“In recent years, with the rapid development of deep learning [12, 13], data-driven methods have provided a new approach for understanding and predicting the complex motion of pedestrians.”
Background
An Overview of Vision Transformers for Image Processing: A Survey
Kameswari, C. S., Kavitha, J., Reddy, T. S., Chinthaguntla, B., Jagatheesaperumal, S. K., Gaftandzhieva, S. N., & Doneva, R. Z. (2023) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2023.0140830 · Cited by 38
“CNNs were once the standard [30] in the Computer Vision domain, but transformers are gaining popularity [29].”
Background
Optimal parameters of random forest for land cover classification with suitable data type and dataset on Google Earth Engine
Sun, J., & Ongsomwang, S. (2023) · Frontiers in Earth Science · doi:10.3389/feart.2023.1188093 · Cited by 32
Breakthrough Solution for Antimicrobial Resistance Detection: Surface‐Enhanced Raman Spectroscopy‐based on Artificial Intelligence
Al‐Shaebi, Z., Akdeniz, M., Ahmed, A. O., Altunbek, M., & Aydin, Ö. F. (2023) · Advanced Materials Interfaces · doi:10.1002/admi.202300664 · Cited by 33
Analysis of the Use of Artificial Intelligence in Software-Defined Intelligent Networks: A Survey
Cifuentes, B. J. O., Suárez, Á., Pineda, V. G., Jaimes, R. A., Benitez, A. O. M., & Bustamante, J. D. G. (2024) · Technologies · doi:10.3390/technologies12070099 · Cited by 32
“The proposal of deep learning (DL) algorithms not only triggered a boom in DL research and applications but also marked the entry of the third wave of AI to date [54].”
Background
Comparative evaluation of data imbalance addressing techniques for CNN-based insider threat detection
Al‐Shehari, T., Kadrie, M., Al‐Mhiqani, M. N., Alfakih, T. M., AlSalman, H., Uddin, M., Ullah, S. S., & Dandoush, A. (2024) · Scientific Reports · doi:10.1038/s41598-024-73510-9 · Cited by 32
FedLSTM: A Federated Learning Framework for Sensor Fault Detection in Wireless Sensor Networks
Khan, R., Saeed, U., & Koo, I. (2024) · Electronics · doi:10.3390/electronics13244907 · Cited by 32
“In highly complex environments with massive amounts of data, DL models generally outperform ML models in terms of accuracy and performance, although they require greater computational power [7].”
Machine learning methods for detecting smart contracts vulnerabilities within Ethereum blockchain − A review
Crisostomo, J., Bação, F., & Lobo, V. J. A. S. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.126353 · Cited by 32
Advancements in Deep Learning Theory and Applications: Perspective in 2020 and beyond
Saadat, M. N., & Shuaib, M. (2020) · IntechOpen eBooks · doi:10.5772/intechopen.92271 · Cited by 31
Data centre day-ahead energy demand prediction and energy dispatch with solar PV integration
Ajayi, O., & Heymann, R. (2021) · Energy Reports · doi:10.1016/j.egyr.2021.06.062 · Cited by 32
Using scientific machine learning for experimental bifurcation analysis of dynamic systems
Beregi, S., Barton, D. A. W., Rezgui, D., & Neild, S. A. (2022) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2022.109649 · Cited by 33
Deep learning in different remote sensing image categories and applications: status and prospects
Bai, Y., Zhao, Y., Shao, Y., Zhang, X., & Yuan, X. (2022) · International Journal of Remote Sensing · doi:10.1080/01431161.2022.2048319 · Cited by 34
“Many articles have introduced and analyzed various deep learning models in detail (Shrestha and Mahmood 2019; Upadhya and Sastry 2019; Mu and Zeng 2019; Khamparia and Singh 2019; Alom et al. 2019).”
Background
Geological Mapping Using Direct Sampling and a Convolutional Neural Network Based on Geochemical Survey Data
Wang, Z., Zuo, R., & Yang, F. (2022) · Mathematical Geosciences · doi:10.1007/s11004-022-10023-z · Cited by 33
“CNN is a recently developed deep learning algorithm designed for computer vision and pattern recognition applications (Lecun and Bengio 1995; Voulodimos et al. 2018; Shrestha and Mahmood 2019; Chai et al. 2021).”
On the Relative Impact of Optimizers on Convolutional Neural Networks with Varying Depth and Width for Image Classification
Dogo, E. M., Afolabi, O. J., & Twala, B. (2022) · Applied Sciences · doi:10.3390/app122311976 · Cited by 48
A Rapid Self-Supervised Deep-Learning-Based Method for Post-Earthquake Damage Detection Using UAV Data (Case Study: Sarpol-e Zahab, Iran)
Takhtkeshha, N., Mohammadzadeh, A., & Salehi, B. (2022) · Remote Sensing · doi:10.3390/rs15010123 · Cited by 32
Integration of Data Science and IoT with Blockchain for Industry 4.0
Gangwani, P., Perez-Pons, A., Joshi, S., Upadhyay, H., & Lagos, L. E. (2023) · Studies in big data · doi:10.1007/978-981-19-8730-4_6 · Cited by 32
EDUBOT-A Chatbot For Education in Covid-19 Pandemic and VQAbot Comparison
Sophia, J. J., & Jacob, T. P. (2021) · International Conference on Electronics and Sustainable Communication Systems (ICESC) · doi:10.1109/icesc51422.2021.9532611 · Cited by 31
“[5] Ajay Shrestha And Ausif Mahmood, “Review of Deep Learning Algorithms and Architectures”, IEEE Access, 2019, doi: 10.”
Background
Tree Species Classification Based on PointNet++ and Airborne Laser Survey Point Cloud Data Enhancement
Fan, Z., Wei, J., Zhang, R., & Zhang, W. (2023) · Forests · doi:10.3390/f14061246 · Cited by 31
A Study of Optimization in Deep Neural Networks for Regression
Chen, C.-H., Lai, J.-P., Chang, Y.-M., Lai, C.-J., & Pai, P. (2023) · Electronics · doi:10.3390/electronics12143071 · Cited by 32
“Furthermore, according to a number of studies [9,17,26,40,56,60,61,76,78,81,84,98–100], Figure 6 presents an overall optimization procedure for DNNR models.”
Controllable Data Generation by Deep Learning: A Review
Wang, S.-Y., Du, Y., Guo, X., Pan, B., Qin, Z. S., & Zhao, L. (2024) · ACM Computing Surveys · doi:10.1145/3648609 · Cited by 48
Bridging prediction and decision: Advances and challenges in data-driven optimization
Wang, Y., Wang, J., Zhang, H., & Song, J. (2025) · Nexus · doi:10.1016/j.ynexs.2025.100057 · Cited by 39
Lion Algorithm-Optimized Long Short-Term Memory Network for Groundwater Level Forecasting in Udupi District, India
Supreetha, B. S., Shenoy, N. K., & Nayak, P. K. (2020) · Applied Computational Intelligence and Soft Computing · doi:10.1155/2020/8685724 · Cited by 36
Optimization of Deep Architectures for EEG Signal Classification: An AutoML Approach Using Evolutionary Algorithms
Aquino-Brítez, D., Ortíz, A., Ortega, J., León, J., Formoso, M. A., Gan, J. Q., & Escobar, J. J. (2021) · Sensors · doi:10.3390/s21062096 · Cited by 35
“algorithms have been developed to solve specific problems [39].”
BackgroundMethod
Memristor-CMOS Hybrid Neuron Circuit with Nonideal-Effect Correction Related to Parasitic Resistance for Binary-Memristor-Crossbar Neural Networks
Nguyen, T. V., An, J., & Min, K. (2021) · Micromachines · doi:10.3390/mi12070791 · Cited by 31
Ultrasonic based concrete defects identification via wavelet packet transform and GA-BP neural network
Hu, T., Zhao, J., Zheng, R., Wang, P., Li, X., & Zhang, Q. (2021) · PeerJ Computer Science · doi:10.7717/peerj-cs.635 · Cited by 31
Kinome-wide polypharmacology profiling of small molecules by multi-task graph isomorphism network approach
Bao, L., Wang, Z., Wu, Z., Luo, H., Yu, J., Kang, Y., Cao, D., & Hou, T. (2022) · Acta Pharmaceutica Sinica B · doi:10.1016/j.apsb.2022.05.004 · Cited by 32
“…51 , single-task extreme gradient boosting (ST-XGBoost) 52 , single-task Deep Neural Networks (ST-DNN) and multi-task Deep Neural Networks (MT-DNN) 38 , and the GNN-based models include single-task Graph Isomorphism Network (ST-GIN) 36 , multi-task Graph Attention network (MT-GAT) 53 , multi-task…”
MethodInfluential
Framework for COVID-19 segmentation and classification based on deep learning of computed tomography lung images
Salama, W. M., & Aly, M. H. (2022) · Journal of Electronic Science and Technology · doi:10.1016/j.jnlest.2022.100161 · Cited by 31
“Impressive results have been achieved by using the deep learning algorithm [8] in the biomedical ﬁ eld, especially the deep convolutional neural networks (CNNs).”
DCAE: A dual conditional autoencoder framework for the reconstruction from EEG into image
Zeng, H., Xia, N., Tao, M., Pan, D., Zheng, H., Wang, C., Xu, F., Zakaria, W., & Dai, G. (2022) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2022.104440 · Cited by 34
Indentation Reverse Algorithm of Mechanical Response for Elastoplastic Coatings Based on LSTM Deep Learning
Long, X., Ding, X., Li, J., Dong, R., Su, Y., & Chang, C. (2023) · Materials · doi:10.3390/ma16072617 · Cited by 31
Building Socially Intelligent AI Systems: Evidence from the Trust Game Using Artificial Agents with Deep Learning
Wu, J. X., Wu, Y. (., Chen, K., & Hua, L. (2023) · Management Science · doi:10.1287/mnsc.2023.4782 · Cited by 33
Image Classification and Restoration of Ancient Textiles Based on Convolutional Neural Network
Sha, S., Li, Y., Wei, W., Liu, Y., Chi, C., Jiang, X., Deng, Z., & Luo, L. (2024) · International Journal of Computational Intelligence Systems · doi:10.1007/s44196-023-00381-9 · Cited by 30
Data-driven initial peak crushing force prediction of hybrid tubes
Liang, H., Lu, C., Pu, Y., Liu, B., & Wang, D. (2024) · International Journal of Mechanical Sciences · doi:10.1016/j.ijmecsci.2024.109141 · Cited by 30
Deep neural networks for speech enhancement and speech recognition: A systematic review
Natarajan, S., Al-Haddad, S. A. R., Ahmad, F. A., Kamil, R., Hassan, M. K., Azrad, S., Macleans, J. F., Abdulhussain, S. H., Mahmmod, B. M., Saparkhojayev, N. P., et al. (2025) · Ain Shams Engineering Journal · doi:10.1016/j.asej.2025.103405 · Cited by 36
AI-Driven predicting and optimizing lignocellulosic sisal fiber-reinforced lightweight foamed concrete: A machine learning and metaheuristic approach for sustainable construction
Sahraoui, M., Laouissi, A., Karmi, Y., Abderazek, H., Hani, M., Chetbani, Y., Belaadi, A., Alshaikh, I. M. H., & Ghernaout, D. (2025) · Results in Engineering · doi:10.1016/j.rineng.2025.105561 · Cited by 31
A survey of machine learning and deep learning methods for vibration-based Bearing fault diagnosis: The need, challenges, and potential future research directions
Puntambekar, R., Vyas, P., Thakkar, A., & Patel, D. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.131628 · Cited by 34
Manufacturing resilience and agility through processes digital twin: design and testing applied in the LPBF case
Papacharalampopoulos, A., Michail, C. K., & Stavropoulos, P. G. (2021) · Procedia CIRP · doi:10.1016/j.procir.2021.10.026 · Cited by 31
A survey on deep learning based sentiment analysis
Joseph, J., Vineetha, S., & Sobhana, N. V. (2022) · Materials Today Proceedings · doi:10.1016/j.matpr.2022.02.483 · Cited by 30
Capturing Eating Behavior from Video Analysis: A Systematic Review
Tufano, M., Lasschuijt, M. P., Chauhan, A., Feskens, E. J. M., & Camps, G. (2022) · Nutrients · doi:10.3390/nu14224847 · Cited by 30
“A multitude of deep neural networks have been proposed in literature which are distinct in design and architecture [53–55].”
Background
An Advanced Decision Tree-Based Deep Neural Network in Nonlinear Data Classification
Arifuzzaman, M., Hasan, M. R., Toma, T. J., Hassan, S. B., & Paul, A. K. (2023) · Technologies · doi:10.3390/technologies11010024 · Cited by 33
Embedded real-time objects’ hardness classification for robotic grippers
Amin, Y., Gianoglio, C., & Valle, M. (2023) · Future Generation Computer Systems · doi:10.1016/j.future.2023.06.002 · Cited by 30
Towards deep learning based smart farming for intelligent weeds management in crops
Saqib, M. A., Aqib, M., Tahir, M. N., & Hafeez, Y. (2023) · Frontiers in Plant Science · doi:10.3389/fpls.2023.1211235 · Cited by 30
“They are mainly built on neural networks and are well known for pattern recognition in image (Szegedy et al., 2015; Guo et al., 2016; Shrestha and Mahmood, 2019).”
A comprehensive health assessment approach using ensemble deep learning model for remote patient monitoring with IoT
Gayathri, R., Maheswari, S. U., Mathivanan, S. K., Shivahare, B. D., Chandan, R. R., & Shah, M. A. (2024) · Scientific Reports · doi:10.1038/s41598-024-66427-w · Cited by 30
Fatigue Crack Evaluation with the Guided Wave–Convolutional Neural Network Ensemble and Differential Wavelet Spectrogram
Chen, J., Wu, W., Ren, Y., & Yuan, S. (2021) · Sensors · doi:10.3390/s22010307 · Cited by 29
“Recently, deep learning has already made a huge impact in areas such as cancer diagnosis, precision medicine, self-driving cars, predictive forecasting, and speech recognition [21].”
Background
Chatbots Development Using Natural Language Processing: A Review
Abdulla, H. M. D., Eltahir, A. M., Alwahaishi, S., Saghair, K., Platoš, J., & Snåšel, V. (2022) · International Conference on Circuits, Systems, Communications and Computers (CSCC) · doi:10.1109/cscc55931.2022.00030 · Cited by 38
“In this paper the authors, Shrestha, A. and Mahmood, A. [15], showed that most of natural language processing technologies are powered by deep learning which is an area of machine learning.”
Background
Application of Computational Intelligence Methods in Agricultural Soil–Machine Interaction: A Review
Badgujar, C., Figueroa, D. M., Das, S., & Flippo, D. K. (2023) · Agriculture · doi:10.3390/agriculture13020357 · Cited by 30
Evaluation of hydraulic fracturing of horizontal wells in tight reservoirs based on the deep neural network with physical constraints
Qu, H.-Y., Zhang, J., Zhou, F., Peng, Y., Pan, Z., & Wu, X.-Y. (2023) · Petroleum Science · doi:10.1016/j.petsci.2023.03.015 · Cited by 29
“The Deep Neural Network (DNN) was developed on the basis of Arti ﬁ cial Neural Network (ANN) (Shrestha and Mahmood, 2019).”
End-to-End Regression Neural Network for Coherent DOA Estimation With Dual-Branch Outputs
Xu, S., Wang, Z., Zhang, W., & He, Z. (2023) · IEEE Sensors Journal · doi:10.1109/jsen.2023.3342796 · Cited by 29
“Neural networks have achieved remarkable success in various signal processing domains, including image processing, speech recognition, and natural language processing [19], [20], [21], [22], [23], [24], [25], [26].”
Background
More eyes on the road: Sensing flooded roads by fusing real-time observations from public data sources
Panakkal, P., & Padgett, J. E. (2024) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2024.110368 · Cited by 30
Fast-training feedforward neural network for multi-scale power quality monitoring in power systems with distributed generation sources
Robles, O. I. C., Barocio, E., Obushevs, A., Korba, P., & Sevilla, F. R. S. (2020) · Measurement · doi:10.1016/j.measurement.2020.108690 · Cited by 29
A Novel Deep Similarity Learning Approach to Electronic Health Records Data
Gupta, V., Sachdeva, S., & Bhalla, S. (2020) · IEEE Access · doi:10.1109/access.2020.3037710 · Cited by 34
“Drug discovery, precision medication, medical imaging, and disease prediction is being actively applied in this domain [15].”
Background
WMNet: A Lossless Watermarking Technique Using Deep Learning for Medical Image Authentication
Chen, Y.-P., Fan, T., & Chao, H.-C. (2021) · Electronics · doi:10.3390/electronics10080932 · Cited by 29
“In recent years, the most popular network architecture for deep learning for images is the CNN and shows very good performance in the application of deep learning [26,27].”
Automated psoriasis lesion segmentation from unconstrained environment using residual U-Net with transfer learning
Raj, R., Londhe, N. D., & Sonawane, R. S. (2021) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2021.106123 · Cited by 33
Advances in artificial intelligence and digital twin for tunnel boring machines
Liu, G., Tang, Y., Zhang, H., Li, R., Wang, H., Liu, B., Zhang, S., Zhu, H., Liu, D., & Ma, S. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11261-3 · Cited by 33
A review of machine learning in hyperspectral imaging for food safety
Das, M., Yeo, W. S., & Saptoro, A. (2025) · Vibrational Spectroscopy · doi:10.1016/j.vibspec.2025.103828 · Cited by 28
Uncertainty Quantification in Machine Learning Modeling for Multi-Step Time Series Forecasting: Example of Recurrent Neural Networks in Discharge Simulations
Song, T., Ding, W., Liu, H., Wu, J., Zhou, H., & Chu, J. (2020) · Water · doi:10.3390/w12030912 · Cited by 32
“Recently, Machine Learning (ML) techniques have obtained growing attention and Deep Learning (DL) techniques have led to substantial changes across many areas of study [1].”
Background
LSTM-RNN Based Approach for Prediction of Dengue Cases in India
Doni, A. R., & Sasipraba, T. (2020) · Ingénierie des systèmes d information · doi:10.18280/isi.250306 · Cited by 32
“The deep learning has its impact in medical field, automobile industry, business forecasting/prediction, computer vision, video classification, pattern recognition, natural language processing, clustering, regression and voice recognition [3].”
Background
An automatic framework for endoscopic image restoration and enhancement
Asif, M., Chen, L., Song, H., Yang, J., & Frangi, A. F. (2020) · Applied Intelligence · doi:10.1007/s10489-020-01923-w · Cited by 28
Unsupervised Adaptive Weight Pruning for Energy-Efficient Neuromorphic Systems
Guo, W., Fouda, M. E., Yantır, H. E., Eltawil, A. M., & Salama, K. N. (2020) · Frontiers in Neuroscience · doi:10.3389/fnins.2020.598876 · Cited by 27
“Neural networks require deep and complex structures to tackle real-world tasks, like pattern recognition, object detection, and motor controls (Shrestha and Mahmood, 2019).”
Background
Feature selection method based on quantum inspired genetic algorithm for Arabic signature verification
Abdulhussien, A. A., Nasrudin, M. F., Darwish, S. M., & Alyasseri, Z. A. A. (2023) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2023.02.005 · Cited by 28
“As the complexity of the issue increases, both the tournament collection and the proportional roulette wheel become more likely to converge early(Razali and Geraghty, 2011; Shrestha and Mahmood, 2019).”
Object Detection of Small Insects in Time-Lapse Camera Recordings
Bjerge, K., Frigaard, C. E., & Karstoft, H. (2023) · Sensors · doi:10.3390/s23167242 · Cited by 28
Artificial intelligence in cardiac computed tomography
Aromiwura, A. A., Settle, T., Umer, M., Joshi, J. K., Shotwell, M., Mattumpuram, J., Vorla, M., Sztukowska, M. N., Contractor, S. G., Amini, A. A., et al. (2023) · Progress in Cardiovascular Diseases · doi:10.1016/j.pcad.2023.09.001 · Cited by 29
Performance Analysis on Artificial Neural Network Based State of Charge Estimation for Electric Vehicles
Manoharan, A., Begam, M., Aparow, V. R., & Sooriamoorthy, D. (2021) · IEEE International Conference on Internet of Things and Intelligence Systems (IoTaIS) · doi:10.1109/iotais53735.2021.9628725 · Cited by 28
A Survey of Deep Q-Networks used for Reinforcement Learning: State of the Art
Hafiz, A. M. (2022) · Lecture notes on data engineering and communications technologies · doi:10.1007/978-981-19-1844-5_30 · Cited by 28
AI-assisted food enzymes design and engineering: a critical review
Wang, X., Yang, P., Zhao, B., & Liu, S. (2022) · Systems Microbiology and Biomanufacturing · doi:10.1007/s43393-022-00138-z · Cited by 29
Improvement of Smart Grid Stability Based on Artificial Intelligence with Fusion Methods
Alaerjan, A., Jabeur, R., Chikha, H. B., Karray, M., & Ksantini, M. (2024) · Symmetry · doi:10.3390/sym16040459 · Cited by 27
“The input and output layers are considered the hidden layers [38].”
Intelligent Sensing: The Emerging Integration of Machine Learning and Soft Sensors Based on Hydrogels and Ionogels
He, W., Lin, R., Kong, S., Qiang, M., Huang, L., Dai, B., Yao, X., Su, L., & Zhang, X. (2025) · Advanced Science · doi:10.1002/advs.202517851 · Cited by 28
Vehicle-Related Scene Understanding Using Deep Learning
Liu, X., Neuyen, M., & Yan, W. Q. (2020) · Communications in computer and information science · doi:10.1007/978-981-15-3651-9_7 · Cited by 26
“The feedback operation exploits a backpropagation algorithm (Rumelhart, Hinton & Williams, 1986) to minimize the error from the last layer to achieve the purpose of updating the parameters of each layer (Shrestha & Mahmood, 2019).”
Method
A Lightweight CNN Model for Refining Moving Vehicle Detection From Satellite Videos
Chen, R., Li, X., & Li, S. (2020) · IEEE Access · doi:10.1109/access.2020.3040977 · Cited by 28
“Deep Learning (DL) has exhibited a powerful ability in many research areas [25]–[27].”
Background
Deep Learning Based on CNN for Emotion Recognition Using EEG Signal
Ahmad, I. S., Zhang, S., Saminu, S., Wang, L., Isselmou, A. E. K., Cai, Z., Javaid, I., Kamhi, S., & Kulsum, U. (2021) · WSEAS Transactions on Signal Processing archive · doi:10.37394/232014.2021.17.4 · Cited by 28
Fusion of handcrafted and deep convolutional neural network features for effective identification of diabetic foot ulcer
Das, S. K., Roy, P., & Mishra, A. K. (2021) · Concurrency and Computation Practice and Experience · doi:10.1002/cpe.6690 · Cited by 27
A Comprehensive Review of Android Security: Threats, Vulnerabilities, Malware Detection, and Analysis
Acharya, S., Rawat, U., & Bhatnagar, R. (2022) · Security and Communication Networks · doi:10.1155/2022/7775917 · Cited by 27
Retracted
Single-channel speech enhancement using implicit Wiener filter for high-quality speech communication
Jaiswal, R. K., Yeduri, S. R., & Cenkeramaddi, L. R. (2022) · International Journal of Speech Technology · doi:10.1007/s10772-022-09987-4 · Cited by 27
“The proposed network in Yuan (2020) works on time-frequency correlation in the improved minima controlled recursive averaging (MCRA)-based feature calculation using long short-term memory (LSTM) and convolutional neural network (CNN) (Shrestha and Mahmood, 2019).”
Background
A review on self-adaptation approaches and techniques in medical image denoising algorithms
Kulathilake, K. A. S. H., Abdullah, N. A., Sabri, A. Q. M., Bandara, A. M. R. R., & Lai, K. W. (2022) · Multimedia Tools and Applications · doi:10.1007/s11042-022-13511-w · Cited by 27
“In training, autoencoders regenerate the input data itself by using the backpropagation algorithm [100].”
Method
A systematic review of intelligent tutoring systems based on Gross body movement detected using computer vision
Ashwin, T. S., Prakash, V., & Rajendran, R. (2023) · Computers and Education Artificial Intelligence · doi:10.1016/j.caeai.2023.100125 · Cited by 33
Interpretable intelligent fault diagnosis strategy for fixed-wing UAV elevator fault diagnosis based on improved cross entropy loss
Li, Y., Jia, Z. Y., Liu, Z., Shao, H., Zhao, W. D., Liu, Z., & Wang, B. (2024) · Measurement Science and Technology · doi:10.1088/1361-6501/ad3666 · Cited by 27
“Transformer [27] is a deep model proposed by Vaswani et al and widely used in research areas such as natural language processing and image analysis.”
A deep learning algorithm for detection of potassium deficiency in a red grapevine and spraying actuation using a raspberry pi3
Ukaegbu, U. F., Tartibu, L. K., Laseinde, O. T., Okwu, M. O., & Olayode, I. O. (2020) · International Conference on Artificial Intelligence, Big Data, Computing and Data Communication Systems (icABCD) · doi:10.1109/icabcd49160.2020.9183810 · Cited by 27
Automated Detection and Localization of Synaptic Vesicles in Electron Microscopy Images
Imbrosci, B., Schmitz, D., & Orlando, M. (2022) · eNeuro · doi:10.1523/eneuro.0400-20.2021 · Cited by 26
“They contribute to the extraordinary advances of many digital applications, such as automatic speech recognition, natural language processing, object recognition, and cancer diagnosis just to mention a few (Shrestha and Mahmood, 2019).”
Background
An integrative review exploring the development of sustainable product design in the technological context of Industry 4.0
Chou, J.-R. (2024) · Advanced Engineering Informatics · doi:10.1016/j.aei.2024.102689 · Cited by 27

Authors: Shrestha, A., & Mahmood, A.
Affiliations: University of Bridgeport (US)
ORCID: Shrestha, Ajay
Mahmood, Ausif
Venue: IEEE Access
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Published: 2019
Volume: 7
Pages: 53040-53065
ISSN: 2169-3536 (ISSN-L)
DOI: 10.1109/access.2019.2912200
OpenAlex: W2942231644
Cited by: 1,883
References: 88
Citations by year: 2026: 116 · 2025: 317 · 2024: 360 · 2023: 375 · 2022: 297 · 2021: 259 · 2020: 139 · 2019: 20
FWCI: 81.55
Type: journal article
Language: en
Open access: gold
doi.org
Licence: ieeexplore.ieee.org/Xplorehelp/downloads/license-information/OAPA.html (vor)
Locations: IEEE Access (publishedVersion)
University of Bridgeport ScholarWorks (University of Bridgeport) (submittedVersion)
DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics: Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
Keywords: deep learning, deep learning architectures, deep convolutional neural networks, ResNet, recurrent neural networks, reinforcement learning, variational autoencoder, feature extraction, hierarchical representation, training time reduction

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Review of Deep Learning Algorithms and Architectures

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks