Cited by 1,883Neural Networks and ApplicationsAdvanced Neural Network ApplicationsStochastic Gradient Optimization Techniques

The DOI matches its Crossref record; 85 of the 88 references it cites were found in a registry or an index.

Review of Deep Learning Algorithms and Architectures

Open access

Shrestha, Ajay, Mahmood, Ausif·2019·IEEE Access

DOI 10.1109/access.2019.2912200

Open this paper in CoChat — free account Open PDF

Abstract

Deep learning (DL) is playing an increasingly important role in our lives. It has already made a huge impact in areas, such as cancer diagnosis, precision medicine, self-driving cars, predictive forecasting, and speech recognition. The painstakingly handcrafted feature extractors used in traditional learning, classification, and pattern recognition systems are not scalable for large-sized data sets. In many cases, depending on the problem complexity, DL can also overcome the limitations of earlier shallow networks that prevented efficient training and abstractions of hierarchical representations of multi-dimensional training data. Deep neural network (DNN) uses multiple (deep) layers of units with highly optimized algorithms and architectures. This paper reviews several optimization methods to improve the accuracy of the training and to reduce training time. We delve into the math behind training algorithms used in recent deep networks. We describe current shortcomings, enhancements, and implementations. The review also covers different types of deep architectures, such as deep convolution networks, deep residual networks, recurrent neural networks, reinforcement learning, variational autoencoders, and others.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1109/access.2019.2912200

Cites

85 of 85 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 5×
“tion with a 152-layer DNN on the ImageNet dataset [62].”
BackgroundMethodInfluential
Long Short-Term Memory
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 2×
“The input, forget and output gates dictates new data going into the cell, what remains in the cell and the cell values used in the calculation of the output of the LSTM block respectively [41], [43].”
Method
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 4×
“Multilayer network training cost on MNIST dataset using different adaptive learning algorithms [58].”
BackgroundMethodInfluential
Deep learning
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 2×
“7-layer architecture of CNN for character recognition [28].”
BackgroundMethod
Adaptation in Natural and Artificial Systems
Holland, J. H. (1992) · The MIT Press eBooks · doi:10.7551/mitpress/1090.001.0001 · Cited by 47,340
Untitled work
(2014) · Cited by 33,419
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818 · cited 2×
“[59] S. Ioffe and C. Szegedy, ‘‘Batch normalization: Accelerating deep network training by reducing internal covariate shift,’’ CoRR, Feb. 2015.”
Background
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545 · cited 1×
“GRUs are smaller in size than LSTM as they don’t include the output gate, and can perform better than LSTM on only some simpler datasets [44], [45].”
Background
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928

Flagged first, then by citations

Show all 88 references

3 of the 88 references couldn’t be matched to a known paper.

Research works printed but not linked: 10

vapnik, 2015, Learning using privileged information: Similarity control and knowledge transfer, J Mach Learn Res, 16, 2023
ng, 2018, Autoencoders
2018, Amazon EC2 P2 & P3 Instances Amazon EC2 Instance Types
simpson, 2015, Uniform learning in a deep neural network via ‘oddball’ stochastic gradient descent, CoRR
minsky, 1969, Perceptrons An Introduction to Computational Geometry Expanded Edition, 258
2019, Top Deep Learning Github Repositories
sastry, 2005, Genetic Algorithms
10.1017/CBO9780511543241 · DOI
wu, 2016, Google’s neural machine translation system: Bridging the gap between human and machine translation, CoRR
metz, 2019, Turing Award Won by 3 Pioneers in Artificial Intelligence, 3b

Sorted by citations

Items 501–600 of 1,001

RTSLPS: Real time server load prediction system for the ever-changing cloud computing environment
Toumi, H., Brahmi, Z., & Gammoudi, M. M. (2019) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2019.12.004 · Cited by 19
“Moreover, DMT and DLT are trained to excel in predeﬁned conditions, thus, they can’t adapt quickly to unpredictable and new circumstances (Moamar, 2016; Shrestha and Mahmood, 2019).”
Deep Learning Based Person Authentication Using Hand Radiographs: A Forensic Approach
Joshi, S. V., & Kanphade, R. D. (2020) · IEEE Access · doi:10.1109/access.2020.2995788 · Cited by 19
“There are different learning algorithms for deep learning, such as Gradient Descent, Stochastic Gradient Descent, Momentum learning, Levenberg–Marquardt algorithm, and Back-propagation through a time learning algorithm [19].”
Method
Review on Supervised and Unsupervised Learning Techniques for Electrical Power Systems: Algorithms and Applications
Chen, S. (2021) · IEEJ Transactions on Electrical and Electronic Engineering · doi:10.1002/tee.23452 · Cited by 26
“As a type of machine learning, unsupervised learning allows model to explore the insight of data and reach certain objectives by itself without guidance [54].”
Background
Beyond the Local Volume. I. Surface Densities of Ultracool Dwarfs in Deep HST/WFC3 Parallel Fields
Aganze, C., Burgasser, A. J., Malkan, M. A., Theissen, C. A., Arevalo, R. A. T., Hsu, C.-C., Gagliuffi, D. C. B., Ryan, R. E., & Holwerda, B. W. (2022) · The Astrophysical Journal · doi:10.3847/1538-4357/ac35ea · Cited by 23
Energy Management Strategy Based on a Novel Speed Prediction Method
Xing, J., Chu, L., Hou, Z., Sun, W. B., & Zhang, Y. (2021) · Sensors · doi:10.3390/s21248273 · Cited by 19
“If the features are not properly selected, there is no guarantee that there is a stable mapping relationship between the input and output of the sample so that the prediction effect is seriously affected [29,30].”
Background
Artificial Intelligence in Fractional-Order Systems Approximation with High Performances: Application in Modelling of an Isotopic Separation Process
Motorga, R.-M., Mureşan, V., Ungureșan, M.-L., Abrudean, M. I., Vălean, H., & Clitan, I. (2022) · Mathematics · doi:10.3390/math10091459 · Cited by 19
Fully Automatic Wheat Disease Detection System by Using Different CNN Models
Kumari, N., & Saini, B. S. (2023) · Advances in intelligent systems and computing · doi:10.1007/978-981-19-5443-6_26 · Cited by 19
Identification of Crown of Thorns Starfish (COTS) using Convolutional Neural Network (CNN) and attention model
Khan, M. H.-M., Makoonlall, A., Nazurally, N., & Mungloo-Dilmohamud, Z. (2023) · PLoS ONE · doi:10.1371/journal.pone.0283121 · Cited by 20
“Deep learning is an efficient approach that is composed of multiple layers that learn to represent data with many levels of abstraction [23].”
Background
Reliability-aware failure recovery for cloud computing based automatic train supervision systems in urban rail transit using deep reinforcement learning
Zhu, L., Zhuang, Q., Jiang, H., Liang, H., Gao, X., & Wang, W. (2023) · Journal of Cloud Computing Advances Systems and Applications · doi:10.1186/s13677-023-00502-x · Cited by 19
Progress on network modeling and analysis of gut microecology: a review
Luo, M., Zhu, J., Jia, J., Zhang, H., & Zhao, J. (2024) · Applied and Environmental Microbiology · doi:10.1128/aem.00092-24 · Cited by 24
“However, it is essential to recognize that RL approaches in microbiome research are still nascent and face challenges, such as the intricate modeling of microbiome-host interactions and the limited availability of large-scale experimental data to train deep RL models (45).”
Background
The application of laser‑induced fluorescence in oil spill detection
Zhang, S., Yuan, Y., Wang, Z., & Li, J. (2024) · Environmental Science and Pollution Research · doi:10.1007/s11356-024-32807-y · Cited by 20
“It is worth noting that one of the advantages of DNNs is their ability to automatically extract data features without relying on manual selection (Cooman et al. 2022; Shrestha & Mahmood 2019), which makes the model’s accuracy independent of previous preprocessing methods.”
Determination and investigation of shear wave velocity based on one deep/machine learning technique
Hazbeh, O., Rajabi, M., Tabasi, S., Lajmorak, S., Ghorbani, H., Radwan, A. E., Alvar, M. A., & Molaei, O. (2024) · Alexandria Engineering Journal · doi:10.1016/j.aej.2024.03.007 · Cited by 21
Fundamentals and recent developments of free-space optical neural networks
McNeil, A. M., Li, Y., Zhang, A., Moebius, M. G., & Liu, Y. (2024) · Journal of Applied Physics · doi:10.1063/5.0215752 · Cited by 21
An improved EnlightenGAN shadow removal framework for images of cracked concrete
Sun, R., Li, X., Law, S., Zhang, L., Hu, L.-Z., & Liu, G. (2024) · Mechanical systems and signal processing · doi:10.1016/j.ymssp.2024.111943 · Cited by 18
Countering Social Media Cybercrime Using Deep Learning: Instagram Fake Accounts Detection
Alharbi, N. H., Alkalifah, B., Alqarawi, G., & Rassam, M. A. (2024) · Future Internet · doi:10.3390/fi16100367 · Cited by 25
State of machine and deep learning in histopathological applications in digestive diseases
Kobayashi, S., Saltz, J. H., & Yang, V. W. (2021) · World Journal of Gastroenterology · doi:10.3748/wjg.v27.i20.2545 · Cited by 20
Estimating heavy metals absorption efficiency in an aqueous solution using nanotube-type halloysite from weathered pegmatites and a novel Harris hawks optimization-based multiple layers perceptron neural network
Bac, B. H., Nguyen, H., Thảo, N. T., Hạnh, V. T., Duyên, L. T., Dũng, N. T., Du, N. K., & Hiệp, N. H. (2021) · Engineering With Computers · doi:10.1007/s00366-021-01459-8 · Cited by 19
Machinery condition monitoring in the era of industry 4.0: A relative degree of contribution feature selection and deep residual network combined approach
Mei, S., Mei, Y., Cui, J. F., Dong, S., & Zhao, J. (2022) · Computers & Industrial Engineering · doi:10.1016/j.cie.2022.108129 · Cited by 20
Advancing Laboratory Medicine Practice With Machine Learning: Swift yet Exact
You, J., Seok, H. S., Kim, S., & Shin, H. (2024) · Annals of Laboratory Medicine · doi:10.3343/alm.2024.0354 · Cited by 22
“A CNN is an artificial neural network containing convolutional layers and is typically utilized for image or sequence data processing because it can capture spatiotemporal features [64-67].”
Advancement of machine learning algorithms in biosensors.
Chin, W.-J., Lim, W., Khor, S. M., Ramakrishnan, N., Chee, P.-S., & Goh, C.-H. (2025) · Clinica chimica acta; international journal of clinical chemistry · doi:10.1016/j.cca.2025.120677 · Cited by 20
End-to-End Deep Learning Model for Steering Angle Control of Autonomous Vehicles
Khanum, A., Lee, C., & Yang, C. (2020) · International Symposium on Computer, Consumer and Control · doi:10.1109/is3c50286.2020.00056 · Cited by 18
“This study author [4] evaluated more than a few optimizations techniques to improve the accuracy of the training and to reduce training time.”
Method
Flexible and Robust Real-Time Intrusion Detection Systems to Network Dynamics
Yu, K., Nguyen, K., & Park, Y. (2022) · IEEE Access · doi:10.1109/access.2022.3199375 · Cited by 21
Inverse design in flat optics
Gao, Y., Chen, Q., Pian, S., & Ma, Y. (2022) · Photonics and Nanostructures - Fundamentals and Applications · doi:10.1016/j.photonics.2022.101074 · Cited by 18
Classification of liquid crystal textures using convolutional neural networks
Dierking, I., Dominguez, J., Harbon, J., & Heaton, J. (2022) · Liquid Crystals · doi:10.1080/02678292.2022.2150790 · Cited by 18
“With the recent explosion in availability of such data and sophisticated computing technology, deep learning has seen a surge in interest and application among several fields [3].”
Background
An End-to-End Steel Surface Classification Approach Based on EDCGAN and MobileNet V2
Jin, G., Liu, Y., Qin, P., Hong, R., Xu, T., & Lu, G. (2023) · Sensors · doi:10.3390/s23041953 · Cited by 19
“This method builds multiple layers of neural network models, to continuously adjust model parameters in the training process, and finally fits the task [10].”
Method
Statistical Process Monitoring of Artificial Neural Networks
Malinovskaya, A., Mozharovskyi, P., & Otto, P. (2023) · Technometrics · doi:10.1080/00401706.2023.2239886 · Cited by 21
“For instance, convolutional ANNs enable image data processing, while recurrent ANNs are suitable for working with temporal sequences (see Shrestha and Mahmood 2019).”
Method
Novel deep learning models with novel integrated activation functions for autism screening: AutiNet and MinAutiNet
Kaviya Elakkiya, M., & Dejey (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.122102 · Cited by 21
Delamination Depth Detection in Composite Plates Using the Lamb Wave Technique Based on Convolutional Neural Networks
Migot, A., Saaudi, A., & Victor Giurgiutiu, B. (2024) · Sensors · doi:10.3390/s24103118 · Cited by 21
“The state-of-the-art machine learning techniques are deep learning (DL) algorithms [27].”
BackgroundMethod
Prediction of Protein–Protein Interactions Based on Integrating Deep Learning and Feature Fusion
Tran, H.-N., Nguyen, P.-X.-Q., Guo, F., & Wang, J. (2024) · International Journal of Molecular Sciences · doi:10.3390/ijms25115820 · Cited by 18
“MLPs consist of multiple layers of neurons paired with non-linear activation functions, allowing them to learn complex patterns [50].”
Background
Electric load forecasting under false data injection attacks via denoising deep learning and generative adversarial networks
Mahmoudnezhad, F., Moradzadeh, A., Mohammadi‐Ivatloo, B., Zare, K., & Ghorbani, R. (2024) · IET Generation Transmission & Distribution · doi:10.1049/gtd2.13273 · Cited by 19
“Autoencoder (AE), as a speciﬁc type of feedforward neural network, aims to derive an effective and robust representation of the training dataset in an unsupervised manner and is employed for data or measurements reconstruction, feature extraction [45].”
RDG-TE: Link reliability-aware DRL-GNN-based traffic engineering in SDN
Farhan, M., Shah, N., Wang, L., Muntean, G., & Song, H. H. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.125963 · Cited by 18
Recent trends on mammogram breast density analysis using deep learning models: neoteric review
Idas, S. J. P., Hemalatha, K., Naveenkumar, J., & Devadas, T. J. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11232-8 · Cited by 18
“The average pooling layer, on the other hand, displays the average of all the values from the region of the image that the kernel has covered (Shrestha and Mahmood 2019 ).”
Multi-scale decomposition based supervised single channel deep speech enhancement
Saleem, N., & Khattak, M. I. (2020) · Applied Soft Computing · doi:10.1016/j.asoc.2020.106666 · Cited by 18
Automated Amharic Hate Speech Posts and Comments Detection Model Using Recurrent Neural NetworkPreprint
Tesfaye, S. G., & Kakeba, K. (2020) · Research Square · doi:10.21203/rs.3.rs-114533/v1 · Cited by 18
Application of Machine Learning and Deep Learning Algorithms for Landslide Susceptibility Assessment in Landslide Prone Himalayan Region
Bhattacharya, S., Ali, T., Chakravortti, S., Pal, T., Majee, B. K., Mondal, A., Pande, C. B., Bilal, M., Rahman, M. T., & Chakrabortty, R. (2024) · Earth Systems and Environment · doi:10.1007/s41748-024-00530-w · Cited by 20
Data-driven deep learning models for predicting off-axis tensile damage of 2.5D woven composites at elevated temperatures
Zhang, C., Bian, Z., Chen, T., Bui, T. Q., Curiel-Sosa, J. L., & Mao, C. (2025) · Thin-Walled Structures · doi:10.1016/j.tws.2025.112944 · Cited by 17
Supervised Learning-Based Fault Classification in Industrial Rotating Equipment Using Multi-Sensor Data
Ovacıklı, A. K., Yağcıoğlu, M., Demircioğlu, S., Kocatekin, T., & Birtane, S. (2025) · Applied Sciences · doi:10.3390/app15137580 · Cited by 22
Comparative analysis of supervised and self-supervised learning with small and imbalanced medical imaging datasets
Espis, A., Marzi, C., & Diciotti, S. (2025) · Scientific Reports · doi:10.1038/s41598-025-99000-0 · Cited by 20
“Figures 3, 4, 5, and 6 represent the mean performance and variability (where applicable) of the learning paradigms on the test sets.”
Estimating the Prognosis of Low-Grade Glioma with Gene Attention Using Multi-Omics and Multi-Modal Schemes
Choi, S. R., & Lee, M. (2022) · Biology · doi:10.3390/biology11101462 · Cited by 17
“The layer receives input from the last layer and multiplies the input and weight matrix [40].”
Background
Artificial intelligence for template-free protein structure prediction: a comprehensive review
Mufassirin, M. M. M., Newton, M. A. H., & Sattar, A. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-022-10350-x · Cited by 19
The Geometry of Feature Space in Deep Learning Models: A Holistic Perspective and Comprehensive Review
Lee, M. (2023) · Mathematics · doi:10.3390/math11102375 · Cited by 21
A machine learning-based colorimetric sensor array for high-precision pathogen identification in household refrigerators
Zhang, Y., Qi, G.-X., Yu, Y., Liu, M.-X., & Chen, S. (2023) · Chemical Communications · doi:10.1039/d3cc01320k · Cited by 17
Application of Image Sensing System in Mineral/Rock Identification: Sensing Mode and Information Process
Zhang, S., Yang, Y., Sun, F., & Fang, B. (2023) · Advanced Intelligent Systems · doi:10.1002/aisy.202300206 · Cited by 17
Deep Learning-Based Multiobjective Optimization for Balancing Effluent Quality, Operational Cost, and Greenhouse Gas Emissions in Wastewater Treatment Plant Control
Lu, H., Meng, Z.-H., Zhang, B., Song, S., Zhan, S.-Y., Li, Y., Wu, Q., Wang, H., & Guo, W.-Q. (2024) · ACS ES&T Water · doi:10.1021/acsestwater.4c00073 · Cited by 17
Disease Detection in Grape Cultivation Using Strategically Placed Cameras and Machine Learning Algorithms With a Focus on Powdery Mildew and Blotches
Khan, K. H., Aljaedi, A., Ishtiaq, M. S., Imam, H., Bassfar, Z., & Jamal, S. S. (2024) · IEEE Access · doi:10.1109/access.2024.3430190 · Cited by 17
“Deep learning operates with neural networks that consist of numerous layers, going beyond the traditional input, output, and hidden layers [18], [19].”
Background
Optimized TD3 algorithm for robust autonomous navigation in crowded and dynamic human-interaction environments
Neamah, H. A., & Mayorga, O. A. (2024) · Results in Engineering · doi:10.1016/j.rineng.2024.102874 · Cited by 17
Forecasting data-driven system strength level for inverter-based resources-integrated weak grid systems using multi-objective machine learning algorithms
Qays, M. O., Ahmad, I., Habibi, D., & Masoum, M. A. S. (2024) · Electric Power Systems Research · doi:10.1016/j.epsr.2024.111112 · Cited by 17
Container Terminal Liner Berthing Time Prediction with Computational Logistics and Deep Learning
Li, B., & He, Y. (2020) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc42975.2020.9282816 · Cited by 17
“Whether in theory or in practice, lots of research progress and landmark achievements have been acquired by deep learning in recent years, especially in terms of classification and prediction [13].”
Background
Ensemble Deep Learning Approaches in Health Care: A Review
Alotaibi, A. (2025) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2025.061998 · Cited by 16
“Reviews of deep learning architectures have been presented in several research papers [2–4].”
Toz yatak füzyon birleştirme eklemeli imalatta kusur tespiti için öğrenme aktarımı kullanan derin öğrenme tabanlı bir yaklaşım
Duman, B., & Özsoy, K. (2021) · Gazi Üniversitesi Mühendislik-Mimarlık Fakültesi Dergisi · doi:10.17341/gazimmfd.870436 · Cited by 17
Recurrent neural network to predict hyperelastic constitutive behaviors of the skeletal muscle
Ballit, A., & Dao, T. T. (2022) · Medical & Biological Engineering & Computing · doi:10.1007/s11517-022-02541-z · Cited by 18
Real-Time Detection of Cyber-Attacks in Modern Power Grids with Uncertainty using Deep Learning
Mohammadpourfard, M., Ghanaatpishe, F., Weng, Y., Genc, I. V. M., & Sandıkkaya, M. T. (2022) · International Conference on Smart Energy Systems and Technologies (SEST) · doi:10.1109/sest53650.2022.9898413 · Cited by 16
“In addition, the LSTM is a time-series method and can perform well in processing the time series and big data [22].”
BackgroundMethod
Improving Streaming Video with Deep Learning-Based Network Throughput Prediction
Biernacki, A. (2022) · Applied Sciences · doi:10.3390/app122010274 · Cited by 16
Classification of Benign and Malignancy in Lung Cancer Using Capsule Networks with Dynamic Routing Algorithm on Computed Tomography Images
Bushara, A. R., S., V. K. R., & Kumar, S. S. (2023) · Journal of Artificial Intelligence and Technology · doi:10.37965/jait.2023.0218 · Cited by 17
Palmprint recognition system based on deep region of interest features with the aid of hybrid approach
Türk, Ö., Çalışkan, A., Acar, E., & Ergen, B. (2023) · Signal Image and Video Processing · doi:10.1007/s11760-023-02612-0 · Cited by 16
“Since deep learning can be considered a subbranch of image processing, it is also very effective in solving image analysis problems [21–24].”
Monthly Runoff Forecasting Using Particle Swarm Optimization Coupled with Flower Pollination Algorithm-Based Deep Belief Networks: A Case Study in the Yalong River Basin
Yue, Z., Liu, H., & Zhou, H. H. (2023) · Water · doi:10.3390/w15152704 · Cited by 16
Implementation of Machine Learning and Deep Learning Techniques for the Detection of Epileptic Seizures Using Intracranial Electroencephalography
Kołodziej, M., Majkowski, A., & Rysz, A. (2023) · Applied Sciences · doi:10.3390/app13158747 · Cited by 19
“Deep learning, as one of the branches of artiﬁcial intelligence, can yield better results than traditional machine learning methods in the case of epileptic seizure detection [101,102].”
Background
Accelerated deep-learning-based process monitoring of microfluidic inkjet printing
Kim, S. J., Choi, E., Won, D. Y., Han, G., Han, G., An, K., & Kim, S. (2023) · CIRP journal of manufacturing science and technology · doi:10.1016/j.cirpj.2023.07.010 · Cited by 16
Advancements in memory technologies for artificial synapses
Sehgal, A., Dhull, S., Roy, S., & Kaushik, B. K. (2024) · Journal of Materials Chemistry C · doi:10.1039/d3tc04131j · Cited by 16
Deep learning-enabled polar code decoders for 5G networks and beyond
Meenalakshmi, M., Chaturvedi, S., & Dwivedi, V. K. (2024) · AEU - International Journal of Electronics and Communications · doi:10.1016/j.aeue.2024.155220 · Cited by 16
Development and Validation of an Explainable Deep Learning Model to Predict In-Hospital Mortality for Patients With Acute Myocardial Infarction: Algorithm Development and Validation Study
Xie, P., Wang, H., Xiao, J., Xu, F., Liu, J., Chen, Z., Zhao, W., Hou, S., Wu, D., Ma, Y., et al. (2024) · Journal of Medical Internet Research · doi:10.2196/49848 · Cited by 17
“Deep learning is the most advanced branch of artificial intelligence, which can automatically learn feature representations from training data and provide new prospects for solving the existing limitations in tree-based models [17].”
Deep learning-based geological map generation using geological routes
Li, C., Li, F., Liu, C., Tang, Z., Fu, S., Lin, M., Lv, X., Liu, S., & Liu, Y. (2024) · Remote Sensing of Environment · doi:10.1016/j.rse.2024.114214 · Cited by 19
Two-stage residual networks for damage identification and location of stiffened composite panel based on guided waves
Tian, T., Yang, L., Liu, W., Yang, Y., Xu, H., Yang, Z., Zhang, J., & Wu, Z. (2024) · NDT & E International · doi:10.1016/j.ndteint.2024.103162 · Cited by 17
Palm Oil Counter: State-of-the-Art Deep Learning Models for Detection and Counting in Plantations
Naftali, M. G., Hugo, G., & Suharjito, S. (2024) · IEEE Access · doi:10.1109/access.2024.3419835 · Cited by 20
“Through Equation (20), we can calculate the mAR by averaging the AR with the number of classes.”
BackgroundMethodInfluential
Harnessing deep reinforcement learning algorithms for image categorization: A multi algorithm approach
Yerramreddy, D. R., Marasani, J., Ponnuru, S. V. G., Min, D., & Don, S. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.108925 · Cited by 16
Digitalised prediction of blue pigment content from Spirulina platensis: Next-generation microalgae bio-molecule detection
Chong, J. W. R., Khoo, K. S., Chew, K. W., Ting, H.-Y., Iwamoto, K., & Show, P. L. (2024) · Algal Research · doi:10.1016/j.algal.2024.103642 · Cited by 17
3-D LiDAR and Monocular Camera Calibration: A Review
Zhang, H., Li, S., Zhu, X., Chen, H., & Yao, W. (2025) · IEEE Sensors Journal · doi:10.1109/jsen.2025.3530925 · Cited by 19
AI-Driven Prognostics for State of Health Prediction in Li-ion Batteries: A Comprehensive Analysis with Validation
Ding, T., Xiang, D., Sun, T., Qi, Y., & Zhao, Z. (2025) · International Conference on Electrical Technology and Automatic Control (ICETAC) · doi:10.1109/icetac65964.2025.11144279 · Cited by 16
A Review of Deep Learning Approaches for Image Analysis
Aslam, Y., & Santhi, N. (2019) · International Conference on Smart Systems and Inventive Technology (ICSSIT) · doi:10.1109/icssit46314.2019.8987922 · Cited by 17
Deep Learning for Assessing Unhealthy Lettuce Hydroponic Using Convolutional Neural Network based on Faster R-CNN with Inception V2
Pratama, I. Y., Wahab, A., & Alaydrus, M. (2020) · International Conference on Informatics and Computing (ICIC) · doi:10.1109/icic50835.2020.9288554 · Cited by 16
“CNN is also used in Natural Language (NL) processing [14].”
Method
Monuments Recognition using Deep Learning VS Machine Learning
Hesham, S., Khaled, R., Yasser, D., Refaat, S., Shorim, N., & Ismail, F. H. (2021) · IEEE Annual Computing and Communication Workshop and Conference (CCWC) · doi:10.1109/ccwc51732.2021.9376029 · Cited by 18
“In this paper, we used the Resnet50 and VGG16 [3] as deep learning algorithms, and we used KNN [4] as the machine learning algorithm.”
Method
The Ethics of Rehabilitation in Virtual Reality: the role of Self-Avatars and Deep Learning
Soccini, A. M., & Cena, F. (2021) · IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR) · doi:10.1109/aivr52153.2021.00068 · Cited by 16
Applications and development of artificial intelligence system from the perspective of system science: A bibliometric review
Jiang, H., Gai, J., Zhao, S., Chaudhry, P. E., & Chaudhry, S. S. (2022) · Systems Research and Behavioral Science · doi:10.1002/sres.2865 · Cited by 18
On Random Matrices Arising in Deep Neural Networks. Gaussian CasePreprint
Pastur, L. A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.06188 · Cited by 28
“Deep neural networks with multiple hidden layers have achieved remarkable performance in a wide variety of domains, see e.g. [2, 3, 4, 9, 25, 27] for reviews.”
Background
Research Challenges, Recent Advances and Benchmark Datasets in Deep-Learning-Based Underwater Marine Object Detection: A ReviewPreprint
Er, M. J., Jie, C., Zhang, Y., & Gao, W. (2022) · doi:10.36227/techrxiv.19350389 · Cited by 16
Machine Learning Methods for Geotechnical Site Characterization and Scour Assessment
Yousefpour, N., Liu, Z., & Zhao, C. (2024) · Transportation Research Record Journal of the Transportation Research Board · doi:10.1177/03611981241257512 · Cited by 19
“DL has revolutionized AI by enabling break-throughs in computer vision, natural language processing, and speech recognition ( 16 ).”
Machine learning algorithms and biomarkers identification for pancreatic cancer diagnosis using multi-omics data integration
Rouzbahani, A. K., Khalili‐Tanha, G., Rajabloo, Y., Khojasteh‐Leylakoohi, F., Garjan, H. S., Nazari, E., & Avan, A. (2024) · Pathology - Research and Practice · doi:10.1016/j.prp.2024.155602 · Cited by 16
Leveraging a deep learning model to improve mid- and long-term operations of hydro-wind-photovoltaic complementary systems
Cheng, Q., Liu, P., Feng, M., Cheng, L., Ming, B., Xie, K., Yang, Z., Zhang, X., Zheng, Y., & Ye, H. (2025) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2025.115986 · Cited by 15
Approximations with deep neural networks in Sobolev time-space
Abdeljawad, A., & Grohs, P. (2022) · Analysis and Applications · doi:10.1142/s0219530522500014 · Cited by 15
Nowcasting of Lumber Futures Price with Google Trends Index Using Machine Learning and Deep Learning Models
He, M., Li, W., Via, B. K., & Zhang, Y. (2021) · Forest Products Journal · doi:10.13073/fpj-d-21-00061 · Cited by 15
Failure mode diagnosis in proton exchange membrane fuel cells using local electrochemical noise
Rubio, M. A., Sánchez, D. G., Gazdzicki, P., Friedrich, K. A., & Urquía, A. (2022) · Journal of Power Sources · doi:10.1016/j.jpowsour.2022.231582 · Cited by 15
Enhancement Digital Forensic Approach for Inter-Frame Video Forgery Detection Using a Deep Learning Technique
Oraibi, M. R., & Radhi, A. M. (2022) · Iraqi Journal of Science · doi:10.24996/ijs.2022.63.6.34 · Cited by 15
“Every layer has units or nodes linked to neighbouring layers [26].”
Background
A deep reinforcement learning method for structural dominant failure modes searching based on self-play strategy
Guan, X., Sun, H., Hou, R., Xu, Y., Bao, Y., & Li, H. (2023) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2023.109093 · Cited by 15
Pilot Maneuvering Performance Analysis and Evaluation with Deep Learning
Zhang, S., Huo, Z., Sun, Y., Li, F., & Jia, B. (2023) · International Journal of Aerospace Engineering · doi:10.1155/2023/6452129 · Cited by 19
Detection of Facial Mask Using Deep Learning Classification Algorithm
Putri, R. K., & Athoillah, M. (2023) · Journal of Data Science and Intelligent Systems · doi:10.47852/bonviewjdsis32021067 · Cited by 16
Detecting the RPL Version Number Attack in IoT Networks using Deep Learning Models
Krari, A., Hajami, A., & Jarmouni, E. (2023) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2023.0141065 · Cited by 15
“This can be useful in a wide range of applications, from forecasting consumer behavior to identifying fraud in financial transactions and detecting cyber-attacks [28-31].”
Background
Marine Equipment Siting Using Machine-Learning-Based Ocean Remote Sensing Data: Current Status and Future Prospects
Zhang, D., Ma, Y., Zhang, H., & Zhang, Y. (2024) · Sustainability · doi:10.3390/su16208889 · Cited by 18
An effective SQL injection detection model using LSTM for imbalanced datasets
Fathi, K. S., Barakat, S., & Rezk, A. (2025) · Computers & Security · doi:10.1016/j.cose.2025.104391 · Cited by 15
Predictions of USA Presidential Parties From 2021 to 2037 Using Historical Data Through Square Wave-Activated WASD Neural Network
Zeng, T., Zhang, Y., Li, Z., Qiu, B., & Ye, C. (2020) · IEEE Access · doi:10.1109/access.2020.2982192 · Cited by 16
“Nowadays, predictions can be generated through machine learning frameworks [13], [14] by learning the historical pattern.”
Background
A Deep Learning-Based Pipeline for Teaching Control Theory: Transforming Feedback Control Systems on Whiteboard Into MATLAB
Erdem, D., Beke, A., & Kumbasar, T. (2020) · IEEE Access · doi:10.1109/access.2020.2992614 · Cited by 17
“For new pattern recognition tasks, the transfer learning approach is widely employed as training of a deep CNNs, like ResNet-50, from scratch is computational expensive and time-consuming [34], [37].”
BackgroundMethodInfluential
Facial Beauty Prediction via Local Feature Fusion and Broad Learning System
Zhai, Y., Yu, C., Qin, C., Zhou, W., Ke, Q., Gan, J., Labati, R. D., Piuri, V., & Scotti, F. (2020) · IEEE Access · doi:10.1109/access.2020.3032515 · Cited by 16
“Although deep learning has realized excellent prediction results in FBP, its imperfections [8] can never be underestimated: (1) time-consuming and costly training process due to a great deal of parameters; (2) poor model generalization [9] arisen from deep learning’s greater sensitivity to image…”
Background
SP-ILC: Concurrent Single-Pixel Imaging, Object Location, and Classification by Deep Learning
Yang, Z., Bai, Y.-M., Sun, L.-D., Huang, K.-X., Liu, J., Ruan, D., & Li, J. (2021) · Photonics · doi:10.3390/photonics8090400 · Cited by 14
“Deep learning has powerful feature extraction capabilities [18,19].”
An Accurate False Data Injection Attack (FDIA) Detection in Renewable-Rich Power Grids
Mohammadpourfard, M., Weng, Y., Genc, I. V. M., & Kim, T. (2022) · Workshop on Modeling and Simulation of Cyber-Physical Energy Systems (MSCPES) · doi:10.1109/mscpes55116.2022.9770151 · Cited by 15
“The RNN-related problems were solved by introducing a deep architecture called long short-term memory (LSTM) [19].”
Background
Optoelectronic integrated circuits for analog optical computing: Development and challenge
Dan, Y., Fan, Z., Chen, Q., Lai, Y., Sun, X., Zhang, T. Y., & Xu, K. (2022) · Frontiers in Physics · doi:10.3389/fphy.2022.1064693 · Cited by 18
A review of the automated timber defect identification approach
Chun, T. H., Hashim, U. R., Ahmad, S., Salahuddin, L. B., Choon, N. H., & Kanchymalay, K. (2022) · International Journal of Power Electronics and Drive Systems/International Journal of Electrical and Computer Engineering · doi:10.11591/ijece.v13i2.pp2156-2166 · Cited by 19
Convolutional neural network based key generation for security of data through encryption with advanced encryption standard
Negabi, I., Asri, S. A. E., Adib, S. E., & Raissouni, N. (2023) · International Journal of Power Electronics and Drive Systems/International Journal of Electrical and Computer Engineering · doi:10.11591/ijece.v13i3.pp2589-2599 · Cited by 15
Online Damage Recovery for Physical Robots with Hierarchical Quality-Diversity
Allard, M., Smith, S. C., Chatzilygeroudis, K. I., Lim, B., & Cully, A. (2023) · ACM Transactions on Evolutionary Learning and Optimization · doi:10.1145/3596912 · Cited by 15
“Some of the learning methods include Deep Learning, Reinforcement Learning and Quality-Diversity (QD) [Chatzilygeroudis et al. 2020; Shrestha and Mahmood 2019; Stanley et al. 2016; Sutton and Barto 2018] to name a few.”
Recognition method of voltage sag causes based on two‐dimensional transform and deep learning hybrid model
Zheng, Z., Qi, L., Wang, H., Pan, A., & Zhou, J. H. (2019) · IET Power Electronics · doi:10.1049/iet-pel.2019.0593 · Cited by 15
“…scenarios of the algorithm, the CNN model is more suitable for dealing with multi-channel two-dimensional (2D) sequences, the DBN model is more suitable for dealing with single-channel 1D sequence problems, and the SSDAE model has better noise immunity for single-channel 1D sequence problems [22].”
Method
Next Position Prediction using LSTM Neural Networks
Violos, J., Tsanakas, S., Androutsopoulou, M., Palaiokrassas, G., & Varvarigou, T. A. (2020) · Hellenic Conference on Artificial Intelligence · doi:10.1145/3411408.3411426 · Cited by 18
“Many DL experts conclude that the accuracy of a DL model has three main factors: The quality of the data [19] that we use, the training methodology that we follow, and the architectural design of the ANN [16].”
BackgroundInfluential
Research Challenges, Recent Advances and Benchmark Datasets in Deep-Learning-Based Underwater Marine Object Detection: A ReviewPreprint
Er, M. J., Chen, J., Zhang, Y., & Gao, W. (2023) · doi:10.36227/techrxiv.19350389.v4 · Cited by 15

Authors: Shrestha, A., & Mahmood, A.
Affiliations: University of Bridgeport (US)
ORCID: Shrestha, Ajay
Mahmood, Ausif
Venue: IEEE Access
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Published: 2019
Volume: 7
Pages: 53040-53065
ISSN: 2169-3536 (ISSN-L)
DOI: 10.1109/access.2019.2912200
OpenAlex: W2942231644
Cited by: 1,883
References: 88
Citations by year: 2026: 116 · 2025: 317 · 2024: 360 · 2023: 375 · 2022: 297 · 2021: 259 · 2020: 139 · 2019: 20
FWCI: 81.55
Type: journal article
Language: en
Open access: gold
doi.org
Licence: ieeexplore.ieee.org/Xplorehelp/downloads/license-information/OAPA.html (vor)
Locations: IEEE Access (publishedVersion)
University of Bridgeport ScholarWorks (University of Bridgeport) (submittedVersion)
DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics: Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
Keywords: deep learning, deep learning architectures, deep convolutional neural networks, ResNet, recurrent neural networks, reinforcement learning, variational autoencoder, feature extraction, hierarchical representation, training time reduction

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Review of Deep Learning Algorithms and Architectures

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks