Cited by 1,883Neural Networks and ApplicationsAdvanced Neural Network ApplicationsStochastic Gradient Optimization Techniques

The DOI matches its Crossref record; 85 of the 88 references it cites were found in a registry or an index.

Review of Deep Learning Algorithms and Architectures

Open access

Shrestha, Ajay, Mahmood, Ausif·2019·IEEE Access

DOI 10.1109/access.2019.2912200

Open this paper in CoChat — free account Open PDF

Abstract

Deep learning (DL) is playing an increasingly important role in our lives. It has already made a huge impact in areas, such as cancer diagnosis, precision medicine, self-driving cars, predictive forecasting, and speech recognition. The painstakingly handcrafted feature extractors used in traditional learning, classification, and pattern recognition systems are not scalable for large-sized data sets. In many cases, depending on the problem complexity, DL can also overcome the limitations of earlier shallow networks that prevented efficient training and abstractions of hierarchical representations of multi-dimensional training data. Deep neural network (DNN) uses multiple (deep) layers of units with highly optimized algorithms and architectures. This paper reviews several optimization methods to improve the accuracy of the training and to reduce training time. We delve into the math behind training algorithms used in recent deep networks. We describe current shortcomings, enhancements, and implementations. The review also covers different types of deep architectures, such as deep convolution networks, deep residual networks, recurrent neural networks, reinforcement learning, variational autoencoders, and others.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1109/access.2019.2912200

Cites

85 of 85 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 5×
“tion with a 152-layer DNN on the ImageNet dataset [62].”
BackgroundMethodInfluential
Long Short-Term Memory
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 2×
“The input, forget and output gates dictates new data going into the cell, what remains in the cell and the cell values used in the calculation of the output of the LSTM block respectively [41], [43].”
Method
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 4×
“Multilayer network training cost on MNIST dataset using different adaptive learning algorithms [58].”
BackgroundMethodInfluential
Deep learning
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 2×
“7-layer architecture of CNN for character recognition [28].”
BackgroundMethod
Adaptation in Natural and Artificial Systems
Holland, J. H. (1992) · The MIT Press eBooks · doi:10.7551/mitpress/1090.001.0001 · Cited by 47,340
Untitled work
(2014) · Cited by 33,419
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818 · cited 2×
“[59] S. Ioffe and C. Szegedy, ‘‘Batch normalization: Accelerating deep network training by reducing internal covariate shift,’’ CoRR, Feb. 2015.”
Background
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545 · cited 1×
“GRUs are smaller in size than LSTM as they don’t include the output gate, and can perform better than LSTM on only some simpler datasets [44], [45].”
Background
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928

Flagged first, then by citations

Show all 88 references

3 of the 88 references couldn’t be matched to a known paper.

Research works printed but not linked: 10

vapnik, 2015, Learning using privileged information: Similarity control and knowledge transfer, J Mach Learn Res, 16, 2023
ng, 2018, Autoencoders
2018, Amazon EC2 P2 & P3 Instances Amazon EC2 Instance Types
simpson, 2015, Uniform learning in a deep neural network via ‘oddball’ stochastic gradient descent, CoRR
minsky, 1969, Perceptrons An Introduction to Computational Geometry Expanded Edition, 258
2019, Top Deep Learning Github Repositories
sastry, 2005, Genetic Algorithms
10.1017/CBO9780511543241 · DOI
wu, 2016, Google’s neural machine translation system: Bridging the gap between human and machine translation, CoRR
metz, 2019, Turing Award Won by 3 Pioneers in Artificial Intelligence, 3b

Sorted by citations

Items 401–500 of 1,001

Single-Molecule Bioelectronic Sensors with AI-Aided Data Analysis: Convergence and Challenges
Yang, Y., Li, Y., Tang, L., & Li, J. (2024) · Precision Chemistry · doi:10.1021/prechem.4c00048 · Cited by 26
“Deep learning 79,80 has seen significant growth driven by the advancement of high-performance computing hardware.”
Background
An Embedded System for Collection and Real-Time Classification of a Tactile Dataset
Kurşun, O., & Patooghy, A. (2020) · IEEE Access · doi:10.1109/access.2020.2996576 · Cited by 26
“Kursun and Favorov [24] have shown that these features can be used to perform efﬁcient texture image classiﬁcation and as in deep convolutional neural networks (CNN) [17], [18], they can be used by subsequent cortical areas to develop gradually more complex and perceptually advanced features.”
Background
A Review of Spam Detection in Social Media
Yurtseven, I., Bağrıyanık, S., & Ayvaz, S. (2021) · International Conference on Computer Science and Engineering (UBMK) · doi:10.1109/ubmk52708.2021.9558993 · Cited by 27
Wind power bidding coordinated with energy storage system operation in real-time electricity market: A maximum entropy deep reinforcement learning approach
Wei, X., Xiang, Y., Li, J., & Liu, J. (2021) · Energy Reports · doi:10.1016/j.egyr.2021.11.216 · Cited by 26
Nonintrusive and automatic quantitative analysis methods for fish behaviour in aquaculture
Liu, J., Bienvenido, F., Yang, X., Zhao, Z., Feng, S., & Zhou, C. (2022) · Aquaculture Research · doi:10.1111/are.15828 · Cited by 28
eICAB: A novel deep learning pipeline for Circle of Willis multiclass segmentation and analysis
Dumais, F., Cáceres, M. P., Janelle, F., Seifeldine, K., Arès-Bruneau, N., Gutierrez, J., Bocti, C., & Whittingstall, K. (2022) · NeuroImage · doi:10.1016/j.neuroimage.2022.119425 · Cited by 26
Deep transfer residual variational autoencoder with multi-sensors fusion for tool condition monitoring in impeller machining
Ou, J., Li, H., Liu, B., & Peng, D. (2022) · Measurement · doi:10.1016/j.measurement.2022.112028 · Cited by 26
Automated counting of white blood cells in thin blood smear images
Escobar, F. I. F., Alipo-on, J. R. T., Novia, J. L. U., Tan, M. J. T., Karim, H. B. A., & AlDahoul, N. (2023) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2023.108710 · Cited by 29
Satellite Aerosol Optical Depth Retrieval Based on Fully Connected Neural Network (FCNN) and a Combine Algorithm of Simplified Aerosol Retrieval Algorithm and Simplified and Robust Surface Reflectance Estimation (SREMARA)
Fan, Y., & Sun, L. (2023) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2023.3281777 · Cited by 26
“However,theseaerosolretrievalalgorithmsbasedonMLare hardtobegivenareasonableinterpretationoftheirbehaviors andsufferfromrepresentativesampleconstructionandeffective training[54],[55],[56].”
Optimization of Storage Location Assignment for Non-Traditional Layout Warehouses Based on the Firework Algorithm
Zhang, X., Mo, T., & Zhang, Y. (2023) · Sustainability · doi:10.3390/su151310242 · Cited by 26
Deep learning-based hair removal for improved diagnostics of skin diseases
El‐Shafai, W., El‐Fattah, I. A., & Taha, T. E. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-16646-6 · Cited by 26
“DL approaches have demonstrated significant advancements in the field of malignant detection, owing to their ability to extract multi-level abstractions from distributed data [30, 31].”
Architectures for Industrial AIoT Applications
Villar, E., Toral, I. M., Calvo, I., Barambones, Ó., & Fernández-Bustamante, P. (2024) · Sensors · doi:10.3390/s24154929 · Cited by 38
“Convolutional Neural Networks (CNNs) are the most representative DL model [40], but other architectures like Restricted Boltzmann Machine (RBM) and Long Short-Term Memory (LSTM) are also being used [41].”
Method
A comprehensive plane-wise review of DDoS attacks in SDN: Leveraging detection and mitigation through machine learning and deep learning
Kalambe, D., Sharma, D., Kadam, P., & Surati, S. (2024) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2024.104081 · Cited by 30
Machine learning-enabled prediction of density and defects in additively manufactured Inconel 718 alloy
Sah, A. K., Agilan, M., Dineshraj, S., Rahul, M. R., & Govind, B. (2022) · Materials Today Communications · doi:10.1016/j.mtcomm.2022.103193 · Cited by 27
Exploration in neo-Hebbian reinforcement learning: Computational approaches to the exploration–exploitation balance with bio-inspired neural networks
Triche, A., Maida, A. S., & Kumar, A. (2022) · Neural Networks · doi:10.1016/j.neunet.2022.03.021 · Cited by 27
An Ensemble Deep Learning Model with a Gene Attention Mechanism for Estimating the Prognosis of Low-Grade Glioma
Lee, M. (2022) · Biology · doi:10.3390/biology11040586 · Cited by 25
“In each layer, the input values are multiplied by a weight matrix, and then a nonlinear operation is performed [40].”
Method
Dynamic Path Planning Based on Neural Networks for Aerial Inspection
de Castro, G. G. R., Pinto, M. F., Biundini, I. Z., Melo, A. G., Marcato, A. L. M., & Haddad, D. B. (2022) · Journal of Control Automation and Electrical Systems · doi:10.1007/s40313-022-00941-y · Cited by 25
“The choice about the number of layers reflects the previous findings that indicate that general accuracy is higher in deep learning networks (Minaee et al. 2021; Shrestha and Mahmood 2019) than in shallow networks.”
Background
Automated detection of schizophrenia using deep learning: a review for the last decade
Sharma, M. D., Patel, R. K., Garg, A., SanTan, R., & Acharya, U. R. (2023) · Physiological Measurement · doi:10.1088/1361-6579/acb24d · Cited by 28
Research on Transformer Voiceprint Anomaly Detection Based on Data-Driven
Da, Y., Zhang, W., & Wang, H. (2023) · Energies · doi:10.3390/en16052151 · Cited by 26
“LSTM introduces the sigmoid function through its three gatings and combines it with the tanh function to increase the summation steps, reduce the possibility of gradient vanishing and gradient explosion, and solve both short-term and long-term dependence problems [23–26].”
ECG heartbeats classification with dilated convolutional autoencoder
Arslan, N. N., Özdemir, D., & Temurtaş, H. (2023) · Signal Image and Video Processing · doi:10.1007/s11760-023-02737-2 · Cited by 25
Reinforcement Learning for Intelligent Healthcare Systems: A Comprehensive SurveyPreprint
Abdellatif, A. A., Mhaisen, N., Chkirbene, Z., Mohamed, A. M. S., Erbad, A., & Guizani, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.04087 · Cited by 39
“[37] It presents a general overview for the neural networks and deep neural networks challenges, training algorithms, architecture, and implementations, without focusing on I-health systems.”
Background
Insider Threat Detection using Deep Learning: A Review
Raut, M., Dhavale, S., Singh, A., & Mehra, A. (2020) · 2020 3rd International Conference on Intelligent Sustainable Systems (ICISS) · doi:10.1109/iciss49785.2020.9315932 · Cited by 27
“Deep Neural Networks (DNNs) use multiple (deep) layers of nodes with highly optimized architectures [5].”
Background
Deep learning algorithms applied to computational chemistry
Pando, A. G., Ramírez-Alonso, G. M. D. J., Arzate-Quintana, C., & Camarillo‐Cisneros, J. (2023) · Molecular Diversity · doi:10.1007/s11030-023-10771-y · Cited by 28
“These findings are consistent with previous analyses of optimization algorithms [145], where the Adam optimizer was the best option to improve accuracy and reduce training time.”
Artificial intelligence for predicting treatment responses in autoimmune rheumatic diseases: advancements, challenges, and future perspectives
Yang, Y., Liu, Y., Chen, Y., Luo, D., Xu, K., & Zhang, L. (2024) · Frontiers in Immunology · doi:10.3389/fimmu.2024.1477130 · Cited by 28
Applications, technologies, and evaluation methods in smart aquaponics: a systematic literature review
Anila, M., & Daramola, O. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-11003-x · Cited by 38
A review of Bayesian-filtering-based techniques in RUL prediction for Lithium-Ion batteries
Khine, M. H. H., Kim, C. G., & Aunsri, N. (2025) · Journal of Energy Storage · doi:10.1016/j.est.2025.115371 · Cited by 26
Privacy-preserving healthcare data in IoT: a synergistic approach with deep learning and blockchain
Bezanjani, B. R., Ghafouri, S. H., & Gholamrezaei, R. (2025) · The Journal of Supercomputing · doi:10.1007/s11227-025-06980-x · Cited by 30
“With its decentralized and immutable nature, blockchain technology provides a robust platform for securing medical data, ensuring transparency, and preventing unauthorized access or tampering [5, 6].”
Multi-view convolutional neural network with leader and long-tail particle swarm optimizer for enhancing heart disease and breast cancer detection
Lan, K., Liu, L., Li, T., Chen, Y., Fong, S. J., Marques, J. A. L., Wong, R. K., & Tang, R. (2020) · Neural Computing and Applications · doi:10.1007/s00521-020-04769-y · Cited by 24
“On the contrary, the gradient descending method can achieve faster convergent speed around global optimum and the convergent accuracy can be higher [38, 42, 43].”
A Hybrid Method for Remaining Useful Life Prediction of Proton Exchange Membrane Fuel Cell Stack
Wang, F., Amogne, Z. E., & Chou, J. (2021) · IEEE Access · doi:10.1109/access.2021.3064684 · Cited by 24
“Shrestha and Mahmood [45] provide a thorough overview of the neural networks and deep neural networks.”
BackgroundInfluential
Deep Learning Model to Detect Diabetes Mellitus Based on DNA Sequence
El-Attar, N. E., Moustafa, B. M., & Awad, W. A. (2021) · Intelligent Automation & Soft Computing · doi:10.32604/iasc.2022.019970 · Cited by 24
Performance Analysis of Deep Neural Networks Using Computer Vision
Sindhwani, N., Anand, R., Meivel, S., Shukla, R., Yadav, M. P., & Yadav, V. (2021) · EAI Endorsed Transactions on Industrial Networks and Intelligent Systems · doi:10.4108/eai.13-10-2021.171318 · Cited by 24
“…recognition, object detection, adversarial networks, reinforcement learning and detailed performance review of these models across three major deep learning frameworks (TensorFlow, MXNet & Cognitive Toolkit) across the various hardware configurations (single-GPU, multi-GPU, and multi-machine) [19].”
Background
Two-Stage Multi-Scale Fault Diagnosis Method for Rolling Bearings with Imbalanced Data
Zheng, M., Chang, Q., Man, J., Liu, Y., & Shen, Y. (2022) · Machines · doi:10.3390/machines10050336 · Cited by 25
“Among them, the recurrent neural network (RNN), convolutional neural networks (CNN), and various variant models are the most widely used, and the actual effect is the most ideal [38].”
Background
Optimized design for modified auxetic structures based on a neural network approach
Bronder, S., Herter, F., Bähre, D., & Jung, A. (2022) · Materials Today Communications · doi:10.1016/j.mtcomm.2022.103931 · Cited by 24
Short-Term Load Forecasting of the Greek Electricity System
Stamatellos, G., & Stamatelos, A. M. (2023) · Applied Sciences · doi:10.3390/app13042719 · Cited by 24
Titanium oxide artificial synaptic device: Nanostructure modeling and synthesis, memristive cross-bar fabrication, and resistive switching investigation
Игоревич, А. В., Tominov, R. V., Vakulov, Z., Zhavoronkov, L. G., & Smirnov, V. A. (2023) · Nano Research · doi:10.1007/s12274-023-5639-5 · Cited by 25
Soft sensor for predicting indoor PM2.5 concentration in subway with adaptive boosting deep learning model
Wang, J., Wang, D., Zhang, F., Yoo, C., & Liu, H. (2023) · Journal of Hazardous Materials · doi:10.1016/j.jhazmat.2023.133074 · Cited by 25
Multiresolution feature fusion for smart diagnosis of schizophrenia in adolescents using EEG signals
Ranjan, R., & Sahana, B. C. (2024) · Cognitive Neurodynamics · doi:10.1007/s11571-024-10120-1 · Cited by 23
“A CNN takes full beneﬁt of the inherent hierarchical patterns in data, assembling more complex patterns by using smaller and smaller patterns (Goodfellow et al. 2016; Shrestha and Mahmood 2019).”
A novel fault diagnosis method for key transmission sections based on Nadam-optimized GRU neural network
Hu, H., Lu, H. F., Shi, R., Fan, X., & Deng, Z. (2024) · Electric Power Systems Research · doi:10.1016/j.epsr.2024.110522 · Cited by 23
Machine learning-assisted identification of environmental pollutants by liquid chromatography coupled with high-resolution mass spectrometry
Wang, H., Zhong, L., Su, W., Ruan, T., & Jiang, G. (2024) · TrAC Trends in Analytical Chemistry · doi:10.1016/j.trac.2024.117988 · Cited by 24
Brain Tumor Classification using Deep Learning: A State-of-the-Art Review
Rasool, M., Noorwali, A., Ghandorh, H., Ismail, N. A., & Yafooz, W. M. S. (2024) · Engineering Technology & Applied Science Research · doi:10.48084/etasr.8298 · Cited by 25
Application of machine learning in breast cancer survival prediction using a multimethod approach
Hamedi, S. Z., Emami, H., Khayamzadeh, M., Rabiei, R., Aria, M., Akrami, M., & Zangouri, V. (2024) · Scientific Reports · doi:10.1038/s41598-024-81734-y · Cited by 28
Dual-channel encoded bidirectional LSTM for multi-building short-term load forecasting
Moudgil, V., Sadiq, R., Brar, J., & Hewage, K. (2024) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2024.144555 · Cited by 23
Swallowing Pattern Classification Method Using Multichannel Surface EMG Signals of Suprahyoid and Infrahyoid Muscles
Suzuki, M., SASAKI, M., Kamata, K., Nakayama, A., Shibamoto, I., & Tamada, Y. (2020) · Advanced Biomedical Engineering · doi:10.14326/abe.9.10 · Cited by 23
Deep Learning Algorithms based Fingerprint Authentication: Systematic Literature Review
Chiroma, H. (2021) · Journal of Artificial Intelligence and Systems · doi:10.33969/ais.2021.31010 · Cited by 29
“The CNNs are the most utilized DL for computer vision, image detection and video recognition [41].”
BackgroundMethod
A lightweight convolutional neural network model with receptive field block for C-shaped root canal detection in mandibular second molars
Zhang, L., Xu, F., Li, Y., Zhang, H., Xi, Z., Xiang, J., & Wang, B. (2022) · Scientific Reports · doi:10.1038/s41598-022-20411-4 · Cited by 25
Deep Learning-Based Path Loss Prediction for Fifth-Generation New Radio Vehicle Communications
Sung, S., Choi, W., Kim, H., & Jung, J.-I. (2023) · IEEE Access · doi:10.1109/access.2023.3297215 · Cited by 23
“Furthermore, an excessive number of neurons in the hidden layer may significantly increase the network training time, making neural network training impractical [17], [18].”
Background
A mask R-CNN approach for detection and classification of brain tumours from MR images
Kordemir, M., Çevik, K. K., & Bozkurt, A. (2024) · Computer Methods in Biomechanics and Biomedical Engineering Imaging & Visualization · doi:10.1080/21681163.2023.2301391 · Cited by 24
“Specifically, deep learning (DL) models (LeCun et al. 2015; Shrestha and Mahmood 2019) have been particularly beneficial in this respect because of the large volume of involved image data (Cui et al. 2020).”
Background
Challenges and practices of deep learning model reengineering: A case study on computer vision
Jiang, W., Banna, V., Vivek, N., Goel, A., Synovic, N. M., Thiruvathukal, G. K., & Davis, J. C. (2024) · Empirical Software Engineering · doi:10.1007/s10664-024-10521-0 · Cited by 33
Try with Simpler - An Evaluation of Improved Principal Component Analysis in Log-based Anomaly Detection
Yang, L., Chen, J., Gao, S., Gong, Z., Zhang, H., Kang, Y., & Li, H. H. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3644386 · Cited by 26
Review on deep learning classifiers for faults diagnosis of rotating industrial machinery
Shaalan, A. A., Mefteh, W., & Frihida, A. (2024) · Service Oriented Computing and Applications · doi:10.1007/s11761-024-00418-7 · Cited by 22
Fusion of machine learning and blockchain-based privacy-preserving approach for healthcare data in the Internet of Things
Bezanjani, B. R., Ghafouri, S. H., & Gholamrezaei, R. (2024) · The Journal of Supercomputing · doi:10.1007/s11227-024-06392-3 · Cited by 22
“On the other hand, DL has garnered significant attention across various domains for its unpar-alleled accuracy in data analysis and pattern recognition [5, 6].”
Application of multiple linear regression model and long short-term memory with compartmental model to forecast dengue cases in Selangor, Malaysia based on climate variables
Lu, X., Teh, S. Y., Tay, C. J., Kassim, N. F. A., Shan, F. P., & Soewono, E. (2024) · Infectious Disease Modelling · doi:10.1016/j.idm.2024.10.007 · Cited by 32
Deep Learning Based Breast Cancer Detection Using Decision Fusion
Manalı, D., Demırel, H., & Eleyan, A. (2024) · Computers · doi:10.3390/computers13110294 · Cited by 22
Machine learning-accelerated exploration on element doping-triggering material performance improvement for energy conversion and storage applications
Wang, H., Zhu, Y., Li, J., Liu, X., Ma, Y., Yao, Y., Zhang, J., & Pan, L. (2025) · Journal of Materials Chemistry A · doi:10.1039/d5ta00922g · Cited by 22
Artificial intelligence assisted non-destructive testing of welding joints: A review of techniques, X-ray image processing and applications
Say, D., Qaisar, S. M., Krichen, M., & Zidi, S. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.111792 · Cited by 22
An Integrated Framework for Autonomous Driving: Object Detection, Lane Detection, and Free Space Detection
Kemsaram, N., Das, A., & Dubbelman, G. (2019) · World Conference on Smart Trends in Systems, Security and Sustainability (WorldS4) · doi:10.1109/worlds4.2019.8904020 · Cited by 22
“This i-CAVE programme is funded by NWO (Netherlands Organisation for Scientiﬁc Research). based approaches [7], [8].”
Method
State of Charge Estimation Method for Lithium-Ion Batteries in All-Electric Ships Based on LSTM Neural Network
Geng, P., Xu, X., & Tarasiuk, T. (2020) · Polish Maritime Research · doi:10.2478/pomr-2020-0051 · Cited by 22
“) represents the activation function, subscript t denotes the time, W is the network weight coefficient and b is the offset vector [29].”
BackgroundMethod
Recognition of Handwritten Arabic and Hindi Numerals Using Convolutional Neural Networks
Alqudah, A., Alqudah, A. M., Alquran, H., Al-Zoubi, H. R., Al-Qodah, M. A., & Al-khassaweneh, M. (2021) · Applied Sciences · doi:10.3390/app11041573 · Cited by 22
“A review of the recent algorithms and architectures that enhance both training and testing in terms of accuracy and time can be found in [25].”
Method
The future of Artificial Intelligence for the BioTech Big Data landscape
Artico, F., Edge, A. L., & Langham, K. (2022) · Current Opinion in Biotechnology · doi:10.1016/j.copbio.2022.102714 · Cited by 22
Informed Chemical Classification of Organophosphorus Compounds via Unsupervised Machine Learning of X-ray Absorption Spectroscopy and X-ray Emission Spectroscopy
Tetef, S., Kashyap, V., Holden, W. M., Velian, A., Govind, N., & Seidler, G. T. (2022) · The Journal of Physical Chemistry A · doi:10.1021/acs.jpca.2c03635 · Cited by 23
Machine Learning Algorithms for 6G Wireless Networks
Patil, A., Iyer, S., & Pandya, R. J. (2022) · Advances in information security, privacy, and ethics book series · doi:10.4018/978-1-6684-3921-0.ch003 · Cited by 22
Accelerated two‐dimensional phase‐contrast for cardiovascular MRI using deep learning‐based reconstruction with complex difference estimation
Oscanoa, J., Middione, M. J., Syed, A., Sandino, C. M., Vasanawala, S. S., & Ennis, D. B. (2022) · Magnetic Resonance in Medicine · doi:10.1002/mrm.29441 · Cited by 22
“34 Additionally, each CNN block is a fully convolutional residual network (ResNet) 35 with (2 + 1)D convolutions as described in Reference 29.”
Method
DeepCF-PPI: improved prediction of protein-protein interactions by combining learned and handcrafted features based on attention mechanisms
Tran, H.-N., Xuan, Q. N. P., & Nguyen, T. T. (2023) · Applied Intelligence · doi:10.1007/s10489-022-04387-2 · Cited by 22
Deep learning-based spam image filtering
Salama, W. M., Aly, M. H., & Abouelseoud, Y. (2023) · Alexandria Engineering Journal · doi:10.1016/j.aej.2023.01.048 · Cited by 22
Blockchain and Deep Learning Integration for Various Application: A Review
Afaq, Y., & Manocha, A. (2023) · Journal of Computer Information Systems · doi:10.1080/08874417.2023.2173330 · Cited by 23
Edge Detection With Direction Guided Postprocessing for Farmland Parcel Extraction
Xie, Y., Zheng, S., Wang, H., Qiu, Y., Xilan, L., & Shi, Q. (2023) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2023.3253779 · Cited by 20
“Deep-learning models are heavily used in different types of image processing tasks, including image classiﬁcation, object detection, and semantic segmentation [15].”
Method
Atrial Fibrillation Detection Algorithm Based on Graph Convolution Network
Ma, H., & Xia, L. (2023) · IEEE Access · doi:10.1109/access.2023.3291352 · Cited by 21
“However, in recent years, deep learning approaches have surpassed traditional feature engineering and machine learning methods in various fields such as computer vision, natural language processing, and speech recognition [6].”
Background
DNABERT-based explainable lncRNA identification in plant genome assemblies
Danilevicz, M. F., Gill, M., Fernandez, C. G. T., Petereit, J., Upadhyaya, S. R., Batley, J., Bennamoun, M., Edwards, D., & Bayer, P. E. (2023) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2023.11.025 · Cited by 21
The importance of machine learning in autonomous actions for surgical decision making
Wagner, M., Bodenstedt, S., Daum, M. T. J., Schulze, A., Younis, R., Brandenburg, J. M., Kolbinger, F. R., Distler, M., Maier-Hein, L., Weitz, J., et al. (2022) · Artificial Intelligence Surgery · doi:10.20517/ais.2022.02 · Cited by 24
Multimodal Age and Gender Estimation for Adaptive Human-Robot Interaction: A Systematic Literature Review
Younis, H. A., Ruhaiyem, N. I. R., Badr, A. A., Abdul-Hassan, A. K., Alfadli, I. M., Binjumah, W. M., Altuwaijri, E. A., & Nasser, M. (2023) · Processes · doi:10.3390/pr11051488 · Cited by 21
End-to-End Automated Guided Modular Vehicle
Curiel-Ramirez, L. A., Ramírez-Mendoza, R. A., Bautista-Montesano, R., Bustamante-Bello, R., González-Hernández, H. G., Reyes-Avendaño, J. A., & Cortes-Gallardo, E. (2020) · Applied Sciences · doi:10.3390/app10124400 · Cited by 21
“Specific architectures have been generated by companies or development groups, such as LeNet, GoogLeNet, VGG, ResNet, Inception, and others [15,16].”
BackgroundMethodInfluential
Analyzing the depression and suicidal tendencies of people affected by COVID-19’s lockdown using sentiment analysis on social networking websites
Sharma, S., & Sharma, S. (2020) · Journal of Statistics and Management Systems · doi:10.1080/09720510.2020.1833453 · Cited by 21
A Neural Question Answering System for Basic Questions about Subroutines
Bansal, A., Eberhart, Z., Wu, L., & McMillan, C. (2021) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner50967.2021.00015 · Cited by 21
Deep Learning Techniques for OFDM Systems
Meenalakshmi, M., Chaturvedi, S., & Dwivedi, V. K. (2021) · IETE Journal of Research · doi:10.1080/03772063.2021.1986154 · Cited by 21
“The gradient descent, stochastic gradient descent, and mini-batch gradient descent are three different types of optimization techniques available for weight optimization [32].”
Method
Auto-detection and segmentation of involved lymph nodes in HPV-associated oropharyngeal cancer using a convolutional deep learning neural network
Taku, N., Wahid, K. A., van Dijk, L. V., Sahlsten, J., Jaskari, J., Kaski, K. K., Fuller, C. D., & Naser, M. A. (2022) · Clinical and Translational Radiation Oncology · doi:10.1016/j.ctro.2022.06.007 · Cited by 20
“Deep learning is a subset of machine learning that uses neural networks to learn and classify data [7].”
Background
Generating 5 km resolution 1981–2018 daily global land surface longwave radiation products from AVHRR shortwave and longwave observations using densely connected convolutional neural networks
Xu, J., Liang, S., Ma, H., & He, T. (2022) · Remote Sensing of Environment · doi:10.1016/j.rse.2022.113223 · Cited by 23
Deep Learning Techniques for the Effective Prediction of Alzheimer’s Disease: A Comprehensive Review
Shastry, K. A., Vijayakumar, V., Manoj Kumar, M., V, Manjunatha, B. A., & Chandrashekhar, B. (2022) · Healthcare · doi:10.3390/healthcare10101842 · Cited by 22
“Overfitting has also contributed immensely to the development of DL [66], with attempts to handle the issue at an individual and collective scale.”
Background
Identification of Mangrove Invasive Plant Derris Trifoliate Using UAV Images and Deep Learning Algorithms
Liu, M., Deng, H., & Dong, W. (2022) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2022.3223227 · Cited by 21
“Developed from machine learning, DL algorithms can generally reach much better performance than conventional methods, by constructing more complex architectures and automatically extracting deeper feature information [22], which have received more attention these years.”
Method
Simultaneous forecasting of wind speed for multiple stations based on attribute-augmented spatiotemporal graph convolutional network and tree-structured parzen estimator
Zhang, C., Qiao, X., Zhang, Z., Wang, Y., Fu, Y., Nazir, M. S., & Tian, P. (2024) · Energy · doi:10.1016/j.energy.2024.131058 · Cited by 23
Depression assessment using integrated multi-featured EEG bands deep neural network models: Leveraging ensemble learning techniques
Chung, K., Chang, Y., Yen, W.-T., Lin, L. N., & Abimannan, S. (2024) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2024.03.022 · Cited by 22
“In the field of Deep Learning applications, various network model architectures have been developed earlier [13], such as RNN (Recurrent Neural Network), CNN (Convolutional Neural Network), LSTM (Long Short-Term Memory), etc.”
Method
Real time evolution with neural-network quantum states
Gutiérrez, I. L., & Mendl, C. B. (2022) · Quantum · doi:10.22331/q-2022-01-20-627 · Cited by 20
“using deeper network architectures with batch normalization and residual blocks [29].”
Method
Data-assisted training of a physics-informed neural network to predict the separated Reynolds-averaged turbulent flow field around an airfoil under variable angles of attack
Harmening, J. H., Pioch, F., Fuhrig, L., Peitzmann, F.-J., Schramm, D. H., & el Moctar, O. (2024) · Neural Computing and Applications · doi:10.1007/s00521-024-09883-9 · Cited by 20
“A vast variety of different architectures and algorithms of NNs have been developed and applied, comprising feed-forward NNs, recurrent NNs, convolutional NNs (CNNs), reinforcement learning (RL), long short-term memory (LSTM), or generative adversarial networks (GANs), among others [8–10].”
Deep learning-driven multi-class classification of brain strokes using computed tomography: A step towards enhanced diagnostic precision
Kulathilake, C. D., Udupihille, J., Abeysundara, S. P., & Senoo, A. (2025) · European Journal of Radiology · doi:10.1016/j.ejrad.2025.112109 · Cited by 20
2D MoTe 2 memristors for energy-efficient artificial synapses and neuromorphic applications
Khan, R., Rehman, N. U., Kalluri, S., Elumalai, S., Saritha, A., Fakhar-e-alam, M., Ikram, M., Abdullaev, S. S., Rahman, N., & Sangaraju, S. (2025) · Nanoscale · doi:10.1039/d5nr01509j · Cited by 22
Foxtsage vs. Adam: Revolution or evolution in optimization?
Aula, S. A., & Rashid, T. A. (2025) · Cognitive Systems Research · doi:10.1016/j.cogsys.2025.101373 · Cited by 20
Network Resource Allocation Strategy Based on Deep Reinforcement Learning
Zhang, S., Wang, C., Zhang, J., Duan, Y., You, X., & Zhang, P. (2020) · IEEE Open Journal of the Computer Society · doi:10.1109/ojcs.2020.3000330 · Cited by 20
“The more classic artificial neural networks include perceptron, BP neural network, radial basis neural network and feedback neural network [22], [23].”
Availability-Aware Channel Allocation for Multi-Cell Cognitive Radio 5G Networks
Halloush, R. D., Salaimeh, R., & Al-Dalqamoni, R. (2022) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2022.3148063 · Cited by 20
“Modern machine learning techniques, such as reinforced and deep learning [17], [18], can also be used to ﬁnd optimal solutions, as they are suited for searching large spaces of possible solutions efﬁciently.”
Background
Developing a fake news identification model with advanced deep language transformers for Turkish covid-19 misinformation data
Bozuyla, M., & Özçift, A. (2021) · TURKISH JOURNAL OF ELECTRICAL ENGINEERING & COMPUTER SCIENCES · doi:10.55730/1300-0632.3818 · Cited by 21
Decision support system for the differentiation of schizophrenia and mood disorders using multiple deep learning models on wearable devices data
Nguyen, D.-K., Chan, C., Li, A., Phan, D., & Lan, C.-H. (2022) · Health Informatics Journal · doi:10.1177/14604582221137537 · Cited by 21
“Application of DL is becoming increasingly popular, and this technique is more widely-used than traditional data mining techniques.(17) In recent years, expenditure on wearable devices has been remarkable owing to their advantages and convenience.”
Method
Characterising surface roughness of Ti-6Al-4V alloy machined using coated and uncoated carbide tools with variable nose radius by machine learning
Mazid, A. M., Imam, T., Ahsan, K. B., & Khandoker, N. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.106546 · Cited by 22
A Review of Smart Photovoltaic Systems Which Are Using Remote-Control, AI, and Cybersecurity Approaches
Comșiț, A.-M. C., Cotfas, D. T., & Cotfas, P. A. (2024) · Applied Sciences · doi:10.3390/app14177838 · Cited by 20
Cancer-Related Gene Signature Selection Based on Boosted Regression for Multilayer Perceptron
Seo, H., & Cho, D. (2020) · IEEE Access · doi:10.1109/access.2020.2985414 · Cited by 21
“Recently, researches have been actively conducted to develop advanced machine learning and artiﬁcial neural networks, which were applied to various problems [25], [26].”
Background
Adaptive cognitive intelligence in analyzing employee feedback using LSTM
Dhanalakshmi, R., & Devi, T. S. (2020) · Journal of Intelligent & Fuzzy Systems · doi:10.3233/jifs-189129 · Cited by 23
Deep similarity learning for disease prediction
Gupta, V., Sachdeva, S., & Dohare, N. (2021) · Elsevier eBooks · doi:10.1016/b978-0-12-822226-3.00008-8 · Cited by 21
Sparse Graph Representation Learning Based on Reinforcement Learning for Personalized Mild Cognitive Impairment (MCI) Diagnosis
Ji, C.-H., Shin, D.-H., Son, Y.-H., & Kam, T. (2024) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2024.3393625 · Cited by 19
“Relying exclusively on supervised learning may limit the model’s capacity to explore and identify distinctive patterns for individual subjects [33], [34], [35].”
BackgroundInfluential
Training of construction robots using imitation learning and environmental rewards
Duan, K., Zou, Z., & Yang, T. Y. (2024) · Computer-Aided Civil and Infrastructure Engineering · doi:10.1111/mice.13394 · Cited by 21
“Recent studies have reported its success in many fields such as manufacturing (Ahny & Park, 2019) and self-driving (Shrestha & Mahmood, 2019).”
Porous Materials for Early Diagnosis of Neurodegenerative Diseases
Arghavani, P., Daneshgar, H., Sojdeh, S., Edrisi, M., Moosavi‐Movahedi, A. A., & Rabiee, N. (2025) · Advanced Healthcare Materials · doi:10.1002/adhm.202404685 · Cited by 19
EHR-based prediction modelling meets multimodal deep learning: A systematic review of structured and textual data fusion methods
Teles, A. S., de Moura, I. R., Silva, F., Roberts, A., & Stahl, D. R. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.102981 · Cited by 23
Enhanced lung cancer subtype classification using attention-integrated DeepCNN and radiomic features from CT images: a focus on feature reproducibility
Alsallal, M., Ahmed, H. H., Kareem, R. A., Yadav, A., Ganesan, S., Shankhyan, A., Gupta, S., Joshi, K. K., Sameer, H. N., Yaseen, A., et al. (2025) · Discover Oncology · doi:10.1007/s12672-025-02115-z · Cited by 19
“Deep learning algorithms, particularly convolutional neural networks (CNNs), have shown promise in automating the feature extraction process by learning complex patterns directly from the raw images [16].”
A Low-Complexity Belief Propagation Based Decoding Scheme for Polar Codes - Decodability Detection and Early Stopping Prediction
Wang, Y., Zhang, S., Zhang, C., Chen, X., & Xu, S. (2019) · IEEE Access · doi:10.1109/access.2019.2950766 · Cited by 19
“First, the indicator function is non-differentiable and non-continuous, and a smooth sigmoid based activation function [22], [23] at the output layer to model I(·) may cause the inaccurate prediction.”
Background

Authors: Shrestha, A., & Mahmood, A.
Affiliations: University of Bridgeport (US)
ORCID: Shrestha, Ajay
Mahmood, Ausif
Venue: IEEE Access
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Published: 2019
Volume: 7
Pages: 53040-53065
ISSN: 2169-3536 (ISSN-L)
DOI: 10.1109/access.2019.2912200
OpenAlex: W2942231644
Cited by: 1,883
References: 88
Citations by year: 2026: 116 · 2025: 317 · 2024: 360 · 2023: 375 · 2022: 297 · 2021: 259 · 2020: 139 · 2019: 20
FWCI: 81.55
Type: journal article
Language: en
Open access: gold
doi.org
Licence: ieeexplore.ieee.org/Xplorehelp/downloads/license-information/OAPA.html (vor)
Locations: IEEE Access (publishedVersion)
University of Bridgeport ScholarWorks (University of Bridgeport) (submittedVersion)
DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics: Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
Keywords: deep learning, deep learning architectures, deep convolutional neural networks, ResNet, recurrent neural networks, reinforcement learning, variational autoencoder, feature extraction, hierarchical representation, training time reduction

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Review of Deep Learning Algorithms and Architectures

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks