Cited by 1,883Neural Networks and ApplicationsAdvanced Neural Network ApplicationsStochastic Gradient Optimization Techniques

The DOI matches its Crossref record; 85 of the 88 references it cites were found in a registry or an index.

Review of Deep Learning Algorithms and Architectures

Open access

Shrestha, Ajay, Mahmood, Ausif·2019·IEEE Access

DOI 10.1109/access.2019.2912200

Open this paper in CoChat — free account Open PDF

Abstract

Deep learning (DL) is playing an increasingly important role in our lives. It has already made a huge impact in areas, such as cancer diagnosis, precision medicine, self-driving cars, predictive forecasting, and speech recognition. The painstakingly handcrafted feature extractors used in traditional learning, classification, and pattern recognition systems are not scalable for large-sized data sets. In many cases, depending on the problem complexity, DL can also overcome the limitations of earlier shallow networks that prevented efficient training and abstractions of hierarchical representations of multi-dimensional training data. Deep neural network (DNN) uses multiple (deep) layers of units with highly optimized algorithms and architectures. This paper reviews several optimization methods to improve the accuracy of the training and to reduce training time. We delve into the math behind training algorithms used in recent deep networks. We describe current shortcomings, enhancements, and implementations. The review also covers different types of deep architectures, such as deep convolution networks, deep residual networks, recurrent neural networks, reinforcement learning, variational autoencoders, and others.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1109/access.2019.2912200

Cites

85 of 85 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 5×
“tion with a 152-layer DNN on the ImageNet dataset [62].”
BackgroundMethodInfluential
Long Short-Term Memory
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 2×
“The input, forget and output gates dictates new data going into the cell, what remains in the cell and the cell values used in the calculation of the output of the LSTM block respectively [41], [43].”
Method
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 4×
“Multilayer network training cost on MNIST dataset using different adaptive learning algorithms [58].”
BackgroundMethodInfluential
Deep learning
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 2×
“7-layer architecture of CNN for character recognition [28].”
BackgroundMethod
Adaptation in Natural and Artificial Systems
Holland, J. H. (1992) · The MIT Press eBooks · doi:10.7551/mitpress/1090.001.0001 · Cited by 47,340
Untitled work
(2014) · Cited by 33,419
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818 · cited 2×
“[59] S. Ioffe and C. Szegedy, ‘‘Batch normalization: Accelerating deep network training by reducing internal covariate shift,’’ CoRR, Feb. 2015.”
Background
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545 · cited 1×
“GRUs are smaller in size than LSTM as they don’t include the output gate, and can perform better than LSTM on only some simpler datasets [44], [45].”
Background
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928

Flagged first, then by citations

Show all 88 references

3 of the 88 references couldn’t be matched to a known paper.

Research works printed but not linked: 10

vapnik, 2015, Learning using privileged information: Similarity control and knowledge transfer, J Mach Learn Res, 16, 2023
ng, 2018, Autoencoders
2018, Amazon EC2 P2 & P3 Instances Amazon EC2 Instance Types
simpson, 2015, Uniform learning in a deep neural network via ‘oddball’ stochastic gradient descent, CoRR
minsky, 1969, Perceptrons An Introduction to Computational Geometry Expanded Edition, 258
2019, Top Deep Learning Github Repositories
sastry, 2005, Genetic Algorithms
10.1017/CBO9780511543241 · DOI
wu, 2016, Google’s neural machine translation system: Bridging the gap between human and machine translation, CoRR
metz, 2019, Turing Award Won by 3 Pioneers in Artificial Intelligence, 3b

Sorted by citations

Items 601–700 of 1,001

Researching the landscape of predictive emissions monitoring system: a review of literature and technology trends
Wang, F., Pang, Y., Bai, L., & Godin, M. (2025) · ENVIRONMENTAL SYSTEMS RESEARCH · doi:10.1186/s40068-025-00403-9 · Cited by 14
“Deep Learning overcomes the limitations of traditional learning systems by using deep neural networks (DNN) with multiple layers and optimized algorithms (Shrestha and Mahmood 2019).”
Experimental motor fault detection and identification of a quadrotor UAV using a hybrid deep learning approach
Khaneghaei, M., Asadi, D., Mowla, M. N., & Dişken, G. (2025) · International Journal of Dynamics and Control · doi:10.1007/s40435-025-01786-4 · Cited by 14
“RMSProp enhances the performance in non-convex optimization problems and facilitates ﬂexible adjustments and faster convergence [41, 44].”
Machine learning-driven insights into the microstructure and properties of high-entropy alloys
Zhang, X., Zhou, W., Li, X., Xu, T., Yu, Y., Zheng, L., Jin, G., & Zhang, S. (2025) · Advanced Powder Materials · doi:10.1016/j.apmate.2025.100331 · Cited by 15
Network of networks: A bibliometric analysis
Karimi, F., Green, D. G., Matouš, P., Varvarigos, E., & Khalilpour, K. R. (2021) · Physica D Nonlinear Phenomena · doi:10.1016/j.physd.2021.132889 · Cited by 15
Min-Max Backpropagation Neural Network to Forecast e-Journal Visitors
Wibawa, A. P., Izdihar, Z. N., Utama, A. B. P., Hernandez, L., & Haviluddin, H. (2021) · International Conference on Artificial Intelligence in Information and Communication (ICAIIC) · doi:10.1109/icaiic51459.2021.9415197 · Cited by 14
“Backpropagation is one of supervised neural network training methods (supervised learning) which consists of three major steps as follows [30]: • Entering data into network input (feed forward).”
Background
Application of Deep Learning Networks to Segmentation of Surface of Railway Tracks
Bojarczak, P., & Nowakowski, W. (2021) · Sensors · doi:10.3390/s21124065 · Cited by 14
UnCNN: A New Directed CNN Model for Isolated Arabic Handwritten Characters Recognition
Khayati, M. E., & Elkettani, Y. (2022) · Arabian Journal for Science and Engineering · doi:10.1007/s13369-022-06652-5 · Cited by 16
“It has even managed to exceed human capabilities in many applications [31].”
Background
Uncertainty Assessment-Based Active Learning for Reliable Fire Detection Systems
Kim, Y.-J., & Kim, W.-T. (2022) · IEEE Access · doi:10.1109/access.2022.3190852 · Cited by 14
Artificial Intelligence-Based Diabetes Diagnosis with Belief Functions Theory
Ellouze, A., Kahouli, O., Ksantini, M., Alsaif, H., Aloui, A., & Kahouli, B. (2022) · Symmetry · doi:10.3390/sym14102197 · Cited by 14
“This eliminates the need for domain expertise and hardware feature extraction [34].”
An immediate analysis of the interaction topic approach to promoting group performance, knowledge convergence, cognitive engagement, and coregulation in online collaborative learning
Zheng, L., Zhong, L., & Fan, Y. (2023) · Education and Information Technologies · doi:10.1007/s10639-023-11588-w · Cited by 17
“DNN is a neural network modeled as a multilayer perceptron trained with algorithms to learn representations without any manual design of feature extractors (Shrestha & Mahmood, 2019).”
BackgroundMethod
An Extensive Review on Deep Learning and Machine Learning Intervention in Prediction and Classification of Types of Aneurysms
Sinnaswamy, R. A., Palanisamy, N., Subramaniam, K., Muthusamy, S., Lamba, R., & Sekaran, S. (2023) · Wireless Personal Communications · doi:10.1007/s11277-023-10532-y · Cited by 14
“[10–12] The classification problems lie under the supervised algorithms (ie) containing labeled classes and unsupervised learning does not contain the labeled classes.”
BackgroundMethod
Memristor Crossbar Circuits Implementing Equilibrium Propagation for On-Device Learning
Oh, S., An, J., Cho, S., Yoon, R., & Min, K. (2023) · Micromachines · doi:10.3390/mi14071367 · Cited by 19
A New Art Design Method Based on AIGC: Analysis from the Perspective of Creation Efficiency
Rao, J., & Xiong, M. (2023) · International Conference on Intelligent Design (ICID) · doi:10.1109/icid60307.2023.10396804 · Cited by 15
“On the one hand, with the breakthrough of the deep learning algorithm[8], AIGC technology with rapid iteration breaks the original formulaic template and small-scale limitations.”
Background
Transforming clinical cardiology through neural networks and deep learning: A guide for clinicians
Sutanto, H. (2024) · Current Problems in Cardiology · doi:10.1016/j.cpcardiol.2024.102454 · Cited by 16
Artificial Intelligence for Quality of Life Study: A Systematic Literature Review
Jannani, A., Sael, N., & Benabbou, F. (2024) · IEEE Access · doi:10.1109/access.2024.3395369 · Cited by 14
“However, when traditional machine learning algorithms are insufficient for solving a problem, DL methods, which consist of neural network techniques utilizing multiple layers to represent data abstractions and build models [20], [21], are employed to improve model accuracy.”
Method
CALSczNet: Convolution Neural Network with Attention and LSTM for the Detection of Schizophrenia Using EEG Signals
Almaghrabi, N., Hussain, M., & Alotaibi, A. (2024) · Mathematics · doi:10.3390/math12131989 · Cited by 14
Enhancing colorectal cancer histology diagnosis using modified deep neural networks optimizer
Elshamy, R., Abu-Elnasr, O., Elhoseny, M., & Elmougy, S. (2024) · Scientific Reports · doi:10.1038/s41598-024-69193-x · Cited by 14
“The model is incentivized to disperse the weights over all the features rather than depending disproportionately on a small number of features by including this term in the loss function 53 .”
Method
A Comprehensive Review of Recent Advances on Intelligence Algorithms and Information Engineering Applications
Zeng, Y., Liao, B., Li, Z., Hua, C., & Li, S. (2024) · IEEE Access · doi:10.1109/access.2024.3461756 · Cited by 18
“For example, in [128], the author extensively reviews fundamental topics such as machine learning and neural network principles, categorization of neural networks (including feedforward, recurrent, radial basis function, self-organizing, modular), various DNN architectures (like convolutional,…”
Background
Beyond connectivity: Artificial intelligence and the internationalisation of digital firms
dos Santos, J. K. F., & Williamson, P. J. (2024) · Information and Organization · doi:10.1016/j.infoandorg.2024.100538 · Cited by 16
Minimum and Maximum Pattern-Based Self-Organized Feature Engineering: Fibromyalgia Detection Using Electrocardiogram Signals
Cambay, V. Y., Hafeez‐Baig, A., Aydemir, E., Tuncer, T., & Doğan, Ş. (2024) · Diagnostics · doi:10.3390/diagnostics14232708 · Cited by 15
A novel deep learning model for spectral analysis: Lightweight ResNet-CNN with adaptive feature compression for oil spill type identification
Zhang, S., Yuan, Y., Wang, Z., Wei, S., Zhang, X., Zhang, T., Song, X., Zou, Y., Wang, J., Chen, F., et al. (2024) · Spectrochimica Acta Part A Molecular and Biomolecular Spectroscopy · doi:10.1016/j.saa.2024.125626 · Cited by 16
Guidelines for clinical trials using artificial intelligence – SPIRIT‐AI and CONSORT‐AI †
McGenity, C., & Treanor, D. E. (2020) · The Journal of Pathology · doi:10.1002/path.5565 · Cited by 14
“Driven by advances in a particular type of neural network called ‘ deep learning ’ [1], computers have achieved human-level performance in a number of tasks previously considered to be some decades in the future [1 – 3].”
A Surrogate Based Computationally Efficient Method to Coordinate Damping Controllers for Enhancement of Probabilistic Small-Signal Stability
Gurung, S., Naetiladdanon, S., & Sangswang, A. (2021) · IEEE Access · doi:10.1109/access.2021.3060502 · Cited by 14
“Deep Neural Network is a machine learning algorithm inspired by the learning ability of our brains and can be used for both regression and classiﬁcation studies [30].”
Method
A milling cutter state recognition method based on multi-source heterogeneous data fusion
Liu, W., Tian, Z., Jiang, X., Liu, S., Zhao, B., Han, Q., Li, J., & Deng, J. (2022) · The International Journal of Advanced Manufacturing Technology · doi:10.1007/s00170-022-10017-5 · Cited by 14
“The dropout method is a kind of effective control measures to overcome the overfitting in deep learning network training [29].”
Method
Fuzzy min–max neural networks: a bibliometric and social network analysis
Kenger, Ö. N., & Özceylan, E. (2023) · Neural Computing and Applications · doi:10.1007/s00521-023-08267-9 · Cited by 14
“…mining techniques such as naive Bayes [3–5], k-nearest neighbour (KNN) [6, 7], decision tree [8, 9], random forest [10, 11], support vector machine (SVM) [12, 13], deep learning [14, 15] and artiﬁcial neural networks (ANNs) [16, 17] are used to transform raw data into meaningful information [18].”
Method
UAV Image High Fidelity Compression Algorithm Based on Generative Adversarial Networks Under Complex Disaster Conditions
Hu, Q., Wu, C., Wu, Y., & Xiong, N. N. (2019) · IEEE Access · doi:10.1109/access.2019.2927809 · Cited by 14
“years, deep learning [13]–[16] has been widely used in data analysis [17], image recognition [18], speech processing [19] and other fields.”
Background
Research on intelligent robot systems for emergency prevention and control of major pandemics
Jiang, Y., Wang, Y., Yi, J., Miao, Z., Liu, L., Zhong, H., & Zhang, H. (2020) · Scientia Sinica Informationis · doi:10.1360/ssi-2020-0107 · Cited by 13
A Scalable System-on-Chip Acceleration for Deep Neural Networks
Shehzad, F., Rashid, M., Sinky, M. H., Alotaibi, S. S., & Zia, M. Y. I. (2021) · IEEE Access · doi:10.1109/access.2021.3094675 · Cited by 14
“…(b) the accuracy requirements in modern ML and AI applications are more demanding [1], [2], (c) the size of neural networks are becoming very large [5], [7] and (d) the network size for various real-life applications is heterogeneous, and therefore, requires a scalable solution [9], [10].”
Background
Supervised deep learning algorithms for process fault detection and diagnosis under different temporal subsequence length of process data
Kai, T. C. Y., Saptoro, A., Putra, Z. A., Lim, K. H., Yeo, W. S., & Sunarso, J. (2025) · Applied Intelligence · doi:10.1007/s10489-025-06711-y · Cited by 14
“In recent years, the noteworthy architectures that were developed and employed in FDD applications were convolution neural networks, long short-term memory, and transformers [17, 47, 48].”
Influential
Biomedical image classification using deep convolutional neural networks – overview
Reddy, M. B. S., & Rana, P. P. (2021) · IOP Conference Series Materials Science and Engineering · doi:10.1088/1757-899x/1022/1/012020 · Cited by 13
“CNN[23] is mostly used for analyzing visual images.”
BackgroundMethod
Deep learning framework for component identification
Sureshkumar, S., Mathan, G., Praseetha, R. I., & Govindarajan, M. (2022) · International Journal of Information Technology · doi:10.1007/s41870-022-00895-z · Cited by 13
Machine Intelligence in Biomedical Data Modeling, Processing, and Analysis
Mujkic, A., Baralic, E., Ombasic, A., Spahić, L., Pokvić, L. G., & Badnjević, A. (2022) · Mediterranean Conference on Embedded Computing (MECO) · doi:10.1109/meco55406.2022.9797164 · Cited by 13
A Synergic Approach of Deep Learning towards Digital Additive Manufacturing: A Review
Pratap, A., Sardana, N., Utomo, S., Ayeelyan, J., Karthikeyan, P., & Hsiung, P. (2022) · Algorithms · doi:10.3390/a15120466 · Cited by 16
Cost-Effective Aerial Inventory of Spruce Seedlings Using Consumer Drones and Deep Learning Techniques with Two-Stage UAV Flight Patterns
Lopatin, E., & Poikonen, P. (2023) · Forests · doi:10.3390/f14050973 · Cited by 13
Citrus Fruit Disease Detection Techniques: A Survey and Comparative Analysis of Relevant Approaches
Dhiman, P., Kaur, A., & Kukreja, V. (2023) · International Journal of Computing and Digital Systems · doi:10.12785/ijcds/140187 · Cited by 13
“…leaf-miner 95.2% and rust mite 96.8% [21] 165 grapefruits 92.70% [24] 210 samples of naval orange having 80 healthy and 130 defective fruits 98.60% [25] Early-ripe citrus fruit images from China (Wenzho) 96.50% [26] For the tests, 100 orange fruits were taken from which 50 were vaguely damaged on…”
BackgroundInfluential
Artificial Intelligence and Innovative Applications in Special Education
Şen, N., & Akbay, T. (2023) · Öğretim Teknolojisi ve Hayat Boyu Öğrenme Dergisi - Instructional Technology and Lifelong Learning · doi:10.52911/itall.1297978 · Cited by 13
A review: Applications of machine learning and deep learning in aerospace engineering and aero-engine engineering
Wang, W., & Ma, J. (2024) · Advances in Engineering Innovation · doi:10.54254/2977-3903/6/2024060 · Cited by 13
“[34] In order to teach deep neural networks, the Backpropagation algorithm (Backpropagation) [35] is commonly used.”
Toward explainable AI in radiology: Ensemble-CAM for effective thoracic disease localization in chest X-ray images using weak supervised learning
Aasem, M., & Iqbal, M. J. (2024) · Frontiers in Big Data · doi:10.3389/fdata.2024.1366415 · Cited by 25
“The most highlighted approach in machine learning is known as deep learning (DL) for its ability to learn complex and meaningful patterns from large volume of data (LeCun et al., 2015; Voulodimos et al., 2018; Shrestha and Mahmood, 2019; Georgiou et al., 2020; Mahony et al., 2020).”
Background
Review of vortex beam orbital angular momentum mode detection methods
Lv, J., Liu, C., Shen, M., Wang, D., Si, G., & Ou, Q. (2024) · Materials Today Communications · doi:10.1016/j.mtcomm.2024.109767 · Cited by 13
A Tutorial on the Use of Physics-Informed Neural Networks to Compute the Spectrum of Quantum Systems
Brevi, L., Mandarino, A., & Prati, E. (2024) · Technologies · doi:10.3390/technologies12100174 · Cited by 17
“The deep learning (DL) approach is one of the most successful machine learning (ML) paradigms, carrying outstanding results on several complex cognitive tasks [1,2].”
Background
Deep learning-based surrogate capacity models and multi-objective fragility estimates for reinforced concrete frames
Xing, L., Gardoni, P., Song, G., & Zhou, Y. M. (2025) · Computer Methods in Applied Mechanics and Engineering · doi:10.1016/j.cma.2025.117928 · Cited by 13
Generalized Zero Shot Learning via Synthesis Pseudo Features
Li, C., Ye, X., Yang, H., Han, Y., Li, X., & Jia, Y. (2019) · IEEE Access · doi:10.1109/access.2019.2925093 · Cited by 13
“Supervised learning methods have achieved signiﬁcant successes in many areas with sufﬁcient labeled training data provided for each class [1], [2].”
Background
Radial Basis Function Network with Differential Privacy
Bugshan, N., Khalil, I., Moustafa, N., Almashor, M., & Abuadbba, A. (2021) · Future Generation Computer Systems · doi:10.1016/j.future.2021.09.013 · Cited by 13
Deep learning in pervasive health monitoring, design goals, applications, and architectures: An overview and a brief synthesis
Boulemtafes, A., Khemissa, H., Derki, M. S., Amira, A., & Djedjig, N. (2021) · Smart Health · doi:10.1016/j.smhl.2021.100221 · Cited by 15
Machine learning and deep learning enabled fuel sooting tendency prediction from molecular structure
Li, R., Herreros, J. M., Tsolakis, A., & Yang, W. (2021) · Journal of Molecular Graphics and Modelling · doi:10.1016/j.jmgm.2021.108083 · Cited by 13
Point Cloud Semantic Segmentation of Concrete Surface Defects Using Dynamic Graph CNN
Bahreini, F., & Amin, H. (2021) · International Symposium on Automation and Robotics in Construction (ISARC) · doi:10.22260/isarc2021/0053 · Cited by 13
“DNNs or deep feedforward networks utilize multiple deep layers along with highly optimized algorithms to learn from trained data sets without the process of manual feature extraction [28].”
Use of Different Hyperparameter Optimization Algorithms in ANN for Predicting the Compressive Strength of Concrete Containing Calcined Clay
Ghanemi, A. H., & Tarighat, A. (2022) · Practice Periodical on Structural Design and Construction · doi:10.1061/(asce)sc.1943-5576.0000683 · Cited by 13
Recipe Bot: The Application of Conversational AI in Home Cooking Assistant
Chu, J. (2021) · International Conference on Big Data & Artificial Intelligence & Software Engineering (ICBASE) · doi:10.1109/icbase53849.2021.00136 · Cited by 13
Hierarchical quality-diversity for online damage recovery
Allard, M., Smith, S. C., Chatzilygeroudis, K. I., & Cully, A. (2022) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/3512290.3528751 · Cited by 13
“Robot learning has significantly advanced in the last years [1, 10, 34] by combining powerful optimisation methods, including Deep Learning, Reinforcement Learning and Quality-Diversity (QD) [5, 48, 52, 54].”
Method
A disaster classification application using convolutional neural network by performing data augmentation
Sobhana, M., Ch, S. C., Indira, D. N. V. S. L. S., & Kumar, K. K. (2022) · Indonesian Journal of Electrical Engineering and Computer Science · doi:10.11591/ijeecs.v27.i3.pp1712-1720 · Cited by 13
Graph Convolutional Networks with POS Gate for Aspect-Based Sentiment Analysis
Kim, D., Kim, Y.-J., & Jeong, Y.-S. (2022) · Applied Sciences · doi:10.3390/app121910134 · Cited by 13
“These different DL models have their own merits, as discussed in [11]; for example, some studies employed CNN to tackle the task of sentiment classification as the CNN is known to be efficient and effective in analyzing local patterns [12,13].”
Method
A comprehensive comparison study of traditional classifiers and deep neural networks for forest fire detection
Akyol, K. (2023) · Cluster Computing · doi:10.1007/s10586-023-04003-z · Cited by 13
“It consists of multiple hidden layers between input and output layers [49].”
An automated framework for evaluation of deep learning models for splice site predictions
Zabardast, A., Tamer, E. G., Son, Y. A., & Yılmaz, A. (2023) · Scientific Reports · doi:10.1038/s41598-023-34795-4 · Cited by 15
Soft Fault Diagnosis for DC–DC Converter Based on Improved ResNet-50
Han, W., Cheng, L., Han, W., Yu, C., Hao, Z., & Yin, Z. (2023) · IEEE Access · doi:10.1109/access.2023.3300692 · Cited by 13
“Deep learning is gaining ground in the field of computer vision [36].”
Background
Cooperative Coupled Generative Networks for Generalized Zero-Shot Learning
Sun, L., Song, J., Wang, Y., & Li, B. (2020) · IEEE Access · doi:10.1109/access.2020.3000347 · Cited by 13
“Due to the availability of big data and the development of artiﬁcial neural networks, supervised learning methods have achieved great success [1].”
Background
Novel pricing strategies for revenue maximization and demand learning using an exploration–exploitation framework
Elreedy, D., Atiya, A. F., & Shaheen, S. I. (2021) · Soft Computing · doi:10.1007/s00500-021-06047-y · Cited by 13
Generative language models potential for requirement engineering applications: insights into current strengths and limitations
Saleem, S., Asim, M. N., van Elst, L., & Dengel, A. R. (2025) · Complex & Intelligent Systems · doi:10.1007/s40747-024-01707-6 · Cited by 13
“Many research studies [29] have concluded that deep learning algorithms produce superior performance with larger training data but struggle to achieve comparable performance on smaller datasets [30].”
Strawberry Diseases Identification From Its Leaf Images Using Convolutional Neural Network
Ramdani, A., & Suyanto, S. (2021) · IEEE International Conference on Industry 4.0, Artificial Intelligence, and Communications Technology (IAICT) · doi:10.1109/iaict52856.2021.9532573 · Cited by 12
Crop Disease Prediction using Deep Learning Techniques - A Review
Sharma, G., & Shrivastava, G. K. (2022) · International Journal of Computer Science and Engineering · doi:10.14445/23488387/ijcse-v9i4p104 · Cited by 12
YOLOv5-PD: A Model for Common Asphalt Pavement Defects Detection
Xu, Y., Sun, F., & Wang, L. (2022) · Journal of Sensors · doi:10.1155/2022/7530361 · Cited by 12
“In order to solve the above problems of processing images, some studies have proposed convolutional neural networks [14].”
Background
Evaluation of Deep Learning Technique on Working Model of Self-driving Car—A Review
Sangwan, S., Singh, G., Bangia, A., & Goswami, V. S. (2023) · Lecture notes in networks and systems · doi:10.1007/978-981-19-9858-4_23 · Cited by 12
Passing Vehicle Road Occupancy Detection Using the Magnetic Sensor Array
Balamutas, J., Navikas, D., Markevičius, V., Čepėnas, M., Valinevičius, A., Žilys, M., Frivaldský, M., Li, Z., & Andriukaitis, D. (2023) · IEEE Access · doi:10.1109/access.2023.3278986 · Cited by 12
“These tires are still popular off-road because they are lighter than radials [32].”
Background
Machine Learning Assisting the Prediction of Clinical Outcomes following Nucleoplasty for Lumbar Degenerative Disc Disease
Chiu, P.-F., Chang, K., Lai, Y.-C., Wu, K.-C., Wang, K.-P., Chiu, Y.-P., Ji, H., Kao, C., & Chiu, C. (2023) · Diagnostics · doi:10.3390/diagnostics13111863 · Cited by 13
“Deep learning (DL) is a branch of ML in which algorithms learn data features by using artificial neural network architecture [43].”
Background
Tomato Plant Disease Detection using Deep Learning Based Techniques: A Comparative study
Deepika, P., & Arthi, B. (2023) · International Conference on Electrical, Electronics, Information and Communication Technologies (ICEEICT) · doi:10.1109/iceeict56924.2023.10157436 · Cited by 12
Techniques and Applications for Deep Learning: A Review
Alhijaj, J. A., & Khudeyer, R. S. (2023) · Journal of Al-Qadisiyah for Computer Science and Mathematics · doi:10.29304/jqcm.2023.15.2.1236 · Cited by 13
“As illustrated in Figure 4 [14], Signals pass through blocks of memory cell state in an LSTM, It is managed by input, forget, and output gates.”
Background
Emotion recognition by skeleton-based spatial and temporal analysis
Oğuz, A., & Ertuğrul, Ö. F. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.121981 · Cited by 12
Applications of Machine Learning in Mobile Networking
Hassan, M. H. H. (2023) · Journal of Smart Internet of Things · doi:10.2478/jsiot-2023-0003 · Cited by 12
Hyperparameter Optimization for Convolutional Neural Networks using the Salp Swarm Algorithm
Abdulsaed, E. H., Alabbas, M., & Khudeyer, R. S. (2023) · Informatica · doi:10.31449/inf.v47i9.5148 · Cited by 12
Deep learning algorithms and their relevance: A review
Nisha, C. M., & Thangarasu, N. (2023) · International Journal of Data Informatics and Intelligent Computing · doi:10.59461/ijdiic.v2i4.78 · Cited by 12
Review on the Application of the Attention Mechanism in Sensing Information Processing for Dynamic Welding Processes
Xu, J., Liu, Q., Xu, Y., Xiao, R., Hou, Z., & Chen, S. (2024) · Journal of Manufacturing and Materials Processing · doi:10.3390/jmmp8010022 · Cited by 15
“…feed-forward neural networks (FNN), convolution neural network (CNN), recurrent neural network (RNN), restricted Boltzmann machine (RBM), etc. [72] In the field of welding sensing, deep learning algorithms are mainly used in extracting characteristic information from sensor data and relating…”
Method
Conglomeration of deep neural network and quantum learning for object detection: Status quo review
Sinha, P. K., & Marimuthu, R. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.111480 · Cited by 14
Ensemble Learning based on CNN and Transformer Models for Leaf Diseases Classification
Li, L.-H., & Tanone, R. (2024) · International Conference on Ubiquitous Information Management and Communication (IMCOM) · doi:10.1109/imcom60618.2024.10418393 · Cited by 14
“This is based on the difference in the number of parameters [8], [9], the training time [10] and dataset, which inherently impacts the accuracy of the model.”
Background
A cost-sensitive deep neural network-based prediction model for the mortality in acute myocardial infarction patients with hypertension on imbalanced data
Zheng, H., Sherazi, S. W. A., & Lee, J. Y. (2024) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2024.1276608 · Cited by 13
“Deep learning (DL), one of the stated methods in ML, has advanced signi ﬁ cantly in the previous ten years due to its powerful computational capacity (19).”
Background
Energy market trading in green microgrids under information vulnerability of renewable energies: A data-driven approach
Sabzevari, K., Habib, S., Tabar, V. S., Shaillan, H. M., Hassan, Q., & Muyeen, S. M. (2024) · Energy Reports · doi:10.1016/j.egyr.2024.03.059 · Cited by 12
MLSea: A Semantic Layer for Discoverable Machine Learning
Dasoulas, I., Yang, D., & Dimou, A. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-60635-9_11 · Cited by 12
“The taxonomy re-uses MEX vocabulary algorithm classes, the Paper with Code methods collection 11 , and ML surveys that organize ML algorithms from diverse ML fields [1, 3, 5, 9, 17, 20, 36, 44, 55, 58, 60, 61, 62, 66].”
Method
A systematic literature analysis of multi-organ cancer diagnosis using deep learning techniques
Kaur, J., & Kaur, P. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.108910 · Cited by 14
A Comprehensive Review to Understand the Definitions, Advantages, Disadvantages and Applications of Machine Learning Algorithms
Rahaman, M. J. (2024) · International Journal of Computer Applications · doi:10.5120/ijca2024923868 · Cited by 12
“Evolutionary learning is extensively used in production related work such as manufacturing, agriculture, power and energy etc. Deep learning also has different types of algorithms for example neural networks based on the nervous system and brain of a human [3].”
Scalable, Advanced Machine Learning Based Approaches for Stellar Flare Identification: Application to TESS Short-cadence Data and Analysis of a New Flare Catalog
Lin, C.-L., Apai, D., Giampapa, M. S., & Ip, W.-H. (2024) · The Astronomical Journal · doi:10.3847/1538-3881/ad7956 · Cited by 17
A review of recent hybridized machine learning methodologies for time series forecasting on water-related variables
Koh, V. K. Z., Li, Y., Kek, X. Y., Shafiee, E., Lin, Z., & Wen, B. (2025) · Journal of Hydrology · doi:10.1016/j.jhydrol.2025.132909 · Cited by 12
Maritime Risk Assessment: A Cutting-Edge Hybrid Model Integrating Automated Machine Learning and Deep Learning with Hydrodynamic and Monte Carlo Simulations
Balas, E. A., & Balas, C. E. (2025) · Journal of Marine Science and Engineering · doi:10.3390/jmse13050939 · Cited by 12
Hybrid AI and semiconductor approaches for power quality improvement
Chinthaginjala, R., Srinivasulu, A., Agrawal, A., Kim, T., Tera, S. P., & Ahmad, S. (2025) · Scientific Reports · doi:10.1038/s41598-025-11116-5 · Cited by 13
Intelligent detection and classification of tetracycline drugs by rare earth fluorescence sensing platform based on deep learning algorithm and STM32 microcontroller
Zhu, Z., Xu, J. B., Chen, X., Li, Y., Zhang, L., Jia, L., Li, J., Zhu, T., & Zhao, T. (2025) · Sensors and Actuators B Chemical · doi:10.1016/j.snb.2025.138638 · Cited by 12
Characterization of complex fluvio–deltaic deposits in Northeast China using multi-modal machine learning fusion
Boateng, C. D., Fu, L., & Danuor, S. K. (2020) · Scientific Reports · doi:10.1038/s41598-020-70382-7 · Cited by 13
Quality Control in Fiore Sardo PDO Cheese: Detection of Heat Treatment Application and Production Chain by MRI Relaxometry and Image Analysis
Anedda, R., Melis, R., & Curti, E. (2021) · Dairy · doi:10.3390/dairy2020023 · Cited by 12
“Deep learning is a subfield of artificial intelligence (AI) concerned with specific algorithms for thinking simulations and data processing inspired by the neurons of the brain [31].”
BackgroundMethod
Deep Learning with Convolutional Neural Network for Proportional Control of Finger Movements from surface EMG Recordings
Mendez, V., Pollina, L., Artoni, F., & Micera, S. (2021) · International IEEE/EMBS Conference on Neural Engineering · doi:10.1109/ner49283.2021.9441095 · Cited by 13
“For the two DL approaches (RAW and FFT), we started from a VGG-like architecture [24] and optimized the shape of the network: the number of convolutional layers before each average pooling layer (up to 3), the number of pooling layer (up to 5) and the number of filters for each convolutional layer…”
Method
Decision-Theoretic Rough Sets based automated scheme for object and background classification in unevenly illuminated images
Wagh, M. P., & Nanda, P. K. (2022) · Applied Soft Computing · doi:10.1016/j.asoc.2022.108596 · Cited by 12
An Online Approach for DNN Model Caching and Processor Allocation in Edge Computing
Chen, Z., Zhang, S., Ma, Z., Zhang, S., Qian, Z., Xiao, M., Wu, J., & Lu, S. (2022) · International Workshop on Quality of Service · doi:10.1109/iwqos54832.2022.9812874 · Cited by 12
“With the breakthrough and development of deep learning [11], artificial intelligence applications are developing rapidly.”
Background
Deep learning based single shot multiple phase derivative retrieval method in multi-wave digital holographic interferometry
Vithin, A. V. S., Ramaiah, J., & Gannavarpu, R. (2022) · Optics and Lasers in Engineering · doi:10.1016/j.optlaseng.2022.107442 · Cited by 12
Comparison of Machine Learning Models in Forecasting Reservoir Water Level
Aquil, M. A. I., & Ishak, W. H. W. (2023) · Journal of Advanced Research in Applied Sciences and Engineering Technology · doi:10.37934/araset.31.3.137144 · Cited by 12
Biomedical Question Answering: A Survey of Methods and Datasets
Kaddari, Z., Mellah, Y., Berrich, J., Bouchentouf, T., & Belkasmi, M. G. (2020) · International Conference on Intelligent Computing in Data Sciences (ICDS) · doi:10.1109/icds50568.2020.9268742 · Cited by 12
“The last ﬁve years have seen an explosion of research in this area in terms of new datasets [3]–[8], and new deep learning [9] based methods and models [10]–[14] that outperformed all predecessors.”
Background
Application of deep neural network (DNN) for experimental liquid-liquid equilibrium data of water + butyric acid + 5-methyl-2-hexanone ternary systems
Bekri, S., Özmen, D., Türkmenoğlu, A., & Özmen, A. (2021) · Fluid Phase Equilibria · doi:10.1016/j.fluid.2021.113094 · Cited by 11
Artificial intelligence and technology in weather forecasting and renewable energy systems
Dutt, V., & Sharma, S. (2022) · Elsevier eBooks · doi:10.1016/b978-0-323-90396-7.00009-2 · Cited by 11
Artificial Intelligence and Information Processing: A Systematic Literature Review
Lin, K.-Y., & Chang, K. (2023) · Mathematics · doi:10.3390/math11112420 · Cited by 19
Development of a Smart Hospital Bed Based on Deep Learning to Monitor Patient Conditions
Ayouni, S., Maddeh, M., Al-Otaibi, S. T., Alazzam, M. B., Alturki, N. M., & Hajjej, F. (2023) · Journal of Disability Research · doi:10.57197/jdr-2023-0017 · Cited by 12
“Recent research suggests that systems based on deep learning may improve the effectiveness, precision, and adaptability of healthcare systems (LeCun et al., 2015; Shrestha and Mahmood, 2019).”
Background
Mean-Median Smoothing Backpropagation Neural Network to Forecast Unique Visitors Time Series of Electronic Journal
Wibawa, A. P. (2023) · Journal of Applied Data Sciences · doi:10.47738/jads.v4i3.97 · Cited by 11
“hidden layer, and the output layer [42].”
Multi-phase hybrid bidirectional deep learning model integrated with Markov chain Monte Carlo bivariate copulas function for streamflow prediction
Iqbal, A., & Siddiqi, T. A. (2023) · Stochastic Environmental Research and Risk Assessment · doi:10.1007/s00477-023-02632-9 · Cited by 11
“The DL framework has many nonlinear feature extraction layers in comparison to shallow learning’s single layer of feature transformations that enhance the model’s capacity to learn the complexity of internal architectures for improved model performance (Shrestha and Mahmood 2019).”
Genetic-Algorithm-Aided Deep Reinforcement Learning for Multi-Agent Drone Delivery
Tarhan, A. F., & Üre, N. K. (2024) · Drones · doi:10.3390/drones8030071 · Cited by 11
Ovarian Cancer Detection Based on Elman Recurrent Neural Network
Farhan, H. R., Kod, M., Taqi, A. M., & Ghazi, S. A. (2024) · Periodica Polytechnica Electrical Engineering and Computer Science · doi:10.3311/ppee.23081 · Cited by 11
“Besides, GD reduces the sum of the squared errors by modifying the function parameters towards the abrupt descent [28].”
BackgroundMethodInfluential
Modeling CO2 Loading Capacity of Diethanolamine (DEA) Aqueous Solutions Using Advanced Deep Learning and Machine Learning Algorithms: Application to Carbon Capture
Mahmoudzadeh, A., Hadavimoghaddam, F., Atashrouz, S., Abedi, A., Abuswer, M. A., Mohaddespour, A., & Hemmati‐Sarapardeh, A. (2024) · Korean Journal of Chemical Engineering · doi:10.1007/s11814-024-00094-5 · Cited by 11
“Deep neural network (DNN) was created using artificial neural network (ANN) as a foundation [79].”
Collaborative approach for swarm robot systems based on distributed DRL
Bar, N. F., & Karaköse, M. (2024) · Engineering Science and Technology an International Journal · doi:10.1016/j.jestch.2024.101701 · Cited by 11

Authors: Shrestha, A., & Mahmood, A.
Affiliations: University of Bridgeport (US)
ORCID: Shrestha, Ajay
Mahmood, Ausif
Venue: IEEE Access
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Published: 2019
Volume: 7
Pages: 53040-53065
ISSN: 2169-3536 (ISSN-L)
DOI: 10.1109/access.2019.2912200
OpenAlex: W2942231644
Cited by: 1,883
References: 88
Citations by year: 2026: 116 · 2025: 317 · 2024: 360 · 2023: 375 · 2022: 297 · 2021: 259 · 2020: 139 · 2019: 20
FWCI: 81.55
Type: journal article
Language: en
Open access: gold
doi.org
Licence: ieeexplore.ieee.org/Xplorehelp/downloads/license-information/OAPA.html (vor)
Locations: IEEE Access (publishedVersion)
University of Bridgeport ScholarWorks (University of Bridgeport) (submittedVersion)
DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics: Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
Keywords: deep learning, deep learning architectures, deep convolutional neural networks, ResNet, recurrent neural networks, reinforcement learning, variational autoencoder, feature extraction, hierarchical representation, training time reduction

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Review of Deep Learning Algorithms and Architectures

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks