The DOI matches its Crossref record; 85 of the 88 references it cites were found in a registry or an index.
Review of Deep Learning Algorithms and Architectures
Open accessAbstract
Deep learning (DL) is playing an increasingly important role in our lives. It has already made a huge impact in areas, such as cancer diagnosis, precision medicine, self-driving cars, predictive forecasting, and speech recognition. The painstakingly handcrafted feature extractors used in traditional learning, classification, and pattern recognition systems are not scalable for large-sized data sets. In many cases, depending on the problem complexity, DL can also overcome the limitations of earlier shallow networks that prevented efficient training and abstractions of hierarchical representations of multi-dimensional training data. Deep neural network (DNN) uses multiple (deep) layers of units with highly optimized algorithms and architectures. This paper reviews several optimization methods to improve the accuracy of the training and to reduce training time. We delve into the math behind training algorithms used in recent deep networks. We describe current shortcomings, enhancements, and implementations. The review also covers different types of deep architectures, such as deep convolution networks, deep residual networks, recurrent neural networks, reinforcement learning, variational autoencoders, and others.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“As consistent with the findings for the second and third periods, ANN, Deep Neural Network (DNN), SVM, GA, LR, and DT are among the prominent AI techniques that gained strong traction in recent years (Şerban and Lytras 2020; Shrestha and Mahmood 2019), so are ML, DL, and cognitive AI models in relation to the planning and development of smart cities (Kamrowska-Załuska 2021; Şerban and Lytras 2020”
From Environmentally sustainable smart cities and their converging AI, IoT, and big data technologies and solutions: an integrated approach to an extensive literature review (2023) · cites this paper as Shrestha and Mahmood 2019
In the citing paper
“Deep learning is gaining traction as a type of ANN with more layers of units (Shrestha and Mahmood, 2019).”
From Artificial intelligence for sustainability: Challenges, opportunities, and a research agenda (2020) · cites this paper as Shrestha and Mahmood, 2019
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 5×
“tion with a 152-layer DNN on the ImageNet dataset [62].”
BackgroundMethodInfluentialHochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 2×
“The input, forget and output gates dictates new data going into the cell, what remains in the cell and the cell values used in the calculation of the output of the LSTM block respectively [41], [43].”
MethodAdam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 4×
“Multilayer network training cost on MNIST dataset using different adaptive learning algorithms [58].”
BackgroundMethodInfluentialLeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 2×
“7-layer architecture of CNN for character recognition [28].”
BackgroundMethodAdaptation in Natural and Artificial Systems
Holland, J. H. (1992) · The MIT Press eBooks · doi:10.7551/mitpress/1090.001.0001 · Cited by 47,340
(2014) · Cited by 33,419
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818 · cited 2×
“[59] S. Ioffe and C. Szegedy, ‘‘Batch normalization: Accelerating deep network training by reducing internal covariate shift,’’ CoRR, Feb. 2015.”
BackgroundLearning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545 · cited 1×
“GRUs are smaller in size than LSTM as they don’t include the output gate, and can perform better than LSTM on only some simpler datasets [44], [45].”
BackgroundReducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
Flagged first, then by citations
3 of the 88 references couldn’t be matched to a known paper.
Research works printed but not linked: 10
- vapnik, 2015, Learning using privileged information: Similarity control and knowledge transfer, J Mach Learn Res, 16, 2023
- ng, 2018, Autoencoders
- 2018, Amazon EC2 P2 & P3 Instances Amazon EC2 Instance Types
- simpson, 2015, Uniform learning in a deep neural network via ‘oddball’ stochastic gradient descent, CoRR
- minsky, 1969, Perceptrons An Introduction to Computational Geometry Expanded Edition, 258
- 2019, Top Deep Learning Github Repositories
- sastry, 2005, Genetic Algorithms
- 10.1017/CBO9780511543241 · DOI
- wu, 2016, Google’s neural machine translation system: Bridging the gap between human and machine translation, CoRR
- metz, 2019, Turing Award Won by 3 Pioneers in Artificial Intelligence, 3b
Cited by
Sorted by citations
Review of deep learning: concepts, CNN architectures, challenges, applications, future directions
Alzubaidi, L., Zhang, J., Humaidi, A. J., Al-Dujaili, A., Duan, Y., Al-Shamma, O., Santamaría, J., Fadhel, M. A., Al-Amidie, M., & Farhan, L. (2021) · Journal of Big Data · doi:10.1186/s40537-021-00444-8 · Cited by 7,906
Scientific Machine Learning Through Physics–Informed Neural Networks: Where we are and What’s Next
Cuomo, S., Cola, V. S. D., Giampaolo, F., Rozza, G., Raissi, M., & Piccialli, F. (2022) · Journal of Scientific Computing · doi:10.1007/s10915-022-01939-z · Cited by 3,262
“The main principle of reinforcement learning is to have an agent learn from its surroundings through exploration and by defining a reward (Shrestha and Mahmood, 2019).”
BackgroundArtificial intelligence for sustainability: Challenges, opportunities, and a research agenda
Nishant, R., Kennedy, M., & Corbett, J. (2020) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2020.102104 · Cited by 1,203
Taye, M. M. (2023) · Computers · doi:10.3390/computers12050091 · Cited by 1,275
“…rules to function; rather, it leverages a massive amount of data to map the provided input to particular labels. is created utilising multiple layers of algorithms (arti fi cial neural networks, or ANNs), each of which delivers a unique interpretation of the data provided to it [25,26].”
BackgroundDeep learning modelling techniques: Current progress, applications, advantages, and challenges
Ahmed, S. F., Alam, M. S. B., Hassan, M., Rozbu, M. R., Ishtiak, T., Rafa, N., Mofijur, M., Shawkat Ali, A. B. M., & Gandomi, A. H. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-023-10466-8 · Cited by 1,079
Alzubaidi, L., Bai, J., Al-Sabaawi, A., Santamaría, J., Albahri, A. S., Al-dabbagh, B. S. N., Fadhel, M. A., Manoufali, M., Zhang, J., Al-Timemy, A. H., et al. (2023) · Journal Of Big Data · doi:10.1186/s40537-023-00727-2 · Cited by 1,057
AI revolutionizing industries worldwide: A comprehensive overview of its diverse applications
Rashid, A. B., & Kausik, M. A. K. (2024) · Hybrid Advances · doi:10.1016/j.hybadv.2024.100277 · Cited by 764
AI applications to medical images: From machine learning to deep learning
Castiglioni, I., Rundo, L., Codari, M., Leo, G. D., Salvatore, C., Interlenghi, M., Gallivanone, F., Cozzi, A., D’Amico, N. C., & Sardanelli, F. (2021) · Physica Medica · doi:10.1016/j.ejmp.2021.02.006 · Cited by 779
Taye, M. M. (2023) · Computation · doi:10.3390/computation11030052 · Cited by 812
Machine Learning for Electrocatalyst and Photocatalyst Design and Discovery
Mai, H., Le, T. C., Chen, D., Winkler, D. A., & Caruso, R. A. (2022) · Chemical Reviews · doi:10.1021/acs.chemrev.2c00061 · Cited by 588
Application of deep learning algorithms in geotechnical engineering: a short critical review
Zhang, W., Li, H., Li, Y., Liu, H., Chen, Y., & Ding, X. (2021) · Artificial Intelligence Review · doi:10.1007/s10462-021-09967-1 · Cited by 556
Loey, M., Smarandache, F., & M. Khalifa, N. E. (2020) · Symmetry · doi:10.3390/sym12040651 · Cited by 538
“Table 2 shows a series of major contributions in the field of the neural network to deep learning [21].”
BackgroundVidal, C., Malysz, P., Kollmeyer, P., & Emadi, A. (2020) · IEEE Access · doi:10.1109/access.2020.2980961 · Cited by 512
“For a more detailed explanation about the methods and its basic math, interested readers are referred to [9], [10].”
BackgroundMethodVali, A., Comai, S., & Matteucci, M. (2020) · Remote Sensing · doi:10.3390/rs12152495 · Cited by 507
“The authors of [136] provide a review discussing the optimisation methods vs.”
BackgroundDeep Learning for Change Detection in Remote Sensing Images: Comprehensive Review and Meta-Analysis
Khelifi, L., & Mignotte, M. (2020) · IEEE Access · doi:10.1109/access.2020.3008036 · Cited by 480
“Hence, this function D(s, a) is represented as the summation of current reward R and future reward α · max[D(s, a)] in the following way [120]:”
BackgroundA review of machine learning in processing remote sensing data for mineral exploration
Shirmard, H., Farahbakhsh, E., Müller, R. D., & Chandra, R. (2021) · Remote Sensing of Environment · doi:10.1016/j.rse.2021.112750 · Cited by 448
“Deep learning methods have the ability to model complex and large datasets and are loosely grouped into feedforward and recurrent neural network architectures (Shrestha and Mahmood, 2019).”
BackgroundHoeser, T., & Kuenzer, C. (2020) · Remote Sensing · doi:10.3390/rs12101667 · Cited by 404
“Such a network then transports values from the input layer via the weights over all hidden layers to the output layer and can be seen as a non-linear function of higher order, mapping input values to outputs [5,25].”
BackgroundApplication of machine learning in groundwater quality modeling - A comprehensive review
Haggerty, R., Sun, J., Yu, H., & Li, Y. (2023) · Water Research · doi:10.1016/j.watres.2023.119745 · Cited by 359
“…Sze et al., 2017; Zhao et al., 2017) optimized for machine learning algorithms (MLAs), especially for deep learning (Emmert-Streib et al., 2020; LeCun et al., 2015; Shrestha and Mahmood, 2019), data-centric ML methods started to emerge in recent years with a remarkable leap forward in performance.”
BackgroundEbayyeh, A. A. R. M. A., & Mousavi, A. (2020) · IEEE Access · doi:10.1109/access.2020.3029127 · Cited by 325
“A special case of DNNs is when the matrix multiplications include convolutional filter operations, which is common in DNNs that are designed for image and video analysis.”
BackgroundInfluentialBibri, S. E., Alexandre, A., Sharifi, A., & Krogstie, J. (2023) · Energy Informatics · doi:10.1186/s42162-023-00259-2 · Cited by 299
“…Neural Network (DNN), SVM, GA, LR, and DT are among the prominent AI techniques that gained strong traction in recent years (Şerban and Lytras 2020; Shrestha and Mahmood 2019), so are ML, DL, and cognitive AI models in relation to the planning and development of smart cities (Kamrowska-Załuska…”
BackgroundNovel Transfer Learning Approach for Medical Imaging with Limited Labeled Data
Alzubaidi, L., Al‐Amidie, M., Al‐Asadi, A., Humaidi, A. J., Al-Shamma, O., Fadhel, M. A., Zhang, J., Santamaría, J., & Duan, Y. (2021) · Cancers · doi:10.3390/cancers13071590 · Cited by 318
“Furthermore, these layers efficiently decrease the network sensitivity to image distortions, scales, and shifts along with shared weights and local receptive fields (inside the same filter) [38].”
BackgroundMethodManoharan, A., Begam, M., Aparow, V. R., & Sooriamoorthy, D. (2022) · Journal of Energy Storage · doi:10.1016/j.est.2022.105384 · Cited by 304
Ren, Z. G., & Du, C. (2023) · Energy Reports · doi:10.1016/j.egyr.2023.01.108 · Cited by 292
The Role of Machine Learning Algorithms for Diagnosing Diseases
Ibrahim, I., & Abdulazeez, A. M. (2021) · Journal of Applied Science and Technology Trends · doi:10.38094/jastt20179 · Cited by 281
“The application of these methods is used for a number of medical functions, such as diagnosis of diseases [25,26].”
Blockchain and AI-Based Solutions to Combat Coronavirus (COVID-19)-Like Epidemics: A Survey
Nguyen, D. C., Ding, M., Pathirana, P. N., & Seneviratne, A. (2021) · IEEE Access · doi:10.1109/access.2021.3093633 · Cited by 286
“adjustment of weight values among perceptrons [62].”
Deploying artificial intelligence for climate change adaptation
Leal Filho, W., Wall, T., Rui Mucova, S. A., Nagy, G. J., Balogun, A.-L., Luetz, J. M., Ng, A. W., Kovaleva, M., Safiul Azam, F. M., Alves, F., et al. (2022) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2022.121662 · Cited by 266
Internet of Things: Evolution, Concerns and Security Challenges
Malhotra, P., Singh, Y., Anand, P., Bangotra, D. K., Singh, P. K., & Hong, W.-C. (2021) · Sensors · doi:10.3390/s21051809 · Cited by 260
“Nowadays, the technique is finding usage in the sector of anomaly detection [93,94], the approach is fused with other methods for anomaly detection, thus providing a profitable proposal in this sector.”
MethodRecent Progresses in Machine Learning Assisted Raman Spectroscopy
Qi, Y., Hu, D., Jiang, Y., Wu, Z., Zheng, M., Chen, E. X., Liang, Y., Sadi, M. A., Zhang, K., & Chen, Y. P. (2023) · Advanced Optical Materials · doi:10.1002/adom.202203104 · Cited by 269
Deep learning in mammography images segmentation and classification: Automated CNN approach
Salama, W. M., & Aly, M. H. (2021) · Alexandria Engineering Journal · doi:10.1016/j.aej.2021.03.048 · Cited by 255
“A. Shrestha et al. developed a new algorithm to describe deep learning [6].”
BackgroundA Deep Learning-Based Phishing Detection System Using CNN, LSTM, and LSTM-CNN
Alshingiti, Z., Alaqel, R., Al-Muhtadi, J. F., Haq, Q. E. U., Saleem, K., & Faheem, M. H. (2023) · Electronics · doi:10.3390/electronics12010232 · Cited by 244
“For training deep learning models, a large amount of labeled data is required, substantial computing power, and neural network architectures that contain numerous layers [15,16].”
BackgroundTransformer Architecture and Attention Mechanisms in Genome Data Analysis: A Comprehensive Review
Choi, S. R., & Lee, M. (2023) · Biology · doi:10.3390/biology12071033 · Cited by 286
“Deep learning, or deep neural networks (DNNs), employs artificial neural networks with multiple layers, a feature that makes it remarkably capable of learning complex patterns from large datasets [23].”
BackgroundMulti-Disease Prediction Based on Deep Learning: A Survey
Xie, S., Yu, Z., & Lv, Z. (2021) · Computer Modeling in Engineering & Sciences · doi:10.32604/cmes.2021.016728 · Cited by 243
Karataş, F., Abedi, F. Y., Ozek Gunyel, F., Karadeniz, D., & Kuzgun, Y. (2024) · Education and Information Technologies · doi:10.1007/s10639-024-12574-6 · Cited by 252
“…architectures and algorithms to efficiently train on large datasets, enabling complex hierarchical representations and transformations that power its capabilities across voice recognition, chatbots, search, and computer vision (Moshayedi et al., 2022; Rizk et al., 2019; Shrestha & Mahmood, 2019).”
A review on social spam detection: Challenges, open issues, and future directions
Rao, S., Verma, A. K., & Bhatia, T. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.115742 · Cited by 237
An Efficient Deep Learning-Based Skin Cancer Classifier for an Imbalanced Dataset
Alam, T. M., Shaukat, K., Khan, W. A., Hameed, I. A., Almuqren, L., Raza, M. A., Aslam, M., & Luo, S. (2022) · Diagnostics · doi:10.3390/diagnostics12092115 · Cited by 207
Jahangir, H., Tayarani, H., Gougheri, S. S., Golkar, M. A., Ahmadian, A., & Elkamel, A. (2020) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2020.3009604 · Cited by 209
Deep learning ensemble 2D CNN approach towards the detection of lung cancer
Shah, A. A., Malik, H. A. M., Muhammad, A., Alourani, A., & Butt, Z. A. (2023) · Scientific Reports · doi:10.1038/s41598-023-29656-z · Cited by 211
“A final solution Deep Ensemble 2D CNN is developed with the help of the Deep Learning Algorithm 22 to detect Lung Nodules from CT Scan images.”
MethodKumar, V., Azamathulla, H. M., Sharma, K. V., Mehta, D., & Tota‐Maharaj, K. (2023) · Sustainability · doi:10.3390/su151310543 · Cited by 217
“The parameters of the model are changed during training to reduce the discrepancy between the model’s predictions and the actual labels in the training data [34].”
BackgroundEmotion recognition from unimodal to multimodal analysis: A review
Ezzameli, K., & Mahersia, H. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101847 · Cited by 206
Pathways and challenges of the application of artificial intelligence to geohazards modelling
Dikshit, A., Pradhan, B., & Alamri, A. M. (2020) · Gondwana Research · doi:10.1016/j.gr.2020.08.007 · Cited by 197
Deep learning in analytical chemistry
Debus, B., Parastar, H., Harrington, P. D. B., & Kirsanov, D. O. (2021) · TrAC Trends in Analytical Chemistry · doi:10.1016/j.trac.2021.116459 · Cited by 208
Deep Neuro-Fuzzy System application trends, challenges, and future perspectives: a systematic survey
Talpur, N., Abdulkadir, S. J., Alhussian, H., Hasan, M. H., Aziz, N., & Bamhdi, A. M. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-022-10188-3 · Cited by 201
“The advancement from shallow to deep learning models has increased the possibility of dealing with complex and nonlinear functions (Shrestha and Mahmood 2019).”
BackgroundA Survey on Kolmogorov-Arnold Network
Somvanshi, S., Javed, S. A., Islam, M. M., Pandit, D., & Das, S. (2025) · ACM Computing Surveys · doi:10.1145/3743128 · Cited by 261
“This structural shift differentiates KANs from Multi-Layer Perceptrons (MLPs), which use fixed activation functions at the nodes, offering KANs the advantage of adaptability and greater alignment with the decomposition of multivariate functions [3].”
Taking the leap between analytical chemistry and artificial intelligence: A tutorial review
Ayres, L. B., Gómez, F. J. V., Linton, J. R., Silva, M. F., & García, C. D. (2021) · Analytica Chimica Acta · doi:10.1016/j.aca.2021.338403 · Cited by 184
An analysis of process fault diagnosis methods from safety perspectives
Arunthavanathan, R., Khan, F. I., Ahmed, S., & Imtiaz, S. A. (2020) · Computers & Chemical Engineering · doi:10.1016/j.compchemeng.2020.107197 · Cited by 177
Molecular design in drug discovery: a comprehensive review of deep generative models
Cheng, Y., Gong, Y., Liu, Y., Song, B., & Zou, Q. (2021) · Briefings in Bioinformatics · doi:10.1093/bib/bbab344 · Cited by 178
A comprehensive survey on optimizing deep learning models by metaheuristics
Akay, B., Karaboğa, D., & Akay, R. (2021) · Artificial Intelligence Review · doi:10.1007/s10462-021-09992-0 · Cited by 182
Machine Learning for Perovskite Solar Cells and Component Materials: Key Technologies and Prospects
Liu, Y., Tan, X., Liang, J., Han, H., Xiang, P., & Yan, W. (2023) · Advanced Functional Materials · doi:10.1002/adfm.202214271 · Cited by 174
AI-Enabled Secure Microservices in Edge Computing: Opportunities and Challenges
Al-Doghman, F., Moustafa, N., Khalil, I., Sohrabi, N., Tari, Z., & Zomaya, A. Y. (2022) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2022.3155447 · Cited by 179
“The usefulness of this operation is that the features do not require to be defined at the beginning and the model would gradually learn to point out the best results by itself [64].”
BackgroundElallid, B. B., Benamar, N., Hafid, A. S., Rachidi, T., & Mrani, N. (2022) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2022.03.013 · Cited by 179
“(Shrestha and Mahmood, 2019).”
Progress and Challenges for Memtransistors in Neuromorphic Circuits and Systems
Yan, X., Qian, J. H., Sangwan, V. K., & Hersam, M. C. (2021) · Advanced Materials · doi:10.1002/adma.202108025 · Cited by 172
Khallaf, R., & Khallaf, M. (2021) · Automation in Construction · doi:10.1016/j.autcon.2021.103760 · Cited by 170
A Comprehensive Overview and Comparative Analysis on Deep Learning Models
Perumal, T., Mustapha, N., Mohamed, R., & Shiri, F. M. (2024) · Journal on artificial intelligence · doi:10.32604/jai.2024.054314 · Cited by 629
“The units sum the inputs after multiplying them by their corresponding weights [4].”
BackgroundEnabling AI in Future Wireless Networks: A Data Life Cycle Perspective
Nguyen, D. C., Cheng, P., Ding, M., Lopez-Perez, D., Pathirana, P. N., Li, J., Seneviratne, A., Li, Y., & Poor, H. V. (2021) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.3024783 · Cited by 160
“1) Convolutional Neural Networks: CNN is a type of neural network used mainly for image processing and recognition with large pixel datasets [53].”
BackgroundConstruction of deep learning-based disease detection model in plants
Jung, M., Song, J. S., Shin, A.-Y., Choi, B., Go, S., Kwon, S., Park, J., Park, S. G., & Kim, Y. (2023) · Scientific Reports · doi:10.1038/s41598-023-34549-2 · Cited by 157
Hybrid CNN-LSTM approaches for identification of type and locations of transmission line faults
Moradzadeh, A., Teimourzadeh, H., Mohammadi‐Ivatloo, B., & Pourhossein, K. (2021) · International Journal of Electrical Power & Energy Systems · doi:10.1016/j.ijepes.2021.107563 · Cited by 170
Deep Transfer Learning Models for Medical Diabetic Retinopathy Detection
Khalifa, N. E. M., Loey, M., Taha, M. H. N., & Mohamed, H. (2019) · Acta Informatica Medica · doi:10.5455/aim.2019.27.327-332 · Cited by 150
“Table 1 shows a series of major contributions in the field of DL (12).”
BackgroundXi, Z., Hopkinson, C., Rood, S. B., & Peddle, D. R. (2020) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2020.08.001 · Cited by 156
Arwa, E. O., & Folly, K. A. (2020) · IEEE Access · doi:10.1109/access.2020.3038735 · Cited by 154
“Many training algorithms have been developed to get the neural network to arrive at the optimal θ , namely, momentum, back propagation, Levenberg–Marquardt algorithm, etc., [75].”
BackgroundApplication of Internet of Things and artificial intelligence for smart fitness: A survey
Farrokhi, A., Farahbakhsh, R., Rezazadeh, J., & Minerva, R. (2021) · Computer Networks · doi:10.1016/j.comnet.2021.107859 · Cited by 158
Yaghoubi, E., Yaghoubi, E., Khamees, A. A., & Vakili, A. H. (2024) · Neural Computing and Applications · doi:10.1007/s00521-024-09893-7 · Cited by 166
“They can automatically acquire hierarchical features [369, 370].”
Artificial Intelligence Revolutionises Weather Forecast, Climate Monitoring and Decadal Prediction
Dewitte, S., Cornelis, J. P., Müller, R. W., & Munteanu, A. (2021) · Remote Sensing · doi:10.3390/rs13163209 · Cited by 165
“backpropagation algorithm has been the most popular one for many years [25].”
BackgroundMathematical Analysis and Performance Evaluation of the GELU Activation Function in Deep Learning
Lee, M. (2023) · Journal of Mathematics · doi:10.1155/2023/4229924 · Cited by 217
“To minimize the loss function, we utilize optimization algorithms [24] that update the weights and biases of the network iteratively.”
MethodZhu, H., Lin, C., Liu, G., Wang, D., Qin, S., Li, A., Xu, J., & He, Y. (2024) · Frontiers in Plant Science · doi:10.3389/fpls.2024.1435016 · Cited by 177
Deep learning-based approaches for robust classification of cervical cancer
Paçal, İ., & Kılıçarslan, S. (2023) · Neural Computing and Applications · doi:10.1007/s00521-023-08757-w · Cited by 148
“…decision tree, Bayesian and logistic regression, which are classical machine methods, offer a good solution to problems that are simple and require little data, but as the problem complexity or data increases, these structures experience a bottleneck and lack the ability to generalize [10, 46, 47].”
BackgroundJahangir, R., Teh, Y. W., Nweke, H. F., Mujtaba, G., Al-Garadi, M. A., & Ali, I. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.114591 · Cited by 152
Artificial intelligence in cardiology: Hope for the future and power for the present
Karatzia, L., Aung, N., & Aksentijević, D. (2022) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2022.945726 · Cited by 147
“Abbreviations: EHRs, electronic health records; CMR, cardiac magnetic resonance; CT, computed tomography; IoT, internet of things; SPECT, single photon-emission computerised tomography. recognition, natural language processing, computer vision, and predictive analysis (13).”
A bird's-eye view of deep learning in bioimage analysis
Meijering, E. (2020) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2020.08.003 · Cited by 126
“Notwithstanding the great arsenal of software toolkits for deep learning available today [54–57], there is still much room for the development of higher-level, user-friendly platforms that make it easier also for non-experts to adopt and use existing DNNs or to design and deploy their own solutions.”
BackgroundThe challenges of modern computing and new opportunities for optics
Li, C., Zhang, X., Li, J., Fang, T., & Dong, X. (2021) · PhotoniX · doi:10.1186/s43074-021-00042-0 · Cited by 137
“Recently, Sorger et al. [52] demonstrated that the optical-electrical hybrid CNN still works well if the phase information in the Fourier filter plane is abandoned.”
MethodInfluentialTrends and gaps in photovoltaic power forecasting with machine learning
Alcañiz, A., Grzebyk, D., Ziar, H., & Isabella, O. (2022) · Energy Reports · doi:10.1016/j.egyr.2022.11.208 · Cited by 160
Sajitha, P., Andrushia, A. D., Anand, N., & Naser, M. Z. (2024) · Journal of Industrial Information Integration · doi:10.1016/j.jii.2024.100572 · Cited by 134
Deep learning technologies for shield tunneling: Challenges and opportunities
Zhou, C., Gao, Y., Chen, E. J., Ding, L., & Qin, W. (2023) · Automation in Construction · doi:10.1016/j.autcon.2023.104982 · Cited by 135
Empowering vertical farming through IoT and AI-Driven technologies: A comprehensive review
Rathor, A. S., Choudhury, S., Sharma, A., Nautiyal, P., & Shah, G. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e34998 · Cited by 130
“Overcoming these challenges will unlock break-throughs across all facets of ML and AI in the field of VF [145].”
BackgroundMachine learning and optimization for production rescheduling in Industry 4.0
Li, Y., Carabelli, S., Fadda, E., Manerba, D., Tadei, R., & Terzo, O. (2020) · The International Journal of Advanced Manufacturing Technology · doi:10.1007/s00170-020-05850-5 · Cited by 143
Survey of Deep Learning Paradigms for Speech Processing
Bhangale, K. B., & Kothandaraman, M. (2022) · Wireless Personal Communications · doi:10.1007/s11277-022-09640-y · Cited by 135
“The learning algorithms have several drawbacks: vanishing gradient problem, local minima, larger training time, over-fitting, etc. [32].”
BackgroundDiabetes detection based on machine learning and deep learning approaches
Wee, B. F., Sivakumar, S., Lim, K. H., Wong, W. K., & Juwono, F. H. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-16407-5 · Cited by 167
“One of the benefits is that deep learning algorithms possess various built in functions such as feature selection and feature extraction [54].”
Sharma, S., Singh, G., & Sharma, M. (2021) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2021.104450 · Cited by 134
Android Mobile Malware Detection Using Machine Learning: A Systematic Review
Senanayake, J., Kalutarage, H., & Al-Kadri, M. O. (2021) · Electronics · doi:10.3390/electronics10131606 · Cited by 126
“It works with models such as artificial neural networks (ANN) and consists of a higher or deeper number of processing layers [31].”
A review on Deep Learning approaches for low-dose Computed Tomography restoration
Kulathilake, K. A. S. H., Abdullah, N. A., Sabri, A. Q. M., & Lai, K. W. (2021) · Complex & Intelligent Systems · doi:10.1007/s40747-021-00405-x · Cited by 125
“The network models based on the discriminative approach represent bottom–up execution to separate learned data based on a decision boundary [61].”
MethodNutakki, M., & Mandava, S. (2023) · Engineering applications of artificial intelligence · doi:10.1016/j.engappai.2022.105721 · Cited by 104
Machine learning for condensed matter physics
Bedolla-Montiel, E. A., Padierna, L. C., & Castañeda-Priego, R. (2020) · Journal of Physics Condensed Matter · doi:10.1088/1361-648x/abb895 · Cited by 134
Deep learning for accelerated and robust MRI reconstruction
Heckel, R., Jacob, M., Chaudhari, A. S., Perlman, O., & Shimron, E. (2024) · Magnetic Resonance Materials in Physics Biology and Medicine · doi:10.1007/s10334-024-01173-8 · Cited by 162
Applications of machine learning in routine laboratory medicine: Current state and future directions
Rabbani, N., Kim, G. Y. E., Suarez, C. J., & Chen, J. H. (2022) · Clinical Biochemistry · doi:10.1016/j.clinbiochem.2022.02.011 · Cited by 118
Yang, X., Guan, J., Ding, L., You, Z., Lee, V. C. S., Hasan, M. R. M., & Cheng, X. (2021) · Journal of Traffic and Transportation Engineering (English Edition) · doi:10.1016/j.jtte.2021.03.005 · Cited by 127
Digital Twin in the Maritime Domain: A Review and Emerging Trends
Madusanka, N. S., Fan, Y., Yang, S., & Xiang, X. (2023) · Journal of Marine Science and Engineering · doi:10.3390/jmse11051021 · Cited by 142
“Diagnostics and predictive forecasting can be incorporated into the DT with its neural network with a deep network from the acquisition of data to the training of the final algorithm [115].”
BackgroundHuang, C., He, R., Ai, B., Molisch, A. F., Lau, B. K., Haneda, K., Liu, B., Wang, C., Yang, M., Oestges, C., et al. (2022) · IEEE Transactions on Antennas and Propagation · doi:10.1109/tap.2022.3149663 · Cited by 115
“Widely used deep neural network (DNN) structures include Convolution Neural Network (CNN), Restricted Boltzmann Machine, Long Short-Term Memory (LSTM), etc [22].”
MethodLifelo, Z., Ding, J., Ning, H., Ain, Q. U., & Dhelim, S. (2024) · Electronics · doi:10.3390/electronics13244874 · Cited by 134
Er, M. J., Chen, J., Zhang, Y., & Gao, W. (2023) · Sensors · doi:10.3390/s23041990 · Cited by 127
Improving the efficiency of RMSProp optimizer by utilizing Nestrove in deep learning
Elshamy, R., Abu-Elnasr, O., Elhoseny, M., & Elmougy, S. (2023) · Scientific Reports · doi:10.1038/s41598-023-35663-x · Cited by 128
“RMSProp can be thought of as an extension of AdaGrad which uses a moving (1) θ = θ − η · ∇F (θ)”
State-of-the-Art of Stress Prediction from Heart Rate Variability Using Artificial Intelligence
Haque, Y., Zawad, M. R. S., Rony, C. S. A., Banna, H. A., Ghosh, T., Kaiser, M. S., & Mahmud, M. (2023) · Cognitive Computation · doi:10.1007/s12559-023-10200-0 · Cited by 125
“T h e a r c h it ec t u r e o f C NN i s b a s e d on t h e v i s u a l c o r t e x o f hu m a n b e i ng s [ 82 ]”
Naghib, A., Gharehchopogh, F. S., & Zamanifar, A. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-024-11101-w · Cited by 135
Deep Learning for Biospectroscopy and Biospectral Imaging: State-of-the-Art and Perspectives
He, H., Yan, S., Lyu, D., Xu, M., Ye, R., Zheng, P., Lu, X., Wang, L., & Ren, B. W. (2021) · Analytical Chemistry · doi:10.1021/acs.analchem.0c04671 · Cited by 114
A Review on the emerging technology of TinyML
Tsoukas, V., Gkogkidis, A., Boumpa, E., & Kakarountas, A. P. (2024) · ACM Computing Surveys · doi:10.1145/3661820 · Cited by 128
“The most widely used DL network types are Recurrent Neural Networks (RNNs) , Convolutional Neural Networks (CNNs) , and Long Short-Term Memory Networks (LSTMs) [190].”
MethodNair, A. K., Sahoo, J., & Raj, E. D. (2023) · Computer Standards & Interfaces · doi:10.1016/j.csi.2023.103720 · Cited by 120
Improved Automatic Summarization of Subroutines via Attention to File Context
Haque, S., LeClair, A., Wu, L., & McMillan, C. (2020) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/3379597.3387449 · Cited by 105
“ds what can be listed in one paper, with thousands of papers discussing overall improvementsaswellasadjustmentsforspecificdomains[10].Nevertheless, this history has been chronicled in several surveys [36, 37, 45]. We include enough details in Section 4 to understand the basic encoder-decoderarchitectureandhowourapproa…”
BackgroundDigital Twin-Driven Remaining Useful Life Prediction for Gear Performance Degradation: A Review
He, B., Liu, L., & Zhang, D. (2021) · Journal of Computing and Information Science in Engineering · doi:10.1115/1.4049537 · Cited by 108
Digital hair removal by deep learning for skin lesion segmentation
Li, W., Raj, A. N. J., Tjahjadi, T., & Zhuang, Z. (2021) · Pattern Recognition · doi:10.1016/j.patcog.2021.107994 · Cited by 116
Hyperspectral Band Selection Using Attention-Based Convolutional Neural Networks
Lorenzo, P. R., Tulczyjew, Ł., Marcinkiewicz, M., & Nalepa, J. (2020) · IEEE Access · doi:10.1109/access.2020.2977454 · Cited by 106
Anomaly detection based on Artificial Intelligence of Things: A Systematic Literature Mapping
Trilles, S., Hammad, S. S., & Iskandaryan, D. (2024) · Internet of Things · doi:10.1016/j.iot.2024.101063 · Cited by 118
Abnormal Driving Detection With Normalized Driving Behavior Data: A Deep Learning Approach
Hu, J., Zhang, X., & Maybank, S. J. (2020) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2020.2993247 · Cited by 104
“The literature shows that deep-layered neural networks can learn more powerful models than shallow ones with a large amount of labeled data [19].”
Background
Metadata
- Authors
- Shrestha, A., & Mahmood, A.
- Affiliations
- University of Bridgeport (US)
- Venue
- IEEE Access
- Published
- 2019
- Volume
- 7
- Pages
- 53040-53065
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W2942231644
- Cited by
- 1,883
- References
- 88
- Citations by year
- 2026: 116 · 2025: 317 · 2024: 360 · 2023: 375 · 2022: 297 · 2021: 259 · 2020: 139 · 2019: 20
- FWCI
- 81.55
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- IEEE Access (publishedVersion)
- University of Bridgeport ScholarWorks (University of Bridgeport) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Keywords
- deep learning, deep learning architectures, deep convolutional neural networks, ResNet, recurrent neural networks, reinforcement learning, variational autoencoder, feature extraction, hierarchical representation, training time reduction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
