The DOI matches its Crossref record; 85 of the 88 references it cites were found in a registry or an index.
Review of Deep Learning Algorithms and Architectures
Open accessAbstract
Deep learning (DL) is playing an increasingly important role in our lives. It has already made a huge impact in areas, such as cancer diagnosis, precision medicine, self-driving cars, predictive forecasting, and speech recognition. The painstakingly handcrafted feature extractors used in traditional learning, classification, and pattern recognition systems are not scalable for large-sized data sets. In many cases, depending on the problem complexity, DL can also overcome the limitations of earlier shallow networks that prevented efficient training and abstractions of hierarchical representations of multi-dimensional training data. Deep neural network (DNN) uses multiple (deep) layers of units with highly optimized algorithms and architectures. This paper reviews several optimization methods to improve the accuracy of the training and to reduce training time. We delve into the math behind training algorithms used in recent deep networks. We describe current shortcomings, enhancements, and implementations. The review also covers different types of deep architectures, such as deep convolution networks, deep residual networks, recurrent neural networks, reinforcement learning, variational autoencoders, and others.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“As consistent with the findings for the second and third periods, ANN, Deep Neural Network (DNN), SVM, GA, LR, and DT are among the prominent AI techniques that gained strong traction in recent years (Şerban and Lytras 2020; Shrestha and Mahmood 2019), so are ML, DL, and cognitive AI models in relation to the planning and development of smart cities (Kamrowska-Załuska 2021; Şerban and Lytras 2020”
From Environmentally sustainable smart cities and their converging AI, IoT, and big data technologies and solutions: an integrated approach to an extensive literature review (2023) · cites this paper as Shrestha and Mahmood 2019
In the citing paper
“Deep learning is gaining traction as a type of ANN with more layers of units (Shrestha and Mahmood, 2019).”
From Artificial intelligence for sustainability: Challenges, opportunities, and a research agenda (2020) · cites this paper as Shrestha and Mahmood, 2019
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 5×
“tion with a 152-layer DNN on the ImageNet dataset [62].”
BackgroundMethodInfluentialHochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 2×
“The input, forget and output gates dictates new data going into the cell, what remains in the cell and the cell values used in the calculation of the output of the LSTM block respectively [41], [43].”
MethodAdam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 4×
“Multilayer network training cost on MNIST dataset using different adaptive learning algorithms [58].”
BackgroundMethodInfluentialLeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 2×
“7-layer architecture of CNN for character recognition [28].”
BackgroundMethodAdaptation in Natural and Artificial Systems
Holland, J. H. (1992) · The MIT Press eBooks · doi:10.7551/mitpress/1090.001.0001 · Cited by 47,340
(2014) · Cited by 33,419
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818 · cited 2×
“[59] S. Ioffe and C. Szegedy, ‘‘Batch normalization: Accelerating deep network training by reducing internal covariate shift,’’ CoRR, Feb. 2015.”
BackgroundLearning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545 · cited 1×
“GRUs are smaller in size than LSTM as they don’t include the output gate, and can perform better than LSTM on only some simpler datasets [44], [45].”
BackgroundReducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
Auto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612 · cited 1×
“AlphaGo, Google’s program that beat the human Go champion also uses reinforcement learning [91].”
BackgroundDermatologist-level classification of skin cancer with deep neural networks
Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017) · Nature · doi:10.1038/nature21056 · Cited by 14,362 · cited 1×
“, [12] reported that an deep learning network was able to learn from 129,450 images of 2,032 diseases and was able to diagnose at the same level as 21 board certified dermatologists.”
Background(2010) · Cited by 14,134
Approximation by superpositions of a sigmoidal function
Cybenko, G. (1989) · Mathematics of Control, Signals, and Systems · doi:10.1007/bf02551274 · Cited by 13,883 · cited 1×
“The universal approximation theorem showed that a single hidden layer was able to solve any continuous problem [3].”
BackgroundExtreme learning machine: Theory and applications
Huang, G.-B., Zhu, Q.-Y., & Siew, C.-K. (2006) · Neurocomputing · doi:10.1016/j.neucom.2005.12.126 · Cited by 13,285
The perceptron: A probabilistic model for information storage and organization in the brain.
Rosenblatt, F. (1958) · Psychological Review · doi:10.1037/h0042519 · Cited by 12,280 · cited 1×
“In 1957, Frank Rosenblatt created the perceptron, the first prototype of what we now know as a neural network [1].”
BackgroundUnsupervised representation learning with deep convolutional generative adversarial networks
Radford, A., Metz, L., & Chintala, S. (2015) · arXiv · doi:10.48550/arxiv.1511.06434 · Cited by 12,235 · cited 1×
“Deep convolutional generative adversarial networks proposed by [33].”
Background(2016) · Cited by 11,679
Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1412.3555 · Cited by 11,008 · cited 1×
“GRUs are smaller in size than LSTM as they don’t include the output gate, and can perform better than LSTM on only some simpler datasets [44], [45].”
BackgroundMachine learning: Trends, perspectives, and prospects
Jordan, M. I., & Mitchell, T. M. (2015) · Science · doi:10.1126/science.aaa8415 · Cited by 9,955 · cited 1×
“ML will continue to impact and disrupt all areas of our lives from education, finance, governance, healthcare, manufacturing, marketing and others [7].”
Background(2010) · Cited by 8,261
Chiang, L. H., Russell, E. L., & Braatz, R. D. (2001) · Advanced textbooks in control and signal processing · doi:10.1007/978-1-4471-0347-9_4 · Cited by 6,684
Learning to Forget: Continual Prediction with LSTM
Gers, F. A., Schmidhuber, J., & Cummins, F. (2000) · Neural Computation · doi:10.1162/089976600300015015 · Cited by 6,122 · cited 2×
“The input, forget and output gates dictates new data going into the cell, what remains in the cell and the cell values used in the calculation of the output of the LSTM block respectively [41], [43].”
MethodApproximation capabilities of multilayer feedforward networks
Hornik, K. (1991) · Neural Networks · doi:10.1016/0893-6080(91)90009-t · Cited by 6,091 · cited 1×
“It was mathematically proven as well [4], which further questioned the validity of DNN.”
BackgroundDimensionality Reduction by Learning an Invariant Mapping
Hadsell, R., Chopra, S., & LeCun, Y. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.100 · Cited by 5,336
Deep Reinforcement Learning: A Brief Survey
Arulkumaran, K., Deisenroth, M. P., Brundage, M., & Bharath, A. A. (2017) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2743240 · Cited by 4,733 · cited 2×
“With DRL, problems that were intractable with reinforcement learning can now be solved with higher number of hidden layers of deep networks and reinforcement learning based Q-learning algorithm that maximizes the reward for actions taken by the agent [13].”
BackgroundConvolutional networks for images, speech, and time series
LeCun, Y., & Bengio, Y. (1998) · HAL · doi:10.5555/303568.303704 · Cited by 4,646
Whitley, D. (1994) · Statistics and Computing · doi:10.1007/bf00175354 · Cited by 4,523 · cited 1×
“of GA can be executed independently in islands and few members exchanged between the island every so often [81].”
BackgroundGenerative Adversarial Networks
Goodfellow, I. J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1406.2661 · Cited by 4,477
Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1406.1078 · Cited by 3,317 · cited 1×
“GRUs are smaller in size than LSTM as they don’t include the output gate, and can perform better than LSTM on only some simpler datasets [44], [45].”
BackgroundUnsupervised Learning of Depth and Ego-Motion from Video
Zhou, T., Brown, M., Snavely, N., & Lowe, D. G. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.700 · Cited by 2,910 · cited 1×
“Reference [22] demonstrated enhance results with unsupervised learning using unstructured video sequences for camera motion estimation and monocular depth.”
BackgroundA Practical Guide to Training Restricted Boltzmann Machines
Hinton, G. E. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-35289-8_32 · Cited by 2,852 · cited 6×
“The probability of every pair of visible and hidden vectors is given by the following [40].”
BackgroundInfluentialDeep learning applications and challenges in big data analytics
Najafabadi, M. M., Villanustre, F., Khoshgoftaar, T. M., Seliya, N., Wald, R., & Muharemagic, E. (2015) · Journal of Big Data · doi:10.1186/s40537-014-0007-7 · Cited by 2,594 · cited 3×
“[47] raises similar challenges learning from real-time streaming data such as credit cards”
BackgroundInfluentialGenerative Adversarial Text to Image Synthesis
Reed, S., Akata, Z., Yan, X., Logeswaran, L., Schiele, B., & Lee, H. (2016) · arXiv · doi:10.48550/arxiv.1605.05396 · Cited by 2,494 · cited 1×
“, [94] were able to successfully synthesize realistic images from text.”
BackgroundConvolutional networks and applications in vision
LeCun, Y., Kavukcuoglu, K., & Farabet, C. (2010) · Proceedings of 2010 IEEE International Symposium on Circuits and Systems · doi:10.1109/iscas.2010.5537907 · Cited by 2,213 · cited 1×
“asserts that one way we could achieve this would be to utilize and integrate both unsupervised and supervised learning [24].”
Background(2001) · Cited by 1,932
Salakhutdinov, R., & Hinton, G. E. (2009) · Cited by 1,890
Buhmann, M. (2003) · Springer eBooks · doi:10.1007/978-3-540-70529-1_306 · Cited by 1,797
An Introduction to Deep Reinforcement Learning
François-Lavet, V., Henderson, P., Islam, R., Bellemare, M. G., & Pineau, J. (2018) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000071 · Cited by 1,512
Pouyanfar, S., Sadiq, S. B., Yan, Y., Tian, H., Tao, Y., Reyes, M. P., Shyu, M., Chen, S., & Iyengar, S. S. (2018) · ACM Computing Surveys · doi:10.1145/3234150 · Cited by 1,510
Extreme Learning Machine for Multilayer Perceptron
Tang, J., Deng, C., & Huang, G.-B. (2015) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2015.2424995 · Cited by 1,469 · cited 1×
“came up with a multilayer version of ELM in 2016 [70] to take on more complex problems.”
BackgroundBig data deep learning: Challenges and perspectives
Chen, X.-W., & Lin, X. (2014) · IEEE Access · doi:10.1109/access.2014.2325029 · Cited by 1,255 · cited 9×
“Below is the equation [23] for change in weights, where c is the momentum factor and α is the learning rate, and v and h are visible and hidden units respectively.”
BackgroundMethodInfluentialHuang, L., Joseph, A. D., Nelson, B., Rubinstein, B. I. P., & Tygar, J. D. (2011) · Proceedings of the 4th ACM workshop on Security and artificial intelligence · doi:10.1145/2046684.2046692 · Cited by 1,194 · cited 1×
“Reference [86] demonstrates these and other vulnerabilities and discusses how application domain, features and data distribution can be used to reduce the risk and impact of such adversarial attacks.”
BackgroundLearning multiple layers of representation
Hinton, G. E. (2007) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2007.09.004 · Cited by 1,050 · cited 3×
“Here is the equation [66] for adjusting the recognition weights rij.”
BackgroundMethodInfluentialVery Deep Convolutional Networks for Text Classification
Conneau, A., Schwenk, H., Barrault, L., & Lecun, Y. (2017) · Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers · doi:10.18653/v1/e17-1104 · Cited by 1,046 · cited 2×
“Reference [62] addressed this issue with shortcut connections between the convolution blocks to let the gradients to propagate more freely and they, along with [74] were able to validate the benefits of the shortcuts with 10/101/152-layers and 49 layers respectively.”
BackgroundMethodThe Design of Innovation: Lessons from and for Competent Genetic Algorithms
Goldberg, D. E. (2002) · Cited by 912
Miikkulainen, R. P., Liang, J., Meyerson, E., Rawal, A., Fink, D., Francon, O., Raju, B., Shahrzad, H., Navruzyan, A., Duffy, N., et al. (2018) · Elsevier eBooks · doi:10.1016/b978-0-12-815480-9.00015-3 · Cited by 871
Genetic Algorithms for the Travelling Salesman Problem: A Review of Representations and Operators
Larrañaga, P., Kuijpers, C. M. H., Murga, R. H., Inza, I., & Dizdarevic, S. (1999) · Artificial Intelligence Review · doi:10.1023/a:1006529012972 · Cited by 849
Deep learning via Hessian-free optimization
Martens, J. (2010) · Cited by 722
Convolutional Learning of Spatio-temporal Features
Taylor, G. W., Fergus, R., LeCun, Y., & Bregler, C. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15567-3_11 · Cited by 652 · cited 1×
“The deep nature of the CNN results in high quality representations while maintaining locality, reduced parameters and invariance to minor variations in the input image [30].”
BackgroundAdvances in Instance Selection for Instance-Based Learning Algorithms
Brighton, H., & Mellish, C. (2002) · Data Mining and Knowledge Discovery · doi:10.1023/a:1014043630878 · Cited by 544 · cited 1×
“Instead of training the network with all the data set, we can preselect a smaller but representative data set from the full training distribution set using instance selection methods [95] or Monte Carlo sampling [48].”
MethodNagpal, K., Foote, D., Liu, Y., Chen, P.-H. C., Wulczyn, E., Tan, F., Olson, N., Smith, J. L., Mohtashamian, A., Wren, J. H., et al. (2019) · npj Digital Medicine · doi:10.1038/s41746-019-0112-2 · Cited by 495 · cited 2×
“Google AI [10] was able to beat the average accuracy of US board certified general pathologists in grading prostate cancer by 70% to 61%.”
BackgroundAutomatic Speech Recognition: A Deep Learning Approach
Yu, D., & Li Juan, D. E. N. G. (2014) · Signals and communication technology · doi:10.1007/978-1-4471-5779-3 · Cited by 475 · cited 2×
“GMM are very effective in extracting speech features and modeling the probability density function to a desired level of accuracy as long as we have sufficient components, and the estimation maximization makes it easy to fit the model [87].”
BackgroundMethodMulti-instance multi-label learning
Zhou, Z., Zhang, M.-L., Huang, S.-J., & Li, Y.-F. (2011) · Artificial Intelligence · doi:10.1016/j.artint.2011.10.002 · Cited by 424
Genetic Algorithm Performance with Different Selection Strategies in Solving TSP
Razali, N. M., & Geraghty, J. (2011) · Cited by 359
A Machine Learning Approach for Non-blind Image Deconvolution
Schuler, C. J., Burger, H. C., Harmeling, S., & Scholkopf, B. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.142 · Cited by 319 · cited 1×
“Reference [32] achieves this by leveraging Fourier transformation to regularize inversion of the blurred images and denoising.”
MethodA Framework for Designing the Architectures of Deep Convolutional Neural Networks
Albelwi, S. A., & Mahmood, A. (2017) · Entropy · doi:10.3390/e19060242 · Cited by 303
Self-citationUnconstrained Face Recognition: Identifying a Person of Interest From a Media Collection
Best-Rowden, L., Han, H., Otto, C., Klare, B. F., & Jain, A. K. (2014) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2014.2359577 · Cited by 205 · cited 1×
“Reference [64] proposes a method to utilize multiple images of the unconstrained source to enhance the recognition process.”
MethodA Multiobjective Sparse Feature Learning Model for Deep Neural Networks
Gong, M., Liu, J., Li, H., Cai, Q., & Su, L. (2015) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2015.2469673 · Cited by 195 · cited 3×
“Their learning procedure continuously iterates between evolutionary optimization step and the stochastic gradient descent to optimize the reconstruction error [71].”
BackgroundMethodInfluentialGenerative or Discriminative? Getting the Best of Both Worlds
Bishop, C., & Lasserre, J. (2007) · doi:10.1093/oso/9780199214655.003.0001 · Cited by 191
Large-scale information retrieval with latent semantic indexing
Letsche, T. A., & Berry, M. W. (1997) · Information Sciences · doi:10.1016/s0020-0255(97)00044-3 · Cited by 187 · cited 1×
“Deep learning can help with semantic indexing to enable information to be more readily accessible in search engines [14], [65].”
BackgroundA Supervised Learning and Control Method to Improve Particle Swarm Optimization Algorithms
Dong, W., & Zhou, M. (2016) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2016.2560128 · Cited by 180 · cited 6×
“The supervised learning module provided real-time feedback with back diffusion (BD) to retain diversity and social attractor renewal to overcome stagnation [76].”
BackgroundInfluentialLean GHTorrent: GitHub data on demand
Gousios, G., Vasilescu, B., Serebrenik, A., & Zaidman, A. (2014) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/2597073.2597126 · Cited by 179 · cited 1×
“Github is the largest hosting service provider of source code in the world [25].”
BackgroundDeepBox: Learning Objectness with Convolutional Networks
Kuo, W., Hariharan, B., & Malik, J. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.285 · Cited by 178 · cited 1×
“Reference [68] proposed a four-layered CNN termed DeepBox that outperformed larger networks in speed and accuracy.”
BackgroundParticle Swarm Model Selection
Escalante, H. J., Montes, M., & Sucar, L. E. (2009) · doi:10.5555/1577069.1577084 · Cited by 172 · cited 2×
“used PSO for hyperparameter optimization to build a competitive model that ranked among the top relative to other comparable methods [52].”
MethodA Survey on Deep Learning in Big Data
Gheisari, M., Wang, G., & Bhuiyan, M. Z. A. (2017) · IEEE International Conference on Computational Science and Engineering (CSE) and IEEE International Conference on Embedded and Ubiquitous Computing (EUC) · doi:10.1109/cse-euc.2017.215 · Cited by 154 · cited 1×
“Deep learning can help with semantic indexing to enable information to be more readily accessible in search engines [14], [65].”
BackgroundA recurrent neural network for classification of unevenly sampled variable stars
Naul, B., Bloom, J. S., Pérez, F., & van der Walt, S. (2017) · Nature Astronomy · doi:10.1038/s41550-017-0321-z · Cited by 148 · cited 1×
“demonstrated LSTM andGRU based autoencoders for automatic feature extractions [46].”
BackgroundRate-coded Restricted Boltzmann Machines for Face Recognition
Teh, Y. W., & Hinton, G. E. (2000) · Cited by 134
A recurrent neural network for classification of unevenly sampled variable stars
Naul, B., Bloom, J. S., Pérez, F., & van der Walt, S. (2018) · eScholarship (California Digital Library) · Cited by 101
Deep Learning-Based Model Reduction for Distributed Parameter Systems
Wang, M., Li, H., Chen, X., & Chen, Y. (2016) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2016.2605159 · Cited by 97 · cited 1×
“[37] successfully implemented a deep autoencoder with stacks of RBM blocks similar to Figure 6 to achieve better modeling accuracy and efficiency than the proper orthogonal decomposition (POD) method for dimensionality reduction of distributed parameter systems (DPSs).”
MethodDeep Learning: A ReviewPreprint
Vargas, R. H., Mosavi, A. H., & Ruiz, R. (2018) · Preprints.org · doi:10.20944/preprints201810.0218.v1 · Cited by 93 · cited 1×
“Other review papers [13]–[16] focus on specific areas and implementations without encompass the full scope of the field.”
BackgroundKhan, S. H., Rahmani, H., Shah, S. A. A., & Bennamoun, M. (2018) · Synthesis lectures on computer vision · doi:10.1007/978-3-031-01821-3_4 · Cited by 79
An Improved Polynomial Neural Network Classifier Using Real-Coded Genetic Algorithm
Lin, C., Prasad, M., & Saxena, A. K. (2015) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2015.2406855 · Cited by 78 · cited 2×
“Standard GA operations including selection, crossover and mutation were used on parameters that included partial descriptions (PDs) of inputs in the first layer, bias and all input features [82].”
MethodSOM: Stochastic initialization versus principal components
Akinduko, A. A., Mirkes, E. M., & Gorban, A. N. (2015) · Information Sciences · doi:10.1016/j.ins.2015.10.013 · Cited by 73 · cited 1×
“The weights of the closest node and its neighbors from the input data are updated to bring them closer to the input data with the formula below [18].”
BackgroundGuo, Y. P., Cai, Q., Samuels, D. C., Ye, F., Long, J., Li, C.-I., Winther, J. F., Tawn, E. J., Stovall, M. A., Lähteenmäki, P. M., et al. (2012) · Mutation Research/Genetic Toxicology and Environmental Mutagenesis · doi:10.1016/j.mrgentox.2012.02.006 · Cited by 57
Multiclass Semisupervised Learning Based Upon Kernel Spectral Clustering
Mehrkanoon, S., Alzate, C., Mall, R., Langone, R., & Suykens, J. A. K. (2015) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2014.2322377 · Cited by 50 · cited 2×
“[72] proposed a multiclass learning algorithm based on Kernel Spectral Clustering (KSC) using both labeled and unlabeled data.”
BackgroundMethodImproving Genetic Algorithm with Fine-Tuned Crossover and Scaled Architecture
Shrestha, A., & Mahmood, A. (2016) · Journal of Mathematics · doi:10.1155/2016/4015845 · Cited by 38 · cited 10×
“[89] A. Shrestha and A.Mahmood, ‘‘Enhancing siamese networks training with importance sampling,’’ in Proc.”
BackgroundMethodInfluentialSelf-citationKernel Spectral Clustering and Applications
Langone, R., Mall, R., Alzate, C., & Suykens, J. A. K. (2016) · Springer eBooks · doi:10.1007/978-3-319-24211-8_6 · Cited by 30
Deep and Modular Neural Networks
Chen, K. (2015) · Springer eBooks · doi:10.1007/978-3-662-43505-2_28 · Cited by 30 · cited 1×
“The smaller networks perform specific task which are later combined as part of a single output of the entire network [19].”
BackgroundML for Flood Forecasting at Scale
Nevo, S., Anisimov, V., Elidan, G., El-Yaniv, R., Giencke, P., Gigi, Y., Hassidim, A., Moshe, Z., Schlesinger, M., Shalev, G., et al. (2019) · arXiv · doi:10.48550/arxiv.1901.09583 · Cited by 23 · cited 1×
“Deep learning has a potential to improve human lives with more accurate diagnosis of diseases like cancer [10], discovery of new drugs, prediction of natural disasters [11].”
BackgroundMultimodal integration learning of object manipulation behaviors using deep neural networks
Noda, K., Arie, H., Suga, Y., & Ogata, T. (2013) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2013.6696582 · Cited by 20 · cited 1×
“It uses greedy layer by layer unsupervised pretraining and fin-tuning with backpropagation [35].”
MethodEnhancing Siamese Networks Training with Importance Sampling
Shrestha, A., & Mahmood, A. (2019) · International Conference on Agents and Artificial Intelligence (ICAART) · doi:10.5220/0007371706100615 · Cited by 6 · cited 1×
“This issue doesn’t apply to siamese network as the number of outputs of the softmax in the twin networks doesn’t have the requirement to match the number of classes [89].”
BackgroundSelf-citationParallel neural network training with OpenCL
Krpan, N., & Jakobović, D. (2012) · International Convention on Information and Communication Technology, Electronics and Microelectronics · Cited by 3
(2014) · doi:10.5555/2627435.2697065 · cited 1×
“Nonetheless [27] evaluated 179 classifiers and concluded that parallel random forest or parRF_t, which is essentially parallel implementation of variation of decision tree, performed the best.”
Background
Research works printed but not linked: 10
- vapnik, 2015, Learning using privileged information: Similarity control and knowledge transfer, J Mach Learn Res, 16, 2023
- ng, 2018, Autoencoders
- 2018, Amazon EC2 P2 & P3 Instances Amazon EC2 Instance Types
- simpson, 2015, Uniform learning in a deep neural network via ‘oddball’ stochastic gradient descent, CoRR
- minsky, 1969, Perceptrons An Introduction to Computational Geometry Expanded Edition, 258
- 2019, Top Deep Learning Github Repositories
- sastry, 2005, Genetic Algorithms
- 10.1017/CBO9780511543241 · DOI
- wu, 2016, Google’s neural machine translation system: Bridging the gap between human and machine translation, CoRR
- metz, 2019, Turing Award Won by 3 Pioneers in Artificial Intelligence, 3b
Cited by
Sorted by citations
Review of deep learning: concepts, CNN architectures, challenges, applications, future directions
Alzubaidi, L., Zhang, J., Humaidi, A. J., Al-Dujaili, A., Duan, Y., Al-Shamma, O., Santamaría, J., Fadhel, M. A., Al-Amidie, M., & Farhan, L. (2021) · Journal of Big Data · doi:10.1186/s40537-021-00444-8 · Cited by 7,906
Scientific Machine Learning Through Physics–Informed Neural Networks: Where we are and What’s Next
Cuomo, S., Cola, V. S. D., Giampaolo, F., Rozza, G., Raissi, M., & Piccialli, F. (2022) · Journal of Scientific Computing · doi:10.1007/s10915-022-01939-z · Cited by 3,262
“The main principle of reinforcement learning is to have an agent learn from its surroundings through exploration and by defining a reward (Shrestha and Mahmood, 2019).”
BackgroundArtificial intelligence for sustainability: Challenges, opportunities, and a research agenda
Nishant, R., Kennedy, M., & Corbett, J. (2020) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2020.102104 · Cited by 1,203
Taye, M. M. (2023) · Computers · doi:10.3390/computers12050091 · Cited by 1,275
“…rules to function; rather, it leverages a massive amount of data to map the provided input to particular labels. is created utilising multiple layers of algorithms (arti fi cial neural networks, or ANNs), each of which delivers a unique interpretation of the data provided to it [25,26].”
BackgroundDeep learning modelling techniques: Current progress, applications, advantages, and challenges
Ahmed, S. F., Alam, M. S. B., Hassan, M., Rozbu, M. R., Ishtiak, T., Rafa, N., Mofijur, M., Shawkat Ali, A. B. M., & Gandomi, A. H. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-023-10466-8 · Cited by 1,079
Metadata
- Authors
- Shrestha, A., & Mahmood, A.
- Affiliations
- University of Bridgeport (US)
- Venue
- IEEE Access
- Published
- 2019
- Volume
- 7
- Pages
- 53040-53065
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W2942231644
- Cited by
- 1,883
- References
- 88
- Citations by year
- 2026: 116 · 2025: 317 · 2024: 360 · 2023: 375 · 2022: 297 · 2021: 259 · 2020: 139 · 2019: 20
- FWCI
- 81.55
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- IEEE Access (publishedVersion)
- University of Bridgeport ScholarWorks (University of Bridgeport) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Keywords
- deep learning, deep learning architectures, deep convolutional neural networks, ResNet, recurrent neural networks, reinforcement learning, variational autoencoder, feature extraction, hierarchical representation, training time reduction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
