Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups
The DOI matches its Crossref record; 68 of the 71 references it cites were found in a registry or an index.
Abstract
Most current speech recognition systems use hidden Markov models (HMMs) to deal with the temporal variability of speech and Gaussian mixture models (GMMs) to determine how well each state of each HMM fits a frame or a short window of frames of coefficients that represents the acoustic input. An alternative way to evaluate the fit is to use a feed-forward neural network that takes several frames of coefficients as input and produces posterior probabilities over HMM states as output. Deep neural networks (DNNs) that have many hidden layers and are trained using new methods have been shown to outperform GMMs on a variety of speech recognition benchmarks, sometimes by a large margin. This article provides an overview of this progress and represents the shared views of four research groups that have had recent successes in using DNNs for acoustic modeling in speech recognition.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Deep Learning has largely alleviated the need for manual feature extraction, achieving state-ofthe-art performance in fields such as computer vision and speech recognition [13,14].”
From EEGNet: a compact convolutional neural network for EEG-based brain-computer interfaces (2018) · cites this paper as [13]
In the citing paper
“In particular, neural networks became competitive, solving problems in speech recognition [68], image recognition [69] and language translation [70].”
From The growing ubiquity of algorithms in society: implications, impacts and innovations (2018) · cites this paper as [68]
In the citing paper
“Apart from images, sequential data such as text and audio can also be processed with DNNs to reach state-of-the-art performance for document classification and speech recognition.”
From Deep learning for time series classification: a review (2019) · cites this paper as Hinton et al., 2012
In the citing paper
“The other more direct reason behind the high accuracies of deep CNNs on time series data is its success in other sequential data such as speech recognition (Hinton et al., 2012) and sentence classification (Kim, 2014)”
From Deep learning for time series classification: a review (2019) · cites this paper as Hinton et al., 2012
In the citing paper
“Recent studies have reported that deep learning methods have been successfully applied to many applications areas e.g., image and video classification [11-13], visual tracking [14], speech recognition [15] and natural language processing [16].”
From Medical image retrieval using deep convolutional neural network (2017) · cites this paper as [15]
Citations
Cites
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892
Self-citationReducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
Self-citationA fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336
Self-citation(2010) · Cited by 14,134
Training Products of Experts by Minimizing Contrastive Divergence
Hinton, G. E. (2002) · Neural Computation · doi:10.1162/089976602760128018 · Cited by 4,879 · cited 3×
“…learning algorithm for a restricted Boltzmann machine (RBM) which consists of a layer of stochastic binary " visible " units that represent binary input data connected to a layer of stochastic binary hidden units that learn to model significant nonindependencies between the visible units [20].”
InfluentialSelf-citation(2010) · Journal of Machine Learning Research · doi:10.5555/1756006.1953039 · Cited by 4,051
Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition
Dahl, G. E., Yu, D., Deng, L., & Acero, A. (2012) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2011.2134090 · Cited by 3,011
Self-citationA Practical Guide to Training Restricted Boltzmann Machines
Hinton, G. E. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-35289-8_32 · Cited by 2,852 · cited 2×
“Learning the standard deviations of a GRBM is problematic for reasons described in [21], so for pretraining using CD 1 , the data are normalized so that each coefficient has zero mean and unit variance, the standard deviations are set to one when computing ( ) v h p ; , and no noise is added to the…”
Self-citationPerceptual linear predictive (PLP) analysis of speech
Hermansky, H. (1990) · The Journal of the Acoustical Society of America · doi:10.1121/1.399423 · Cited by 2,536
Acoustic Modeling Using Deep Belief Networks
Mohamed, A.-R., Dahl, G. E., & Hinton, G. (2012) · IEEE Transactions on Audio, Speech, and Language Processing · doi:10.1109/tasl.2011.2109382 · Cited by 1,727
Self-citation
Research works printed but not linked: 5
- sainath, 2011, Improvements in using deep belief networks for large vocabulary continuous speech recognition, Speech and Language Algorithm Group IBM Yorktown Heights NY Tech Rep UTML TR 2010-003
- rifai, 0, Contractive autoencoders: Explicit invariance during feature extraction, Proc 28th Int Conf Machine Learning, 833
- yu, 2011, Discriminative pretraining of deep neural networks, U S Patent Filing
- mohamed, 0, Deep belief networks for phone recognition, Proc NIPS Workshop Deep Learning for Speech Recognition and Related Applications
- pearl, 1988, Probabilistic Inference in Intelligent Systems Networks of Plausible Inference
Cited by
Items 201–300 of 1,106
Stable architectures for deep neural networks
Haber, E., & Ruthotto, L. (2018) · Inverse Problems · doi:10.1088/1361-6420/aa9a90 · Cited by 397
Cross-language knowledge transfer using multilingual deep neural network with shared hidden layers
Huang, J.-T., Li, J., Yu, D., Deng, L., & Gong, Y. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6639081 · Cited by 595
Self-citationDeepChain: Auditable and Privacy-Preserving Deep Learning with Blockchain-based Incentive
Weng, J., Weng, J., Zhang, J., Li, M., Zhang, Y., & Luo, W. (2019) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2019.2952332 · Cited by 610
Metaheuristic design of feedforward neural networks: A review of two decades of research
Ojha, V. K., Abraham, A., & Snášel, V. (2017) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2017.01.013 · Cited by 606
Streaming End-to-end Speech Recognition for Mobile Devices
He, Y., Sainath, T. N., Prabhavalkar, R., McGraw, I., Alvarez, R., Zhao, D., Rybach, D., Kannan, A., Wu, Y., Pang, R., et al. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8682336 · Cited by 601
Self-citationConvolutional Recurrent Neural Networks for Polyphonic Sound Event Detection
Cakir, E., Parascandolo, G., Heittola, T., Huttunen, H., & Virtanen, T. (2017) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2017.2690575 · Cited by 604
Protein interface prediction using graph convolutional networks
Fout, A., Ben‐Hur, A., Anderson, C., Chitsaz, H., & Zhou, W. (2017) · Digital Collections of Colorado (Colorado State University) · doi:10.25675/3.021826 · Cited by 513
Spoofing and countermeasures for speaker verification: A survey
Wu, Z., Evans, N., Kinnunen, T., Yamagishi, J., Alegre, F., & Li, H. (2015) · Speech Communication · doi:10.1016/j.specom.2014.10.005 · Cited by 601
HIN2Vec: Explore Meta-paths in Heterogeneous Information Networks for Representation Learning
Fu, T.-Y., Lee, W.-C., & Lei, Z. (2017) · Proceedings of the 2017 ACM on Conference on Information and Knowledge Management · doi:10.1145/3132847.3132953 · Cited by 594
Low-rank matrix factorization for Deep Neural Network training with high-dimensional output targets
Sainath, T. N., Kingsbury, B., Sindhwani, V., Arisoy, E., & Ramabhadran, B. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6638949 · Cited by 593
Self-citationA high-performance speech neuroprosthesis
Willett, F. R., Kunz, E. M., Fan, C., Avansino, D. T., Wilson, G. H., Choi, E. Y., Kamdar, F., Glasser, M. F., Hochberg, L. R., Druckmann, S., et al. (2023) · Nature · doi:10.1038/s41586-023-06377-x · Cited by 581
Deep-learning-enabled self-adaptive microwave cloak without human intervention
Qian, C., Zheng, B., Shen, Y., Jing, L., Li, E., Shen, L., & Chen, H. (2020) · Nature Photonics · doi:10.1038/s41566-020-0604-2 · Cited by 585
Audio-visual speech recognition using deep learning
Noda, K., Yamaguchi, Y., Nakadai, K., Okuno, H. G., & Ogata, T. (2015) · Applied Intelligence · doi:10.1007/s10489-014-0629-7 · Cited by 579
An Overview of Noise-Robust Automatic Speech Recognition
Li, J., Deng, L., Gong, Y., & Haeb-Umbach, R. (2014) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2014.2304637 · Cited by 568
Self-citationMater, A. C., & Coote, M. L. (2019) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.9b00266 · Cited by 565
“(6) Graph Updated Features Hidden Weights”
Deep Learning of Representations: Looking Forward
Bengio, Y. (2013) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-39593-2_1 · Cited by 513
Harnessing Deep Neural Networks with Logic Rules
Hu, Z., Ma, X., Liu, Z., Hovy, E., & Xing, E. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1228 · Cited by 545
Deep Learning for Entity Matching: A Design Space Exploration
Mudgal, S., Li, H., Rekatsinas, T., Doan, A., Park, Y., Krishnan, G., Deep, R., Arcaute, E., & Raghavendra, V. (2018) · Proceedings of the 2018 International Conference on Management of Data · doi:10.1145/3183713.3196926 · Cited by 550
Protecting Intellectual Property of Deep Neural Networks with Watermarking
Zhang, J., Gu, Z., Jang, J., Wu, H., Stoecklin, M. P., Huang, H., & Molloy, I. (2018) · Proceedings of the 2018 on Asia Conference on Computer and Communications Security · doi:10.1145/3196494.3196550 · Cited by 540
Evaluating deep learning architectures for Speech Emotion Recognition
Fayek, H. M., Lech, M., & Cavedon, L. (2017) · Neural Networks · doi:10.1016/j.neunet.2017.02.013 · Cited by 554
“Since DNNs displaced Gaussian Mixture Models (GMMs) for acoustic modeling in ASR (Hinton et al., 2012; Mohamed, Dahl, & Hinton, 2012), researchers have attempted to employ DNNs for other speech applications as well, and specifically for SER. Stuhlsatz et al. (2011) proposed a DNN Generalized…”
InfluentialA genetic programming approach to designing convolutional neural network architectures
Suganuma, M., Shirakawa, S., & Nagao, T. (2017) · Proceedings of the Genetic and Evolutionary Computation Conference · doi:10.1145/3071178.3071229 · Cited by 491
Quo vadis artificial intelligence?
Jiang, Y., Li, X., Luo, H., Yin, S., & Kaynak, O. (2022) · Discover Artificial Intelligence · doi:10.1007/s44163-022-00022-8 · Cited by 544
“The application fields of AI are extensive, covering (but not limited to) speech recognition [4, 5], image processing [6, 7], natural language processing [8, 9], smart robots [10, 11], autonomous vehicles [12, 13], energy systems [14, 15], healthcare [16, 17], Fintech [18], etc.”
BackgroundFink, O., Wang, Q., Svensén, M., Dersin, P., Lee, W.-J., & Ducoffe, M. (2020) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2020.103678 · Cited by 549
“These first systems used deep belief networks (DBNs) (Hinton et al., 2012) in conjunction with hidden Markov models (HMM) (Rabiner, 1989) to deliver a significant performance improvement on the until then prevailing technology, which had used Gaussian mixture models (Bishop, 2007) in place of the…”
Unmasking Clever Hans predictors and assessing what machines really learn
Lapuschkin, S., Wäldchen, S., Binder, A., Montavon, G., Samek, W., & Müller, K. (2019) · RePEc: Research Papers in Economics · Cited by 580
Cai, Y., Guan, K., Peng, J., Wang, S., Seifert, C., Wardlow, B., & Li, Z. (2018) · Remote Sensing of Environment · doi:10.1016/j.rse.2018.02.045 · Cited by 540
Parallel Recurrent Neural Network Architectures for Feature-rich Session-based Recommendations
Hidasi, B., Quadrana, M., Karatzoglou, A., & Tikk, D. (2016) · Proceedings of the 10th ACM Conference on Recommender Systems · doi:10.1145/2959100.2959167 · Cited by 523
Coucke, A., Saade, A., Ball, A., Bluche, T., Caulier, A., Leroy, D., Doumouro, C., Gisselbrecht, T., Caltagirone, F., Lavril, T., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.10190 · Cited by 272
MS MARCO: A Human Generated MAchine Reading COmprehension Dataset.
Nguyen, T. G., Rosenberg, M., Xia, S., Gao, J., Tiwary, S., Majumder, R., & Deng, L. (2016) · neural information processing systems · Cited by 1,961
Self-citationFlexible Neuromorphic Electronics for Computing, Soft Robotics, and Neuroprosthetics
Park, H., Lee, Y., Kim, N., Seo, D., Go, G., & Lee, T. (2020) · Advanced Materials · doi:10.1002/adma.201903558 · Cited by 533
Rifaioglu, A. S., Atas, H., Martin, M. J., Cetin-Atalay, R., Atalay, V., & Doğan, T. (2019) · Briefings in Bioinformatics · doi:10.1093/bib/bby061 · Cited by 529
Artificial intelligence: Implications for the future of work
Howard, J. (2019) · American Journal of Industrial Medicine · doi:10.1002/ajim.23037 · Cited by 522
Wang, H., Liu, Z., Peng, D., & Qin, Y. (2020) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2019.2955540 · Cited by 519
Rodríguez-Pérez, R., & Bajorath, J. (2020) · Journal of Medicinal Chemistry · doi:10.1021/acs.jmedchem.9b01101 · Cited by 512
The Marabou Framework for Verification and Analysis of Deep Neural Networks
Katz, G., Huang, D. A., Ibeling, D., Julian, K., Lazarus, C., Lim, R., Shah, P., Thakoor, S., Wu, H., Zeljić, A., et al. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-25540-4_26 · Cited by 517
A Very Brief Introduction to Machine Learning With Applications to Communication Systems
Simeone, O. (2018) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2018.2881442 · Cited by 516
Samuel, N., Diskin, T., & Wiesel, A. (2019) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2019.2899805 · Cited by 515
Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks
Hernández-Lobato, J. M., & Adams, R. P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1502.05336 · Cited by 504
Multimodal intelligence: Representation learning, information fusion, and applications
Zhang, C., Yang, Z., He, X., & Deng, L. (2020) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2020.2987728 · Cited by 511
Self-citationMultimodal Classification of Remote Sensing Images: A Review and Future Directions
Gomez-Chova, L., Tuia, D., Moser, G., & Camps-Valls, G. (2015) · Proceedings of the IEEE · doi:10.1109/jproc.2015.2449668 · Cited by 508
From Feedforward to Recurrent LSTM Neural Networks for Language Modeling
Sundermeyer, M., Ney, H., & Schluter, R. (2015) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2015.2400218 · Cited by 507
Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionality
Ma, X., Li, B., Wang, Y., Erfani, S., Wijewickrema, S. N. R., Schoenebeck, G. R., Song, D. X., Houle, M. E., & Bailey, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.02613 · Cited by 440
Machine Learning Methods for Small Data Challenges in Molecular Science
Dou, B., Zhu, Z., Merkurjev, E., Ke, L., Chen, L., Jiang, J., Zhu, Y., Liu, J., Zhang, B., & Wei, G.-W. (2023) · Chemical Reviews · doi:10.1021/acs.chemrev.3c00189 · Cited by 499
A novel scheme for speaker recognition using a phonetically-aware deep neural network
Lei, Y., Scheffer, N., Ferrer, L., & McLaren, M. (2014) · 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2014.6853887 · Cited by 496
FPGA-Based Accelerators of Deep Learning Networks for Learning and Classification: A Review
Shawahna, A., Sait, S. M., & El-Maleh, A. (2019) · IEEE Access · doi:10.1109/access.2018.2890150 · Cited by 496
“and throughput as they are highly efficient in floating-point matrix-based operations [57]–[59].”
BackgroundWallach, I., Dzamba, M., & Heifets, A. (2015) · arXiv · doi:10.48550/arxiv.1510.02855
Could Machine Learning Break the Convection Parameterization Deadlock?
Gentine, P., Pritchard, M., Rasp, S., Reinaudi, G., & Yacalis, G. (2018) · Geophysical Research Letters · doi:10.1029/2018gl078202 · Cited by 497
Deep Convolutional Neural Networks for Large-scale Speech Tasks
Sainath, T. N., Kingsbury, B., Saon, G., Soltau, H., Mohamed, A.-R., Dahl, G., & Ramabhadran, B. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.08.005 · Cited by 496
Self-citationFügener, A., Grahl, J., Gupta, A., & Ketter, W. (2022) · Information Systems Research · doi:10.1287/isre.2021.1079 · Cited by 478
DeepFix: A Fully Convolutional Neural Network for Predicting Human Eye Fixations
Kruthiventi, S. S. S., Ayush, K., & Babu, R. V. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2710620 · Cited by 478
Gokmen, T., & Vlasov, Y. (2016) · Frontiers in Neuroscience · doi:10.3389/fnins.2016.00333 · Cited by 491
Artificial intelligence: a survey on evolution, models, applications and future trends
Lu, Y. (2019) · Journal of Management Analytics · doi:10.1080/23270012.2019.1570365 · Cited by 641
Wetzel, S. J. (2017) · Physical Review E · doi:10.1103/physreve.96.022140 · Cited by 488
A Systematic DNN Weight Pruning Framework Using Alternating Direction Method of Multipliers
Zhang, T., Ye, S., Zhang, K., Tang, J., Wen, W., Fardad, M., & Wang, Y. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01237-3_12 · Cited by 473
Deep Neural Networks Motivated by Partial Differential Equations
Ruthotto, L., & Haber, E. (2020) · Journal of Mathematical Imaging and Vision · doi:10.1007/s10851-019-00903-1 · Cited by 460
Bui, D. T., Tsangaratos, P., Nguyen, V.-T., Liem, N. V., & Trinh, P. T. (2020) · CATENA · doi:10.1016/j.catena.2019.104426 · Cited by 467
A review on neural networks with random weights
Cao, W., Wang, X., Ming, Z., & Gao, J. (2018) · Neurocomputing · doi:10.1016/j.neucom.2017.08.040 · Cited by 466
Multi-key privacy-preserving deep learning in cloud computing
Li, P., Li, J., Huang, Z., Li, T., Gao, C.-Z., Yiu, S.-M., & Chen, K. (2017) · Future Generation Computer Systems · doi:10.1016/j.future.2017.02.006 · Cited by 462
Using Recurrent Neural Networks for Slot Filling in Spoken Language Understanding
Mesnil, G., Dauphin, Y., Yao, K., Bengio, Y., Deng, L., Hakkani-Tur, D., He, X., Heck, L., Tur, G., Yu, D., et al. (2015) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2014.2383614 · Cited by 461
Self-citationDeep learning in microscopy image analysis: a survey
Xing, F., Xie, Y., Su, H., Liu, F., & Yang, L. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2017.2766168 · Cited by 460
Artificial intelligence in clinical and genomic diagnostics
Dias, R., & Torkamani, A. (2019) · Genome Medicine · doi:10.1186/s13073-019-0689-8 · Cited by 456
Machine Learning Paradigms for Speech Recognition: An Overview
Deng, L., & Li, X. (2013) · IEEE Transactions on Audio, Speech, and Language Processing · doi:10.1109/tasl.2013.2244083 · Cited by 456
Self-citationThe future of human-AI collaboration: a taxonomy of design knowledge for hybrid intelligence systems
Dellermann, D., Calma, A., Lipusch, N., Weber, T., Weigel, S., & Ebel, P. (2021) · arXiv · doi:10.48550/arxiv.2105.03354 · Cited by 456
Big data and artificial intelligence modeling for drug discovery
Zhu, H. (2020) · Annual Review of Pharmacology and Toxicology · doi:10.1146/annurev-pharmtox-010919-023324 · Cited by 455
Supervised Machine Learning for Population Genetics: A New Paradigm
Schrider, D. R., & Kern, A. D. (2018) · Trends in Genetics · doi:10.1016/j.tig.2017.12.005 · Cited by 453
Guiding Deep Learning System Testing Using Surprise Adequacy
Kim, J., Feldt, R., & Yoo, S. (2019) · 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE) · doi:10.1109/icse.2019.00108 · Cited by 430
DeepX: A Software Accelerator for Low-Power Deep Learning Inference on Mobile Devices
Lane, N. D., Bhattacharya, S., Georgiev, P., Forlivesi, C., Jiao, L., Qendro, L., & Kawsar, F. (2016) · 2016 15th ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN) · doi:10.1109/ipsn.2016.7460664 · Cited by 450
De novo peptide sequencing by deep learning
Tran, N. H., Zhang, X., Xin, L., Shan, B., & Li, M. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1705691114 · Cited by 441
Minerva: enabling low-power, highly-accurate deep neural network accelerators
Reagen, B., Whatmough, P., Adolf, R., Rama, S., Lee, H., Lee, S. K., Hernández-Lobato, J. M., Wei, G.-Y., & Brooks, D. (2016) · ACM SIGARCH Computer Architecture News · doi:10.1145/3007787.3001165 · Cited by 446
“These methods have gained popularity over the last ten years thanks to empirical achievements on a wide range of tasks including speech recognition [2] [3], computer vision [4], and natural language processing [5] [6].”
BackgroundRobust Physical-World Attacks on Deep Learning Models
Eykholt, K., Evtimov, I., Fernandes, E., Li, B., Rahmati, A., Xiao, C., Prakash, A., Kohno, T., & Song, D. X. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.08945 · Cited by 444
Zhang, S., Zhang, S., Huang, T., & Gao, W. (2018) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2017.2766843 · Cited by 441
Describing Multimedia Content Using Attention-Based Encoder-Decoder Networks
Cho, K., Courville, A., & Bengio, Y. (2015) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2015.2477044 · Cited by 338
DeepHunter: a coverage-guided fuzz testing framework for deep neural networks
Xie, X., Ma, L., Juefei-Xu, F., Xue, M., Chen, H., Liu, Y., Zhao, J., Li, B., Yin, J., & See, S. (2019) · Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis · doi:10.1145/3293882.3330579 · Cited by 426
The physics of brain network structure, function and control
Lynn, C. W., & Bassett, D. S. (2019) · Nature Reviews Physics · doi:10.1038/s42254-019-0040-8 · Cited by 400
Learning Deep Generative Models
Salakhutdinov, R. (2015) · Annual Review of Statistics and Its Application · doi:10.1146/annurev-statistics-010814-020120 · Cited by 436
TTS synthesis with bidirectional LSTM based recurrent neural networks
Fan, Y., Qian, Y., Xie, F.-L., & Soong, F. K. (2014) · Interspeech 2014 · doi:10.21437/interspeech.2014-443 · Cited by 426
Deep Neural Network for Structural Prediction and Lane Detection in Traffic Scene
Li, J., Mei, X., Prokhorov, D., & Tao, D. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2522428 · Cited by 435
Stock closing price prediction based on sentiment analysis and LSTM
Jin, Z., Yang, Y., & Liu, Y. (2020) · Neural Computing and Applications · doi:10.1007/s00521-019-04504-2 · Cited by 434
“In [17], the author constructed an autoregressive dynamic Bayesian network (AR-DBN) based on dynamic Bayesian network (DBN) [18] and inferred the market index, which improves the predictability of stock market volatility.”
Parallel physics-informed neural networks via domain decomposition
Shukla, K., Jagtap, A. D., & Karniadakis, G. E. (2021) · Journal of Computational Physics · doi:10.1016/j.jcp.2021.110683 · Cited by 431
Using deep learning to model the hierarchical structure and function of a cell
Ma, J., Yu, M. K., Fong, S., Ono, K., Sage, E., Demchak, B., Sharan, R., & Ideker, T. (2018) · Nature Methods · doi:10.1038/nmeth.4627 · Cited by 435
Optimizing deep learning hyper-parameters through an evolutionary algorithm
Young, S. R., Rose, D. C., Karnowski, T. P., Lim, S.-H., & Patton, R. M. (2015) · Proceedings of the Workshop on Machine Learning in High-Performance Computing Environments · doi:10.1145/2834892.2834896 · Cited by 424
Self-Balancing Federated Learning With Global Imbalanced Data in Mobile Systems
Duan, M., Liu, D., Chen, X., Liu, R., Tan, Y., & Liang, L. (2021) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2020.3009406 · Cited by 432
Medical image retrieval using deep convolutional neural network
Qayyum, A., Anwar, S. M., Awais, M., & Majid, M. (2017) · Neurocomputing · doi:10.1016/j.neucom.2017.05.025 · Cited by 433
Wang, S., Song, J., Lien, J., Poupyrev, I., & Hilliges, O. (2016) · Proceedings of the 29th Annual Symposium on User Interface Software and Technology · doi:10.1145/2984511.2984565 · Cited by 433
Optimal approximation of piecewise smooth functions using deep ReLU neural networks
Petersen, P., & Voigtlaender, F. (2018) · Neural Networks · doi:10.1016/j.neunet.2018.08.019 · Cited by 432
Gene expression inference with deep learning
Chen, Y., Li, Y., Narayan, R., Subramanian, A., & Xie, X. (2016) · Bioinformatics · doi:10.1093/bioinformatics/btw074 · Cited by 431
Very deep convolutional networks for end-to-end speech recognition
Zhang, Y., Chan, W., & Jaitly, N. (2017) · 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2017.7953077 · Cited by 407
Self-citationDeepPF: A deep learning based architecture for metro passenger flow prediction
Liu, Y., Liu, Z., & Jia, R. (2019) · Transportation Research Part C: Emerging Technologies · doi:10.1016/j.trc.2019.01.027 · Cited by 426
Emerging Memristive Artificial Synapses and Neurons for Energy‐Efficient Neuromorphic Computing
Choi, S., Yang, J., & Wang, G. (2020) · Advanced Materials · doi:10.1002/adma.202004659 · Cited by 424
A Deep Learning Scheme for Motor Imagery Classification based on Restricted Boltzmann Machines
Lu, N., Li, T., Ren, X., & Miao, H. (2017) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2016.2601240 · Cited by 419
Yu, D., Yao, K., Su, H., Li, G., & Seide, F. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6639201 · Cited by 408
Self-citationDeep residual learning-based fault diagnosis method for rotating machinery
Zhang, W., Li, X., & Ding, Q. (2019) · ISA Transactions · doi:10.1016/j.isatra.2018.12.025 · Cited by 416
Biswas, A., & Chandrakasan, A. P. (2019) · IEEE Journal of Solid-State Circuits · doi:10.1109/jssc.2018.2880918 · Cited by 414
Neural network analysis of sleep stages enables efficient diagnosis of narcolepsy
Stephansen, J. B., Olesen, A. N., Olsen, M., Ambati, A., Leary, E. B., Moore, H. E., Carrillo, O., Lin, L., Han, F., Yan, H., et al. (2018) · Nature Communications · doi:10.1038/s41467-018-07229-3 · Cited by 413
Artificial intelligence: foundations of computational agents
(2010) · Choice Reviews Online · doi:10.5860/choice.48-2130 · Cited by 415
Learned Step Size Quantization
Esser, S. K., McKinstry, J. L., Bablani, D., Appuswamy, R., & Modha, D. S. (2019) · arXiv · doi:10.48550/arxiv.1902.08153 · Cited by 415
“Deep networks are emerging as components of a number of revolutionary technologies, including image recognition (Krizhevsky et al., 2012), speech recognition (Hinton et al., 2012), and driving assistance (Xu et al., 2017).”
BackgroundEmoNets: Multimodal deep learning approaches for emotion recognition in video
Kahou, S. E., Bouthillier, X., Lamblin, P., Gulcehre, C., Michalski, V., Konda, K., Jean, S., Froumenty, P., Dauphin, Y., Boulanger-Lewandowski, N., et al. (2016) · Journal on Multimodal User Interfaces · doi:10.1007/s12193-015-0195-2 · Cited by 412
Pourbabaee, B., Roshtkhari, M. J., & Khorasani, K. (2018) · IEEE Transactions on Systems, Man, and Cybernetics: Systems · doi:10.1109/tsmc.2017.2705582 · Cited by 411
The Challenge of Machine Learning in Space Weather: Nowcasting and Forecasting
Camporeale, E. (2019) · Space Weather · doi:10.1029/2018sw002061 · Cited by 403
“Maybe because of the computational cost of training a LSTM network, only three inputs were used in all experiments (one of which is always the flux itself).”
MethodInfluentialGShard: Scaling Giant Models with Conditional Computation and Automatic Sharding
Lepikhin, D., Lee, H., Xu, Y., Chen, D., Firat, O., Huang, Y., Krikun, M., Shazeer, N., & Chen, Z. (2020) · arXiv · doi:10.48550/arxiv.2006.16668 · Cited by 412
“For years, the fields have been continuously reporting new state of the art results using varieties of model architectures for computer vision tasks [57, 58, 7], for natural language understanding tasks [59, 60, 61], for speech recognition and synthesis tasks [62, 63, 64, 65, 66].”
BackgroundReal-time classification and sensor fusion with a spiking deep belief network
O'Connor, P., Neil, D., Liu, S.-C., Delbruck, T., & Pfeiffer, M. (2013) · Frontiers in Neuroscience · doi:10.3389/fnins.2013.00178 · Cited by 408
Metadata
- Authors
- Hinton, G., Deng, L., Yu, D., Dahl, G., Mohamed, A.-R., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T., & Kingsbury, B.
- Affiliations
- University of Toronto (CA)
- University of Waterloo (CA)
- Microsoft (United States) (US)
- Google (United States) (US)
- IBM (United States) (US)
- Michigan State University (US)
- Venue
- IEEE Signal Processing Magazine
- Published
- 2012-11
- Volume
- 29
- Issue
- 6
- Pages
- 82-97
- ISSN
- 1053-5888 (ISSN-L)
- OpenAlex
- W2160815625
- Cited by
- 10,260
- References
- 71
- Citations by year
- 2026: 253 · 2025: 526 · 2024: 610 · 2023: 696 · 2022: 837 · 2021: 978 · 2020: 1232 · 2019: 1313 · 2018: 1220 · 2017: 967 · 2016: 712 · 2015: 498 · 2014: 276 · 2013: 128 · 2012: 9
- FWCI
- 284.71
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Signal Processing Magazine (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Keywords
- deep neural network, acoustic modeling, speech recognition, hidden Markov models, Gaussian mixture model, feedforward neural networks, posterior probability
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.