Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups
The DOI matches its Crossref record; 68 of the 71 references it cites were found in a registry or an index.
Abstract
Most current speech recognition systems use hidden Markov models (HMMs) to deal with the temporal variability of speech and Gaussian mixture models (GMMs) to determine how well each state of each HMM fits a frame or a short window of frames of coefficients that represents the acoustic input. An alternative way to evaluate the fit is to use a feed-forward neural network that takes several frames of coefficients as input and produces posterior probabilities over HMM states as output. Deep neural networks (DNNs) that have many hidden layers and are trained using new methods have been shown to outperform GMMs on a variety of speech recognition benchmarks, sometimes by a large margin. This article provides an overview of this progress and represents the shared views of four research groups that have had recent successes in using DNNs for acoustic modeling in speech recognition.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Deep Learning has largely alleviated the need for manual feature extraction, achieving state-ofthe-art performance in fields such as computer vision and speech recognition [13,14].”
From EEGNet: a compact convolutional neural network for EEG-based brain-computer interfaces (2018) · cites this paper as [13]
In the citing paper
“In particular, neural networks became competitive, solving problems in speech recognition [68], image recognition [69] and language translation [70].”
From The growing ubiquity of algorithms in society: implications, impacts and innovations (2018) · cites this paper as [68]
In the citing paper
“Apart from images, sequential data such as text and audio can also be processed with DNNs to reach state-of-the-art performance for document classification and speech recognition.”
From Deep learning for time series classification: a review (2019) · cites this paper as Hinton et al., 2012
In the citing paper
“The other more direct reason behind the high accuracies of deep CNNs on time series data is its success in other sequential data such as speech recognition (Hinton et al., 2012) and sentence classification (Kim, 2014)”
From Deep learning for time series classification: a review (2019) · cites this paper as Hinton et al., 2012
In the citing paper
“Recent studies have reported that deep learning methods have been successfully applied to many applications areas e.g., image and video classification [11-13], visual tracking [14], speech recognition [15] and natural language processing [16].”
From Medical image retrieval using deep convolutional neural network (2017) · cites this paper as [15]
Citations
Cites
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892
Self-citationReducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
Self-citationA fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336
Self-citation(2010) · Cited by 14,134
Training Products of Experts by Minimizing Contrastive Divergence
Hinton, G. E. (2002) · Neural Computation · doi:10.1162/089976602760128018 · Cited by 4,879 · cited 3×
“…learning algorithm for a restricted Boltzmann machine (RBM) which consists of a layer of stochastic binary " visible " units that represent binary input data connected to a layer of stochastic binary hidden units that learn to model significant nonindependencies between the visible units [20].”
InfluentialSelf-citation(2010) · Journal of Machine Learning Research · doi:10.5555/1756006.1953039 · Cited by 4,051
Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition
Dahl, G. E., Yu, D., Deng, L., & Acero, A. (2012) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2011.2134090 · Cited by 3,011
Self-citationA Practical Guide to Training Restricted Boltzmann Machines
Hinton, G. E. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-35289-8_32 · Cited by 2,852 · cited 2×
“Learning the standard deviations of a GRBM is problematic for reasons described in [21], so for pretraining using CD 1 , the data are normalized so that each coefficient has zero mean and unit variance, the standard deviations are set to one when computing ( ) v h p ; , and no noise is added to the…”
Self-citationPerceptual linear predictive (PLP) analysis of speech
Hermansky, H. (1990) · The Journal of the Acoustical Society of America · doi:10.1121/1.399423 · Cited by 2,536
Acoustic Modeling Using Deep Belief Networks
Mohamed, A.-R., Dahl, G. E., & Hinton, G. (2012) · IEEE Transactions on Audio, Speech, and Language Processing · doi:10.1109/tasl.2011.2109382 · Cited by 1,727
Self-citation
Research works printed but not linked: 5
- sainath, 2011, Improvements in using deep belief networks for large vocabulary continuous speech recognition, Speech and Language Algorithm Group IBM Yorktown Heights NY Tech Rep UTML TR 2010-003
- rifai, 0, Contractive autoencoders: Explicit invariance during feature extraction, Proc 28th Int Conf Machine Learning, 833
- yu, 2011, Discriminative pretraining of deep neural networks, U S Patent Filing
- mohamed, 0, Deep belief networks for phone recognition, Proc NIPS Workshop Deep Learning for Speech Recognition and Related Applications
- pearl, 1988, Probabilistic Inference in Intelligent Systems Networks of Plausible Inference
Cited by
Items 101–200 of 1,106
A review on the application of deep learning in system health management
Khan, S., & Yairi, T. (2018) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2017.11.024 · Cited by 1,133
Bao, W., Yue, J., & Rao, Y. (2017) · PLOS ONE · doi:10.1371/journal.pone.0180944 · Cited by 1,124
End-to-end attention-based large vocabulary speech recognition
Bahdanau, D., Chorowski, J., Serdyuk, D., Brakel, P., & Bengio, Y. (2016) · 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2016.7472618 · Cited by 1,096
Towards artificial general intelligence with hybrid Tianjic chip architecture
Pei, J., Deng, L., Song, S., Zhao, M., Zhang, Y., Wu, S., Wang, G., Zou, Z., Wu, Z., He, W., et al. (2019) · Nature · doi:10.1038/s41586-019-1424-8 · Cited by 1,082
Self-citationKather, J. N., Krisam, J., Charoentong, P., Luedde, T., Herpel, E., Weis, C.-A., Gaiser, T., Marx, A., Valous, N. A., Ferber, D., et al. (2019) · PLOS Medicine · doi:10.1371/journal.pmed.1002730 · Cited by 1,082
“Applications of CNNsrangewidely,from speechrecognition[14,15],facerecognition[16], or traffic sign classification[17] to masteringtheJapanese gameof Go [18].”
BackgroundDeep learning modelling techniques: Current progress, applications, advantages, and challenges
Ahmed, S. F., Alam, M. S. B., Hassan, M., Rozbu, M. R., Ishtiak, T., Rafa, N., Mofijur, M., Shawkat Ali, A. B. M., & Gandomi, A. H. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-023-10466-8 · Cited by 1,079
Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentation
Liu, C., Chen, L.-C., Schroff, F., Adam, H., Hua, W., Yuille, A. L., & Fei-Fei, L. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00017 · Cited by 1,025
Deep neural networks for small footprint text-dependent speaker verification
Variani, E., Lei, X., McDermott, E., Moreno, I. L., & Gonzalez-Dominguez, J. (2014) · 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2014.6854363 · Cited by 1,035
Deep convolutional neural networks for LVCSR
Sainath, T. N., Mohamed, A.-R., Kingsbury, B., & Ramabhadran, B. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6639347 · Cited by 1,053
Self-citationMachine learning in materials science
Wei, J., Chu, X., Sun, X., Xu, K., Deng, H., Chen, J., Wei, Z., & Lei, M. (2019) · InfoMat · doi:10.1002/inf2.12028 · Cited by 1,069
Emerging Artificial Intelligence Methods in Structural Engineering
Salehi, H., & Burgueño, R. (2018) · Engineering Structures · doi:10.1016/j.engstruct.2018.05.084 · Cited by 1,068
xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems
Lian, J., Zhou, X., Zhang, F., Chen, Z., Xie, X., & Sun, G. (2018) · Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3219819.3220023 · Cited by 1,040
A Unifying Review of Deep and Shallow Anomaly Detection
Ruff, L., Kauffmann, J. R., Vandermeulen, R. A., Montavon, G., Samek, W., Kloft, M., Dietterich, T. G., & Muller, K.-R. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2021.3052449 · Cited by 914
MagNet: A Two-Pronged Defense against Adversarial Examples
Meng, D., & Chen, H. (2017) · Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3133956.3134057 · Cited by 1,006
Deep Convolutional Neural Network Based Regression Approach for Estimation of Remaining Useful Life
Sateesh Babu, G., Zhao, P., & Li, X.-L. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-32025-0_14 · Cited by 962
Compressing Neural Networks with the Hashing Trick
Chen, W., Wilson, J. T., Tyree, S., Weinberger, K. Q., & Chen, Y. (2015) · arXiv · doi:10.48550/arxiv.1504.04788 · Cited by 974
High-performance brain-to-text communication via handwriting
Willett, F. R., Avansino, D. T., Hochberg, L. R., Henderson, J. M., & Shenoy, K. V. (2021) · Nature · doi:10.1038/s41586-021-03506-2 · Cited by 1,024
DeepARG: a deep learning approach for predicting antibiotic resistance genes from metagenomic data
Arango-Argoty, G., Garner, E., Pruden, A., Heath, L. S., Vikesland, P., & Zhang, L. (2018) · Microbiome · doi:10.1186/s40168-018-0401-z · Cited by 958
Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks
Liang, S., Li, Y., & Srikant, R. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.02690 · Cited by 611
Deep content-based music recommendation
van den Oord, A., Dieleman, S., & Schrauwen, B. (2013) · Ghent University Academic Bibliography (Ghent University) · Cited by 1,425
Deep-Learning-Enabled On-Demand Design of Chiral Metamaterials
Ma, W., Cheng, F., & Liu, Y. (2018) · ACS Nano · doi:10.1021/acsnano.8b03569 · Cited by 925
Accurate De Novo Prediction of Protein Contact Map by Ultra-Deep Learning Model
Wang, S., Sun, S., Li, Z., Zhang, R., & Xu, J. (2017) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1005324 · Cited by 931
Nweke, H. F., Teh, Y. W., Al-garadi, M. A., & Alo, U. R. (2018) · Expert Systems with Applications · doi:10.1016/j.eswa.2018.03.056 · Cited by 930
Using recurrent neural network models for early detection of heart failure onset
Choi, E., Schuetz, A., Stewart, W. F., & Sun, J. (2017) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocw112 · Cited by 928
“Since then, variations of neural network application have explored deep architectures in computer vision, 14–16 audio processing, 17,18 and natural language processing (NLP), 11,19–21 among other fields.”
BackgroundAn Experimental Study on Speech Enhancement Based on Deep Neural Networks
Xu, Y., Du, J., Dai, L.-R., & Lee, C.-H. (2014) · IEEE Signal Processing Letters · doi:10.1109/lsp.2013.2291240 · Cited by 924
A Survey on Neural Network Interpretability
Zhang, Y., Tino, P., Leonardis, A., & Tang, K. (2021) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2021.3100641 · Cited by 923
A survey of machine learning for big data processing
Qiu, J., Wu, Q., Ding, G., Xu, Y., & Feng, S. (2016) · EURASIP Journal on Advances in Signal Processing · doi:10.1186/s13634-016-0355-x · Cited by 894
Long Short-Term Memory Networks for Accurate State-of-Charge Estimation of Li-ion Batteries
Chemali, E., Kollmeyer, P. J., Preindl, M., Ahmed, R., & Emadi, A. (2018) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2017.2787586 · Cited by 906
Adieu features? End-to-end speech emotion recognition using a deep convolutional recurrent network
Trigeorgis, G., Ringeval, F., Brueckner, R., Marchi, E., Nicolaou, M. A., Schuller, B., & Zafeiriou, S. (2016) · 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2016.7472669 · Cited by 904
Neural Networks and Deep Learning
Theodoridis, S. (2015) · Machine Learning · doi:10.1016/b978-0-12-801522-3.00018-5 · Cited by 900
Automatic diagnosis of the 12-lead ECG using a deep neural network
Ribeiro, A. H., Ribeiro, M. H., Paixão, G. M. M., Oliveira, D. M., Gomes, P. R., Canazart, J. A., Ferreira, M. P. S., Andersson, C. R., Macfarlane, P. W., Meira, W., et al. (2020) · Nature Communications · doi:10.1038/s41467-020-15432-4 · Cited by 894
Semantically Conditioned LSTM-based Natural Language Generation for Spoken Dialogue Systems
Wen, T.-H., Gasic, M., Mrkšić, N., Su, P.-H., Vandyke, D., & Young, S. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1199 · Cited by 874
Deep Learning for Audio Signal Processing
Purwins, H., Li, B., Virtanen, T., Schluter, J., Chang, S.-Y., & Sainath, T. (2019) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2019.2908700 · Cited by 855
“feedforward neural networks [3], [5], convolutional neural networks (CNNs, [6]) and long short-term memory (LSTM, [7]).”
BackgroundMethodSelf-citationJoint CTC-attention based end-to-end speech recognition using multi-task learning
Kim, S., Hori, T., & Watanabe, S. (2017) · 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2017.7953075 · Cited by 844
DeepPath: A Reinforcement Learning Method for Knowledge Graph Reasoning
Xiong, W., Hoang, T., & Wang, W. Y. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1060 · Cited by 812
Permutation invariant training of deep models for speaker-independent multi-talker speech separation
Yu, D., Kolbaek, M., Tan, Z.-H., & Jensen, J. (2017) · 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2017.7952154 · Cited by 839
Self-citationBrain tumor classification for MR images using transfer learning and fine-tuning
Swati, Z. N. K., Zhao, Q., Kabir, M., Ali, F., Ali, Z., Ahmed, S., & Lu, J. (2019) · Computerized Medical Imaging and Graphics · doi:10.1016/j.compmedimag.2019.05.001 · Cited by 840
Adaptive Graph Convolutional Neural Networks
Li, R., Wang, S., Zhu, F., & Huang, J. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11691 · Cited by 842
Palangi, H., Deng, L., Shen, Y., Gao, J., He, X., Chen, J., Song, X., & Ward, R. (2016) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2016.2520371 · Cited by 618
Self-citationAutomated machine learning: Review of the state-of-the-art and opportunities for healthcare
Waring, J., Lindvall, C., & Umeton, R. (2020) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2020.101822 · Cited by 834
E, W., Han, J., & Jentzen, A. (2017) · Communications in Mathematics and Statistics · doi:10.1007/s40304-017-0117-6 · Cited by 822
Hypergraph convolution and hypergraph attention
Bai, S., Zhang, F., & Torr, P. H. S. (2021) · Pattern Recognition · doi:10.1016/j.patcog.2020.107637 · Cited by 809
A Deep Relevance Matching Model for Ad-hoc Retrieval
Guo, J., Fan, Y., Ai, Q., & Croft, W. B. (2016) · Proceedings of the 25th ACM International on Conference on Information and Knowledge Management · doi:10.1145/2983323.2983769 · Cited by 806
Kolbaek, M., Yu, D., Tan, Z.-H., & Jensen, J. (2017) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2017.2726762 · Cited by 820
Self-citationChemali, E., Kollmeyer, P. J., Preindl, M., & Emadi, A. (2018) · Journal of Power Sources · doi:10.1016/j.jpowsour.2018.06.104 · Cited by 818
Statistical parametric speech synthesis using deep neural networks
Zen, H., Senior, A., & Schuster, M. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6639215 · Cited by 809
Self-citationDeep Learning for Content-Based Image Retrieval: A Comprehensive Study
Wan, J., Wang, D., Hoi, S. C. H., Wu, P., Zhu, J., Zhang, Y., & Li, J. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654948 · Cited by 811
Measuring human perceptions of a large-scale urban region using machine learning
Zhang, F., Zhou, B., Liu, L., Liu, Y., Fung, H. H., Lin, H., & Ratti, C. (2018) · Landscape and Urban Planning · doi:10.1016/j.landurbplan.2018.08.020 · Cited by 810
Generative adversarial networks: introduction and outlook
Wang, K., Gou, C., Duan, Y., Lin, Y., Zheng, X., & Wang, F.-Y. (2017) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2017.7510583 · Cited by 806
Personal Sensing: Understanding Mental Health Using Ubiquitous Sensors and Machine Learning
Mohr, D. C., Zhang, M., & Schueller, S. M. (2017) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-032816-044949 · Cited by 802
Deep Audio-Visual Speech Recognition
Afouras, T., Chung, J. S., Senior, A., Vinyals, O., & Zisserman, A. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2889052 · Cited by 801
Self-citationRecent advances in deep learning for speech research at Microsoft
Deng, L., Li, J., Huang, J.-T., Yao, K., Yu, D., Seide, F., Seltzer, M., Zweig, G., He, X., Williams, J., et al. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6639345 · Cited by 801
Self-citationHybrid CTC/Attention Architecture for End-to-End Speech Recognition
Watanabe, S., Hori, T., Kim, S., Hershey, J. R., & Hayashi, T. (2017) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2017.2763455 · Cited by 800
DeepStack: Expert-level artificial intelligence in heads-up no-limit poker
Moravčík, M., Schmid, M., Burch, N., Lisý, V., Morrill, D., Bard, N., Davis, T., Waugh, K., Johanson, M., & Bowling, M. (2017) · Science · doi:10.1126/science.aam6960 · Cited by 786
A hybrid deep learning based traffic flow prediction method and its understanding
Wu, Y., Tan, H., Qin, L., Ran, B., & Jiang, Z. (2018) · Transportation Research Part C: Emerging Technologies · doi:10.1016/j.trc.2018.03.001 · Cited by 798
Data Augmentation Generative Adversarial Networks
Antoniou, A., Storkey, A., & Edwards, H. S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.04340 · Cited by 747
von Rueden, L., Mayer, S., Beckh, K., Georgiev, B., Giesselbach, S., Heese, R., Kirsch, B., Walczak, M., Pfrommer, J., Pick, A., et al. (2021) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3079836 · Cited by 793
Xiao, C., Choi, E., & Sun, J. (2018) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocy068 · Cited by 792
AISHELL-1: An open-source Mandarin speech corpus and a speech recognition baseline
Bu, H., Du, J., Na, X., Wu, B., & Zheng, H. (2017) · 2017 20th Conference of the Oriental Chapter of the International Coordinating Committee on Speech Databases and Speech I/O Systems and Assessment (O-COCOSDA) · doi:10.1109/icsda.2017.8384449 · Cited by 791
Deep Learning for Intelligent Wireless Networks: A Comprehensive Survey
Mao, Q., Hu, F., & Hao, Q. (2018) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2018.2846401 · Cited by 789
Automatic Image-Based Plant Disease Severity Estimation Using Deep Learning
Wang, G., Sun, Y., & Wang, J. (2017) · Computational Intelligence and Neuroscience · doi:10.1155/2017/2917536 · Cited by 746
End-to-End Multimodal Emotion Recognition using Deep Neural Networks
Tzirakis, P., Trigeorgis, G., Nicolaou, M. A., Schuller, B., & Zafeiriou, S. (2017) · arXiv · doi:10.48550/arxiv.1704.08619 · Cited by 752
Recurrent Neural Networks: A Comprehensive Review of Architectures, Variants, and Applications
Mienye, I. D., Swart, T. G., & Obaido, G. (2024) · Information · doi:10.3390/info15090517 · Cited by 732
Ball, J. E., Anderson, D. T., & Chan, C. S. (2017) · Journal of Applied Remote Sensing · doi:10.1117/1.jrs.11.042609 · Cited by 746
From Eliza to XiaoIce: Challenges and opportunities with social chatbots
Shum, H.-Y., He, X.-D., & Li, D. (2018) · Frontiers of Information Technology & Electronic Engineering · doi:10.1631/fitee.1700826 · Cited by 749
“, 2013, 2015), speech recognition and synthesis (Hinton et al., 2012; Deng et al., 2013; Qian et al., 2014; van den Oord et al., 2016; Xiong et al., 2016), computer vision (Krizhevsky et al.”
BackgroundGeneralized Latent Multi-View Subspace Clustering
Zhang, C., Fu, H., Hu, Q., Cao, X., Xie, Y., Tao, D., & Xu, D. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2877660 · Cited by 739
“Neural network-based methods [51], [52] can flexibly learn highly non-linear mappings, so here we employ neural networks to address complex relationships between the latent representation and the features from individual views, and the non-linear interactions among multiple views.”
Cheng, J.-Z., Ni, D., Chou, Y.-H., Qin, J., Tiu, C.-M., Chang, Y.-C., Huang, C.-S., Shen, D., & Chen, C.-M. (2016) · Scientific Reports · doi:10.1038/srep24454 · Cited by 755
“Although deep learning has been successfully applied to many image-and acoustic-related problems 46 , it is less explored in the context of CADx.”
Medical Image Denoising Using Convolutional Denoising Autoencoders
Gondara, L. (2016) · 2016 IEEE 16th International Conference on Data Mining Workshops (ICDMW) · doi:10.1109/icdmw.2016.0041 · Cited by 737
“IN most cases, y is assumed to be generated from a well defined process.”
BackgroundGenerative Adversarial Networks for Hyperspectral Image Classification
Zhu, L., Chen, Y., Ghamisi, P., & Benediktsson, J. A. (2018) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2018.2805286 · Cited by 734
A Hierarchical Latent Variable Encoder-Decoder Model for Generating Dialogues
Serban, I., Sordoni, A., Lowe, R., Charlin, L., Pineau, J., Courville, A., & Bengio, Y. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10983 · Cited by 697
A tutorial survey of architectures, algorithms, and applications for deep learning
Deng, L. (2014) · APSIPA Transactions on Signal and Information Processing · doi:10.1017/atsip.2013.9 · Cited by 725
Self-citationApplications for deep learning in ecology
Christin, S., Hervet, É., & Lecomte, N. (2019) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.13256 · Cited by 724
Jo, T., Nho, K., & Saykin, A. J. (2019) · Frontiers in Aging Neuroscience · doi:10.3389/fnagi.2019.00220 · Cited by 724
Plasmonic nanostructure design and characterization via Deep Learning
Malkiel, I., Mrejen, M., Nagler, A., Arieli, U., Wolf, L., & Suchowski, H. (2018) · Light: Science & Applications · doi:10.1038/s41377-018-0060-7 · Cited by 715
EESEN: End-to-end speech recognition using deep RNN models and WFST-based decoding
Miao, Y., Gowayyed, M., & Metze, F. (2015) · 2015 IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru.2015.7404790 · Cited by 707
Ribeiro, A. H., Ribeiro, M. H., Paixao, G. M., Oliveira, D. M., Gomes, P. R., Canazart, J. A., Ferreira, M. P. S., Andersson, C. R., Macfarlane, P. W., Meira, W., et al. (2020) · RePEc: Research Papers in Economics · doi:10.5281/zenodo.3625006 · Cited by 715
DeepGauge: multi-granularity testing criteria for deep learning systems
Ma, L., Juefei-Xu, F., Zhang, F., Sun, J., Xue, M., Li, B., Chen, C., Su, T., Li, L., Liu, Y., et al. (2018) · Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering · doi:10.1145/3238147.3238202 · Cited by 588
Structured Pruning of Deep Convolutional Neural Networks
Anwar, S., Hwang, K., & Sung, W. (2017) · ACM Journal on Emerging Technologies in Computing Systems · doi:10.1145/3005348 · Cited by 698
Computational Grounded Theory: A Methodological Framework
Nelson, L. K. (2020) · Sociological Methods & Research · doi:10.1177/0049124117729703 · Cited by 697
“2013; Monroe, Colaresi, and Quinn 2008), psychologists (Tausczik and Pennebaker 2010; Yu and Ho 2014), and linguists (P. F. Brown et al. 1993; Hinton et al. 2012), are using computer-assisted text analysis techniques to bolster, and in some cases completely replace, traditional content analysis methods in their discip…”
MethodAdversarial Attacks and Defenses in Images, Graphs and Text: A Review
Xu, H., Ma, Y., Liu, H.-C., Deb, D., Liu, H., Tang, J.-L., & Jain, A. K. (2020) · International Journal of Automation and Computing · doi:10.1007/s11633-019-1211-x · Cited by 679
Speech Emotion Recognition Using Deep Learning Techniques: A Review
Khalil, R. A., Jones, E., Babar, M. I., Jan, T., Zafar, M. H., & Alhussain, T. (2019) · IEEE Access · doi:10.1109/access.2019.2936124 · Cited by 702
A sparse auto-encoder-based deep neural network approach for induction motor faults classification
Sun, W., Shao, S., Zhao, R., Yan, R., Zhang, X., & Chen, X. (2016) · Measurement · doi:10.1016/j.measurement.2016.04.007 · Cited by 698
Designing neural networks through neuroevolution
Stanley, K. O., Clune, J., Lehman, J., & Miikkulainen, R. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-018-0006-z · Cited by 689
Understanding deep convolutional networks
Mallat, S. (2016) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2015.0203 · Cited by 688
Tsubaki, M., Tomii, K., & Sese, J. (2019) · Bioinformatics · doi:10.1093/bioinformatics/bty535 · Cited by 688
Atzori, M., Cognolato, M., & Müller, H. (2016) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2016.00009 · Cited by 684
“Since 2010, the application of deep learning techniques to speech recognition has allowed a quick and impressive reduction of error rate (Dahl et al., 2010; Deng et al., 2010b, 2013; Hinton et al., 2012; Goodfellow et al., 2016).”
BackgroundIntrusion Detection System Using Deep Neural Network for In-Vehicle Network Security
Kang, M.-J., & Kang, J.-W. (2016) · PLOS ONE · doi:10.1371/journal.pone.0155781 · Cited by 678
Bengio, Y., Lecun, Y., & Hinton, G. (2021) · Communications of the ACM · doi:10.1145/3448250 · Cited by 661
“In 2010, essentially the same deep network was shown to beat the SOTA for large vocabulary speech recognition without requiring speaker-dependent training 28,46 and by 2012, Google had engineered a production version that significantly improved voice search on Android.”
Self-citationAnden, J., & Mallat, S. (2014) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2014.2326991 · Cited by 666
Deep learning and artificial intelligence methods for Raman and surface-enhanced Raman scattering
Lussier, F., Thibault, V., Charron, B., Wallace, G. Q., & Masson, J.-F. (2020) · TrAC Trends in Analytical Chemistry · doi:10.1016/j.trac.2019.115796 · Cited by 666
DeepSynergy: predicting anti-cancer drug synergy with Deep Learning
Preuer, K., Lewis, R. P. I., Hochreiter, S., Bender, A., Bulusu, K. C., & Klambauer, G. (2018) · Bioinformatics · doi:10.1093/bioinformatics/btx806 · Cited by 653
Kim, Y., & Moon, T. (2016) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2015.2491329 · Cited by 647
Lago, J., De Ridder, F., & De Schutter, B. (2018) · Applied Energy · doi:10.1016/j.apenergy.2018.02.069 · Cited by 642
Using machine learning to replicate chaotic attractors and calculate Lyapunov exponents from data
Pathak, J., Lu, Z., Hunt, B. R., Girvan, M., & Ott, E. (2017) · Chaos: An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/1.5010300 · Cited by 620
Closed-Loop Turbulence Control: Progress and Challenges
Brunton, S. L., & Noack, B. R. (2015) · Applied Mechanics Reviews · doi:10.1115/1.4031175 · Cited by 623
Deep learning approach for active classification of electrocardiogram signals
Rahhal, M. M. A., Bazi, Y., AlHichri, H., Alajlan, N., Melgani, F., & Yager, R. R. (2016) · Information Sciences · doi:10.1016/j.ins.2016.01.082 · Cited by 627
Deep Learning and Its Applications in Biomedicine
Cao, C., Liu, F., Tan, H., Song, D., Shu, W., Li, W., Zhou, Y., Bo, X., & Xie, Z. (2018) · Genomics, Proteomics & Bioinformatics · doi:10.1016/j.gpb.2017.07.003 · Cited by 624
Speaker Recognition by Machines and Humans: A tutorial review
Hansen, J. H. L., & Hasan, T. (2015) · IEEE Signal Processing Magazine · doi:10.1109/msp.2015.2462851 · Cited by 621
Ma, W., Cheng, F., Xu, Y., Wen, Q., & Liu, Y. (2019) · Advanced Materials · doi:10.1002/adma.201901111 · Cited by 615
“With the rapidly accumulated data and thus booming of deep learning (DL), the state of the art in many research domains, such as speech recognition[29], computer vision[30, 31], natural language processing [32] and decision making [33], has been pushed far beyond conventional methods.”
BackgroundGenerative Moment Matching Networks
Li, Y., Swersky, K., & Zemel, R. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1502.02761 · Cited by 517
Metadata
- Authors
- Hinton, G., Deng, L., Yu, D., Dahl, G., Mohamed, A.-R., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T., & Kingsbury, B.
- Affiliations
- University of Toronto (CA)
- University of Waterloo (CA)
- Microsoft (United States) (US)
- Google (United States) (US)
- IBM (United States) (US)
- Michigan State University (US)
- Venue
- IEEE Signal Processing Magazine
- Published
- 2012-11
- Volume
- 29
- Issue
- 6
- Pages
- 82-97
- ISSN
- 1053-5888 (ISSN-L)
- OpenAlex
- W2160815625
- Cited by
- 10,260
- References
- 71
- Citations by year
- 2026: 253 · 2025: 526 · 2024: 610 · 2023: 696 · 2022: 837 · 2021: 978 · 2020: 1232 · 2019: 1313 · 2018: 1220 · 2017: 967 · 2016: 712 · 2015: 498 · 2014: 276 · 2013: 128 · 2012: 9
- FWCI
- 284.71
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Signal Processing Magazine (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Keywords
- deep neural network, acoustic modeling, speech recognition, hidden Markov models, Gaussian mixture model, feedforward neural networks, posterior probability
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.