Machine Learning Paradigms for Speech Recognition: An Overview
The DOI matches its Crossref record; 229 of the 229 references it cites were found in a registry or an index.
Abstract
Automatic Speech Recognition (ASR) has historically been a driving force behind many machine learning (ML) techniques, including the ubiquitously used hidden Markov model, discriminative learning, structured sequence learning, Bayesian learning, and adaptive learning. Moreover, ML can and occasionally does use ASR as a large-scale, realistic application to rigorously test the effectiveness of a given technique, and to inspire new problems arising from the inherently sequential and dynamic nature of speech. On the other hand, even though ASR is available commercially for some applications, it is largely an unsolved problem—for almost all applications, the performance of ASR is not on par with human performance. New insight from modern ML methodology shows great promise to advance the state-of-the-art in ASR technology. This overview article provides readers with an overview of modern ML techniques as utilized in the current and as relevant to future ASR research and systems. The intent is to foster further cross-pollination between the ML and ASR communities than has occurred in the past. The article is organized according to the major ML paradigms that are either popular already or have potential for making significant contributions to ASR technology. The paradigms presented and elaborated in this overview include: generative and discriminative learning; supervised, unsupervised, semi-supervised, and active learning; adaptive and multi-task learning; and Bayesian learning. These learning paradigms are motivated and discussed in the context of ASR technology and applications. We finally present and analyze recent developments of deep learning and learning with sparse representations, focusing on their direct relevance to advancing ASR technology.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Cortes, C., & Vapnik, V. (1995) · Machine Learning · doi:10.1007/bf00994018 · Cited by 59,421
Maximum Likelihood from Incomplete Data Via the EM Algorithm
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1977.tb01600.x · Cited by 50,444
Wu, Y. (1999) · Technometrics · doi:10.1080/00401706.1999.10485951 · Cited by 27,221
A tutorial on hidden Markov models and selected applications in speech recognition
Rabiner, L. R. (1989) · Proceedings of the IEEE · doi:10.1109/5.18626 · Cited by 22,743 · cited 1×
“Use of vector quantization (VQ) to provide discrete inputs to ASR is one early successful application of unsupervised learning to ASR [138].”
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
Statistical Analysis With Missing Data
Lahiff, M., Little, R. J. A., & Rubin, D. B. (1989) · Journal of the American Statistical Association · doi:10.2307/2289883 · Cited by 17,265
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336
Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
Lafferty, J., McCallum, A., & Pereira, F. C. N. (2001) · ScholarlyCommons (University of Pennsylvania) · Cited by 12,935
Hinton, G., Deng, L., Yu, D., Dahl, G., Mohamed, A.-R., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T., et al. (2012) · IEEE Signal Processing Magazine · doi:10.1109/msp.2012.2205597 · Cited by 10,260
Self-citationLearning Deep Architectures for AI
Bengio, Y. (2009) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000006 · Cited by 9,239
Fundamentals of speech recognition
Rabiner, L. R., & Juang, B.-H. F. (1993) · Cited by 7,680
Caruana, R. (1997) · Machine Learning · doi:10.1023/a:1007379606734 · Cited by 6,636
(2009) · Cited by 6,153
Combining labeled and unlabeled data with co-training
Blum, A., & Mitchell, T. (1998) · Proceedings of the eleventh annual conference on Computational learning theory · doi:10.1145/279943.279962 · Cited by 5,495
Learning Deep Architectures for AI
Bengio, Y. (2009) · now publishers, Inc. eBooks · doi:10.1561/9781601982957 · Cited by 4,941
(2009) · Minds at UW (University of Wisconsin) · Cited by 4,791
Statistical Analysis With Missing Data
Ghosh, S. (1988) · Technometrics · doi:10.1080/00401706.1988.10488446 · Cited by 4,576
Semi-Supervised Learning Literature Survey
Zhu, X. (2005) · Minds at UW (University of Wisconsin) · Cited by 4,505
Large Scale Distributed Deep Networks
Dean, J. B., Corrado, G. S., Monga, R., Chen, K., Devin, M., Mao, M., Ranzato, M., Senior, A., Tucker, P. A., Yang, K., et al. (2012) · neural information processing systems · Cited by 4,489
Learning with Local and Global Consistency
Zhou, D., Bousquet, O., Lal, T. N., Weston, J., & Schölkopf, B. (2003) · MPG.PuRe (Max Planck Society) · Cited by 3,864
(2006) · Cited by 3,624
Semi-supervised learning using Gaussian fields and harmonic functions
Zhu, X., Zoubin Ghahramani, A., & Lafferty, J. (2003) · Cited by 3,509
Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition
Dahl, G. E., Yu, D., Deng, L., & Acero, A. (2012) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2011.2134090 · Cited by 3,011 · cited 1×
“(It turns out that such context dependency also plays a critical role in the more recent advance of ASR in the area of discriminative-based deep learning [20], to be discussed in Section VII-A.)”
BackgroundSelf-citationBilmes, J. A. (1998) · CTIT technical reports series · Cited by 2,853
Statistical Inference for Probabilistic Functions of Finite State Markov Chains
Baum, L. E., & Petrie, T. (1966) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177699147 · Cited by 2,775
(2011) · Cited by 2,752
Text classification from labeled and unlabeled documents using EM
Nigam, K., Mccallum, A. K., Thrun, S., & Mitchell, T. (2000) · Machine Learning · doi:10.1023/a:1007692713085 · Cited by 2,736 · cited 1×
“One way of solving this optimization problem is applying the EM algorithm or its variations to unlabeled data [143], [144].”
MethodTransductive Inference for Text Classification using Support Vector Machines
Joachims, T. (1999) · Cited by 2,723
Unsupervised word sense disambiguation rivaling supervised methods
Yarowsky, D. (1995) · Proceedings of the 33rd annual meeting on Association for Computational Linguistics - · doi:10.3115/981658.981684 · Cited by 2,385
On the algorithmic implementation of multiclass kernel-based vector machines
Crammer, K., & Singer, Y. (2002) · Journal of Machine Learning Research · Cited by 2,311
Semi-supervised Learning by Entropy Minimization
Grandvalet, Y., & Bengio, Y. (2004) · neural information processing systems · Cited by 2,268
Leggetter, C. J., & Woodland, P. C. (1995) · Computer Speech & Language · doi:10.1006/csla.1995.0010 · Cited by 2,192
Deep Neural Networks for Acoustic Modeling in Speech Recognition
Hinton, G. E., Deng, L., Yu, D., Dahl, G. E., Mohamed, A., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T. N., et al. (2012) · IEEE Signal Processing Magazine · Cited by 2,115
Self-citationMaximum a posteriori estimation for multivariate Gaussian mixture observations of Markov chains
Gauvain, J.-L., & Lee, C.-H. (1994) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/89.279278 · Cited by 2,108
Acoustic Modeling Using Deep Belief Networks
Mohamed, A.-R., Dahl, G. E., & Hinton, G. (2012) · IEEE Transactions on Audio, Speech, and Language Processing · doi:10.1109/tasl.2011.2109382 · Cited by 1,727
Seung, H. S., Opper, M., & Sompolinsky, H. (1992) · Proceedings of the fifth annual workshop on Computational learning theory · doi:10.1145/130385.130417 · Cited by 1,651
Learning from labeled and unlabeled data with label propagation
Zhu, X., & Ghahramani, Z. (2002) · Cited by 1,534
A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data
Ando, R. K., & Zhang, T. (2005) · doi:10.5555/1046920.1194905 · Cited by 1,311
Exploiting Generative Models in Discriminative Classifiers
Jaakkola, T., & Haussler, D. H. (1998) · Cited by 1,271
Support vector machine learning for interdependent and structured output spaces
Tsochantaridis, I., Hofmann, T., Joachims, T., & Altun, Y. (2004) · Twenty-first international conference on Machine learning - ICML '04 · doi:10.1145/1015330.1015341 · Cited by 1,243
(1998) · Springer eBooks · doi:10.1007/978-1-4615-5529-2 · Cited by 1,235
Deng, L., Hinton, G., & Kingsbury, B. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6639344 · Cited by 1,178
Self-citationHeterogeneous Uncertainty Sampling for Supervised Learning
Lewis, D. D., & Catlett, J. (1994) · Machine Learning Proceedings 1994 · doi:10.1016/b978-1-55860-335-6.50026-x · Cited by 1,150
Selective Sampling Using the Query by Committee Algorithm
Freund, Y., Seung, H. S., Shamir, E., & Tishby, N. (1997) · Machine Learning · doi:10.1023/a:1007330508534 · Cited by 1,097
Deep, Big, Simple Neural Nets for Handwritten Digit Recognition
Cireşan, D. C., Meier, U., Gambardella, L. M., & Schmidhuber, J. (2010) · Neural Computation · doi:10.1162/neco_a_00052 · Cited by 1,078
Convexity, Classification, and Risk Bounds
Bartlett, P. L., Jordan, M. I., & McAuliffe, J. D. (2006) · Journal of the American Statistical Association · doi:10.1198/016214505000000907 · Cited by 1,037
Analyzing the effectiveness and applicability of co-training
Nigam, K., & Ghani, R. (2000) · Proceedings of the ninth international conference on Information and knowledge management · doi:10.1145/354756.354805 · Cited by 1,005
An analysis of active learning strategies for sequence labeling tasks
Settles, B., & Craven, M. (2008) · Proceedings of the Conference on Empirical Methods in Natural Language Processing - EMNLP '08 · doi:10.3115/1613715.1613855 · Cited by 972
Learning from Labeled and Unlabeled Data using Graph Mincuts
Blum, A. L., & Chawla, S. (2018) · doi:10.1184/r1/6606860 · Cited by 928
Continuous speech recognition by statistical methods
Jelinek, F. (1976) · Proceedings of the IEEE · doi:10.1109/proc.1976.10159 · Cited by 920
Applying Convolutional Neural Networks concepts to hybrid NN-HMM model for speech recognition
Abdel-Hamid, O., Mohamed, A.-R., Jiang, H., & Penn, G. (2012) · 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2012.6288864 · Cited by 876
On the algorithmic implementation of multiclass kernel-based vector machines
CrammerKoby, & SingerYoram (2002) · Journal of Machine Learning Research · doi:10.5555/944790.944813 · Cited by 874
Semi-Supervised Support Vector Machines
Bennett, K. P., & Demiriz, A. (1998) · Cited by 826
Domain Adaptation for Statistical Classifiers
Daume III, H., & Marcu, D. (2006) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1872 · Cited by 816
Learning Multiple Tasks with Kernel Methods
Evgeniou, T., Micchelli, C. A., & Pontil, M. (2005) · doi:10.5555/1046920.1088693 · Cited by 799
Support vector machines for multi-class pattern recognition.
Weston, J., & Watkins, C. (1999) · Cited by 796
A Model of Inductive Bias Learning
Baxter, J. (2000) · Cited by 730
Active learning using pre-clustering
Nguyen, H. T., & Smeulders, A. (2004) · Twenty-first international conference on Machine learning - ICML '04 · doi:10.1145/1015330.1015349 · Cited by 658
Transductive learning via spectral graph partitioning
Joachims, T. (2003) · Cited by 633
Minimum Phone Error and I-smoothing for improved discriminative training
Povey, D., & Woodland, P. C. (2002) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2002.5743665 · Cited by 622
Minimum classification error rate methods for speech recognition
Juang, B.-H., Hou, W., & Lee, C.-H. (1997) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/89.568732 · Cited by 618
From HMM's to segment models: a unified view of stochastic modeling for speech recognition
Ostendorf, M., Digalakis, V. V., & Kimball, O. A. (1996) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/89.536930 · Cited by 587
Connectionist Speech Recognition
Bourlard, H. A., & Morgan, N. (1994) · Springer eBooks · doi:10.1007/978-1-4615-3210-1 · Cited by 585
A model of inductive bias learning
Baxter, J. (2000) · Journal of Artificial Intelligence Research · doi:10.1613/jair.731 · Cited by 582 · cited 2×
“lated [197], [220]–[223].”
Partially labeled classification with Markov Random Walks
Szummer, M., & Jaakkola, T. (2001) · Cited by 558
Active Hidden Markov Models for Information Extraction
Scheffer, T., Decomain, C., & Wrobel, S. (2001) · Lecture Notes in Computer Science · doi:10.1007/3-540-44816-0_31 · Cited by 534
Collobert, R., Sinz, F. H., Weston, J., & Bottou, L. (2006) · Journal of Machine Learning Research · doi:10.5555/1248547.1248609 · Cited by 527
Incremental learning with support vector machines
Ruping, S. (2002) · Proceedings 2001 IEEE International Conference on Data Mining · doi:10.1109/icdm.2001.989589 · Cited by 524
Domain Adaptation: Learning Bounds and Algorithms
Mansour, Y., Mohri, M., & Rostamizadeh, A. (2009) · arXiv · doi:10.48550/arxiv.0902.3430 · Cited by 504
Committee-Based Sampling For Training Probabilistic Classifiers
Dagan, I., & Engelson, S. P. (1995) · Machine Learning Proceedings 1995 · doi:10.1016/b978-1-55860-377-6.50027-x · Cited by 499
Rapid speaker adaptation in eigenvoice space
Kuhn, R., Junqua, J.-C., Nguyen, P., & Niedzielski, N. (2000) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/89.876308 · Cited by 488
Multi-task learning for classification with dirichlet process priors
Xue, Y., Liao, X., Carin, L., Krishnapuram, B., & Bartlett, P. (2007) · Cited by 485
A compact model for speaker-adaptive training
Anastasakos, T., McDonough, J., Schwartz, R., & Makhoul, J. (2002) · Proceeding of Fourth International Conference on Spoken Language Processing. ICSLP '96 · doi:10.1109/icslp.1996.607807 · Cited by 476
McAllester, D. A. (1998) · Proceedings of the eleventh annual conference on Computational learning theory · doi:10.1145/279943.279989 · Cited by 464
A Bayesian/Information Theoretic Model of Learning to Learn via Multiple Task Sampling
Baxter, J. (1997) · Machine Learning · doi:10.1023/a:1007327622663 · Cited by 462
Robust continuous speech recognition using parallel model combination
Gales, M., & Young, S. J. (1996) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/89.536929 · Cited by 439
Exploiting Task Relatedness for Multiple Task Learning
Ben-David, S., & Schuller, R. (2003) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-45167-9_41 · Cited by 425
Mean and variance adaptation within the MLLR framework
Gales, M. J. F., & Woodland, P. C. (1996) · Computer Speech & Language · doi:10.1006/csla.1996.0013 · Cited by 393
Learning Bounds for Domain Adaptation
Blitzer, J. C., Crammer, K., Kulesza, A., Pereira, F. M. B., & Wortman, J. R. (2007) · Cited by 393
Learning Gaussian processes from multiple tasks
Yu, K., Tresp, V., & Schwaighofer, A. (2005) · Proceedings of the 22nd international conference on Machine learning - ICML '05 · doi:10.1145/1102351.1102479 · Cited by 380
Binary coding of speech spectrograms using a deep auto-encoder
Deng, L., Seltzer, M. L., Yu, D., Acero, A., Mohamed, A.-R., & Hinton, G. (2010) · Interspeech 2010 · doi:10.21437/interspeech.2010-487 · Cited by 375
Self-citationSupport Vector Machine Active Learning with Application sto Text Classification
Tong, S., & Koller, D. (2000) · International Conference on Machine Learning · Cited by 369
Translated Learning: Transfer Learning across Different Feature Spaces
Dai, W., Chen, Y., Xue, G.-R., Yang, Q., & Yu, Y. H. (2008) · neural information processing systems · Cited by 340
Minimum Bayes-Risk Decoding for Statistical Machine Translation
Kumar, S., Byrne, B., et al. (2004) · Cited by 328
Exemplar-Based Sparse Representations for Noise Robust Automatic Speech Recognition
Gemmeke, J. F., Virtanen, T., & Hurmalainen, A. (2011) · IEEE Transactions on Audio, Speech, and Language Processing · doi:10.1109/tasl.2011.2112350 · Cited by 328
Large scale discriminative training of hidden Markov models for speech recognition
Woodland, P. C., & Povey, D. (2002) · Computer Speech & Language · doi:10.1006/csla.2001.0182 · Cited by 306
Improving SVM accuracy by training on auxiliary data sources
Wu, P., & Dietterich, T. G. (2004) · Twenty-first international conference on Machine learning - ICML '04 · doi:10.1145/1015330.1015436 · Cited by 301
Hidden conditional random fields for phone classification
Gunawardana, A., Mahajan, M., Acero, A., & Platt, J. C. (2005) · Interspeech 2005 · doi:10.21437/interspeech.2005-126 · Cited by 299
A Mixture of Experts Classifier with Learning Based on Both Labelled and Unlabelled Data
Miller, D. J., & Uyar, H. S. (1996) · neural information processing systems · Cited by 292
Deep Belief Networks using discriminative features for phone recognition
Mohamed, A.-R., Sainath, T. N., Dahl, G., Ramabhadran, B., Hinton, G. E., & Picheny, M. A. (2011) · 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2011.5947494 · Cited by 286
HMM adaptation using vector taylor series for noisy speech recognition
Acero, A., Deng, L., Kristjansson, T., & Zhang, J. (2000) · 6th International Conference on Spoken Language Processing (ICSLP 2000) · doi:10.21437/icslp.2000-672 · Cited by 278
Self-citationLightly supervised and unsupervised acoustic model training
Lamel, L., Gauvain, J.-L., & Adda, G. (2002) · Computer Speech & Language · doi:10.1006/csla.2001.0186 · Cited by 258
Cluster adaptive training of hidden Markov models
Gales, M. J. F. (2000) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/89.848223 · Cited by 255
Semisupervised alignment of manifolds.
Ham, J., Lee, D. D., & Saul, L. K. (2005) · International Conference on Artificial Intelligence and Statistics · Cited by 252
Maximum Entropy Discrimination
Jebara, T. (2004) · Springer eBooks · doi:10.1007/978-1-4419-9011-2_3 · Cited by 252
Maximum-Likelihood Estimation for Mixture Multivariate Stochastic Observations of Markov Chains
Juang, B.-H. (1985) · AT&T Technical Journal · doi:10.1002/j.1538-7305.1985.tb00273.x · Cited by 236
Maximum likelihood estimation for multivariate mixture observations of markov chains (Corresp.)
Juang, B.-H., Levinson, S., & Sondhi, M. (1986) · IEEE Transactions on Information Theory · doi:10.1109/tit.1986.1057145 · Cited by 235
Developments and directions in speech recognition and understanding, Part 1 [DSP Education]
Baker, J., Deng, L., Glass, J., Khudanpur, S., Lee, C.-H., Morgan, N., & O'Shaughnessy, D. (2009) · IEEE Signal Processing Magazine · doi:10.1109/msp.2009.932166 · Cited by 228
Self-citationConnectionist probability estimators in HMM speech recognition
Renals, S., Morgan, N., Bourlard, H., Cohen, M., & Franco, H. (1994) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/89.260359 · Cited by 228
Large-vocabulary speech recognition under adverse acoustic environments
Deng, L., Acero, A., Plumpe, M., & Huang, X. (2000) · 6th International Conference on Spoken Language Processing (ICSLP 2000) · doi:10.21437/icslp.2000-657 · Cited by 222
Self-citationManifold alignment without correspondence
Wang, C., & Mahadevan, S. (2009) · International Joint Conference on Artificial Intelligence · Cited by 221
Kingsbury, B., Sainath, T. N., & Soltau, H. (2012) · Interspeech 2012 · doi:10.21437/interspeech.2012-3 · Cited by 220
A Spectral Regularization Framework for Multi-Task Structure Learning
Argyriou, A. A., Pontil, M., Ying, Y., & Micchelli, C. A. (2007) · Cited by 215
Learning Internal Representations
Baxter, J. (1995) · Cited by 211
Investigation of full-sequence training of deep belief networks for speech recognition
Mohamed, A.-R., Yu, D., & Deng, L. (2010) · Interspeech 2010 · doi:10.21437/interspeech.2010-304 · Cited by 210
Self-citationScalable stacking and learning for building deep architectures
Deng, L., Yu, D., & Platt, J. (2012) · 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2012.6288333 · Cited by 210
Self-citationYu, D., Deng, L., & Dahl, G. E. (2010) · neural information processing systems · Cited by 189
Self-citationSpeech production knowledge in automatic speech recognition
King, S., Frankel, J., Livescu, K., McDermott, E., Richmond, K., & Wester, M. (2007) · The Journal of the Acoustical Society of America · doi:10.1121/1.2404622 · Cited by 184
Template-Based Continuous Speech Recognition
Wachter, M. D., Matton, M., Demuynck, K., Wambacq, P., Cools, R. F. A., & Compernolle, D. V. (2007) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2007.894524 · Cited by 184
Auto-encoder bottleneck features using deep belief networks
Sainath, T. N., Kingsbury, B., & Ramabhadran, B. (2012) · 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2012.6288833 · Cited by 182
Adaptation of maximum entropy capitalizer: Little data can help a lot
Chelba, C., & Acero, A. (2006) · Computer Speech & Language · doi:10.1016/j.csl.2005.05.005 · Cited by 180
A continuation method for semi-supervised SVMs
Chapelle, O., Chi, M., & Zien, A. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143868 · Cited by 178
Large vocabulary continuous speech recognition with context-dependent DBN-HMMS
Dahl, G. E., Yu, D., Deng, L., & Acero, A. (2011) · 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2011.5947401 · Cited by 178
Self-citationLarge Margin Hidden Markov Models for Automatic Speech Recognition
Sha, F., & Saul, L. K. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0161 · Cited by 177
Speaker-adaptation for hybrid HMM-ANN continuous speech recognition system
Neto, J., Almeida, L., Hochberg, M., Martins, C., Nunes, L., Renals, S., & Robinson, T. (1995) · 4th European Conference on Speech Communication and Technology (Eurospeech 1995) · doi:10.21437/eurospeech.1995-411 · Cited by 175
BILMES, J. A. (2006) · IEICE Transactions on Information and Systems · doi:10.1093/ietisy/e89-d.3.869 · Cited by 172 · cited 1×
“In ASR, the most common generative learning approach is based on Gaussian-Mixture-Model based Hidden Markov models, or GMM-HMM; e.g., [15]–[18].”
MethodMinimum Bayes-risk automatic speech recognition
Goel, V., & Byrne, W. J. (2000) · Computer Speech & Language · doi:10.1006/csla.2000.0138 · Cited by 169
Active learning: theory and applications to automatic speech recognition
Riccardi, G., & Hakkani-Tur, D. (2005) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/tsa.2005.848882 · Cited by 167
Discriminative Training for Large-Vocabulary Speech Recognition Using Minimum Classification Error
McDermott, E., Hazen, T. J., Le Roux, J., Nakamura, A., & Katagiri, S. (2007) · IEEE Transactions on Audio, Speech and Language Processing · doi:10.1109/tasl.2006.876778 · Cited by 166
Hutchinson, B., Deng, L., & Yu, D. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.268 · Cited by 154
Self-citationDeng, L., Aksmanovic, M., Sun, X., & Wu, C. (1994) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/89.326610 · Cited by 152
Self-citationDeng, L., & Sun, D. X. (1994) · The Journal of the Acoustical Society of America · doi:10.1121/1.409839 · Cited by 144
Self-citationDiscriminative learning in sequential pattern recognition
He, X., Deng, L., & Chou, W. (2008) · IEEE Signal Processing Magazine · doi:10.1109/msp.2008.926652 · Cited by 143
Self-citationUnsupervised language model adaptation
Bacchiani, M., & Roark, B. (2003) · 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03). · doi:10.1109/icassp.2003.1198758 · Cited by 138
Unsupervised training of acoustic models for large vocabulary continuous speech recognition
Wessel, F., & Ney, H. (2004) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/tsa.2004.838537 · Cited by 137
Exploiting sparseness in deep neural networks for large vocabulary speech recognition
Yu, D., Seide, F., Li, G., & Deng, L. (2012) · 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2012.6288897 · Cited by 137
Self-citationHybrid SVM/HMM architectures for speech recognition
Ganapathiraju, A., Hamaker, J., & Picone, J. (2000) · International Conference on Spoken Language Processing (ICSLP) · doi:10.21437/icslp.2000-859 · Cited by 137
On adaptive decision rules and decision parameter adaptation for automatic speech recognition
Lee, C., & Huo, Q. (2000) · Proceedings of the IEEE · doi:10.1109/5.880082 · Cited by 130
Deep and Wide: Multiple Layers in Automatic Speech Recognition
Morgan, N. (2012) · IEEE Transactions on Audio, Speech, and Language Processing · doi:10.1109/tasl.2011.2116010 · Cited by 128
Semi-supervised conditional random fields for improved sequence segmentation and labeling
Jiao, F., Wang, S., Lee, C., Greiner, R., & Schuurmans, D. (2006) · International Conference on Computational Linguistics · doi:10.3115/1220175.1220202 · Cited by 127
Large margin hidden Markov models for speech recognition
Jiang, H., Li, X., & Liu, C. (2006) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2006.879805 · Cited by 126
Self-citationStoyanov, V., Ropson, A., & Eisner, J. M. (2011) · International Conference on Artificial Intelligence and Statistics · Cited by 124
Maximum a posteriori linear regression for hidden Markov model adaptation
Chesta, C., Siohan, O., & Lee, C. (1999) · European Conference on Speech Communication and Technology (Eurospeech) · doi:10.21437/eurospeech.1999-56 · Cited by 122
Large Margin Gaussian Mixture Modeling for Phonetic Classification and Recognition
Sha, F., & Saul, L. K. (2006) · 2006 IEEE International Conference on Acoustics Speed and Signal Processing Proceedings · doi:10.1109/icassp.2006.1660008 · Cited by 122
Adaptation of Maximum Entropy Capitalizer: Little Data Can Help a Lo.
Chelba, C. I., & Acero, A. (2004) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · Cited by 121
Exploiting deep neural networks for detection-based speech recognition
Siniscalchi, S. M., Yu, D., Deng, L., & Lee, C. (2012) · Neurocomputing · doi:10.1016/j.neucom.2012.11.008 · Cited by 120
Self-citationGeneralized Expectation Criteria for Semi-Supervised Learning of Conditional Random Fields
Mann, G., & McCallum, A. (2008) · Scholarworks (University of Massachusetts Amherst) · Cited by 120
Graphical model architectures for speech recognition
Bilmes, J. A., & Bartels, C. D. (2005) · IEEE Signal Processing Magazine · doi:10.1109/msp.2005.1511827 · Cited by 115
The Deep Tensor Neural Network With Applications to Large Vocabulary Speech Recognition
Yu, D., Deng, L., & Seide, F. (2013) · IEEE Transactions on Audio, Speech, and Language Processing · doi:10.1109/tasl.2012.2227738 · Cited by 113
Self-citationNatural-quality background noise coding using residual substitution
El-Maleh, K., & Kabal, P. (1999) · European Conference on Speech Communication and Technology (Eurospeech) · doi:10.21437/eurospeech.1999-599 · Cited by 112
Landmark-Based Speech Recognition: Report of the 2004 Johns Hopkins Summer Workshop
Hasegawa‐Johnson, M., Baker, J. M., Borys, S., Chen, K., Coogan, E., Greenberg, S. S., Juneja, A., Kirchhoff, K., Livescu, K., Mohan, S., et al. (2006) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2005.1415088 · Cited by 111
A study on multilingual acoustic modeling for large vocabulary ASR
Lin, H. Z., Deng, L., Yu, D., Gong, Y., Acero, A., & Lee, C. (2009) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2009.4960588 · Cited by 110
Self-citationDeng, L. (1992) · Signal Processing · doi:10.1016/0165-1684(92)90112-a · Cited by 109
Self-citationComparison of discriminative training criteria and optimization methods for speech recognition
Schlüter, R., Macherey, W., Müller, B. P., & Ney, H. (2001) · Speech Communication · doi:10.1016/s0167-6393(00)00035-2 · Cited by 108
Bayesian Multitask Learning with Latent HierarchiesPreprint
Daumé, H. (2009) · arXiv (Cornell University) · doi:10.48550/arxiv.0907.0783 · Cited by 104
Continuous speech recognition by connectionist statistical methods
Bourlard, H., & Morgan, N. (1993) · IEEE Transactions on Neural Networks · doi:10.1109/72.286885 · Cited by 102
Frey, B. J., Deng, L., Acero, A., & Kristjansson, T. T. (2001) · European Conference on Speech Communication and Technology (Eurospeech) · doi:10.21437/eurospeech.2001-273 · Cited by 96
Self-citationPushing the envelope - aside [speech recognition
Morgan, N., Zhu, Q., Stolcke, A., Sonmez, K., Sivadas, S., Shinozaki, T., Ostendorf, M., Jain, P., Hermansky, H., Ellis, D., et al. (2005) · IEEE Signal Processing Magazine · doi:10.1109/msp.2005.1511826 · Cited by 95
Empirical Bayes for Learning to Learn
Heskes, T. (2000) · Radboud Repository (Radboud University) · doi:10.5555/645529.658133 · Cited by 94
Li, J., Deng, L., Yu, D., Gong, Y., & Acero, A. (2009) · Computer Speech & Language · doi:10.1016/j.csl.2009.02.001 · Cited by 93
Self-citationMulti-conditional learning: generative/discriminative training for clustering and classification
McCallum, A., Pal, C., Druck, G., & Wang, X. (2006) · ScholarWorks@UMassAmherst (University of Massachusetts Amherst) · Cited by 92
Graphical models and automatic speech recognition
Bilmes, J. (2002) · The Journal of the Acoustical Society of America · doi:10.1121/1.4779134 · Cited by 90
Deng, L., & Ma, J. (2000) · The Journal of the Acoustical Society of America · doi:10.1121/1.1315288 · Cited by 89
Self-citationBayesian Nonparametric Methods for Learning Markov Switching Processes
Fox, E. B., Sudderth, E. B., Jordan, M. I., & Willsky, A. S. (2010) · IEEE Signal Processing Magazine · doi:10.1109/msp.2010.937999 · Cited by 88
Deng, L. (1998) · Speech Communication · doi:10.1016/s0167-6393(98)00023-5 · Cited by 88
Self-citationProduction models as a structural basis for automatic speech recognition
Deng, L., Ramsay, G., & Sun, D. (1997) · Speech Communication · doi:10.1016/s0167-6393(97)00018-6 · Cited by 84
Self-citationLi, D., Yu, D., & Acero, A. (2006) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2006.878265 · Cited by 84
Probabilistic-trajectory segmental HMMs
Holmes, W. J., & Russell, M. J. (1999) · Computer Speech & Language · doi:10.1006/csla.1998.0048 · Cited by 83
Bayesian Multitask Learning with Latent HierarchiesPreprint
Daumé, H. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1408.2032 · Cited by 83
Speech Recognition Using Augmented Conditional Random Fields
Hifny, Y., & Renals, S. (2009) · IEEE Transactions on Audio, Speech, and Language Processing · doi:10.1109/tasl.2008.2010286 · Cited by 81
Analysis of MLP-Based Hierarchical Phoneme Posterior Probability Estimator
Pinto, J., Garimella, S., Magimai.-Doss, M., Hynek Hermansky, B., & Bourlard, H. A. (2010) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2010.2045943 · Cited by 80
Noise Adaptive Training for Robust Automatic Speech Recognition
Kalinli, O., Seltzer, M. L., Droppo, J., & Acero, A. (2010) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2010.2040522 · Cited by 80
An application of discriminative feature extraction to filter-bank-based speech recognition
Biem, A., Katagiri, S., McDermott, E., & Juang, B.-H. F. (2001) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/89.902277 · Cited by 78
Unsupervised training of a speech recognizer: recent experiments
Kemp, T., & Waibel, A. (1999) · European Conference on Speech Communication and Technology (Eurospeech) · doi:10.21437/eurospeech.1999-682 · Cited by 77
Speaker clustering and transformation for speaker adaptation in speech recognition systems
Padmanabhan, M., Bahl, L. R., Nahamoo, D., & Picheny, M. (1998) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/89.650313 · Cited by 76
Exemplar-Based Sparse Representation Features: From TIMIT to LVCSR
Sainath, T. N., Ramabhadran, B., Picheny, M., Nahamoo, D., & Kanevsky, D. (2011) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2011.2155060 · Cited by 75
Graphical Models for Time-Series
Barber, D., & Cemgil, A. T. (2010) · IEEE Signal Processing Magazine · doi:10.1109/msp.2010.938028 · Cited by 74
Chengalvarayan, R., & Deng, L. (1997) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/89.568731 · Cited by 73
Self-citationMultiple Source Adaptation and the Renyi Divergence
Mansour, Y., Mohri, M., & Rostamizadeh, A. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1205.2628 · Cited by 71
Sun, J., & Deng, L. (2002) · The Journal of the Acoustical Society of America · doi:10.1121/1.1420380 · Cited by 71
Self-citationLee, C.-H. (2004) · Interspeech (USB) · Cited by 70
Connectionist speaker normalization and adaptation
Abrash, V., Franco, H., Sankar, A., & Cohen, M. (1995) · 4th European Conference on Speech Communication and Technology (Eurospeech 1995) · doi:10.21437/eurospeech.1995-414 · Cited by 69
Learning internal representations
Baxter, J. (1995) · Proceedings of the eighth annual conference on Computational learning theory - COLT '95 · doi:10.1145/225298.225336 · Cited by 69
Switching Linear Dynamical Systems for Noise Robust Speech Recognition
Mesot, B., & Barber, D. (2007) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2007.901312 · Cited by 68
Lin, H. Z., & Bilmes, J. (2009) · Interspeech (USB) · doi:10.21437/interspeech.2009-730 · Cited by 67
Discriminative versus generative parameter and structure learning of Bayesian network classifiers
Pernkopf, F., & Bilmes, J. (2005) · International Conference on Machine Learning · doi:10.1145/1102351.1102434 · Cited by 67
Noise robust speech recognition with a switching linear dynamic model
Droppo, J., & Acero, A. (2004) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2004.1326145 · Cited by 67
Interactive Submodular Set CoverPreprint
Guillory, A., & Bilmes, J. (2010) · arXiv (Cornell University) · doi:10.48550/arxiv.1002.3345 · Cited by 66
Yu, D., Siniscalchi, S. M., Deng, L., & Lee, C. (2012) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2012.6288837 · Cited by 66
Self-citationSparse Multilayer Perceptron for Phoneme Recognition
Sivaram, G. S. V. S., & Hermansky, H. (2012) · IEEE Transactions on Audio, Speech, and Language Processing · doi:10.1109/tasl.2011.2129510 · Cited by 65
Dirichlet Class Language Models for Speech Recognition
Chien, J., & Chueh, C.-H. (2010) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2010.2050717 · Cited by 64
Maximum Expected BLEU Training of Phrase and Lexicon Translation Models
He, X., & Deng, L. (2012) · Cited by 60
Self-citationLearning with Recursive Perceptual Representations
Vinyals, O., Jia, Y., Deng, L., & Darrell, T. (2012) · Cited by 60
Self-citationUpdated MINDS report on speech recognition and understanding, Part 2 [DSP Education]
Baker, J. M., Deng, L., Khudanpur, S. P., Lee, C., Glass, J. R., Morgan, N., & O’Shaughnessy, D. D. (2009) · IEEE Signal Processing Magazine · doi:10.1109/msp.2009.932707 · Cited by 59
Self-citationAn introduction of trajectory model into HMM-based speech synthesis.
Zen, H., Tokuda, K., & Kitamura, T. (2004) · Cited by 59
An Overview of Modern Speech Recognition.
Huang, X., & Deng, L. (2010) · Cited by 58
Self-citationEntropic Graph Regularization in Non-Parametric Semi-Supervised Classification
Subramanya, A., & Bilmes, J. (2009) · Cited by 57
Combining phonetic attributes using conditional random fields
Morris, J., & Fosler‐Lussier, E. (2006) · Interspeech (USB) · doi:10.21437/interspeech.2006-218 · Cited by 54
Bridle, J. S. (1998) · Medical Entomology and Zoology · Cited by 53
Computational Models for Speech Production
Deng, L. (1999) · Springer eBooks · doi:10.1007/978-3-642-60087-6_20 · Cited by 51
Self-citationSegmental Minimum Bayes-Risk Decoding for Automatic Speech Recognition
Goel, V., Kumar, S., & Byrne, B. (2004) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/tsa.2004.825678 · Cited by 51
Deng, L., Kenny, P. J., Lennig, M., Gupta, V., Seitz, F., & Mermelstein, P. G. (1991) · IEEE Transactions on Signal Processing · doi:10.1109/78.134406 · Cited by 51
Self-citationProduct of Experts for Statistical Parametric Speech Synthesis
Zen, H., Gales, M., Nankaku, Y., & Tokuda, K. (2011) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2011.2165280 · Cited by 50
Exploiting comparable corpora and bilingual dictionaries for cross-language text categorization
Gliozzo, A., & Strapparava, C. (2006) · International Conference on Computational Linguistics · doi:10.3115/1220175.1220245 · Cited by 49
Mathematical Foundations of Speech and Language Processing
Johnson, M., Khudanpur, S., Mari Ostendorf, E., & Rosenfeld, R. (2004) · The IMA volumes in mathematics and its applications · doi:10.1007/978-1-4419-9017-4 · Cited by 47
Buried Markov models: a graphical-modeling approach to automatic speech recognition
Bilmes, J. (2003) · Computer Speech & Language · doi:10.1016/s0885-2308(03)00010-x · Cited by 47
Hutchinson, B., Deng, L., & Yu, D. (2012) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2012.6288994 · Cited by 46
Self-citationSWITCHING DYNAMIC SYSTEM MODELS FOR SPEECH ARTICULATION AND ACOUSTICS
Denc, L. (2004) · Cited by 43
Large vocabulary word recognition using context-dependent allophonic hidden Markov models
Deng, L., Lennig, M., Seitz, F., & Mermelstein, P. (1990) · Computer Speech & Language · doi:10.1016/0885-2308(90)90015-x · Cited by 40
Self-citationBilmes, J. (2010) · IEEE Signal Processing Magazine · doi:10.1109/msp.2010.938078 · Cited by 40
Cited by
Deep Learning: Methods and Applications
Deng, L., & Yu, D. (2014) · Foundations and Trends® in Signal Processing · doi:10.1561/2000000039 · Cited by 3,480
“, Chapter 28 in [264], and Reference [95], it may be natural to just classify deep learning techniques into deep discriminative models (e.”
BackgroundSelf-citationDeep learning for healthcare: review, opportunities and challenges
Miotto, R., Wang, F., Wang, S., Jiang, X., & Dudley, J. T. (2018) · Briefings in Bioinformatics · doi:10.1093/bib/bbx044 · Cited by 3,076
1D convolutional neural networks and applications: A survey
Kiranyaz, S., Avci, O., Abdeljaber, O., Ince, T., Gabbouj, M., & Inman, D. J. (2021) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2020.107398 · Cited by 2,835
Convolutional Neural Networks for Speech Recognition
Abdel-Hamid, O., Mohamed, A.-R., Jiang, H., Deng, L., Penn, G., & Yu, D. (2014) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2014.2339736 · Cited by 2,262
Self-citationBuilding End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models
Serban, I., Sordoni, A., Bengio, Y., Courville, A., & Pineau, J. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.9883 · Cited by 1,615
Metadata
- Authors
- Deng, L., & Li, X.
- Venue
- IEEE Transactions on Audio, Speech, and Language Processing
- Published
- 2013-05
- Volume
- 21
- Issue
- 5
- Pages
- 1060-1089
- ISSN
- 1558-7916 (ISSN-L)
- 1558-7924
- OpenAlex
- W2161742217
- Cited by
- 456
- References
- 229
- FWCI
- 29.82
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.