The DOI matches its Crossref record; 22 of the 22 references it cites were found in a registry or an index.
Sequence-discriminative training of deep neural networks
Open accessAbstract
Sequence-discriminative training of deep neural networks (DNNs) is investigated on a 300 hour American English conversational telephone speech task. Different sequence-discriminative criteria ndash;- maximum mutual information (MMI), minimum phone error (MPE), state-level minimum Bayes risk (sMBR), and boosted MMI ndash;- are compared. Two different heuristics are investigated to improve the performance of the DNNs trained using sequence-based criteria ndash;- lattices are re-generated after the first iteration of training; and, for MMI and BMMI, the frames where the numerator and denominator hypotheses are disjoint are removed from the gradient computation. Starting from a competitive DNN baseline trained using cross-entropy, different sequence-discriminative criteria are shown to lower word error rates by 8-9% relative, on average. Little difference is noticed between the different sequence-based criteria that are investigated. The experiments are done using the open-source Kaldi toolkit, which makes it possible for the wider community to reproduce these results.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336
Kaldi Speech Recognition Toolkit
Povey, D. (2024) · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · doi:10.57702/jb3fvbn9 · Cited by 4,609
Self-citationLarge Scale Distributed Deep Networks
Dean, J. B., Corrado, G. S., Monga, R., Chen, K., Devin, M., Mao, M., Ranzato, M., Senior, A., Tucker, P. A., Yang, K., et al. (2012) · Cited by 3,199
Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition
Dahl, G. E., Yu, D., Deng, L., & Acero, A. (2012) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2011.2134090 · Cited by 3,011
Connectionist Speech Recognition: A Hybrid Approach
Bourlard, H. A., & Morgan, N. H. (1993) · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · Cited by 1,115
Maximum mutual information estimation of hidden Markov model parameters for speech recognition
Bahl, L., Brown, P., de Souza, P., & Mercer, R. (2005) · ICASSP '86. IEEE International Conference on Acoustics, Speech, and Signal Processing · doi:10.1109/icassp.1986.1169179 · Cited by 817
Learning Representations by Back-Propagating Errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (2002) · Cognitive Modeling · doi:10.7551/mitpress/1888.003.0013 · Cited by 759
Seide, F., Li, G., Chen, X., & Yu, D. (2011) · 2011 IEEE Workshop on Automatic Speech Recognition & Understanding · doi:10.1109/asru.2011.6163899 · Cited by 632
Semi-tied covariance matrices for hidden Markov models
Gales, M. J. F. (1999) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/89.759034 · Cited by 537
An application of recurrent nets to phone probability estimation
Robinson, A. J. (1994) · IEEE Transactions on Neural Networks · doi:10.1109/72.279192 · Cited by 441
Boosted MMI for model and feature-space discriminative training
Povey, D., Kanevsky, D., Kingsbury, B., Ramabhadran, B., Saon, G., & Visweswariah, K. (2008) · 2008 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2008.4518545 · Cited by 341
Self-citationApplication of pretrained deep neural networks to large vocabulary speech recognition
Jaitly, N., Nguyen, P., Senior, A., & Vanhoucke, V. (2012) · Interspeech 2012 · doi:10.21437/interspeech.2012-10 · Cited by 243
Lattice-based optimization of sequence classification criteria for neural-network acoustic modeling
Kingsbury, B. (2009) · 2009 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2009.4960445 · Cited by 232
Connectionist probability estimators in HMM speech recognition
Renals, S., Morgan, N., Bourlard, H., Cohen, M., & Franco, H. (1994) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/89.260359 · Cited by 228
Kingsbury, B., Sainath, T. N., & Soltau, H. (2012) · Interspeech 2012 · doi:10.21437/interspeech.2012-3 · Cited by 220
Making Deep Belief Networks effective for large vocabulary continuous speech recognition
Sainath, T. N., Kingsbury, B., Ramabhadran, B., Fousek, P., Novak, P., & Mohamed, A.-R. (2011) · 2011 IEEE Workshop on Automatic Speech Recognition & Understanding · doi:10.1109/asru.2011.6163900 · Cited by 189
MMIE training of large vocabulary recognition systems
Valtchev, V, Odell, J. J., Woodland, P. C., & Young, S. J. (1997) · Speech Communication · doi:10.1016/s0167-6393(97)00029-0 · Cited by 175
A novel loss function for the overall risk criterion based discriminative training of HMM models
Kaiser, J., Horvat, B., & Kačič, Z. (2000) · International Conference on Spoken Language Processing (ICSLP) · doi:10.21437/icslp.2000-412 · Cited by 75
Evaluation of Proposed Modifications to MPE for Large Scale Discriminative Training
Povey, D., & Kingsbury, B. E. D. (2007) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2007.366914 · Cited by 73
Self-citationKrogh, A., & Riis, S. K. (1999) · Neural Computation · doi:10.1162/089976699300016764 · Cited by 71
Sequential classification criteria for NNs in automatic speech recognition
Wang, G., & Sim, K. C. (2011) · Interspeech (USB) · doi:10.21437/interspeech.2011-170 · Cited by 21
Cited by
SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition
Park, D. S., Chan, W., Zhang, Y., Chiu, C.-C., Zoph, B., Cubuk, E. D., & Le, Q. V. (2019) · Interspeech 2019 · doi:10.21437/interspeech.2019-2680 · Cited by 3,558
Deep Learning: Methods and Applications
Deng, L., & Yu, D. (2014) · Foundations and Trends® in Signal Processing · doi:10.1561/2000000039 · Cited by 3,480
SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient
Yu, L., Zhang, W., Wang, J., & Yu, Y. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10804 · Cited by 2,285
Listen, attend and spell: A neural network for large vocabulary conversational speech recognition
Chan, W., Jaitly, N., Le, Q., & Vinyals, O. (2016) · 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2016.7472621 · Cited by 2,226
Deep Learning: Methods and Applications
Deng, L. (2014) · now publishers, Inc. eBooks · doi:10.1561/9781601988157 · Cited by 1,759
Metadata
- Authors
- Veselý, K., Ghoshal, A., Burget, L., & Povey, D.
- Venue
- Interspeech 2013
- Publisher
- ISCA
- Published
- 2013-08-25
- Pages
- 2345-2349
- ISSN
- 2308-457X (ISSN-L)
- OpenAlex
- W2131342762
- Cited by
- 657
- References
- 22
- FWCI
- 56.81
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.