The DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.
Multilingual acoustic models using distributed deep neural networks
Abstract
Today's speech recognition technology is mature enough to be useful for many practical applications. In this context, it is of paramount importance to train accurate acoustic models for many languages within given resource constraints such as data, processing power, and time. Multilingual training has the potential to solve the data issue and close the performance gap between resource-rich and resource-scarce languages. Neural networks lend themselves naturally to parameter sharing across languages, and distributed implementations have made it feasible to train large networks. In this paper, we present experimental results for cross- and multi-lingual network training of eleven Romance languages on 10k hours of data in total. The average relative gains over the monolingual baselines are 4%/2% (data-scarce/data-rich languages) for cross- and 7%/2% for multi-lingual training. However, the additional gain from jointly training the languages on all data comes at an increased training time of roughly four weeks, compared to two weeks (monolingual) and one week (crosslingual).
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336
Hinton, G., Deng, L., Yu, D., Dahl, G., Mohamed, A.-R., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T., et al. (2012) · IEEE Signal Processing Magazine · doi:10.1109/msp.2012.2205597 · Cited by 10,260
Self-citation(2010) · Cited by 8,261
Caruana, R. (1997) · Machine Learning · doi:10.1023/a:1007379606734 · Cited by 6,636
Large Scale Distributed Deep Networks
Dean, J. B., Corrado, G. S., Monga, R., Chen, K., Devin, M., Mao, M., Ranzato, M., Senior, A., Tucker, P. A., Yang, K., et al. (2012) · neural information processing systems · Cited by 4,489
Self-citationContext-dependent pre-trained deep neural networks for large-vocabulary speech recognition
Dahl, G. E., Yu, D., Deng, L., & Acero, A. (2012) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2011.2134090 · Cited by 3,011 · cited 4×
“Recently, excellent results have been achieved using Deep Neural Networks (DNNs) for acoustic modeling [2, 3, 4, 5, 6].”
BackgroundMethodInfluentialBuilding high-level features using large scale unsupervised learning
Le, Q. V. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6639343 · Cited by 1,695 · cited 3×
“Section 3 summarizes the distributed DNN implementation from [11, 12].”
BackgroundMethodDeng, L., Hinton, G., & Kingsbury, B. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6639344 · Cited by 1,178
Caruana, R. (1998) · Cited by 1,110
Conversational speech transcription using context-dependent deep neural networks
Seide, F., Li, G., & Yu, D. (2011) · Interspeech 2011 · doi:10.21437/interspeech.2011-169 · Cited by 852
Building high-level features using large scale unsupervised learning
Ranzato, M., Monga, R., Devin, M., Chen, K., Corrado, G. S., Dean, J., Le, Q. V., & Ng, A. Y. (2012) · International Conference on Machine Learning · Cited by 823
Self-citationA Model of Inductive Bias Learning
Baxter, J. (2000) · Cited by 730
Cross-language knowledge transfer using multilingual deep neural network with shared hidden layers
Huang, J.-T., Li, J., Yu, D., Deng, L., & Gong, Y. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6639081 · Cited by 595
Stochastic Gradient Learning in Neural Networks
Bottou, L. (1991) · Cited by 584
A model of inductive bias learning
Baxter, J. (2000) · Journal of Artificial Intelligence Research · doi:10.1613/jair.731 · Cited by 582
(1995) · Cited by 520
Application of pretrained deep neural networks to large vocabulary speech recognition
Jaitly, N., Nguyen, P., Senior, A., & Vanhoucke, V. (2012) · Interspeech 2012 · doi:10.21437/interspeech.2012-10 · Cited by 243
Self-citation“Your Word is my Command”: Google Search by Voice: A Case Study
Schalkwyk, J., Beeferman, D., Beaufays, F., Byrne, B., Chelba, C., Cohen, M., Kamvar, M., & Strope, B. (2010) · Advances in Speech Recognition · doi:10.1007/978-1-4419-5951-5_4 · Cited by 232
Transfer learning for Latin and Chinese characters with Deep Neural Networks
Ciresan, D. C., Meier, U., & Schmidhuber, J. (2012) · The 2012 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2012.6252544 · Cited by 209
Building high-level features using large scale unsupervised learning
Le, Q. V., Ranzato, M., Monga, R., Devin, M., Chen, K., Corrado, G. S., Dean, J., & Ng, A. Y. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1112.6209 · Cited by 163
Self-citationMultilingual acoustic modeling for speech recognition based on subspace Gaussian Mixture Models
Burget, L., Schwarz, P., Agarwal, M., Akyazi, P., Feng, K., Ghoshal, A., Glembek, O., Goel, N., Karafiat, M., Povey, D., et al. (2010) · 2010 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2010.5495646 · Cited by 162
Conversational speech transcription using context-dependent deep neural networks
Yu, D., Seide, F. T. B., & Li, G. (2012) · International Conference on Machine Learning · Cited by 131
Unsupervised cross-lingual knowledge transfer in DNN-based LVCSR
Świętojański, P., Ghoshal, A. K., & Renals, S. J. (2012) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt.2012.6424230 · Cited by 116
Multilingual MLP features for low-resource LVCSR systems
Thomas, S., Ganapathy, S., & Hynek Hermansky, B. (2012) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2012.6288862 · Cited by 110
A study on multilingual acoustic modeling for large vocabulary ASR
Lin, H. Z., Deng, L., Yu, D., Gong, Y., Acero, A., & Lee, C. (2009) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2009.4960588 · Cited by 110
Cross-Domain and Cross-Language Portability of Acoustic Features Estimated by Multilayer Perceptrons
Stolcke, A., Grézl, F., Hwang, M.-Y., Lei, X., Morgan, N., & Vergyri, D. (2006) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2006.1660022 · Cited by 110
Multilingual and crosslingual speech recognition
Schultz, T., & Waibel, A. (1998) · doi:10.5445/ir/44598 · Cited by 86
Cross-lingual and multi-stream posterior features for low resource LVCSR systems
Thomas, S., Ganapathy, S., & Heřmanský, H. (2010) · Interspeech (USB) · doi:10.21437/interspeech.2010-295 · Cited by 67
Vu, N. T., Breiter, W., Metze, F., & Schultz, T. (2018) · Figshare · doi:10.1184/r1/6473039 · Cited by 35
Vu, N. T., Breiter, W., Metze, F., & Schultz, T. (2012) · Interspeech (USB) · doi:10.21437/interspeech.2012-12 · Cited by 24
Multitask Learning for Bayesian Neural Networks
Kova, K. (2005) · Cited by 6
Cited by
TensorFlow: A system for large-scale machine learning
Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.08695 · Cited by 11,822
Self-citationTensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., et al. (2016) · arXiv · doi:10.48550/arxiv.1603.04467 · Cited by 9,408
Self-citationDeep Learning: Methods and Applications
Deng, L. (2014) · now publishers, Inc. eBooks · doi:10.1561/9781601988157 · Cited by 1,759
A survey on deep learning for big data
Zhang, Q., Yang, L. T., Chen, Z., & Li, P. (2018) · Information Fusion · doi:10.1016/j.inffus.2017.10.006 · Cited by 1,226
Deng, L., Hinton, G., & Kingsbury, B. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6639344 · Cited by 1,178
Metadata
- Authors
- Heigold, G., Vanhoucke, V., Senior, A., Nguyen, P., Ranzato, M., Devin, M., & Dean, J.
- Venue
- 2013 IEEE International Conference on Acoustics, Speech and Signal Processing
- Publisher
- IEEE
- Published
- 2013-05
- Pages
- 8619-8623
- ISSN
- 1520-6149 (ISSN-L)
- OpenAlex
- W1978660892
- Cited by
- 273
- References
- 31
- FWCI
- 20.81
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.