Learning hidden unit contributions for unsupervised speaker adaptation of neural network acoustic models
Open accessAbstract
This paper proposes a simple yet effective model-based neural network speaker adaptation technique that learns speaker-specific hidden unit contributions given adaptation data, without requiring any form of speaker-adaptive training, or labelled adaptation data. An additional amplitude parameter is defined for each hidden unit; the amplitude parameters are tied for each speaker, and are learned using unsupervised adaptation. We conducted experiments on the TED talks data, as used in the International Workshop on Spoken Language Translation (IWSLT) evaluations. Our results indicate that the approach can reduce word error rates on standard IWSLT test sets by about 8-15% relative compared to unadapted systems, with a further reduction of 4-6% relative when combined with feature-space maximum likelihood linear regression (fMLLR). The approach can be employed in most existing feed-forward neural network architectures, and we report results using various hidden unit activation functions: sigmoid, maxout, and rectifying linear units (ReLU).
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Cites
(2010) · Cited by 14,134 · see citing works
(2010) · International Conference on Machine Learning · Cited by 13,995 · see citing works
Hinton, G., Deng, L., Yu, D., Dahl, G., Mohamed, A.-R., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T., et al. (2012) · IEEE Signal Processing Magazine · doi:10.1109/msp.2012.2205597 · Cited by 10,260 · see citing works
Kaldi Speech Recognition Toolkit
Povey, D. (2024) · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · doi:10.57702/jb3fvbn9 · Cited by 4,837 · see citing works
Front-End Factor Analysis for Speaker Verification
Dehak, N., Kenny, P. J., Dehak, R., Dumouchel, P., & Ouellet, P. (2011) · IEEE Transactions on Audio, Speech, and Language Processing · doi:10.1109/tasl.2010.2064307 · Cited by 3,549 · see citing works
Hierarchical models of object recognition in cortex
Riesenhuber, M., & Poggio, T. (1999) · Nature Neuroscience · doi:10.1038/14819 · Cited by 3,309 · see citing works
Catastrophic forgetting in connectionist networks
French, R. (1999) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(99)01294-2 · Cited by 2,311 · see citing works
Leggetter, C. J., & Woodland, P. C. (1995) · Computer Speech & Language · doi:10.1006/csla.1995.0010 · Cited by 2,210 · see citing works
Maximum likelihood linear transformations for HMM-based speech recognition
Gales, M. J. F. (1998) · Computer Speech & Language · doi:10.1006/csla.1998.0043 · Cited by 1,482 · see citing works
Connectionist Speech Recognition: A Hybrid Approach
Bourlard, H. A., & Morgan, N. H. (1993) · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · Cited by 1,115 · see citing works
Tandem connectionist feature extraction for conventional HMM systems
Heřmanský, H., Ellis, D. P. W., & Sharma, S. (2002) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2000.862024 · Cited by 697 · see citing works
An investigation of deep neural networks for noise robust speech recognition
Seltzer, M. L., Yu, D., & Wang, Y. (2013) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2013.6639100 · Cited by 658 · see citing works
Seide, F. T. B., Li, G., Xie, C., & Yu, D. (2011) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru.2011.6163899 · Cited by 641 · see citing works
Connectionist Speech Recognition
Bourlard, H. A., & Morgan, N. H. (1994) · Springer eBooks · doi:10.1007/978-1-4615-3210-1 · Cited by 587 · see citing works
Speaker adaptation of neural network acoustic models using i-vectors
Saon, G., Soltau, H., Nahamoo, D., & Picheny, M. (2013) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru.2013.6707705 · Cited by 586 · see citing works
Yu, D., Yao, K., Su, H., Li, G., & Seide, F. T. B. (2013) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2013.6639201 · Cited by 417 · see citing works
Probabilistic and Bottle-Neck Features for LVCSR of Meetings
Grézl, F., Karafiát, M., Kontar, S., & Černocký, J. (2007) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2007.367023 · Cited by 340 · see citing works
Pylearn2: a machine learning research libraryPreprint
Goodfellow, I., Warde-Farley, D., Lamblin, P., Dumoulin, V., Mirza, M., Pascanu, R., Bergstra, J., Bastien, F., & Bengio, Y. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1308.4214 · Cited by 245 · see citing works
Speaker adaptation of context dependent deep neural networks
Liao, H. (2013) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2013.6639212 · Cited by 233 · see citing works
Connectionist probability estimators in HMM speech recognition
Renals, S. J., Morgan, N., Bourlard, H., Cohen, M., & Franco, H. (1994) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/89.260359 · Cited by 228 · see citing works
Self-citationAbdel‐Hamid, O., & Jiang, H. (2013) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2013.6639211 · Cited by 224 · see citing works
Adaptation of context-dependent deep neural networks for automatic speech recognition
Yao, K., Yu, D., Seide, F. T. B., Su, H., Deng, L., & Gong, Y. (2012) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt.2012.6424251 · Cited by 194 · see citing works
Auto-encoder bottleneck features using deep belief networks
Sainath, T. N., Kingsbury, B. E. D., & Ramabhadran, B. (2012) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2012.6288833 · Cited by 185 · see citing works
Speaker-adaptation for hybrid HMM-ANN continuous speech recognition system
Neto, J. P. D. S., Almeida, L. B., Hochberg, M. M., et al. (1995) · European Conference on Speech Communication and Technology (Eurospeech) · doi:10.21437/eurospeech.1995-411 · Cited by 181 · see citing works
Xue, J., Li, J., Yu, D., Seltzer, M., & Gong, Y. (2014) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2014.6854828 · Cited by 162 · see citing works
Linear hidden transformations for adaptation of hybrid ANN/HMM models
Gemello, R., Mana, F., Scanzio, S., Laface, P., & Mori, R. D. (2007) · Speech Communication · doi:10.1016/j.specom.2006.11.005 · Cited by 147 · see citing works
I-vector-based speaker adaptation of deep neural networks for French broadcast audio transcription
Gupta, V., Kenny, P. J., Ouellet, P., et al. (2014) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2014.6854823 · Cited by 121 · see citing works
Transcribing Meetings With the AMIDA Systems
Hain, T., Burget, L., Dines, J., et al. (2011) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2011.2163395 · Cited by 121 · see citing works
Networks with trainable amplitude of activation functions
Trentin, E. (2001) · Neural Networks · doi:10.1016/s0893-6080(01)00028-4 · Cited by 107 · see citing works
Speaker adaptation using combined transformation and Bayesian methods
Digalakis, V., & Neumeyer, L. (1996) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/89.506933 · Cited by 101 · see citing works
iVector-based discriminative adaptation for automatic speech recognition
Karafiát, M., Burget, L., Matějka, P., et al. (2011) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru.2011.6163922 · Cited by 90 · see citing works
Catastrophic Forgetting in Connectionist Networks: Causes, Consequences and Solutions
French, R. M. (1999) · Cited by 75 · see citing works
Hermitian Polynomial for Speaker Adaptation of Connectionist Speech Recognition Systems
Siniscalchi, S. M., Li, J., & Lee, C. (2013) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2013.2270370 · Cited by 72 · see citing works
Connectionist speaker normalization and adaptation
Abrash, V., Franco, H., Sankar, A., & Cohen, M. (1995) · European Conference on Speech Communication and Technology (Eurospeech) · doi:10.21437/eurospeech.1995-414 · Cited by 70 · see citing works
Using neural network front-ends on far field multiple microphones based speech recognition
Liu, Y., Zhang, P., & Hain, T. (2014) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2014.6854663 · Cited by 68 · see citing works
Adaptation of deep neural network acoustic models using factorised i-vectors
Karanasou, P., Wang, Y., Gales, M., et al. (2014) · Interspeech (USB) · doi:10.21437/interspeech.2014-488 · Cited by 64 · see citing works
Speaker Adaptive Training using Deep Neural Networks
Ochiai, T., Matsuda, S., Lu, X., et al. (2014) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2014.6854826 · Cited by 59 · see citing works
Revisiting hybrid and GMM-HMM system combination techniques
Świętojański, P., Ghoshal, A. K., & Renals, S. J. (2013) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2013.6638967 · Cited by 58 · see citing works
Self-citationInvestigation of maxout networks for speech recognition
Świętojański, P., Li, J., & Huang, J.-T. (2014) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2014.6855088 · Cited by 52 · see citing works
Multi-level adaptive networks in tandem and hybrid ASR systems
Bell, P., Świętojański, P., & Renals, S. J. (2013) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2013.6639014 · Cited by 39 · see citing works
Self-citationThe IWSLT 2011 Evaluation Campaign on Automatic Talk Translation
Federico, M., Stüker, S., Bentivogli, L., Paul, M. D., Cettolo, M., Herrmann, T., Niehues, J., & Moretti, G. (2012) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/5ms6auiwwnq2 · Cited by 37 · see citing works
A lecture transcription system combining neural network acoustic and language models
Bell, P., Yamamoto, H., Świętojański, P., et al. (2013) · Interspeech (USB) · doi:10.21437/interspeech.2013-673 · Cited by 25 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Personalization of end-to-end speech recognition on mobile devices for named entities
Sim, K. C., Beaufays, F., Benard, A., Guliani, D., Kabel, A., Khare, N., Lucassen, T., Zadrazil, P., Zhang, H., Johnson, L., et al. (2019) · 2019 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) · doi:10.1109/asru46091.2019.9003775 · Cited by 54 · see citing works
Geng, M., Xie, X., Ye, Z., Wang, T., Li, G., Hu, S., Liu, X., & Meng, H. (2022) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2022.3195113 · Cited by 48 · see citing works
User adaptation of convolutional neural network for human activity recognition
Matsui, S., Inoue, N., Akagi, Y., Nagino, G., & Shinoda, K. (2017) · 2017 25th European Signal Processing Conference (EUSIPCO) · doi:10.23919/eusipco.2017.8081308 · Cited by 46 · see citing works
On-device personalization of automatic speech recognition models for disordered speech
Tomanek, K., Beaufays, F., Cattiau, J., Chandorkar, A., & Sim, K. C. (2021) · arXiv · doi:10.48550/arxiv.2106.10259 · Cited by 14 · see citing works
- Authors
- Swietojanski, P., & Renals, S.
- Affiliations
- University of Edinburgh (GB)
- ORCID
- Renals, Steve
- Venue
- 2014 IEEE Spoken Language Technology Workshop (SLT)
- Publisher
- IEEE
- Published
- 2014-12
- Pages
- 171-176
- OpenAlex
- W2094147890
- Cited by
- 218
- References
- 42
- Citations by year
- 2026: 3 · 2025: 1 · 2024: 3 · 2023: 7 · 2022: 13 · 2021: 15 · 2020: 20 · 2019: 18 · 2018: 36 · 2017: 31 · 2016: 43 · 2015: 26 · 2014: 2
- FWCI
- 20.02
- Type
- conference paper
- Language
- en
- Open access
- Licence
- other-oa
- Locations
- IEEE Spoken Language Technology Workshop (SLT) (publishedVersion)
- Edinburgh Research Explorer (University of Edinburgh) (submittedVersion, other-oa)
- Funders
- Engineering and Physical Sciences Research Council (EP/I031022/1)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Keywords
- unsupervised speaker adaptation, model-based adaptation, TED Talks, speech translation, word error rate reduction, rectified linear units
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review