CoChat
Sign Up
Cited by 133Speech Recognition and SynthesisSpeech and Audio ProcessingPhonetics and Phonology Research

The DOI matches its Crossref record; the references it cites are not checked yet.

Learning Hidden Unit Contributions for Unsupervised Acoustic Model Adaptation

Open access
Swietojanski, Pawel, Li, Jinyu, Renals, Steve·2016·IEEE/ACM Transactions on Audio, Speech, and Language Processing
DOI 10.1109/taslp.2016.2560534
Open this paper in CoChat — free accountOpen PDF

Abstract

This work presents a broad study on the adaptation of neural network acoustic models by means of learning hidden unit contributions (LHUC) - a method that linearly re-combines hidden units in a speaker- or environment-dependent manner using small amounts of unsupervised adaptation data. We also extend LHUC to a speaker adaptive training (SAT) framework that leads to a more adaptable DNN acoustic model, working both in a speaker-dependent and a speaker-independent manner, without the requirements to maintain auxiliary speaker-dependent feature extractors or to introduce significant speaker-dependent changes to the DNN structure. Through a series of experiments on four different speech recognition benchmarks (TED talks, Switchboard, AMI meetings, and Aurora4) comprising 270 test speakers, we show that LHUC in both its test-only and SAT variants results in consistent word error rate reductions ranging from 5% to 23% relative depending on the task and the degree of mismatch between training and test data. In addition, we have investigated the effect of the amount of adaptation data per speaker, the quality of unsupervised adaptation targets, the complementarity to other adaptation techniques, one-shot adaptation, and an extension to adapting DNNs trained in a sequence discriminative manner.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/taslp.2016.2560534

Cites

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

The citation lists haven’t been fetched yet

References 96 · Cited by 133

Research works printed but not linked: 82

  1. 10.1109/ICASSP.2009.4960445 · DOI
  2. kaiser, 0, A novel loss function for the overall risk criterion based discriminative training of HMM models, Proc Int Conf Spoken Lang Process, 887
  3. 10.1016/S1364-6613(99)01294-2 · DOI · Paper page
  4. 10.1109/ICASSP.1992.225837 · DOI
  5. samarakoon, 0, On combining i-vectors and discriminative adaptation methods for unsupervised speaker normalization in dnn acoustic models, Proc IEEE Int Conf Acoust Speech Signal Process, 5275
  6. wester, 0, Human vs machine spoofing detection on wideband and narrowband data, Proc Annu Conf Int Speech Commun Assoc, 2047
  7. huang, 0, The NICT ASR system for IWSLT 2013, Proc Int Workshop Spoken Lang Translation
  8. huang, 0, Regularized sequence-level deep neural network model adaptation, Proc Annu Conf Int Speech Commun Assoc, 1081
  9. 10.1109/ASRU.2015.7404807 · DOI
  10. 10.1109/ICASSP.2013.6639201 · DOI
  11. fiscus, 0, 2000 NIST evaluation of conversational speech recognition over the telephone: English and Mandarin performance results, Proc Speech Transcription Workshop
  12. 10.1109/ICASSP.2014.6854662 · DOI
  13. 10.1109/ICASSP.2014.6854663 · DOI
  14. 10.1109/TASLP.2015.2457612 · DOI
  15. karanasou, 0, Adaptation of deep neural network acoustic models using factorised i-vectors, Proc Annu Conf Int Speech Commun Assoc, 2180
  16. 10.1109/ICASSP.2014.6854823 · DOI
  17. 10.1109/ICASSP.2013.6639212 · DOI
  18. xue, 2014, Fast adaptation of deep neural network based on discriminant codes for speech recognition, IEEE/ACM Trans Audio Speech Lang Process, 22, 1713, 10.1109/TASLP.2014.2346313 · DOI
  19. 10.1109/ICASSP.2013.6639211 · DOI
  20. bridle, 0, Recnorm: Simultaneous normalisation and classification applied to speech recognition, Proc Adv Neural Inf Process Sys, 234
  21. 10.1109/ASRU.2007.4430116 · DOI
  22. bell, 0, The UEDIN ASR systems for the IWSLT 2014 evaluation, Proc Int Workshop Spoken Lang Translation, 26
  23. federico, 0, Overview of the IWSLT 2012 evaluation campaign, Proc Int Workshop Spoken Lang Translation, 12
  24. 10.1109/ASRU.2013.6707744 · DOI
  25. 10.1109/ASRU.2011.6163922 · DOI
  26. 10.1109/ASRU.2013.6707749 · DOI
  27. 10.1109/TASL.2010.2064307 · DOI · Paper page
  28. lecun, 1998, Efficient backprop, Neural Networks Tricks of the Trade, 10.1007/3-540-49430-8_2 · DOI
  29. 10.1109/ICASSP.2012.6288864 · DOI
  30. 10.1109/ASRU.2013.6707705 · DOI
  31. 10.1109/LSP.2014.2325781 · DOI
  32. vesely, 0, Sequence-discriminative training of deep neural networks, Proc Annu Conf Int Speech Commun Assoc, 2345
  33. povey, 0, The Kaldi speech recognition toolkit, Proc IEEE Workshop Autom Speech Recog and Understanding
  34. 10.1007/978-1-4615-3210-1 · DOI
  35. 10.1109/ICASSP.2011.5947494 · DOI
  36. 10.1109/ICASSP.2012.6288833 · DOI
  37. 10.1109/TASL.2011.2163395 · DOI
  38. neto, 0, Speaker adaptation for hybrid HMM–ANN continuous speech recognition system, Proc 4th Eur Conf Speech Commun Technol, 2171
  39. 10.1109/ICASSP.2013.6639014 · DOI
  40. li, 0, Comparison of discriminative input and output transformations for speaker adaptation in the hybrid nn/hmm systems, Proc 11th Annu Conf Int Speech Commun Assoc
  41. abrash, 0, Connectionist speaker normalization and adaptation, Proc 4th Eur Conf Speech Commun Technol, 2183
  42. 10.1109/SLT.2014.7078566 · DOI
  43. 10.1016/0893-6080(89)90020-8 · DOI · Paper page
  44. 10.1109/ICSLP.1996.607807 · DOI
  45. 10.1109/ICASSP.2015.7178829 · DOI
  46. 10.1109/ICASSP.2015.7178787 · DOI
  47. 10.1109/ICASSP.2015.7178785 · DOI
  48. 10.1016/S0893-6080(01)00028-4 · DOI
  49. zhang, 0, Parameterised sigmoid and ReLU hidden activation functions for DNN acoustic modelling, Proc Annu Conf Int Speech Commun Assoc, 3224
  50. 10.1109/18.256500 · DOI
  51. 10.1016/0893-6080(91)90009-T · DOI · Paper page
  52. 10.1109/ASRU.2015.7404856 · DOI
  53. woodland, 0, Speaker adaptation for continuous density HMMs: A review, Proc ISCA ITR Workshop Adaptation Methods Speech Recognition, 11
  54. 10.1109/ICASSP.2014.6854828 · DOI
  55. abdel-hamid, 0, Rapid and effective speaker adaptation of convolutional neural network based models for speech recognition, Proc Annu Conf Int Speech Commun Assoc, 1248
  56. 10.1109/SLT.2014.7078569 · DOI · Paper page
  57. swietojanski, 0, SAT-LHUC: Speaker adaptive training for learning hidden unit contributions, Proc IEEE Int Conf Acoust Speech Signal Process, 5010
  58. cettolo, 0, Wit $^3$: Web inventory of transcribed and translated talks, Proc 16th Conf Eur Assoc Mach Translat, 261
  59. goodfellow, 0, Maxout networks, Proc Int Conf Mach Learn, 1319
  60. 10.1007/s10579-007-9040-x · DOI
  61. 10.1109/SLT.2014.7078567 · DOI
  62. van der maaten, 2008, Visualizing high-dimensional data using t-sne, J Mach Learn Res, 9, 2579
  63. parihar, 0, Performance analysis of the Aurora large vocabulary baseline system, Proc 12th Eur Signal Process Conf, 553
  64. srivastava, 0, Dropout: 2014 A simple way to prevent neural networks from overfitting, J Mach Learn Res, 1929
  65. 10.1006/csla.1998.0043 · DOI · Paper page
  66. 10.1109/ICASSP.2013.6639100 · DOI
  67. 10.1109/ASRU.2011.6163899 · DOI
  68. 10.1109/89.260359 · DOI
  69. 10.1109/ICASSP.2000.862024 · DOI
  70. 10.1109/TASL.2011.2134090 · DOI · Paper page
  71. 10.1109/ICASSP.2013.6639347 · DOI
  72. 10.1109/ICASSP.2007.367023 · DOI
  73. swietojanski, 0, Structured output layer with auxiliary targets for context-dependent acoustic modelling, Proc Annu Conf Int Speech Commun Assoc, 3605
  74. 10.1109/ICASSP.2013.6638967 · DOI
  75. 10.1109/TASL.2013.2270370 · DOI
  76. 10.1109/ICASSP.2015.7178783 · DOI
  77. huang, 0, Rapid adaptation for deep neural networks through multi-task learning, Proc Annu Conf Int Speech Commun Assoc, 3625
  78. huang, 0, Maximum a posteriori adaptation of network parameters in deep models, Proc Annu Conf Int Speech Commun Assoc, 1076
  79. 10.1109/ICASSP.2014.6854826 · DOI
  80. 10.1109/ASRU.2015.7404787 · DOI
  81. 10.1109/ICASSP.2015.7178784 · DOI
  82. 10.1109/SLT.2012.6424251 · DOI

Metadata

Authors
Swietojanski, P., Li, J., & Renals, S.
Affiliations
  • Microsoft (United States) (US)
  • University of Edinburgh (GB)
ORCID
Venue
IEEE/ACM Transactions on Audio, Speech, and Language Processing
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Published
2016-08
Volume
24
Issue
8
Pages
1450-1463
ISSN
  • 2329-9290 (ISSN-L)
  • 2329-9304
DOI
10.1109/taslp.2016.2560534
OpenAlex
W2239847623
Cited by
133
References
96
Citations by year
2026: 8 · 2025: 11 · 2024: 11 · 2023: 15 · 2022: 6 · 2021: 9 · 2020: 16 · 2019: 17 · 2018: 17 · 2017: 17 · 2016: 6
FWCI
15.65
Type
journal article
Language
en
Open access
Licence
ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html (vor)
Locations
  • IEEE/ACM Transactions on Audio Speech and Language Processing (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • Edinburgh Research Explorer (University of Edinburgh) (submittedVersion, other-oa)
  • arXiv (Cornell University)
Funders
  • EPSRC (EP/I031022/1)
  • Natural Speech Technology (NST)
  • European Union (H2020)
  • SUMMA (688139)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
  • Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Phonetics and Phonology Research
Keywords
speaker adaptive training, word error rate reduction

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review