CoChat
Sign Up
PreprintCited by 20Speech Recognition and SynthesisDomain Adaptation and Few-Shot LearningSpeech and Audio Processing

The DOI matches its DataCite record.

Cross-Language Transfer Learning, Continuous Learning, and Domain Adaptation for End-to-End Automatic Speech Recognition

Open access
Huang, Jocelyn, Kuchaiev, Oleksii, O'Neill, Patrick, Lavrukhin, Vitaly, Li, Jason, Flores, Adriana, Kucsko, Georg, Ginsburg, Boris·2020·arXiv
DOI 10.48550/arxiv.2005.04290arXiv 2005.04290
Open this paper in CoChat — free accountOpen PDF

Abstract

In this paper, we demonstrate the efficacy of transfer learning and continuous learning for various automatic speech recognition (ASR) tasks. We start with a pre-trained English ASR model and show that transfer learning can be effectively and easily performed on: (1) different English accents, (2) different languages (German, Spanish and Russian) and (3) application-specific domains. Our experiments demonstrate that in all three cases, transfer learning from a good base model has higher accuracy than a model trained from scratch. It is preferred to fine-tune large models than small pre-trained models, even if the dataset for fine-tuning is small. Moreover, transfer learning significantly speeds up convergence for both very small and very large target datasets.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2005.04290

Cites

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • In earlier research works, transfer learning has been used to cope with cross-language text classification.

    “In earlier research works, transfer learning has been used to cope with cross-language text classification [147][148][149][150].”

    From Online grooming detection: A comprehensive survey of child exploitation in chat logs (2023) · cites this paper as [149]

    Section 2.1 (p. 1) reports earlier uses of cross-language transfer learning, but the examples are for automatic speech recognition and acoustic modeling. The source does not support the sentence’s specific claim about cross-language text classification. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

  • Distribution variation in feature space where the data language changes can lead to a performance drop.

    “Distribution variation in feature space where the data language changes can lead to a performance drop [147][148][149][150].”

    From Online grooming detection: A comprehensive survey of child exploitation in chat logs (2023) · cites this paper as [149]

    Section 4.1 (pp. 2–3) shows that an ASR model trained on one English dataset can have higher word error rates on another, supporting the general point that a change in data can hurt performance. Section 5.3 (p. 4) studies transfer from English to German, Spanish, and Russian, but does not establish that a change in feature-space distribution caused a performance drop when the language changed. (LLM, conf=0.78)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

No references could be resolved yet.

Research works printed but not linked: 21

  1. [1][1] D. Wang and T. F. Zheng, 'Transfer learning for speech and language processing,' in Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA) , 2015, pp. 1225-1237. -
  2. [2][2] T. Schultz and A. H. Waibel, 'Language-independent and language-adaptive acoustic modeling for speech recognition,' Speech Commun. , vol. 35, pp. 31-51, 2001. -
  3. [3][3] S. Kriman, S. Beliaev, B. Ginsburg, J. Huang, O. Kuchaiev, V. Lavrukhin, R. Leary, J. Li, and Y . Zhang, 'QuartzNet: Deep automatic speech recognition with 1D time-channel separable convolutions,' ICASSP , 2020. -
  4. [4][4] O. Kuchaiev, J. Li, H. Nguyen, O. Hrinchuk, R. Leary, B. Ginsburg, S. Kriman, S. Beliaev, V. Lavrukhin, J. Cook, P. Castonguay, M. Popova, J. Huang, and J. M. Cohen, 'Nemo: a toolkit for building AI applications using neural modules,' arXiv:1909.09577 , 2019. - · arXiv
  5. [5][5] O. Anderson, P. Dalsgaard, and W. Barry, 'On the use of datadriven clustering technique for identification of poly- and monophonemes for four european languages,' in ICASSP , 1994. -
  6. [6][6] J.-T. Huang, J. Li, D. Yu, L. Deng, and Y. Gong, 'Cross-language knowledge transfer using multilingual deep neural network with shared hidden layers,' in ICASSP , 2013. -
  7. [7][7] P. Swietojanski, A. Ghoshal, and S. Renals, 'Unsupervised crosslingual knowledge transfer in DNN-based LVCSR,' in Spoken Language Technology Workshop (SLT) , 2012. -
  8. [8][8] A. Ghoshal, P. Swietojanski, and S. Renals, 'Multilingual training of deep neural networks,' in ICASSP , 2013. -
  9. [9][9] J. Kunze, L. Kirsch, I. Kurenkov, A. Krug, J. Johannsmeier, and S. Stober, 'Transfer learning for speech recognition on a budget,' in Rep4NLP@ACL , 2017. -
  10. [11][11] S. Tong, P. Garner, and H. Bourlard, 'Multilingual training and cross-lingual adaptation on CTC-based acoustic model,' Speech Communication , vol. 104, 2017. -
  11. [12][12] S. Ueno, T. Moriya, M. Mimura, S. Sakai, Y. Shinohara, Y. Yamaguchi, Y. Aono, and T. Kawahara, 'Encoder transfer for attention-based acoustic-to-word speech recognition,' in INTERSPEECH , 2018. -
  12. [13][13] W. Chan, N. Jaitly, Q. V. Le, and O. Vinyals, 'Listen, attend and spell,' in ICASSP , 2016. -
  13. [14][14] T. Moriya, R. Masumura, T. Asami, Y. Shinohara, M. Delcroix, Y. Yamaguchi, and Y. Aono, 'Progressive neural network-based knowledge transfer in acoustic models,' in Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) , 2018. -
  14. [15][15] A. Graves, S. Fern´ andez, F. Gomez, and J. Schmidhuber, 'Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks,' in ICML , 2006. -
  15. [16][16] F. Chollet, 'Xception: Deep learning with depthwise separable convolutions,' in CVPR , 2017. -
  16. [17][17] J. X. Koh, A. Mislan, K. Khoo, B. Ang, W. Ang, C. Ng, and Y.-Y. Tan, 'Building the Singapore English National Speech Corpus,' in INTERSPEECH , 2019. -
  17. [19][19] V. Panayotov, G. Chen, D. Povey, and S. Khudanpur, 'Librispeech: an ASR corpus based on public domain audio books,' in ICASSP , 2015. -
  18. [20][20] R. Ardila, M. Branson, K. Davis, M. Henretty, M. Kohler, J. Meyer, R. Morais, L. Saunders, F. M. Tyers, and G. Weber, 'Common Voice: A massively-multilingual speech corpus,' arXiv:1912.06670 , 2019. - · arXiv
  19. [21][21] D. B. Paul and J. M. Baker, 'The design for the Wall Street Journal based CSR corpus,' in Proceedings of the workshop on Speech and Natural Language . ACL, 1992, pp. 357-362. -
  20. [22][22] C. Cieri, D. Miller, and K. Walker, 'The Fisher corpus: a resource for the next generations of speech-to-text,' in LREC , 2004. -
  21. [24][24] B. Ginsburg, P. Castonguay, O. Hrinchuk, O. Kuchaiev, V. Lavrukhin, R. Leary, J. Li, H. Nguyen, and J. M. Cohen, 'Stochastic gradient methods with layer-wise adaptive moments for training of deep networks,' arXiv:1905.11286 , 2019. · arXiv

Cited by

Sorted by citations

Metadata

Authors
Huang, J., Kuchaiev, O., O'Neill, P., Lavrukhin, V., Li, J., Flores, A., Kucsko, G., & Ginsburg, B.
Affiliations
Nvidia (United Kingdom) (GB)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2020
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2005.04290
arXiv
2005.04290
OpenAlex
W3023256384
Cited by
20
References
0
Citations by year
2025: 1 · 2024: 3 · 2023: 3 · 2022: 4 · 2021: 7 · 2020: 2
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
Keywords
continuous learning, domain adaptation, end-to-end speech recognition, fine-tuning, English accents, low-resource languages, convergence rate

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review