CoChat

Meta-TTS: Meta-Learning for Few-Shot Speaker Adaptive Text-to-Speech

Open access
Huang, Sung-Feng, Lin, Chyi-Jiunn, Liu, Da-Rong, Chen, Yi-Chen, Lee, Hung-yi · 2022 · IEEE/ACM Transactions on Audio, Speech, and Language Processing
Cited by 59

The DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.

Abstract

Personalizing a speech synthesis system is a highly desired application, where the system can generate speech with the user’s voice with rare enrolled recordings. There are two main approaches to build such a system in recent works: speaker adaptation and speaker encoding. On the one hand, speaker adaptation methods fine-tune a trained multi-speaker text-to-speech (TTS) model with few enrolled samples. However, they require at least thousands of fine-tuning steps for high-quality adaptation, making it hard to apply on devices. On the other hand, speaker encoding methods encode enrollment utterances into a speaker embedding. The trained TTS model can synthesize the user’s speech conditioned on the corresponding speaker embedding. Nevertheless, the speaker encoder suffers from the generalization gap between the seen and unseen speakers. In this paper, we propose applying a meta-learning algorithm to the speaker adaptation method. More specifically, we use Model Agnostic Meta-Learning (MAML) as the training algorithm of a multi-speaker TTS model, which aims to find a great meta-initialization to adapt the model to any few-shot speaker adaptation tasks quickly. Therefore, we can also adapt the meta-trained TTS model to unseen speakers efficiently. Our experiments compare the proposed method (Meta-TTS) with two baselines: a speaker adaptation method baseline and a speaker encoding method baseline. The evaluation results show that Meta-TTS can synthesize high speaker-similarity speech from few enrollment samples with fewer adaptation steps than the speaker adaptation baseline and outperforms the speaker encoding baseline under the same training scheme. When the speaker encoder of the baseline is pre-trained with extra 8371 speakers of data, Meta-TTS can still outperform the baseline on LibriTTS dataset and achieve comparable results on VCTK dataset.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/taslp.2022.3167258

Cites

12 of 12 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 12 references

Research works printed but not linked: 25

  1. Ren, 2021, FastSpeech 2: Fast and high-quality end-to-end text-to-speech
  2. Gibiansky, 2017, Deep voice 2: Multi-speaker neural text-to-speech, 30, 2966
  3. Ping, 2018, Deep voice 3: 2000-speaker neural text-to-speech, 214
  4. Arik, 2018, Neural voice cloning with a few samples, 10040
  5. Chen, 2019, Sample efficient adaptive text-to-speech
  6. 10.21437/Interspeech.2020-1745 · DOI · Paper page
  7. Chen, 2021, AdaSpeech: Adaptive text to speech for custom voice
  8. 10.1109/ICASSP39728.2021.9414727 · DOI
  9. Taigman, 2018, VoiceLoop: Voice fitting and synthesis via a phonological loop
  10. 10.21437/Interspeech.2020-1737 · DOI
  11. 10.21437/Interspeech.2020-1032 · DOI
  12. 10.1109/ICASSP39728.2021.9413880 · DOI
  13. Finn, 2017, Model-agnostic meta-learning for fast adaptation of deep networks, 1126
  14. Raghu, 2020, Rapid learning or feature reuse? Towards understanding the effectiveness of MAML
  15. 10.1109/ICASSP.2014.6854363 · DOI · Paper page
  16. 10.1109/ICASSP.2016.7472652 · DOI · Paper page
  17. 10.1109/ICASSP.2018.8462665 · DOI · Paper page
  18. 10.21437/Interspeech.2019-2003 · DOI · Paper page
  19. 10.1109/ICASSP39728.2021.9413877 · DOI
  20. 10.21437/Interspeech.2021-2013 · DOI
  21. Baevski, 2020, Wav2Vec 2.0: A framework for self-supervised learning of speech representations, 33, 12449
  22. 10.1109/TASLP.2021.3095662 · DOI · Paper page
  23. 10.21437/Interspeech.2019-2441 · DOI · Paper page
  24. Yamagishi, 2019, CSTR VCTK Corpus: English multi-speaker corpus for CSTR voice cloning toolkit
  25. 10.1109/ICASSP.2018.8461368 · DOI · Paper page

Cited by

See all 59

Metadata

Authors
Huang, S.-F., Lin, C.-J., Liu, D.-R., Chen, Y.-C., & Lee, H.-Y.
Venue
IEEE/ACM Transactions on Audio, Speech, and Language Processing
Published
2022
Volume
30
Pages
1558-1571
ISSN
  • 2329-9290 (ISSN-L)
  • 2329-9304
OpenAlex
W3213544594
Cited by
59
References
12
FWCI
4.52
Type
journal article
Language
en
Open access
green
Funders
Ministry of Science and Technology, Taiwan (111-2628-E-002-013, 110-2223-E-002 -007-MY3)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Meta-TTS: Meta-Learning for Few-Shot Speaker Adaptive Text-to-Speech. https://app.cochat.ai/papers/10.1109/taslp.2022.3167258
The paper
Huang, S.-F., Lin, C.-J., Liu, D.-R., Chen, Y.-C., & Lee, H.-Y. (2022). Meta-TTS: Meta-Learning for Few-Shot Speaker Adaptive Text-to-Speech. IEEE/ACM Transactions on Audio, Speech, and Language Processing. https://doi.org/10.1109/taslp.2022.3167258

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.