Attentron: Few-Shot Text-to-Speech Utilizing Attention-Based Variable-Length Embedding
Open accessThe DOI matches its Crossref record; 30 of the 30 references it cites were found in a registry or an index.
Abstract
On account of growing demands for personalization, the need for a so-called few-shot TTS system that clones speakers with only a few data is emerging.To address this issue, we propose Attentron, a few-shot TTS model that clones voices of speakers unseen during training.It introduces two special encoders, each serving different purposes.A fine-grained encoder extracts variable-length style information via an attention mechanism, and a coarse-grained encoder greatly stabilizes the speech synthesis, circumventing unintelligible gibberish even for synthesizing speech of unseen speakers.In addition, the model can scale out to an arbitrary number of reference audios to improve the quality of the synthesized speech.According to our experiments, including a human evaluation, the proposed model significantly outperforms state-of-the-art models when generating speech for unseen speakers in terms of speaker similarity and quality.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Due to rapid research progress, it is possible to produce a realistic-sounding clone using only a few audio samples [3].”
From Warning: Humans cannot reliably detect speech deepfakes (2023) · cites this paper as [3]
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Exploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
librosa: Audio and Music Signal Analysis in Python
McFee, B., Raffel, C., Liang, D., Ellis, D., McVicar, M., Battenberg, E., & Nieto, O. (2015) · Proceedings of the Python in Science Conference · doi:10.25080/majora-7b98e3ed-003 · Cited by 3,033
Natural TTS Synthesis by Conditioning Wavenet on MEL Spectrogram Predictions
Shen, J., Pang, R., Weiss, R. J., Schuster, M., Jaitly, N., Yang, Z., Chen, Z., Zhang, Y., Wang, Y., Skerrv-Ryan, R., et al. (2018) · 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2018.8461368 · Cited by 2,791
Mel-cepstral distance measure for objective speech quality assessment
Kubichek, R. (2002) · Proceedings of IEEE Pacific Rim Conference on Communications Computers and Signal Processing · doi:10.1109/pacrim.1993.407206 · Cited by 534
Transfer learning from speaker verification to multispeaker text-to-speech synthesis
Jia, Y., Zhang, Y., Weiss, R. J., Wang, Q., Shen, J., Ren, F., Chen, Z., Nguyen, P., Pang, R., Moreno, I. L., et al. (2018) · arXiv · doi:10.48550/arxiv.1806.04558 · Cited by 519
Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesis
Wang, Y., Stanton, D., Zhang, Y., Skerry-Ryan, R., Battenberg, E., Shor, J., Xiao, Y., Ren, F., Ye, J., & Saurous, R. A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.09017 · Cited by 497
SUPERSEDED - CSTR VCTK Corpus: English Multi-speaker Corpus for CSTR Voice Cloning Toolkit
Veaux, C., Yamagishi, J., & MacDonald, K. (2016) · University of Edinburgh. The Centre for Speech Technology Research (CSTR) · doi:10.7488/ds/1495 · Cited by 434
ClariNet: Parallel Wave Generation in End-to-End Text-to-SpeechPreprint
Ping, W., Peng, K., & Chen, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.07281 · Cited by 350
Cited by
A survey on neural speech synthesis
Tan, X., Qin, T., Soong, F., & Liu, T.-Y. (2021) · arXiv · doi:10.48550/arxiv.2106.15561 · Cited by 216
Warning: Humans cannot reliably detect speech deepfakes
Mai, K. T., Bray, S., Davies, T., & Griffin, L. D. (2023) · PLOS ONE · doi:10.1371/journal.pone.0285333 · Cited by 98
“Due to rapid research progress, it is possible to produce a realistic-sounding clone using only a few audio samples (Choi et al., 2020).”
BackgroundSC-GlowTTS: An Efficient Zero-Shot Multi-Speaker Text-To-Speech Model
Casanova, E., Shulby, C., Gölge, E., Müller, N. M., Oliveira, F. S. D., Candido Jr., A., Soares, A. D. S., Aluisio, S. M., & Ponti, M. A. (2021) · Interspeech 2021 · doi:10.21437/interspeech.2021-1774 · Cited by 83
Meta-TTS: Meta-Learning for Few-Shot Speaker Adaptive Text-to-Speech
Huang, S.-F., Lin, C.-J., Liu, D.-R., Chen, Y.-C., & Lee, H.-Y. (2022) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2022.3167258 · Cited by 59
AV-Deepfake1M: A Large-Scale LLM-Driven Audio-Visual Deepfake Dataset
Cai, Z., Ghosh, S., Adatia, A. P., Hayat, M., Dhall, A., Gedeon, T., & Stefanov, K. (2024) · Proceedings of the 32nd ACM International Conference on Multimedia · doi:10.1145/3664647.3680795 · Cited by 59
Metadata
- Authors
- Choi, S., Han, S., Kim, D., & Ha, S.
- Venue
- Interspeech 2020
- Publisher
- ISCA
- Published
- 2020-10-25
- Pages
- 2007-2011
- ISSN
- 2308-457X (ISSN-L)
- OpenAlex
- W3095199334
- Cited by
- 56
- References
- 29
- Citations by year
- 2026: 2 · 2025: 9 · 2024: 13 · 2023: 10 · 2022: 15 · 2021: 7
- FWCI
- 3.06
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Interspeech (USB) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Health Sciences › Health Professions › Speech and Hearing › Voice and Speech Disorders
- Keywords
- attention mechanism, speaker similarity, synthetic speech quality
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Attentron: Few-Shot Text-to-Speech Utilizing Attention-Based Variable-Length Embedding. https://app.cochat.ai/papers/10.21437/interspeech.2020-2096
- The paper
- Choi, S., Han, S., Kim, D., & Ha, S. (2020). Attentron: Few-Shot Text-to-Speech Utilizing Attention-Based Variable-Length Embedding. Interspeech 2020. https://doi.org/10.21437/interspeech.2020-2096
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.