Thousands of Voices for HMM-Based Speech Synthesis–Analysis and Application of TTS Systems Built on Various ASR Corpora
Open accessThe DOI matches its Crossref record; 66 of the 73 references it cites were found in a registry or an index.
Abstract
In conventional speech synthesis, large amounts of phonetically balanced speech data recorded in highly controlled recording studio environments are typically required to build a voice. Although using such data is a straightforward solution for high quality synthesis, the number of voices available will always be limited, because recording costs are high. On the other hand, our recent experiments with HMM-based speech synthesis systems have demonstrated that speaker-adaptive HMM-based speech synthesis (which uses an “average voice model” plus model adaptation) is robust to non-ideal speech data that are recorded under various conditions and with varying microphones, that are not perfectly clean, and/or that lack phonetic balance. This enables us to consider building high-quality voices on “non-TTS” corpora such as ASR corpora. Since ASR corpora generally include a large number of speakers, this leads to the possibility of producing an enormous number of voices automatically. In this paper, we demonstrate the thousands of voices for HMM-based speech synthesis that we have made from several popular ASR corpora such as the Wall Street Journal (WSJ0, WSJ1, and WSJCAM0), Resource Management, Globalphone, and SPEECON databases. We also present the results of associated analysis based on perceptual evaluation, and discuss remaining issues.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Furthermore, these MCDs are comparable to those observed with text-to-speech synthesizers16 and better than those in previous neural-decoding work with participants that were able to speak naturally11.”
From A high-performance neuroprosthesis for speech decoding and avatar control (2023) · cites this paper as [16]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Maximum Likelihood from Incomplete Data Via the EM Algorithm
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1977.tb01600.x · Cited by 50,444
Kawahara, H., Masuda-Katsuse, I., & de Cheveigné, A. (1999) · Speech Communication · doi:10.1016/s0167-6393(98)00085-5 · Cited by 1,831
Maximum likelihood linear transformations for HMM-based speech recognition
Gales, M. J. F. (1998) · Computer Speech & Language · doi:10.1006/csla.1998.0043 · Cited by 1,482
Attractive Faces Are Only Average
Langlois, J. H., & Roggman, L. A. (1990) · Psychological Science · doi:10.1111/j.1467-9280.1990.tb00079.x · Cited by 1,317
Pitch-synchronous waveform processing techniques for text-to-speech synthesis using diphones
Moulines, E., & Charpentier, F. (1990) · Speech Communication · doi:10.1016/0167-6393(90)90021-z · Cited by 1,244
Statistical parametric speech synthesis
Zen, H., Tokuda, K., & Black, A. W. (2009) · Speech Communication · doi:10.1016/j.specom.2009.04.004 · Cited by 1,195
Self-citationThe design for the wall street journal-based CSR corpus
Paul, D. B., & Baker, J. M. (1992) · Proceedings of the workshop on Speech and Natural Language - HLT '91 · doi:10.3115/1075527.1075614 · Cited by 1,099
Cox, T., & Cox, M. (2000) · Chapman & Hall/CRC eBooks · doi:10.1201/9780367801700 · Cited by 745
Tree-based state tying for high accuracy acoustic modelling
Young, S. J., Odell, J. J., & Woodland, P. C. (1994) · Proceedings of the workshop on Human Language Technology - HLT '94 · doi:10.3115/1075812.1075885 · Cited by 664
A robust parser for spoken language understanding
Wang, Y.-Y. (1999) · 6th European Conference on Speech Communication and Technology · doi:10.21437/eurospeech.1999-513 · Cited by 653
Research works printed but not linked: 9
- guan, 2009, evaluation of flat start labeling for phoneme based mandarin hts system, Proc ORIENTAL-COCOSDA-09, 187
- 2008, Deliverable Report D2 1
- guan, 2010, an efficient and unified approach of mandarin hts system, Proc ICASSP96
- kawai, 2006, ximera: a concatenative speech synthesis system with large scale corpora, IEICE Trans Inf Syst, j89 d ii, 2688
- tsakalidis, 2005, Cross-corpus normalization of diverse acoustic training data for robust HMM training
- pallet, 1990, darpa resource management bench, Proc Workshop Speech Natural Lang, 298
- tokuda, 0, The HMM-Based Speech Synthesis System (HTS) Version 2 1
- 10.1109/TASL.2009.2016394 · DOI · Paper page
- 10.1109/JSTSP.2010.2079315 · DOI
Cited by
A high-performance neuroprosthesis for speech decoding and avatar control
Metzger, S. L., Littlejohn, K. T., Silva, A. B., Moses, D. A., Seaton, M. P., Wang, R., Dougherty, M. E., Liu, J. R., Wu, P., Berger, M. A., et al. (2023) · Nature · doi:10.1038/s41586-023-06443-4 · Cited by 522
“The final 1D transpose convolution had the same number of kernels as the input signal (16).”
BackgroundVeaux, C., Yamagishi, J., & King, S. (2013) · 2013 International Conference Oriental COCOSDA held jointly with 2013 Conference on Asian Spoken Language Research and Evaluation (O-COCOSDA/CASLRE) · doi:10.1109/icsda.2013.6709856 · Cited by 398
“This technique, referred to as speaker adaptation, has already been successfully applied to “non traditional TTS” corpora such as ASR corpora, which contains a large number of speakers [4].”
MethodSelf-citationSpeech Synthesis Based on Hidden Markov Models
Tokuda, K., Nankaku, Y., Toda, T., Zen, H., Yamagishi, J., & Oura, K. (2013) · Proceedings of the IEEE · doi:10.1109/jproc.2013.2251852 · Cited by 404
“Speaker-adaptive HMMbased synthesis requires as little as 5–7 min of recorded speech from a target speaker to generate a personalized synthetic voice [24]; hence, the average voice model can be easily transformed into a synthetic voice for any number of speakers [25].”
BackgroundSelf-citationEvaluation of Speaker Verification Security and Detection of HMM-Based Synthetic Speech
De Leon, P. L., Pucher, M., Yamagishi, J., Hernaez, I., & Saratxaga, I. (2012) · IEEE Transactions on Audio, Speech, and Language Processing · doi:10.1109/tasl.2012.2201472 · Cited by 231
Self-citationVoice Conversion Using Dynamic Kernel Partial Least Squares Regression
Helander, E. E., Silén, H., Virtanen, T. I., & Gabbouj, M. (2011) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2011.2165944 · Cited by 126
“the eigenvoice approach [11] or speaker adaptation in HMM-base d speech synthesis [16].”
Method
Metadata
- Authors
- Yamagishi, J., Usabaev, B., King, S., Watts, O., Dines, J., Tian, J., Guan, Y., Hu, R., Oura, K., Wu, Y.-J., Tokuda, K., Karhila, R., & Kurimo, M.
- Affiliations
- University of Edinburgh (GB)
- University of Tübingen (DE)
- Idiap Research Institute (CH)
- Nokia (China) (CN)
- Nagoya Institute of Technology (JP)
- Microsoft (United States) (US)
- Venue
- IEEE Transactions on Audio, Speech, and Language Processing
- Published
- 2010-07
- Volume
- 18
- Issue
- 5
- Pages
- 984-1004
- ISSN
- 1558-7916 (ISSN-L)
- 1558-7924
- OpenAlex
- W2165143604
- Cited by
- 78
- References
- 73
- Citations by year
- 2026: 1 · 2025: 2 · 2023: 4 · 2022: 3 · 2021: 2 · 2020: 4 · 2019: 1 · 2018: 2 · 2017: 7 · 2016: 7 · 2015: 4 · 2014: 9 · 2013: 6 · 2012: 15
- FWCI
- 10.83
- Type
- journal article
- Language
- en
- Open access
- green
- hdl.handle.net
- Locations
- IEEE Transactions on Audio Speech and Language Processing (publishedVersion)
- ERA (submittedVersion)
- Edinburgh Research Explorer (University of Edinburgh) (submittedVersion)
- Funders
- Helsingin Yliopisto
- European Commission (213845)
- Universidad Politécnica de Madrid
- Engineering and Physical Sciences Research Council (FP7/2007–2013)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Phonetics and Phonology Research
- Keywords
- HMM-based speech synthesis, model adaptation, perceptual evaluation, Wall Street Journal, resource management
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Thousands of Voices for HMM-Based Speech Synthesis–Analysis and Application of TTS Systems Built on Various ASR Corpora. https://app.cochat.ai/papers/10.1109/tasl.2010.2045237
- The paper
- Yamagishi, J., Usabaev, B., King, S., Watts, O., Dines, J., Tian, J., Guan, Y., Hu, R., Oura, K., Wu, Y.-J., Tokuda, K., Karhila, R., & Kurimo, M. (2010). Thousands of Voices for HMM-Based Speech Synthesis–Analysis and Application of TTS Systems Built on Various ASR Corpora. IEEE Transactions on Audio, Speech, and Language Processing. https://doi.org/10.1109/tasl.2010.2045237
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.