Speech Synthesis Evaluation — State-of-the-Art Assessment and Suggestion for a Novel Research Program
Open accessThe DOI matches its Crossref record; 40 of the 40 references it cites were found in a registry or an index.
Abstract
Wagner P, Beskow J, Betz S, et al. Speech Synthesis Evaluation—State-of-the-Art Assessment and Suggestion for a Novel Research Program. In: Proceedings of the 10th Speech Synthesis Workshop (SSW10). 2019.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“It is preferred to correlation-based metrics, such as computing a Pearson correlation between ground truth and decoded acoustic waveforms, as MCD weighs the features in the produced sound that are most perceptually relevant for conveying information151.”
From The speech neuroprosthesis (2024) · cites this paper as [151]
Citations
Cites
WaveNet: a generative model for raw audio
Oord, A. V. D., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1609.03499 · Cited by 4,911 · cited 3×
“How can novel machine learning and high quality synthesis such as WaveNet be put to use in TTS evaluation?”
MethodBartneck, C., Kulić, D., Croft, E., & Zoghbi, S. (2009) · International Journal of Social Robotics · doi:10.1007/s12369-008-0001-3 · Cited by 3,193 · cited 1×
“To overcome this problem, typical surveys employed in HCI or HRI tend to be very extensive [35], thereby trying to address all potential quality dimensions a user may have employed in her or his assessment.”
BackgroundEncyclopedia of Survey Research Methods
Lavrakas, P. (2008) · doi:10.4135/9781412963947 · Cited by 2,689
Explaining Phonetic Variation: A Sketch of the H&H Theory
Lindblom, B. (1990) · Speech Production and Speech Modelling · doi:10.1007/978-94-009-2037-8_16 · Cited by 1,824
The Meaning of Intonational Contours in the Interpretation of Discourse
Pierrehumbert, J., & Hirschberg, J. (1990) · Intentions in Communication · doi:10.7551/mitpress/3839.003.0016 · Cited by 1,433
The Meaning of Intonational Contours in the Interpretation of Discourse
Pierrehumbert, J. B., & Hirschberg, J. (1990) · doi:10.7916/d8kd24fp · Cited by 830 · cited 1×
“…(tutoring, chatting, arguing, counseling...), individual (hearing problems, attitude, level of distraction, motivation, familiarity), linguistic (frequency, predictability, suprisal, importance) or environmental settings (external noise, mutual visibility, ...) [11, 12, 13, 14, 15, 16, 17, 18].”
Aylett, M., & Turk, A. (2004) · Language and Speech · doi:10.1177/00238309040470010201 · Cited by 781 · cited 1×
“…(tutoring, chatting, arguing, counseling...), individual (hearing problems, attitude, level of distraction, motivation, familiarity), linguistic (frequency, predictability, suprisal, importance) or environmental settings (external noise, mutual visibility, ...) [11, 12, 13, 14, 15, 16, 17, 18].”
Probabilistic relations between words: Evidence from reduction in lexical production
Jurafsky, D., Bell, A., Gregory, M., & Raymond, W. D. (2001) · Typological Studies in Language · doi:10.1075/tsl.45.13jur · Cited by 762
A Bayesian explanation of the ‘Uncanny Valley’ effect and related psychological phenomena
Moore, R. K. (2012) · Scientific Reports · doi:10.1038/srep00864 · Cited by 212 · cited 1×
“This is in line with the analysis by [20, 21], claiming that long-known problems of human machine interaction such as the uncanny valley can be modeled as a mismatch between a user’s expectations and a machine’s actual expression.”
ResultQuality-Net: An End-to-End Non-intrusive Speech Quality Assessment Model Based on BLSTM
Fu, S.-W., Tsao, Y., Hwang, H.-T., & Wang, H.-M. (2018) · Interspeech 2018 · doi:10.21437/interspeech.2018-1802 · Cited by 178
Cited by
Saka, A. B., Oyedele, L. O., Akanbi, L. A., Ganiyu, S. A., Chan, D. W. M., & Bello, S. A. (2023) · Advanced Engineering Informatics · doi:10.1016/j.aei.2022.101869 · Cited by 161
Conversational AI: Dialogue Systems, Conversational Agents, and Chatbots
McTear, M. (2021) · Synthesis Lectures on Human Language Technologies · doi:10.1007/978-3-031-02176-3 · Cited by 155
Kühne, K., Fischer, M. H., & Zhou, Y. (2020) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2020.593732 · Cited by 123
“While most synthesized voices have reached a high level of intelligibility, it is the perceived quality that still requires clarification (Polkosky and Lewis, 2003; Wagner et al., 2019).”
BackgroundSilva, A. B., Littlejohn, K. T., Liu, J. R., Moses, D. A., & Chang, E. F. (2024) · Nature Reviews Neuroscience · doi:10.1038/s41583-024-00819-9 · Cited by 103
“The performance of TTS systems is often described using mel-cepstral distortion (MCD) and open-ended human-transcribed WER 149,150 .”
Choice of Voices: A Large-Scale Evaluation of Text-to-Speech Voice Quality for Long-Form Content
Cambre, J., Colnago, J., Maddock, J., Tsai, J., & Kaye, J. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376789 · Cited by 65
“However, speech technology is advancing so rapidly that the state-of-the-art is likely outpacing our own publishing cycles [41,49].”
BackgroundInfluentialNot My Voice! A Taxonomy of Ethical and Safety Harms of Speech Generators
Hutiri, W. T., Papakyriakopoulos, O., & Xiang, A. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3658911 · Cited by 113
“Instead, current evaluations rely on listener tests, which are subjective and oftentimes unreliable [27, 53].”
BackgroundSocial Media through Voice: Synthesized Voice Qualities and Self-presentation
Zhang, L., Jiang, L., Washington, N., Liu, A. A., Shao, J., Fourney, A., Morris, M. R., & Findlater, L. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3449235 · Cited by 48
“, audio books) are evaluated primarily based on intelligibility, comprehensibility, and other subjective measures, such as likeability [7, 76].”
BackgroundInfluentialIs Natural Necessary? Human Voice versus Synthetic Voice for Intelligent Virtual Agents
Abdulrahman, A., & Richards, D. (2022) · Multimodal Technologies and Interaction · doi:10.3390/mti6070051 · Cited by 58
“Although synthetic voices with higher quality have been introduced [57,58], we opted to use the basic synthesized voice of Hazel for two reasons.”
MethodJung, T. H., Bae, S., Moorhouse, N., & Kwon, O. (2023) · Information Technology and People · doi:10.1108/itp-01-2023-0100 · Cited by 42
A review on subjective and objective evaluation of synthetic speech
Cooper, E., Huang, W.-C., Tsao, Y., Wang, H., Toda, T., & Yamagishi, J. (2024) · Nippon Onkyo Gakkaishi/Acoustical science and technology/Nihon Onkyo Gakkaishi · doi:10.1250/ast.e24.12 · Cited by 75
A review of differentiable digital signal processing for music and speech synthesis
Hayes, B., Shier, J., Fazekas, G., McPherson, A., & Saitis, C. (2024) · Frontiers in Signal Processing · doi:10.3389/frsip.2023.1284100 · Cited by 90
Choi, Y., Jung, Y., & Kim, H. (2021) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt48900.2021.9383533 · Cited by 28
“However, since there is no “explicit answer” for the speech generation tasks, researchers still rely on a subjective mean opinion score (MOS) test to evaluate the quality of the synthesized speech, which is expensive and time-consuming [7, 8].”
BackgroundAugmented Datasheets for Speech Datasets and Ethical Decision-Making
Papakyriakopoulos, O., Choi, A. S. G., Thong, W. E., Zhao, D., Andrews, J. T. A., Bourke, R., Xiang, A., & Koenecke, A. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3594049 · Cited by 59
“As another example, models of speech recognition or synthesis in affective computing might support users to understand or express their feelings [14, 93, 157, 169, 260].”
BackgroundMethodRefining the evaluation of speech synthesis: A summary of the Blizzard Challenge 2023
Perrotin, O., Stephenson, B., Gerber, S., Bailly, G., & King, S. (2024) · Computer Speech & Language · doi:10.1016/j.csl.2024.101747 · Cited by 45
“Due to lack of time and resources, this Challenge edition did not address the evaluation of speech synthesis in context (cf. example of applications in Wagner et al. (2019, Table 1)) which has already been discussed and sometimes performed in the most recent literature.”
InfluentialDeep MOS Predictor for Synthetic Speech Using Cluster-Based Modeling
Choi, Y., Jung, Y., & Kim, H. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-2111 · Cited by 25
Williams, J. A., Rownicka, J., Gallegos, P. O., & King, S. (2020) · The Speaker and Language Recognition Workshop (Odyssey) · doi:10.21437/odyssey.2020-32 · Cited by 30
The limits of the Mean Opinion Score for speech synthesis evaluation
Maguer, S. L., King, S., & Harte, N. (2023) · Computer Speech & Language · doi:10.1016/j.csl.2023.101577 · Cited by 67
“In the spirit of [69], we consider that a paradigm shift is in order as we need to answer this fundamental question: what constitutes ”good” synthetic speech?”
BackgroundBuilding and Designing Expressive Speech Synthesis
Aylett, M. P., Clark, L., Cowan, B. R., & Torre, I. (2021) · ACM eBooks · doi:10.1145/3477322.3477329 · Cited by 22
Torre, I., & Maguer, S. L. (2020) · IEEE RO-MAN · doi:10.1109/ro-man47096.2020.9223599 · Cited by 28
“Additionally, speech synthesis cannot be taken as a general problem with one solution fitting everyone [34].”
BackgroundTo Rate or Not To Rate: Investigating Evaluation Methods for Generated Co-Speech Gestures
Wolfert, P., Girard, J. M., Kucherenko, T., & Belpaeme, T. (2021) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3462244.3479889 · Cited by 18
“Thus, the “final stretch” in quality evaluations still relies heavily on subjective evaluations [34].”
BackgroundShi, Z., Chen, H., Velentza, A.-M., Liu, S., Dennler, N. S., O'Connell, A., & Matarić, M. J. (2023) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1145/3568162.3576987 · Cited by 26
“In the speech synthesis community, TTS voices have been commonly evaluated for their clarity [2–4, 15, 33] and intelligibility (i.e., how easy it is to accurately hear the content of the speech) [12, 15, 33] in the context of reading.”
BackgroundResultInfluentialTranslatotron 2: Robust direct speech-to-speech translation.Preprint
Ye, J., Ramanovich, M. T., Remez, T., & Pomerantz, R. (2021) · arXiv (Cornell University) · Cited by 24
Heiselberg, L., Blom, J. N., & van Dalen, A. (2022) · Journalism Studies · doi:10.1080/1461670x.2022.2052346 · Cited by 21
“” (Wagner et al. 2019, 1).”
BackgroundSynthesizing intelligible utterances from EEG of imagined speech
Xiong, W., Ma, L., & Li, H. (2025) · Frontiers in Neuroscience · doi:10.3389/fnins.2025.1565848 · Cited by 15
Perrotin, O., Stephenson, B., Gerber, S., & Bailly, G. (2023) · doi:10.21437/blizzard.2023-1 · Cited by 25
“Besides speech quality, speaker similarity, and speech intelligibility, the number of dimensions along which TTS can be sub-jectively evaluated is large [20].”
BackgroundAutomatic imitation of human and computer-generated vocal stimuli
Wilt, H., Wu, Y., Trotter, A. S., & Adank, P. (2022) · Psychonomic Bulletin & Review · doi:10.3758/s13423-022-02218-6 · Cited by 12
“The Klatt synthesizer used was unsophisticated compared with current speech synthesizers (Wagner et al., 2019).”
MethodUsability Of Text-To-Speech Technology in Creating News Podcasts using Portuguese Of Portugal
Afonso, M., Almeida, P., Beça, P., Silva, T., & Covalenco, I. (2022) · ACM International Conference on Interactive Media Experiences · doi:10.1145/3505284.3532968 · Cited by 9
“Considering the degree of subjectivity of the appreciation of TTS in relation to its function [14], a question that aimed to evaluate the TTS in its adaptability to the intended function, the oralization of news, was included.”
BackgroundLeblebici, D. (2024) · Multilingua · doi:10.1515/multi-2023-0072 · Cited by 22
Synthesizing a human-like voice is the easy way
Maguer, S. L., & Cowan, B. R. (2021) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3469595.3469614 · Cited by 8
Evaluating Speech–Phoneme Alignment and its Impact on Neural Text-To-Speech Synthesis
Zalkow, F., Govalkar, P., Müller, M., Habets, E. A. P., & Dittmar, C. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10097248 · Cited by 9
Bruder, C., Breda, P., & Larrouy-Maestri, P. (2025) · Computers in Human Behavior Artificial Humans · doi:10.1016/j.chbah.2025.100211 · Cited by 8
Cuciniello, M., Amorese, T., Greco, C., Callejas, Z., Vogel, C. M., Cordasco, G., & Esposito, A. A. (2023) · Human Behavior and Emerging Technologies · doi:10.1155/2023/8858268 · Cited by 7
“In this regard, an interesting paper by [16] argues against the idea of a single “ neutral ” or “ perfectly natural ” speaking style as a reference for evaluating synthetic speech, challenging the common assumption that human-read speech serves as the gold standard.”
BackgroundTTSDS - Text-to-Speech Distribution Score
Minixhofer, C., Klejch, O., & Bell, P. (2024) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt61566.2024.10832178 · Cited by 32
“We also include environment as a factor due to the prevalence of artifacts present in speech synthesis [19], and the difficulty of some TTS system to generate speech with realistic recording conditions [8].”
Park, K. S., & Etnier, J. L. (2021) · Contemporary Clinical Trials · doi:10.1016/j.cct.2021.106500 · Cited by 6
Effects of gender neutralization on the anthropomorphism of natural and synthetic voices
Kuch, J. M., Melchior, F., & Becker-Asano, C. (2023) · IEEE RO-MAN · doi:10.1109/ro-man57019.2023.10309479 · Cited by 9
Kim, J., & Hong, J.-H. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713911 · Cited by 9
A Comparative Study of Self-Supervised Speech Representations in Read and Spontaneous TTS
Wang, S., Henter, G. E., Gustafson, J., & Székely, É. (2023) · IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops (ICASSPW) · doi:10.1109/icasspw59220.2023.10193157 · Cited by 9
“However, we note that, (1) the specific TTS models could have bias towards which SSLs they work better with, (2) unlike other more easily benchmark-able downstream tasks, TTS is known to be complex to evaluate quantitatively or subjectively [26].”
BackgroundSelf-citationDo, P., Coler, M., Dijkstra, J., & Klabbers, E. (2023) · Interspeech (USB) · doi:10.21437/interspeech.2023-2158 · Cited by 9
The evaluation of prosody in speech synthesis: a systematic review
Galdino, J. C., Matos, A. N., Svartman, F. R. F., & Aluísio, S. M. (2025) · Journal of the Brazilian Computer Society · doi:10.5753/jbcs.2025.5468 · Cited by 13
Gutierrez, E., Gallegos, P. O., & Lai, C. (2021) · ISCA Speech Synthesis Workshop (SSW) · doi:10.21437/ssw.2021-5 · Cited by 13
Evaluating the usage of Text-To-Speech in K12 education
Dai, L., Kritskaia, V., van der Velden, E., Jung, M. M., Postma, M., & Louwerse, M. M. (2022) · International Conference on Education and E-Learning (ICEEL) · doi:10.1145/3578837.3578864 · Cited by 8
“Interestingly,[36] alsosuggestedthatsynthesizedspeechshouldbeevaluatedinthe environmentinwhichitisintendedtobeusedbecausethenoise intheenvironmentislikelytoimpacttheTTSperception.”
BackgroundRe-examining the quality dimensions of synthetic speech
Seebauer, F., Kuhlmann, M. T., Haeb‐Umbach, R., & Wagner, P. (2023) · ISCA Speech Synthesis Workshop (SSW) · doi:10.21437/ssw.2023-6 · Cited by 4
“This need for a more contextaware and time-sensitive method of evaluation has also been discussed in [15] and the importance of the specific wording in a synthetic speech evaluation has been pointed out in [16].”
MethodSelf-citationIntroduction to Audio Deepfake Generation: Academic Insights for Non-Experts
Choi, J., Schäfer, K., & Zmudzinski, S. (2024) · ACM International Workshop on Multimedia AI against Disinformation (MAD) · doi:10.1145/3643491.3660286 · Cited by 9
“The objective evaluation metrics are inexpensive, but do not match human perception well [72].”
BackgroundChoi, Y., Jung, Y., & Kim, H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.08267 · Cited by 4
“However, since there is no “explicit answer” for the speech generation tasks, researchers still rely on a subjective mean opinion score (MOS) test to evaluate the quality of the synthesized speech, which is expensive and time-consuming [7, 8].”
BackgroundHi robot, it’s not what you say, it’s how you say it
Miniota, J., Wang, S., Beskow, J., Gustafson, J., Székely, É., & Pereiral, A. (2023) · IEEE RO-MAN · doi:10.1109/ro-man57019.2023.10309427 · Cited by 6
“These findings highlight the difficulty in contextualizing evaluations, as has been called for when assessing the quality of modern TTS systems [32].”
BackgroundSelf-citationGonzalez, A. G. C., Lo, W.-S., & Mizuuchi, I. (2022) · IEEE RO-MAN · doi:10.1109/ro-man53752.2022.9900685 · Cited by 3
“An example of what is a natural conversation needs to be provided for future experiments, since what is considered to be normal depends on the context [44].”
BackgroundCorrelations of Evaluation Metrics for Voice Conversion: An Experimental Analysis
Nandi, A., Ghosh, S., Akhter, M. T., Dhar, S., & Jana, N. D. (2024) · International Conference on Computing, Communication and Networking Technologies (ICCCNT) · doi:10.1109/icccnt61001.2024.10725801 · Cited by 3
“On the other hand, the subjective assessment [30] refers to the human perception for the similarity between the original and the converted features.”
BackgroundAugmented Prompt Selection for Evaluation of Spontaneous Speech Synthesis
Székely, É., Edlund, J., & Gustafson, J. (2020) · KTH Publication Database DiVA (KTH Royal Institute of Technology) · Cited by 3
Self-citationImplementing Concatenative Text-To-Speech Synthesis System for Marathi Language using Python
Bairagi, V. K., Joshi, S. L., & Bharambe, V. (2022) · International Journal of Electrical and Electronics Engineering · doi:10.14445/23488379/ijeee-v9i9p101 · Cited by 3
Song, J., Kim, M., & Park, J. (2023) · PLoS ONE · doi:10.1371/journal.pone.0293222 · Cited by 3
Gustafson, J., Székely, É., & Beskow, J. (2023) · ACM International Conference on Intelligent Virtual Agents (IVA) · doi:10.1145/3570945.3607289 · Cited by 4
“As pointed out by Wagner at al. when we evaluate our TTS systems we need "to assess and take into account listeners’ application-specific needs and expectations" [56].”
BackgroundSelf-citationWhich Prosodic Features Matter Most for Pragmatics?
Ward, N., Marco, D., & Fuentes, O. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10889382 · Cited by 6
“…to serve as “knobs” for human-in-the-loop specification or post-editing of the prosody of speech synthesizer output [6]. d) To enable better evaluation of the quality of the prosody output of a generative system, for practical needs, including: d1) The evaluation of speech synthesizer output [7].”
Yaegashi, M., Saito, S., Nakano, T., & Ogawa, T. (2022) · Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) · doi:10.23919/apsipaasc55919.2022.9979812 · Cited by 2
“Synthetic speeches are known to have three primary aspects in their quality assessment: naturalness, intelligibility, and speaker similarity [6], [7].”
BackgroundProceedings of the Fifth Workshop on Teaching NLP
(2021) · doi:10.18653/v1/2021.teachingnlp-1 · Cited by 3
Can Auditory Nerve Models Tell us What’s Different About WaveNet Vocoded Speech?
Maguer, S. L., & Harte, N. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-2596 · Cited by 1
“in [10], the evaluation and, hence, the analysis of synthetic speech is no easy task.”
BackgroundChoi, I., & Zu, J. (2022) · ETS Research Report Series · doi:10.1002/ets2.12347 · Cited by 1
Thoughts on the usage of audible smiling in speech synthesis applications
Trouvain, J., & Weiß, B. (2022) · Frontiers in Computer Science · doi:10.3389/fcomp.2022.885657 · Cited by 1
Implementation of Synthetic Artificial Human Like Voice Over Robotic Voice
Mithun, S., Niharika, P., & Bharamagoudra, M. R. (2022) · International Conference on Smart Generation Computing, Communication and Networking (SMART GENCON) · doi:10.1109/smartgencon56628.2022.10084239 · Cited by 1
“The perceived quality of most synthesised voices still needs to be clarified, despite the fact that they have generally attained a high level of understandability (Wagner et al., 2019) [2].”
BackgroundSchettino, L., Origlia, A., & Cutugno, F. (2023) · Computer Speech & Language · doi:10.1016/j.csl.2023.101585 · Cited by 4
Investigation of Auditory Nerve Model Based Analysis for Vocoded Speech Synthesis
Maguer, S. L., & Harte, N. (2020) · International Conference on Quality of Multimedia Experience (QoMEX) · doi:10.1109/qomex48832.2020.9123101 · Cited by 1
“A more dedicated and detailed subjective analysis should be conducted, but, as [3] pointed out, the development of new protocols is necessary to do so.”
BackgroundMethodInfluentialKnock-Knock! Who’s There? The Laughter-Enhanced Virtual Real-Estate Agent
Ludusan, B., & Wagner, P. (2021) · PUB – Publications at Bielefeld University (Bielefeld University) · Cited by 1
Self-citationNeural Speech Synthesis for EstonianPreprint
Rätsep, L., Piits, L., Pajupuu, H., Hein, I., & Fishel, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.02636 · Cited by 1
“Sobivuse hindamise testides lähtuti [8] ja [9] põhimõtetest.”
Synthesising Expressive Speech – Which Synthesiser for VOCAs?
Wülfing, J.-O., Dang, C. T., & André, E. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58323-1_43
Williams, S. (2022) · Springer eBooks · doi:10.1007/978-3-031-12488-4_4 · Cited by 2
Spectral and latent representation distortion for TTS evaluation
Kongthaworn, T. (2021) · Chula Digiverse · doi:10.58837/chula.the.2021.101
Combining Text-to-Speech Services with Conventional Voiceover for News Oralization
Afonso, M., & Almeida, P. (2023) · Communications in computer and information science · doi:10.1007/978-3-031-45611-4_5
Refining the Evaluation of Speech SynthesisPreprint
Perrotin, O., Stephenson, B., Gerber, S., Bailly, G., & King, S. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4870649
Bellinghausen, C., Schröder, B., Rauh, R., Riedel, A., Dahmen, P., Birkholz, P., van Elst, L. T., & Fangmeier, T. (2024) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2024.1347913 · Cited by 2
Special Issue on Speech Diversity and Its Applications
Mori, H. (2024) · Nippon Onkyo Gakkaishi/Acoustical science and technology/Nihon Onkyo Gakkaishi · doi:10.1250/ast.e25.001
Data Augmentation for Deep Learning-Based Speech Reconstruction Using FOC-Based Methods
Yazgaç, B. G., & Kırcı, M. (2025) · Fractal and Fractional · doi:10.3390/fractalfract9020056
Phonetic reduction is associated with positive assessment and other pragmatic functions
Ward, N. G., Gomez, R. O., Ortega, C. A., & Bugarini, G. (2025) · Speech Communication · doi:10.1016/j.specom.2025.103305
Egbunu, C. O., Rosdi, F., Nasrudin, M. F., & Rahman, A. H. A. (2025) · doi:10.1109/iceei68459.2025.11331223
“[7], [18] highlighted a fragmented understanding due to evaluating isolated aspects in TTS. Notably, [19] stressed the need for comprehensive multi-metric evaluation of TTS systems.”
Lenglet, M., Perrotin, O., & Bailly, G. (2026) · Computer Speech & Language · doi:10.1016/j.csl.2026.101985 · Cited by 4
Can self-supervised speech models predict the perceived acceptability of prosodic variation?
Wallbridge, S., Adigwe, A., & Bell, P. (2025) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru65441.2025.11434724
“…recognise that widely used subjective paradigms like Mean Opinion Score (MOS) or Multiple Stimuli with Hidden Reference and Anchor (MUSHRA)—which aim to capture fidelity or fluency—cannot assess whether a particular prosodic variant is more, less, or equally acceptable to another [6], [7], [8].”
Metadata
- Authors
- Wagner, P., Beskow, J., Betz, S., Edlund, J., Gustafson, J., Eje Henter, G., Le Maguer, S., Malisz, Z., Székely, É., Tånnander, C., & Voße, J.
- Affiliations
- Bielefeld University (DE)
- KTH Royal Institute of Technology (SE)
- Corresponding author
- Wagner, Petra
- ORCID
- Venue
- 10th ISCA Workshop on Speech Synthesis (SSW 10)
- Publisher
- ISCA
- Published
- 2019-09-20
- Pages
- 105-110
- OpenAlex
- W3011535310
- Cited by
- 74
- References
- 40
- Citations by year
- 2026: 1 · 2025: 9 · 2024: 12 · 2023: 17 · 2022: 12 · 2021: 11 · 2020: 12
- FWCI
- 3.35
- Type
- conference paper
- Language
- en
- Open access
- green
- nbn-resolving.org
- Licence
- cc-by-nc-nd
- Locations
- ISCA Speech Synthesis Workshop (SSW) (publishedVersion)
- PUB – Publications at Bielefeld University (Bielefeld University) (submittedVersion, cc-by-nc-nd)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Health Sciences › Health Professions › Speech and Hearing › Voice and Speech Disorders
- Keywords
- speech synthesis evaluation, speech quality assessment, state-of-the-art assessment, research program proposal, speech synthesis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.