Tacotron: Towards End-to-End Speech Synthesis
The DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.
Abstract
A text-to-speech synthesis system typically consists of multiple stages, such as a text analysis frontend, an acoustic model and an audio synthesis module.Building these components often requires extensive domain expertise and may contain brittle design choices.In this paper, we present Tacotron, an end-to-end generative text-to-speech model that synthesizes speech directly from characters.Given pairs, the model can be trained completely from scratch with random initialization.We present several key techniques to make the sequence-tosequence framework perform well for this challenging task.Tacotron achieves a 3.82 subjective 5-scale mean opinion score on US English, outperforming a production parametric system in terms of naturalness.In addition, since Tacotron generates speech at the frame level, it's substantially faster than sample-level autoregressive methods.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The frameworks including their code as well as the datasets used that are available publicly (Sotelo et al., 2017; Wang et al., 2017; Ping et al., 2018; Shen et al., 2018; Vasquez and Lewis, 2019) are more likely to be used for nefarious purposes or research, so they will be more developed.”
From Audio deepfakes: A survey (2023) · cites this paper as Wang et al., 2017
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 2×
“We further pass the processed sequence to a few fixed-width 1-D convolutions, whose outputs are added with the original input sequence via residual connections [17].”
BackgroundAdam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 2×
“We use the Adam optimizer (Kingma & Ba, 2015) with learning rate decay, which starts from 0.001 and is reduced to 0.0005, 0.0003, and 0.0001 after 500K, 1M and 2M global steps, respectively.”
MethodInfluentialBatch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818 · cited 2×
“Batch normalization [18] is used for all convolutional layers.”
MethodSequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766 · cited 2×
“In this paper, we propose Tacotron, an end-to-end generative TTS model based on the sequence-to-sequence (seq2seq) (Sutskever et al., 2014) with attention paradigm (Bahdanau et al., 2014).”
MethodSelf-citationNeural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213 · cited 4×
“The backbone of Tacotron is a seq2seq model with attention [7, 14].”
MethodInfluentialEmpirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1412.3555 · Cited by 11,008 · cited 7×
“Second, we compare with a model with the CBHG encoder replaced by a 2-layer residual GRU encoder.”
BackgroundInfluentialTensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., et al. (2016) · arXiv · doi:10.48550/arxiv.1603.04467 · Cited by 9,408 · cited 2×
“We implemented Griffin-Lim in TensorFlow [22] hence it’s also part of the model.”
MethodSelf-citationGoogle's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
Wu, Y., Schuster, M., Chen, Z., Le, Q. V., Norouzi, M., Macherey, W., Krikun, M., Cao, Y., Gao, Q., Macherey, K., et al. (2016) · arXiv · doi:10.48550/arxiv.1609.08144 · Cited by 5,319 · cited 4×
“C L ] 6 A pr or machine translation (Wu et al., 2016), TTS outputs are continuous, and output sequences are usually much longer than those of the input.”
BackgroundMethodInfluentialSelf-citationWaveNet: a generative model for raw audio
Oord, A. V. D., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1609.03499 · Cited by 4,911 · cited 4×
“It could be used to predict alternative targets such as vocoder parameters, or as a WaveNetlike neural vocoder [9, 13, 10] that synthesizes waveform samples directly.”
BackgroundMethodInfluentialListen, attend and spell: A neural network for large vocabulary conversational speech recognition
Chan, W., Jaitly, N., Le, Q., & Vinyals, O. (2016) · 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2016.7472621 · Cited by 2,226 · cited 2×
“Moreover, unlike end-to-end speech recognition [4] or machine translation [5], TTS outputs are continuous, and output sequences are usually much longer than those of the input.”
BackgroundSelf-citation
Cited by
Items 501–600 of 1,002
Building and Designing Expressive Speech Synthesis
Aylett, M. P., Clark, L., Cowan, B. R., & Torre, I. (2021) · ACM eBooks · doi:10.1145/3477322.3477329 · Cited by 22
Integrated Speech and Gesture Synthesis
Wang, S., Alexanderson, S., Gustafson, J., Beskow, J., Henter, G. E., & Székely, É. (2021) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3462244.3479914 · Cited by 28
“The use of machine learning to generate gestures is relatively more recent, and there are also hybrid systems that combine learned and procedural approaches, e.g., by learning when to produce a gesture from a fixed set of pre-animated gesture clips [10, 20, 27, 39].”
Stanton, D., Shannon, M., Mariooryad, S., Skerry-Ryan, R., Battenberg, E., Bagby, T., & Kao, D. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747345 · Cited by 36
Self-citationLee, J. W., Kim, E., Koo, J., & Lee, K. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-11460 · Cited by 33
“Text-to-speech (TTS) [5, 6, 7] and voice conversion (VC) [8, 9, 10] studies have advanced to the extent that they could produce perceptually almost indistinguishable from bona fide human speech in terms of naturalness and similarity.”
BackgroundInfluentialAttacker Attribution of Audio Deepfakes
Müller, N., Diekmann, F., & Williams, J. R. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-129 · Cited by 30
SQuId: Measuring Speech Naturalness in Many Languages
Sellam, T., Bapna, A., Camp, J., Mackinnon, D., Parikh, A. P., & Riesa, J. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10094909 · Cited by 32
“The target MOS ratings are linearly rescaled from [1, 5] to [0, 1].”
BackgroundShirahata, Y., Yamamoto, R., Song, E., Terashima, R., Kim, J.-M., & Tachibana, K. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10096480 · Cited by 32
DFADD: The Diffusion and Flow-Matching Based Audio Deepfake Dataset
Du, J., Lin, I.-M., Chiu, I.-H., Chen, X., Wu, H., Ren, W., Tsao, Y., Lee, H.-Y., & Jang, J. R. (2024) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt61566.2024.10832250 · Cited by 54
“Tacotron 1/2 [2, 3] are early RNN-based TTS systems that significantly improved speech quality compared to previous meth-ods.”
BackgroundLiu, D.-R., Yang, C.-Y., Wu, S.-L., & Lee, H.-Y. (2018) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt.2018.8639672 · Cited by 25
“Post processing net, which is a CBHG module [4], generates the linear spectrogram ẑ from ŷ.”
BackgroundMethodResultInfluentialParametric Resynthesis With Neural Vocoders
Maiti, S., & Mandel, M. (2019) · IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA) · doi:10.1109/waspaa.2019.8937165 · Cited by 21
EMPHASIS: An Emotional Phoneme-based Acoustic Model for Speech Synthesis System
Li, H., Kang, Y., & Wang, Z. (2018) · Interspeech (USB) · doi:10.21437/interspeech.2018-1511 · Cited by 21
Emotional Speech Synthesis Based on Style Embedded Tacotron2 Framework
Kwon, O., Jang, I., Ahn, C., & Kang, H.-G. (2019) · International Technical Conference on Circuits/Systems, Computers and Communications (ITC-CSCC) · doi:10.1109/itc-cscc.2019.8793393 · Cited by 21
“Tacotron1 framework, proposed as a complete toward end-to-end TTS system [7], directly generates mel-spectrogram from given input text sequence.”
MethodLearning Hierarchical Representations for Expressive Speaking Style in End-to-End Speech Synthesis
An, X., Wang, Y., Yang, S., Ma, Z., & Xie, L. (2019) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru46091.2019.9003859 · Cited by 25
“We are now able to produce natural prosody with high audio fidelity using a much simplified voice building pipeline [1, 2, 3, 4].”
BackgroundMethodInfluentialSelf-citationCode-Switched Speech Synthesis Using Bilingual Phonetic Posteriorgram with Only Monolingual Corpora
Cao, Y., Liu, S., Wu, X., Kang, S., Liu, P., Wu, Z., Liu, X., Su, D., Yu, D., & Meng, H. M. L. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9053094 · Cited by 21
Zhang, J., Liu, L., Chen, Y., Hu, Y.-J., Jiang, Y., Ling, Z.-H., & Dai, L.-R. (2020) · doi:10.21437/vcc_bc.2020-16 · Cited by 20
Multilingual Speech Synthesis for Voice Cloning
Seong, J.-W., Lee, W., & Lee, S. (2021) · IEEE International Conference on Big Data and Smart Computing (BigComp) · doi:10.1109/bigcomp51126.2021.00067 · Cited by 21
“The next series, Deep Voice2[5], was expanded to show better sound quality while learning the voices of multiple speakers by using vectors representing the speaker's information, and also presented how to extend the Tacotron[6] model to multi-speaker.”
BackgroundExtracting and Predicting Word-Level Style Variations for Speech Synthesis
Zhang, Y., & Ling, Z.-H. (2021) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2021.3074757 · Cited by 23
Enriching Source Style Transfer in Recognition-Synthesis Based Non-Parallel Voice Conversion
Wang, Z., Zhou, X., Yang, F., Li, T., Du, H., Xie, L., Gan, W., Chen, H., & Li, H. H. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-1351 · Cited by 26
NANSY++: Unified Voice Synthesis with Neural Analysis and Synthesis
Choi, H.-S., Yang, J., Lee, J., & Kim, H. (2022) · arXiv · doi:10.48550/arxiv.2211.09407 · Cited by 20
“), and converting the mel spectrogram into a waveform using a vocoder (Wang et al., 2017; Kim et al., 2020; Jeong et al., 2021; Lee et al., 2019; Liu et al., 2022).”
MethodVisualtts: TTS with Accurate Lip-Speech Synchronization for Automatic Voice Over
Lu, J., Şişman, B., Liu, R., Zhang, M., & Li, H. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9746421 · Cited by 35
Data Augmentation for Low-Resource Quechua ASR Improvement
Zevallos, R., Bel, N., Cámbara, G., Farrús, M., & Luque, J. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-770 · Cited by 28
A real-time voice cloning system with multiple algorithms for speech quality improvement
Hu, W., & Zhu, X. (2023) · PLoS ONE · doi:10.1371/journal.pone.0283440 · Cited by 26
“In 2017, Google reached a major milestone in speech synthesis, developing the first end-to-end speech model called Tacotron [6].”
BackgroundImaginary Voice: Face-Styled Diffusion Model for Text-to-Speech
Lee, J.-Y., Chung, J. S., & Chung, S.-W. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10094745 · Cited by 60
“There have been many approaches using generative modeling methods [1, 2, 19], and Tacotron-based models [3, 20] incorporate a sequence-to-sequence model to transform the text sequences into the acoustic representations.”
BackgroundMethodEnd-to-End Code-Switching TTS with Cross-Lingual Language Model
Zhou, X., Tian, X., Lee, G., Das, R. K., & Li, H. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9054722 · Cited by 30
“In the context of CS TTS [10], two kinds of encoders are explored to handle alphabetic inputs from different languages: shared multilingual encoder with explicit language embedding and separated monolingual encoder, both of which produce more natural CS utterances than Tacotron [11].”
BackgroundMethodGeneric Indic Text-to-Speech Synthesisers with Rapid Adaptation in an End-to-End Framework
Prakash, A., & Murthy, H. A. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-2663 · Cited by 20
“The current state-of-the-art text-to-speech (TTS) paradigm is end-to-end speech (E2E) synthesis [1, 2, 3].”
A Preprocessing Strategy for Denoising of Speech Data Based on Speech Segment Detection
Lee, S., & Kwon, H.-Y. (2020) · Applied Sciences · doi:10.3390/app10207385 · Cited by 20
“Wavenet is an internal speech DNN model within a voice synthesis model, Tacotron [18], to create raw sound waveforms.”
MethodSpeech Bert Embedding for Improving Prosody in Neural TTS
Chen, L., Deng, Y., Wang, X., Soong, F. K., & He, L. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9413864 · Cited by 25
“In recent years, the fast development of neural end-to-end models[1, 2, 3, 4] has boosted TTS performance.”
BackgroundResMax: Detecting Voice Spoofing Attacks with Residual Network and Max Feature Map
Kwak, I., Kwag, S., Lee, J., Huh, J. H., Lee, C.-H., Jeon, Y., Hwang, J., & Yoon, J. W. (2021) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr48806.2021.9412165 · Cited by 27
“IK was supported by the National Research Foundation of Korea(NRF) grant funded by Ministry of Science and ICT (2020R1C1C1A01013020) References [1] Y. Wang, R. Skerry-Ryan, D. Stanton, Y. Wu, R. J. Weiss, N. Jaitly, Z. Yang, Y. Xiao, Z. Chen, S. Bengio, Q. Le, Y. Agiomyrgiannakis, R. Clark, and R. A. Saurou, “Tacotron…”
MethodZhang, J., Richmond, K., Ling, Z.-H., & Dai, L.-R. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i16.17693 · Cited by 23
Rich Prosody Diversity Modelling with Phone-Level Mixture Density Network
Du, C., & Yu, K. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-802 · Cited by 26
“Neural text-to-speech(TTS) synthesis models with sequence-tosequence architecture [1, 2, 3] can be applied to generate naturally sounding speech.”
BackgroundTransformer-Based Text-to-Speech with Weighted Forced Attention
Okamoto, T., Toda, T., Shiga, Y., & Kawai, H. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9053915 · Cited by 20
“A reduction factor provided in [5] is also efficient for Transformer-based TTS.”
MethodTalkNet: Fully-Convolutional Non-Autoregressive Speech Synthesis ModelPreprint
Beliaev, S., Rebryk, Y., & Ginsburg, B. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.05514 · Cited by 26
“Tacotron [12, 4] is an end-to-end NN which takes characters as input and directly outputs the melspectrogram.”
BackgroundA review of deep learning based multimodal forgery detection for video and audio
Tan, D., Yang, Y., Niu, C., Li, S., Yang, D., & Tan, B. (2025) · Discover Applied Sciences · doi:10.1007/s42452-025-07629-3 · Cited by 23
“Speech synthesis technology aims to generate natural and fluent voice content from text or voice features, such as models based on Tacotron [28] and FastSpeech [29].”
Saleem, N., Gao, J., Irfan, M., Verdú, E., & Fuente, J. P. (2022) · Image and Vision Computing · doi:10.1016/j.imavis.2022.104389 · Cited by 21
Cross-Speaker Emotion Transfer Through Information Perturbation in Emotional Speech Synthesis
Lei, Y., Yang, S., Zhu, X., Xie, L., & Su, D. (2022) · IEEE Signal Processing Letters · doi:10.1109/lsp.2022.3203888 · Cited by 24
“B ENEFITING from the recent advances of neural sequence-to-sequencemodels,text-to-speech(TTS)hasmadesignif-icant process in generating high-quality natural speech [1], [2], [3], [4], [5], [6].”
BackgroundSimple and Effective Unsupervised Speech Synthesis
Liu, A. H., Lai, C.-I., Hsu, W.-N., Auli, M., Baevski, A., & Glass, J. R. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-11071 · Cited by 24
Localizing Fake Segments in Speech
Zhang, B., & Sim, T. (2022) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr56361.2022.9956134 · Cited by 28
“In the literature, there are 4 types of spoofing attack: impersonation [3], synthetic speech [4] [5], voice conversion [6] and replay attack [7].”
BackgroundECOGEN : Bird sounds generation using deep learning
Guei, A., Christin, S., Lecomte, N., & Hervet, É. (2023) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.14239 · Cited by 24
“Currently, the most efficient deep learning methods in these fields are as fol - lows: Wavenet (Oord, Dieleman, et al., 2016 ), Tacatron 1 (Wang et al., 2017 ), Tacotron 2 (Elias et al., 2021 ) and lately Jukebox (Dhariwal et al., 2020 ).”
MethodEfficientTTS 2: Variational End-to-End Text-to-Speech Synthesis and Voice Conversion
Miao, C., Zhu, Q., Chen, M., Ma, J., Wang, S., & Xiao, J. Z. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3369528 · Cited by 20
“The conventional neural TTS systems cascade two separate models: an acoustic model that transforms the input text sequences into acoustic features (e.g., mel-spectrogram) [1], [5], followed by a neural vocoder that transforms the acoustic features into audio waveforms [7], [8].”
BackgroundMethodJules, T. D., & Salajan, F. D. (2024) · Comparative Education Review · doi:10.1086/728688 · Cited by 30
“Such language models include GPT-2, GPT-3, ChatGPT, and Text-to-Speech models like Tacotron (Wang et al. 2017).”
MethodPreference Alignment Improves Language Model-Based TTS
Tian, J., Zhang, C., Shi, J., Zhang, H., Yu, J., Watanabe, S., & Yu, D. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10890510 · Cited by 38
On Using Backpropagation for Speech Texture Generation and Voice Conversion
Chorowski, J., Weiss, R. J., Saurous, R. A., & Samy Bengio, A. (2018) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2018.8461282 · Cited by 19
Self-citationA Comparison of Letters and Phones as Input to Sequence-to-Sequence Models for Speech Synthesis
Fong, J., Taylor, J., Richmond, K., & King, S. (2019) · ISCA Speech Synthesis Workshop (SSW) · doi:10.21437/ssw.2019-40 · Cited by 27
Fast Query-by-example Speech Search using Attention-based Deep Binary Embeddings
Yuan, Y., Xie, L., Leung, C.-C., Chen, H., & Ma, B. (2020) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2020.2998277 · Cited by 19
“Note that the use of attention mechanism has recently pushed the frontiers of many tasks, including neural machine translation (NMT) [19]–[21], image captioning [22], [23], speech recognition [24]–[27] and synthesis [28]–[30].”
BackgroundExploring Emotional Prototypes in a High Dimensional TTS Latent Space
van Rijn, P., Mertes, S., Schiller, D., Harrison, P. M. C., Larrouy-Maestri, P., André, E., & Jacoby, N. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-1538 · Cited by 21
SoundChoice: Grapheme-to-Phoneme Models with Semantic Disambiguation
Ploujnikov, A., & Ravanelli, M. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-11066 · Cited by 19
CatETA: A Categorical Approximate Approach for Estimating Time of Arrival
Ye, Y., Zhu, Y., Markos, C., & Yu, J. J. Q. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2022.3207894 · Cited by 19
“In this study, we follow the design of previous ETA studies [13], [42] that utilize bidirectional recurrent layers as the feature extraction component and stack two GRU layers to capture both forward and backward data propagation in time [43].”
MethodNovitasari, S., Tjandra, A., Sakti, S., & Nakamura, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.02128 · Cited by 19
STATNet: Spectral and Temporal features based Multi-Task Network for Audio Spoofing Detection
Ranjan, R., Vatsa, M., & Singh, R. (2022) · 2022 IEEE International Joint Conference on Biometrics (IJCB) · doi:10.1109/ijcb54206.2022.10007949 · Cited by 18
“The ongoing research in TTS [11, 12, 13] systems and Voice cloning [14, 15, 16] systems have made it easier for attackers to generate high quality spoofed samples.”
BackgroundReal-time Detection of AI-Generated Speech for DeepFake Voice Conversion
Bird, J. J., & Lotfi, A. (2023) · arXiv · doi:10.48550/arxiv.2308.12734 · Cited by 18
“Google introduced a speech synthesis model named Tacotron in 2017 [12], and subsequent research introduced prosody [13] (intonation and stress), style control[14], multi-speaker synthesis [15], among many more advances.”
MethodA Survey of Large Language Model for Drug Research and Development
Guo, H., Xing, X., Zhou, Y., Jiang, W., Chen, X., Wang, T., Jiang, Z., Wang, Y., Hou, J., Jiang, Y., et al. (2025) · IEEE Access · doi:10.1109/access.2025.3552256 · Cited by 24
“Tacotron [121] and Tacotron-2 [100], designed for text-to-speech (TTS) synthesis, follow the encoder-decoder architecture but differ from conventional Seq2Seq models by incorporating a Post-net component to enhance output quality.”
Self-citationBhushan, S., Mishra, V. P., Rishiwal, V., Sharmila, A., & Agarwal, U. (2025) · IEEE Access · doi:10.1109/access.2025.3605236 · Cited by 22
“Later innovations improved naturalness and intelligibility using FastSpeech [4] and Tacotron [22], advancing the synthesis process.”
Introduction to Voice Presentation Attack Detection and Recent Advances
Sahidullah, M., Delgado, H., Todisco, M., Nautsch, A., Wang, X., Kinnunen, T., Evans, N., Yamagishi, J., & Lee, K. A. (2023) · Advances in computer vision and pattern recognition · doi:10.1007/978-981-19-5288-3_13 · Cited by 22
Wu, Y.-C., Hayashi, T., Tobing, P. L., Kobayashi, K., & Toda, T. (2019) · Interspeech (USB) · doi:10.21437/interspeech.2019-1232 · Cited by 18
Self-citationCampNet: Context-Aware Mask Prediction for End-to-End Text-Based Speech Editing
Wang, T., Yi, J., Fu, R., Tao, J., & Wen, Z. (2022) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2022.3190717 · Cited by 32
Human-computer interaction for virtual-real fusion
Tao, J., Gong, J., Gao, N., Fu, S., Liang, S., & Yu, C. (2023) · Journal of Image and Graphics · doi:10.11834/jig.230020 · Cited by 22
E3 TTS: Easy End-to-End Diffusion-Based Text To Speech
Gao, Y., Morioka, N., Zhang, Y., & Chen, N. (2023) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru57964.2023.10389766 · Cited by 53
“Like the typical approach in autoregressive TTS [31, 32], we adopt a cross-attention to extract information from BERT output in the top downsampling/upsampling blocks.”
MethodAccurate synthesis of dysarthric Speech for ASR data augmentation
Soleymanpour, M., Johnson, M. T., Soleymanpour, R., & Berry, J. (2024) · Speech Communication · doi:10.1016/j.specom.2024.103112 · Cited by 28
SAFARI: Speech-Associated Facial Authentication for AR/VR Settings via Robust VIbration Signatures
Zhang, T., Ji, Q., Ye, Z., Akanda, M. M. R. R., Mahdad, A. T., Shi, C., Wang, Y., Saxena, N., & Chen, Y. J. (2024) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3658644.3670358 · Cited by 19
Self-citationImproving FFTNet Vocoder with Noise Shaping and Subband Approaches
Okamoto, T., Toda, T., Shiga, Y., & Kawai, H. (2018) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt.2018.8639687 · Cited by 18
“…feature extraction [9], glottal vocoder [10, 11], power spectrum reconstruction for vocoded speech [12], and speech wave-form generation [13, 14] from power spectrums estimated using the Griffin–Lim algorithm [15], have been investigated as in deep learning-based acoustic models for SPSS and…”
MethodEnhancing Hybrid Self-attention Structure with Relative-position-aware Bias for Speech Synthesis
Yang, S., Lu, H., Shi-ying, K., Xie, L., & Yu, D. (2019) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2019.8682861 · Cited by 18
“The emergence of these seq2seq based speech synthesis models such as Tacotron [8] discard the independently designed text analyzer, and replace the traditional aligned linguistic frame to acoustic frame mapping networks by an encoder-decoder paradigm.”
BackgroundMethodInfluentialWang, Y., Wang, W., Liang, W. G., & Yu, L.-F. (2019) · ACM Transactions on Graphics · doi:10.1145/3355089.3356487 · Cited by 18
“GSTisdevisedupona populartext-to-speechmodel,Tacotron[Shen et al. 2018; Wang et al. 2017], which predicts mel spectrograms directly from grapheme inputs.”
BackgroundSelf-citationHigh-Fidelity Audio Generation and Representation Learning With Guided Adversarial Autoencoder
Haque, K. N., Rana, R. K., & Schuller, B. W. (2020) · IEEE Access · doi:10.1109/access.2020.3040797 · Cited by 20
“Other researchers have utilised the seq2seq model for TTS such as Char2Wav [52] and TACOTRON [53].”
MethodImproved Parallel Wavegan Vocoder with Perceptually Weighted Spectrogram Loss
Song, E., Yamamoto, R., Hwang, M.-J., Kim, J., Kwon, O., & Kim, J.-M. (2021) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt48900.2021.9383549 · Cited by 18
TTS-Portuguese Corpus: a corpus for speech synthesis in Brazilian Portuguese
Casanova, E., Cândido, A., Shulby, C., de Oliveira, F. S., Teixeira, J. P., Ponti, M. A., & Aluísio, S. M. (2022) · Language Resources and Evaluation · doi:10.1007/s10579-021-09570-4 · Cited by 18
Social robotic application to support active and healthy ageing
Cooper, S., Fava, A. D., Villacañas, Ó., Silva, T., Fernández-Carbajales, V., Unzueta, L., Serras, M., Marchionni, L., & Ferro, F. (2021) · IEEE RO-MAN · doi:10.1109/ro-man50785.2021.9515432 · Cited by 21
Disentangling Style and Speaker Attributes for TTS Style Transfer
An, X., Soong, F. K., & Xie, L. (2022) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2022.3145297 · Cited by 26
“E ND-TO-END neural TTS models, such as Tacotron 2 [1], can produce high quality speech with naturalness close to that of human speakers [2]–[5].”
Language Model-Based Emotion Prediction Methods for Emotional Speech Synthesis Systems
Yoon, H.-W., Kwon, O., Lee, H., Yamamoto, R., Song, E., Kim, J., & Hwang, M.-J. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-11133 · Cited by 23
ParaTTS: Learning Linguistic and Prosodic Cross-Sentence Information in Paragraph-Based TTS
Xue, L., Soong, F. K., Zhang, S., & Xie, L. (2022) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2022.3202126 · Cited by 26
“The development of sequence-to-sequence (seq2seq) based neural acoustic models [1, 2, 3, 4, 5, 6] and neural vocoders [7, 8, 9, 10], brought a significant improvement to the text-tospeech (TTS) synthesis quality, which allows to automatically”
BackgroundLi, J., Meng, Y., Wu, X., Wu, Z., Jia, J., Meng, H. M. L., Tian, Q., Wang, Y., & Wang, Y. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3547831 · Cited by 24
“The local encoder consists of the pre-net and CBHG networks in Tacotron [37].”
MethodSelf-citationUnsupervised TTS Acoustic Modeling for TTS With Conditional Disentangled Sequential VAE
Lian, J., Zhang, C., Anumanchipalli, G. K., & Yu, D. (2023) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2023.3290423 · Cited by 24
“Conditional Disentangled Sequential VAE Jiachen Lian , Member, IEEE , Chunlei Zhang , Member, IEEE , Gopala K. Anumanchipalli , Member, IEEE , and Dong Yu , Fellow, IEEE Abstract —”
BackgroundMonteiro, J., Alam, J., & Falk, T. H. (2020) · ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp40776.2020.9054558 · Cited by 17
“The latest LA attacks have taken advantage of recent advances in speech synthesis and voice conversion based on auto-regressive waveform modeling or generative adversarial networks (GANs) [3, 4, 5].”
BackgroundAdapting TTS models For New Speakers using Transfer LearningPreprint
Neekhara, P., Li, J., & Ginsburg, B. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.05798 · Cited by 23
“While such neural TTS models [7, 3] have shown promise in synthesizing natural sounding speech, they cannot be directly used to synthesize speech for a new speaker.”
BackgroundWu, P., Pan, J., Xu, C., Zhang, J., Wu, L., Yin, X., & Ma, Z. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.04153 · Cited by 18
U Can't Gen This? A Survey of Intellectual Property Protection Methods for Data in Generative AIPreprint
Šarčević, T., Karlowicz, A., Mayer, R. L., Baeza-Yates, R. A., & Rauber, A. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2406.15386 · Cited by 23
“TTS techniques are more complex systems consisting of text analysis, acoustic model and audio synthesis [57], [58].”
MethodPronunciation trainer for second language learning using generative AI
Sungkur, R. K., & Shibdeen, N. (2025) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-025-00561-x · Cited by 17
“Tacotron and Tacotron 2 are two end-to-end neural network models that transform text into mel-spectrograms, which are then turned into speech using vocoders such as WaveNet (Paneru et al., 2024; Shen et al., 2018).”
InfluentialAzizah, K., & Jatmiko, W. (2022) · IEEE Access · doi:10.1109/access.2022.3141200 · Cited by 22
LightGrad: Lightweight Diffusion Probabilistic Model for Text-to-Speech
Chen, J., Song, X., Peng, Z., Zhang, B., Pan, F., & Wu, Z. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10096710 · Cited by 26
“Although autoregressive neural TTS models, such as Tacotron [1] and Transformer TTS [2] have shown superior performance, they suffer from slow inference speed and a lack of robustness.”
MethodCML-TTS: A Multilingual Dataset for Speech Synthesis in Low-Resource Languages
de Oliveira, F. S., Casanova, E., Cândido, A., Soares, A. S., & Filho, A. R. G. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-40498-6_17 · Cited by 38
Doctors’ perceptions of using their digital twins in patient care
Zalake, M. S. (2023) · Scientific Reports · doi:10.1038/s41598-023-48747-5 · Cited by 18
Xue, J., Deng, Y., Gao, Y., & Li, Y. (2024) · Interspeech (USB) · doi:10.21437/interspeech.2024-980 · Cited by 23
Hashmi, A., Shahzad, S. A., Lin, C., Tsao, Y., & Wang, H. (2026) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2026.3656145 · Cited by 23
“Other DL-based methods include WaveNet [42], Deep Voice [43], Tacotron [44], and NaturalSpeech [45].”
Improving Mandarin End-to-End Speech Synthesis by Self-Attention and Learnable Gaussian Bias
Yang, F., Yang, S., Zhu, P., Yan, P., & Xie, L. (2019) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru46091.2019.9003949 · Cited by 17
“For the encoder part in SAE-Tacotron, it contains 3 feedforward layers with Relu activation as pre-net and dropout regularization as described in Tacotron [4].”
BackgroundMethodGraphTTS: Graph-to-Sequence Modelling in Neural Text-to-Speech
Sun, A., Wang, J., Cheng, N., Peng, H., Zeng, Z., & Xiao, J. Z. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9053355 · Cited by 23
“Neural approaches, specifically sequence-tosequence models, make the development of TTS to a higher level after the announcement of Google’s Tacotron[1] and Wavenet[2][3] model.”
MethodSelf-supervised Pitch Detection by Inverse Audio Synthesis
Engel, J., Swavely, R., Hantrakul, L., Roberts, A. P., & Hawthorne, C. (2020) · Cited by 17
Unsupervised Acoustic Unit Representation Learning for Voice Conversion Using WaveNet Auto-Encoders
Chen, M., & Hain, T. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-1785 · Cited by 17
Onoma-to-wave: Environmental Sound Synthesis from Onomatopoeic Words
Okamoto, Y., Imoto, K., Takamichi, S., Yamanishi, R., Fukumori, T., & Yamashita, Y. (2022) · APSIPA Transactions on Signal and Information Processing · doi:10.1561/116.00000049 · Cited by 23
Light-TTS: Lightweight Multi-Speaker Multi-Lingual Text-to-Speech
Li, S., Ouyang, B., Li, L., & Hong, Q. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414400 · Cited by 23
“Secondly, we employ a multi-frame output strategy [2] that makes the decoder output two Mel-spectrum frames at each time step, so as to enhance the contextual relevance and semantic integrity of the Mel-spectrum generated by the speech synthesis model.”
BackgroundMethodAudiovisual Speech Synthesis using Tacotron2
Abdelaziz, A. H., Kumar, A. P., Seivwright, C., Fanelli, G., Binder, J., Stylianou, Y., & Kajareker, S. (2021) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3462244.3479883 · Cited by 17
“Deep neural networks, in particular sequence-to-sequence architectures [22, 33], produce almost human-like synthesized speech.”
BackgroundSource Tracing: Detecting Voice Spoofing
Zhu, T., Wang, X., Qin, X., & Li, M. (2022) · Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) · doi:10.23919/apsipaasc55919.2022.9980129 · Cited by 32
“Many notable high-performance TTS and VC systems includes Tacotron systems [4], [5], Fast Speech systems [6], [7], VITS system [8], DelightfulTTS [9], etc.”
BackgroundExploring End-To-End Attention-Based Neural Networks For Native Language Identification
Ubale, R., Qian, Y., & Evanini, K. (2018) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt.2018.8639689 · Cited by 19
“Furthermore, end-to-end sequence-to-sequence networks have been applied successfully to text-to-speech synthesis [20, 21, 22] to convert character sequences to mel spectrogram which is then converted to speech.”
MethodDeep Generative Models for Musical Audio Synthesis
Huzaifah, M., & Wyse, L. L. (2021) · Springer eBooks · doi:10.1007/978-3-030-72116-9_22 · Cited by 17
Modern speech synthesis for phonetic sciences: a discussion and an evaluationPreprint
Malisz, Z., Henter, G. E., Valentini-Botinhao, C., Watts, O., Beskow, J., & Gustafson, J. (2020) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/dxvhc · Cited by 17
“Since 2016, important breakthroughs have come from introducing deep learning for generating audio wave-forms, i.e., neural vocoders like WaveNet [26], and for analysing text input (Tacotron [36]).”
MethodInfluentialCFAD: A Chinese Dataset for Fake Audio DetectionPreprint
Ma, H., Yi, J., Wang, C., Yan, X., Tao, J., Wang, T., Wang, S., & Fu, R. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2207.12308 · Cited by 23
“But the acoustic model of F11 is Tacotron2 [48].”
BackgroundSpeakStream: Streaming Text-to-Speech with Interleaved DataPreprint
Bai, R. H., Gu, Z., Likhomanenko, T., & Jaitly, N. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2505.19206 · Cited by 19
“Traditional TTS systems [11]–[15] process complete text to generate complete audio.”
Self-citationText-Free Prosody-Aware Generative Spoken Language Modeling
Kharitonov, E., Lee, A., Polyak, A., Adi, Y., Copet, J., Lakhotia, K., Nguyen, T.-A., Rivière, M., Mohamed, A., Dupoux, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2109.03264 · Cited by 16
“As FastSpeech and FastPitch are designed to improve the inference-time efficiency from auto-regressive models like Tacotron (Wang et al., 2017), they predict prosodic features and spectrograms without introducing dependency between time steps.”
MethodThe Oxford Handbook of Expressivity
Foolen, A. P. (2025) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780198869450.001.0001 · Cited by 71
More than Words: In-the-Wild Visually-Driven Prosody for Text-to-Speech
Hassid, M., Ramanovich, M. T., Shillingford, B., Wang, M., Ye, J., & Remez, T. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01033 · Cited by 28
“Methods have shifted from parametric models towards increasingly end-to-end neural networks [6, 7].”
MethodImprove Few-Shot Voice Cloning Using Multi-Modal Learning
Zhang, H., & Lin, Y. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9746233 · Cited by 18
An Overview & Analysis of Sequence-to-Sequence Emotional Voice Conversion
Yang, Z., Jing, X., Triantafyllopoulos, A., Song, M., Aslan, I., & Schuller, B. W. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-10636 · Cited by 23
Self-citation
Metadata
- Authors
- Wang, Y., Skerry-Ryan, R. J., Stanton, D., Wu, Y., Weiss, R. J., Jaitly, N., Yang, Z., Xiao, Y., Chen, Z., Bengio, S., Le, Q., Agiomyrgiannakis, Y., Clark, R., & Saurous, R. A.
- Affiliations
- Google (United States) (US)
- Corresponding author
- Wang, Yuxuan
- ORCID
- Venue
- Interspeech 2017
- Publisher
- ISCA
- Published
- 2017-08-20
- Pages
- 4006-4010
- ISSN
- 2308-457X (ISSN-L)
- OpenAlex
- W2963609956
- Cited by
- 1,746
- References
- 25
- Citations by year
- 2026: 134 · 2025: 187 · 2024: 183 · 2023: 215 · 2022: 243 · 2021: 301 · 2020: 252 · 2019: 176 · 2018: 45 · 2017: 9
- FWCI
- 60.12
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Interspeech (USB) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Health Sciences › Health Professions › Speech and Hearing › Voice and Speech Disorders
- Keywords
- Tacotron, end-to-end speech synthesis, text-to-speech, sequence-to-sequence models, naturalness
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.