Tacotron: Towards End-to-End Speech Synthesis
The DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.
Abstract
A text-to-speech synthesis system typically consists of multiple stages, such as a text analysis frontend, an acoustic model and an audio synthesis module.Building these components often requires extensive domain expertise and may contain brittle design choices.In this paper, we present Tacotron, an end-to-end generative text-to-speech model that synthesizes speech directly from characters.Given pairs, the model can be trained completely from scratch with random initialization.We present several key techniques to make the sequence-tosequence framework perform well for this challenging task.Tacotron achieves a 3.82 subjective 5-scale mean opinion score on US English, outperforming a production parametric system in terms of naturalness.In addition, since Tacotron generates speech at the frame level, it's substantially faster than sample-level autoregressive methods.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The frameworks including their code as well as the datasets used that are available publicly (Sotelo et al., 2017; Wang et al., 2017; Ping et al., 2018; Shen et al., 2018; Vasquez and Lewis, 2019) are more likely to be used for nefarious purposes or research, so they will be more developed.”
From Audio deepfakes: A survey (2023) · cites this paper as Wang et al., 2017
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 2×
“We further pass the processed sequence to a few fixed-width 1-D convolutions, whose outputs are added with the original input sequence via residual connections [17].”
BackgroundAdam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 2×
“We use the Adam optimizer (Kingma & Ba, 2015) with learning rate decay, which starts from 0.001 and is reduced to 0.0005, 0.0003, and 0.0001 after 500K, 1M and 2M global steps, respectively.”
MethodInfluentialBatch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818 · cited 2×
“Batch normalization [18] is used for all convolutional layers.”
MethodSequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766 · cited 2×
“In this paper, we propose Tacotron, an end-to-end generative TTS model based on the sequence-to-sequence (seq2seq) (Sutskever et al., 2014) with attention paradigm (Bahdanau et al., 2014).”
MethodSelf-citationNeural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213 · cited 4×
“The backbone of Tacotron is a seq2seq model with attention [7, 14].”
MethodInfluentialEmpirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1412.3555 · Cited by 11,008 · cited 7×
“Second, we compare with a model with the CBHG encoder replaced by a 2-layer residual GRU encoder.”
BackgroundInfluentialTensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., et al. (2016) · arXiv · doi:10.48550/arxiv.1603.04467 · Cited by 9,408 · cited 2×
“We implemented Griffin-Lim in TensorFlow [22] hence it’s also part of the model.”
MethodSelf-citationGoogle's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
Wu, Y., Schuster, M., Chen, Z., Le, Q. V., Norouzi, M., Macherey, W., Krikun, M., Cao, Y., Gao, Q., Macherey, K., et al. (2016) · arXiv · doi:10.48550/arxiv.1609.08144 · Cited by 5,319 · cited 4×
“C L ] 6 A pr or machine translation (Wu et al., 2016), TTS outputs are continuous, and output sequences are usually much longer than those of the input.”
BackgroundMethodInfluentialSelf-citationWaveNet: a generative model for raw audio
Oord, A. V. D., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1609.03499 · Cited by 4,911 · cited 4×
“It could be used to predict alternative targets such as vocoder parameters, or as a WaveNetlike neural vocoder [9, 13, 10] that synthesizes waveform samples directly.”
BackgroundMethodInfluentialListen, attend and spell: A neural network for large vocabulary conversational speech recognition
Chan, W., Jaitly, N., Le, Q., & Vinyals, O. (2016) · 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2016.7472621 · Cited by 2,226 · cited 2×
“Moreover, unlike end-to-end speech recognition [4] or machine translation [5], TTS outputs are continuous, and output sequences are usually much longer than those of the input.”
BackgroundSelf-citation
Cited by
Items 401–500 of 1,002
Yasuda, Y., Wang, X., & Yamagishi, J. (2020) · Computer Speech & Language · doi:10.1016/j.csl.2020.101183 · Cited by 36
Deng, Y., He, L., & Soong, F. K. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.05253 · Cited by 30
“Tacotron+WaveNet [1–3], Char2Wav [4], DeepVoice [5] and VoiceLoop [6].”
End-to-end Text-to-speech for Low-resource Languages by Cross-Lingual Transfer LearningPreprint
Tu, T., Chen, Y.-J., Yeh, C.-C., & Lee, H.-Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.06508 · Cited by 33
“[1] Y. Wang, R. Skerry-Ryan, D. Stanton, Y. Wu, R. J. Weiss, N. Jaitly, Z. Yang, Y. Xiao, Z. Chen, S. Bengio et al., “Tacotron: Towards end-to-end speech synthesis,” Interspeech, pp. 4006– 4010, 2017.”
BackgroundMethodInfluentialVARA-TTS: Non-Autoregressive Text-to-Speech Synthesis based on Very Deep VAE with Residual AttentionPreprint
Liu, P., Cao, Y., Liu, S., Hu, N., Li, G., Weng, C., & Su, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.06431 · Cited by 31
“We can see that inference speed of our VARA-TTS is 16x faster than that of Tacotron 2, and at the same scale as that of BVAE-TTS. trained with different values of β .”
InfluentialAhmad, S. R. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2401.01511 · Cited by 33
Hello me, meet the real me: Voice synthesis attacks on voice assistants
Bilika, D., Michopoulou, N., Alepis, E., & Patsakis, C. (2024) · Computers & Security · doi:10.1016/j.cose.2023.103617 · Cited by 27
Shahzad, S. A., Hashmi, A., Peng, Y., Tsao, Y., & Wang, H. (2025) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2025.3618409 · Cited by 31
“Audio deepfakes involve the acoustic manipulation of audio using voice conversion and text-to-speech techniques (such as WaveNet [9], WaveGlow [10], MelNet [11], and Tacotron [12]).”
Statistical Parametric Speech Synthesis Using Deep Gaussian Processes
Koriyama, T., & Kobayashi, T. (2019) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2019.2905167 · Cited by 27
“After the proposal of DNNbased speech synthesis, various extensions, such as the long short-term memory recurrent neural network (LSTM-RNN)-[2] and attention-based synthesis [3], [4], have been developed.”
MethodBuilding a Mixed-Lingual Neural TTS System with Only Monolingual Data
Xue, L., Song, W., Xu, G., Xie, L., & Wu, Z. (2019) · Interspeech (USB) · doi:10.21437/interspeech.2019-3191 · Cited by 28
Deep Audio Priors Emerge From Harmonic Convolutional Networks
Zhang, Z., Wang, Y., Gan, C., Wu, J., Tenenbaum, J. B., Torralba, A. B., & Freeman, W. T. (2020) · International Conference on Learning Representations · Cited by 27
Self-citationTrojan Attack on Deep Generative Models in Autonomous Driving
Ding, S., Tian, Y., Xu, F., Li, Q., & Zhong, S. (2019) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-030-37228-6_15 · Cited by 27
Korean Singing Voice Synthesis Based on Auto-Regressive Boundary Equilibrium Gan
Choi, S., Kim, W., Park, S., Yong, S., & Nam, J. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9053950 · Cited by 27
“Recently, end-to-end learning models based on deep neural networks have achieved remarkable results in speech synthesis [4, 5, 6], and this in turn has affected the development of neural singing synthesis [7].”
BackgroundMethodZhao, S., Nguyen, T. H., Wang, H., & Ma, B. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-1163 · Cited by 27
Prosody-TTS: An End-to-End Speech Synthesis System with Prosody Control
Pamisetty, G., & Murty, K. S. R. (2022) · Circuits Systems and Signal Processing · doi:10.1007/s00034-022-02126-z · Cited by 27
USAF: Multimodal Chinese named entity recognition using synthesized acoustic features
Liu, Y., Huang, S., Li, R., Yan, N., & Du, Z. (2023) · Information Processing & Management · doi:10.1016/j.ipm.2023.103290 · Cited by 29
Enhancing Speech Intelligibility in Text-To-Speech Synthesis Using Speaking Style Conversion
Paul, D., Shifas, M. P. V., Pantazis, Y., & Stylianou, Y. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-2793 · Cited by 27
Preech: A System for Privacy-Preserving Speech TranscriptionPreprint
Ahmed, S., Chowdhury, A. R., Fawaz, K., & Ramanathan, P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.04198 · Cited by 54
“Furthermore, extracting the biometric information from S enables critical attacks like speech synthesis and impersonation attacks [34, 55, 62, 63].”
MethodGLM-TTS Technical ReportPreprint
Cui, J., Yang, Z., Li, N., Tian, J., Ma, X., Zhang, Y., Chen, G., Yang, R., Cheng, Y., Zhou, Y., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2512.14291 · Cited by 33
“…neural acoustic models that directly predict continuous speech representations—such as attention-based sequence-to-sequence models (e.g., Tacotron (Wang et al., 2017), Transformer-TTS (Li et al., 2019)) and non-autoregressive feed-forward architectures (e.g., Fastspeech (Ren et al., 2019) series)…”
Self-citationLarge Language Model-Driven 3D Hyper-Realistic Interactive Intelligent Digital Human System
Song, Y., & Xiong, W. (2025) · Sensors · doi:10.3390/s25061855 · Cited by 26
“To facilitate TTS research, we have assembled a number of open-source implementations, such as DelightfulTTS [30,31], ESPnet-TTS [32,33], FastSpeech [34,35], Glow-TTS [36], NaturalSpeech [37,38], SpeedySpeech [39], and Tacotron [40,41].”
Collapsed Speech Segment Detection and Suppression for WaveNet Vocoder
Wu, Y.-C., Kobayashi, K., Hayashi, T., Tobing, P. L., & Toda, T. (2018) · Interspeech (USB) · doi:10.21437/interspeech.2018-1210 · Cited by 29
Self-citationEffective Deep Learning Models for Automatic Diacritization of Arabic Text
Madhfar, M. A. H., & Qamar, A. M. (2020) · IEEE Access · doi:10.1109/access.2020.3041676 · Cited by 26
“The encoder-decoder is a powerful architecture that has been used to solve many problems such as neural machine translation [1], [2], image captioning [28], and text-to-speech synthesis [3], [27].”
BackgroundMethodInfluentialCong, J., Yang, S., Xie, L., & Su, D. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-414 · Cited by 30
Li, J., Meng, Y., Li, C., Wu, Z., Meng, H. M. L., Weng, C., & Su, D. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747837 · Cited by 39
“[1] Yuxuan Wang, R. J. Skerry-Ryan, Daisy Stanton, Yonghui Wu, Ron J. Weiss, Navdeep Jaitly, Zongheng Yang, Ying Xiao, Zhifeng Chen, Samy Bengio, Quoc Le, Yannis Agiomyrgiannakis, Rob Clark, and Rif A. Saurous, “Tacotron: Towards End-to-End Speech Synthesis,” arXiv:1703.10135 [cs], Mar. 2017, arXiv: 1703.10135.”
BackgroundMethodInfluentialDuration Controllable Voice Conversion via Phoneme-Based Information Bottleneck
Lee, S.-H., Noh, H.-R., Nam, W.-J., & Lee, S. (2022) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2022.3156757 · Cited by 33
SyntaSpeech: Syntax-Aware Generative Adversarial Text-to-Speech
Ye, Z., Zhao, Z., Ren, Y., & Wu, F. Q. (2022) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/620 · Cited by 31
Kwak, I., Kwag, S., Lee, J., Jeon, Y., Hwang, J., Choi, H.-J., Yang, J., Han, S.-Y., Huh, J. H., Lee, C.-H., et al. (2023) · IEEE Access · doi:10.1109/access.2023.3275790 · Cited by 30
“To train voice models, you can use free resources like Google’s Wavenet, and Tacotron [1], [2], [3].”
MethodMilestones in speaker recognition
Sharma, R. K., Govind, D., Mishra, J., Dubey, A. K., Deepak, K. T., & Prasanna, S. R. M. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-023-10688-w · Cited by 35
“Due to recent advancements in TTS methods such as DNN-based TTS and Generative Adversarial Networks (GANs) (Saito et al. 2017), Tacotron (Wang et al. 2017), Wavenet (Oord et al. 2016), etc., a high-quality speech similar to natural speech containing prosodic content can be generated.”
Controllable Accented Text-to-Speech Synthesis With Fine and Coarse-Grained Intensity Rendering
Liu, R., Şişman, B., Gao, G., & Li, H. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3378110 · Cited by 40
Oh, H.-S., Lee, S.-H., & Lee, S. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3395994 · Cited by 40
Phase Reconstruction Based On Recurrent Phase Unwrapping With Deep Neural Networks
Masuyama, Y., Yatabe, K., Koizumi, Y., Oikawa, Y., & Harada, N. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9053234 · Cited by 26
Using Personalized Speech Synthesis and Neural Language Generator for Rapid Speaker Adaptation
Huang, Y., He, L., Wei, W., Gale, W., Li, J., & Gong, Y. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9053104 · Cited by 26
“In this paper, we propose to utilize the personalized speech synthesis [22–25] and the neural language generator [26,27] to directly address the general data sparsity issue in the rapid model adaptation.”
MethodCan Speaker Augmentation Improve Multi-Speaker End-to-End TTS?
Cooper, E., Lai, C.-I., Yasuda, Y., & Yamagishi, J. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-1229 · Cited by 29
“Recent advances in end-to-end text-to-speech (TTS) synthesis enable the production of synthetic speech of high quality and good speaker similarity [1, 2, 3, 4].”
BackgroundProsodic Feature-Based Discriminatively Trained Low Resource Speech Recognition System
Hasija, T., Kadyan, V., Guleria, K., Alharbi, A., Alyami, H., & Goyal, N. (2022) · Sustainability · doi:10.3390/su14020614 · Cited by 26
“[15] proposed an end-to-end methodology that generated synthetic speech.”
BackgroundMethodAdaDurIAN: Few-shot Adaptation for Neural Text-to-Speech with DurIANPreprint
Zhang, Z., Tian, Q., Lu, H., Chen, L.-H., & Liu, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.05642 · Cited by 29
“Instead of CBHG [3] module, we adopt a vanilla LSTM layer with a time delay of 5 frames as post-net.”
BackgroundMethodInfluentialUnified Mandarin TTS Front-end Based on Distilled BERT ModelPreprint
Zhang, Y., Deng, L., & Wang, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.15404 · Cited by 30
“The aim of this work is to model PSP and G2P simultaneously in a unified manner and provide all the necessary prosodic and pronunciation information for the TTS backend (e.g., the acoustic model Tacotron [10] and Wavenet vocoder [11]).”
MethodInfluentialSelf-citationMOSS-TTS Technical ReportPreprint
Gong, Y., Jiang, B., Zhao, Y., Yuan, Y., Chen, K., Jiang, Y. L., Chang, C., Hong, D., Chen, M., Li, R., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2603.18090 · Cited by 34
Self-citationSemi-Supervised Multimodal Emotion Recognition with Expression MAE
Cheng, Z., Lin, Y., Chen, Z., Li, X., Mao, S., Zhang, F., Ding, D., Zhang, B., & Peng, X. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612840 · Cited by 40
Self-citationGosai, A., Vuong, T., Tyagi, U., Li, S., You, W., Bavare, M., Uçar, A., 方众望, Jang, B., Liu, B., et al. (2026) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2026.acl-long.1654 · Cited by 33
Neufa: Neural Network Based End-to-End Forced Alignment with Bidirectional Attention Mechanism
Li, J., Meng, Y., Wu, Z., Meng, H. M. L., Tian, Q., Wang, Y., & Wang, Y. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747085 · Cited by 25
Self-citationDeep Learning Approaches in Topics of Singing Information Processing
Gupta, C., Li, H., & Goto, M. (2022) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2022.3190732 · Cited by 25
MetaMGC: a music generation framework for concerts in metaverse
Jin, C., Wu, F., Wang, J., Liu, Y., Guan, Z., & Han, Z. (2022) · EURASIP Journal on Audio Speech and Music Processing · doi:10.1186/s13636-022-00261-8 · Cited by 25
“More recently, neural network techniques have gained attention in audio generation as a result of the success of neural networks in speech synthesis [27].”
BackgroundAnyoneNet: Synchronized Speech and Talking Head Generation for Arbitrary Persons
Wang, X., Xie, Q., Zhu, J., Xie, L., & Scharenborg, O. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3214100 · Cited by 35
“4) follows the Tacotron’s [59] encoder that consists of a pre-net and a CBHG block.”
BackgroundMethodTriAAN-VC: Triple Adaptive Attention Normalization for Any-to-Any Voice Conversion
Park, H. J., Yang, S. W., Kim, J. S., Shin, W., & Han, S. W. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10096642 · Cited by 40
UMETTS: A Unified Framework for Emotional Text-to-Speech Synthesis with Multimodal Prompts
Li, X., Cheng, Z.-Q., He, J.-Y., Chen, J., Fan, X., Peng, X., & Hauptmann, A. G. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10889012 · Cited by 28
Yeh, C.-C., Hsu, P.-C., Chou, J.-C., Lee, H.-Y., & Lee, L.-S. (2018) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt.2018.8639647 · Cited by 29
Attention-based Wavenet Autoencoder for Universal Voice Conversion
Polyak, A., & Wolf, L. (2019) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2019.8682589 · Cited by 32
Building Multilingual End-to-End Speech Synthesisers for Indian Languages
Prakash, A., Thomas, A. L., Umesh, S., & Murthy, H. A. (2019) · ISCA Speech Synthesis Workshop (SSW) · doi:10.21437/ssw.2019-35 · Cited by 25
“With the success of attention-based sequence to sequence (seq2seq) models in an end-to-end framework, building a TTS system is simple [1, 2, 3].”
BackgroundMethodResultHUI-Audio-Corpus-German: A High Quality TTS Dataset
Puchtler, P., Wirth, J., & Peinl, R. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-87626-5_15 · Cited by 35
Multistage Deep Transfer Learning for EmIoT-Enabled Human–Computer Interaction
Liu, R., Liu, Q., Zhu, H., & Cao, H. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2022.3148766 · Cited by 25
FMFCC-A: A Challenging Mandarin Dataset for Synthetic Speech Detection
Zhang, Z., Gu, Y., Yi, X., & Zhao, X. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-95398-0_9 · Cited by 24
Deepfakes in digital media forensics: Generation, AI-based detection and challenges
Bendiab, G., Haiouni, H., Moulas, I., & Shiaeles, S. N. (2024) · Journal of Information Security and Applications · doi:10.1016/j.jisa.2024.103935 · Cited by 35
Towards Universal Text-to-Speech
Yang, J., & He, L. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-1590 · Cited by 33
“Recent years, end-to-end (E2E) models have been widely used in speech synthesis [6, 7, 8, 9, 10, 11, 12], where the models can be directly trained on text-speech pairs with minimal engineering efforts.”
MethodTan, D., & Lee, T. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-1129 · Cited by 26
Non-Autoregressive Sequence-To-Sequence Voice Conversion
Hayashi, T., Huang, W.-C., Kobayashi, K., & Toda, T. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9413973 · Cited by 30
“To reduce the length of the Mel-spectrogram, we introduce a reduction factor scheme [22], which concatenates the several frames into one frame.”
MethodGenerating synthetic dysarthric speech to overcome dysarthria acoustic data scarcity
Hu, A., Phadnis, D., & Shahamiri, S. R. (2021) · Journal of Ambient Intelligence and Humanized Computing · doi:10.1007/s12652-021-03542-w · Cited by 24
“Although it was reported that Griffin-Lim in Tacotron 1 was faster to train (Wang et al. 2017), it produced less natural speech compared to WaveNet.”
BackgroundResultInfluentialDeep Performer: Score-to-Audio Music Performance Synthesis
Dong, H. K., Zhou, C., Berg-Kirkpatrick, T., & McAuley, J. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747217 · Cited by 24
WeSinger: Data-augmented Singing Voice Synthesis with Auxiliary Losses
Zhang, Z., Zheng, Y., Li, X., & Lü, L. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-454 · Cited by 24
Autoregressive Predictive Coding: A Comprehensive Study
Yang, G.-P., Yeh, S.-L., Chung, Y.-A., Glass, J. R., & Tang, H. (2022) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3203608 · Cited by 24
“In fact, the original APC uses the (cid:2) 1 norm [15], drawing inspiration from end-to-end TTS systems [17].”
MethodTowards Building Text-to-Speech Systems for the Next Billion Users
Kumar, G. K., Praveen, S. V., Kumar, P., Khapra, M. M., & Nandakumar, K. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10096069 · Cited by 24
“Tacotron [7] was the first successful neural acoustic model to generate spectrograms from the text directly.”
BackgroundAi, Y., & Ling, Z.-H. (2023) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2023.3277276 · Cited by 24
“[2] Y. Wang et al., “Tacotron: Towards end-to-end speech synthesis,” in Proc.”
BackgroundLi, T., Hu, C., Cong, J., Zhu, X., Li, J., Tian, Q., Wang, Y., & Xie, L. (2023) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2023.3313413 · Cited by 28
“Skerry-Ryan et al. [19] integrate the Tacotron [51], [52] model with an extra prosody encoder, denoted as Reference Encoder, in which the reference is encapsulated into a fixed-length embedding that is directly concatenated with the linguistic representations.”
MethodSelf-citationPlanning the development of text-to-speech synthesis models and datasets with dynamic deep learning
Ahmad, H. A., & Rashid, T. A. (2024) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2024.102131 · Cited by 27
“A lower RMSE is desirable than a greater one (Zen and Senior, 2014; Shechtman and Mordechay, 2018; Luong et al., 2017; Chen et al., 2018; Choi et al., 2019; Lee et al., 2018; Wang et al., 2017; Sun et al., 2015).”
InfluentialSpeech Chain for Semi-Supervised Learning of Japanese-English Code-Switching ASR and TTS
Nakayama, S., Tjandra, A., Sakti, S., & Nakamura, S. (2018) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt.2018.8639674 · Cited by 24
Expediting TTS Synthesis with Adversarial Vocoding
Neekhara, P., Donahue, C., Puckette, M., Dubnov, S., & McAuley, J. (2019) · Interspeech (USB) · doi:10.21437/interspeech.2019-3099 · Cited by 24
Using generative modelling to produce varied intonation for speech synthesis
Hodari, Z., Watts, O., & King, S. (2019) · ISCA Speech Synthesis Workshop (SSW) · doi:10.21437/ssw.2019-43 · Cited by 27
Tacotron-Based Acoustic Model Using Phoneme Alignment for Practical Neural Text-to-Speech Systems
Okamoto, T., Toda, T., Shiga, Y., & Kawai, H. (2019) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru46091.2019.9003956 · Cited by 24
“Although conventional SPSS systems separately train duration and AMs, seq2seq models jointly train them at once without a pipeline structure, and several end-to-end methods have been initially investigated [10–12].”
MethodLow-Resource End-to-end Sanskrit TTS using Tacotron2, WaveGlow and Transfer Learning
Debnath, A., Patil, S. S., Nadiger, G., & Ramakrishnan, A. G. (2020) · IEEE India Conference · doi:10.1109/indicon49873.2020.9342071 · Cited by 36
Attacking Speaker Recognition With Deep Generative ModelsPreprint
Cai, W., Doshi, A. P., & Valle, R. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.02384 · Cited by 24
Enhancing Low-Resource ASR through Versatile TTS: Bridging the Data Gap
Yang, G., Yu, F., Ma, Z., Du, Z., Gao, Z., Zhang, S. S., & Chen, X. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10889894 · Cited by 34
“Traditional TTS models [7]–[9] commonly train on only dozens to hundreds of hours of single-speaker or multi-speaker data, producing speech with constrained timbre, prosody, and speaking styles.”
Speech Prediction in Silent Videos Using Variational Autoencoders
Yadav, R. K., Sardana, A., Namboodiri, V. P., & Hegde, R. M. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414040 · Cited by 23
“Text-to-speech systems [1, 2], which uses text or phonemes as inputs to generate audio, have been widely useful in ubiquitous applications.”
BackgroundLi, X., Ma, D., & Yin, B. (2020) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2020.105908 · Cited by 23
Fake visual content detection using two-stream convolutional neural networks
Yousaf, B., Usama, M., Sultani, W., Mahmood, A., & Qadir, J. (2022) · Neural Computing and Applications · doi:10.1007/s00521-022-06902-5 · Cited by 23
Bidirectional Variational Inference for Non-Autoregressive Text-to-Speech
Lee, Y., Shin, J., & Jung, K. (2021) · International Conference on Learning Representations · Cited by 45
Sequence-to-Sequence Emotional Voice Conversion With Strength Control
Choi, H., & Hahn, M. (2021) · IEEE Access · doi:10.1109/access.2021.3065460 · Cited by 41
“A prosody embedding vector is computed through a prosody encoder and is used as an additional input to Tacotron [45] so that not monotonous but expressive speech can be synthesized.”
MethodA Comparative Study of Self-Supervised Speech Representation Based Voice Conversion
Huang, W.-C., Yang, S.-W., Hayashi, T., & Toda, T. (2022) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3193761 · Cited by 32
“Dropout is essential in the AR loop to avoid exposure bias brought by teacher-forcing [58], [59].”
BackgroundUnsupervised Polyglot Text-to-speech
Nachmani, E., & Wolf, L. (2019) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2019.8683519 · Cited by 38
Applications of Deep Learning to Audio Generation
Zhao, Y., Xia, X., & Togneri, R. B. (2019) · IEEE Circuits and Systems Magazine · doi:10.1109/mcas.2019.2945210 · Cited by 28
“This Tacotron 2 was composed of a recurrent network, which was for mapping character embeddings to Mel-scale spectrograms, and a modified WaveNet model, which acted as a vocoder to yield time-domain waveforms from those spectrograms.”
MethodShiga, Y., Ni, J., Tachibana, K., & Okamoto, T. (2019) · SpringerBriefs in computer science · doi:10.1007/978-981-15-0595-9_3 · Cited by 23
Situation-Aware Emotion Regulation of Conversational Agents with Kinetic Earables
Katayama, S., Mathur, A., den Broeck, M. V., Okoshi, T., Nakazawa, J., & Kawsar, F. (2019) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii.2019.8925449 · Cited by 30
“On Speech Synthesis, a collection of work, such as Wavenet [11], Tacotron [12], and Deep Voice [13] looked at synthesising voice using reference acoustic representation for the desired prosody.”
MethodN-Singer: A Non-Autoregressive Korean Singing Voice Synthesis System for Pronunciation Enhancement
Lee, G.-H., Kim, T., Bae, H., Lee, M., Kim, Y.-I., & Cho, H. Y. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-239 · Cited by 26
Variational Bi-LSTMsPreprint
Shabanian, S., Arpit, D., Trischler, A., & Bengio, Y. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.05717 · Cited by 23
“…have become the standard models for sequential prediction tasks with state of the art performance in a number of applications like sequence prediction, language translation, machine comprehension, and speech synthesis (Arik et al., 2017; Wang et al., 2017; Mehri et al., 2016; Sotelo et al., 2017).”
BackgroundConditional WaveGANPreprint
Lee, C. Y., Toffy, A., Jung, G. J., & Han, W.-J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.10636 · Cited by 25
“Tacotron [26] is an end-to-end generative text-to-speech model that synthesizes speech directly from characters.”
Multi-Speaker End-to-End Speech SynthesisPreprint
Park, J., Zhao, K., Peng, K., & Ping, W. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.04462 · Cited by 23
“C L ] 9 J ul 2 01 Deep Voice 3 (Ping et al., 2018), Tacotron (Wang et al., 2017), Tacotron 2 (Shen et al., 2018), Char2Wav (Sotelo et al., 2017), and ParaNet (Peng et al., 2019) are attention-based seq2seq models, which convert text to acoustic features (e.g., mel spectrogram) and have much more…”
BackgroundInfluentialDeep Learning for Mandarin-Tibetan Cross-Lingual Speech Synthesis
Zhang, W., Yang, H., Bu, X., & Wang, L. (2019) · IEEE Access · doi:10.1109/access.2019.2954342 · Cited by 22
“Recent research on attention-based sequence-to-sequence (seq2seq) models are demonstrated to outperform the SPSS by several end-toend TTS systems [11], [12].”
Transferring Source Style in Non-Parallel Voice Conversion
Liu, S., Cao, Y., Kang, S., Hu, N., Liu, X., Su, D., Yu, D., & Meng, H. M. L. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-2412 · Cited by 28
“[18] Y. Wang, R. Skerry-Ryan, D. Stanton, Y. Wu, R. J. Weiss, N. Jaitly, Z. Yang, Y. Xiao, Z. Chen, S. Bengio et al., “Tacotron: Towards end-to-end speech synthesis,” Proc.”
BackgroundMethodRyanSpeech: A Corpus for Conversational Text-to-Speech Synthesis
Zandie, R., Mahoor, M. H., Madsen, J., & Emamian, E. S. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-341 · Cited by 25
MIST-Tacotron: End-to-End Emotional Speech Synthesis Using Mel-Spectrogram Image Style Transfer
Moon, S., Kim, S., & Choi, Y. (2022) · IEEE Access · doi:10.1109/access.2022.3156093 · Cited by 26
“Tacotron [9] is the first end-to-end generative TTS model based on the sequence-to-sequence neural network model [18] with attention module.”
BackgroundMethodZhao, B., Zhang, X., Wang, J., Cheng, N., & Xiao, J. Z. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9746875 · Cited by 24
SANE-TTS: Stable And Natural End-to-End Multilingual Text-to-Speech
Cho, H., Jung, W., Lee, J., & Woo, S. H. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-46 · Cited by 34
“However, Tacotron-based models have several issues while they synthesize speeches in an autoregressive manner that utilizes attention [8] to align input text and target speech.”
MethodPolyGlotFake: A Novel Multilingual and Multimodal DeepFake Dataset
Hou, Y., Fu, H., Chen, C., Li, Z., Zhang, H., & Zhao, J. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-78341-8_12 · Cited by 40
Investigating very deep highway networks for parametric speech synthesis
Wang, X., Takaki, S., & Yamagishi, J. (2017) · Speech Communication · doi:10.1016/j.specom.2017.11.002 · Cited by 24
Incremental Text-to-Speech Synthesis Using Pseudo Lookahead With Large Pretrained Language Model
Saeki, T., Takamichi, S., & Saruwatari, H. (2021) · IEEE Signal Processing Letters · doi:10.1109/lsp.2021.3073869 · Cited by 29
“In recent years, the quality of TTS has dramatically improved with the shift from cascade statistical parametric speech synthesis [7], [8], [9] to end-to-end TTS [10], [4], [11], which directly generates a mel-spectrogram of output speech from”
BackgroundMethodKurihara, K., Seiyama, N., & Kumano, T. (2021) · IEICE Transactions on Information and Systems · doi:10.1587/transinf.2020edp7104 · Cited by 22
“Tacotron 2 uses a sequence-to-sequence (seq2seq) model, as do similar methods such as Tacotron [4], Char2Wav [5], Voiceloop [6], Deep Voice 3 [7], and Transformer-based TTS [8].”
BackgroundMethodMulti-Modal Emotion Recognition Using Speech Features and Text-Embedding
Byun, S.-W., Kim, J.-H., & Lee, S.-P. (2021) · Applied Sciences · doi:10.3390/app11177967 · Cited by 23
Multi-Task Learning Improves Synthetic Speech Detection
Mo, Y., & Wang, S. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9746059 · Cited by 30
Efficient Convolutional Neural Network Training with Direct Feedback AlignmentPreprint
Han, D., & Yoo, H. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.01986 · Cited by 22
Mel-spectrogram augmentation for sequence to sequence voice conversionPreprint
Hwang, Y., Cho, H. M., Yang, H., Won, D.-O., Oh, I., & Lee, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.01401 · Cited by 31
“Recently developed speech-synthesis techniques [2, 3] can produce synthesized speech close to that of the target speaker.”
BackgroundNeural Dubber: Dubbing for Videos According to ScriptsPreprint
Hu, C., Tian, Q., Li, T., Wang, Y.-P., Wang, Y., & Zhao, H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.08243 · Cited by 57
“The results of extensive quantitative and qualitative evaluations show that in terms of speech quality, Neural Dubber is on par with state-of-the-art TTS models [9, 6].”
InfluentialSelf-citationA Survey of Deep Learning Audio Generation MethodsPreprint
Božić, M., & Horvat, M. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2406.00146 · Cited by 23
A Unified Sequence-to-Sequence Front-End Model for Mandarin Text-to-Speech Synthesis
Pan, J., Yin, X., Zhang, Z., Liu, S., Zhang, Y., Ma, Z., & Wang, Y. (2020) · ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp40776.2020.9053390 · Cited by 21
“We implemented two front-end models, the pipeline-based and the unified (CWS+POS auxiliary) with Tacotron [1] and WaveRNN [20] in two Mandarin TTS systems separately.”
MethodInfluentialSelf-citation
Metadata
- Authors
- Wang, Y., Skerry-Ryan, R. J., Stanton, D., Wu, Y., Weiss, R. J., Jaitly, N., Yang, Z., Xiao, Y., Chen, Z., Bengio, S., Le, Q., Agiomyrgiannakis, Y., Clark, R., & Saurous, R. A.
- Affiliations
- Google (United States) (US)
- Corresponding author
- Wang, Yuxuan
- ORCID
- Venue
- Interspeech 2017
- Publisher
- ISCA
- Published
- 2017-08-20
- Pages
- 4006-4010
- ISSN
- 2308-457X (ISSN-L)
- OpenAlex
- W2963609956
- Cited by
- 1,746
- References
- 25
- Citations by year
- 2026: 134 · 2025: 187 · 2024: 183 · 2023: 215 · 2022: 243 · 2021: 301 · 2020: 252 · 2019: 176 · 2018: 45 · 2017: 9
- FWCI
- 60.12
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Interspeech (USB) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Health Sciences › Health Professions › Speech and Hearing › Voice and Speech Disorders
- Keywords
- Tacotron, end-to-end speech synthesis, text-to-speech, sequence-to-sequence models, naturalness
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.