Tacotron: Towards End-to-End Speech Synthesis
The DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.
Abstract
A text-to-speech synthesis system typically consists of multiple stages, such as a text analysis frontend, an acoustic model and an audio synthesis module.Building these components often requires extensive domain expertise and may contain brittle design choices.In this paper, we present Tacotron, an end-to-end generative text-to-speech model that synthesizes speech directly from characters.Given pairs, the model can be trained completely from scratch with random initialization.We present several key techniques to make the sequence-tosequence framework perform well for this challenging task.Tacotron achieves a 3.82 subjective 5-scale mean opinion score on US English, outperforming a production parametric system in terms of naturalness.In addition, since Tacotron generates speech at the frame level, it's substantially faster than sample-level autoregressive methods.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The frameworks including their code as well as the datasets used that are available publicly (Sotelo et al., 2017; Wang et al., 2017; Ping et al., 2018; Shen et al., 2018; Vasquez and Lewis, 2019) are more likely to be used for nefarious purposes or research, so they will be more developed.”
From Audio deepfakes: A survey (2023) · cites this paper as Wang et al., 2017
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 2×
“We further pass the processed sequence to a few fixed-width 1-D convolutions, whose outputs are added with the original input sequence via residual connections [17].”
BackgroundAdam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 2×
“We use the Adam optimizer (Kingma & Ba, 2015) with learning rate decay, which starts from 0.001 and is reduced to 0.0005, 0.0003, and 0.0001 after 500K, 1M and 2M global steps, respectively.”
MethodInfluentialBatch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818 · cited 2×
“Batch normalization [18] is used for all convolutional layers.”
MethodSequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766 · cited 2×
“In this paper, we propose Tacotron, an end-to-end generative TTS model based on the sequence-to-sequence (seq2seq) (Sutskever et al., 2014) with attention paradigm (Bahdanau et al., 2014).”
MethodSelf-citationNeural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213 · cited 4×
“The backbone of Tacotron is a seq2seq model with attention [7, 14].”
MethodInfluentialEmpirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1412.3555 · Cited by 11,008 · cited 7×
“Second, we compare with a model with the CBHG encoder replaced by a 2-layer residual GRU encoder.”
BackgroundInfluentialTensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., et al. (2016) · arXiv · doi:10.48550/arxiv.1603.04467 · Cited by 9,408 · cited 2×
“We implemented Griffin-Lim in TensorFlow [22] hence it’s also part of the model.”
MethodSelf-citationGoogle's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
Wu, Y., Schuster, M., Chen, Z., Le, Q. V., Norouzi, M., Macherey, W., Krikun, M., Cao, Y., Gao, Q., Macherey, K., et al. (2016) · arXiv · doi:10.48550/arxiv.1609.08144 · Cited by 5,319 · cited 4×
“C L ] 6 A pr or machine translation (Wu et al., 2016), TTS outputs are continuous, and output sequences are usually much longer than those of the input.”
BackgroundMethodInfluentialSelf-citationWaveNet: a generative model for raw audio
Oord, A. V. D., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1609.03499 · Cited by 4,911 · cited 4×
“It could be used to predict alternative targets such as vocoder parameters, or as a WaveNetlike neural vocoder [9, 13, 10] that synthesizes waveform samples directly.”
BackgroundMethodInfluentialListen, attend and spell: A neural network for large vocabulary conversational speech recognition
Chan, W., Jaitly, N., Le, Q., & Vinyals, O. (2016) · 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2016.7472621 · Cited by 2,226 · cited 2×
“Moreover, unlike end-to-end speech recognition [4] or machine translation [5], TTS outputs are continuous, and output sequences are usually much longer than those of the input.”
BackgroundSelf-citation
Cited by
Items 701–800 of 1,002
Exploring the Role of Language Families for Building Indic Speech Synthesisers
Prakash, A., & Murthy, H. A. (2022) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2022.3230453 · Cited by 22
“W ITH the advent of neural network-based end-to-end (E2E) approaches, training a text-to-speech (TTS) synthesiser has become easier when a large amount of data is available for a language [1], [2].”
BackgroundECAPA-TDNN for Multi-speaker Text-to-speech Synthesis
Xue, J., Deng, Y., Han, Y., Li, Y., Sun, J., & Liang, J. (2022) · International Symposium on Chinese Spoken Language Processing (ISCSLP) · doi:10.1109/iscslp57327.2022.10037956 · Cited by 23
SR-TTS: a rhyme-based end-to-end speech synthesis system
Yao, Y., Tao, L., Feng, R., Shi, K., Yu, J., Wang, W., & Li, J. (2024) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2024.1322312 · Cited by 13
“We evaluated the SR-TTS model on an English dataset LJ Speech (Ito and Johnson, 2017) and compared the synthesized speech with the mainstream models, including Tacotron2 and FastSpeech2.”
BackgroundInfluentialProblem-Agnostic Speech Embeddings for Multi-Speaker Text-to-Speech with SampleRNN
Álvarez, D., Pascual, S., & Bonafonte, A. (2019) · ISCA Speech Synthesis Workshop (SSW) · doi:10.21437/ssw.2019-7 · Cited by 14
Fast Griffin Lim based waveform generation strategy for text-to-speech synthesis
Sharma, A., Kumar, P., Maddukuri, V., Madamshettib, N., Kg, K., Kavurub, S. S. S., Raman, B., & Roy, P. P. (2020) · Multimedia Tools and Applications · doi:10.1007/s11042-020-09321-7 · Cited by 15
“The recent neural TTS systems include Wavenet [40], Char2Wav [35], Tacotron [42], Tacotron 2 [33], DeepVoice [2], DeepVoice 2 [9], DeepVoice 3 [28], VoiceLoop [37].”
BackgroundMethodInfluentialWhispered and Lombard Neural Speech Synthesis
Hu, Q., Bleisch, T., Petkov, P. N., Raitio, T., Marchi, E., & Lakshminarasimhan, V. (2021) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt48900.2021.9383454 · Cited by 19
“To investigate the feasibility of synthesizing whisper and Lombard speech with neural TTS, while keeping in mind the difficulties involved in data collection for whisper and Lombard speech, we created neural TTS systems based on the Tacotron 2 [1] architecture capable of synthesizing different speaking styles using a…”
BackgroundMethodInfluentialEnd-to-End Speech Recognition Sequence Training With Reinforcement Learning
Tjandra, A., Sakti, S., & Nakamura, S. (2019) · IEEE Access · doi:10.1109/access.2019.2922617 · Cited by 13
“INTRODUCTION End-to-end sequence modeling has been successfully developed into many different applications, such as: image captioning [32], [39], machine translation [1], [27], abstractive summarization [17], speech synthesis [33] and speech recognition [2], [3].”
BackgroundHooliGAN: Robust, High Quality Neural VocodingPreprint
McCarthy, O., & Ahmed, Z. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.02493 · Cited by 13
“Most of these vocoders consume log mel spectrograms that are predicted by a text to acoustic model such as Tacotron [13, 14].”
MethodInfluentialFastS2S-VC: Streaming Non-Autoregressive Sequence-to-Sequence Voice ConversionPreprint
Kameoka, H., Tanaka, K., & Kaneko, T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.06900 · Cited by 20
“The S2S learning framework offers a general and powerful solution to a broad class of sequential mapping problems, including machine translation (MT), automatic speech recognition (ASR) [27], and text-to-speech (TTS) [38]–[43].”
BackgroundMethodMeta-Voice: Fast few-shot style transfer for expressive voice cloning using meta learningPreprint
Liu, S., Su, D., & Yu, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.07218 · Cited by 15
Chen, J., Byun, J.-S., Elsner, M., & Perrault, A. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2405.14632 · Cited by 13
“…application of online RL to fine-tune TTS diffusion models (Appendix A), especially whether RL techniques can similarly improve the naturalness and sound quality of diffusion TTS models that are directly trained on waveforms and assess the impact of RL on waveform generation in diffusion models.”
Towards Signal Processing In Large Language ModelsPreprint
Verma, P., & Pilancı, M. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2406.10254 · Cited by 14
Continuous vocoder applied in deep neural network based voice conversion
Al-Radhi, M. S., Csapó, T. G., & Németh, G. (2019) · Multimedia Tools and Applications · doi:10.1007/s11042-019-08198-5 · Cited by 12
“…various studies in voice conversion are still considering some of these vocoders [23], such as STRAIGHT [5, 53, 59], mixed excitation [34], Harmonic plus Noise Model [35], glottal source modeling [6], or even with more complex end-to-end acoustic models like adaptive WAVENET [54], or Tacotron [62].”
MethodBang, C.-W., & Chun, C. (2023) · Sensors · doi:10.3390/s23239591 · Cited by 16
“An example of a neural network-based speech synthesis model is Tacotron, which uses an attention-based sequence-to-sequence structure [4,5].”
MethodTwo-stage and Self-supervised Voice Conversion for Zero-Shot Dysarthric Speech Reconstruction
Liu, D., Lin, Y., Bu, H., & Li, M. (2024) · International Conference on Asian Language Processing · doi:10.1109/ialp63756.2024.10661160 · Cited by 12
“With the development of deep learning, sequence-to-sequence (seq2seq) based TTS synthesis [24] and knowledge distillation (KD) techniques [13], [27] have emerged as promising solutions for DSR. Wang et al. [23] proposed an end-to-end voice conversion (E2E-VC) based DSR sys-tem with cross-modal…”
BackgroundMitigating Unauthorized Speech Synthesis for Voice Protection
Zhang, Z., Yang, Q., Wang, D., Huang, P., Cao, Y., Ye, K., & Hao, J. (2023) · doi:10.1145/3689217.3690615 · Cited by 20
TTSDS - Text-to-Speech Distribution Score
Minixhofer, C., Klejch, O., & Bell, P. (2024) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt61566.2024.10832178 · Cited by 31
“Later, the ”Back to the Future” (BTTF) [5] compared unit selection, hybrid and statistical parametric HMM-based systems from the Blizzard 2013 challenge [37] with deep learning systems inspired by the Blizzard 2021 challenge [38] based on FastPitch [39] and Tacotron [40].”
ECE-TTS: A Zero-Shot Emotion Text-to-Speech Model with Simplified and Precise Control
Liang, S., Zhou, R., & Yuan, Q. (2025) · Applied Sciences · doi:10.3390/app15095108 · Cited by 20
A survey of threats against voice authentication and anti-spoofing systems
Kamel, K., Sood, K., Dutta, H. S., & Aryal, S. (2026) · Artificial Intelligence Review · doi:10.1007/s10462-026-11709-0 · Cited by 17
Forward-Backward Decoding for Regularizing End-to-End TTS
Zheng, Y., Wang, X., He, L., Pan, S., Soong, F. K., Wen, Z., & Tao, J. (2019) · Interspeech (USB) · doi:10.21437/interspeech.2019-2325 · Cited by 14
What the Future Brings: Investigating the Impact of Lookahead for Incremental Neural TTS
Stephenson, B., Besacier, L., Girin, L., & Hueber, T. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-2103 · Cited by 19
Dahmani, S., Colotte, V., Girard, V., & Ouni, S. (2021) · Neural Networks · doi:10.1016/j.neunet.2021.04.021 · Cited by 12
“Recently, end-to-end systems for acoustic speech synthesis emerged ([17], [18]).”
BackgroundDeep Learning End to End Speech Synthesis: A Review
Nazir, O., & Malik, A. (2021) · International Conference on Secure Cyber Computing and Communications (ICSCCC) · doi:10.1109/icsccc51823.2021.9478125 · Cited by 12
Liu, S., Yang, S., Su, D., & Yu, D. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9746858 · Cited by 13
Limited text speech synthesis with electroglottograph based on Bi-LSTM and modified Tacotron-2
Chen, L., Ren, J., Chen, P., Mao, X., & Zhao, Q. (2022) · Applied Intelligence · doi:10.1007/s10489-021-03075-x · Cited by 12
“Another widely-used model named Tacotron [25] was proposed by Google, which is an end-to-end generative text to speech model and achieves to directly learn the mapping between the text and speech pair.”
MethodDuration Modeling of Neural TTS for Automatic Dubbing
Effendi, J., Virkar, Y. S., Barra-Chicote, R., & Federico, M. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747158 · Cited by 25
“Compared with the attention-based Tacotron model [27, 28] which models the duration implicitly through attention, explicit duration modelling is used in FastSpeech [29, 30] and Parallel Tacotron [31, 32].”
MethodCross-speaker Emotion Transfer Based On Prosody Compensation for End-to-End Speech Synthesis
Li, T., Wang, X., Xie, Q., Wang, Z., Jiang, M., & Xie, L. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-610 · Cited by 20
Online Phase Reconstruction via DNN-Based Phase Differences Estimation
Masuyama, Y., Yatabe, K., Nagatomo, K., & Oikawa, Y. (2022) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2022.3221041 · Cited by 12
The Sound of Silence: Efficiency of First Digit Features in Synthetic Audio Detection
Mari, D., Latora, F., & Milani, S. (2022) · IEEE International Workshop on Information Forensics and Security · doi:10.1109/wifs55849.2022.9975404 · Cited by 17
“To improve upon this, neural networks have also been used to substitute either the acoustic model [22] or the vocoder [23], [24] later leading to the first end to end TTS generation algorithms [25], [26].”
MethodGopalakrishnan, T., Imam, S. A., & Aggarwal, A. (2022) · International Conference on Data Science and Information System (ICDSIS) · doi:10.1109/icdsis55133.2022.9915932 · Cited by 18
“According to the experiments in [8], Deep Voice 3 took approximately 500k iterations to train, whereas Tacotron [21] took almost 2 million iterations.”
MethodLiVoAuth: Liveness Detection in Voiceprint Authentication With Random Challenges and Detection Modes
Zhang, R., Yan, Z., Wang, X., & Deng, R. H. (2022) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2022.3213830 · Cited by 12
“We attempted this attack based on some existing speech synthesis method [24], [25].”
BackgroundMethodDictionary Attacks on Speaker Verification
Marras, M., Korus, P., Jain, A., & Memon, N. D. (2022) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2022.3229583 · Cited by 16
“The system uses Tacotron [59] for waveform synthesis, WaveRNN [60] as a vocoder, and an LSTM-based encoder [34].”
MethodProsody Modelling With Pre-Trained Cross-Utterance Representations for Improved Speech Synthesis
Zhang, Y.-J., Zhang, C., Song, W. G., Zhang, Z., Wu, Y., & He, X. (2023) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2023.3278184 · Cited by 16
“Popularend-to-end TTSmethods[1],[2],[3],[4],[5],[6],[7]canbedividedinto twocategories:autoregressivemodels,suchasTacotron[1], andnon-autoregressivemodels,suchasFastspeech[5].”
Self-citationDual-branch network with fused Mel features for logic-manipulated speech detection
Yang, H., Yan, X., & Wang, H. (2024) · Applied Acoustics · doi:10.1016/j.apacoust.2024.110047 · Cited by 12
Recognition and Processing of Speech Signals Using Neural Networks
O’Shaughnessy, D. D. (2019) · Circuits Systems and Signal Processing · doi:10.1007/s00034-019-01081-6 · Cited by 12
“Google’s Tacotron [53] is a possible view of future TTS, which has exceptional quality, but is still far from practical, owing to the need for massive computation.”
BackgroundLiu, Y., & Zheng, J. J. (2019) · Information · doi:10.3390/info10040131 · Cited by 12
“We employ an internal US English dataset [1], which contains 24.”
BackgroundMethodSaito, Y., Takamichi, S., & Saruwatari, H. (2021) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2021.3059114 · Cited by 17
“…modeling has been accomplished thanks to the developments of DNN-based speech waveform modeling [11], [12], rich acoustic models (e.g., Tacotron [13], FastSpeech [14], [15], and MelNet [16]), and sophisticated acoustic model training (e.g., generative adversarial network (GAN)-based meth-ods…”
MethodEnd-to-End Text-to-Speech Using Latent Duration Based on VQ-VAE
Yasuda, Y., Wang, X., & Yamagishd, J. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414499 · Cited by 18
Fine-Grained Prosody Modeling in Neural Speech Synthesis Using ToBI Representation
Zou, Y., Liu, S., Yin, X., Lin, H., Wang, C., Zhang, H., & Ma, Z. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-883 · Cited by 12
“Recently, neural text-to-speech (TTS) systems [1, 2, 3, 4] can generate speech that is indistinguishable from natural speech.”
BackgroundTatanov, O., Beliaev, S., & Ginsburg, B. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9746107 · Cited by 20
“The improvement in training and inference speed is mostly related to switching from sequential, autoregressive models [1, 2, 3]) to parallel, non-autoregressive models [4, 5, 6, 7].”
MethodFeature reinforcement with word embedding and parsing information in neural TTSPreprint
Ming, H., He, L., Guo, H., & Soong, F. K. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.00707 · Cited by 12
DeviceTTS: A Small-Footprint, Fast, Stable Network for On-Device Text-to-SpeechPreprint
Huang, Z., Li, H., & Lei, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.15311 · Cited by 12
“In recent years, many excellent TTS systems have been proposed to simplify the traditional TTS pipeline and remove the dependencies of hand crafted intermediate representations [2–8].”
TranssionADD: A multi-frame reinforcement based sequence tagging model for audio deepfake detectionPreprint
Liu, J., Su, Z., Huang, H. F., Wan, C., Wang, Q., Hong, J., Tang, B., & Zhu, F. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.15212 · Cited by 21
An overview of text-to-speech systems and media applicationsPreprint
Hasanabadi, M. R. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.14301 · Cited by 17
“Each of these properties can cause errors which accumulate quickly [2].”
BackgroundMethodInfluentialSparks of Large Audio Models: A Survey and Outlook
Latif, S., Shoukat, M., Shamshad, F., Usama, M., Ren, Y., Cuayáhuitl, H., Wang, W., Zhang, X., Togneri, R., Cambria, E., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.12792 · Cited by 17
“Popular TTs models include Tacotron [168], Deep Voice model [169], and Clarinet [170], and many other [26].”
MethodMultimodal data-driven machine learning for the prediction of surface topography in end milling
Hu, L., Phan, H., Srinivasan, S. S., Cooper, C., Zhang, J., Yuan, B., Gao, R. X., & Guo, Y. B. (2024) · Production Engineering · doi:10.1007/s11740-023-01253-z · Cited by 11
“In contrast to using direct learning on the raw waveforms in the previous ML work for audio [51, 52], the audio signals are used to create a time–frequency map from the power magnitude obtained through Short-time Fourier Transform (STFT), which captures the changes over time of each frequency…”
Reddy, P. V., Rohith, D., Dhanush, S., Kumar, T. S. G., Lal, G. J., & Premjith, B. (2023) · International Conference on Electronics, Communication and Aerospace Technology (ICECA) · doi:10.1109/iceca58529.2023.10395349 · Cited by 11
“MelGAN consists of a generative network that effectively upsamples the spectrogram on the mel scale using an extended convolutional neural network (CNN) architecture [6].”
MethodInfluentialNeural Effects of One’s Own Voice on Self-Talk for Emotion Regulation
Jo, H.-J., Park, C., Lee, E., Lee, J. H., Kim, J., Han, S.-J., Kim, J., Kim, E. J., Kim, E., & Kim, J. (2024) · Brain Sciences · doi:10.3390/brainsci14070637 · Cited by 15
Improving Mispronunciation Detection Using Speech Reconstruction
Das, A., & Gutiérrez‐Osuna, R. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3434497 · Cited by 14
“The architecture of the model follows from [8], which has been shown to generate natural-sounding speech.”
BackgroundMethodInfluentialShen, Y., Cao, G., Dong, H.-Y., Dong, B., & Lee, C.-M. (2025) · Applied Sciences · doi:10.3390/app15158413 · Cited by 11
Zhang, H., Liu, Y., Guan, X., Cai, J., & Carroll, J. M. (2026) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2026.103950 · Cited by 13
Prediction model for stock price trend based on convolution neural network
Lin, H., Zhao, J., Liang, S., & Kang, H. (2020) · Journal of Intelligent & Fuzzy Systems · doi:10.3233/jifs-179985 · Cited by 11
“…1] to avoid the situation where large orders of magnitude cover small orders of information. b) The discrete technical indicators-the trend of the [7] pointed out that some technical indicators that can predict the future movement of stocks, so continuous technical indicators are cited in part.”
Enhancing Sequence-to-Sequence Text-to-Speech with Morphology
Taylor, J., & Richmond, K. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-1547 · Cited by 11
“Tacotron [7]) which is now often considered state-of-theart in acoustic modelling [8].”
BackgroundFcl-Taco2: Towards Fast, Controllable and Lightweight Text-to-Speech Synthesis
Wang, D., Deng, L., Zhang, Y., Zheng, N., Yeung, Y. T., Chen, X. D., Liu, X., & Meng, H. M. L. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414870 · Cited by 12
“Advances in deep learning have enabled TTS to achieve remarkable progress via sequence-to-sequence (seq2seq) acoustic models [7, 9-11], where the attention mechanism is used to align the text and speech automatically to achieve satisfactory synthesis results.”
Cheng, G., Wang, L., Ling, Z.-H., Guo, S., Ju, Z., & Dang, J. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414720 · Cited by 11
“Conventional approaches [1, 2] to end-to-end speech synthesis typically consider the spectral features in which the fundamental frequency (F0) is implicit as targets.”
BackgroundMethodReinforce-Aligner: Reinforcement Alignment Search for Robust End-to-End Text-to-Speech
Chung, H., Lee, S., & Lee, S. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-831 · Cited by 14
An, X., Soong, F. K., Yang, S., & Xie, L. (2021) · Neural Networks · doi:10.1016/j.neunet.2021.06.006 · Cited by 11
Saeki, T., Tachibana, K., & Yamamoto, R. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-294 · Cited by 13
Zhou, Y., Song, C., Li, J., Wu, Z., Bian, Y., Su, D., & Meng, H. M. L. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-10061 · Cited by 12
“As deep learning techniques develop, neural TTS systems [2, 3, 4] have shown great advantages over the conventional TTS systems [5, 6].”
BackgroundDetecting Spoofed Speeches via Segment-Based Word CQCC and Average ZCR for Embedded Systems
Zhan, J., Pu, Z., Jiang, W., Wu, J., & Yang, Y. (2022) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2022.3197531 · Cited by 12
“Text-to-speech attacks can convert text to the speeches of the target speaker [9], [10], [11].”
BackgroundMethodEnd to End Text to Speech Synthesis for Malay Language using Tacotron and Tacotron 2
Aziz, A. F. A., Tiun, S. B., & Ruslan, N. (2023) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2023.0140644 · Cited by 11
“Numerous studies focusing on Tacotron and Tacotron 2 models for TTS applications have been carried out, predominantly targeting languages like English [6][7], Chinese [10], Spanish, Korean, Japanese, Mongolian [11], and Myanmar [12][13].”
BackgroundMethodInfluentialUncovering the Deceptions: An Analysis on Audio Spoofing Detection and Future Prospects
Ranjan, R., Vatsa, M., & Singh, R. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/756 · Cited by 14
Wang, W., Huang, J., Huang, J.-T., Chen, C., Gu, J., He, P., & Lyu, M. R.-T. (2023) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase56229.2023.00189 · Cited by 11
“Sentiment analysis [70], [71] enables online review recommendation; Sequence-to-sequence models make substantial progress in machine translation [72]–[74] and text-to-speech synthesis [75], [76].”
Protecting Your Voice from Speech Synthesis Attacks
Liu, Z., Zhang, Y., & Miao, C. (2023) · Annual Computer Security Applications Conference · doi:10.1145/3627106.3627183 · Cited by 19
Du, H.-P., Lu, Y.-X., Ai, Y., & Ling, Z.-H. (2024) · Communications in computer and information science · doi:10.1007/978-981-97-0601-3_6 · Cited by 29
FT-GAN: Fine-Grained Tune Modeling for Chinese Opera Synthesis
Zheng, M., Bai, P., Shi, X., Zhou, X. S., & Yan, Y. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i17.29943 · Cited by 16
“Notably, most autoregressive SVS models at this stage are based on the Tacotron (Wang et al. 2017).”
MethodThe USTC System for Blizzard Challenge 2017
Hu, Y.-J., Ding, C., Liu, L.-J., Ling, Z.-H., & Dai, L.-R. (2017) · The Blizzard Challenge · doi:10.21437/blizzard.2017-13 · Cited by 11
Zhou, X., Zhang, M., Zhou, Y., Wu, Z., & Li, H. (2026) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2026.3657306 · Cited by 13
“Over the years, TTS technology has made significant progress, from statistical parametric modeling [1], [2] to end-to-end (E2E) architectures [3], [4], [5], [6], [7], [8], [9], which generate high-quality and human-like speech directly from text inputs.”
Saito, Y., Takamichi, S., & Saruwatari, H. (2019) · ISCA Speech Synthesis Workshop (SSW) · doi:10.21437/ssw.2019-10 · Cited by 14
Audio Dequantization for High Fidelity Audio Generation in Flow-Based Neural Vocoder
Yoon, H.-W., Lee, S.-H., Noh, H.-R., & Lee, S. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-1226 · Cited by 14
MASS: Multi-task anthropomorphic speech synthesis framework
Chen, J., Ye, L., & Ming, Z. (2021) · Computer Speech & Language · doi:10.1016/j.csl.2021.101243 · Cited by 12
Nam, K. H., Song, Y. J., & Yun, I. D. (2021) · IEEE Access · doi:10.1109/access.2021.3112545 · Cited by 12
“RNNs have made many advances in various sequential data domains, including speech processing such as speech recognition [11]–[15] and speech synthesis [16], [17], and language processing such as machine translation [10], [18].”
BackgroundMethodManipulating Voice Attributes by Adversarial Learning of Structured Disentangled Representations
Benaroya, L., Obin, N., & Röebel, A. (2023) · Entropy · doi:10.3390/e25020375 · Cited by 14
Comparison of Deepfakes Detection Techniques
Salman, S., & Shamsi, J. A. (2023) · doi:10.1109/icai58407.2023.10136659 · Cited by 14
“Synthetic deepfake audio is generated using applications such as Google Tacotron2 [8] and DeepVoice [9].”
MethodFu, R., Tao, J., Wen, Z., & Zheng, Y. (2019) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2019.8682535 · Cited by 11
“Statistical parametric speech synthesis (SPSS) [1] is more robust and apt to control the speaking style of generated speech using a small database at current stage, compared with unit selection methods [2] and end-to-end methods [3,4].”
BackgroundAnanthabhotla, I., Ewert, S., & Paradiso, J. A. (2019) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3343031.3351148 · Cited by 11
“Tacotron: Towards end-to-end speech synthesis.”
MethodSpeech-To-Singing Conversion in an Encoder-Decoder Framework
Parekh, J., Rao, P., & Yang, Y. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9054473 · Cited by 11
“This reduces artifacts in the final time domain predictions [23].”
BackgroundMethodSpeaker/Style-Dependent Neural Network Speech Synthesis Based on Speaker/Style Embedding
Sečujski, M., Pekar, D. J., Suzić, S., Smirnov, A., & Nosek, T. (2020) · JUCS - Journal of Universal Computer Science · doi:10.3897/jucs.2020.023 · Cited by 11
On Designing Expressive Robot Behavior: The Effect of Affective Cues on Interaction
Aly, A., & Tapus, A. (2020) · SN Computer Science · doi:10.1007/s42979-020-00263-3 · Cited by 12
Enhanced feature network for monaural singing voice separation
Yuan, W., He, B., Wang, S., Wang, J., & Unoki, M. (2018) · Speech Communication · doi:10.1016/j.specom.2018.11.004 · Cited by 11
Voice Imitating Text-to-Speech Neural NetworksPreprint
Lee, Y., Kim, T., & Lee, S.-Y. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.00927 · Cited by 11
“Report- edly, trimming silence is important for training Tacotron (Arık et al., 2017b; Wang et al., 2017).”
BackgroundMethodNoise Robust TTS for Low Resource Speakers using Pre-trained Model and Speech EnhancementPreprint
Dai, D., Chen, L., Wang, Y., Wang, M., Xia, R., Song, X., Wu, Z., & Wang, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.12531 · Cited by 11
“To leverage low-quality crowd-sourced data to train multispeaker TTS models that can synthesize clean speech for all speakers, based on Tacotron2, [11] introduced conditional generative reference encoders and adversarial training to learn disentangled representations to independently control the speaker identity and b…”
BackgroundMethodInfluentialSelf-citationAl-Radhi, M. S., Csapó, T. G., & Németh, G. (2019) · Applied Sciences · doi:10.3390/app9122460 · Cited by 10
A Comprehensive Interaction in Multiscale Multichannel EEG Signals for Emotion Recognition
Guo, Y., Zhang, B., Fan, X., Shen, X., & Peng, X. (2024) · Mathematics · doi:10.3390/math12081180 · Cited by 11
Semantic and Technical Noise Modeling in Semantic Image Communication Over Wireless Channels
Samarathunga, P., Pollwaththage, N., Fernando, T., Jayasinghe, U., Ihalagamage, N., Ganearachchi, Y., & Fernando, A. (2025) · IEEE Access · doi:10.1109/access.2025.3600320 · Cited by 10
“Speech Synthesis and Encoding can, on the receiver side, use deep generative models (WaveNet [50], Tacotron [51], etc.) to convert textual or abstract representations back into audible speech, enabling a full end-to-end system where meaning is transmitted and speech is reconstructed.”
Xiaomingbot: A Multilingual Robot News Reporter
Xu, R., Cao, J., Wang, M., Chen, J., Zhou, H., Zeng, Y., Wang, Y., Chen, L., Yin, X., Zhang, X., et al. (2020) · Annual Meeting of the Association for Computational Linguistics: System Demonstrations (ACL Demo) · doi:10.18653/v1/2020.acl-demos.1 · Cited by 13
Self-citationATT: Attention-based Timbre Transfer
Jain, D. K., Kumar, A., Cai, L., Singhal, S., & Kumar, V. (2020) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn48605.2020.9207146 · Cited by 10
“While simplistic, the approach is known to introduce powerful artificial features in the final output as observed by [1].”
BackgroundReformer-TTS: Neural Speech Synthesis with Reformer Network
Ihm, H. R., Lee, J. Y., Choi, B. J., Cheon, S. J., & Kim, N. S. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-2189 · Cited by 10
“To resolve this problem, integrated end-to-end TTS models like Deep Voice[4, 5], Tacotrons [6, 7] has been proposed.”
MethodTone Learning in Low-Resource Bilingual TTS
Liu, R., Wen, X., Chunhui, L., & Chen, X. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-2180 · Cited by 10
“Recent progress in end-to-end monolingual TTS [8–12] prompted studies to extend these systems to the multilingual task [13–17].”
BackgroundHierarchical Transfer Learning for Text-to-Speech in Indonesian, Javanese, and Sundanese Languages
Azizah, K., & Adriani, M. (2020) · International Conference on Advanced Computer Science and Information Systems · doi:10.1109/icacsis51025.2020.9263086 · Cited by 10
“Some BPSS studies include WaveNet [30], [31], DeepVoice [32], DeepVoice-2 [33], and DeepVoice-3 [34], Tacotron [9] and Tacotron-2 [4], and the modification and adaptation of Tacotron-based TTS [5], [35], [36].”
MethodResultZhang, G., Qiu, S., Qin, Y. H., & Lee, T. (2021) · International Symposium on Chinese Spoken Language Processing (ISCSLP) · doi:10.1109/iscslp49672.2021.9362098 · Cited by 11
“The system architecture for emotional prosody transfer follows that of [12] with some modifications, which is based on the Tacotron 1 [22] TTS system.”
MethodDua, M., Kadyan, V., Banthia, N., Bansal, A., & Agarwal, T. (2022) · Applied Acoustics · doi:10.1016/j.apacoust.2022.108643 · Cited by 10
Adversarial Multi-Task Learning for Disentangling Timbre and Pitch in Singing Voice Synthesis
Kim, T., Kang, M.-S., & Lee, G.-H. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-10994 · Cited by 11
WeSinger 2: Fully Parallel Singing Voice Synthesis via Multi-Singer Conditional Adversarial Training
Zhang, Z., Zheng, Y., Li, X., & Lü, L. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10095102 · Cited by 14
“As a consequence of the improvements [1, 2] in the text-to-speech (TTS) field, many researchers employed deep neural networks subsequently for acoustic modeling in SVS systems [3–5], which undoubtedly outperform the traditional SVS methods [6, 7] largely concerning the quality and naturalness.”
Vocoder Detection of Spoofing Speech Based on GAN Fingerprints and Domain Generalization
Li, F., Chen, Y., Liu, H., Zhao, Z., Yao, Y., & Liao, X. (2023) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3630751 · Cited by 13
Choi, J., Park, S. J., Kim, M., & Ro, Y. M. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02580 · Cited by 26
“One possible solution could be generating AV translation data, by separately synthesizing speech and video by applying TTS [44–46] and TFG [47–49] onto the existing NMT, A2T, and A2A datasets.”
MethodLhasa-Tibetan Speech Synthesis Using End-to-End Model
Zhao, Y., Hu, P., Xu, X., Wu, L., & Li, X. (2019) · IEEE Access · doi:10.1109/access.2019.2940125 · Cited by 12
“To solve the problem of sequence-to-sequence, an encoderdecoder framework has been proposed in [24].”
MethodAcoustic and articulatory analysis and synthesis of shouted vowels
Xue, Y., Marxen, M., Akagi, M., & Birkholz, P. (2020) · Computer Speech & Language · doi:10.1016/j.csl.2020.101156 · Cited by 10
“Tacotron [34]) can currently achieve the best speech quality, but not without some limitations.”
BackgroundNeural Style Transfer Based Voice Mimicking for Personalized Audio Stories
Fatima, S. M., Shehzad, M., Murtuza, S. S., & Raza, S. S. (2020) · International Workshop on AI for Smart TV Content Production, Access and Delivery (AI4TV) · doi:10.1145/3422839.3423063 · Cited by 10
“Generally the approach used for audio cloning and mimicking in literature focuses on deep neural networks, mainly convolutional neural network[15] because CNN based sequences can be much faster than RNN based techniques[16].”
MethodDian: Duration Informed Auto-Regressive Network for Voice Cloning
Song, W. G., Yuan, X., Zhang, Z., Zhang, C., Wu, Y., He, X., & Zhou, B. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414727 · Cited by 15
Self-citation
Metadata
- Authors
- Wang, Y., Skerry-Ryan, R. J., Stanton, D., Wu, Y., Weiss, R. J., Jaitly, N., Yang, Z., Xiao, Y., Chen, Z., Bengio, S., Le, Q., Agiomyrgiannakis, Y., Clark, R., & Saurous, R. A.
- Affiliations
- Google (United States) (US)
- Corresponding author
- Wang, Yuxuan
- ORCID
- Venue
- Interspeech 2017
- Publisher
- ISCA
- Published
- 2017-08-20
- Pages
- 4006-4010
- ISSN
- 2308-457X (ISSN-L)
- OpenAlex
- W2963609956
- Cited by
- 1,746
- References
- 25
- Citations by year
- 2026: 134 · 2025: 187 · 2024: 183 · 2023: 215 · 2022: 243 · 2021: 301 · 2020: 252 · 2019: 176 · 2018: 45 · 2017: 9
- FWCI
- 60.12
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Interspeech (USB) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Health Sciences › Health Professions › Speech and Hearing › Voice and Speech Disorders
- Keywords
- Tacotron, end-to-end speech synthesis, text-to-speech, sequence-to-sequence models, naturalness
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.