Tacotron: Towards End-to-End Speech Synthesis
The DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.
Abstract
A text-to-speech synthesis system typically consists of multiple stages, such as a text analysis frontend, an acoustic model and an audio synthesis module.Building these components often requires extensive domain expertise and may contain brittle design choices.In this paper, we present Tacotron, an end-to-end generative text-to-speech model that synthesizes speech directly from characters.Given pairs, the model can be trained completely from scratch with random initialization.We present several key techniques to make the sequence-tosequence framework perform well for this challenging task.Tacotron achieves a 3.82 subjective 5-scale mean opinion score on US English, outperforming a production parametric system in terms of naturalness.In addition, since Tacotron generates speech at the frame level, it's substantially faster than sample-level autoregressive methods.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The frameworks including their code as well as the datasets used that are available publicly (Sotelo et al., 2017; Wang et al., 2017; Ping et al., 2018; Shen et al., 2018; Vasquez and Lewis, 2019) are more likely to be used for nefarious purposes or research, so they will be more developed.”
From Audio deepfakes: A survey (2023) · cites this paper as Wang et al., 2017
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 2×
“We further pass the processed sequence to a few fixed-width 1-D convolutions, whose outputs are added with the original input sequence via residual connections [17].”
BackgroundAdam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 2×
“We use the Adam optimizer (Kingma & Ba, 2015) with learning rate decay, which starts from 0.001 and is reduced to 0.0005, 0.0003, and 0.0001 after 500K, 1M and 2M global steps, respectively.”
MethodInfluentialBatch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818 · cited 2×
“Batch normalization [18] is used for all convolutional layers.”
MethodSequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766 · cited 2×
“In this paper, we propose Tacotron, an end-to-end generative TTS model based on the sequence-to-sequence (seq2seq) (Sutskever et al., 2014) with attention paradigm (Bahdanau et al., 2014).”
MethodSelf-citationNeural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213 · cited 4×
“The backbone of Tacotron is a seq2seq model with attention [7, 14].”
MethodInfluentialEmpirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1412.3555 · Cited by 11,008 · cited 7×
“Second, we compare with a model with the CBHG encoder replaced by a 2-layer residual GRU encoder.”
BackgroundInfluentialTensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., et al. (2016) · arXiv · doi:10.48550/arxiv.1603.04467 · Cited by 9,408 · cited 2×
“We implemented Griffin-Lim in TensorFlow [22] hence it’s also part of the model.”
MethodSelf-citationGoogle's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
Wu, Y., Schuster, M., Chen, Z., Le, Q. V., Norouzi, M., Macherey, W., Krikun, M., Cao, Y., Gao, Q., Macherey, K., et al. (2016) · arXiv · doi:10.48550/arxiv.1609.08144 · Cited by 5,319 · cited 4×
“C L ] 6 A pr or machine translation (Wu et al., 2016), TTS outputs are continuous, and output sequences are usually much longer than those of the input.”
BackgroundMethodInfluentialSelf-citationWaveNet: a generative model for raw audio
Oord, A. V. D., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1609.03499 · Cited by 4,911 · cited 4×
“It could be used to predict alternative targets such as vocoder parameters, or as a WaveNetlike neural vocoder [9, 13, 10] that synthesizes waveform samples directly.”
BackgroundMethodInfluentialListen, attend and spell: A neural network for large vocabulary conversational speech recognition
Chan, W., Jaitly, N., Le, Q., & Vinyals, O. (2016) · 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2016.7472621 · Cited by 2,226 · cited 2×
“Moreover, unlike end-to-end speech recognition [4] or machine translation [5], TTS outputs are continuous, and output sequences are usually much longer than those of the input.”
BackgroundSelf-citation
Cited by
Items 801–900 of 1,002
Improving Multi-Speaker TTS Prosody Variance with a Residual Encoder and Normalizing Flows
Vallés-Ṕerez, I., Roth, J., Beringer, G., Barra-Chicote, R., & Droppo, J. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-562 · Cited by 12
Cross-lingual Transfer of Phonological Features for Low-resource Speech Synthesis
Wells, D., & Richmond, K. (2021) · ISCA Speech Synthesis Workshop (SSW) · doi:10.21437/ssw.2021-28 · Cited by 13
“We use a modified Tacotron 2 [11] architecture to predict acoustic features from text, based on the Mozilla TTS implementation [21].”
MethodImprove Cross-Lingual Text-To-Speech Synthesis on Monolingual Corpora with Pitch Contour Information
Zhan, H., Zhang, H., Ou, W., & Lin, Y. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-474 · Cited by 13
“We adopt Tacotron [1] as our baseline end-to-end cross-lingual TTS model, which consists of an encoder-decoder-with-attention framework.”
BackgroundMethodTransformer-Based Acoustic Modeling for Streaming Speech Synthesis
Wu, C., Xiu, Z., Shi, Y., Kalinli, O., Fuegen, C., Koehler, T., & He, Q. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-1655 · Cited by 10
“Existing acoustic models such as Tacotron2 [1, 2] use the Bi-directional Long Short-term Memory (BLSTM) recurrent networks.”
MethodDeep Learning for Videoconferencing: A Brief Examination of Speech to Text and Speech Synthesis
Tanberk, S., Dağlı, V., & Gürkan, M. K. (2021) · International Conference on Computer Science and Engineering (UBMK) · doi:10.1109/ubmk52708.2021.9558954 · Cited by 14
Data augmentation based non-parallel voice conversion with frame-level speaker disentangler
Chen, B., Xu, Z., & Yu, K. (2021) · Speech Communication · doi:10.1016/j.specom.2021.10.001 · Cited by 12
AC-VC: Non-Parallel Low Latency Phonetic Posteriorgrams Based Voice Conversion
Ronssin, D., & Cerňak, M. (2021) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru51503.2021.9688277 · Cited by 14
Accuth: Anti-Spoofing Voice Authentication via Accelerometer
Han, F., Yang, P., Du, H., & Li, X. (2022) · ACM Conference on Embedded Networked Sensor Systems (SenSys) · doi:10.1145/3560905.3568522 · Cited by 14
“, voice replay attack [29], voice conversion attack [5], and voice synthesis attack [49]), many defense mechanisms have been developed to distinguish human speakers and loudspeakers.”
BackgroundMulti-speaker Multi-style Speech Synthesis with Timbre and Style Disentanglement
Song, W. G., Yue, Y., Zhang, Y., Zhang, Z., Wu, Y., & He, X. (2023) · Communications in computer and information science · doi:10.1007/978-981-99-2401-1_12 · Cited by 12
Self-citationDriver Head Pose Detection From Naturalistic Driving Data
Chai, W., Chen, J., Wang, J., Velipasalar, S., Venkatachalapathy, A., Adu-Gyamfi, Y. O., Merickel, J., & Sharma, A. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3275070 · Cited by 10
“[43], [44] have been proposed in this area.”
BackgroundMethodRegeneration Learning: A Learning Paradigm for Data Generation
Tan, X., Qin, T., Bian, J., Liu, T., & Bengio, Y. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i20.30271 · Cited by 21
Adversarially Trained Autoencoders for Parallel-data-free Voice Conversion
Öçal, O., Elibol, O. H., Keskin, G., Stephenson, C., Thomas, A., & Ramchandran, K. (2019) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2019.8683204 · Cited by 10
“To complete the loop on voice conversion, after the conversion of spectrograms using the neural network, we use Griffin-Lim’s algorithm [13] to construct the audio signal similar to the way Tacotron [14] does.”
MethodSergio, G. C., & Lee, M. (2020) · Mathematical Problems in Engineering · doi:10.1155/2020/8478527 · Cited by 10
“In order to mitigate this issue and achieve smoother transitions between audio segments, our music generating module can be improved by using a network that is able to model audio in its raw waveform [41, 42].”
BackgroundText-to-Speech Synthesis Techniques for MIDI-to-Audio Synthesis
Cooper, E., Wang, X., & Yamagishi, J. (2021) · ISCA Speech Synthesis Workshop (SSW) · doi:10.21437/ssw.2021-23 · Cited by 10
Luong, H.-T., & Yamagishi, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.07414 · Cited by 11
“Strategy Layer(s) Number of speaker parameters Speaker bias Speaker scale Total A1b A1 128 0 128x1 A1B A1 full 0 256x1 A3a A3 128 128 256x1 A3A A3 full full 512x1 B1b B1 128 0 128x1 B1B B1 full 0 512x1 B8a B8 128 128 256x1 B8A B8 full full 1024x1 Bab B[1-8] 64 0 64x8 BaB B[1-8] full 0 512x8 Baa B[1-8] 64 64 128x8 BaA…”
BackgroundControllable and Lossless Non-Autoregressive End-to-End Text-to-SpeechPreprint
Liu, Z., Tian, Q., Hu, C., Liu, X., Wu, M., Wang, Y., Zhao, H., & Wang, Y. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2207.06088 · Cited by 10
“The general pipeline of two-stage methods is: first, generate the acoustic features (e.g., mel-spectrograms) from text autoregressively [36, 29, 24, 19, 34] or non-autoregressively [27, 26, 14, 23], then synthesize the raw waveforms conditioned on the acoustic features [10, 22, 25, 18].”
Self-citationA Proposal for Foley Sound Synthesis ChallengePreprint
Choi, K., Oh, S., Kang, M.-S., & McFee, B. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2207.10760 · Cited by 13
“In the more recent, data-driven approaches, the target sound categories have expanded significantly by researchers taking advantage of deep generative models such as GANs and WaveNet [15, 2, 3].”
BackgroundPathway to Future Symbiotic CreativityPreprint
Guo, Y., Liu, Q., Chen, J., Xue, W., Fu, J., Jensen, H. J., Rosas, F. E., Shaw, J., Wu, X., Zhang, J., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2209.02388 · Cited by 11
“RNN based generative models, such as Tacotron [33], DurIAN [34], and the self-attention based ones including TransformerTTS [35], FastSpeech [36], AdaSpeech [37], have become the mainstream approaches for building acoustic models, which tackles the problem using seq2seq modelling.”
MethodInfluentialText-to-Video: a Two-stage Framework for Zero-shot Identity-agnostic Talking-head GenerationPreprint
Wang, Z., Dai, M., & Lundgaard, K. T. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2308.06457 · Cited by 10
“A significant milestone in this domain was the development of Tacotron, which consisted of three key elements: an encoder, a decoder, and a vocoder [15].”
BackgroundMethodWave-U-Mamba: An End-To-End Framework For High-Quality And Efficient Speech Super Resolution
Lee, Y., & Kim, C. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10890511 · Cited by 9
“This two-step approach offers the advantage of decomposing the SSR task into simpler modules, akin to the traditional two-stage modeling employed in tasks such as Text-to-Speech (TTS) [10]–[12].”
SpeakEasy: Enhancing Text-to-Speech Interactions for Expressive Content Creation
Brade, S., Anderson, S., Kumar, R., Jin, Z., & Truong, A. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714263 · Cited by 9
“Rapid advancements came with the application of neural networks to audio synthesis like WaveNet [42], which presented a deep architecture for raw waveform generation, and later TacoTron [46], which achieved end-to-end single-speaker speech synthesis by leveraging a sequence-to-sequence approach.”
Dubbing Movies via Hierarchical Phoneme Modeling and Acoustic Diffusion Denoising
Li, L., Cong, G., Qi, Y., Zha, Z.-J., Wu, Q., Sheng, Q. Z., Huang, Q., & Yang, M. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3597267 · Cited by 17
Recalibrated Bandpass Filtering On Temporal Waveform For Audio Spoof Detection
Ren, Y., Liu, W., Liu, D., & Wang, L. (2021) · 2021 IEEE International Conference on Image Processing (ICIP) · doi:10.1109/icip42928.2021.9506427 · Cited by 9
“With the advancement of deep learning technology, recent TTS works like Wavenet [ 4], Tacotron [5] can already generate perceptually indistinguishable speech.”
MethodKim, J.-W., Yoon, H., & Jung, H. (2021) · IEEE Access · doi:10.1109/access.2021.3115608 · Cited by 10
“In other words, L spec influences the post-processing network which is trained to speculate the tendency of linear spectral magnitude obtained by estimating sampled spectral characteristics in the Mel-frequency scale with a linear frequency scale [44].”
BackgroundMethodInfluentialExpressive, Variable, and Controllable Duration Modelling in TTS
Abbas, S. A., Merritt, T., Moinet, A., Karlapati, S., Muszyńska, E., Slangen, S., Gatti, E., & Drugman, T. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-384 · Cited by 13
Choi, B. J., Jeong, M., Lee, J. Y., & Kim, N. S. (2022) · IEEE Signal Processing Letters · doi:10.1109/lsp.2022.3226655 · Cited by 17
“A S THE sample quality of the recently proposed neural text-to-speech (TTS) models, [1], [2], [3], [4], [5], [6], [7], [8], [9], [10], [11], approaches to the natural speech, the research interest has extended to high fidelity multi-speaker TTS systems, which enables the speech generation of multiple speakers via a si…”
BackgroundQiang, C., Yang, P., Che, H., Zhang, Y., Wang, X., & Wang, Z. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10095840 · Cited by 9
“With the development of deep learning, speech synthesis technology has rapidly advanced[1, 2, 3].”
BackgroundContextSpeech: Expressive and Efficient Text-to-Speech for Paragraph Reading
Xiao, Y., Zhang, S., Wang, X., Tan, X., He, L., Zhao, S., Soong, F. K., & Lee, T. (2023) · Interspeech (USB) · doi:10.21437/interspeech.2023-122 · Cited by 12
“Representative neural network-based acoustic models in TTS evolve from autoregressive structures ( e . g ., Tacotron [3, 4], Deepvoice [5], TransformerTTS [6]) to non-autoregressive frameworks ( e . g ., FastSpeech [7, 8], GlowTTS [9]) to achieve high quality generation efficiently.”
BackgroundSelf-citationAugmentation Techniques for Adult-Speech to Generate Child-Like Speech Data Samples at Scale
Yiwere, M. Y., Barcovschi, A., Jain, R., Cucu, H., & Corcoran, P. (2023) · IEEE Access · doi:10.1109/access.2023.3317360 · Cited by 9
“TTS [9], [10], [11], [12], [13], [14] and Speaker Recognition”
BackgroundInfluentialAdvancing AI Voice Synthesis: Integrating Emotional Expression in Multi-Speaker Voice Generation
Kolekar, S. S., Richter, D. J., Bappi, M. I., & Kim, K. (2024) · International Conference on Artificial Intelligence in Information and Communication (ICAIIC) · doi:10.1109/icaiic60209.2024.10463204 · Cited by 16
“In Tacotron [1] which is a auto-regressive model,WaveNet [10] is included as an attempt to directly synthesize phoneme sequences to speech inference.”
MethodT5G2P: Text-to-Text Transfer Transformer Based Grapheme-to-Phoneme Conversion
Řezáčková, M., Tihelka, D., & Matoušek, J. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3426332 · Cited by 9
“W ITH the rise of DNN-based speech synthesis and end-to-end systems such as Tacotron(2) [1], [2], Fast-Speech(2) [3], [4] or VITS [5] to name some, the grapheme-to-phoneme conversion (G2P), sometimes referred to phonetic transcription , is usually considered as “just working”.”
BackgroundMixed-EVC: Mixed Emotion Synthesis and Control in Voice Conversion
Zhou, K., Şişman, B., Busso, C., Ma, B., & Li, H. (2024) · The Speaker and Language Recognition Workshop (Odyssey) · doi:10.21437/odyssey.2024-26 · Cited by 11
Speech Generation for Indigenous Language Education
Pine, A., Cooper, E., Guzmán, D., Joanis, E., Kazantseva, A., Krekoski, R., Kühn, R., Larkin, S., Littell, P., Lothian, D., et al. (2024) · Computer Speech & Language · doi:10.1016/j.csl.2024.101723 · Cited by 23
Han, D., Cui, M., Kang, J., Wu, X., Liu, X., & Meng, H. M. L. (2024) · International Symposium on Chinese Spoken Language Processing (ISCSLP) · doi:10.1109/iscslp63861.2024.10800392 · Cited by 16
“Driven by the rapid development of deep learning, neural TTS systems such as [1, 2, 3, 4] have been widely applied in practical human-machine interactions.”
StyleSpeech: Parameter-efficient Fine Tuning for Pre-trained Controllable Text-to-Speech
Lou, H., Paik, H.-Y., Hu, W. S., & Yao, L. (2024) · ACM International Conference on Multimedia in Asia (MMAsia) · doi:10.1145/3696409.3700163 · Cited by 16
“FastSpeech [15, 16] family was subsequently proposed to address the word-skipping issue in long sequence inputs and to enhance the controllability of the synthesized speech.”
TTSlow: Slow Down Text-to-Speech With Efficiency Robustness Evaluations
Gao, X., Chen, Y., Yue, X., Tsao, Y., & Chen, N. F. (2025) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2025.3533357 · Cited by 13
“autoregressive TTS models [5], [6], [7], [8], [9] to the non-autoregressive TTS approaches [10], [11], [12], [13], [14], [15], [16], [17], [18], [19], neural vocoders [20], [21], [22], [23], [24] and the integration of diffusion models [25], [26], [27], [28], [29] as well as the speech style…”
Kim, M., Jung, J.-W., Rha, H., Maiti, S., Arora, S., Chang, X., Watanabe, S., & Ro, Y. M. (2026) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2026.3659297 · Cited by 10
“In this viewpoint, NMT techniques, back translation and network architecture, are incorporated in TMT.”
Phone-Level Embeddings for Unit Selection Speech Synthesis
Perquin, A., Lecorvé, G., Lolive, D., & Amsaleg, L. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00810-9_3 · Cited by 9
“End-to-end approaches such as [9] go further by predicting these waveforms directly from text, that is linguistic features are removed.”
BackgroundA Comparison of Expressive Speech Synthesis Approaches based on Neural Network
Xue, L., Zhu, X., An, X., & Xie, L. (2018) · doi:10.1145/3267935.3267947 · Cited by 9
Exploiting Deep Sentential Context for Expressive End-to-End Speech Synthesis
Yang, F., Yang, S., Wu, Q., Wang, Y., & Xie, L. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-2423 · Cited by 9
Self-citationLearning Syllable-Level Discrete Prosodic Representation for Expressive Speech Generation
Zhang, G., Qin, Y. H., & Lee, T. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-2228 · Cited by 9
Multi-Rate Attention Architecture for Fast Streamable Text-to-Speech Spectrum Modeling
He, Q., Xiu, Z., Koehler, T., & Wu, J. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414809 · Cited by 9
“Conventional high-quality spectrum models are usually based on the encoder-decoder attention framework [1,2,4].”
Hono, Y., Takaki, S., Hashimoto, K., Oura, K., Nankaku, Y., & Tokuda, K. (2021) · IEEE Access · doi:10.1109/access.2021.3118033 · Cited by 10
SUST TTS Corpus: A phonetically-balanced corpus for Bangla text-to-speech synthesis
Ahmad, A., Selim, M. R., Iqbal, M. Z., & Rahman, M. S. (2021) · Nippon Onkyo Gakkaishi/Acoustical science and technology/Nihon Onkyo Gakkaishi · doi:10.1250/ast.42.326 · Cited by 17
“TTS models based on Recurrent Neural Networks (RNNs) such as Tacotron [6], Tacotron 2 [7], Deep Voice [8], Deep Voice 2 [9] have shown impressive performance.”
MethodAi, Y., Ling, Z.-H., Wu, W.-L., & Li, A. (2022) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2022.3182268 · Cited by 9
HiFi-GAN based Text-to-Speech Synthesis in Serbian
Suzić, S., Pekar, D. J., Sečujski, M., Nosek, T., & Delić, V. (2022) · European Signal Processing Conference (EUSIPCO) · doi:10.23919/eusipco55093.2022.9909548 · Cited by 12
Xie, Q., Li, T., Wang, X., Wang, Z., Xie, L., Yu, G., & Wan, G. (2022) · International Symposium on Chinese Spoken Language Processing (ISCSLP) · doi:10.1109/iscslp57327.2022.10038056 · Cited by 9
“In recent years, enormous progress has been made in the neural text-to-speech (TTS), which benefits from the development of sequence-to-sequence (seq2seq) neural models [1, 2], making it possible to synthesize highly intelligible and natural speech [3, 4, 5, 6].”
BackgroundTranslating Braille into Chinese based on improved CBHG model
Huang, T., Su, W., Liu, L., Zhang, J., Cai, C., Yuan, Y., & Xu, C. (2023) · Displays · doi:10.1016/j.displa.2023.102445 · Cited by 14
Singing for the Missing: Bringing the Body Back to AI Voice and Speech Technologies
Cotton, K., de Vries, K., & Tatar, K. (2024) · International Conference on Movement and Computing (MOCO) · doi:10.1145/3658852.3659065 · Cited by 9
“[103, 154, 160] Automatic Speech Recognition is the processing of human speech into text.”
BackgroundFCTalker: Fine and Coarse Grained Context Modeling for Expressive Conversational Speech Synthesis
Hu, Y., Liu, R., Gao, G., & Li, H. (2024) · International Symposium on Chinese Spoken Language Processing (ISCSLP) · doi:10.1109/iscslp63861.2024.10800061 · Cited by 15
“With the advent of deep learning, neural TTS [5–8], i.e. Tacotron [5, 6], FastSpeech [7, 8] based models, has gained remarkable performance over the traditional statistical parametric speech synthesis methods [9, 10] in terms of Fig.”
Investigating Context Features Hidden in End-to-end TTS
Mametani, K., Kato, T., & Yamamoto, S. (2019) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2019.8683857 · Cited by 9
“Since the first release of the HTS, acoustic modeling in this approach has markedly improved due to the progress of its generative model from HMM to deep neural network [4, 5] and recurrent neural network, especially long short-term memory (LSTM) [6, 7].”
BackgroundMethodFong, J., Gallegos, P. O., Hodari, Z., & King, S. (2019) · Interspeech (USB) · doi:10.21437/interspeech.2019-1824 · Cited by 9
“Training with over 24 hours of data can produce voices with almost perfect naturalness [2].”
BackgroundThe Theory behind Controllable Expressive Speech Synthesis: A Cross-Disciplinary Approach
Tits, N., Haddad, K. E., & Dutoit, T. (2019) · IntechOpen eBooks · doi:10.5772/intechopen.89849 · Cited by 9
Modeling of Rakugo Speech and Its Limitations: Toward Speech Synthesis That Entertains Audiences
Kato, S., Yasuda, Y., Wang, X., Cooper, E., Takaki, S., & Yamagishi, J. (2020) · IEEE Access · doi:10.1109/access.2020.3011975 · Cited by 11
“In order to take prosodic characteristics of Japanese speech into consideration, we further enhance Tacotron 2 with self-attention (SA-Tacotron) for modeling rakugo speech because it is reported that the combination of Tacotron 1 2 [16] and self-attention outperforms Tacotron 1 in Japanese [17].”
BackgroundMethodAnalysis and Assessment of Controllability of an Expressive Deep Learning-Based TTS System
Tits, N., Haddad, K. E., & Dutoit, T. (2021) · Informatics · doi:10.3390/informatics8040084 · Cited by 11
Machine Learning Mitigants for Speech Based Cyber Risk
Campi, M., Peters, G. W., Azzaoui, N., & Matsui, T. (2020) · IEEE Access · doi:10.1109/access.2021.3117080 · Cited by 11
A Melody-Unsupervision Model for Singing Voice Synthesis
Choi, S., & Nam, J. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747422 · Cited by 15
Self-attention generative adversarial networks applied to conditional music generation
Neves, P. L. T., Fornari, J., & Florindo, J. B. (2022) · Multimedia Tools and Applications · doi:10.1007/s11042-022-12116-7 · Cited by 10
“Text-to-Speech models, such as those from the Tacotron family [35, 38, 39], receive textual inputs and generate speech corresponding to these inputs, often using intermediate representations to accomplish the task.”
BackgroundOn the Generalizability of Two-dimensional Convolutional Neural Networks for Fake Speech Detection
Papastergiopoulos, C., Vafeiadis, A., Papadimitriou, I., Votis, K., & Tzovaras, D. K. (2022) · doi:10.1145/3512732.3533585 · Cited by 10
“For the fake audio part, half of the unique sentences were converted to audio with Google Text-to-Speech (gTTS) library 8 and the other half with Tacotron-2 [27] and MelGAN [28] TTS pipeline.”
MethodInfluentialA Multi-Scale Time-Frequency Spectrogram Discriminator for GAN-based Non-Autoregressive TTS
Guo, H., Lu, H. M., Wu, X., & Meng, H. M. L. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-52 · Cited by 11
MnTTS: An Open-Source Mongolian Text-to-Speech Synthesis Dataset and Accompanied Baseline
Hu, Y., Yin, P., Liu, R., Bao, F., & Gao, G. (2022) · International Conference on Asian Language Processing · doi:10.1109/ialp57159.2022.9961271 · Cited by 11
“The recent proliferation of human-machine interaction applications, such as smart speaker, smart home, smart car assistant, has attracted a great deal of attention to TTS research from both academia and industry [9]–[11].”
BackgroundInfluentialReducing over-smoothness in speech synthesis using Generative Adversarial Networks
Sheng, L., & Pavlovskiy, E. (2019) · International Multi-Conference on Engineering, Computer and Information Sciences (SIBIRCON) · doi:10.1109/sibircon48586.2019.8957862 · Cited by 9
“Deep neural network based TTS system includes WaveNet [3], Char2Wav [4], DeepVoice1&2 [5], [6], and Tacotron1&2 [7], [8].”
BackgroundMethodContinuous Wavelet Vocoder-Based Decomposition of Parametric Speech Waveform Synthesis
Al-Radhi, M. S., Csapó, T. G., Zainkó, C., & Németh, G. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-1600 · Cited by 9
An ASR Guided Speech Intelligibility Measure for TTS Model SelectionPreprint
Baby, A., Vinnaitherthan, S., Adiga, N., Jawale, P., Badam, S., Adavanne, S., & Konjeti, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.01463 · Cited by 11
Continual Speaker Adaptation for Text-to-Speech SynthesisPreprint
Hemati, H., & Borth, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.14512 · Cited by 11
“With the success of recent neural TTS models [9, 10, 11], the desire for continually adding new speakers to an existing TTS model and proper methods for measuring forgetting, backward and forward transfer, etc.”
Learning De-identified Representations of Prosody from Raw AudioPreprint
Weston, J., Lenain, R., Meepegama, U., & Fristed, E. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.08248 · Cited by 15
“By contrast, wav2vec, vqwav2vec, wav2vec 2.0, TRILL, Tacotron and CHiVE all use fixed-length audio input (Schneider et al., 2019; Baevski et al., 2019; 2020; Shor et al., 2020; Wang et al., 2017; Wan et al., 2019).”
InfluentialText-To-Speech Data Augmentation for Low Resource Speech RecognitionPreprint
Zevallos, R. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2204.00291 · Cited by 12
FoundationTTS: Text-to-Speech for ASR Customization with Generative Language ModelPreprint
Xue, R., Liu, Y., He, L., Tan, X., Liu, L., Lin, E., & Zhao, S. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2303.02939 · Cited by 12
“Neural speech synthesis [9, 10, 11, 12, 13, 14, 15, 16, 17, 18] have showed close to human quality in terms of naturalness and audio fidelity in the past years, with cascaded or end-to-end transformer or LSTM architectures.”
BackgroundDEX-TTS: Diffusion-based EXpressive Text-to-Speech with Style Modeling on Time VariabilityPreprint
Park, H. J., Kim, J. S., Shin, W., & Han, S. W. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2406.19135 · Cited by 13
“Text-to-Speech (TTS) [1, 2, 3, 4] is the task of synthesizing natural speech from a given text, which is applied to various applications such as voice assistant services.”
BackgroundLee, S., Yoon, H.-W., Noh, H.-R., Kim, J., & Lee, S. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i14.17559 · Cited by 25
“We compared our models with GT, the converted audio from the mel-spectrogram of the GT, and other TTS models (Tacotron2, GST, Tansformer-based TTS, and FastSpeech2).”
MethodInfluentialCNN Based Automatic Speech Recognition: A Comparative Study
Ilgaz, H., Akkoyun, B., Alpay, Ö., & Akcayol, M. A. (2024) · ADCAIJ ADVANCES IN DISTRIBUTED COMPUTING AND ARTIFICIAL INTELLIGENCE JOURNAL · doi:10.14201/adcaij.29191 · Cited by 8
“Tacotron is a seq2seq model with attention.”
BackgroundMCDubber: Multimodal Context-Aware Expressive Video Dubbing
Zhao, Y., Jia, Z., Liu, R., Hu, D., Bao, F., & Gao, G. (2024) · Communications in computer and information science · doi:10.1007/978-981-96-1045-7_14 · Cited by 13
Power grid fault diagnosis based on an improved BERT model
Zheng, Y., Zheng, Y., Zhang, Z., Liu, B., Guo, Z., & Wei, J. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.128648 · Cited by 8
Wang, X., Delgado, H., Evans, N., Liu, X., Kinnunen, T., Tak, H., Lee, K. A., Kukanov, I., Sahidullah, M., Todisco, M., et al. (2026) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2026.3682962 · Cited by 12
“ASVspoof 2019 was the first to consider the detection of DNN-based spoofing attacks, i.e. deepfakes, generated using, e.g., WaveNet [18] and Tacotron [19].”
Liu, D.-R., Hsu, P.-C., Chen, Y., Huang, S.-F., Chuang, S.-P., Wu, D.-Y., & Lee, H.-Y. (2021) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2021.3138720 · Cited by 9
How Much Does Lookahead Matter for Disambiguation? Partial Arabic Diacritization Case Study
Esmail, S., Bar, K., & Dershowitz, N. (2022) · Computational Linguistics · doi:10.1162/coli_a_00456 · Cited by 8
“An encoder-decoder network using a Tacotron CBHG module (Wang et al. 2017) as part of the encoder has been introduced (Madhfar and Qamar 2020).”
MethodControllable speech synthesis by learning discrete phoneme-level prosodic representations
Ellinas, N., Christidou, M., Vioni, A., Sung, J. S., Chalamandaris, A., Tsiakoulis, P., & Mastorocostas, P. (2022) · Speech Communication · doi:10.1016/j.specom.2022.11.006 · Cited by 8
Lei, S., Zhou, Y., Chen, L., Wu, Z., Wu, X., Kang, S., & Meng, H. M. L. (2023) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2023.3301217 · Cited by 13
“Our proposed method can be easily extended to other TTS models, such as Tacotron2 [13].”
BackgroundInfluentialFastDiff 2: Revisiting and Incorporating GANs and Diffusion Models in High-Fidelity Speech Synthesis
Huang, R., Ren, Y., Jiang, Z. K., Cui, C., Liu, J., & Zhao, Z. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.437 · Cited by 20
“Most models (Wang et al., 2017; Ren et al., 2019; Huang et al.; Cui et al., 2021; Huang et al., 2023b; Ye et al., 2022) first convert input text or phoneme sequence into mel-spectrogram, and then transform it to waveform using a separately trained vocoder (Kumar et al., 2019; Kong et al., 2020a;…”
MethodEzzine, K., Martino, J. D., & Frikha, M. (2023) · Applied Sciences · doi:10.3390/app132111988 · Cited by 9
Zheng, J. P., Zhou, J., Zheng, W., Tao, L., & Kwan, H. K. (2024) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2024.3412152 · Cited by 9
“The rapid development of deep learning models has significantly advanced the text-to-speech (TTS) system to produce synthetic speech with high intelligibility from a linguistic perspective [3], [4], [5], [6], [7].”
Speech Audio Deepfake Detection via Convolutional Neural Networks
Valente, L. P., de Souza, M. M. S., & da Rocha, A. M. (2024) · IEEE Conference on Evolving and Adaptive Intelligent Systems (EAIS) · doi:10.1109/eais58494.2024.10569111 · Cited by 13
“Works as [14] are able to clone voices from five-seconds samples and [15] proposes a Tacotron neural network for speech synthesis with four parts: a CBHG Module, Encoder, Decoder [16] and a Post-Processing and Waveform Synthesis Network.”
BackgroundKim, S., & Lee, K. (2024) · IEEE Access · doi:10.1109/access.2024.3420433 · Cited by 8
Kurdish end-to-end speech synthesis using deep neural networks
Muhamad, S. S., Veisi, H., Mahmudi, A., Abdullah, A. A., & Rahimi, F. (2024) · Natural Language Processing Journal · doi:10.1016/j.nlp.2024.100096 · Cited by 17
U-Style: Cascading U-Nets With Multi-Level Speaker and Style Modeling for Zero-Shot Voice Cloning
Li, T., Wang, Z., Zhu, X., Cong, J., Tian, Q., Wang, Y., & Xie, L. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3453606 · Cited by 8
“T EXT-TO-SPEECH(TTS)systemshavemadearemarkable process in generating high-quality human-like speech for both the single-and multi-speaker scenarios [1], [2].”
BackgroundSelf-citationEnd-to-End Mandarin Speech Reconstruction Based on Ultrasound Tongue Images Using Deep Learning
Li, F., Shen, F., Ma, D., Zhou, J., Zhang, S., Wang, L., Fan, F., Liu, T., Chen, X., Toda, T., et al. (2024) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2024.3520498 · Cited by 8
SILA: Signal-to-Language Augmentation for Enhanced Control in Text-to-Audio Generation
Kumar, S., Seetharaman, P., Salamon, J., Manocha, D., & Nieto, O. (2025) · IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA) · doi:10.1109/waspaa66052.2025.11230964 · Cited by 10
“Recent advancements in audio generation predominantly utilize deep generative models, such as Generative Adversarial Networks (GANs) [11], [12] and Variational Autoencoders (VAEs) [13], to synthesize high-quality audio signals.”
Expressivity and Speech Synthesis
Triantafyllopoulos, A., & Schuller, B. W. (2026) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780198869450.013.0034 · Cited by 8
Perceptual Salience of Voice Source Parameters in Signaling Focal Prominence
Yanushevskaya, I., Murphy, A., Gobl, C., & Chasaide, A. N. (2016) · Interspeech (USB) · doi:10.21437/interspeech.2016-1160 · Cited by 9
Zhang, H., Ro, J. H., & Sproat, R. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.411 · Cited by 8
Taco-VC: A Single Speaker Tacotron based Voice Conversion with Limited Data
Leshem, R. L., & Giryes, R. (2020) · European Signal Processing Conference (EUSIPCO) · doi:10.23919/eusipco47968.2020.9287448 · Cited by 9
“TTS research has gained significant progress over the last years, mainly due to the adaptation of sequence-to-sequence (Seq2Seq) models such as Tacotron [19], [20].”
MethodMelGlow: Efficient Waveform Generative Network Based On Location-Variable Convolution
Zeng, Z., Wang, J., Cheng, N., & Xiao, J. Z. (2021) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt48900.2021.9383603 · Cited by 8
“Currently, the general speech synthesis pipeline is composed of two components [1, 2, 3, 4, 5, 6, 7, 8, 9]: (1) a speech feature prediction model that transforms the text into time-aligned speech features, for example mel-spectrum, and (2) a neural vocoder that generates raw waveform from speech features.”
Speech Synthesis Method Based on Tacotron2
Yang, L., Qin, D., & Zhang, J. (2021) · International Conference on Advanced Computational Intelligence (ICACI) · doi:10.1109/icaci52617.2021.9435882 · Cited by 8
“Tacotron [5] and Tacotron2 are conditional autoregressive models (CAR) with Teacher Forcing mechanism.”
BackgroundDetection and Analysis of Attention Errors in Sequence-to-Sequence Text-to-Speech
Valentini-Botinhao, C., & King, S. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-286 · Cited by 8
“[6] Y. Wang, R. Skerry-Ryan, D. Stanton, Y. Wu, R. J. Weiss, N. Jaitly, Z. Yang, Y. Xiao, Z. Chen, S. Bengio et al., “Tacotron: Towards end-to-end speech synthesis,” arXiv preprint arXiv:1703.10135, 2017.”
MethodInfluentialProsodyspeech: Towards Advanced Prosody Model for Neural Text-to-Speech
Yi, Y., He, L., Pan, S., Wang, X., & Xiao, Y. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9746744 · Cited by 12
Self-citationLow-Latency Incremental Text-to-Speech Synthesis with Distilled Context Prediction Network
Saeki, T., Takamichi, S., & Saruwatari, H. (2021) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru51503.2021.9687904 · Cited by 10
CASIA Voice Conversion System for the Voice Conversion Challenge 2020
Lian, Z., Tao, J., Wen, Z., & Zhong, R. (2020) · doi:10.21437/vcc_bc.2020-19 · Cited by 8
“The CBHG [18] conversion model has a bank of 1-”
BackgroundMethodInfluentialSpeaker-Independent Emotional Voice Conversion via Disentangled Representations
Chen, X., Xu, X., Chen, J., Zhang, Z., Takiguchi, T., & Hancock, E. R. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3222646 · Cited by 14
“To enlarge the receptive field and capture long-time-scale information, we also employed the ConvBank layer [43], which stacks convolution layers of different kernel sizes.”
MethodLearning Accent Representation with Multi-Level VAE Towards Controllable Speech Synthesis
Melechovský, J., Mehrish, A., Herremans, D., & Şişman, B. (2023) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt54892.2023.10023072 · Cited by 11
“The introduction of the attention mechanism [16] has allowed for autoregressive end-to-end models such as Tacotron [17], Tacotron2 [18], as well as non-autoregressive models such as FastSpeech [19], FastSpeech2 [20], and LightSpeech [21] to arise and push the performance boundaries.”
MethodDo, P., Coler, M., Dijkstra, J., & Klabbers, E. (2023) · Interspeech (USB) · doi:10.21437/interspeech.2023-2158 · Cited by 9
Metadata
- Authors
- Wang, Y., Skerry-Ryan, R. J., Stanton, D., Wu, Y., Weiss, R. J., Jaitly, N., Yang, Z., Xiao, Y., Chen, Z., Bengio, S., Le, Q., Agiomyrgiannakis, Y., Clark, R., & Saurous, R. A.
- Affiliations
- Google (United States) (US)
- Corresponding author
- Wang, Yuxuan
- ORCID
- Venue
- Interspeech 2017
- Publisher
- ISCA
- Published
- 2017-08-20
- Pages
- 4006-4010
- ISSN
- 2308-457X (ISSN-L)
- OpenAlex
- W2963609956
- Cited by
- 1,746
- References
- 25
- Citations by year
- 2026: 134 · 2025: 187 · 2024: 183 · 2023: 215 · 2022: 243 · 2021: 301 · 2020: 252 · 2019: 176 · 2018: 45 · 2017: 9
- FWCI
- 60.12
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Interspeech (USB) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Health Sciences › Health Professions › Speech and Hearing › Voice and Speech Disorders
- Keywords
- Tacotron, end-to-end speech synthesis, text-to-speech, sequence-to-sequence models, naturalness
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.