Tacotron: Towards End-to-End Speech Synthesis
The DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.
Abstract
A text-to-speech synthesis system typically consists of multiple stages, such as a text analysis frontend, an acoustic model and an audio synthesis module.Building these components often requires extensive domain expertise and may contain brittle design choices.In this paper, we present Tacotron, an end-to-end generative text-to-speech model that synthesizes speech directly from characters.Given pairs, the model can be trained completely from scratch with random initialization.We present several key techniques to make the sequence-tosequence framework perform well for this challenging task.Tacotron achieves a 3.82 subjective 5-scale mean opinion score on US English, outperforming a production parametric system in terms of naturalness.In addition, since Tacotron generates speech at the frame level, it's substantially faster than sample-level autoregressive methods.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The frameworks including their code as well as the datasets used that are available publicly (Sotelo et al., 2017; Wang et al., 2017; Ping et al., 2018; Shen et al., 2018; Vasquez and Lewis, 2019) are more likely to be used for nefarious purposes or research, so they will be more developed.”
From Audio deepfakes: A survey (2023) · cites this paper as Wang et al., 2017
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 2×
“We further pass the processed sequence to a few fixed-width 1-D convolutions, whose outputs are added with the original input sequence via residual connections [17].”
BackgroundAdam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 2×
“We use the Adam optimizer (Kingma & Ba, 2015) with learning rate decay, which starts from 0.001 and is reduced to 0.0005, 0.0003, and 0.0001 after 500K, 1M and 2M global steps, respectively.”
MethodInfluentialBatch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818 · cited 2×
“Batch normalization [18] is used for all convolutional layers.”
MethodSequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766 · cited 2×
“In this paper, we propose Tacotron, an end-to-end generative TTS model based on the sequence-to-sequence (seq2seq) (Sutskever et al., 2014) with attention paradigm (Bahdanau et al., 2014).”
MethodSelf-citationNeural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213 · cited 4×
“The backbone of Tacotron is a seq2seq model with attention [7, 14].”
MethodInfluentialEmpirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1412.3555 · Cited by 11,008 · cited 7×
“Second, we compare with a model with the CBHG encoder replaced by a 2-layer residual GRU encoder.”
BackgroundInfluentialTensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., et al. (2016) · arXiv · doi:10.48550/arxiv.1603.04467 · Cited by 9,408 · cited 2×
“We implemented Griffin-Lim in TensorFlow [22] hence it’s also part of the model.”
MethodSelf-citationGoogle's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
Wu, Y., Schuster, M., Chen, Z., Le, Q. V., Norouzi, M., Macherey, W., Krikun, M., Cao, Y., Gao, Q., Macherey, K., et al. (2016) · arXiv · doi:10.48550/arxiv.1609.08144 · Cited by 5,319 · cited 4×
“C L ] 6 A pr or machine translation (Wu et al., 2016), TTS outputs are continuous, and output sequences are usually much longer than those of the input.”
BackgroundMethodInfluentialSelf-citationWaveNet: a generative model for raw audio
Oord, A. V. D., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1609.03499 · Cited by 4,911 · cited 4×
“It could be used to predict alternative targets such as vocoder parameters, or as a WaveNetlike neural vocoder [9, 13, 10] that synthesizes waveform samples directly.”
BackgroundMethodInfluentialListen, attend and spell: A neural network for large vocabulary conversational speech recognition
Chan, W., Jaitly, N., Le, Q., & Vinyals, O. (2016) · 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2016.7472621 · Cited by 2,226 · cited 2×
“Moreover, unlike end-to-end speech recognition [4] or machine translation [5], TTS outputs are continuous, and output sequences are usually much longer than those of the input.”
BackgroundSelf-citation
Cited by
Items 601–700 of 1,002
RAMP: Retrieval-Augmented MOS Prediction via Confidence-based Dynamic Weighting
Wang, H., Zhao, S., Zheng, X., & Qin, Y. (2023) · Interspeech · doi:10.21437/interspeech.2023-851 · Cited by 27
“It consists of 20K TTS audio files generated from several Tacotron-like acoustic models [26] and an LPCNet vocoder [27].”
MethodQiang, C., Li, H., Ni, H., Qu, H., Fu, R., Wang, T., Wang, L., & Dang, J. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10446203 · Cited by 21
“Traditional speech synthesis methods have achieved satisfactory results[1, 2, 3, 4, 5, 6].”
BackgroundIbrahim, K. M., Perzo, A., & Leglaive, S. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10445740 · Cited by 21
“Recently, there has been a shift towards using end-to-end speech synthesis for its several advantages, e.g. alleviating the need for extensive feature engineering, as well as allowing rich conditioning on various attributes, such as speaker and emotion [21].”
Voice Presentation Attack Detection through Text-Converted Voice Command Analysis
Kwak, I., Huh, J. H., Han, S. T., Kim, I., & Yoon, J. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300828 · Cited by 16
“The attacker uses existing, easily accessible voice training tools like Google’s Tacotron [30, 34] or Baidu’s DeepVoice [2, 28], trains the victim’s voice biometric models, and generates a series of new, malicious voice commands using that model.”
MethodNeural iTTS: Toward Synthesizing Speech in Real-time with End-to-end Neural Text-to-Speech Framework
Yanagita, T., Sakti, S., & Nakamura, S. (2019) · ISCA Speech Synthesis Workshop (SSW) · doi:10.21437/ssw.2019-33 · Cited by 16
“Recently, end-to-end TTS systems have been proposed [18, 19], based on seq2seq with attention [20].”
BackgroundMethodSinging Synthesis: With a Little Help from my Attention
Angelini, O., Moinet, A., Yanagisawa, K., & Drugman, T. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-1399 · Cited by 16
Speaker Adaptation of a Multilingual Acoustic Model for Cross-Language Synthesis
Himawan, I., Aryal, S., Ouyang, I. C., Kang, S., Lanchantin, P., & King, S. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9053642 · Cited by 18
“such as Char2Wav [5] and Tacotron [6] directly map the input text into acoustic features.”
BackgroundGenerating Multilingual Voices Using Speaker Space Translation Based on Bilingual Speaker Data
Maiti, S., Marchi, E., & Conkie, A. D. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9054305 · Cited by 21
Liu, Z., & Mak, B. K.-W. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-1464 · Cited by 16
“Char2wav [3] and Tacotron [4] and its improved version Tacotron2 [5] resort to a totally end-to-end neural model1 that uses an attention mechanism to convert a sequence of text directly to its corresponding sequence of vocoder features, from which speech audios may be generated using a vocoder.”
BackgroundMethodInfluentialResearch on Voice Cloning with a Few Samples
Li, Z., & Chen, F. (2020) · International Conference on Computer Network, Electronic and Automation (ICCNEA) · doi:10.1109/iccnea50255.2020.00073 · Cited by 16
“The synthesizer is based on optimized Tacotron 2.”
BackgroundMethodInfluentialExemplar-Based Emotive Speech Synthesis
Wu, X., Cao, Y., Lu, H., Liu, S., Kang, S., Wu, Z., Liu, X., & Meng, H. M. L. (2021) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2021.3052688 · Cited by 16
Development and Evaluation of Speech Synthesis System Based on Deep Learning Models
Valizada, A., Jafarova, S., Sultanov, E., & Rustamov, S. (2021) · Symmetry · doi:10.3390/sym13050819 · Cited by 17
“According to [16], the architecture of the model consists of an encoder, an attention-based decoder, and a post-processing net (Figure 1).”
MethodProsody and Voice Factorization for Few-Shot Speaker Adaptation in the Challenge M2voc 2021
Wang, T., Fu, R., Yi, J., Tao, J., Wen, Z., Qiang, C., & Wang, S. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414427 · Cited by 16
“With the development of deep neural networks (DNN) [1], a significant amout of efforts has been made to improve the naturalness of text-to-speech (TTS), such as Tacotron [2–4], wavenet [5].”
BackgroundNeural HMMS Are All You Need (For High-Quality Attention-Free TTS)
Mehta, S., Székely, É., Beskow, J., & Henter, G. E. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9746686 · Cited by 16
Glow-WaveGAN 2: High-quality Zero-shot Text-to-speech Synthesis and Any-to-any Voice Conversion
Lei, Y., Yang, S., Cong, J., Xie, L., & Su, D. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-684 · Cited by 16
Yasuda, Y., & Toda, T. (2022) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3190672 · Cited by 16
“In [53], mel-spectrogram was used to pre-train BERT, which is used as a pre-net network [30] in TTS.”
MethodAdapter-Based Extension of Multi-Speaker Text-To-Speech Model for New Speakers
Hsieh, C.-P., Ghosh, S., & Ginsburg, B. (2023) · Interspeech (USB) · doi:10.21437/interspeech.2023-2313 · Cited by 20
Comparison of the Ability of Neural Network Model and Humans to Detect a Cloned Voice
Milewski, K., Zaporowski, S., & Czyżewski, A. (2023) · Electronics · doi:10.3390/electronics12214458 · Cited by 18
Influence of Artificial Intelligence on Activities and Competitiveness of an Organization
Wodecki, A. (2018) · Springer eBooks · doi:10.1007/978-3-319-91596-8_3 · Cited by 16
Bird, J. J., Faria, D. R., Premebida, C., Ekárt, A., & Ayrosa, P. P. S. (2020) · IEEE International Conference on Autonomous Robot Systems and Competitions · doi:10.1109/icarsc49921.2020.9096166 · Cited by 16
“Models such as Tacotron [21] learn to produce audio spectrograms from training data in order to perform realistic text-to-speech from either textual representation or internationally recognised phonemes.”
MethodSequence-to-Sequence Articulatory Inversion Through Time Convolution of Sub-Band Frequency Signals
Shahrebabaki, A. S., Siniscalchi, S. M., Salvi, G., & Svendsen, T. K. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-1140 · Cited by 16
“ASR [24, 25], speech synthesis [26], and machine translation [27].”
Multi-Speaker Emotional Speech Synthesis with Fine-Grained Prosody Modeling
Chunhui, L., Wen, X., Liu, R., & Chen, X. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9413398 · Cited by 16
Prosody Transfer in Neural Text to Speech Using Global Pitch and Loudness FeaturesPreprint
Gururani, S., Gupta, K., Shah, D., Shakeri, Z., & Pinto, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.09645 · Cited by 16
“The learned prosody embedding is used as a global conditioning vector for Tacotron’s decoder [9].”
MethodImproving Accent Conversion with Reference Encoder and End-To-End Text-To-SpeechPreprint
Li, W., Tang, B., Yin, X., Zhao, Y., Li, W., Wang, K., Huang, H. M., Wang, Y., & Ma, Z. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.09271 · Cited by 16
“Additionally, to further improve performance, we optimize model architecture by using the GMM attention [19] and CBHG encoder [20], which can get stable alignment and powerful feature representation.”
MethodSelf-citationOne TTS Alignment To Rule Them AllPreprint
Badlani, R., Łańcucki, A., Shih, K. J., Valle, R., Ping, W., & Catanzaro, B. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.10447 · Cited by 16
Takin: A Cohort of Superior Quality Zero-shot Speech Generation ModelsPreprint
Chen, S., Feng, Y., He, L., He, T., He, W., Hu, Y., Lin, B., Lin, Y., Pan, Y., Tan, P., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2409.12139 · Cited by 16
“Takin TTS shows a large superiority comparing to conventional neural speech synthesis methods [55, 56, 57, 58, 59], which revolutionizes the field of AI audiobook generation.”
BackgroundProsodic Representation Learning and Contextual Sampling for Neural Text-to-Speech
Karlapati, S., Abbas, A., Hodari, Z., Moinet, A., Joly, A., Karanasou, P., & Drugman, T. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9413696 · Cited by 15
“Neural text-to-speech (NTTS) techniques have significantly improved the naturalness of speech produced by TTS systems [1–6].”
BackgroundOut Domain Data Augmentation on Punjabi Children Speech Recognition using Tacotron
Hasija, T., Kadyan, V., & Guleria, K. (2021) · Journal of Physics Conference Series · doi:10.1088/1742-6596/1950/1/012044 · Cited by 15
“Tacotron can be used for designing new sounds from the text which will help us in the ASR system [12][13].”
BackgroundMethodRobust Audio Watermarking Against Manipulation Attacks Based on Deep Learning
Wen, S., Zhang, Q., Hu, T., & Li, J. (2024) · IEEE Signal Processing Letters · doi:10.1109/lsp.2024.3501285 · Cited by 17
“Modern speech synthesis models such as WaveNet [2], Tacotron [3], and other Deepfake techniques are capable of generating highly realistic synthetic audio and video content [4].”
RUSLAN: Russian Spoken Language Corpus for Speech Synthesis
Gabdrakhmanov, L., Garaev, R., & Razinkov, E. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-26061-3_12 · Cited by 15
Learning Neural Trans-Dimensional Random Field Language Models with Noise-Contrastive Estimation
Wang, B., & Ou, Z. (2018) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2018.8461813 · Cited by 15
Advancing Speech Synthesis using EEG
Krishna, G., Tran, C., Carnahan, M., & Tewfik, A. H. (2021) · International IEEE/EMBS Conference on Neural Engineering · doi:10.1109/ner49283.2021.9441306 · Cited by 19
“The attention models have been used before for tasks like automatic speech recognition (ASR) [12] and speech synthesis (producing sound from the text) [13] where the inputs and outputs are of different lengths, whereas in our case the input and output features are of the same length since both”
MethodBenchmarking Deep Learning Models for Classification of Book Covers
Lucieri, A., Sabir, H., Siddiqui, S. A., Rizvi, S. T. R., Iwana, B. K., Uchida, S., Dengel, A. R., & Ahmed, S. (2020) · SN Computer Science · doi:10.1007/s42979-020-00132-z · Cited by 15
Prosodic Prominence and Boundaries in Sequence-to-Sequence Speech Synthesis
Suni, A., Kakouros, S., Vainio, M., & Šimko, J. (2020) · Speech prosody · doi:10.21437/speechprosody.2020-192 · Cited by 15
“) have been replaced by “end-to-end” systems, such as Tacotron and its derivatives [1, 2, 3].”
BackgroundMethodInfluentialTransfer Learning in Speaker’s Age and Gender Recognition
Markitantov, M. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-60276-5_32 · Cited by 15
Towards Fine-Grained Prosody Control for Voice Conversion
Lian, Z., Zhong, R., Wen, Z., Liu, B., & Tao, J. (2021) · International Symposium on Chinese Spoken Language Processing (ISCSLP) · doi:10.1109/iscslp49672.2021.9362110 · Cited by 15
“Then, a CBHG [12] conversion model is trained to map the input F to the output acoustic features.”
MethodInfluentialRecurrent Phase Reconstruction Using Estimated Phase Derivatives from Deep Neural Networks
Thieling, L., Wilhelm, D., & Jax, P. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9413722 · Cited by 15
Cloning One’s Voice Using Very Limited Data in the Wild
Dai, D., Chen, Y., Chen, L., Tu, M., Liu, L., Xia, R., Tian, Q., Wang, Y., & Wang, Y. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747319 · Cited by 18
Self-citationWang, C., Li, Z., Tang, B., Yin, X., Wan, Y., Yu, Y., & Ma, Z. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-10316 · Cited by 15
On-Device Neural Speech Synthesis
Achanta, S., Antony, A., Golipour, L., Li, J., Raitio, T., Rasipuram, R., Rossi, F., Shi, J., Upadhyay, J., Winarsky, D., et al. (2021) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru51503.2021.9688154 · Cited by 22
Vocbench: A Neural Vocoder Benchmark for Speech Synthesis
AlBadawy, E. A., Gibiansky, A., He, Q., Wu, J., Chang, M., & Lyu, S. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9746698 · Cited by 22
Decoding Knowledge Transfer for Neural Text-to-Speech Training
Liu, R., Şişman, B., Gao, G., & Li, H. (2022) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2022.3171974 · Cited by 18
First Attempt of Gender-free Speech Style Transfer for Genderless Robot
Yu, C., Fu, C., Chen, R., & Tapus, A. (2022) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1109/hri53351.2022.9889533 · Cited by 19
“Recently, supervised and unsupervised text-to-speech (TTS) synthesis and speech style transfer have attracted many AI and Robotics researchers’ attention to the development of speech processing research and Natural Language Processing (NLP) research [6] [7] [8] [9].”
BackgroundImproving the Efficiency of Dysarthria Voice Conversion System Based on Data Augmentation
Zheng, W.-Z., Han, J.-Y., Chen, C.-Y., Chang, Y.-J., & Lai, Y.-H. (2023) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2023.3331524 · Cited by 15
“More specifically, text-to-speech (TTS) [24] technology is used to synthesize the speech of multiple normal target speakers.”
BackgroundSemi-Supervised Speaker Adaptation for End-to-End Speech Synthesis with Pretrained Models
Inoue, K., Hara, S., Abe, M., Hayashi, T., Yamamoto, R., & Watanabe, S. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9053371 · Cited by 19
“As an end-to-end TTS, Tacotron [1] that addresses the mapping between character and Mel-spectrogram was proposed.”
BackgroundLaughter Synthesis: Combining Seq2seq Modeling with Transfer Learning
Tits, N., Haddad, K. E., & Dutoit, T. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-1423 · Cited by 15
Speaking Speed Control of End-to-End Speech Synthesis Using Sentence-Level Conditioning
Bae, J.-S., Bae, H., Joo, Y.-S., Lee, J., Lee, G.-H., & Cho, H. Y. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-1361 · Cited by 15
Peking Opera Synthesis via Duration Informed Attention Network
Wu, Y., Li, S., Yu, C., Lu, H., Weng, C., Zhang, L., & Yu, D. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-1724 · Cited by 15
Self-citationEnd-to-End Image-to-Speech Generation for Untranscribed Unknown Languages
Effendi, J., Sakti, S., & Nakamura, S. (2021) · IEEE Access · doi:10.1109/access.2021.3071541 · Cited by 15
“As a simple baseline, we combined two popular neural network models: ResNet-50 [24] and Tacotron [26].”
MethodInfluentialDesot, T., Portet, F., & Vacher, M. (2022) · Computer Speech & Language · doi:10.1016/j.csl.2022.101369 · Cited by 24
PJS: phoneme-balanced Japanese singing voice corpusPreprint
Koguchi, J., & Takamichi, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.02959 · Cited by 15
“Because singing-voice synthesis typically uses explicit temporal alignments (unlike end-to-end text-to-speech synthesis), we also made a temporal alignment between a singing voice and a phoneme sequence [27], [28].”
BackgroundUnsupervised Audiovisual Synthesis via Exemplar AutoencodersPreprint
Deng, K.-L., Bansal, A., & Ramanan, D. (2020) · International Conference on Learning Representations · doi:10.48550/arxiv.2001.04463 · Cited by 18
“Sequence-to-sequence (Seq2seq) structures (Sutskever et al., 2014) have led to significant advancements in TTS systems (Oord et al., 2016; Shen et al., 2018; Wang et al., 2017).”
BackgroundGELP: GAN-Excited Linear Prediction for Speech Synthesis from Mel-Spectrogram
Juvela, L., Bollepalli, B., Yamagishi, J., & Alku, P. (2019) · Interspeech (USB) · doi:10.21437/interspeech.2019-2008 · Cited by 14
Emotional Prosody Control for Speech Generation
Sivaprasad, S., Kosgi, S., & Gandhi, V. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-307 · Cited by 18
“Neural network models have made significant improvements in enhancing the quality of generated speech, and most state-of-the-art TTS systems like Deep Voice[1], Tacotron[2], and Fastspeech2[3] generate natural sounding voice.”
BackgroundInfluentialContribution of Timbre and Shimmer Features to Deepfake Speech Detection
Chaiwongyen, A., Songsriboonsit, N., Duangpummet, S., Karnjana, J., Kongprawechnon, W., & Unoki, M. (2022) · Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) · doi:10.23919/apsipaasc55919.2022.9980281 · Cited by 15
Uzbek Speech Synthesis Using Deep Learning Algorithms
Abdullaeva, M., Juraev, D., Ochilov, M., & Rakhimov, M. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-27199-1_5 · Cited by 14
OverFlow: Putting flows on top of neural transducers for better TTS
Mehta, S., Kirkland, A., Lameris, H., Beskow, J., Székely, É., & Henter, G. E. (2023) · Interspeech (USB) · doi:10.21437/interspeech.2023-1996 · Cited by 17
“Our experiments use the decoder from Glow-TTS, which leverages an architecture based on non-causal CNNs, evoking similarities to the post-nets in attention-based neural TTS [13, 12, 14].”
MethodM 3 TTS: Multi-modal text-to-speech of multi-scale style control for dubbing
Liu, Y., Wei, L.-F., Qian, X., Zhang, T., Chen, S.-L., & Yin, X.-C. (2024) · Pattern Recognition Letters · doi:10.1016/j.patrec.2024.02.005 · Cited by 18
Zero-Shot Voice Cloning Text-to-Speech for Dysphonia Disorder Speakers
Azizah, K. (2024) · IEEE Access · doi:10.1109/access.2024.3396377 · Cited by 21
“Several deep neural network (DNN)-based TTS studies have achieved excellent quality in recent years [6], [7], [8], [9], [10], [11], [12], [13], [14], [15], [16], [17].”
BackgroundMulti-Scale Alignment and Contextual History for Attention Mechanism in Sequence-to-Sequence Model
Tjandra, A., Sakti, S., & Nakamura, S. (2018) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt.2018.8639528 · Cited by 14
Comprehensive Evaluation of Statistical Speech Waveform Synthesis
Merritt, T., Putrycz, B., Nadolski, A., Ye, T., Korzekwa, D., Dolecki, W., Drugman, T., Klimkov, V., Moinet, A., Breen, A., et al. (2018) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt.2018.8639556 · Cited by 14
Spoken Language Understanding of Human-Machine Conversations for Language Learning Applications
Qian, Y., Ubale, R., Lange, P. L., Evanini, K., Ramanarayanan, V., & Soong, F. K. (2019) · Journal of Signal Processing Systems · doi:10.1007/s11265-019-01484-3 · Cited by 15
Efficient clustering for aggregate loads: An unsupervised pretraining based method
Ruhang, X. (2020) · Energy · doi:10.1016/j.energy.2020.118617 · Cited by 14
Effective Emotion Transplantation in an End-to-End Text-to-Speech System
Joo, Y.-S., Bae, H., Kim, Y.-I., Cho, H. Y., & Kang, H.-G. (2020) · IEEE Access · doi:10.1109/access.2020.3021758 · Cited by 14
“In particular, end-to-end framework-based TTS models that infer acoustic features directly from input character sequences without laborious feature-engineering tasks have shown great success [6]–[8].”
BackgroundSpoofprint: A New Paradigm for Spoofing Attacks Detection
Chen, T., & Khoury, E. (2021) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt48900.2021.9383572 · Cited by 15
“Extremely high quality spoof voice can be generated by deep learning based text-to-speech synthesis (TTS) [1, 2] and voice conversion (VC) techniques [3, 4].”
BackgroundConvolutional neural network of deep learning in computer vision and image classification problems
Tokarev, K. E., Зотов, В. М., Николаевна, Х. В., & Родионова, О. Н. (2021) · IOP Conference Series Earth and Environmental Science · doi:10.1088/1755-1315/786/1/012040 · Cited by 14
“Researchers attribute this degradation problem to the complexity of optimizing deep neural networks [1-5, 10].”
BackgroundLearning Explicit Prosody Models and Deep Speaker Embeddings for Atypical Voice Conversion
Wang, D., Liu, S., Sun, L., Wu, X., Liu, X., & Meng, H. M. L. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-285 · Cited by 14
VAENAR-TTS: Variational Auto-Encoder Based Non-AutoRegressive Text-to-Speech Synthesis
Lu, H., Wu, Z., Wu, X., Li, X., Kang, S., Liu, X., & Meng, H. M. L. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-2121 · Cited by 20
Cross-Lingual Low Resource Speaker Adaptation Using Phonological Features
Maniati, G., Ellinas, N., Μαρκόπουλος, Κ., Vamvoukakis, G., Sung, J. S., Park, H.-M., Chalamandaris, A., & Tsiakoulis, P. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-327 · Cited by 14
“Recently proposed neural architectures [1, 2] have shown that an efficient end-to-end acoustic model is possible by directly consuming text characters.”
BackgroundNovitasari, S., Sakti, S., & Nakamura, S. (2022) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2022.3196879 · Cited by 15
“Under clean conditions, neural TTS successfully synthesize a highly natural speech given only a text [17]–[20].”
BackgroundMethodAudio-Driven Dubbing for User Generated Contents via Style-Aware Semi-Parametric Synthesis
Song, L., Wu, W., Fu, C., Loy, C. C., & He, R. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3210002 · Cited by 24
“We also adopt Conv1d layers to handle the time-serial relevance of the produced phoneme distributions, and a ConvBank layer to capture the short-term and long-term speech information [50].”
MethodTDASS: Target Domain Adaptation Speech Synthesis Framework for Multi-speaker Low-Resource TTS
Zhang, X., Wang, J., Cheng, N., & Xiao, J. Z. (2022) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn55064.2022.9892596 · Cited by 17
“Tacotron is the first end-to-end TTS model [6].”
BackgroundMethodTransformer-Based Speech Synthesizer Attribution in an Open Set Scenario
Bartusiak, E. R., & Delp, E. J. (2022) · International Conference on Machine Learning and Applications · doi:10.1109/icmla55696.2022.00054 · Cited by 22
“However, these initial investigations were unsuccessful at attributing Tacotron 2 signals.”
MethodInfluentialMixGAN-TTS: Efficient and Stable Speech Synthesis Based on Diffusion Model
Deng, Y., Wu, N., Qiu, C., Luo, Y., & Chen, Y. (2023) · IEEE Access · doi:10.1109/access.2023.3283772 · Cited by 16
“problems such as word skipping and repeating [8], [9], [10].”
BackgroundSubband wavenet with overlapped single-sideband filterbanks
Okamoto, T., Tachibana, K., Toda, T., Shiga, Y., & Kawai, H. (2017) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru.2017.8269005 · Cited by 14
“Furthermore, the proposed method can be directly applied to other raw audio generative models, such as SampleRNN [20] and Tacotron [23].”
BackgroundMethodInfluentialPhonemic-level Duration Control Using Attention Alignment for Natural Speech Synthesis
Park, J., Han, K., Jeong, Y., & Lee, S. W. (2019) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2019.8683827 · Cited by 14
Knowledge-Based Linguistic Encoding for End-to-End Mandarin Text-to-Speech Synthesis
Li, J., Wu, Z., Li, R., Zhi, P., Yang, S., & Meng, H. M. L. (2019) · Interspeech (USB) · doi:10.21437/interspeech.2019-1118 · Cited by 14
“[4] Y. Wang, R. J. Skerry-Ryan, D. Stanton, Y. Wu, R. J. Weiss, N. Jaitly, Z. Yang, Y. Xiao, Z. Chen, S. Bengio, Q. Le, Y. Agiomyrgiannakis, R. Clark, and R. A. Saurous, “Tacotron: Towards End-to-End Speech Synthesis,” arXiv:1703.10135 [cs], Mar. 2017, arXiv: 1703.10135.”
BackgroundMethodInfluentialMonteiro, J., & Alam, J. (2019) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru46091.2019.9003792 · Cited by 17
“With recent advances in speech synthesis and voice conversion approaches based on deep neural networks, including, for instance, end-to-end direct waveform modeling, and generative adversarial networks (GANS) [1, 2, 3, 4], and due to the availability of open-source tool-kits online, voice-based attacks generated by sp…”
BackgroundA Bangla Text-to-Speech System using Deep Neural Networks
Raju, R. S., Bhattacharjee, P., Ahmad, A., & Rahman, M. S. (2019) · International Conference on Bangla Speech and Language Processing (ICBSLP) · doi:10.1109/icbslp47725.2019.202055 · Cited by 14
“Example of such attempts are WaveNet [23], Tacotron [24], and Tacotron 2 [25] from Google, Deep Voice 1 [26], 2 [27], 3 [28] from Baidu, Char2Wav [29] from MILA, FastSpeech [30] from Microsoft, etc.”
BackgroundYang, S., Wang, Y., & Xie, L. (2020) · IEEE Signal Processing Letters · doi:10.1109/lsp.2020.3025410 · Cited by 14
Self-citationArabic Speech Synthesis using Deep Neural Networks
Ali, A. H., Magdy, M., Alfawzy, M., Ghaly, M., & Abbas, H. M. (2021) · International Conference on Communications, Signal Processing, and their Applications (ICCSPA) · doi:10.1109/iccspa49915.2021.9385731 · Cited by 14
Litesing: Towards Fast, Lightweight and Expressive Singing Voice Synthesis
Zhuang, X., Jiang, T., Chou, S.-Y., Wu, B., Hu, P. F., & Lui, S. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414043 · Cited by 19
NatiQ: An End-to-end Text-to-Speech System for Arabic
Abdelalí, A., Durrani, N., Demiroğlu, C., Dalvi, F., Mubarak, H., & Darwish, K. M. (2022) · Arabic Natural Language Processing Workshop (WANLP) · doi:10.18653/v1/2022.wanlp-1.38 · Cited by 14
Bollepalli, B., Juvela, L., Airaksinen, M., Valentini-Botinhao, C., & Alku, P. (2019) · Speech Communication · doi:10.1016/j.specom.2019.04.008 · Cited by 14
“However, recent more advanced neural networks, such as sequence-to-sequence models, are able to learn the duration implicitly within models (for example, see Tacotron [64], DeepVoice [6], and Char2Wav [47]).”
BackgroundNeumann Optimizer: A Practical Optimization Algorithm for Deep Neural NetworksPreprint
Krishnan, S. M., Xiao, Y., & Saurous, R. A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.03298 · Cited by 16
“4 Negative result for sequence-to-sequence RNNs We also tried our algorithm on a large-scale sequence-to-sequence speech-synthesis model called Tacotron ([40]), where we were unable to obtain any speedup or quality improvements.”
MethodSelf-citationAdaSpeech 3: Adaptive Text to Speech for Spontaneous StylePreprint
Yan, Y., Tan, X., Li, B., Zhang, G., Qin, T., Zhao, S., Shen, Y., Zhang, W.-Q., & Liu, T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.02530 · Cited by 18
“While many models have been designed [2, 4] to synthesize reading-style speech and good quality has been achieved, the synthesis of spontaneous style, which usually occurs in scenarios like conversation, talk show or podcast [7], has not been well studied.”
BackgroundDistinguishable Speaker Anonymization Based on Formant and Fundamental Frequency Scaling
Yao, J., Wang, Q., Lei, Y., Guo, P., Xie, L., Wang, N., & Liu, J. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10095120 · Cited by 23
“We use the CBHG module [17], which does not contain an attention mechanism, as the encoder because the BN and the acoustic features are initially aligned.”
MethodESPnet-ST-v2: Multipurpose Spoken Language Translation Toolkit
Yan, B., Shi, J., Tang, Y., Inaguma, H., Peng, Y., Dalmia, S., Polák, P., Fernandes, P., Berrebbi, D., Hayashi, T., et al. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-demo.38 · Cited by 13
Huang, R., Zhang, C., Ren, Y., Zhao, Z., & Yu, D. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.508 · Cited by 20
“Most works (Wang et al., 2017; Min et al., 2021; Yang et al., 2021a) utilize regression losses (e.”
BackgroundSpeaker Adaptive Text-to-Speech With Timbre-Normalized Vector-Quantized Feature
Du, C., Guo, Y., Chen, X., & Yu, K. (2023) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2023.3308374 · Cited by 18
“Compared with traditional statistical parametric speech synthesis [21], neural TTS model [22], [23] based on deep neural network shows a better performance.”
MethodRestoration of Bone-Conducted Speech With U-Net-Like Model and Energy Distance Loss
Li, C., Yang, F., & Yang, J. J. (2023) · IEEE Signal Processing Letters · doi:10.1109/lsp.2023.3347149 · Cited by 13
“As aforementioned, the restoration of bone-conducted speech is a conditional generation task, which is similar to text-to-speech synthesis (TTS) but is quite different from the speech denoising [25].”
BackgroundReflow-TTS: A Rectified Flow Model for High-Fidelity Text-to-Speech
Guan, W., Su, Q., Zhou, H., Miao, S., Xie, X., Li, L., & Hong, Q. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10447822 · Cited by 34
The Effect of Voice and Repair Strategy on Trust Formation and Repair in Human-Robot Interaction
Romeo, M., Torre, I., Maguer, S. L., Sleat, A., Cangelosi, A., & Leite, I. (2025) · ACM Transactions on Human-Robot Interaction · doi:10.1145/3711938 · Cited by 14
AI-determined similarity increases likability and trustworthiness of human voices
Jaggy, O., Schwan, S., & Meyerhoff, H. S. (2025) · PLoS ONE · doi:10.1371/journal.pone.0318890 · Cited by 14
“For the first as well as the following Experiments, we used an open-source encoder [65] based on research conducted by [22,66,67].”
Dubiel, M., Halvey, M. J., Azzopardi, L. A., & Daronnat, S. (2018) · Strathprints: The University of Strathclyde institutional repository (University of Strathclyde) · Cited by 23
Aso, M., Takamichi, S., Takamune, N., & Saruwatari, H. (2020) · Speech Communication · doi:10.1016/j.specom.2020.09.003 · Cited by 13
Haque, K. N., Rana, R. K., Liu, J., Hansen, J. H. L., Cummins, N., Busso, C., & Schuller, B. W. (2021) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2021.3098764 · Cited by 14
“Researchers have also used seq2seq-based models for acoustic modelling (text to acoustic features) such as TACOTRON (1 & 2) [19], [20] and Char2Wav [21].”
FastPitchFormant: Source-Filter Based Decomposed Modeling for Speech Synthesis
Bak, T., Bae, J.-S., Bae, H., Kim, Y.-I., & Cho, H. Y. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-866 · Cited by 19
Futamata, K., Park, B., Yamamoto, R., & Tachibana, K. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-252 · Cited by 20
Residual-Guided Personalized Speech Synthesis based on Face Image
Wang, J., Wang, Z., Hu, X., Li, X., Fang, Q., & Liu, L. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9746808 · Cited by 23
Metadata
- Authors
- Wang, Y., Skerry-Ryan, R. J., Stanton, D., Wu, Y., Weiss, R. J., Jaitly, N., Yang, Z., Xiao, Y., Chen, Z., Bengio, S., Le, Q., Agiomyrgiannakis, Y., Clark, R., & Saurous, R. A.
- Affiliations
- Google (United States) (US)
- Corresponding author
- Wang, Yuxuan
- ORCID
- Venue
- Interspeech 2017
- Publisher
- ISCA
- Published
- 2017-08-20
- Pages
- 4006-4010
- ISSN
- 2308-457X (ISSN-L)
- OpenAlex
- W2963609956
- Cited by
- 1,746
- References
- 25
- Citations by year
- 2026: 134 · 2025: 187 · 2024: 183 · 2023: 215 · 2022: 243 · 2021: 301 · 2020: 252 · 2019: 176 · 2018: 45 · 2017: 9
- FWCI
- 60.12
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Interspeech (USB) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Health Sciences › Health Professions › Speech and Hearing › Voice and Speech Disorders
- Keywords
- Tacotron, end-to-end speech synthesis, text-to-speech, sequence-to-sequence models, naturalness
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.