MultiSpeech: Multi-Speaker Text to Speech with Transformer
Open accessAbstract
Transformer-based text to speech (TTS) model (e.g., Transformer TTS [1], FastSpeech [2]) has shown the advantages of training and inference efficiency over Tacotron [3]) due to its parallel computation in training and/or inference.However, the parallel computation increases the difficulty while learning the alignment between text and speech in Transformer, which is further magnified in the multi-speaker scenario with noisy data and diverse speakers, and hinders the applicability of Transformer for multi-speaker TTS.In this paper, we develop a robust and high-quality multispeaker Transformer TTS system called MultiSpeech, with several specially designed components/techniques to improve textto-speech alignment: 1) a diagonal constraint on the weight matrix of encoder-decoder attention in both training and inference; 2) layer normalization on phoneme embedding in encoder to better preserve position information; 3) a bottleneck in decoder pre-net to prevent copy between consecutive speech frames.Experiments on VCTK and LibriTTS multi-speaker datasets demonstrate the effectiveness of MultiSpeech: 1) it synthesizes more robust and better quality multi-speaker voice than naive Transformer based TTS; 2) with a MutiSpeech model as the teacher, we obtain a strong multi-speaker FastSpeech model with almost zero quality degradation while enjoying extremely fast inference speed.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Several ideas introduced recently include random sampling, learning concept dictionary, uniform masking, patch projection, joint learning of a region detector, and representation aligning before mask prediction.
“Several ideas introduced recently include random sampling [113], learning concept dictionary [203], uniform masking [270], patch projection [192], joint learning of a region detector [271], and representation aligning before mask prediction [263].”
From Multimodal learning with transformers: A survey (2023) · cites this paper as [113]
Recent remedies for fine-grained semantic alignment include random sampling, learning a concept dictionary, jointly learning a region detector, and representation aligning before mask prediction.
“Several remedies introduced recently include random sampling [113], learning concept dictionary [203], jointly learning a region detector [271], and representation aligning before mask prediction [263].”
From Multimodal learning with transformers: A survey (2023) · cites this paper as [113]
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
Jakubův, J., Chvalovský, K., Goertzel, Z., et al. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,648 · see citing works
Layer NormalizationPreprint
Ba, J., Kiros, J., & Hinton, G. E. (2016) · arXiv.org · doi:10.48550/arxiv.1607.06450 · Cited by 13,745 · see citing works
“Librispeech: an ASR corpus based on public domain audio books,”
Panayotov, V., Chen, G., Povey, D., & Khudanpur, S. (2015) · 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2015.7178964 · Cited by 6,072 · see citing works
WaveNet: a generative model for raw audio
Oord, A. V. D., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1609.03499 · Cited by 4,911 · see citing works
Natural TTS Synthesis by Conditioning Wavenet on MEL Spectrogram Predictions
Shen, J., Pang, R., Weiss, R. J., Schuster, M., Jaitly, N., Yang, Z., Chen, Z., Zhang, Y., Wang, Y., Skerrv-Ryan, R., et al. (2018) · 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2018.8461368 · Cited by 2,791 · see citing works
Attention-Based Models for Speech Recognition
Chorowski, J., Bahdanau, D., Serdyuk, D., Cho, K., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1506.07503 · Cited by 2,021 · see citing works
Tacotron: towards end-to-end speech synthesis
Wang, Y., Skerry-Ryan, R. J., Stanton, D., Wu, Y., Weiss, R. J., Jaitly, N., Yang, Z., Xiao, Y., Chen, Z., Bengio, S., et al. (2017) · Interspeech 2017 · doi:10.21437/interspeech.2017-1452 · Cited by 1,746 · see citing works
Morgan, M. C. (2025) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.17575540 · Cited by 1,375 · see citing works
LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speech
Zen, H., Dang, V., Clark, R., Zhang, Y., Weiss, R. J., Ye, J., Chen, Z., & Wu, Y. (2019) · Interspeech (USB) · doi:10.21437/interspeech.2019-2441 · Cited by 828 · see citing works
Neural Speech Synthesis with Transformer Network
Li, N., Liu, S., Liu, Y., Zhao, S., & Liu, M. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33016706 · Cited by 754 · see citing works
Self-citationFastSpeech: Fast, Robust and Controllable Text to SpeechPreprint
Ren, Y., Ruan, Y., Tan, X., Qin, T., Zhao, S., Zhao, Z., & Liu, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.09263 · Cited by 724 · see citing works
Self-citationTransfer learning from speaker verification to multispeaker text-to-speech synthesis
Jia, Y., Zhang, Y., Weiss, R. J., Wang, Q., Shen, J., Ren, F., Chen, Z., Nguyen, P., Pang, R., Moreno, I. L., et al. (2018) · arXiv · doi:10.48550/arxiv.1806.04558 · Cited by 519 · see citing works
SUPERSEDED - CSTR VCTK Corpus: English Multi-speaker Corpus for CSTR Voice Cloning Toolkit
Veaux, C., Yamagishi, J., & MacDonald, K. (2016) · University of Edinburgh. The Centre for Speech Technology Research (CSTR) · doi:10.7488/ds/1495 · Cited by 434 · see citing works
Towards End-to-End Prosody Transfer for Expressive Speech Synthesis with TacotronPreprint
Skerry-Ryan, R., Battenberg, E., Xiao, Y., Wang, Y., Stanton, D., Shor, J., Weiss, R. J., Clark, R., & Saurous, R. A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.09047 · Cited by 348 · see citing works
Deep voice 3: scaling text-to-speech with convolutional sequence learning
Ping, W., Peng, K., Gibiansky, A., Arik, S. O., Kannan, A., Narang, S., Raiman, J., & Miller, J. (2017) · arXiv · doi:10.48550/arxiv.1710.07654 · Cited by 313 · see citing works
Tachibana, H., Uenoyama, K., & Aihara, S. (2018) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2018.8461829 · Cited by 277 · see citing works
FastSpeech: Fast, Robust and Controllable Text to Speech
Ren, Y. (2024) · TIB Data Manager · doi:10.57702/jd8hw0cw · Cited by 256 · see citing works
Self-citationHierarchical Generative Modeling for Controllable Speech SynthesisPreprint
Hsu, W.-N., Zhang, Y., Weiss, R. J., Zen, H., Wu, Y., Wang, Y., Cao, Y. J., Ye, J., Chen, Z., Shen, J., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.07217 · Cited by 165 · see citing works
Tacotron: Towards End-to-End Speech SynthesisPreprint
Wang, Y., Skerry-Ryan, R., Stanton, D., Wu, Y., Weiss, R. J., Jaitly, N., Yang, Z., Xiao, Y., Chen, Z., Samy Bengio, A., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.10135 · Cited by 123 · see citing works
Forward Attention in Sequence- To-Sequence Acoustic Modeling for Speech Synthesis
Zhang, J., Ling, Z.-H., & Dai, L.-R. (2018) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2018.8462020 · Cited by 82 · see citing works
Robust Sequence-to-Sequence Acoustic Modeling with Stepwise Monotonic Attention for Neural TTS
He, M., Deng, Y., & He, L. (2019) · Interspeech (USB) · doi:10.21437/interspeech.2019-1972 · Cited by 76 · see citing works
Token-Level Ensemble Distillation for Grapheme-to-Phoneme Conversion
Sun, H., Tan, X., Gan, J.-W., Liu, H., Zhao, S., Qin, T., & Liu, T. (2019) · Interspeech (USB) · doi:10.21437/interspeech.2019-1208 · Cited by 51 · see citing works
Self-citationLibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speech
Zen, H., Dang, V., Clark, R., Zhang, Y., Weiss, R. J., Jia, Y., Chen, Z., & Wu, Y. (2019) · arXiv · doi:10.48550/arxiv.1904.02882 · Cited by 23 · see citing works
Yasuda, Y., Wang, X., & Yamagishi, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.11535 · Cited by 5 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Multimodal learning with transformers: A survey
Xu, P., Zhu, X., & Clifton, D. A. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3275156 · Cited by 1,072 · see citing works
A survey on neural speech synthesis
Tan, X., Qin, T., Soong, F., & Liu, T.-Y. (2021) · arXiv · doi:10.48550/arxiv.2106.15561 · Cited by 216 · see citing works
Self-citationHi-Fi Multi-Speaker English TTS Dataset
Bakhturina, E., Lavrukhin, V., Ginsburg, B., & Zhang, Y. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-1599 · Cited by 82 · see citing works
AI-Synthesized Voice Detection Using Neural Vocoder Artifacts
Sun, C., Jia, S., Hou, S., & Lyu, S. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00097 · Cited by 81 · see citing works
Transformers in speech processing: Overcoming challenges and paving the future
Latif, S., Zaidi, S. A. M., Cuayáhuitl, H., Shamshad, F., Shoukat, M., Usama, M., & Qadir, J. (2025) · Computer Science Review · doi:10.1016/j.cosrev.2025.100768 · Cited by 68 · see citing works
StyleTTS: A Style-Based Generative Model for Natural and Diverse Text-to-Speech Synthesis
Li, Y. A., Han, C., & Mesgarani, N. (2025) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2025.3530171 · Cited by 64 · see citing works
Adaspeech 2: Adaptive Text to Speech with Untranscribed Data
Yan, Y., Tan, X., Li, B., Qin, T., Zhao, S., Shen, Y., & Liu, T. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414872 · Cited by 45 · see citing works
Self-citationFine-Grained Style Control In Transformer-Based Text-To-Speech Synthesis
Chen, L., & Rudnicky, A. I. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747747 · Cited by 35 · see citing works
ESPnet2-TTS: Extending the Edge of TTS ResearchPreprint
Hayashi, T., Yamamoto, R., Yoshimura, T., Wu, P., Shi, J., Saeki, T., Ju, Y., Yasuda, Y., Takamichi, S., & Watanabe, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.07840 · Cited by 35 · see citing works
Grad-StyleSpeech: Any-Speaker Adaptive Text-to-Speech Synthesis with Diffusion Models
Kang, M., Min, D., & Hwang, S. J. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10095515 · Cited by 34 · see citing works
WSRGlow: A Glow-Based Waveform Generative Model for Audio Super-Resolution
Zhang, K., Ren, Y., Xu, C., & Zhao, Z. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-892 · Cited by 33 · see citing works
Self-citationEMOVIE: A Mandarin Emotion Speech Dataset with a Simple Emotional Text-to-Speech Model
Cui, C., Ren, Y., Liu, J., Chen, F., Huang, R., Lei, M., & Zhao, Z. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-1148 · Cited by 32 · see citing works
Self-citationChen, W., Tan, X., Xia, Y., Qin, T., Wang, Y., & Liu, T.-Y. (2020) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3394171.3413623 · Cited by 28 · see citing works
Self-citationGANSpeech: Adversarial Training for High-Fidelity Multi-Speaker Speech Synthesis
Yang, J., Bae, J.-S., Bak, T., Kim, Y.-I., & Cho, H. Y. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-971 · Cited by 27 · see citing works
Jain, R., Yiwere, M. Y., Bigioi, D., Corcoran, P., & Cucu, H. (2022) · IEEE Access · doi:10.1109/access.2022.3170836 · Cited by 26 · see citing works
Controllable Accented Text-to-Speech Synthesis With Fine and Coarse-Grained Intensity Rendering
Liu, R., Şişman, B., Gao, G., & Li, H. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3378110 · Cited by 25 · see citing works
Accented Text-to-Speech Synthesis With Limited Data
Zhou, X., Zhang, M., Zhou, Y., Wu, Z., & Li, H. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3363414 · Cited by 24 · see citing works
Multilingual Byte2Speech Models for Scalable Low-resource Speech SynthesisPreprint
He, M., Yang, J., He, L., & Soong, F. K. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.03541 · Cited by 24 · see citing works
Zhou, Y., Song, C., Li, X., Zhang, L., Wu, Z., Bian, Y., Su, D., & Meng, H. M. L. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-10054 · Cited by 21 · see citing works
MIST-Tacotron: End-to-End Emotional Speech Synthesis Using Mel-Spectrogram Image Style Transfer
Moon, S., Kim, S., & Choi, Y. (2022) · IEEE Access · doi:10.1109/access.2022.3156093 · Cited by 21 · see citing works
Neural Dubber: Dubbing for Videos According to ScriptsPreprint
Hu, C., Tian, Q., Li, T., Wang, Y.-P., Wang, Y., & Zhao, H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.08243 · Cited by 21 · see citing works
fairseq Sˆ2: A Scalable and Integrable Speech Synthesis Toolkit
Wang, C., Hsu, W.-N., Adi, Y., Polyak, A., Lee, A., Chen, P., Gu, J., & Pino, J. (2021) · Conference on Empirical Methods in Natural Language Processing: System Demonstrations (EMNLP) · doi:10.18653/v1/2021.emnlp-demo.17 · Cited by 19 · see citing works
Speech Bert Embedding for Improving Prosody in Neural TTS
Chen, L., Deng, Y., Wang, X., Soong, F. K., & He, L. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9413864 · Cited by 19 · see citing works
AraSpell: A Deep Learning Approach for Arabic Spelling CorrectionPreprint
Salhab, M., & Abu-Khzam, F. N. (2023) · Research Square · doi:10.21203/rs.3.rs-2974359/v1 · Cited by 18 · see citing works
MM-TTS: Multi-Modal Prompt Based Style Transfer for Expressive Text-to-Speech Synthesis
Guan, W., Li, Y., Li, T., Huang, H., Wang, F., Lin, J., Huang, L., Li, L., & Hong, Q. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i16.29769 · Cited by 17 · see citing works
Bhushan, S., Mishra, V. P., Rishiwal, V., Sharmila, A., & Agarwal, U. (2025) · IEEE Access · doi:10.1109/access.2025.3605236 · Cited by 16 · see citing works
TDASS: Target Domain Adaptation Speech Synthesis Framework for Multi-speaker Low-Resource TTS
Zhang, X., Wang, J., Cheng, N., & Xiao, J. Z. (2022) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn55064.2022.9892596 · Cited by 14 · see citing works
Novitasari, S., Sakti, S., & Nakamura, S. (2022) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2022.3196879 · Cited by 12 · see citing works
Federated Learning with Dynamic Transformer for Text to Speech
Hong, Z., Wang, J., Qu, X., Liu, J., Zhao, C., & Xiao, J. (2021) · Interspeech 2021 · doi:10.21437/interspeech.2021-2039 · Cited by 11 · see citing works
MuSE-SVS: Multi-Singer Emotional Singing Voice Synthesizer That Controls Emotional Intensity
Kim, S. Y., Kim, Y., Jun, J., & Kim, I. (2023) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2023.3294712 · Cited by 11 · see citing works
Zero-Shot Normalization Driven Multi-Speaker Text to Speech Synthesis
Kumar, N., Narang, A., & Lall, B. (2022) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2022.3169634 · Cited by 11 · see citing works
Learning to Maximize Speech Quality Directly Using MOS Prediction for Neural Text-to-Speech
Choi, Y., Jung, Y., Suh, Y.-J., & Kim, H. (2022) · IEEE Access · doi:10.1109/access.2022.3175810 · Cited by 11 · see citing works
Mandeel, A. R., Al-Radhi, M. S., & Csapó, T. G. (2022) · Multimedia Tools and Applications · doi:10.1007/s11042-022-14005-5 · Cited by 11 · see citing works
SC-CNN: Effective Speaker Conditioning Method for Zero-Shot Multi-Speaker Text-to-Speech Systems
Yoon, H., Kim, C., Um, S., Yoon, H.-W., & Kang, H.-G. (2023) · IEEE Signal Processing Letters · doi:10.1109/lsp.2023.3277786 · Cited by 11 · see citing works
CrossSpeech++: Cross-Lingual Speech Synthesis With Decoupled Language and Speaker Generation
Kim, J.-H., Yang, H.-S., Ju, Y.-C., Kim, I.-H., Kim, B.-Y., & Chung, J. S. (2025) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2025.3547231 · Cited by 7 · see citing works
Ahn, Y., Chae, J.-W., & Shin, J. W. (2025) · IEEE Signal Processing Letters · doi:10.1109/lsp.2025.3537949 · Cited by 7 · see citing works
Self supervised learning for robust voice cloning
Klapsas, K., Ellinas, N., Nikitaras, K., Vamvoukakis, G., Kakoulidis, P., Μαρκόπουλος, Κ., Raptis, S., Sung, J. S., Jho, G., Chalamandaris, A., et al. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-10856 · Cited by 6 · see citing works
Generating the Voice of the Interactive Virtual Assistant
Stan, A., & Lőrincz, B. (2021) · IntechOpen eBooks · doi:10.5772/intechopen.95510 · Cited by 6 · see citing works
Lee, S., Yoon, H.-W., Noh, H.-R., Kim, J., & Lee, S. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i14.17559 · Cited by 5 · see citing works
Low-Resource Speech Synthesis with Speaker-Aware Embedding
Yang, L.-J., Yeh, I.-P., & Chien, J. (2022) · International Symposium on Chinese Spoken Language Processing (ISCSLP) · doi:10.1109/iscslp57327.2022.10038221 · Cited by 5 · see citing works
VITS: Quality Vs. Speed Analysis
Matoušek, J., & Tihelka, D. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-40498-6_19 · Cited by 5 · see citing works
Lőrincz, B., Stan, A., & Giurgiu, M. (2021) · Procedia Computer Science · doi:10.1016/j.procs.2021.08.078 · Cited by 5 · see citing works
VoiceX as a Design Tool for Virtual Agents' Voices
Don, D. S. W., Lingenfelser, F., Kuch, J. M., Grothe, O., Schlagowski, R., André, E., & Mertes, S. (2025) · ACM International Conference on Intelligent Virtual Agents (IVA) · doi:10.1145/3717511.3749296 · Cited by 4 · see citing works
Towards An Integrated Approach for Expressive Piano Performance Synthesis from Music Scores
Tang, J., Cooper, E., Wang, X., Yamagishi, J., & Fazekas, G. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10890623 · Cited by 4 · see citing works
Speaker verification-derived loss and data augmentation for DNN-based multispeaker speech synthesis
Lőrincz, B., Stan, A., & Giurgiu, M. (2021) · European Signal Processing Conference (EUSIPCO) · doi:10.23919/eusipco54536.2021.9616305 · Cited by 4 · see citing works
Zhou, X., Zhang, M., Zhou, Y., Wu, Z., & Li, H. (2026) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2026.3657306 · Cited by 3 · see citing works
Tan, X. (2023) · Artificial intelligence: foundations, theory, and algorithms/Artificial intelligence: Foundations, theory, and algorithms · doi:10.1007/978-981-99-0827-1_5 · Cited by 3 · see citing works
Self-citationData Augmentation based Cross-Lingual Multi-Speaker TTS using DL with Sentiment Analysis
Lalitha, B., Madhurima, V., Ch, N., Babu, J. S., Sekhar, J. N. C., & Reddy, P. V. (2023) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3628428 · Cited by 3 · see citing works
Advanced Trans-BiGRU-QA Fusion Model for Atmospheric Mercury Prediction
Shih, D., Chung, F.-I., Wu, T.-W., Wang, B.-H., & Shih, M. (2024) · Mathematics · doi:10.3390/math12223547 · Cited by 3 · see citing works
Wu, Q., Shen, Q., Luan, J., & Wang, Y. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747032 · Cited by 3 · see citing works
Meta-StyleSpeech: Multi-speaker adaptive text-to-speech generation
Min, D. (2025) · TIB Data Manager · doi:10.57702/fqzsmwin · Cited by 3 · see citing works
Dubbing Movies via Hierarchical Phoneme Modeling and Acoustic Diffusion Denoising
Li, L., Cong, G., Qi, Y., Zha, Z.-J., Wu, Q., Sheng, Q. Z., Huang, Q., & Yang, M. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3597267 · Cited by 2 · see citing works
Novitasari, S., Sakti, S., & Nakamura, S. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10096128 · Cited by 2 · see citing works
ACCELERATION OF TRANSFORMER ARCHITECTURES ON JETSON XAVIER USING TENSORRT
Nikoghosyan, K. H., Khachatryan, T. B., Harutyunyan, E. A., & Galstyan, D. M. (2023) · doi:10.53297/18293336-2023.2-30 · Cited by 2 · see citing works
Munthe, M. V. R., Sitanggang, A., Herman, H., & Mobo, F. D. (2023) · Dharmas Education Journal (DE_Journal) · doi:10.56667/dejournal.v4i2.1064 · Cited by 2 · see citing works
Facetron: A Multi-Speaker Face-to-Speech Model Based on Cross-Modal Latent Representations
Um, S., Kim, J., Lee, J., & Kang, H.-G. (2023) · European Signal Processing Conference (EUSIPCO) · doi:10.23919/eusipco58844.2023.10290115 · Cited by 2 · see citing works
Al-Radhi, M. S., Ibrahim, O., Mandeel, A. R., Csapó, T. G., & Németh, G. (2023) · International Conference on Speech Technology and Human-Computer Dialogue (SpeD) · doi:10.1109/sped59241.2023.10314948 · Cited by 2 · see citing works
Improving Language Model-Based Zero-Shot Text-to-Speech Synthesis with Multi-Scale Acoustic Prompts
Lei, S., Zhou, Y., Chen, L., Luo, D., Wu, Z., Wu, X., Kang, S., Jiang, T., Zhou, Y., Han, Y., et al. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10447815 · Cited by 2 · see citing works
FastSpeech2 Based Japanese Emotional Speech Synthesis
Ikeda, M., & Markov, K. (2024) · IEEE International Conference on Intelligent Systems · doi:10.1109/is61756.2024.10705252 · Cited by 2 · see citing works
DeepMine-multi-TTS: a Persian speech corpus for multi-speaker text-to-speech
Adibian, M., Zeinali, H., & Barmaki, S. (2025) · Language Resources and Evaluation · doi:10.1007/s10579-025-09807-6 · Cited by 2 · see citing works
End-to-End Multi-Speaker FastSpeech2 With Hierarchical Decoder
Adibian, M., & Zeinali, H. (2025) · IEEE Access · doi:10.1109/access.2025.3589120 · Cited by 2 · see citing works
Maghfur, N. M., Ibrohim, M. O., Fahmi, J., Putera, A. S., & Riandi, O. (2021) · International Conference of Computer and Informatics Engineering (IC2IE) · doi:10.1109/ic2ie53219.2021.9649417 · Cited by 2 · see citing works
FCH-TTS: Fast, Controllable and High-quality Non-Autoregressive Text-to-Speech Synthesis
Zhou, X. Y., Zhou, Z., & Shi, X. (2022) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn55064.2022.9892512 · Cited by 2 · see citing works
Document-Level Neural TTS Using Curriculum Learning and Attention Masking
Hwang, S.-W., & Chang, J. (2021) · IEEE Access · doi:10.1109/access.2020.3049073 · Cited by 2 · see citing works
StoryForge: AI-Powered Narrative with Dynamic Imagery and Voice
Sri Chandrasekharendra Saraswathi Viswa Mahavidyalaya Enathur, K., & DThirunavukkarasu, M. (2024) · INTERANTIONAL JOURNAL OF SCIENTIFIC RESEARCH IN ENGINEERING AND MANAGEMENT · doi:10.55041/ijsrem32973 · Cited by 1 · see citing works
Enhanced Prosody Modeling and Character Voice Controlling for Audiobook Speech Synthesis
Wu, N.-Q., & Ling, Z.-H. (2025) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3749644 · Cited by 1 · see citing works
TrInk: Ink Generation with Transformer Network
Jin, Z., Desai, S., Chen, X., Fang, B., Huang, Z., Li, Z., Gan, C.-X., Tu, X., Mak, M., Lu, Y., et al. (2025) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2025.emnlp-main.244 · Cited by 1 · see citing works
RFGETT-TTS: Robust Fine-Grained Expressivity Transfer With Transformer for Text-to-Speech Synthesis
Hamed, M., Missaoui, I., & Lachiri, Z. (2025) · IEEE Access · doi:10.1109/access.2025.3648672 · Cited by 1 · see citing works
Zero-Shot Voice Cloning Using Variational Embedding with Attention Mechanism
Lee, J., Kim, J., & Chang, J. (2021) · IEEE International Conference on Network Infrastructure and Digital Content (IC-NIDC) · doi:10.1109/ic-nidc54101.2021.9660599 · Cited by 1 · see citing works
A Contrastive Analysis Of Prefix In English And Batak
Sidabalok, F. Y., Purba, D. N., Sihombing, K., & Sitanggang, A. (2023) · PIJAR Jurnal Pendidikan dan Pengajaran · doi:10.58540/pijar.v1i3.381 · Cited by 1 · see citing works
Expressivity Transfer In Transformer-Based Text-To-Speech Synthesis
Hamed, M., & Lachiri, Z. (2024) · International Conference on Advanced Technologies for Signal and Image Processing · doi:10.1109/atsip62566.2024.10638975 · Cited by 1 · see citing works
Fine-Grained Prosody Transfer Text-to-Speech Synthesis with Transformer
Hamed, M., & Lachiri, Z. (2024) · International Conference in Electronic Engineering & Information Technology (EEITE) · doi:10.1109/eeite61750.2024.10654450 · Cited by 1 · see citing works
Rhythm-controllable Attention with High Robustness for Long Sentence Speech Synthesis
Ke, D., Deng, Y., Jia, Y., Xue, J., Luo, Q., Li, Y., Sun, J., Liang, J., & Lin, B. (2022) · International Symposium on Chinese Spoken Language Processing (ISCSLP) · doi:10.1109/iscslp57327.2022.10037822 · Cited by 1 · see citing works
Speech BERT Embedding For Improving Prosody in Neural TTSPreprint
Chen, L., Deng, Y., Wang, X., Soong, F. K., & He, L. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.04312
Effective and Differentiated Use of Control Information for Multi-speaker Speech Synthesis.Preprint
Wu, Q., Shen, Q., Luan, J., & Wang, Y. (2021) · arXiv (Cornell University)
Mispronunciation Detection and Correction via Discrete Acoustic Units.Preprint
Zhang, Z., Wang, Y., & Yang, J. (2021) · arXiv (Cornell University)
Perceptually Guided End-to-End Text-to-Speech With MOS PredictionPreprint
Choi, Y., Jung, Y., Suh, Y.-J., & Kim, H. (2020) · arXiv (Cornell University)
Tan, X. (2023) · Artificial intelligence: foundations, theory, and algorithms/Artificial intelligence: Foundations, theory, and algorithms · doi:10.1007/978-981-99-0827-1_10
Self-citationExpressive and Controllable TTS
Tan, X. (2023) · Artificial intelligence: foundations, theory, and algorithms/Artificial intelligence: Foundations, theory, and algorithms · doi:10.1007/978-981-99-0827-1_8
Self-citationTan, X. (2023) · Artificial intelligence: foundations, theory, and algorithms/Artificial intelligence: Foundations, theory, and algorithms · doi:10.1007/978-981-99-0827-1_9
Self-citationPruning Self-Attention for Zero-Shot Multi-Speaker Text-to-Speech
Yoon, H., Kim, C., Song, E., Yoon, H.-W., & Kang, H.-G. (2023) · Interspeech (USB) · doi:10.21437/interspeech.2023-1301
Generating Speech with Prosodic Prominence based on SSL-Visually Grounded Models
Hartanti, B. S. I., Tanaya, D., Azizah, K., Lestari, D. P., Purwarianti, A., & Sakti, S. (2023) · International Conference on Speech Database and Assessments · doi:10.1109/o-cocosda60357.2023.10482965
Bridging Facial Imagery and Vocal Reality: Stable Diffusion-Enhanced Voice Generation
Lin, Y., Liu, D., Xu, Y., Suo, H., & Li, M. (2024) · International Symposium on Chinese Spoken Language Processing (ISCSLP) · doi:10.1109/iscslp63861.2024.10800185
Dynamic Speech Generation to Enhance Intelligibility in Noisy Environments
Simantiraki, O., Markaki, M., & Pantazis, Y. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10887886
Briefing of Text-to-Speech Trends Toward Zero-Shot Multi-Speaker Synthesis
Song, D., & Lee, J. (2025) · International Conference on Consumer Electronics · doi:10.1109/icce63647.2025.10930170
Deshmukh, R., Bhangale, H., Khambe, H., Shinde, A., & Gurav, A. (2025) · Lecture notes in networks and systems · doi:10.1007/978-981-96-2694-6_15
Zero-Voice: A Low-Resource Approach For Zero-shot Text-to-SpeechPreprint
Wang, Z., Hou, F., & Wang, R. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.6590685
Detecting Computer Network Attacks in Time Series Network Data Using Transformer-Based Models
Fath, N. M. N., Shiddiqi, A. M., Mogi, I. K. A., Rizki, A., Syaefudin, M. A., & Rahma, F. (2025) · International Conference on Information & Communication Technology and System (ICTS) · doi:10.1109/icts67612.2025.11369713
Shih, D., Chan, P.-Y., Shih, P.-Y., Chen, B.-R., & Shih, M. (2026) · Mathematics · doi:10.3390/math14101674
Fast speech style adaptation with adjustable prosody and variable duration
Hua, Z., Yu, L., Liu, C., Sun, D., & Xie, H. (2026) · JUSTC · doi:10.52396/justc-2023-0135
Adewale, S., Ubur, S. D., Chandrashekar, N. D., Emebo, O., & Gračanin, D. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-31421-5_1
Multi-speaker recognition with AdaptoNet and hybrid optimization approach
Mohanty, A., & Cherukuri, R. C. (2026) · Multimedia Tools and Applications · doi:10.1007/s11042-026-21837-y
- Authors
- Chen, M., Tan, X., Ren, Y., Xu, J., Sun, H., Zhao, S., & Qin, T.
- Affiliations
- Peking University (CN)
- Microsoft Research Asia (China) (CN)
- Zhejiang University (CN)
- Tsinghua University (CN)
- Microsoft (United States) (US)
- Venue
- Interspeech 2020
- Publisher
- ISCA
- Published
- 2020-10-25
- Pages
- 4024-4028
- ISSN
- 2308-457X (ISSN-L)
- OpenAlex
- W3095545636
- Cited by
- 92
- References
- 23
- Citations by year
- 2026: 3 · 2025: 16 · 2024: 10 · 2023: 20 · 2022: 14 · 2021: 27 · 2020: 2
- FWCI
- 5.93
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- public-domain
- Locations
- Interspeech (USB) (publishedVersion)
- arXiv (Cornell University) (submittedVersion, public-domain)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Health Sciences › Medicine › Physiology › Voice and Speech Disorders
- Keywords
- multi-speaker text-to-speech, inference speed, VCTK, LibriTTS
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review