The DOI matches its DataCite record; 55 of the 55 references it cites were found in a registry or an index.
MelNet: A Generative Model for Audio in the Frequency Domain
Open accessAbstract
Capturing high-level structure in audio waveforms is challenging because a single second of audio spans tens of thousands of timesteps. While long-range dependencies are difficult to model directly in the time domain, we show that they can be more tractably modelled in two-dimensional time-frequency representations such as spectrograms. By leveraging this representational advantage, in conjunction with a highly expressive probabilistic model and a multiscale generation procedure, we design a model capable of generating high-fidelity audio samples which capture structure at timescales that time-domain models have yet to achieve. We apply our model to a variety of audio generation tasks, including unconditional speech generation, music generation, and text-to-speech synthesis---showing improvements over previous approaches in both density estimates and human judgments.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“MelNet (Vasquez and Lewis, 2019), which has used a 140 h single speaker dataset, as well as VoxCeleb2 (Chung et al., 2018) multi-speaker dataset, has a better performance than the previous works.”
From Audio deepfakes: A survey (2023) · cites this paper as Vasquez and Lewis, 2019
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
GAN（Generative Adversarial Nets）
淳司, 柴. (2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 20,507
Auto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067
WaveNet: a generative model for raw audio
Oord, A. V. D., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1609.03499 · Cited by 4,911
Progressive Growing of GANs for Improved Quality, Stability, and Variation
Karras, T., Aila, T., Laine, S., & Lehtinen, J. (2017) · arXiv · doi:10.48550/arxiv.1710.10196 · Cited by 4,727
Generating Sequences With Recurrent Neural Networks
Graves, A. (2013) · arXiv · doi:10.48550/arxiv.1308.0850 · Cited by 2,894
Natural TTS Synthesis by Conditioning Wavenet on MEL Spectrogram Predictions
Shen, J., Pang, R., Weiss, R. J., Schuster, M., Jaitly, N., Yang, Z., Chen, Z., Zhang, Y., Wang, Y., Skerrv-Ryan, R., et al. (2018) · 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2018.8461368 · Cited by 2,791
Neural Discrete Representation Learning
Oord, A. V. D., Vinyals, O., & Kavukcuoglu, K. (2017) · arXiv · doi:10.48550/arxiv.1711.00937 · Cited by 2,729
Research works printed but not linked: 4
- [14][14] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in neural information processing systems , pages 2672-2680, 2014. -
- [22][22] Sepp Hochreiter and J¨ urgen Schmidhuber. Long shortterm memory. Neural computation , 9(8):1735-1780, 1997. -
- [31][31] Durk P Kingma and Prafulla Dhariwal. Glow: Generative flow with invertible 1x1 convolutions. In Advances in Neural Information Processing Systems , pages 10236-10245, 2018. -
- [46][46] Tijmen Tieleman and Geoffrey Hinton. Lecture 6.5rmsprop: Divide the gradient by a running average of its recent magnitude. COURSERA: Neural networks for machine learning , 4(2):26-31, 2012. -
Cited by
Sorted by citations
Jukebox: A Generative Model for MusicPreprint
Dhariwal, P., Jun, H., Payne, C., Kim, J. W., Radford, A., & Sutskever, I. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.00341 · Cited by 1,101
WaveGrad: Estimating Gradients for Waveform GenerationPreprint
Chen, N., Zhang, Y., Zen, H., Weiss, R. J., Norouzi, M., & Chan, W. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.00713 · Cited by 603
A survey on neural speech synthesis
Tan, X., Qin, T., Soong, F., & Liu, T.-Y. (2021) · arXiv · doi:10.48550/arxiv.2106.15561 · Cited by 216
SuperNNova: an open-source framework for Bayesian, neural network-based supernova classification
Möller, A., & de Boissière, T. (2020) · Monthly Notices of the Royal Astronomical Society · doi:10.1093/mnras/stz3312 · Cited by 191
Deepfake generation and detection: Case study and challenges
Patel, Y., Tanwar, S., Gupta, R., Bhattacharya, P., Davidson, I. E., Nyameko, R., Aluvala, S., & Vimal, V. (2023) · IEEE Access · doi:10.1109/access.2023.3342107 · Cited by 186
High Fidelity Speech Synthesis with Adversarial NetworksPreprint
Bińkowski, M., Donahue, J., Dieleman, S., Clark, A., Elsen, E., Casagrande, N., Cobo, L. C., & Simonyan, K. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.11646 · Cited by 227
3‑D Inorganic Crystal Structure Generation and Property Prediction via Representation Learning
Court, C. J., Yildirim, B., Jain, A., & Cole, J. M. (2020) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.0c00464 · Cited by 138
End-to-End Adversarial Text-to-SpeechPreprint
Donahue, J., Dieleman, S., Binkowski, M., Elsen, E., & Simonyan, K. (2020) · International Conference on Learning Representations · doi:10.48550/arxiv.2006.03575 · Cited by 200
Khanjani, Z., Watson, G., & Janeja, V. P. (2023) · Frontiers in Big Data · doi:10.3389/fdata.2022.1001063 · Cited by 121
Testing human ability to detect ‘deepfake’ images of human faces
Bray, S. D., Johnson, S. D., & Kleinberg, B. (2023) · Journal of Cybersecurity · doi:10.1093/cybsec/tyad011 · Cited by 96
VQVC+: One-Shot Voice Conversion by Vector Quantization and U-Net Architecture
Wu, D.-Y., Chen, Y.-H., & Lee, H.-Y. (2020) · Interspeech 2020 · doi:10.21437/interspeech.2020-1443 · Cited by 88
Any-to-Many Voice Conversion With Location-Relative Sequence-to-Sequence Modeling
Liu, S., Cao, Y., Wang, D., Wu, X., Liu, X., & Meng, H. M. L. (2021) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2021.3076867 · Cited by 92
Ji, S., Luo, J., & Yang, X. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.06801 · Cited by 165
Frid, E., Gomes, C., & Jin, Z. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376514 · Cited by 58
GACELA: A Generative Adversarial Context Encoder for Long Audio Inpainting of Music
Marafioti, A., Majdak, P., Holighaus, N., & Perraudin, N. (2020) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2020.3037506 · Cited by 53
Nistal, J., Lattner, S., & Gaël Richard, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.12073 · Cited by 60
The Ethical Implications of Generative Audio Models: A Systematic Literature Review
Barnett, J. (2023) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3600211.3604686 · Cited by 61
Zhang, L., Jiang, L., Washington, N. L., Liu, A. A., Shao, J., Fourney, A., Morris, M., & Findlater, L. (2021) · Proc. ACM Hum. Comput. Interact. · doi:10.1145/3449235 · Cited by 34
Revisiting Over-Smoothness in Text to Speech
Ren, Y., Tan, X., Qin, T., Zhao, Z., & Liu, T. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.564 · Cited by 39
Unsupervised Style and Content Separation by Minimizing Mutual Information for Speech Synthesis
Hu, T.-Y., Shrivastava, A., Tuzel, O., & Dhir, C. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9054591 · Cited by 35
Audio representations for deep learning in sound synthesis: A review
Natsiou, A., & O’Leary, S. (2021) · IEEE/ACS International Conference on Computer Systems and Applications (AICCSA) · doi:10.1109/aiccsa53542.2021.9686838 · Cited by 35
Unconditional Audio Generation with Generative Adversarial Networks and Cycle Regularization
Liu, J.-Y., Chen, Y.-H., Yeh, Y.-C., & Yang, Y. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-1137 · Cited by 33
Conditional Sound Generation Using Neural Discrete Time-Frequency Representation Learning
Liu, X., Iqbal, T., Zhao, J., Huang, Q., Plumbley, M. D., & Wang, W. (2021) · Machine learning for signal processing ... · doi:10.1109/mlsp52302.2021.9596430 · Cited by 32
Semi-Supervised Generative Modeling for Controllable Speech SynthesisPreprint
Habib, R., Mariooryad, S., Shannon, M., Battenberg, E., Skerry-Ryan, R., Stanton, D., Kao, D., & Bagby, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.01709 · Cited by 41
Controllable Data Generation by Deep Learning: A Review
Wang, S.-Y., Du, Y., Guo, X., Pan, B., Qin, Z. S., & Zhao, L. (2024) · ACM Computing Surveys · doi:10.1145/3648609 · Cited by 48
Music Generation Using Deep Learning and Generative AI: A Systematic Review
Mitra, R., & Zualkernan, I. A. (2025) · IEEE Access · doi:10.1109/access.2025.3531798 · Cited by 41
Paul, D., Pantazis, Y., & Stylianou, Y. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-2786 · Cited by 30
A Convolutional Deep Markov Model for Unsupervised Speech Representation Learning
Khurana, S., Laurent, A., Hsu, W.-N., Chorowski, J., Łańcucki, A., Marxer, R., & Glass, J. R. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-3084 · Cited by 32
Seq-U-Net: A One-Dimensional Causal U-Net for Efficient Sequence Modelling
Stoller, D., Tian, M., Ewert, S., & Dixon, S. (2020) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/400 · Cited by 28
Ghose, S., & Prevost, J. J. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3177894 · Cited by 28
SYNT++: Utilizing Imperfect Synthetic Data to Improve Speech Recognition
Hu, T.-Y., Armandpour, M., Shrivastava, A., Chang, J.-H. R., Koppula, H. S., & Tuzel, O. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9746217 · Cited by 28
High Quality Streaming Speech Synthesis with Low, Sentence-Length-Independent Latency
Ellinas, N., Vamvoukakis, G., Μαρκόπουλος, Κ., Chalamandaris, A., Maniati, G., Kakoulidis, P., Raptis, S., Sung, J. S., Park, H.-M., & Tsiakoulis, P. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-2464 · Cited by 26
Yasuda, Y., Wang, X., & Yamagishi, J. (2020) · Computer Speech & Language · doi:10.1016/j.csl.2020.101183 · Cited by 23
Score and Lyrics-Free Singing Voice GenerationPreprint
Liu, J.-Y., Chen, Y.-H., Yeh, Y.-C., & Yang, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.11747 · Cited by 23
ESeismic-GAN: A Generative Model for Seismic Events From Cotopaxi Volcano
Grijalva, F., Ramos, W., Pérez, N., Benítez, D. S., Lara-Cueva, R., & Ruiz, M. C. (2021) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2021.3095270 · Cited by 22
Review of end-to-end speech synthesis technology based on deep learningPreprint
Mu, Z., Yang, X., & Dong, Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.09995 · Cited by 34
Controllable deep melody generation via hierarchical music structure representationPreprint
Dai, S., Jin, Z., Gomes, C., & Dannenberg, R. B. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.00663 · Cited by 22
How Deep Are the Fakes? Focusing on Audio Deepfake: A SurveyPreprint
Khanjani, Z., Watson, G., & Janeja, V. P. (2021) · arXiv (Cornell University) · doi:10.13016/m2ivas-ogyu · Cited by 22
Phase Reconstruction Based On Recurrent Phase Unwrapping With Deep Neural Networks
Masuyama, Y., Yatabe, K., Koizumi, Y., Oikawa, Y., & Harada, N. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9053234 · Cited by 21
Adaptive Multi-Scale Detection of Acoustic Events
Ding, W., & He, L. J. (2019) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2019.2953350 · Cited by 20
Learnable MFCCs for Speaker Verification
Liu, X., Sahidullah, M., & Kinnunen, T. (2021) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas51556.2021.9401593 · Cited by 21
Multi-Instrumentalist Net: Unsupervised Generation of Music from Body MovementsPreprint
Su, K., Liu, X., & Shlizerman, E. (2020) · arXiv.org · doi:10.48550/arxiv.2012.03478 · Cited by 33
High Quality Audio Coding with Mdctnet
Davidson, G. A., Vinton, M. S., Ekstrand, P., Zhou, C., Villemoes, L. F., & Lu, L. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10096056 · Cited by 13
Conditional image generation with One-Vs-All classifier
Xu, X., Li, Y., & Cao, Y. (2021) · Neurocomputing · doi:10.1016/j.neucom.2020.12.091 · Cited by 13
Time-Frequency Phase Retrieval for Audio—The Effect of Transform Parameters
Marafioti, A., Holighaus, N., & Majdak, P. (2021) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2021.3088581 · Cited by 12
Dual Learning Music Composition and Dance Choreography
Wu, S., Liu, Z., Lu, S., & Cheng, L. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3475180 · Cited by 11
GANStrument: Adversarial Instrument Sound Synthesis with Pitch-Invariant Instance Conditioning
Narita, G., Shimizu, J., & Akama, T. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10097250 · Cited by 11
Saito, Y., Takamichi, S., & Saruwatari, H. (2021) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2021.3059114 · Cited by 11
Learning audio representations via phase predictionPreprint
Quitry, F. D. C., Tagliasacchi, M., & Roblek, D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.11910 · Cited by 10
Incremental Text to Speech for Neural Sequence-to-Sequence Models Using Reinforcement Learning
Mohan, D. S. R., Lenain, R., Foglianti, L., Teh, T. H., Staib, M., Torresquintero, A., & Gao, J. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-1822 · Cited by 11
Jayalakshmy, S., Lakshmipriya, B., & Sudha, G. F. (2021) · Elsevier eBooks · doi:10.1016/b978-0-12-823519-5.00024-5 · Cited by 9
Modeling of Rakugo Speech and Its Limitations: Toward Speech Synthesis That Entertains Audiences
Kato, S., Yasuda, Y., Wang, X., Cooper, E., Takaki, S., & Yamagishi, J. (2020) · IEEE Access · doi:10.1109/access.2020.3011975 · Cited by 10
CSTNet: Contrastive Speech Translation Network for Self-Supervised Speech Representation LearningPreprint
Khurana, S., Laurent, A., & Glass, J. R. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.02814 · Cited by 11
FastS2S-VC: Streaming Non-Autoregressive Sequence-to-Sequence Voice ConversionPreprint
Kameoka, H., Tanaka, K., & Kaneko, T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.06900 · Cited by 8
Energy Consumption of Deep Generative Audio ModelsPreprint
Douwes, C., Esling, P., & Briot, J.-P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.02621 · Cited by 27
VMD-Informer-DCC for Photovoltaic Power Prediction
Wu, Y., Pan, X., & Yang, J. (2024) · IEICE Transactions on Communications · doi:10.23919/transcom.2023ebp3170 · Cited by 7
Fingerprint Spoof Generation Using Style Transfer
Wone, A., Manno, J. D., Charrier, C., & Rosenberger, C. (2025) · IEEE Transactions on Biometrics Behavior and Identity Science · doi:10.1109/tbiom.2025.3545308 · Cited by 7
Controllable speech synthesis by learning discrete phoneme-level prosodic representations
Ellinas, N., Christidou, M., Vioni, A., Sung, J. S., Chalamandaris, A., Tsiakoulis, P., & Mastorocostas, P. (2022) · Speech Communication · doi:10.1016/j.specom.2022.11.006 · Cited by 8
Speech-to-Singing Conversion Based on Boundary Equilibrium GAN
Wu, D.-Y., & Yang, Y. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-1984 · Cited by 6
Enabling an IoT System of Systems through Auto Sound Synthesis in Silent Video with DNN
Ghose, S., & Prevost, J. J. (2020) · International Conference on System of Systems Engineering · doi:10.1109/sose50414.2020.9130483 · Cited by 6
Prosodic Clustering for Phoneme-Level Prosody Control in End-to-End Speech Synthesis
Vioni, A., Christidou, M., Ellinas, N., Vamvoukakis, G., Kakoulidis, P., Kim, T., Sung, J. S., Park, H.-M., Chalamandaris, A., & Tsiakoulis, P. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9413604 · Cited by 6
Gamma Boltzmann Machine for Audio Modeling
Nakashika, T., & Yatabe, K. (2021) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2021.3095656 · Cited by 6
Comparing Representations for Audio Synthesis Using Generative Adversarial Networks
Nistal, J., Lattner, S., & Gaël Richard, A. (2020) · European Signal Processing Conference (EUSIPCO) · doi:10.23919/eusipco47968.2020.9287799 · Cited by 5
Comprehensive Performance Analysis of Object Detection Algorithms Across Diverse Sensor Modalities
Acharjee, S., Ganguly, S., Ali, A., & Frnda, J. (2025) · IEEE Sensors Reviews · doi:10.1109/sr.2025.3584745 · Cited by 6
A Neural Lip-Sync Framework for Synthesizing Photorealistic Virtual News Anchors
Zheng, R., Zhou, Z., Song, B., & Ji, C. (2021) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr48806.2021.9412187 · Cited by 7
Phase reconstruction based on recurrent phase unwrapping with deep neural networksPreprint
Masuyama, Y., Yatabe, K., Koizumi, Y., Oikawa, Y., & Harada, N. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.05832 · Cited by 5
WaveNODE: A Continuous Normalizing Flow for Speech SynthesisPreprint
Kim, H., Lee, H. S., Kang, W. H., Cheon, S. J., Choi, B. J., & Kim, N. S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.04598 · Cited by 8
MP3net: coherent, minute-long music generation from raw audio with a simple convolutional GANPreprint
van den Broek, K. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.04785 · Cited by 5
DarkGAN: Exploiting Knowledge Distillation for Comprehensible Audio Synthesis With GANsPreprint
Nistal, J., Lattner, S., & Gaël Richard, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.01216 · Cited by 7
A Survey on Audio Synthesis and Audio-Visual Multimodal ProcessingPreprint
Shi, Z. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.00443 · Cited by 18
Phase-Aware Transformations in Variational Autoencoders for Audio Effects
Cámara, M., & Blanco, J. L. (2022) · Journal of the Audio Engineering Society · doi:10.17743/jaes.2022.0042 · Cited by 4
CycleDRUMS: automatic drum arrangement for bass lines using CycleGAN
Barnabò, G., Trappolini, G., Lastilla, L., Campagnano, C., Fan, A., Petroni, F., & Silvestri, F. (2023) · Discover Artificial Intelligence · doi:10.1007/s44163-023-00047-7 · Cited by 4
Multi-speaker Chinese news broadcasting system based on improved Tacotron2
Zhao, W., Lian, Y., Chai, J., & Tu, Z. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-15279-z · Cited by 5
Synthesising Expressiveness in Peking Opera via Duration Informed Attention NetworkPreprint
Wu, Y., Li, S., Yu, C., Lu, H., Weng, C., Zhang, L., & Yu, D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.12010 · Cited by 4
Generative Models for Security: Attacks, Defenses, and OpportunitiesPreprint
Bauer, L. A., & Bindschaedler, V. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.10139 · Cited by 14
A Benchmarking Initiative for Audio-Domain Music Generation Using the Freesound Loop DatasetPreprint
Hung, T. M., Chen, B., Yeh, Y.-T., & Yang, Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.01576 · Cited by 4
TTSlow: Slow Down Text-to-Speech With Efficiency Robustness Evaluations
Gao, X., Chen, Y., Yue, X., Tsao, Y., & Chen, N. F. (2025) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2025.3533357 · Cited by 3
Sound of Story: Multi-modal Storytelling with Audio
Bae, J., Jeong, S., Kang, S., Han, N., Lee, J.-Y., Kim, H., & Kim, T.-H. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.898 · Cited by 6
Mapping the Audio Landscape for Innovative Music Sample Generation
Limberg, C., & Zhang, Z. (2024) · International Conference on Multimedia Retrieval · doi:10.1145/3652583.3657586 · Cited by 2
FCH-TTS: Fast, Controllable and High-quality Non-Autoregressive Text-to-Speech Synthesis
Zhou, X. Y., Zhou, Z., & Shi, X. (2022) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn55064.2022.9892512 · Cited by 2
Unsupervised Style and Content Separation by Minimizing Mutual Information for Speech SynthesisPreprint
Hu, T.-Y., Shrivastava, A., Tuzel, O., & Dhir, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.06227 · Cited by 2
Gamma Boltzmann Machine for Simultaneously Modeling Linear- and Log-amplitude SpectraPreprint
Nakashika, T., & Yatabe, K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.13590 · Cited by 2
Generative Adversarial Networks in Human Emotion Synthesis: A Review
Hajarolasvadi, N., Ramírez, M. A., Beccaro, W., & Demirel, H. (2020) · IEEE Access · doi:10.1109/access.2020.3042328 · Cited by 37
MDCTNet: A Hybrid Approach to Neural Audio Coding
Villemoes, L. F., Vinton, M. S., Ekstrand, P., Lu, L., Davidson, G. A., & Zhou, C. (2024) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2024.3482721 · Cited by 1
Jeon, M.-J., & Kim, J. (2021) · Journal of Digital Contents Society · doi:10.9728/dcs.2021.22.3.375 · Cited by 1
Multi-Track Music Generation Network Based on a Hybrid Learning Module
Zhang, D., Hu, Z., Li, X., Tie, Y., & Qi, L. (2023) · IEEE International Conference on Multimedia and Expo workshops · doi:10.1109/icmew59549.2023.00062 · Cited by 1
Exploring the Impact of Speech AI: A Comparative Analysis of ML Models on Arabic Dataset
Mna, M. K. B., & Letaïfa, A. B. (2023) · International Conference on Communications and Networking (ComNet) · doi:10.1109/comnet60156.2023.10366659 · Cited by 1
García, S. K. B., Lucero, E. S., Méndez, B. E. P., Hernández, J. C. H., Huerta, E. B., & Cruz, J. F. R. (2020) · International Conference in Software Engineering Research and Innovation (CONISOFT) · doi:10.1109/conisoft50191.2020.00041 · Cited by 1
A sinusoidal signal reconstruction method for the inversion of the mel-spectrogram
Natsiou, A., & O’Leary, S. (2021) · IEEE International Symposium on Multimedia (ISM) · doi:10.1109/ism52913.2021.00048 · Cited by 1
Conditional Sound Generation Using Neural Discrete Time-Frequency Representation LearningPreprint
Liu, X., Iqbal, T., Zhao, J., Huang, Q., Plumbley, M. D., & Wang, W. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.09998 · Cited by 1
Guided-TTS:Text-to-Speech with Untranscribed SpeechPreprint
Kim, H., Kim, S., & Yoon, S. (2021) · arXiv (Cornell University) · Cited by 1
Individually amplified text-to-speech.Preprint
Schlittenlacher, J., & Baer, T. (2020) · arXiv (Cornell University)
AudioViewer : learning to visualize soundPreprint
Zhang, Y. (2021) · arXiv (Cornell University) · doi:10.14288/1.0395431
Text-to-speech for the hearing impairedPreprint
Schlittenlacher, J., & Baer, T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.02174
Improving the Expressiveness of Neural Vocoding with Non-Affine Normalizing Flows
Gabryś, A., Jiao, Y., Klimkov, V., Korzekwa, D., & Barra-Chicote, R. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-1555
Multi-Scale Spectrogram Modelling for Neural Text-to-Speech
Abbas, A., Bollepalli, B., Moinet, A., Joly, A., Karanasou, P., Makarov, P., Slangens, S., Karlapati, S., & Thomas Drugman, A. (2021) · ISCA Speech Synthesis Workshop (SSW) · doi:10.21437/ssw.2021-31
Musical Speech: A Transformer-based Composition ToolPreprint
d’Eon, J., Dumpala, S. H., Sastry, C. S., Oore, D., & Oore, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.01043
Research on the Methods of Speech Synthesis Technology
Hu, J., & Hamdulla, A. (2021) · IEEE International Conference on Pattern Recognition and Machine Learning (PRML) · doi:10.1109/prml52754.2021.9520718
Analyzing Emotional Speech and Text: A Special Focus on Bengali Language
Majumder, K., & Das, D. (2022) · Signals and communication technology · doi:10.1007/978-3-031-18444-4_15
Metadata
- Authors
- Vasquez, S., & Lewis, M.
- ORCID
- Lewis, Mike
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2019
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1906.01083
- OpenAlex
- W2948211236
- Cited by
- 107
- References
- 50
- Citations by year
- 2025: 5 · 2024: 5 · 2023: 14 · 2022: 7 · 2021: 35 · 2020: 34 · 2019: 7
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Signal Processing › Music and Audio Processing
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Keywords
- audio generation, spectrogram, time-frequency representation, long-range dependencies, music generation, text-to-speech
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
