Adaptive Wavenet Vocoder for Residual Compensation in GAN-Based Voice Conversion
The DOI matches its Crossref record; 44 of the 44 references it cites were found in a registry or an index.
Abstract
In this paper, we propose to use generative adversarial networks (GAN) together with a WaveNet vocoder to address the over-smoothing problem arising from the deep learning approaches to voice conversion, and to improve the vocoding quality over the traditional vocoders. As GAN aims to minimize the divergence between the natural and converted speech parameters, it effectively alleviates the over-smoothing problem in the converted speech. On the other hand, WaveNet vocoder allows us to leverage from the human speech of a large speaker population, thus improving the naturalness of the synthetic voice. Furthermore, for the first time, we study how to use WaveNet vocoder for residual compensation to improve the voice conversion performance. The experiments show that the proposed voice conversion framework consistently outperforms the baselines.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Benefiting from advancements in neural vocoders, modern VC systems have redefined their architecture. Since a neural vocoder itself is trainable, it enables joint training with conversion models [57] and, in some cases, speech analysis modules [58], achieving an E2E structure.”
From Audio Deepfake Detection: What Has Been Achieved and What Lies Ahead (2025) · cites this paper as [57]
Citations
Cites
WaveNet: a generative model for raw audio
Oord, A. V. D., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1609.03499 · Cited by 4,911 · cited 4×
“WaveNet [28] is a well-known deep neural network that can generates raw audio waveforms.”
BackgroundInfluentialStackGAN: Text to Photo-Realistic Image Synthesis with Stacked Generative Adversarial Networks
Zhang, H., Xu, T., Li, H., Zhang, S., Wang, X., Huang, X., & Metaxas, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.629 · Cited by 3,021 · cited 1×
“GANs have recently been shown to be an effective training method and have been used for image generation [30], image synthesis [30], speech enhancement [23, 24], language identification [31], and text-to-speech synthesis [25, 26].”
MethodSelf-citationNatural TTS Synthesis by Conditioning Wavenet on MEL Spectrogram Predictions
Shen, J., Pang, R., Weiss, R. J., Schuster, M., Jaitly, N., Yang, Z., Chen, Z., Zhang, Y., Wang, Y., Skerrv-Ryan, R., et al. (2018) · 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2018.8461368 · Cited by 2,791
SEGAN: Speech Enhancement Generative Adversarial Network
Pascual, S., Bonafonte, A., & Serrà, J. (2017) · Interspeech 2017 · doi:10.21437/interspeech.2017-1428 · Cited by 1,297 · cited 2×
“GAN has shown to be effective in many fields including speech enhancement [23, 24], speech synthesis [25,26].”
BackgroundMethodVoice Conversion Based on Maximum-Likelihood Estimation of Spectral Parameter Trajectory
Toda, T., Black, A. W., & Tokuda, K. (2007) · IEEE Transactions on Audio, Speech and Language Processing · doi:10.1109/tasl.2007.907344 · Cited by 937
Mel-cepstral distance measure for objective speech quality assessment
Kubichek, R. (2002) · Proceedings of IEEE Pacific Rim Conference on Communications Computers and Signal Processing · doi:10.1109/pacrim.1993.407206 · Cited by 534 · cited 7×
“Such MCD doesn’t evaluate the effect of vocoders.”
MethodInfluentialVoice conversion through vector quantization
Abe, M., Nakamura, S., Shikano, K., & Kuwabara, H. (2003) · ICASSP-88., International Conference on Acoustics, Speech, and Signal Processing · doi:10.1109/icassp.1988.196671 · Cited by 415 · cited 2×
“There are 30 hidden layers in our WaveNet Vocoder, and dilations are [1, 2, 4, 8, 16, 32, 64, 128, 256, 512]× 3 respectively.”
BackgroundMethodSelf-citationDigital speech processing, synthesis, and recognition
(1989) · Speech Communication · doi:10.1016/0167-6393(89)90009-5 · Cited by 412
“StarGAN-VC: non-parallel many-to-many voice conversion using star generative adversarial networks,”
Kameoka, H., Kaneko, T., Tanaka, K., & Hojo, N. (2018) · 2018 IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt.2018.8639535 · Cited by 371
Hsu, C.-C., Hwang, H.-T., Wu, Y.-C., Tsao, Y., & Wang, H.-M. (2017) · Interspeech 2017 · doi:10.21437/interspeech.2017-63 · Cited by 320 · cited 2×
“To address the quality problem, GAN was proposed [17, 21, 22] as a solution.”
Background
Cited by
An Overview of Voice Conversion and Its Challenges: From Statistical Modeling to Deep Learning
Sisman, B., Yamagishi, J., King, S., & Li, H. (2021) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2020.3038524 · Cited by 357
“GANs offer a general framework for training a data generator in such a way that it can deceive a real/fake discriminator that attempts to distinguish real data and fake data produced by the generator [242]–[244].”
BackgroundMethodInfluentialSelf-citationEmotional voice conversion: Theory, databases and ESD
Zhou, K., Sisman, B., Liu, R., & Li, H. (2022) · Speech Communication · doi:10.1016/j.specom.2021.11.006 · Cited by 181
Self-citationExpressive TTS Training With Frame and Style Reconstruction Loss
Liu, R., Şişman, B., Gao, G., & Li, H. (2021) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2021.3076369 · Cited by 97
“a neural vocoder [5], [16]–[21], neural TTS generates naturalsounding and human-like speech which achieves state-of-the-art”
BackgroundSelf-citationTransforming Spectrum and Prosody for Emotional Voice Conversion with Non-Parallel Training Data
Zhou, K., Şişman, B., & Li, H. (2020) · The Speaker and Language Recognition Workshop (Odyssey) · doi:10.21437/odyssey.2020-33 · Cited by 93
Self-citationVQVAE Unsupervised Unit Discovery and Multi-Scale Code2Spec Inverter for Zerospeech Challenge 2019
Tjandra, A., Sisman, B., Zhang, M., Sakti, S., Li, H., & Nakamura, S. (2019) · Interspeech 2019 · doi:10.21437/interspeech.2019-3232 · Cited by 65
Self-citation
Metadata
- Authors
- Sisman, B., Zhang, M., Sakti, S., Li, H., & Nakamura, S.
- Affiliations
- National University of Singapore (SG)
- Nara Institute of Science and Technology (JP)
- Venue
- 2018 IEEE Spoken Language Technology Workshop (SLT)
- Publisher
- IEEE
- Published
- 2018-12
- Pages
- 282-289
- OpenAlex
- W2911340057
- Cited by
- 36
- References
- 44
- Citations by year
- 2025: 1 · 2023: 1 · 2022: 2 · 2021: 7 · 2020: 14 · 2019: 11
- FWCI
- 3.25
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE Spoken Language Technology Workshop (SLT) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Keywords
- voice conversion, generative adversarial networks, residual compensation, over-smoothing, synthetic speech naturalness, speech synthesis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Adaptive Wavenet Vocoder for Residual Compensation in GAN-Based Voice Conversion. https://app.cochat.ai/papers/10.1109/slt.2018.8639507
- The paper
- Sisman, B., Zhang, M., Sakti, S., Li, H., & Nakamura, S. (2018). Adaptive Wavenet Vocoder for Residual Compensation in GAN-Based Voice Conversion. 2018 IEEE Spoken Language Technology Workshop (SLT). https://doi.org/10.1109/slt.2018.8639507
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.