The DOI matches its Crossref record; 56 of the 58 references it cites were found in a registry or an index.
CycleGAN-VC2: improved CycleGAN-based non-parallel voice conversion.
Abstract
Non-parallel voice conversion (VC) is a technique for learning the mapping from source to target speech without relying on parallel data. This is an important task, but it has been challenging due to the disadvantages of the training conditions. Recently, CycleGAN-VC has provided a breakthrough and performed comparably to a parallel VC method without relying on any extra data, modules, or time alignment procedures. However, there is still a large gap between the real target and converted speech, and bridging this gap remains a challenge. To reduce the gap, we propose CycleGAN-VC2, which is an improved version of CycleGAN-VC incorporating three new techniques: an improved objective (two-step adversarial losses), improved generator (2-1-2D CNN), and improved discriminator (PatchGAN). We evaluated our method on a non-parallel VC task and analyzed the effect of each technique in detail. An objective evaluation showed that these techniques help bring the converted feature sequence closer to the target in terms of both global and local structures, which we assess by using Mel-cepstral distortion and modulation spectra distance, respectively. A subjective evaluation showed that CycleGAN-VC2 outperforms CycleGAN-VC in terms of naturalness and similarity for every speaker pair, including intra-gender and inter-gender pairs.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“To improve the stability of the training process, an additional cycle-consistency loss function was proposed to optimize the structure of GAN, which was defined as cycle GANs (Kaneko et al., 2019).”
From Data Augmentation for Deep Neural Networks Model in EEG Classification Task: A Review (2021) · cites this paper as Kaneko et al., 2019
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
Unpaired image-to-image translation using cycle-consistent adversarial networks
Zhu, J.-Y., Park, T., Isola, P., & Efros, A. A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.244 · Cited by 23,602
Image-to-Image Translation with Conditional Adversarial Networks
Isola, P., Zhu, J.-Y., Zhou, T., & Efros, A. A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.632 · Cited by 22,930
Labaca-Castro, R. (2023) · Springer eBooks · doi:10.1007/978-3-658-40442-0_9 · Cited by 21,119
GAN（Generative Adversarial Nets）
淳司, 柴. (2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 20,507
Shi, W., Caballero, J., Huszar, F., Totz, J., Aitken, A. P., Bishop, R., Rueckert, D., & Wang, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.207 · Cited by 7,570
“Least squares generative adversarial networks,”
Mao, X., Li, Q., Xie, H., Lau, R. Y. K., Wang, Z., & Smolley, S. P. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.304 · Cited by 5,437
WaveNet: a generative model for raw audio
Oord, A. V. D., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1609.03499 · Cited by 4,911
Instance Normalization: The Missing Ingredient for Fast StylizationPreprint
Ulyanov, D., Vedaldi, A., & Lempitsky, V. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1607.08022 · Cited by 4,325
DualGAN: Unsupervised Dual Learning for Image-to-Image Translation
Yi, Z., Zhang, H., Tan, P., & Gong, M. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.310 · Cited by 2,262
Language Modeling with Gated Convolutional NetworksPreprint
Dauphin, Y., Fan, A., Auli, M., & Grangier, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.08083 · Cited by 1,765
Learning to Discover Cross-Domain Relations with Generative Adversarial Networks
Kim, T., Cha, M., Kim, H., Lee, J. K., & Kim, J. (2017) · arXiv · doi:10.48550/arxiv.1703.05192 · Cited by 1,716
Precomputed Real-Time Texture Synthesis with Markovian Generative Adversarial Networks
Li, C., & Wand, M. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46487-9_43 · Cited by 1,525
WORLD: A Vocoder-Based High-Quality Speech Synthesis System for Real-Time Applications
Morise, M., Yokomori, F., & Ozawa, K. (2016) · IEICE Trans. Inf. Syst. · doi:10.1587/transinf.2015edp7457 · Cited by 1,228
Unsupervised Cross-Domain Image GenerationPreprint
Taigman, Y., Polyak, A., & Wolf, L. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.02200 · Cited by 956
Voice Conversion Based on Maximum-Likelihood Estimation of Spectral Parameter Trajectory
Toda, T., Black, A. W., & Tokuda, K. (2007) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2007.907344 · Cited by 938
Continuous probabilistic transform for voice conversion
Stylianou, Y., Cappé, O., & Moulines, É. (1998) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/89.661472 · Cited by 933
(2017) · International Conference on Machine Learning · Cited by 758
“StarGAN-VC: non-parallel many-to-many voice conversion using star generative adversarial networks,”
Kameoka, H., Kaneko, T., Tanaka, K., & Hojo, N. (2018) · 2018 IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt.2018.8639535 · Cited by 371
Self-citationCycleGAN-VC: Non-parallel Voice Conversion Using Cycle-Consistent Adversarial Networks
Kaneko, T., & Kameoka, H. (2018) · 2018 26th European Signal Processing Conference (EUSIPCO) · doi:10.23919/eusipco.2018.8553236 · Cited by 358
Self-citationLearning Dense Correspondence via 3D-Guided Cycle Consistency
Zhou, T., Krähenbühl, P., Aubry, M., Huang, Q., & Efros, A. A. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.20 · Cited by 353
Hsu, C.-C., Hwang, H.-T., Wu, Y.-C., Tsao, Y., & Wang, H. (2017) · Interspeech (USB) · doi:10.21437/interspeech.2017-63 · Cited by 337
The Voice Conversion Challenge 2018: Promoting Development of Parallel and Nonparallel Methods
Lorenzo-Trueba, J., Yamagishi, J., Toda, T., Saito, D., Villavicencio, F., Kinnunen, T., & Ling, Z. (2018) · The Speaker and Language Recognition Workshop (Odyssey 2018) · doi:10.21437/odyssey.2018-28 · Cited by 305
Speaker-Dependent WaveNet Vocoder
Tamamori, A., Hayashi, T., Kobayashi, K., Takeda, K., & Toda, T. (2017) · Interspeech (USB) · doi:10.21437/interspeech.2017-314 · Cited by 276
Voice conversion using deep Bidirectional Long Short-Term Memory based Recurrent Neural Networks
Sun, L., Kang, S., Li, K., & Meng, H. M. L. (2015) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2015.7178896 · Cited by 250
Spectral Mapping Using Artificial Neural Networks for Voice Conversion
Desai, S., Black, A. W., Yegnanarayana, B., & Prahallad, K. S. (2010) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2010.2047683 · Cited by 224
Voice Conversion Using Deep Neural Networks With Layer-Wise Generative Training
Chen, L.-H., Ling, Z.-H., Liu, L., & Dai, L.-R. (2014) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2014.2353991 · Cited by 210
Voice Conversion Using Partial Least Squares Regression
Helander, E. E., Virtanen, T. I., Nurminen, J., & Gabbouj, M. (2010) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2010.2041699 · Cited by 207
Statistical Voice Conversion Techniques for Body-Conducted Unvoiced Speech Enhancement
Toda, T., Nakagiri, M., & Shikano, K. (2012) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2012.2205241 · Cited by 195
Speaking-aid systems using GMM-based voice conversion for electrolaryngeal speech
Nakamura, K., Toda, T., Saruwatari, H., & Shikano, K. (2011) · Speech Communication · doi:10.1016/j.specom.2011.07.007 · Cited by 171
Exemplar-Based Sparse Representation With Residual Compensation for Voice Conversion
Wu, Z., Virtanen, T. I., Chng, E. S., & Li, H. (2014) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2014.2333242 · Cited by 166
Parallel-Data-Free Voice Conversion Using Cycle-Consistent Adversarial NetworksPreprint
Kaneko, T., & Kameoka, H. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.11293 · Cited by 163
Self-citationImproving the intelligibility of dysarthric speech
Kain, A. B., Hosom, J.-P., Niu, X., van Santen, J. P. H., Fried-Oken, M. B., & Staehely, J. (2007) · Speech Communication · doi:10.1016/j.specom.2007.05.001 · Cited by 157
“High-quality nonparallel voice conversion based on cycle-consistent adversarial network,”
Fang, F., Yamagishi, J., Echizen, I., & Lorenzo-Trueba, J. (2018) · 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2018.8462342 · Cited by 128
Generative adversarial network-based postfilter for statistical parametric speech synthesis
Kaneko, T., Kameoka, H., Hojo, N., Ijima, Y., Hiramatsu, K., & Kashino, K. (2017) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2017.7953090 · Cited by 121
Self-citationEigenvoice conversion based on Gaussian mixture model
Toda, T., Ohtani, Y., & Shikano, K. (2006) · Interspeech (USB) · doi:10.21437/interspeech.2006-613 · Cited by 119
Saito, Y., Ijima, Y., Nishida, K., & Takamichi, S. (2018) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2018.8461384 · Cited by 115
Kaneko, T., Kameoka, H., Hiramatsu, K., & Kashino, K. (2017) · Interspeech (USB) · doi:10.21437/interspeech.2017-970 · Cited by 110
Self-citationVoice conversion using deep neural networks with speaker-independent pre-training
Mohammadi, S. H., & Kain, A. B. (2014) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt.2014.7078543 · Cited by 105
Nonparallel training for voice conversion based on a parameter adaptation approach
Mouchtaris, A., der Spiegel, J. V., & Müeller, P. (2006) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tsa.2005.857790 · Cited by 104
Map-based adaptation for speech conversion using adaptation data selection and non-parallel training
Lee, C.-H., & Wu, C. (2006) · Interspeech (USB) · doi:10.21437/interspeech.2006-579 · Cited by 100
Data-driven emotion conversion in spoken English
Inanoglu, Z., & Young, S. J. (2008) · Speech Communication · doi:10.1016/j.specom.2008.09.006 · Cited by 92
One-to-many voice conversion based on tensor representation of speaker space
Saito, D., Yamamoto, K., Minematsu, N., & Hirose, K. (2011) · Interspeech (USB) · doi:10.21437/interspeech.2011-268 · Cited by 86
A KL Divergence and DNN-Based Approach to Voice Conversion without Parallel Training Sentences
Xie, F., Soong, F. K., & Li, H. (2016) · Interspeech (USB) · doi:10.21437/interspeech.2016-116 · Cited by 81
Liu, K., Zhang, J., & Yan, Y. (2007) · International Conference on Fuzzy Systems and Knowledge Discovery (FSKD) · doi:10.1109/fskd.2007.347 · Cited by 72
Generative Adversarial Network-Based Postfilter for STFT Spectrograms
Kaneko, T., Takaki, S., Kameoka, H., & Yamagishi, J. (2017) · Interspeech (USB) · doi:10.21437/interspeech.2017-962 · Cited by 66
Self-citationNakashika, T., Takiguchi, T., & Ariki, Y. (2014) · Interspeech (USB) · doi:10.21437/interspeech.2014-447 · Cited by 64
Non-Parallel Training in Voice Conversion Using an Adaptive Restricted Boltzmann Machine
Nakashika, T., Takiguchi, T., & Minami, Y. (2016) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2016.2593263 · Cited by 63
Exemplar-Based Voice Conversion Using Sparse Representation in Noisy Environments
Takashima, R., Takiguchi, T., & Ariki, Y. (2013) · IEICE Transactions on Fundamentals of Electronics Communications and Computer Sciences · doi:10.1587/transfun.e96.a.1946 · Cited by 44
Voice Conversion Based on Speaker-Dependent Restricted Boltzmann Machines
Nakashika, T., Takiguchi, T., & Ariki, Y. (2014) · IEICE Transactions on Information and Systems · doi:10.1587/transinf.e97.d.1403 · Cited by 43
Kameoka, H., Kaneko, T., Tanaka, K., & Hojo, N. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.05092 · Cited by 41
Self-citationSynthetic-to-Natural Speech Waveform Conversion Using Cycle-Consistent Adversarial Networks
Tanaka, K., Kaneko, T., Hojo, N., & Kameoka, H. (2018) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt.2018.8639636 · Cited by 37
Self-citationOyamada, K., Kameoka, H., Kaneko, T., Ando, H., Hiramatsu, K., & Kashino, K. (2017) · Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) · doi:10.1109/apsipa.2017.8282025 · Cited by 20
Self-citationKobayashi, K., Toda, T., & Nakamura, S. (2016) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt.2016.7846338 · Cited by 16
Cited by
Sorted by citations
Masood, M., Nawaz, M., Malik, K. M., Javed, A., Irtaza, A., & Malik, H. (2023) · Applied Intelligence · doi:10.1007/s10489-022-03766-z · Cited by 543
An Overview of Voice Conversion and Its Challenges: From Statistical Modeling to Deep Learning
Şişman, B., Yamagishi, J., King, S., & Li, H. (2020) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2020.3038524 · Cited by 399
StarGAN-VC2: Rethinking Conditional Methods for StarGAN-Based Voice Conversion
Kaneko, T., Kameoka, H., Tanaka, K., & Hojo, N. (2019) · Interspeech (USB) · doi:10.21437/interspeech.2019-2236 · Cited by 170
Self-citationIntelligent Pneumonia Identification From Chest X-Rays: A Systematic Literature Review
Khan, W., Zaki, N., & Ali, L. (2021) · IEEE Access · doi:10.1109/access.2021.3069937 · Cited by 133
A Review of Synthetic Image Data and Its Use in Computer Vision
Man, K., & Chahl, J. (2022) · Journal of Imaging · doi:10.3390/jimaging8110310 · Cited by 132
Metadata
- Authors
- Kaneko, T., Kameoka, H., Tanaka, K., & Hojo, N.
- Affiliations
- NTT (Japan) (JP)
- Venue
- ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
- Publisher
- IEEE
- Published
- 2019-05
- Pages
- 6820-6824
- ISSN
- 1520-6149 (ISSN-L)
- OpenAlex
- W2937579788
- Cited by
- 272
- References
- 58
- Citations by year
- 2026: 14 · 2025: 34 · 2024: 36 · 2023: 57 · 2022: 39 · 2021: 43 · 2020: 34 · 2019: 15
- FWCI
- 12.53
- Type
- conference paper
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE International Conference on Acoustics Speech and Signal Processing (publishedVersion)
- Funders
- Virginia Agricultural Experiment Station, Virginia Polytechnic Institute and State University
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Health Sciences › Health Professions › Speech and Hearing › Voice and Speech Disorders
- Keywords
- non-parallel voice conversion, adversarial training, PatchGAN discriminator, Mel cepstral distortion, speech naturalness
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
