CycleGAN-VC2: improved CycleGAN-based non-parallel voice conversion.
Abstract
Non-parallel voice conversion (VC) is a technique for learning the mapping from source to target speech without relying on parallel data. This is an important task, but it has been challenging due to the disadvantages of the training conditions. Recently, CycleGAN-VC has provided a breakthrough and performed comparably to a parallel VC method without relying on any extra data, modules, or time alignment procedures. However, there is still a large gap between the real target and converted speech, and bridging this gap remains a challenge. To reduce the gap, we propose CycleGAN-VC2, which is an improved version of CycleGAN-VC incorporating three new techniques: an improved objective (two-step adversarial losses), improved generator (2-1-2D CNN), and improved discriminator (PatchGAN). We evaluated our method on a non-parallel VC task and analyzed the effect of each technique in detail. An objective evaluation showed that these techniques help bring the converted feature sequence closer to the target in terms of both global and local structures, which we assess by using Mel-cepstral distortion and modulation spectra distance, respectively. A subjective evaluation showed that CycleGAN-VC2 outperforms CycleGAN-VC in terms of naturalness and similarity for every speaker pair, including intra-gender and inter-gender pairs.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
An additional cycle-consistency loss function improves the stability of the training process and defines cycle GANs.
“To improve the stability of the training process, an additional cycle-consistency loss function was proposed to optimize the structure of GAN, which was defined as cycle GANs (Kaneko et al., 2019).”
From Data Augmentation for Deep Neural Networks Model in EEG Classification Task: A Review (2021) · cites this paper as Kaneko et al., 2019
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834 · see citing works
Unpaired image-to-image translation using cycle-consistent adversarial networks
Zhu, J.-Y., Park, T., Isola, P., & Efros, A. A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.244 · Cited by 23,602 · see citing works
Image-to-Image Translation with Conditional Adversarial Networks
Isola, P., Zhu, J.-Y., Zhou, T., & Efros, A. A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.632 · Cited by 22,930 · see citing works
Labaca-Castro, R. (2023) · Springer eBooks · doi:10.1007/978-3-658-40442-0_9 · Cited by 21,100 · see citing works
GAN（Generative Adversarial Nets）
淳司, 柴. (2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,268 · see citing works
Shi, W., Caballero, J., Huszar, F., Totz, J., Aitken, A. P., Bishop, R., Rueckert, D., & Wang, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.207 · Cited by 7,570 · see citing works
“Least squares generative adversarial networks,”
Mao, X., Li, Q., Xie, H., Lau, R. Y. K., Wang, Z., & Smolley, S. P. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.304 · Cited by 5,437 · see citing works
WaveNet: a generative model for raw audio
Oord, A. V. D., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1609.03499 · Cited by 4,911 · see citing works
Instance Normalization: The Missing Ingredient for Fast StylizationPreprint
Ulyanov, D., Vedaldi, A., & Lempitsky, V. (2016) · arXiv.org · doi:10.48550/arxiv.1607.08022 · Cited by 4,239 · see citing works
Language Modeling with Gated Convolutional NetworksPreprint
Dauphin, Y., Fan, A., Auli, M., & Grangier, D. (2016) · International Conference on Machine Learning · doi:10.48550/arxiv.1612.08083 · Cited by 3,097 · see citing works
DualGAN: Unsupervised Dual Learning for Image-to-Image Translation
Yi, Z., Zhang, H., Tan, P., & Gong, M. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.310 · Cited by 2,226 · see citing works
Learning to Discover Cross-Domain Relations with Generative Adversarial Networks
Kim, T., Cha, M., Kim, H., Lee, J. K., & Kim, J. (2017) · arXiv · doi:10.48550/arxiv.1703.05192 · Cited by 1,716 · see citing works
Precomputed Real-Time Texture Synthesis with Markovian Generative Adversarial Networks
Li, C., & Wand, M. (2016) · European Conference on Computer Vision · doi:10.1007/978-3-319-46487-9_43 · Cited by 1,535 · see citing works
WORLD: A Vocoder-Based High-Quality Speech Synthesis System for Real-Time Applications
Morise, M., Yokomori, F., & Ozawa, K. (2016) · IEICE Trans. Inf. Syst. · doi:10.1587/transinf.2015edp7457 · Cited by 1,228 · see citing works
Continuous probabilistic transform for voice conversion
Stylianou, Y., Cappé, O., & Moulines, É. (1998) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/89.661472 · Cited by 933 · see citing works
Voice Conversion Based on Maximum-Likelihood Estimation of Spectral Parameter Trajectory
Toda, T., Black, A. W., & Tokuda, K. (2007) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2007.907344 · Cited by 928 · see citing works
Unsupervised Cross-Domain Image GenerationPreprint
Taigman, Y., Polyak, A., & Wolf, L. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.02200 · Cited by 901 · see citing works
(2017) · International Conference on Machine Learning · Cited by 758 · see citing works
“StarGAN-VC: non-parallel many-to-many voice conversion using star generative adversarial networks,”
Kameoka, H., Kaneko, T., Tanaka, K., & Hojo, N. (2018) · 2018 IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt.2018.8639535 · Cited by 371 · see citing works
Self-citationCycleGAN-VC: Non-parallel Voice Conversion Using Cycle-Consistent Adversarial Networks
Kaneko, T., & Kameoka, H. (2018) · 2018 26th European Signal Processing Conference (EUSIPCO) · doi:10.23919/eusipco.2018.8553236 · Cited by 358 · see citing works
Self-citationLearning Dense Correspondence via 3D-Guided Cycle Consistency
Zhou, T., Krähenbühl, P., Aubry, M., Qi-xing Huang, A., & Efros, A. A. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.20 · Cited by 351 · see citing works
Hsu, C.-C., Hwang, H.-T., Wu, Y.-C., Tsao, Y., & Wang, H. (2017) · Interspeech (USB) · doi:10.21437/interspeech.2017-63 · Cited by 317 · see citing works
The Voice Conversion Challenge 2018: Promoting Development of Parallel and Nonparallel Methods
Lorenzo-Trueba, J., Yamagishi, J., Toda, T., Saito, D., Villavicencio, F., Kinnunen, T., & Ling, Z. (2018) · The Speaker and Language Recognition Workshop (Odyssey 2018) · doi:10.21437/odyssey.2018-28 · Cited by 305 · see citing works
Speaker-Dependent WaveNet Vocoder
Tamamori, A., Hayashi, T., Kobayashi, K., Takeda, K., & Toda, T. (2017) · Interspeech (USB) · doi:10.21437/interspeech.2017-314 · Cited by 276 · see citing works
Voice conversion using deep Bidirectional Long Short-Term Memory based Recurrent Neural Networks
Sun, L., Kang, S., Li, K., & Meng, H. M. L. (2015) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2015.7178896 · Cited by 261 · see citing works
Spectral Mapping Using Artificial Neural Networks for Voice Conversion
Desai, S., Black, A. W., Yegnanarayana, B., & Prahallad, K. S. (2010) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2010.2047683 · Cited by 224 · see citing works
Voice Conversion Using Deep Neural Networks With Layer-Wise Generative Training
Chen, L.-H., Ling, Z.-H., Liu, L., & Dai, L.-R. (2014) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2014.2353991 · Cited by 210 · see citing works
Voice Conversion Using Partial Least Squares Regression
Helander, E. E., Virtanen, T. I., Nurminen, J., & Gabbouj, M. (2010) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2010.2041699 · Cited by 207 · see citing works
Statistical Voice Conversion Techniques for Body-Conducted Unvoiced Speech Enhancement
Toda, T., Nakagiri, M., & Shikano, K. (2012) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2012.2205241 · Cited by 195 · see citing works
Speaking-aid systems using GMM-based voice conversion for electrolaryngeal speech
Nakamura, K., Toda, T., Saruwatari, H., & Shikano, K. (2011) · Speech Communication · doi:10.1016/j.specom.2011.07.007 · Cited by 171 · see citing works
Exemplar-Based Sparse Representation With Residual Compensation for Voice Conversion
Wu, Z., Virtanen, T. I., Chng, E. S., & Li, H. (2014) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2014.2333242 · Cited by 166 · see citing works
Parallel-Data-Free Voice Conversion Using Cycle-Consistent Adversarial NetworksPreprint
Kaneko, T., & Kameoka, H. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.11293 · Cited by 163 · see citing works
Self-citationImproving the intelligibility of dysarthric speech
Kain, A. B., Hosom, J.-P., Niu, X., van Santen, J. P. H., Fried-Oken, M. B., & Staehely, J. (2007) · Speech Communication · doi:10.1016/j.specom.2007.05.001 · Cited by 157 · see citing works
“High-quality nonparallel voice conversion based on cycle-consistent adversarial network,”
Fang, F., Yamagishi, J., Echizen, I., & Lorenzo-Trueba, J. (2018) · 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2018.8462342 · Cited by 128 · see citing works
Generative adversarial network-based postfilter for statistical parametric speech synthesis
Kaneko, T., Kameoka, H., Hojo, N., Ijima, Y., Hiramatsu, K., & Kashino, K. (2017) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2017.7953090 · Cited by 121 · see citing works
Self-citationEigenvoice conversion based on Gaussian mixture model
Toda, T., Ohtani, Y., & Shikano, K. (2006) · Interspeech (USB) · doi:10.21437/interspeech.2006-613 · Cited by 119 · see citing works
Saito, Y., Ijima, Y., Nishida, K., & Takamichi, S. (2018) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2018.8461384 · Cited by 117 · see citing works
Kaneko, T., Kameoka, H., Hiramatsu, K., & Kashino, K. (2017) · Interspeech (USB) · doi:10.21437/interspeech.2017-970 · Cited by 110 · see citing works
Self-citationVoice conversion using deep neural networks with speaker-independent pre-training
Mohammadi, S. H., & Kain, A. B. (2014) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt.2014.7078543 · Cited by 105 · see citing works
Nonparallel training for voice conversion based on a parameter adaptation approach
Mouchtaris, A., der Spiegel, J. V., & Müeller, P. (2006) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tsa.2005.857790 · Cited by 104 · see citing works
Map-based adaptation for speech conversion using adaptation data selection and non-parallel training
Lee, C.-H., & Wu, C. (2006) · Interspeech (USB) · doi:10.21437/interspeech.2006-579 · Cited by 100 · see citing works
Data-driven emotion conversion in spoken English
Inanoglu, Z., & Young, S. J. (2008) · Speech Communication · doi:10.1016/j.specom.2008.09.006 · Cited by 92 · see citing works
One-to-many voice conversion based on tensor representation of speaker space
Saito, D., Yamamoto, K., Minematsu, N., & Hirose, K. (2011) · Interspeech (USB) · doi:10.21437/interspeech.2011-268 · Cited by 86 · see citing works
A KL Divergence and DNN-Based Approach to Voice Conversion without Parallel Training Sentences
Xie, F., Soong, F. K., & Li, H. (2016) · Interspeech (USB) · doi:10.21437/interspeech.2016-116 · Cited by 81 · see citing works
Liu, K., Zhang, J., & Yan, Y. (2007) · International Conference on Fuzzy Systems and Knowledge Discovery (FSKD) · doi:10.1109/fskd.2007.347 · Cited by 72 · see citing works
Generative Adversarial Network-Based Postfilter for STFT Spectrograms
Kaneko, T., Takaki, S., Kameoka, H., & Yamagishi, J. (2017) · Interspeech (USB) · doi:10.21437/interspeech.2017-962 · Cited by 66 · see citing works
Self-citationNakashika, T., Takiguchi, T., & Ariki, Y. (2014) · Interspeech (USB) · doi:10.21437/interspeech.2014-447 · Cited by 64 · see citing works
Non-Parallel Training in Voice Conversion Using an Adaptive Restricted Boltzmann Machine
Nakashika, T., Takiguchi, T., & Minami, Y. (2016) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2016.2593263 · Cited by 63 · see citing works
Exemplar-Based Voice Conversion Using Sparse Representation in Noisy Environments
Takashima, R., Takiguchi, T., & Ariki, Y. (2013) · IEICE Transactions on Fundamentals of Electronics Communications and Computer Sciences · doi:10.1587/transfun.e96.a.1946 · Cited by 44 · see citing works
Voice Conversion Based on Speaker-Dependent Restricted Boltzmann Machines
Nakashika, T., Takiguchi, T., & Ariki, Y. (2014) · IEICE Transactions on Information and Systems · doi:10.1587/transinf.e97.d.1403 · Cited by 43 · see citing works
Kameoka, H., Kaneko, T., Tanaka, K., & Hojo, N. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.05092 · Cited by 41 · see citing works
Self-citationSynthetic-to-Natural Speech Waveform Conversion Using Cycle-Consistent Adversarial Networks
Tanaka, K., Kaneko, T., Hojo, N., & Kameoka, H. (2018) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt.2018.8639636 · Cited by 36 · see citing works
Self-citationOyamada, K., Kameoka, H., Kaneko, T., Ando, H., Hiramatsu, K., & Kashino, K. (2017) · Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) · doi:10.1109/apsipa.2017.8282025 · Cited by 20 · see citing works
Self-citationKobayashi, K., Toda, T., & Nakamura, S. (2016) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt.2016.7846338 · Cited by 16 · see citing works
Flagged first, then by citations
2 of the 58 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Masood, M., Nawaz, M., Malik, K. M., Javed, A., Irtaza, A., & Malik, H. (2023) · Applied Intelligence · doi:10.1007/s10489-022-03766-z · Cited by 543 · see citing works
An Overview of Voice Conversion and Its Challenges: From Statistical Modeling to Deep Learning
Şişman, B., Yamagishi, J., King, S., & Li, H. (2020) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2020.3038524 · Cited by 374 · see citing works
StarGAN-VC2: Rethinking Conditional Methods for StarGAN-Based Voice Conversion
Kaneko, T., Kameoka, H., Tanaka, K., & Hojo, N. (2019) · Interspeech (USB) · doi:10.21437/interspeech.2019-2236 · Cited by 154 · see citing works
Self-citationIntelligent Pneumonia Identification From Chest X-Rays: A Systematic Literature Review
Khan, W., Zaki, N., & Ali, L. (2021) · IEEE Access · doi:10.1109/access.2021.3069937 · Cited by 129 · see citing works
A Review of Synthetic Image Data and Its Use in Computer Vision
Man, K., & Chahl, J. (2022) · Journal of Imaging · doi:10.3390/jimaging8110310 · Cited by 121 · see citing works
Fetal ECG Extraction From Maternal ECG Using Attention-Based CycleGAN
Mohebbian, M. R., Vedaei, S. S., Wahid, K. A., Dinh, A., Marateb, H. R., & Tavakolian, K. (2021) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2021.3111873 · Cited by 105 · see citing works
Data Augmentation for Deep Neural Networks Model in EEG Classification Task: A Review
He, C., Liu, J., Zhu, Y., & Du, W. (2021) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2021.765525 · Cited by 105 · see citing works
Zhang, J.-X., Ling, Z.-H., & Dai, L.-R. (2019) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2019.2960721 · Cited by 105 · see citing works
Again-VC: A One-Shot Voice Conversion Using Activation Guidance and Adaptive Instance Normalization
Chen, Y.-H., Wu, D.-Y., Wu, T.-H., & Lee, H.-Y. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414257 · Cited by 105 · see citing works
CycleGAN-VC3: Examining and Improving CycleGAN-VCs for Mel-Spectrogram Conversion
Kaneko, T., Kameoka, H., Tanaka, K., & Hojo, N. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-2280 · Cited by 96 · see citing works
Self-citationLuleci, F., Çatbaş, F. N., & Avcı, O. (2023) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2023.110370 · Cited by 95 · see citing works
Understanding GANs: fundamentals, variants, training challenges, applications, and open problems
Ahmad, Z., Jaffri, Z. U. A., Chen, M., & Bao, S. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-024-19361-y · Cited by 92 · see citing works
An Overview of Affective Speech Synthesis and Conversion in the Deep Learning Era
Triantafyllopoulos, A., Schuller, B. W., İymen, G., Sezgin, T. M., He, X., Yang, Z., Tzirakis, P., Liu, S., Mertes, S., André, E., et al. (2023) · Proceedings of the IEEE · doi:10.1109/jproc.2023.3250266 · Cited by 89 · see citing works
Nonparallel Emotional Speech Conversion
Gao, J., Chakraborty, D., Tembiné, H., & Olaleye, O. (2019) · Interspeech (USB) · doi:10.21437/interspeech.2019-2878 · Cited by 75 · see citing works
Maskcyclegan-VC: Learning Non-Parallel Voice Conversion with Filling in Frames
Kaneko, T., Kameoka, H., Tanaka, K., & Hojo, N. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414851 · Cited by 75 · see citing works
Self-citationTransforming Spectrum and Prosody for Emotional Voice Conversion with Non-Parallel Training Data
Zhou, K., Şişman, B., & Li, H. (2020) · The Speaker and Language Recognition Workshop (Odyssey) · doi:10.21437/odyssey.2020-33 · Cited by 67 · see citing works
TIMIT-TTS: A Text-to-Speech Dataset for Multimodal Synthetic Media Detection
Salvi, D., Hosler, B., Bestagini, P., Stamm, M. C., & Tubaro, S. (2023) · IEEE Access · doi:10.1109/access.2023.3276480 · Cited by 65 · see citing works
Defending Your Voice: Adversarial Attack on Voice Conversion
Huang, C., Lin, Y. Y., Lee, H.-Y., & Lee, L.-S. (2021) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt48900.2021.9383529 · Cited by 57 · see citing works
Basak, P., Sakib, A. H., Chowdhury, M. E. H., Al-Emadi, N. A., Yalcin, H. C., Pedersen, S., Mahmud, S., Serkan Kiranyaz, B., & Al-Maadeed, S. A. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.121196 · Cited by 55 · see citing works
Xiang, Y., & Bao, C. (2020) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2020.2997118 · Cited by 52 · see citing works
Low-Resource Expressive Text-To-Speech Using Data Augmentation
Huybrechts, G., Merritt, T., Comini, G., Perz, B., Shah, R., & Lorenzo-Trueba, J. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9413466 · Cited by 50 · see citing works
Huang, W.-C., Luo, H., Hwang, H.-T., Lo, C.-C., Peng, Y.-H., Tsao, Y., & Wang, H.-M. (2020) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2020.2977678 · Cited by 48 · see citing works
SINGAN: Singing Voice Conversion with Generative Adversarial Networks
Şişman, B., Vijayan, K., Dong, M., & Li, H. (2019) · Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) · doi:10.1109/apsipaasc47483.2019.9023162 · Cited by 46 · see citing works
Wang, M., Xie, P., Du, Y., & Hu, X. (2023) · Applied Sciences · doi:10.3390/app13127111 · Cited by 46 · see citing works
Personalized Adversarial Data Augmentation for Dysarthric and Elderly Speech Recognition
Jin, Z., Geng, M., Deng, J., Wang, T., Hu, S., Li, G., & Liu, X. (2023) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2023.3323888 · Cited by 46 · see citing works
Gao, Y., Jin, Y., Chauhan, J., Choi, S., Li, J., & Jin, Z. (2021) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3448113 · Cited by 45 · see citing works
Detecting Voice Cloning Attacks via Timbre Watermarking
Liu, C., Zhang, J., Zhang, T., Yang, X., Zhang, W., & Yu, N. (2024) · Proceedings 2024 Network and Distributed System Security Symposium · doi:10.14722/ndss.2024.24200 · Cited by 40 · see citing works
Many-To-Many Voice Conversion Using Conditional Cycle-Consistent Adversarial Networks
Lee, S., Ko, B., Lee, K., Yoo, I.-C., & Yook, D. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9053726 · Cited by 38 · see citing works
Seismic Data Augmentation Based on Conditional Generative Adversarial Networks
Li, Y., Ku, B., Zhang, S., Ahn, J.-K., & Ko, H. (2020) · Sensors · doi:10.3390/s20236850 · Cited by 38 · see citing works
Generative Adversarial Networks in Human Emotion Synthesis: A Review
Hajarolasvadi, N., Ramírez, M. A., Beccaro, W., & Demirel, H. (2020) · IEEE Access · doi:10.1109/access.2020.3042328 · Cited by 34 · see citing works
MelGAN-VC: Voice Conversion and Audio Style Transfer on arbitrarily long samples using SpectrogramsPreprint
Pasini, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.03713 · Cited by 38 · see citing works
An Adaptive-Learning-Based Generative Adversarial Network for One-to-One Voice Conversion
Dhar, S., Jana, N. D., & Das, S. (2022) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2022.3149858 · Cited by 28 · see citing works
Nonparallel Voice Conversion With Augmented Classifier Star Generative Adversarial Networks
Kameoka, H., Kaneko, T., Tanaka, K., & Hojo, N. (2020) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2020.3036784 · Cited by 28 · see citing works
Self-citationEmotionless: Privacy-Preserving Speech Analysis for Voice Assistants
Aloufi, R., Haddadi, H., & Boyle, D. (2019) · arXiv · Cited by 39 · see citing works
Duration Controllable Voice Conversion via Phoneme-Based Information Bottleneck
Lee, S.-H., Noh, H.-R., Nam, W.-J., & Lee, S. (2022) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2022.3156757 · Cited by 25 · see citing works
Sequence-to-Sequence Emotional Voice Conversion With Strength Control
Choi, H., & Hahn, M. (2021) · IEEE Access · doi:10.1109/access.2021.3065460 · Cited by 23 · see citing works
An Overview of Voice Conversion and its Challenges: From Statistical Modeling to Deep LearningPreprint
Şişman, B., Yamagishi, J., King, S., & Li, H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.03648 · Cited by 23 · see citing works
Voice Conversion with Transformer Network
Liu, R., Chen, X., & Wen, X. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9054523 · Cited by 22 · see citing works
Matsubara, K., Okamoto, T., Takashima, R., Takiguchi, T., Toda, T., Shiga, Y., & Kawai, H. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414136 · Cited by 21 · see citing works
VoiceGrad: Non-Parallel Any-to-Many Voice Conversion With Annealed Langevin Dynamics
Kameoka, H., Kaneko, T., Tanaka, K., Hojo, N., & Seki, S. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3379901 · Cited by 21 · see citing works
Self-citationTGAVC: Improving Autoencoder Voice Conversion with Text-Guided and Adversarial Training
Tang, H., Zhang, X., Wang, J., Cheng, N., Zeng, Z., Xiao, E., & Xiao, J. (2021) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru51503.2021.9688088 · Cited by 19 · see citing works
Dance Style Transfer with Cross-modal Transformer
Yin, W., Yin, H., Baraka, K., Kragić, D., & Björkman, M. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00503 · Cited by 19 · see citing works
Chu, M., Yang, M., Xu, C., Ma, Y., Wang, J., Fan, Z., Tao, Z., & Wu, D. (2023) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2023.3239551 · Cited by 19 · see citing works
STATNet: Spectral and Temporal features based Multi-Task Network for Audio Spoofing Detection
Ranjan, R., Vatsa, M., & Singh, R. (2022) · 2022 IEEE International Joint Conference on Biometrics (IJCB) · doi:10.1109/ijcb54206.2022.10007949 · Cited by 18 · see citing works
Expressive Voice Conversion: A Joint Framework for Speaker Identity and Emotional Style Transfer
Du, Z., Şişman, B., Zhou, K., & Li, H. (2021) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru51503.2021.9687906 · Cited by 18 · see citing works
Multimodal dance style transfer
Yin, W., Yin, H., Baraka, K., Kragić, D., & Björkman, M. (2023) · Machine Vision and Applications · doi:10.1007/s00138-023-01399-x · Cited by 18 · see citing works
Wang, Y., Yu, G., Wang, J., Wang, H., & Zhang, Q. (2020) · IEEE Access · doi:10.1109/access.2020.3029417 · Cited by 18 · see citing works
MS-GAN: multi-scale GAN with parallel class activation maps for image reconstruction
Rao, J., Ke, A., Liu, G., & Ming, Y. (2022) · The Visual Computer · doi:10.1007/s00371-022-02468-4 · Cited by 18 · see citing works
Data Augmentation Using CycleGAN for End-to-End Children ASR
Singh, D. K., Amin, P. P., Sailor, H. B., & Patil, H. A. (2021) · European Signal Processing Conference (EUSIPCO) · doi:10.23919/eusipco54536.2021.9616228 · Cited by 17 · see citing works
Wani, T. M., Qadri, S. A. A., Wani, F. A., & Amerini, I. (2024) · Foundations and Trends® in Privacy and Security · doi:10.1561/3300000048 · Cited by 16 · see citing works
Kobayashi, K., Huang, W.-C., Wu, Y.-C., Tobing, P. L., Hayashi, T., & Toda, T. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9413959 · Cited by 16 · see citing works
Aloufi, R., Haddadi, H., & Boyle, D. (2019) · doi:10.1145/3362743.3362960 · Cited by 16 · see citing works
Face-based Voice Conversion: Learning the Voice behind a Face
Lu, H.-H., Weng, S.-E., Yen, Y.-F., Shuai, H.-H., & Cheng, W.-H. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3475198 · Cited by 15 · see citing works
Cai, D., Cai, Z., & Li, M. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10096733 · Cited by 15 · see citing works
VoicePM: A Robust Privacy Measurement on Voice Anonymity
Zhang, S., Li, Z., & Das, A. (2023) · ACM Conference on Security and Privacy in Wireless and Mobile Networks (WiSec) · doi:10.1145/3558482.3590175 · Cited by 15 · see citing works
Transforming Spectrum and Prosody for Emotional Voice Conversion with Non-Parallel Training DataPreprint
Zhou, K., Şişman, B., & Li, H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.00198 · Cited by 15 · see citing works
Cyclegean: Cycle Generative Enhanced Adversarial Network for Voice Conversion
Zhang, X., Wang, J., Cheng, N., Xiao, E., & Xiao, J. Z. (2021) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru51503.2021.9687948 · Cited by 14 · see citing works
Many-to-Many Voice Conversion Using Cycle-Consistent Variational Autoencoder with Multiple Decoders
Yook, D., Leem, S.-G., Lee, K., & Yoo, I.-C. (2020) · The Speaker and Language Recognition Workshop (Odyssey) · doi:10.21437/odyssey.2020-31 · Cited by 14 · see citing works
Learning the Beauty in Songs: Neural Singing Voice Beautifier
Liu, J., Li, C., Ren, Y., Zhu, Z., & Zhao, Z. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.549 · Cited by 14 · see citing works
Style Equalization: Unsupervised Learning of Controllable Generative Sequence ModelsPreprint
Chang, J.-H. R., Shrivastava, A., Koppula, H. S., Zhang, X., & Tuzel, O. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.02891 · Cited by 15 · see citing works
PAVITS: Exploring Prosody-Aware VITS for End-to-End Emotional Voice Conversion
Qi, T., Zheng, W., Lu, C., Zong, Y., & Lian, H. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10446191 · Cited by 13 · see citing works
CycleGAN-VC-GP: Improved CycleGAN-based Non-parallel Voice Conversion
Wang, C., & Yu, Y. (2020) · IEEE International Conference on Communication Technology (ICCT) · doi:10.1109/icct50939.2020.9295938 · Cited by 13 · see citing works
Zero-Shot Voice Conversion with Adjusted Speaker Embeddings and Simple Acoustic Features
Tan, Z., Wei, J., Xu, J., He, Y., & Lu, W. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414975 · Cited by 13 · see citing works
Akhter, M. T., Banerjee, P., Dhar, S., Ghosh, S., & Jana, N. D. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-48309-7_20 · Cited by 12 · see citing works
GLGAN-VC: A Guided Loss-Based Generative Adversarial Network for Many-to-Many Voice Conversion
Dhar, S., Jana, N. D., & Das, S. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3335119 · Cited by 12 · see citing works
Towards Low-Resource Stargan Voice Conversion Using Weight Adaptive Instance Normalization
Chen, M., Shi, Y., & Hain, T. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9415042 · Cited by 12 · see citing works
Voice privacy using CycleGAN and time-scale modification
Prajapati, G. P., Singh, D. K., Amin, P. P., & Patil, H. A. (2022) · Computer Speech & Language · doi:10.1016/j.csl.2022.101353 · Cited by 12 · see citing works
The Sound of Silence: Efficiency of First Digit Features in Synthetic Audio Detection
Mari, D., Latora, F., & Milani, S. (2022) · IEEE International Workshop on Information Forensics and Security · doi:10.1109/wifs55849.2022.9975404 · Cited by 12 · see citing works
CVC: Contrastive Learning for Non-Parallel Voice Conversion
Li, T., Liu, Y., Hu, C., & Zhao, H. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-137 · Cited by 12 · see citing works
End-to-End Zero-Shot Voice Conversion with Location-Variable Convolutions
Kang, W., Hasegawa‐Johnson, M., & Roy, D. K. (2023) · Interspeech (USB) · doi:10.21437/interspeech.2023-2298 · Cited by 13 · see citing works
Conan's Bow Tie: A Streaming Voice Conversion for Real-Time VTuber Livestreaming
Chen, Q., Gu, Z. P., Lü, L., Xu, X., Ba, Z., Lin, F., Liu, Z., & Ren, K. (2024) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3640543.3645146 · Cited by 11 · see citing works
Voice Conversion with Denoising Diffusion Probabilistic GAN Models
Zhang, X., Wang, J., Cheng, N., & Xiao, J. Z. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-46674-8_11 · Cited by 11 · see citing works
A Modularized Neural Network with Language-Specific Output Layers for Cross-Lingual Voice Conversion
Zhou, Y., Tian, X., Yılmaz, E., Das, R. K., & Li, H. (2019) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru46091.2019.9003798 · Cited by 11 · see citing works
Yin, J., Li, Z., Cui, L., Zhang, W., Wang, Q., & Si, G. (2023) · IEEE International Conference on Power, Intelligent Computing and Systems (ICPICS) · doi:10.1109/icpics58376.2023.10235612 · Cited by 10 · see citing works
Any-to-Any Voice Conversion with F0 and Timbre Disentanglement and Novel Timbre Conditioning
Kovela, S., Valle, R., Dantrey, A., & Catanzaro, B. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10096220 · Cited by 10 · see citing works
RefXVC: Cross-Lingual Voice Conversion With Enhanced Reference Leveraging
Zhang, M., Zhou, Y., Ren, Y., Zhang, C., Yin, X., & Li, H. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3439996 · Cited by 9 · see citing works
CycleGAN Voice Conversion of Spectral Envelopes using Adversarial Weights
Ferro, R., Obin, N., & Röebel, A. (2020) · European Signal Processing Conference (EUSIPCO) · doi:10.23919/eusipco47968.2020.9287643 · Cited by 10 · see citing works
Jia, N., & Zheng, C. (2021) · Arabian Journal for Science and Engineering · doi:10.1007/s13369-021-06090-9 · Cited by 9 · see citing works
DGC-Vector: A New Speaker Embedding for Zero-Shot Voice Conversion
Xiao, R., Zhang, H., & Lin, Y. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9746278 · Cited by 9 · see citing works
HiFi-SVC: Fast High Fidelity Cross-Domain Singing Voice Conversion
Zhou, Y., & Lu, X. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9746812 · Cited by 9 · see citing works
Unsupervised Audiovisual Synthesis via Exemplar AutoencodersPreprint
Deng, K., Bansal, A., & Ramanan, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.04463 · Cited by 12 · see citing works
CASIA Voice Conversion System for the Voice Conversion Challenge 2020
Lian, Z., Tao, J., Wen, Z., & Zhong, R. (2020) · doi:10.21437/vcc_bc.2020-19 · Cited by 8 · see citing works
Feature Map Regularized CycleGAN for Domain Transfer
Krstanović, L., Popovic, B. Z., Janev, M. B., & Brkljač, B. (2023) · Mathematics · doi:10.3390/math11020372 · Cited by 8 · see citing works
Gu, Y., Zhang, Z., Yi, X., & Zhao, X. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.02500 · Cited by 8 · see citing works
Transfer Learning From Speech Synthesis to Voice Conversion With Non-Parallel Training Data
Zhang, M., Zhou, Y., Li, Z., & Li, H. (2021) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2021.3066047 · Cited by 7 · see citing works
CycleDiffusion: Voice Conversion Using Cycle-Consistent Diffusion Models
Yook, D., Han, G., Chang, H.-P., & Yoo, I.-C. (2024) · Applied Sciences · doi:10.3390/app14209595 · Cited by 7 · see citing works
Reconstructing Dual Learning for Neural Voice Conversion Using Relatively Few Samples
Sun, A., Wang, J., Cheng, N., Tantrawenith, M., Wu, Z., Meng, H. M. L., Xiao, E., & Xiao, J. Z. (2021) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru51503.2021.9687965 · Cited by 7 · see citing works
Dhar, S., Banerjee, P., Jana, N. D., & Das, S. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10095069 · Cited by 7 · see citing works
A Style Transfer Approach to Source Separation
Venkataramani, S., Tzinis, E., & Smaragdis, P. (2019) · IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA) · doi:10.1109/waspaa.2019.8937203 · Cited by 7 · see citing works
Random Cycle Loss and Its Application to Voice Conversion
Sun, H., Wang, D., Li, L., Chen, C., & Zheng, T. F. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3257839 · Cited by 7 · see citing works
Chun, C., Lee, Y. H., Lee, G. W., Jeon, M., & Kim, H. K. (2023) · IEEE Consumer Communications and Networking Conference · doi:10.1109/ccnc51644.2023.10060510 · Cited by 7 · see citing works
MFCCGAN: A Novel MFCC-Based Speech Synthesizer Using Adversarial Learning
Hasanabadi, M. R., Behdad, M., & Gharavian, D. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10095873 · Cited by 7 · see citing works
One-Shot Voice Conversion Based on Speaker Aware Module
Zhang, Y., Che, H., Li, J., Li, C., Wang, X., & Wang, Z. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414081 · Cited by 7 · see citing works
Attention-based Multimodal learning framework for Generalized Audio-Visual Deepfake Detection
Masood, M., Javed, A., & Irtaza, A. (2023) · Preprint (Research Square) · doi:10.21203/rs.3.rs-3415144/v1 · Cited by 6 · see citing works
Bargum, A. R., Serafin, S., & Erkut, C. (2024) · Frontiers in Signal Processing · doi:10.3389/frsip.2024.1339159 · Cited by 9 · see citing works
CycleGANAS: Differentiable Neural Architecture Search for CycleGAN
An, T., & Joo, C. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00172 · Cited by 6 · see citing works
MADD: A Multi-Lingual Multi-Speaker Audio Deepfake Detection Dataset
Qi, X., Gu, H., Yi, J., Tao, J., Ren, Y., He, J., & Zeng, S. (2024) · International Symposium on Chinese Spoken Language Processing (ISCSLP) · doi:10.1109/iscslp63861.2024.10800535 · Cited by 7 · see citing works
Singing Voice Conversion Based on WD-GAN Algorithm
Zhao, W., Wang, W., Sun, Y., & Tang, T. (2019) · IEEE Advanced Information Technology, Electronic and Automation Control Conference (IAEAC) · doi:10.1109/iaeac47372.2019.8997824 · Cited by 6 · see citing works
Effectiveness of Transfer Learning on Singing Voice Conversion in the Presence of Background Music
Rajpura, D. G., Shah, J., Patel, M., Malaviya, H., Phatnani, K. S., & Patil, H. A. (2020) · International Conference on Signal Processing and Communications (SPCOM) · doi:10.1109/spcom50965.2020.9179583 · Cited by 6 · see citing works
Dhar, S., Akhter, M. T., Banerjee, P., Jana, N. D., & Das, S. (2023) · Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) · doi:10.1109/apsipaasc58517.2023.10317438 · Cited by 6 · see citing works
- Authors
- Kaneko, T., Kameoka, H., Tanaka, K., & Hojo, N.
- Affiliations
- NTT (Japan) (JP)
- Venue
- ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
- Publisher
- IEEE
- Published
- 2019-05
- Pages
- 6820-6824
- ISSN
- 1520-6149 (ISSN-L)
- OpenAlex
- W2937579788
- Cited by
- 272
- References
- 58
- Citations by year
- 2026: 14 · 2025: 34 · 2024: 36 · 2023: 57 · 2022: 39 · 2021: 43 · 2020: 34 · 2019: 15
- FWCI
- 12.53
- Type
- conference paper
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE International Conference on Acoustics Speech and Signal Processing (publishedVersion)
- Funders
- Virginia Agricultural Experiment Station, Virginia Polytechnic Institute and State University
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Health Sciences › Medicine › Physiology › Voice and Speech Disorders
- Keywords
- non-parallel voice conversion, adversarial training, PatchGAN discriminator, Mel cepstral distortion, speech naturalness
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
