“StarGAN-VC: non-parallel many-to-many voice conversion using star generative adversarial networks,”
The DOI matches its Crossref record; 55 of the 55 references it cites were found in a registry or an index.
Abstract
This paper proposes a method that allows non-parallel many-to-many voice conversion (VC) by using a variant of a generative adversarial network (GAN) called StarGAN. Our method, which we call StarGAN-VC, is noteworthy in that it (1) requires no parallel utterances, transcriptions, or time alignment procedures for speech generator training, (2) simultaneously learns many-to-many mappings across different attribute domains using a single generator network, (3) is able to generate converted speech signals quickly enough to allow real-time implementations and (4) requires only several minutes of training examples to generate reasonably realistic sounding speech. Subjective evaluation experiments on a non-parallel many-to-many speaker identity conversion task revealed that the proposed method obtained higher sound quality and speaker similarity than a state-of-the-art method based on variational autoencoding GANs.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“For example, CycleGAN and StarGAN are two frameworks for image deepfake generation that are used as the base of two audio deepfake frameworks (Fang et al., 2018; Kameoka et al., 2018), which can work with non-parallel data not just parallel ones.”
From Audio deepfakes: A survey (2023) · cites this paper as Kameoka et al., 2018
Citations
Cites
Unpaired image-to-image translation using cycle-consistent adversarial networks
Zhu, J.-Y., Park, T., Isola, P., & Efros, A. A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.244 · Cited by 23,602
Image-to-Image Translation with Conditional Adversarial Networks
Isola, P., Zhu, J.-Y., Zhou, T., & Efros, A. A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.632 · Cited by 22,930
Auto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067
Labaca-Castro, R. (2023) · Machine Learning under Malware Attack · doi:10.1007/978-3-658-40442-0_9 · Cited by 19,272
GAN(Generative Adversarial Nets)
(2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,270
WaveNet: a generative model for raw audio
Oord, A. V. D., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1609.03499 · Cited by 4,911
StarGAN: Unified Generative Adversarial Networks for Multi-domain Image-to-Image Translation
Choi, Y., Choi, M., Kim, M., Ha, J.-W., Kim, S., & Choo, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00916 · Cited by 4,163
Front-End Factor Analysis for Speaker Verification
Dehak, N., Kenny, P. J., Dehak, R., Dumouchel, P., & Ouellet, P. (2011) · IEEE Transactions on Audio, Speech, and Language Processing · doi:10.1109/tasl.2010.2064307 · Cited by 3,549
Neural Discrete Representation Learning
Oord, A. V. D., Vinyals, O., & Kavukcuoglu, K. (2017) · arXiv · doi:10.48550/arxiv.1711.00937 · Cited by 2,729
DualGAN: Unsupervised Dual Learning for Image-to-Image Translation
Yi, Z., Zhang, H., Tan, P., & Gong, M. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.310 · Cited by 2,134
Autoencoding beyond pixels using a learned similarity metric
Larsen, A., Sønderby, S. K., Larochelle, H., & Winther, O. (2016) · Research at the University of Copenhagen (University of Copenhagen) · Cited by 1,989
Learning to Discover Cross-Domain Relations with Generative Adversarial Networks
Kim, T., Cha, M., Kim, H., Lee, J. K., & Kim, J. (2017) · arXiv · doi:10.48550/arxiv.1703.05192 · Cited by 1,716
Semi-Supervised Learning with Deep Generative Models
Kingma, D. P., Rezende, D. J., Mohamed, S., & Welling, M. (2014) · UvA-DARE (University of Amsterdam) · doi:10.48550/arxiv.1406.5298 · Cited by 1,415
SEGAN: Speech Enhancement Generative Adversarial Network
Pascual, S., Bonafonte, A., & Serrà, J. (2017) · Interspeech 2017 · doi:10.21437/interspeech.2017-1428 · Cited by 1,297
Jointly Embedding Multiple Single-Cell Omics Measurements
Liu, J., Huang, Y., Singh, R., Vert, J., & Noble, W. S. (2019) · Leibniz international proceedings in informatics · doi:10.4230/lipics.wabi.2019.10 · Cited by 1,141
WORLD: A Vocoder-Based High-Quality Speech Synthesis System for Real-Time Applications
MORISE, M., YOKOMORI, F., & OZAWA, K. (2016) · IEICE Transactions on Information and Systems · doi:10.1587/transinf.2015edp7457 · Cited by 1,092
Language Modeling with Gated Convolutional Networks
Dauphin, Y., Fan, A., Auli, M., & Grangier, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.08083 · Cited by 1,080
Voice Conversion Based on Maximum-Likelihood Estimation of Spectral Parameter Trajectory
Toda, T., Black, A. W., & Tokuda, K. (2007) · IEEE Transactions on Audio, Speech and Language Processing · doi:10.1109/tasl.2007.907344 · Cited by 937
Continuous probabilistic transform for voice conversion
Stylianou, Y., Cappe, O., & Moulines, E. (1998) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/89.661472 · Cited by 931
(2017) · International Conference on Machine Learning · Cited by 758
Spectral voice conversion for text-to-speech synthesis
Kain, A., & Macon, M. W. (2002) · Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181) · doi:10.1109/icassp.1998.674423 · Cited by 552
Parallel WaveNet: Fast High-Fidelity Speech Synthesis
van den Oord, A., Li, Y., Babuschkin, I., Simonyan, K., Vinyals, O., Kavukcuoglu, K., van den Driessche, G., Lockhart, E., Cobo, L. C., Stimberg, F., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.10433 · Cited by 330
Hsu, C.-C., Hwang, H.-T., Wu, Y.-C., Tsao, Y., & Wang, H.-M. (2017) · Interspeech 2017 · doi:10.21437/interspeech.2017-63 · Cited by 320
The Voice Conversion Challenge 2018: Promoting Development of Parallel and Nonparallel Methods
Lorenzo-Trueba, J., Yamagishi, J., Toda, T., Saito, D., Villavicencio, F., Kinnunen, T., & Ling, Z. (2018) · The Speaker and Language Recognition Workshop (Odyssey 2018) · doi:10.21437/odyssey.2018-28 · Cited by 305
Voice conversion from non-parallel corpora using variational auto-encoder
Hsu, C.-C., Hwang, H.-T., Wu, Y.-C., Tsao, Y., & Wang, H.-M. (2016) · 2016 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA) · doi:10.1109/apsipa.2016.7820786 · Cited by 289
Voice conversion using deep Bidirectional Long Short-Term Memory based Recurrent Neural Networks
Sun, L., Kang, S., Li, K., & Meng, H. (2015) · 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2015.7178896 · Cited by 257
Spectral Mapping Using Artificial Neural Networks for Voice Conversion
Desai, S., Black, A. W., Yegnanarayana, B., & Prahallad, K. (2010) · IEEE Transactions on Audio, Speech, and Language Processing · doi:10.1109/tasl.2010.2047683 · Cited by 223
Statistical Parametric Speech Synthesis Incorporating Generative Adversarial Networks
Saito, Y., Takamichi, S., & Saruwatari, H. (2018) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2017.2761547 · Cited by 223
Voice Conversion Using Deep Neural Networks With Layer-Wise Generative Training
Chen, L.-H., Ling, Z.-H., Liu, L.-J., & Dai, L.-R. (2014) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2014.2353991 · Cited by 210
Voice Conversion Using Partial Least Squares Regression
Helander, E., Virtanen, T., Nurminen, J., & Gabbouj, M. (2010) · IEEE Transactions on Audio, Speech, and Language Processing · doi:10.1109/tasl.2010.2041699 · Cited by 207
Statistical Voice Conversion Techniques for Body-Conducted Unvoiced Speech Enhancement
Toda, T., Nakagiri, M., & Shikano, K. (2012) · IEEE Transactions on Audio, Speech, and Language Processing · doi:10.1109/tasl.2012.2205241 · Cited by 182
Speaking-aid systems using GMM-based voice conversion for electrolaryngeal speech
Nakamura, K., Toda, T., Saruwatari, H., & Shikano, K. (2012) · Speech Communication · doi:10.1016/j.specom.2011.07.007 · Cited by 169
Exemplar-Based Sparse Representation With Residual Compensation for Voice Conversion
Wu, Z., Virtanen, T., Chng, E. S., & Li, H. (2014) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2014.2333242 · Cited by 166
Improving the intelligibility of dysarthric speech
Kain, A. B., Hosom, J.-P., Niu, X., van Santen, J. P. H., Fried-Oken, M., & Staehely, J. (2007) · Speech Communication · doi:10.1016/j.specom.2007.05.001 · Cited by 158
Parallel-Data-Free Voice Conversion Using Cycle-Consistent Adversarial Networks
Kaneko, T., & Kameoka, H. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.11293 · Cited by 158
Self-citationKaneko, T., Kameoka, H., Hiramatsu, K., & Kashino, K. (2017) · Interspeech (USB) · doi:10.21437/interspeech.2017-970 · Cited by 127
Self-citationGenerative adversarial network-based postfilter for statistical parametric speech synthesis
Kaneko, T., Kameoka, H., Hojo, N., Ijima, Y., Hiramatsu, K., & Kashino, K. (2017) · 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2017.7953090 · Cited by 119
Self-citationSaito, Y., Ijima, Y., Nishida, K., & Takamichi, S. (2018) · 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2018.8461384 · Cited by 117
Voice conversion using deep neural networks with speaker-independent pre-training
Mohammadi, S. H., & Kain, A. (2014) · 2014 IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt.2014.7078543 · Cited by 94
Modeling and Transforming Speech Using Variational Autoencoders
Blaauw, M., & Bonada, J. (2016) · Interspeech (USB) · doi:10.21437/interspeech.2016-1183 · Cited by 86
Data-driven emotion conversion in spoken English
Inanoglu, Z., & Young, S. J. (2008) · Speech Communication · doi:10.1016/j.specom.2008.09.006 · Cited by 79
Liu, K., Zhang, J., & Yan, Y. (2007) · Fourth International Conference on Fuzzy Systems and Knowledge Discovery (FSKD 2007) · doi:10.1109/fskd.2007.347 · Cited by 66
A KL Divergence and DNN-Based Approach to Voice Conversion without Parallel Training Sentences
Xie, F., Soong, F. K., & Li, H. (2016) · Interspeech (USB) · doi:10.21437/interspeech.2016-116 · Cited by 66
The Voice Conversion Challenge 2018: Promoting Development of Parallel and Nonparallel MethodsPreprint
Lorenzo-Trueba, J., Yamagishi, J., Toda, T., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.04262 · Cited by 66
Generative Adversarial Network-Based Postfilter for STFT Spectrograms
Kaneko, T., Takaki, S., Kameoka, H., & Yamagishi, J. (2017) · Interspeech 2017 · doi:10.21437/interspeech.2017-962 · Cited by 65
Self-citationKinnunen, T., Juvela, L., Alku, P., & Yamagishi, J. (2017) · 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2017.7953215 · Cited by 62
Nakashika, T., Takiguchi, T., & Ariki, Y. (2014) · Interspeech 2014 · doi:10.21437/interspeech.2014-447 · Cited by 57
Evaluation of Expressive Speech Synthesis With Voice Conversion and Copy Resynthesis Techniques
Turk, O., & Schroder, M. (2010) · IEEE Transactions on Audio, Speech, and Language Processing · doi:10.1109/tasl.2010.2041113 · Cited by 56
StarGAN-VC: Non-parallel many-to-many voice conversion with star generative adversarial networksPreprint
Kameoka, H., Kaneko, T., Tanaka, K., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.02169 · Cited by 41
Self-citationExemplar-Based Voice Conversion Using Sparse Representation in Noisy Environments
TAKASHIMA, R., TAKIGUCHI, T., & ARIKI, Y. (2013) · IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences · doi:10.1587/transfun.e96.a.1946 · Cited by 39
Voice Conversion Based on Speaker-Dependent Restricted Boltzmann Machines
NAKASHIKA, T., TAKIGUCHI, T., & ARIKI, Y. (2014) · IEICE Transactions on Information and Systems · doi:10.1587/transinf.e97.d.1403 · Cited by 36
Generative adversarial network-based approach to signal reconstruction from magnitude spectrogram
Oyamada, K., Kameoka, H., Kaneko, T., Tanaka, K., Hojo, N., & Ando, H. (2018) · European Signal Processing Conference (EUSIPCO) · doi:10.23919/eusipco.2018.8553396 · Cited by 36
Self-citationVoice Conversion Using Input-to-Output Highway Networks
Saito, Y., Takamichi, S., & Saruwatari, H. (2017) · IEICE Transactions on Information and Systems · doi:10.1587/transinf.2017edl8034 · Cited by 29
Deep Clustering with Gated Convolutional Networks
Li, L., & Kameoka, H. (2018) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2018.8461746 · Cited by 23
Self-citationHashimoto, T., Uchida, H., Saito, D., & Minematsu, N. (2017) · Interspeech (USB) · doi:10.21437/interspeech.2017-961 · Cited by 5
Research works printed but not linked: 1
- nakashika, 2015, Parallel-data-free, many-to-many voice conversion using an adaptive restricted Boltzmann machine, Proc MLSP
Cited by
Masood, M., Nawaz, M., Malik, K. M., Javed, A., Irtaza, A., & Malik, H. (2023) · Applied Intelligence · doi:10.1007/s10489-022-03766-z · Cited by 543
An Overview of Voice Conversion and Its Challenges: From Statistical Modeling to Deep Learning
Sisman, B., Yamagishi, J., King, S., & Li, H. (2021) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2020.3038524 · Cited by 357
AUTOVC: Zero-Shot Voice Style Transfer with Only Autoencoder Loss
Qian, K., Zhang, Y., Chang, S., Yang, X., & Hasegawa-Johnson, M. (2019) · arXiv · doi:10.48550/arxiv.1905.05879 · Cited by 336
CycleGAN-VC2: improved CycleGAN-based non-parallel voice conversion.
Kaneko, T., Kameoka, H., Tanaka, K., & Hojo, N. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8682897 · Cited by 272
Self-citationChou, J.-C., & Lee, H.-Y. (2019) · Interspeech 2019 · doi:10.21437/interspeech.2019-2663 · Cited by 232
Metadata
- Authors
- Kameoka, H., Kaneko, T., Tanaka, K., & Hojo, N.
- Affiliations
- NTT (Japan) (JP)
- Venue
- 2018 IEEE Spoken Language Technology Workshop (SLT)
- Publisher
- IEEE
- Published
- 2018-12
- Volume
- 2018
- Pages
- 266-273
- OpenAlex
- W2963539064
- Cited by
- 371
- References
- 54
- Citations by year
- 2026: 23 · 2025: 26 · 2024: 42 · 2023: 55 · 2022: 57 · 2021: 76 · 2020: 58 · 2019: 32 · 2018: 2
- FWCI
- 17.14
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE Spoken Language Technology Workshop (SLT) (publishedVersion)
- IEEE Conference Proceedings
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Keywords
- voice conversion, StarGAN, generative adversarial networks, speaker similarity, sound quality
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.