Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Sign Up
Cited by 357

The DOI matches its Crossref record; 248 of the 248 references it cites were found in a registry or an index.

An Overview of Voice Conversion and Its Challenges: From Statistical Modeling to Deep Learning

Open access
Sisman, Berrak, Yamagishi, Junichi, King, Simon, Li, Haizhou·2021·IEEE/ACM Transactions on Audio, Speech, and Language Processing
DOI 10.1109/taslp.2020.3038524
Open this paper in CoChat — free account

Abstract

Speaker identity is one of the important characteristics of human speech. In voice conversion, we change the speaker identity from one to another, while keeping the linguistic content unchanged. Voice conversion involves multiple speech processing techniques, such as speech analysis, spectral conversion, prosody conversion, speaker characterization, and vocoding. With the recent advances in theory and practice, we are now able to produce human-like voice quality with high speaker similarity. In this article, we provide a comprehensive overview of the state-of-the-art of voice conversion techniques and their performance evaluation methods from the statistical approaches to deep learning, and discuss their promise and limitations. We will also report the recent Voice Conversion Challenges (VCC), the performance of the current state of technology, and provide a summary of the available resources for voice conversion research.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/taslp.2020.3038524

Cites

248 of 248 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 248 references

Research works printed but not linked: 66

  1. 10.1109/ICASSP.2018.8461960 · doi:10.1109/icassp.2018.8461960
  2. recommendation, 2001, 1534-1, Method for the subjective assessment of intermediate sound quality (MUSHRA), Int Telecommun Union
  3. benisty, 2011, Voice conversion using GMM with enhanced global variance, Proc 12th Annu Conf Int Speech Commun Assoc
  4. zielinski, 2007, Potential biases in MUSHRA listening tests, Proc 123rd Audio Eng Soc Convers
  5. 10.1016/j.asoc.2012.05.027 · doi:10.1016/j.asoc.2012.05.027
  6. song, 0, Text-independent voice conversion using speaker model alignment method from non-parallel speech, Proc Annu Conf Int Speech Commun Assoc INTERSPEECH, 2308
  7. tamura, 0, Speaker adaptation for HMM-based speech synthesis system using MLLR, Proc 3rd ESCA/COCOSDA Workshop (ETRW) Speech Synth
  8. remes, 2013, Objective evaluation measures for speaker-adaptive HMM-TTS systems, Proc 8th ISCA Workshop Speech Synth
  9. ping, 0, Deep voice 3: 2000-speaker neural text-to-speech, Proc ICLR, 214
  10. huang, 0, Prediction of perceived sound quality of synthetic speech, Proc Asia-Pacific Signal Inf Process Assoc Annu Summit Conf
  11. cernak, 0, An evaluation of synthetic speech using the PESQ measure, Proc Eur Congr Acoust, 2725
  12. erro, 0, Frame alignment method for cross-lingual voice conversion, Proc INTERSPEECH
  13. 10.1109/TASL.2010.2041688 · doi:10.1109/tasl.2010.2041688
  14. mashimo, 2002, Cross-language voice conversion evaluation using bilingual databases, Proc Int Conf Spoken Lang Process
  15. sundermann, 0, VTLN-based crosslanguage voice conversion, Proc IEEE Autom Speech Recognit Understanding Workshop
  16. 10.1109/ICASSP.2015.7178897 · doi:10.1109/icassp.2015.7178897
  17. 10.21437/Odyssey.2018-30 · doi:10.21437/odyssey.2018-30
  18. 10.21437/Odyssey.2018-34 · DOI
  19. 10.1109/TASLP.2019.2892235 · DOI
  20. fujii, 2007, High individuality voice conversion based on concatenative speech synthesis, Int J Elect Comput Energetic Electron Commun Eng, 1, 1617
  21. sagisaka, 0, Proc Int Conf Spoken Lang Process
  22. ney, 0, A first step towards text-independent voice conversion, Proc Int Conf Spoken Lang Process
  23. bi?kowski, 2018, Demystifying MMD GANs, arXiv 1801 01401
  24. 10.1109/ISCSLP.2014.6936599 · doi:10.1109/iscslp.2014.6936599
  25. 10.21437/Interspeech.2017-247 · doi:10.21437/interspeech.2017-247
  26. lorenzo-trueba, 2018, The voice conversion challenge: Promoting development of parallel and nonparallel methods, arXiv 1804 04262
  27. 10.1016/S0167-6393(99)00015-1 · doi:10.1016/s0167-6393(99)00015-1
  28. 10.23919/EUSIPCO.2019.8902651 · doi:10.23919/eusipco.2019.8902651
  29. kalchbrenner, 2018, Efficient neural audio synthesis, arXiv 1802 08435
  30. kim, 0, Flowavenet: A generative flow for raw audio, Proc Int Conf Mach Learn, 3370
  31. toda, 0, The voice conversion challenge, Proc INTERSPEECH, 1632
  32. 10.21437/Interspeech.2016-1331 · DOI
  33. 10.1109/ICCV.2017.244 · DOI · Paper page
  34. 10.21437/Interspeech.2018-1528 · doi:10.21437/interspeech.2018-1528
  35. 10.1007/978-3-030-00794-2_48 · doi:10.1007/978-3-030-00794-2_48
  36. 10.1109/ASRU46091.2019.9003801 · DOI
  37. 10.21437/Interspeech.2019-1514 · doi:10.21437/interspeech.2019-1514
  38. 10.1109/TASLP.2018.2835720 · DOI
  39. huang, 0, TimbreTron: A wavenet (cycleGAN (CQT (audio))) pipeline for musical timbre transfer, Proc Int Conf Learn Representations
  40. 10.1088/1742-6596/1607/1/012046 · doi:10.1088/1742-6596/1607/1/012046
  41. 10.21437/Odyssey.2020-35 · DOI
  42. 10.1007/978-3-642-35292-8_4 · doi:10.1007/978-3-642-35292-8_4
  43. 10.1109/APSIPA.2013.6694179 · doi:10.1109/apsipa.2013.6694179
  44. 10.5923/j.ajsp.20120205.06 · doi:10.5923/j.ajsp.20120205.06
  45. ribeiro, 0, A perceptual investigation of wavelet-based decomposition of f0 for text-to-speech synthesis, Proc 16th Annu Conf Int Speech Commun Assoc
  46. 10.1109/ICASSP.2016.7472734 · doi:10.1109/icassp.2016.7472734
  47. 10.1109/TASL.2009.2038663 · DOI · Paper page
  48. 10.1109/CHINSL.2008.ECP.44 · doi:10.1109/chinsl.2008.ecp.44
  49. 10.1016/j.specom.2006.08.001 · doi:10.1016/j.specom.2006.08.001
  50. 10.1109/ICASSP.2004.1325911 · doi:10.1109/icassp.2004.1325911
  51. 10.1007/978-3-642-34584-5_24 · doi:10.1007/978-3-642-34584-5_24
  52. 10.1038/110311a0 · doi:10.1038/110311a0
  53. wu, 0, Examplar-based voice conversion using non-negative spectrogram deconvolution, Proc 8th ISCA Speech Synth Workshop
  54. liu, 2016, Coupled generative adversarial networks, Proc Adv Neural Inf Process Syst, 469
  55. zou, 2020, Edge-GAN: Edge conditioned multi-view face image generation, Proc IEEE Int Conf Image Process, 2401
  56. 10.1016/0167-6393(94)00053-D · doi:10.1016/0167-6393(94)00053-d
  57. 10.1109/ICSP.2016.7877819 · doi:10.1109/icsp.2016.7877819
  58. 10.1109/TASL.2011.2165944 · DOI
  59. 10.1109/ICASSP.2005.1415037 · doi:10.1109/icassp.2005.1415037
  60. 10.1250/ast.14.353 · doi:10.1250/ast.14.353
  61. 10.1109/89.661472 · DOI · Paper page
  62. xuan, 0, Em algorithms of Gaussian mixture model and hidden Markov model, Proc Int Conf Image Process, 145
  63. 10.1109/APSIPA.2014.7041540 · doi:10.1109/apsipa.2014.7041540
  64. 10.1109/APSIPA.2017.8282110 · doi:10.1109/apsipa.2017.8282110
  65. watanabe, 0, Transformation of spectral envelope for voice conversion based on radial basis function networks, Proc Int Conf Spoken Lang Process
  66. ramani, 0, Cross-lingual voice conversion-based polyglot speech synthesizer for indian languages, Proc 15th Annu Conf Int Speech Commun Assoc

Cited by

See all 357

Metadata

Authors
Sisman, B., Yamagishi, J., King, S., & Li, H.
Venue
IEEE/ACM Transactions on Audio, Speech, and Language Processing
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Published
2021
Volume
29
Pages
132-157
ISSN
  • 2329-9290 (ISSN-L)
  • 2329-9304
DOI
10.1109/taslp.2020.3038524
OpenAlex
W3098557217
Cited by
357
References
248
FWCI
15.63
Type
journal article
Language
en
Open access
hybrid
Licence
creativecommons.org/licenses/by/4.0/legalcode (vor)
Funders
  • SUTD Start-up Grant Artificial Intelligence for Human Voice Conversion (SRG ISTD 2020 158)
  • SUTD AI
  • The Understanding and Synthesis of Expressive Speech by AI (PIE-SGP-AI-2020-02)
  • National Research Foundation Singapore (AISG-GC-2019-002, AISG-100E-2018-006)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: An Overview of Voice Conversion and Its Challenges: From Statistical Modeling to Deep Learning. https://app.cochat.ai/papers/10.1109/taslp.2020.3038524
The paper
Sisman, B., Yamagishi, J., King, S., & Li, H. (2021). An Overview of Voice Conversion and Its Challenges: From Statistical Modeling to Deep Learning. IEEE/ACM Transactions on Audio, Speech, and Language Processing. https://doi.org/10.1109/taslp.2020.3038524

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.