A Spoofing Benchmark for the 2018 Voice Conversion Challenge: Leveraging from Spoofing Countermeasures for Speech Artifact Assessment
Open accessThe DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.
Abstract
Voice conversion (VC) aims at conversion of speaker characteristic without altering content. Due to training data limitations and modeling imperfections, it is difficult to achieve believable speaker mimicry without introducing processing artifacts; performance assessment of VC, therefore, usually involves both speaker similarity and quality evaluation by a human panel. As a time-consuming, expensive, and non-reproducible process, it hinders rapid prototyping of new VC technology. We address artifact assessment using an alternative, objective approach leveraging from prior work on spoofing countermeasures (CMs) for automatic speaker verification. Therein, CMs are used for rejecting `fake' inputs such as replayed, synthetic or converted speech but their potential for automatic speech artifact assessment remains unknown. This study serves to fill that gap. As a supplement to subjective results for the 2018 Voice Conversion Challenge (VCC'18) data, we configure a standard constant-Q cepstral coefficient CM to quantify the extent of processing artifacts. Equal error rate (EER) of the CM, a confusability index of VC samples with real human speech, serves as our artifact measure. Two clusters of VCC'18 entries are identified: low-quality ones with detectable artifacts (low EERs), and higher quality ones with less artifacts. None of the VCC'18 systems, however, is perfect: all EERs are < 30 % (the `ideal' value would be 50 %). Our preliminary findings suggest potential of CMs outside of their original application, as a supplemental optimization and benchmarking tool to enhance VC technology.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Spoofing and countermeasures for speaker verification: A survey
Wu, Z., Evans, N., Kinnunen, T., Yamagishi, J., Alegre, F., & Li, H. (2015) · Speech Communication · doi:10.1016/j.specom.2014.10.005 · Cited by 601
Self-citationA comparison of features for synthetic speech detection
Sahidullah, M., Kinnunen, T., & Hanilçi, C. (2015) · Interspeech 2015 · doi:10.21437/interspeech.2015-472 · Cited by 382
Self-citationThe Voice Conversion Challenge 2018: Promoting Development of Parallel and Nonparallel Methods
Lorenzo-Trueba, J., Yamagishi, J., Toda, T., Saito, D., Villavicencio, F., Kinnunen, T., & Ling, Z. (2018) · The Speaker and Language Recognition Workshop (Odyssey 2018) · doi:10.21437/odyssey.2018-28 · Cited by 305
Self-citationAudio replay attack detection with deep learning frameworks
Lavrentyeva, G., Novoselov, S., Malykh, E., Kozlov, A., Kudashev, O., & Shchemelinin, V. (2017) · Interspeech 2017 · doi:10.21437/interspeech.2017-360 · Cited by 292
The Voice Conversion Challenge 2016
Toda, T., Chen, L.-H., Saito, D., Villavicencio, F., Wester, M., Wu, Z., & Yamagishi, J. (2016) · Interspeech 2016 · doi:10.21437/interspeech.2016-1066 · Cited by 156
Self-citationImpact of Bandwidth and Channel Variation on Presentation Attack Detection for Speaker Verification
Delgado, H., Todisco, M., Evans, N., Sahidullah, M., Liu, W. M., Alegre, F., Kinnunen, T., & Fauve, B. (2017) · 2017 International Conference of the Biometrics Special Interest Group (BIOSIG) · doi:10.23919/biosig.2017.8053510 · Cited by 8
Self-citation
Cited by
ASVspoof 2019: A large-scale public database of synthesized, converted and replayed speech
Wang, X., Yamagishi, J., Todisco, M., Delgado, H., Nautsch, A., Evans, N., Sahidullah, M., Vestman, V., Kinnunen, T., Lee, K. A., et al. (2020) · Computer Speech & Language · doi:10.1016/j.csl.2020.101114 · Cited by 505
Self-citationGeneralization of Audio Deepfake Detection
Chen, T., Kumar, A., Nagarsheth, P., Sivaraman, G., & Khoury, E. (2020) · The Speaker and Language Recognition Workshop (Odyssey 2020) · doi:10.21437/odyssey.2020-19 · Cited by 200
Synthetic speech detection through short-term and long-term prediction traces
Borrelli, C., Bestagini, P., Antonacci, F., Sarti, A., & Tubaro, S. (2021) · EURASIP Journal on Information Security · doi:10.1186/s13635-021-00116-3 · Cited by 80
Speech is Silver, Silence is Golden: What do ASVspoof-trained Models Really Learn?
Müller, N., Dieckmann, F., Czempin, P., Canals, R., Böttinger, K., & Williams, J. (2021) · 2021 Edition of the Automatic Speaker Verification and Spoofing Countermeasures Challenge · doi:10.21437/asvspoof.2021-9 · Cited by 69
Pindrop Labs' Submission to the ASVspoof 2021 Challenge
Chen, T., Khoury, E., Phatak, K., & Sivaraman, G. (2021) · 2021 Edition of the Automatic Speaker Verification and Spoofing Countermeasures Challenge · doi:10.21437/asvspoof.2021-14 · Cited by 30
Metadata
- Authors
- Kinnunen, T., Lorenzo-Trueba, J., Yamagishi, J., Toda, T., Saito, D., Villavicencio, F., & Ling, Z.
- Venue
- The Speaker and Language Recognition Workshop (Odyssey 2018)
- Publisher
- ISCA
- Published
- 2018-06-26
- Pages
- 187-194
- OpenAlex
- W2963127219
- Cited by
- 23
- References
- 6
- FWCI
- 1.66
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: A Spoofing Benchmark for the 2018 Voice Conversion Challenge: Leveraging from Spoofing Countermeasures for Speech Artifact Assessment. https://app.cochat.ai/papers/10.21437/odyssey.2018-27
- The paper
- Kinnunen, T., Lorenzo-Trueba, J., Yamagishi, J., Toda, T., Saito, D., Villavicencio, F., & Ling, Z. (2018). A Spoofing Benchmark for the 2018 Voice Conversion Challenge: Leveraging from Spoofing Countermeasures for Speech Artifact Assessment. The Speaker and Language Recognition Workshop (Odyssey 2018). https://doi.org/10.21437/odyssey.2018-27
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.