The DOI matches its Crossref record; 55 of the 66 references it cites were found in a registry or an index, and 3 are not checked yet; 18 of the 41 sources its claims cite were checked for support.
ASVspoof 2021: Towards Spoofed and Deepfake Speech Detection in the Wild
Open accessAbstract
Benchmarking initiatives support the meaningful comparison of competing solutions to prominent problems in speech and language processing. Successive benchmarking evaluations typically reflect a progressive evolution from ideal lab conditions towards to those encountered in the wild. ASVspoof, the spoofing and deepfake detection initiative and challenge series, has followed the same trend. This article provides a summary of the ASVspoof 2021 challenge and the results of 54 participating teams that submitted to the evaluation phase. For the logical access (LA) task, results indicate that countermeasures are robust to newly introduced encoding and transmission effects. Results for the physical access (PA) task indicate the potential to detect replay attacks in real, as opposed to simulated physical spaces, but a lack of robustness to variations between simulated and real acoustic environments. The Deepfake (DF) task, new to the 2021 edition, targets solutions to the detection of manipulated, compressed speech data posted online. While detection solutions offer some resilience to compression effects, they lack generalization across different source datasets. In addition to a summary of the top-performing systems for each task, new analyses of influential data factors and results for hidden data subsets, the article includes a review of post-challenge results, an outline of the principal challenge limitations and a road-map for the future of ASVspoof.
Verified 2026-10-05 · 3 references not checked yet
Overview
CoChat Summary
## TL;DR
ASVspoof 2021 benchmarked spoofing and deepfake speech detection under more realistic “in the wild” conditions across three tasks: logical access, physical access, and deepfake. The main takeaway is that logical-access detection remained fairly robust under transmission distortions, while physical-access replay detection was hardest and deepfake detection suffered from poor generalization across source datasets.
## Key findings
- The challenge drew submissions from 54 teams; 34 teams submitted the required system description.
- Logical access (LA): many systems outperformed the best baseline B03; the best system T23 achieved 0.1815 min t-DCF / 0.89% EER on the progress set and 0.2177 min t-DCF / 1.32% EER on the evaluation set.
- Physical access (PA): all systems were substantially worse than the ASV floor of 0.12; the best system T07 had 0.6736 min t-DCF / 23.60% EER on progress and 0.6824 / 24.25% on evaluation.
- Deepfake (DF): performance dropped sharply from progress to evaluation; the best system T23 went from 0.24% EER on progress to 15.64% EER on evaluation, and 23 of 33 systems had progress-set EERs below 10%, while all evaluation-set EERs exceeded 15%.
- Hidden-subset analyses showed that removing non-speech harmed performance: for example, in LA the baseline B04 rose from 0.426 min t-DCF with non-speech to 0.908 when non-speech at the two ends was removed, and to 0.961 on the hidden track without any non-speech.
## Methods
- The paper reports the ASVspoof 2021 challenge as three independent sub-challenges: LA (spoofed speech over telephony), PA (replay attacks in real physical spaces), and DF (deepfake detection in compressed audio).
- Data were distributed in FLAC, 16 kHz, 16-bit format; participants trained on ASVspoof 2019 partitions, with task-specific evaluation databases for 2021.
- Main evaluation metrics were min t-DCF for LA and PA, and EER for DF; the paper analyzes challenge results, hidden subsets, influential data factors, and post-challenge baseline experiments on features, backends, losses, and augmentation.
## Limitations
- The paper explicitly notes that ASVspoof 2021 is not without limitations and points to future work on more realistic audio conditions, including background noise for LA, human-talker recordings instead of loudspeakers for PA, and real social-media data for DF.
- It also states that non-speech intervals can be a cue, which is a limitation because they may reflect database artifacts rather than robust spoofing cues.
- Evident from the text: the challenge uses fixed training/data policies and largely English read speech from VCTK, which the paper itself links to limited generalization and over-fitting across source domains.
Model-written summary of a shortened version of the paper’s full text (it is longer than we read). Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
15 Valid · 5 Not valid · 8 Not checked · 129 Own results and interpretations
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“Furthermore, the past ASVspoof training, development and evaluation sets all originate from the same source corpus: either VCTK [10] or RedDots [11].”
What the sources say
- Not checked — no full text of the sourcecites [10] [10] C. Veaux, J. Yamagishi, and K. MacDonald, 'CSTR VCTK corpus: English multi-speaker corpus for CSTR voice cloning toolkit,' 2017, http://dx.doi.org/10.7488/ds/1994. - source
- Supportedcites [11] [11] K. A. Lee, A. Larcher, G. Wang, et al. , 'The reddots data collection for speaker recognition,' in Proc. Interspeech , 2015, pp. 2996-3000. - source
Citation [11] identifies RedDots as one of the source corpora named in the sentence. The paper describes the RedDots speech corpus in its database-development section (p. 2), so it supports that identification. It does not independently document which ASVspoof sets used RedDots. (LLM, conf=0.88)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“The full DF evaluation database is generated using in excess of 100 different spoofing attack algorithms generated by the large number of teams or individuals who contributed to the ASVspoof 2019 database [18], or by VCC participants.”
What the sources say
- Could not verifycites [18] [18] X. Wang, J. Yamagishi, M. Todisco, et al. , 'ASVspoof 2019: A largescale public database of synthesized, converted and replayed speech,' Computer Speech & Language , vol. 64, p. 101 114, Nov. 2020. - sourceExperimental — accuracy not yet measured
Section 2.2.2 (p. 5) describes an ASVspoof 2019 logical-access evaluation partition with 2 known and 11 unknown attacks. Our copy does not describe the DF evaluation database or establish that it uses more than 100 algorithms; searches for that number and for the claimed contributors found no supporting passage. (LLM, conf=0.76)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“B IOMETRIC systems implemented with automatic speaker verification (ASV) technology [1] are vulnerable to spoofing attacks [2] whereby an adversary attempts to masquerade as another individual through the presentation of manipulated speech data [3].”
What the sources say
- Supportedcites [1] [1] Z. Bai and X.-L. Zhang, 'Speaker recognition based on deep learning: An overview,' Neural Networks , vol. 140, pp. 65-99, 2021. - source
The citation to this paper is attached to “automatic speaker verification (ASV) technology,” not to the subsequent claims about spoofing, which have separate citations. The Introduction describes automatic speaker recognition and its use in voice-based authentication, while Section 2 discusses speaker verification systems. (LLM, conf=0.90)
- Not matchedcites [2] [2] 'ISO/IEC 30107. Information technology -biometric presentation attack detection,' Standard, 2016. -
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites [3] [3] Z. Wu, N. Evans, T. Kinnunen, et al. , 'Spoofing and countermeasures for speaker verification: A survey,' speech communication , vol. 66, pp. 130-153, 2015. -
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“These observations show that A18, an attack based on a non-parallel VC system (details in Section 3.1, [18]) which is already difficult to detect even in the absence of encoding, becomes disproportionately more difficult to detect after low bandwidth encoding.”
What the sources say
- Supportedcites [18] [18] X. Wang, J. Yamagishi, M. Todisco, et al. , 'ASVspoof 2019: A largescale public database of synthesized, converted and replayed speech,' Computer Speech & Language , vol. 64, p. 101 114, Nov. 2020. - source
The citation is attached to the description of A18, not to the citing authors’ observation about low-bandwidth encoding. Section 3.1 (p. 9) identifies A18 as a non-parallel voice-conversion system; the paper is not being cited for the encoding comparison. (LLM, conf=0.91)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“The summary of the results indicate the fragility of synthesis detection model against lower quality and unseen fake audio, especially when partially-spoofed audio is presented.”
What the sources say
- Not checked — no full text of the sourcecites [50] [50] J. Yi, R. Fu, J. Tao, et al. , 'ADD 2022: The first audio deep synthesis detection challenge,' in Proc. ICASSP , 2022, pp. 9216-9220. - source
- Partly supportedcites [51] [51] L. Zhang, X. Wang, E. Cooper, et al. , 'The partialspoof database and countermeasures for the detection of short generated audio segments embedded in a speech utterance,' ArXiv , vol. abs/2204.05177, 2022. - sourceExperimental — accuracy not yet measured
The paper says existing countermeasures need adaptation for partially spoofed speech (p. 1), and its Construction Policies (p. 3) anticipate spoofing methods unseen in training. Its ADD Challenge section (p. 6) identifies a partially spoofed detection track, but does not summarize results showing fragility against lower-quality or unseen fake audio. (LLM, conf=0.84)
hedgedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
77 more claims · 4 with their cited sources checked
Create a free account to see all 157 claims and every source verdict.
- It may state two separate claims as one. 42
- Probably not a substantial claim. 41
- The quote only partly states this claim. 2
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“ASVspoof 2021 [74,75] built upon the ASVspoof 2019 framework by introducing, for the first time, a Speech Deepfake (DF) detection task aimed at identifying spoofed speech outside ASV scenarios.”
From Audio Deepfake Detection: What Has Been Achieved and What Lies Ahead (2025) · cites this paper as [75]
The cited source says what the claim attributes to it.
In the citing paper
“Until this point manipulations are mainly applied only to the visual modality, and later, audio manipulations [39] and audio-visual manipulations [31] have been proposed to increase the complexity of the task.”
From AV-Deepfake1M: A Large-Scale LLM-Driven Audio-Visual Deepfake Dataset (2024) · cites this paper as [39]
The cited source says what the claim attributes to it.
In the citing paper
“While detection systems offer some resilience to compression effects, they lack generalization across different source datasets2.”
From Deepfake forensics: a survey of digital forensic methods for multimodal deepfake identification on social media (2024) · cites this paper as Liu et al., 2023
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Deepfake technology is advancing at an unprecedented rate, with new methods and techniques emerging regularly.”
From Deepfake forensics: a survey of digital forensic methods for multimodal deepfake identification on social media (2024) · cites this paper as Liu et al., 2023
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“For instance, the development of AI-based techniques for synthesizing human images, known as Deep Fakes, has made it possible to create highly realistic fake videos and images that leave few traces of manipulation (Liu et al., 2023).”
From Deepfake forensics: a survey of digital forensic methods for multimodal deepfake identification on social media (2024) · cites this paper as Liu et al., 2023
Experimental — accuracy not yet measured
Citations
Cites
Squeeze-and-Excitation Networks
Hu, J., Shen, L., Albanie, S., Sun, G., & Wu, E. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2913372 · Cited by 12,157
SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition
Park, D. S., Chan, W., Zhang, Y., Chiu, C.-C., Zoph, B., Cubuk, E. D., & Le, Q. V. (2019) · Interspeech 2019 · doi:10.21437/interspeech.2019-2680 · Cited by 3,558
An overview of text-independent speaker recognition: From features to supervectors
Kinnunen, T., & Li, H. (2010) · Speech Communication · doi:10.1016/j.specom.2009.08.009 · Cited by 1,434 · cited 1×
“An energy-based voice activity detector (VAD) with the recommended threshold settings [38, Section 5.1] is then used to remove segments labeled as non-speech in addition to any speech segments of less than 50 ms duration.”
MethodInfluentialSelf-citationSpoofing and countermeasures for speaker verification: A survey
Wu, Z., Evans, N., Kinnunen, T., Yamagishi, J., Alegre, F., & Li, H. (2015) · Speech Communication · doi:10.1016/j.specom.2014.10.005 · Cited by 601 · cited 2×
“have hence received greater attention [3].”
BackgroundSelf-citationASVspoof 2019: A large-scale public database of synthesized, converted and replayed speech
Wang, X., Yamagishi, J., Todisco, M., Delgado, H., Nautsch, A., Evans, N., Sahidullah, M., Vestman, V., Kinnunen, T., Lee, K. A., et al. (2020) · Computer Speech & Language · doi:10.1016/j.csl.2020.101114 · Cited by 505 · cited 4×
“1, [18]) which is already difficult to detect even in the absence of encoding, becomes disproportionately more difficult to detect after low bandwidth encoding.”
BackgroundMethodInfluentialSelf-citationASVspoof 2015: the first automatic speaker verification spoofing and countermeasures challenge
Wu, Z., Kinnunen, T., Evans, N., Yamagishi, J., Hanilçi, C., Sahidullah, M., & Sizov, A. (2015) · Interspeech 2015 · doi:10.21437/interspeech.2015-462 · Cited by 484 · cited 2×
“The first edition of ASVspoof in 2015 [8] focused on the development of CMs for the detection of VC and TTS attacks.”
BackgroundSelf-citationEnd-to-End anti-spoofing with RawNet2
Tak, H., Patino, J., Todisco, M., Nautsch, A., Evans, N., & Larcher, A. (2021) · ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp39728.2021.9414234 · Cited by 402 · cited 1×
“Originating from [39] and based on RawNet [40] and graph attention networks, AASIST [41] is representative of progress in the use of E2E architectures.”
MethodSelf-citationAASIST: Audio Anti-Spoofing Using Integrated Spectro-Temporal Graph Attention Networks
Jung, J.-W., Heo, H.-S., Tak, H., Shim, H.-J., Chung, J. S., Lee, B.-J., Yu, H.-J., & Evans, N. (2022) · ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp43922.2022.9747766 · Cited by 399
Self-citationSpeaker recognition based on deep learning: An overview
Bai, Z., & Zhang, X.-L. (2021) · Neural Networks · doi:10.1016/j.neunet.2021.03.004 · Cited by 386 · cited 1×
“B IOMETRIC systems implemented with automatic speaker verification (ASV) technology [1] are vulnerable to spoofing attacks [2] whereby an adversary attempts to masquerade as another individual through the presentation of manipulated speech data [3].”
BackgroundASVspoof 2021: accelerating progress in spoofed and deepfake speech detection
Yamagishi, J., Wang, X., Todisco, M., Sahidullah, M., Patino, J., Nautsch, A., Liu, X., Lee, K. A., Kinnunen, T., Evans, N., et al. (2021) · 2021 Edition of the Automatic Speaker Verification and Spoofing Countermeasures Challenge · doi:10.21437/asvspoof.2021-8 · Cited by 380 · cited 3×
“TABLE V EMPIRICAL DESCRIPTION OF TOP-5 SUBMISSIONS FOR EACH TASK, AND THE BASELINE SYSTEMS (BOTTOM) [14]”
BackgroundMethodInfluentialSelf-citation
Cited by
Deepfake: definitions, performance metrics and standards, datasets, and a meta-review
Altuncu, E., Franqueira, V. N. L., & Li, S. (2024) · Frontiers in Big Data · doi:10.3389/fdata.2024.1400024 · Cited by 134
A Survey on Speech Deepfake Detection
Li, M., Ahmadiadli, Y., & Zhang, X.-P. (2025) · ACM Computing Surveys · doi:10.1145/3714458 · Cited by 101
ASVspoof 5: crowdsourced speech data, deepfakes, and adversarial attacks at scale
Wang, X., Delgado, H., Tak, H., Jung, J.-W., Shim, H.-J., Todisco, M., Kukanov, I., Liu, X., Sahidullah, M., Kinnunen, T. H., et al. (2024) · The Automatic Speaker Verification Spoofing Countermeasures Workshop (ASVspoof 2024) · doi:10.21437/asvspoof.2024-1 · Cited by 101
Self-citationQureshi, S. M., Saeed, A., Almotiri, S. H., Ahmad, F., & Al Ghamdi, M. A. (2024) · PeerJ Computer Science · doi:10.7717/peerj-cs.2037 · Cited by 71
Audio Deepfake Detection: What Has Been Achieved and What Lies Ahead
Zhang, B., Cui, H., Nguyen, V., & Whitty, M. (2025) · Sensors · doi:10.3390/s25071989 · Cited by 72
Metadata
- Authors
- Liu, X., Wang, X., Sahidullah, M., Patino, J., Delgado, H., Kinnunen, T., Todisco, M., Yamagishi, J., Evans, N., Nautsch, A., & Lee, K. A.
- Affiliations
- Centre National de la Recherche Scientifique (FR)
- National Institute of Informatics (JP)
- EURECOM (FR)
- Nuance Communications, Madrid, Spain
- University of Eastern Finland (FI)
- Agency for Science, Technology and Research (SG)
- ORCID
- Venue
- IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Published
- 2023
- Volume
- 31
- Pages
- 2507-2522
- ISSN
- 2329-9290 (ISSN-L)
- 2329-9304
- OpenAlex
- W4381198892
- Semantic Scholar
- a3dc949e83746a6c64969d535a5d8f968ded83f7
- Cited by
- 312
- References
- 58
- Citations by year
- 2026: 93 · 2025: 118 · 2024: 71 · 2023: 28
- FWCI
- 29.14
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE/ACM Transactions on Audio Speech and Language Processing (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- HAL (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Funders
- Agence Nationale de la Recherche (ANR-18-JSTS-0001)
- Japan Science and Technology Agency through Project VoicePersonae (JPMJCR18A6)
- Academy of Finland (349605)
- SPEECHFAKES
- MEXT KAKENHI (21K17775, 21H04906)
- Region Grand Est, France, joint doctoral funding from Inria and UEF
- Science and Engineering Research Council, Agency for Science, Technology and Research (A*STAR), Singapore
- Council Research Fund (CR-2021-005)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Physical Sciences › Computer Science › Signal Processing › Biometric Identification and Security
- Keywords
- anti-spoofing, speech deepfake detection, replay attacks, speaker verification security
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-09-29, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-09-29 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: ASVspoof 2021: Towards Spoofed and Deepfake Speech Detection in the Wild. https://app.cochat.ai/papers/10.1109/taslp.2023.3285283
- The paper
- Liu, X., Wang, X., Sahidullah, M., Patino, J., Delgado, H., Kinnunen, T., Todisco, M., Yamagishi, J., Evans, N., Nautsch, A., & Lee, K. A. (2023). ASVspoof 2021: Towards Spoofed and Deepfake Speech Detection in the Wild. IEEE/ACM Transactions on Audio, Speech, and Language Processing. https://doi.org/10.1109/taslp.2023.3285283
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.