Attack Agnostic Dataset: Towards Generalization and Stabilization of Audio DeepFake Detection
Open accessThe DOI matches its Crossref record; 30 of the 30 references it cites were found in a registry or an index; 7 of the 8 sources its claims cite were checked for support.
Abstract
Audio DeepFakes allow the creation of high-quality, convincing utterances and therefore pose a threat due to its potential applications such as impersonation or fake news. Methods for detecting these manipulations should be characterized by good generalization and stability leading to robustness against attacks conducted with techniques that are not explicitly included in the training. In this work, we introduce Attack Agnostic Dataset - a combination of two audio DeepFakes and one anti-spoofing datasets that, thanks to the disjoint use of attacks, can lead to better generalization of detection methods. We present a thorough analysis of current DeepFake detection methods and consider different audio features (front-ends). In addition, we propose a model based on LCNN with LFCC and mel-spectrogram front-end, which not only is characterized by a good generalization and stability results but also shows improvement over LFCC-based mode - we decrease standard deviation on all folds and EER in two folds by up to 5%.
Overview
CoChat Summary
## TL;DR
The paper proposes an “Attack Agnostic Dataset” that combines three audio DeepFake/spoofing datasets with disjoint attack splits to study and improve generalization and stability in audio DeepFake detection. Across benchmarks, LCNN with LFCC plus a mel-spectrogram front-end gave the best overall generalization/stability, with standard deviation below 0.64% on all folds and EER reduced by up to nearly 5% versus the LFCC baseline.
## Key findings
- The proposed Attack Agnostic Dataset combines 31,083 real and 222,035 fake samples covering 27 methods.
- In the main benchmark, LCNN achieved the best generalization, with EERs of 9.525%, 9.523%, and 2.370% across the three folds, and the paper says it also had the lowest standard deviation, indicating the best stability.
- Compared with the LFCC baseline, LCNN + LFCC + spectrogram improved EER in 2 out of 3 folds and had std values of 0.321%, 0.639%, and 0.335%.
- The conclusion states the proposed LCNN with LFCC and mel-spectrogram achieved a standard deviation that does not exceed 0.64% on all folds, and reduced baseline EER by nearly 5% by enhancing the input with a spectrogram.
## Methods
- The paper introduces Attack Agnostic Dataset, combining FakeAVCeleb, WaveFake, and ASVspoof 2019 LA, with attacks disjointly divided across train/test/eval subsets in three different folds to probe generalization and stability.
- Models evaluated include LCNN, MesoInception-4, XceptionNet, RawNet2, and GMM; training used Adam, binary cross-entropy, batch size 128, 5 epochs, and 3 fixed randomness seeds.
- Inputs/front-ends compared include LFCC, MFCC, spectrogram, and their concatenations; the audio was resampled to 16 kHz mono, silences longer than 0.2 s were trimmed, and samples were padded or trimmed to 4 s.
## Limitations
- The paper states that ASVspoof 2021 DF subset training information and LA attack types were not available at the time, so that dataset was not used; this limits coverage of the newest attacks.
- The text makes evident that results can vary substantially by fold and attack split: it reports large fluctuations in test accuracy and says training stability is generally low; this is a limitation of the task and evaluation setup as discussed by the authors.
- The paper also notes that some datasets contain only one or a few generation methods (for example, FakeAVCeleb uses one method), which can make fold results less informative about broader attack diversity.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
3 Valid · 1 Not valid · 0 Not checked · 41 Own results and interpretations
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“For FakeAVCeleb dataset, there is only one method used, thus the EER results are very similar for all folds.”
What the sources say
- Disputedcites [14] [14] H. Khalid, S. Tariq, M. Kim, and S. S. Woo, 'FakeAVCeleb: A novel audio-video multimodal deepfake dataset,' in Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2) , 2021. - sourceExperimental — accuracy not yet measured
Section 3.1 (p. 7) explicitly says FakeAVCeleb uses four deepfake-generation methods: FaceSwap, FSGAN, Wav2Lip, and SV2TTS. SV2TTS is the voice-cloning method, but it is not the dataset’s only method. The cited paper does not establish the proposed explanation for similar EER results across folds. (LLM, conf=0.90)
Checked by a reasoning model — may be wrong.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Neural networks (as well as other machine learning models) applied to the DeepFake detection problem are particularly prone to two common issues - generalization and stability.”
What the sources say
- Partly supportedcites [15] [15] X. Wang, J. Yamagishi, M. Todisco, H. Delgado, A. Nautsch, N. Evans, M. Sahidullah, V. Vestman, T. Kinnunen, K. A. Lee, L. Juvela, P. Alku, Y.-H. Peng, H.-T. Hwang, Y. Tsao, H.M. Wang, S. L. Maguer, M. Becker, F. Henderson, R. Clark, Y. Zhang, Q. Wang, Y. Jia, K. Onuma, K. Mushika, T. Kaneda, Y. Jiang, L.-J. Liu, Y.-C. Wu, W.-C. Huang, T. Toda, K. Tanaka, H. Kameoka, I. Steiner, D. Matrouf, J.-F. Bonastre, A. Govender, S. Ronanki, J.-X. Zhang, and Z.-H. Ling, 'Asvspoof 2019: A large-scale public database of synthesized, converted and replayed speech,' Computer Speech & Language , vol. 64, sourceExperimental — accuracy not yet measured
Section 6.1.2 (p. 17) reports substantially poorer spoofing-countermeasure performance on evaluation attacks not included in training, supporting a generalization concern in speech-fake detection. But the countermeasures examined there are GMM-based, not neural-network detectors, and the paper does not establish that such detectors are particularly prone to stability issues. (LLM, conf=0.84)
- Partly supportedcites [16] [16] H. Delgado, N. W. D. Evans, T. H. Kinnunen, K.-A. Lee, X. Liu, A. Nautsch, J. Patino, M. Sahidullah, M. Todisco, X. Wang, and J. Yamagishi, 'Asvspoof 2021: Automatic speaker verification spoofing and countermeasures challenge evaluation plan,' ArXiv , vol. abs/2109.00535, 2021. - sourceExperimental — accuracy not yet measured
Our copy of the ASVspoof 2021 evaluation plan treats generalisation as a central problem for spoofing/deepfake countermeasures — p. 1 ('technically more challenging data to promote countermeasure generalisation'), p. 2 Technical objectives ('promotion of generalised countermeasures', 'robust to channel variability'), and p. 4 DF task ('detection of speech deepfakes across multiple and unknown conditions', 'new data for the purpose of evaluating robustness to domain mismatch'). However, it never attributes particular proneness to these problems to neural networks (or ML models) as opposed to other countermeasures — the baselines in section 7.2 (p. 6) mix GMM and neural systems without such a comparison — and it makes no claim about 'stability' as a second issue; the closest passage (Appendix, p. 13) only says pooled scoring encourages countermeasures that 'calibrate well' and whose scores are 'mutually compatible across a wide range of conditions'. So the generalisation concern in deepfake/speech-spoofing detection is supported, but the neural-network-specific proneness and the stability part are not. (LLM, conf=0.62)
- Not checked — no full text of the sourcecites [17] [17] R. Białobrzeski, M. Ko´ smider, M. Matuszewski, M. Plata, and A. Rakowski, 'Robust Bayesian and Light Neural Networks for Voice Spoofing Detection,' in Proc. Interspeech 2019 , 2019, pp. 1028-1032. [Online]. Available: http://dx.doi.org/10.21437/ Interspeech.2019-2676 - source
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“It is a combination of two DeepFake datasets - FakeAVCeleb [14] and WaveFake [18], and one dataset originally prepared for spoofing detection ASVspoof 2019 LA subset [19].”
What the sources say
- Supportedcites [14] [14] H. Khalid, S. Tariq, M. Kim, and S. S. Woo, 'FakeAVCeleb: A novel audio-video multimodal deepfake dataset,' in Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2) , 2021. - source
The citation identifies FakeAVCeleb as one of the datasets used in the citing paper's combination. The Introduction (p. 2) identifies the cited work as presenting FakeAVCeleb, an audio-video deepfake dataset; the combination with WaveFake and ASVspoof is the citing paper's own description, not a finding attributed to this source. (LLM, conf=0.93)
- Supportedcites [18] [18] J. Frank and L. Sch¨ onherr, 'WaveFake: A Data Set to Facilitate Audio Deepfake Detection,' in Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track , 2021. - source
The citation [18] identifies WaveFake as one of the datasets used in the citing authors’ combination. The source title (p. 1) identifies the work as the WaveFake dataset, and the Discussion (p. 9) refers to it as “our data set.” The combination itself is the citing authors’ description, not a claim attributed to WaveFake. (LLM, conf=0.91)
- Supportedcites [19] [19] M. Todisco, X. Wang, V. Vestman, M. Sahidullah, H. Delgado, A. Nautsch, J. Yamagishi, N. Evans, T. Kinnunen, and K. A. Lee, 'Asvspoof 2019: Future horizons in spoofed and fake audio detection,' arXiv preprint arXiv:1904.05441 , 2019. - source
The citation is attached only to the identification of the ASVspoof 2019 LA subset, so the judging target is whether our copy is that dataset and whether it was originally prepared for spoofing detection. Section 2 (Database, p. 1) and 2.1 (Logical access, p. 1) describe the LA partition of the ASVspoof 2019 database (VCTK-derived, bona fide plus 17 TTS/VC spoofed conditions split into known/unknown attacks). The Abstract (p. 1) and Introduction (p. 1) frame the whole challenge as developing countermeasures to protect ASV from spoofing, i.e. spoofing/fake audio detection. The other two datasets named in the citing sentence (FakeAVCeleb, WaveFake) and their combination are not attributed to this source and are not judged here. (LLM, conf=0.90)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, treating LFCC as a baseline, each of the other front-ends (apart from spectrogram) provides improvements on at least one fold.”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Spectrogram features present the worst results across all front-ends; however attaching it to other features enhances the results (apart from the fold 3 of LFCC+Spectrogram).”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
23 more claims
Create a free account to see all 45 claims and every source verdict.
- It may state two separate claims as one. 12
- The quote only partly states this claim. 1
- Probably not a substantial claim. 5
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 1×
“Every model was trained using gradient descent based on Adam optimizer [29] and binary cross–entropy loss function with a batch size of 128.”
MethodXception: Deep Learning with Depthwise Separable Convolutions
Chollet, F. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.195 · Cited by 20,380 · cited 5×
“During tests, we used the following solutions: LCNN [10], MesoInception– 4 [27], XceptionNet [28], RawNet2 [12] and GMM [9].”
MethodInfluentialLabaca-Castro, R. (2023) · Machine Learning under Malware Attack · doi:10.1007/978-3-658-40442-0_9 · Cited by 19,272
WaveNet: a generative model for raw audio
Oord, A. V. D., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1609.03499 · Cited by 4,911 · cited 1×
“There has been a vast improvement in the topics utilized by DeepFake creation methods: Text–To–Speech (TTS) [1], vocoders [2, 3, 4, 5], Voice Conversion [6] and Voice Cloning [7].”
MethodNatural TTS Synthesis by Conditioning Wavenet on MEL Spectrogram Predictions
Shen, J., Pang, R., Weiss, R. J., Schuster, M., Jaitly, N., Yang, Z., Chen, Z., Zhang, Y., Wang, Y., Skerrv-Ryan, R., et al. (2018) · 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2018.8461368 · Cited by 2,791 · cited 6×
“Attacks were created using text– to-speech (TTS), voice conversion (VC) and hybrid algorithms (VC with synthetic speech as an input).”
MethodInfluentialMesoNet: a Compact Facial Video Forgery Detection Network
Afchar, D., Nozick, V., Yamagishi, J., & Echizen, I. (2018) · 2018 IEEE International Workshop on Information Forensics and Security (WIFS) · doi:10.1109/wifs.2018.8630761 · Cited by 1,872 · cited 1×
“During tests, we used the following solutions: LCNN [10], MesoInception– 4 [27], XceptionNet [28], RawNet2 [12] and GMM [9].”
Method“Waveglow: a flow-based generative network for speech synthesis,”
Prenger, R., Valle, R., & Catanzaro, B. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8683143 · Cited by 1,004 · cited 3×
“All solutions except WaveGlow are based on Generative Adversarial Networks (GANs) [24].”
MethodYamamoto, R., Song, E., & Kim, J.-M. (2020) · ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp40776.2020.9053795 · Cited by 827 · cited 1×
“It is created basing on two languages (English and Japanese) with the use of 7 spoof generating neural network architectures: MelGAN (with MelGAN Large) [5], ParallelWaveGAN [21], Multi–band MelGAN, (with Full–band MelGAN) [22], HiFi–GAN [23] and WaveGlow [2].”
MethodMelGAN: generative adversarial networks for conditional waveform synthesis
Kumar, K., Kumar, R., de Boissiere, T., Gestin, L., Teoh, W. Z., Sotelo, J., de Brebisson, A., Bengio, Y., & Courville, A. (2019) · arXiv · doi:10.48550/arxiv.1910.06711 · Cited by 722 · cited 4×
“Moreover, some methods are interdependent (e.g. MelGAN, MelGAN Large), thus only 4 of them are unique.”
MethodInfluentialHiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis
Kong, J., Kim, J., & Bae, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.05646 · Cited by 722 · cited 1×
“It is created basing on two languages (English and Japanese) with the use of 7 spoof generating neural network architectures: MelGAN (with MelGAN Large) [5], ParallelWaveGAN [21], Multi–band MelGAN, (with Full–band MelGAN) [22], HiFi–GAN [23] and WaveGlow [2].”
Method
Research works printed but not linked: 1
- [22][22] G. Yang, S. Yang, K. Liu, P. Fang, W. Chen, and L. Xie, 'Multiband melgan: Faster waveform generation for high-quality textto-speech,' in 2021 IEEE Spoken Language Technology Workshop (SLT) . IEEE, 2021, pp. 492-498. -
Cited by
Qureshi, S. M., Saeed, A., Almotiri, S. H., Ahmad, F., & Al Ghamdi, M. A. (2024) · PeerJ Computer Science · doi:10.7717/peerj-cs.2037 · Cited by 71
Domain Generalization via Aggregation and Separation for Audio Deepfake Detection
Xie, Y., Cheng, H., Wang, Y., & Ye, L. (2024) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2023.3324724 · Cited by 59
“In [14], a combination of three deepfake and spoofing databases named the Attack Agnostic Dataset was used to increase the training stability and general diversity of the combined datasets.”
BackgroundMethodMCL: Multimodal Contrastive Learning for Deepfake Detection
Liu, X., Yu, Y., Li, X., & Zhao, Y. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3312738 · Cited by 84
“For DFDC, there are no audio labels, thus the current state-of-the-art audio Deepfake detection method [76] is utilized to label the audio data of DFDC. Notice that, to avoid misclassification, only high confidence data (probability ∈ [ 0 , 0 .”
MethodSpecRNet: Towards Faster and More Accessible Audio DeepFake Detection
Kawa, P., Plata, M., & Syga, P. (2022) · 2022 IEEE International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom) · doi:10.1109/trustcom56396.2022.00111 · Cited by 32
“Thanks to its use of different attacks in various dataset folds, Attack Agnostic Dataset provides a good overview of the potential model’s performance in the face of new DeepFake generation algorithms.”
BackgroundMethodInfluentialSelf-citationAnomaly Detection of Deepfake Audio Based on Real Audio Using Generative Adversarial Network Model
Song, D., Lee, N., Kim, J., & Choi, E. (2024) · IEEE Access · doi:10.1109/access.2024.3506973 · Cited by 28
Metadata
- Authors
- Kawa, P., Plata, M., & Syga, P.
- Affiliations
- Wrocław University of Science and Technology (PL)
- Venue
- Interspeech 2022
- Publisher
- ISCA
- Published
- 2022-09-18
- Pages
- 4023-4027
- ISSN
- 2308-457X (ISSN-L)
- OpenAlex
- W4296069136
- Cited by
- 33
- References
- 26
- Citations by year
- 2026: 9 · 2025: 10 · 2024: 10 · 2023: 3 · 2022: 1
- FWCI
- 3.32
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Interspeech (USB) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Keywords
- audio deepfake detection, generalization, detection stability, anti-spoofing, spoofing attacks, Mel spectrogram
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Attack Agnostic Dataset: Towards Generalization and Stabilization of Audio DeepFake Detection. https://app.cochat.ai/papers/10.21437/interspeech.2022-10078
- The paper
- Kawa, P., Plata, M., & Syga, P. (2022). Attack Agnostic Dataset: Towards Generalization and Stabilization of Audio DeepFake Detection. Interspeech 2022. https://doi.org/10.21437/interspeech.2022-10078
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.