CoChat

Exploring Self-supervised Embeddings and Synthetic Data Augmentation for Robust Audio Deepfake Detection

Open access
Martín-Doñas, Juan M., Álvarez, Aitor, Rosello, Eros, Gomez, Angel M., Peinado, Antonio M. · 2024 · Interspeech 2024

The DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index; 4 of the 11 sources its claims cite were checked for support.

Abstract

This work explores the performance of large speech self- supervised models as robust audio deepfake detectors. Despite the current trend of fine-tuning the upstream network, in this paper, we revisit the use of pre-trained models as feature extractors to adapt specialized downstream audio deepfake classifiers. The goal is to keep the general knowledge of the audio foundation model to extract discriminative features to feed up a simplified deepfake classifier. In addition, the generalization capabilities of the system are improved by augmenting the training corpora using additional synthetic data from different vocoder algorithms. This strategy is also complemented by various data augmentations covering challenging acoustic conditions. Our proposal is evaluated under different benchmark datasets for audio deepfake and anti-spoofing tasks, showing state-of-the-art performance. Furthermore, we analyze the relevant parts of the downstream classifier to achieve a robust system.

Overview

CoChat Summary

## TL;DR

The paper studies audio deepfake detection using frozen self-supervised Wav2Vec2 XLS-R embeddings plus a lightweight fine-tuned downstream classifier, and it adds synthetic spoofed training data from vocoders to improve robustness. It reports state-of-the-art results across multiple in-domain and out-of-domain benchmarks, with the best overall pooled EER at 5.13%.

## Key findings

  • Using the frozen Wav2Vec2 XLS-R-128 upstream as a feature extractor and training only a small downstream classifier gave strong results; the upstream accounts for nearly 99% of the parameters, while the trainable blocks are small (e.g., Adapter 25 params, Proj 262K, NN 328K).
  • Adding vocoder-generated spoofed data improved generalization: the paper reports the best overall pooled EER of 5.13% for NN-ASP with trimming + vocoded data, compared with 9.90% for NN-ASP without augmentation.
  • On the benchmark comparison, the proposed method beat the cited NII systems overall, achieving 5.13% vs 9.98% EER on the pooled test setting.
  • Table results show that data augmentation and architecture choices matter across datasets; for example, NN-ASP + vocoded data achieved 0.14% EER on ASVspoof 2019 LA eval and 2.82% on WaveFake, while the no-augmentation baseline was 0.22% and 9.33%, respectively.

## Methods

  • The system uses Wav2Vec2 XLS-R-128 (315M parameters, 24 Transformer layers) as a frozen self-supervised upstream and feeds hidden-layer embeddings into a fine-tuned downstream classifier.
  • The downstream includes an adapter that combines SSL layers, a frame-wise block (either Proj or NN), time pooling (SP, ASP, or ACP), and a final scoring layer; the paper evaluates combinations such as NN-ASP, NN-ACP, and Proj-ASP.
  • Training used ASVspoof 2019 LA train/dev plus 10K additional spoof samples from Voc.v4 partition 1 (4 vocoders: HiFiGAN, WaveGlow, Hn-NSF, and Hn-NSF+HiFiGAN), with RawBoost and silence trimming as optional augmentations; optimization used one-class softmax loss, Adam with learning rate 3×10^-4, batch size 8 with gradient accumulation to an effective batch of 64, and evaluation by EER on multiple in- and out-of-domain test sets.

## Limitations

  • The paper explicitly states future work is needed to evaluate ensembled downstream networks and active learning for selecting fine-tuning data, which indicates the current study does not cover those approaches.
  • The text itself suggests a limitation in generalization: improvements from added vocoder data are not evident on ASVspoof 2021 LA, likely because that degradation is mainly due to telephonic channels, which the new vocoders do not address.
  • The paper uses a frozen upstream and only fine-tunes the downstream classifier; while this is a design choice rather than a stated flaw, the text makes evident that adaptation is limited to the downstream model only.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.21437/interspeech.2024-942

Cites

0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

4 Valid · 1 Not valid · 5 Not checked · 25 Own results and interpretations

20 more claims

Create a free account to see all 35 claims and to verify the 1 reference still open.

See all claims — free account
  • It may state two separate claims as one. 5
  • Probably not a substantial claim. 5

Create a free account to see these claims.

Citations

Cites

The publisher hasn’t deposited this paper’s reference list.

Cited by

See all 19

Metadata

Authors
Martín-Doñas, J. M., Álvarez, A., Rosello, E., Gomez, A. M., & Peinado, A. M.
Venue
Interspeech 2024
Publisher
ISCA
Published
2024-09-01
Pages
2085-2089
ISSN
2308-457X (ISSN-L)
OpenAlex
W4402112185
Cited by
18
References
0
Citations by year
2026: 13 · 2025: 5
FWCI
5.29
Type
conference paper
Language
en
Open access
Licence
cc-by-nc-nd
Locations
  • Interspeech (USB) (publishedVersion)
  • Institutional Repository of the University of Granada (University of Granada) (submittedVersion, cc-by-nc-nd)
Funders
  • European Commission (13039/501100011033, 501100011033, 101073928)
  • HORIZON EUROPE Framework Programme (101073928)
  • European Regional Development Fund (13039/501100011033, 501100011033)
  • Agencia Estatal de Investigación (501100011033, 13039, .13039/501100011033, AEI/10)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
  • Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
  • Physical Sciences › Computer Science › Signal Processing › Music and Audio Processing
Keywords
audio deepfake detection, self-supervised speech models, pre-trained feature extraction, synthetic data augmentation, anti-spoofing

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-09-30, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-09-30 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Exploring Self-supervised Embeddings and Synthetic Data Augmentation for Robust Audio Deepfake Detection. https://app.cochat.ai/papers/10.21437/interspeech.2024-942
The paper
Martín-Doñas, J. M., Álvarez, A., Rosello, E., Gomez, A. M., & Peinado, A. M. (2024). Exploring Self-supervised Embeddings and Synthetic Data Augmentation for Robust Audio Deepfake Detection. Interspeech 2024. https://doi.org/10.21437/interspeech.2024-942

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.