Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 41 of the 41 references it cites were found in a registry or an index; none of the 4 sources its claims cite is checked for support yet.

Benchmarking Audio Deepfake Detection Robustness in Real-World Communication Scenarios

Open access
Shi, Haohan, Shi, Xiyu, Dogan, Safak, Alzubi, Saif, Huang, Tianjin, Zhang, Yunxiao · 2025 · 2025 33rd European Signal Processing Conference (EUSIPCO)

Abstract

Existing Audio Deepfake Detection (ADD) systems often struggle to generalise effectively due to the significantly degraded audio quality caused by audio codec compression and channel transmission effects in real-world communication scenarios. To address this challenge, we developed a rigorous benchmark to evaluate the performance of the ADD system under such scenarios. We introduced ADD-C, a new test dataset to evaluate the robustness of ADD systems under diverse communication conditions, including different combinations of audio codecs for compression and packet loss rates. Benchmarking three baseline ADD models on the ADD-C dataset demonstrated a significant decline in robustness under such conditions. A novel Data Augmentation (DA) strategy was proposed to improve the robustness of ADD systems. Experimental results demonstrated that the proposed approach significantly enhances the performance of ADD systems on the proposed ADD-C dataset. Our benchmark can assist future efforts towards building practical and robustly generalisable ADD systems.

Overview

CoChat Summary

TL;DR

The paper introduces ADD-C, a benchmark and test dataset for evaluating audio deepfake detection (ADD) robustness under simulated real-world communication conditions involving codec compression and packet loss. It shows that baseline ADD models degrade significantly under these conditions, and proposes a data augmentation strategy that improves robustness.

Key findings

  • Benchmarking three baseline ADD models (GMM, LCNN, AASIST) on ADD-C showed an average degradation from clean condition C0 to communication condition C1 of 5.30% in EER, 3.16% in AUC, and 3.34% in F1-score.
  • Proposed models trained on the Original dataset also degraded from C0 to C1, with average drops of 2.33% (EER), 0.49% (AUC), and 1.40% (F1-score).
  • Proposed models trained on the Augmented dataset remained stable across conditions: EER was unchanged with a slight improvement of 0.003% from C0 to C1, while AUC and F1-score showed a negligible decrease of 0.1%.
  • The ADD-C test dataset contains six conditions: one clean condition and five communication conditions combining six speech codecs with five packet loss rates (0%, 1%, 5%, 10%, 20%).

Methods

  • ADD-C was built from six public speech datasets: Fake-or-Real (FoR), Wavefake, LJSpeech, MLAAD, M-AILABS, and ASVspoof2021 Logical Access (ASV), totaling 130,041 real and 240,373 fake utterances across 36 deepfake algorithms.
  • Real-world communication was simulated using six codecs (AMR-WB, EVS, IVAS, OPUS, Speex WB, SILK) and five packet loss rates; ADD-C has 4,000 utterances in C0 and 24,000 utterances in each of C1–C5.
  • The proposed data augmentation partitioned the Original dataset into six equal subsets, applied codec compression and packet loss simulation, and merged the five PLR-generated datasets into an Augmented dataset of 1,832,070 utterances (640,205 real, 1,191,865 fake). Models used 4-second mono audio, 80%:20% train/validation split, five epochs, batch size 256, Adam, early stopping with patience 3, and cross-entropy loss.

Limitations

  • The paper does not state an explicit limitations section.
  • Evident from the text: evaluation is based on simulated codec compression and packet loss rather than live real-world communication deployments.
  • Evident from the text: the selected datasets are all English-language, and the benchmark uses a fixed set of six codecs, five PLRs, and three baseline models, which may limit generalization to other languages, codecs, or network conditions.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.23919/eusipco63237.2025.11226601

Cites

41 of 41 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

0 Valid · 0 Not valid · 4 Not checked · 24 Own results and interpretations

9 more claims

Create a free account to see all 28 claims and every source verdict.

See all claims — free account
  • The quote only partly states this claim. 1
  • It may state two separate claims as one. 7
  • Probably not a substantial claim. 8

Create a free account to see these claims.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

1 of 1

Citations

Cites

Show all 33 references

Research works printed but not linked: 8

  1. Brewster, 2021, Fraudsters cloned company director’s voice in $ 35 million heist, police find, Forbes
  2. 10.1145/3658664.3659647 · DOI
  3. Blue, Who are you(i really wanna know)? detecting audio deepfakes through vocal tract reconstruction, 2691
  4. Molisch, 2012, Wireless communications, 34
  5. 2024, Lte; 5g; codec for immersive voice and audio services - detailed algorithmic description incl. rtp payload format and sdp parameter definitions
  6. Astrom, 2009, Rtp payload format and file storage format for silk speech and audio codec
  7. Ito, 2017, The lj speech dataset
  8. 2024, The m-ailabs speech dataset

Cited by

See all 8

Metadata

Authors
Shi, H., Shi, X., Dogan, S., Alzubi, S., Huang, T., & Zhang, Y.
Affiliations
  • Loughborough University (GB)
  • University of Exeter (GB)
Venue
2025 33rd European Signal Processing Conference (EUSIPCO)
Publisher
IEEE
Published
2025-09-08
Pages
566-570
OpenAlex
W4417052665
Cited by
8
References
33
Citations by year
2026: 7 · 2025: 1
FWCI
7.63
Type
conference paper
Language
en
Open access
Locations
  • European Signal Processing Conference (EUSIPCO) (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • Figshare (submittedVersion, cc-by)
  • arXiv (Cornell University)
Funders
  • Loughborough University (GS1016)
  • China Scholarship Council (202208060237)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
  • Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
Keywords
audio deepfake detection, benchmarking, packet loss, data augmentation, robustness, generalization, audio quality degradation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-10-12, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-12 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Benchmarking Audio Deepfake Detection Robustness in Real-World Communication Scenarios. https://app.cochat.ai/papers/10.23919/eusipco63237.2025.11226601
The paper
Shi, H., Shi, X., Dogan, S., Alzubi, S., Huang, T., & Zhang, Y. (2025). Benchmarking Audio Deepfake Detection Robustness in Real-World Communication Scenarios. 2025 33rd European Signal Processing Conference (EUSIPCO). https://doi.org/10.23919/eusipco63237.2025.11226601

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.