CoChat
Sign Up
PreprintCited by 7Speech Recognition and SynthesisMusic and Audio ProcessingSpeech and Audio Processing

The DOI matches its DataCite record.

Exploring Green AI for Audio Deepfake Detection

Open access
Saha, Subhajit, Sahidullah, Md, Das, Swagatam·2024·arXiv
DOI 10.48550/arxiv.2403.14290arXiv 2403.14290
Open this paper in CoChat — free accountOpen PDF

Abstract

The state-of-the-art audio deepfake detectors leveraging deep neural networks exhibit impressive recognition performance. Nonetheless, this advantage is accompanied by a significant carbon footprint. This is mainly due to the use of high-performance computing with accelerators and high training time. Studies show that average deep NLP model produces around 626k lbs of CO\textsubscript{2} which is equivalent to five times of average US car emission at its lifetime. This is certainly a massive threat to the environment. To tackle this challenge, this study presents a novel framework for audio deepfake detection that can be seamlessly trained using standard CPU resources. Our proposed framework utilizes off-the-shelve self-supervised learning (SSL) based models which are pre-trained and available in public repositories. In contrast to existing methods that fine-tune SSL models and employ additional deep neural networks for downstream tasks, we exploit classical machine learning algorithms such as logistic regression and shallow neural networks using the SSL embeddings extracted using the pre-trained model. Our approach shows competitive results compared to the commonly used high-carbon footprint approaches. In experiments with the ASVspoof 2019 LA dataset, we achieve a 0.90\% equal error rate (EER) with less than 1k trainable model parameters. To encourage further research in this direction and support reproducible results, the Python code will be made publicly accessible following acceptance. Github: https://github.com/sahasubhajit/Speech-Spoofing-

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2403.14290

Cites

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Large SSL models have shown very good performance over the deepfake detection task.

    “As shown above, large SSL models have shown very good performance over the deepfake detection task.”

    From WavLM model ensemble for audio deepfake detection (2024) · cites this paper as [7]

  • Methods based on self-supervised representations provide a lighter and more robust alternative compared to previous generation approaches such as ResNet on LFCC or RawNet.

    “This is also the case for our task of interest, audio deepfake detection, where methods based on self-supervised representations provide a lighter [7] and more robust [8] alternative compared to the previous generation of approaches, such as ResNet on linear frequency cepstral coefficients (LFCC) [14] or RawNet [15].”

    From WavLM model ensemble for audio deepfake detection (2024) · cites this paper as [7]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

No references could be resolved yet.

Research works printed but not linked: 33

  1. Y. LeCun, Y. Bengio, and G. Hinton, "Deep learning," Nature, vol. 521, no. 7553, pp. 436-444, 2015.
  2. S. J. Prince, Understanding Deep Learning. MIT Press, 2023.
  3. E. Strubell, A. Ganesh, and A. Mccallum, "Energy and policy consider- ations for deep learning in NLP," in Proc. ACL, pp. 3645-3650, 2019.
  4. R. Schwartz, J. Dodge, N. A. Smith, and O. Etzioni, "Green AI," Communications of the ACM, vol. 63, no. 12, pp. 54-63, 2020.
  5. E. Strubell, A. Ganesh, and A. McCallum, "Energy and policy consid- erations for modern deep learning research," Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 13693-13696, Apr. 2020.
  6. R. Verdecchia, J. Sallou, and L. Cruz, "A systematic review of Green AI," Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, vol. 13, no. 4, p. e1507, 2023.
  7. M. R. Kamble, H. B. Sailor, H. A. Patil, and H. Li, "Advances in anti-spoofing: from the perspective of ASVspoof challenges," APSIPA Transactions on Signal and Information Processing, vol. 9, p. e2, 2020.
  8. X. Liu et al., "ASVspoof 2021: Towards spoofed and deepfake speech detection in the wild," IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2023.
  9. X. Wang and J. Yamagishi, "A comparative study on recent neural spoofing countermeasures for synthetic speech detection," in Proc. INTERSPEECH, pp. 4259-4263, 2021.
  10. J.-w. Jung et al., "AASIST: Audio anti-spoofing using integrated spectro- temporal graph attention networks," in Proc. ICASSP, 2022.
  11. H. Tak et al., "Automatic speaker verification spoofing and deepfake detection using wav2vec 2.0 and data augmentation," in Proc. ODYSSEY, 2022.
  12. S. Sinha, S. Dey, and G. Saha, "Improving self-supervised learning model for audio spoofing detection with layer-conditioned embedding fusion," Computer Speech & Language, vol. 86, p. 101599, 2024.
  13. T. Kang et al., "Experimental study: Enhancing voice spoofing detection models with wav2vec 2.0," arXiv preprint arXiv:2402.17127, 2024. · arXiv
  14. S. Borzì et al., "Is synthetic voice detection research going into the right direction?," in Proc. CVPR, 2022.
  15. S. wen Yang et al., "SUPERB: Speech processing universal performance benchmark," in Proc. INTERSPEECH, 2021.
  16. A. Mohamed et al., "Self-supervised speech representation learning: A review," IEEE Journal of Selected Topics in Signal Processing, vol. 16, no. 6, pp. 1179-1210, 2022.
  17. A. Baevski et al., "wav2vec 2.0: A framework for self-supervised learning of speech representations," in Advances in Neural Information Processing Systems, 2020.
  18. R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification. John Wiley & Sons, 2006.
  19. G. James, D. Witten, T. Hastie, and R. Tibshirani, Introduction to Statistical Learning: with Applications in R. Springer, 2013.
  20. C. J. Burges, "A tutorial on support vector machines for pattern recogni- tion," Data Mining and Knowledge Discovery, vol. 2, no. 2, pp. 121-167, 1998.
  21. G. I. Webb, Naïve Bayes, pp. 713-714. Boston, MA: Springer US, 2010.
  22. J. R. Quinlan, "Induction of decision trees," Machine learning, vol. 1, pp. 81-106, 1986.
  23. S. Haykin, Neural Networks and Learning Machines. Pearson Education, 3rd ed., 2009.
  24. F. Pedregosa et al., "Scikit-learn: Machine learning in Python," The Journal of Machine Learning Research, vol. 12, pp. 2825-2830, 2011.
  25. X. Wang et al., "ASVspoof 2019: A large-scale public database of synthesized, converted and replayed speech," Computer Speech & Lan- guage, vol. 64, p. 101114, 2020.
  26. V. Panayotov et al., "LibriSpeech: an ASR corpus based on public domain audio books," in Proc. ICASSP, 2015.
  27. H. Tak et al., "End-to-end spectro-temporal graph attention networks for speaker verification anti-spoofing and speech deepfake detection," in Proc. ASVspoof, pp. 1-8, 2021.
  28. Y. Zhang et al., "The effect of silence and dual-band fusion in anti- spoofing system," in Proc. Interspeech, 2021.
  29. G. Hua, A. B. J. Teoh, and H. Zhang, "Towards end-to-end synthetic speech detection," IEEE Signal Processing Letters, vol. 28, pp. 1265- 1269, 2021.
  30. W. Ge et al., "Raw differentiable architecture search for speech deepfake and spoofing detection," in Proc. ASVspoof, pp. 22-28, 2021.
  31. X. Wang and J. Yamagishi, "A comparative study on recent neural spoofing countermeasures for synthetic speech detection," in Proc. INTERSPEECH, 2021.
  32. C. Wang et al., "Fully automated end-to-end fake audio detection," in Proceedings of the 1st International Workshop on Deepfake Detection for Audio Multimedia, p. 27-33, 2022.
  33. X. Wang and J. Yamagishi, "Investigating self-supervised front ends for speech spoofing countermeasures," in Proc. ODYSSEY, pp. 100-106, 2022.

Cited by

Sorted by citations

Metadata

Authors
Saha, S., Sahidullah, M., & Das, S.
Affiliations
TCG Crest (IN)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2024
Pages
186-190
DOI
10.48550/arxiv.2403.14290
arXiv
2403.14290
OpenAlex
W4404609074
Cited by
7
References
0
Citations by year
2026: 3 · 2025: 3 · 2024: 1
FWCI
2.02
Type
conference paper
Language
en
Open access
Licence
Locations
  • European Signal Processing Conference (EUSIPCO) (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
  • Physical Sciences › Computer Science › Signal Processing › Music and Audio Processing
  • Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
Keywords
audio deepfake detection, green AI, self-supervised learning, logistic regression, shallow neural networks, low-resource training, environmental impact of machine learning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review