Audio Deepfake Detection with Self-Supervised XLS-R and SLS Classifier
The DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.
Abstract
Generative AI technologies, including text-to-speech (TTS) and voice conversion (VC), frequently become indistinguishable from genuine samples, posing challenges for individuals in discerning between real and synthetic content. This indistinguishability undermines trust in media, and the arbitrary cloning of personal voice signals presents significant challenges to privacy and security. In the field of deepfake audio detection, the majority of models achieving higher detection accuracy currently employ self-supervised pre-trained models. However, with the ongoing development of deepfake audio generation algorithms, maintaining high discrimination accuracy against new algorithms grows more challenging. To enhance the sensitivity of deepfake audio features, we propose a deepfake audio detection model that incorporates an SLS (Sensitive Layer Selection) module. Specifically, utilizing the pre-trained XLS-R enables our model to extract diverse audio features from its various layers, each providing distinct discriminative information. Utilizing the SLS classifier, our model captures sensitive contextual information across different layer levels of audio features, effectively employing this information for fake audio detection. Experimental results show that our method achieves state-of-the-art (SOTA) performance on both the ASVspoof 2021 DF and In-the-Wild datasets, with a specific Equal Error Rate (EER) of 1.92% on the ASVspoof 2021 DF dataset and 7.46% on the In-the-Wild dataset. Codes and data can be found at https://github.com/QiShanZhang/SLSforADD.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units
Hsu, W.-N., Bolte, B., Tsai, Y.-H. H., Lakhotia, K., Salakhutdinov, R., & Mohamed, A. (2021) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2021.3122291 · Cited by 2,983
Robust Speech Recognition via Large-Scale Weak Supervision
Radford, A., Kim, J. W., Xu, T., Brockman, G., McLeavey, C., & Sutskever, I. (2022) · arXiv · doi:10.48550/arxiv.2212.04356 · Cited by 1,953
WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing
Chen, S., Wang, C., Chen, Z., Wu, Y., Liu, S., Chen, Z., Li, J., Kanda, N., Yoshioka, T., Xiao, X., et al. (2022) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3188113 · Cited by 1,926
Speaker Recognition from Raw Waveform with SincNet
Ravanelli, M., & Bengio, Y. (2018) · 2018 IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt.2018.8639585 · Cited by 896
ASVspoof 2019: Future Horizons in Spoofed and Fake Audio Detection
Todisco, M., Wang, X., Vestman, V., Sahidullah, M., Delgado, H., Nautsch, A., Yamagishi, J., Evans, N., Kinnunen, T., & Lee, K. A. (2019) · arXiv · doi:10.48550/arxiv.1904.05441 · Cited by 688
ASVspoof 2019: Future Horizons in Spoofed and Fake Audio Detection
Todisco, M., Wang, X., Vestman, V., Sahidullah, M., Delgado, H., Nautsch, A., Yamagishi, J., Evans, N., Kinnunen, T. H., & Lee, K. A. (2019) · Interspeech 2019 · doi:10.21437/interspeech.2019-2249 · Cited by 687
XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale
Babu, A., Wang, C., Tjandra, A., Lakhotia, K., Xu, Q., Goyal, N., Singh, K., von Platen, P., Saraf, Y., Pino, J., et al. (2022) · Interspeech 2022 · doi:10.21437/interspeech.2022-143 · Cited by 586
ASVspoof 2019: A large-scale public database of synthesized, converted and replayed speech
Wang, X., Yamagishi, J., Todisco, M., Delgado, H., Nautsch, A., Evans, N., Sahidullah, M., Vestman, V., Kinnunen, T., Lee, K. A., et al. (2020) · Computer Speech & Language · doi:10.1016/j.csl.2020.101114 · Cited by 505 · cited 1×
“We utilized the ASVspoof 2019 LA database [25, 32] training part for training, opting not to use the validation part, because we believe the validation set is traditionally used to prevent overfitting, but the validation part, compared to the train set, only differs in the speaker, we posit that…”
End-to-End anti-spoofing with RawNet2
Tak, H., Patino, J., Todisco, M., Nautsch, A., Evans, N., & Larcher, A. (2021) · ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp39728.2021.9414234 · Cited by 402 · cited 1×
“Tak et al. [16] achieved significantly improved performance in the field of spoofing detection by applying the XLS-R model with a Rawnet2 [27] encoder and AASIST [28] backend, achieving an EER of 2.85% on the ASVspoof 2021 DF eval set.”
InfluentialAASIST: Audio Anti-Spoofing Using Integrated Spectro-Temporal Graph Attention Networks
Jung, J.-W., Heo, H.-S., Tak, H., Shim, H.-J., Chung, J. S., Lee, B.-J., Yu, H.-J., & Evans, N. (2022) · ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp43922.2022.9747766 · Cited by 399
ASVspoof 2021: accelerating progress in spoofed and deepfake speech detection
Yamagishi, J., Wang, X., Todisco, M., Sahidullah, M., Patino, J., Nautsch, A., Liu, X., Lee, K. A., Kinnunen, T., Evans, N., et al. (2021) · 2021 Edition of the Automatic Speaker Verification and Spoofing Countermeasures Challenge · doi:10.21437/asvspoof.2021-8 · Cited by 380
ASVspoof 2021: Towards Spoofed and Deepfake Speech Detection in the Wild
Liu, X., Wang, X., Sahidullah, M., Patino, J., Delgado, H., Kinnunen, T., Todisco, M., Yamagishi, J., Evans, N., Nautsch, A., et al. (2023) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2023.3285283 · Cited by 312 · cited 1×
“The latest ASVspoof challenge is [7], which has introduced a new task: Deepfake (DF).”
Does Audio Deepfake Detection Generalize?
Müller, N., Czempin, P., Diekmann, F., Froghyar, A., & Böttinger, K. (2022) · Interspeech 2022 · doi:10.21437/interspeech.2022-108 · Cited by 258
Tak, H., Todisco, M., Wang, X., Jung, J.-W., Yamagishi, J., & Evans, N. (2022) · The Speaker and Language Recognition Workshop (Odyssey 2022) · doi:10.21437/odyssey.2022-16 · Cited by 237
ADD 2022: the first Audio Deep Synthesis Detection Challenge
Yi, J., Fu, R., Tao, J., Nie, S., Ma, H., Wang, C., Wang, T., Tian, Z., Bai, Y., Fan, C., et al. (2022) · ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp43922.2022.9746939 · Cited by 165
Investigating Self-Supervised Front Ends for Speech Spoofing Countermeasures
Wang, X., & Yamagishi, J. (2022) · The Speaker and Language Recognition Workshop (Odyssey 2022) · doi:10.21437/odyssey.2022-14 · Cited by 138
ASVspoof 2021: Automatic Speaker Verification Spoofing and Countermeasures Challenge Evaluation Plan
Delgado, H., Evans, N., Kinnunen, T., Lee, K. A., Liu, X., Nautsch, A., Patino, J., Sahidullah, M., Todisco, M., Wang, X., et al. (2021) · arXiv · doi:10.48550/arxiv.2109.00535 · Cited by 129
The Vicomtech Audio Deepfake Detection System Based on Wav2vec2 for the 2022 ADD Challenge
Martin-Donas, J. M., & Alvarez, A. (2022) · ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp43922.2022.9747768 · Cited by 100
Audio Deepfake Detection With Self-Supervised WavLM And Multi-Fusion Attentive Classifier
Guo, Y., Huang, H., Chen, X., Zhao, H., & Wang, Y. (2024) · ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp48485.2024.10447923 · Cited by 95
Deepfake audio detection by speaker verification
Pianese, A., Cozzolino, D., Poggi, G., & Verdoliva, L. (2022) · 2022 IEEE International Workshop on Information Forensics and Security (WIFS) · doi:10.1109/wifs55849.2022.9975428 · Cited by 74
A Robust Audio Deepfake Detection System via Multi-View Feature
Yang, Y., Qin, H., Zhou, H., Wang, C., Guo, T., Han, K., & Wang, Y. (2024) · ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp48485.2024.10446560 · Cited by 61 · cited 2×
“Compared to prior work[16, 30], our model architecture achieves high accuracy despite its simplicity.”
Audio Deepfake Detection: A Survey
Yi, J., Wang, C., Tao, J., Zhang, X., Zhang, C. Y., & Zhao, Y. (2023) · arXiv · doi:10.48550/arxiv.2308.14970 · Cited by 40
Multi-Scale Permutation Entropy for Audio Deepfake Detection
Wang, C., He, J., Yi, J., Tao, J., Zhang, C. Y., & Zhang, X. (2024) · ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp48485.2024.10448095 · Cited by 31
ADD 2023: the Second Audio Deepfake Detection Challenge
Yi, J., Tao, J., Fu, R., Yan, X., Wang, C., Wang, T., Zhang, C. Y., Zhang, X., Zhao, Y., Ren, Y., et al. (2023) · arXiv · doi:10.48550/arxiv.2305.13774 · Cited by 20
Spoofing and countermeasures for speaker verification: A review
Anjum, Z. K., & Swamy, R. K. (2017) · 2017 International Conference on Wireless Communications, Signal Processing and Networking (WiSPNET) · doi:10.1109/wispnet.2017.8299800 · Cited by 9 · cited 1×
“There are also many biometric identity authentication applications, such as access control systems, telephone banking, and forensic scenarios [5].”
Research works printed but not linked: 14
- 10.1007/s11042-022-13943-4 · doi:10.1007/s11042-022-13943-4
- 10.1109/ASRU57964.2023.10389671 · DOI
- Massimiliano Todisco Héctor Delgado Kong Aik Lee Md Sahidullah Nicholas Evans Tomi Kinnunen and Junichi Yamagishi. Integrated presentation attack detection and automatic speaker verification: Common features and gaussian back-end fusion. In Interspeech 2018--19th Annual Conference of the International Speech Communication Association. ISCA 2018. · DOI · Paper page
- 10.1109/ISCSLP.2010.5684887 · DOI
- Massimiliano Todisco, Héctor Delgado, and Nicholas WD Evans. A new feature for automatic speaker verification anti-spoofing: Constant q cepstral coefficients. In Odyssey, volume 2016, pages 283--290, 2016.
- 10.1109/ICASSP43922.2022.9746722 · DOI
- Yang Xie, Zhenchuan Zhang, and Yingchun Yang. Siamese network with wav2vec feature for spoofing speech detection. In Interspeech, pages 4269--4273, 2021.
- 10.1109/ICASSP43922.2022.9747605 · DOI · Paper page
- Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. wav2vec 2.0: A framework for self-supervised learning of speech representations. Advances in neural information processing systems, 33:12449--12460, 2020.
- Chenglong Wang, Jiangyan Yi, Jianhua Tao, Chu Yuan Zhang, Shuai Zhang, and Xun Chen. Detection of Cross-Dataset Fake Audio Based on Prosodic and Pronunciation Features. In Proc. INTERSPEECH 2023, pages 3844--3848, 2023.
- 10.1109/CVPR.2018.00745 · DOI · Paper page
- Niko Brümmer and Edward De Villiers. The bosaris toolkit: Theory, algorithms and code for surviving the new dcf. arXiv preprint arXiv:1304.2865, 2013. · arXiv
- Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. nature, 521(7553):436--444, 2015.
- 10.1109/ICASSP43922.2022.9746213 · DOI
Cited by
Audio Deepfake Detection: What Has Been Achieved and What Lies Ahead
Zhang, B., Cui, H., Nguyen, V., & Whitty, M. (2025) · Sensors · doi:10.3390/s25071989 · Cited by 72
AWaveFormer: Audio Wavelet Transformer Network for Generalized Audio Deepfake Detection
Wang, R., Chen, Z., Wang, B., Ba, Z., & Ren, K. (2025) · IEEE Transactions on Audio, Speech and Language Processing · doi:10.1109/taslpro.2025.3611229 · Cited by 10
Benchmarking Audio Deepfake Detection Robustness in Real-World Communication Scenarios
Shi, H., Shi, X., Dogan, S., Alzubi, S., Huang, T., & Zhang, Y. (2025) · 2025 33rd European Signal Processing Conference (EUSIPCO) · doi:10.23919/eusipco63237.2025.11226601 · Cited by 8
“Additionally, other acoustic features, including Constant-Q Cepstral Coefficients (CQCC), Linear-Frequency Cep-stral Coefficients (LFCC), Mel-spectrograms, and constant-Q-transform [9]–[11], have also been extensively utilised.”
Metadata
- Authors
- Zhang, Q., Wen, S., & Hu, T.
- Venue
- Proceedings of the 32nd ACM International Conference on Multimedia
- Publisher
- ACM
- Published
- 2024-10-28
- Pages
- 6765-6773
- OpenAlex
- W4403791925
- Cited by
- 56
- References
- 25
- FWCI
- 17.04
- Type
- conference paper
- Language
- en
- Open access
- closed
- Funders
- Natural Science Foundation of Hubei Province of China (2023AFD061)
- Training program of high level scientific research achievements of Hubei Minzu University (PY22011)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.