Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 45 of the 54 references it cites were found in a registry or an index.

SafeEar: Content Privacy-Preserving Audio Deepfake Detection

Open access
Li, Xinfeng, Li, Kai, Zheng, Yifan, Yan, Chen, Ji, Xiaoyu, Xu, Wenyuan · 2024 · Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security

Abstract

Text-to-Speech (TTS) and Voice Conversion (VC) models have exhibited remarkable performance in generating realistic and natural audio. However, their dark side, audio deepfake poses a significant threat to both society and individuals. Existing countermeasures largely focus on determining the genuineness of speech based on complete original audio recordings, which however often contain private content. This oversight may refrain deepfake detection from many applications, particularly in scenarios involving sensitive information like business secrets. In this paper, we propose SafeEar, a novel framework that aims to detect deepfake audios without relying on accessing the speech content within. Our key idea is to devise a neural audio codec into a novel decoupling model that well separates the semantic and acoustic information from audio samples, and only use the acoustic information (e.g., prosody and timbre) for deepfake detection. In this way, no semantic content will be exposed to the detector. To overcome the challenge of identifying diverse deepfake audio without semantic clues, we enhance our deepfake detector with real-world codec augmentation. Extensive experiments conducted on four benchmark datasets demonstrate SafeEar's effectiveness in detecting various deepfake techniques with an equal error rate (EER) down to 2.02%. Simultaneously, it shields five-language speech content from being deciphered by both machine and human auditory analysis, demonstrated by word error rates (WERs) all above 93.93% and our user study. Furthermore, our benchmark constructed for anti-deepfake and anti-content recovery evaluation helps provide a basis for future research in the realms of audio privacy preservation and deepfake detection.

Overview

Integrity signals

Registry

Crossref · 10.1145/3658644.3670285

Cites

45 of 45 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 54 references

Research works printed but not linked: 34

  1. Shimaa Ahmed, Amrita Roy Chowdhury, Kassem Fawaz, and Parmesh Ramanathan. 2020. Preech: A System for Privacy-Preserving Speech Transcription. In 29th USENIX Security Symposium, USENIX Security. 2703--2720.
  2. R. Ardila, M. Branson, K. Davis, M. Henretty, M. Kohler, J. Meyer, R. Morais, L. Saunders, F. M. Tyers, and G. Weber. 2020. Common Voice: A Massively-Multilingual Speech Corpus. In Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020). 4211--4215.
  3. Fahimeh Bahmaninezhad, Chunlei Zhang, and John H. L. Hansen. 2018. Convolutional Neural Network Based Speaker De-Identification. In Odyssey 2018: The Speaker and Language Recognition Workshop, 26--29 June 2018, Les Sables d'Olonne. 255--260.
  4. Nicholas Carlini and David A. Wagner. 2018. Audio Adversarial Examples: Targeted Attacks on Speech-to-Text. In 2018 IEEE Security and Privacy Workshops, SP Workshops 2018. 1--7.
  5. Ouba"ida Chouchane, Baptiste Brossier, Jorge Esteban Gamboa Gamboa, Thomas Lardy, Hemlata Tak, Orhan Ermis, Madhu R. Kamble, Jose Patino, Nicholas W. D. Evans, Melek Önen, and Massimiliano Todisco. 2021. Privacy-Preserving Voice Anti-Spoofing Using Secure Multi-Party Computation. In 22nd Annual Conference of the International Speech Communication Association, Interspeech 2021. 856--860.
  6. Jiangyi Deng, Fei Teng, Yanjiao Chen, Xiaofu Chen, Zhaohui Wang, and Wenyuan Xu. 2023. V-Cloak: Intelligibility-, Naturalness- & Timbre-Preserving Real-Time Voice Anonymization. In 32nd USENIX Security Symposium, USENIX Security 2023, Anaheim. 5181--5198.
  7. Yuchen Fan, Yao Qian, Feng-Long Xie, and Frank K. Soong. 2014. TTS Synthesis With Bidirectional LS™ Based Recurrent Neural Networks. In 15th Annual Conference of the International Speech Communication Association, INTERSPEECH 2014. 1964--1968.
  8. Xiaolin Hu, Kai Li, Weiyi Zhang, Yi Luo, Jean-Marie Lemercier, and Timo Gerkmann. 2021. Speech Separation Using an Asynchronous Fully Recurrent Convolutional Neural Network. In Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS. 22509--22522.
  9. Jee-weon Jung, Hee-Soo Heo, Hemlata Tak, Hye-jin Shim, Joon Son Chung, Bong-Jin Lee, Ha-Jin Yu, and Nicholas W. D. Evans. 2022. AASIST: Audio Anti-Spoofing Using Integrated Spectro-Temporal Graph Attention Networks. In IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2022. 6367--6371.
  10. Tadej Justin, Vitomir Struc, Simon Dobrisek, Bostjan Vesnicer, Ivo Ipsic, and France Mihelic. 2015. Speaker De-Identification Using Diphone Recognition and Speech Synthesis. In 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition, FG 2015. 1--7.
  11. Jaehyeon Kim, Jungil Kong, and Juhee Son. 2021. Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech. In Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18--24 July 2021 (Proceedings of Machine Learning Research, Vol. 139). 5530--5540.
  12. Jungil Kong, Jaehyeon Kim, and Jaekyoung Bae. 2020. HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020.
  13. Kundan Kumar, Rithesh Kumar, Thibault de Boissiere, Lucas Gestin, Wei Zhen Teoh, Jose Sotelo, Alexandre de Brébisson, Yoshua Bengio, and Aaron C. Courville. 2019. MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8--14, 2019, Vancouver, BC. 14881--14892.
  14. Kai Li, Xiaolin Hu, and Yi Luo. 2022. On the Use of Deep Mask Estimation Module for Neural Source Separation Systems. In 23rd Annual Conference of the International Speech Communication Association, Interspeech 2022. 5328--5332.
  15. Kai Li and Yi Luo. 2023. On The Design and Training Strategies for Rnn-Based Online Neural Speech Separation Systems. In IEEE International Conference on Acoustics, Speech and Signal Processing ICASSP 2023. 1--5.
  16. Kai Li, Runxuan Yang, and Xiaolin Hu. 2023. An Efficient Encoder-Decoder Architecture With Top-Down Attention For Speech Separation. In The Eleventh International Conference on Learning Representations, ICLR 2023.
  17. Menglu Li, Yasaman Ahmadiadli, and Xiao-Ping Zhang. 2022. A Comparative Study on Physical and Perceptual Features for Deepfake Audio Detection. In DDAM at MM 2022: Proceedings of the 1st International Workshop on Deepfake Detection for Audio Multimedia, Lisboa. 35--41.
  18. Xinfeng Li, Xiaoyu Ji, Chen Yan, Chaohao Li, Yichen Li, Zhenning Zhang, and Wenyuan Xu. 2023. Learning Normality is Enough: A Software-based Mitigation against Inaudible Voice Attacks. In 32nd USENIX Security Symposium, USENIX Security 2023, Anaheim. 2455--2472.
  19. Xinfeng Li, Chen Yan, Xuancun Lu, Zihan Zeng, Xiaoyu Ji, and Wenyuan Xu. 2024. Inaudible Adversarial Perturbation: Manipulating the Recognition of User Speech in Real Time. In 31st Annual Network and Distributed System Security Symposium, NDSS 2024.
  20. Xinfeng Li, Zhicong Zheng, Chen Yan, Chaohao Li, Xiaoyu Ji, and Wenyuan Xu. 2023 d. Towards Pitch-Insensitive Speaker Verification via Soundfield. IEEE Internet of Things Journal (2023).
  21. Xiaohui Liu, Meng Liu, Longbiao Wang, Kong Aik Lee, Hanyi Zhang, and Jianwu Dang. 2023. Leveraging Positional-Related Local-Global Dependency for Synthetic Speech Detection. In IEEE International Conference on Acoustics, Speech and Signal Processing ICASSP 2023. 1--5.
  22. Andrew Cameron Morris, Viktoria Maier, and Phil D. Green. 2004. From WER and RIL to MER and WIL: Improved Evaluation Measures For Connected Speech Recognition. In 8th International Conference on Spoken Language Processing, INTERSPEECH-ICSLP 2004. 2765--2768.
  23. Ahmed Mustafa, Nicola Pia, and Guillaume Fuchs. 2021. StyleMelGAN: An Efficient High-Fidelity Adversarial Vocoder with Temporal Adaptive Normalization. In IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2021. 6034--6038.
  24. Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur. 2015. Librispeech: An ASR Corpus Based on Public Domain Audio Books. In 2015 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2015. 5206--5210.
  25. Jose Patino, Natalia A. Tomashenko, Massimiliano Todisco, Andreas Nautsch, and Nicholas W. D. Evans. 2021. Speaker Anonymisation Using the McAdams Coefficient. In 22nd Annual Conference of the International Speech Communication Association, Interspeech 2021. 1099--1103.
  26. Mirco Ravanelli, Titouan Parcollet, Peter Plantinga, Aku Rouhe, Samuele Cornell, Loren Lugosch, Cem Subakan, Nauman Dawalatabad, Abdelwahab Heba, Jianyuan Zhong, Ju-Chieh Chou, Sung-Lin Yeh, Szu-Wei Fu, Chien-Feng Liao, Elena Rastorgueva, Franccois Grondin, William Aris, Hwidong Na, Yan Gao, Renato De Mori, and Yoshua Bengio. 2021. SpeechBrain: A General-Purpose Speech Toolkit., Vol. abs/2106.04624 (2021). showeprint[arXiv]2106.04624 · arXiv
  27. Yi Ren, Chenxu Hu, Xu Tan, Tao Qin, Sheng Zhao, Zhou Zhao, and Tie-Yan Liu. 2021. FastSpeech 2: Fast and High-Quality End-to-End Text to Speech. In 9th International Conference on Learning Representations, ICLR 2021.
  28. Yi Ren, Yangjun Ruan, Xu Tan, Tao Qin, Sheng Zhao, Zhou Zhao, and Tie-Yan Liu. 2019. FastSpeech: Fast, Robust and Controllable Text to Speech. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8--14, 2019, Vancouver, BC. 3165--3174.
  29. Aäron van den Oord, Oriol Vinyals, and Koray Kavukcuoglu. 2017. Neural Discrete Representation Learning. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4--9, 2017, Long Beach, CA,. 6306--6315.
  30. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is All you Need. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4--9, 2017, Long Beach, CA,. 5998--6008.
  31. Junichi Yamagishi, Xin Wang, Massimiliano Todisco, Md. Sahidullah, Jose Patino, Andreas Nautsch, Xuechen Liu, Kong Aik Lee, Tomi Kinnunen, Nicholas W. D. Evans, and Héctor Delgado. 2021. ASVspoof 2021: Accelerating Progress in Spoofed and Deepfake Speech Detection., Vol. abs/2109.00537 (2021). showeprint[arXiv]2109.00537 · arXiv
  32. Ryuichi Yamamoto, Eunwoo Song, and Jae-Min Kim. 2020. Parallel Wavegan: A Fast Waveform Generation Model Based on Generative Adversarial Networks with Multi-Resolution Spectrogram. In 2020 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2020. 6199--6203.
  33. Geng Yang, Shan Yang, Kai Liu, Peng Fang, Wei Chen, and Lei Xie. 2021. Multi-Band Melgan: Faster Waveform Generation For High-Quality Text-To-Speech. In IEEE Spoken Language Technology Workshop, SLT 2021. 492--498.
  34. Heiga Zen, Andrew W. Senior, and Mike Schuster. 2013. Statistical Parametric Speech Synthesis Using Deep Neural Networks. In IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2013, Vancouver, BC. 7962--7966.

Cited by

See all 54

Metadata

Authors
Li, X., Li, K., Zheng, Y., Yan, C., Ji, X., & Xu, W.
Affiliations
  • Zhejiang University (CN)
  • Tsinghua University (CN)
Venue
Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security
Publisher
ACM
Published
2024-12-02
Pages
3585-3599
ISSN
1543-7221 (ISSN-L)
OpenAlex
W4405182531
Cited by
39
References
54
Citations by year
2026: 16 · 2025: 21 · 2024: 2
FWCI
8.08
Type
conference paper
Language
en
Open access
Locations
  • ACM SIGSAC Conference on Computer and Communications Security (CCS) (publishedVersion, public-domain)
  • arXiv (Cornell University) (submittedVersion, cc-by-nc-nd)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
  • Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
  • Physical Sciences › Computer Science › Signal Processing › Music and Audio Processing
Keywords
audio deepfake detection, content privacy, audio forensics, deepfake audio, privacy-preserving detection

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: SafeEar: Content Privacy-Preserving Audio Deepfake Detection. https://app.cochat.ai/papers/10.1145/3658644.3670285
The paper
Li, X., Li, K., Zheng, Y., Yan, C., Ji, X., & Xu, W. (2024). SafeEar: Content Privacy-Preserving Audio Deepfake Detection. Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security. https://doi.org/10.1145/3658644.3670285

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.