CoChat
Sign Up
Cited by 15Speech Recognition and SynthesisDigital Media Forensic DetectionMisinformation and Its Impacts

"Better Be Computer or I'm Dumb": A Large-Scale Evaluation of Humans as Audio Deepfake Detectors

Open access
Warren, Kevin, Tucker, Tyler, Crowder, Anna, Olszewski, Daniel, Lu, Allison, Fedele, Caroline, Pasternak, Magdalena, Layton, Seth +3 more·2024·Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security
DOI 10.1145/3658644.3670325
Open this paper in CoChat — free accountOpen PDF

Abstract

Audio deepfakes represent a rising threat to trust in our daily communications. In response to this, the research community has developed a wide array of detection techniques aimed at preventing such attacks from deceiving users. Unfortunately, the creation of these defenses has generally overlooked the most important element of the system - the user themselves. As such, it is not clear whether current mechanisms augment, hinder, or simply contradict human classification of deepfakes. In this paper, we perform the first large-scale user study on deepfake detection. We recruit over 1,200 users and present them with samples from the three most widely-cited deepfake datasets. We then quantitatively compare performance and qualitatively conduct thematic analysis to motivate and understand the reasoning behind user decisions and differences from machine classifications. Our results show that users correctly classify human audio at significantly higher rates than machine learning models, and rely on linguistic features and intuition when performing classification. However, users are also regularly misled by pre-conceptions about the capabilities of generated audio (e.g., that accents and background sounds are indicative of humans). Finally, machine learning models suffer from significantly higher false positive rates, and experience false negatives that humans correctly classify when issues of quality or robotic characteristics are reported. By analyzing user behavior across multiple deepfake datasets, our study demonstrates the need to more tightly compare user and machine learning performance, and to target the latter towards areas where humans are less likely to successfully identify threats.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3658644.3670325

Cites

38 of 38 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Successful spoofing attacks are not necessarily implemented using the latest techniques and might not require the highest quality synthetic speech to fool a CM or an ASV system.

    “However, successful spoofing attacks are not necessarily implemented using the latest techniques, and might not even require the highest quality synthetic speech to fool a CM (Warren et al., 2024) or an ASV system (Jung et al., 2024).”

    From ASVspoof 5: Design, collection and validation of resources for spoofing, deepfake, and adversarial attack detection using crowdsourced speech (2026) · cites this paper as Warren et al., 2024

    In the Case (1) analysis (page unavailable), the paper reports that models sometimes missed deepfakes that humans described as robotic or poor quality. This supports the point that high-quality synthetic speech is not always needed to fool the tested detection models. The cited paper does not establish that successful attacks need not use the latest generation techniques; the ASV-system portion is attributed to a different citation. (LLM, conf=0.85)

    Experimental — accuracy not yet measured

  • In a large-scale user study involving over 1200 participants, human overall performance in classifying deepfake audio remains suboptimal in certain scenarios, particularly in complex environments.

    “Another large-scale user study [27] involving over 1200 participants showed that while humans can rely on intuition and linguistic features for classification, their overall performance remains suboptimal in certain scenarios, particularly in complex environments.”

    From Audio Deepfake Detection: What Has Been Achieved and What Lies Ahead (2025) · cites this paper as [27]

    The Introduction reports a study of over 1,200 participants with 73% classification accuracy, and §4.1 reports lower accuracy on some datasets. The paper also says participants rely on linguistic features and intuition (Abstract; Finding 3). It does not establish that performance is particularly suboptimal in complex environments; the tested conditions were samples from three audio-deepfake datasets, and no page numbers are available in our copy. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • Existing machine learning models have not consistently outperformed human intuition in deepfake audio detection.

    “Furthermore, while existing machine learning models have achieved considerable progress, they have not consistently outperformed human intuition [27].”

    From Audio Deepfake Detection: What Has Been Achieved and What Lies Ahead (2025) · cites this paper as [27]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 31

  1. [4]4. Ehab A AlBadawy, Siwei Lyu, and Hany Farid. 2019. Detecting AI-Synthesized Speech Using Bispectral Analysis. In CVPR Workshops.
  2. [8]8. Logan Blue, Kevin Warren, Hadi Abdullah, Cassidy Gibson, Luis Vargas, Jessica O'Dell, Kevin Butler, and Patrick Traynor. 2022. Who Are You (I Really Wanna Know)? Detecting Audio DeepFakes Through Vocal Tract Reconstruction. In Proceedings of the USENIX Security Symposium.
  3. [9]9. Harry Cheng, Yangyang Guo, Tianyi Wang, Qi Li, Xiaojun Chang, and Liqiang Nie. 2022. Voice-Face Homogeneity Tells Deepfake. arXiv (2022).
  4. [11]11. François Chollet. 2017. Xception: Deep Learning with Depthwise Separable Convolutions. arXiv (2017).
  5. [14]14. Davide Cozzolino, Matthias Nießner, and Luisa Verdoliva. 2022. Audio-Visual Person-of-Interest DeepFake Detection. arXiv (2022).
  6. [18]18. Chao Feng, Ziyang Chen, and Andrew Owens. 2023. Self-Supervised Video Forensics by Audio-Visual Anomaly Detection. arXiv (2023).
  7. [21]21. Joel Frank and Lea Schönherr. 2021. WaveFake: A Data Set to Facilitate Audio DeepFake Detection. In Proceedings of the Conference on Neural Information Processing Systems (NeurIPS).
  8. [24]24. Wajih Ul Hassan, Shengjian Guo, Ding Li, Zhengzhang Chen, Kangkook Jee, Zhichun Li, and Adam Bates. 2019. NoDoze: Combatting Threat Alert Fatigue with Automated Provenance Triage. Network and Distributed Systems Security Symposium (2019).
  9. [29]29. Ye Jia, Yu Zhang, Ron Weiss, Quan Wang, Jonathan Shen, Fei Ren, zhifeng Chen, Patrick Nguyen, Ruoming Pang, Ignacio Lopez Moreno, and Yonghui Wu. 2018. Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis. In Advances in Neural Information Processing Systems.
  10. [30]30. Ye Jia, Yu Zhang, Ron J. Weiss, Quan Wang, Jonathan Shen, Fei Ren, Zhifeng Chen, Patrick Nguyen, Ruoming Pang, Ignacio Lopez Moreno, and Yonghui Wu. 2019. Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis. arXiv (2019).
  11. [32]32. Piotr Kawa, Marcin Plata, and Piotr Syga. 2022. Defense Against Adversarial Attacks on Audio DeepFake Detection. arXiv (2022).
  12. [33]33. Piotr Kawa, Marcin Plata, and Piotr Syga. 2022. SpecRNet: Towards Faster and More Accessible Audio DeepFake Detection. arXiv (2022).
  13. [34]34. Hasam Khalid, Minha Kim, Shahroz Tariq, and Simon S. Woo. 2021. Evaluation of an Audio-Video Multimodal Deepfake Dataset Using Unimodal and Multimodal Detectors. In Proceedings of the Workshop on Synthetic Multimedia -Audiovisual Deepfake Generation and Detection.
  14. [35]35. Hasam Khalid, Shahroz Tariq, Minha Kim, and Simon S. Woo. 2022. FakeAVCeleb: A Novel Audio-Video Multimodal Deepfake Dataset. In Proceedings of the Neu- ral Information Processing Systems Track on Datasets and Benchmarks (NeurIPS Datasets and Benchmarks).
  15. [36]36. Pavel Korshunov and Sébastien Marcel. 2020. Deepfake Detection: Humans vs. Machines. arXiv (2020).
  16. [38]38. Seth Layton, Tyler Tucker, Daniel Olszewski, Kevin Warren, Kevin Butler, and Patrick Traynor. 2024. SoK: The Good, The Bad, and The Unbalanced: Measuring Structural Limitations of Deepfake Datasets. In Proceedings of the USENIX Security Symposium (Security).
  17. [39]39. Sangjun Lee, Donggeun Ko, Jinyong Park, Saebyeol Shin, Donghee Hong, and Simon S. Woo. 2022. Deepfake Detection for Fake Images with Facemasks. In Proceedings of the Workshop on Security Implications of Deepfakes and Cheapfakes.
  18. [41]41. Haoxin Ma, Jiangyan Yi, Chenglong Wang, Xinrui Yan, Jianhua Tao, Tao Wang, Shiming Wang, and Ruibo Fu. 2022. CFAD: A Chinese Dataset for Fake Audio Detection. arXiv (2022).
  19. [43]43. Hafiz Malik. 2019. Securing Voice-Driven Interfaces Against Fake (Cloned) Audio Attacks. In 2019 IEEE Conference on Multimedia Information Processing and Retrieval (MIPR).
  20. [47]47. Jaron Mink, Licheng Luo, Natã M. Barbosa, Olivia Figueira, Yang Wang, and Gang Wang. 2022. DeepPhish: Understanding User Trust Towards Artificially Generated Profiles in Online Social Networks. In Proceedings of the USENIX Security Symposium.
  21. [48]48. Daniel Mas Montserrat, Hanxiang Hao, S. K. Yarlagadda, Sriram Baireddy, Ruiting Shao, János Horváth, Emily Bartusiak, Justin Yang, David Güera, Fengqing Zhu, and Edward J. Delp. 2020. Deepfakes Detection with Automatic Face Weighting. arXiv (2020).
  22. [54]54. Daniel Olszewski, Allison Lu, Carson Stillman, Kevin Warren, Cole Kitroser, Alejandro Pascual, Divyajyoti Ukirde, Kevin Butler, and Patrick Traynor. 2023. "Get In Researchers; We're Measuring Reproducibility": A Reproducibility Study of Machine Learning Papers in Tier 1 Security Conferences. In Proceedings of the ACM SIGSAC Conference on Computer and Communications Security (CCS).
  23. [56]56. Kaizhi Qian, Yang Zhang, Shiyu Chang, Xuesong Yang, and Mark Hasegawa- Johnson. 2019. AUTOVC: Zero-Shot Voice Style Transfer with Only Autoencoder Loss. In Proceedings of the International Conference on Machine Learning.
  24. [60]60. Steffen Schneider, Alexei Baevski, Ronan Collobert, and Michael Auli. 2019. Wav2vec: Unsupervised Pre-training for Speech Recognition. arXiv (2019).
  25. [63]63. Ryosuke Sonobe, Shinnosuke Takamichi, and Hiroshi Saruwatari. 2017. JSUT Cor- pus: Free Large-Scale Japanese Speech Corpus for End-to-End Speech Synthesis. arXiv (2017).
  26. [64]64. Chengzhe Sun, Shan Jia, Shuwei Hou, Ehab AlBadawy, and Siwei Lyu. 2023. Exposing AI-Synthesized Human Voices Using Neural Vocoder Artifacts. arXiv (2023).
  27. [69]69. Qian Wang, Xiu Lin, Man Zhou, Yanjiao Chen, Cong Wang, Qi Li, and Xiangyang Luo. 2019. VoicePop: A Pop Noise Based Anti-spoofing System for Voice Authen- tication on Smartphones. In IEEE INFOCOM 2019 -IEEE Conference on Computer Communications.
  28. [71]71. Xin Wang and Junichi Yamagishi. 2023. Investigating Active-Learning-Based Training Data Selection for Speech Spoofing Countermeasure. In 2022 IEEE Spoken Language Technology Workshop (SLT).
  29. [72]72. Xin Wang and Junichi Yamagishi. 2023. Spoofed Training Data for Speech Spoofing Countermeasure Can Be Efficiently Created Using Neural Vocoders. arXiv (2023).
  30. [74]74. Emily Wenger, Max Bronckers, Christian Cianfarani, Jenna Cryan, Angela Sha, Haitao Zheng, and Ben Y. Zhao. 2021. "Hello, It's Me": Deep Learning-based Speech Synthesis Attacks in the Real World. In Proceedings of the ACM SIGSAC Conference on Computer and Communications Security (CCS).
  31. [81]81. Linghan Zhang, Sheng Tan, Jie Yang, and Yingying Chen. 2016. VoiceLive: A Phoneme Localization Based Liveness Detection for Voice Authentication on Smartphones. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security.

Cited by

Sorted by citations

Related

Authors
Warren, K., Tucker, T., Crowder, A., Olszewski, D., Lu, A., Fedele, C., Pasternak, M., Layton, S., Butler, K., Gates, C., & Traynor, P.
Affiliations
  • University of Florida (US)
  • Dalhousie University (CA)
ORCID
Venue
Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security
Publisher
ACM
Published
2024-12-02
Pages
2696-2710
ISSN
1543-7221 (ISSN-L)
DOI
10.1145/3658644.3670325
OpenAlex
W4405181098
Cited by
15
References
37
Citations by year
2026: 8 · 2025: 7
FWCI
4.43
Type
conference paper
Language
en
Open access
Licence
www.acm.org/publications/policies/copyright_policy#Background (vor, 372 d)
Locations
ACM SIGSAC Conference on Computer and Communications Security (CCS) (publishedVersion)
Funders
  • National Science Foundation (CNS-1933208, CNS-2206950)
  • Office of Naval Research (ONR-OTA N00014-21-1-2658)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
  • Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
Keywords
deepfake audio, deepfake detection, user study, human-machine comparison, linguistic features, false positives, deepfake datasets, trust in communication

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review