The DOI matches its Crossref record; 40 of the 40 references it cites were found in a registry or an index; 27 of the 43 sources its claims cite were checked for support.
Warning: Humans cannot reliably detect speech deepfakes
Open accessAbstract
Speech deepfakes are artificial voices generated by machine learning models. Previous literature has highlighted deepfakes as one of the biggest security threats arising from progress in artificial intelligence due to their potential for misuse. However, studies investigating human detection capabilities are limited. We presented genuine and deepfake audio to n = 529 individuals and asked them to identify the deepfakes. We ran our experiments in English and Mandarin to understand if language affects detection performance and decision-making rationale. We found that detection capability is unreliable. Listeners only correctly spotted the deepfakes 73% of the time, and there was no difference in detectability between the two languages. Increasing listener awareness by providing examples of speech deepfakes only improves results slightly. As speech synthesis algorithms improve and become more realistic, we can expect the detection task to become harder. The difficulty of detecting speech deepfakes confirms their potential for misuse and signals that defenses against this threat are needed.
Overview
CoChat Summary
## TL;DR
This study tested 529 people on whether they could distinguish genuine from synthetic speech in English and Mandarin. Human detection was unreliable: participants identified deepfakes correctly 73% of the time in the unary setting, with no significant language difference and only a small boost from familiarization.
## Key findings
- In the unary task, participants classified clips correctly 70.35% of the time overall; they spotted deepfakes correctly 73% of the time and bona fide clips correctly 67.78% of the time.
- In the binary task, performance improved: participants correctly recognized deepfake audio in 85.59% of trials, and this improvement from contextual reference audio was statistically significant (p < 0.001).
- Familiarization with five deepfake examples improved detection accuracy by 3.84% on average (p = 0.001); in the unary and binary regressions the gains were 3.76% (p = 0.017) and 3.85% (p = 0.032), respectively.
- Detection did not significantly differ by language: Mandarin-speaking participants outperformed English-speaking participants by 1.79%, but this was not statistically significant (p = 0.202).
- Human performance was stable over time and not helped by repeated listening: clip replays had no relationship with accuracy (ρ = −0.05, p < 0.001), and total task time also showed no meaningful improvement (ρ = 0.10, p = 0.018).
## Methods
- Online experiment with 529 Prolific participants fluent in English or Mandarin; mean age was 28.9 years and 50.6% identified as male.
- Stimuli were 100 clips total: bona fide speech from LJSpeech (English) and CSMSC (Mandarin), plus deepfakes generated from the same sentences using pre-trained VITS text-to-speech models; clip lengths ranged from 2 to 11 seconds.
- Participants were randomly assigned to unary vs binary judging, and to familiarization vs control; the main analysis used linear regression on confidence-weighted accuracy scores, with additional comparisons to automated LFCC-LCNN detectors and AUROC/EER metrics.
## Limitations
- The paper states that the setup “does not necessarily reflect more realistic scenarios” because the deepfake proportion was balanced 50/50, whereas AI-generated content is still rare, and participants were warned about deepfakes in advance.
- The stimuli minimized contextual information, so the study does not test cases where speaker identity, emotional status, conversational context, or political affiliation might help detection.
- The deepfakes were generated with an older approach and “may not faithfully reflect performance under the most current conditions” — stated in the paper.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures










Verification
Claims the paper makes
22 Valid · 0 Not valid · 13 Not checked · 63 Own results and interpretations
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“Existing speech deepfake detection research focuses on developing machine learning systems in the context of voice authentication [7–9].”
§ Warning: Humans cannot reliably detect speech deepfakes > Introduction
What the sources say
- Not checked — no full text of the sourcecites [7] Z Wu, 2017, ASVspoof: The Automatic Speaker Verification Spoofing and Countermeasures Challenge, IEEE Journal of Selected Topics in Signal Processing source
- Supportedcites [8] A Nautsch, 2021, ASVspoof 2019: Spoofing Countermeasures for the Detection of Synthesized, Converted and Replayed Speech, IEEE Transactions on Biometrics, Behavior, and Identity Science source
The Introduction (p. 1) describes research on automatic spoofing-detection systems intended to protect automatic speaker verification, a voice-authentication technology. The Conclusions (p. 11) report a shift toward deep architectures and ensemble systems, supporting the claim's characterization of this research as developing machine-learning systems. (LLM, conf=0.88)
- Supportedcites [9] Yamagishi J, Wang X, Todisco M, Sahidullah M, Patino J, Nautsch A, et al. ASVspoof 2021: accelerating progress in spoofed and deepfake speech detection. arXiv preprint arXiv:210900537. 2021 Sep 1 [Cited 2023 Jan 19]. source
The Abstract (p. 1) describes a challenge devoted to developing countermeasures that protect automatic speaker verification systems and includes a speech-deepfake detection task. The Introduction (p. 1) likewise places detection-system development in the speaker-verification context, while the CM baselines section (p. 4) describes trained detection models. (LLM, conf=0.86)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Multiple studies deem other uses of speech deepfakes as more concerning, such as misleading people through voice impersonations [5, 6].”
§ Warning: Humans cannot reliably detect speech deepfakes > Introduction
What the sources say
- Supportedcites [5] M Caldwell, 2020, AI-enabled future crime, Crime Science source
The Introduction describes deceptive deepfake impersonations, including phone calls used to gain access to secure systems, and reports that audio/video impersonation ranked as the most concerning crime considered. The paper rates audio and video impersonation together rather than speech alone, but it supports the cited example of concerning voice impersonation. (LLM, conf=0.87)
- Partly supportedcites [6] Y Mirsky, 2022, The threat of offensive ai to organizations, Computers & Security sourceExperimental — accuracy not yet measured
The ‘Impersonation (Identity Theft)’ section describes deepfakes that reenact a victim’s voice for scams and phishing. ‘From the Literature Survey’ ranks AI-enabled impersonation among the highest threats, but compares it with other AI threats, not specifically with other uses of speech deepfakes. The paper therefore supports the concern about misleading voice impersonation, but not the claim’s narrower comparison. (LLM, conf=0.82)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“crowd performance is on par with the in-domain detector.”
§ Warning: Humans cannot reliably detect speech deepfakes > Results > Comparing human performance to automated detectors
What the sources say
- Supportedcites [24] M Groh, 2021, Deepfake detection by human crowds, machines, and machine-informed crowds, Proceedings of the National Academy of Sciences source
The Results (Experiment 2, page not provided) explicitly say the crowd mean was on par with the leading model on sampled holdout videos. The claim is supported for that comparison; the passage does not present it as a result across all videos. (LLM, conf=0.94)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“The ASVspoof 2021 challenge used LFCC-LCNNs as baseline models for spoof detection [9].”
§ Warning: Humans cannot reliably detect speech deepfakes > Materials and methods > Benchmarking against automated deepfake detectors
What the sources say
- Supportedcites [9] Yamagishi J, Wang X, Todisco M, Sahidullah M, Patino J, Nautsch A, et al. ASVspoof 2021: accelerating progress in spoofed and deepfake speech detection. arXiv preprint arXiv:210900537. 2021 Sep 1 [Cited 2023 Jan 19]. source
Section 4.2 (p. 4) identifies an LFCC-LCNN system as baseline B3 among the four countermeasure baselines used for ASVspoof 2021. (LLM, conf=0.95)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“There is no improvement when evaluating videos either [24–26].”
§ Warning: Humans cannot reliably detect speech deepfakes > Background > Related work on human deepfake detection capabilities
What the sources say
- Partly supportedcites [24] M Groh, 2021, Deepfake detection by human crowds, machines, and machine-informed crowds, Proceedings of the National Academy of Sciences sourceExperimental — accuracy not yet measured
The Results for Experiment 1 report no evidence that participants improved at detecting deepfakes over the first 10 videos they saw (P = 0.112), and the Discussion repeats that conclusion. The claim turns that limited, evidence-based finding into an unqualified statement that there is no improvement when evaluating videos. (LLM, conf=0.85)
- Partly supportedcites [25] NC Köbis, 2021, Fooled twice: People cannot detect deepfakes but think they can, iScience sourceExperimental — accuracy not yet measured
The Results section, “Interventions to increase accuracy” (page unavailable), reports that raising awareness and offering financial incentives did not significantly improve participants’ accuracy when classifying deepfake videos. That supports a limited no-improvement finding, but not the broader claim that evaluating videos yields no improvement under other conditions or over time. (LLM, conf=0.84)
- Not checked — no full text of the sourcecites [26] Tahir R, Batool B, Jamshed H, Jameel M, Anwar M, Ahmed F, et al. Seeing is believing: Exploring perceptual differences in deepfake videos. In: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems; 2021. p. 1–16. source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
39 more claims · 2 with their cited sources checked
Create a free account to see all 98 claims and every source verdict.
- Probably not a substantial claim. 41
- It may state two separate claims as one. 18
- The quote only partly states this claim. 3
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“A study [26] found that humans could correctly identify deepfake audio with an accuracy of only 73%.”
From Audio Deepfake Detection: What Has Been Achieved and What Lies Ahead (2025) · cites this paper as [26]
Experimental — accuracy not yet measured
Citations
Cites
SciPy 1.0: fundamental algorithms for scientific computing in Python
Virtanen, P., Gommers, R., Oliphant, T. E., Haberland, M., Reddy, T., Cournapeau, D., Burovski, E., Peterson, P., Weckesser, W., Bright, J., et al. (2020) · Nature Methods · doi:10.1038/s41592-019-0686-2 · Cited by 38,430 · cited 1×
“We completed the analysis using the SciPy [45] and statsmodels [46] Python packages.”
MethodSciPy 1.0: fundamental algorithms for scientific computing in Python
Virtanen, P., Gommers, R., Oliphant, T. E., Haberland, M., Reddy, T., Cournapeau, D., Burovski, E., Peterson, P., Weckesser, W., Bright, J., et al. (2019) · Apollo - University of Cambridge Repository · doi:10.17863/cam.64134 · Cited by 10,460
Statsmodels: Econometric and Statistical Modeling with Python
Seabold, S., & Perktold, J. (2010) · Proceedings of the Python in Science Conference · doi:10.25080/majora-92bf1922-011 · Cited by 6,465 · cited 1×
“We completed the analysis using the SciPy [45] and statsmodels [46] Python packages.”
MethodAnalyzing and Improving the Image Quality of StyleGAN
Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J., & Aila, T. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00813 · Cited by 6,332 · cited 2×
“Modality Year Author Deepfake stimuli Image 2021 Nightingale & Farid [23] Faces generated using StyleGAN2 [31] 2022 Bray et al.”
MethodFaceForensics++: Learning to Detect Manipulated Facial Images
Rossler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., & Niessner, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00009 · Cited by 3,314
Proceedings of The 32nd International Conference on Machine Learning
Germain, M., Gregor, K., Murray, I., & Larochelle, H. (2015) · Cited by 2,020
Celeb-DF: A Large-Scale Challenging Dataset for DeepFake Forensics
Li, Y., Yang, X., Sun, P., Qi, H., & Lyu, S. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00327 · Cited by 1,846 · cited 2×
“[26] Face-swap videos from Celeb-DF [33], FaceForensics++ [34] and DeepFaceLab [35] 2022 Appel & Prietzel [21] Face-swap videos of politicians”
Method(2018) · doi:10.21437/interspeech.2018-1456 · Cited by 1,366
Leskovec, J., Rajaraman, A., & Ullman, J. D. (2014) · Cambridge University Press eBooks · doi:10.1017/cbo9781139924801 · Cited by 1,069
Proceedings of Interspeech 2013
Lu, L., Ghoshal, A. K., & Renals, S. J. (2013) · Proceedings - International Symposium on Computer Architecture/Proceedings · Cited by 1,003
Research works printed but not linked: 10
- I Goodfellow, 2016, Deep Learning
- N Schick, 2020, Deep Fakes and the Infocalypse: What You Urgently Need To Know
- Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, et al. Generative Adversarial Nets. In: Advances in Neural Information Processing Systems; 2014. Available from: https://proceedings.neurips.cc/paper_files/paper/2014/file/5ca3e9b122f61f8f06494c97b1afccf3-Paper.pdf
- Geirhos T, Zimmerman RS, Bilodeau BL, Brendal W, Kim, B. Don’t trust your eyes: on the (un)reliability of feature visualizations. arXiv preprint arXiv: 230604719. 2023 Jun 13 [Cited 2023 Jun 13].
- Watson G, Khanjani Z, Janeja VP. Audio Deepfake Perceptions in College Going Populations. arXiv preprint arXiv:211203351. 2021 Dec 6 [Cited 2023 Jan 19].
- Dolhansky B, Bitton J, Pflaum B, Lu J, Howes R, Wang M, et al. The Deepfake Detection Challenge Dataset. arXiv preprint arXiv:200607397. [Cited 2023 Mar 27]. Dataset.
- K Kumar, 2019, Proceedings of Advances in Neural Information Processing Systems
- Kim J, Kong J, Son J. Conditional variational autoencoder with adversarial learning for end-to-end text-to-speech. In: International Conference on Machine Learning. PMLR; 2021. p. 5530–5540. Available from: https://proceedings.mlr.press/v139/kim21f.html.
- Ma H, Yi J, Wang C, Yan X, Tao J, Wang T, et al. FAD: A Chinese Dataset for Fake Audio Detection. arXiv preprint arXiv:220712308. 2022 Jul 12. [Cited 2023 Jan 19].
- Chesney R, Citron D. Deepfakes and the new disinformation war: The coming age of post-truth geopolitics. Foreign Affairs. 2019 Jan/Feb;98:147.
Cited by
Creative partnerships with generative AI. Possibilities for education and beyond
Creely, E., & Blannin, J. (2024) · Thinking Skills and Creativity · doi:10.1016/j.tsc.2024.101727 · Cited by 145
Human performance in detecting deepfakes: A systematic review and meta-analysis of 56 papers
Diel, A., Lalgi, T., Schröter, I. C., MacDorman, K. F., Teufel, M., & Bäuerle, A. (2024) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2024.100538 · Cited by 64
“…voice banking; Judge & Hayton, 2022), AI-generated voices can be misused to impersonate, commit fraud or scientific misconduct, or spread misinformation (Farid, 2022; Khanjani et al., 2023; Mai, Bray, Davies, & Griffin, 2023; Strupp, 2019; Suwajanakorn, Seitz, & Kemelmacher-Shlizerman, 2017).”
Audio Deepfake Detection: What Has Been Achieved and What Lies Ahead
Zhang, B., Cui, H., Nguyen, V., & Whitty, M. (2025) · Sensors · doi:10.3390/s25071989 · Cited by 72
“A study [26] found that humans could correctly identify deepfake audio with an accuracy of only 73%.”
People are poorly equipped to detect AI-powered voice clones
Barrington, S., Cooper, E. A., & Farid, H. (2025) · Scientific Reports · doi:10.1038/s41598-025-94170-3 · Cited by 131
“This study reveals that, generally speaking, people are poorly equipped to identify AI-generated voice clones, both in terms of identity matching and naturalness.”
Labeling AI-Generated Content: Promises, Perils, and Future Directions
Wittenberg, C., Epstein, Z., Berinsky, A. J., & Rand, D. G. (2024) · An MIT Exploration of Generative AI · doi:10.21428/e4baedd9.0319e3a6 · Cited by 73
Metadata
- Authors
- Mai, K. T., Bray, S., Davies, T., & Griffin, L. D.
- Affiliations
- University College London (GB)
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2023-08-02
- Volume
- 18
- Issue
- 8
- Pages
- e0285333
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC10395974
- OpenAlex
- W4385483249
- Cited by
- 98
- References
- 37
- Citations by year
- 2026: 42 · 2025: 34 · 2024: 18 · 2023: 4
- FWCI
- 9.21
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Funders
- Engineering and Physical Sciences Research Council (EP/R513143/1, EP/S022503/1)
- Dawes Centre for Future Crime
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Keywords
- deepfake detection, synthetic speech, speech synthesis, AI misuse, decision-making rationale
- MeSH terms
- Artificial Intelligence, Humans, Language, Phonetics, Speech, Speech Perception
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-09-30, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-09-30 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Warning: Humans cannot reliably detect speech deepfakes. https://app.cochat.ai/papers/10.1371/journal.pone.0285333
- The paper
- Mai, K. T., Bray, S., Davies, T., & Griffin, L. D. (2023). Warning: Humans cannot reliably detect speech deepfakes. PLOS ONE. https://doi.org/10.1371/journal.pone.0285333
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.