Deepfake detection by human crowds, machines, and machine-informed crowds
Open accessThe DOI matches its Crossref record; 103 of the 103 references it cites were found in a registry or an index; 13 of the 41 sources its claims cite were checked for support.
Abstract
Significance: The recent emergence of deepfake videos raises theoretical and practical questions. Are humans or the leading machine learning model more capable of detecting algorithmic visual manipulations of videos? How should content moderation systems be designed to detect and flag video-based misinformation? We present data showing that ordinary humans perform in the range of the leading machine learning model on a large set of minimal context videos. While we find that a system integrating human and model predictions is more accurate than either humans or the model alone, we show inaccurate model predictions often lead humans to incorrectly update their responses. Finally, we demonstrate that specialized face processing and the ability to consider context may specially equip humans for deepfake detection.
Overview
CoChat Summary
TL;DR
In two online experiments, ordinary people performed near or above the leading 2020 deepfake-detection model on minimal-context DFDC videos, and crowd averages matched or exceeded the model. Combining human and model predictions improved accuracy overall, but incorrect model predictions often led participants to update toward wrong answers.
Key findings
- Experiment 1: 5,524 individuals completed 26,820 trials; among 882 participants who saw at least 10 pairs, 82% outperformed the leading model, which achieved 65% accuracy on the holdout dataset. Experiment 2: recruited participants accurately identified deepfakes in 66% of attempts, nonrecruited participants in 69% of trials (72% for those seeing at least 10 videos), versus the leading model’s 80% on sampled videos; 13% of recruited, 27% of nonrecruited with ≥10 videos, and 37% of nonrecruited with <10 videos outperformed the model.
- Crowd wisdom: the crowd mean accurately identified 74% of videos for recruited participants, 80% for nonrecruited participants, and 86% for the 1,879 nonrecruited participants who saw at least 10 videos, compared with 80% for the leading model. After seeing the model’s prediction, recruited participants’ accurate identification increased from 66% to 73% (P < 0.001), but on 10 videos with incorrect or equivocal model predictions, participants became 2.7% less accurate on average; on political deepfakes the model was confidently wrong (2%, 8%, and 1% deepfake probability) and participants’ accurate identification dropped from 56% to 34% for Kim Jong-un and 70% to 55% for Vladimir Putin.
- Specialized face processing: inversion, misalignment, and eye occlusion reduced human accuracy by 4.3 percentage points (P = 0.002), 6.3 percentage points (P < 0.001), and 4.4 percentage points (P = 0.004) on a base rate of 65.5%; the model was affected only by inversion, dropping 12.1 percentage points (P = 0.005). Anger elicitation had no overall significant effect (P = 0.280), but on real videos the anger group underperformed controls by 5.2 percentage points (P = 0.032).
- Video features: a flickering face increased recruited participants’ accuracy by 24.2 percentage points (P < 0.001) and the model’s by 21.7 percentage points (P = 0.170); two people increased recruited participants’ accuracy by 7.6% (P < 0.001) but decreased the model’s accuracy by 21.9% (P = 0.023) on real videos; dark-skin presence decreased recruited participants’ accuracy by 8.8% (P < 0.001) and the model’s by 12.0% (P = 0.192).
Methods
- Two randomized online experiments on the Detect Fakes website. Experiment 1 used a two-alternative forced-choice design with 56 pairs of videos from the DFDC training dataset (n = 5,524; 26,820 trials; analysis focused on 882 participants who saw at least 10 pairs). Experiment 2 used a single-video design (n = 9,492: 304 recruited from Prolific and 9,188 organic visitors) with 50 videos randomly sampled from the DFDC holdout set (half deepfake, half nonmanipulated), 4 political-leader videos, and a deepfake attention check; participants reported confidence from 50% to 100% in 1% increments and could update after seeing the model’s prediction.
- Embedded randomized interventions tested anger elicitation and obstruction of specialized face processing via inversion, misalignment, and occlusion. Analyses used linear regression with robust SEs clustered on participants unless otherwise stated; accuracy was normalized between 0 and 1, with crowd means, ROC/AUC comparisons, and hand-labeled video-level features.
Limitations
- Paper states: the study evaluated 167 videos (84 deepfake and 83 authentic); only the two political deepfake videos include lip-syncing manipulations; no expert fact-checkers or forensic analysts were recruited, so results generalize only to ordinary people; the comparison is limited to the best model in 2020; minimal-context videos may not resemble the most problematic deepfakes; the half-real/half-deepfake base rate does not reflect real misinformation base rates; and emotion results were mixed, so no firm conclusions are drawn.
- Paper states: future work is needed on persuasive manipulated video, larger samples of political deepfakes, social media context, and additional or integral emotions.
- Evident from text: the nonrecruited participants were not preregistered and arrived organically/self-selected via the public website, while recruited participants came from Prolific.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures


Verification
Claims the paper makes
5 Valid · 1 Not valid · 18 Not checked · 170 Own results and interpretations
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“adults are less accurate at recognizing faces when images are inverted or contain misaligned parts (60–62).”
§ Deepfake detection by human crowds, machines, and machine-informed crowds > Abstract
What the sources say
- Not checked yetcites [60] R. K. Yin, Looking at upside-down faces. J. Exp. Psychol. 81, 141– 145 (1969). source
- Not checked yetcites [61] G. Rhodes, S. Brake, A. P. Atkinson, What’s lost in inverted faces? Cognition 47, 25– 57 (1993). 8482070 source
- Disputedcites [62] J. J. Richler, O. S. Cheung, I. Gauthier, Holistic processing predicts face recognition. Psychol. Sci. 22, 464– 471 (2011). 21393576 PMC3077885 sourceExperimental — accuracy not yet measured
Our copy never reports an inversion cost for face recognition — 'inverted' appears only inside reference titles (Richler, Mack, Palmeri & Gauthier, 2011; Goffaux, 2009; Duchaine & Nakayama, 2006), which is not the paper's own claim. On alignment, the paper states the opposite direction: in the Method > Composite task section it writes that in the partial design 'Accuracy is greater or reaction time (RT) is faster in misaligned trials than in aligned trials,' and in Results and discussion that 'when face parts are misaligned, selective attention to a part is more successful, as evidenced by the smaller congruency effect on misaligned trials.' Its face-recognition measures (CFMT, face-identification task) used intact upright faces varying in lighting and pose, with no misaligned-parts or inverted condition, so the attributed decrement is not what this source says. (LLM, conf=0.65)
Checked by a reasoning model — may be wrong.
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“visual processing of faces appears to proceed holistically for the vast majority of people (64, 65).”
§ Deepfake detection by human crowds, machines, and machine-informed crowds > Abstract
What the sources say
- Partly supportedcites [64] J. J. Richler, I. Gauthier, A meta-analysis and review of holistic face processing. Psychol. Bull. 140, 1281– 1302 (2014). 24956123 PMC4152424 sourceExperimental — accuracy not yet measured
The Holistic Processing section describes face processing as holistic, and the Meta-Analysis Results report an average holistic-processing effect across studies. However, those group-level findings do not establish that the effect occurs in the vast majority of individual people. (LLM, conf=0.82)
- Supportedcites [65] A. W. Young, A. M. Burton, Are we face experts? Trends Cogn. Sci. 22, 100– 110 (2018). 29254899 source
In “The facial expertise debate” (page not provided), the paper calls holistic processing a well-known property of face recognition. Its “Holistic processing” entry defines this as processing faces as a whole. Although it gives no percentage, these passages support the citing sentence’s broad, hedged description of typical face processing. (LLM, conf=0.83)
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Likewise, these results mirror other recent research revealing human–AI collaborative decision-making does not necessarily lead to more accurate results than either humans or AI alone (72–76).”
§ Deepfake detection by human crowds, machines, and machine-informed crowds > Discussion
What the sources say
- Not checked yetcites [72] M. Vaccaro, J. Waldo, The effects of mixing machine learning and human judgment. Commun. ACM 62, 104– 110 (2019). source
- Not checked yetcites [73] P. Tschandl., Human-computer collaboration for skin cancer recognition. Nat. Med. 26, 1229– 1234 (2020). 32572267 source
- Partly supportedcites [74] S. Gaube., Do as AI say: Susceptibility in deployment of clinical decision-aids. NPJ Digit. Med. 4, 31 (2021). 33608629 PMC7896064 sourceExperimental — accuracy not yet measured
Our copy supports the 'collaboration does not necessarily yield better accuracy' half: the Results section 'Diagnostic accuracy is similarly impacted by human/AI advice' reports accuracy was worse with inaccurate advice regardless of purported source, and the Discussion states 'physicians were not able to effectively filter inaccurate advice'. However, the comparison 'than either humans or AI alone' is not tested here — the Methods 'Experimental design' describes a 2 × 2 design varying only advice source (AI vs human label) and advice accuracy, with no unaided-physician or AI-alone condition, and the Discussion instead says outcomes 'may improve overall' only 'if the underlying model has a higher diagnostic accuracy than the physicians using it'. The related statement that implemented systems 'showed no improved clinical outcomes' (Introduction) is itself credited to other works (refs 6,7) rather than shown by this study. (LLM, conf=0.78)
- Partly supportedcites [75] A. Abeliuk, D. M. Benjamin, F. Morstatter, A. Galstyan, Quantifying machine influence over human forecasters. Sci. Rep. 10, 15940 (2020). 32994447 PMC7524768 sourceExperimental — accuracy not yet measured
The Discussion (para. on implications) states that 'algorithms are far from perfect and do not necessarily beat humans in some deeply uncertain settings' and that 'overconfidence on algorithms may lead to detrimental results', and the Impact on performance / Discussion sections report that confirmation bias and trust in low-quality models decrease forecast accuracy ('users would do better to trust their prior'), so the source does support the idea that pairing humans with machine models does not necessarily beat humans alone. However, the paper never compares the hybrid against the machine/AI component alone; its own headline conclusion is the more positive 'machine models can improve the judgment of a human pool but highlight the importance of accounting for trust and cognitive biases' (Abstract), and the Machine predictions section frames machines as under-performing humans rather than the team being worse than machines. The 'or AI alone' half of the claim is therefore not supported by this source. (LLM, conf=0.75)
- Supportedcites [76] M. Jacobs., How machine-learning recommendations influence clinician treatment selections: The example of the antidepressant selection. Transl. Psychiatry 11, 108 (2021). 33542191 PMC7862671 source
The Results section 'No change in overall accuracy between clinicians and clinician-ML collaboration' (p. Results) reports clinician-ML collaborative accuracy (M = 0.356) statistically indistinguishable from clinicians alone (M = 0.357; p = 0.925) and significantly below the simulated ML system acting alone (0.667; p < 0.0001). The Discussion restates this: 'interacting with ML recommendations did not improve treatment selection accuracy' and 'Our work challenges the validity of this assumption' that humans interacting with ML tools will perform better than either actor individually. This matches the cited sentence's paraphrase that human-AI collaboration does not necessarily yield more accurate results than either humans or AI alone. (LLM, conf=0.90)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Eighty-two percent of participants outperform the leading model, which achieves 65% accuracy on the holdout dataset (36).”
§ Deepfake detection by human crowds, machines, and machine-informed crowds > Results > Experiment 1: Two-Alternative Forced Choice (n = 5,524).
What the sources say
- Not checked yetcites [36] B. Dolhansky., Deepfake detection challenge results: An open initiative to advance AI (2020). Accessed 27 January 2021.
plainn = 882- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Half of the stimuli set (28 of 56 pairs of videos) was identified correctly by over 83% of participants”
§ Deepfake detection by human crowds, machines, and machine-informed crowds > Results > Experiment 1: Two-Alternative Forced Choice (n = 5,524).
plainn = 882- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
126 more claims
Create a free account to see all 194 claims and to verify the 28 references still open.
- It may state two separate claims as one. 34
- Probably not a substantial claim. 31
- The quote only partly states this claim. 2
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“There is no improvement when evaluating videos either [24–26].”
From Warning: Humans cannot reliably detect speech deepfakes (2023) · cites this paper as [24]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“On a sample of 166 videos from the largest publicly available dataset of deepfake videos to date from the Deepfake Detection Contest (DFDC)48, people are significantly better than chance but far from perfect at discerning whether an unknown actor’s face has been visually manipulated by a deepfake algorithm49.”
From Human detection of political speech deepfakes across transcripts, audio, and video (2024) · cites this paper as [49]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Furthermore, Groh et al. [2] point out that the brain possesses a specialized region for processing faces holistically. In their research, humans were adversely a ff ected by manipulations obstructing specialized face processing (e.g., inversion, misalignment, and eye occlusion), whereas computer performance remained una ff ected.”
From Human Performance in Deepfake Detection: A Systematic Review (2025) · cites this paper as [2]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Groh et al. [2] compared human performance with the winning model using videos from DFDC and found that, among the participants who saw at least 10 videos, 82% outperformed the winning model.”
From Human Performance in Deepfake Detection: A Systematic Review (2025) · cites this paper as [2]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Second, while lower-quality deepfakes are generally easier to detect [20, 35, 36, 42], imperfections in authentic videos (e.g., lighting inconsistencies or unusual framing) can lead viewers to miscategorize genuine content as manipulated [2, 22, 27, 35, 37].”
From Human Performance in Deepfake Detection: A Systematic Review (2025) · cites this paper as [2]
Experimental — accuracy not yet measured
12 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
§ Deepfake detection by human crowds, machines, and machine-informed crowds > Results > Experiment 2: Single-Video Design (n = 9,492).
“For nonrecruited participants, the crowd mean accurately identifies 80% of videos.”
§ Deepfake detection by human crowds, machines, and machine-informed crowds > Results > Experiment 2: Single-Video Design (n = 9,492).
“the crowd wisdom of nonrecruited participants and the model is correct on 10 videos. When we examine the 36 videos that are neither grainy, blurry, nor extremely dark, the crowd wisdom of recruited participants is correct on 29 out of 36 videos, the crowd wisdom of nonrecruited participants is correct on 32 out of 36 videos”
Citations
Cites
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
Tan, M., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1905.11946 · Cited by 17,072 · cited 2×
“While there are many proposed techniques for algorithmically detecting fakes (including affective computing approaches like examining heart rate and breathing rate (37) and looking for emotion-congruent speech and facial expressions) (38, 39), the most accurate computer vision model in the contest (40) focused on loca…”
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Dermatologist-level classification of skin cancer with deep neural networks
Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017) · Nature · doi:10.1038/nature21056 · Cited by 14,362
A Style-Based Generator Architecture for Generative Adversarial Networks
Karras, T., Laine, S., & Aila, T. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00453 · Cited by 11,714 · cited 3×
“The deepfakes examined here contain only visual manipulations produced using seven synthetic techniques: two deepfake autoencoders, a neural network face swap model 32 , the NTH talking heads model 33 , the FSGAN method for reenactment and inpainting 34 , StyleGAN 35 , and sharpening refinement on blended faces 31 .”
BackgroundMethodMastering the game of Go without human knowledge
Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., et al. (2017) · Nature · doi:10.1038/nature24270 · Cited by 9,084 · cited 1×
“Recent advances in training neural networks for computer vision reveal that algorithms are capable of surpassing the performance of human experts in some complex strategy games (5, 6) and medical diagnoses (7, 8), so we might expect algorithms to be similarly capable of outperforming people at deepfake detection.”
The spread of true and false news online
Vosoughi, S., Roy, D., & Aral, S. (2018) · Science · doi:10.1126/science.aap9559 · Cited by 8,847 · cited 1×
“The role of emotion in deepfake detection is of practical concern because people share misinformation, especially political misinformation, because of its novelty and emotional content (55).”
The fusiform face area: a module in human extrastriate cortex specialized for face perception
Kanwisher, N., McDermott, J., & Chun, M. M. (1997) · The Journal of Neuroscience · doi:10.1523/jneurosci.17-11-04302.1997 · Cited by 7,738 · cited 1×
“For example, there is a region of the brain specialized for processing faces (57).”
Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks
Zhang, K., Zhang, Z., Li, Z., & Qiao, Y. (2016) · IEEE Signal Processing Letters · doi:10.1109/lsp.2016.2603342 · Cited by 6,214
Lazer, D. M. J., Baum, M. A., Benkler, Y., Berinsky, A. J., Greenhill, K. M., Menczer, F., Metzger, M. J., Nyhan, B., Pennycook, G., Rothschild, D., et al. (2018) · Science · doi:10.1126/science.aao2998 · Cited by 4,160
(2018) · Cited by 3,510
Prolific.ac—A subject pool for online experiments
Palan, S., & Schitter, C. (2018) · Journal of Behavioral and Experimental Finance · doi:10.1016/j.jbef.2017.12.004 · Cited by 3,450
FaceForensics++: Learning to Detect Manipulated Facial Images
Rossler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., & Niessner, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00009 · Cited by 3,314
International evaluation of an AI system for breast cancer screening
McKinney, S. M., Sieniek, M., Godbole, V., Godwin, J., Antropova, N., Ashrafian, H., Back, T., Chesus, M., Corrado, G. S., Darzi, A., et al. (2020) · Nature · doi:10.1038/s41586-019-1799-6 · Cited by 3,287 · cited 1×
“Recent advances in training neural networks for computer vision reveal that algorithms are capable of surpassing the performance of human experts in some complex strategy games (5, 6) and medical diagnoses (7, 8), so we might expect algorithms to be similarly capable of outperforming people at deepfake detection.”
Albumentations: fast and flexible image augmentations
Buslaev, A., Iglovikov, V. I., Khvedchenya, E., Parinov, A., Druzhinin, M., & Kalinin, A. A. (2020) · Information · doi:10.3390/info11020125 · Cited by 2,714
Yin, R. K. (1969) · Journal of Experimental Psychology · doi:10.1037/h0027474 · Cited by 2,624
Celeb-DF: A Large-Scale Challenging Dataset for DeepFake Forensics
Li, Y., Yang, X., Sun, P., Qi, H., & Lyu, S. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00327 · Cited by 1,846
When Should You Adjust Standard Errors for Clustering?
Abadie, A., Athey, S., Imbens, G., & Wooldridge, J. (2017) · National Bureau of Economic Research · doi:10.3386/w24003 · Cited by 1,360
Pennycook, G., & Rand, D. G. (2021) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2021.02.007 · Cited by 1,360 · cited 1×
“Moreover, metadata from an individual’s decision to share on social media may not be a reliable predictor of media’s veracity because social media tends to focus people’s attention on factors other than truth and accuracy (17, 18).”
Deepfakes and beyond: A Survey of face manipulation and fake detection
Tolosana, R., Vera-Rodriguez, R., Fierrez, J., Morales, A., & Ortega-Garcia, J. (2020) · Information Fusion · doi:10.1016/j.inffus.2020.06.014 · Cited by 1,237
Exposing Deep Fakes Using Inconsistent Head Poses
Yang, X., Li, Y., & Lyu, S. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8683164 · Cited by 1,196
Visual Following and Pattern Discrimination of Face-like Stimuli by Newborn Infants
Goren, C. C., Sarty, M., & Wu, P. Y. K. (1975) · Pediatrics · doi:10.1542/peds.56.4.544 · Cited by 1,158 · cited 1×
“Human infants show sensitivity to faces even before being exposed to them (58, 59), and adults are less accurate at recognizing faces when images are inverted or contain misaligned parts (60–62).”
BackgroundShifting attention to accuracy can reduce misinformation online
Pennycook, G., Epstein, Z., Mosleh, M., Arechar, A. A., Eckles, D., & Rand, D. G. (2021) · Nature · doi:10.1038/s41586-021-03344-2 · Cited by 1,155
Self-citationGALTON, F. (1907) · Nature · doi:10.1038/075450a0 · Cited by 1,080
Synthesizing Obama: learning lip sync from audio
Suwajanakorn, S., Seitz, S. M., & Kemelmacher-Shlizerman, I. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073640 · Cited by 1,057
Human-computer collaboration for skin cancer recognition
Tschandl, P., Rinner, C., Apalla, Z., Argenziano, G., Codella, N., Halpern, A., Janda, M., Lallas, A., Longo, C., Malvehy, J., et al. (2020) · Nature Medicine · doi:10.1038/s41591-020-0942-0 · Cited by 1,024
Handbook of Theories of Social Psychology
Van Lange, P., Kruglanski, A., & Higgins, E. (2012) · SAGE Publications Ltd eBooks · doi:10.4135/9781446249222 · Cited by 982
Media Forensics and DeepFakes: An Overview
Verdoliva, L. (2020) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2020.3002101 · Cited by 979
Fighting misinformation on social media using crowdsourced judgments of news source quality
Pennycook, G., & Rand, D. G. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1806781116 · Cited by 947
A Lip Sync Expert Is All You Need for Speech to Lip Generation In the Wild
Prajwal, K. R., Mukhopadhyay, R., Namboodiri, V. P., & Jawahar, C. V. (2020) · Proceedings of the 28th ACM International Conference on Multimedia · doi:10.1145/3394171.3413532 · Cited by 897
Deep Fakes: A Looming Challenge for Privacy, Democracy, and National Security
Chesney, R., & Citron, D. K. (2018) · SSRN Electronic Journal · doi:10.2139/ssrn.3213954 · Cited by 880
Feelings-as-Information Theory
Schwarz, N. (2012) · Handbook of Theories of Social Psychology: Volume 1 · doi:10.4135/9781446249215.n15 · Cited by 864
The Creation and Detection of Deepfakes: A Survey
Mirsky, Y., & Lee, W. (2022) · ACM Computing Surveys · doi:10.1145/3425780 · Cited by 863 · cited 1×
“As a consequence, society will require more than just visual-based classification algorithms to protect against the potentially harmful threats that deepfakes pose 27 .”
BackgroundThe Creation and Detection of Deepfakes: A Survey
Mirsky, Y., & Lee, W. (2020) · arXiv · doi:10.48550/arxiv.2004.11138 · Cited by 863
FSGAN: Subject Agnostic Face Swapping and Reenactment
Nirkin, Y., Keller, Y., & Hassner, T. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00728 · Cited by 706
Few-Shot Adversarial Learning of Realistic Neural Talking Head Models
Zakharov, E., Shysheya, A., Burkov, E., & Lempitsky, V. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00955 · Cited by 687
Face Recognition by Humans: Nineteen Results All Computer Vision Researchers Should Know About
Sinha, P., Balas, B., Ostrovsky, Y., & Russell, R. (2006) · Proceedings of the IEEE · doi:10.1109/jproc.2006.884093 · Cited by 674 · cited 1×
“Research in visual neuroscience and perceptual psychology has shown that the human visual system is equipped with mechanisms dedicated for face perception (56).”
DeeperForensics-1.0: A Large-Scale Dataset for Real-World Face Forgery Detection
Jiang, L., Li, R., Wu, W., Qian, C., & Loy, C. C. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00296 · Cited by 599
Exposure to untrustworthy websites in the 2016 US election
Guess, A. M., Nyhan, B., & Reifler, J. (2020) · Nature Human Behaviour · doi:10.1038/s41562-020-0833-x · Cited by 587
Do as AI say: susceptibility in deployment of clinical decision-aids
Gaube, S., Suresh, H., Raue, M., Merritt, A., Berkowitz, S. J., Lermer, E., Coughlin, J. F., Guttag, J. V., Colak, E., & Ghassemi, M. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00385-9 · Cited by 543
Evaluating the fake news problem at the scale of the information ecosystem
Allen, J., Howland, B., Mobius, M., Rothschild, D., & Watts, D. J. (2020) · Science Advances · doi:10.1126/sciadv.aay3539 · Cited by 528
DeepFakes: a New Threat to Face Recognition? Assessment and Detection
Korshunov, P., & Marcel, S. (2018) · arXiv · doi:10.48550/arxiv.1812.08685 · Cited by 497
What's lost in inverted faces?
Rhodes, G., Brake, S., & Atkinson, A. P. (1993) · Cognition · doi:10.1016/0010-0277(93)90061-y · Cited by 487
Reliance on emotion promotes belief in fake news
Martel, C., Pennycook, G., & Rand, D. G. (2020) · Cognitive Research: Principles and Implications · doi:10.1186/s41235-020-00252-3 · Cited by 470
Face recognition accuracy of forensic examiners, superrecognizers, and face recognition algorithms
Phillips, P. J., Yates, A. N., Hu, Y., Hahn, C. A., Noyes, E., Jackson, K., Cavazos, J. G., Jeckeln, G., Ranjan, R., Sankaranarayanan, S., et al. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1721355115 · Cited by 461 · cited 1×
“Similarly, face recognition algorithms often outperform forensic examiners (who are significantly better than ordinary people) at identifying whether pairs of face images show the same or different people (13).”
The deepfake detection challenge (dfdc) dataset
Dolhansky, B., Bitton, J., Pflaum, B., Lu, J., Howes, R., Wang, M., & Ferrer, C. C. (2020) · arXiv · doi:10.48550/arxiv.2006.07397 · Cited by 427 · cited 4×
“…two deepfake autoen-coders, a neural network face swap model (32), the neural talking heads (NTH) model (33), the face swapping generative adversarial network (FSGAN) for reenactment and inpainting (34), StyleGAN for generating synthetic faces (35), and sharpening refinement on blended faces (31).”
InfluentialThe Human Fetus Preferentially Engages with Face-like Visual Stimuli
Reid, V. M., Dunn, K., Young, R. J., Amu, J., Donovan, T., & Reissland, N. (2017) · Current Biology · doi:10.1016/j.cub.2017.05.044 · Cited by 421
(2019) · Cited by 415
The Deepfake Detection Challenge (DFDC) Preview Dataset
Dolhansky, B., Howes, R., Pflaum, B., Baram, N., & Ferrer, C. C. (2019) · arXiv · doi:10.48550/arxiv.1910.08854 · Cited by 413
Detection of GAN-Generated Fake Images over Social Networks
Marra, F., Gragnaniello, D., Cozzolino, D., & Verdoliva, L. (2018) · 2018 IEEE Conference on Multimedia Information Processing and Retrieval (MIPR) · doi:10.1109/mipr.2018.00084 · Cited by 399
Affective feelings as feedback: Some cognitive consequences.
Clore, G. L., Wyer, R. S., Dienes, B. P. A., Gasper, K., Gohm, C. L., & Isbell, L. M. (2001) · doi:10.4324/9781410605733-7 · Cited by 365
Holistic Processing Predicts Face Recognition
Richler, J. J., Cheung, O. S., & Gauthier, I. (2011) · Psychological Science · doi:10.1177/0956797611401753 · Cited by 341 · cited 1×
“Human infants show sensitivity to faces even before being exposed to them (58, 59), and adults are less accurate at recognizing faces when images are inverted or contain misaligned parts (60–62).”
A meta-analysis and review of holistic face processing.
Richler, J. J., & Gauthier, I. (2014) · Psychological Bulletin · doi:10.1037/a0037004 · Cited by 340
FaceForensics: A Large-scale Video Dataset for Forgery Detection in Human Faces
Rössler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., & Nießner, M. (2018) · arXiv · doi:10.48550/arxiv.1803.09179 · Cited by 339
Emotions Don't Lie: An Audio-Visual Deepfake Detection Method using Affective Cues
Mittal, T., Bhattacharya, U., Chandra, R., Bera, A., & Manocha, D. (2020) · Proceedings of the 28th ACM International Conference on Multimedia · doi:10.1145/3394171.3413570 · Cited by 319 · cited 1×
“While there are many proposed techniques for algorithmically detecting fakes (including affective computing approaches like examining heart rate and breathing rate (37) and looking for emotion-congruent speech and facial expressions) (38, 39), the most accurate computer vision model in the contest (40) focused on loca…”
MethodA solution to the single-question crowd wisdom problem
Prelec, D., Seung, H. S., & McCoy, J. (2017) · Nature · doi:10.1038/nature21054 · Cited by 314
Brashier, N. M., & Marsh, E. J. (2020) · Annual Review of Psychology · doi:10.1146/annurev-psych-010419-050807 · Cited by 288
DeepRhythm: Exposing DeepFakes with Attentional Visual Heartbeat Rhythms
Qi, H., Guo, Q., Juefei-Xu, F., Xie, X., Ma, L., Feng, W., Liu, Y., & Zhao, J. (2020) · arXiv · doi:10.48550/arxiv.2006.07634 · Cited by 280
Do (microtargeted) deepfakes have real effects on political attitudes?
Dobber, T., Metoui, N., Trilling, D., Helberger, N., & de Vreese, C. (2021) · The International Journal of Press/Politics · doi:10.1177/1940161220944364 · Cited by 256
On being happy and gullible: Mood effects on skepticism and the detection of deception
Forgas, J. P., & East, R. (2008) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2008.04.010 · Cited by 247 · cited 1×
“Recent research in social psychology suggests that negative emotions can reduce gullibility (50, 51), which could perhaps improve individual’s discernment of videos.”
BackgroundJacobs, M., Pradier, M. F., McCoy, T. H., Perlis, R. H., Doshi-Velez, F., & Gajos, K. Z. (2021) · Translational Psychiatry · doi:10.1038/s41398-021-01224-x · Cited by 247
Detecting Deep-Fake Videos from Appearance and Behavior
Agarwal, S., Farid, H., El-Gaaly, T., & Lim, S.-N. (2020) · 2020 IEEE International Workshop on Information Forensics and Security (WIFS) · doi:10.1109/wifs49906.2020.9360904 · Cited by 242 · cited 1×
“While there are many proposed techniques for algorithmically detecting fakes (including affective computing approaches like examining heart rate and breathing rate (37) and looking for emotion-congruent speech and facial expressions) (38, 39), the most accurate computer vision model in the contest (40) focused on loca…”
MethodYoung, A. W., & Burton, A. M. (2018) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2017.11.007 · Cited by 235
Deepfake Detection: Current Challenges and Next Steps
Lyu, S. (2020) · 2020 IEEE International Conference on Multimedia & Expo Workshops (ICMEW) · doi:10.1109/icmew46912.2020.9105991 · Cited by 234
Detecting Deep-Fake Videos from Phoneme-Viseme Mismatches
Agarwal, S., Farid, H., Fried, O., & Agrawala, M. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00338 · Cited by 234
Scaling up fact-checking using the wisdom of crowds
Allen, J., Arechar, A. A., Pennycook, G., & Rand, D. G. (2021) · Science Advances · doi:10.1126/sciadv.abf4393 · Cited by 234
Emotional Policy: Personal Sadness and Anger Shape Judgments about a Welfare Case
Small, D. A., & Lerner, J. S. (2008) · Political Psychology · doi:10.1111/j.1467-9221.2008.00621.x · Cited by 222
Chen, P., Liu, S., Zhao, H., Wang, X., & Jia, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.04086 · Cited by 221
DeepRhythm: Exposing DeepFakes with Attentional Visual Heartbeat Rhythms
Qi, H., Guo, Q., Juefei-Xu, F., Xie, X., Ma, L., Feng, W., Liu, Y., & Zhao, J. (2020) · Proceedings of the 28th ACM International Conference on Multimedia · doi:10.1145/3394171.3413707 · Cited by 220 · cited 1×
“We find that manipulations designed to disrupt visual processing of faces hinder human participants’ performance while mostly not affecting the model’s performance, suggesting a role for specialized cognitive capacities in explaining human deepfake detection performance.”
(2020) · arXiv (Cornell University) · Cited by 204
Performance vs. competence in human–machine comparisons
Firestone, C. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1905334117 · Cited by 150 · cited 1×
“The lack of additional context creates a level playing field for a reasonably fair comparison of human and machine vision, because humans cannot also reason about contextual, conceptual clues in these videos (70).”
Self-citationEstimating the reliability of eyewitness identifications from police lineups
Wixted, J. T., Mickes, L., Dunn, J. C., Clark, S. E., & Wells, W. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1516814112 · Cited by 147
Can people identify original and manipulated photos of real-world scenes?
Nightingale, S. J., Wade, K. A., & Watson, D. G. (2017) · Cognitive Research: Principles and Implications · doi:10.1186/s41235-017-0067-2 · Cited by 135
Theories of mood and cognition: a user's handbook
(2001) · Choice Reviews Online · doi:10.5860/choice.39-0614 · Cited by 123
Measuring the news and its impact on democracy
Watts, D. J., Rothschild, D. M., & Mobius, M. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1912443118 · Cited by 121 · cited 1×
“In 2021, less than a fraction of a percent of news was misinformation (85).”
The (minimal) persuasive advantage of political video over text
Wittenberg, C., Tappin, B. M., Berinsky, A. J., & Rand, D. G. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2114388118 · Cited by 114
Celeb-DF: A Large-scale Challenging Dataset for DeepFake ForensicsPreprint
Li, Y., Yang, X., Sun, P., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.12962 · Cited by 104
Paris, B. S., & Donovan, J. (2019) · Cited by 74
Epstein, Z., Pennycook, G., & Rand, D. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376232 · Cited by 67 · cited 1×
“We find that manipulations designed to disrupt visual processing of faces hinder human participants’ performance while mostly not affecting the model’s performance, suggesting a role for specialized cognitive capacities in explaining human deepfake detection performance.”
Self-citationImage forensic analyses that elude the human visual system
Farid, H., & Bravo, M. J. (2010) · SPIE Proceedings · doi:10.1117/12.837788 · Cited by 66
Happy Believers and Sad Skeptics? Affective Influences on Gullibility
Forgas, J. P. (2019) · Current Directions in Psychological Science · doi:10.1177/0963721419834543 · Cited by 61 · cited 1×
“Moreover, priming people with emotion has been demonstrated to both increase and decrease people’s gullibility depending on the category of emotion 53 and hinder people’s ability to discern real from fake news 54 .”
BackgroundScaling Up Fact-Checking Using the Wisdom of CrowdsPreprint
Allen, J. N. L., Arechar, A. A., Pennycook, G., et al. (2020) · doi:10.31234/osf.io/9qdza · Cited by 60
The effects of mixing machine learning and human judgment
Vaccaro, M., & Waldo, J. (2019) · Communications of the ACM · doi:10.1145/3359338 · Cited by 49
A visual search advantage for illusory faces in objects
Keys, R. T., Taubert, J., & Wardle, S. G. (2021) · Attention, Perception, & Psychophysics · doi:10.3758/s13414-021-02267-4 · Cited by 48
Facial Action Transfer with Personalized Bilinear Regression
Huang, D., & la Torre, F. D. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33709-3_11 · Cited by 43
Ternovski, J., Kalla, J., & Aronow, P. M. (2021) · doi:10.31219/osf.io/dta97 · Cited by 37
Seeing Is Believing: How People Fail to Identify Fake Images on the Web
Kasra, M., Shen, C., & O'Brien, J. F. (2018) · Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3170427.3188604 · Cited by 36
Farzmahdi, A., Rajaei, K., Ghodrati, M., Ebrahimpour, R., & Khaligh‐Razavi, S. (2016) · Scientific Reports · doi:10.1038/srep25025 · Cited by 34
The Human Fetus Preferentially Engages with Face-like Visual Stimuli
Reid, V. M., Dunn, K., Young, R. J., Amu, J., Donovan, T., & Reissland, N. (2018) · Current Biology · doi:10.1016/j.cub.2018.02.025 · Cited by 31
Seeing Is Believing: How People Fail to Identify Fake Images on the WebPreprint
Kasra, M., Shen, C., & O’Brien, J. F. (2016) · SSRN Electronic Journal · doi:10.2139/ssrn.3051561 · Cited by 30
Evaluating the Fake News Problem at the Scale of the Information EcosystemPreprint
Allen, J. N. L., Howland, B., Möbius, M., Rothschild, D. M., & Watts, D. J. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3502581 · Cited by 30
Epstein, Z., Pennycook, G., & Rand, D. G. (2019) · doi:10.31234/osf.io/z3s5k · Cited by 28
Self-citationHaut, K. G., Wohn, C., Antony, V. N., Goldfarb, A., Welsh, M., Sumanthiran, D., Jang, J.-Z., Ali, M. R., & Hoque, M. E. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.08054 · Cited by 26
The (Minimal) Persuasive Advantage of Political Video over TextPreprint
Wittenberg, C., Tappin, B. M., Berinsky, A. J., et al. (2020) · doi:10.31234/osf.io/r5yun · Cited by 25
Towards Understanding Perceptual Differences between Genuine and Face-Swapped Videos
Wöhler, L., Zembaty, M., Castillo, S., & Magnor, M. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445627 · Cited by 24
Leibowicz, C. R., McGregor, S., & Ovadya, A. (2021) · Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3461702.3462584 · Cited by 23
Human detection of machine-manipulated media
Groh, M., Epstein, Z., Obradovich, N., Cebrian, M., & Rahwan, I. (2021) · Communications of the ACM · doi:10.1145/3445972 · Cited by 22
Self-citationQuantifying machine influence over human forecasters
Abeliuk, A., Benjamin, D. M., Morstatter, F., & Galstyan, A. (2020) · Scientific Reports · doi:10.1038/s41598-020-72690-4 · Cited by 18
Emotions Don't Lie: A Deepfake Detection Method using Audio-Visual Affective Cues
Mittal, T., Bhattacharya, U., Chandra, R., Bera, A., & Manocha, D. (2020) · Cited by 15
Can people detect errors in shadows and reflections?
Nightingale, S. J., Wade, K. A., Farid, H., & Watson, D. G. (2019) · Attention, Perception, & Psychophysics · doi:10.3758/s13414-019-01773-w · Cited by 14
Farid, H. (2019) · doi:10.7551/mitpress/11736.001.0001 · Cited by 11
Leibowicz, C., McGregor, S., & Ovadya, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.06109 · Cited by 8
Anger and Strategic Behavior: A Level-K AnalysisPreprint
Castagnetti, A., & Proto, E. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3687138 · Cited by 6
Identify Experts through Revealed Confidence: Application to Wisdom of CrowdsPreprint
Zhang, Y. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3739192 · Cited by 3
Cited by
When combinations of humans and AI are useful: A systematic review and meta-analysis
Vaccaro, M., Almaatouq, A., & Malone, T. (2024) · Nature Human Behaviour · doi:10.1038/s41562-024-02024-1 · Cited by 570
Deep learning-aided decision support for diagnosis of skin disease across skin tones
Groh, M., Badri, O., Daneshjou, R., Koochek, A., Harris, C., Soenksen, L. R., Doraiswamy, P. M., & Picard, R. (2024) · Nature Medicine · doi:10.1038/s41591-023-02728-3 · Cited by 160
Self-citationAlgorithm-mediated social learning in online social networks
Brady, W. J., Jackson, J. C., Lindström, B., & Crockett, M. J. (2023) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2023.06.008 · Cited by 132
Abbas, F., & Taeihagh, A. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.124260 · Cited by 186
Danry, V., Pataranutaporn, P., Mao, Y., & Maes, P. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580672 · Cited by 110
“a member of a minority group which can lead to over-policing of marginalized communities, or a social media information recommendation system might falsely claim incorrect or misleading news stories to be true which can lead to political instability and social unrest [17, 18, 46].”
Background
Metadata
- Authors
- Groh, M., Epstein, Z., Firestone, C., & Picard, R.
- Affiliations
- MIT Media Lab (US)
- Johns Hopkins University (US)
- Corresponding author
- Groh, Matthew
- Venue
- Proceedings of the National Academy of Sciences
- Publisher
- National Academy of Sciences
- Published
- 2021-12-28
- Volume
- 119
- Issue
- 1
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- PMC
- PMC8740705
- OpenAlex
- W3209972423
- Cited by
- 248
- References
- 100
- Citations by year
- 2026: 64 · 2025: 55 · 2024: 65 · 2023: 43 · 2022: 20 · 2021: 1
- FWCI
- 11.59
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Proceedings of the National Academy of Sciences (publishedVersion, cc-by-nc-nd)
- PubMed Central (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- PhilPapers (PhilPapers Foundation) (submittedVersion)
- arXiv (Cornell University)
- Funders
- MIT Media Lab member companies (NA)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Life Sciences › Neuroscience › Cognitive Neuroscience › Face Recognition and Perception
- Keywords
- deepfake detection, human-machine comparison, face processing
- MeSH terms
- Facial Recognition, Artificial Intelligence, Communication, Humans, Deception, Video Recording, Forensic Sciences, Social Media
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash and gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.