CoChat
Sign Up
Cited by 67Digital Media Forensic DetectionMisinformation and Its ImpactsExplainable Artificial Intelligence (XAI)

Deepfake forensics analysis: An explainable hierarchical ensemble of weakly supervised models

Open access
Silva, Samuel Henrique, Bethany, Mazal, Votto, Alexis Megan, Scarff, Ian Henry, Beebe, Nicole, Najafirad, Peyman·2022·Forensic Science International: Synergy
DOI 10.1016/j.fsisyn.2022.100217PMC PMC8808059
Open this paper in CoChat — free accountOpen PDF

Abstract

Deepfakes have become exponentially more common and sophisticated in recent years, so much so that forensic specialists, policy makers, and the public alike are anxious about their role in spreading disinformation. Recently, the detection and creation of such forgery became a popular research topic, leading to significant growth in publications related to the creation of deepfakes, detection methods, and datasets containing the latest deepfake creation methods. The most successful approaches in identifying and preventing deepfakes are deep learning methods that rely on convolutional neural networks as the backbone for a binary classification task. A convolutional neural network extracts the underlying patterns from the input frames. It feeds these to a binary classification fully connected network, which classifies these patterns as trustworthy or untrustworthy. We claim that this method is not ideal in a scenario in which the generation algorithms constantly evolve since the detection algorithm is not robust enough to detect comparably minor artifacts introduced by the generation algorithms. This work proposes a hierarchical explainable forensics algorithm that incorporates humans in the detection loop. We curate the data through a deep learning detection algorithm and share an explainable decision to humans alongside a set of forensic analyses on the decision region. On the detection side, we propose an attention-based explainable deepfake detection algorithm. We address this generalization issue by implementing an ensemble of standard and attention-based data-augmented detection networks. We use the attention blocks to evaluate the face regions where the model focuses its decision. We simultaneously drop and enlarge the region to push the model to base its decision on more regions of the face, while maintaining a specific focal point for its decision. In this case, we use an ensemble of models to improve the generalization. We also evaluate the models' decision using Grad-CAM explanation to focus on the attention maps. The region uncovered by the explanation layer is cropped and undergoes a series of frequency and statistical analyses that help humans decide if the frame is real or fake. We evaluate our model in one of the most challenging datasets, the DFDC, and achieve an accuracy of 92.4%. We successfully maintain this accuracy in datasets not used in the training process.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.fsisyn.2022.100217

Cites

74 of 74 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Research has yet to extensively explore the ethical implications of deepfake detection.

    “Finally, research has yet to extensively explore the ethical implications of deepfake detection.”

    From Deepfake forensics: a survey of digital forensic methods for multimodal deepfake identification on social media (2024) · cites this paper as Silva et al., 2022

    The Background section on deepfake countermeasures describes detection methods, while the Introduction discusses harms caused by deepfakes. Neither addresses whether research has extensively explored the ethical implications of detecting them. Searches for “ethical implications,” “ethical,” and related terms found no passage making that claim. (LLM, conf=0.75)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

4 of the 78 references couldn’t be matched to a known paper.

Research works printed but not linked: 15

  1. [8]D. P. Kingma, M. Welling, Auto-encoding Variational Bayes, arXiv preprint arXiv:1312.6114. · arXiv
  2. [11]N. Bendre, H. T. Marín, P. Najafirad, Learning from Few Samples: A Survey, arXiv preprint arXiv:2007.15484. · arXiv
  3. [28]Y. Li, X. Yang, B. Wu, S. Lyu, Hiding Faces in Plain Sight: Disrupting ai Face Synthesis with Adversarial Perturbations, arXiv preprint arXiv:1906.09288. · arXiv
  4. [31]B. Dolhansky, J. Bitton, B. Pflaum, J. Lu, R. Howes, M. Wang, C. C. Ferrer, The Deepfake Detection Challenge Dataset, arXiv preprint arXiv:2006.07397. · arXiv
  5. [35]R. Durall, M. Keuper, F.-J. Pfreundt, J. Keuper, Unmasking Deepfakes with Simple Features, arXiv preprint arXiv:1911.00686. · arXiv
  6. [36]Y. Li, S. Lyu, Exposing Deepfake Videos by Detecting Face Warping Artifacts, arXiv preprint arXiv:1811.00656. · arXiv
  7. [45]Korshunov P. Marcel S. Speaker inconsistency detection in tampered video 2018 26th European Signal Processing Conference (EUSIPCO) 2018 IEEE 2375 2379
  8. [57]T. Fernando, C. Fookes, S. Denman, S. Sridharan, Exploiting Human Social Cognition for the Detection of Fake and Fraudulent Faces via Memory Networks, arXiv preprint arXiv:1911.07844. · arXiv
  9. [59]O. de Lima, S. Franklin, S. Basu, B. Karwoski, A. George, Deepfake Detection Using Spatiotemporal Convolutional Networks, arXiv preprint arXiv:2006.14749. · arXiv
  10. [60]R. Wang, F. Juefei-Xu, L. Ma, X. Xie, Y. Huang, J. Wang, Y. Liu, Fakespotter: A Simple yet Robust Baseline for Spotting ai-Synthesized Fake Faces, arXiv preprint arXiv:1909.06122. · arXiv
  11. [63]B. Dolhansky, R. Howes, B. Pflaum, N. Baram, C. C. Ferrer, The Deepfake Detection Challenge (Dfdc) Preview Dataset, arXiv preprint arXiv:1910.08854. · arXiv
  12. [68]Z. Allen-Zhu, Y. Li, Towards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep Learning, arXiv preprint arXiv:2012.09816. · arXiv
  13. [69]J. Deng, J. Guo, Y. Zhou, J. Yu, I. Kotsia, S. Zafeiriou, Retinaface: Single-Stage Dense Face Localisation in the Wild, arXiv preprint arXiv:1905.00641. · arXiv
  14. [75]T. Hu, H. Qi, Q. Huang, Y. Lu, See Better before Looking Closer: Weakly Supervised Data Augmentation Network for Fine-Grained Visual Classification, arXiv preprint arXiv:1901.09891. · arXiv
  15. [78]N. Bendre, K. Desai, P. Najafirad, Show Why the Answer is Correct! towards Explainable Ai Using Compositional Temporal Attention, arXiv preprint arXiv:2105.07141. · arXiv

Cited by

Sorted by citations

Related

Authors
Silva, S. H., Bethany, M., Votto, A. M., Scarff, I. H., Beebe, N., & Najafirad, P.
Affiliations
The University of Texas at San Antonio (US)
Corresponding author
Najafirad, Peyman
ORCID
Venue
Forensic Science International: Synergy
Publisher
Elsevier BV
Published
2022
Volume
4
Pages
100217
ISSN
2589-871X (ISSN-L)
DOI
10.1016/j.fsisyn.2022.100217
PMC
PMC8808059
OpenAlex
W4210595036
Cited by
66
References
78
Citations by year
2026: 13 · 2025: 13 · 2024: 25 · 2023: 9 · 2022: 6
FWCI
4.34
Type
journal article
Language
en
Open access
Licence
Locations
  • Forensic Science International Synergy (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
U.S. Department of Homeland Security (70US0920D70090004)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
  • Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
Keywords
deepfake detection, deepfake forensics, explainable AI, weakly supervised learning, ensemble learning, Grad-CAM, human-in-the-loop decision making, disinformation, data augmentation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review