From Black Boxes to Glass Boxes: Explainable AI for Trustworthy Deepfake Forensics
Open accessThe DOI matches its Crossref record; 66 of the 66 references it cites were found in a registry or an index; 11 of the 32 sources its claims cite were checked for support.
Abstract
Systematic review of explainable deepfake detection methods, categorizing forensic, model-centric, multimodal, and natural-language explanation approaches and emphasizing trustworthy forensics.
Overview
CoChat Summary
## TL;DR
This is a survey paper on explainable AI for deepfake forensics, arguing that high detection accuracy alone is not enough for high-stakes use because detectors must also provide trustworthy, human-understandable explanations. It organizes the field into forensic-analysis methods, model-centric explainability, and multimodal/natural-language explanations, and also reviews evaluation metrics and specialized datasets.
## Key findings
- The paper categorizes explainable deepfake detection into three main approaches: forensic analysis, model-centric methods, and multimodal/natural-language explanations.
- It argues that the field has evolved from physical/algorithmic artifact detection (e.g., PRNU, convolutional traces, residuals) toward model-centric explanations and then to large multimodal models that generate human-readable reasoning.
- For evaluation, it highlights quantitative explainability metrics proposed by Baldassarre et al. that assess visual quality (smoothness, spatial locality, sparsity) and informativeness, and notes that standard deepfake metrics like Accuracy, AUC, and EER do not measure interpretability.
- On data, it notes that newer explainability-oriented datasets such as DDL contain 1.8 million samples, 75 manipulation types, and fine-grained annotations, while ExDDV adds clicks and textual descriptions.
## Methods
- The paper is a systematic review/survey of explainable deepfake detection methods, not an experimental study.
- It synthesizes prior work across forensic traces, post hoc XAI, interpretable architectures, multimodal frameworks, metrics, and datasets.
- Its main analysis is a taxonomy and conceptual comparison of methods, discussing strengths, limitations, and research directions grounded in the cited literature.
## Limitations
- The paper does not report original experiments or new benchmark results; this is evident from the text because it is a review and does not present a study sample or empirical protocol.
- It explicitly states that many existing explanations are still evaluated subjectively or with metrics not tailored to deepfake forensics, and that explanation fidelity remains a concern.
- It also states that generalization to unseen forgery techniques and robustness against anti-forensic/adversarial attacks remain open problems.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
9 Valid · 5 Not valid · 19 Not checked · 58 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“Baldassarre et al. [47] formalized evaluation by proposing metrics such as sparsity and spatial locality, enabling quantitative comparisons of the quality of visual explanations.”
What the sources say
- Supportedcites [47] Baldassarre, F., Debard, Q., Pontiveros, G.F., and Wijaya, T.K. (2022, January 23–27). Quantitative metrics for evaluating explanations of video deepfake detectors. Proceedings of the European Conference on Computer Vision Workshops (ECCVW), Tel Aviv, Israel. source
The Introduction says the paper proposes metrics for sparsity and locality within a quantitative framework for evaluating DeepFake explanations. Section 3.2 says it formulates metrics for evaluating human-centric properties, and the Conclusion says these enable quantitative comparisons of explanation quality. Page numbers are not provided in our copy. (LLM, conf=0.94)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Foundational benchmarks have been instrumental in advancing the accuracy of deepfake detectors.”
What the sources say
- Supportedcites [48] Li, Y., Yang, X., Sun, P., Qi, H., and Lyu, S. (2020, January 13–19). Celeb-DF: A Large-Scale Challenging Dataset for DeepFake Forensics. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA. source
The Introduction (p. 1) calls large-scale DeepFake datasets an enabling factor in developing detection methods. Results and Analysis (p. 7) reports progress in detection performance, supporting the broad claim, while cautioning that high accuracy on earlier datasets may not transfer to more challenging videos. (LLM, conf=0.85)
- Partly supportedcites [49] Dolhansky, B., Bitton, J., Pflaum, B., Lu, J., Howes, R., Wang, M., and Canton-Ferrer, C. (2020). The DeepFake Detection Challenge (DFDC) Dataset. arXiv. sourceExperimental — accuracy not yet measured
The Introduction (p. 1) says deepfake datasets are used to train and evaluate detectors and that the DFDC challenge aims to invigorate detection research. It does not establish that benchmarks have already advanced detector accuracy, as the claim asserts. (LLM, conf=0.84)
- Not checked — no full text of the sourcecites [50] Zi, B., Chang, M., Chen, J., Ma, X., and Jiang, Y. (2020, January 12–16). WildDeepfake: A Challenging Real-World Dataset for Deepfake Detection. Proceedings of the 28th ACM International Conference on Multimedia, Seattle, WA, USA. source
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Recent datasets such as DDL [51] and ExDDV [52] are explicitly designed with explainability in mind, combining binary labels, pixel-level masks, and human-generated textual descriptions.”
What the sources say
- Partly supportedcites [51] Miao, C., Chen, Y., Zhang, Z., Liu, Z., Wang, Z., Wang, S., and Feng, J. (2024). DDL: A Dataset for Interpretable Deepfake Detection and Localization in Real-World Scenarios. arXiv. sourceExperimental — accuracy not yet measured
The Datasets section says DDL provides spatial forgery-region masks and temporal segment labels to support localization and interpretation, consistent with the claim's explainability focus. The Introduction discusses binary classification alongside localization, but our copy does not say DDL includes human-generated textual descriptions. This verdict concerns DDL, not the separately cited ExDDV. (LLM, conf=0.86)
- Partly supportedcites [52] Hondru, V., Hogea, E., Onchis, D.M., and Ionescu, R.T. (2025). ExDDV: A New Dataset for Explainable Deepfake Detection in Video. arXiv. sourceExperimental — accuracy not yet measured
Our copy confirms ExDDV was built for explainability and carries binary (real/fake) labels plus human-written textual explanations: Related Work (p. 2) contrasts prior sets that 'contain the binary label (real or fake)' with ExDDV's 'textual explanations for the artifacts observed by human annotators, along with clicks (points) that indicate artifact locations', and the Abstract/Introduction (pp. 1-2) repeat the manual text-description annotation. However, the localization annotation is clicks — relative pixel coordinates plus timestamps (Section 7, p. 12) — not pixel-level masks; the only masks in the paper are the soft/hard input-masking heuristics the authors derive around a click ROI for training (Section 4.4, p. 6), not dataset annotations. So the 'pixel-level masks' component of the claim is not what ExDDV provides. (LLM, conf=0.82)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Despite significant accuracy improvements in existing detection models, their inherent opacity limits their practical application in high-risk areas such as forensic investigations and news verification.”
What the sources say
- Supportedcites [1] Tolosana, 2020, Deepfakes and beyond: A Survey of face manipulation and fake detection, Inf. Fusion source
The Manipulation Detection section says recent deep-learning methods were applied to improve detection and reports accuracies above 90% for one approach. It also reports near-100% AUC on some first-generation datasets, while cautioning that performance degrades on newer datasets. The attributed claim concerns accuracy improvements, not the citing sentence’s separate assertion about opacity. (LLM, conf=0.86)
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“models such as TruthLens and M2F2-Det perform detection through the process of explanation itself [40,41].”
What the sources say
- Supportedcites [40] Kundu, R., Jia, S., Mohanty, V., Balachandran, A., and Roy-Chowdhury, A.K. (2024). TruthLens: Explainable DeepFake Detection for Face Manipulated and Fully Synthetic Data. arXiv. source
Our copy states exactly this for its own model in §3.1 Problem Setup: the objective is 'to generate a detailed answer ... that determines whether the image is real or manipulated and provides an explanation for the decision', reinforced in §A3 ('For a system whose deployment value is the explanation itself') and the Abstract ('accurate predictions along with detailed, fine-grained reasoning'). M2F2-Det (Guo et al., 2025) is placed in §1 Related Work under 'Face Manipulation Reasoning' — methods that 'generate natural language explanations through finetuned MLLMs' — explicitly contrasted with the 'Detection without Reasoning' thread that 'produces only binary labels', so the grouping of the two models as explanation-based detectors is what our copy says. Caveat: our copy never prints the name 'TruthLens'; it labels its own framework FORGE (metadata title aside), so the model name in the claim is not verifiable from the text. (LLM, conf=0.80)
- Not checked — no full text of the sourcecites [41] Guo, X., Song, X., Zhang, Y., Liu, X., and Liu, X. (2024). Rethinking Vision-Language Model in Face Forensics: Multi-Modal Interpretable Forged Face Detector. arXiv. source
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
32 more claims · 1 with their cited sources checked
Create a free account to see all 91 claims and to verify the 13 references still open.
- Probably not a substantial claim. 33
- It may state two separate claims as one. 35
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Abbreviations
“| LLMs | Large Multimodal Models |”
§ References
“| LLM | Large Language Model |”
Citations
Cites
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising
Zhang, K., Zuo, W., Chen, Y., Meng, D., & Zhang, L. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2662206 · Cited by 8,983 · cited 1×
“Zhang et al. [18] pioneered this approach with DnCNN, a CNN designed to predict the residual (noise) rather than the clean image.”
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-019-01228-7 · Cited by 6,909 · cited 1×
“Among the most popular of these are gradient-based attribution methods such as Gradient-weighted class activation mapping (Grad-CAM) [26].”
FaceForensics++: Learning to Detect Manipulated Facial Images
Rossler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., & Niessner, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00009 · Cited by 3,314 · cited 2×
“Within the context of Deepfake detection, explain-ability transcends a simple classification label; It is the comprehensive ability to provide the reasoning for a decision, localize the evidence artifacts, and even shed light on the manipulation methodology [2].”
Celeb-DF: A Large-Scale Challenging Dataset for DeepFake Forensics
Li, Y., Yang, X., Sun, P., Qi, H., & Lyu, S. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00327 · Cited by 1,846 · cited 1×
“Key resources include FaceForensics++, a widely adopted dataset for benchmarking, and Celeb-DF [48], known for its high-realism forgeries.”
Digital Camera Identification From Sensor Pattern Noise
Luka, J., Fridrich, J., & Goljan, M. (2006) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2006.873602 · Cited by 1,328 · cited 2×
“The seminal work by Lukas et al. [9] systematically introduced the use of sensor pattern noise, particularly PRNU, as a unique identifier for the source camera.”
Deepfakes and beyond: A Survey of face manipulation and fake detection
Tolosana, R., Vera-Rodriguez, R., Fierrez, J., Morales, A., & Ortega-Garcia, J. (2020) · Information Fusion · doi:10.1016/j.inffus.2020.06.014 · Cited by 1,237 · cited 1×
“In response, the research community has developed a vast array of detection models, achieving remarkable success in classification accuracy [1].”
Exposing DeepFake Videos By Detecting Face Warping Artifacts
Li, Y., & Lyu, S. (2018) · arXiv · doi:10.48550/arxiv.1811.00656 · Cited by 785 · cited 1×
“More recently, advances have emerged in multimodal and natural language explanations, where Large Multimodal Models (LMMs) are leveraged to generate rich, human-readable reports that articulate the reasoning behind detection outcomes [3].”
Noiseprint: A CNN-Based Camera Model Fingerprint
Cozzolino, D., & Verdoliva, L. (2020) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2019.2916364 · Cited by 499 · cited 2×
“Cozzolino et al. [14] introduced Noiseprint, which improves on traditional hand-crafted PRNU features by replacing the fixed noise model with a learned representation.”
WildDeepfake: A Challenging Real-World Dataset for Deepfake Detection
Zi, B., Chang, M., Chen, J., Ma, X., & Jiang, Y.-G. (2020) · Proceedings of the 28th ACM International Conference on Multimedia · doi:10.1145/3394171.3413769 · Cited by 448 · cited 1×
“Additionally, large-scale efforts such as the Deepfake Detection Challenge (DFDC) [49] and the WildDeepfake [50] dataset have driven the development of more generalizable models.”
Research works printed but not linked: 6
- Koopman, M., Rodriguez, A.M., and Geradts, Z. (2018, January 29–31). Detection of deepfake video manipulation. Proceedings of the 20th Irish Machine Vision and Image Processing Conference (IMVIP), Belfast, UK.
- Cozzolino, D., Poggi, G., and Verdoliva, L. (2019, January 16–20). Extracting camera-based fingerprints for video forensics. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Long Beach, CA, USA.
- Khan, S. (2022). Explainable Deepfake Detection Using Deep Learning. [Master’s Thesis, NED University of Engineering and Technology].
- Pellcier, A.L., Li, Y., and Angelov, P. (2023, January 18–24). PUDD: Prototype-based Unified Framework for Deepfake Detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.
- Jia, S., Kundu, R., Balachandran, A., Moreira, D., and Roy-Chowdhury, A.K. (2024). Common Sense Reasoning for Deepfake Detection. arXiv.
- Yan, H., Han, J., Jiang, Y., Yan, B., Liu, J., Wang, X., and Lyu, S. (2024). X2-DFD: A framework for eXplainable and eXtendable Deepfake Detection. arXiv.
Cited by
Thakur, G., & Dutta, K. (2026) · Archives of Computational Methods in Engineering · doi:10.1007/s11831-026-10523-9 · Cited by 3
Vijay, B., Chandra, J., Nagendra, N., et al. (2026) · doi:10.1109/icdsaai69492.2026.11505190
“Xu et al. (2024) proposed the Fake Shield, a multimodal big language model system, and it is designed to detect and localize explainable image forgery [9].”
Deepfake Forensics: Threats and Detection of AI-Manipulated Media
Cardoso, P. M. C., Oliveira, R. S., Gomes, T. F. N., Bastos, E. S., & Lopes, S. F. S. F. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-7660-8.ch015 · Cited by 1
Towards Secure Media Deepfake Detection Using Explainable AI
Rajathi, S., Hemalatha, J., Rajeswari, S., Parameswaran, S., Ramya, D., & Balamurugan, S. (2026) · IFIP advances in information and communication technology · doi:10.1007/978-3-032-28939-1_24
Aydın, Y. (2026) · International Journal of Intelligent Systems · doi:10.1155/int/9932351
Position Paper: Neuro-Fuzzy Fusion for Dataset-Agnostic Deepfake Detection
Chattopadhyay, S., Priya, B., & Muhuri, P. K. (2026) · IEEE International Conference on Fuzzy Systems (FUZZ-IEEE) · doi:10.1109/fuzz69877.2026.11626363
Hasen, S. A., Mohammed Ismail, B., & Ram, N. S. (2026) · International Conference on Smart Systems and Inventive Technology (ICSSIT) · doi:10.1109/icssit69151.2026.11656466
“…become a key area of exploration, providing researchers visual cues (Grad-CAM, SHAP, LIME) that can assist journalists and analysts by providing reasoning for the results which can help in quick decision-making, and classifications via the model architectures, heuristics, and algorithms [6], [9].”
Influential
Metadata
- Authors
- Qian, H., Xia, L., Ge, R., Fan, Y., Wang, Q., & Jing, Z.
- Affiliations
- Department of Computer Information and Cybersecurity, Jiangsu Police Institute, Nanjing 210031, China
- College of Information Science and Technology & Artificial Intelligence, Nanjing Forestry University, Nanjing 210037, China
- Jiangsu University of Technology (CN)
- Corresponding author
- Xia, Lingling
- Venue
- Cryptography
- Publisher
- MDPI AG
- Published
- 2025-09-26
- Volume
- 9
- Issue
- 4
- Pages
- 61
- ISSN
- 2410-387X (ISSN-L)
- OpenAlex
- W4414554223
- Semantic Scholar
- 515c2b155d88a961435a2412679031ba3d7151f3
- Cited by
- 10
- References
- 36
- Citations by year
- 2026: 10
- FWCI
- 3.47
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Cryptography (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- MOE Industry-University Collaborative Education Project
- Philosophy and Social Sciences Research Project of Universities in Jiangsu Province (2024SJYB0345, 2023SJYB0464, 2023SJYB0468)
- “Cyberspace Security” construction project of key disciplines in Jiangsu Province during the “14th Five-Year Plan”
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Keywords
- deepfake detection, explainable AI, deepfake forensics, model interpretability, post-hoc explanation, multimodal explanations, natural language explanations, online misinformation, trustworthy AI, forensic analysis, evaluation dataset
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.