The DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index; 7 of the 17 sources its claims cite were checked for support.
Beyond Accuracy: A Deployment-Oriented Benchmark of Deepfake Detection Models
Open accessAbstract
The increasing sophistication of deepfake generation techniques has created urgent challenges, especially as detection systems struggle to meet critical real-world demands, including performance, latency, dataset quality, computational costs, and cross-domain generalization, all of which are essential for practical deployment. To address these challenges, this study compares state-of-the-art deepfake detection techniques by evaluating CNN and transformer-based architectures in terms of performance, computational efficiency, and resource requirements for realtime deployment in image and video-based deepfake detection. CNN-based models such as XceptionNet, ResNet-50, MesoNet, Inception-v3, and EfficientNetV2 and transformer-based models such as ViT (16/32), Swin V2, CrossViT and MaxViT are evaluated for scalability. The experimental findings indicate that transformer-based models, such as Swin V2, achieve stateof-the-art accuracy in controlled environments. Swin V2 provides optimal performance on high-resolution images, achieving 99.80% accuracy, precision, recall, and F1 score. However, it requires substantial computational resources, including 20.27 ms of inference time, 520 MB of RAM, 220.6 MB of GPU, and 4.5% of CPU. In contrast, CNN-based models maintain competitive performance (≥94%) with substantially lower computational costs. These findings clarify the inherent trade-offs between performance and efficiency, providing actionable guidance for selecting architectures under practical constraints, and establishing design principles for scalable, resource-efficient systems.
Overview
CoChat Summary
TL;DR
This preprint benchmarks five CNN and five transformer deepfake detectors on DFC, Celeb-DF(v2), and FaceForensics++ for accuracy and deployment cost. It finds transformer models, especially Swin V2 and MaxViT, lead intra-dataset accuracy but incur high latency/resources, while cross-domain accuracy collapses toward chance.
Key findings
- The abstract reports Swin V2 achieves 99.80% accuracy, precision, recall, and F1 on high-resolution images, but requires 20.27 ms inference time, 520 MB RAM, 220.6 MB GPU, and 4.5% CPU.
- CNN models are faster: EfficientNetV2 (1.04–1.37 ms) and MesoNet (0.86–2.43 ms) show optimal inference speed, while MaxViT has the highest latency (80.02–88.72 ms); CrossViT and Swin V2 are GPU-efficient at 210 MB and 220.6 MB.
- Cross-domain testing shows severe degradation: accuracy often approaches 50%, Swin V2 reaches only 37.94% when trained on FF++ and tested on Celeb-DF, and MaxViT achieves the highest transfer accuracy at 78.30%.
- Intra-dataset results show MaxViT at 99.63% on Celeb-DF(v2) and 87.51% on FF++, while XceptionNet is lowest across datasets (83.53% DFC, 89.25% Celeb-DF, 73.77% FF++).
Methods
- Evaluated five CNN models (XceptionNet, ResNet-50, MesoNet, Inception-v3, EfficientNetV2) and five transformer models (ViT-16/32, Swin V2, CrossViT, MaxViT) on DFC, Celeb-DF(v2), and FaceForensics++.
- Preprocessing extracted frames at 5-second intervals, cropped faces with a DNN-based detector, and resized to 160×160×3; reported splits include DFC 6,300 train/3,000 test/700 validation, Celeb-DF(v2) 12,490 train/1,389 validation/5,950 test, and FF++ 11,288 train/1,256 validation/5,379 test. CNNs trained for 100 epochs and transformers for 10 due to computational constraints.
- Metrics included accuracy, precision, recall, F1, inference time, RAM, GPU, CPU, and a normalized Resource Index; hardware was Intel Core i9-14900HX, 32 GB RAM, NVIDIA RTX 4060 8 GB, Windows 11 Pro, Python 3.9, TensorFlow 2.10, CUDA 11.3.
Limitations
- Paper-stated: DFC is limited to static images rather than video; Celeb-DF(v2) is relatively smaller scale and may restrict generalization; FF++ artifacts may be less representative of state-of-the-art deepfake generation. The paper also states cross-domain generalization failure, transformer scalability issues from quadratic self-attention, and that MaxViT is not practical for limited-compute environments.
- Paper-stated: computational constraints forced transformers to train for only 10 epochs versus 100 for CNNs, which may affect comparisons.
- Evident from the text: intra-dataset accuracies near 99% contrast sharply with cross-domain accuracies near chance, and the abstract’s claim that CNN models maintain competitive performance (≥94%) is not supported for FF++ in Table V, where CNN accuracies range from 73.8% to 87.5%. The dataset-splitting description is also internally inconsistent, mentioning both 60/30/10 and 70/30 partitions.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
6 Valid · 5 Not valid · 9 Not checked · 113 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“The Deepfake Challenge dataset [23] combines authentic CelebA and FFHQ images with GAN-generated manipulations (StarGAN, AttGAN, StyleGAN) for attribute/face synthesis.”
§ Beyond Accuracy: A Deployment-Oriented Benchmark of Deepfake Detection Models > C. Dataset Explanation
What the sources say
- Supportedcites [23] 23. L. Guarnera, O. Giudice, F. Guarnera, A. Ortis, G. Puglisi, A. Paratore, L. M. Q. Bui, M. Fontani, D. A. Coccomini, R. Caldelli, F. Falchi, C. Gennaro, N. Messina, G. Amato, G. Perelli, S. Concas, C. Cuccu, G. Orrù, G. L. Marcialis, S. Battiato, The face deepfake detection challenge, Journal of Imaging 8 (2022) 263. source
Section 3.1 says the challenge used real CelebA and FFHQ images alongside deepfake images generated with StarGAN, AttGAN, and StyleGAN. It describes CelebA images being manipulated with StarGAN and AttGAN, and StyleGAN images being created from FFHQ; the cited examples are not an exhaustive list of the architectures used. (LLM, conf=0.94)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Overall, Swin V2 and CrossViT demonstrate consistently superior GPU efficiency across all datasets.”
§ Beyond Accuracy: A Deployment-Oriented Benchmark of Deepfake Detection Models > A. Intra-Dataset Testing Performance
boostedstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As shown in Fig. 6, Transformer-based models achieve superior precision across datasets.”
§ Beyond Accuracy: A Deployment-Oriented Benchmark of Deepfake Detection Models > A. Intra-Dataset Testing Performance
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The lowest accuracy has been shown by XceptionNet across all datasets (83.53% on DFC, 89.25% on Celeb-DF, 73.77% on FF++), confirming its limitations as a baseline architecture.”
§ Beyond Accuracy: A Deployment-Oriented Benchmark of Deepfake Detection Models > A. Intra-Dataset Testing Performance
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The experimental findings indicate that transformer-based models, such as Swin V2, achieve stateof-the-art accuracy in controlled environments.”
§ Beyond Accuracy: A Deployment-Oriented Benchmark of Deepfake Detection Models > Abstract
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
47 more claims
Create a free account to see all 133 claims and every source verdict.
- Probably not a substantial claim. 28
- It may state two separate claims as one. 66
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
§ Beyond Accuracy: A Deployment-Oriented Benchmark of Deepfake Detection Models > Abstract
“In contrast, CNN-based models maintain competitive performance (≥94%) with substantially lower computational costs.”
§ Beyond Accuracy: A Deployment-Oriented Benchmark of Deepfake Detection Models > A. Intra-Dataset Testing Performance
“Xception Net 83.5 84.0 84.0 83.0 89.3 89.0 89.0 89.0 73.8 71.0 74.0 72.0”
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
§ Beyond Accuracy: A Deployment-Oriented Benchmark of Deepfake Detection Models > C. Dataset Explanation
“This involved extracting frames at 5-second intervals, cropping the facial region using a DNN-based model, and resizing the images to a standard 160×160 px resolution.”
§ Beyond Accuracy: A Deployment-Oriented Benchmark of Deepfake Detection Models > C. Dataset Explanation
“The 5-second frame sampling interval optimizes temporal coverage while mitigating redundancy, preserving ≈20% of sequential data (30fps basis) for efficient analysis.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Beyond Accuracy: A Deployment-Oriented Benchmark of Deepfake Detection Models > A. Intra-Dataset Testing Performance
“Swin V2 achieves the highest precision (0.9927)”
§ Beyond Accuracy: A Deployment-Oriented Benchmark of Deepfake Detection Models > A. Intra-Dataset Testing Performance
“closely followed by MaxViT (0.9963).”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Beyond Accuracy: A Deployment-Oriented Benchmark of Deepfake Detection Models > B. Cross-Domain Testing Performance
“Performance improved slightly to 45.10% when the model was trained”
§ Beyond Accuracy: A Deployment-Oriented Benchmark of Deepfake Detection Models > B. Cross-Domain Testing Performance
“Swin V2 70.17 67.26 51.87 34.54 45.10 37.94”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Beyond Accuracy: A Deployment-Oriented Benchmark of Deepfake Detection Models > C. Dataset Explanation
“We Split the dataset into 60% training, 30% testing, and 10% percent validation.”
§ Beyond Accuracy: A Deployment-Oriented Benchmark of Deepfake Detection Models > C. Dataset Explanation
“For DFC datasets, Balanced 5,000 real (2,500 CelebA + 2,500 FFHQ) and 5,000 fake images were partitioned into training (6,300), testing (3,000), and validation (700) sets.”
3 more findings
Create a free account to see every finding of this check.
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 3
- [23]23. L. Guarnera, O. Giudice, F. Guarnera, A. Ortis, G. Puglisi, A. Paratore, L. M. Q. Bui, M. Fontani, D. A. Coccomini, R. Caldelli, F. Falchi, C. Gennaro, N. Messina, G. Amato, G. Perelli, S. Concas, C. Cuccu, G. Orrù, G. L. Marcialis, S. Battiato, The face deepfake detection challenge, Journal of Imaging 8 (2022) 263.
- [25]25. Huansheng Ning (Senior Member, IEEE) received the B.S. degree from Anhui University, Hefei, China, in 1996, and the Ph.D. degree from Beihang University, Beijing, China, in 2001. He is currently a Professor with the School of Computer and Communication Engineering, University of Science and Technology Beijing, China, and the Founder and the Principal with Beijing Cyberspace International Science and Technology Cooperation Base. His current research interests include the IoT, general cyberspace, metaverse, smart education, cyber-syndrome, and cyber-health.
- [27]27. Mahmoud Daneshmand (Life Senior Member, IEEE) received the B.S. and M.S. degrees in mathematics from the University of Tehran, Iran, and the M.S. and Ph.D. degrees in statistics from the University of California at Berkeley, USA. He is currently an Industry Professor at Stevens Institute of Technology, USA, with over 40 years of experience in academia and industry, including positions at Bell Laboratories, AT&T Shannon Labs, and multiple universities. He has published more than 300 journal and conference papers and authored three books.
Cited by
AT-HSTNet: An Efficient Hierarchical Action-Transformer Framework for Deepfake Video Detection
Javaid, S., Rai, M. C. E., Elkhouly, A., Al-Khatib, O., Far, A. B., & Barachi, M. E. (2026) · Applied Sciences · doi:10.3390/app16073450 · Cited by 2
Asim, A., Umair, M., Mazhar, N., & Asghar, M. N. (2026) · Applied Sciences · doi:10.3390/app16168223
Metadata
- Authors
- Ain, Q. U., Ning, H., Philipo, A. G., Daneshmand, M., & Ding, J.
- Affiliations
- Blekinge Institute of Technology (SE)
- FernUniversität in Hagen (DE)
- Corresponding author
- Ning, Huansheng
- Venue
- Preprint (TechRxiv)
- Published
- 2025-11-14
- OpenAlex
- W4416220209
- Cited by
- 2
- References
- 0
- Citations by year
- 2026: 2
- Type
- posted content
- Language
- en
- Open access
- green
- doi.org
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Keywords
- deepfake detection, benchmarking, deployment efficiency, computational cost, domain generalization, vision transformer, convolutional neural network architecture, real-time inference, performance-efficiency trade-off
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-09-30, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-09-30 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Beyond Accuracy: A Deployment-Oriented Benchmark of Deepfake Detection Models. https://app.cochat.ai/papers/10.36227/techrxiv.176315939.91577661/v1
- The paper
- Ain, Q. U., Ning, H., Philipo, A. G., Daneshmand, M., & Ding, J. (2025). Beyond Accuracy: A Deployment-Oriented Benchmark of Deepfake Detection Models. Preprint (TechRxiv). https://doi.org/10.36227/techrxiv.176315939.91577661/v1
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.