Exposing Fake Faces Through Deep Neural Networks Combining Content and Trace Feature Extractors
Open accessThe DOI matches its Crossref record; 17 of the 17 references it cites were found in a registry or an index.
Abstract
With the breakthrough of computer vision and deep learning, there has been a surge of realistic-looking fake face media manipulated by AI such as DeepFake or Face2Face that manipulate facial identities or expressions. The fake faces were mostly created for fun, but abuse has caused social unrest. For example, some celebrities have become victims of fake pornography made by DeepFake. There are also growing concerns about fake political speech videos created by Face2Face. To maintain individual privacy as well as social, political, and international security, it is imperative to develop models that detect fake faces in media. Previous research can be divided into general-purpose image forensics and face image forensics. While the former has been studied for several decades and focuses on extracting hand-crafted features of traces left in the image after manipulation, the latter is based on convolutional neural networks mainly inspired by object detection models specialized to extract images’ content features. This paper proposes a hybrid face forensics framework based on a convolutional neural network combining the two forensics approaches to enhance the manipulation detection performance. To validate the proposed framework, we used a public Face2Face dataset and a custom DeepFake dataset collected on our own. Experimental results using the two datasets showed that the proposed model is more accurate and robust at various video compression rates compared to the previous methods. Throughout class activation map visualization, the proposed framework provided information on which face parts are considered important and revealed the tempering traces invisible to naked eyes.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Image Style Transfer Using Convolutional Neural Networks
Gatys, L. A., Ecker, A. S., & Bethge, M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.265 · Cited by 6,025
An introduction to biometric recognition
Jain, A. K., Ross, A., & Prabhakar, S. (2004) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2003.818349 · Cited by 4,974 · cited 1×
“In other applications like biometrics, fake faces may fool an access control system [2], [3].”
BackgroundFaceForensics++: Learning to Detect Manipulated Facial Images
Rossler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., & Niessner, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00009 · Cited by 3,314 · cited 2×
“Among the object detection CNNs, Xception [10] showed the most powerful performance [12]–[14].”
BackgroundMethodRich Models for Steganalysis of Digital Images
Fridrich, J., & Kodovsky, J. (2012) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2012.2190402 · Cited by 2,144 · cited 3×
“Thus, hand-designed feature extraction methods such as [28], [29] seems to be difficult to apply to a large amount of data, which will benefit from the adaptive feature extraction method. alized (the size of feature maps were adjusted).”
MethodMesoNet: a Compact Facial Video Forgery Detection Network
Afchar, D., Nozick, V., Yamagishi, J., & Echizen, I. (2018) · 2018 IEEE International Workshop on Information Forensics and Security (WIFS) · doi:10.1109/wifs.2018.8630761 · Cited by 1,872
Face2Face: Real-Time Face Capture and Reenactment of RGB Videos
Thies, J., Zollhofer, M., Stamminger, M., Theobalt, C., & Niessner, M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.262 · Cited by 1,747
Bulat, A., & Tzimiropoulos, G. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.116 · Cited by 1,464
Cited by
Deepfake detection using deep learning methods: A systematic and comprehensive review
Heidari, A., Jafari Navimipour, N., Dag, H., & Unal, M. (2024) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1520 · Cited by 349
“Also, transfer learning (TL) is critical since the model must be trained in a reasonable period and with little resources to achieve the necessary accuracy for classification over a wide range of instances in the dataset (Garg & Mago, 2021; Kim & Cho, 2021; Zong & Wan, 2022; Zong & Wang, 2022).”
BackgroundA Novel Blockchain-Based Deepfake Detection Method Using Federated and Deep Learning Models
Heidari, A., Navimipour, N. J., Dag, H., Talebi, S., & Unal, M. (2024) · Cognitive Computation · doi:10.1007/s12559-024-10255-7 · Cited by 164
DeepFake on Face and Expression Swap: A Review
Waseem, S., Abu Bakar, S. A. R. S., Ahmed, B. A., Omar, Z., Eisa, T. A. E., & Dalam, M. E. E. (2023) · IEEE Access · doi:10.1109/access.2023.3324403 · Cited by 65
Vardhan, H., Varshney, N., Manoj Kiran, R., Pradeep, R., & Dr. Latha, N. R. (2024) · International Research Journal on Advanced Engineering Hub (IRJAEH) · doi:10.47392/irjaeh.2024.0117 · Cited by 4
Metadata
- Authors
- Kim, E., & Cho, S.
- Venue
- IEEE Access
- Published
- 2021
- Volume
- 9
- Pages
- 123493-123503
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W3200712287
- Cited by
- 58
- References
- 17
- FWCI
- 2.65
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Exposing Fake Faces Through Deep Neural Networks Combining Content and Trace Feature Extractors. https://app.cochat.ai/papers/10.1109/access.2021.3110859
- The paper
- Kim, E., & Cho, S. (2021). Exposing Fake Faces Through Deep Neural Networks Combining Content and Trace Feature Extractors. IEEE Access. https://doi.org/10.1109/access.2021.3110859
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.