Deepfake Video Detection Using Recurrent Neural Networks
The DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.
Abstract
In recent months a machine learning based free software tool has made it easy to create believable face swaps in videos that leaves few traces of manipulation, in what are known as "deepfake" videos. Scenarios where these realistic fake videos are used to create political distress, blackmail someone or fake terrorism events are easily envisioned. This paper proposes a temporal-aware pipeline to automatically detect deepfake videos. Our system uses a convolutional neural network (CNN) to extract frame-level features. These features are then used to train a recurrent neural network (RNN) that learns to classify if a video has been subject to manipulation or not. We evaluate our method against a large set of deepfake videos collected from multiple video websites. We show how our system can achieve competitive results in this task while using a simple architecture.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Their experiments with a large collection of digitally altered videos revealed that a simple convolutional LSTM structure could indeed effectively forecast whether a video has been manipulated or not with as little as 2 s of video data.”
From Deepfake detection using deep learning methods: A systematic and comprehensive review (2024) · cites this paper as Güera and Delp (2018)
In the citing paper
“Korshunov and Marcel [63] produced a notable deep- fake dataset consisting of 620 videos based on the GAN model using the open source code Faceswap-GAN [58].”
From Deep learning for deepfakes creation and detection: A survey (2022) · cites this paper as [63]
In the citing paper
“Güera and Delp (2018) claims they can take advantage of the time differences by building a pipeline that starts with a CNN and ends with a recurrent neural network (RNN). Their approach extracts frame-level information using a CNN. These features are then used to train an RNN to detect video manipulation.”
From Deepfake video detection: challenges and opportunities (2024) · cites this paper as Güera and Delp 2018
In the citing paper
“Deepfakes, which convincingly replace parts of original content with fabricated elements, pose significant threats ranging from personal identity theft to the propagation of disinformation (Güera & Delp, 2018).”
From Deepfake forensics: a survey of digital forensic methods for multimodal deepfake identification on social media (2024) · cites this paper as Güera & Delp, 2018
In the citing paper
“They achieved precision in accurately capturing temporal inconsistencies between frames due to face swapping.”
From A Comprehensive Review of DeepFake Detection Using Advanced Machine Learning and Fusion Methods (2023) · cites this paper as [36]
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Rethinking the Inception Architecture for Computer Vision
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.308 · Cited by 32,138
Unpaired image-to-image translation using cycle-consistent adversarial networks
Zhu, J.-Y., Park, T., Isola, P., & Efros, A. A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.244 · Cited by 23,602
Image-to-Image Translation with Conditional Adversarial Networks
Isola, P., Zhu, J.-Y., Zhou, T., & Efros, A. A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.632 · Cited by 22,930
Labaca-Castro, R. (2023) · Machine Learning under Malware Attack · doi:10.1007/978-3-658-40442-0_9 · Cited by 19,272
GAN(Generative Adversarial Nets)
(2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,270
TensorFlow: A system for large-scale machine learning
Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.08695 · Cited by 11,822
Progressive Growing of GANs for Improved Quality, Stability, and Variation
Karras, T., Aila, T., Laine, S., & Lehtinen, J. (2017) · arXiv · doi:10.48550/arxiv.1710.10196 · Cited by 4,727
Learning realistic human actions from movies
Laptev, I., Marszalek, M., Schmid, C., & Rozenfeld, B. (2008) · 2008 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2008.4587756 · Cited by 3,444
Face2Face: Real-Time Face Capture and Reenactment of RGB Videos
Thies, J., Zollhofer, M., Stamminger, M., Theobalt, C., & Niessner, M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.262 · Cited by 1,747
Long-Term Recurrent Convolutional Networks for Visual Recognition and Description
Donahue, J., Hendricks, L. A., Rohrbach, M., Venugopalan, S., Guadarrama, S., Saenko, K., & Darrell, T. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2599174 · Cited by 1,547
The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation
Brundage, M., Avin, S., Clark, J., Toner, H., Eckersley, P., Garfinkel, B., Dafoe, A., Scharre, P., Zeitzoff, T., Filar, B., et al. (2018) · Apollo - University of Cambridge Repository · doi:10.17863/cam.22520 · Cited by 849
The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation
Brundage, M., Avin, S., Clark, J., Toner, H., Eckersley, P., Garfinkel, B., Dafoe, A., Scharre, P., Zeitzoff, T., Filar, B., et al. (2018) · arXiv · doi:10.48550/arxiv.1802.07228 · Cited by 849
Two-Stream Neural Networks for Tampered Face Detection
Zhou, P., Han, X., Morariu, V. I., & Davis, L. S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.229 · Cited by 700
Video Rewrite: driving visual speech with audio
Bregler, C., Covell, M., & Slaney, M. (1997) · Proceedings of the 24th annual conference on Computer graphics and interactive techniques - SIGGRAPH '97 · doi:10.1145/258734.258880 · Cited by 669
Demo of Face2Face: real-time face capture and reenactment of RGB videos
Thies, J., Zollhöfer, M., Stamminger, M., Theobalt, C., & Nießner, M. (2016) · ACM SIGGRAPH 2016 Emerging Technologies · doi:10.1145/2929464.2929475 · Cited by 589
Recurrent Convolutional Network for Video-Based Person Re-identification
McLaughlin, N., Martinez del Rincon, J., & Miller, P. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.148 · Cited by 564
Face aging with conditional generative adversarial networks
Antipov, G., Baccouche, M., & Dugelay, J.-L. (2017) · 2017 IEEE International Conference on Image Processing (ICIP) · doi:10.1109/icip.2017.8296650 · Cited by 515
MoFA: Model-Based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstruction
Tewari, A., Zollhofer, M., Kim, H., Garrido, P., Bernard, F., Perez, P., & Theobalt, C. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.401 · Cited by 343
FaceForensics: A Large-scale Video Dataset for Forgery Detection in Human Faces
Rössler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., & Nießner, M. (2018) · arXiv · doi:10.48550/arxiv.1803.09179 · Cited by 339
Distinguishing computer graphics from natural images using convolution neural networks
Rahmouni, N., Nozick, V., Yamagishi, J., & Echizen, I. (2017) · 2017 IEEE Workshop on Information Forensics and Security (WIFS) · doi:10.1109/wifs.2017.8267647 · Cited by 318
Deep Feature Interpolation for Image Content Changes
Upchurch, P., Gardner, J., Pleiss, G., Pless, R., Snavely, N., Bala, K., & Weinberger, K. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.645 · Cited by 296
Fader Networks: Manipulating Images by Sliding Attributes
Lample, G., Zeghidour, N., Usunier, N., Bordes, A., Denoyer, L., & Ranzato, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.00409 · Cited by 271
MoFA: Model-Based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstruction
Tewari, A., Zollhofer, M., Kim, H., Garrido, P., Bernard, F., Perez, P., & Theobalt, C. (2017) · 2017 IEEE International Conference on Computer Vision Workshops (ICCVW) · doi:10.1109/iccvw.2017.153 · Cited by 269
Transferable Deep-CNN Features for Detecting Digital and Print-Scanned Morphed Face Images
Raghavendra, R., Raja, K. B., Venkatesh, S., & Busch, C. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.228 · Cited by 204
Averbuch-Elor, H., Cohen-Or, D., Kopf, J., & Cohen, M. F. (2017) · ACM Transactions on Graphics · doi:10.1145/3130800.3130818 · Cited by 187
Dale, K., Sunkavalli, K., Johnson, M. K., Vlasic, D., Matusik, W., & Pfister, H. (2011) · Proceedings of the 2011 SIGGRAPH Asia Conference · doi:10.1145/2024156.2024164 · Cited by 166
Local tampering detection in video sequences
Bestagini, P., Milani, S., Tagliasacchi, M., & Tubaro, S. (2013) · IEEE International Workshop on Multimedia Signal Processing (MMSP) · doi:10.1109/mmsp.2013.6659337 · Cited by 151
Sencar, H. T., & Memon, N. D. (2012) · Springer eBooks · doi:10.1007/978-1-4614-0757-7 · Cited by 146
Exposing Digital Forgeries in Interlaced and Deinterlaced Video
Wang, W., & Farid, H. (2007) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2007.902661 · Cited by 137
Farid, H. (2016) · The MIT Press eBooks · doi:10.7551/mitpress/10451.001.0001 · Cited by 130
Dale, K. M., Sunkavalli, K., Johnson, M. K., Vlasic, D., Matusik, W., & Pfister, H. (2011) · ACM Transactions on Graphics · doi:10.1145/2070781.2024164 · Cited by 128
A Counter-Forensic Method for CNN-Based Camera Model Identification
Güera, D., Wang, Y., Bondi, L., Bestagini, P., Tubaro, S., & Delp, E. J. (2017) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.230 · Cited by 78
Self-citationPhysiologically-based detection of computer generated faces in video
Conotter, V., Bodnari, E., Boato, G., & Farid, H. (2014) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2014.7025049 · Cited by 65
Graph Regularized Auto-Encoders for Image Representation
Liao, Y., Wang, Y., & Liu, Y. (2016) · IEEE Transactions on Image Processing · doi:10.1109/tip.2016.2605010 · Cited by 56
Conditional CycleGAN for Attribute Guided Face Image GenerationPreprint
Lu, Y., Tai, Y., & Tang, C. (2017) · arXiv (Cornell University) · Cited by 54
Qian, Y., Chen, K., Nikkanen, J., Kämäräinen, J., & Matas, J. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.582 · Cited by 52
Recent Progress of Face Image Synthesis
Lu, Z., Li, Z., Cao, J., He, R., & Sun, Z. (2017) · Asian Conference on Pattern Recognition · doi:10.1109/acpr.2017.2 · Cited by 32
Reliability Map Estimation for CNN-Based Camera Model Attribution
Güera, D., Zhu, F. M., Yarlagadda, S. K., Tubaro, S., Bestagini, P., & Delp, E. J. (2018) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2018.00111 · Cited by 29
Self-citationRecent Progress of Face Image SynthesisPreprint
Lu, Z., Li, Z., Cao, J., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.04717 · Cited by 12
Lincoln, a picture story of his life
Lorant, S. (1952) · Medical Entomology and Zoology · Cited by 8
Cited by
FaceForensics++: Learning to Detect Manipulated Facial Images
Rossler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., & Niessner, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00009 · Cited by 3,314
Celeb-DF: A Large-Scale Challenging Dataset for DeepFake Forensics
Li, Y., Yang, X., Sun, P., Qi, H., & Lyu, S. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00327 · Cited by 1,846
“, [6, 17, 35, 36]) are data-driven, which directly employ various types of DNNs trained on real and DeepFake videos, not relying on any specific artifact.”
MethodDeepfakes and beyond: A Survey of face manipulation and fake detection
Tolosana, R., Vera-Rodriguez, R., Fierrez, J., Morales, A., & Ortega-Garcia, J. (2020) · Information Fusion · doi:10.1016/j.inffus.2020.06.014 · Cited by 1,237
Exposing Deep Fakes Using Inconsistent Head Poses
Yang, X., Li, Y., & Lyu, S. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8683164 · Cited by 1,196
Media Forensics and DeepFakes: An Overview
Verdoliva, L. (2020) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2020.3002101 · Cited by 979
Metadata
- Authors
- Guera, D., & Delp, E. J.
- Affiliations
- Purdue University West Lafayette (US)
- ORCID
- Delp, Edward J.
- Venue
- 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS)
- Publisher
- IEEE
- Published
- 2018-11
- Pages
- 1-6
- OpenAlex
- W2911424785
- Cited by
- 1,285
- References
- 42
- Citations by year
- 2026: 121 · 2025: 200 · 2024: 217 · 2023: 185 · 2022: 198 · 2021: 173 · 2020: 146 · 2019: 42 · 2018: 3
- FWCI
- 27.01
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS) (publishedVersion)
- Funders
- Defense Advanced Research Projects Agency
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Keywords
- deepfake detection, video manipulation, face swapping, recurrent neural networks, convolutional neural networks, temporal features, video authenticity, misinformation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.