Exploiting Prediction Error Inconsistencies through LSTM-based Classifiers to Detect Deepfake Videos
The DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
Abstract
The ability of artificial intelligence techniques to build synthesized brand new videos or to alter the facial expression of already existing ones has been efficiently demonstrated in the literature. The identification of such new threat generally known as Deepfake, but consisting of different techniques, is fundamental in multimedia forensics. In fact this kind of manipulated information could undermine and easily distort the public opinion on a certain person or about a specific event. Thus, in this paper, a new technique able to distinguish synthetic generated portrait videos from natural ones is introduced by exploiting inconsistencies due to the prediction error in the re-encoding phase. In particular, features based on inter-frame prediction error have been investigated jointly with a Long Short-Term Memory (LSTM) model network able to learn the temporal correlation among consecutive frames. Preliminary results have demonstrated that such sequence-based approach, used to distinguish between original and manipulated videos, highlights promising performances.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“the re- cent common underlying mechanism for deepfake cre- ation is deep learning models such as autoencoders and generative adversarial networks (GANs)”
From Deep learning for deepfakes creation and detection: A survey (2022) · cites this paper as [6]
In the citing paper
“Existing video-level approaches are not specially designed for this inconsistency and can not properly capture the spatiotemporal cues which hide in deepfake videos, which makes these approaches poorly generalized and vulnerable, even cannot achieve comparable performance to imagelevel approaches.”
From Deepfake Video Detection with Spatiotemporal Dropout Transformer (2022) · cites this paper as [2]
Citations
Cites
Greff, K., Srivastava, R. K., Koutnik, J., Steunebrink, B. R., & Schmidhuber, J. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2582924 · Cited by 6,955 · cited 1×
“As prediction error should show temporal dependencies, our idea is to employ a LSTM model as in [15] capable of learning long-term dependencies, in order to capture the evolution of such a feature throughout the video sequence.”
MethodFaceForensics++: Learning to Detect Manipulated Facial Images
Rossler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., & Niessner, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00009 · Cited by 3,314 · cited 5×
“The dataset used for the experiments is the FaceForensics++ dataset introduced in [27] composed by 1000 YouTube videos with different resolutions and aspect ratios.”
BackgroundMethodInfluentialRich Models for Steganalysis of Digital Images
Fridrich, J., & Kodovsky, J. (2012) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2012.2190402 · Cited by 2,144 · cited 1×
“By comparing with the state of the art (sota) approaches reported in [27], our method performs comparably to the methods which train their model from scratch [5, 11, 14, 25] as well as done in the proposed one and, especially for the compressed videos, there is also a slight improvement.”
MethodMesoNet: a Compact Facial Video Forgery Detection Network
Afchar, D., Nozick, V., Yamagishi, J., & Echizen, I. (2018) · 2018 IEEE International Workshop on Information Forensics and Security (WIFS) · doi:10.1109/wifs.2018.8630761 · Cited by 1,872
Long-Term Recurrent Convolutional Networks for Visual Recognition and Description
Donahue, J., Hendricks, L. A., Rohrbach, M., Venugopalan, S., Guadarrama, S., Saenko, K., & Darrell, T. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2599174 · Cited by 1,547
Deepfake Video Detection Using Recurrent Neural Networks
Guera, D., & Delp, E. J. (2018) · 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS) · doi:10.1109/avss.2018.8639163 · Cited by 1,285 · cited 1×
“In [16, 28] recurrent neural networks have been employed.”
MethodDetermining Image Origin and Integrity Using Sensor Noise
Chen, M., Fridrich, J., Goljan, M., & Lukas, J. (2008) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2007.916285 · Cited by 893 · cited 1×
“Pioneering works in this field are forensic techniques proposed to classify computer generated and digital camera images [3, 17] using the characteristic of the Sensor Pattern Noise of digital CCD sensors [8].”
MethodA Deep Learning Approach to Universal Image Manipulation Detection Using a New Convolutional Layer
Bayar, B., & Stamm, M. C. (2016) · Proceedings of the 4th ACM Workshop on Information Hiding and Multimedia Security · doi:10.1145/2909827.2930786 · Cited by 838 · cited 1×
“By comparing with the state of the art (sota) approaches reported in [27], our method performs comparably to the methods which train their model from scratch [5, 11, 14, 25] as well as done in the proposed one and, especially for the compressed videos, there is also a slight improvement.”
MethodExploiting Visual Artifacts to Expose Deepfakes and Face Manipulations
Matern, F., Riess, C., & Stamminger, M. (2019) · 2019 IEEE Winter Applications of Computer Vision Workshops (WACVW) · doi:10.1109/wacvw.2019.00020 · Cited by 824 · cited 1×
“Again a single frame evaluation, as well as the previous technique, is given as well in [22].”
MethodChan, C., Ginosar, S., Zhou, T., & Efros, A. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00603 · Cited by 737
Research works printed but not linked: 7
- Darius Afchar Vincent Nozick Junichi Yamagishi and I Echizen. 2018. MesoNet: a Compact Facial Video Forgery Detection Network. 1--7. https://doi.org/10.1109/WIFS.2018.8630761 · DOI · Paper page
- Shruti Agarwal , Hany Farid , Yuming Gu , Mingming He , Koki Nagano , and Hao Li . 2019 . Protecting World Leaders Against Deep Fakes. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops. Shruti Agarwal, Hany Farid, Yuming Gu, Mingming He, Koki Nagano, and Hao Li. 2019. Protecting World Leaders Against Deep Fakes. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops.
- Francois Chollet . 2016 . Xception: Deep Learning with Depthwise Separable Convolutions. arxiv: cs.CV/1610.02357 Francois Chollet. 2016. Xception: Deep Learning with Depthwise Separable Convolutions. arxiv: cs.CV/1610.02357
- D. Dang-Nguyen , G. Boato , and F. G. B. De Natale . 2012 . Discrimination between computer generated and natural human faces based on asymmetry information . In 2012 Proceedings of the 20th European Signal Processing Conference (EUSIPCO). 1234--1238 . D. Dang-Nguyen, G. Boato, and F. G. B. De Natale. 2012. Discrimination between computer generated and natural human faces based on asymmetry information. In 2012 Proceedings of the 20th European Signal Processing Conference (EUSIPCO). 1234--1238.
- N. Khanna , G. T.-C. Chiu , J. P. Allebach , and E. J. Delp . 2008. Forensic techniques for classifying scanner, computer generated and digital camera images . In Proc. of IEEE ICASSP . Las Vegas, USA. N. Khanna, G. T.-C. Chiu, J. P. Allebach, and E. J. Delp. 2008. Forensic techniques for classifying scanner, computer generated and digital camera images. In Proc. of IEEE ICASSP. Las Vegas, USA.
- Andreas Rö ssler, Davide Cozzolino , Luisa Verdoliva , Christian Riess , Justus Thies , and Matthias Nießner . 2018. FaceForensics: A Large-scale Video Dataset for Forgery Detection in Human Faces. CoRR , Vol. abs/ 1803 .09179 ( 2018 ). arxiv: 1803.09179 http://arxiv.org/abs/1803.09179 Andreas Rö ssler, Davide Cozzolino, Luisa Verdoliva, Christian Riess, Justus Thies, and Matthias Nießner. 2018. FaceForensics: A Large-scale Video Dataset for Forgery Detection in Human Faces. CoRR, Vol. abs/1803.09179 (2018). arxiv: 1803.09179 http://arxiv.org/abs/1803.09179 · arXiv
- Sheng-Yu Wang , Oliver Wang , Richard Zhang , Andrew Owens , and Alexei A . Efros . 2019 . CNN-generated images are surprisingly easy to spot... for now. arxiv: cs.CV/1912.11035 Sheng-Yu Wang, Oliver Wang, Richard Zhang, Andrew Owens, and Alexei A. Efros. 2019. CNN-generated images are surprisingly easy to spot... for now. arxiv: cs.CV/1912.11035
Cited by
The Creation and Detection of Deepfakes: A Survey
Mirsky, Y., & Lee, W. (2022) · ACM Computing Surveys · doi:10.1145/3425780 · Cited by 863
“Later, the same authors use an LSTM to predict the next frame and expose deepfakes when the reconstruction error is high [10].”
BackgroundThe Creation and Detection of Deepfakes: A Survey
Mirsky, Y., & Lee, W. (2020) · arXiv · doi:10.48550/arxiv.2004.11138 · Cited by 863
“Later, the same authors use an LSTM to predict the next frame and expose deepfakes when the reconstruction error is high [10].”
BackgroundMasood, M., Nawaz, M., Malik, K. M., Javed, A., Irtaza, A., & Malik, H. (2023) · Applied Intelligence · doi:10.1007/s10489-022-03766-z · Cited by 543
“The temporal artifacts involve detecting variation in a person’s behavior [79, 87, 268], physiological signals [74, 75, 83, 89], coherence [269, 270], and video frame synchronization [72, 82, 94, 134].”
BackgroundDeep learning for deepfakes creation and detection: A survey
Nguyen, T. T., Nguyen, Q. V. H., Nguyen, D. T., Nguyen, D. T., Huynh-The, T., Nahavandi, S., Nguyen, T. T., Pham, Q.-V., & Nguyen, C. M. (2022) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2022.103525 · Cited by 464
Deep Learning for Deepfakes Creation and Detection: A Survey
Nguyen, T. T., Nguyen, Q. V. H., Nguyen, D. T., Nguyen, D. T., Huynh-The, T., Nahavandi, S., Nguyen, T. T., Pham, Q.-V., & Nguyen, C. M. (2019) · arXiv · doi:10.48550/arxiv.1909.11573 · Cited by 464
Metadata
- Authors
- Amerini, I., & Caldelli, R.
- Affiliations
- Sapienza University of Rome (IT)
- Consorzio Nazionale Interuniversitario per le Telecomunicazioni (IT)
- Venue
- Proceedings of the 2020 ACM Workshop on Information Hiding and Multimedia Security
- Publisher
- ACM
- Published
- 2020-06-22
- Pages
- 97-102
- OpenAlex
- W3037675456
- Semantic Scholar
- 566a1ad5afc0f5a080e041cd5174ff923d11e0f4
- Cited by
- 64
- References
- 26
- Citations by year
- 2026: 3 · 2025: 6 · 2024: 12 · 2023: 11 · 2022: 23 · 2021: 7 · 2020: 2
- FWCI
- 3.32
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM Workshop on Information Hiding and Multimedia Security (IH&MMSec) (publishedVersion)
- IRIS Research product catalog (Sapienza University of Rome) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image and Video Quality Assessment
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Video Analysis and Summarization
- Keywords
- deepfake detection, video forensics, prediction error, synthetic video, LSTM classifier, temporal correlation, video manipulation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Exploiting Prediction Error Inconsistencies through LSTM-based Classifiers to Detect Deepfake Videos. https://app.cochat.ai/papers/10.1145/3369412.3395070
- The paper
- Amerini, I., & Caldelli, R. (2020). Exploiting Prediction Error Inconsistencies through LSTM-based Classifiers to Detect Deepfake Videos. Proceedings of the 2020 ACM Workshop on Information Hiding and Multimedia Security. https://doi.org/10.1145/3369412.3395070
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.