CoChat
Sign Up
PreprintCited by 339Digital Media Forensic DetectionGenerative Adversarial Networks and Image SynthesisAdvanced Steganography and Watermarking Techniques

FaceForensics: A Large-scale Video Dataset for Forgery Detection in Human Faces

Open access
Rössler, Andreas, Cozzolino, Davide, Verdoliva, Luisa, Riess, Christian, Thies, Justus, Nießner, Matthias·2018·arXiv
DOI 10.48550/arxiv.1803.09179arXiv 1803.09179
Open this paper in CoChat — free accountOpen PDF

Abstract

With recent advances in computer vision and graphics, it is now possible to generate videos with extremely realistic synthetic faces, even in real time. Countless applications are possible, some of which raise a legitimate alarm, calling for reliable detectors of fake videos. In fact, distinguishing between original and manipulated video can be a challenge for humans and computers alike, especially when the videos are compressed or have low resolution, as it often happens on social networks. Research on the detection of face manipulations has been seriously hampered by the lack of adequate datasets. To this end, we introduce a novel face manipulation dataset of about half a million edited images (from over 1000 videos). The manipulations have been generated with a state-of-the-art face editing approach. It exceeds all existing video manipulation datasets by at least an order of magnitude. Using our new dataset, we introduce benchmarks for classical image forensic tasks, including classification and segmentation, considering videos compressed at various quality levels. In addition, we introduce a benchmark evaluation for creating indistinguishable forgeries with known ground truth; for instance with generative refinement models.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.1803.09179

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • FaceForensics (FF) contains 1,004 face videos with over 500,000 frames, covering various quality levels and two types of facial manipulation using Face2Face.

    “FaceForensics (FF) (Rössler et al., 2018): This dataset contains 1,004 face videos with over 500,000 frames, covering various quality levels and two types of facial manipulation using the Face2Face approach.”

    From Deepfake: definitions, performance metrics and standards, datasets, and a meta-review (2024) · cites this paper as Rössler et al., 2018

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 54

  1. [1]1. Frith, C.: Role of facial expressions in social interactions. Philosophical Transactions of the Royal Society B: Biological Sciences 364 (1535) (December 2009)
  2. [2]2. Thies, J., Zollh¨ ofer, M., Stamminger, M., Theobalt, C., Nießner, M.: Face2Face: Real-Time Face Capture and Reenactment of RGB Videos. In: IEEE Conference on Computer Vision and Pattern Recognition. (June 2016) 2387-2395
  3. [3]3. Suwajanakorn, S., Seitz, S.M., Kemelmacher-Shlizerman, I.: Synthesizing Obama: learning lip sync from audio. ACM Transactions on Graphics (TOG) 36 (4) (2017) Fig. 11. Forgery segmentation examples on easy-compressed videos. For each frame, we show the heatmaps for the original video (first row) and the manipulated one (second row). From left to right: input frame, ground truth mask (only for the fake input), results of Rahmouni et al. [39], Cozzolino et al. [27], and the XceptionNet-based method. <!-- image -->
  4. [4]4. Averbuch-Elor, H., Cohen-Or, D., Kopf, J., Cohen, M.F.: Bringing portraits to life. ACM Transactions on Graphics (Proceeding of SIGGRAPH Asia 2017) 36 (4) (2017) to appear
  5. [5]5. Jain, A.K., Ross, A., Prabhakar, S.: An introduction to biometric recognition. IEEE Transactions on Circuits and Systems for Video Technology 14 (1) (January 2004) 4-20
  6. [6]6. Ferrara, M., Franco, A., Maltoni, D.: The magic passport. In: IEEE International Joint Conference in Biometrics. (2014) 1-7
  7. [7]7. Shrivastava, A., Pfister, T., Tuzel, O., Susskind, J., Wang, W., Webb, R.: Learning from simulated and unsupervised images through adversarial training. In: IEEE Conference on Computer Vision and Pattern Recognition. Volume 3. (2017) 2107-2116
  8. [8]8. Bregler, C., Covell, M., Slaney, M.: Video rewrite: Driving visual speech with audio. In: Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques. SIGGRAPH '97, New York, NY, USA, ACM Press/Addison-Wesley Publishing Co. (1997) 353-360
  9. [9]9. Dale, K., Sunkavalli, K., Johnson, M.K., Vlasic, D., Matusik, W., Pfister, H.: Video face replacement. ACM Trans. Graph. 30 (6) (December 2011) 130:1-130:10 Fig. 12. Forgery segmentation examples on hard-compressed videos. For each frame, we show the heatmaps for the original video (first row) and the manipulated one (second row). From left to right: input frame, ground truth mask (only for the fake input), results of Rahmouni et al. [39], Cozzolino et al. [27], and the XceptionNet-based method. <!-- image -->
  10. [10]10. Garrido, P., Valgaerts, L., Rehmsen, O., Thormaehlen, T., Perez, P., Theobalt, C.: Automatic face reenactment. In: IEEE Conference on Computer Vision and Pattern Recognition. (2014) 4217-4224
  11. [12]12. Thies, J., Zollh¨ ofer, M., Nießner, M., Valgaerts, L., Stamminger, M., Theobalt, C.: Realtime expression transfer for facial reenactment. ACM Transactions on Graphics (TOG) Proceedings of ACM SIGGRAPH Asia 2015 34 (6) (2015) Art. No. 183
  12. [13]13. Thies, J., Zollh¨ ofer, M., Stamminger, M., Theobalt, C., Nießner, M.: Facevr: Real-time gazeaware facial reenactment in virtual reality. ACM Transactions on Graphics 2018 (TOG) (2018)
  13. [14]14. Kholgade, N., Simon, T., Efros, A., Sheikh, Y.: 3d object manipulation in a single photograph using stock 3d models. ACM Transactions on Graphics (TOG) 33 (4) (2014) 127
  14. [16]16. Haouchine, N., Roy, F., Courtecuisse, H., Nießner, M., Cotin, S.: Calipso: Physics-based image and video editing through cad model proxies. arXiv preprint arXiv:1708.03748 (2017) · arXiv
  15. [17]17. Lu, Z., Li, Z., Cao, J., He, R., Sun, Z.: Recent progress of face image synthesis. CoRR abs/1706.04717 (2017) · arXiv
  16. [18]18. Antipov, G., Baccouche, M., Dugelay, J.: Face aging with conditional generative adversarial networks. CoRR abs/1702.01983 (2017) · arXiv
  17. [19]19. Huang, R., Zhang, S., Li, T., He, R.: Beyond face rotation: Global and local perception GAN for photorealistic and identity preserving frontal view synthesis. CoRR abs/1704.04086 (2017) · arXiv
  18. [20]20. Lu, Y., Tai, Y., Tang, C.: Conditional cyclegan for attribute guided face image generation. CoRR abs/1705.09966 (2017) · arXiv
  19. [21]21. Upchurch, P., Gardner, J.R., Bala, K., Pless, R., Snavely, N., Weinberger, K.Q.: Deep feature interpolation for image content changes. CoRR abs/1611.05507 (2016) · arXiv
  20. [22]22. Lample, G., Zeghidour, N., Usunier, N., Bordes, A., Denoyer, L., Ranzato, M.: Fader networks: Manipulating images by sliding attributes. CoRR abs/1706.00409 (2017) · arXiv
  21. [23]23. Karras, T., Aila, T., Laine, S., Lehtinen, J.: Progressive Growing of GANs for Improved Quality, Stability, and Variation. CoRR abs/1710.10196 (2017) · arXiv
  22. [24]24. Farid, H.: Photo Forensics. The MIT Press (2016)
  23. [25]25. Sencar, H.T., Memon, N.: Digital Image Forensics - There is More to a Picture than Meets the Eye. Springer (2013)
  24. [26]26. Bayar, B., Stamm, M.: A deep learning approach to universal image manipulation detection using a new convolutional layer. In: ACM Workshop on Information Hiding and Multimedia Security. (2016) 5-10
  25. [27]27. Cozzolino, D., Poggi, G., Verdoliva, L.: Recasting residual-based local descriptors as convolutional neural networks: an application to image forgery detection. In: ACM Workshop on Information Hiding and Multimedia Security. (2017) 1-6
  26. [28]28. Bondi, L., Lameri, S., G¨ uera, D., Bestagini, P., Delp, E., Tubaro, S.: Tampering Detection and Localization through Clustering of Camera-Based CNN Features. In: IEEE Computer Vision and Pattern Recognition Workshops. (2017)
  27. [29]29. Bappy, J., Roy-Chowdhury, A., Bunk, J., Nataraj, L., Manjunath, B.: Exploiting spatial structure for localizing manipulated image regions. In: IEEE International Conference on Computer Vision. (2017) 4970-4979
  28. [30]30. Wang, W., Farid, H.: Exposing Digital Forgeries in Interlaced and Deinterlaced Video. IEEE Transactions on Information Forensics and Security 2 (3) (September 2007) 438-449
  29. [31]31. Gironi, A., Fontani, M., Bianchi, T., Piva, A., Barni, M.: A video forensic technique for detection frame deletion and insertion. In: IEEE International Conference on Acoustics, Speech and Signal Processing. (2014) 6226-6230
  30. [32]32. Long, C., Smith, E., Basharat, A., Hoogs, A.: A C3D-based Convolutional Neural Network for Frame Dropping Detection in a Single Video Shot. In: IEEE Computer Vision and Pattern Recognition Workshops. (2017) 1898-1906
  31. [33]33. Ding, X., Gaobo, Y., Li, R., Zhang, L., Li, Y ., Sun, X.: Identification of Motion-Compensated Frame Rate Up-Conversion Based on Residual Signal. IEEE Transactions on Circuits and Systems for Video Technology, in press (2017)
  32. [34]34. Bestagini, P., Milani, S., Tagliasacchi, M., Tubaro, S.: Local tampering detection in video sequences. In: IEEE International Workshop on Multimedia Signal Processing. (October 2013) 488-493
  33. [35]35. D'Amiano, L., Cozzolino, D., Poggi, G., Verdoliva, L.: A PatchMatch-based Dense-field Algorithm for Video Copy-Move Detection and Localization. IEEE Transactions on Circuits and Systems for Video Technology, in press (2018)
  34. [36]36. Mullan, P., Cozzolino, D., Verdoliva, L., Riess, C.: Residual-based forensic comparison of video sequences. In: IEEE International Conference on Image Processing. (2017)
  35. [37]37. Dang-Nguyen, D.T., Boato, G., De Natale, F.: Identify computer generated characters by analysing facial expressions variation. In: IEEE International Workshop on Information Forensics and Security. (2012) 252-257
  36. [38]38. Conotter, V., Bodnari, E., Boato, G., Farid, H.: Physiologically-based detection of computer generated faces in video. In: IEEE International Conference on Image Processing. (Oct 2014) 1-5
  37. [39]39. Rahmouni, N., Nozick, V., Yamagishi, J., Echizeny, I.: Distinguishing computer graphics from natural images using convolution neural networks. In: IEEE Workshop on Information Forensics and Security. (2017) 1-6
  38. [40]40. Bharati, A., Singh, R., Vatsa, M., Bowyer, K.: Detecting facial retouching using supervised deep learning. IEEE Transactions on Information Forensics and Security 11 (9) (Sep 2016) 1903-1913
  39. [41]41. Raghavendra, R., Raja, K., Venkatesh, S., Busch, C.: Transferable Deep-CNN features for detecting digital and print-scanned morphed face images. In: IEEE Computer Vision and Pattern Recognition Workshops. (2017)
  40. [42]42. Zhou, P., Han, X., Morariu, V., Davis, L.: Two-stream neural networks for tampered face detection. In: IEEE Computer Vision and Pattern Recognition Workshops. (2017) 18311839
  41. [43]43. B¨ ohme, R., Kirchner, M.: Counter-forensics: attacking image forensics. In Sencar, H., Memon, N., eds.: Digital Image Forensics. Springer (2012)
  42. [45]45. Gloe, T., B¨ ohme, R.: The 'Dresden Image Database' for benchmarking digital image forensics. In: Proceedings of the 25th Annual ACM Symposium On Applied Computing (SAC 2010). Volume 2., Sierre, Switzerland (March 2010) 1585-1591
  43. [46]46. Shullani, D., Fontani, M., Iuliani, M., Al Shaya, O., Piva, A.: VISION: a video and image dataset for source identification. EURASIP Journal on Information Security (December 2017) 2017:15
  44. [47]47. Amerini, I., Ballan, L., Caldelli, R., Del Bimbo, A., Serra, G.: A SIFT-based forensic method for copy-move attack detection and transformation recovery. IEEE Transactions on Information Forensics and Security 6 (3) (March 2011) 1099-1110
  45. [48]48. Zampoglou, M., Papadopoulos, S., , Kompatsiaris, Y.: Detecting image splicing in the wild (Web). In: IEEE International Conference on Multimedia &amp; Expo Workshops (ICMEW). (2015)
  46. [49]49. Korus, P., Huang, J.: Multi-scale Analysis Strategies in PRNU-based Tampering Localization. IEEE Transactions on Information Forensics and Security 12 (4) (April 2017) 809-824
  47. [52]52. Abu-El-Haija, S., Kothari, N., Lee, J., Natsev, P., Toderici, G., Varadarajan, B., Vijayanarasimhan, S.: Youtube-8m: A large-scale video classification benchmark. arXiv preprint arXiv:1609.08675 (2016) · arXiv
  48. [53]53. Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: IEEE Conference on Computer Vision and Pattern Recognition. Volume 1., IEEE (2001) I-I
  49. [54]54. Chollet, F.: Xception: Deep Learning with Depthwise Separable Convolutions. In: IEEE Conference on Computer Vision and Pattern Recognition. (2017)
  50. [55]55. Fridrich, J., Kodovsk´ y, J.: Rich Models for Steganalysis of Digital Images. IEEE Transactions on Information Forensics and Security 7 (3) (June 2012) 868-882
  51. [56]56. Cozzolino, D., Gragnaniello, D., L.Verdoliva: Image forgery detection through residualbased local descriptors and block-matching. In: IEEE International Conference on Image Processing. (October 2014) 5297-5301
  52. [58]58. Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) · arXiv
  53. [59]59. Cao, Q., Shen, L., Xie, W., Parkhi, O.M., Zisserman, A.: VGGFace2: A dataset for recognising faces across pose and age. arXiv preprint arXiv:1710.08092 (2017) · arXiv
  54. [60]60. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. In: Advances in neural information processing systems. (2014) 2672-2680

Cited by

Sorted by citations

Authors
Rössler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., & Nießner, M.
ORCID
Venue
arXiv
Publisher
arXiv
Published
2018
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.1803.09179
arXiv
1803.09179
OpenAlex
W2794857359
Cited by
339
References
0
Citations by year
2026: 1 · 2025: 24 · 2024: 41 · 2023: 59 · 2022: 39 · 2021: 53 · 2020: 57 · 2019: 51 · 2018: 14
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Steganography and Watermarking Techniques
Keywords
face forensics, video forgery detection, face manipulation, synthetic face generation, image forensics, video compression, low-resolution video, face editing, benchmark dataset, image segmentation, image classification

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review