CoChat
Sign Up
PreprintCited by 221Digital Media Forensic DetectionAdvanced Steganography and Watermarking TechniquesBenford’s Law and Fraud Detection

ForensicTransfer: Weakly-supervised Domain Adaptation for Forgery Detection

Open access
Cozzolino, Davide, Thies, Justus, Rössler, Andreas, Riess, Christian, Nießner, Matthias, Verdoliva, Luisa·2018·arXiv
DOI 10.48550/arxiv.1812.02510arXiv 1812.02510
Open this paper in CoChat — free accountOpen PDF

Abstract

Distinguishing manipulated from real images is becoming increasingly difficult as new sophisticated image forgery approaches come out by the day. Naive classification approaches based on Convolutional Neural Networks (CNNs) show excellent performance in detecting image manipulations when they are trained on a specific forgery method. However, on examples from unseen manipulation approaches, their performance drops significantly. To address this limitation in transferability, we introduce Forensic-Transfer (FT). We devise a learning-based forensic detector which adapts well to new domains, i.e., novel manipulation methods and can handle scenarios where only a handful of fake examples are available during training. To this end, we learn a forensic embedding based on a novel autoencoder-based architecture that can be used to distinguish between real and fake imagery. The learned embedding acts as a form of anomaly detector; namely, an image manipulated from an unseen method will be detected as fake provided it maps sufficiently far away from the cluster of real images. Comparing to prior works, FT shows significant improvements in transferability, which we demonstrate in a series of experiments on cutting-edge benchmarks. For instance, on unseen examples, we achieve up to 85% in terms of accuracy, and with only a handful of seen examples, our performance already reaches around 95%.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.1812.02510

Cites

8 of 8 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The method proposed in [3] is very effective on face-swapping manipulations, much less on facial reenactment.

    “This approach turns out to be very effective on face-swapping manipulations, much less on facial reenactment ones.”

    From Audio-Visual Person-of-Interest DeepFake Detection (2023) · cites this paper as [3]

    The comparison runs in the opposite direction: the Datasets section (p. 5) uses Face2Face reenactment as the source domain and FaceSwap as the target. The Results (p. 8) report strong performance on seen source-domain manipulations but describe transfer to FaceSwap as a more challenging experiment with slower improvement. This does not support the claimed advantage for face-swapping over reenactment. (LLM, conf=0.84)

    Checked by a reasoning model — may be wrong.

    Experimental — accuracy not yet measured

  • Domain adaptation, meta-learning and self-supervised pretraining have been explored to mitigate cross-dataset performance drops, but these solutions often involve complex adaptation pipelines or external supervision.

    “Domain adaptation [27], meta-learning [28] and self-supervised pretraining [ 29 ] have been explored to mitigate these issues, but these solutions often involve complex adaptation pipelines or external supervision.”

    From Cross-dataset video deepfake detection using Transformer and CNN architectures (2026) · cites this paper as [27]

    The Generalization and Adaptation analyses (Section 5.2, p. 8) document performance losses on new manipulation datasets and show that ForensicTransfer improves accuracy through adaptation. The Proposed Method (Section 3, p. 4) uses a few target-domain images for fine-tuning, but the paper does not establish the broader claim that these solutions often require complex adaptation pipelines or external supervision. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • Characterizing pristine faces based on semantic features such as soft biometrics is particularly effective for POI-based detection.

    “This approach turns out to be particularly effective if the characterization of pristine faces is based on semantic features, such as soft biometrics, leading to Person-of-Interest (POI) based detection [3,5-7,12,20].”

    From Audio-Visual Person-of-Interest DeepFake Detection (2023) · cites this paper as [3]

    The Related work section (p. 2) mentions facial-manipulation detection, and D. Activation maps (p. 12) discusses detecting manipulation artifacts. Neither addresses whether semantic features such as soft biometrics make characterization of pristine faces particularly effective for Person-of-Interest detection. Searches for “soft biometrics” and identity-specific detection found no passage supporting that point. (LLM, conf=0.72)

    Experimental — accuracy not yet measured

  • Prior POI methods rely on video-only features, sometimes complemented by categorical information, neglecting audio information.

    “These methods present some limitations: in particular, they rely on video-only features, sometimes complemented by categorical information, neglecting precious audio information.”

    From Audio-Visual Person-of-Interest DeepFake Detection (2023) · cites this paper as [3]

    The Related work section (p. 2) discusses image-forgery methods that use visual clues, but does not characterize prior POI methods or their use of categorical information. Searches for POI, categorical information, and audio did not find a passage assessing whether those methods neglect audio. (LLM, conf=0.75)

    Experimental — accuracy not yet measured

  • DARPA initiated a research scheme in media forensics named Media Forensics or MediFor to accelerate the development of fake digital visual media detection methods.

    “the United States Defense Advanced Re- search Projects Agency (DARPA) initiated a research scheme in media forensics (named Media Forensics or MediFor) to accelerate the development of fake digital visual media detection methods [30].”

    From Deep learning for deepfakes creation and detection: A survey (2022) · cites this paper as [30]

    The Introduction (p. 1) says reliable detection of image forgeries is important, and Related work (p. 2) surveys media-forensics methods. Neither those passages nor searches for DARPA, MediFor, and the alleged research scheme address whether DARPA initiated the program or its purpose. (LLM, conf=0.76)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 47

  1. [3][3] D. Afchar, V. Nozick, J. Yamagishi, and I. Echizen. MesoNet: a compact facial video forgery detection network . In IEEE International Workshop on Information Forensics and Security (WIFS) , 2018. 2, 8, 12 -
  2. [4][4] S. Agarwal and H. Farid. Photo Forensics from JPEG Dimples. In IEEE International Workshop on Information Forensics and Security (WIFS) , 2017. 2 -
  3. [5][5] Hadar Averbuch-Elor, Daniel Cohen-Or, Johannes Kopf, and Michael F. Cohen. Bringing portraits to life. ACM Transactions on Graphics (Proceeding of SIGGRAPH Asia 2017) , 36(4):to appear, 2017. 1 -
  4. [6][6] J.H. Bappy, A.K. Roy-Chowdhury, J. Bunk, L. Nataraj, and B.S. Manjunath. Exploiting spatial structure for localizing manipulated image regions. In Computer Vision and Pattern Recognition Workshops , pages 4970-4979, 2017. 2 -
  5. [7][7] B. Bayar and M.C. Stamm. A deep learning approach to universal image manipulation detection using a new convolutional layer. In ACM Workshop on Information Hiding and Multimedia Security , pages 5-10, 2016. 2, 8, 12 -
  6. [8][8] M. Chen, Z. Xu, K.Q. Weinberger, and F. Sha. Marginalized denoising autoencoders for domain adaptation. In International Conference on Machine Learning , pages 1627-1634, 2012. 3 -
  7. [9][9] W.-Y. Chen, Y.-C. Liu, Z. Kira, Y.-C. Frank Wang, and J.-B. Huang. A closer look at few shot classification. In International Conference on Learning Representations , 2019. 3, 4, 11 -
  8. [13][13] D. Cozzolino and L. Verdoliva. Noiseprint: a CNN-based camera model fingerprint. IEEE Trans. Inf. Forensics Security , 15:144-159, Jan. 2020. 2 -
  9. [14][14] Gabriela Csurka. A Comprehensive Survey on Domain Adaptation for Visual Applications , pages 1-35. Springer International Publishing, 2017. 2 -
  10. [15][15] T. de Carvalho, C. Riess, E. Angelopoulou, H. Pedrini, and A. Rocha. Exposing digital image forgeries by illumination color classification. IEEE Transactions on Information Forensics and Security , 8(7):1182-1194, 2013. 2 -
  11. [16][16] H. Farid. Photo Forensics . The MIT Press, 2016. 1 -
  12. [17][17] P. Ferrara, T. Bianchi, A. De Rosa, and A. Piva. Image Forgery Localization via Fine-Grained Analysis of CFA Artifacts. IEEE Transactions on Information Forensics and Security , 7(5):1566-1577, Oct 2012. 2 -
  13. [18][18] S. Gidaris and N. Komodakis. Dynamic few-shot visual learning without forgetting. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , June 2018. 11 -
  14. [19][19] X. Glorot, A. Bordes, and Y. Bengio. Domain adaptation for large-scale sentiment classification: A deep learning approach. In International Conference on Machine Learning , pages 513520, 2011. 3 -
  15. [20][20] D. G¨ uera and E.J. Delp. Deepfake video detection using recurrent neural networks. In IEEE International Conference on Advanced Video and Signal Based Surveillance , 2018. 2 -
  16. [21][21] Judy Hoffman, Eric Tzeng, Taesung Park, Jun-Yan Zhu, Phillip Isola, Kate Saenko, Alexei A Efros, and Trevor Darrell. Cycada: Cycle-consistent adversarial domain adaptation. arXiv preprint arXiv:1711.03213 , 2017. 2 - · arXiv
  17. [22][22] J. Hu, J. Lu, and Y. Tan. Deep transfer metric learning. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages 325-333, June 2015. 4 -
  18. [23][23] M. Huh, A. Liu, A. Owens, and A.A. Efros. Fighting fake news: Image splice detection via learned self-consistency. In European Conference on Computer Vision (ECCV) , 2018. 2 -
  19. [24][24] Satoshi Iizuka, Edgar Simo-Serra, and Hiroshi Ishikawa. Globally and Locally Consistent Image Completion. ACM Transactions on Graphics (Proc. of SIGGRAPH 2017) , 36(4):107:1-107:14, 2017. 1, 5, 6, 7, 11, 12 -
  20. [25][25] Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progressive Growing of GANs for Improved Quality, Stability, and Variation. In International Conference on Learning Representations , 2018. 1, 5, 6, 7 -
  21. [27][27] H. Kim, P. Garrido, A. Tewari, W. Xu, J. Thies M. Nießner, P. P´ erez, C. Richardt, M. Zollh¨ ofer, and C. Theobalt. Deep Video Portraits. ACM Transactions on Graphics (TOG) , 2018. 1 -
  22. [28][28] D.P. Kingma and P. Dhariwal. Glow: Generative flow with invertible 1x1 convolutions. arXiv preprint arXiv:1807.03039 , 2018. 1, 5, 7, 8 - · arXiv
  23. [29][29] E. Kodirov, T. Xiang, and S. Gong. Semantic autoencoder for zero-shot learning. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages 4447-4456, 2017. 3 -
  24. [30][30] X. Liang, H. Zhang, L. Lin, and E. Xing. Generative semantic manipulation with mask-contrasting gan. In European Computer Vision Conference (ECCV) , 2018. 1 -
  25. [31][31] W. Liu et al. Large-margin softmax loss for convolutional neural networks. In ICML , 2016. 6 -
  26. [32][32] Y. Liu, Q. Guan, X. Zhao, and Y. Cao. Image forgery localization based on multi-scale convolutional neural networks. In ACM Workshop on Information Hiding and Multimedia Security , 2018. 2 -
  27. [33][33] Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In IEEE International Conference on Computer Vision (ICCV) , pages 37303738, Dec 2015. 5 -
  28. [34][34] S. Lyu, X. Pan, and X. Zhang. Exposing region splicing forgeries with blind local noise estimation. International Journal of Computer Vision , 110(2):202-221, 2014. 2 -
  29. [36][36] S. Motiian, Q. Jones, S. Iranmanesh, and G. Doretto. Fewshot adversarial domain adaptation. In Advances in Neural Information Processing Systems (NIPS) , pages 6670-6680. 2017. 11 -
  30. [37][37] A. Pumarola, A. Agudo, A.M. Martinez, A. Sanfeliu, and F. Moreno-Noguer. GANimation: Anatomically-aware Facial Animation from a Single Image. In European Conference on Computer Vision (ECCV) , 2018. 1 -
  31. [38][38] N. Rahmouni, V. Nozick, J. Yamagishi, and I. Echizeny. Distinguishing computer graphics from natural images using convolution neural networks. In IEEE Workshop on Information Forensics and Security , pages 1-6, 2017. 2, 8, 12 -
  32. [39][39] Y. Rao and J. Ni. A deep learning approach to detection of splicing and copy-move forgeries in images. In IEEE International Workshop on Information Forensics and Security (WIFS) , pages 1-6, 2016. 2 -
  33. [40][40] A. R¨ ossler, D. Cozzolino, L. Verdoliva, C. Riess, J. Thies, and M. Nießner. Faceforensics: A large-scale video dataset for forgery detection in human faces. arXiv preprint arXiv:1803.09179 , 2018. 2, 5, 8 - · arXiv
  34. [41][41] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision , 115(3):211-252, Dec 2015. 5 -
  35. [42][42] R. Salloum, Y. Ren, and C. C. Jay Kuo. Image Splicing Localization using a Multi-task Fully Convolutional Network (MFCN). Journal of Visual Communication and Image Representation , 51:201-209, 2018. 2 -
  36. [43][43] J. Snell, K. Swersky, and R. Zemel. Prototypical networks for few-shot learning. In Advances in Neural Information Processing Systems (NIPS) , pages 4077-4087. 2017. 11 -
  37. [44][44] F. Sung, Y. Yang, L. Zhang, T. Xiang, P. H.S. Torr, and T. M. Hospedales. Learning to compare: Relation network for fewshot learning. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , June 2018. 11 -
  38. [45][45] Supasorn Suwajanakorn, Steven M Seitz, and Ira Kemelmacher-Shlizerman. Synthesizing Obama: learning lip sync from audio. ACM Transactions on Graphics (TOG) , 36(4):95, 2017. 1 -
  39. [46][46] C. Szegedy, Wei Liu, Yangqing Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich. Going deeper with convolutions. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages 1-9, June 2015. 12 -
  40. [47][47] J. Thies, M. Zollh¨ ofer, M. Stamminger, C. Theobalt, and M. Nießner. Face2Face: Real-Time Face Capture and Reenactment of RGB Videos. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages 2387-2395, June 2016. 1, 5, 7, 12, 13 -
  41. [48][48] M. Tschannen, O. Bachem, and M. Lucic. Recent advances in autoencoder-based representation learning. arXiv preprint arXiv:1812.05069v1 , 2018. 3 - · arXiv
  42. [50][50] O. Vinyals, C. Blundell, T. Lillicrap, K. Kavukcuoglu, and D. Wierstra. Matching networks for one shot learning. In Advances in Neural Information Processing Systems (NIPS) , pages 3630-3638. 2016. 11 -
  43. [51][51] I. Yerushalmy and H. Hel-Or. Digital image forgery detection based on lens and sensor aberration. International Journal of Computer Vision , 92(1):71-91, 2018. 2 -
  44. [52][52] Jiahui Yu, Zhe Lin, Jimei Yang, Xiaohui Shen, Xin Lu, and Thomas S Huang. Generative image inpainting with contextual attention. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2018. 5, 6, 7, 11, 12 -
  45. [54][54] Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. Learning deep features for discriminative localization. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages 2921-2929, 2016. 12 -
  46. [55][55] P. Zhou, X. Han, V.I. Morariu, and L.S. Davis. Learning rich features for image manipulation detection. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2018. 2 -
  47. [57][57] J.-Y. Zhu, P. Kr¨ ahenb¨ uhl, E. Shechtman, and A.A. Efros. Learning a Discriminative Model for the Perception of Realism in Composite Images. In IEEE International Conference on Computer Vision (ICCV) , 2015. 1

Cited by

Sorted by citations

Authors
Cozzolino, D., Thies, J., Rössler, A., Riess, C., Nießner, M., & Verdoliva, L.
ORCID
Venue
arXiv
Publisher
arXiv
Published
2018
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.1812.02510
arXiv
1812.02510
OpenAlex
W2902304528
Cited by
221
References
8
Citations by year
2026: 1 · 2025: 12 · 2024: 28 · 2023: 32 · 2022: 32 · 2021: 53 · 2020: 41 · 2019: 20 · 2018: 2
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Steganography and Watermarking Techniques
  • Physical Sciences › Mathematics › Statistics and Probability › Benford’s Law and Fraud Detection
Keywords
image forgery detection, anomaly detection, transferability, forensic image analysis

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review