CoChat
Sign Up
Cited by 93Digital Media Forensic DetectionMisinformation and Its ImpactsImage and Video Quality Assessment

LAA-Net: Localized Artifact Attention Network for Quality-Agnostic and Generalizable Deepfake Detection

Open access
NGUYEN, Dat, MEJRI, Nesryne, SINGH, Inder Pal, KULESHOVA, Polina, ASTRID, Marcella, KACEM, Anis, GHORBEL, Enjie, AOUADA, Djamila·2024·2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
DOI 10.1109/cvpr52733.2024.01647
Open this paper in CoChat — free accountOpen PDF

Abstract

This paper introduces a novel approach for high-quality deepfake detection called Localized Artifact Attention Net-work (LAA-Net). Existing methods for high-quality deep-fake detection are mainly based on a supervised binary classifier coupled with an implicit attention mechanism. As a result, they do not generalize well to unseen ma-nipulations. To handle this issue, two main contributions are made. First, an explicit attention mechanism within a multi-task learning framework is proposed. By combining heatmap-based and self-consistency attention strate-gies, LAA-Net is forced to focus on a few small artifact-prone vulnerable regions. Second, an Enhanced Feature Pyramid Network (E-FPN) is proposed as a simple and ef-fective mechanism for spreading discriminative low-level features into the final feature output, with the advantage of limiting redundancy. Experiments performed on sev-eral benchmarks show the superiority of our approach in terms of Area Under the Curve (AUC) and Average Preci-sion (AP). The code is available at https://github.com/10Ring/LAA-Net.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/cvpr52733.2024.01647

Cites

15 of 15 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • LAA-Net (w/ SBI) achieves an AUC of 45.4% on collected AI avatar videos.

    “Our method achieves an AUC of 86.1%, while the RealForensics method achieves an AUC of 84.9%, and the LAA-Net (w/ SBI) method achieves only 45.4% AUC.”

    From Towards More General Video-based Deepfake Detection through Facial Component Guided Adaptation for Foundation Model (2024) · cites this paper as [32]

    Section 4.1 (page unavailable) lists the datasets used to evaluate LAA-Net, but does not include collected AI avatar videos. I searched our copy for 45.4% AUC, AI avatar videos, and RealForensics; no passage reports the claimed result on that dataset. (LLM, conf=0.76)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 37

  1. Darius Afchar, Vincent Nozick, Junichi Yamagishi, and Isao Echizen. Mesonet: a compact facial video forgery detection network. CoRR, abs/1809.00888, 2018. · arXiv
  2. Babajide O Ayinde, Tamer Inanc, and Jacek M Zurada. Reg- ularizing deep neural networks by enhancing diversity in fea- ture extraction. IEEE transactions on neural networks and learning systems, 30(9):2650-2661, 2019. 2, 5
  3. Weiming Bai, Yufan Liu, Zhipeng Zhang, Bing Li, and Weiming Hu. Aunet: Learning relations between action units for face forgery detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 24709-24719, 2023. 2, 3, 6, 7
  4. Zhixi Cai, Shreya Ghosh, Kalin Stefanov, Abhinav Dhall, Jianfei Cai, Hamid Rezatofighi, Reza Haffari, and Munawar Hayat. Marlin: Masked autoencoder for facial video repre- sentation learning. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 1493-1504, 2023. 2
  5. Shen Chen, Taiping Yao, Yang Chen, Shouhong Ding, Jilin Li, and R. Ji. Local relation learning for face forgery de- tection. In AAAI Conference on Artificial Intelligence, 2021. 6
  6. Franc ¸ois Chollet. Xception: Deep learning with depthwise separable convolutions. In Proceedings of the IEEE con- ference on computer vision and pattern recognition, pages 1251-1258, 2017. 2, 3, 7
  7. Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, K. Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248-255, 2009. 6
  8. Pierre Foret, Ariel Kleiner, Hossein Mobahi, and Behnam Neyshabur. Sharpness-aware minimization for efficiently improving generalization. CoRR, abs/2010.01412, 2020. 7 · arXiv
  9. Ying Guo, Cheng Zhen, and Pengfei Yan. Controllable guide-space for generalizable face forgery detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 20818-20827, 2023. 6
  10. Baojin Huang, Zhongyuan Wang, Jifan Yang, Jiaxin Ai, Qin Zou, Qian Wang, and Dengpan Ye. Implicit identity driven deepfake face swapping detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4490-4499, 2023. 6
  11. Hei Law and Jia Deng. Cornernet: Detecting objects as paired keypoints. International Journal of Computer Vision, 128:642-656, 2018. 4
  12. Lingzhi Li, Jianmin Bao, Ting Zhang, Hao Yang, Dong Chen, Fang Wen, and Baining Guo. Face x-ray for more general face forgery detection. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020. 1, 2, 3, 4, 6, 7
  13. Yuezun Li, Xin Yang, Pu Sun, Honggang Qi, and Siwei Lyu. Celeb-df: A new dataset for deepfake forensics. CoRR, abs/1909.12962, 2019. 1, 2, 6 · arXiv
  14. Tsung-Yi Lin, Priya Goyal, Ross B. Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. CoRR, abs/1708.02002, 2017. 4, 5, 6, 8 · arXiv
  15. Tsung-Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie. Feature pyra- mid networks for object detection. In Proceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 2117-2125, 2017. 2, 5, 8
  16. Baoping Liu, Bo Liu, Ming Ding, Tianqing Zhu, and Xin Yu. Ti2net: Temporal identity inconsistency network for deepfake detection. In Proceedings of the IEEE/CVF Win- ter Conference on Applications of Computer Vision (WACV), pages 4691-4700, 2023. 6
  17. Liqian Ma, Xu Jia, Qianru Sun, Bernt Schiele, Tinne Tuyte- laars, and Luc Van Gool. Pose guided person image gener- ation. Advances in neural information processing systems, 30, 2017. 2, 6
  18. Youssef Mansour and Reinhard Heckel. Zero-shot noise2noise: Efficient image denoising without any data. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition (CVPR), pages 14018-14027, 2023.
  19. Falko Matern, Christian Riess, and Marc Stamminger. Ex- ploiting visual artifacts to expose deepfakes and face manip- ulations. In 2019 IEEE Winter Applications of Computer Vision Workshops (WACVW), pages 83-92, 2019. 2
  20. Nesryne Mejri, Konstantinos Papadopoulos, and Djamila Aouada. Leveraging high-frequency components for deep- fake detection. In 2021 IEEE 23rd International Workshop on Multimedia Signal Processing (MMSP), pages 1-6, 2021.
  21. Nesryne Mejri, Enjie Ghorbel, and Djamila Aouada. Untag: Learning generic features for unsupervised type-agnostic deepfake detection. In ICASSP 2023 -2023 IEEE Interna- tional Conference on Acoustics, Speech and Signal Process- ing (ICASSP), pages 1-5, 2023. 2
  22. Rafael Müller, Simon Kornblith, and Geoffrey E. Hinton. When does label smoothing help? CoRR, abs/1906.02629, 2019. 7 · arXiv
  23. Huy H. Nguyen, Junichi Yamagishi, and Isao Echizen. Capsule-forensics: Using capsule networks to detect forged images and videos. CoRR, abs/1810.11215, 2018. 2 · arXiv
  24. Thanh Thi Nguyen, Cuong M. Nguyen, Dung Tien Nguyen, Duc Thanh Nguyen, and Saeid Nahavandi. Deep learning for deepfakes creation and detection. CoRR, abs/1909.11573, 2019. 2 · arXiv
  25. Andreas Rössler, Davide Cozzolino, Luisa Verdoliva, Chris- tian Riess, Justus Thies, and Matthias Nießner. FaceForen- sics++: Learning to detect manipulated facial images. In International Conference on Computer Vision (ICCV), 2019. 1, 2, 5, 6, 7, 8
  26. Selim S. Seferbekov, Vladimir I. Iglovikov, Alexander V. Buslaev, and Alexey A. Shvets. Feature pyramid network for multi-class land segmentation. CoRR, abs/1806.03510, 2018. 5 · arXiv
  27. Ramprasaath R. Selvaraju, Abhishek Das, Ramakrishna Vedantam, Michael Cogswell, Devi Parikh, and Dhruv Ba- tra. Grad-cam: Why did you say that? visual explanations from deep networks via gradient-based localization. CoRR, abs/1610.02391, 2016. 7, 8 · arXiv
  28. Zehua Sheng, Zhu Yu, Xiongwei Liu, Si-Yuan Cao, Yuqi Liu, Hui-Liang Shen, and Huaqi Zhang. Structure aggre- gation for cross-spectral stereo image guided denoising. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition (CVPR), pages 13997-14006, 2023. 7
  29. Inder Pal Singh, Nesryne Mejri, van Dat Nguyen, Enjie Ghorbel, and Djamila Aouada. Multi-label deepfake classi- fication. IEEE Workshop on Multimedia Signal Processing, 2023. 2
  30. Zekun Sun, Yujie Han, Zeyu Hua, Na Ruan, and Weijia Jia. Improving the efficiency and robustness of deepfakes detec- tion through precise geometric features. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 3608-3617, 2021. 6
  31. Justus Thies, Michael Zollhöfer, and Matthias Nießner. De- ferred neural rendering: Image synthesis using neural tex- tures. CoRR, abs/1904.12356, 2019. 5 · arXiv
  32. Justus Thies, Michael Zollhöfer, Marc Stamminger, Chris- tian Theobalt, and Matthias Nießner. Face2face: Real- time face capture and reenactment of RGB videos. CoRR, abs/2007.14808, 2020. 5 · arXiv
  33. Run Wang, Lei Ma, Felix Juefei-Xu, Xiaofei Xie, Jian Wang, and Yang Liu. Fakespotter: A simple baseline for spotting ai-synthesized fake faces. CoRR, abs/1909.06122, 2019. 2 · arXiv
  34. Yuan Wang, Kun Yu, Chen Chen, Xiyuan Hu, and Silong Peng. Dynamic graph learning with content-guided spatial- frequency relation reasoning for deepfake detection. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 7278-7287, 2023. 2, 3, 6, 7
  35. Zhiyuan Yan, Yong Zhang, Yanbo Fan, and Baoyuan Wu. Ucf: Uncovering common features for generalizable deep- fake detection. In Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision (ICCV), pages 22412-22423, 2023. 6
  36. Hanqing Zhao, Wenbo Zhou, Dongdong Chen, Tianyi Wei, Weiming Zhang, and Nenghai Yu. Multi-attentional deep- fake detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2185- 2194, 2021. 1, 2, 3, 5, 6, 7, 8
  37. Bojia Zi, Minghao Chang, Jingjing Chen, Xingjun Ma, and Yu-Gang Jiang. Wilddeepfake: A challenging real-world dataset for deepfake detection. Proceedings of the 28th ACM International Conference on Multimedia, 2020. 2, 6

Cited by

Sorted by citations

Authors
NGUYEN, D., MEJRI, N., SINGH, I. P., KULESHOVA, P., ASTRID, M., KACEM, A., GHORBEL, E., & AOUADA, D.
Affiliations
  • University of Luxembourg (LU)
  • National School of Computer Science (TN)
ORCID
Venue
2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Publisher
IEEE
Published
2024-06-16
Pages
17395-17405
ISSN
1063-6919 (ISSN-L)
DOI
10.1109/cvpr52733.2024.01647
OpenAlex
W4402715910
Cited by
93
References
15
Citations by year
2026: 45 · 2025: 40 · 2024: 8
FWCI
17.30
Type
conference paper
Language
en
Open access
Licence
Locations
  • IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, cc-by-nc-nd)
  • Open Repository and Bibliography (University of Luxembourg) (submittedVersion, other-oa)
  • arXiv (Cornell University)
Funders
Fonds National de la Recherche Luxembourg (BRIDGES2021/IS/16353350/FaKeDeTeR)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
  • Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image and Video Quality Assessment
Keywords
deepfake detection, attention mechanism, multi-task learning, feature pyramid network, video manipulation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review