The DOI matches its Crossref record; 160 of the 160 references it cites were found in a registry or an index.
The Creation and Detection of Deepfakes: A Survey
Open accessAbstract
Generative deep learning algorithms have progressed to a point where it is difficult to tell the difference between what is real and what is fake. In 2018, it was discovered how easy it is to use this technology for unethical and malicious applications, such as the spread of misinformation, impersonation of political leaders, and the defamation of innocent individuals. Since then, these “deepfakes” have advanced significantly. In this article, we explore the creation and detection of deepfakes and provide an in-depth view as to how these architectures work. The purpose of this survey is to provide the reader with a deeper understanding of (1) how deepfakes are created and detected, (2) the current trends and advancements in this domain, (3) the shortcomings of the current defense solutions, and (4) the areas that require further research and attention.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“The survey lacks discussion on the technical challenges associated with generation and detection systems.”
From Deepfake video detection: challenges and opportunities (2024) · cites this paper as Mirsky and Lee 2021
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Furthermore, it has been proven that deep neural networks tend to achieve better performance than traditional image forensic tools [9].”
From DFDT: An End-to-End DeepFake Detection Framework Using Vision Transformer (2022) · cites this paper as [9]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Faceswapping can also be difficult to identify it as a deepfake when the pictures artifacts are hidden behind the compression artifacts (Mirsky and Lee, 2021).”
From Audio deepfakes: A survey (2023) · cites this paper as Mirsky and Lee, 2021
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“While there are several surveys of passive DF detection that examine specific modalities in isolation [5-9], the field requires a holistic analysis that transcends these single approaches.”
From Deepfake detection across image, video, and audio: a comprehensive survey with empirical evaluation of generalization and robustness (2026) · cites this paper as [6]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Current survey papers [5-9, 15, 15, 16] primarily focus on reviewing detection methods based on their accuracy performance within the same dataset (within-domain evaluation), which achieves high accuracy (often exceeding 95%).”
From Deepfake detection across image, video, and audio: a comprehensive survey with empirical evaluation of generalization and robustness (2026) · cites this paper as [6]
Experimental — accuracy not yet measured
8 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Image-to-Image Translation with Conditional Adversarial Networks
Isola, P., Zhu, J.-Y., Zhou, T., & Efros, A. A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.632 · Cited by 22,930 · cited 1×
“One popular version is the pix2pix framework which enables translations from one image domain to another [12].”
BackgroundPerceptual Losses for Real-Time Style Transfer and Super-Resolution
Johnson, J., Alahi, A., & Fei-Fei, L. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46475-6_43 · Cited by 11,283
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
Finn, C., Abbeel, P., & Levine, S. (2017) · arXiv · doi:10.48550/arxiv.1703.03400 · Cited by 8,512 · cited 1×
“4) Few-Shot Learning: In [85], the authors demonstrate the few-shot learning technique of [86] on a pix2pixHD network and the network of [82].”
BackgroundAccurate Image Super-Resolution Using Very Deep Convolutional Networks
Kim, J., Lee, J. K., & Lee, K. M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.182 · Cited by 7,870 · cited 1×
“s [111] can be evaded by reenacting the entire head and by learning the mannerisms from the same databases. Models which identify blurred content [115] are affected by noise and sharpening GANs [42], [151], and models which search for the boundary where the face was blended in [112]–[117] do not work on deepfakes pass…”
BackgroundImproved Techniques for Training GANs
Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., & Chen, X. (2016) · arXiv · doi:10.48550/arxiv.1606.03498 · Cited by 6,965 · cited 1×
“) Similar to Lperc , there is a feature matching loss (LFM ) [133] which uses the last output of a network.”
BackgroundAnalyzing and Improving the Image Quality of StyleGAN
Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J., & Aila, T. (2019) · arXiv · doi:10.48550/arxiv.1912.04958 · Cited by 6,332 · cited 1×
“Human face and body synthesis techniques such as [78] (used in Fig.”
MethodAnalyzing and Improving the Image Quality of StyleGAN
Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J., & Aila, T. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00813 · Cited by 6,332
A morphable model for the synthesis of 3D faces
Blanz, V., & Vetter, T. (1999) · Proceedings of the 26th annual conference on Computer graphics and interactive techniques - SIGGRAPH '99 · doi:10.1145/311535.311556 · Cited by 4,710
High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs
Wang, T.-C., Liu, M.-Y., Zhu, J.-Y., Tao, A., Kautz, J., & Catanzaro, B. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00917 · Cited by 4,332
StarGAN: Unified Generative Adversarial Networks for Multi-domain Image-to-Image Translation
Choi, Y., Choi, M., Kim, M., Ha, J.-W., Kim, S., & Choo, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00916 · Cited by 4,163
FaceForensics++: Learning to Detect Manipulated Facial Images
Rossler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., & Niessner, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00009 · Cited by 3,314 · cited 2×
“In [106, 119, 131], it was shown that deep neural networks tend to perform beer than traditional image forensic tools on compressed imagery.”
BackgroundMesoNet: a Compact Facial Video Forgery Detection Network
Afchar, D., Nozick, V., Yamagishi, J., & Echizen, I. (2018) · 2018 IEEE International Workshop on Information Forensics and Security (WIFS) · doi:10.1109/wifs.2018.8630761 · Cited by 1,872
End-to-End Recovery of Human Shape and Pose
Kanazawa, A., Black, M. J., Jacobs, D. W., & Malik, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00744 · Cited by 1,870
DensePose: Dense Human Pose Estimation in the Wild.
Güler, R. A., Neverova, N., & Kokkinos, I. (2018) · UCL Discovery (University College London) · Cited by 1,421
Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
Papernot, N., McDaniel, P., & Goodfellow, I. (2016) · arXiv · doi:10.48550/arxiv.1605.07277 · Cited by 1,326 · cited 1×
“Advances in adversarial machine learning has shown that these attacks transfer across multiple models regardless of the training data used [125].”
BackgroundFace X-Ray for More General Face Forgery Detection
Li, L., Bao, J., Zhang, T., Yang, H., Chen, D., Wen, F., & Guo, B. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00505 · Cited by 1,211
CNN-Generated Images Are Surprisingly Easy to Spot… for Now
Wang, S.-Y., Wang, O., Zhang, R., Owens, A., & Efros, A. A. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00872 · Cited by 1,210
Exposing Deep Fakes Using Inconsistent Head Poses
Yang, X., Li, Y., & Lyu, S. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8683164 · Cited by 1,196
In Ictu Oculi: Exposing AI Created Fake Videos by Detecting Eye Blinking
Li, Y., Chang, M.-C., & Lyu, S. (2018) · 2018 IEEE International Workshop on Information Forensics and Security (WIFS) · doi:10.1109/wifs.2018.8630787 · Cited by 1,089 · cited 3×
“Regarding deepfakes, [30] monitored blood volume paerns (pulse) under the skin, and [97] took a more robust approach by monitoring irregular eye blinking paerns.”
BackgroundInfluentialMoCoGAN: Decomposing Motion and Content for Video Generation
Tulyakov, S., Liu, M.-Y., Yang, X., & Kautz, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00165 · Cited by 1,076
Synthesizing Obama: learning lip sync from audio
Suwajanakorn, S., Seitz, S. M., & Kemelmacher-Shlizerman, I. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073640 · Cited by 1,057 · cited 7×
“Later that year, the authors of [89] presented ObamaNet: a network that reenacts an individual’s mouth and voice using text as input instead of audio like [152].”
BackgroundMethodInfluentialCapsule-forensics: Using Capsule Networks to Detect Forged Images and Videos
Nguyen, H. H., Yamagishi, J., & Echizen, I. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8682602 · Cited by 951
Deep Fakes: A Looming Challenge for Privacy, Democracy, and National Security
Chesney, R., & Citron, D. K. (2018) · SSRN Electronic Journal · doi:10.2139/ssrn.3213954 · Cited by 880 · cited 1×
“Although human image editing and synthesis are active research topics, reenactment and replacement deepfakes are the greatest concern because they give an aacker control over one’s identity[12, 28, 66].”
BackgroundExposing DeepFake Videos By Detecting Face Warping Artifacts
Li, Y., & Lyu, S. (2018) · arXiv · doi:10.48550/arxiv.1811.00656 · Cited by 785 · cited 2×
“4 [153] 2019 • • • • CNN 1024x1024 • 94 [30] 2019 • • • • CNN 128x128 • • 96 [99] 2019 • • • • • CNN • 224x224 • 93.”
BackgroundOn the Detection of Digital Face Manipulation
Dang, H., Liu, F., Stehouwer, J., Liu, X., & Jain, A. K. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00582 · Cited by 764
Chan, C., Ginosar, S., Zhou, T., & Efros, A. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00603 · Cited by 737
FSGAN: Subject Agnostic Face Swapping and Reenactment
Nirkin, Y., Keller, Y., & Hassner, T. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00728 · Cited by 706 · cited 10×
“L1: xg, xt ′ L1: e I , Ƹ e L1: e , Ƹ e Lpixel−verif: xg, xt , xs LCE De, C : y, ො x Ladv 1: y1, Τ x xg Ladv 2: y2, Τ x xg Ladv I1: yI1, Τ x xg Ladv I2: yI2, Τ x xg 2÷ 2÷ [121] FSGAN: m: segmentation mask (face, hair, other), l: 3D facial landmarks, H: n passes through H while interpolating ls to lt”
BackgroundMethodInfluentialFew-Shot Adversarial Learning of Realistic Neural Talking Head Models
Zakharov, E., Shysheya, A., Burkov, E., & Lempitsky, V. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00955 · Cited by 687 · cited 7×
“Ladv: y, kt , ൗ xg xt (i) LFM: xg ∙ vst, xt (i) ∙ vst [183] Neural TalkingHeads:”
BackgroundInfluentialWang, T.-C., Liu, M.-Y., Zhu, J.-Y., Liu, G., Tao, A., Kautz, J., & Catanzaro, B. (2018) · arXiv · doi:10.48550/arxiv.1808.06601 · Cited by 594 · cited 1×
“of an RNN T Ú :5; T Ú :6; T Ú :Ä; V A ç )4 7 5 )4 7 6 )4 7 Ä A ë :5; A ë :6; A ë :Ä; U æ ) Í ) Í ) Í Sample 1 Sample T L: Ú : ç ñ & Ï U Ï & Â U Â Image Disc. Video Disc. [21] Vid2Vid: L adv;L FM;L H æ :Ü?Å;áå æ :Ü; :Ü?Å ;áå 5 T æ :Ü?Å ;áå T .' * Intermediate Synth. Occlusion Masking Warp Field Pred. / 9 D…”
Demo of Face2Face: real-time face capture and reenactment of RGB videos
Thies, J., Zollhöfer, M., Stamminger, M., Theobalt, C., & Nießner, M. (2016) · ACM SIGGRAPH 2016 Emerging Technologies · doi:10.1145/2929464.2929475 · Cited by 589
First Order Motion Model for Image Animation
Siarohin, A., Lathuilière, S., Tulyakov, S., Ricci, E., & Sebe, N. (2020) · arXiv · doi:10.48550/arxiv.2003.00196 · Cited by 585 · cited 6×
“For example, Vid2Vid [168, 169], MocoGAN [162], and others [144, 145].”
BackgroundMethodInfluentialCVAE-GAN: Fine-Grained Image Generation through Asymmetric Training
Bao, J., Chen, D., Wen, F., Li, H., & Hua, G. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.299 · Cited by 569
Defending Against Neural Fake News
Zellers, R., Holtzman, A., Rashkin, H., Bisk, Y., Farhadi, A., Roesner, F., & Choi, Y. (2019) · arXiv · doi:10.48550/arxiv.1905.12616 · Cited by 569 · cited 1×
“Finally, it has been shown how deepfakes of news articles can be generated [184] and that deepfake tweets exist as well [50].”
BackgroundMulti-task Learning For Detecting and Segmenting Manipulated Facial Images and Videos
Nguyen, H. H., Fang, F., Yamagishi, J., & Echizen, I. (2019) · arXiv · doi:10.48550/arxiv.1906.06876 · Cited by 551 · cited 2×
“9 [118] 2019 • • • • • CNN-DE • 256x256 • • 92.”
BackgroundTransfer learning from speaker verification to multispeaker text-to-speech synthesis
Jia, Y., Zhang, Y., Weiss, R. J., Wang, Q., Shen, J., Ren, F., Chen, Z., Nguyen, P., Pang, R., Moreno, I. L., et al. (2018) · arXiv · doi:10.48550/arxiv.1806.04558 · Cited by 519 · cited 3×
“For example, by finding a bound on a model’s delay can help detect realtime attacks such as [154], and determining the limits of GANs like [159] can help us devise the appropriate strategies.”
BackgroundMethodInfluentialTwo-branch Recurrent Network for Isolating Deepfakes in Videos
Masi, I., Killekar, A., Mascarenhas, R., Gurudatt, S. P., & Abdalmageed, W. (2020) · European Conference on Computer Vision · doi:10.48550/arxiv.2008.03412 · Cited by 506 · cited 2×
“To focus on the residuals, the authors of [108] use a two stream ED to encode the color image and a frequency enhanced version using “Laplacian of Gaussian layers” (LoG).”
MethodFast Face-Swap Using Convolutional Neural Networks
Korshunova, I., Shi, W., Dambre, J., & Theis, L. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.397 · Cited by 501
FakeCatcher: Detection of Synthetic Portrait Videos using Biological Signals
Ciftci, U. A., Demir, I., & Yin, L. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3009287 · Cited by 499
DeepFakes: a New Threat to Face Recognition? Assessment and Detection
Korshunov, P., & Marcel, S. (2018) · arXiv · doi:10.48550/arxiv.1812.08685 · Cited by 497 · cited 1×
“For example, residuals from face warping processes [97, 98, 100], lighting [149], and varying fidelity [86] can indicate the presence of generated content.”
BackgroundDeformable GANs for Pose-Based Human Image Generation
Siarohin, A., Sangineto, E., Lathuiliere, S., & Sebe, N. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00359 · Cited by 489
Kim, H., Garrido, P., Tewari, A., Xu, W., Thies, J., Niessner, M., Pérez, P., Richardt, C., Zollhöfer, M., & Theobalt, C. (2018) · ACM Transactions on Graphics · doi:10.1145/3197517.3201283 · Cited by 475 · cited 9×
“video 1 1 1 1 • • • • text - • 256x256 [83] 2018 Deep Video Portr.”
BackgroundMethodResultInfluentialTwo-Branch Recurrent Network for Isolating Deepfakes in Videos
Masi, I., Killekar, A., Mascarenhas, R. M., Gurudatt, S. P., & AbdAlmageed, W. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58571-6_39 · Cited by 471
Neural Voice Puppetry: Audio-driven Facial Reenactment
Thies, J., Elgharib, M. A., Tewari, A., Theobalt, C., & Niessner, M. (2019) · European Conference on Computer Vision · doi:10.48550/arxiv.1912.05566 · Cited by 464 · cited 7×
“, [55, 154, 170]).”
BackgroundMethodInfluentialRecurrent convolutional strategies for face manipulation detection in videos
Sabir, E., Cheng, J., Jaiswal, A., AbdAlmageed, W., Masi, I., & Natarajan, P. (2019) · arXiv · doi:10.48550/arxiv.1905.00582 · Cited by 443
Hierarchical Cross-Modal Talking Face Generation With Dynamic Pixel-Wise Loss
Chen, L., Maddox, R. K., Duan, Z., & Xu, C. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00802 · Cited by 437 · cited 3×
“[27] ATVGnet:”
BackgroundAttributing Fake Images to GANs: Learning and Analyzing GAN Fingerprints
Yu, N., Davis, L., & Fritz, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00765 · Cited by 437 · cited 3×
“[156] 2019 • • • • • CNN 128x128 • • • 99.”
X2Face: A Network for Controlling Face Generation Using Images, Audio, and Pose Codes
Wiles, O., Koepke, A. S., & Zisserman, A. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01261-8_41 · Cited by 435
Deepfake Video Detection through Optical Flow Based CNN
Amerini, I., Galteri, L., Caldelli, R., & Del Bimbo, A. (2019) · 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW) · doi:10.1109/iccvw.2019.00152 · Cited by 424 · cited 2×
“In [25] a classier is trained pairs of sequential frames and in [11] the authors rene the network’s focus by monitoring the frames’ optical ow.”
Talking Face Generation by Adversarially Disentangled Audio-Visual Representation
Zhou, H., Liu, Y., Liu, Z., Luo, P., & Wang, X. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33019299 · Cited by 416
The Deepfake Detection Challenge (DFDC) Preview Dataset
Dolhansky, B., Howes, R., Pflaum, B., Baram, N., & Ferrer, C. C. (2019) · arXiv · doi:10.48550/arxiv.1910.08854 · Cited by 413
Detection of GAN-Generated Fake Images over Social Networks
Marra, F., Gragnaniello, D., Cozzolino, D., & Verdoliva, L. (2018) · 2018 IEEE Conference on Multimedia Information Processing and Retrieval (MIPR) · doi:10.1109/mipr.2018.00084 · Cited by 399 · cited 2×
“In [106, 119, 131], it was shown that deep neural networks tend to perform beer than traditional image forensic tools on compressed imagery.”
BackgroundProgressive Pose Attention Transfer for Person Image Generation
Zhu, Z., Huang, T., Shi, B., Yu, M., Wang, B., & Bai, X. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00245 · Cited by 376 · cited 2×
“authors also propose a novel nearest neighbor loss instead of using L1 or L2 losses. To modify unseen identities at test time, an encoding of x is passed to the decoder’s inner layers. The authors of [79] propose using a GAN that uses novel Pose-Attentional Transfer blocks (PATB) inside the generator. The architecture…”
Real-time expression transfer for facial reenactment
Thies, J., Zollhöfer, M., Nießner, M., Valgaerts, L., Stamminger, M., & Theobalt, C. (2015) · ACM Transactions on Graphics · doi:10.1145/2816795.2818056 · Cited by 369 · cited 1×
“t a 3D model [53]. Later, between 2015 and 2018, Thies et al. demonstrated how 3D parametric models can be used to achieve high quality and real-time results with depth sensing and ordinary cameras ( [54] and [55], [56]). Regardless, today deep learning approaches are recognized as the simplest way to generate believa…”
BackgroundDo GANs leave artificial fingerprints?
Marra, F., Gragnaniello, D., Verdoliva, L., & Poggi, G. (2019) · 2019 IEEE Conference on Multimedia Information Processing and Retrieval (MIPR) · doi:10.1109/mipr.2019.00103 · Cited by 361
Animating Arbitrary Objects via Deep Motion Transfer
Siarohin, A., Lathuilière, S., Tulyakov, S., Ricci, E., & Sebe, N. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00248 · Cited by 349 · cited 3×
“In contrast, the authors of [144] propose Monkey-Net: a self supervised network for driving an image with an arbitrary video sequence.”
BackgroundCombating Deepfake Videos Using Blockchain and Smart Contracts
Hasan, H. R., & Salah, K. (2019) · IEEE Access · doi:10.1109/access.2019.2905689 · Cited by 339 · cited 2×
“In contrast, [68] proposes that the content should authenticated and managed as a global le system over Etherium smart contracts.”
BackgroundFaceForensics: A Large-scale Video Dataset for Forgery Detection in Human Faces
Rössler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., & Nießner, M. (2018) · arXiv · doi:10.48550/arxiv.1803.09179 · Cited by 339
Face swapping: automatically replacing faces in photographs
Bitouk, D., Kumar, N., Dhillon, S., Belhumeur, P., & Nayar, S. K. (2008) · ACM Transactions on Graphics · doi:10.1145/1360612.1360638 · Cited by 333
Recycle-GAN: Unsupervised Video Retargeting
Bansal, A., Ma, S., Ramanan, D., & Sheikh, Y. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01228-1_8 · Cited by 328
Synthesizing Images of Humans in Unseen Poses
Balakrishnan, G., Zhao, A., Dalca, A. V., Durand, F., & Guttag, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00870 · Cited by 317
DeepFake Detection Based on Discrepancies Between Faces and Their Context
Nirkin, Y., Wolf, L., Keller, Y., & Hassner, T. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3093446 · Cited by 312
Deep fake image detection based on pairwise learning
Hsu, C.-C., Zhuang, Y.-X., & Lee, C.-Y. (2020) · Applied Sciences · doi:10.3390/app10010370 · Cited by 304 · cited 1×
“In [69], the authors train the CNN as a Siamese network using contrasting examples of real and fake images.”
BackgroundDensePose: Dense Human Pose Estimation in the Wild
Guler, R. A., Neverova, N., & Kokkinos, I. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00762 · Cited by 290
On Face Segmentation, Face Swapping, and Face Perception
Nirkin, Y., Masi, I., Tran Tuan, A., Hassner, T., & Medioni, G. (2018) · 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018) · doi:10.1109/fg.2018.00024 · Cited by 285
Liu, W., Piao, Z., Min, J., Luo, W., Ma, L., & Gao, S. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00600 · Cited by 266 · cited 1×
“[103], the authors counter this issue with their Liquid Warping GAN.”
BackgroundTowards Open-Set Identity Preserving Face Synthesis
Bao, J., Chen, D., Wen, F., Li, H., & Hua, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00702 · Cited by 257
FaceShifter: Towards High Fidelity And Occlusion Aware Face Swapping
Li, L., Bao, J., Yang, H., Chen, D., & Wen, F. (2019) · arXiv · doi:10.48550/arxiv.1912.13457 · Cited by 236 · cited 3×
“Other methods may be slowed down due to their pre/post-processing steps, such as warping [60, 62, 185], UV mapping or segmentation prediction [23, 112, 123, 180], and the use of refinement networks [25, 60, 83, 93, 111, 153].”
MethodDetecting Deep-Fake Videos from Phoneme-Viseme Mismatches
Agarwal, S., Farid, H., Fried, O., & Agrawala, M. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00338 · Cited by 234 · cited 1×
“[5], the authors refined the approach by detecting when visemes (mouth shapes) are inconsistent with the spoken phonemes (utternaces).”
BackgroundForensicTransfer: Weakly-supervised Domain Adaptation for Forgery Detection
Cozzolino, D., Thies, J., Rössler, A., Riess, C., Nießner, M., & Verdoliva, L. (2018) · arXiv · doi:10.48550/arxiv.1812.02510 · Cited by 221 · cited 3×
“In our SoK, we observed that the majority deepfake detection algorithms assume a static game with the adversary: They are either focused on identifying a specific artifact, or do not generalize well to new distributions and unseen attacks [143].”
BackgroundDeepfake Detection using Spatiotemporal Convolutional NetworksPreprint
de Lima, O., Franklin, S., Basu, S., Karwoski, B., & George, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.14749 · Cited by 217 · cited 1×
“In [36], the authors tried a variety of dierent classic spatio-temproal networks and feature extractors as a baseline for temporal deepfake detection.”
BackgroundWorkman, M. (2008) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.20779 · Cited by 213
Adversarial Deepfakes: Evaluating Vulnerability of Deepfake Detectors to Adversarial Examples
Hussain, S., Neekhara, P., Jere, M., Koushanfar, F., & McAuley, J. (2021) · 2021 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv48630.2021.00339 · Cited by 204
ReenactGAN: Learning to Reenact Faces via Boundary Transfer
Wu, W., Zhang, Y., Li, C., Qian, C., & Loy, C. C. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01246-5_37 · Cited by 201
Neverova, N., Alp Güler, R., & Kokkinos, I. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01219-9_8 · Cited by 196
DwNet: Dense warp-based network for pose-guided human video generationPreprint
Zablotskaia, P., Siarohin, A., Zhao, B., & Sigal, L. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.09139 · Cited by 193 · cited 4×
“, [83, 93, 103, 113, 114, 121, 182, 191]).”
MethodInfluentialFace transfer with multilinear models
Vlasic, D., Brand, M., Pfister, H., & Popovic, J. (2006) · SIGGRAPH Courses · doi:10.1145/1185657.1185864 · Cited by 192 · cited 1×
“Later, fully automated methods were proposed in [34, 80, 163] and [122] using methods such as warping and reconstructed 3D morphable face models.”
MethodPose-Guided Photorealistic Face Rotation
Hu, Y., Wu, X., Yu, B., He, R., & Sun, Z. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00876 · Cited by 187 · cited 2×
“Compared to [70], [159] has the exibility of manipulating the encodings for dierent tasks and the authors improve the quality of xд by averaging multiple examples of the identity encoding before passing it through the decoder (similar to [62, 168, 183]).”
BackgroundMethodOn the Generalization of GAN Image Forensics
Xuan, X., Peng, B., Wang, W., & Dong, J. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-31456-9_15 · Cited by 184
ImaGINator: Conditional Spatio-Temporal GAN for Video Generation
Wang, Y., Biliński, P., Brémond, F., & Dantcheva, A. (2020) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv45572.2020.9093492 · Cited by 184 · cited 3×
“[170] propose ImaGINator: a conditional GAN that fuses both motion and content and uses them with transposed 3D convolutions to capture the distinct spatio-temporal relationships.”
BackgroundUnmasking DeepFakes with simple Features
Durall, R., Keuper, M., Pfreundt, F.-J., & Keuper, J. (2019) · arXiv · doi:10.48550/arxiv.1911.00686 · Cited by 184 · cited 2×
“To help emphasize these artifacts to a learner, researchers have proposed edge detectors, quality measures, and frequency analysis [4, 8, 42, 110, 186].”
BackgroundFake Faces Identification via Convolutional Neural Network
Mo, H., Chen, B., & Luo, W. (2018) · Proceedings of the 6th ACM Workshop on Information Hiding and Multimedia Security · doi:10.1145/3206004.3206009 · Cited by 175 · cited 3×
“Models which identify blurred content [94] are affected by noise and sharpening GANs [63, 71], and models which search for the boundary where the face was blended in [4, 8, 38, 81, 94, 163] do not work on deepfakes passed through refiner networks or those which output full frames (e.”
A survey on image tampering and its detection in real-world photos
Zheng, L., Zhang, Y., & Thing, V. L. L. (2019) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2018.12.022 · Cited by 174 · cited 1×
“1 Detection e subject of image forgery detection is a well researched subject [188].”
BackgroundFaceID-GAN: Learning a Symmetry Three-Player GAN for Identity-Preserving Face Synthesis
Shen, Y., Luo, P., Yan, J., Wang, X., & Tang, X. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00092 · Cited by 174 · cited 3×
“In [25], the authors propose FaceID-GAN which can reenacts t at oblique poses and high resolution.”
BackgroundInfluentialGarrido, P., Valgaerts, L., Rehmsen, O., Thormaehlen, T., Perez, P., & Theobalt, C. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.537 · Cited by 172 · cited 1×
“In 2005, it was shown how this can be done without a 3D model [26], and through warping with matching similar textures [58].”
BackgroundEvading Deepfake-Image Detectors with White- and Black-Box Attacks
Carlini, N., & Farid, H. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00337 · Cited by 170 · cited 1×
“how these attacks not only work on deepfakes classifiers [115] but also work with no knowledge of the classifier or its training set [24].”
BackgroundDale, K., Sunkavalli, K., Johnson, M. K., Vlasic, D., Matusik, W., & Pfister, H. (2011) · Proceedings of the 2011 SIGGRAPH Asia Conference · doi:10.1145/2024156.2024164 · Cited by 166 · cited 1×
“Later, fully automated methods were proposed in [34, 80, 163] and [122] using methods such as warping and reconstructed 3D morphable face models.”
MethodpaGAN: real-time avatars using dynamic textures
Nagano, K., Seo, J., Xing, J., Wei, L., Li, Z., Saito, S., Agarwal, A., Fursund, J., & Li, H. (2018) · ACM Transactions on Graphics · doi:10.1145/3272127.3275075 · Cited by 165 · cited 7×
“, [83, 93, 103, 113, 114, 121, 182, 191]).”
BackgroundMethodInfluentialNeural Rendering and Reenactment of Human Actor Videos
Liu, L., Xu, W., Zollhöfer, M., Kim, H., Bernard, F., Habermann, M., Wang, W., & Theobalt, C. (2019) · ACM Transactions on Graphics · doi:10.1145/3333002 · Cited by 163 · cited 4×
“, [83, 93, 103, 113, 114, 121, 182, 191]).”
BackgroundInfluentialDeepMasterPrints: Generating MasterPrints for Dictionary Attacks via Latent Variable Evolution
Bontrager, P., Roy, A., Togelius, J., Memon, N., & Ross, A. (2018) · 2018 IEEE 9th International Conference on Biometrics Theory, Applications and Systems (BTAS) · doi:10.1109/btas.2018.8698539 · Cited by 161 · cited 1×
“The authors of [21] have shown how deep learning can generate realistic human fingerprints that can unlock multiple users’ devices.”
Representation Learning by Rotating Your Faces
Tran, L., Yin, X., & Liu, X. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2868350 · Cited by 158 · cited 4×
“Compared to [70], [159] has the exibility of manipulating the encodings for dierent tasks and the authors improve the quality of xд by averaging multiple examples of the identity encoding before passing it through the decoder (similar to [62, 168, 183]).”
BackgroundMethodInfluentialMarioNETte: Few-Shot Face Reenactment Preserving Identity of Unseen Targets
Ha, S., Kersner, M., Kim, B., Seo, S., & Kim, D. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6721 · Cited by 157
Fake Face Detection Methods: Can They Be Generalized?
Khodabakhsh, A., Ramachandra, R., Raja, K., Wasnik, P., & Busch, C. (2018) · 2018 International Conference of the Biometrics Special Interest Group (BIOSIG) · doi:10.23919/biosig.2018.8553251 · Cited by 151 · cited 1×
“ABLE III: Summary of Deepfake Detection Models Type Modality Content Method Eval. Dataset Performance* Image Replacement Reenactment ideo Audio Feature ace art Image Model Resolution Area [120] [126] [161] [139] [160] [124] Custom DB CC EER UC [113] 2017 SVM-RBF 250x250 92.9 [112] 2017 SVM * 18.2 [110] 2018 SVM * 0.97…”
InfluentialWarp-guided GANs for single-photo facial animation
Geng, J., Shao, T., Zheng, Y., Weng, Y., & Zhou, K. (2018) · ACM Transactions on Graphics · doi:10.1145/3272127.3275043 · Cited by 150 · cited 5×
“To overcome issue of occlusions in the eyes and mouth, the authors of [62] use multiple images of t as a reference, in contrast to [60] and [186] which only use one.”
BackgroundMethodInfluentialA Hybrid Model for Identity Obfuscation by Face Replacement
Sun, Q., Tewari, A., Xu, W., Fritz, M., Theobalt, C., & Schiele, B. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01246-5_34 · Cited by 145
Headon: real-time reenactment of human portrait videos
Thies, J., Zollhöfer, M., Theobalt, C., Stamminger, M., & Nießner, M. (2018) · ACM Transactions on Graphics · doi:10.1145/3197517.3201350 · Cited by 133
DeepWarp: Photorealistic Image Resynthesis for Gaze Manipulation
Ganin, Y., Kononenko, D., Sungatullina, D., & Lempitsky, V. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46475-6_20 · Cited by 132 · cited 3×
“, [79, 135, 145, 162, 172, 174]) are more likely to be used by researchers and criminals compared to those which are unavailable [2, 55, 65, 83, 121, 127, 154, 171, 185] or require highly specic or private datasets [57, 113, 181].”
BackgroundMethodCeleb-DF: A Large-scale Challenging Dataset for DeepFake ForensicsPreprint
Li, Y., Yang, X., Sun, P., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.12962 · Cited by 104 · cited 4×
“Concretely, the three most recent benchmark datasets (DFD by Google [120], DFDC by Facebook [40], and Celeb-DF by [101]) were released within one month of each other at the end of 2019.”
BackgroundInfluentialImproving Few-Shot User-Specific Gaze Adaptation via Gaze Redirection Synthesis
Yu, Y., Liu, G., & Odobez, J. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01221 · Cited by 98 · cited 2×
“In [76], the authors propose the Gaze Redireciton Network (GRN).”
BackgroundSpeaker Inconsistency Detection in Tampered Video
Korshunov, P., & Marcel, S. (2018) · 2018 26th European Signal Processing Conference (EUSIPCO) · doi:10.23919/eusipco.2018.8553270 · Cited by 97 · cited 2×
“ted faces [105]. Regarding deepfakes, [106] monitored irregular eye blinking patterns and [107] monitored blood volume patterns (pulse) under the skin. Inconsistencies are also a revealing factor. In [108] and [109], the authors noticed that video dubbing attacks can be detected my correlating the speech to landmarks…”
BackgroundInfluentialPhotorealistic facial expression synthesis by the conditional difference adversarial autoencoder
Zhou, Y., & Shi, B. E. (2017) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii.2017.8273626 · Cited by 91 · cited 2×
“x g subjected to L 1 and L adv losses. The challenge of using a CGAN was that the training data had to be paired (images of different identities with the same expression). Going one step further, in [61] the authors reenacted full portraits at low resolutions. Their approach was to decoupling the identities was to use…”
MethodRealistic Dynamic Facial Textures from a Single Image Using GANs
Olszewski, K., Li, Z., Yang, C., Zhou, Y., Yu, R., Huang, Z., Xiang, S., Saito, S., Kohli, P., & Li, H. (2017) · IEEE International Conference on Computer Vision · doi:10.1109/iccv.2017.580 · Cited by 90 · cited 5×
“One of the rst identity agnostic methods was [124], mentioned in Section 4.”
BackgroundMethodInfluentialDeferred Neural Rendering: Image Synthesis using Neural TexturesPreprint
Thies, J., Zollhöfer, M., & Nießner, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.12356 · Cited by 90 · cited 1×
“[153] had a different approach: (1) animate the reconstructed 3D head with the predicted blend shape parameters from as using a DeepSpeech model for feature extraction, (2) use Deferred Neural Rendering [154] to generate the mouth region, and then (3) use a network to blend the mouth into the original frame.”
MethodICface: Interpretable and Controllable Face Reenactment Using GANs
Tripathy, S., Kannala, J., & Rahtu, E. (2020) · 2020 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv45572.2020.9093474 · Cited by 88 · cited 1×
“[159], the authors present a model called ICFace where the expression, pose, mouth, eye, and eyebrows of xt can be driven independently.”
BackgroundObamaNet: Photo-realistic lip-sync from textPreprint
Kumar, R., Sotelo, J., Kumar, K., de Brébisson, A., & Bengio, Y. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.01442 · Cited by 87 · cited 4×
“, [25, 55, 71, 73, 89, 105, 152, 174]).”
InfluentialSWAPPED! Digital face presentation attack detection via weighted local magnitude pattern
Agarwal, A., Singh, R., Vatsa, M., & Noore, A. (2017) · 2017 IEEE International Joint Conference on Biometrics (IJCB) · doi:10.1109/btas.2017.8272754 · Cited by 85 · cited 2×
“Models which identify blurred content [94] are affected by noise and sharpening GANs [63, 71], and models which search for the boundary where the face was blended in [4, 8, 38, 81, 94, 163] do not work on deepfakes passed through refiner networks or those which output full frames (e.”
RSGAN: Face Swapping and Editing using Face and Hair Representation in Latent SpacesPreprint
Natsume, R., Yatagawa, T., & Morishima, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.03447 · Cited by 84 · cited 3×
“LCE EnI : yI , x L2 EnA : xg, xt -weighted LKL: e A Ladv D : y, xt/xg LFM De :Dl∙ xg , Dl∙(xt) LCE(DI): yI , x /xg LFM De :DIl∙ xg , DIl∙(x′) [115] RSGAN:”
BackgroundResultKemelmacher-Shlizerman, I. (2016) · ACM Transactions on Graphics · doi:10.1145/2897824.2925871 · Cited by 79
Physiologically-based detection of computer generated faces in video
Conotter, V., Bodnari, E., Boato, G., & Farid, H. (2014) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2014.7025049 · Cited by 76 · cited 1×
“In 2014, researchers hypothesized that generated content will lack physiological signals and identied computer generated faces by monitoring their heart rate [32].”
BackgroundLimits of Deepfake Detection: A Robust Estimation ViewpointPreprint
Agarwal, S., & Varshney, L. R. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.03493 · Cited by 76 · cited 1×
“For example, by nding a bound on a model’s delay can help detect real-time aacks such as [75], and determining the limits of GANs like [7] can help us devise the appropriate strategies.”
Background3D Aided Duet GANs for Multi-View Face Image Synthesis
Cao, J., Hu, Y., Yu, B., He, R., & Sun, Z. (2019) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2019.2891116 · Cited by 70
Exploiting Prediction Error Inconsistencies through LSTM-based Classifiers to Detect Deepfake Videos
Amerini, I., & Caldelli, R. (2020) · Proceedings of the 2020 ACM Workshop on Information Hiding and Multimedia Security · doi:10.1145/3369412.3395070 · Cited by 64 · cited 1×
“Later, the same authors use an LSTM to predict the next frame and expose deepfakes when the reconstruction error is high [10].”
BackgroundTransferable videorealistic speech animation
Chang, Y.-J., & Ezzat, T. (2005) · Symposium on Computer Animation · doi:10.1145/1073368.1073388 · Cited by 60 · cited 1×
“ression reenactment has been around long before deepfakes were popularized. In 2003, researchers morphed models of 3D scanned heads [52]. In 2005, it was shown how this can be done without a 3D model [53]. Later, between 2015 and 2018, Thies et al. demonstrated how 3D parametric models can be used to achieve high qual…”
BackgroundFighting Against Deepfake: Patch&Pair Convolutional Neural Networks (PPCNN)
Li, X., Yu, K., Ji, S., Wang, Y., Wu, C., & Xue, H. (2020) · International World Wide Web Conference (WWW) · doi:10.1145/3366424.3382711 · Cited by 54 · cited 1×
“In [96], the authors follow a dierent approach by contrasting the generated foreground to the (untampered) background using a patch and pair CNN.”
MethodDetecting Deepfake Videos using Attribution-Based Confidence Metric
Fernandes, S. L., Raj, S., Ewetz, R., Pannu, J. S., Jha, S. K., Ortiz, E., Vintila, I., & Salter, M. (2020) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00162 · Cited by 52 · cited 1×
“[51] use a state-of-the-art attribution-based confidence (ABC) metric.”
One-shot Face ReenactmentPreprint
Zhang, Y., Zhang, S., He, Y. L., Li, C., Loy, C. C., & Liu, Z. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.03251 · Cited by 52 · cited 4×
“Other methods may be slowed down due to their pre/post-processing steps, such as warping [60, 62, 186], UV mapping or segmentation prediction [23, 113, 124, 181], and the use of renement networks [25, 60, 83, 93, 112, 154].”
MethodInfluentialFaceFeat-GAN: a Two-Stage Approach for Identity-Preserving Face SynthesisPreprint
Shen, Y., Zhou, B., Luo, P., & Tang, X. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.01288 · Cited by 49 · cited 1×
“[140] proposed FaceFeat-GAN, which improves the diversty of the faces while preserving the identity [141].”
BackgroundPham, H., Wang, Y., & Pavlović, V. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.07716 · Cited by 45 · cited 4×
“Similar to [127], another w rk called GANimation [128] reenacts faces through AU value inputs estimated from xs .”
BackgroundMethodInfluentialHiding Faces in Plain Sight: Disrupting AI Face Synthesis with Adversarial PerturbationsPreprint
Li, Y., Yang, X., Wu, B., & Lyu, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.09288 · Cited by 44 · cited 2×
“For example, the establishment of content provenance and authenticity frameworks for online videos [44, 54, 68], and proactive defenses such as the use of adversarial machine learning to protect content from tampering [102].”
BackgroundAdversarial Learning of Deepfakes in Accounting
Schreyer, M., Sattarov, T., Reimer, B., & Borth, D. (2019) · Alexandria (UniSG) (University of St.Gallen) · doi:10.48550/arxiv.1910.03810 · Cited by 44 · cited 1×
“[135], it was shown how deepfakes can be applied to financial records to evade the detection of auditors.”
BackgroundOn the Detection of Digital Face ManipulationPreprint
Dang, H., Liu, F., Stehouwer, J., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.01717 · Cited by 41 · cited 2×
“To localize the tampered areas, some works train networks to predicting masks learned from a ground truth dataset, or by mapping the neural activations back to the raw image [41, 92, 118, 149].”
Face Transfer with Generative Adversarial NetworkPreprint
Xu, R., Zhou, Z., Zhang, W., & Yu, Y. H. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.06090 · Cited by 40 · cited 2×
“In 2017, the authors of [176] proposed using a CycleGAN for facial reenactment, without the need for data pairing.”
BackgroundSpeech-Driven Facial Reenactment Using Conditional Generative Adversarial NetworksPreprint
Jalalifar, S. A., Hasani, H., & Aghajan, H. K. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.07461 · Cited by 34 · cited 17×
“[75], and determining the limits of GANs like Ref.”
BackgroundInfluentialEmotions Don't Lie: An Audio-Visual Deepfake Detection Method Using Affective CuesPreprint
Mittal, T., Bhattacharya, U., Chandra, R., Bera, A., & Manocha, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.06711 · Cited by 33
FakeSpotter: A Simple yet Robust Baseline for Spotting AI-Synthesized Fake FacesPreprint
Wang, R., Juefei-Xu, F., Ma, L., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.06122 · Cited by 32 · cited 3×
“Alternatively, to overcome noise and other distortions, the authors of [144] measured the neural activation (coverage) of a face recognition network to obtain a stronger signal from than just using the raw pixels.”
InfluentialMetaPix: Few-Shot Video RetargetingPreprint
Lee, J., Ramanan, D., & Girdhar, R. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.04742 · Cited by 29 · cited 1×
“[91], the authors demonstrate the few-shot learning technique of Ref.”
BackgroundDeepFake Detection Based on the Discrepancy Between the Face and its ContextPreprint
Nirkin, Y., Wolf, L., Keller, Y., & Hassner, T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.12262 · Cited by 28 · cited 2×
“e authors of [123] also contrast the fore/background but enable a network to identify the distinguishing features automatically.”
BackgroundTriple consistency loss for pairing distributions in GAN-based face synthesisPreprint
Sánchez, E., & Valstar, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.03492 · Cited by 24 · cited 2×
“[134] propose GANnotation, which uses facial landmark images.”
BackgroundHigh Fidelity Face Manipulation with Extreme Pose and ExpressionPreprint
Fu, C.-Y., Hu, Y., Wu, X., Wang, G.-L., Zhang, Q., & He, R. (2019) · arXiv.org · doi:10.48550/arxiv.1903.12003 · Cited by 24 · cited 4×
“contrast to other facial conversion methods [56, 185], FSGAN uses fewer neural networks enabling real time reenactment at 30fps.”
BackgroundMethodResultInfluentialTalking heads synthesis from audio with deep neural networks
Shimba, T., Sakurai, R., Yamazoe, H., & Lee, J. (2015) · IEEE/SICE International Symposium on System Integration (SII) · doi:10.1109/sii.2015.7404961 · Cited by 23 · cited 2×
“o 1 2 1 1 portrait/landmarks 1-3 portraits 256x256 Many-to-One [64] 2019 Few-shot Vid2Vid or None 1-10 portraits o 3 3 2 4 portrait/body video 1-10 portr./bodies 2048x1024 [69] 2015 Shimba et al. None None 0 0 0 1 audio face database * [75] 2016 DeepWarp None None 0 0 0 2 gaze eye image >40x50 [59] 2017 CVAE-GAN No…”
MethodMining Audio, Text and Visual Information for Talking Face Generation
Yu, L., Yu, J., & Ling, Q. (2019) · Proceedings · doi:10.1109/icdm.2019.00089 · Cited by 23 · cited 1×
“rames are generated by (1) encoding each MFCC audio segment a(i) s and x t with separate encoders, (2) passing the encodings through an RNN, and (3) decoding the outputs as x(i) g using a decoder. In [71] the authors try to improve lip sync with textual context. A time-delayed LSTM is used to predict mouth landmarks g…”
MethodPhotorealistic Monocular Gaze Redirection Using Machine Learning
Kononenko, D., Ganin, Y., Sungatullina, D., & Lempitsky, V. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2737423 · Cited by 22 · cited 1×
“is is used to improve photographs or to automatically maintain eye contact during video interviews [45].”
MethodA Conditional Deep Generative Model of People in Natural Images
de, R., Ghosh, A., Boukhayma, A., Ajanthan, T., Siddharth, N., & Torr, P. H. S. (2019) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2019.00159 · Cited by 22 · cited 1×
“To improve the control over the compositing, the authors of [35] used a CVAE-GAN.”
MethodFace X-ray for More General Face Forgery DetectionPreprint
Li, L., Bao, J., Zhang, T., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.13458 · Cited by 21
Talking Face Generation by Conditional Recurrent Adversarial NetworkPreprint
Song, Y., Zhu, J., Li, D., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.04786 · Cited by 20 · cited 2×
“Compared to direct models such as the direct models of Refs [146] and [178], the authors of Ref.”
MethodZooming into Face Forensics: A Pixel-level AnalysisPreprint
Li, J., Shen, T., Zhang, W., Ren, H., Zeng, D., & Mei, T. (2019) · arXiv.org · doi:10.48550/arxiv.1912.05790 · Cited by 19 · cited 2×
“To localize the tampered areas, some works train networks to predicting masks learned from a ground truth dataset, or by mapping the neural activations back to the raw image [41, 92, 118, 149].”
BackgroundXiao, F., Liu, H., & Lee, Y. J. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00711 · Cited by 17 · cited 4×
“[175] IHPT:”
BackgroundInfluentialExploiting Human Social Cognition for the Detection of Fake and Fraudulent Faces via Memory NetworksPreprint
Fernando, T., Fookes, C., Denman, S., & Sridharan, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.07844 · Cited by 13 · cited 2×
“Instead of focusing on a specific artifact, some authors train deep neural networks as generic classifiers, and let the network decide which features to analyze [133]–[137].”
BackgroundTowards Generalizable Deepfake Detection with Locality-aware AutoEncoderPreprint
Du, M., Pentyala, S., Li, Y., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.05999 · Cited by 12 · cited 1×
“5 [119] 2019 • • • • • CNN AE GAN • 256x256 • • 99.”
Çiftçi, U. A., Demir, İ., & Yin, L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.11363 · Cited by 10
Using subject face brightness assessment to detect ‘deep fakes’ (Conference Presentation)
Straub, J. (2019) · Real-Time Image Processing and Deep Learning · doi:10.1117/12.2520546 · Cited by 9 · cited 2×
“ral activations to the raw image [118]–[121]. The content of a fake face can be anomalous in context to the rest of the frame. For example, residuals from face warping processes [122]–[124], lighting [125], and varying fidelity [126] indicate the presence of generated content. In [127] and [128], the authors found that…”
InfluentialWorkmanMichael (2008) · Journal of the American Society for Information Science and Technology · doi:10.5555/1345501.1345514 · Cited by 7
Video synthesis of human upper body with realistic facePreprint
Liu, Z., Hu, H., Wang, Z., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.06607 · Cited by 7 · cited 5×
“Ladv: ls, yt , xg, xs Lperc: I1,l∙ xg , I1,l∙ xs Lperc: I2,l∙ xg , I2,l∙ xs LFM: Dl xg , Dl xs Lmse: LMT [105] Liu et al.”
BackgroundMethodInfluentialSwapped Face Detection using Deep Learning and Subjective AssessmentPreprint
Ding, X., Raziei, Z., Larson, E. C., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.04217 · Cited by 7 · cited 3×
“18 [38] 2019 • • • • CNN • 98.”
BackgroundFake face detection via adaptive manipulation traces extraction networkPreprint
Guo, Z., Yang, G., Chen, J., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.04945 · Cited by 7 · cited 1×
“[64] use a network to emphasize the residuals and suppress the imperfections in a preprocessing step for a classifier.”
MethodFReeNet: Multi-Identity Face ReenactmentPreprint
Zhang, J., Zeng, X., Wang, M., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.11805 · Cited by 6 · cited 4×
“contrast to other facial conversion methods [56, 185], FSGAN uses fewer neural networks enabling real time reenactment at 30fps.”
BackgroundResultInfluentialMixNMatch: Multifactor Disentanglement and Encoding for Conditional Image GenerationPreprint
Li, Y., Singh, K. K., Ojha, U., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.11758 · Cited by 5 · cited 2×
“However neither [175] or [100] were explicitly performed on faces.”
Dance Dance Generation: Motion Transfer for Internet VideosPreprint
Zhou, Y., Wang, Z., Chen, F., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.00129 · Cited by 1 · cited 3×
“, Refs [83], [93], [102], [112], [113], [120], [181], and [190]).”
BackgroundMethodA Neural Algorithm of Artistic Style
Gatys, L. A., Ecker, A. S., & Bethge, M. (2015) · arXiv · doi:10.48550/arxiv.1508.06576 · cited 1×
“Another common loss is a type of content loss (LC ) [59], which is used to help the generator create realistic features, based on the perspective of a perceptual model.”
BackgroundTweepFake: about Detecting Deepfake Tweets
Fagni, T., Falchi, F., Gambini, M., et al. (2020) · arXiv · doi:10.48550/arxiv.2008.00036 · cited 1×
“Finally, it has been shown how deepfakes of news articles can be generated [183] and that deepfake tweets exist as well [50].”
BackgroundFaceForensics++: Learning to Detect Manipulated Facial Images
Rössler, A., Cozzolino, D., Verdoliva, L., et al. (2019) · arXiv · doi:10.48550/arxiv.1901.08971 · cited 2×
“In [106, 119, 131], it was shown that deep neural networks tend to perform beer than traditional image forensic tools on compressed imagery.”
BackgroundFakeCatcher: Detection of Synthetic Portrait Videos using Biological Signals
Ciftci, U. A., & Demir, I. (2019) · arXiv · doi:10.48550/arxiv.1901.02212 · cited 2×
“For example, G can generate the biological signals monitored by Refs [30] and [96] by adding a discriminator that monitors these signals.”
BackgroundAdversarial Deepfakes: Evaluating Vulnerability of Deepfake Detectors to Adversarial Examples
Hussain, S., Neekhara, P., Jere, M., Koushanfar, F., & McAuley, J. (2020) · arXiv · doi:10.48550/arxiv.2002.12749 · cited 1×
“Recent works have shown how these aacks not only work on deepfakes classiers [116] but also work with no knowledge of the classier or it’s training set [24].”
BackgroundText-based Editing of Talking-head Video
Fried, O., Tewari, A., Zollhöfer, M., Finkelstein, A., Shechtman, E., Goldman, D. B., Genova, K., Jin, Z., Theobalt, C., & Agrawala, M. (2019) · arXiv · doi:10.48550/arxiv.1906.01524 · cited 4×
“, see Refs [25], [55], [71], [73], [89], [104], [151], and [173]).”
BackgroundMethodInfluentialFraga-Lamas, P., & Fernández-Caramés, T. M. (2019) · arXiv · doi:10.48550/arxiv.1904.05386
Speech-driven facial animation using polynomial fusion of features
Kefalas, T., Vougioukas, K., Panagakis, Y., Petridis, S., Kossaifi, J., & Pantic, M. (2019) · arXiv · doi:10.48550/arxiv.1912.05833 · cited 3×
“2 • ◦ ◦ None None • 5 1 3 3 • • • audio portrait • 96x128 [79] 2019 Speech D.”
BackgroundICface: Interpretable and Controllable Face Reenactment Using GANs
Tripathy, S., Kannala, J., & Rahtu, E. (2019) · arXiv · doi:10.48550/arxiv.1904.01909 · cited 1×
“[159], the authors present a model called ICFace where the expression, pose, mouth, eye, and eyebrows of xt can be driven independently.”
BackgroundDeep Transfer Across Domains for Face Anti-spoofing
Tu, X., Zhang, H., Xie, M., Luo, Y., Zhang, Y., & Ma, Z. (2019) · arXiv · doi:10.48550/arxiv.1901.05633 · cited 2×
“To improve performance of their models, some add noise to their training data [142] and others try to develop models which generalize better to unseen attacks/generators through disentanglement [143] and semi-supervised learning [144].”
BackgroundRealistic Speech-Driven Facial Animation with GANs
Vougioukas, K., Petridis, S., & Pantic, M. (2019) · arXiv · doi:10.48550/arxiv.1906.06337 · cited 2×
“[164], the same authors improve the temporal coherence by splitting the video discriminator into two: (1) for temporal realism in mouth to audio synchronization and (2) for temporal realism in overall facial expressions.”
BackgroundResultFew-Shot Adversarial Learning of Realistic Neural Talking Head Models
Zakharov, E., Shysheya, A., Burkov, E., & Lempitsky, V. (2019) · arXiv · doi:10.48550/arxiv.1905.08233 · cited 7×
“Ladv: y, kt , ൗ xg xt (i) LFM: xg ∙ vst, xt (i) ∙ vst [183] Neural TalkingHeads:”
BackgroundInfluentialOtberdout, N., Daoudi, M., Kacem, A., Ballihi, L., & Berretti, S. (2019) · arXiv · doi:10.48550/arxiv.1907.10087 · cited 4×
“A challenge with works such as [125] and [171] is that they are label driven and produce videos with a set number of frames.”
BackgroundMethodInfluential
Research works printed but not linked: 49
- 2020, Retrieved
- Aberman Kfir, Computer Graphics Forum
- Agarwal Shruti, 2019, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 38--45
- Agarwal Sakshi, 2019, Varshney
- Antinori Arije, 2019, ECIAIR 2019 European Conference on the Impact of Artificial Intelligence and Robotics. Academic Conferences and Publishing Limited, 23
- Averbuch-Elor Hadar, 2017, Cohen
- Blanz Volker, Computer Graphics Forum
- Blanz Volker, Computer Graphics Forum
- Chan Caroline, Proceedings of the IEEE International Conference on Computer Vision. 5933--5942
- Chi-Ying, Forbes 2019 19th International Conference on Computational Science and Its Applications (ICCSA). IEEE, 132--137
- Liqian, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 99--108
- Pavel, International Conference on Machine Learning.
- Shengju, Proceedings of the IEEE International Conference on Computer Vision.
- 10.5555/3305381.3305498 · doi:10.5555/3305381.3305498
- Fraga-Lamas Paula, 2019, Fernandez-Carames
- Ian Goodfellow Jean Pouget-Abadie Mehdi Mirza Bing Xu David Warde-Farley Sherjil Ozair Aaron Courville and Yoshua Bengio. 2014. Generative adversarial nets. In Advances in Neural Information Processing Systems. 2672--2680. Ian Goodfellow Jean Pouget-Abadie Mehdi Mirza Bing Xu David Warde-Farley Sherjil Ozair Aaron Courville and Yoshua Bengio. 2014. Generative adversarial nets. In Advances in Neural Information Processing Systems. 2672--2680.
- Gu Kuangxiao, 2020, Huang
- Guera David, IEEE Conference on Advanced Video and Signal Based Surveillance (AVSS). IEEE, 1--6.
- Hall Holly Kathleen, 2018, Deepfake videos: When seeing isn’t believing. Cath. UJL 8 Tech 27
- Hao Karen, The biggest threat of deepfakes isn
- 2019, Retrieved
- Isola Phillip, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1125--1134
- Jamaludin Amir, 2019, Joon Son Chung, and Andrew Zisserman
- Jia Ye, 2018, Yonghui Wu, et al.
- Khalid Hasam, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. 656--657
- Koopman Marissa, 2018, Conference: IMVIP.
- Li Yuezun, 2019, Retrieved
- Li Yuezun, 2019, IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).
- Li Yuheng, 2019, Utkarsh Ojha, and Yong Jae Lee.
- Masi Iacopo, 2020, Shenoy Pratik Gurudatt
- Mirsky Yisroel, 2019, USENIX Security Symposium
- Joel Ruben Antony Moniz Christopher Beckham Simon Rajotte Sina Honari and Chris Pal. 2018. Unsupervised depth estimation 3D face rotation and replacement. In Advances in Neural Information Processing Systems. 9736--9746. Joel Ruben Antony Moniz Christopher Beckham Simon Rajotte Sina Honari and Chris Pal. 2018. Unsupervised depth estimation 3D face rotation and replacement. In Advances in Neural Information Processing Systems. 9736--9746.
- Natsume Ryota, 2018, Asian Conference on Computer Vision. Springer, 117--132
- Pumarola Albert, 2019, International Journal of Computer Vision
- 2020, Shohel Rana and Andrew H. Sung
- Schwartz Oscar, 2020, You thought fake news was bad? The Guardian. Retrieved
- Shan Shawn, Proceedings of the 29th {USENIX} Security Symposium ({USENIX} Security 20)
- 2018, Retrieved
- Siarohin Aliaksandr, Advances in Neural Information Processing Systems 32. Curran Associates
- Sotelo Jose, 2017, Kyle Kastner, Aaron Courville, and Yoshua Bengio.
- Tariq Shahroz, Proceedings of the 2nd International Workshop on Multimedia Privacy and Security. ACM, 81--87
- Vougioukas Konstantinos, 2019, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 37--40
- Wang Sheng-Yu, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 7
- Ting-Chun Wang Ming-Yu Liu Andrew Tao Guilin Liu Jan Kautz and Bryan Catanzaro. 2019. Few-shot video-to-video synthesis. In Advances in Neural Information Processing Systems (NeurIPS). Ting-Chun Wang Ming-Yu Liu Andrew Tao Guilin Liu Jan Kautz and Bryan Catanzaro. 2019. Few-shot video-to-video synthesis. In Advances in Neural Information Processing Systems (NeurIPS).
- Zellers Rowan, Advances in Neural Information Processing Systems 32. Curran Associates
- Zhang Yunxuan, 2019, Chen Change Loy, and Ziwei Liu
- Zhang Ying, Proceedings of the 2017 IEEE 2nd International Conference on Signal and Image Processing (ICSIP). IEEE, 15--19
- Zhou Yipin, 2019, Berg
- Zhu Jun-Yan, Proceedings of the IEEE International Conference on Computer Vision. 2223--2232
Cited by
Feuerriegel, S., Hartmann, J., Janiesch, C., & Zschech, P. (2024) · Business & Information Systems Engineering · doi:10.1007/s12599-023-00834-7 · Cited by 2,267
“Notwithstanding, AI-generated disinformation has previously evolved as so-called deepfakes (Mirsky and Lee 2021), but recent advances in generative AI reduce the cost of creating such disinformation and allow for unprecedented personalization.”
Generative artificial intelligence
Banh, L., & Strobel, G. (2023) · Electronic Markets · doi:10.1007/s12525-023-00680-1 · Cited by 579
“…GAI models provides starting points for new applications and business models for misuse and criminals, ultimately being leveraged to spread misinformation and influence the media and politics, or to defraud individuals and businesses (Hart-mann et al., 2023; Kreps et al., 2022; Mirsky & Lee, 2022).”
BackgroundChallenges in deploying machine learning: A survey of case studies
Paleyes, A., Urma, R.-G., & Lawrence, N. D. (2023) · ACM Computing Surveys · doi:10.1145/3533378 · Cited by 638
“Closely related to this question is the growing concern of fake content being generated with ML, such as deepfake images and video, which can be easily used for the wrong purposes [95].”
BackgroundDeepfake Detection: A Systematic Literature Review
Rana, M. S., Nobi, M. N., Murali, B., & Sung, A. H. (2022) · IEEE Access · doi:10.1109/access.2022.3154404 · Cited by 573
Masood, M., Nawaz, M., Malik, K. M., Javed, A., Irtaza, A., & Malik, H. (2023) · Applied Intelligence · doi:10.1007/s10489-022-03766-z · Cited by 543
Metadata
- Authors
- Mirsky, Y., & Lee, W.
- Affiliations
- Georgia Institute of Technology (US)
- Venue
- ACM Computing Surveys
- Published
- 2021-01-02
- Volume
- 54
- Issue
- 1
- Pages
- 1-41
- ISSN
- 0360-0300 (ISSN-L)
- 1557-7341
- OpenAlex
- W3019200173
- Semantic Scholar
- 97ac037b7589d9d0f2cb5109aa2b095ad06067fd
- Cited by
- 863
- References
- 81
- Citations by year
- 2026: 202 · 2025: 213 · 2024: 163 · 2023: 131 · 2022: 90 · 2021: 53
- FWCI
- 39.29
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- ACM Computing Surveys (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Keywords
- deepfakes, deepfake detection, deepfake generation, generative adversarial networks, misinformation, impersonation, defamation, synthetic media
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.