Creating, Using, Misusing, and Detecting Deep Fakes
Open accessThe DOI matches its Crossref record; 46 of the 46 references it cites were found in a registry or an index.
Abstract
Synthetic media—so-called deep fakes—have captured the imagination of some and struck fear in others. Although they vary in their form and creation, deep fakes refer to text, image, audio, or video that has been automatically synthesized by a machine-learning system. Deep fakes are the latest in a long line of techniques used to manipulate reality, yet their introduction poses new opportunities and risks due to the democratized access to what would have historically been the purview of Hollywood-style studios. This review describes how synthetic media is created, how it is being used and misused, and if (and how) it can be perceptually and forensically distinguished from reality.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293 · cited 1×
“Even more powerfully, this stylizing can be driven by text-based descriptions alone (Razavi, Van den Oord, and Vinyals 2019; Radford et al. 2021; Bau et al. 2021).”
BackgroundA Style-Based Generator Architecture for Generative Adversarial Networks
Karras, T., Laine, S., & Aila, T. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00453 · Cited by 11,714
Analyzing and Improving the Image Quality of StyleGAN
Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J., & Aila, T. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00813 · Cited by 6,332
Analyzing and Improving the Image Quality of StyleGAN
Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J., & Aila, T. (2019) · arXiv · doi:10.48550/arxiv.1912.04958 · Cited by 6,332
WaveNet: a generative model for raw audio
Oord, A. V. D., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1609.03499 · Cited by 4,911
Colman (2022) · doi:10.1073/pnas · Cited by 4,370
FaceForensics++: Learning to Detect Manipulated Facial Images
Rossler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., & Niessner, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00009 · Cited by 3,314 · cited 2×
“FaceForensics++ (Rossler et al. 2019) consisting of 1,000 YouTube videos of 1,000 different people, mostly news anchors and video bloggers, from which four types of deep-fake videos are created: faceswap17, FaceSwap18, Face2Face (Justus Thies et al. 2016), and Neural Textures (Thies, Zollhöfer, and…”
BackgroundMesoNet: a Compact Facial Video Forgery Detection Network
Afchar, D., Nozick, V., Yamagishi, J., & Echizen, I. (2018) · 2018 IEEE International Workshop on Information Forensics and Security (WIFS) · doi:10.1109/wifs.2018.8630761 · Cited by 1,872
Face2Face: Real-Time Face Capture and Reenactment of RGB Videos
Thies, J., Zollhofer, M., Stamminger, M., Theobalt, C., & Niessner, M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.262 · Cited by 1,747
Face X-Ray for More General Face Forgery Detection
Li, L., Bao, J., Zhang, T., Yang, H., Chen, D., Wen, F., & Guo, B. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00505 · Cited by 1,211
CNN-Generated Images Are Surprisingly Easy to Spot… for Now
Wang, S.-Y., Wang, O., Zhang, R., Owens, A., & Efros, A. A. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00872 · Cited by 1,210
Exposing Deep Fakes Using Inconsistent Head Poses
Yang, X., Li, Y., & Lyu, S. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8683164 · Cited by 1,196
In Ictu Oculi: Exposing AI Created Fake Videos by Detecting Eye Blinking
Li, Y., Chang, M.-C., & Lyu, S. (2018) · 2018 IEEE International Workshop on Information Forensics and Security (WIFS) · doi:10.1109/wifs.2018.8630787 · Cited by 1,089
Deep Fakes: A Looming Challenge for Privacy, Democracy, and National Security
Chesney, R., & Citron, D. K. (2018) · SSRN Electronic Journal · doi:10.2139/ssrn.3213954 · Cited by 880
Chan, C., Ginosar, S., Zhou, T., & Efros, A. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00603 · Cited by 737
Generating diverse high-fidelity images with vq-vae-2
Razavi, A., Oord, A. V. D., & Vinyals, O. (2019) · arXiv · doi:10.48550/arxiv.1906.00446 · Cited by 709
FSGAN: Subject Agnostic Face Swapping and Reenactment
Nirkin, Y., Keller, Y., & Hassner, T. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00728 · Cited by 706
Two-Stream Neural Networks for Tampered Face Detection
Zhou, P., Han, X., Morariu, V. I., & Davis, L. S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.229 · Cited by 700
Video Rewrite: driving visual speech with audio
Bregler, C., Covell, M., & Slaney, M. (1997) · Proceedings of the 24th annual conference on Computer graphics and interactive techniques - SIGGRAPH '97 · doi:10.1145/258734.258880 · Cited by 669
First Order Motion Model for Image Animation
Siarohin, A., Lathuilière, S., Tulyakov, S., Ricci, E., & Sebe, N. (2020) · arXiv · doi:10.48550/arxiv.2003.00196 · Cited by 585 · cited 1×
“VC predates deep fakes by 65 years, although recent efforts have incorporated deep learning (Sisman et al. 2020).”
BackgroundDefending Against Neural Fake News
Zellers, R., Holtzman, A., Rashkin, H., Bisk, Y., Farhadi, A., Roesner, F., & Choi, Y. (2019) · arXiv · doi:10.48550/arxiv.1905.12616 · Cited by 569 · cited 1×
“While AI-synthesized text is beyond the scope of this review, recent breakthroughs in natural-language processing have made it possible to automatically synthesize detailed and cogent prose (Zellers et al. 2019; Brown et al. 2020).”
BackgroundASVspoof 2019: A large-scale public database of synthesized, converted and replayed speech
Wang, X., Yamagishi, J., Todisco, M., Delgado, H., Nautsch, A., Evans, N., Sahidullah, M., Vestman, V., Kinnunen, T., Lee, K. A., et al. (2020) · Computer Speech & Language · doi:10.1016/j.csl.2020.101114 · Cited by 505 · cited 2×
“Wang et al.’s forensic classifier (S.-Y. Wang et al. 2020), for example, is based on a standard neural-network architecture (ResNet-50), pre-trained on a standard image database (ImageNet), and then refined to classify an image as real or synthesized.”
MethodFakeCatcher: Detection of Synthetic Portrait Videos using Biological Signals
Ciftci, U. A., Demir, I., & Yin, L. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3009287 · Cited by 499
The Deepfake Detection Challenge (DFDC) Preview Dataset
Dolhansky, B., Howes, R., Pflaum, B., Baram, N., & Ferrer, C. C. (2019) · arXiv · doi:10.48550/arxiv.1910.08854 · Cited by 413 · cited 4×
“By comparison, the leading DFDC predictor correctly identified 80% of the deep-fake videos, although this accuracy is significantly higher than the some 65% accuracy achieved by the topperforming model on the entire DFDC holdout dataset of 4,000 videos.”
BackgroundInfluentialAI-synthesized faces are indistinguishable from real faces and more trustworthy
Nightingale, S. J., & Farid, H. (2022) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2120481119 · Cited by 397 · cited 1×
“For example, in mid-May of 2020, in the midst of the global pandemic, 20% of the US public believed Bill Gates was planning to use COVID-19 to implement a mandatory vaccine program with tracking microchips (Nightingale and Farid 2020).”
BackgroundSelf-citationLeveraging Frequency Analysis for Deep Fake Image Recognition
Frank, J., Eisenhofer, T., Schönherr, L., Fischer, A., Kolossa, D., & Holz, T. (2020) · arXiv · doi:10.48550/arxiv.2003.08685 · Cited by 390 · cited 2×
“Using a similar computational approach, the forensic classifier of Frank et al. (Frank et al. 2020) yields a similarly accurate and generalizable classifier.”
BackgroundMethodASVspoof 2021: accelerating progress in spoofed and deepfake speech detection
Yamagishi, J., Wang, X., Todisco, M., Sahidullah, M., Patino, J., Nautsch, A., Liu, X., Lee, K. A., Kinnunen, T., Evans, N., et al. (2021) · 2021 Edition of the Automatic Speaker Verification and Spoofing Countermeasures Challenge · doi:10.21437/asvspoof.2021-8 · Cited by 380
Deep voice 3: scaling text-to-speech with convolutional sequence learning
Ping, W., Peng, K., Gibiansky, A., Arik, S. O., Kannan, A., Narang, S., Raiman, J., & Miller, J. (2017) · arXiv · doi:10.48550/arxiv.1710.07654 · Cited by 313
The Prevalence of Inappropriate Image Duplication in Biomedical Research Publications
Bik, E. M., Casadevall, A., & Fang, F. C. (2016) · mBio · doi:10.1128/mbio.00809-16 · Cited by 287
Text-based Editing of Talking-head Video
Fried, O., Tewari, A., Zollhöfer, M., Finkelstein, A., Shechtman, E., Goldman, D. B., Genova, K., Jin, Z., Theobalt, C., & Agrawala, M. (2019) · ACM Transactions on Graphics · doi:10.1145/3306346.3323028 · Cited by 270 · cited 1×
“A related text-based video editing technique permits the addition, removal, and alteration of words in a video from, for example “she sells seashells by the sea shore” to “she sells ice cream by the sea shore” (Fried et al. 2019).”
MethodDeepfake detection by human crowds, machines, and machine-informed crowds
Groh, M., Epstein, Z., Firestone, C., & Picard, R. (2022) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2110013119 · Cited by 248
Detecting Deep-Fake Videos from Appearance and Behavior
Agarwal, S., Farid, H., El-Gaaly, T., & Lim, S.-N. (2020) · 2020 IEEE International Workshop on Information Forensics and Security (WIFS) · doi:10.1109/wifs49906.2020.9360904 · Cited by 242
Self-citationA survey on neural speech synthesis
Tan, X., Qin, T., Soong, F., & Liu, T.-Y. (2021) · arXiv · doi:10.48550/arxiv.2106.15561 · Cited by 216 · cited 1×
“A prototypical text-to-speech system (Taylor 2009; Shi 2021; Tan et al. 2021) consists of two basic parts.”
BackgroundThe Social Impact of Deepfakes
Hancock, J. T., & Bailenson, J. N. (2021) · Cyberpsychology, Behavior, and Social Networking · doi:10.1089/cyber.2021.29208.jth · Cited by 214
Sundar, S. S., Molina, M. D., & Cho, E. (2021) · Journal of Computer-Mediated Communication · doi:10.1093/jcmc/zmab010 · Cited by 212
Generalization of Audio Deepfake Detection
Chen, T., Kumar, A., Nagarsheth, P., Sivaraman, G., & Khoury, E. (2020) · The Speaker and Language Recognition Workshop (Odyssey 2020) · doi:10.21437/odyssey.2020-19 · Cited by 200 · cited 1×
“These three steps can also be incorporated into one end-to-end, learning-based system typically using deep neural networks (see, for example, (Chen et al. 2020)).”
BackgroundEvading Deepfake-Image Detectors with White- and Black-Box Attacks
Carlini, N., & Farid, H. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00337 · Cited by 170
Self-citationWaveFake: A Data Set to Facilitate Audio Deepfake Detection
Frank, J., & Schönherr, L. (2021) · arXiv · doi:10.48550/arxiv.2111.02813 · Cited by 76
Human Perception of Audio Deepfakes
Müller, N. M., Pizzi, K., & Williams, J. (2022) · Proceedings of the 1st International Workshop on Deepfake Detection for Audio Multimedia · doi:10.1145/3552466.3556531 · Cited by 68
Video Rewrite: Driving Visual Speech with Audio
Bregler, C., Covell, M., & Slaney, M. (2023) · Seminal Graphics Papers: Pushing the Boundaries, Volume 2 · doi:10.1145/3596711.3596787 · Cited by 67
Deepfakes: A new threat to image fabrication in scientific publications?
Wang, L., Zhou, L., Yang, W., & Yu, R. (2022) · Patterns · doi:10.1016/j.patter.2022.100509 · Cited by 60 · cited 1×
“Twenty years later, the same GAN machinery that can be used to synthesize faces (see above) can be adapted to synthesize different types of biomedical imagery (L. Wang et al. 2022).”
BackgroundBioscience-scale automated detection of figure element reuse
Acuna, D. E., Brookes, P. S., & Kording, K. P. (2018) · doi:10.1101/269415 · Cited by 54
Persuasive Synthetic Speech: Voice Perception and User Behaviour
Dubiel, M., Halvey, M., Gallegos, P. O., & King, S. (2020) · Proceedings of the 2nd Conference on Conversational User Interfaces · doi:10.1145/3405755.3406120 · Cited by 28 · cited 2×
“Persuasiveness may be desirable (although ethically questionable) for advertising and marketing (see (Dubiel et al. 2020) for an examination of the persuasiveness of synthetic voices.)”
BackgroundWatch Those Words: Video Falsification Detection Using Word-Conditioned Facial Motion
Agarwal, S., Hu, L., Ng, E., Darrell, T., Li, H., & Rohrbach, A. (2023) · 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv56688.2023.00469 · Cited by 28
Detecting CNN-Generated Facial Images in Real-World Scenarios
Hulzebosch, N., Ibrahimi, S., & Worring, M. (2020) · arXiv · doi:10.48550/arxiv.2005.05632 · Cited by 18
Cited by
Deepfake Detection: A Comprehensive Survey from the Reliability Perspective
Wang, T., Liao, X., Chow, K. P., Lin, X., & Wang, Y. (2025) · ACM Computing Surveys · doi:10.1145/3699710 · Cited by 107
Testing human ability to detect ‘deepfake’ images of human faces
Bray, S. D., Johnson, S. D., & Kleinberg, B. (2023) · Journal of Cybersecurity · doi:10.1093/cybsec/tyad011 · Cited by 96
Non-Consensual Synthetic Intimate Imagery: Prevalence, Attitudes, and Knowledge in 10 Countries
Umbach, R., Henry, N., Beard, G. F., & Berryessa, C. M. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642382 · Cited by 73
Human performance in detecting deepfakes: A systematic review and meta-analysis of 56 papers
Diel, A., Lalgi, T., Schröter, I. C., MacDorman, K. F., Teufel, M., & Bäuerle, A. (2024) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2024.100538 · Cited by 64
Studying the Online Deepfake Community
Timmerman, B., Mehta, P., Deb, P., Gallagher, K., Dolan-Gavitt, B., Garg, S., & Greenstadt, R. (2023) · Journal of Online Trust and Safety · doi:10.54501/jots.v2i1.126 · Cited by 17
Metadata
- Authors
- Farid, H.
- Venue
- Journal of Online Trust and Safety
- Publisher
- Stanford Internet Observatory
- Published
- 2022-09-20
- Volume
- 1
- Issue
- 4
- ISSN
- 2770-3142 (ISSN-L)
- OpenAlex
- W4297878838
- Cited by
- 89
- References
- 46
- FWCI
- 5.50
- Type
- journal article
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.