PreprintCited by 463Digital Media Forensic DetectionFace recognition and analysisGenerative Adversarial Networks and Image Synthesis

The deepfake detection challenge (dfdc) dataset

Open access
Dolhansky, Brian, Bitton, Joanna, Pflaum, Ben, Lu, Jikuo, Howes, Russ, Wang, Menglin, Ferrer, Cristian Canton·2020·arXiv

Abstract

Deepfakes are a recent off-the-shelf manipulation technique that allows anyone to swap two identities in a single video. In addition to Deepfakes, a variety of GAN-based face swapping methods have also been published with accompanying code. To counter this emerging threat, we have constructed an extremely large face swap video dataset to enable the training of detection models, and organized the accompanying DeepFake Detection Challenge (DFDC) Kaggle competition. Importantly, all recorded subjects agreed to participate in and have their likenesses modified during the construction of the face-swapped dataset. The DFDC dataset is by far the largest currently and publicly available face swap video dataset, with over 100,000 total clips sourced from 3,426 paid actors, produced with several Deepfake, GAN-based, and non-learned methods. In addition to describing the methods used to construct the dataset, we provide a detailed analysis of the top submissions from the Kaggle contest. We show although Deepfake detection is extremely difficult and still an unsolved problem, a Deepfake detection model trained only on the DFDC can generalize to real "in-the-wild" Deepfake videos, and such a model can be a valuable analysis tool when analyzing potentially Deepfaked videos. Training, validation and testing corpuses can be downloaded from https://ai.facebook.com/datasets/dfdc.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2006.07397

Cites

24 of 24 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Facebook hosted the DeepFake Detection Challenge (DFDC) in December 2019 with 2114 worldwide participants generating more than 35,000 models.

    “Facebook fosters the development of high-performance DeepFake detection tools by hosting the DeepFake Detection Challenge (DFDC) (Dolhansky et al. 2020) in December 2019 with 2114 worldwide participants generating more than 35,000 models.”

    From Countering Malicious DeepFakes: Survey, Battleground, and Horizon (2022) · cites this paper as Dolhansky et al. 2020

    The Abstract (p. 1) says the authors organized the DFDC Kaggle competition, and §6.1 (p. 10) reports 2,114 participating teams—not necessarily 2,114 individual participants. Our copy does not substantiate the December 2019 date, worldwide participation, or the claim that participants generated more than 35,000 models. (LLM, conf=0.87)

    Experimental — accuracy not yet measured
  • The datasets used by DFDC and DeeperForensics Challenge are significantly larger than previous datasets, with 100,000 videos and 100,000,000 frames.

    “The dataset DeeperForensics-1.0 (Jiang et al. 2020b) and DFDC (Dolhansky et al. 2020) used by these two challenges are significantly larger than the previous datasets. They have 100,000 videos and 100,000,000 numbers of frames.”

    From Countering Malicious DeepFakes: Survey, Battleground, and Horizon (2022) · cites this paper as Dolhansky et al. 2020

    The Abstract (p. 1) reports over 100,000 DFDC clips, and Previous work (p. 3) describes DFDC and DeeperForensics-1.0 as larger than earlier dataset generations. But the comparison table on p. 3 lists 60,000 total videos for DeeperForensics-1.0, not 100,000, and the cited passages do not substantiate the claimed 100,000,000 frames. (LLM, conf=0.88)

    Experimental — accuracy not yet measured
  • DFDC has 128,514 videos, over 100,000 total clips sourced from 3426 actors.

    “The DFDC is a publicly-available face swap video dataset, with 128,514 videos, over 100,000 total clips sourced from 3426 actors.”

    From Countering Malicious DeepFakes: Survey, Battleground, and Horizon (2022) · cites this paper as Dolhansky et al. 2020

    The Introduction (p. 2) says the source footage came from 3,426 paid actors, and Dataset contents (p. 6) reports more than 100,000 clips. But Future work (p. 10) says only 960 of the roughly 3,500 original identities were included in the dataset, so 3,426 is a source-footage count, not a count of actors represented in DFDC. The paper’s dataset counts do not establish the claimed total of 128,514 videos. (LLM, conf=0.90)

    Experimental — accuracy not yet measured
  • Existing detection methods struggle to recognize Deepfakes generated by newer synthesis techniques.

    “many existing detection methods struggle to recognize Deepfakes, synthetic facial images and videos, generated by newer synthesis techniques [8, 12, 26, 27, 43, 54].”

    From Towards More General Video-based Deepfake Detection through Facial Component Guided Adaptation for Foundation Model (2024) · cites this paper as [8]

    Section 6.2 (p. 10) reports that detection performance dropped on real, in-the-wild videos compared with DFDC videos. That supports the general point that detectors struggle, but the paper does not establish that newer synthesis techniques caused the difficulty. (LLM, conf=0.82)

    Experimental — accuracy not yet measured
  • ML-based detection approaches incorporate conventional machine learning techniques, and identifying patterns, anomalies, or inconsistencies in media information is typically accomplished using statistical models and algorithms.

    “ML-based detection approaches incorporate conventional machine learning techniques. Identifying patterns, anomalies, or inconsistencies in media information is typically accomplished using statistical models and algorithms (Dolhansky etal. 2020).”

    From Deepfake video detection: challenges and opportunities (2024) · cites this paper as Dolhansky et al. 2020

    The Introduction (p. 1) describes computer-vision and multimodal models for detecting deepfakes, and the analysis of submitted models (p. 10) describes models used for feature extraction and detection. The paper does not establish that detection typically uses conventional machine-learning or statistical models to identify patterns, anomalies, or inconsistencies in media generally. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Authors
Dolhansky, B., Bitton, J., Pflaum, B., Lu, J., Howes, R., Wang, M., & Ferrer, C. C.
Venue
arXiv
Publisher
arXiv
Published
2020
ISSN
2331-8422 (ISSN-L)
OpenAlex
W3095986900
Cited by
427
References
24
Citations by year
2026: 15 · 2025: 86 · 2024: 135 · 2023: 91 · 2022: 57 · 2021: 40 · 2020: 3
Type
posted content
Language
en
Open access
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
  • TIB Data Manager
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
Keywords
deepfake detection, face swapping, video manipulation, DFDC, synthetic video dataset

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review