Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 84 of the 91 references it cites were found in a registry or an index; 15 of the 22 sources its claims cite were checked for support.

Do You Really Mean That? Content Driven Audio-Visual Deepfake Dataset and Multimodal Method for Temporal Forgery Localization

Open access
Cai, Zhixi, Stefanov, Kalin, Dhall, Abhinav, Hayat, Munawar · 2022 · 2022 International Conference on Digital Image Computing: Techniques and Applications (DICTA)

Abstract

Introduces LAV-DF, a content-driven audio-visual deepfake dataset and Boundary Aware Temporal Forgery Detection baseline for temporal forgery localization.

Overview

CoChat Summary

TL;DR

The paper introduces LAV-DF, a content-driven audio-visual deepfake dataset where short audio/video segments are manipulated to change the sentiment polarity of otherwise real videos, and proposes BA-TFD, a multimodal boundary-aware method for temporal forgery localization. BA-TFD outperforms prior temporal action localization and deepfake detection baselines on the proposed dataset.

Key findings

  • LAV-DF contains 136,304 videos: 36,431 completely real and 99,873 with fake segments, from 153 unique identities; there are 114,253 fake segments, with duration in [0–1.6] seconds, average length 0.65 seconds, and 89.26% shorter than 1 second.
  • On the full test set, BA-TFD (multimodal) achieves 76.90 AP@0.5 and 66.90 AR@100, outperforming MDS (12.78 AP@0.5), AGT (17.85), BMN (24.01), BMN with I3D (10.56), and AVFusion (65.38).
  • On the visual-only subset, BA-TFD (multimodal) reaches 85.20 AP@0.5 and 67.34 AR@100; the visual-only BA-TFD improves from 58.55 to 83.55 AP@0.5, and the unimodal–multimodal gap decreases from 18.35 to 1.65 AP@0.5.
  • For deepfake classification, BA-TFD obtains AUC 0.990 on LAV-DF, outperforming F3Net (0.520), MDS (0.828), and EfficientViT (0.965); on a DFDC subset it obtains 0.846, below MDS (0.916). Ablation shows frame classification loss contributes most to localization performance.

Methods

  • Dataset generation: real videos are sourced from VoxCeleb2, filtered to English, transcribed with Google Speech-to-Text, and manipulated by replacing tokens with antonyms chosen to maximize sentiment change using NLTK; fake audio is generated with SV2TTS and fake video with Wav2Lip. Three fake variants are included: fake audio + fake video, fake audio + real video, and real audio + fake video.
  • BA-TFD: a 3DCNN video encoder and 2DCNN audio encoder extract frame-level features; training uses contrastive loss, frame classification loss, and boundary matching loss, followed by a multimodal fusion module and S-NMS post-processing at inference.
  • Evaluation: temporal forgery localization is measured with AP at IoU thresholds 0.5, 0.75, 0.95 and AR with 100, 50, 20, 10 proposals; deepfake classification is measured with AUC. The dataset is split identity-independently into training (78,703 videos, 91 identities), validation (31,501 videos, 31 identities), and testing (26,100 videos, 31 identities).

Limitations

  • The paper states: the audio reenactment method does not always generate the reference style; dataset resolution is constrained by the source videos; and the high classification score indicates the need to improve the video reenactment method.
  • Failure analysis in the paper notes that outputs can be noisy for very short video manipulations (≤0.5 sec) with corresponding real audio, especially when the visual transition from real to fake and back is smooth.
  • The paper also notes ethical concerns: the dataset contains celebrities and could be used for unethical purposes such as fake rumours, and the generation pipeline can be used to create fake videos.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1109/dicta56598.2022.10034605

Cites

84 of 84 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

12 Valid · 7 Not valid · 3 Not checked · 42 Own results and interpretations

27 more claims · 3 with their cited sources checked

Create a free account to see all 64 claims and to verify the 2 references still open.

See all claims — free account
  • It may state two separate claims as one. 18
  • Probably not a substantial claim. 15
  • The quote only partly states this claim. 4

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

4 more claims about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

1 of 1

Citations

Cites

Show all 91 references

Research works printed but not linked: 6

  1. Schwartz, 2018, You thought fake news was bad? Deep fakes are where truth goes to die, The Guardian
  2. Brandon, 2019, There Are Now 15,000 Deepfake Videos on Social Media. Yes, You Should Worry, Forbes
  3. 2020, What are deepfakes - and how can you spot them?, The Guardian
  4. Thomas, 2020, Deepfakes: A threat to democracy or just a bit of fun?
  5. Nick, 2019, Contributing Data to Deepfake Detection Research
  6. Bird, 2009, Natural Language Processing with Python: Analyzing Text with the Natural Language Toolkit

Cited by

See all 99

Metadata

Authors
Cai, Z., Stefanov, K., Dhall, A., & Hayat, M.
Affiliations
Monash University (AU)
Venue
2022 International Conference on Digital Image Computing: Techniques and Applications (DICTA)
Publisher
IEEE
Published
2022-11-30
Pages
1-10
OpenAlex
W4319978495
Cited by
99
References
91
Citations by year
2026: 23 · 2025: 42 · 2024: 21 · 2023: 12 · 2022: 1
FWCI
6.89
Type
conference paper
Language
en
Open access
Locations
  • International Conference on Digital Image Computing: Techniques and Applications (DICTA) (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
  • Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
Keywords
deepfake detection, 3D convolutional neural networks, contrastive loss

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-29, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-29 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Do You Really Mean That? Content Driven Audio-Visual Deepfake Dataset and Multimodal Method for Temporal Forgery Localization. https://app.cochat.ai/papers/10.1109/dicta56598.2022.10034605
The paper
Cai, Z., Stefanov, K., Dhall, A., & Hayat, M. (2022). Do You Really Mean That? Content Driven Audio-Visual Deepfake Dataset and Multimodal Method for Temporal Forgery Localization. 2022 International Conference on Digital Image Computing: Techniques and Applications (DICTA). https://doi.org/10.1109/dicta56598.2022.10034605

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.