The DOI matches its Crossref record; 84 of the 91 references it cites were found in a registry or an index; 15 of the 22 sources its claims cite were checked for support.
Do You Really Mean That? Content Driven Audio-Visual Deepfake Dataset and Multimodal Method for Temporal Forgery Localization
Open accessAbstract
Introduces LAV-DF, a content-driven audio-visual deepfake dataset and Boundary Aware Temporal Forgery Detection baseline for temporal forgery localization.
Overview
CoChat Summary
TL;DR
The paper introduces LAV-DF, a content-driven audio-visual deepfake dataset where short audio/video segments are manipulated to change the sentiment polarity of otherwise real videos, and proposes BA-TFD, a multimodal boundary-aware method for temporal forgery localization. BA-TFD outperforms prior temporal action localization and deepfake detection baselines on the proposed dataset.
Key findings
- LAV-DF contains 136,304 videos: 36,431 completely real and 99,873 with fake segments, from 153 unique identities; there are 114,253 fake segments, with duration in [0–1.6] seconds, average length 0.65 seconds, and 89.26% shorter than 1 second.
- On the full test set, BA-TFD (multimodal) achieves 76.90 AP@0.5 and 66.90 AR@100, outperforming MDS (12.78 AP@0.5), AGT (17.85), BMN (24.01), BMN with I3D (10.56), and AVFusion (65.38).
- On the visual-only subset, BA-TFD (multimodal) reaches 85.20 AP@0.5 and 67.34 AR@100; the visual-only BA-TFD improves from 58.55 to 83.55 AP@0.5, and the unimodal–multimodal gap decreases from 18.35 to 1.65 AP@0.5.
- For deepfake classification, BA-TFD obtains AUC 0.990 on LAV-DF, outperforming F3Net (0.520), MDS (0.828), and EfficientViT (0.965); on a DFDC subset it obtains 0.846, below MDS (0.916). Ablation shows frame classification loss contributes most to localization performance.
Methods
- Dataset generation: real videos are sourced from VoxCeleb2, filtered to English, transcribed with Google Speech-to-Text, and manipulated by replacing tokens with antonyms chosen to maximize sentiment change using NLTK; fake audio is generated with SV2TTS and fake video with Wav2Lip. Three fake variants are included: fake audio + fake video, fake audio + real video, and real audio + fake video.
- BA-TFD: a 3DCNN video encoder and 2DCNN audio encoder extract frame-level features; training uses contrastive loss, frame classification loss, and boundary matching loss, followed by a multimodal fusion module and S-NMS post-processing at inference.
- Evaluation: temporal forgery localization is measured with AP at IoU thresholds 0.5, 0.75, 0.95 and AR with 100, 50, 20, 10 proposals; deepfake classification is measured with AUC. The dataset is split identity-independently into training (78,703 videos, 91 identities), validation (31,501 videos, 31 identities), and testing (26,100 videos, 31 identities).
Limitations
- The paper states: the audio reenactment method does not always generate the reference style; dataset resolution is constrained by the source videos; and the high classification score indicates the need to improve the video reenactment method.
- Failure analysis in the paper notes that outputs can be noisy for very short video manipulations (≤0.5 sec) with corresponding real audio, especially when the visual transition from real to fake and back is smooth.
- The paper also notes ethical concerns: the dataset contains celebrities and could be used for unethical purposes such as fake rumours, and the generation pipeline can be used to create fake videos.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
12 Valid · 7 Not valid · 3 Not checked · 42 Own results and interpretations
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“However, the manipulations present in that dataset are only identity modifications that do not necessarily alter the meaning of the content.”
What the sources say
- Disputedcites [27] sourceExperimental — accuracy not yet measured
Section 3.2.1 (p. 4) distinguishes identity-replaced from identity-remained forgery approaches. Figure 5 identifies reenactment as manipulating intrinsic attributes and editing as manipulating external attributes. ForgeryNet therefore does not contain only identity modifications; the passages do not establish whether those changes alter the meaning of the content. (LLM, conf=0.90)
Checked by a reasoning model — may be wrong.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“The multimodal MDS method is not designed for temporal forgery localization tasks”
What the sources say
- Disputedcites [28] sourceExperimental — accuracy not yet measured
The Introduction identifies temporal forgery localization as a key contribution and says chunk-wise learning enables localization of transient video forgeries. Section 4.6 demonstrates the MDS method assigning segment-wise scores to locate forged portions of videos, contrary to the claim. (LLM, conf=0.92)
Checked by a reasoning model — may be wrong.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
Every source the paper cites for this claim was checked and supports it.
In the paper
“Only the latest one, ForgeryNet, provides examples of the important problem of temporal forgery localization since it includes random face-swapping applied to parts of some videos.”
What the sources say
- Supportedcites [27] source
The Video Forgery Classification & Temporal Forgery Localization passage (page not available) says ForgeryNet randomly selects one to four manipulated video segments, fills the rest with real footage, and annotates the segments for temporal localization. Figure 3 identifies face swapping among ForgeryNet’s forgery approaches. The paper contrasts its untrimmed, partly manipulated videos with existing datasets, supporting the citing sentence’s comparison. (LLM, conf=0.88)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“All previous datasets provide face manipulations that occur in most of the frames of the video [28].”
What the sources say
- Partly supportedcites [28] sourceExperimental — accuracy not yet measured
Section 4.6, “Temporal Forgery Localization” (page not available), says that most datasets, including those used in the paper, have manipulation across practically the entire video. It does not say this is true of all previous datasets or that every such manipulation is facial; Section 4.1 notes that the DFDC dataset can contain audio manipulation. (LLM, conf=0.88)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“The importance of multimodality was demonstrated recently by AVFusion [46].”
What the sources say
- Partly supportedcites [46] Bagchi, 2021, Hear Me Out: Fusional Approaches for Audio Augmented Temporal Action Localization, arXiv sourceExperimental — accuracy not yet measured
The Abstract (p. 1) reports that combining audio and video improves temporal action localization over video-only approaches, supporting the general point about multimodality. The paper studies action localization, not temporal forgery localization, so it does not demonstrate the claim in that setting. (LLM, conf=0.86)
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
27 more claims · 3 with their cited sources checked
Create a free account to see all 64 claims and to verify the 2 references still open.
- It may state two separate claims as one. 18
- Probably not a substantial claim. 15
- The quote only partly states this claim. 4
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“there are currently only two datasets [3, 7] specifically designed for the TFL task, both of which are English-centric.”
From CLFormer: a cross-lingual transformer framework for temporal forgery localization (2025) · cites this paper as [3]
The cited source says what the claim attributes to it.
In the citing paper
“Following previous deepfake dataset generation research [6, 31], the dataset includes three different combinations of modified modalities in the generated fake videos.”
From AV-Deepfake1M: A Large-Scale LLM-Driven Audio-Visual Deepfake Dataset (2024) · cites this paper as [6]
The cited source says what the claim attributes to it.
In the citing paper
“In 2022, LAV-DF [6] was introduced to become the first contentdriven deepfake dataset for temporal localization.”
From AV-Deepfake1M: A Large-Scale LLM-Driven Audio-Visual Deepfake Dataset (2024) · cites this paper as [6]
The cited source says what the claim attributes to it.
In the citing paper
“There is a significant drop in BA-TFD [6] temporal localization performance as compared to LAV-DF (Table 8).”
From AV-Deepfake1M: A Large-Scale LLM-Driven Audio-Visual Deepfake Dataset (2024) · cites this paper as [6]
The cited source says what the claim attributes to it.
In the citing paper
“Deepfake localization approaches are either anchor-based [21, 22] that utilize a sliding window approach to detect deepfake segments or boundary prediction based [6, 10, 9, 64] that focuses on predicting the boundary of fake segments in a video using boundary matching loss.”
From Contextual Cross-Modal Attention for Audio-Visual Deepfake Detection and Localization (2024) · cites this paper as [10]
4 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ II. RELATED WORK · p. 2
“Multiple face manipulation methods generated 128,154 videos, including real videos of 3000 actors.”
§ I. INTRODUCTION · p. 2
“| DFDC [16] | 2020 | Cla | AV | FS | 960 | 23,654 | 104,500 | 128,154 |”
Citations
Cites
PyTorch: An Imperative Style, High-Performance Deep Learning Library
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al. (2019) · arXiv · doi:10.48550/arxiv.1912.01703 · Cited by 31,473
Generative adversarial networks
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., & Bengio, Y. (2020) · Communications of the ACM · doi:10.1145/3422622 · Cited by 18,384
Learning Internal Representations by Error Propagation
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1985) · doi:10.21236/ada164453 · Cited by 16,172
Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset
Carreira, J., & Zisserman, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.502 · Cited by 9,570 · cited 1×
“For comparison, we also trained BMN with I3D features [69] (i.”
MethodWaveNet: a generative model for raw audio
Oord, A. V. D., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1609.03499 · Cited by 4,911 · cited 2×
“The SV2TTS comprises three modules 1) An encoder for extracting style embedding of the reference speaker, 2) Tacotron 2 [59] based spectrogram generated using the replacement tokens and the speaker style embedding, and 3) WaveNet [60] based vocoder for generating realistic audio using the spectrogram.”
MethodFaceForensics++: Learning to Detect Manipulated Facial Images
Rossler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., & Niessner, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00009 · Cited by 3,314 · cited 3×
“This has led to the release of several benchmark datasets [14], [15], [16] and methods [17] for fake content detection.”
BackgroundLearning internal representations by error propagation
RUMELHART, D. E., HINTON, G. E., & WILLIAMS, R. J. (1988) · Readings in Cognitive Science · doi:10.1016/b978-1-4832-1446-7.50035-2 · Cited by 3,002
Natural TTS Synthesis by Conditioning Wavenet on MEL Spectrogram Predictions
Shen, J., Pang, R., Weiss, R. J., Schuster, M., Jaitly, N., Yang, Z., Chen, Z., Zhang, Y., Wang, Y., Skerrv-Ryan, R., et al. (2018) · 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2018.8461368 · Cited by 2,791
Dlib-ml: A Machine Learning Toolkit
King, D. E. (2009) · doi:10.5555/1577069.1755843 · Cited by 2,777
ActivityNet: A large-scale video benchmark for human activity understanding
Heilbron, F. C., Escorcia, V., Ghanem, B., & Niebles, J. C. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298698 · Cited by 2,578
Research works printed but not linked: 6
- Schwartz, 2018, You thought fake news was bad? Deep fakes are where truth goes to die, The Guardian
- Brandon, 2019, There Are Now 15,000 Deepfake Videos on Social Media. Yes, You Should Worry, Forbes
- 2020, What are deepfakes - and how can you spot them?, The Guardian
- Thomas, 2020, Deepfakes: A threat to democracy or just a bit of fun?
- Nick, 2019, Contributing Data to Deepfake Detection Research
- Bird, 2009, Natural Language Processing with Python: Analyzing Text with the Natural Language Toolkit
Cited by
AVoiD-DF: Audio-Visual Joint Learning for Detecting Deepfake
Yang, W., Zhou, X., Chen, Z., Guo, B., Ba, Z., Xia, Z., Cao, X., & Ren, K. (2023) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2023.3262148 · Cited by 198
MARLIN: Masked Autoencoder for facial video Representation LearnINg
Cai, Z., Ghosh, S., Stefanov, K., Dhall, A., Cai, J., Rezatofighi, H., Haffari, R., & Hayat, M. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00150 · Cited by 165
Self-citationVoice-Face Homogeneity Tells Deepfake
Cheng, H. H., Guo, Y., Wang, T., Li, Q., Chang, X., & Nie, L. (2023) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3625231 · Cited by 171
Cai, Z., Ghosh, S., Dhall, A., Gedeon, T., Stefanov, K., & Hayat, M. (2023) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2023.103818 · Cited by 63
Self-citationPVASS-MDD: Predictive Visual-Audio Alignment Self-Supervision for Multimodal Deepfake Detection
Yu, Y., Liu, X., Ni, R., Yang, S., Zhao, Y., & Kot, A. C. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3309899 · Cited by 63
Metadata
- Authors
- Cai, Z., Stefanov, K., Dhall, A., & Hayat, M.
- Affiliations
- Monash University (AU)
- Venue
- 2022 International Conference on Digital Image Computing: Techniques and Applications (DICTA)
- Publisher
- IEEE
- Published
- 2022-11-30
- Pages
- 1-10
- OpenAlex
- W4319978495
- Cited by
- 99
- References
- 91
- Citations by year
- 2026: 23 · 2025: 42 · 2024: 21 · 2023: 12 · 2022: 1
- FWCI
- 6.89
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- International Conference on Digital Image Computing: Techniques and Applications (DICTA) (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Keywords
- deepfake detection, 3D convolutional neural networks, contrastive loss
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-29, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-29 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Do You Really Mean That? Content Driven Audio-Visual Deepfake Dataset and Multimodal Method for Temporal Forgery Localization. https://app.cochat.ai/papers/10.1109/dicta56598.2022.10034605
- The paper
- Cai, Z., Stefanov, K., Dhall, A., & Hayat, M. (2022). Do You Really Mean That? Content Driven Audio-Visual Deepfake Dataset and Multimodal Method for Temporal Forgery Localization. 2022 International Conference on Digital Image Computing: Techniques and Applications (DICTA). https://doi.org/10.1109/dicta56598.2022.10034605
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.