The DOI matches its Crossref record; 43 of the 43 references it cites were found in a registry or an index; 7 of the 14 sources its claims cite were checked for support.
CLFormer: a cross-lingual transformer framework for temporal forgery localization
Open accessAbstract
Abstract Temporal forgery localization (TFL) is crucial in deepfake detection. It focuses on identifying subtle temporal manipulations within video content. However, the generalization capabilities of current TFL methods are limited, especially across different languages, which limits their performance in diverse environments. This limitation stems from two key factors. First, most existing datasets are English-centric. Second, there is inadequate learning from multi-modal information, where visual features are often prioritized over audio features. To address this gap, we created the Chinese audio-visual deepfake (CHAV-DF) dataset, which is the first dataset designed for the TFL in the Chinese context. This dataset provides a valuable benchmark for evaluating the TFL methods in cross-lingual settings. Additionally, we introduced a cross-lingual transformer framework (CLFormer), which prioritizes audio features and utilizes a pre-trained multi-lingual Wav2Vec2 to enhance cross-lingual generalization, while incorporating visual features to further refine TFL. Moreover, we incorporated a refinement module into CLFormer to enhance the accuracy of forgery localization. Experiments on the LAV-DF, CHAV-DF, and AV-Deepfake1M datasets demonstrate that CLFormer performs well in both same-language and cross-language settings. Specifically, CLFormer achieves an average precision (AP) of 57.68% at temporal intersection over union (tIoU) of 0.50 when trained on CHAV-DF and tested on LAV-DF, surpassing the state-of-the-art method by 47.59%, and validating its cross-language generalization capability.
Overview
CoChat Summary
## TL;DR
The paper introduces CHAV-DF, a Chinese audio-visual deepfake dataset for temporal forgery localization, and CLFormer, a cross-lingual transformer framework that emphasizes audio features via Wav2Vec2 plus a refinement module. The authors report strong same-language and cross-language results, including 57.68% AP at tIoU 0.50 when trained on CHAV-DF and tested on LAV-DF, which they say is 47.59% above the state of the art.
## Key findings
- CHAV-DF contains 12,951 videos: 1,080 authentic and 11,871 with one manipulated segment, totaling 27.78 h; it is split into 9,099 training, 1,920 validation, and 1,932 test videos.
- On LAV-DF, CLFormer reaches 99.33% AP at tIoU 0.50 and 70.07% AP at tIoU 0.95, outperforming the second-best method (VIGO) by 2.71% at 0.50 and 5.03% at 0.95, and improving average AR by 3.51%.
- On AV-Deepfake1M, CLFormer improves over UMMAFormer by 14.42% in AP at tIoU 0.95 and by 6.71% in average AP.
- In cross-language testing, training on CHAV-DF and testing on LAV-DF yields 57.68% AP at tIoU 0.50, surpassing the state of the art by 47.59%; the paper also reports 13.06% average AP and 56.72% average AR when trained on LAV-DF and tested on CHAV-DF.
## Methods
- The paper constructs CHAV-DF from CUC-RED real videos (12 broadcasting/hosting students, 90 sentences each) and generates manipulations by modifying text, synthesizing fake audio with three Chinese TTS systems, and creating local fake visuals with Wav2Lip.
- CLFormer extracts visual features with TSN and audio features with pre-trained Wav2Vec2, concatenates them, applies a multi-level transformer-based TFL detector, and then refines boundaries with a refinement module inspired by RefineTAD.
- Experiments evaluate CLFormer on LAV-DF, AV-Deepfake1M, and CHAV-DF using AP and AR at tIoU thresholds 0.50, 0.75, and 0.95, with training at 25 fps and audio sampled at 16 kHz.
## Limitations
- The paper states that current TFL methods still struggle with high-precision temporal localization and increasingly sophisticated forgery scenarios.
- The authors note that CHAV-DF is Chinese-only and say future work should expand the dataset to more languages from different linguistic families.
- Evident from the text: many reported tables are partially garbled in the provided excerpt, but the paper itself still clearly states the headline results and dataset statistics.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
3 Valid · 1 Not valid · 7 Not checked · 51 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“the generalization capabilities of current TFL methods are limited, especially across different languages”
- Also stated in: § 1 Introduction · boosted
“the effectiveness of existing methods [3-6] is significantly constrained by their generalization capabilities, particularly in cross-lingual contexts.”
What the sources say
- Could not verifycites [3] Cai, Z., Stefanov, K., Dhall, A., & Hayat, M. (2022). Do you really mean that? Content driven audio-visual deepfake dataset and multimodal method for temporal forgery localization. In Proceedings of the 2022 international conference on digital image computing: techniques and applications (pp. 1–10). Piscataway: IEEE. sourceExperimental — accuracy not yet measured
The Results (p. 6) compare methods on the proposed dataset, but do not test their generalization across languages. The Proposed Dataset section (p. 3) says only English-speaking videos were selected, so the paper provides no cross-language evidence for the claim. Searches for language and generalization did not locate such a finding. (LLM, conf=0.73)
- Could not verifycites [4] Cai, Z., Ghosh, S., Dhall, A., Gedeon, T., Stefanov, K., & Hayat, M. (2023). Glitch in the matrix: a large scale benchmark for content driven audio-visual forgery detection and localization. Computer Vision and Image Understanding, 236, 103818. sourceExperimental — accuracy not yet measured
Our copy contains no statement about the generalization capability of temporal forgery localization (TFL) methods across languages. The only 'generalization' remark is in Sec. 2.1 (Deepfake Detection Datasets, p. 2), where the small size of early deepfake detection datasets is said to give them 'low generalization capability', and that point is credited to reference [56] (Celeb-DF), not made about TFL methods or languages. The paper's actual criticisms of prior work are that detection methods are classification-centric and lack temporal localization (Sec. 2.2, p. 2) and that temporal action localization methods are visual-only (Sec. 2.3, p. 3); its only mention of language is that the LAV-DF dataset itself includes English-speaking videos only (Sec. 3.1, p. 3). Searched: generalization across languages, cross-lingual/language diversity, unseen/cross-dataset generalization of TFL methods — nothing speaks to the claim. (LLM, conf=0.70)
- Not matchedcites [5] Cheng, H., Yu, H., Fang, L., & Ye, L. (2023). MABC-Net: multimodal mixed attentional network with balanced class for temporal forgery localization. In G. Zhai, J. Zhou, L. Ye, H. Yang, P. An, & X. Yang (Eds.), Proceedings of the 20th international forum on digital TV and wireless multimedia communications (pp. 269–283). Cham: Springer.
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Partly supportedcites [6] Zhang, R., Wang, H., Du, M., Liu, H., Zhou, Y., & Zeng, Q. (2023). Ummaformer: a universal multimodal-adaptive transformer framework for temporal forgery localization. In A. El-Saddik, T. Mei, R. Cucchiara, M. Bertini, D. P. T. Vallejo, P. K. Atrey, & M. S. Hossain (Eds.), Proceedings of the 31st ACM international conference on multimedia (pp. 8749–8759). New York: ACM. sourceExperimental — accuracy not yet measured
Our copy does state that current TFL/temporal-forgery work is limited in reach: the Introduction ('Original Frames Modified Frames' section) says real-world scenarios involve audio-only, visual-only and audio-visual data 'requiring separate models for manipulation detection', that only a few studies validated TFL on a single dataset each (Lav-DF, Psynd), and that 'this narrow scope limits the potential applications for forgery detection and TFL'; Related Work 2.2 adds 'Some studies [6,11,21,58] have focused on this task, but there is still room for significant improvement.' However, the limitation the paper identifies is across modalities, scenarios and datasets — no passage attributes limited generalization across different languages to prior TFL methods. The nearest mention is §5.4, which only notes 'LFSS is the only work so far focused on localizing voice cloning partially faked English speech,' a coverage observation, not a cross-language generalization claim. The 'especially across different languages' qualifier is therefore not supported by this source. (LLM, conf=0.72)
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 1 Introduction · boosted
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“CLFormer achieves an average precision (AP) of 57.68% at temporal intersection over union (tIoU) of 0.50 when trained on CHAV-DF and tested on LAV-DF”
- Also stated in: § 1 Introduction · plain
“achieving average precision (AP) of 57.68% at temporal intersection over union (tIoU) of 0.50 on LAV-DF [3] when trained on CHAV-DF”
What the sources say
- Could not verifycites [3] Cai, Z., Stefanov, K., Dhall, A., & Hayat, M. (2022). Do you really mean that? Content driven audio-visual deepfake dataset and multimodal method for temporal forgery localization. In Proceedings of the 2022 international conference on digital image computing: techniques and applications (pp. 1–10). Piscataway: IEEE. sourceExperimental — accuracy not yet measured
The Introduction (p. 2) identifies LAV-DF, and the Experiments section (p. 6) describes AP at an IoU threshold of 0.5. The Results (pp. 6–7) report results for other methods, but no passage found by searching for CLFormer, CHAV-DF, or 57.68 reports the claimed cross-dataset result. (LLM, conf=0.78)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 1 Introduction · plain
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“there are currently only two datasets [3, 7] specifically designed for the TFL task, both of which are English-centric.”
- Also stated in: § 2.1 Deepfake detection datasets · plain
“both LAV-DF and AV-Deepfake1M are primarily English-language datasets”
What the sources say
- Supportedcites [3] Cai, Z., Stefanov, K., Dhall, A., & Hayat, M. (2022). Do you really mean that? Content driven audio-visual deepfake dataset and multimodal method for temporal forgery localization. In Proceedings of the 2022 international conference on digital image computing: techniques and applications (pp. 1–10). Piscataway: IEEE. source
The paper identifies LAV-DF as a dataset designed for temporal forgery localization (p. 1). In “III. Proposed Dataset” (p. 3), it says only English-speaking videos were selected for LAV-DF, supporting this citation as an example of an English-centric TFL dataset. (LLM, conf=0.90)
- Partly supportedcites [7] Cai, Z., Ghosh, S., Adatia, A. P, Hayat, M., Dhall, A., Gedeon, T., & Stefanov, K. (2024). AV-Deepfake1M: a large-scale LLM-driven audio-visual deepfake dataset. In J. Cai, M. S. Kankanhalli, B. Prabhakaran, S. Boll, R. Subramanian, L. Zheng, V. K. Singh, P. César, L. Xie, & D. Xu (Eds.), Proceedings of the 32nd ACM international conference on multimedia (pp. 7414–7423). New York: ACM. sourceExperimental — accuracy not yet measured
The Related Work section (page unavailable) identifies LAV-DF as the only other publicly available dataset designed for the same temporal-localization task, supporting the count of two when AV-Deepfake1M is included. Our copy does not state that either dataset is English-centric; a search for English and language diversity found no passage establishing that part. (LLM, conf=0.84)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 2.1 Deepfake detection datasets · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Moreover, these features significantly increase the recall rate and reduce the number of false positive detections.”
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“On CHAV-DF, CLFormer achieves an impressive 8.58% improvement in the AP at tIoU of 0.95 compared with UMMAFormer”
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
26 more claims
Create a free account to see all 62 claims and to verify the 4 references still open.
- Probably not a substantial claim. 10
- It may state two separate claims as one. 25
Citations
Cites
Proceedings of IEEE Conference on Computer Vision and Pattern Recognition
(1993) · doi:10.1109/cvpr.1993 · Cited by 17,667
Temporal Segment Networks: Towards Good Practices for Deep Action Recognition
Wang, L., Xiong, Y., Wang, Z., Qiao, Y., Lin, D., Tang, X., & Van Gool, L. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46484-8_2 · Cited by 3,958
ActivityNet: A large-scale video benchmark for human activity understanding
Heilbron, F. C., Escorcia, V., Ghanem, B., & Niebles, J. C. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298698 · Cited by 2,578
wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations
Baevski, A., Zhou, H., Mohamed, A., & Auli, M. (2020) · arXiv · doi:10.48550/arxiv.2006.11477 · Cited by 2,369
Proceedings IEEE International Conference on Acoustics Speech and Signal Processing
Zheng, F., McLaughlin, S., & Mulgrew, B. (1991) · Cited by 1,905
Diffusion Models in Vision: A Survey
Croitoru, F.-A., Hondru, V., Ionescu, R. T., & Shah, M. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3261988 · Cited by 1,859
Proceedings for the 5th International Conference on Learning Representations
Srivastava, A., & Sutton, C. A. (2017) · Cited by 1,837
Proceedings of the 3rd International Conference on Neural Information Processing Systems
Lippmann, R. P., Moody, J. E., & Touretzky, D. S. (1990) · Cited by 1,211
Proceedings Of The International Joint Conference On Neural Networks 2003
(2003) · Proceedings of the International Joint Conference on Neural Networks, 2003. · doi:10.1109/ijcnn.2003.1223944 · Cited by 950
A Lip Sync Expert Is All You Need for Speech to Lip Generation In the Wild
Prajwal, K. R., Mukhopadhyay, R., Namboodiri, V. P., & Jawahar, C. V. (2020) · Proceedings of the 28th ACM International Conference on Multimedia · doi:10.1145/3394171.3413532 · Cited by 897
Cited by
Armağan, S., Gündoğan, E., Kaya, M., & Alhajj, R. (2026) · Ain Shams Engineering Journal · doi:10.1016/j.asej.2026.104233 · Cited by 3
AI-Powered Deepfake Detection System for Audio and Video Integrity
Yesugade, A. K., Yesugade, K. D., Sawant, S., Jadhav, P. J., Bhagwat, P., Bombe, A., Gogawale, S., & Ghanwat, E. (2026) · International Conference on Computing for Sustainable Global Development · doi:10.23919/indiacom70271.2026.11525659
“Visual techniques primarily focused on the detection of artifacts like abnormal facial textures, unnatural blinking, or frame-level inconsistencies [5],[6],[14].”
Influential
Metadata
- Authors
- Cheng, H., Liu, H., Cai, J., & Ye, L.
- Affiliations
- State Key Laboratory of Media Convergence and Communication (CN)
- Communication University of China (CN)
- Corresponding author
- Cai, Juanjuan
- Venue
- Visual Intelligence
- Published
- 2025-07-11
- Volume
- 3
- Issue
- 1
- ISSN
- 2731-9008 (ISSN-L)
- 2097-3330
- OpenAlex
- W4412193028
- Cited by
- 2
- References
- 33
- Citations by year
- 2026: 2
- FWCI
- 0.70
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Visual Intelligence (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- National Natural Science Foundation of China (62201524)
- Beijing Natural Science Foundation (4252011)
- Fundamental Research Funds for the Central Universities (CUC25CGJ01)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Keywords
- deepfake detection, cross-lingual generalization, wav2vec 2.0, audiovisual integration
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: CLFormer: a cross-lingual transformer framework for temporal forgery localization. https://app.cochat.ai/papers/10.1007/s44267-025-00084-z
- The paper
- Cheng, H., Liu, H., Cai, J., & Ye, L. (2025). CLFormer: a cross-lingual transformer framework for temporal forgery localization. Visual Intelligence. https://doi.org/10.1007/s44267-025-00084-z
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.