MINTIME: Multi-Identity Size-Invariant Video Deepfake Detection
Open accessThe DOI matches its Crossref record; 71 of the 71 references it cites were found in a registry or an index.
Abstract
In this paper, we present MINTIME, a video deepfake detection method that effectively captures spatial and temporal inconsistencies in videos that depict multiple individuals and varying face sizes. Unlike previous approaches that either employ simplistic a-posteriori aggregation schemes, i.e., averaging or max operations, or only focus on the largest face in the video, our proposed method learns to accurately detect spatio-temporal inconsistencies across multiple identities in a video through a Spatio-Temporal Transformer combined with a Convolutional Neural Network backbone. This is achieved through an Identity-aware Attention mechanism that applies a masking operation on the face sequence to process each identity independently, which enables effective video-level aggregation. Furthermore, our system incorporates two novel embedding schemes: (i) the Temporal Coherent Positional Embedding, which encodes the temporal information of the face sequences of each identity, and (ii) the Size Embedding, which captures the relative sizes of the faces to the video frames. MINTIME achieves state-of-the-art performance on the ForgeryNet dataset, with a remarkable improvement of up to 14% AUC in videos containing multiple people. Moreover, it demonstrates very robust generalization capabilities in cross-forgery and cross-dataset settings. The code is publicly available at: https://github.com/davide-coccomini/MINTIME-Multi-Ident ity-size-iNvariant-TIMEsformer-for-Video-Deepfake-Detection.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Deepfake detection has been approached so far by means of passive techniques [185,186,187] that operate after the fact, meaning they are applied after the forged content has been created, distributed, and potentially manipulated.”
From Deepfake Media Forensics: Status and Future Challenges (2025) · cites this paper as [185]
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 3×
“It is capable of receiving as input a sequence of N faces containing one or more identities and detect whether the video has been manipulated.”
BackgroundMethodHochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 1×
“…people in the video comes from [14] in which identities are classified separately and if even one of them is manipulated then the whole video is considered fake but this method is frame-based and relies on an ad hoc aggregation scheme (average or max) also requiring a forward pass for each frame.”
MethodExploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
Xception: Deep Learning with Depthwise Separable Convolutions
Chollet, F. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.195 · Cited by 20,380 · cited 3×
“Consider M I i ∈ { 0 , 1 } ( NHW +1) × 1 as the mask of identity I i indicating to the model which faces attend to an identity and which to another during the attention calculation.”
BackgroundMethodHigh-Resolution Image Synthesis with Latent Diffusion Models
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01042 · Cited by 20,302 · cited 1×
“The impact of the deepfake detection strategy in a case of video containing multiple identities and variations in the face-frame area ratio.”
MethodEfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
Tan, M., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1905.11946 · Cited by 17,072 · cited 3×
“…(red) and vice versa, while all refer to the global CLS. Formally, let Z ∈ R ( NHW +1) × D be the set of all the tokens in which Z I 0 ∈ R 1 × D is the global CLS token and Z I i ∈ R N Ii HW × D is the set of all tokens related to identity I i , where N I i is the number of faces for identity I i .”
BackgroundMethodFaceNet: A unified embedding for face recognition and clustering
Schroff, F., Kalenichenko, D., & Philbin, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298682 · Cited by 15,267 · cited 2×
“Firstly, the features of each face are extracted via an InceptionResnetV1 pretrained on FaceNet [41].”
MethodQuo Vadis, Action Recognition? A New Model and the Kinetics Dataset
Carreira, J., & Zisserman, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.502 · Cited by 9,570
Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks
Zhang, K., Zhang, Z., Li, Z., & Qiao, Y. (2016) · IEEE Signal Processing Letters · doi:10.1109/lsp.2016.2603342 · Cited by 6,214 · cited 2×
“Our proposed model takes as input a sequence composed of faces detected by means of a state-of-the-art face detector, i.e. MTCNN [48] with an additional 30% surface area to include a portion of the background as done in the literature [5, 14].”
MethodEmerging Properties in Self-Supervised Vision Transformers
Caron, M., Touvron, H., Misra, I., Jegou, H., Mairal, J., Bojanowski, P., & Joulin, A. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00951 · Cited by 5,638
Research works printed but not linked: 5
- Goodfellow, Generative adversarial nets, 27, 1
- Dosovitskiy, 2021, An image is worth 16×16 words: Transformers for image recognition at scale, arXiv:2010.11929 · arXiv
- Lei Ba, 2016, Layer normalization, arXiv:1607.06450 · arXiv
- 10.5555/3298023.3298188 · doi:10.5555/3298023.3298188
- Seferbekov, 2020, DFDC 1st Place Solution
Cited by
DeMamba: AI-generated video detection on million-scale GenVideo benchmark
Chen, H., Yan, H., Huang, Z., Xu, Z., Gu, Z., Li, Y., Lan, J., Zhu, H., Zhang, J., Wang, W., et al. (2026) · Science China Information Sciences · doi:10.1007/s11432-024-4894-0 · Cited by 127
“Three types of image-level methods and seven video-level methods are treated as baselines for comparison: F3Net [16], NPR [17], CLIP [88], STIL [20], XCLIP [21], VideoMAE [95], VideoMamba [24], MINTIME [96], FTCN [97], TALL [18].”
Deepfake Media Forensics: Status and Future Challenges
Amerini, I., Barni, M., Battiato, S., Bestagini, P., Boato, G., Bruni, V., Caldelli, R., De Natale, F., De Nicola, R., Guarnera, L., et al. (2025) · Journal of Imaging · doi:10.3390/jimaging11030073 · Cited by 113
“Active Deepfake Detection Deepfake detection has been approached so far by means of passive techniques [185–187] that operate after the fact , meaning they are applied after the forged content has been created, distributed, and potentially manipulated.”
Self-citationDeep convolutional pooling transformer for deepfake detection
Wang, T., Cheng, H., Chow, K. P., & Nie, L. (2023) · ACM Transactions on Multimedia Computing, Communications, and Applications · doi:10.1145/3588574 · Cited by 111
“It is also worth noting that a Transformer based approach, CViT [59], has been proposed with source code published and has shown promising results, and several latest models [8, 9, 26] are further designed following similar ideas but devoting better detection performance.”
MethodD3: Training-Free AI-Generated Video Detection Using Second-Order Features
Zheng, C., Suo, R., Lin, C., Zhao, Z., Yang, L., Liu, S., Yang, M., Wang, C., & Shen, C. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.01194 · Cited by 55
“We report the results of the baselines [16, 19, 24, 34, 50, 58] on GenVideo dataset from [16].”
Generative AI and Disinformation: Recent Advances, Challenges, and Opportunities
Bontcheva, K., Papadopoulous, S., Tsalakanidou, F., Gallotti, R., Dutkiewicz, L., Krack, N., Teyssou, D., Severio Nucci, F., Spangenberg, J., Srba, I., et al. (2024) · Lirias (KU Leuven) · Cited by 54
Metadata
- Authors
- Coccomini, D. A., Zilos, G. K., Amato, G., Caldelli, R., Falchi, F., Papadopoulos, S., & Gennaro, C.
- Affiliations
- Istituto di Scienza e Tecnologie dell'Informazione "Alessandro Faedo" (IT)
- Faculty of Electrical Engineering, Czech Technical University in Prague, Prague, Czech Republic
- Mercatorum University (IT)
- Centre for Research and Technology Hellas (GR)
- ORCID
- Venue
- IEEE Transactions on Information Forensics and Security
- Published
- 2024
- Volume
- 19
- Pages
- 6084-6096
- ISSN
- 1556-6013 (ISSN-L)
- 1556-6021
- OpenAlex
- W4399310664
- Cited by
- 33
- References
- 70
- Citations by year
- 2026: 16 · 2025: 13 · 2024: 3
- FWCI
- 4.47
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- IEEE Transactions on Information Forensics and Security (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- European Commission (P20227PEPK, PE00000014, 951911)
- Italian MUR
- Grantová Agentura České Republiky (21-28830M)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
- Keywords
- spatio-temporal transformer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: MINTIME: Multi-Identity Size-Invariant Video Deepfake Detection. https://app.cochat.ai/papers/10.1109/tifs.2024.3409054
- The paper
- Coccomini, D. A., Zilos, G. K., Amato, G., Caldelli, R., Falchi, F., Papadopoulos, S., & Gennaro, C. (2024). MINTIME: Multi-Identity Size-Invariant Video Deepfake Detection. IEEE Transactions on Information Forensics and Security. https://doi.org/10.1109/tifs.2024.3409054
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.