A New Deep Learning-Based Methodology for Video Deepfake Detection Using XGBoost
Open accessThe DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.
Abstract
Currently, face-swapping deepfake techniques are widely spread, generating a significant number of highly realistic fake videos that threaten the privacy of people and countries. Due to their devastating impacts on the world, distinguishing between real and deepfake videos has become a fundamental issue. This paper presents a new deepfake detection method: you only look once–convolutional neural network–extreme gradient boosting (YOLO-CNN-XGBoost). The YOLO face detector is employed to extract the face area from video frames, while the InceptionResNetV2 CNN is utilized to extract features from these faces. These features are fed into the XGBoost that works as a recognizer on the top level of the CNN network. The proposed method achieves 90.62% of an area under the receiver operating characteristic curve (AUC), 90.73% accuracy, 93.53% specificity, 85.39% sensitivity, 85.39% recall, 87.36% precision, and 86.36% F1-measure on the CelebDF-FaceForencics++ (c23) merged dataset. The experimental study confirms the superiority of the presented method as compared to the state-of-the-art methods.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
XGBoost: A Scalable Tree Boosting System
Chen, T., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939785 · Cited by 54,569
You Only Look Once: Unified, Real-Time Object Detection
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.91 · Cited by 53,803
The meaning and use of the area under a receiver operating characteristic (ROC) curve.
Hanley, J. A., & McNeil, B. J. (1982) · Radiology · doi:10.1148/radiology.143.1.7063747 · Cited by 21,856 · cited 1×
“It is a single scalar value that evaluates the performance of a binary classifier [52].”
BackgroundXception: Deep Learning with Depthwise Separable Convolutions
Chollet, F. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.195 · Cited by 20,380
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
Tan, M., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1905.11946 · Cited by 17,072
Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks
Zhang, K., Zhang, Z., Li, Z., & Qiao, Y. (2016) · IEEE Signal Processing Letters · doi:10.1109/lsp.2016.2603342 · Cited by 6,214
FaceForensics++: Learning to Detect Manipulated Facial Images
Rossler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., & Niessner, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00009 · Cited by 3,314
MesoNet: a Compact Facial Video Forgery Detection Network
Afchar, D., Nozick, V., Yamagishi, J., & Echizen, I. (2018) · 2018 IEEE International Workshop on Information Forensics and Security (WIFS) · doi:10.1109/wifs.2018.8630761 · Cited by 1,872
Celeb-DF: A Large-Scale Challenging Dataset for DeepFake Forensics
Li, Y., Yang, X., Sun, P., Qi, H., & Lyu, S. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00327 · Cited by 1,846 · cited 3×
“The Celeb-DF test set is especially utilized for testing due to its fake videos being created using an enhanced deepfake algorithm [25].”
MethodRetinaFace: Single-Shot Multi-Level Face Localisation in the Wild
Deng, J., Guo, J., Ververas, E., Kotsia, I., & Zafeiriou, S. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00525 · Cited by 1,681
Exposing Deep Fakes Using Inconsistent Head Poses
Yang, X., Li, Y., & Lyu, S. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8683164 · Cited by 1,196
In Ictu Oculi: Exposing AI Created Fake Videos by Detecting Eye Blinking
Li, Y., Chang, M.-C., & Lyu, S. (2018) · 2018 IEEE International Workshop on Information Forensics and Security (WIFS) · doi:10.1109/wifs.2018.8630787 · Cited by 1,089
Capsule-forensics: Using Capsule Networks to Detect Forged Images and Videos
Nguyen, H. H., Yamagishi, J., & Echizen, I. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8682602 · Cited by 951
On the Detection of Digital Face Manipulation
Dang, H., Liu, F., Stehouwer, J., Liu, X., & Jain, A. K. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00582 · Cited by 764
DeeperForensics-1.0: A Large-Scale Dataset for Real-World Face Forgery Detection
Jiang, L., Li, R., Wu, W., Qian, C., & Loy, C. C. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00296 · Cited by 599
Multi-task Learning for Detecting and Segmenting Manipulated Facial Images and Videos
Nguyen, H. H., Fang, F., Yamagishi, J., & Echizen, I. (2019) · 2019 IEEE 10th International Conference on Biometrics Theory, Applications and Systems (BTAS) · doi:10.1109/btas46853.2019.9185974 · Cited by 549
Two-Branch Recurrent Network for Isolating Deepfakes in Videos
Masi, I., Killekar, A., Mascarenhas, R. M., Gurudatt, S. P., & AbdAlmageed, W. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58571-6_39 · Cited by 471
WildDeepfake: A Challenging Real-World Dataset for Deepfake Detection
Zi, B., Chang, M., Chen, J., Ma, X., & Jiang, Y.-G. (2020) · Proceedings of the 28th ACM International Conference on Multimedia · doi:10.1145/3394171.3413769 · Cited by 448 · cited 1×
“0 [27], and WildDeep-fake [28].”
Recurrent convolutional strategies for face manipulation detection in videos
Sabir, E., Cheng, J., Jaiswal, A., AbdAlmageed, W., Masi, I., & Natarajan, P. (2019) · arXiv · doi:10.48550/arxiv.1905.00582 · Cited by 443 · cited 1×
“Sabir et al. [38] employ face masks [17] for detecting the face region from frames.”
BackgroundDeepfakes Detection with Automatic Face Weighting
Montserrat, D. M., Hao, H., Yarlagadda, S. K., Baireddy, S., Shao, R., Horvath, J., Bartusiak, E., Yang, J., Guera, D., Zhu, F., et al. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00342 · Cited by 181 · cited 5×
“YOLO + (EfficientNet-B5 [43]) + (SVM [23]) 83.”
MethodInfluentialDetecting Deepfakes with Metric Learning
Kumar, A., Bhavsar, A., & Verma, R. (2020) · 2020 8th International Workshop on Biometrics and Forensics (IWBF) · doi:10.1109/iwbf49977.2020.9107962 · Cited by 90 · cited 3×
“(MTCNN [34,36,43]) + InceptionResNetV2 XGBoost 86.”
MethodiCaps-Dfake: An Integrated Capsule-Based Model for Deepfake Image and Video Detection
Khalil, S. S., Youssef, S. M., & Saleh, S. N. (2021) · Future Internet · doi:10.3390/fi13040093 · Cited by 67 · cited 1×
“The authors in Khalil et al. [35] use YOLO v3 for face detection.”
BackgroundNguyen, X. H., Tran, T. S., Le, V. T., Nguyen, K. D., & Truong, D.-T. (2021) · Forensic Science International: Digital Investigation · doi:10.1016/j.fsidi.2021.301108 · Cited by 53
The Deepfake Challenges and Deepfake Video Detection
Lecturer at Wolkite University, W. E., & Wubet*, W. M. (2020) · International Journal of Innovative Technology and Exploring Engineering · doi:10.35940/ijitee.e2779.049620 · Cited by 27
Video Face Manipulation Detection Through Ensemble of CNNs
Bonettini, N., Cannas, E. D., Mandelli, S., Bondi, L., Bestagini, P., & Tubaro, S. (2021) · 2020 25th International Conference on Pattern Recognition (ICPR) · doi:10.1109/icpr48806.2021.9412711 · Cited by 27
Research works printed but not linked: 33
- Akhtar, Z., Dasgupta, D., and Banerjee, B. (2019, January 15–17). Face Authenticity: An overview of face manipulation generation, detection and recognition. Proceedings of the International Conference on Communication and Information Processing (ICCIP), Chongqing, China. · DOI
- Vezzetti, 2016, Application of geometry to rgb images for facial landmark localisation-a preliminary approach, Int. J. Biom., 8, 216
- Zhang, 2014, Multiview facial landmark localization in RGB-D images via hierarchical regression with binary patterns, IEEE Trans. Circuits Syst. Video Technol., 24, 1475, 10.1109/TCSVT.2014.2308639 · DOI
- Viola, P., and Jones, M. (2001, January 8–14). Rapid object detection using a boosted cascade of simple features. Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, (CVPR 2001), Kauai, HI, USA.
- King, 2009, Dlib-ml: A machine learning toolkit, J. Mach. Learn. Res., 10, 1755
- Bazarevsky, V., Kartynnik, Y., Vakunov, A., Raveendran, K., and Grundmann, M. (2019). Blazeface: Sub-millisecond neural face detection on mobile gpus. arXiv.
- Nguyen, T.T., Nguyen, C.M., Nguyen, D.T., Nguyen, D.T., and Nahavandi, S. (2019). Deep learning for deepfakes creation and detection. arXiv.
- Hui, K., Wang, J., He, H., and Ip, W.H. (2021). A multilevel single stage network for face detection. Wirel. Commun. Mob. Comput., 2021. · DOI
- Garg, D., Goel, P., Pandya, S., Ganatra, A., and Kotecha, K. (December, January 30). A deep learning approach for face detection using YOLO. Proceedings of the 2018 IEEE Punecon, Pune, India. · DOI
- Qi, D., Tan, W., Yao, Q., and Liu, J. (2021). YOLO5Face: Why reinventing a face detector. arXiv.
- Aralikatti, A., Appalla, J., Kushal, S., Naveen, G.S., Lokesh, S., and Jayasri, B.S. (2020, January 13–14). Real-time object detection and face recognition system to assist the visually impaired. Proceedings of the First International Conference on Advances in Physical Sciences and Materials, Coimbatore, India. · DOI
- Dave, 2021, An amalgamation of YOLOv4 and XGBoost for next-gen smart traffic management system, PeerJ Comput. Sci., 7, e586, 10.7717/peerj-cs.586 · DOI
- Kumar, R., Arora, R., Bansal, V., Sahayasheela, V.J., Buckchash, H., Imran, J., Narayanan, N., Pandian, G.N., and Raman, B. (2020). Accurate prediction of COVID-19 using chest X-Ray images through deep feature learning model with SMOTE and machine learning classifiers. MedRxiv. · DOI · Paper page
- Li, Y., and Lyu, S. (2018). Exposing deepfake videos by detecting face warping artifacts. arXiv.
- Güera, D., and Delp, E.J. (2018, January 27–30). Deepfake video detection using recurrent neural networks. Proceedings of the 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), Auckland, New Zealand. · DOI · Paper page
- Wang, Z., She, Q., and Ward, T.E. (2019). Generative adversarial networks in computer vision: A survey and taxonomy. arXiv.
- Korshunov, P., and Marcel, S. (2018). Deepfakes: A new threat to face recognition? assessment and detection. arXiv.
- Dufour, 2019, Contributing data to deepfake detection research, Google AI Blog, 1, 3
- Dolhansky, B., Howes, R., Pflaum, B., Baram, N., and Ferrer, C.C. (2020). The deepfake detection challenge (dfdc) dataset. arXiv.
- Li, X., Yu, K., Ji, S., Wang, Y., Wu, C., and Xue, H. (2020, January 20–24). Fighting against deepfake: Patch & pair convolutional neural networks (ppcnn). Proceedings of the Companion Proceedings of the Web Conference 2020, Taipei, Taiwan. · DOI · Paper page
- Charitidis, P., Kordopatis-Zilos, G., Papadopoulos, S., and Kompatsiaris, I. (2020). A face preprocessing approach for improved deepfake detection. arXiv.
- Wodajo, D., and Atnafu, S. (2021). Deepfake video detection using convolutional vision transformer. arXiv.
- Singh, 2020, DeepFake video detection: A time-distributed approach, SN Comput. Sci., 1, 1, 10.1007/s42979-020-00225-9 · DOI
- De Lima, O., Franklin, S., Basu, S., Karwoski, B., and George, A. (2020). Deepfake detection using spatiotemporal convolutional networks. arXiv.
- Mehra, A. (2020). Deepfake Detection Using Capsule Networks with Long Short-Term Memory Networks. [Master’s Thesis, University of Twenty]. · DOI
- Chen, W., Huang, H., Peng, S., Zhou, C., and Zhang, C. (2020). YOLO-face: A Real-Time Face Detector. Vis. Comput., 1–9. Available online: https://link.springer.com/article/10.1007/s00371-020-01831-7. · DOI · Paper page
- Szegedy, C., Ioffe, S., Banjouke, V., and Alemi, A. (2017, January 4–9). Inception-v4, inception-resnet and the impact of residual connections on learning. Proceedings of the AAAI Conference on Artificial Intelligence, San Francisco, CA, USA. · DOI · Paper page
- Srivastava, 2014, Dropout: A simple way to prevent neural networks from overfitting, J. Mach. Learn. Res., 15, 1929
- Zhang, L., and Zhan, C. (2017, January 17–20). Machine learning in rock facies classification: An application of XGBoost. Proceedings of the International Geophysical Conference, Qingdao, China. · DOI
- Fawcett, 2006, An introduction to ROC analysis, Pattern Recognit. Lett., 27, 861, 10.1016/j.patrec.2005.10.010 · DOI · Paper page
- Dozat, T. (2020, August 14). Incorporating Nesterov Momentum into Adam; 2016. Available online: https://openreview.net/forum?id=OM0jvwB8jIp57ZJjtNEZ.
- Xing, Y., Lv, C., and Cao, D. (2020). Advanced Driver Intention Inference: Theory and Design, Elsevier. · DOI
- Kaati, L., Omer, E., Prucha, N., and Shrestha, A. (2015, January 14–17). Detecting multipliers of jihadism on twitter. Proceedings of the 2015 IEEE International Conference on Data Mining Workshop (ICDMW), Washington, DC, USA. · DOI
Cited by
A Novel Blockchain-Based Deepfake Detection Method Using Federated and Deep Learning Models
Heidari, A., Navimipour, N. J., Dag, H., Talebi, S., & Unal, M. (2024) · Cognitive Computation · doi:10.1007/s12559-024-10255-7 · Cited by 164
Deepfake detection using convolutional vision transformers and convolutional neural networks
Soudy, A. H., Sayed, O., Tag-Elser, H., Ragab, R., Mohsen, S., Mostafa, T., Abohany, A. A., & Slim, S. O. (2024) · Neural Computing and Applications · doi:10.1007/s00521-024-10181-7 · Cited by 129
“It is now possible to easily create hyper-realistic digital pictures using a few tools and straightforward how-to guides that are easily available online [9].”
Enhanced deepfake detection with densenet and cross-vit
Siddiqui, F., Yang, J., Xiao, S., & Fahad, M. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.126150 · Cited by 62
Singh, S., & Dhumane, A. (2025) · MethodsX · doi:10.1016/j.mex.2025.103632 · Cited by 24
Xu, S., Yang, X., & Xie, Z. (2025) · Peer-to-Peer Networking and Applications · doi:10.1007/s12083-025-02048-y · Cited by 2
Metadata
- Authors
- Ismail, A., Elpeltagy, M., S. Zaki, M., & Eldahshan, K.
- Venue
- Sensors
- Publisher
- MDPI AG
- Published
- 2021-08-10
- Volume
- 21
- Issue
- 16
- Pages
- 5413
- ISSN
- 1424-8220 (ISSN-L)
- OpenAlex
- W3189760401
- Cited by
- 135
- References
- 26
- FWCI
- 7.05
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.