Countering Spoof: Towards Detecting Deepfake with Multidimensional Biological Signals
Open accessThe DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index.
Abstract
The deepfake technology is conveniently abused with the low technology threshold, which may bring the huge social security risks. As GAN-based synthesis technology is becoming stronger, various methods are difficult to classify the fake content effectively. However, although the fake content generated by GANs can deceive the human eyes, it ignores the biological signals hidden in the face video. In this paper, we proposed a novel video forensics method with multidimensional biological signals, which extracting the difference of the biological signal between real and fake videos from three dimensions. The experimental results show that our method achieves 98% accuracy on the main public dataset. Compared with other technologies, the proposed method only extracts fake video information and is not limited to a specific generation method, so it is not affected by synthetic methods and has good adaptability.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Besides, methods based on hard-to-find biological signals, such as heart rate, would be far less accurate due to video compression and other processing [39].”
From Multisemantic Path Neural Network for Deepfake Detection (2022) · cites this paper as [39]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Rapid object detection using a boosted cascade of simple features
Viola, P., & Jones, M. (2005) · Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001 · doi:10.1109/cvpr.2001.990517 · Cited by 18,300
FaceForensics++: Learning to Detect Manipulated Facial Images
Rossler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., & Niessner, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00009 · Cited by 3,314
MesoNet: a Compact Facial Video Forgery Detection Network
Afchar, D., Nozick, V., Yamagishi, J., & Echizen, I. (2018) · 2018 IEEE International Workshop on Information Forensics and Security (WIFS) · doi:10.1109/wifs.2018.8630761 · Cited by 1,872
Deepfake Video Detection Using Recurrent Neural Networks
Guera, D., & Delp, E. J. (2018) · 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS) · doi:10.1109/avss.2018.8639163 · Cited by 1,285
Face X-Ray for More General Face Forgery Detection
Li, L., Bao, J., Zhang, T., Yang, H., Chen, D., Wen, F., & Guo, B. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00505 · Cited by 1,211
Exposing Deep Fakes Using Inconsistent Head Poses
Yang, X., Li, Y., & Lyu, S. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8683164 · Cited by 1,196
In Ictu Oculi: Exposing AI Created Fake Videos by Detecting Eye Blinking
Li, Y., Chang, M.-C., & Lyu, S. (2018) · 2018 IEEE International Workshop on Information Forensics and Security (WIFS) · doi:10.1109/wifs.2018.8630787 · Cited by 1,089
Multi-attentional Deepfake Detection
Zhao, H., Wei, T., Zhou, W., Zhang, W., Chen, D., & Yu, N. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00222 · Cited by 1,019
Capsule-forensics: Using Capsule Networks to Detect Forged Images and Videos
Nguyen, H. H., Yamagishi, J., & Echizen, I. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8682602 · Cited by 951
A Deep Learning Approach to Universal Image Manipulation Detection Using a New Convolutional Layer
Bayar, B., & Stamm, M. C. (2016) · Proceedings of the 4th ACM Workshop on Information Hiding and Multimedia Security · doi:10.1145/2909827.2930786 · Cited by 838
Exposing DeepFake Videos By Detecting Face Warping Artifacts
Li, Y., & Lyu, S. (2018) · arXiv · doi:10.48550/arxiv.1811.00656 · Cited by 785
Spatial-Phase Shallow Learning: Rethinking Face Forgery Detection in Frequency Domain
Liu, H., Li, X., Zhou, W., Chen, Y., He, Y., Xue, H., Zhang, W., & Yu, N. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00083 · Cited by 600
Detecting Pulse from Head Motions in Video
Balakrishnan, G., Durand, F., & Guttag, J. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.440 · Cited by 553
FakeCatcher: Detection of Synthetic Portrait Videos using Biological Signals
Ciftci, U. A., Demir, I., & Yin, L. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3009287 · Cited by 499
DeepFakes: a New Threat to Face Recognition? Assessment and Detection
Korshunov, P., & Marcel, S. (2018) · arXiv · doi:10.48550/arxiv.1812.08685 · Cited by 497 · cited 1×
“)e recent research study by Korshunov [1] shows that fake videos can easily deceive the face recognition system, and some serious security risks, such as fake news, have been raised by them.”
BackgroundRecurrent convolutional strategies for face manipulation detection in videos
Sabir, E., Cheng, J., Jaiswal, A., AbdAlmageed, W., Masi, I., & Natarajan, P. (2019) · arXiv · doi:10.48550/arxiv.1905.00582 · Cited by 443 · cited 1×
“[18] also proposed the CNN+RNN method, but they used face alignment and bidirectional recurrency.”
MethodRhythmNet: End-to-End Heart Rate Estimation From Face via Spatial-Temporal Representation
Niu, X., Shan, S., Han, H., & Chen, X. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2947204 · Cited by 420
(2019) · Cited by 415
Tulyakov, S., Alameda-Pineda, X., Ricci, E., Yin, L., Cohn, J. F., & Sebe, N. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.263 · Cited by 340
FaceForensics: A Large-scale Video Dataset for Forgery Detection in Human Faces
Rössler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., & Nießner, M. (2018) · arXiv · doi:10.48550/arxiv.1803.09179 · Cited by 339
Distinguishing computer graphics from natural images using convolution neural networks
Rahmouni, N., Nozick, V., Yamagishi, J., & Echizen, I. (2017) · 2017 IEEE Workshop on Information Forensics and Security (WIFS) · doi:10.1109/wifs.2017.8267647 · Cited by 318
FakeSpotter: A Simple yet Robust Baseline for Spotting AI-Synthesized Fake Faces
Wang, R., Juefei-Xu, F., Ma, L., Xie, X., Huang, Y., Wang, J., & Liu, Y. (2020) · Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/476 · Cited by 250
Remote heart rate measurement using low-cost RGB face video: a technical literature review
Rouast, P. V., Adam, M. T. P., Chiong, R., Cornforth, D., & Lux, E. (2018) · Frontiers of Computer Science · doi:10.1007/s11704-016-6243-6 · Cited by 187
A Serial Image Copy-Move Forgery Localization Scheme With Source/Target Distinguishment
Chen, B., Tan, W., Coatrieux, G., Zheng, Y., & Shi, Y.-Q. (2021) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2020.3026868 · Cited by 159 · cited 1×
“Recently, a bunch of high-efficient detectors with the new algorithms have been proposed to improve the performance of tampering detection and localization [8, 9].”
BackgroundBounded Kalman filter method for motion-robust, non-contact heart rate estimation
Prakash, S. K. A., & Tucker, C. S. (2018) · Biomedical Optics Express · doi:10.1364/boe.9.000873 · Cited by 64
Physiologically-based detection of computer generated faces in video
Conotter, V., Bodnari, E., Boato, G., & Farid, H. (2014) · 2014 IEEE International Conference on Image Processing (ICIP) · doi:10.1109/icip.2014.7025049 · Cited by 61
Capsule-Forensics: Using Capsule Networks to Detect Forged Images and VideosPreprint
Nguyen, H. H., Yamagishi, J., & Echizen, I. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.11215 · Cited by 49
Generative Adversarial Ensemble Learning for Face Forensics
Baek, J.-Y., Yoo, Y., & Bae, S. (2020) · IEEE Access · doi:10.1109/access.2020.2968612 · Cited by 40
FakeSpotter: A Simple yet Robust Baseline for Spotting AI-Synthesized Fake FacesPreprint
Wang, R., Juefei-Xu, F., Ma, L., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.06122 · Cited by 32
Video Face Manipulation Detection Through Ensemble of CNNs
Bonettini, N., Cannas, E. D., Mandelli, S., Bondi, L., Bestagini, P., & Tubaro, S. (2021) · 2020 25th International Conference on Pattern Recognition (ICPR) · doi:10.1109/icpr48806.2021.9412711 · Cited by 27
Spatial-Phase Shallow Learning: Rethinking Face Forgery Detection in Frequency DomainPreprint
Liu, H., Li, X., Zhou, W., Chen, Y., He, Y., Xue, H., Zhang, W., & Yu, N. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.01856 · Cited by 22
Image splicing localization using residual image and residual-based fully convolutional network
Chen, B., Qi, X., Zhou, Y., Yang, G., Zheng, Y., & Xiao, B. (2020) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2020.102967 · Cited by 19
Multi-attentional Deepfake DetectionPreprint
Zhao, H., Zhou, W., Chen, D., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.02406 · Cited by 15
Deep Face Forgery DetectionPreprint
Dogonadze, N., Obernosterer, J., & Hou, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.11804 · Cited by 8
Video Face Manipulation Detection Through Ensemble of CNNs
Bonettini, N., Cannas, E. D., Mandelli, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2004.07676
FakeCatcher: Detection of Synthetic Portrait Videos using Biological Signals
Ciftci, U. A., & Demir, I. (2019) · arXiv · doi:10.48550/arxiv.1901.02212
Cited by
A Review of Image Processing Techniques for Deepfakes
Shahzad, H. F., Rustam, F., Flores, E. S., Luís Vidal Mazón, J., de la Torre Diez, I., & Ashraf, I. (2022) · Sensors · doi:10.3390/s22124556 · Cited by 79
“Intuitively, computer experts have designed algorithms that can measure biological signals using features from the video data such as changes in color, motion, subtle head movements, etc. [82].”
BackgroundA systematic literature review on the effectiveness of deepfake detection techniques
Stroebel, L., Llewellyn, M., Hartley, T., Ip, T. S., & Ahmed, M. (2023) · Journal of Cyber Security Technology · doi:10.1080/23742917.2023.2192888 · Cited by 66
Deepfakes: evolution and trends
Gil, R., Virgili‐Gomà, J., López, J. M., & García, R. (2023) · Soft Computing · doi:10.1007/s00500-023-08605-y · Cited by 50
“…Lu et al. (2021), Su et al. (2021), Luo et al. (2021), Biswas et al. (2021), Jiang et al. (2021), Korshunov and Marcel (2021), Chen and Tan (2021), Jin et al. (2021), Khalil et al. (2021), Gu et al. (2021), Yang et al. (2021c), Chintha et al. (2020a), Younus and Hasan (2020b), Korshunov and…”
Multimodal Neurosymbolic Approach for Explainable Deepfake Detection
Haq, I. U., Malik, K. M., & Muhammad, K. (2023) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3624748 · Cited by 37
“For instance, Jin et al. [ 18 ] used a multidimensional feature space from the RGB, Y U V, and chromatic dimensions to extract heart rate from a visual modality.”
Video authentication detection using deep learning: a systematic literature review
Alrawahneh, A. A.-M., Abdullah, S. N. H. S., Abdullah, S. N. H. S., Kamarudin, N. H., & Taylor, S. K. (2024) · Applied Intelligence · doi:10.1007/s10489-024-05997-8 · Cited by 31
“C) of 98.07% for Xcep‑ tionNet and 99.83% for MesoNet in detecting deepfakes However, [45] used a DNN to analyze biological sig‑ nals for deepfake detection.”
Influential
Metadata
- Authors
- Jin, X., Ye, D., & Chen, C.
- Affiliations
- Wuhan University (CN)
- Corresponding author
- Ye, Dengpan
- Venue
- Security and Communication Networks
- Publisher
- Wiley
- Published
- 2021-04-22
- Volume
- 2021
- Pages
- 1-8
- ISSN
- 1939-0114 (ISSN-L)
- 1939-0122
- OpenAlex
- W3155922523
- Cited by
- 17
- References
- 32
- Citations by year
- 2025: 3 · 2024: 3 · 2023: 6 · 2022: 4 · 2021: 1
- FWCI
- 1.26
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Security and Communication Networks (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Funders
- National Key Research Development Program of China (2019QY(Y)0206, 62072343, U1736211)
- National Natural Science Foundation of China (2019QY(Y)0206, 62072343, U1736211)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Physical Sciences › Computer Science › Signal Processing › Biometric Identification and Security
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Keywords
- deepfake detection, video forensics, biological signals, facial video, anti-spoofing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.