DeepRhythm: Exposing DeepFakes with Attentional Visual Heartbeat Rhythms
The DOI matches its Crossref record; 57 of the 57 references it cites were found in a registry or an index.
Abstract
As the GAN-based face image and video generation techniques, widely known as DeepFakes, have become more and more matured and realistic, there comes a pressing and urgent demand for effective DeepFakes detectors. Motivated by the fact that remote visual photoplethysmography (PPG) is made possible by monitoring the minuscule periodic changes of skin color due to blood pumping through the face, we conjecture that normal heartbeat rhythms found in the real face videos will be disrupted or even entirely broken in a DeepFake video, making it a potentially powerful indicator for DeepFake detection. In this work, we propose DeepRhythm, a DeepFake detection technique that exposes DeepFakes by monitoring the heartbeat rhythms. DeepRhythm utilizes dual-spatial-temporal attention to adapt to dynamically changing face and fake types. Extensive experiments on FaceForensics++ and DFDC-preview datasets have confirmed our conjecture and demonstrated not only the effectiveness, but also the generalization capability of DeepRhythm over different datasets by various DeepFakes generation techniques and multifarious challenging degradations.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Xception: Deep Learning with Depthwise Separable Convolutions
Chollet, F. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.195 · Cited by 20,380 · cited 5×
“pFakes by monitoring the heartbeat rhythms. More specifically, DeepRhythm outperforms four state-of-the-art DeepFake detection methods including Bayer’s method [3], Inception ResNet V1 [59], Xception [9], and MesoNet [1] in the FaceForensics++ benchmark [57] and exhibits high robustness to JPEG compression, noise, and…”
BackgroundMethodInfluentialA Style-Based Generator Architecture for Generative Adversarial Networks
Karras, T., Laine, S., & Aila, T. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00453 · Cited by 11,714 · cited 1×
“ion, Thies et al. [60] further use neural textures and defer neural render to generate the forgeries. Besides the above face-wrap-based methods, recent DeepFake approaches, i.e., PGGAN [36], StyleGAN [37], and StyleGAN2 [38], employ the generative adversarial network (GAN) [22] for the nearrealistic face synthesis. Mo…”
MethodAnalyzing and Improving the Image Quality of StyleGAN
Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J., & Aila, T. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00813 · Cited by 6,332
Analyzing and Improving the Image Quality of StyleGAN
Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J., & Aila, T. (2019) · arXiv · doi:10.48550/arxiv.1912.04958 · Cited by 6,332 · cited 1×
“ther uses neural textures and deferred neural render to generate the forgeries. Besides above face-wrap-based methods, more and more DeepFake approaches, i.e., PGGAN [32], StyleGAN [33] and StyleGAN2 [34], employ the generative adversarial network (GAN) [21] to realize the near-realistic face synthesis. Moreover, some…”
Method2009 IEEE Conference on Computer Vision and Pattern Recognition
(2009) · doi:10.1109/cvpr14067.2009 · Cited by 5,550
Progressive Growing of GANs for Improved Quality, Stability, and Variation
Karras, T., Aila, T., Laine, S., & Lehtinen, J. (2017) · arXiv · doi:10.48550/arxiv.1710.10196 · Cited by 4,727 · cited 1×
“model. Then, Thies et al. [53] further uses neural textures and deferred neural render to generate the forgeries. Besides above face-wrap-based methods, more and more DeepFake approaches, i.e., PGGAN [32], StyleGAN [33] and StyleGAN2 [34], employ the generative adversarial network (GAN) [21] to realize the near-realis…”
MethodStarGAN: Unified Generative Adversarial Networks for Multi-domain Image-to-Image Translation
Choi, Y., Choi, M., Kim, M., Ha, J.-W., Kim, S., & Choo, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00916 · Cited by 4,163
FaceForensics++: Learning to Detect Manipulated Facial Images
Rossler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., & Niessner, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00009 · Cited by 3,314 · cited 4×
“We select FaceForensics++ [57] as our training and testing datasets, and use DeepFake Detection Challenge preview (DFDCpreview) [15] as an additional testing dataset to evaluate ourmethod’s cross-dataset generalization capability.”
MethodInfluentialRich Models for Steganalysis of Digital Images
Fridrich, J., & Kodovsky, J. (2012) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2012.2190402 · Cited by 2,144 · cited 1×
“etecting forgeries. Panet al. [46] detect image splicing by estimating local noise variances. Buchana et al. [5] apply shallow methods such as correlation filters to spot the forgery. Fridrich et al. [16] assemble many diverse submodels to build steganography detectors. And Goljan et al. [19] use 3D co-occurrences to…”
BackgroundMesoNet: a Compact Facial Video Forgery Detection Network
Afchar, D., Nozick, V., Yamagishi, J., & Echizen, I. (2018) · 2018 IEEE International Workshop on Information Forensics and Security (WIFS) · doi:10.1109/wifs.2018.8630761 · Cited by 1,872
Non-contact, automated cardiac pulse measurements using video imaging and blind source separation
Poh, M.-Z., McDuff, D. J., & Picard, R. W. (2010) · Optics Express · doi:10.1364/oe.18.010762 · Cited by 1,643 · cited 1×
“sk with DeepFake detection and aims to determine whether an input video contains a live face or a face photo or mask. As the remote heart rhythm (HR) measuring techniques achieving great developments [2, 6, 31, 35, 47, 48, 62], many researchers use rPPG to detect face spoofing attacks. For example, Li et al. [36] use…”
Background2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
(2016) · doi:10.1109/cvpr33180.2016 · Cited by 1,534
Advancements in Noncontact, Multiparameter Physiological Measurements Using a Webcam
Poh, M.-Z., McDuff, D. J., & Picard, R. W. (2011) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2010.2086456 · Cited by 1,486 · cited 1×
“sk with DeepFake detection and aims to determine whether an input video contains a live face or a face photo or mask. As the remote heart rhythm (HR) measuring techniques achieving great developments [2, 6, 31, 35, 47, 48, 62], many researchers use rPPG to detect face spoofing attacks. For example, Li et al. [36] use…”
BackgroundDeferred neural rendering: image synthesis using neural textures
Thies, J., Zollhöfer, M., & Nießner, M. (2019) · ACM Transactions on Graphics · doi:10.1145/3306346.3323035 · Cited by 1,356 · cited 1×
“nactment [61] and propose the Face2Face method [62] that tracks target and source facial expressions to build a face 3D model and re-renders source face on the target model. In addition, Thies et al. [60] further use neural textures and defer neural render to generate the forgeries. Besides the above face-wrap-based m…”
MethodCapsule-forensics: Using Capsule Networks to Detect Forged Images and Videos
Nguyen, H. H., Yamagishi, J., & Echizen, I. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8682602 · Cited by 951
Learning Dynamic Siamese Network for Visual Object Tracking
Guo, Q., Feng, W., Zhou, C., Huang, R., Wan, L., & Wang, S. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.196 · Cited by 869
Self-citationA Deep Learning Approach to Universal Image Manipulation Detection Using a New Convolutional Layer
Bayar, B., & Stamm, M. C. (2016) · Proceedings of the 4th ACM Workshop on Information Hiding and Multimedia Security · doi:10.1145/2909827.2930786 · Cited by 838 · cited 4×
“For Bayer’s method [3], we do not find publicly available code, so we re-implement it via Keras.”
BackgroundMethodInfluentialAttGAN: Facial attribute editing by only changing what you want
He, Z., Zuo, W., Kan, M., Shan, S., & Chen, X. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2916751 · Cited by 833 · cited 1×
“e adversarial network (GAN) [22] for the nearrealistic face synthesis. Moreover, some methods can even alter face attributes, e.g., changing or removing the color of the hair, adding glasses or scars [8, 21, 30, 63], and modifying persons’ facial expression [41]. Overall, GANs have shown great potential in this area a…”
MethodOn the Detection of Digital Face Manipulation
Dang, H., Liu, F., Stehouwer, J., Liu, X., & Jain, A. K. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00582 · Cited by 764
Learning Deep Models for Face Anti-Spoofing: Binary or Auxiliary Supervision
Liu, Y., Jourabloo, A., & Liu, X. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00048 · Cited by 712
Self-citationTwo-Stream Neural Networks for Tampered Face Detection
Zhou, P., Han, X., Morariu, V. I., & Davis, L. S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.229 · Cited by 700 · cited 1×
“[71] combine a DNN-based face classification stream with a steganalysis-based triplet stream, yielding good performance.”
MethodVideo Rewrite: driving visual speech with audio
Bregler, C., Covell, M., & Slaney, M. (1997) · Proceedings of the 24th annual conference on Computer graphics and interactive techniques - SIGGRAPH '97 · doi:10.1145/258734.258880 · Cited by 669
Recurrent convolutional strategies for face manipulation detection in videos
Sabir, E., Cheng, J., Jaiswal, A., AbdAlmageed, W., Masi, I., & Natarajan, P. (2019) · arXiv · doi:10.48550/arxiv.1905.00582 · Cited by 443 · cited 1×
“ned with the CNN to extract image and temporal features to distinguish real and fake videos. For example, Güera et al. [24] use CNN features to train a RNN to classify videos. Similarly, Sabir et al. [51] use a sequence of spatio-temporal faces, as RNN’s input to classify the videos. Further more, Dong et al. [12] uti…”
MethodRhythmNet: End-to-End Heart Rate Estimation From Face via Spatial-Temporal Representation
Niu, X., Shan, S., Han, H., & Chen, X. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2947204 · Cited by 420 · cited 2×
“s for DeepFake detection is to use existing HR representations that are designed for the remote HR estimation. For example, we can use the spatial-temporal representation (STR) proposed by Niu et al. [48] for representing HR signals and feed them to a classifier for DeepFake detection. However, it is hard to achieve h…”
MethodInfluentialThe Deepfake Detection Challenge (DFDC) Preview Dataset
Dolhansky, B., Howes, R., Pflaum, B., Baram, N., & Ferrer, C. C. (2019) · arXiv · doi:10.48550/arxiv.1910.08854 · Cited by 413 · cited 1×
“accuracy & robustness. 4.1 Dataset and Experiment Setting Dataset. We choose FaceForensics++ [50] as our training and testing datasets, and use DeepFake Detection Challenge preview (DFDC-preview) [15] as an additional testing dataset to evaluate our method’s cross-dataset generalization ability. FaceForensics++ da…”
MethodInfluentialReal-time expression transfer for facial reenactment
Thies, J., Zollhöfer, M., Nießner, M., Valgaerts, L., Stamminger, M., & Theobalt, C. (2015) · ACM Transactions on Graphics · doi:10.1145/2816795.2818056 · Cited by 369 · cited 1×
“Thies et al . develop a real-time expression transfer for facial reenactment [54] and propose the Face2Face method [55] that tracks target and source facial expressions to build a face 3D model and re-renders source face on the target model.”
MethodCozzolino, D., Poggi, G., & Verdoliva, L. (2017) · Proceedings of the 5th ACM Workshop on Information Hiding and Multimedia Security · doi:10.1145/3082031.3083247 · Cited by 349
Distinguishing computer graphics from natural images using convolution neural networks
Rahmouni, N., Nozick, V., Yamagishi, J., & Echizen, I. (2017) · 2017 IEEE Workshop on Information Forensics and Security (WIFS) · doi:10.1109/wifs.2017.8267647 · Cited by 318 · cited 1×
“also employ SOTA deep neural networks (DNNs) to detect forgery images. Cozzolino etal. [10] use residual-based local features and achieve significant performance. Bayar et al. [3] and Rahmouni et al. [56] propose novel DNNs to detect manipulated images. Zhou et al. [71] combine a DNN-based face classification stream w…”
Deep Feature Interpolation for Image Content Changes
Upchurch, P., Gardner, J., Pleiss, G., Pless, R., Snavely, N., Bala, K., & Weinberger, K. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.645 · Cited by 296 · cited 1×
“e adversarial network (GAN) [22] for the nearrealistic face synthesis. Moreover, some methods can even alter face attributes, e.g., changing or removing the color of the hair, adding glasses or scars [8, 21, 30, 63], and modifying persons’ facial expression [41]. Overall, GANs have shown great potential in this area a…”
MethodFakeSpotter: A Simple yet Robust Baseline for Spotting AI-Synthesized Fake Faces
Wang, R., Juefei-Xu, F., Ma, L., Xie, X., Huang, Y., Wang, J., & Liu, Y. (2020) · Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/476 · Cited by 250 · cited 1×
“, color cues [39], monitoring neuron behaviors [58], and employing classifiers, e.”
MethodSelf-citationDetecting GAN-Generated Imagery Using Saturation Cues
McCloskey, S., & Albright, M. (2019) · 2019 IEEE International Conference on Image Processing (ICIP) · doi:10.1109/icip.2019.8803661 · Cited by 220
Humphreys, K., Ward, T., & Markham, C. (2007) · Review of Scientific Instruments · doi:10.1063/1.2724789 · Cited by 217
Generalized face anti-spoofing by detecting pulse from face videos
Li, X., Komulainen, J., Zhao, G., Yuen, P.-C., & Pietikainen, M. (2016) · 2016 23rd International Conference on Pattern Recognition (ICPR) · doi:10.1109/icpr.2016.7900300 · Cited by 170 · cited 1×
“r mask. As the remote heart rhythm (HR) measuring techniques achieving great developments [2, 6, 31, 35, 47, 48, 62], many researchers use rPPG to detect face spoofing attacks. For example, Li et al. [36] use the difference of pulse between real faces and printed faces to defend spoofing attacks. Nowaraet al. [42] com…”
BackgroundLearning-Based Video Motion Magnification
Oh, T.-H., Jaroensri, R., Kim, C., Elgharib, M., Durand, F., Freeman, W. T., & Matusik, W. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01225-0_39 · Cited by 167
DeepCT: Tomographic Combinatorial Testing for Deep Learning Systems
Ma, L., Juefei-Xu, F., Xue, M., Li, B., Li, L., Liu, Y., & Zhao, J. (2019) · 2019 IEEE 26th International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner.2019.8668044 · Cited by 162 · cited 1×
“, color cues [46], monitoring neuron behaviors [43–45, 66, 68], and employing classifiers, e.”
MethodSelf-citationDynamic Saliency-Aware Regularization for Correlation Filter-Based Object Tracking
Feng, W., Han, R., Guo, Q., Zhu, J., & Wang, S. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2895411 · Cited by 143
Self-citationVDub: Modifying Face Video of Actors for Plausible Visual Alignment to a Dubbed Audio Track
Garrido, P., Valgaerts, L., Sarmadi, H., Steiner, I., Varanasi, K., Pérez, P., & Theobalt, C. (2015) · Computer graphics forum (Print) · doi:10.1111/cgf.12552 · Cited by 137 · cited 1×
“Later, they propose a new photo-realistically replacement based on high-quality monocular capture [18].”
BackgroundDeepSonar: Towards Effective and Robust Detection of AI-Synthesized Fake Voices
Wang, R., Juefei-Xu, F., Huang, Y., Guo, Q., Xie, X., Ma, L., & Liu, Y. (2020) · Proceedings of the 28th ACM International Conference on Multimedia · doi:10.1145/3394171.3413716 · Cited by 131
Self-citationFacial Soft Biometrics for Recognition in the Wild: Recent Works, Annotation, and COTS Evaluation
González-Sosa, E., Fiérrez, J., Vera-Rodríguez, R., & Alonso‐Fernandez, F. (2018) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2018.2807791 · Cited by 129 · cited 1×
“e adversarial network (GAN) [22] for the nearrealistic face synthesis. Moreover, some methods can even alter face attributes, e.g., changing or removing the color of the hair, adding glasses or scars [8, 21, 30, 63], and modifying persons’ facial expression [41]. Overall, GANs have shown great potential in this area a…”
MethodDale, K., Sunkavalli, K., Johnson, M. K., Vlasic, D., Matusik, W., & Pfister, H. (2011) · ACM Transactions on Graphics · doi:10.1145/2070781.2024164 · Cited by 125
Use of a Capsule Network to Detect Fake Images and Videos
Nguyen, H. H., Yamagishi, J., & Echizen, I. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.12467 · Cited by 121 · cited 1×
“More recently, researchers are trying to apply much more complex and advanced DNNs on video forgery detection, such as Inception-ResNet [59], MesoNet [1], capsule networks [47], and Xception [9].”
BackgroundExposing image splicing with inconsistent local noise variances
Pan, X., Zhang, X., & Lyu, S. (2012) · 2012 IEEE International Conference on Computational Photography (ICCP) · doi:10.1109/iccphot.2012.6215223 · Cited by 120 · cited 1×
“e-art digital forensics. To alleviate this challenge, researchers are exploring effective solutions to identify fake videos. Early attempts focus on detecting forgeries via hand-crafted features,e.g.,[5,17,20,53].However,thesehand-craftedmodelscan be strenuous due to the realistic faces generated by SOTA DeepFake meth…”
MethodHeart rate monitoring via remote photoplethysmography with motion artifacts reduction
Cennini, G., Arguel, J., Akşit, K., & van Leest, A. J. (2010) · Optics Express · doi:10.1364/oe.18.004867 · Cited by 111 · cited 1×
“sk with DeepFake detection and aims to determine whether an input video contains a live face or a face photo or mask. As the remote heart rhythm (HR) measuring techniques achieving great developments [2, 6, 31, 35, 47, 48, 62], many researchers use rPPG to detect face spoofing attacks. For example, Li et al. [36] use…”
BackgroundFakeLocator: Robust Localization of GAN-Based Face Manipulations
Huang, Y., Juefei-Xu, F., Guo, Q., Liu, Y., & Pu, G. (2022) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2022.3141262 · Cited by 102 · cited 1×
“In future work, one of the interesting directions worth pursuing is to study the combined effort of our proposed DeepRhythm with other DeepFake detectors [29, 30, 57, 59].”
BackgroundSelf-citationPPGSecure: Biometric Presentation Attack Detection Using Photopletysmograms
Nowara, E. M., Sabharwal, A., & Veeraraghavan, A. (2017) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2017.16 · Cited by 95 · cited 1×
“[49] compare PPGs of face and background to decide whether the face is live or not.”
BackgroundFakePolisher: Making DeepFakes More Detection-Evasive by Shallow Reconstruction
Huang, Y., Juefei-Xu, F., Wang, R., Guo, Q., Ma, L., Xie, X., Li, J., Miao, W., Liu, Y., & Pu, G. (2020) · Proceedings of the 28th ACM International Conference on Multimedia · doi:10.1145/3394171.3413732 · Cited by 71 · cited 1×
“One interesting future direction is to study the combined effort of DeepRhythm with other DeepFake detectors [33, 34, 65, 66].”
BackgroundSelf-citationVideo Rewrite: Driving Visual Speech with Audio
Bregler, C., Covell, M., & Slaney, M. (2023) · Seminal Graphics Papers: Pushing the Boundaries, Volume 2 · doi:10.1145/3596711.3596787 · Cited by 67 · cited 1×
“ed widespread attention and been used in generating pornographic videos, fake news, and hoaxes, etc. Some early studies use face-wrap-based methods to generate fake videos. For example, Bregler etal. [4] tracks the movement of the speaker’s mouth and morph the input video. Dale et al. [11] present a video face replace…”
BackgroundTime Analysis of Pulse-Based Face Anti-Spoofing in Visible and NIR
Hernandez‐Ortega, J., Fiérrez, J., Morales, A., & Pedro Tome, A. (2018) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2018.00096 · Cited by 60 · cited 1×
“araetal. [49] compare PPGs of face and background to decide whether the face is live or not. Heusch et al. [32] use the long-term statistical spectral on the pulse signals and Hernandez-Ortega et al. [31] employ the near infrared against realistic artifacts. Moreover, combining with DNNs, Liu et al. [42] extract spati…”
BackgroundTwo-Stream Neural Networks for Tampered Face DetectionPreprint
Zhou, P., Han, X., Morariu, V. I., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.11276 · Cited by 57 · cited 1×
“[71] combine a DNN-based face classification stream with a steganalysis-based triplet stream, yielding good performance.”
MethodDetecting GAN-generated Imagery using Color Cues
McCloskey, S., & Albright, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.08247 · Cited by 53 · cited 1×
“erated by SOTA DeepFake methods (e.g., FaceApp, Reflect, and ZAO). Later, researchers regard the DeepFake detection as a classification problem by extracting discriminative features, e.g., color cues [46], monitoring neuron behaviors [43–45, 66, 68], and employing classifiers,e.g., support vector machine (SVM), to tel…”
MethodSelective Spatial Regularization by Reinforcement Learned Decision Making for Object Tracking
Guo, Q., Han, R., Feng, W., Chen, Z., & Wan, L. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2955292 · Cited by 45
Self-citationStructure-Regularized Compressive Tracking With Online Data-Driven Sampling
Guo, Q., Feng, W., Zhou, C., Pun, C., & Wu, B. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2745205 · Cited by 44
Self-citationOn the Detection of Digital Face ManipulationPreprint
Dang, H., Liu, F., Stehouwer, J., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.01717 · Cited by 41 · cited 1×
“[12] utilize an attention mechanism to generate and improve the feature maps, which highlight the informative regions.”
MethodPulse-based Features for Face Presentation Attack Detection
Heusch, G., & Marcel, S. (2018) · IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS) · doi:10.1109/btas.2018.8698579 · Cited by 29 · cited 1×
“[32] use the long-term statistical spectral on the pulse signals and Hernandez-Ortega et al.”
MethodSimultaneous forgery identification and localization in paintings using advanced correlation filters
Buchana, P., Cazan, I., Diaz-Granados, M., Juefei-Xu, F., & Savvides, M. (2016) · 2016 IEEE International Conference on Image Processing (ICIP) · doi:10.1109/icip.2016.7532336 · Cited by 21 · cited 1×
“loring effective solutions to identify fake videos. Early attempts in this field focus on detecting forgeries. Panet al. [46] detect image splicing by estimating local noise variances. Buchana et al. [5] apply shallow methods such as correlation filters to spot the forgery. Fridrich et al. [16] assemble many diverse s…”
BackgroundSelf-citationSelective object and context tracking
Zhou, C., Guo, Q., Wan, L., & Feng, W. (2017) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2017.7952496 · Cited by 13
Self-citationIEEE International Workshop on Information Forensics and Security (WIFS)
(2009) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2009.2015031 · Cited by 8
Research works printed but not linked: 19
- Yupeng Cheng , Qing Guo , Felix Juefei-Xu , Xiaofei Xie , Shang-Wei Lin , Weisi Lin , Wei Feng , and Yang Liu . 2020 . Pasadena: Perceptually Aware and Stealthy Adversarial Denoise Attack. arXiv preprint (2020). Yupeng Cheng, Qing Guo, Felix Juefei-Xu, Xiaofei Xie, Shang-Wei Lin, Weisi Lin, Wei Feng, and Yang Liu. 2020. Pasadena: Perceptually Aware and Stealthy Adversarial Denoise Attack. arXiv preprint (2020).
- Pablo Garrido , Levi Valgaerts , Ole Rehmsen , Thorsten Thormählen , Patrick Pérez , and Christian Theobalt . 2014 . Automatic Face Reenactment. 2014 IEEE Conference on Computer Vision and Pattern Recognition (2014), 4217--4224. Pablo Garrido, Levi Valgaerts, Ole Rehmsen, Thorsten Thormählen, Patrick Pérez, and Christian Theobalt. 2014. Automatic Face Reenactment. 2014 IEEE Conference on Computer Vision and Pattern Recognition (2014), 4217--4224.
- P. Garrido Levi Valgaerts Hamid Sarmadi I. Steiner Kiran Varanasi P. Pérez and C. Theobalt. 2015. VDub: Modifying Face Video of Actors for Plausible Visual Alignment to a Dubbed Audio Track. Computer Graphics Forum 34 (05 2015). https://doi.org/10.1111/cgf.12552 · DOI
- Miroslav Goljan and Jessica Fridrich . 2015 . CFA-aware features for steganalysis of color images . Proceedings of SPIE - The International Society for Optical Engineering 9409 (03 2015). Miroslav Goljan and Jessica Fridrich. 2015. CFA-aware features for steganalysis of color images. Proceedings of SPIE - The International Society for Optical Engineering 9409 (03 2015).
- Ian J. Goodfellow Jean Pouget-Abadie Mehdi Mirza Bing Xu David WardeFarley Sherjil Ozair Aaron Courville and Yoshua Bengio. 2014. Generative Adversarial Networks. arXiv:stat.ML/1406.2661 Ian J. Goodfellow Jean Pouget-Abadie Mehdi Mirza Bing Xu David WardeFarley Sherjil Ozair Aaron Courville and Yoshua Bengio. 2014. Generative Adversarial Networks. arXiv:stat.ML/1406.2661
- Qing Guo , Wei Feng , Ce Zhou , Chi-Man Pun , and Bin Wu . 2017 . StructureRegularized Compressive Tracking With Online Data-Driven Sampling . IEEE TIP 26 , 12 (2017), 5692 -- 5705 . Qing Guo, Wei Feng, Ce Zhou, Chi-Man Pun, and Bin Wu. 2017. StructureRegularized Compressive Tracking With Online Data-Driven Sampling. IEEE TIP 26, 12 (2017), 5692--5705.
- Qing Guo , Felix Juefei-Xu , Xiaofei Xie , Lei Ma , Jian Wang , Bing Yu , Wei Feng , and Yang Liu . 2020. Watch out! Motion is Blurring the Vision of Your Deep Neural Networks. arXiv preprint arXiv:2002.03500 ( 2020 ). Qing Guo, Felix Juefei-Xu, Xiaofei Xie, Lei Ma, Jian Wang, Bing Yu, Wei Feng, and Yang Liu. 2020. Watch out! Motion is Blurring the Vision of Your Deep Neural Networks. arXiv preprint arXiv:2002.03500 (2020). · arXiv
- Qing Guo , Xiaofei Xie , Felix Juefei-Xu , Lei Ma , Zhongguo Li , Wanli Xue , Wei Feng , and Yang Liu . 2020 . SPARK: Spatial-aware Online Incremental Attack Against Visual Tracking. European Conference on Computer Vision (ECCV) (2020). Qing Guo, Xiaofei Xie, Felix Juefei-Xu, Lei Ma, Zhongguo Li, Wanli Xue, Wei Feng, and Yang Liu. 2020. SPARK: Spatial-aware Online Incremental Attack Against Visual Tracking. European Conference on Computer Vision (ECCV) (2020).
- D. Güera and E. J. Delp . 2018 . Deepfake Video Detection Using Recurrent Neural Networks. In 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS). 1--6. D. Güera and E. J. Delp. 2018. Deepfake Video Detection Using Recurrent Neural Networks. In 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS). 1--6.
- X. Li , J. Chen , G. Zhao , and M. Pietikäinen . 2014. Remote Heart Rate Measurement from Face Videos under Realistic Situations . In 2014 IEEE Conference on Computer Vision and Pattern Recognition. 4264--4271 . X. Li, J. Chen, G. Zhao, and M. Pietikäinen. 2014. Remote Heart Rate Measurement from Face Videos under Realistic Situations. In 2014 IEEE Conference on Computer Vision and Pattern Recognition. 4264--4271.
- Ming Liu , Yukang Ding , Min Xia , Xiao Liu , Errui Ding , Wangmeng Zuo , and Shilei Wen . 2019 . STGAN: A Unified Selective Transfer Network for Arbitrary Image Attribute Editing. arXiv:cs.CV/1904.09709 Ming Liu, Yukang Ding, Min Xia, Xiao Liu, Errui Ding, Wangmeng Zuo, and Shilei Wen. 2019. STGAN: A Unified Selective Transfer Network for Arbitrary Image Attribute Editing. arXiv:cs.CV/1904.09709
- Lei Ma , Felix Juefei-Xu , Jiyuan Sun , Chunyang Chen , Ting Su , Fuyuan Zhang , Minhui Xue , Bo Li , Li Li , Yang Liu , Jianjun Zhao , and Yadong Wang . 2018 . DeepGauge: Multi-Granularity Testing Criteria for Deep Learning Systems. In The 33rd IEEE/ACM International Conference on Automated Software Engineering (ASE). Lei Ma, Felix Juefei-Xu, Jiyuan Sun, Chunyang Chen, Ting Su, Fuyuan Zhang, Minhui Xue, Bo Li, Li Li, Yang Liu, Jianjun Zhao, and Yadong Wang. 2018. DeepGauge: Multi-Granularity Testing Criteria for Deep Learning Systems. In The 33rd IEEE/ACM International Conference on Automated Software Engineering (ASE).
- Lei Ma , Fuyuan Zhang , Jiyuan Sun , Minhui Xue , Bo Li , Felix Juefei-Xu , Chao Xie , Li Li , Yang Liu , Jianjun Zhao , and Yadong Wang . 2018 . DeepMutation: Mutation Testing of Deep Learning Systems. In The 29th IEEE International Symposium on Software Reliability Engineering (ISSRE). Lei Ma, Fuyuan Zhang, Jiyuan Sun, Minhui Xue, Bo Li, Felix Juefei-Xu, Chao Xie, Li Li, Yang Liu, Jianjun Zhao, and Yadong Wang. 2018. DeepMutation: Mutation Testing of Deep Learning Systems. In The 29th IEEE International Symposium on Software Reliability Engineering (ISSRE).
- Christian Szegedy , Sergey Ioffe , Vincent Vanhoucke , and Alexander Alemi . 2016 . Inception-v4 , Inception-ResNet and the Impact of Residual Connections on Learning. AAAI Conference on Artificial Intelligence (02 2016). Christian Szegedy, Sergey Ioffe, Vincent Vanhoucke, and Alexander Alemi. 2016. Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning. AAAI Conference on Artificial Intelligence (02 2016).
- J. Thies , M. Zollhöfer , M. Stamminger , C. Theobalt , and M. Nießner . 2016. Face2Face: Real-Time Face Capture and Reenactment of RGB Videos . In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2387--2395 . J. Thies, M. Zollhöfer, M. Stamminger, C. Theobalt, and M. Nießner. 2016. Face2Face: Real-Time Face Capture and Reenactment of RGB Videos. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2387--2395.
- Run Wang , Felix Juefei-Xu , Qing Guo , Yihao Huang , Xiaofei Xie , Lei Ma , and Yang Liu . 2020 . Amora: Black-box Adversarial Morphing Attack. ACM International Conference on Multimedia (ACM MM) (2020). Run Wang, Felix Juefei-Xu, Qing Guo, Yihao Huang, Xiaofei Xie, Lei Ma, and Yang Liu. 2020. Amora: Black-box Adversarial Morphing Attack. ACM International Conference on Multimedia (ACM MM) (2020).
- Hao-Yu Wu , Michael Rubinstein , Eugene Shih , John Guttag , Frédo Durand , and William T. Freeman . 2012 . Eulerian Video Magnification for Revealing Subtle Changes in the World. ACM Transactions on Graphics (Proc. SIGGRAPH 2012) 31, 4 ( 2012 ). Hao-Yu Wu, Michael Rubinstein, Eugene Shih, John Guttag, Frédo Durand, and William T. Freeman. 2012. Eulerian Video Magnification for Revealing Subtle Changes in the World. ACM Transactions on Graphics (Proc. SIGGRAPH 2012) 31, 4 (2012).
- Xiaofei Xie , Lei Ma , Felix Juefei-Xu , Minhui Xue , Hongxu Chen , Yang Liu , Jianjun Zhao , Bo Li , Jianxiong Yin , and Simon See . 2019 . DeepHunter: A CoverageGuided Fuzz Testing Framework for Deep Neural Networks. In ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA). Xiaofei Xie, Lei Ma, Felix Juefei-Xu, Minhui Xue, Hongxu Chen, Yang Liu, Jianjun Zhao, Bo Li, Jianxiong Yin, and Simon See. 2019. DeepHunter: A CoverageGuided Fuzz Testing Framework for Deep Neural Networks. In ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA).
- Zitong Yu Wei Peng Xiaobai Li Xiaopeng Hong and Guoying Zhao. 2019. Remote Heart Rate Measurement from Highly Compressed Facial Videos: an End-toend Deep Learning Solution with Video Enhancement. arXiv:eess.IV/1907.11921 Zitong Yu Wei Peng Xiaobai Li Xiaopeng Hong and Guoying Zhao. 2019. Remote Heart Rate Measurement from Highly Compressed Facial Videos: an End-toend Deep Learning Solution with Video Enhancement. arXiv:eess.IV/1907.11921
Cited by
Deepfake Detection: A Systematic Literature Review
Rana, M. S., Nobi, M. N., Murali, B., & Sung, A. H. (2022) · IEEE Access · doi:10.1109/access.2022.3154404 · Cited by 573
Li, J., Xie, H., Li, J., Wang, Z., & Zhang, Y. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00639 · Cited by 357
“After that, the research mainstream is gradually turning to meth-ods that introduce different information and prior knowledge into backbone networks [10, 33, 29].”
BackgroundMethodInfluentialGenerative AI Literacy: Twelve Defining Competencies
Annapureddy, R., Fornaroli, A., & Gatica-Perez, D. (2025) · Digital Government: Research and Practice · doi:10.1145/3685680 · Cited by 333
“…of generative AI, which may vary from case to case; moreover, although the ethical and legal contexts are important, the user should also consider the social and professional context, which may determine to what extent it is appropriate and useful to use generative AI for given scenarios.”
DeepFake Detection Based on Discrepancies Between Faces and Their Context
Nirkin, Y., Wolf, L., Keller, Y., & Hassner, T. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3093446 · Cited by 312
“Recently, detection methods have been proposed that focus on liveliness and other specific authenticity signals such as heartbeat [13], [14] and specular highlights [15].”
BackgroundDeepFake Detection for Human Face Images and Videos: A Survey
Malik, A., Kuribayashi, M., Abdullahi, S. M., & Khan, A. N. (2022) · IEEE Access · doi:10.1109/access.2022.3151186 · Cited by 297
“Qi et al. [106] proposed a Deep-Rhythm model that also exposes DeepFake videos using heartbeat rhythms.”
Background
Metadata
- Authors
- Qi, H., Guo, Q., Juefei-Xu, F., Xie, X., Ma, L., Feng, W., Liu, Y., & Zhao, J.
- Affiliations
- Kyushu University (JP)
- Nanyang Technological University (SG)
- Alibaba Group (United States) (US)
- Tianjin University (CN)
- Venue
- Proceedings of the 28th ACM International Conference on Multimedia
- Publisher
- ACM
- Published
- 2020-10-12
- Pages
- 4318-4327
- OpenAlex
- W3092997524
- Cited by
- 220
- References
- 50
- Citations by year
- 2026: 24 · 2025: 48 · 2024: 42 · 2023: 42 · 2022: 32 · 2021: 23 · 2020: 9
- FWCI
- 8.09
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM International Conference on Multimedia (ACM MM) (publishedVersion)
- Funders
- National Key Research and Development Project (2016YFB0800403)
- JST-Mirai Program (JPMJMI18BB)
- National Satellite of Excellence in Trustworthy Software System (NRF2018NCR- NSOE003-0001)
- NRF Investigatorship (NRFI06-2020-0022)
- JSPS KAKENHI (20H04168)
- Singapore National Cybersecurity R&D Program (NRF2018NCR-NCR005-0001)
- National Natural Science Foundation of China (61871258)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Keywords
- deepfake detection, remote photoplethysmography, deepfakes, FaceForensics++, DFDC
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.