DeepRhythm: Exposing DeepFakes with Attentional Visual Heartbeat Rhythms
The DOI matches its Crossref record; 57 of the 57 references it cites were found in a registry or an index.
Abstract
As the GAN-based face image and video generation techniques, widely known as DeepFakes, have become more and more matured and realistic, there comes a pressing and urgent demand for effective DeepFakes detectors. Motivated by the fact that remote visual photoplethysmography (PPG) is made possible by monitoring the minuscule periodic changes of skin color due to blood pumping through the face, we conjecture that normal heartbeat rhythms found in the real face videos will be disrupted or even entirely broken in a DeepFake video, making it a potentially powerful indicator for DeepFake detection. In this work, we propose DeepRhythm, a DeepFake detection technique that exposes DeepFakes by monitoring the heartbeat rhythms. DeepRhythm utilizes dual-spatial-temporal attention to adapt to dynamically changing face and fake types. Extensive experiments on FaceForensics++ and DFDC-preview datasets have confirmed our conjecture and demonstrated not only the effectiveness, but also the generalization capability of DeepRhythm over different datasets by various DeepFakes generation techniques and multifarious challenging degradations.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Xception: Deep Learning with Depthwise Separable Convolutions
Chollet, F. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.195 · Cited by 20,380 · cited 5×
“pFakes by monitoring the heartbeat rhythms. More specifically, DeepRhythm outperforms four state-of-the-art DeepFake detection methods including Bayer’s method [3], Inception ResNet V1 [59], Xception [9], and MesoNet [1] in the FaceForensics++ benchmark [57] and exhibits high robustness to JPEG compression, noise, and…”
BackgroundMethodInfluentialA Style-Based Generator Architecture for Generative Adversarial Networks
Karras, T., Laine, S., & Aila, T. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00453 · Cited by 11,714 · cited 1×
“ion, Thies et al. [60] further use neural textures and defer neural render to generate the forgeries. Besides the above face-wrap-based methods, recent DeepFake approaches, i.e., PGGAN [36], StyleGAN [37], and StyleGAN2 [38], employ the generative adversarial network (GAN) [22] for the nearrealistic face synthesis. Mo…”
MethodAnalyzing and Improving the Image Quality of StyleGAN
Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J., & Aila, T. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00813 · Cited by 6,332
Analyzing and Improving the Image Quality of StyleGAN
Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J., & Aila, T. (2019) · arXiv · doi:10.48550/arxiv.1912.04958 · Cited by 6,332 · cited 1×
“ther uses neural textures and deferred neural render to generate the forgeries. Besides above face-wrap-based methods, more and more DeepFake approaches, i.e., PGGAN [32], StyleGAN [33] and StyleGAN2 [34], employ the generative adversarial network (GAN) [21] to realize the near-realistic face synthesis. Moreover, some…”
Method2009 IEEE Conference on Computer Vision and Pattern Recognition
(2009) · doi:10.1109/cvpr14067.2009 · Cited by 5,550
Progressive Growing of GANs for Improved Quality, Stability, and Variation
Karras, T., Aila, T., Laine, S., & Lehtinen, J. (2017) · arXiv · doi:10.48550/arxiv.1710.10196 · Cited by 4,727 · cited 1×
“model. Then, Thies et al. [53] further uses neural textures and deferred neural render to generate the forgeries. Besides above face-wrap-based methods, more and more DeepFake approaches, i.e., PGGAN [32], StyleGAN [33] and StyleGAN2 [34], employ the generative adversarial network (GAN) [21] to realize the near-realis…”
MethodStarGAN: Unified Generative Adversarial Networks for Multi-domain Image-to-Image Translation
Choi, Y., Choi, M., Kim, M., Ha, J.-W., Kim, S., & Choo, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00916 · Cited by 4,163
FaceForensics++: Learning to Detect Manipulated Facial Images
Rossler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., & Niessner, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00009 · Cited by 3,314 · cited 4×
“We select FaceForensics++ [57] as our training and testing datasets, and use DeepFake Detection Challenge preview (DFDCpreview) [15] as an additional testing dataset to evaluate ourmethod’s cross-dataset generalization capability.”
MethodInfluentialRich Models for Steganalysis of Digital Images
Fridrich, J., & Kodovsky, J. (2012) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2012.2190402 · Cited by 2,144 · cited 1×
“etecting forgeries. Panet al. [46] detect image splicing by estimating local noise variances. Buchana et al. [5] apply shallow methods such as correlation filters to spot the forgery. Fridrich et al. [16] assemble many diverse submodels to build steganography detectors. And Goljan et al. [19] use 3D co-occurrences to…”
BackgroundMesoNet: a Compact Facial Video Forgery Detection Network
Afchar, D., Nozick, V., Yamagishi, J., & Echizen, I. (2018) · 2018 IEEE International Workshop on Information Forensics and Security (WIFS) · doi:10.1109/wifs.2018.8630761 · Cited by 1,872
Research works printed but not linked: 19
- Yupeng Cheng , Qing Guo , Felix Juefei-Xu , Xiaofei Xie , Shang-Wei Lin , Weisi Lin , Wei Feng , and Yang Liu . 2020 . Pasadena: Perceptually Aware and Stealthy Adversarial Denoise Attack. arXiv preprint (2020). Yupeng Cheng, Qing Guo, Felix Juefei-Xu, Xiaofei Xie, Shang-Wei Lin, Weisi Lin, Wei Feng, and Yang Liu. 2020. Pasadena: Perceptually Aware and Stealthy Adversarial Denoise Attack. arXiv preprint (2020).
- Pablo Garrido , Levi Valgaerts , Ole Rehmsen , Thorsten Thormählen , Patrick Pérez , and Christian Theobalt . 2014 . Automatic Face Reenactment. 2014 IEEE Conference on Computer Vision and Pattern Recognition (2014), 4217--4224. Pablo Garrido, Levi Valgaerts, Ole Rehmsen, Thorsten Thormählen, Patrick Pérez, and Christian Theobalt. 2014. Automatic Face Reenactment. 2014 IEEE Conference on Computer Vision and Pattern Recognition (2014), 4217--4224.
- P. Garrido Levi Valgaerts Hamid Sarmadi I. Steiner Kiran Varanasi P. Pérez and C. Theobalt. 2015. VDub: Modifying Face Video of Actors for Plausible Visual Alignment to a Dubbed Audio Track. Computer Graphics Forum 34 (05 2015). https://doi.org/10.1111/cgf.12552 · DOI
- Miroslav Goljan and Jessica Fridrich . 2015 . CFA-aware features for steganalysis of color images . Proceedings of SPIE - The International Society for Optical Engineering 9409 (03 2015). Miroslav Goljan and Jessica Fridrich. 2015. CFA-aware features for steganalysis of color images. Proceedings of SPIE - The International Society for Optical Engineering 9409 (03 2015).
- Ian J. Goodfellow Jean Pouget-Abadie Mehdi Mirza Bing Xu David WardeFarley Sherjil Ozair Aaron Courville and Yoshua Bengio. 2014. Generative Adversarial Networks. arXiv:stat.ML/1406.2661 Ian J. Goodfellow Jean Pouget-Abadie Mehdi Mirza Bing Xu David WardeFarley Sherjil Ozair Aaron Courville and Yoshua Bengio. 2014. Generative Adversarial Networks. arXiv:stat.ML/1406.2661
- Qing Guo , Wei Feng , Ce Zhou , Chi-Man Pun , and Bin Wu . 2017 . StructureRegularized Compressive Tracking With Online Data-Driven Sampling . IEEE TIP 26 , 12 (2017), 5692 -- 5705 . Qing Guo, Wei Feng, Ce Zhou, Chi-Man Pun, and Bin Wu. 2017. StructureRegularized Compressive Tracking With Online Data-Driven Sampling. IEEE TIP 26, 12 (2017), 5692--5705.
- Qing Guo , Felix Juefei-Xu , Xiaofei Xie , Lei Ma , Jian Wang , Bing Yu , Wei Feng , and Yang Liu . 2020. Watch out! Motion is Blurring the Vision of Your Deep Neural Networks. arXiv preprint arXiv:2002.03500 ( 2020 ). Qing Guo, Felix Juefei-Xu, Xiaofei Xie, Lei Ma, Jian Wang, Bing Yu, Wei Feng, and Yang Liu. 2020. Watch out! Motion is Blurring the Vision of Your Deep Neural Networks. arXiv preprint arXiv:2002.03500 (2020). · arXiv
- Qing Guo , Xiaofei Xie , Felix Juefei-Xu , Lei Ma , Zhongguo Li , Wanli Xue , Wei Feng , and Yang Liu . 2020 . SPARK: Spatial-aware Online Incremental Attack Against Visual Tracking. European Conference on Computer Vision (ECCV) (2020). Qing Guo, Xiaofei Xie, Felix Juefei-Xu, Lei Ma, Zhongguo Li, Wanli Xue, Wei Feng, and Yang Liu. 2020. SPARK: Spatial-aware Online Incremental Attack Against Visual Tracking. European Conference on Computer Vision (ECCV) (2020).
- D. Güera and E. J. Delp . 2018 . Deepfake Video Detection Using Recurrent Neural Networks. In 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS). 1--6. D. Güera and E. J. Delp. 2018. Deepfake Video Detection Using Recurrent Neural Networks. In 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS). 1--6.
- X. Li , J. Chen , G. Zhao , and M. Pietikäinen . 2014. Remote Heart Rate Measurement from Face Videos under Realistic Situations . In 2014 IEEE Conference on Computer Vision and Pattern Recognition. 4264--4271 . X. Li, J. Chen, G. Zhao, and M. Pietikäinen. 2014. Remote Heart Rate Measurement from Face Videos under Realistic Situations. In 2014 IEEE Conference on Computer Vision and Pattern Recognition. 4264--4271.
- Ming Liu , Yukang Ding , Min Xia , Xiao Liu , Errui Ding , Wangmeng Zuo , and Shilei Wen . 2019 . STGAN: A Unified Selective Transfer Network for Arbitrary Image Attribute Editing. arXiv:cs.CV/1904.09709 Ming Liu, Yukang Ding, Min Xia, Xiao Liu, Errui Ding, Wangmeng Zuo, and Shilei Wen. 2019. STGAN: A Unified Selective Transfer Network for Arbitrary Image Attribute Editing. arXiv:cs.CV/1904.09709
- Lei Ma , Felix Juefei-Xu , Jiyuan Sun , Chunyang Chen , Ting Su , Fuyuan Zhang , Minhui Xue , Bo Li , Li Li , Yang Liu , Jianjun Zhao , and Yadong Wang . 2018 . DeepGauge: Multi-Granularity Testing Criteria for Deep Learning Systems. In The 33rd IEEE/ACM International Conference on Automated Software Engineering (ASE). Lei Ma, Felix Juefei-Xu, Jiyuan Sun, Chunyang Chen, Ting Su, Fuyuan Zhang, Minhui Xue, Bo Li, Li Li, Yang Liu, Jianjun Zhao, and Yadong Wang. 2018. DeepGauge: Multi-Granularity Testing Criteria for Deep Learning Systems. In The 33rd IEEE/ACM International Conference on Automated Software Engineering (ASE).
- Lei Ma , Fuyuan Zhang , Jiyuan Sun , Minhui Xue , Bo Li , Felix Juefei-Xu , Chao Xie , Li Li , Yang Liu , Jianjun Zhao , and Yadong Wang . 2018 . DeepMutation: Mutation Testing of Deep Learning Systems. In The 29th IEEE International Symposium on Software Reliability Engineering (ISSRE). Lei Ma, Fuyuan Zhang, Jiyuan Sun, Minhui Xue, Bo Li, Felix Juefei-Xu, Chao Xie, Li Li, Yang Liu, Jianjun Zhao, and Yadong Wang. 2018. DeepMutation: Mutation Testing of Deep Learning Systems. In The 29th IEEE International Symposium on Software Reliability Engineering (ISSRE).
- Christian Szegedy , Sergey Ioffe , Vincent Vanhoucke , and Alexander Alemi . 2016 . Inception-v4 , Inception-ResNet and the Impact of Residual Connections on Learning. AAAI Conference on Artificial Intelligence (02 2016). Christian Szegedy, Sergey Ioffe, Vincent Vanhoucke, and Alexander Alemi. 2016. Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning. AAAI Conference on Artificial Intelligence (02 2016).
- J. Thies , M. Zollhöfer , M. Stamminger , C. Theobalt , and M. Nießner . 2016. Face2Face: Real-Time Face Capture and Reenactment of RGB Videos . In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2387--2395 . J. Thies, M. Zollhöfer, M. Stamminger, C. Theobalt, and M. Nießner. 2016. Face2Face: Real-Time Face Capture and Reenactment of RGB Videos. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2387--2395.
- Run Wang , Felix Juefei-Xu , Qing Guo , Yihao Huang , Xiaofei Xie , Lei Ma , and Yang Liu . 2020 . Amora: Black-box Adversarial Morphing Attack. ACM International Conference on Multimedia (ACM MM) (2020). Run Wang, Felix Juefei-Xu, Qing Guo, Yihao Huang, Xiaofei Xie, Lei Ma, and Yang Liu. 2020. Amora: Black-box Adversarial Morphing Attack. ACM International Conference on Multimedia (ACM MM) (2020).
- Hao-Yu Wu , Michael Rubinstein , Eugene Shih , John Guttag , Frédo Durand , and William T. Freeman . 2012 . Eulerian Video Magnification for Revealing Subtle Changes in the World. ACM Transactions on Graphics (Proc. SIGGRAPH 2012) 31, 4 ( 2012 ). Hao-Yu Wu, Michael Rubinstein, Eugene Shih, John Guttag, Frédo Durand, and William T. Freeman. 2012. Eulerian Video Magnification for Revealing Subtle Changes in the World. ACM Transactions on Graphics (Proc. SIGGRAPH 2012) 31, 4 (2012).
- Xiaofei Xie , Lei Ma , Felix Juefei-Xu , Minhui Xue , Hongxu Chen , Yang Liu , Jianjun Zhao , Bo Li , Jianxiong Yin , and Simon See . 2019 . DeepHunter: A CoverageGuided Fuzz Testing Framework for Deep Neural Networks. In ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA). Xiaofei Xie, Lei Ma, Felix Juefei-Xu, Minhui Xue, Hongxu Chen, Yang Liu, Jianjun Zhao, Bo Li, Jianxiong Yin, and Simon See. 2019. DeepHunter: A CoverageGuided Fuzz Testing Framework for Deep Neural Networks. In ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA).
- Zitong Yu Wei Peng Xiaobai Li Xiaopeng Hong and Guoying Zhao. 2019. Remote Heart Rate Measurement from Highly Compressed Facial Videos: an End-toend Deep Learning Solution with Video Enhancement. arXiv:eess.IV/1907.11921 Zitong Yu Wei Peng Xiaobai Li Xiaopeng Hong and Guoying Zhao. 2019. Remote Heart Rate Measurement from Highly Compressed Facial Videos: an End-toend Deep Learning Solution with Video Enhancement. arXiv:eess.IV/1907.11921
Cited by
Deepfake Detection: A Systematic Literature Review
Rana, M. S., Nobi, M. N., Murali, B., & Sung, A. H. (2022) · IEEE Access · doi:10.1109/access.2022.3154404 · Cited by 573
Li, J., Xie, H., Li, J., Wang, Z., & Zhang, Y. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00639 · Cited by 357
“After that, the research mainstream is gradually turning to meth-ods that introduce different information and prior knowledge into backbone networks [10, 33, 29].”
BackgroundMethodInfluentialGenerative AI Literacy: Twelve Defining Competencies
Annapureddy, R., Fornaroli, A., & Gatica-Perez, D. (2025) · Digital Government: Research and Practice · doi:10.1145/3685680 · Cited by 333
“…of generative AI, which may vary from case to case; moreover, although the ethical and legal contexts are important, the user should also consider the social and professional context, which may determine to what extent it is appropriate and useful to use generative AI for given scenarios.”
DeepFake Detection Based on Discrepancies Between Faces and Their Context
Nirkin, Y., Wolf, L., Keller, Y., & Hassner, T. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3093446 · Cited by 312
“Recently, detection methods have been proposed that focus on liveliness and other specific authenticity signals such as heartbeat [13], [14] and specular highlights [15].”
BackgroundDeepFake Detection for Human Face Images and Videos: A Survey
Malik, A., Kuribayashi, M., Abdullahi, S. M., & Khan, A. N. (2022) · IEEE Access · doi:10.1109/access.2022.3151186 · Cited by 297
“Qi et al. [106] proposed a Deep-Rhythm model that also exposes DeepFake videos using heartbeat rhythms.”
Background
Metadata
- Authors
- Qi, H., Guo, Q., Juefei-Xu, F., Xie, X., Ma, L., Feng, W., Liu, Y., & Zhao, J.
- Affiliations
- Kyushu University (JP)
- Nanyang Technological University (SG)
- Alibaba Group (United States) (US)
- Tianjin University (CN)
- Venue
- Proceedings of the 28th ACM International Conference on Multimedia
- Publisher
- ACM
- Published
- 2020-10-12
- Pages
- 4318-4327
- OpenAlex
- W3092997524
- Cited by
- 220
- References
- 50
- Citations by year
- 2026: 24 · 2025: 48 · 2024: 42 · 2023: 42 · 2022: 32 · 2021: 23 · 2020: 9
- FWCI
- 8.09
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM International Conference on Multimedia (ACM MM) (publishedVersion)
- Funders
- National Key Research and Development Project (2016YFB0800403)
- JST-Mirai Program (JPMJMI18BB)
- National Satellite of Excellence in Trustworthy Software System (NRF2018NCR- NSOE003-0001)
- NRF Investigatorship (NRFI06-2020-0022)
- JSPS KAKENHI (20H04168)
- Singapore National Cybersecurity R&D Program (NRF2018NCR-NCR005-0001)
- National Natural Science Foundation of China (61871258)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Keywords
- deepfake detection, remote photoplethysmography, deepfakes, FaceForensics++, DFDC
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: DeepRhythm: Exposing DeepFakes with Attentional Visual Heartbeat Rhythms. https://app.cochat.ai/papers/10.1145/3394171.3413707
- The paper
- Qi, H., Guo, Q., Juefei-Xu, F., Xie, X., Ma, L., Feng, W., Liu, Y., & Zhao, J. (2020). DeepRhythm: Exposing DeepFakes with Attentional Visual Heartbeat Rhythms. Proceedings of the 28th ACM International Conference on Multimedia. https://doi.org/10.1145/3394171.3413707
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.