CoChat

DeepRhythm: Exposing DeepFakes with Attentional Visual Heartbeat Rhythms

Qi, Hua, Guo, Qing, Juefei-Xu, Felix, Xie, Xiaofei, Ma, Lei, Feng, Wei, Liu, Yang, Zhao, Jianjun · 2020 · Proceedings of the 28th ACM International Conference on Multimedia

The DOI matches its Crossref record; 57 of the 57 references it cites were found in a registry or an index.

Abstract

As the GAN-based face image and video generation techniques, widely known as DeepFakes, have become more and more matured and realistic, there comes a pressing and urgent demand for effective DeepFakes detectors. Motivated by the fact that remote visual photoplethysmography (PPG) is made possible by monitoring the minuscule periodic changes of skin color due to blood pumping through the face, we conjecture that normal heartbeat rhythms found in the real face videos will be disrupted or even entirely broken in a DeepFake video, making it a potentially powerful indicator for DeepFake detection. In this work, we propose DeepRhythm, a DeepFake detection technique that exposes DeepFakes by monitoring the heartbeat rhythms. DeepRhythm utilizes dual-spatial-temporal attention to adapt to dynamically changing face and fake types. Extensive experiments on FaceForensics++ and DFDC-preview datasets have confirmed our conjecture and demonstrated not only the effectiveness, but also the generalization capability of DeepRhythm over different datasets by various DeepFakes generation techniques and multifarious challenging degradations.

Overview

Integrity signals

Registry

Crossref · 10.1145/3394171.3413707

Cites

57 of 57 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Citations

Cites

Show all 57 references

Research works printed but not linked: 19

  1. Yupeng Cheng , Qing Guo , Felix Juefei-Xu , Xiaofei Xie , Shang-Wei Lin , Weisi Lin , Wei Feng , and Yang Liu . 2020 . Pasadena: Perceptually Aware and Stealthy Adversarial Denoise Attack. arXiv preprint (2020). Yupeng Cheng, Qing Guo, Felix Juefei-Xu, Xiaofei Xie, Shang-Wei Lin, Weisi Lin, Wei Feng, and Yang Liu. 2020. Pasadena: Perceptually Aware and Stealthy Adversarial Denoise Attack. arXiv preprint (2020).
  2. Pablo Garrido , Levi Valgaerts , Ole Rehmsen , Thorsten Thormählen , Patrick Pérez , and Christian Theobalt . 2014 . Automatic Face Reenactment. 2014 IEEE Conference on Computer Vision and Pattern Recognition (2014), 4217--4224. Pablo Garrido, Levi Valgaerts, Ole Rehmsen, Thorsten Thormählen, Patrick Pérez, and Christian Theobalt. 2014. Automatic Face Reenactment. 2014 IEEE Conference on Computer Vision and Pattern Recognition (2014), 4217--4224.
  3. P. Garrido Levi Valgaerts Hamid Sarmadi I. Steiner Kiran Varanasi P. Pérez and C. Theobalt. 2015. VDub: Modifying Face Video of Actors for Plausible Visual Alignment to a Dubbed Audio Track. Computer Graphics Forum 34 (05 2015). https://doi.org/10.1111/cgf.12552 · DOI
  4. Miroslav Goljan and Jessica Fridrich . 2015 . CFA-aware features for steganalysis of color images . Proceedings of SPIE - The International Society for Optical Engineering 9409 (03 2015). Miroslav Goljan and Jessica Fridrich. 2015. CFA-aware features for steganalysis of color images. Proceedings of SPIE - The International Society for Optical Engineering 9409 (03 2015).
  5. Ian J. Goodfellow Jean Pouget-Abadie Mehdi Mirza Bing Xu David WardeFarley Sherjil Ozair Aaron Courville and Yoshua Bengio. 2014. Generative Adversarial Networks. arXiv:stat.ML/1406.2661 Ian J. Goodfellow Jean Pouget-Abadie Mehdi Mirza Bing Xu David WardeFarley Sherjil Ozair Aaron Courville and Yoshua Bengio. 2014. Generative Adversarial Networks. arXiv:stat.ML/1406.2661
  6. Qing Guo , Wei Feng , Ce Zhou , Chi-Man Pun , and Bin Wu . 2017 . StructureRegularized Compressive Tracking With Online Data-Driven Sampling . IEEE TIP 26 , 12 (2017), 5692 -- 5705 . Qing Guo, Wei Feng, Ce Zhou, Chi-Man Pun, and Bin Wu. 2017. StructureRegularized Compressive Tracking With Online Data-Driven Sampling. IEEE TIP 26, 12 (2017), 5692--5705.
  7. Qing Guo , Felix Juefei-Xu , Xiaofei Xie , Lei Ma , Jian Wang , Bing Yu , Wei Feng , and Yang Liu . 2020. Watch out! Motion is Blurring the Vision of Your Deep Neural Networks. arXiv preprint arXiv:2002.03500 ( 2020 ). Qing Guo, Felix Juefei-Xu, Xiaofei Xie, Lei Ma, Jian Wang, Bing Yu, Wei Feng, and Yang Liu. 2020. Watch out! Motion is Blurring the Vision of Your Deep Neural Networks. arXiv preprint arXiv:2002.03500 (2020). · arXiv
  8. Qing Guo , Xiaofei Xie , Felix Juefei-Xu , Lei Ma , Zhongguo Li , Wanli Xue , Wei Feng , and Yang Liu . 2020 . SPARK: Spatial-aware Online Incremental Attack Against Visual Tracking. European Conference on Computer Vision (ECCV) (2020). Qing Guo, Xiaofei Xie, Felix Juefei-Xu, Lei Ma, Zhongguo Li, Wanli Xue, Wei Feng, and Yang Liu. 2020. SPARK: Spatial-aware Online Incremental Attack Against Visual Tracking. European Conference on Computer Vision (ECCV) (2020).
  9. D. Güera and E. J. Delp . 2018 . Deepfake Video Detection Using Recurrent Neural Networks. In 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS). 1--6. D. Güera and E. J. Delp. 2018. Deepfake Video Detection Using Recurrent Neural Networks. In 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS). 1--6.
  10. X. Li , J. Chen , G. Zhao , and M. Pietikäinen . 2014. Remote Heart Rate Measurement from Face Videos under Realistic Situations . In 2014 IEEE Conference on Computer Vision and Pattern Recognition. 4264--4271 . X. Li, J. Chen, G. Zhao, and M. Pietikäinen. 2014. Remote Heart Rate Measurement from Face Videos under Realistic Situations. In 2014 IEEE Conference on Computer Vision and Pattern Recognition. 4264--4271.
  11. Ming Liu , Yukang Ding , Min Xia , Xiao Liu , Errui Ding , Wangmeng Zuo , and Shilei Wen . 2019 . STGAN: A Unified Selective Transfer Network for Arbitrary Image Attribute Editing. arXiv:cs.CV/1904.09709 Ming Liu, Yukang Ding, Min Xia, Xiao Liu, Errui Ding, Wangmeng Zuo, and Shilei Wen. 2019. STGAN: A Unified Selective Transfer Network for Arbitrary Image Attribute Editing. arXiv:cs.CV/1904.09709
  12. Lei Ma , Felix Juefei-Xu , Jiyuan Sun , Chunyang Chen , Ting Su , Fuyuan Zhang , Minhui Xue , Bo Li , Li Li , Yang Liu , Jianjun Zhao , and Yadong Wang . 2018 . DeepGauge: Multi-Granularity Testing Criteria for Deep Learning Systems. In The 33rd IEEE/ACM International Conference on Automated Software Engineering (ASE). Lei Ma, Felix Juefei-Xu, Jiyuan Sun, Chunyang Chen, Ting Su, Fuyuan Zhang, Minhui Xue, Bo Li, Li Li, Yang Liu, Jianjun Zhao, and Yadong Wang. 2018. DeepGauge: Multi-Granularity Testing Criteria for Deep Learning Systems. In The 33rd IEEE/ACM International Conference on Automated Software Engineering (ASE).
  13. Lei Ma , Fuyuan Zhang , Jiyuan Sun , Minhui Xue , Bo Li , Felix Juefei-Xu , Chao Xie , Li Li , Yang Liu , Jianjun Zhao , and Yadong Wang . 2018 . DeepMutation: Mutation Testing of Deep Learning Systems. In The 29th IEEE International Symposium on Software Reliability Engineering (ISSRE). Lei Ma, Fuyuan Zhang, Jiyuan Sun, Minhui Xue, Bo Li, Felix Juefei-Xu, Chao Xie, Li Li, Yang Liu, Jianjun Zhao, and Yadong Wang. 2018. DeepMutation: Mutation Testing of Deep Learning Systems. In The 29th IEEE International Symposium on Software Reliability Engineering (ISSRE).
  14. Christian Szegedy , Sergey Ioffe , Vincent Vanhoucke , and Alexander Alemi . 2016 . Inception-v4 , Inception-ResNet and the Impact of Residual Connections on Learning. AAAI Conference on Artificial Intelligence (02 2016). Christian Szegedy, Sergey Ioffe, Vincent Vanhoucke, and Alexander Alemi. 2016. Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning. AAAI Conference on Artificial Intelligence (02 2016).
  15. J. Thies , M. Zollhöfer , M. Stamminger , C. Theobalt , and M. Nießner . 2016. Face2Face: Real-Time Face Capture and Reenactment of RGB Videos . In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2387--2395 . J. Thies, M. Zollhöfer, M. Stamminger, C. Theobalt, and M. Nießner. 2016. Face2Face: Real-Time Face Capture and Reenactment of RGB Videos. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2387--2395.
  16. Run Wang , Felix Juefei-Xu , Qing Guo , Yihao Huang , Xiaofei Xie , Lei Ma , and Yang Liu . 2020 . Amora: Black-box Adversarial Morphing Attack. ACM International Conference on Multimedia (ACM MM) (2020). Run Wang, Felix Juefei-Xu, Qing Guo, Yihao Huang, Xiaofei Xie, Lei Ma, and Yang Liu. 2020. Amora: Black-box Adversarial Morphing Attack. ACM International Conference on Multimedia (ACM MM) (2020).
  17. Hao-Yu Wu , Michael Rubinstein , Eugene Shih , John Guttag , Frédo Durand , and William T. Freeman . 2012 . Eulerian Video Magnification for Revealing Subtle Changes in the World. ACM Transactions on Graphics (Proc. SIGGRAPH 2012) 31, 4 ( 2012 ). Hao-Yu Wu, Michael Rubinstein, Eugene Shih, John Guttag, Frédo Durand, and William T. Freeman. 2012. Eulerian Video Magnification for Revealing Subtle Changes in the World. ACM Transactions on Graphics (Proc. SIGGRAPH 2012) 31, 4 (2012).
  18. Xiaofei Xie , Lei Ma , Felix Juefei-Xu , Minhui Xue , Hongxu Chen , Yang Liu , Jianjun Zhao , Bo Li , Jianxiong Yin , and Simon See . 2019 . DeepHunter: A CoverageGuided Fuzz Testing Framework for Deep Neural Networks. In ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA). Xiaofei Xie, Lei Ma, Felix Juefei-Xu, Minhui Xue, Hongxu Chen, Yang Liu, Jianjun Zhao, Bo Li, Jianxiong Yin, and Simon See. 2019. DeepHunter: A CoverageGuided Fuzz Testing Framework for Deep Neural Networks. In ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA).
  19. Zitong Yu Wei Peng Xiaobai Li Xiaopeng Hong and Guoying Zhao. 2019. Remote Heart Rate Measurement from Highly Compressed Facial Videos: an End-toend Deep Learning Solution with Video Enhancement. arXiv:eess.IV/1907.11921 Zitong Yu Wei Peng Xiaobai Li Xiaopeng Hong and Guoying Zhao. 2019. Remote Heart Rate Measurement from Highly Compressed Facial Videos: an End-toend Deep Learning Solution with Video Enhancement. arXiv:eess.IV/1907.11921

Cited by

See all 280

Metadata

Authors
Qi, H., Guo, Q., Juefei-Xu, F., Xie, X., Ma, L., Feng, W., Liu, Y., & Zhao, J.
Affiliations
  • Kyushu University (JP)
  • Nanyang Technological University (SG)
  • Alibaba Group (United States) (US)
  • Tianjin University (CN)
Venue
Proceedings of the 28th ACM International Conference on Multimedia
Publisher
ACM
Published
2020-10-12
Pages
4318-4327
OpenAlex
W3092997524
Cited by
220
References
50
Citations by year
2026: 24 · 2025: 48 · 2024: 42 · 2023: 42 · 2022: 32 · 2021: 23 · 2020: 9
FWCI
8.09
Type
conference paper
Language
en
Open access
closed
Locations
ACM International Conference on Multimedia (ACM MM) (publishedVersion)
Funders
  • National Key Research and Development Project (2016YFB0800403)
  • JST-Mirai Program (JPMJMI18BB)
  • National Satellite of Excellence in Trustworthy Software System (NRF2018NCR- NSOE003-0001)
  • NRF Investigatorship (NRFI06-2020-0022)
  • JSPS KAKENHI (20H04168)
  • Singapore National Cybersecurity R&D Program (NRF2018NCR-NCR005-0001)
  • National Natural Science Foundation of China (61871258)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
Keywords
deepfake detection, remote photoplethysmography, deepfakes, FaceForensics++, DFDC

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: DeepRhythm: Exposing DeepFakes with Attentional Visual Heartbeat Rhythms. https://app.cochat.ai/papers/10.1145/3394171.3413707
The paper
Qi, H., Guo, Q., Juefei-Xu, F., Xie, X., Ma, L., Feng, W., Liu, Y., & Zhao, J. (2020). DeepRhythm: Exposing DeepFakes with Attentional Visual Heartbeat Rhythms. Proceedings of the 28th ACM International Conference on Multimedia. https://doi.org/10.1145/3394171.3413707

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.