CoChat
Sign Up
PreprintCited by 14Biometric Identification and SecurityDigital Media Forensic DetectionUser Authentication and Security Systems

The DOI matches its DataCite record.

Seeing is Living? Rethinking the Security of Facial Liveness Verification in the Deepfake Era

Open access
Li, Changjiang, Wang, Li, Ji, Shouling, Zhang, Xuhong, Xi, Zhaohan, Guo, Shanqing, Wang, Ting·2022·arXiv
DOI 10.48550/arxiv.2202.10673arXiv 2202.10673
Open this paper in CoChat — free accountOpen PDF

Abstract

Facial Liveness Verification (FLV) is widely used for identity authentication in many security-sensitive domains and offered as Platform-as-a-Service (PaaS) by leading cloud vendors. Yet, with the rapid advances in synthetic media techniques (e.g., deepfake), the security of FLV is facing unprecedented challenges, about which little is known thus far. To bridge this gap, in this paper, we conduct the first systematic study on the security of FLV in real-world settings. Specifically, we present LiveBugger, a new deepfake-powered attack framework that enables customizable, automated security evaluation of FLV. Leveraging LiveBugger, we perform a comprehensive empirical assessment of representative FLV platforms, leading to a set of interesting findings. For instance, most FLV APIs do not use anti-deepfake detection; even for those with such defenses, their effectiveness is concerning (e.g., it may detect high-quality synthesized videos but fail to detect low-quality ones). We then conduct an in-depth analysis of the factors impacting the attack performance of LiveBugger: a) the bias (e.g., gender or race) in FLV can be exploited to select victims; b) adversarial training makes deepfake more effective to bypass FLV; c) the input quality has a varying influence on different deepfake techniques to bypass FLV. Based on these findings, we propose a customized, two-stage approach that can boost the attack success rate by up to 70%. Further, we run proof-of-concept attacks on several representative applications of FLV (i.e., the clients of FLV APIs) to illustrate the practical implications: due to the vulnerability of the APIs, many downstream applications are vulnerable to deepfake. Finally, we discuss potential countermeasures to improve the security of FLV. Our findings have been confirmed by the corresponding vendors.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2202.10673

Cites

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

No references could be resolved yet.

Research works printed but not linked: 42

  1. [1][1] Z. Ming, J. Chazalon, M. M. Luqman, M. Visani, and J.-C. Burie, 'Facelivenet: End-to-end networks combining face verification with interactive facial expression-based liveness detection,' in 2018 24th International Conference on Pattern Recognition (ICPR) . IEEE, 2018, pp. 3507-3512. -
  2. [2][2] E. Ivanova and G. Borzunov, 'Improving the security of the facial biometrics system using the liveness detection module,' in Advanced Technologies in Robotics and Intelligent Systems . Springer, 2020, pp. 199-205. -
  3. [3][3] M. Smith-Creasey, F. A. Albalooshi, and M. Rajarajan, 'Continuous face authentication scheme for mobile devices with tracking and liveness detection,' Microprocessors and Microsystems , vol. 63, pp. 147-157, 2018. -
  4. [4][4] T. Yang, W.-G. Chen, Y.-N. Zhang, and M.-R. Wu, 'Empirical study on web-based facial recognition services,' in 2018 IEEE 4th International Conference on Computer and Communications (ICCC) . IEEE, 2018, pp. 1463-1467. -
  5. [9][9] R. Ramachandra and C. Busch, 'Presentation attack detection methods for face recognition systems: A comprehensive survey,' ACMComputing Surveys (CSUR) , vol. 50, no. 1, pp. 1-37, 2017. -
  6. [10][10] Z. Boulkenafet, J. Komulainen, and A. Hadid, 'Face antispoofing based on color texture analysis,' in 2015 IEEE international conference on image processing (ICIP) . IEEE, 2015, pp. 2636-2640. -
  7. [11][11] X. Li, J. Komulainen, G. Zhao, P.-C. Yuen, and M. Pietikäinen, 'Generalized face anti-spoofing by detecting pulse from face videos,' in 2016 23rd International Conference on Pattern Recognition (ICPR) . IEEE, 2016, pp. 4244-4249. -
  8. [12][12] S.-Q. Liu, X. Lan, and P. C. Yuen, 'Remote photoplethysmography correspondence feature for 3d mask face presentation attack detection,' in Proceedings of the European Conference on Computer Vision (ECCV) , 2018, pp. 558-573. -
  9. [13][13] X. Tan, Y. Li, J. Liu, and L. Jiang, 'Face liveness detection from a single image with sparse low rank bilinear discriminative model,' in European Conference on Computer Vision . Springer, 2010, pp. 504-517. -
  10. [14][14] J. Määttä, A. Hadid, and M. Pietikäinen, 'Face spoofing detection from single images using micro-texture analysis,' in 2011 international joint conference on Biometrics (IJCB) . IEEE, 2011, pp. 1-7. -
  11. [15][15] A. George, Z. Mostaani, D. Geissenbuhler, O. Nikisins, A. Anjos, and S. Marcel, 'Biometric face presentation attack detection with multi-channel convolutional neural network,' IEEE Transactions on Information Forensics and Security , vol. 15, pp. 42-55, 2019. -
  12. [16][16] G. Wang, H. Han, S. Shan, and X. Chen, 'Cross-domain face presentation attack detection via multi-domain disentangled representation learning,' in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2020, pp. 6678-6687. -
  13. [17][17] Y. Mirsky and W. Lee, 'The creation and detection of deepfakes: A survey,' ACM Computing Surveys (CSUR) , vol. 54, no. 1, pp. 1-41, 2021. -
  14. [18][18] E. Uzun, S. P. H. Chung, I. Essa, and W. Lee, 'rtcaptcha: A realtime CAPTCHA based liveness detection system,' in NDSS , 2018. -
  15. [19][19] S. A. Buo, 'The emerging threats of deepfake attacks and countermeasures,' arXiv preprint arXiv:2012.07989 , 2020. - · arXiv
  16. [22][22] Y. Nirkin, Y. Keller, and T. Hassner, 'Fsgan: Subject agnostic face swapping and reenactment,' in Proceedings of the IEEE international conference on computer vision , 2019, pp. 71847193. -
  17. [24][24] Y. Zhang, Z. Yin, Y. Li, G. Yin, J. Yan, J. Shao, and Z. Liu, 'Celeba-spoof: Large-scale face anti-spoofing dataset with rich annotations,' in European Conference on Computer Vision . Springer, 2020, pp. 70-85. -
  18. [25][25] Y. Liu, J. Stehouwer, A. Jourabloo, and X. Liu, 'Deep tree learning for zero-shot face anti-spoofing,' in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2019, pp. 4680-4689. -
  19. [26][26] O. Wiles, A. Sophia Koepke, and A. Zisserman, 'X2face: A network for controlling face generation using images, audio, and pose codes,' in Proceedings of the European conference on computer vision (ECCV) , 2018, pp. 670-686. -
  20. [27][27] S. Tripathy, J. Kannala, and E. Rahtu, 'Icface: Interpretable and controllable face reenactment using gans,' in The IEEE Winter Conference on Applications of Computer Vision , 2020, pp. 3385-3394. -
  21. [28][28] A. Siarohin, S. Lathuilière, S. Tulyakov, E. Ricci, and N. Sebe, 'First order motion model for image animation,' in Advances in Neural Information Processing Systems , 2019, pp. 7137-7147. -
  22. [29][29] L. Li, J. Bao, H. Yang, D. Chen, and F. Wen, 'Faceshifter: Towards high fidelity and occlusion aware face swapping,' arXiv preprint arXiv:1912.13457 , 2019. - · arXiv
  23. [30][30] A. Bulat and G. Tzimiropoulos, 'How far are we from solving the 2d & 3d face alignment problem? (and a dataset of 230,000 3d facial landmarks),' in International Conference on Computer Vision , 2017. -
  24. [31][31] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. WardeFarley, S. Ozair, A. Courville, and Y. Bengio, 'Generative adversarial nets,' Advances in neural information processing systems , vol. 27, pp. 2672-2680, 2014. -
  25. [32][32] M. Sharif, S. Bhagavatula, L. Bauer, and M. K. Reiter, 'Accessorize to a crime: Real and stealthy attacks on state-of-the-art face recognition,' in Proceedings of the 2016 acm sigsac conference on computer and communications security , 2016, pp. 1528-1540. -
  26. [33][33] B. Zhao, S. Ji, W.-H. Lee, C. Lin, H. Weng, J. Wu, P. Zhou, L. Fang, and R. Beyah, 'A large-scale empirical study on the vulnerability of deployed iot devices,' IEEE Transactions on Dependable and Secure Computing , 2020. -
  27. [34][34] C. Shi, X. Xu, S. Ji, K. Bu, J. Chen, R. Beyah, and T. Wang, 'Adversarial captchas,' IEEE Transactions on Cybernetics , 2021. -
  28. [35][35] B. Zhao, H. Weng, S. Ji, J. Chen, T. Wang, Q. He, and R. Beyah, 'Towards evaluating the security of real-world deployed image captchas,' in Proceedings of the 11th ACM Workshop on Artificial Intelligence and Security , 2018, pp. 85-96. -
  29. [36][36] S. Shan, E. Wenger, J. Zhang, H. Li, H. Zheng, and B. Y. Zhao, 'Fawkes: Protecting privacy against unauthorized deep learning models,' in 29th { USENIX } Security Symposium ( { USENIX } Security 20) , 2020, pp. 1589-1604. -
  30. [37][37] T.-C. Wang, M.-Y. Liu, A. Tao, G. Liu, J. Kautz, and B. Catanzaro, 'Few-shot video-to-video synthesis,' arXiv preprint arXiv:1910.12713 , 2019. - · arXiv
  31. [38][38] E. Zakharov, A. Shysheya, E. Burkov, and V. Lempitsky, 'Fewshot adversarial learning of realistic neural talking head models,' in Proceedings of the IEEE International Conference on Computer Vision , 2019, pp. 9459-9468. -
  32. [40][40] Y. Li, K. Xu, Q. Yan, Y. Li, and R. H. Deng, 'Understanding osn-based facial disclosure against face authentication systems,' in Proceedings of the 9th ACM symposium on Information, computer and communications security , 2014, pp. 413-424. -
  33. [41][41] A. Benlamoudi, D. Samai, A. Ouafi, S. E. Bekhouche, A. TalebAhmed, and A. Hadid, 'Face spoofing detection using local binary patterns and fisher score,' in 2015 3rd International Conference on Control, Engineering & Information Technology (CEIT) . IEEE, 2015, pp. 1-5. -
  34. [42][42] S. Parveen, S. M. S. Ahmad, N. H. Abbas, W. A. W. Adnan, M. Hanafi, and N. Naeem, 'Face liveness detection using dynamic local ternary pattern (dltp),' Computers , vol. 5, no. 2, p. 10, 2016. -
  35. [43][43] G. Pan, L. Sun, Z. Wu, and S. Lao, 'Eyeblink-based antispoofing in face recognition from a generic webcamera,' in 2007 IEEE 11th International Conference on Computer Vision . IEEE, 2007, pp. 1-8. -
  36. [44][44] A. Jourabloo, Y. Liu, and X. Liu, 'Face de-spoofing: Antispoofing via noise modeling,' in Proceedings of the European Conference on Computer Vision (ECCV) , 2018, pp. 290-306. -
  37. [45][45] A. George and S. Marcel, 'Deep pixel-wise binary supervision for face presentation attack detection,' in 2019 International Conference on Biometrics (ICB) . IEEE, 2019, pp. 1-8. -
  38. [46][46] X. Yang, W. Luo, L. Bao, Y. Gao, D. Gong, S. Zheng, Z. Li, and W. Liu, 'Face anti-spoofing: Model matters, so does data,' in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2019, pp. 3507-3516. -
  39. [47][47] G. Chetty and M. Wagner, 'Multi-level liveness verification for face-voice biometric authentication,' in 2006 Biometrics Symposium: Special Session on Research at the Biometric Consortium Conference . IEEE, 2006, pp. 1-6. -
  40. [48][48] T. Choudhury, B. Clarkson, T. Jebara, and A. Pentland, 'Multimodal person recognition using unconstrained audio and video,' in Proceedings, International Conference on Audio-and VideoBased Person Authentication . Citeseer, 1999, pp. 176-181. -
  41. [49][49] R. W. Frischholz and A. Werner, 'Avoiding replay-attacks in a face recognition system using head-pose estimation,' in 2003 IEEE International SOI Conference. Proceedings (Cat. No. 03CH37443) . IEEE, 2003, pp. 234-235. -
  42. [50][50] D. Tang, Z. Zhou, Y. Zhang, and K. Zhang, 'Face flashing: a secure liveness detection protocol based on light reflections,' in NDSS , 2018.

Cited by

Sorted by citations

Metadata

Authors
Li, C., Wang, L., Ji, S., Zhang, X., Xi, Z., Guo, S., & Wang, T.
Affiliations
  • Shandong University (CN)
  • Zhejiang University (CN)
  • Pennsylvania State University (US)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2022
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2202.10673
arXiv
2202.10673
OpenAlex
W4221161648
Cited by
14
References
0
Citations by year
2025: 2 · 2024: 7 · 2023: 4 · 2022: 1
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • National Science Foundation (1953813, 1951729, 1953893)
  • National Natural Science Foundation of China (U1936215, 62102360, U1836202)
  • National Key Research and Development Program of China (2020AAA0140004)
  • Natural Science Foundation of Zhejiang Province (LR19F020003)
Topics
  • Physical Sciences › Computer Science › Signal Processing › Biometric Identification and Security
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
  • Physical Sciences › Computer Science › Information Systems › User Authentication and Security Systems
Keywords
deepfake attacks, synthetic media, adversarial training, input quality, countermeasures

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review