CoChat
Sign Up
Cited by 326Digital Media Forensic DetectionBiometric Identification and SecurityBenford’s Law and Fraud Detection

DeepVision: Deepfakes Detection Using Human Eye Blinking Pattern

Open access
Jung, Tackhyun, Kim, Sangwon, Kim, Keecheon·2020·IEEE Access
DOI 10.1109/access.2020.2988660
Open this paper in CoChat — free accountOpen PDF

Abstract

In this paper, we propose a new approach to detect Deepfakes generated through the generative adversarial network (GANs) model via an algorithm called DeepVision to analyze a significant change in the pattern of blinking, which is a voluntary and spontaneous action that does not require conscious effort. Human eye blinking pattern has been known to significantly change according to the person's overall physical conditions, cognitive activities, biological factors, and information processing level. For example, an individual's gender or age, the time of day, or the person's emotional state or degree of alertness can all influence the pattern. As a result, Deepfakes can be determined through integrity verification by tracking significant changes in the eye blinking patterns in deepfakes by means of a heuristic method based on the results of medicine, biology, and brain engineering research, as well as machine learning and various algorithms based on engineering and statistical knowledge. This means we can perform integrity verification through tracking significant changes in the eye blinking pattern of a subject in a video. The proposed method called DeepVision is implemented as a measure to verify an anomaly based on the period, repeated number, and elapsed eye blink time when eye blinks were continuously repeated within a very short period of time. DeepVision accurately detected Deepfakes in seven out of eight types of videos (87.5% accuracy rate), suggesting we can overcome the limitations of integrity verification algorithms performed only on the basis of pixels.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/access.2020.2988660

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

References 27 · Cited by 326

Research works printed but not linked: 30

  1. I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio ''Generative adversarial nets,'' in Proc. Adv. Neural Inf. Process. Syst., 2014, pp. 2672-2680.
  2. A. Radford, L. Metz, and S. Chintala, ''Unsupervised representation learning with deep convolutional generative adversarial networks,'' 2015, arXiv:1511.06434. [Online]. Available: http://arxiv.org/abs/1511.06434 · arXiv
  3. S. Suwajanakorn, S. M. Seitz, and I. Kemelmacher-Shlizerman, ''Synthe- sizing Obama: Learning lip sync from audio,'' ACM Trans. Graph., vol. 36, no. 4, pp. 1-13, Jul. 2017.
  4. C.-C. Hsu, C.-Y. Lee, and Y.-X. Zhuang, ''Learning to detect fake face images in the wild,'' in Proc. Int. Symp. Comput., Consum. Control (IS3C), Taichung, Taiwan, Dec. 2018, pp. 388-391.
  5. B. Bayar and M. C. Stamm, ''A deep learning approach to universal image manipulation detection using a new convolutional layer,'' in Proc. 4th ACM Workshop Inf. Hiding Multimedia Secur. (IH&MMSec), 2016, pp. 5-10.
  6. A. Rössler, D. Cozzolino, L. Verdoliva, C. Riess, J. Thies, and M. Nießner, ''FaceForensics++: Learning to detect manipulated facial images,'' 2019, arXiv:1901.08971. [Online]. Available: http://arxiv.org/abs/1901.08971 · arXiv
  7. D. H. Kim, S. W. Choi, and S. Y. Kwak, ''Deep learning based fake face detection,'' J. Korea Ind. Inf. Syst. Res., vol. 23, no. 5, pp. 9-17, Oct. 2018.
  8. Y. Li, M.-C. Chang, and S. Lyu, ''In ictu oculi: Exposing AI generated fake face videos by detecting eye blinking,'' 2018, arXiv:1806.02877. [Online]. Available: http://arxiv.org/abs/1806.02877 · arXiv
  9. S. Lawrence, C. L. Giles, A. Chung Tsoi, and A. D. Back, ''Face recog- nition: A convolutional neural-network approach,'' IEEE Trans. Neural Netw., vol. 8, no. 1, pp. 98-113, Jan. 1997.
  10. J. G. Lawrenson, R. Birhah, and P. J. Murphy, ''Tear-film lipid layer mor- phology and corneal sensation in the development of blinking in neonates and infants,'' J. Anatomy, vol. 206, no. 3, pp. 265-270, Mar. 2005.
  11. A. J. Zametkin, J. R. Stevens, and R. Pittman, ''Ontogeny of spontaneous blinking and of habituation of the blink reflex,'' Ann. Neurol., vol. 5, no. 5, pp. 453-457, May 1979.
  12. P. J. De Jong and H. Merckelbach, ''Eyeblink frequency, rehearsal activity, and sympathetic arousal,'' Int. J. Neurosci., vol. 51, nos. 1-2, pp. 89-94, Jan. 1990.
  13. J. Oh and J. Jeong, ''Potential significance of eyeblinks as a behavior marker of neuropsychiatric disorders,'' Korean J. Biol. Psychiatry, vol. 19, no. 1, pp. 9-20, 2012.
  14. E. Ponder and W. P. Kennedy, ''On the act of blinking,'' Quart. J. Exp. Physiol., vol. 18, no. 2, pp. 89-110, Jul. 1927.
  15. L. C. Dang, G. R. Samanez-Larkin, J. J. Castrellon, S. F. Perkins, R. L. Cowan, P. A. Newhouse, and D. H. Zald, ''Spontaneous eye blink rate (EBR) is uncorrelated with dopamine D2 receptor availability and unmodulated by dopamine agonism in healthy adults,'' eNeuro, vol. 4, no. 5, pp. 1-11, Sep. 2017.
  16. G. Barbato, G. Ficca, G. Muscettola, M. Fichele, M. Beatrice, and F. Rinaldi, ''Diurnal variation in spontaneous eye-blink rate,'' Psychiatry Res., vol. 93, no. 2, pp. 145-151, Mar. 2000.
  17. A. R. Bentivoglio, S. B. Bressman, E. Cassetta, D. Carretta, P. Tonali, and A. Albanese, ''Analysis of blink rate patterns in normal subjects,'' Movement Disorders, vol. 12, no. 6, pp. 1028-1034, Nov. 1997.
  18. R. Ranjan, V. M. Patel, and R. Chellappa, ''HyperFace: A deep multi- task learning framework for face detection, landmark localization, pose estimation, and gender recognition,'' 2016, arXiv:1603.01249. [Online]. Available: http://arxiv.org/abs/1603.01249 · arXiv
  19. T. Soukupová and J. Cech, ''Real-time eye blink detection using facial landmarks,'' in Proc. Comput. Vis. Winter Workshop (CVWW), 2016, pp. 42-50.
  20. D. von Cramon and U. Schuri, ''Blink frequency and speech motor activ- ity,'' Neuropsychologia, vol. 18, nos. 4-5, pp. 603-606, Jan. 1980.
  21. D. Guera and E. J. Delp, ''Deepfake video detection using recurrent neural networks,'' in Proc. 15th IEEE Int. Conf. Adv. Video Signal Based Surveill. (AVSS), Nov. 2018, pp. 1-6.
  22. M. Koopman, A. M. Rodriguez, and Z. Geradts, ''Detection of deepfake video manipulation,'' in Proc. IMVIP, Belfast, U.K., 2018, pp. 133-136.
  23. W. J. Freed, J. E. Kleinman, C. N. Karson, S. G. Potkin, D. L. Murphy, and R. J. Wyatt, ''Eye-blink rates and platelet monoamine oxidase activ- ity in chronic schizophrenic patients,'' Biol. psychiatry., vol. 15, no. 2, pp. 329-332, 1980.
  24. Kaggle, ''Eye blinking prediction,'' in Proc. CompOmics Summer Competition, 2018. [Online]. Available: https://www.kaggle.com/c/ compomicssummer2018/data
  25. O. Rösler and D. Suendermann, ''A first step towards eye state prediction using EEG,'' in Proc. AIHLS, Istanbul, Turkey, 2013.
  26. B. L. Sean and J. Taylor, ''Forecasting at scale,'' Amer. Statistician, vol. 72, no. 1, pp. 37-45, 2017.
  27. G. E. P. Box, G. M. Jenkins, and G. C. Reinsel, Time Series Analysis: Forecasting and Control. Hoboken, NJ, USA: Wiley, 2016.
  28. D. Afchar, V. Nozick, J. Yamagishi, and I. Echizen, ''MesoNet: A compact facial video forgery detection network,'' in Proc. IEEE Int. Workshop Inf. Forensics Secur. (WIFS), Dec. 2018, pp. 1-7.
  29. N. Rahmouni, V. Nozick, J. Yamagishi, and I. Echizen, ''Distinguishing computer graphics from natural images using convolution neural net- works,'' in Proc. IEEE Workshop Inf. Forensics Secur. (WIFS), Dec. 2017, pp. 1-6.
  30. A. Rössler, D. Cozzolino, L. Verdoliva, C. Riess, J. Thies, and M. Nießner, ''FaceForensics: A large-scale video dataset for forgery detection in human faces,'' 2018, arXiv:1803.09179. [Online]. Available: http://arxiv.org/abs/1803.09179 TACKHYUN JUNG is currently pursuing the master's degree in IT convergence information security with the IN&S Lab, Konkuk University. Before entering Konkuk University, he was a Mas- ter Sergeant with the Republic of Korea (ROK) Air Force, and he worked on national infor- mation security for four and a half years. His research interests are cybersecurity, AI, and com- puter vision. · arXiv

Authors
Jung, T., Kim, S., & Kim, K.
Affiliations
Konkuk University (KR)
ORCID
Venue
IEEE Access
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Published
2020
Volume
8
Pages
83144-83154
ISSN
2169-3536 (ISSN-L)
DOI
10.1109/access.2020.2988660
OpenAlex
W3017837134
Cited by
326
References
27
Citations by year
2026: 34 · 2025: 70 · 2024: 87 · 2023: 60 · 2022: 52 · 2021: 16 · 2020: 7
FWCI
13.13
Type
journal article
Language
en
Open access
Licence
Locations
  • IEEE Access (publishedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
Konkuk University, in 2019
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
  • Physical Sciences › Computer Science › Signal Processing › Biometric Identification and Security
  • Physical Sciences › Mathematics › Statistics and Probability › Benford’s Law and Fraud Detection
Keywords
deepfake detection, generative adversarial networks, video integrity verification, heuristic detection

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review