Gliding and saccadic gaze gesture recognition in real time
The DOI matches its Crossref record; 34 of the 34 references it cites were found in a registry or an index.
Abstract
Eye movements can be consciously controlled by humans to the extent of performing sequences of predefined movement patterns, or 'gaze gestures'. Gaze gestures can be tracked noninvasively employing a video-based eye tracking system. Gaze gestures hold the potential to become an emerging input paradigm in the context of human-computer interaction (HCI) as low-cost eye trackers become more ubiquitous. The viability of gaze gestures as an innovative way to control a computer rests on how easily they can be assimilated by potential users and also on the ability of machine learning algorithms to discriminate in real time intentional gaze gestures from typical gaze activity performed during standard interaction with electronic devices. In this work, through a set of experiments and user studies, we evaluate the performance of two different gaze gestures modalities, gliding gaze gestures and saccadic gaze gestures, and their corresponding real-time recognition algorithms, Hierarchical Temporal Memory networks and the Needleman-Wunsch algorithm for sequence alignment. Our results show that a specific combination of gaze gesture modality, namely saccadic gaze gestures, and recognition algorithm, Needleman-Wunsch, allows for reliable usage of intentional gaze gestures to interact with a computer with accuracy rates higher than 95% and completion speeds of around 1.5 to 2.5 seconds per gesture. The optimal gaze gesture modality and recognition algorithm do not interfere with otherwise standard human-computer gaze interaction, generating very few false positives during real time recognition and positive feedback from the users. These encouraging results and the low cost eye tracking equipment used, open up a new HCI paradigm for the fields of accessibility and interaction with smartphones, tablets, projected displays and traditional desktop computers.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“In addition, eye-control research has focused on young individuals (Rozado et al., 2012).”
From The impact of usage experience and input modality on trust experience and cognitive load in older adults (2025) · cites this paper as Rozado et al., 2012
Citations
Cites
Proceedings of IEEE Conference on Computer Vision and Pattern Recognition
(1993) · doi:10.1109/cvpr.1993 · Cited by 17,667
Needleman, S. B., & Wunsch, C. D. (1970) · Journal of Molecular Biology · doi:10.1016/0022-2836(70)90057-4 · Cited by 11,288 · cited 2×
“The Needleman-Wunsch algorithm for sequence alignment [Needleman and Wunsch 1970] is an NP-hard algorithm based on dynamic programming.”
MethodEye Tracking Methodology: Theory and Practice
Duchowski, A. T. (2003) · Springer eBooks · doi:10.1007/978-1-4471-3750-4 · Cited by 2,242
The use of eye movements in human-computer interaction techniques: what you look at is what you get
Jacob, R. J. K. (1991) · ACM Transactions on Information Systems · doi:10.1145/123078.128728 · Cited by 845
The mechanics of human smooth pursuit eye movement.
Robinson, D. A. (1965) · The Journal of Physiology · doi:10.1113/jphysiol.1965.sp007718 · Cited by 602
Towards a mathematical theory of cortical micro-circuits
George, D., & Hawkins, J. (2009) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000532 · Cited by 404
Latent-Dynamic Discriminative Models for Continuous Gesture Recognition
Morency, L.-P., Quattoni, A., & Darrell, T. (2007) · 2007 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2007.383299 · Cited by 374
Fast hands-free writing by gaze direction
Ward, D. J., & MacKay, D. J. C. (2002) · Nature · doi:10.1038/418838a · Cited by 329
A hierarchical bayesian model of invariant pattern recognition in the visual cortex
George, D., & Hawkins, J. (2006) · Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005. · doi:10.1109/ijcnn.2005.1556155 · Cited by 223
openEyes: a low-cost head-mounted eye-tracking solution
Li, D., Babcock, J., & Parkhurst, D. J. (2006) · Proceedings of the 2006 symposium on Eye tracking research & applications - ETRA '06 · doi:10.1145/1117309.1117350 · Cited by 218
Fast Hands-free Writing by Gaze Direction
Ward, D. J., & Mackay, D. (2002) · PubMed · doi:10.48550/arxiv.cs/0204030 · Cited by 214
Real Time Eye Tracking and Blink Detection with USB Cameras
Chau, M. N. Y., & Betke, M. (2005) · Cited by 198
Interacting with the Computer Using Gaze Gestures
Drewes, H., & Schmidt, A. (2007) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-74800-7_43 · Cited by 176
Evaluation of a low-cost open-source gaze tracker
San Agustin, J., Skovsgaard, H., Mollenbach, E., Barret, M., Tall, M., Hansen, D. W., & Hansen, J. P. (2010) · Proceedings of the 2010 Symposium on Eye-Tracking Research & Applications - ETRA '10 · doi:10.1145/1743666.1743685 · Cited by 164
Self-citationLongitudinal evaluation of discrete consecutive gaze gestures for text entry
Wobbrock, J. O., Rubinstein, J., Sawyer, M. W., & Duchowski, A. T. (2008) · Proceedings of the 2008 symposium on Eye tracking research & applications - ETRA '08 · doi:10.1145/1344471.1344475 · Cited by 162
Conversing with the user based on eye-gaze patterns
Qvarfordt, P., & Zhai, S. (2005) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1054972.1055004 · Cited by 160
Eye Gaze Tracking for Human Computer Interaction
Drewes, H. (2010) · Electronic Theses of LMU Munich (Ludwig-Maximilians-Universität München) · doi:10.5282/edoc.11591 · Cited by 140
Gesture-based interaction and communication: automated classification of hand gesture contours
Gupta, L., & Ma, S. (2001) · IEEE Transactions on Systems Man and Cybernetics Part C (Applications and Reviews) · doi:10.1109/5326.923274 · Cited by 126
Text input methods for eye trackers using off-screen targets
Isokoski, P. (2000) · Proceedings of the symposium on Eye tracking research & applications - ETRA '00 · doi:10.1145/355017.355020 · Cited by 104
Designing gaze gestures for gaming
Istance, H. O., Hyrskykari, A., Immonen, L., Mansikkamaa, S., & Vickers, S. (2010) · ACM Symposium on Eye Tracking Research and Applications (ETRA) · doi:10.1145/1743666.1743740 · Cited by 101
Writing with Your Eye: A Dwell Time Free Writing System Adapted to the Nature of Human Eye Gaze
Bee, N., & André, E. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-69369-7_13 · Cited by 96
It's in your eyes: towards context-awareness and mobile HCI using wearable EOG goggles
Bulling, A., Roggen, D., & Tröster, G. (2008) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/1409635.1409647 · Cited by 95
New technological windows into mind: there is more in eyes and brains for human-computer interaction
Velichkovsky, B. M., & Hansen, J. P. (1996) · Proceedings of the SIGCHI conference on Human factors in computing systems common ground - CHI '96 · doi:10.1145/238386.238619 · Cited by 70
Enhanced interactive gaming by blending full-body tracking and gesture animation
Bleiweiss, A., Eshar, D., Kutliroff, G., Lerner, A., Oshrat, Y., & Yanai, Y. (2010) · ACM SIGGRAPH ASIA 2010 Sketches · doi:10.1145/1899950.1899984 · Cited by 64
Bernhaupt, R., Dalvi, G., Joshi, A. N., Balkrishnan, D. K., O’Neill, J., & Winckler, M. A. A. (2017) · HAL · Cited by 64
Møllenbach, E., Lillholm, M., Gail, A., & Hansen, J. P. (2010) · ACM Symposium on Eye Tracking Research and Applications (ETRA) · doi:10.1145/1743666.1743710 · Cited by 45
Fixation Precision in High-Speed Noncontact Eye-Gaze Tracking
Hennessey, C., Noureddin, B., & Lawrence, P. (2008) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2007.911378 · Cited by 43
Simple reaction time with very small imperative- stimulus probabilities
Näätänen, R., & Koskinen, P. (1975) · Acta Psychologica · doi:10.1016/0001-6918(75)90020-7 · Cited by 34
Not Typing but Writing: Eye-based Text Entry Using Letter-like Gestures
Wobbrock, J. O., Rubinstein, J., Sawyer, M. W., & Duchowski, A. T. (2007) · Cited by 32
Møllenbach, E., Hansen, J. P., Lillholm, M., & Gale, A. G. (2009) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/1520340.1520699 · Cited by 31
Off-the-Shelf Gaze Interaction
Agustin, J. S. (2010) · IT University Of Copenhagen (IT University of Copenhagen) · Cited by 23
Self-citationOptimizing Hierarchical Temporal Memory for Multivariable Time Series
Rozado, D., Rodrı́guez, F. B., & Varona, P. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15822-3_62 · Cited by 20
Self-citationGaze Gesture Recognition with Hierarchical Temporal Memory Networks
Rozado, D., Rodrı́guez, F. B., & Varona, P. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-21501-8_1 · Cited by 12
Self-citationSelection strategies in gaze interaction
Møllenbach, E. (2010) · Loughborough University Institutional Repository (Loughborough University) · Cited by 6
Research works printed but not linked: 2
- Holden , E. , Owens , R. , and Geoffrey , G . 1999. Adaptive fuzzy expert system for sign recognition . In Proceedings of the International Conference on Signal and Image Processing (SIP'00) . Holden, E., Owens, R., and Geoffrey, G. 1999. Adaptive fuzzy expert system for sign recognition. In Proceedings of the International Conference on Signal and Image Processing (SIP'00).
- Mackay , D. 2003 . Dasher - an efficient keyboard alternative . Adv. Clin. Newr. Rehab. 3 , 2 , 24. Mackay, D. 2003. Dasher - an efficient keyboard alternative. Adv. Clin. Newr. Rehab. 3, 2, 24.
Cited by
Toward Everyday Gaze Input: Accuracy and Precision of Eye Tracking and Implications for Design
Feit, A. M., Williams, S., Toledo, A., Paradiso, A., Kulkarni, H., Kane, S., & Morris, M. R. (2017) · Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025599 · Cited by 234
Eye-tracker algorithms to detect saccades during static and dynamic tasks: a structured review
Stuart, S., Hickey, A., Vitório, R., Welman, K. E., Foo, S. W. L., Keen, D., & Godfrey, A. (2019) · Physiological Measurement · doi:10.1088/1361-6579/ab02ab · Cited by 58
SMOOVS: Towards calibration-free text entry by gaze using smooth pursuit movements
Lutz, O. H.-M., Venjakob, A. C., & Ruff, S. (2015) · Journal of Eye Movement Research · doi:10.16910/jemr.8.1.2 · Cited by 50
Controlling a Smartphone Using Gaze Gestures as the Input Mechanism
Rozado, D., moreno, T. B., Agustin, J. S., Rodrı́guez, F. B., & Varona, P. (2013) · Human-Computer Interaction · doi:10.1080/07370024.2013.870385 · Cited by 44
Self-citationAn End-to-End Review of Gaze Estimation and its Interactive Applications on Handheld Mobile Devices
Lei, Y., He, S., Khamis, M., & Ye, J. (2023) · ACM Computing Surveys · doi:10.1145/3606947 · Cited by 60
Metadata
- Authors
- Rozado, D., Agustin, J. S., Rodriguez, F. B., & Varona, P.
- Affiliations
- Universidad Autónoma de Madrid (ES)
- IT University of Copenhagen (DK)
- Universidad Autónoma de Madrid, Spain
- Venue
- ACM Transactions on Interactive Intelligent Systems
- Published
- 2012-01
- Volume
- 1
- Issue
- 2
- Pages
- 1-27
- ISSN
- 2160-6455 (ISSN-L)
- 2160-6463
- OpenAlex
- W2006567158
- Cited by
- 32
- References
- 33
- Citations by year
- 2026: 1 · 2025: 2 · 2024: 3 · 2023: 4 · 2022: 1 · 2021: 1 · 2019: 5 · 2017: 4 · 2016: 4 · 2015: 5 · 2013: 1
- FWCI
- 3.53
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- ACM Transactions on Interactive Intelligent Systems (publishedVersion)
- IT University Of Copenhagen (IT University of Copenhagen) (submittedVersion)
- Funders
- European Social Fund (MICINN BFU2009-08473 and TIN2010-19607)
- Topics
- Physical Sciences › Computer Science › Human-Computer Interaction › Gaze Tracking and Assistive Technology
- Physical Sciences › Computer Science › Human-Computer Interaction › Interactive and Immersive Displays
- Physical Sciences › Computer Science › Human-Computer Interaction › Hand Gesture Recognition Systems
- Keywords
- gaze gestures, real-time recognition, video-based eye tracking, hierarchical temporal memory, Needleman-Wunsch algorithm, sequence alignment, accessibility, smartphone interaction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.