The DOI matches its Crossref record; 67 of the 67 references it cites were found in a registry or an index.
Your Eyes Tell: Leveraging Smooth Pursuit for Assessing Cognitive Workload
Abstract
A common objective for context-aware computing systems is to predict how user interfaces impact user performance regarding their cognitive capabilities. Existing approaches such as questionnaires or pupil dilation measurements either only allow for subjective assessments or are susceptible to environmental influences and user physiology. We address these challenges by exploiting the fact that cognitive workload influences smooth pursuit eye movements. We compared three trajectories and two speeds under different levels of cognitive workload within a user study (N=20). We found higher deviations of gaze points during smooth pursuit eye movements for specific trajectory types at higher cognitive workload levels. Using an SVM classifier, we predict cognitive workload through smooth pursuit with an accuracy of 99.5% for distinguishing between low and high workload as well as an accuracy of 88.1% for estimating workload between three levels of difficulty. We discuss implications and present use cases of how cognition-aware systems benefit from inferring cognitive workload in real-time by smooth pursuit eye movements.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Data Mining: Practical Machine Learning Tools and Techniques
(2011) · Elsevier eBooks · doi:10.1016/c2009-0-19715-5 · Cited by 25,437
Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research
(1988) · Advances in Psychology · doi:10.1016/s0166-4115(08)62386-9 · Cited by 15,003
NASA-Task Load Index (NASA-TLX); 20 Years Later
Hart, S. G. (2006) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/154193120605000909 · Cited by 4,297 · cited 15×
“After completing a trial, participants were asked to fill out a NASA-TLX questionnaire to assess subjectively perceived cognitive workload.”
MethodInfluentialThe Neurology of Eye Movements
Leigh, R. J., & Zee, D. S. (2015) · Oxford University Press eBooks · doi:10.1093/med/9780199969289.001.0001 · Cited by 3,820
Paced Auditory Serial-Addition Task: A Measure of Recovery from Concussion
Gronwall, D. M. A. (1977) · Perceptual and Motor Skills · doi:10.2466/pms.1977.44.2.367 · Cited by 2,395 · cited 1×
“Tsai et al. [55] investigated how eyes behave under cognitive workload while performing a paced auditory serial addition task [18], however, their work did not investigate the impact on smooth pursuit eye movements.”
BackgroundIs working memory capacity task dependent?
Turner, M. L., & Engle, R. W. (1989) · Journal of Memory and Language · doi:10.1016/0749-596x(89)90040-5 · Cited by 2,299
Eye Tracking Methodology: Theory and Practice
Duchowski, A. T. (2003) · Springer eBooks · doi:10.1007/978-1-4471-3750-4 · Cited by 2,242
Eye fixations and cognitive processes
Just, M. A., & Carpenter, P. A. (1976) · Cognitive Psychology · doi:10.1016/0010-0285(76)90015-3 · Cited by 1,730
Eye Tracking in Human-Computer Interaction and Usability Research
Jacob, R. J. K., & Karn, K. S. (2003) · The Mind's Eye · doi:10.1016/b978-044451020-4/50031-1 · Cited by 1,567
Support vector machine learning for interdependent and structured output spaces
Tsochantaridis, I., Hofmann, T., Joachims, T., & Altun, Y. (2004) · Twenty-first international conference on Machine learning - ICML '04 · doi:10.1145/1015330.1015341 · Cited by 1,243
Pupil Size in Relation to Mental Activity during Simple Problem-Solving
Hess, E. H., & Polt, J. M. (1964) · Science · doi:10.1126/science.143.3611.1190 · Cited by 1,169
A breadth-first survey of eye-tracking applications
Duchowski, A. T. (2002) · Behavior Research Methods, Instruments, & Computers · doi:10.3758/bf03195475 · Cited by 1,061
EEG correlates of task engagement and mental workload in vigilance, learning, and memory tasks.
Berka, C., Levendowski, D. J., Lumicao, M. N., Yau, A., Davis, G., Zivkovic, V. T., Olmstead, R. E., Tremoulet, P. D., & Craven, P. L. (2007) · PubMed · Cited by 1,050
SELECTIVE DISTURBANCE OF MOVEMENT VISION AFTER BILATERAL BRAIN DAMAGE
ZIHL, J., VON CRAMON, D., & MAI, N. (1983) · Brain · doi:10.1093/brain/106.2.313 · Cited by 943
Working memory, attention control, and the N-back task: a question of construct validity
Kane, M. J., Conway, A. R. A., Miura, T. K., & Colflesh, G. J. H. (2007) · Journal of Experimental Psychology: Learning, Memory, and Cognition · doi:10.1037/0278-7393.33.3.615 · Cited by 933 · cited 1×
“Throughout the study, we use the N -back task to manipulate cognitive workload by demanding working memory with different difficulty levels [27].”
MethodNasa-task load index (NASA-TLX); 20 years later: (577632012-009)
Hart, S. G. (2006) · PsycEXTRA Dataset · doi:10.1037/e577632012-009 · Cited by 838
The brainstem control of saccadic eye movements
Sparks, D. L. (2002) · Nature Reviews Neuroscience · doi:10.1038/nrn986 · Cited by 717
What you look at is what you get: eye movement-based interaction techniques
Jacob, R. J. K. (1990) · Proceedings of the SIGCHI conference on Human factors in computing systems Empowering people - CHI '90 · doi:10.1145/97243.97246 · Cited by 712
Eye tracking in HCI and usability research
Poole, A., & Ball, L. J. (2006) · Encyclopedia of Human Computer Interaction · doi:10.4018/978-1-59140-562-7.ch034 · Cited by 690 · cited 1×
“Focusing on usability, related research shows how eye tracking technologies evolved until they became usable in human-computer interaction research [24, 46].”
BackgroundVogel, D., & Balakrishnan, R. (2004) · Proceedings of the 17th annual ACM symposium on User interface software and technology · doi:10.1145/1029632.1029656 · Cited by 660
Human-computer interaction using eye-gaze input
Hutchinson, T. E., White, K. P., Martin, W. N., Reichert, K. C., & Frey, L. A. (1989) · IEEE Transactions on Systems, Man, and Cybernetics · doi:10.1109/21.44068 · Cited by 540
Evaluation of eye gaze interaction
Sibert, L. E., & Jacob, R. J. K. (2000) · Proceedings of the SIGCHI conference on Human Factors in Computing Systems · doi:10.1145/332040.332445 · Cited by 539
Collewijn, H., & Tamminga, E. P. (1984) · The Journal of Physiology · doi:10.1113/jphysiol.1984.sp015242 · Cited by 511
THE NEUROLOGY OF EYE MOVEMENTS
Leigh, R. J., & Zee, D. S. (1984) · Optometry and Vision Science · doi:10.1097/00006324-198402000-00014 · Cited by 498
Sensitivity of eye-movement measures to in-vehicle task difficulty
Victor, T. W., Harbluk, J. L., & Engström, J. A. (2005) · Transportation Research Part F: Traffic Psychology and Behaviour · doi:10.1016/j.trf.2005.04.014 · Cited by 492
Twenty years of eye typing: systems and design issues
Majaranta, P., & Räihä, K.-J. (2002) · Proceedings of the symposium on Eye tracking research & applications - ETRA '02 · doi:10.1145/507072.507076 · Cited by 483
It's Mine, Don't Touch!: interactions at a large multi-touch display in a city centre
Peltonen, P., Kurvinen, E., Salovaara, A., Jacucci, G., Ilmonen, T., Evans, J., Oulasvirta, A., & Saarikko, P. (2008) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1357054.1357255 · Cited by 462
Estimating workload using EEG spectral power and ERPs in the n-back task
Brouwer, A.-M., Hogervorst, M. A., van Erp, J. B. F., Heffelaar, T., Zimmerman, P. H., & Oostenveld, R. (2012) · Journal of Neural Engineering · doi:10.1088/1741-2560/9/4/045008 · Cited by 407
Pupillary Responses During Information Processing Vary with Scholastic Aptitude Test Scores
Ahern, S., & Beatty, J. (1979) · Science · doi:10.1126/science.472746 · Cited by 391
Rapid eye movement sleep behavior disorder. A treatable parasomnia affecting older adults
Schenck, C. H. (1987) · JAMA · doi:10.1001/jama.257.13.1786 · Cited by 330
Non-contact heart rate and heart rate variability measurements: A review
Kranjec, J., Beguš, S., Geršak, G., & Drnovšek, J. (2014) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2014.03.004 · Cited by 318
Using eye movement activity as a correlate of cognitive workload
Ahlstrom, U., & Friedman-Berg, F. J. (2006) · International Journal of Industrial Ergonomics · doi:10.1016/j.ergon.2006.04.002 · Cited by 309
Driver workload and eye blink duration
Benedetto, S., Pedrotti, M., Minin, L., Baccino, T., Re, A., & Montanari, R. (2011) · Transportation Research Part F: Traffic Psychology and Behaviour · doi:10.1016/j.trf.2010.12.001 · Cited by 303
Cognitive processes involved in smooth pursuit eye movements
Barnes, G. R. (2008) · Brain and Cognition · doi:10.1016/j.bandc.2008.08.020 · Cited by 300
Vidal, M., Bulling, A., & Gellersen, H. (2013) · Proceedings of the 2013 ACM international joint conference on Pervasive and ubiquitous computing · doi:10.1145/2493432.2493477 · Cited by 276
Orbits: Gaze Interaction for Smart Watches using Smooth Pursuit Eye Movements
Esteves, A., Velloso, E., Bulling, A., & Gellersen, H. (2015) · Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology · doi:10.1145/2807442.2807499 · Cited by 264
A method to assess the driver mental workload: The driving activity load index (DALI)
Pauzié, A. (2008) · IET Intelligent Transport Systems · doi:10.1049/iet-its:20080023 · Cited by 246
Measuring Cognitive Load using Eye Tracking Technology in Visual Computing
Zagermann, J., Pfeil, U., & Reiterer, H. (2016) · Proceedings of the Sixth Workshop on Beyond Time and Errors on Novel Evaluation Methods for Visualization · doi:10.1145/2993901.2993908 · Cited by 224
Measuring the task-evoked pupillary response with a remote eye tracker
Klingner, J., Kumar, R., & Hanrahan, P. (2008) · Proceedings of the 2008 symposium on Eye tracking research & applications - ETRA '08 · doi:10.1145/1344471.1344489 · Cited by 222
A Model Relating Pupil Diameter to Mental Workload and Lighting Conditions
Pfleging, B., Fekety, D. K., Schmidt, A., & Kun, A. L. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858117 · Cited by 179
Task difficulty in mental arithmetic affects microsaccadic rates and magnitudes
Siegenthaler, E., Costela, F. M., McCamy, M. B., Di Stasi, L. L., Otero‐Millan, J., Sonderegger, A., Groner, R., Macknik, S., & Martinez‐Conde, S. (2014) · European Journal of Neuroscience · doi:10.1111/ejn.12395 · Cited by 166
Pursuit calibration: making gaze calibration less tedious and more flexible
Pfeuffer, K., Vidal, M., Turner, J., Bulling, A., & Gellersen, H. W. (2013) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/2501988.2501998 · Cited by 162
Cognitive Heat: Exploring the Usage of Thermal Imaging to Unobtrusively Estimate Cognitive Load
Abdelrahman, Y., Velloso, E., Dingler, T., Schmidt, A., & Vetere, F. (2017) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3130898 · Cited by 154
Measuring the impact of subtitles on cognitive load: eye tracking and dynamic audiovisual texts
Kruger, J., Hefer, E., & Matthew, G. (2013) · doi:10.1145/2509315.2509331 · Cited by 153
Evaluation of eye-gaze interaction methods for security enhanced PIN-entry
Luca, A. D., Weiss, R., & Drewes, H. (2007) · Australian Conference on Computer-Human Interaction (OzCHI) · doi:10.1145/1324892.1324932 · Cited by 114
Text input methods for eye trackers using off-screen targets
Isokoski, P. (2000) · ACM Symposium on Eye Tracking Research and Applications (ETRA) · doi:10.1145/355017.355020 · Cited by 108
The effect of cognitive load on saccadic eye movements
Stuyven, E., Van der Goten, K., Vandierendonck, A., Claeys, K., & Crevits, L. (2000) · Acta Psychologica · doi:10.1016/s0001-6918(99)00054-2 · Cited by 106
Hammoud, R. I. (2008) · Springer eBooks · doi:10.1007/978-3-540-75412-1 · Cited by 102
Now Dasher! Dash away! longitudinal study of fast text entry by Eye Gaze
Tuisku, O., Majaranta, P., Isokoski, P., & Räihä, K. (2008) · ACM Symposium on Eye Tracking Research and Applications (ETRA) · doi:10.1145/1344471.1344476 · Cited by 101
Writing with Your Eye: A Dwell Time Free Writing System Adapted to the Nature of Human Eye Gaze
Bee, N., & André, E. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-69369-7_13 · Cited by 96
SideWays: a gaze interface for spontaneous interaction with situated displays
Zhang, Y., Bulling, A., & Gellersen, H. W. (2013) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2470654.2470775 · Cited by 90
Bulling, A., & Zander, T. O. (2014) · IEEE Pervasive Computing · doi:10.1109/mprv.2014.42 · Cited by 85
TextPursuits: using text for pursuits-based interaction and calibration on public displays
Khamis, M., Saltuk, O., Hang, A., Stolz, K., Bulling, A., & Alt, F. (2016) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/2971648.2971679 · Cited by 84
Self-citationTypes of Eye Movements and Their Functions
Purves, D., Augustine, G. J., Fitzpatrick, D., Katz, L. C., LaMantia, A., McNamara, J. O., & Williams, S. M. (2001) · Cited by 74
Velloso, E., Wirth, M., Weichel, C., Esteves, A., & Gellersen, H. W. (2016) · Designing Interactive Systems Conference · doi:10.1145/2901790.2901867 · Cited by 74
Surrogate Distraction Measurement Techniques: The Lane Change Test
Mattes, S., & Hallén, A. (2009) · Cited by 70
Features of eye gaze interface for selection tasks
Ohno, T. (2002) · Proceedings. 3rd Asia Pacific Computer Human Interaction (Cat. No.98EX110) · doi:10.1109/apchi.1998.704190 · Cited by 57 · cited 1×
“Recent research has addressed the use of eye gaze for selection tasks [40, 50], which can be used by physiologically impaired individuals to perform input on computers [22].”
BackgroundA field study on spontaneous gaze-based interaction with a public display using pursuits
Khamis, M., Alt, F., & Bulling, A. (2015) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/2800835.2804335 · Cited by 51
Self-citationExploiting Eye Tracking for Smartphone Authentication
Liu, D., Dong, B., Gao, X. H., & Wang, H. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-28166-7_22 · Cited by 51
Thermal Imaging as a Way to Classify Cognitive Workload
Stemberger, J., Allison, R. S., & Schnell, T. (2010) · Canadian Conference on Computer and Robot Vision · doi:10.1109/crv.2010.37 · Cited by 47
Effect of cognitive load on eye-target synchronization during smooth pursuit eye movement
Contreras, R. P., Ghajar, J., Bahar, S., & Suh, M. (2011) · Brain Research · doi:10.1016/j.brainres.2011.05.004 · Cited by 42
Schenk, S., Dreiser, M., Rigoll, G., & Dörr, M. (2017) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025455 · Cited by 33
Driver Cognitive Distraction Detection Using Eye Movements
Liang, Y., & Lee, J. D. (2008) · Springer eBooks · doi:10.1007/978-3-540-75412-1_13 · Cited by 20
Study of pupil diameter and eye movements to enhance flight safety
Peysakhovich, V. (2016) · theses.fr (ABES) · doi:10.70675/1da398f6z1cc7z4530zad0bz9506aa6bc733 · Cited by 16
Cognition-aware systems as mobile personal assistants
Dingler, T. (2016) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/2968219.2968565 · Cited by 12
A Comparison of Smooth Pursuit- and Dwell-based Selection at Multiple Levels of Spatial Accuracy
Lohr, D. J., & Komogortsev, O. V. (2017) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3027063.3053233 · Cited by 11
Peysakhovich, V. (2016) · Thèses en ligne de l'Université Toulouse III (Université Toulouse III) · Cited by 4
Research works printed but not linked: 7
- Alan D. Baddeley and Graham Hitch. 1974. Working Memory. Psychology of Learning and Motivation, Vol. 8. Academic Press, 47 -- 89.
- 10.5555/3134162 · doi:10.5555/3134162
- Bruce Mehler, Bryan Reimer, and JA Dusek. 2011. MIT AgeLab delayed digit recall task (n-back). Cambridge, MA: Massachusetts Institute of Technology (2011).
- 10.5555/786112.786297 · doi:10.5555/786112.786297
- 10.1001/jama.1987.03390130104038 · DOI · Paper page
- Yi-Fang Tsai, Erik Viirre, Christopher Strychacz, Bradley Chase, and Tzyy-Ping Jung. 2007. Task performance and eye activity: predicting behavior relating to cognitive workload. Aviation, space, and environmental medicine 78, Supplement 1 (2007), B176--B185.
- 10.5555/3086818 · doi:10.5555/3086818
Cited by
Review of Eye Tracking Metrics Involved in Emotional and Cognitive Processes
Skaramagkas, V., Giannakakis, G., Ktistakis, E., Manousos, D., Karatzanis, I., Tachos, N., Tripoliti, E., Marias, K., Fotiadis, D. I., & Tsiknakis, M. (2023) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2021.3066072 · Cited by 419
A Survey on Measuring Cognitive Workload in Human-Computer Interaction
Kosch, T., Karolus, J., Zagermann, J., Reiterer, H., Schmidt, A., & Woźniak, P. W. (2023) · ACM Computing Surveys · doi:10.1145/3582272 · Cited by 304
“Previous work has also shown that smooth pursuit eye movements deviate from a given trajectory in situations with a higher cognitive demand [133] while being less prone to changes in light intensity (cf. pupil diameter).”
Self-citationSouchet, A. D., Philippe, S., Lourdeaux, D., & Leroy, L. (2022) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2021.1976509 · Cited by 204
Gaze-based interaction: A 30 year retrospective
Duchowski, A. T. (2018) · Computers & Graphics · doi:10.1016/j.cag.2018.04.002 · Cited by 163
Evaluation of Appearance-Based Methods and Implications for Gaze-Based Applications
Zhang, X., Sugano, Y., & Bulling, A. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300646 · Cited by 156
Metadata
- Authors
- Kosch, T., Hassib, M., Woźniak, P. W., Buschek, D., & Alt, F.
- Affiliations
- Ludwig-Maximilians-Universität München (DE)
- University of Stuttgart (DE)
- Munich University of Applied Sciences (DE)
- Venue
- Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2018-04-21
- Pages
- 1-13
- OpenAlex
- W2796389891
- Cited by
- 79
- References
- 66
- Citations by year
- 2026: 1 · 2025: 10 · 2024: 13 · 2023: 10 · 2022: 8 · 2021: 10 · 2020: 10 · 2019: 10 · 2018: 7
- FWCI
- 7.45
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- Funders
- German Federal Ministry of Education and Research (16SV7599K)
- Topics
- Physical Sciences › Computer Science › Human-Computer Interaction › Gaze Tracking and Assistive Technology
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Life Sciences › Neuroscience › Cognitive Neuroscience › EEG and Brain-Computer Interfaces
- Keywords
- mental workload, smooth pursuit eye movements, gaze tracking, workload estimation, context-aware computing, user interface evaluation, gaze trajectories
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.