Look into my Eyes: Using Pupil Dilation to Estimate Mental Workload for Task Complexity Adaptation
The DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
Abstract
Cognition-aware systems acquire physiological data to derive implications about physical and mental states. Pupil dilation has recently attracted attention in the HCI community as an indicator for mental workload. The impact of mental workload on pupillary behavior has been extensively examined. However, systems making use of these measurements to alleviate mental workload have been scarcely evaluated. Our work investigates the expediency of task complexity adaption based on pupillary data in real-time. By conducting math tasks with different complexities, we calibrate a complexity adjustment system. In a pilot study (N=6), we evaluate the feasibility of changing task complexity using two different complexities. Our findings show less perceived mental workload during task complexity adaptation compared to presenting high task complexities only. We show the potential of pupil dilation as a valid metric for assessing mental workload as a modality for cognition-aware user interfaces.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research
(1988) · Advances in Psychology · doi:10.1016/s0166-4115(08)62386-9 · Cited by 15,003 · cited 4×
“This is supported by averaged pupil diameter mea-Figure surements and subjective feedback provided through NASA-TLX questionnaires.”
MethodInfluentialNASA-Task Load Index (NASA-TLX); 20 Years Later
Hart, S. G. (2006) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/154193120605000909 · Cited by 4,297 · cited 1×
“To specify the complexity of experimental conditions, questionnaires such as the NASA-TLX [7, 8] or the Driver Activity Load Index [13], are used to derive cognitive implications which depend on subjective perceptions of participants.”
MethodA tutorial on human activity recognition using body-worn inertial sensors
Bulling, A., Blanke, U., & Schiele, B. (2014) · ACM Computing Surveys · doi:10.1145/2499621 · Cited by 1,668 · cited 1×
“Sensing emotional states has attracted attention as a modality for context-aware computing and has been researched thoroughly in the domain of affective computing [4, 15].”
BackgroundPupil Diameter and Load on Memory
Kahneman, D., & Beatty, J. (1966) · Science · doi:10.1126/science.154.3756.1583 · Cited by 1,617 · cited 1×
“Demanding the short-term memory frequently over a timespan causes cognitive effort [1, 2, 9, 10, 11], a mental process which extends the pupil diameter (see Figure 2).”
BackgroundThe relationships among working memory, math anxiety, and performance
Ashcraft, M. H., & Kirk, E. P. (2001) · Journal of Experimental Psychology: General · doi:10.1037/0096-3445.130.2.224 · Cited by 1,445 · cited 2×
“Demanding the short-term memory frequently over a timespan causes cognitive effort [1, 2, 9, 10, 11], a mental process which extends the pupil diameter (see Figure 2).”
BackgroundPupil Size in Relation to Mental Activity during Simple Problem-Solving
Hess, E. H., & Polt, J. M. (1964) · Science · doi:10.1126/science.143.3611.1190 · Cited by 1,169 · cited 2×
“Demanding the short-term memory frequently over a timespan causes cognitive effort [1, 2, 9, 10, 11], a mental process which extends the pupil diameter (see Figure 2).”
BackgroundWorking memory, math performance, and math anxiety
Ashcraft, M. H., & Krause, J. A. (2007) · Psychonomic Bulletin & Review · doi:10.3758/bf03194059 · Cited by 802 · cited 2×
“Demanding the short-term memory frequently over a timespan causes cognitive effort [1, 2, 9, 10, 11], a mental process which extends the pupil diameter (see Figure 2).”
BackgroundTao, J., & Tan, T. (2005) · Lecture Notes in Computer Science · doi:10.1007/11573548_125 · Cited by 494 · cited 1×
“Sensing emotional states has attracted attention as a modality for context-aware computing and has been researched thoroughly in the domain of affective computing [4, 15].”
BackgroundA method to assess the driver mental workload: The driving activity load index (DALI)
Pauzié, A. (2008) · IET Intelligent Transport Systems · doi:10.1049/iet-its:20080023 · Cited by 246 · cited 1×
“To specify the complexity of experimental conditions, questionnaires such as the NASA-TLX [7, 8] or the Driver Activity Load Index [13], are used to derive cognitive implications which depend on subjective perceptions of participants.”
MethodA Model Relating Pupil Diameter to Mental Workload and Lighting Conditions
Pfleging, B., Fekety, D. K., Schmidt, A., & Kun, A. L. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858117 · Cited by 179 · cited 1×
“Since pupillary measurements are error prone to lighting conditions, Pfleging et al. [14] proposed a model for classifying cognitive workload of pupil dilation under different lighting conditions.”
BackgroundSelf-citation
Research works printed but not linked: 1
- Simone Benedetto, Marco Pedrotti, Luca Minin, Thierry Baccino, Alessandra Re, and Roberto Montanari. 2011. Driver workload and eye blink duration. Transportation research part F: traffic psychology and behaviour 14, 3 (2011), 199--208.
Cited by
A Survey on Measuring Cognitive Workload in Human-Computer Interaction
Kosch, T., Karolus, J., Zagermann, J., Reiterer, H., Schmidt, A., & Woźniak, P. W. (2023) · ACM Computing Surveys · doi:10.1145/3582272 · Cited by 304
Self-citationSurvey and Synthesis of State of the Art in Driver Monitoring
Halin, A., Verly, J., & Droogenbroeck, M. V. (2021) · Sensors · doi:10.3390/s21165558 · Cited by 107
“The pupil diameter is considered to be an indicator of mental workload (61; 105; 173).”
Integrating Eye-Tracking to Augmented Reality System for Surgical Training
Lu, S., Sanchez, Y. P., Jiang, X., & Zheng, B. (2020) · Journal of Medical Systems · doi:10.1007/s10916-020-01656-w · Cited by 62
“Eye-tracking technology provides useful pupillary and eye motion data, revealing cognitive information including humanmental workload [15] and attention shift [25].”
BackgroundVirtual Reality Adaptation Using Electrodermal Activity to Support the User Experience
Chiossi, F., Welsch, R., Villa, S., Chuang, L. L., & Mayer, S. (2022) · Big Data and Cognitive Computing · doi:10.3390/bdcc6020055 · Cited by 58
“For instance, when we perform cognitively demanding tasks, the physiological arousal is heightened, manifesting in increased physiological activity such as skin conductance level (SCL) [12], heart rate [13], or pupil dilation [14].”
BackgroundStress generation and non-intrusive measurement in virtual environments using eye tracking
Hirt, C., Eckard, M., & Kunz, A. M. (2020) · Journal of Ambient Intelligence and Humanized Computing · doi:10.1007/s12652-020-01845-y · Cited by 52
Objective Measures of Cognitive Load Using Deep Multi-Modal Learning: A Use-Case in Aviation
Wilson, J. C., Nair, S., Scielzo, S., & Larson, E. C. (2021) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3448111 · Cited by 52
“Until now no one has shown significant classification accuracy across all subjects [1, 4, 17, 31, 41, 57, 64, 78].”
BackgroundKosch, T., Funk, M., Schmidt, A., & Chuang, L. L. (2018) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3229093 · Cited by 46
“Therefore, physiological methods are increasingly employed as a means to estimate mental states, such as cognitive workload [34, 45, 47].”
MethodSelf-citationNotiBike: Assessing Target Selection Techniques for Cyclist Notifications in Augmented Reality
Kosch, T., Matviienko, A., Müller, F., Bersch, J., Katins, C., Schön, D., & Mühlhäuser, M. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3546732 · Cited by 47
“Further factors, including emotion measurements [5, 39, 41], cognitive assessments through gaze [38, 40] and cortical behavior [37, 42, 43] will be investigated for adjusting Dwell Times dynamically in real-time.”
BackgroundSelf-citationInvestigating Methods for Cognitive Workload Estimation for Assistive Robots
Aygun, A., Nguyen, T. D., Haga, Z., Aeron, S., & Scheutz, M. J. (2022) · Sensors · doi:10.3390/s22186834 · Cited by 40
“Additionally, some recent works evaluated eye gaze parameters for workload prediction in real-time applications [37,38].”
Enhanced cognitive workload evaluation in 3D immersive environments with TOPSIS model
Wang, Y., Chardonnet, J.-R., & Merienne, F. (2020) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2020.102572 · Cited by 32
“Pupil dilation has recently been used as an indicator for mental workload in the HCI community since eye gaze signals indicate a user ’ s cognitive state (Kosch et al., 2018).”
MethodA Survey of Augmented Piano Prototypes: Has Augmentation Improved Learning Experiences?
Deja, J. A., Mayer, S., Pucihar, K. Č., & Kljun, M. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3567719 · Cited by 30
“In the next section, we provide recommendations for future development, implementations, and research of the augmented piano prototypes.”
BackgroundCognitive Workload Assessment via Eye Gaze and EEG in an Interactive Multi-Modal Driving Task
Aygun, A., Lyu, B., Nguyen, T. D., Haga, Z., Aeron, S., & Scheutz, M. J. (2022) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3536221.3556610 · Cited by 33
“There have been relatively few studies that utilize gaze parameters for real-time workload prediction [5, 33].”
BackgroundEffects of Neuro-Cognitive Load on Learning Transfer Using a Virtual Reality-Based Driving System
Abdurrahman, U. A., Yeh, S.-C., Wong, Y., & Wei, L. (2021) · Big Data and Cognitive Computing · doi:10.3390/bdcc5040054 · Cited by 24
John, A. R., Singh, A. K., Do, T., Eidels, A., Nalivaiko, E., Gavgani, A. M., Brown, S. D., Bennett, M. S., Lal, S., Simpson, A. M., et al. (2022) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2022.3157446 · Cited by 27
Chiossi, F., Gruenefeld, U., Hou, B. J., Newn, J., Ou, C., Liao, R., Welsch, R., & Mayer, S. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3676528 · Cited by 31
Emotions on the Go: Mobile Emotion Assessment in Real-Time using Facial Expressions
Kosch, T., Hassib, M. E., Reutter, R., & Alt, F. (2020) · International Conference on Advanced Visual Interfaces (AVI) · doi:10.1145/3399715.3399928 · Cited by 32
“We will further utilize other emerging physiological measures to extend the contextual space of emotion classification in-the-wild, such as eye tracking [29, 30] and brain-computer interfaces [32] for workload assessments as well as electrodermal activity for arousal measurements [2, 31].”
MethodSelf-citationAutomated Filtering of Eye Movements Using Dynamic AOI in Multiple Granularity Levels
Jayawardena, G., & Jayarathna, S. (2021) · International Journal of Multimedia Data Engineering and Management · doi:10.4018/ijmdem.2021010104 · Cited by 17
Grootjen, J. W., Thalhammer, P., & Kosch, T. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3676531 · Cited by 14
Self-citationAbdrabou, Y., Abdelrahman, Y., Khamis, M., & Alt, F. (2021) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3411763.3451636 · Cited by 16
“Pupil dilation was found to be higher for more challenging tasks [11, 26].”
BackgroundCognitive Load during Eye-typing
Bafna, T., Hansen, J. P., & Bækgaard, P. (2020) · ACM Symposium on Eye Tracking Research and Applications (ETRA) · doi:10.1145/3379155.3391333 · Cited by 16
“Blink duration, also an indicator for cognitive load, reduces in length with increasing task difficulty [Bækgaard et al. 2016; Kosch et al. 2018].”
BackgroundReal-Time Workload Estimation Using Eye Tracking: A Bayesian Inference Approach
Luo, R., Weng, Y., Jayakumar, P., Brudnak, M., Paul, V., Desaraju, V. R., Stein, J. L., Ersal, T., & Yang, X. J. (2023) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2023.2205274 · Cited by 14
“For example, previous studies showed that SVMs could be used with human pupil dilation (Kosch, Hassib, Buschek, & Schmidt, 2018) and fixation features (i.e., fixation duration) (Liang et al., 2007) to estimate human workload.”
BackgroundMethodInfluentialRAEMAP: Real-Time Advanced Eye Movements Analysis Pipeline
Jayawardena, G. (2020) · ACM Symposium on Eye Tracking Research and Applications (ETRA) · doi:10.1145/3379157.3391992 · Cited by 15
“Studies have shown that the size of the pupil diameter correlates to the task complexity [Kosch et al. 2018], thus enabling the use of features extracted from eye movements and pupillary behavior as bio-markers of mental workload when completing a task.”
BackgroundMaggi, P., & Nocera, F. D. (2021) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2021.642535 · Cited by 10
“Pupil Diameter Since the seminal studies by Beatty and Kahneman (1966), several authors have shown that pupil dilation may be related to cognitive processing and to the mental effort required to perform a given task (Othman and Romli, 2016; Kosch et al., 2018; van der Wel and van Steenbergen, 2018).”
BackgroundAttallah, O., Mamdouh, M., & Al-Kabbany, A. (2025) · AI · doi:10.3390/ai6040079 · Cited by 10
Lagunes-Ramírez, D. A., González, G., López-Sánchez, M., Olivia, F.-D., Castro-Sánchez, N. A., & Olivares-Rojas, J. C. (2020) · IEEE International Autumn Meeting on Power, Electronics and Computing (ROPEC) · doi:10.1109/ropec50909.2020.9258693 · Cited by 11
“…around eyes behavior correlate different movements and eye states to changes in emotional stimuli(images, video and audio) [3], [4], [5], [6] and cognitive tasks, these includes math resolution [7], playing videogames [8], reading [9], vigilance [10] and pattern identification [11], [12], [13].”
BackgroundHuman-centered Behavioral and Physiological Security
Alt, F., Hassib, M. E., & Distler, V. (2023) · New Security Paradigms Workshop · doi:10.1145/3633500.3633504 · Cited by 12
“Prior research has shown all these eye features can be used to predict cognitive load [53–55].”
BackgroundSelf-citationBläsing, D., & Bornewasser, M. (2019) · Advances in intelligent systems and computing · doi:10.1007/978-3-030-11051-2_2 · Cited by 9
A task-independent design and development process for cognitive products in industrial applications
Murauer, M., Jungwirth, F., Anzengruber, B., Abbas, A., Ahmad, A., Azadi, B., Cho, J., Ennsbrunner, H., Ferscha, A., Gerhard, D., et al. (2019) · International Conference on PErvasive Technologies Related to Assistive Environments (PETRA) · doi:10.1145/3316782.3322748 · Cited by 9
“The examination of gaze behavior [7, 8] enables to build cognition systems [9], that adjust to cognitive processes [30, 36], task-evoked cognitive workload, perceived task difficulty or skill levels [20, 22, 41, 65].”
BackgroundAutomated Filtering of Eye Gaze Metrics from Dynamic Areas of Interest
Jayawardena, G., & Jayarathna, S. (2020) · IEEE International Conference on Information Reuse and Integration (IRI) · doi:10.1109/iri49571.2020.00018 · Cited by 9
“Studies have shown that the size of the pupil diameter correlates to the task complexity [7] enabling the use of pupillary behavior as bio-markers of mental workload when completing a task.”
BackgroundSchneegass, C., Kosch, T., Schmidt, A., & Hußmann, H. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-29387-1_17 · Cited by 11
Self-citationInfluence of Complexity and Noise on Mental Workload During a Manual Assembly Task
Bläsing, D., & Bornewasser, M. (2020) · Communications in computer and information science · doi:10.1007/978-3-030-62302-9_10 · Cited by 11
Towards smarter workplaces: integrating real-time cognitive feedback in collaborative robotics
Boschetti, G., Calzavara, M., Faccio, M., Granata, I., Sinico, T., & Trevisani, A. (2025) · International Journal of Production Research · doi:10.1080/00207543.2025.2561188 · Cited by 5
“Similarly, Kosch et al. (2018) highlighted the potential of real-time pupil dilation monitoring to adapt task complexity and workload in collaborative environments.”
Scherger, A.-L., Urbanczik, G., Ludwigs, T., & Kizilirmak, J. M. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.717379 · Cited by 7
Workload-Aware Systems and Interfaces for Cognitive Augmentation
Kosch, T. (2020) · Elektronische Hochschulschriften der LMU München (Ludwig-Maximilians-Universität München) · doi:10.48550/arxiv.2010.07703 · Cited by 4
“Therefore, physiological methods are increasingly employed as a means to estimate mental states, such as cognitive workload [156, 223, 226].”
MethodSelf-citationmobEYEle: an embedded eye tracking platform for industrial assistance
Jungwirth, F., Murauer, M., Selymes, J., Haslgrübler, M., Gollan, B., & Ferscha, A. (2019) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/3341162.3350842 · Cited by 2
Review on Prevailing Difficulties Using IriS Recognition
Divya, C. D., & Rajendra, A. B. (2020) · Advances in intelligent systems and computing · doi:10.1007/978-3-030-37218-7_74 · Cited by 1
Enhancing Marine Navigation Performance Using the Head-Up Interface
Singh, A. K., Zhou, J., Lin, C., Lal, S., Eidels, A., Jiang, X., & Brown, S. D. (2024) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc54092.2024.10831001 · Cited by 1
Scherger, A.-L., Urbanczik, G., Ludwigs, T., & Kizilirmak, J. M. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.790449
Throndsen, T. U., & Anmarkrud, Ø. (2026) · Frontiers in Education · doi:10.31219/osf.io/n2bze
Doumanis, I., & Economou, D. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-73990-3_6
Baber, C., & Cassenti, D. N. (2025) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/10711813251370728
“Variation in pupil size can indicate workload levels (Ma et al., 2024) which can support adaptative user interfaces (Kosch et al., 2018).”
Wang, Y., & Chardonnet, J.-R. (2026) · Springer eBooks · doi:10.1007/978-3-032-03296-6_2
A Platform for Physiological and Behavioral Security
Dietz, F., Heubl, P., Haliburton, L., Bothe, D., Hörnemann, J., Sasse, M. A., & Alt, F. (2025) · New Security Paradigms Workshop · doi:10.1145/3774761.3774915
From Scores to Strategies: Towards Gaze-Informed Diagnostic Assessment for Visualization LiteracyPreprint
Schnizer, K. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2603.21898
“Task difficulty has been shown to systematically modulate pupil size [22], a signal that has been lever-aged to adapt system behavior in real time [10, 15].”
Brazzolotto, P. (2019) · theses.fr (ABES) · doi:10.70675/6be88775z3a53z4c21z8f36z87023aad5089
Navigation intelligente dans des environnements virtuels
Wang, Y. (2021) · theses.fr (ABES) · doi:10.70675/0b65bda7z46bez4616za661z766b86b14a9e
Watanabe, K., Pol, P., Großmann, N., Ishimaru, S., & Dengel, A. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2604.23562
“Due to these exciting observations, eye-tracking has been used to detect fatigue [2, 65], mental workload [18, 32], task difficulty [4, 10, 14], mind wandering [7, 22], lie detection [19, 36, 40], and other cognitive activity recognition.”
Beyond Overload: Assessing Cognitive Load to Facilitate Learning Transfer in Virtual Environments
ABDURRAHMAN, U. A., ROGO, A. A., Bichi, A. A., & Abdullahi, A. M. (2025) · FUDMA Journal of Engineering and Technology · doi:10.33003/8n3xzk56
“In addition, pupil diameter, a well-established cognitive load metric within Human-Computer Interaction (HCI), has been shown to increase in response to heightened cognitive demand, providing an index of task difficulty and mental workload [9, 2].”
Metadata
- Authors
- Kosch, T., Hassib, M., Buschek, D., & Schmidt, A.
- Affiliations
- Ludwig-Maximilians-Universität München (DE)
- Venue
- Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2018-04-20
- Pages
- 1-6
- OpenAlex
- W2798664662
- Cited by
- 43
- References
- 15
- Citations by year
- 2026: 1 · 2025: 4 · 2024: 4 · 2023: 4 · 2022: 6 · 2021: 7 · 2020: 11 · 2019: 5 · 2018: 1
- FWCI
- 10.74
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) (publishedVersion)
- Funders
- German Federal Ministry of Education and Research (16SV7599K)
- Topics
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Physical Sciences › Computer Science › Human-Computer Interaction › Gaze Tracking and Assistive Technology
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Keywords
- pupil dilation, mental workload, real-time adaptation, physiological signals, user interface design
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.