Towards an index of opportunity: understanding changes in mental workload during task execution
The DOI matches its Crossref record; 37 of the 41 references it cites were found in a registry or an index.
Abstract
To contribute to systems that reason about human attention, our work empirically demonstrates how a user's mental workload changes during task execution. We conducted a study where users performed interactive, hierarchical tasks while mental workload was measured through the use of pupil size. Results show that (i) different types of subtasks impose different mental workload, (ii) workload decreases at subtask boundaries, (iii) workload decreases more at boundaries higher in a task model and less at boundaries lower in the model, (iv) workload changes among subtask boundaries within the same level of a task model, and (v) effective understanding of why changes in workload occur requires that the measure be tightly coupled to a validated task model. From the results, we show how to map mental workload onto a computational Index of Opportunity that systems can use to better reason about human attention.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“In prior work [8, 9], we developed workload-aligned models for several interactive tasks.”
From Investigating the effectiveness of mental workload as a predictor of opportune moments for interruption (2005) · cites this paper as [8]
In the citing paper
“For example, in the route-planning task in [24], information had to be maintained as a problem state during a subtask, but not between subtasks.”
From What Makes Interruptions Disruptive?: A Process-Model Account of the Effects of the Problem State Bottleneck on Task Interruption and Resumption (2015) · cites this paper as [24]
In the citing paper
“It was shown that this was related to higher workload [24], as well as to higher resumption costs when interrupted during the subtask [25].”
From What Makes Interruptions Disruptive?: A Process-Model Account of the Effects of the Problem State Bottleneck on Task Interruption and Resumption (2015) · cites this paper as [24]
Citations
Cites
Picard, R. W. (1997) · MIT Press · doi:10.7551/mitpress/1140.001.0001 · Cited by 7,382 · cited 1×
“Physiological signals can provide systems with a continuous measure of the state of the user [29], e.”
BackgroundThe Psychology of Human-Computer Interaction
Card, S. K., Moran, T. P., & Newell, A. (2018) · CRC Press eBooks · doi:10.1201/9780203736166 · Cited by 4,103
Agents that reduce work and information overload
Maes, P. (1994) · Communications of the ACM · doi:10.1145/176789.176792 · Cited by 2,321 · cited 1×
“E-mail notifications [18], instant messages [6], agent requests [24], and system alerts all contribute to this burgeoning epidemic of interruption that negatively affects almost every user.”
BackgroundTask-evoked pupillary responses, processing load, and the structure of processing resources.
Beatty, J. (1982) · Psychological Bulletin · doi:10.1037/0033-2909.91.2.276 · Cited by 1,959 · cited 5×
“Though we did not balance for gender, previous research has shown that gender does not influence a user’s pupillary response to mental workload [3].”
BackgroundMethodResultInfluentialUser Centered System Design: New Perspectives on Human-Computer Interaction
Anderson, N. S., Norman, D. A., & Draper, S. W. (1988) · The American Journal of Psychology · doi:10.2307/1422802 · Cited by 1,840
The psychology of human-computer interaction
Huesmann, L. R., Card, S. K., Moran, T. P., & Newell, A. (1984) · The American Journal of Psychology · doi:10.2307/1422176 · Cited by 1,177 · cited 4×
“Interactive tasks, however, are composed of hierarchical patterns of goal formulation, execution, and evaluation, creating many levels of subtask boundaries in a task model [ 5 ].”
BackgroundMethodResultInfluentialPupil Size in Relation to Mental Activity during Simple Problem-Solving
Hess, E. H., & Polt, J. M. (1964) · Science · doi:10.1126/science.143.3611.1190 · Cited by 1,169 · cited 2×
“Research shows that pupil size is a reliable measure of mental workload [3, 11, 22].”
BackgroundMethodSpeier, C., Valacich, J. S., & Vessey, I. (1999) · Decision Sciences · doi:10.1111/j.1540-5915.1999.tb01613.x · Cited by 701 · cited 1×
“When a background application interrupts a user at an inopportune moment during task execution, the user performs tasks slower [2, 29], commits more errors [24], makes worse decisions [34], and experiences more frustration, annoyance, and anxiety [1, 2, 4] than if it had interrupted at a more opportune moment.”
BackgroundThe Lumiere Project: Bayesian User Modeling for Inferring the Goals and Needs of Software Users
Horvitz, E., Breese, J. S., Heckerman, D. E., Hovel, D., & Rommelse, K. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.7385 · Cited by 654 · cited 1×
“The IOP, for example, could serve as an evidence variable in a Bayesian network [14].”
BackgroundIf not now, when?: the effects of interruption at different moments within task execution
Adamczyk, P. D., & Bailey, B. P. (2004) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/985692.985727 · Cited by 531
Self-citationWhat makes interruptions disruptive? A study of length, similarity, and complexity
Gillie, T., & Broadbent, D. (1989) · Psychological Research · doi:10.1007/bf00309260 · Cited by 490 · cited 2×
“Researchers have long argued that opportune moments for interruption occur during periods of low mental workload [2, 7, 10], and posit that these periods occur at subtask boundaries during task execution [30].”
BackgroundResultPhysiological metrics of mental workload: A review of recent progress
Kramer, A. F. (2020) · Multiple-task performance · doi:10.1201/9781003069447-14 · Cited by 459
Temporal factors in mental work: Effects of interrupted activities
Zijlstra, F. R. H., Roe, R. A., Leonora, A. B., & Krediet, I. (1999) · Journal of Occupational and Organizational Psychology · doi:10.1348/096317999166581 · Cited by 429 · cited 1×
“When a background application interrupts a user at an inopportune moment during task execution, the user performs tasks slower [2, 7, 26], commits more errors [23], makes worse decisions [32], and experiences more frustration, annoyance, and anxiety [1, 2, 36] than if it had interrupted at a more opportune moment.”
BackgroundHyönä, J., Tommola, J., & Alaja, A.-M. (1995) · The Quarterly Journal of Experimental Psychology Section A · doi:10.1080/14640749508401407 · Cited by 371
Pupillary dilation as a measure of attention: a quantitative system analysis
Hoeks, B., & Levelt, W. J. M. (1993) · Behavior Research Methods, Instruments, & Computers · doi:10.3758/bf03204445 · Cited by 342 · cited 1×
“Use of Pupil Size to Measure Mental Workload Under conditions of controlled illumination, research shows that pupil size is an effective and reliable measure of mental workload [10, 11, 20, 21, 28, 33], where increases in pupil size correlate with increases in mental workload.”
Method(2001) · Cited by 341
Task-evoked pupillary response to mental workload in human-computer interaction
Iqbal, S. T., Zheng, X. S., & Bailey, B. P. (2004) · CHI '04 Extended Abstracts on Human Factors in Computing Systems · doi:10.1145/985921.986094 · Cited by 317
Self-citationHorvitz, E., Jacobs, A., & Hovel, D. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.6707 · Cited by 311 · cited 2×
“To mitigate the disruptive effects of interruption, researchers are investigating systems that reason about when to interrupt users [13, 15, 16].”
BackgroundPredicting human interruptibility with sensors: a Wizard of Oz feasibility study
Hudson, S., Fogarty, J., Atkeson, C., Avrahami, D., Forlizzi, J., Kiesler, S., Lee, J., & Yang, J. (2003) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/642611.642657 · Cited by 295 · cited 2×
“To mitigate the disruptive effects of interruption, researchers are investigating systems that reason about when to interrupt users [13, 15, 16].”
BackgroundInstant Messaging and Interruption: Influence of Task Type on Performance
Czerwinski, M. P., Cutrell, E., & Horvitz, E. (2000) · Cited by 288
(2001) · Cited by 269
Self-citationLearning and reasoning about interruption
Horvitz, E., & Apacible, J. (2003) · Proceedings of the 5th international conference on Multimodal interfaces · doi:10.1145/958432.958440 · Cited by 267 · cited 3×
“To mitigate the disruptive effects of interruption, researchers are investigating systems that reason about when to interrupt users [13, 15, 16].”
BackgroundPupil Size as a Measure of Arousal during Information Processing
BRADSHAW, J. (1967) · Nature · doi:10.1038/216515a0 · Cited by 205
(2000) · Cited by 192
Heart rate variability: indicator of user state as an aid to human-computer interaction
Rowe, D. W., Sibert, J., & Irwin, D. (1998) · Proceedings of the SIGCHI conference on Human factors in computing systems - CHI '98 · doi:10.1145/274644.274709 · Cited by 192 · cited 1×
“Although researchers have also investigated the use of eye movement [35], blink rate [23], and heart rate variance [32] to approximate mental workload, pupil size offers an immediate quantitative measure, which simplifies analysis of the response data.”
Background(1999) · International Conference on Human-Computer Interaction · Cited by 168
Pupillary responses in a pitch-discrimination task
Hahnemann, D., & Beatty, J. (1967) · Perception & Psychophysics · doi:10.3758/bf03210302 · Cited by 161 · cited 2×
“Research shows that pupil size is a reliable measure of mental workload [3, 11, 22].”
BackgroundMethodThe cost of email interruption
Jackson, T., Dawson, R., & Wilson, D. (2001) · Journal of Systems and Information Technology · doi:10.1108/13287260180000760 · Cited by 161 · cited 1×
“E-mail notifications [19], instant messages [7], agent requests [25], and system alerts all contribute to this burgeoning epidemic of interruption that negatively affects almost every user.”
BackgroundPhysiological Metrics of Mental Workload: A Review of Recent Progress
Kramer, A. F. (1990) · doi:10.21236/ada223701 · Cited by 157
The act of task difficulty and eye-movement frequency for the 'Oculo-motor indices'
Nakayama, M., Takahashi, K., & Shimizu, Y. (2002) · ACM Symposium on Eye Tracking Research and Applications (ETRA) · doi:10.1145/507072.507080 · Cited by 85 · cited 1×
“Use of Pupil Size to Measure Mental Workload Under conditions of controlled illumination, research shows that pupil size is an effective and reliable measure of mental workload [11, 12, 22, 31, 35], where the increase in pupil size correlates with the increase in mental workload.”
MethodInterruption as a test of the user-computer interface
Kreifeldt, J. G., & McCarthy, M. E. (1981) · NASA Technical Reports Server (NASA) · Cited by 78
Interacting with groups of computers
Shell, J. S., Selker, T., & Vertegaal, R. (2003) · Communications of the ACM · doi:10.1145/636772.636796 · Cited by 68 · cited 1×
“We believe that future technology will provide cost-effective, non-intrusive means to effectively measure pupil size, e.g., remote eye trackers built into LCD monitors [36] or even eye glasses [ 33 ], thus justifying our use of pupil size in the present work.”
BackgroundJohn, B. E., Vera, A. H., Matessa, M., Freed, M. C., & Remington, R. W. (2002) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/503376.503404 · Cited by 61 · cited 2×
“This is likely because the cognitive and motor systems may execute in parallel [ 20 ], but with cognitive function preceding motor function.”
BackgroundMethodThe pupillary response to mental overload
Juris, M., & Velden, M. (1977) · Physiological Psychology · doi:10.3758/bf03337847 · Cited by 31 · cited 2×
“Use of Pupil Size to Measure Mental Workload Under conditions of controlled illumination, research shows that pupil size is an effective and reliable measure of mental workload [10, 11, 20, 21, 28, 33], where increases in pupil size correlate with increases in mental workload.”
MethodThe response of eye-movement and pupil size to audio instruction while viewing a moving target
Takahashi, K., Nakayama, M., & Shimizu, Y. (2000) · Proceedings of the symposium on Eye tracking research & applications - ETRA '00 · doi:10.1145/355017.355043 · Cited by 30 · cited 2×
“Although researchers have also investigated the use of eye movement [35], blink rate [23], and heart rate variance [32] to approximate mental workload, pupil size offers an immediate quantitative measure, which simplifies analysis of the response data.”
BackgroundMethodWilson, G. M., & Sasse, M. A. (2000) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/633292.633360 · Cited by 8
New techniques for evaluating innovative interfaces with eye tracking
Marshall, S. P. (2003) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/958432.958434 · Cited by 5 · cited 2×
“UI researchers are already using pupil size to evaluate the mental workload imposed by user interface designs [27].”
Background
Cited by
Bailey, B. P., & Konstan, J. A. (2006) · Computers in Human Behavior · doi:10.1016/j.chb.2005.12.009 · Cited by 600
“If those boundaries could be identified, perhaps by linking physiological measures of workload to models of task execution (Iqbal et al., 2005), effects of interruption similar to the effects produced in this experiment may be achieved.”
BackgroundSelf-citationEstimating cognitive load using remote eye tracking in a driving simulator
Palinko, O., Kun, A. L., Shyrokov, A., & Heeman, P. (2010) · Proceedings of the 2010 Symposium on Eye-Tracking Research & Applications - ETRA '10 · doi:10.1145/1743666.1743701 · Cited by 371
Psycho-physiological measures for assessing cognitive load
Haapalainen, E., Kim, S., Forlizzi, J. F., & Dey, A. K. (2010) · Proceedings of the 12th ACM international conference on Ubiquitous computing · doi:10.1145/1864349.1864395 · Cited by 382
“Earlier work has shown the value of tracking eye movements and changes in pupil size as measures of cognitive load [3, 21, 22 , 52].”
BackgroundMethodInfluentialUsing eye movement activity as a correlate of cognitive workload
Ahlstrom, U., & Friedman-Berg, F. J. (2006) · International Journal of Industrial Ergonomics · doi:10.1016/j.ergon.2006.04.002 · Cited by 309
Bailey, B. P., & Iqbal, S. T. (2008) · ACM Transactions on Computer-Human Interaction · doi:10.1145/1314683.1314689 · Cited by 325
Self-citation
Metadata
- Authors
- Iqbal, S. T., Adamczyk, P. D., Zheng, X. S., & Bailey, B. P.
- Affiliations
- University of Illinois Urbana-Champaign (US)
- Venue
- Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2005-04-02
- Pages
- 311-320
- OpenAlex
- W1966518476
- Cited by
- 196
- References
- 41
- Citations by year
- 2026: 1 · 2025: 3 · 2024: 2 · 2023: 5 · 2022: 3 · 2021: 11 · 2020: 7 · 2019: 12 · 2018: 14 · 2017: 14 · 2016: 12 · 2015: 14 · 2014: 10 · 2013: 10 · 2012: 6
- FWCI
- 21.30
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- Topics
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Physical Sciences › Computer Science › Human-Computer Interaction › Gaze Tracking and Assistive Technology
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Keywords
- mental workload, pupil size, human attention, task modeling, computational model of attention, cognitive load assessment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.