Using pen input features as indices of cognitive load
The DOI matches its Crossref record; 7 of the 7 references it cites were found in a registry or an index.
Abstract
Multimodal interfaces are known to be useful in map-based applications, and in complex, time-pressure based tasks. Cognitive load variations in such tasks have been found to impact multimodal behaviour. For example, users become more multimodal and tend towards semantic complementarity as cognitive load increases. The richness of multimodal data means that systems could monitor particular input features to detect experienced load variations. In this paper, we present our attempt to induce controlled levels of load and solicit natural speech and pen-gesture inputs. In particular, we analyse for these features in the pen gesture modality. Our experimental design relies on a map-based Wizard of Oz, using a tablet PC. This paper details analysis of pen-gesture interaction across subjects, and presents suggestive trends of increases in the degree of degeneration of pen-gestures in some subjects, and possible trends in gesture kinematics, when cognitive load increases.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
The magical number seven, plus or minus two: Some limits on our capacity for processing information.
Miller, G. A. (1994) · Psychological Review · doi:10.1037/0033-295x.101.2.343 · Cited by 17,161
Cognitive Load Theory and Instructional Design: Recent Developments
Paas, F., Renkl, A., & Sweller, J. (2003) · Educational Psychologist · doi:10.1207/s15326985ep3801_1 · Cited by 3,041
Cognitive Load Measurement as a Means to Advance Cognitive Load Theory
Paas, F., Tuovinen, J. E., Tabbers, H., & Van Gerven, P. W. M. (2003) · Educational Psychologist · doi:10.1207/s15326985ep3801_8 · Cited by 2,527 · cited 2×
“They are more suited to discreet sequential tasks, as seen in traditional learning environments, rather than continuous interaction with an interface [5].”
BackgroundSpecifying gestures by example
Rubine, D. (1991) · Proceedings of the 18th annual conference on Computer graphics and interactive techniques · doi:10.1145/122718.122753 · Cited by 778 · cited 2×
“The result is the number of standard deviations a pen-gesture is away from the mean of its classification [6].”
BackgroundMethodWhen do we interact multimodally?: cognitive load and multimodal communication patterns
Oviatt, S., Coulston, R., & Lunsford, R. (2004) · Proceedings of the 6th international conference on Multimodal interfaces · doi:10.1145/1027933.1027957 · Cited by 285 · cited 2×
“For example, increased use of multimodal input for more complex tasks is suggested to be a form of selfmanagement of cognitive load by using more than one modal channel to communicate [4].”
BackgroundInterpreting Symptoms of Cognitive Load in Speech Input
Berthold, A., & Jameson, A. (1999) · Courses and lectures · doi:10.1007/978-3-7091-2490-1_23 · Cited by 102 · cited 1×
“Significant variations in levels of disfluency across a number of modality channels, including gesture and speech, have also been shown in users experiencing low versus high cognitive load [2].”
BackgroundExamining the redundancy of multimodal input
Ruiz, N., Taib, R., & Chen, F. (2006) · Australian Conference on Computer-Human Interaction (OzCHI) · doi:10.1145/1228175.1228254 · Cited by 24 · cited 1×
“Another example of behaviour exhibited at high levels of load is a reduction in the degree of semantic redundancy between input modalities, where users tend to be more semantically concise when tasks become more difficult [7].”
BackgroundSelf-citation
Research works printed but not linked: 1
- Baddeley , A. , Working Memory. Science , 1992 . 255: 556 -- 559 . Baddeley, A., Working Memory. Science, 1992. 255: 556--559.
Cited by
Integrating cognitive load theory and concepts of human–computer interaction
Hollender, N., Hofmann, C., Deneke, M., & Schmitz, B. (2010) · Computers in Human Behavior · doi:10.1016/j.chb.2010.05.031 · Cited by 328
A Survey on Measuring Cognitive Workload in Human-Computer Interaction
Kosch, T., Karolus, J., Zagermann, J., Reiterer, H., Schmidt, A., & Woźniak, P. W. (2023) · ACM Computing Surveys · doi:10.1145/3582272 · Cited by 304
“During handwriting with a stylus, the velocity [211, 260], frequency [210], trajectory duration [209], and pressure of strokes [260] were studied as indicators for cognitive workload.”
Multimodal behavior and interaction as indicators of cognitive load
Chen, F., Ruiz, N., Choi, E., Epps, J., Khawaja, M. A., Taib, R., Yin, B., & Wang, Y. (2012) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/2395123.2395127 · Cited by 141
Self-citationEstimating cognitive workload using wavelet entropy-based features during an arithmetic task
Zarjam, P., Epps, J., Chen, F., & Lovell, N. H. (2013) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2013.08.021 · Cited by 141
Self-citationBeyond Subjective Self-Rating: EEG Signal Classification of Cognitive Workload
Zarjam, P., Epps, J., & Lovell, N. H. (2015) · IEEE Transactions on Autonomous Mental Development · doi:10.1109/tamd.2015.2441960 · Cited by 140
Metadata
- Authors
- Ruiz, N., Taib, R., Shi, Y. (., Choi, E., & Chen, F.
- Affiliations
- UNSW Sydney (AU)
- Data61 (AU)
- Venue
- Proceedings of the 9th international conference on Multimodal interfaces
- Publisher
- ACM
- Published
- 2007-11-12
- Pages
- 315-318
- OpenAlex
- W2017617574
- Cited by
- 34
- References
- 7
- Citations by year
- 2025: 2 · 2024: 1 · 2023: 3 · 2022: 1 · 2020: 2 · 2019: 2 · 2017: 3 · 2016: 3 · 2015: 3 · 2013: 7 · 2012: 1
- FWCI
- 1.18
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Human-Computer Interaction › Interactive and Immersive Displays
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Physical Sciences › Computer Science › Human-Computer Interaction › Gaze Tracking and Assistive Technology
- Keywords
- cognitive load, pen-based input, gesture kinematics, multimodal interfaces, map-based interface, Wizard of Oz, tablets, speech input, time pressure
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.