Using language complexity to measure cognitive load for adaptive interaction design
The DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.
Abstract
An adaptive interaction system, which is aware of the users' current cognitive load, can change its response, presentation and interaction flow to improve users' experience and their task performance. In this paper, we propose a novel speech content analysis approach for measuring users' cognitive load, based on their language and dialogue complexity. We have analysed the transcribed speech of operators working in computerized incident control rooms and involved in highly complex bushfire management tasks in Australia. The resulting patterns of language complexity show significant differences between the speech from cognitively low load and high load tasks. We also discuss the value of using this approach of cognitive load measurement for user interface evaluation and interaction design improvement.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Here, speech features such as the flow of words (e.g., tempo [42] or pauses [117,118]), but also more nuanced features such as the lexical density [115], the spectral domain [259], or the pitch contour [39] while talking were shown to be indicators of changes in participants' cognitive workload.”
From A Survey on Measuring Cognitive Workload in Human-Computer Interaction (2023) · cites this paper as [115]
In the citing paper
“Average word length is known to be associated with higher cognitive (Khawaja, Chen, & Marcus, 2010), conceptual complexity (Lewis & Frank, 2016), education, and social class (Hartley, Pennebaker, & Fox, 2003; Tausczik & Pennebaker, 2010).”
From World Trade Center responders in their own words: predicting PTSD symptom trajectories with AI-based language analyses of interviews (2023) · cites this paper as Khawaja, Chen, & Marcus, 2010
Citations
Cites
Mayer, R. (2020) · doi:10.1017/9781316941355 · Cited by 7,777
Flesch, R. (1948) · Journal of Applied Psychology · doi:10.1037/h0057532 · Cited by 5,391 · cited 1×
“Flesch-Kincaid Grade calculates the language difficulty using average sentence lengths and average syllables per word [25].”
MethodCognitive Load Theory and Instructional Design: Recent Developments
Paas, F., Renkl, A., & Sweller, J. (2003) · Educational Psychologist · doi:10.1207/s15326985ep3801_1 · Cited by 3,041
Cognitive Load Theory and the Format of Instruction
Chandler, P., & Sweller, J. (1991) · Cognition and Instruction · doi:10.1207/s1532690xci0804_2 · Cited by 2,894 · cited 9×
“When a person performs a particular problem-solving task, it imposes a mental load on the person’s working memory, known as Cognitive Load [1].”
BackgroundInfluentialCognitive Load Measurement as a Means to Advance Cognitive Load Theory
Paas, F., Tuovinen, J. E., Tabbers, H., & Van Gerven, P. W. M. (2003) · Educational Psychologist · doi:10.1207/s15326985ep3801_8 · Cited by 2,527
Mayer, R. E. (2001) · doi:10.1017/cbo9781139164603 · Cited by 2,304
SMOG Grading - A New Readability Formula.
McLaughlin, G. H. (1969) · The Journal of Reading · Cited by 2,081
The Technique of Clear Writing.
Gunning, R. C. (1968) · CERN Document Server (European Organization for Nuclear Research) · Cited by 1,641
Word Use in the Poetry of Suicidal and Nonsuicidal Poets
Wiltsey Stirman, S., & Pennebaker, J. W. (2001) · Psychosomatic Medicine · doi:10.1097/00006842-200107000-00001 · Cited by 547
Physiological metrics of mental workload: A review of recent progress
Kramer, A. F. (2020) · Multiple-task performance · doi:10.1201/9781003069447-14 · Cited by 459
Analyzing Cockpit Communications: The Links Between Language, Performance, Error, and Workload
Sexton, J. B., & Helmreich, R. L. (2000) · Journal of Human Performance in Extreme Environments · doi:10.7771/2327-2937.1007 · Cited by 222
THE LEXILE FRAMEWORK AS AN APPROACH FOR READING MEASUREMENT AND SUCCESS
Lennon, C., & BURDICK, H. O. (2004) · Cited by 161
Ark, W., Dryer, D. C., & Lu, D. J. (1999) · International Conference on Human-Computer Interaction · Cited by 139
On the Cognitive Status of Pauses in Discourse Production
Schilperoord, J. (2002) · Studies in writing · doi:10.1007/978-94-010-0468-8_4 · Cited by 120
Interpreting Symptoms of Cognitive Load in Speech Input
Berthold, A., & Jameson, A. (1999) · Courses and lectures · doi:10.1007/978-3-7091-2490-1_23 · Cited by 102
Speech-based cognitive load monitoring system
Yin, B., Chen, F., Ruiz, N., & Ambikairajah, E. (2008) · 2008 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2008.4518041 · Cited by 87
Self-citationRecognizing Time Pressure and Cognitive Load on the Basis of Speech: An Experimental Study
Müller, C. A., Großmann-Hutter, B., Jameson, A., Rummer, R., & Wittig⋆, F. (2001) · Lecture notes in computer science · doi:10.1007/3-540-44566-8_3 · Cited by 86
Using linguistic features to measure presence in computer-mediated communication
Kramer, A. D. I., Oh, L. M., & Fussell, S. R. (2006) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1124772.1124907 · Cited by 51
Exploiting prosodic structuring of coverbal gesticulation
Kettebekov, S. (2004) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/1027933.1027953 · Cited by 28
Potential speech features for cognitive load measurement
Khawaja, M. A., Ruiz, N., & Chen, F. (2007) · Proceedings of the 19th Australasian conference on Computer-Human Interaction: Entertaining User Interfaces · doi:10.1145/1324892.1324902 · Cited by 26
Self-citationCognitive Load Measurement from User’s Linguistic Speech Features for Adaptive Interaction Design
Khawaja, M. A., Chen, F., Owen, C., & Hickey, G. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-03655-2_54 · Cited by 21
Self-citationComparative Analysis of Text Data in Successful Face-to-Face and Electronic Negotiations
Sokolova, M., Shah, M., & Śzpakowicz, S. (2006) · Group Decision and Negotiation · doi:10.1007/s10726-006-9024-z · Cited by 21
A Comparison of Two Mental Workload Instruments in Multimedia Instruction
Windell, D. T., Wiebe, E., Converse, S. A., & Beith, B. H. (2006) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/154193120605001709 · Cited by 11
Using Psychophysiological Measurements in Physically Demanding Virtual Environments
Novak, D., Mihelj, M., & Munih, M. (2009) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-03655-2_55 · Cited by 9 · cited 1×
“Many studies have attempted to measure cognitive load using several methods including physiological, performance, and self-reporting subjective measures [1,4-7].”
A comparison of two mental workload instruments in multimedia instruction
Windell, D. T., Wiebe, E., Converse, S. A., & Beith, B. H. (2006) · PsycEXTRA Dataset · doi:10.1037/e577712012-009 · Cited by 3
Research works printed but not linked: 5
- 10.5555/647664.733413 · doi:10.5555/647664.733413
- Reck , P. and et el., Generating and rendering readability scores for Project Gutenberg texts , In Proc. Corpus Linguistics Conference , Birmingham, UK , 2007 , Article #120. Reck, P. and et el., Generating and rendering readability scores for Project Gutenberg texts, In Proc. Corpus Linguistics Conference, Birmingham, UK, 2007, Article #120.
- Kun-Moo Rhee and Eunil Kim, A Statistical Analysis of Text for Inferring Authenticity , Proc. 53rd Session of International Statistical Institute , Seoul , 2001 . http://isi.cbs.nl/iamamember/index.htm Kun-Moo Rhee and Eunil Kim, A Statistical Analysis of Text for Inferring Authenticity, Proc. 53rd Session of International Statistical Institute, Seoul, 2001. http://isi.cbs.nl/iamamember/index.htm
- Ure , J. Lexical density and register differentiation. Applications of Linguistics, Perren and Trim (Eds) , London : Cambridge University Press . 1971 , pp. 443 -- 452 Ure, J. Lexical density and register differentiation. Applications of Linguistics, Perren and Trim (Eds), London: Cambridge University Press. 1971, pp. 443--452
- Advanced Text Analyser , UsingEnglish , http://www.usingenglish.com ; Last accessed: Sept. 2009 Advanced Text Analyser, UsingEnglish, http://www.usingenglish.com; Last accessed: Sept. 2009
Cited by
A Survey on Measuring Cognitive Workload in Human-Computer Interaction
Kosch, T., Karolus, J., Zagermann, J., Reiterer, H., Schmidt, A., & Woźniak, P. W. (2023) · ACM Computing Surveys · doi:10.1145/3582272 · Cited by 304
“Here, speech features such as the flow of words (e.g., tempo [42] or pauses [117, 118]), but also more nuanced features such as the lexical density [115], the spectral domain [259], or the pitch contour [39] while talking were shown to be indicators of changes in participants’ cognitive workload.”
Reluctant Disclosure and Transparency: Evidence from Environmental Disclosures
Fabrizio, K. R., & Kim, E.-H. (2019) · Organization Science · doi:10.1287/orsc.2019.1298 · Cited by 175
The worked example effect, the generation effect, and element interactivity.
Chen, O., Kalyuga, S., & Sweller, J. (2015) · Journal of Educational Psychology · doi:10.1037/edu0000018 · Cited by 167
Multimodal behavior and interaction as indicators of cognitive load
Chen, F., Ruiz, N., Choi, E., Epps, J., Khawaja, M. A., Taib, R., Yin, B., & Wang, Y. (2012) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/2395123.2395127 · Cited by 141
Self-citationZierau, N., Hildebrand, C., Bergner, A., Busquet, F., Schmitt, A., & Marco Leimeister, J. (2023) · Journal of the Academy of Marketing Science · doi:10.1007/s11747-022-00868-5 · Cited by 109
Metadata
- Authors
- Khawaja, M. A., Chen, F., & Marcus, N.
- Affiliations
- UNSW Sydney (AU)
- Venue
- Proceedings of the 15th international conference on Intelligent user interfaces
- Publisher
- ACM
- Published
- 2010-02-07
- Pages
- 333-336
- OpenAlex
- W2139006235
- Cited by
- 32
- References
- 25
- Citations by year
- 2025: 2 · 2024: 1 · 2023: 1 · 2022: 3 · 2021: 2 · 2020: 5 · 2019: 3 · 2017: 1 · 2016: 4 · 2015: 2 · 2014: 1 · 2013: 3 · 2012: 2
- FWCI
- 5.46
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM International Conference on Intelligent User Interfaces (IUI) (publishedVersion)
- Funders
- University of Tasmania
- Topics
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Keywords
- cognitive load assessment, speech content analysis, linguistic complexity, wildfire management, user interface evaluation, transcribed speech
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.