Automatic cognitive load detection from speech features
Abstract
Cognitive load variations have been found to impact multimodal behaviour, in particular, features of spoken input. In this paper, we present a design and implementation of a user study aimed at soliciting natural speech at three different levels of cognitive load. Some of the speech data produced was then used to train a number of models to automatically detect cognitive load. We describe a classification approach, the cognitive load levels were detected and output as discrete level ranges. The final system achieved a 71.1% accuracy for 3 levels classification in a speaker-independent setting. The ability to detect and manage a user's cognitive load can help us to adapt intelligent interfaces that ensure optimal user performance
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Cites
When do we interact multimodally?: cognitive load and multimodal communication patterns
Oviatt, S., Coulston, R., & Lunsford, R. (2004) · Proceedings of the 6th international conference on Multimodal interfaces · doi:10.1145/1027933.1027957 · Cited by 285 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Integrating cognitive load theory and concepts of human–computer interaction
Hollender, N., Hofmann, C., Deneke, M., & Schmitz, B. (2010) · Computers in Human Behavior · doi:10.1016/j.chb.2010.05.031 · Cited by 328 · see citing works
A Survey on Measuring Cognitive Workload in Human-Computer Interaction
Kosch, T., Karolus, J., Zagermann, J., Reiterer, H., Schmidt, A., & Woźniak, P. W. (2023) · ACM Computing Surveys · doi:10.1145/3582272 · Cited by 304 · see citing works
“The fixation duration was found to indicate an increased strain on cognitive workload.”
Think before you talk: an empirical study of relationship between speech pauses and cognitive load
Khawaja, M. A., Ruiz, N., & Chen, F. (2008) · Proceedings of the 20th Australasian Conference on Computer-Human Interaction: Designing for Habitus and Habitat · doi:10.1145/1517744.1517814 · Cited by 30 · see citing works
Self-citationCosic, K., Popovic, S., Sarlija, M., Mijic, I., Kokot, M., Kesedzic, I., Strangman, G., Ivkovic, V., & Zhang, Q. (2019) · IEEE Access · doi:10.1109/access.2019.2957357 · Cited by 29 · see citing works
Vizer, L. M., & Sears, A. (2017) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2017.03.001 · Cited by 12 · see citing works
Do speech features for detecting cognitive load depend on specific languages?
Chen, R., Xie, T., Xie, Y., Lin, T., & Tang, N. (2016) · Proceedings of the 18th ACM International Conference on Multimodal Interaction · doi:10.1145/2993148.2993149 · Cited by 8 · see citing works
- Authors
- Yin, B., Ruiz, N., Chen, F., & Khawaja, M. A.
- Affiliations
- UNSW Sydney (AU)
- Data61 (AU)
- ORCID
- Chen, Fang
- Venue
- Proceedings of the 19th Australasian conference on Computer-Human Interaction: Entertaining User Interfaces
- Publisher
- ACM
- Published
- 2007-11-28
- Pages
- 249-255
- OpenAlex
- W2043745365
- Cited by
- 55
- References
- 1
- Citations by year
- 2025: 2 · 2024: 1 · 2023: 2 · 2022: 1 · 2021: 2 · 2020: 4 · 2019: 6 · 2018: 4 · 2017: 3 · 2016: 2 · 2015: 2 · 2014: 4 · 2013: 2 · 2012: 2
- FWCI
- 6.83
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Australian Conference on Computer-Human Interaction (OzCHI) (publishedVersion)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Emotion and Mood Recognition
- Social Sciences › Psychology › Social Psychology › Human-Automation Interaction and Safety
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Keywords
- cognitive load detection, speech features, speech classification, user study, speaker-independent classification, intelligent user interfaces, adaptive user interfaces
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
