Inferring mental overload based on postural behavior and gestures
The DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
Abstract
Humans rely on body gestures and posture when communicating. This topic has been covered in great detail by researchers from various fields. Within this work, approaches to transfer findings in psychology and behavioral studies regarding the relation between gestures and emotions to machine learning methods, will be investigated. Knowledge about the users emotional state is important to achieve human like, natural HCI in modern technical systems. The main focus lies on discriminating between mental overload and mental underload, when completing a given task, which for instance can be useful in an e-tutorial system. Mental underload is a new term used to describe the state a person is in when completing a dull or boring task. A suggestion how the affective states of overload and underload can be expressed using the established notation in the Valence, Arousal and Dominance (V,A,D) space will be given. In a further step it will be shown how to select suitable features, such as gestures, movement and postural behavior patterns. Based on the features selected, a classifier is designed and trained capable of deciding whether a person is experiencing mental overload or underload.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Picard, R. W. (1997) · MIT Press · doi:10.7551/mitpress/1140.001.0001 · Cited by 7,382
A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions
Zeng, Z., Pantic, M., Roisman, G. I., & Huang, T. S. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.52 · Cited by 2,797
Facial and vocal expressions of emotion
Russell, J. A., Bachorowski, J.-A., & Fernández-Dols, J.-M. (2003) · Annual Review of Psychology · doi:10.1146/annurev.psych.54.101601.145102 · Cited by 873
Robust facial expression recognition using local binary patterns
Shan, C., Gong, S., & McOwan, P. W. (2005) · IEEE International Conference on Image Processing 2005 · doi:10.1109/icip.2005.1530069 · Cited by 414
Recognition of Affective Communicative Intent in Robot-Directed Speech
Breazeal, C., & Aryananda, L. (2002) · Autonomous Robots · doi:10.1023/a:1013215010749 · Cited by 243
Gesture-Based Affective Computing on Motion Capture Data
Kapur, A., Kapur, A., Virji-Babul, N., Tzanetakis, G., & Driessen, P. F. (2005) · Lecture Notes in Computer Science · doi:10.1007/11573548_1 · Cited by 202
Perceiving facial and vocal expressions of emotion in individuals with Williams syndrome
Plesa-Skwerer, D., Faja, S., Schofield, C., Verbalis, A., & Tager-Flusberg, H. (2006) · American Journal on Mental Retardation · doi:10.1352/0895-8017(2006)111[15:pfaveo]2.0.co;2 · Cited by 136 · cited 1×
“There has a been a lot of research in this area [6, 2, 8].”
BackgroundModeling naturalistic affective states via facial and vocal expressions recognition
Caridakis, G., Malatesta, L., Kessous, L., Amir, N., Raouzaiou, A., & Karpouzis, K. (2006) · Proceedings of the 8th international conference on Multimodal interfaces · doi:10.1145/1180995.1181029 · Cited by 131
Gesture and emotion: Can basic gestural form features discriminate emotions?
Kipp, M., & Martin, J. (2009) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii.2009.5349544 · Cited by 130
A Companion Technology for Cognitive Technical Systems
Wendemuth, A., & Biundo, S. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-34584-5_7 · Cited by 115
Research works printed but not linked: 3
- Bianchi-Berthouze N., 2006, Emotion and HCI workshop at BCS HCI London
- Schüssel F., 2016, HCI International
- Shan C., Image Processing, 2005. ICIP 2005. IEEE International Conference on, 2, 370
Cited by
A Survey on Measuring Cognitive Workload in Human-Computer Interaction
Kosch, T., Karolus, J., Zagermann, J., Reiterer, H., Schmidt, A., & Woźniak, P. W. (2023) · ACM Computing Surveys · doi:10.1145/3582272 · Cited by 304
“So far, previous research has investigated (1) postural behavior and gestures to assess phases of cognitive underload or overload [95], (2) mouse movements such as traveled distances [12] or pause/break activities [119], and (3) differences in glance duration at input modalities and output devices…”
Metadata
- Authors
- Hihn, H., Meudt, S., & Schwenker, F.
- Affiliations
- Universität Ulm (DE)
- Venue
- Proceedings of the 2nd workshop on Emotion Representations and Modelling for Companion Systems
- Publisher
- ACM
- Published
- 2016-11-16
- Pages
- 1-4
- OpenAlex
- W2557016173
- Cited by
- 8
- References
- 13
- Citations by year
- 2026: 1 · 2024: 1 · 2023: 2 · 2020: 1 · 2018: 1 · 2017: 2
- FWCI
- 1.16
- Type
- conference paper
- Language
- en
- Open access
- closed
- Funders
- Deutsche Forschungsgemeinschaft
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Human-Computer Interaction › Gaze Tracking and Assistive Technology
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Keywords
- cognitive overload, postural behavior, gesture, affective computing, emotion recognition, feature selection, classifier design, human-computer interaction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Inferring mental overload based on postural behavior and gestures. https://app.cochat.ai/papers/10.1145/3009960.3009961
- The paper
- Hihn, H., Meudt, S., & Schwenker, F. (2016). Inferring mental overload based on postural behavior and gestures. Proceedings of the 2nd workshop on Emotion Representations and Modelling for Companion Systems. https://doi.org/10.1145/3009960.3009961
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.