Context-based recognition during human interactions: automatic feature selection and encoding dictionary
The DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Abstract
During face-to-face conversation, people use visual feedback such as head nods to communicate relevant information and to synchronize rhythm between participants. In this paper we describe how contextual information from other participants can be used to predict visual feedback and improve recognition of head gestures in human-human interactions. For example, in a dyadic interaction, the speaker contextual cues such as gaze shifts or changes in prosody will influence listener backchannel feedback (e.g., head nod). To automatically learn how to integrate this contextual information into the listener gesture recognition framework, this paper addresses two main challenges: optimal feature representation using an encoding dictionary and automatic selection of optimal feature-encoding pairs. Multimodal integration between context and visual observations is performed using a discriminative sequential model (Latent-Dynamic Conditional Random Fields) trained on previous interactions. In our experiments involving 38 storytelling dyads, our context-based recognizer significantly improved head gesture recognition performance over a vision-only recognizer.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The nature of rapport and its nonverbal correlates
Tickle-Degnen, L., & Rosenthal, R. (1990) · Psychological Inquiry · doi:10.1207/s15327965pli0104_1 · Cited by 960
Bavelas, J. B., Coates, L., & Johnson, T. (2000) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.79.6.941 · Cited by 829 · cited 1×
“Such feedback is an essential and predictable aspect of natural conversation and its absence can significantly disrupt participants ability to communicate [3, 23].”
BackgroundPower moves: complementarity in dominant and submissive nonverbal behavior
Tiedens, L. Z., & Fragale, A. R. (2003) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.84.3.558 · Cited by 634
On the semantics and pragmatics of linguistic feedback
ALLWOOD, J., NIVRE, J., & AHLSÉN, E. (1992) · Journal of Semantics · doi:10.1093/jos/9.1.1 · Cited by 442 · cited 1×
“Rather they react to the current topic, previous utterances and the speaker’s current verbal and nonverbal behavior [1].”
BackgroundA shallow model of backchannel continuers in spoken dialogue
Cathcart, N., Carletta, J., & Klein, E. (2003) · Proceedings of the tenth conference on European chapter of the Association for Computational Linguistics - EACL '03 · doi:10.3115/1067807.1067816 · Cited by 88 · cited 3×
“Pauses Listener feedback often follows speaker pauses or filled pauses such as“um”(see [5]).”
BackgroundInfluential
Cited by
Clinical Virtual Reality tools to advance the prevention, assessment, and treatment of PTSD
Rizzo, A. ‘., & Shilling, R. (2017) · European Journal of Psychotraumatology · doi:10.1080/20008198.2017.1414560 · Cited by 241
A probabilistic multimodal approach for predicting listener backchannels
Morency, L.-P., de Kok, I., & Gratch, J. (2010) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-009-9092-y · Cited by 162
Self-citationMULTIMODAL AFFECT MODELING AND RECOGNITION FOR EMPATHIC ROBOT COMPANIONS
CASTELLANO, G., LEITE, I., PEREIRA, A., MARTINHO, C., PAIVA, A., & MCOWAN, P. W. (2013) · International Journal of Humanoid Robotics · doi:10.1142/s0219843613500102 · Cited by 51
“Each slice of the interaction is described by multimodal data: a frontal video capturing the face and the upper body of the children, a lateral video capturing their lateral posture and full-body movements, a video capturing the iCat, and a set of synchronised contextual features that describe the…”
Method
Metadata
- Authors
- Morency, L.-P., de Kok, I., & Gratch, J.
- Venue
- Proceedings of the 10th international conference on Multimodal interfaces
- Publisher
- ACM
- Published
- 2008-10-20
- Pages
- 181-188
- OpenAlex
- W2138858393
- Cited by
- 51
- References
- 5
- FWCI
- 3.61
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Context-based recognition during human interactions: automatic feature selection and encoding dictionary. https://app.cochat.ai/papers/10.1145/1452392.1452426
- The paper
- Morency, L.-P., de Kok, I., & Gratch, J. (2008). Context-based recognition during human interactions: automatic feature selection and encoding dictionary. Proceedings of the 10th international conference on Multimodal interfaces. https://doi.org/10.1145/1452392.1452426
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.