The DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index.
Contextual recognition of head gestures,
Abstract
Head pose and gesture offer several key conversational grounding cues and are used extensively in face-to-face interaction among people. We investigate how dialog context from an embodied conversational agent (ECA) can improve visual recognition of user gestures. We present a recognition framework which (1) extracts contextual features from an ECA's dialog manager, (2) computes a prediction of head nod and head shakes, and (3) integrates the contextual predictions with the visual observation of a vision-based head gesture recognizer. We found a subset of lexical, punctuation and timing features that are easily available in most ECA architectures and can be used to learn how to predict user feedback. Using a discriminative approach to contextual prediction and multi-modal integration, we were able to improve the performance of head gesture detection even when the topic of the test set was significantly different than the training set.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“In their study, they found that classification of nods and shakes was possible to do with high accuracy when the visual data from a camera sensor was combined with a prediction from dialogue context.”
From Modeling Feedback in Interaction With Conversational Agents—A Review (2022) · cites this paper as Morency et al., 2005
Citations
Cites
Context-based vision system for place and object recognition
Torralba, Murphy, M., Freeman, & Рубин (2003) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2003.1238354 · Cited by 875
Explorations in engagement for humans and robots
Sidner, C. L., Lee, C. W., Kidd, C. D., Lesh, N. B., & Rich, C. (2005) · Artificial Intelligence · doi:10.1016/j.artint.2005.03.005 · Cited by 559
Self-citationThe power of a nod and a glance: envelope vs. emotional feedback in animated conversational agents
Cassell, J., & Thorisson, K. R. (1999) · Applied Artificial Intelligence · doi:10.1080/088395199117360 · Cited by 383
Collagen: applying collaborative discourse theory to human-computer interaction
Rich, C., Sidner, C. L., & Lesh, N. B. (2001) · doi:10.1609/aimag.v22i4.1589 · Cited by 289
Self-citationEmbodied agents for multi-party dialogue in immersive virtual worlds
Traum, D. R., & Rickel, J. W. (2002) · Adaptive Agents and Multi-Agents Systems · doi:10.1145/544862.544922 · Cited by 236
Towards a model of face-to-face grounding
Nakano, Y., Reinstein, G., Stocky, T., & Cassell, J. (2003) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.3115/1075096.1075166 · Cited by 202
A real-time head nod and shake detector
Kapoor, A., & Picard, R. W. (2001) · Workshop on Perceptive User Interfaces · doi:10.1145/971478.971509 · Cited by 156
Tracking focus of attention in meetings
Stiefelhagen, R. (2003) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1109/icmi.2002.1167006 · Cited by 152
Teaching and Working with Robots as a Collaboration
Breazeal, C., Hoffman, G., & Thomaz, A. L. (2004) · doi:10.1109/aamas.2004.258 · Cited by 136
Adaptive view-based appearance models
Morency, L.-P., Rahimi, A., & Darrell, T. J. (2003) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2003.1211435 · Cited by 107
Self-citation
Research works printed but not linked: 6
- Tim Bickmore and Justine Cassell . J. van Kuppevelt , L. Dybkjaer , and N. Bernsen (eds.), Natural, Intelligent and Effective Interaction with Multimodal Dialogue Systems , chapter Social Dialogue with Embodied Conversational Agents. Kluwer Academic , 2004 . Tim Bickmore and Justine Cassell. J. van Kuppevelt, L. Dybkjaer, and N. Bernsen (eds.), Natural, Intelligent and Effective Interaction with Multimodal Dialogue Systems, chapter Social Dialogue with Embodied Conversational Agents. Kluwer Academic, 2004.
- 10.5555/1018411.1018871 · DOI
- Dillman, Ehrenmann, Steinhaus, Rogalla, and R. Zoellner . Human friendly programming of humanoid robots--the German Collaborative Research Center . In The Third IARP Intenational Workshop on Humanoid and Human-Friendly Robotics, Tsukuba Research Centre , Japan , December 2002 . Dillman, Ehrenmann, Steinhaus, Rogalla, and R. Zoellner. Human friendly programming of humanoid robots--the German Collaborative Research Center. In The Third IARP Intenational Workshop on Humanoid and Human-Friendly Robotics, Tsukuba Research Centre, Japan, December 2002.
- Lemon, Gruenstein, and Stanley Peters . Collaborative activities and multi-tasking in dialogue systems. Traitement Automatique des Langues (TAL), special issue on dialogue, 43(2):131--154 , 2002 . Lemon, Gruenstein, and Stanley Peters. Collaborative activities and multi-tasking in dialogue systems. Traitement Automatique des Langues (TAL), special issue on dialogue, 43(2):131--154, 2002.
- 10.5555/874061.875409 · DOI
- 10.5555/946247.946665 · DOI
Cited by
Sorted by citations
Items 101–130 of 130
Feedback facial expressions and emotions
Navarretta, C. (2013) · Journal on Multimodal User Interfaces · doi:10.1007/s12193-013-0145-9 · Cited by 4
Leveraging the robot dialog state for visual focus of attention recognition
Sheikhi, S., Khalidov, V., Klotz, D., Wrede, B., & Odobez, J. (2013) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/2522848.2522881 · Cited by 3
Emotional Recognition and towards Context based Decision
Butalia, A., Ramani, A. K., & Kulkarni, P. A. (2010) · International Journal of Computer Applications · doi:10.5120/1362-1838 · Cited by 3
Virtual Humans for the Study of Rapport in Cross Cultural Settings
Gratch, J. S., Okhmatovskaia, A., & Duncan, S. (2006) · Cited by 3
CCDb-HG: Novel Annotations and Gaze-Aware Representations for Head Gesture Recognition
Vuillecard, P., Farkhondeh, A., Villamizar, M., & Odobez, J. (2024) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg59268.2024.10581954 · Cited by 2
Contextual and Human Factors in Information Fusion
Garcı́a, J., Patricio, M. A., Molina Jos eacute, M., & Cardoso Luiz, A. (2010) · NATO science for peace and security series. D, Information and communication security · doi:10.3233/978-1-60750-621-8-79 · Cited by 2
Singh, A., & Bansal, A. K. (2023) · Studies in computational intelligence · doi:10.1007/978-3-031-44127-1_3 · Cited by 2
Automated Real-Time Recognition of Non-emotional Conversational Head-Gestures for Social Robots
Singh, A., & Bansal, A. K. (2022) · Lecture notes in networks and systems · doi:10.1007/978-3-031-18344-7_29 · Cited by 2
Design of an entertainment robot with multimodal human-robot interactions
Jean, J.-H., Chen, K., Shih, K.-Y., & Lin, H.-L. (2008) · SICE Annual Conference (SICE) · doi:10.1109/sice.2008.4654884 · Cited by 2
Enhancing Informal Learning Experiences with Affect-Aware Technologies
Calvo, R. A., D’Mello, S. K., Gratch, J. S., Kappas, A., & Lane, H. C. (2014) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780199942237.013.033 · Cited by 2
BEST EMOTIONAL FEATURE SELECTION CRITERIA BASED ON ROUGH SET THEORY
Butalia, A., Ingle, M., & Kulkarni, P. A. (2012) · Cited by 2
Rich, C., & Sidner, C. L. (2006) · Cited by 2
Self-citationHead Nod Detection in Dyadic Conversations
Numanoğlu, T., Erzin, E., Yemezy, Y., & Sezgin, T. M. (2019) · Signal Processing and Communications Applications Conference (SIU) · doi:10.1109/siu.2019.8806231 · Cited by 2
Poster: Head gesture 3D interface using a head mounted camera
Moteki, A., Hara, N., Murase, T., Ozawa, N., Nakai, T., Matsuda, T., & Fujimoto, K. (2012) · IEEE Symposium on 3D User Interfaces (3DUI) · doi:10.1109/3dui.2012.6184206 · Cited by 2
Behavior-grounded multi-sensory object perception and exploration by a humanoid robot
Sinapov, J. (2013) · Iowa State University Digital Repository (Iowa State University) · doi:10.31274/etd-180810-3415 · Cited by 2
A Study on Head Gesture Recognition System using Wearable Sensor
Shogo, K. I. S. A., & Horiuchi, T. (2016) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.28.986 · Cited by 1
Parasocial consensus sampling: modeling human nonverbal behaviors from multiple perspectives
Huang, L. (2013) · University of Southern California Digital Library · doi:10.25549/usctheses-c3-233719 · Cited by 1
Proceedings of the Sixth Workshop on Vision and Language
(2017) · doi:10.18653/v1/w17-20 · Cited by 1
Object’s Interaction Management by Means of a Fuzzy System within a Context-Based Tracking System
Sánchez, A. M., Patricio, M. A., & Garcı́a, J. (2008) · Advances in soft computing · doi:10.1007/978-3-540-85863-8_85
Mühl, C., Heylen, D., Nijholt, A., Cohn, J., Nijholt, A., & Pantić, M. (2009) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii.2009.5349457
Development of a two layer classification system for situations in meetings
Tyaneva, A. (2011) · OPUS Publication Server of the University of Stuttgart (University of Stuttgart) · doi:10.18419/opus-2801
Atrash, A., & Mead, R. (2013) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.5555/2447556.2447709
Classification performance of different classifiers on head gestures and facial expressions
Akakın, H. Ç., & Sankur, B. (2010) · Signal Processing and Communications Applications Conference (SIU) · doi:10.1109/siu.2010.5650829
Interacting with a Self-portrait Camera Using Gestures
褚, 少. (2013) · Terrestrial Environment Research Center (University of Tsukuba)
Paggio, P., Navarretta, C., Jongejan, B., & Agirrezabal, M. (2021) · doi:10.18653/v1/2021.law-1.16
Atrash, A., & Mead, R. (2013) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1109/hri.2013.6483635
Said, S. (2020) · theses.fr (ABES) · doi:10.70675/dd9c45ddza2aaz46bfzaff8z04f45452183d · Cited by 1
Visual Feedback Design for Voice Assistants: Applying Nonverbal Backchannel and Turn-Taking Cues
Park, Y., Kim, D., & Song, H. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2026.2695442
Ni, Z., & Yang, L. (2026) · Companion Publication of the International Conference on Multimodal Interaction (ICMI Companion) · doi:10.1145/3776591.3832517
Metadata
- Authors
- Morency, L.-P., Sidner, C., Lee, C., & Darrell, T.
- Affiliations
- Massachusetts Institute of Technology (US)
- Mitsubishi Electric Research Laboratories (United States) (US)
- Venue
- Proceedings of the 7th international conference on Multimodal interfaces
- Publisher
- ACM
- Published
- 2005-10-04
- Pages
- 18-24
- OpenAlex
- W2112618815
- Cited by
- 130
- References
- 18
- Citations by year
- 2026: 2 · 2024: 2 · 2023: 3 · 2022: 4 · 2021: 2 · 2020: 4 · 2019: 1 · 2018: 3 · 2017: 5 · 2016: 4 · 2015: 5 · 2014: 4 · 2013: 12 · 2012: 7
- FWCI
- 7.61
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Physical Sciences › Computer Science › Human-Computer Interaction › Hand Gesture Recognition Systems
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Keywords
- head gesture recognition, head orientation, head nodding, head shake, embodied conversational agents, contextual features, multimodal integration, lexical features, timing features, conversational grounding
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
