Starting engagement detection towards a companion robot using multimodal features
Open accessThe DOI matches its Crossref record; 4 of the 4 references it cites were found in a registry or an index.
Abstract
Recognition of intentions is a subconscious cognitive process vital to human communication. This skill enables anticipation and increases the quality of interactions between humans. Within the context of engagement, non-verbal signals are used to communicate the intention of starting the interaction with a partner. In this paper, we investigated methods to detect these signals in order to allow a robot to know when it is about to be addressed. Originality of our approach resides in taking inspiration from social and cognitive sciences to perform our perception task. We investigate meaningful features, i.e. human readable features, and elicit which of these are important for recognizing someone's intention of starting an interaction. Classically, spatial information like the human position and speed, the human-robot distance are used to detect the engagement. Our approach integrates multimodal features gathered using a companion robot equipped with a Kinect. The evaluation on our corpus collected in spontaneous conditions highlights its robustness and validates the use of such a technique in a real environment. Experimental validation shows that multimodal features set gives better precision and recall than using only spatial and speed features. We also demonstrate that 7 selected features are sufficient to provide a good starting engagement detection score. In our last investigation, we show that among our full 99 features set, the space reduction is not a solved task. This result opens new researches perspectives on multimodal engagement detection. Multimodal approach for starting engagement detection using non-explicit cues.Results show that our approach performs better than spatial one in all conditions.MRMR strategy reduces the features space to 7 features without a performance loss.Validation of Schegloff (sociologist) meaningful features for engagement detection.A robot centered labeled corpus of 4 hours in a home-like environment.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Mehrabian, A. (1996) · Current Psychology · doi:10.1007/bf02686918 · Cited by 1,493
Social signal processing: Survey of an emerging domain
Vinciarelli, A., Pantic, M., & Bourlard, H. (2009) · Image and Vision Computing · doi:10.1016/j.imavis.2008.11.007 · Cited by 842
Toward an affect-sensitive multimodal human-computer interaction
Pantic, M., & Rothkrantz, L. J. M. (2003) · Proceedings of the IEEE · doi:10.1109/jproc.2003.817122 · Cited by 791
Automatic analysis of affective postures and body motion to detect engagement with a game companion,
Sanghvi, J., Castellano, G., Leite, I., Pereira, A., McOwan, P. W., & Paiva, A. (2011) · Proceedings of the 6th international conference on Human-robot interaction · doi:10.1145/1957656.1957781 · Cited by 250 · cited 1×
“Bull claims the importance of posture and gesture in non verbal communication where these channels have been neglected compared to facial features and speech cues.”
Background
Cited by
Engagement in human-agent interaction: An overview
Oertel, C., Castellano, G., Chetouani, M., Nasir, J., Obaid, M., Pelachaud, C., & Peters, C. (2020) · Frontiers in Robotics and AI · doi:10.3389/frobt.2020.00092 · Cited by 160
Strazdas, D., Hintz, J., Khalifa, A., Abdelrahman, A. A., Hempel, T., & Al-Hamadi, A. (2022) · Sensors · doi:10.3390/s22030923 · Cited by 53
Modelling adaptive presentations in human-robot interaction using behaviour trees,
Axelsson, N., & Skantze, G. (2019) · Proceedings of the 20th Annual SIGdial Meeting on Discourse and Dialogue · doi:10.18653/v1/w19-5940 · Cited by 15
“…these sub-trees have had their chance to change the presentation agenda to address negative evidence of hearing, understanding and acceptance (see (Vaufreydaz et al., 2016; Aly and Tapus, 2015; Sidner et al., 2006; Skantze et al., 2014) for examples on how to measure these), the system then…”
Background
Metadata
- Authors
- Vaufreydaz, D., Johal, W., & Combe, C.
- Venue
- Robotics and Autonomous Systems
- Publisher
- Elsevier BV
- Published
- 2016-01
- Volume
- 75
- Pages
- 4-16
- ISSN
- 0921-8890 (ISSN-L)
- OpenAlex
- W2086012981
- Cited by
- 63
- References
- 4
- FWCI
- 6.26
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.