Enabling Multimodal Human–Robot Interaction for the Karlsruhe Humanoid Robot
Open accessThe DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Abstract
In this paper, we present our work in building technologies for natural multimodal human-robot interaction. We present our systems for spontaneous speech recognition, multimodal dialogue processing, and visual perception of a user, which includes localization, tracking, and identification of the user, recognition of pointing gestures, as well as the recognition of a person's head orientation. Each of the components is described in the paper and experimental results are presented. We also present several experiments on multimodal human-robot interaction, such as interaction using speech and gestures, the automatic determination of the addressee during human-human-robot interaction, as well on interactive learning of dialogue strategies. The work and the components presented here constitute the core building blocks for audiovisual perception of humans and multimodal human-robot interaction used for the humanoid robot developed within the German research project (Sonderforschungsbereich) on humanoid cooperative robots.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Robust Real-Time Face Detection
Viola, P., & Jones, M. J. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000013087.49260.fb · Cited by 14,097
Turk, M., & Pentland, A. (1991) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.1991.3.1.71 · Cited by 13,733
CONDENSATION—Conditional Density Propagation for Visual Tracking
Isard, M., & Blake, A. (1998) · International Journal of Computer Vision · doi:10.1023/a:1008078328650 · Cited by 4,886
A survey of socially interactive robots
Fong, T., Nourbakhsh, I., & Dautenhahn, K. (2003) · Robotics and Autonomous Systems · doi:10.1016/s0921-8890(02)00372-x · Cited by 3,162 · cited 1×
“…much research effort has been fo-cused on the development of humanoid robots, and great progress has been made in developing robots with human-like torsi, including legs, arms, head etc., as well as some human-like motoric skills, such as walking, grasping, dancing [3], [4], [29], [33], [41], [46].”
BackgroundAn extended set of Haar-like features for rapid object detection
Lienhart, R., & Maydt, J. (2003) · Proceedings. International Conference on Image Processing · doi:10.1109/icip.2002.1038171 · Cited by 2,977 · cited 1×
“In the face detection algorithm proposed by [54] and extended by [35], a variable-size search window is repeatedly shifted over the image, and overlapping detections are combined to a single detection.”
Cited by
A review of verbal and non-verbal human–robot interactive communication
Mavridis, N. (2015) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2014.09.031 · Cited by 453
Do travelers trust intelligent service robots?
Tussyadiah, I. P., Zach, F. J., & Wang, J. (2020) · Annals of Tourism Research · doi:10.1016/j.annals.2020.102886 · Cited by 329
A Communication Robot in a Shopping Mall
Kanda, T., Shiomi, M., Miyashita, Z., Ishiguro, H., & Hagita, N. (2010) · IEEE Transactions on Robotics · doi:10.1109/tro.2010.2062550 · Cited by 309
Localization of sound sources in robotics: A review
Rascon, C., & Meza, I. (2017) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2017.07.011 · Cited by 271
Robust Multimodal Emotion Recognition from Conversation with Transformer-Based Crossmodality Fusion
Xie, B., Sidulova, M., & Park, C. H. (2021) · Sensors · doi:10.3390/s21144913 · Cited by 108
Metadata
- Authors
- Stiefelhagen, R., Ekenel, H. K., Fugen, C., Gieselmann, P., Holzapfel, H., Kraft, F., Nickel, K., Voit, M., & Waibel, A.
- Venue
- IEEE Transactions on Robotics
- Published
- 2007-10
- Volume
- 23
- Issue
- 5
- Pages
- 840-851
- ISSN
- 1552-3098 (ISSN-L)
- 1941-0468
- OpenAlex
- W2016565202
- Cited by
- 136
- References
- 5
- FWCI
- 10.64
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.