Cited by 14Social Robot Interaction and HRISpeech and dialogue systemsEmotion and Mood Recognition

Multimodal User Feedback During Adaptive Robot-Human Presentations

Open access
Axelsson, Agnes, Skantze, Gabriel·2022·Frontiers in Computer Science

Abstract

Feedback is an essential part of all communication, and agents communicating with humans must be able to both give and receive feedback in order to ensure mutual understanding. In this paper, we analyse multimodal feedback given by humans towards a robot that is presenting a piece of art in a shared environment, similar to a museum setting. The data analysed contains both video and audio recordings of 28 participants, and the data has been richly annotated both in terms of multimodal cues (speech, gaze, head gestures, facial expressions, and body pose), as well as the polarity of any feedback (negative, positive, or neutral). We train statistical and machine learning models on the dataset, and find that random forest models and multinomial regression models perform well on predicting the polarity of the participants' reactions. An analysis of the different modalities shows that most information is found in the participants' speech and head gestures, while much less information is found in their facial expressions, body pose and gaze. An analysis of the timing of the feedback shows that most feedback is given when the robot makes pauses (and thereby invites feedback), but that the more exact timing of the feedback does not affect its meaning.

Integrity signals

Registry

Crossref · 10.3389/fcomp.2021.741148

Cites

95 of 95 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Axelsson and Skantze (2022) found that individuals interacting with a system presenting a painting to them generally used feedback in the head and speech modalities.

    “Recently, Axelsson and Skantze (2022) showed that individuals interacting with a system presenting a painting to them generally used feedback in the head and speech modalities”

    From Modeling Feedback in Interaction With Conversational Agents—A Review (2022) · cites this paper as Axelsson and Skantze (2022)

  • Axelsson and Skantze (2022) found that feedback in the facial, body pose and gaze modalities was not important for classifying their response as positive, negative, or neutral.

    “and that feedback in the facial, body pose and gaze modalities was not important for classifying their response as positive, negative, or neutral.”

    From Modeling Feedback in Interaction With Conversational Agents—A Review (2022) · cites this paper as Axelsson and Skantze (2022)

Cites

Flagged first, then by citations

6 of the 101 references couldn’t be matched to a known paper.

Cited by

Sorted by citations

Authors
Axelsson, A., & Skantze, G.
Affiliations
KTH Royal Institute of Technology (SE)
Corresponding author
  • Axelsson, Agnes
  • Skantze, Gabriel
Venue
Frontiers in Computer Science
Published
2022-01-03
Volume
3
ISSN
2624-9898 (ISSN-L)
OpenAlex
W4205976400
Cited by
14
References
101
Citations by year
2026: 1 · 2025: 8 · 2024: 1 · 2023: 3 · 2022: 1
FWCI
3.07
Type
journal article
Language
en
Open access
Locations
  • Frontiers in Computer Science (publishedVersion, cc-by)
  • KTH Publication Database DiVA (KTH Royal Institute of Technology) (submittedVersion, other-oa)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
Stiftelsen för Strategisk Forskning
Topics
  • Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
  • Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Emotion and Mood Recognition
Keywords
multimodal feedback, human-robot interaction, adaptive presentation, museum robots, speech analysis, head gestures, facial expressions, gaze, body pose, feedback timing

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review