CoChat
Sign Up
Cited by 185Speech and dialogue systemsEmotion and Mood RecognitionSocial Robot Interaction and HRI

Building autonomous sensitive artificial listeners

Open access
Schroder, M., Bevacqua, E., Cowie, R., Eyben, F., Gunes, H., Heylen, D., ter Maat, M., McKeown, G. +7 more·2012·IEEE Transactions on Affective Computing
DOI 10.1109/t-affc.2011.34
Open this paper in CoChat — free accountOpen PDF

Abstract

This paper describes a substantial effort to build a real-time interactive multimodal dialogue system with a focus on emotional and nonverbal interaction capabilities. The work is motivated by the aim to provide technology with competences in perceiving and producing the emotional and nonverbal behaviors required to sustain a conversational dialogue. We present the Sensitive Artificial Listener (SAL) scenario as a setting which seems particularly suited for the study of emotional and nonverbal behavior since it requires only very limited verbal understanding on the part of the machine. This scenario allows us to concentrate on nonverbal capabilities without having to address at the same time the challenges of spoken language understanding, task modeling, etc. We first report on three prototype versions of the SAL scenario in which the behavior of the Sensitive Artificial Listener characters was determined by a human operator. These prototypes served the purpose of verifying the effectiveness of the SAL scenario and allowed us to collect data required for building system components for analyzing and synthesizing the respective behaviors. We then describe the fully autonomous integrated real-time system we created, which combines incremental analysis of user behavior, dialogue management, and synthesis of speaker and listener behavior of a SAL character displayed as a virtual agent. We discuss principles that should underlie the evaluation of SAL-type systems. Since the system is designed for modularity and reuse and since it is publicly available, the SAL system has potential as a joint research tool in the affective computing research community.

Integrity signals

Registry

Crossref · 10.1109/t-affc.2011.34

Cites

126 of 126 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • In evaluation studies, users preferred the affect-sensitive listener to a control and it led to higher engagement, felt engagement, and flow.

    “In evaluation studies it was shown that users preferred the affect-sensitive listener to a control and that it led to higher engagement, felt engagement, and flow.”

    From Modeling Feedback in Interaction With Conversational Agents—A Review (2022) · cites this paper as Schröder et al., 2012

  • In studies on agent backchannels, typical domains include story telling, instructions, or interview scenarios, where the user is talking most of the time.

    “In studies on agent backchannels, typical domains include story telling (Schröder et al., 2012), instructions (Meena et al., 2014) or interview scenarios (Johansson et al., 2016), where the user is talking most of the time.”

    From Modeling Feedback in Interaction With Conversational Agents—A Review (2022) · cites this paper as Schröder et al., 2012

Cites

Flagged first, then by citations

4 of the 130 references couldn’t be matched to a known paper.

Cited by

Sorted by citations

Related

Authors
Schroder, M., Bevacqua, E., Cowie, R., Eyben, F., Gunes, H., Heylen, D., ter Maat, M., McKeown, G., Pammi, S., Pantic, M., Pelachaud, C., Schuller, B., de Sevin, E., Valstar, M., & Wollmer, M.
Affiliations
  • École nationale d'ingénieurs de Brest (FR)
  • Queen's University Belfast (GB)
  • Technical University of Munich (DE)
  • University of London (GB)
  • University of Twente (NL)
  • German Research Centre for Artificial Intelligence (DE)
  • Imperial College London (GB)
ORCID
Venue
IEEE Transactions on Affective Computing
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Published
2012-04
Volume
3
Issue
2
Pages
165-183
ISSN
1949-3045 (ISSN-L)
DOI
10.1109/t-affc.2011.34
OpenAlex
W2042170119
Cited by
185
References
130
Citations by year
2026: 6 · 2024: 4 · 2023: 9 · 2022: 13 · 2021: 9 · 2020: 6 · 2019: 11 · 2018: 18 · 2017: 9 · 2016: 22 · 2015: 25 · 2014: 18 · 2013: 25 · 2012: 9
FWCI
15.61
Type
journal article
Language
en
Open access
Licence
Locations
  • IEEE Transactions on Affective Computing (publishedVersion)
  • Research Portal (Queen's University Belfast) (submittedVersion, other-oa)
  • HAL (submittedVersion)
  • mediaTUM – the media and publications repository of the Technical University Munich (Technical University Munich) (submittedVersion, other-oa)
  • University of Twente Research Information (submittedVersion)
Funders
European Commission (211486)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Emotion and Mood Recognition
  • Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
Keywords
multimodal dialogue systems, emotional interaction, nonverbal behavior, virtual agents, dialogue management, user behavior analysis, affective computing, real-time interactive systems, system evaluation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review