CoChat

Equipping social robots with culturally-sensitive facial expressions of emotion using data-driven methods

Open access
Chen, Chaona, Hensel, Laura B., Duan, Yaocong, Ince, Robin A. A., Garrod, Oliver G. B., Beskow, Jonas, Jack, Rachael E., Schyns, Philippe G. · 2019 · 2019 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019)

The DOI matches its Crossref record; 52 of the 52 references it cites were found in a registry or an index.

Abstract

Social robots must be able to generate realistic and recognizable facial expressions to engage their human users. Many social robots are equipped with standardized facial expressions of emotion that are widely considered to be universally recognized across all cultures. However, mounting evidence shows that these facial expressions are not universally recognized - for example, they elicit significantly lower recognition accuracy in East Asian cultures than they do in Western cultures. Therefore, without culturally sensitive facial expressions, state-of-the-art social robots are restricted in their ability to engage a culturally diverse range of human users, which in turn limits their global marketability. To develop culturally sensitive facial expressions, novel data-driven methods are used to model the dynamic face movement patterns that convey basic emotions (e.g., happy, sad, anger) in a given culture using cultural perception. Here, we tested whether such dynamic facial expression models, derived in an East Asian culture and transferred to a popular social robot, improved the social signalling generation capabilities of the social robot with East Asian participants. Results showed that, compared to the social robot's existing set of facial `universal' expressions, the culturally-sensitive facial expression models are recognized with generally higher accuracy and judged as more human-like by East Asian participants. We also detail the specific dynamic face movements (Action Units) that are associated with high recognition accuracy and judgments of human-likeness, including those that further boost performance. Our results therefore demonstrate the utility of using data-driven methods that employ human cultural perception to derive culturally-sensitive facial expressions that improve the social face signal generation capabilities of social robots. We anticipate that these methods will continue to inform the design of social robots and broaden their usability and global marketability.

Overview

Integrity signals

Registry

Crossref · 10.1109/fg.2019.8756570

Cites

52 of 52 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Citations

Cites

Research works printed but not linked: 2

  1. 10.1037/0022-3514.53.4.712 · DOI · Paper page
  2. ekman, 1978, Manual for the Facial Action Coding System

Cited by

See all 41

Metadata

Authors
Chen, C., Hensel, L. B., Duan, Y., Ince, R. A. A., Garrod, O. G. B., Beskow, J., Jack, R. E., & Schyns, P. G.
Affiliations
University of Glasgow (GB)
Venue
2019 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019)
Publisher
IEEE
Published
2019-05
Pages
1-8
ISSN
1541-5058 (ISSN-L)
OpenAlex
W2922943851
Cited by
41
References
52
Citations by year
2026: 4 · 2025: 2 · 2024: 5 · 2023: 6 · 2022: 3 · 2021: 12 · 2020: 8 · 2019: 1
FWCI
9.90
Type
conference paper
Language
en
Open access
Locations
  • International Conference on Automatic Face and Gesture Recognition (publishedVersion)
  • ENLIGHTEN (Jurnal Bimbingan dan Konseling Islam) (acceptedVersion, cc-by-sa)
  • OSF Preprints (OSF Preprints) (submittedVersion)
Funders
  • Engineering and Physical Sciences Research Council (EP/N019261/1)
  • Economic and Social Research Council (1944569)
Topics
  • Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
  • Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Face Recognition and Perception
Keywords
social robots, emotional facial expressions, cultural perception, East Asian culture, facial action units, universal facial expressions, human-likeness, social signaling, robot expressiveness

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.