Equipping social robots with culturally-sensitive facial expressions of emotion using data-driven methods
Open accessThe DOI matches its Crossref record; 52 of the 52 references it cites were found in a registry or an index.
Abstract
Social robots must be able to generate realistic and recognizable facial expressions to engage their human users. Many social robots are equipped with standardized facial expressions of emotion that are widely considered to be universally recognized across all cultures. However, mounting evidence shows that these facial expressions are not universally recognized - for example, they elicit significantly lower recognition accuracy in East Asian cultures than they do in Western cultures. Therefore, without culturally sensitive facial expressions, state-of-the-art social robots are restricted in their ability to engage a culturally diverse range of human users, which in turn limits their global marketability. To develop culturally sensitive facial expressions, novel data-driven methods are used to model the dynamic face movement patterns that convey basic emotions (e.g., happy, sad, anger) in a given culture using cultural perception. Here, we tested whether such dynamic facial expression models, derived in an East Asian culture and transferred to a popular social robot, improved the social signalling generation capabilities of the social robot with East Asian participants. Results showed that, compared to the social robot's existing set of facial `universal' expressions, the culturally-sensitive facial expression models are recognized with generally higher accuracy and judged as more human-like by East Asian participants. We also detail the specific dynamic face movements (Action Units) that are associated with high recognition accuracy and judgments of human-likeness, including those that further boost performance. Our results therefore demonstrate the utility of using data-driven methods that employ human cultural perception to derive culturally-sensitive facial expressions that improve the social face signal generation capabilities of social robots. We anticipate that these methods will continue to inform the design of social robots and broaden their usability and global marketability.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
The weirdest people in the world?
Henrich, J., Heine, S. J., & Norenzayan, A. (2010) · Behavioral and Brain Sciences · doi:10.1017/s0140525x0999152x · Cited by 12,641
Nonparametric permutation tests for functional neuroimaging: A primer with examples
Nichols, T. E., & Holmes, A. P. (2002) · Human Brain Mapping · doi:10.1002/hbm.1058 · Cited by 6,256
Hall, E. T. (1973) · Leonardo · doi:10.2307/1572461 · Cited by 4,859
A survey of socially interactive robots
Fong, T., Nourbakhsh, I., & Dautenhahn, K. (2003) · Robotics and Autonomous Systems · doi:10.1016/s0921-8890(02)00372-x · Cited by 3,162
Elements of Information Theory
Nalewajski, R. F. (2011) · Perspectives in Electronic Structure Theory · doi:10.1007/978-3-642-20180-6_8 · Cited by 2,309
Universals and cultural differences in facial expressions of emotion.
Ekman, P. (1972) · Medical Entomology and Zoology · Cited by 2,273
On the universality and cultural specificity of emotion recognition: a meta-analysis
Elfenbein, H. A., & Ambady, N. (2002) · Psychological Bulletin · doi:10.1037/0033-2909.128.2.203 · Cited by 2,049
Automatic analysis of facial expressions: the state of the art
Pantie, M., & Rothkrantz, L. J. M. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.895976 · Cited by 1,778
Pan-cultural elements in facial displays of emotion
Ekman, P., Sorenson, E. R., & Friesen, W. V. (1969) · Science · doi:10.1126/science.164.3875.86 · Cited by 1,775
Universals and cultural differences in the judgments of facial expressions of emotion.
Ekman, P., Friesen, W. V., O'Sullivan, M., Chan, A., Diacoyanni-Tarlatzis, I., Heider, K., Krause, R., LeCompte, W. A., Pitcairn, T., Ricci-Bitti, P. E., et al. (1987) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.53.4.712 · Cited by 1,763
Research works printed but not linked: 2
- 10.1037/0022-3514.53.4.712 · DOI · Paper page
- ekman, 1978, Manual for the Facial Action Coding System
Cited by
Conversational Affective Social Robots for Ageing and Dementia Support
Lima, M. R., Wairagkar, M., Gupta, M., Rodriguez y Baena, F., Barnaghi, P., Sharp, D. J., & Vaidyanathan, R. (2022) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2021.3115228 · Cited by 75
Zhan, J., Liu, M., Garrod, O., Daube, C., Ince, R. A. A., Jack, R. E., & Schyns, P. G. (2021) · Current Biology · doi:10.1016/j.cub.2021.03.013 · Cited by 56
Self-citationHuman-robot facial coexpression
Hu, Y., Chen, B., Lin, J., Wang, Y., Wang, Y., Mehlman, C., & Lipson, H. (2024) · Science Robotics · doi:10.1126/scirobotics.adi4724 · Cited by 73
Liu, M., Duan, Y., Ince, R. A. A., Chen, C., Garrod, O., Schyns, P. G., & Jack, R. E. (2021) · Current Biology · doi:10.1016/j.cub.2021.10.035 · Cited by 48
Self-citationCultural facial expressions dynamically convey emotion category and intensity information
Chen, C., Messinger, D. S., Cheng, C., Yan, H., Duan, Y., Ince, R. A. A., Garrod, O., Schyns, P. G., & Jack, R. E. (2023) · Current Biology · doi:10.1016/j.cub.2023.12.001 · Cited by 37
Self-citation
Metadata
- Authors
- Chen, C., Hensel, L. B., Duan, Y., Ince, R. A. A., Garrod, O. G. B., Beskow, J., Jack, R. E., & Schyns, P. G.
- Affiliations
- University of Glasgow (GB)
- ORCID
- Venue
- 2019 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019)
- Publisher
- IEEE
- Published
- 2019-05
- Pages
- 1-8
- ISSN
- 1541-5058 (ISSN-L)
- OpenAlex
- W2922943851
- Cited by
- 41
- References
- 52
- Citations by year
- 2026: 4 · 2025: 2 · 2024: 5 · 2023: 6 · 2022: 3 · 2021: 12 · 2020: 8 · 2019: 1
- FWCI
- 9.90
- Type
- conference paper
- Language
- en
- Open access
- green
- eprints.gla.ac.uk
- Licence
- Locations
- International Conference on Automatic Face and Gesture Recognition (publishedVersion)
- ENLIGHTEN (Jurnal Bimbingan dan Konseling Islam) (acceptedVersion, cc-by-sa)
- OSF Preprints (OSF Preprints) (submittedVersion)
- Funders
- Engineering and Physical Sciences Research Council (EP/N019261/1)
- Economic and Social Research Council (1944569)
- Topics
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Life Sciences › Neuroscience › Cognitive Neuroscience › Face Recognition and Perception
- Keywords
- social robots, emotional facial expressions, cultural perception, East Asian culture, facial action units, universal facial expressions, human-likeness, social signaling, robot expressiveness
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Equipping social robots with culturally-sensitive facial expressions of emotion using data-driven methods. https://app.cochat.ai/papers/10.1109/fg.2019.8756570
- The paper
- Chen, C., Hensel, L. B., Duan, Y., Ince, R. A. A., Garrod, O. G. B., Beskow, J., Jack, R. E., & Schyns, P. G. (2019). Equipping social robots with culturally-sensitive facial expressions of emotion using data-driven methods. 2019 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019). https://doi.org/10.1109/fg.2019.8756570
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.