CoChat
Sign Up
Cited by 1,748Face Recognition and PerceptionEmotion and Mood RecognitionEvolutionary Psychology and Human Behavior

Emotional expressions reconsidered: Challenges to inferring emotion from human facial movements

Open access
Barrett, Lisa Feldman, Adolphs, Ralph, Marsella, Stacy, Martinez, Aleix M., Pollak, Seth D.·2019·Psychological Science in the Public Interest
DOI 10.1177/1529100619832930PMC PMC6640856
Open this paper in CoChat — free account

Abstract

It is commonly assumed that a person’s emotional state can be readily inferred from his or her facial movements, typically called emotional expressions or facial expressions . This assumption influences legal judgments, policy decisions, national security protocols, and educational practices; guides the diagnosis and treatment of psychiatric illness, as well as the development of commercial applications; and pervades everyday social interactions as well as research in other scientific fields such as artificial intelligence, neuroscience, and computer vision. In this article, we survey examples of this widespread assumption, which we refer to as the common view , and we then examine the scientific evidence that tests this view, focusing on the six most popular emotion categories used by consumers of emotion research: anger, disgust, fear, happiness, sadness, and surprise. The available scientific evidence suggests that people do sometimes smile when happy, frown when sad, scowl when angry, and so on, as proposed by the common view, more than what would be expected by chance. Yet how people communicate anger, disgust, fear, happiness, sadness, and surprise varies substantially across cultures, situations, and even across people within a single situation. Furthermore, similar configurations of facial movements variably express instances of more than one emotion category. In fact, a given configuration of facial movements, such as a scowl, often communicates something other than an emotional state. Scientists agree that facial movements convey a range of information and are important for social communication, emotional or otherwise. But our review suggests an urgent need for research that examines how people actually move their faces to express emotions and other social information in the variety of contexts that make up everyday life, as well as careful study of the mechanisms by which people perceive instances of emotion in one another. We make specific research recommendations that will yield a more valid picture of how people move their faces to express emotions and how they infer emotional meaning from facial movements in situations of everyday life. This research is crucial to provide consumers of emotion research with the translational information they require.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1177/1529100619832930

Cites

446 of 446 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Multimodal recognition extends inference to speech prosody, facial movement and physiology, but facial expressions are context-sensitive social cues rather than emotion readouts.

    “Multimodal recognition extends inference to speech prosody, facial movement and physiology, but facial expressions are context-sensitive social cues rather than emotion readouts [29]”

    From Caring Without Feeling: Affective Dynamics as the Control Layer of Human-AI Agent Collaboration (2026) · cites this paper as [29]

    The Summary and Recommendations section describes facial movements as context-dependent social information, not reliable readouts of particular emotions. Its Recommendations for Future Scientific Research also propose combining facial movement, tone of voice, and physiological measurements, but present this as a research approach rather than an established multimodal recognition capability. Page numbers are not provided in our copy. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • The findings allow no conclusions about whether the smiles were genuine expressions of felt joy.

    “However, our findings allow no conclusions about whether the smiles were “genuine” as expressions of felt joy (Feldman Barrett et al., 2019).”

    From Existential threat and responses to emotional displays of ingroup and outgroup members (2023) · cites this paper as Feldman Barrett et al., 2019

  • The notion that discrete emotion categories can be reliably identified by specific behavioural patterns and/or physiological features has been seriously called into doubt by marked variability within instances of the same category as well as similarities across categories.

    “The notion that discrete emotion categories can be reliably identified by specific behavioural patterns and/or physiological features has been seriously called into doubt by the marked variability within instances of the same category as well as similarities across categories.”

    From A new science of emotion: implications for functional neurological disorder (2022) · cites this paper as [57]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 56

  1. Adolphs R & Anderson D (2018). The Neuroscience of Emotion: A New Synthesis. Princeton University Press.
  2. Barrett LF (2013). Psychological construction: A Darwinian approach to the science of emotion. Emotion Review, 5, 379– 389.
  3. Barrett LF, Wilson-Mendenhall CD, & Barsalou LW (2015). The conceptual act theory: A road map Chapter in Barrett LF and Russell JA (Eds.), The psychological construction of emotion (p. 83– 110). New York: Guilford.
  4. Benitez-Quiroz F, Srinivasan R, & Martinez AM (in press). Discriminant Functional Learning of Color Features for the Recognition of Facial Action Units and their Intensities. IEEE transactions on pattern analysis and machine intelligence. PMC6880652 30188814
  5. Cacioppo JT, Berntson GG, Larsen JH, Poehlmann KM, & Ito TA (2000). The psychophysiology of emotion In Lewis R & Haviland-Jones JM (Eds.), The handbook of emotion (2 ed., pp. 173– 191). New York: Guilford Press.
  6. Calvo MG, Gutierrez-Garcia A, Fernandez-Martin A, and Nummenmaa L (2014). Recognition of facial expressions of emotion is related to their frequency in everyday life. Journal of Nonverbal Behavior, 38, 549– 567.
  7. Campos JJ, Campos RG, & Barrett KC (1989). Emergent themes in the study of emotional development and emotion regulation. Developmental Psychology, 25(3), 394.
  8. Camras LA (1991). Conceptualizing early infant affect: View II and Reply In Strongman K (Ed.), International review of studies on emotion (pp. 16– 28, 33–36). New York: Wiley
  9. Camras Fatani, Fraumeni & Shuster, 2016. The development of facial expressions In Barrett LF, Lewis M, & Haviland-Jones JM (Eds.). (2016). Handbook of Emotions, 4th Edition (255– 271). New York: Guilford.
  10. Ceulemans E, Kuppens P, & Mechelen IV (2012). Capturing the Structure of Distinct Types of Individual Differences in the Situation-specific Experience of Emotions: The Case of Anger. European Journal of Personality, 26(5), 484– 495.
  11. Clore GL, & Ortony A (2008). Appraisal theories: How cognition shapes affect into emotion In Lewis M, Haviland-Jones JM, & Barrett LF (Eds.), Handbook of Emotions (3rd ed., pp. 628– 642). New York, NY: Guilford Press.
  12. Craig AD (2015). How do you feel?: an interoceptive moment with your neurobiological self. Princeton University Press.
  13. Darwin C (1859/1964). On the Origin of Species (A facsimile of the first edition. ed.). Cambridge, MA: Harvard University Press.
  14. Dashiell JF (1927). A new method of measuring reactions to facial expression of emotion. Psychological Bulletin, 24, 174– 175.
  15. de Gelder B (2016). Emotions and the body. New York: Oxford University Press.
  16. Denham SA (1998). Emotional development in young children. Guilford Press.
  17. Dondi M, Gervasi MT, Valente A, Vacca T, Bogana G, De Beilis I, … & Oster H. (2012, 7). Spontaneous facial expressions of distress in fetuses. In 14th European Conference on facial Expression (pp. 16– 18).
  18. Duran JI, Reisenzein R, & Fernandez-Dols J-M. (2017). Coherence between emotions and facial expressions: A research synthesis In Fernandez-Dols J-M & Russell JA (Eds.), The science of facial expression (p. 107– 129). NY: Oxford U Press.
  19. Ekman P (2017). Facial expressions In Femandez-Dols J-M & Russell JA (Eds.), The science of facial expression (p. 39– 56). NY: Oxford U Press.
  20. Ekman P, Friesen WV (1976) Pictures of Facial Affect (Consulting Psychologists Press, Palo Alto, CA).
  21. Ekman P, Friesen WV (1978) Facial Action Coding System: A Technique for the Measurement of Facial Movement (Consulting Psychologists Press, Palo Alto, CA).
  22. Feng D, Jeong D, Krämer N, Miller L and Marsella S " Is It Just Me?: Evaluating Attribution of Negative Feedback as a Function of Virtual Instructor's Gender and Proxemics", in Proceedings of the 16th Conference on Autonomous Agents and MultiAgent Systems, 5 2017, pp. 810– 818, International Foundation for Autonomous Agents and Multiagent Systems
  23. Gross D, & Harris PL (1988). False beliefs about emotion: Children's understanding of misleading emotional displays. International Journal of Behavioral Development, 11(4), 475– 488.
  24. Hess U, & Hareli S (2017). The social signal value of emotions: The role of contextual factors in social inferences drawn from emotion displays In Fernandez-Dols J-M & Russell JA (Eds.), The science of facial expression (p. 375– 392). NY: Oxford U Press.
  25. Izard CE (2010). The many meanings/aspects of emotion: Definitions, functions, activation, and regulation. Emotion Review, 2(4), 363– 370.
  26. Kärtner J, Holodynski M, & Wörmann V (2013). Parental ethnotheories, social practice and the culture-specific development of social smiling in infants. Mind, Culture, and Activity, 20(1), 79– 95.
  27. Kim J, Hill R, Durlach P, Lane H, Forbell E, Core M, Marsella S, Pynadath D, and and Hart J (2009). " BiLAT: A Game-Based Environment for Practicing Negotiation in a Cultural Context. Special Issue on Ill-Defined Domains", International Journal of AI in Education, vol. 19, no. 3, pp. 289– 308
  28. Landau B, & Shipley E (2001). Labelling patterns and object naming. Developmental Science, 4(1), 109– 118.
  29. Lazarus R (1991). Emotion and Adaptation. NY, Oxford University Press.
  30. Lee DH & Anderson AK (2017). Form and function of facial expressive origins In Fernandez-Dols J-M & Russell JA(Eds.), The science of facial expression (p. 173– 194). NY: Oxford U Press.
  31. Lindquist K, & Barrett LF (2008). Emotional complexity. Chapter in Lewis M, Haviland-Jones JM, and Barrett LF (Eds.), The handbook of emotion, 3rd Edition (p. 513– 530). New York: Guilford.
  32. Marsella S & Gratch J (2016). Computational models of emotion as psychological tools In Barrett LF, Lewis M and Haviliand-Jones J (Eds.) Handbook of Emotions, 4th Edition. New York: Guilford Press.
  33. Matsumoto D, Keltner D, Shiota M, Frank M, & O'Sullivan M (2008). Facial expressions of emotions In Lewis M, Haviland-Jones JM, & Barrett LF (Eds.), Handbook of Emotion, 3rd Ed. (pp. 211– 234). New York: Macmillan.
  34. Mayr E (2004). What makes biology unique? Considerations on the autonomy of a scientific discipline. New York: Cambridge University Press.
  35. Miller GA (1995). WordNet: a lexical database for English. Communications of the ACM, 38(11), 39– 41.
  36. Montagu J (1994). The expression of the passions: The origin and influence of Charles Le Brun's lecture on general and particular expression. London: Yale University Press.
  37. Neidle Carol, Kegl Judy A., Dawn MacLaughlin Benjamin Bahan & Lee Robert G. 2000 The syntax of American Sign Language. Cambridge: MIT Press.
  38. Nelson NL, & Russell JA (2013). Universality revisited. Emotion Review, 5, 8– 15.
  39. Ortony A, Clore GL, Collins A (1990). The cognitive structure of emotions. Cambridge University Press.
  40. Oster H (2005). The repertoire of infant facial expressions: an ontogenetic perspective In Nadel J & Muir D (Eds.), Emotional development: Recent research advances (pp. 261– 292). New York, NY, US: Oxford University Press.
  41. Panksepp J (1998). Affective neuroscience: The foundations of human and animal emotions. New York: Oxford University Press.
  42. Pollak SD (2009) The emergence of emotion: Experience, development and biology. In Cicchetti D and Gunnar MR (Eds.), Minnesota Symposia on Child Psychology: Meeting the challenge of translational research in child psychology (Vol 35, p. 189– 238). Hoboken, NJ: Wiley.
  43. Rosenberg EL, & Ekman P (1995). Conceptual and methodological issues in the judgment of facial expressions of emotion. Motivation and Emotion, 19(2), 111– 138.
  44. Röttger-Rössler B, Scheidecker G, Funk L, & Holodynski M (2015). Learning (by) feeling: A cross-cultural comparison of the socialization and development of emotions. Ethos, 43(2), 187– 220.
  45. Russell JA, & Widen SC (2002a). A label superiority effect in children's categorization of facial expressions. Social Development, 11(1), 30– 52.
  46. Russell JA, & Widen SC (2002b). Words versus faces in evoking preschool children’s knowledge of the causes of emotions. International Journal of Behavioral Development, 26(2), 97– 103.
  47. Scarantino A (2016). The philosophy of emotions and its impact on affective science In Barrett LF, Lewis M and Haviland-Jones JM (Eds.), Handbook of Emotions, 4th Ed. (p. 3– 48). New York: Guilford Press.
  48. Scarantino A, & Griffiths P (2011). Don’t give up on basic emotions. Emotion Review, 3(4), 444– 454.
  49. Searle JR (1992). The rediscovery of the mind: MIT press.
  50. Searle JR (2004). Mind: a brief introduction: Oxford University Press.
  51. Valliant-Molina M & Bahrick LE (2012) The role of intersensory redundancy in the emergence of social referencing in 5½-month-old infants. Developmental Psychology, 48, 1– 9. 21895359 PMC3966294
  52. Walker-Andrews AS, & Lennon E (1991). Infants’ discrimination of vocal expressions: Contributions of auditory and visual information. Infant Behavior and Development, 14(2), 131– 142.
  53. Wallbott HG (1988). In and out of context: Influences of facial expression and context information on emotion attributions. British Journal of Social Psychology, 27, 357– 369.
  54. Widen SC (2016). The development of children's concepts of emotion In Barrett LF, Lewis M, J. M., & Haviland-Jones (Eds.), Handbook of emotions (4th ed., pp. 307– 318). New York: Guilford Press.
  55. Yin Y, Nabian M, Ostadabbas S, Fan M, Chou C, & Gendron M (2018). Facial expression and peripheral physiology fusion to decode individualized affective experiences. 2nd IJCAI Workshop on Artificial Intelligence in Affective Computing: Proceedings of Marching Learning Research, 1, 1– 16.
  56. Zafeiriou S, Papaioannou A, Kotsia I, Nicolaou M, & Zhao G (2016). Facial Affect‘‘In-The-Wild. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (pp. 36– 47).

Cited by

Sorted by citations

Related

Authors
Barrett, L. F., Adolphs, R., Marsella, S., Martinez, A. M., & Pollak, S. D.
Affiliations
  • Northeastern University (US)
  • California Institute of Technology (US)
  • Department of Electrical and Computer Engineering and Center for Cognitive and Brain Sciences, The Ohio State University
  • University of Wisconsin–Madison (US)
Corresponding author
Barrett, Lisa Feldman
ORCID
Venue
Psychological Science in the Public Interest
Publisher
SAGE Publications
Published
2019-07
Volume
20
Issue
1
Pages
1-68
ISSN
  • 1529-1006 (ISSN-L)
  • 2160-0031
DOI
10.1177/1529100619832930
PMC
PMC6640856
OpenAlex
W2912483755
Semantic Scholar
c489b6787c5af8aca97f4761343a66f3f189b35d
Cited by
1,748
References
446
Citations by year
2026: 238 · 2025: 266 · 2024: 230 · 2023: 302 · 2022: 269 · 2021: 227 · 2020: 186 · 2019: 26 · 2018: 3 · 2015: 1
FWCI
102.82
Type
journal article
Language
en
Update notices
correction (2019-11-15)
Open access
Licence
journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
Locations
  • Psychological Science in the Public Interest (publishedVersion)
  • PubMed (publishedVersion)
  • PubMed Central (acceptedVersion)
  • ENLIGHTEN (Jurnal Bimbingan dan Konseling Islam) (acceptedVersion, cc-by-sa)
  • CaltechAUTHORS (California Institute of Technology) (submittedVersion)
Funders
  • National Institute of Mental Health (R01MH113234, P50MH094258, R01MH109464, R01MH061285)
  • National Eye Institute (R01EY020834)
  • National Cancer Institute (U01CA193632)
  • National Institute on Deafness and Other Communication Disorders (R01DC014498)
  • Eunice Kennedy Shriver National Institute of Child Health and Human Development (U54HD090256)
Topics
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Face Recognition and Perception
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Emotion and Mood Recognition
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Evolutionary Psychology and Human Behavior
Keywords
facial expressions, emotion recognition, facial motion, emotion categories, mixed emotions, social communication, emotion perception, anger, disgust, fear, happiness, sadness, surprise
MeSH terms
Facial Recognition, Emotions, Facial Expression, Female, Humans, Interpersonal Relations, Judgment, Male, Movement, Psychomotor Performance

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review