The DOI matches its Crossref record; 45 of the 47 references it cites were found in a registry or an index.
Measuring facial expressions by computer image analysis
Abstract
Facial expressions provide an important behavioral measure for the study of emotion, cognitive processes, and social interaction. The Facial Action Coding System (Ekman & Friesen, 1978) is an objective method for quantifying facial movement in terms of component actions. We applied computer image analysis to the problem of automatically detecting facial actions in sequences of images. Three approaches were compared: holistic spatial analysis, explicit measurement of features such as wrinkles, and estimation of motion flow fields. The three methods were combined in a hybrid system that classified six upper facial actions with 91% accuracy. The hybrid system outperformed human nonexperts on this task and performed as well as highly trained experts. An automated system would make facial expression measurement more widely accessible as a research tool in behavioral science and investigations of the neural substrates of emotion.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Ekman defined six such categories, referred to as the basic emotions: happiness, sadness, surprise, fear, anger, and disgust [19].”
From Automatic analysis of facial expressions: the state of the art (2000) · cites this paper as [19]
In the citing paper
“He described each basic emotion in terms of a facial expression that uniquely characterizes that emotion.”
From Automatic analysis of facial expressions: the state of the art (2000) · cites this paper as [19]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Neural Networks: A Comprehensive Foundation
Haykin, S. (1998) · Cited by 30,070
Turk, M., & Pentland, A. (1991) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.1991.3.1.71 · Cited by 13,733
Eigenfaces vs. Fisherfaces: recognition using class specific linear projection
Belhumeur, P. N., Hespanha, J. P., & Kriegman, D. J. (1997) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.598228 · Cited by 11,798
Horn, B. K. P., & Schunck, B. G. (1981) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.965761 · Cited by 7,443
Autonomic nervous system activity distinguishes among emotions
Ekman, P., Levenson, R. W., & Friesen, W. V. (1983) · Science · doi:10.1126/science.6612338 · Cited by 2,692
Self-citationFace recognition: features versus templates
Brunelli, R., & Poggio, T. (1993) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.254061 · Cited by 2,452
A differential neural response in the human amygdala to fearful and happy facial expressions
Morris, J. S., Frith, C. D., Perrett, D. I., Rowland, D., Young, A. W., Calder, A. J., & Dolan, R. J. (1996) · Nature · doi:10.1038/383812a0 · Cited by 2,013
Ekman, P., & Rosenberg, E. L. (2005) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780195179644.001.0001 · Cited by 1,971
Self-citationResponse and Habituation of the Human Amygdala during Visual Processing of Facial Expression
Breiter, H. C., Etcoff, N. L., Whalen, P. J., Kennedy, W. A., Rauch, S. L., Buckner, R. L., Strauss, M. M., Hyman, S. E., & Rosen, B. R. (1996) · Neuron · doi:10.1016/s0896-6273(00)80219-6 · Cited by 1,711
Approach-withdrawal and cerebral asymmetry: Emotional expression and brain physiology: I.
Davidson, R. J., Ekman, P., Saron, C. D., Senulis, J. A., & Friesen, W. V. (1990) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.58.2.330 · Cited by 1,684
Self-citationA specific neural substrate for perceiving facial expressions of disgust
Phillips, M. L., Young, A. W., Senior, C., Brammer, M., Andrew, C., Calder, A. J., Bullmore, E. T., Perrett, D. I., Rowland, D., Williams, S. C. R., et al. (1997) · Nature · doi:10.1038/39051 · Cited by 1,609
Ekman, P., & Oster, H. (1979) · Annual Review of Psychology · doi:10.1146/annurev.ps.30.020179.002523 · Cited by 1,228
Self-citationAdolphs, R., Tranel, D., Damasio, H., & Damasio, A. (1995) · The Journal of Neuroscience · doi:10.1523/jneurosci.15-09-05879.1995 · Cited by 1,138
(1984) · doi:10.4324/9781315798806-24 · Cited by 1,061
Self-citationDonato, G., Bartlett, M. S., Hager, J. C., Ekman, P., & B, T. J. S. A. (1999) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.799905 · Cited by 999
Self-citationCoding, analysis, interpretation, and recognition of facial expressions
Essa, I. A., & Pentland, A. (1997) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.598232 · Cited by 917
Cortical Systems for the Recognition of Emotion in Facial Expressions
Adolphs, R., Damasio, H., Tranel, D., & Damasio, A. R. (1996) · Journal of Neuroscience · doi:10.1523/jneurosci.16-23-07678.1996 · Cited by 726
Bassili, J. N. (1979) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.37.11.2049 · Cited by 663
Hasselmo, M. E., Rolls, E. T., & Baylis, G. C. (1989) · Behavioural Brain Research · doi:10.1016/s0166-4328(89)80054-3 · Cited by 657
Automatic interpretation and coding of face images using flexible models
Lanitis, A., Taylor, C., & Cootes, T. F. (1997) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.598231 · Cited by 629
Analysis and synthesis of facial image sequences using physical and anatomical models
Terzopoulos, D., & Waters, K. (1993) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.216726 · Cited by 538
Handbook of methods in nonverbal behavior research
Scherer, K. R., & Ekman, P. (1982) · Cited by 521
Self-citationRecognizing human facial expressions from long image sequences using optical flow
Yacoob, Y., & Davis, L. S. (1996) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.506414 · Cited by 463
Linear object classes and image synthesis from a single example image
Vetter, T., & Poggio, T. (1997) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.598230 · Cited by 410
Recognition of Facial Expression from Optical Flow
Mase, K. (1991) · IEICE Transactions on Information and Systems · Cited by 404
Face recognition under varying pose
Beymer (1994) · doi:10.1109/cvpr.1994.323893 · Cited by 376
Connectionist models of face processing: A survey
Valentin, D., Abdi, H., O’Toole, A. J., & Cottrell, G. W. (1994) · Pattern Recognition · doi:10.1016/0031-3203(94)90006-x · Cited by 370
3-D motion estimation in model-based facial image coding
Li, H., Roivainen, P., & Forchheimer, R. (1993) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.216724 · Cited by 335
The argument and evidence about universals in facial expressions of emotion.
Ekman, P. (1989) · Cited by 311
Self-citationStructural aspects of face recognition and the other-race effect
O’Toole, A. J., Deffenbacher, K. A., Valentin, D., & Abdi, H. (1994) · Memory & Cognition · doi:10.3758/bf03208892 · Cited by 308
Tranel, D., Damásio, A. R., & Damásio, H. (1988) · Neurology · doi:10.1212/wnl.38.5.690 · Cited by 305
Cohn, J. F., Zlochower, A. J., Lien, J. J.-J., & Takeo Kanade, A. (1999) · Psychophysiology · doi:10.1017/s0048577299971184 · Cited by 287
Human expression recognition from motion using a radial basis function network architecture
Rosenblum, M., Yacoob, Y., & Davis, L. S. (1996) · IEEE Transactions on Neural Networks · doi:10.1109/72.536309 · Cited by 284
Face processing: Human perception and principal components analysis
Hancock, P., Burton, A. M., & Bruce, V. (1996) · Memory & Cognition · doi:10.3758/bf03197270 · Cited by 242
Computer recognition of human faces
Kanade (1977) · Cited by 208
Yaxley, S., Rolls, E. T., & Sienkiewicz, Z. J. (1988) · Physiology & Behavior · doi:10.1016/0031-9384(88)90074-1 · Cited by 206
Functional anatomy of taste perception in the human brain studied with positron emission tomography
Kinomura, S., Kawashima, R., Yamada, K., Ono, S., Itoh, M., Yoshioka, S., Yamaguchi, T., Matsui, H., Miyazawa, H., Itoh, H., et al. (1994) · Brain Research · doi:10.1016/0006-8993(94)90890-7 · Cited by 174
Representing Face Images for Emotion Classification
Padgett, C. W., & Cottrell, G. W. (1996) · neural information processing systems · Cited by 171
Visual Speech Recognition with Stochastic Networks
Movellan, J. R. (1994) · Cited by 147
Li, C.-C., & Lien, J.-J. J. (1998) · Cited by 99
Facial Expressions Used by Children in a Conflict Situation
Camras, L. A. (1977) · Child Development · doi:10.1111/j.1467-8624.1977.tb03950.x · Cited by 97
Bartlett, M. S., Viola, P. A., B, T. J. S. A., Golomb, B. A., Larsen, J. O., Hager, J. C., & Ekman, P. (1995) · Cited by 97
Self-citationAutomated coding of facial behavior in human-computer interactions with facs
Kaiser, S., & Wehrle, T. (1992) · Journal of Nonverbal Behavior · doi:10.1007/bf00990323 · Cited by 47
Human Face Perception and Identification
Bruce, V., Hancock, P., & Burton, A. M. (1998) · Springer eBooks · doi:10.1007/978-3-642-72201-1_3 · Cited by 46
Effects of distortion of spatial and temporal resolution of video stimuli on emotion attributions
Wallbott, H. G. (1992) · Journal of Nonverbal Behavior · doi:10.1007/bf00986876 · Cited by 34
Flagged first, then by citations
2 of the 47 references couldn’t be matched to a known paper.
Research works printed but not linked: 11
- Beymer, D., Shashua, A., & Poggio, T. ~ 1993 ! . Example based image analysis and synthesis ~ AI Memo No. 1431 ! . Cambridge, MA: MIT Press.
- Cottrell, G. W., & Fleming, M. K. ~ 1990 ! . Face recognition using unsupervised feature extraction. Proceedings of the International Neural Network Conference ~ pp. 322-325 ! . Paris: Kluwer.
- Cottrell, G., & Metcalfe, J. ~ 1991 ! . EMPATH: Face, gender and emotion recognition using holons. In R. P. Lippman, J. Moody, & D. Touretzky ~ Eds. ! , Advances in neural information processing systems ~ Vol. 3, pp. 564-571 ! . San Mateo, CA: Morgan & Kaufman.
- Ekman, P. ~ Ed. ! . ~ 1982a ! . Emotion in the human face ~ 2nd ed. ! . New York: Cambridge University Press.
- Ekman, P., & Friesen, W. ~ 1978 ! . Facial Action Coding System: A technique for the measurement of facial movement. Palo Alto, CA: Consulting Psychologists Press.
- Ekman, P., Huang, T., Sejnowski, T., & Hager, J. ~ 1992 ! . Final Report to NSF of the Planning Workshop on Facial Expression Understanding @ on-line # . Available: http: 00 alt.xmission.com 0 nirc1 0 Research 0 research.html
- Hager, J., & Ekman, P. ~ 1995 ! . The essential behavioral science of the face and gesture that computer scientists need to know. In M. Bichsel ~ Ed. ! , Proceedings of the International Workshop on Automatic Face and Gesture Recognition ~ pp. 7-11 ! . Available: http: 00 alt.xmission.com 0 nirc1 0 Research 0 research.html
- Himer, W., Schneider, F., Kost, G., & Heimann, H. ~ 1991 ! . Computerbased analysis of facial action: A new approach. Journal of Psychophysiology , 5 , 189-195.
- Jain, R., Kasturi, R., & Schunk, B. G. ~ 1995 ! . Machine vision. New York: McGraw-Hill.
- Tukey, J. W. ~ 1958 ! . Bias and confidence in not-quite large samples. Annals of Mathematical Statistics , 29 , 614.
- Zajonc, R. B. ~ 1984 ! . The interaction of affect and cognition. In K. Scherer & P. Ekman ~ Eds. ! , Approaches to emotion ~ pp. 239-246 ! . Hillsdale, NJ: Erlbaum.
Cited by
Sorted by citations
Comprehensive database for facial expression analysis
Takeo Kanade, A., Cohn, J. F., & Tian, Y. (2002) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/afgr.2000.840611 · Cited by 2,644
Emotion recognition in human-computer interaction
Cowie, R., Douglas-Cowie, E., Tsapatsoulis, N., Votsis, G., Kollias, S., Fellenz, W., & Taylor, J. G. (2001) · IEEE Signal Processing Magazine · doi:10.1109/79.911197 · Cited by 2,603
Toward machine emotional intelligence: analysis of affective physiological state
Picard, R. W., Vyzas, E., & Healey, J. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.954607 · Cited by 2,334
Automatic analysis of facial expressions: the state of the art
Pantie, M., & Rothkrantz, L. J. M. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.895976 · Cited by 1,778
Recognizing action units for facial expression analysis
Tian, Y.-I., Kanade, T., & Cohn, J. F. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.908962 · Cited by 1,745
Metadata
- Authors
- Bartlett, M. S., Hager, J. C., Ekman, P., & Sejnowski, T. J.
- Affiliations
- Salk Institute for Biological Studies (US)
- University of California, San Francisco (US)
- Corresponding author
- Bartlett, Marian Stewart
- Venue
- Psychophysiology
- Publisher
- Wiley
- Published
- 1999-03
- Volume
- 36
- Issue
- 2
- Pages
- 253-263
- ISSN
- 0048-5772 (ISSN-L)
- 1469-8986
- OpenAlex
- W2007431552
- Cited by
- 424
- References
- 47
- Citations by year
- 2026: 7 · 2025: 8 · 2024: 13 · 2023: 11 · 2022: 34 · 2021: 23 · 2020: 16 · 2019: 23 · 2018: 14 · 2017: 20 · 2016: 14 · 2015: 17 · 2014: 19 · 2013: 12 · 2012: 12
- FWCI
- 19.99
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychophysiology (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
- Life Sciences › Neuroscience › Cognitive Neuroscience › Face Recognition and Perception
- Keywords
- facial expressions, facial action units, facial action coding system, computer image analysis, wrinkles, hybrid classification system, emotion measurement
- MeSH terms
- Adult, Cues, Facial Expression, Female, Humans, Image Processing, Computer-Assisted, Male, Middle Aged, Databases as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
