The DOI matches its Crossref record; 45 of the 47 references it cites were found in a registry or an index.
Measuring facial expressions by computer image analysis
Abstract
Facial expressions provide an important behavioral measure for the study of emotion, cognitive processes, and social interaction. The Facial Action Coding System (Ekman & Friesen, 1978) is an objective method for quantifying facial movement in terms of component actions. We applied computer image analysis to the problem of automatically detecting facial actions in sequences of images. Three approaches were compared: holistic spatial analysis, explicit measurement of features such as wrinkles, and estimation of motion flow fields. The three methods were combined in a hybrid system that classified six upper facial actions with 91% accuracy. The hybrid system outperformed human nonexperts on this task and performed as well as highly trained experts. An automated system would make facial expression measurement more widely accessible as a research tool in behavioral science and investigations of the neural substrates of emotion.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Ekman defined six such categories, referred to as the basic emotions: happiness, sadness, surprise, fear, anger, and disgust [19].”
From Automatic analysis of facial expressions: the state of the art (2000) · cites this paper as [19]
In the citing paper
“He described each basic emotion in terms of a facial expression that uniquely characterizes that emotion.”
From Automatic analysis of facial expressions: the state of the art (2000) · cites this paper as [19]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Neural Networks: A Comprehensive Foundation
Haykin, S. (1998) · Cited by 30,070
Turk, M., & Pentland, A. (1991) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.1991.3.1.71 · Cited by 13,733
Eigenfaces vs. Fisherfaces: recognition using class specific linear projection
Belhumeur, P. N., Hespanha, J. P., & Kriegman, D. J. (1997) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.598228 · Cited by 11,798
Horn, B. K. P., & Schunck, B. G. (1981) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.965761 · Cited by 7,443
Autonomic nervous system activity distinguishes among emotions
Ekman, P., Levenson, R. W., & Friesen, W. V. (1983) · Science · doi:10.1126/science.6612338 · Cited by 2,692
Self-citationFace recognition: features versus templates
Brunelli, R., & Poggio, T. (1993) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.254061 · Cited by 2,452
A differential neural response in the human amygdala to fearful and happy facial expressions
Morris, J. S., Frith, C. D., Perrett, D. I., Rowland, D., Young, A. W., Calder, A. J., & Dolan, R. J. (1996) · Nature · doi:10.1038/383812a0 · Cited by 2,013
Ekman, P., & Rosenberg, E. L. (2005) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780195179644.001.0001 · Cited by 1,971
Self-citationResponse and Habituation of the Human Amygdala during Visual Processing of Facial Expression
Breiter, H. C., Etcoff, N. L., Whalen, P. J., Kennedy, W. A., Rauch, S. L., Buckner, R. L., Strauss, M. M., Hyman, S. E., & Rosen, B. R. (1996) · Neuron · doi:10.1016/s0896-6273(00)80219-6 · Cited by 1,711
Approach-withdrawal and cerebral asymmetry: Emotional expression and brain physiology: I.
Davidson, R. J., Ekman, P., Saron, C. D., Senulis, J. A., & Friesen, W. V. (1990) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.58.2.330 · Cited by 1,684
Self-citation
Research works printed but not linked: 11
- Beymer, D., Shashua, A., & Poggio, T. ~ 1993 ! . Example based image analysis and synthesis ~ AI Memo No. 1431 ! . Cambridge, MA: MIT Press.
- Cottrell, G. W., & Fleming, M. K. ~ 1990 ! . Face recognition using unsupervised feature extraction. Proceedings of the International Neural Network Conference ~ pp. 322-325 ! . Paris: Kluwer.
- Cottrell, G., & Metcalfe, J. ~ 1991 ! . EMPATH: Face, gender and emotion recognition using holons. In R. P. Lippman, J. Moody, & D. Touretzky ~ Eds. ! , Advances in neural information processing systems ~ Vol. 3, pp. 564-571 ! . San Mateo, CA: Morgan & Kaufman.
- Ekman, P. ~ Ed. ! . ~ 1982a ! . Emotion in the human face ~ 2nd ed. ! . New York: Cambridge University Press.
- Ekman, P., & Friesen, W. ~ 1978 ! . Facial Action Coding System: A technique for the measurement of facial movement. Palo Alto, CA: Consulting Psychologists Press.
- Ekman, P., Huang, T., Sejnowski, T., & Hager, J. ~ 1992 ! . Final Report to NSF of the Planning Workshop on Facial Expression Understanding @ on-line # . Available: http: 00 alt.xmission.com 0 nirc1 0 Research 0 research.html
- Hager, J., & Ekman, P. ~ 1995 ! . The essential behavioral science of the face and gesture that computer scientists need to know. In M. Bichsel ~ Ed. ! , Proceedings of the International Workshop on Automatic Face and Gesture Recognition ~ pp. 7-11 ! . Available: http: 00 alt.xmission.com 0 nirc1 0 Research 0 research.html
- Himer, W., Schneider, F., Kost, G., & Heimann, H. ~ 1991 ! . Computerbased analysis of facial action: A new approach. Journal of Psychophysiology , 5 , 189-195.
- Jain, R., Kasturi, R., & Schunk, B. G. ~ 1995 ! . Machine vision. New York: McGraw-Hill.
- Tukey, J. W. ~ 1958 ! . Bias and confidence in not-quite large samples. Annals of Mathematical Statistics , 29 , 614.
- Zajonc, R. B. ~ 1984 ! . The interaction of affect and cognition. In K. Scherer & P. Ekman ~ Eds. ! , Approaches to emotion ~ pp. 239-246 ! . Hillsdale, NJ: Erlbaum.
Cited by
Sorted by citations
Items 401–425 of 425
Synthesizing Natural and Believable Emotional Expressions
Tian, L., Oviatt, S. L., Muszyński, M., Chamberlain, B. C., Healey, J., & Sano, A. (2022) · ACM eBooks · doi:10.1145/3502398.3502406
Addressing Ethical Issues of Affective Computing
Tian, L., Oviatt, S. L., Muszyński, M., Chamberlain, B. C., Healey, J., & Sano, A. (2022) · ACM eBooks · doi:10.1145/3502398.3502410
Emotions as Studied in Psychology and Cognitive Science
Tian, L., Oviatt, S. L., Muszyński, M., Chamberlain, B. C., Healey, J., & Sano, A. (2022) · ACM eBooks · doi:10.1145/3502398.3502401
Facial Image Processing in Computer Vision
Yap, M. H., & Ugail, H. (2011) · IGI Global eBooks · doi:10.4018/978-1-60960-477-6.ch011
Emotion Recognition Expressed on the Face By Multimodal Method using Deep Learning
(2019) · International Journal of Engineering and Advanced Technology · doi:10.35940/ijeat.a1825.129219
(2004) · Kluwer Academic Publishers eBooks · doi:10.1007/b116723
Facial Expression Recognition using PCA Algorithm
Patil, S., & Katariya, S. (2015) · Communications on Applied Electronics · doi:10.5120/cae2015651904
Nobel Face Expression Recognition Technique
Pandey, A. K., & Gandhi, A. (2022) · International Conference on Contemporary Computing and Informatics (IC3I) · doi:10.1109/ic3i56241.2022.10072758
Head Movement Quantification and Its Role in Facial Expression Study
Yusoff, F. H., Rahmat, R. W. O. K., Sulaiman, M. N., Shaharom, M. H., & Majid, H. S. A. (2009) · Lecture notes in electrical engineering · doi:10.1007/978-90-481-3517-2_10
Normalized margin loss for action unit detection
Racoviţeanu, A., Florea, C., Florea, L., & Vertan, C. (2023) · Machine Vision and Applications · doi:10.1007/s00138-023-01490-3
Emotion in the Neutral Face: Applications for Computer Vision and Aesthetics
Albohn, D. N., & Brandenburg, J. C. (2023) · Springer eBooks · doi:10.1007/978-3-031-50269-9_9
Truffarelli, T., Valenti, C., Tribbiani, G., Pagano, S., Marsili, R., & Rossi, G. L. (2024) · Journal of Physics Conference Series · doi:10.1088/1742-6596/2802/1/012003
Facial Action Units and Personality Traits: An Explorative Approach through Computer Vision
Sibin Raj, K., & Mandal, M. K. (2025) · Journal of Contemporary Behavioural and Social Research · doi:10.4103/jcbsr.jcbsr_5_24
Wen, Z. (2004) · Illinois Digital Environment for Access to Learning and Scholarship (University of Illinois at Urbana-Champaign)
Databasing the Face: The Rise of Data-Centrism in Computerized Facial Expression Recognition
Kim, H. (2025) · IEEE Annals of the History of Computing · doi:10.1109/mahc.2025.3645105
L'influence de la texture d'un emballage : une approche par des méthodes explicites et implicites
Ferreira, B. E. M. (2014) · theses.fr (ABES) · doi:10.70675/a124d052z6c00z49ceza06az68aa3a37ae25 · Cited by 2
Analysis of Micro-Expressions based on the Riesz Pyramid : Application to Spotting and Recognition
Duque, C. A. (2018) · theses.fr (ABES) · doi:10.70675/7a5de57ezd0d3z4109zbcbezee32fab1669b · Cited by 1
Pavlić, I., Puh, B., & Bilandžić, A. (2026) · Ekonomska misao i praksa · doi:10.17818/emip/2026/24
Deception Detection Meets Vision-Language Models
Zhu, D., Hu, R., Wang, M., Guo, X., Liao, L., & Ye, M. (2026) · International Journal of Computer Vision · doi:10.1007/s11263-026-02932-x
Lee, E., Sim, S. H., Park, C., Kim, H., Ahn, W., & Park, C. H. K. (2026) · medRxiv · doi:10.64898/2026.07.19.26358451
Affect and mental state detection for mental health care
Lefter, I., & Sano, A. (2026) · Elsevier eBooks · doi:10.1016/b978-0-443-33151-0.00016-2
Scholz, S., & Kißler, J. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.08.05.742947
Fully Convolutional Masked Autoencoding with ConvNext V2 for Facial Action Unit Detection
Anuar, M. H. A., Bohani, F. A., & Diah, N. M. (2026) · IEEE International Conference on Signal and Image Processing Applications (ICSIPA) · doi:10.1109/icsipa69622.2026.11720131
Metadata
- Authors
- Bartlett, M. S., Hager, J. C., Ekman, P., & Sejnowski, T. J.
- Affiliations
- Salk Institute for Biological Studies (US)
- University of California, San Francisco (US)
- Corresponding author
- Bartlett, Marian Stewart
- Venue
- Psychophysiology
- Publisher
- Wiley
- Published
- 1999-03
- Volume
- 36
- Issue
- 2
- Pages
- 253-263
- ISSN
- 0048-5772 (ISSN-L)
- 1469-8986
- OpenAlex
- W2007431552
- Cited by
- 424
- References
- 47
- Citations by year
- 2026: 7 · 2025: 8 · 2024: 13 · 2023: 11 · 2022: 34 · 2021: 23 · 2020: 16 · 2019: 23 · 2018: 14 · 2017: 20 · 2016: 14 · 2015: 17 · 2014: 19 · 2013: 12 · 2012: 12
- FWCI
- 19.99
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychophysiology (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
- Life Sciences › Neuroscience › Cognitive Neuroscience › Face Recognition and Perception
- Keywords
- facial expressions, facial action units, facial action coding system, computer image analysis, wrinkles, hybrid classification system, emotion measurement
- MeSH terms
- Adult, Cues, Facial Expression, Female, Humans, Image Processing, Computer-Assisted, Male, Middle Aged, Databases as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
