Measuring facial expressions by computer image analysis
Abstract
Facial expressions provide an important behavioral measure for the study of emotion, cognitive processes, and social interaction. The Facial Action Coding System (Ekman & Friesen, 1978) is an objective method for quantifying facial movement in terms of component actions. We applied computer image analysis to the problem of automatically detecting facial actions in sequences of images. Three approaches were compared: holistic spatial analysis, explicit measurement of features such as wrinkles, and estimation of motion flow fields. The three methods were combined in a hybrid system that classified six upper facial actions with 91% accuracy. The hybrid system outperformed human nonexperts on this task and performed as well as highly trained experts. An automated system would make facial expression measurement more widely accessible as a research tool in behavioral science and investigations of the neural substrates of emotion.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Ekman defined six basic emotion categories: happiness, sadness, surprise, fear, anger, and disgust.
“Ekman defined six such categories, referred to as the basic emotions: happiness, sadness, surprise, fear, anger, and disgust [19].”
From Automatic analysis of facial expressions: the state of the art (2000) · cites this paper as [19]
Ekman described each basic emotion in terms of a facial expression that uniquely characterizes that emotion.
“He described each basic emotion in terms of a facial expression that uniquely characterizes that emotion.”
From Automatic analysis of facial expressions: the state of the art (2000) · cites this paper as [19]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Neural Networks: A Comprehensive Foundation
Haykin, S. (1998) · Cited by 29,735 · see citing works
Turk, M., & Pentland, A. (1991) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.1991.3.1.71 · Cited by 13,733 · see citing works
Eigenfaces vs. Fisherfaces: recognition using class specific linear projection
Belhumeur, P. N., Hespanha, J. P., & Kriegman, D. (1997) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.598228 · Cited by 11,776 · see citing works
Horn, B. K. P., & Schunck, B. G. (1981) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.965761 · Cited by 7,443 · see citing works
Autonomic nervous system activity distinguishes among emotions
Ekman, P., Levenson, R. W., & Friesen, W. V. (1983) · Science · doi:10.1126/science.6612338 · Cited by 2,692 · see citing works
Self-citationFace recognition: features versus templates
Brunelli, R., & Poggio, T. (1993) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.254061 · Cited by 2,452 · see citing works
A differential neural response in the human amygdala to fearful and happy facial expressions
Morris, J. S., Frith, C. D., Perrett, D. I., Rowland, D., Young, A. W., Calder, A. J., & Dolan, R. J. (1996) · Nature · doi:10.1038/383812a0 · Cited by 2,013 · see citing works
Ekman, P., & Rosenberg, E. L. (2005) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780195179644.001.0001 · Cited by 1,975 · see citing works
Self-citationResponse and Habituation of the Human Amygdala during Visual Processing of Facial Expression
Breiter, H. C., Etcoff, N. L., Whalen, P. J., Kennedy, W. A., Rauch, S. L., Buckner, R. L., Strauss, M. M., Hyman, S. E., & Rosen, B. R. (1996) · Neuron · doi:10.1016/s0896-6273(00)80219-6 · Cited by 1,711 · see citing works
Approach-withdrawal and cerebral asymmetry: Emotional expression and brain physiology: I.
Davidson, R. J., Ekman, P., Saron, C. D., Senulis, J. A., & Friesen, W. V. (1990) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.58.2.330 · Cited by 1,638 · see citing works
Self-citationA specific neural substrate for perceiving facial expressions of disgust
Phillips, M. L., Young, A. W., Senior, C., Brammer, M., Andrew, C., Calder, A. J., Bullmore, E. T., Perrett, D. I., Rowland, D., Williams, S. C. R., et al. (1997) · Nature · doi:10.1038/39051 · Cited by 1,609 · see citing works
Ekman, P., & Oster, H. (1979) · Annual Review of Psychology · doi:10.1146/annurev.ps.30.020179.002523 · Cited by 1,197 · see citing works
Self-citationAdolphs, R., Tranel, D., Damasio, H., & Damasio, A. (1995) · The Journal of Neuroscience · doi:10.1523/jneurosci.15-09-05879.1995 · Cited by 1,138 · see citing works
(1984) · doi:10.4324/9781315798806-24 · Cited by 1,061 · see citing works
Self-citationDonato, G., Bartlett, M. S., Hager, J. C., Ekman, P., & B, T. J. S. A. (1999) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.799905 · Cited by 999 · see citing works
Self-citationCoding, analysis, interpretation, and recognition of facial expressions
Essa, I. A., & Pentland, A. (1997) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.598232 · Cited by 906 · see citing works
Cortical Systems for the Recognition of Emotion in Facial Expressions
Adolphs, R., Damasio, H., Tranel, D., & Damasio, A. R. (1996) · Journal of Neuroscience · doi:10.1523/jneurosci.16-23-07678.1996 · Cited by 719 · see citing works
Hasselmo, M. E., Rolls, E. T., & Baylis, G. C. (1989) · Behavioural Brain Research · doi:10.1016/s0166-4328(89)80054-3 · Cited by 657 · see citing works
Automatic interpretation and coding of face images using flexible models
Lanitis, A., Taylor, C., & Cootes, T. F. (1997) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.598231 · Cited by 629 · see citing works
Analysis and synthesis of facial image sequences using physical and anatomical models
Terzopoulos, D., & Waters, K. (1993) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.216726 · Cited by 538 · see citing works
Bassili, J. N. (1979) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.37.11.2049 · Cited by 531 · see citing works
Handbook of methods in nonverbal behavior research
Scherer, K. R., & Ekman, P. (1982) · Cited by 515 · see citing works
Self-citationRecognizing human facial expressions from long image sequences using optical flow
Yacoob, Y., & Davis, L. S. (1996) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.506414 · Cited by 458 · see citing works
Linear object classes and image synthesis from a single example image
Vetter, T., & Poggio, T. (1997) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.598230 · Cited by 410 · see citing works
Recognition of Facial Expression from Optical Flow
Mase, K. (1991) · IEICE Transactions on Information and Systems · Cited by 404 · see citing works
Face recognition under varying pose
Beymer (1994) · doi:10.1109/cvpr.1994.323893 · Cited by 376 · see citing works
Connectionist models of face processing: A survey
Valentin, D., Abdi, H., O’Toole, A. J., & Cottrell, G. W. (1994) · Pattern Recognition · doi:10.1016/0031-3203(94)90006-x · Cited by 366 · see citing works
3-D motion estimation in model-based facial image coding
Li, H., Roivainen, P., & Forchheimer, R. (1993) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.216724 · Cited by 336 · see citing works
The argument and evidence about universals in facial expressions of emotion.
Ekman, P. (1989) · Cited by 311 · see citing works
Self-citationStructural aspects of face recognition and the other-race effect
O’Toole, A. J., Deffenbacher, K. A., Valentin, D., & Abdi, H. (1994) · Memory & Cognition · doi:10.3758/bf03208892 · Cited by 308 · see citing works
Tranel, D., Damásio, A. R., & Damásio, H. (1988) · Neurology · doi:10.1212/wnl.38.5.690 · Cited by 305 · see citing works
Cohn, J. F., Zlochower, A. J., Lien, J. J.-J., & Takeo Kanade, A. (1999) · Psychophysiology · doi:10.1017/s0048577299971184 · Cited by 287 · see citing works
Human expression recognition from motion using a radial basis function network architecture
Rosenblum, M., Yacoob, Y., & Davis, L. S. (1996) · IEEE Transactions on Neural Networks · doi:10.1109/72.536309 · Cited by 284 · see citing works
Face processing: Human perception and principal components analysis
Hancock, P., Burton, A. M., & Bruce, V. (1996) · Memory & Cognition · doi:10.3758/bf03197270 · Cited by 242 · see citing works
Computer recognition of human faces
Kanade (1977) · Cited by 208 · see citing works
Yaxley, S., Rolls, E. T., & Sienkiewicz, Z. J. (1988) · Physiology & Behavior · doi:10.1016/0031-9384(88)90074-1 · Cited by 206 · see citing works
Functional anatomy of taste perception in the human brain studied with positron emission tomography
Kinomura, S., Kawashima, R., Yamada, K., Ono, S., Itoh, M., Yoshioka, S., Yamaguchi, T., Matsui, H., Miyazawa, H., Itoh, H., et al. (1994) · Brain Research · doi:10.1016/0006-8993(94)90890-7 · Cited by 174 · see citing works
Representing Face Images for Emotion Classification
Padgett, C. W., & Cottrell, G. W. (1996) · neural information processing systems · Cited by 171 · see citing works
Visual Speech Recognition with Stochastic Networks
Movellan, J. R. (1994) · Cited by 147 · see citing works
Li, C.-C., & Lien, J.-J. J. (1998) · Cited by 99 · see citing works
Bartlett, M. S., Viola, P. A., B, T. J. S. A., Golomb, B. A., Larsen, J. O., Hager, J. C., & Ekman, P. (1995) · Cited by 97 · see citing works
Self-citationFacial Expressions Used by Children in a Conflict Situation
Camras, L. A. (1977) · Child Development · doi:10.1111/j.1467-8624.1977.tb03950.x · Cited by 82 · see citing works
Automated coding of facial behavior in human-computer interactions with facs
Kaiser, S., & Wehrle, T. (1992) · Journal of Nonverbal Behavior · doi:10.1007/bf00990323 · Cited by 47 · see citing works
Human Face Perception and Identification
Bruce, V., Hancock, P., & Burton, A. M. (1998) · Springer eBooks · doi:10.1007/978-3-642-72201-1_3 · Cited by 46 · see citing works
Effects of distortion of spatial and temporal resolution of video stimuli on emotion attributions
Wallbott, H. G. (1992) · Journal of Nonverbal Behavior · doi:10.1007/bf00986876 · Cited by 34 · see citing works
Flagged first, then by citations
2 of the 47 references couldn’t be matched to a known paper.
Research works printed but not linked: 11
- Beymer, D., Shashua, A., & Poggio, T. ~ 1993 ! . Example based image analysis and synthesis ~ AI Memo No. 1431 ! . Cambridge, MA: MIT Press.
- Cottrell, G. W., & Fleming, M. K. ~ 1990 ! . Face recognition using unsupervised feature extraction. Proceedings of the International Neural Network Conference ~ pp. 322-325 ! . Paris: Kluwer.
- Cottrell, G., & Metcalfe, J. ~ 1991 ! . EMPATH: Face, gender and emotion recognition using holons. In R. P. Lippman, J. Moody, & D. Touretzky ~ Eds. ! , Advances in neural information processing systems ~ Vol. 3, pp. 564-571 ! . San Mateo, CA: Morgan & Kaufman.
- Ekman, P. ~ Ed. ! . ~ 1982a ! . Emotion in the human face ~ 2nd ed. ! . New York: Cambridge University Press.
- Ekman, P., & Friesen, W. ~ 1978 ! . Facial Action Coding System: A technique for the measurement of facial movement. Palo Alto, CA: Consulting Psychologists Press.
- Ekman, P., Huang, T., Sejnowski, T., & Hager, J. ~ 1992 ! . Final Report to NSF of the Planning Workshop on Facial Expression Understanding @ on-line # . Available: http: 00 alt.xmission.com 0 nirc1 0 Research 0 research.html
- Hager, J., & Ekman, P. ~ 1995 ! . The essential behavioral science of the face and gesture that computer scientists need to know. In M. Bichsel ~ Ed. ! , Proceedings of the International Workshop on Automatic Face and Gesture Recognition ~ pp. 7-11 ! . Available: http: 00 alt.xmission.com 0 nirc1 0 Research 0 research.html
- Himer, W., Schneider, F., Kost, G., & Heimann, H. ~ 1991 ! . Computerbased analysis of facial action: A new approach. Journal of Psychophysiology , 5 , 189-195.
- Jain, R., Kasturi, R., & Schunk, B. G. ~ 1995 ! . Machine vision. New York: McGraw-Hill.
- Tukey, J. W. ~ 1958 ! . Bias and confidence in not-quite large samples. Annals of Mathematical Statistics , 29 , 614.
- Zajonc, R. B. ~ 1984 ! . The interaction of affect and cognition. In K. Scherer & P. Ekman ~ Eds. ! , Approaches to emotion ~ pp. 239-246 ! . Hillsdale, NJ: Erlbaum.
Cited by
Sorted by citations
Comprehensive database for facial expression analysis
Takeo Kanade, A., Cohn, J. F., & Tian, Y. (2002) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/afgr.2000.840611 · Cited by 2,612 · see citing works
Emotion recognition in human-computer interaction
Cowie, R., Douglas-Cowie, E., Tsapatsoulis, N., Votsis, G., Kollias, S., Fellenz, W., & Taylor, J. G. (2001) · IEEE Signal Processing Magazine · doi:10.1109/79.911197 · Cited by 2,603 · see citing works
Toward machine emotional intelligence: analysis of affective physiological state
Picard, R. W., Vyzas, E., & Healey, J. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.954607 · Cited by 2,334 · see citing works
Automatic analysis of facial expressions: the state of the art
Pantie, M., & Rothkrantz, L. J. M. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.895976 · Cited by 1,778 · see citing works
Recognizing action units for facial expression analysis
Tian, Y.-I., Kanade, T., & Cohn, J. F. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.908962 · Cited by 1,745 · see citing works
A review of affective computing: From unimodal analysis to multimodal fusion
Poria, S., Cambria, E., Bajpai, R., & Hussain, A. (2017) · Information Fusion · doi:10.1016/j.inffus.2017.02.003 · Cited by 1,644 · see citing works
A 3D Facial Expression Database For Facial Behavior Research
Yin, L., Wei, X., Sun, Y., Wang, J., & Rosato, M. J. (2006) · 7th International Conference on Automatic Face and Gesture Recognition (FGR06) · doi:10.1109/fgr.2006.6 · Cited by 1,171 · see citing works
Donato, G., Bartlett, M. S., Hager, J. C., Ekman, P., & B, T. J. S. A. (1999) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.799905 · Cited by 999 · see citing works
Self-citationToward an affect-sensitive multimodal human-computer interaction
Pantic, M., & Rothkrantz, L. J. M. (2003) · Proceedings of the IEEE · doi:10.1109/jproc.2003.817122 · Cited by 791 · see citing works
Deciphering the Enigmatic Face
Ambadar, Z., Schooler, J. W., & Cohn, J. F. (2005) · Psychological Science · doi:10.1111/j.0956-7976.2005.01548.x · Cited by 649 · see citing works
Pantic, M., & Patras, I. (2006) · IEEE Transactions on Systems, Man and Cybernetics, Part B (Cybernetics) · doi:10.1109/tsmcb.2005.859075 · Cited by 580 · see citing works
Affective Learning — A Manifesto
Picard, R. W., Papert, S. A., Bender, W. R., Blumberg, B. M., Breazeal, C., Cavallo, D. P., Machover, T., Resnick, M., Roy, D. K., & Strohecker, C. (2004) · BT Technology Journal · doi:10.1023/b:bttj.0000047603.37042.33 · Cited by 511 · see citing works
Emotion-Induced Engagement in Internet Video Advertisements
Teixeira, T. S., Wedel, M., & Pieters, R. G. M. (2011) · Journal of Marketing Research · doi:10.1509/jmr.10.0207 · Cited by 439 · see citing works
Facial Action Recognition for Facial Expression Analysis From Static Face Images
Pantić, M., & Rothkrantz, L. J. M. (2004) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2004.825931 · Cited by 419 · see citing works
Tian, Y., Takeo Kanade, A., & Cohn, J. F. (2005) · Springer eBooks · doi:10.1007/0-387-27257-7_12 · Cited by 407 · see citing works
Observer-Based Measurement of Facial Expression with the Facial Action Coding System
Cohn, J. F., Ambadar, Z., & Ekman, P. (2007) · doi:10.1093/oso/9780195169157.003.0014 · Cited by 398 · see citing works
Self-citationAutomatic Analysis of Facial Actions: A Survey
Martínez, B., Valstar, M., Jiang, B., & Pantić, M. (2017) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2017.2731763 · Cited by 354 · see citing works
A real-time automated system for the recognition of human facial expressions
Anderson, K., & McOwan, P. W. (2006) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2005.854502 · Cited by 351 · see citing works
Toward Emotion Recognition in Car-Racing Drivers: A Biosignal Processing Approach
Katsis, C. D., Katertsidis, N. S., Ganiatsas, G., & Fotiadis, D. I. (2008) · IEEE Transactions on Systems Man and Cybernetics - Part A Systems and Humans · doi:10.1109/tsmca.2008.918624 · Cited by 332 · see citing works
Côté, J., Ericsson, K. A., & Law, M. P. (2005) · Journal of Applied Sport Psychology · doi:10.1080/10413200590907531 · Cited by 326 · see citing works
Detection, tracking, and classification of action units in facial expression
Lien, J. J.-J., Takeo Kanade, A., Cohn, J. F., & Li, C.-C. (2000) · Robotics and Autonomous Systems · doi:10.1016/s0921-8890(99)00103-7 · Cited by 310 · see citing works
Fully Automatic Recognition of the Temporal Phases of Facial Actions
Valstar, M., & Pantić, M. (2011) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2011.2163710 · Cited by 304 · see citing works
Facial expression analysis with AFFDEX and FACET: A validation study
Stöckli, S., Schulte‐Mecklenbeck, M., Borer, S., & Samson, A. C. (2017) · Behavior Research Methods · doi:10.3758/s13428-017-0996-1 · Cited by 298 · see citing works
Differences in facial expressions of four universal emotions
Kohler, C. G., Turner, T. H., Stolar, N., Bilker, W. B., Brensinger, C. M., Gur, R. E., & Gur, R. C. (2004) · Psychiatry Research · doi:10.1016/j.psychres.2004.07.003 · Cited by 276 · see citing works
Tian, Y., Takeo Kanade, A., & Cohn, J. F. (2011) · Springer eBooks · doi:10.1007/978-0-85729-932-1_19 · Cited by 268 · see citing works
Machine Analysis of Facial Expressions
Pantic, M., & Stewart, M. (2007) · Face Recognition · doi:10.5772/4847 · Cited by 248 · see citing works
A Psychometric Evaluation of the Facial Action Coding System for Assessing Spontaneous Expression
Sayette, M. A., Cohn, J. F., Wertz, J. M., Perrott, M. A., & Parrott, D. J. (2001) · Journal of Nonverbal Behavior · doi:10.1023/a:1010671109788 · Cited by 235 · see citing works
Takeo Kanade, A. (2005) · Lecture notes in computer science · doi:10.1007/11564386_1 · Cited by 226 · see citing works
Tian, Y., Takeo Kanade, A., & Cohn, J. F. (2003) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/afgr.2002.1004159 · Cited by 224 · see citing works
Recognition of facial expressions using Gabor wavelets and learning vector quantization
Bashyal, S., & Venayagamoorthy, G. K. (2008) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2007.11.010 · Cited by 222 · see citing works
Learning Multiscale Active Facial Patches for Expression Analysis
Zhong, L., Liu, Q., Yang, P., Huang, J., & Metaxas, D. (2014) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2354351 · Cited by 217 · see citing works
Skiendziel, T., Rösch, A. G., & Schultheiss, O. C. (2019) · PLoS ONE · doi:10.1371/journal.pone.0223905 · Cited by 201 · see citing works
Lisetti, C. L., & Schiano, D. J. (2000) · Pragmatics & Cognition · doi:10.1075/pc.8.1.09lis · Cited by 189 · see citing works
Siddiqi, M. H., Ali, R., Khan, A., Park, Y.-T., & Lee, S. (2015) · IEEE Transactions on Image Processing · doi:10.1109/tip.2015.2405346 · Cited by 183 · see citing works
Robust full‐motion recovery of head by dynamic templates and re‐registration techniques
Xiao, J., Moriyama, T., Takeo Kanade, A., & Cohn, J. F. (2003) · International Journal of Imaging Systems and Technology · doi:10.1002/ima.10048 · Cited by 180 · see citing works
Spontaneous vs. posed facial behavior: automatic analysis of brow actions
Valstar, M. F., Pantic, M., Ambadar, Z., & Cohn, J. F. (2006) · Proceedings of the 8th international conference on Multimodal interfaces · doi:10.1145/1180995.1181031 · Cited by 168 · see citing works
Multimodal Video Sentiment Analysis Using Deep Learning Approaches, a Survey
Abdu, S. A., Yousef, A. H., & Salem, A. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.06.003 · Cited by 147 · see citing works
Video Content Marketing: The Making of Clips
Liu, X., Shi, S. W., Teixeira, T. S., & Wedel, M. (2018) · Journal of Marketing · doi:10.1509/jm.16.0048 · Cited by 143 · see citing works
Automated Assessment of Children’s Postoperative Pain Using Computer Vision
Sikka, K., Ahmed, A. A., Díaz, D., Goodwin, M. S., Craig, K. D., Bartlett, M. S., & Huang, J. S. (2015) · PEDIATRICS · doi:10.1542/peds.2015-0029 · Cited by 137 · see citing works
Self-citationFacial expressions in school-aged children are a good indicator of ‘dislikes’, but not of ‘likes’
Zeinstra, G. G., Koelen, M. A., Colindres, D., Kok, F. J., & de Graaf, C. (2009) · Food Quality and Preference · doi:10.1016/j.foodqual.2009.07.002 · Cited by 137 · see citing works
Fully automatic upper facial action recognition
Kapoor, A., Qi, Y., & Picard, R. W. (2004) · IEEE International SOI Conference (SOI) · doi:10.1109/amfg.2003.1240843 · Cited by 135 · see citing works
Automatic Facial Expression Recognition in Standardized and Non-standardized Emotional Expressions
Küntzler, T., Höfling, T. T. A., & Alpers, G. W. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.627561 · Cited by 139 · see citing works
Boosted multi-resolution spatiotemporal descriptors for facial expression recognition
Zhao, G., & Pietikäinen, M. (2009) · Pattern Recognition Letters · doi:10.1016/j.patrec.2009.03.018 · Cited by 129 · see citing works
A comprehensive survey on automatic facial action unit analysis
Zhi, R., Liu, M., & Zhang, D. (2019) · The Visual Computer · doi:10.1007/s00371-019-01707-5 · Cited by 127 · see citing works
Facial expressivity in the course of schizophrenia and depression
Gäebel, W., & W�lwer, W. (2004) · European Archives of Psychiatry and Clinical Neuroscience · doi:10.1007/s00406-004-0510-5 · Cited by 114 · see citing works
Motion history for facial action detection in video
Valstar, M., Pantić, M., & Patras, I. (2005) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/icsmc.2004.1398371 · Cited by 111 · see citing works
Face Image Analysis by Unsupervised Learning
Bartlett, M. S. (2001) · Springer eBooks · doi:10.1007/978-1-4615-1637-8 · Cited by 109 · see citing works
Self-citationKatsis, C. D., Katertsidis, N. S., & Fotiadis, D. I. (2010) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2010.12.001 · Cited by 110 · see citing works
Evaluating affective interactions: Alternatives to asking what users feel
Picard, R. W., & Daily, S. B. (2005) · Cited by 107 · see citing works
Manfredonia, J., Bangerter, A., Manyakov, N. V., Ness, S. L., Lewin, D., Skalkin, A., Boice, M., Goodwin, M. S., Dawson, G., Hendren, R. L., et al. (2018) · Journal of Autism and Developmental Disorders · doi:10.1007/s10803-018-3757-9 · Cited by 108 · see citing works
Machine analysis of facial behaviour: naturalistic and dynamic behaviour
Pantić, M. (2009) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2009.0135 · Cited by 103 · see citing works
Service with a smiley face: Emojional contagion in digitally mediated relationships
Smith, L., & Rose, R. L. (2020) · International Journal of Research in Marketing · doi:10.1016/j.ijresmar.2019.09.004 · Cited by 94 · see citing works
Khan, M. M., Ward, R. D., & Ingleby, M. (2009) · ACM Transactions on Applied Perception · doi:10.1145/1462055.1462061 · Cited by 104 · see citing works
Examining the Use of Nonverbal Communication in Virtual Agents
Wang, I., & Ruiz, J. (2021) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2021.1898851 · Cited by 103 · see citing works
Representing emotive meaning in visual images: A social semiotic approach
Feng, D. W., & O’Halloran, K. L. (2012) · Journal of Pragmatics · doi:10.1016/j.pragma.2012.10.003 · Cited by 98 · see citing works
Generating Facial Expressions with Deep Belief Nets
Joshua, M., Geoffrey, E., Javier, R. A. V., & Adam, K. (2008) · doi:10.5772/6167 · Cited by 97 · see citing works
Capturing subtle facial motions in 3D face tracking
Wen, Z., & Huang (2003) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2003.1238646 · Cited by 97 · see citing works
Modeling Multimodal Expression of User’s Affective Subjective Experience
Bianchi‐Berthouze, N., & Lisetti, C. (2002) · User Modeling and User-Adapted Interaction · doi:10.1023/a:1013365332180 · Cited by 96 · see citing works
Recognizing lower face action units for facial expression analysis
Tian, Y., Takeo Kanade, A., & Cohn, J. F. (2002) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/afgr.2000.840678 · Cited by 94 · see citing works
Automated video-based facial expression analysis of neuropsychiatric disorders
Wang, P., Barrett, F. S., Martin, E. A., Milonova, M., Gur, R. E., Gur, R. C., Kohler, C. G., & Verma, R. (2007) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2007.09.030 · Cited by 90 · see citing works
Le, T. M., Borghi, J., Kujawa, A., Klein, D. N., & Leung, H. (2017) · NeuroImage Clinical · doi:10.1016/j.nicl.2017.01.004 · Cited by 91 · see citing works
Facial Expressions of Basic Emotions in Japanese Laypeople
Sato, W., Hyniewska, S., Minemoto, K., & Yoshikawa, S. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.00259 · Cited by 91 · see citing works
Dinges, D. F., Rider, R., Dorrian, J., McGlinchey, E. L., Rogers, N. L., Cizman, Z., Goldenstein, S. K., Vogler, C., Venkataraman, S., & Metaxas, D. (2005) · PubMed · Cited by 89 · see citing works
Automatic analysis and recognition of brow actions and head motion in spontaneous facial behavior
Cohn, J. F., Reed, L. I., Ambadar, Z., Xiao, J., & Moriyama, T. (2005) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/icsmc.2004.1398367 · Cited by 88 · see citing works
Robust full-motion recovery of head by dynamic templates and re-registration techniques
Xiao, J., Takeo Kanade, A., & Cohn, J. F. (2003) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/afgr.2002.1004149 · Cited by 87 · see citing works
Towards Personalised Gaming via Facial Expression Recognition
Blom, P. M., Bakkes, S. C. J., Tan, C. T., Whiteson, S., Roijers, D. M., Valenti, R., & Gevers, T. (2014) · Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment · doi:10.1609/aiide.v10i1.12707 · Cited by 81 · see citing works
Computational Model of Believable Conversational Agents
Pélachaud, C., & Bilvi, M. (2003) · Lecture notes in computer science · doi:10.1007/978-3-540-44972-0_17 · Cited by 81 · see citing works
Static topographic modeling for facial expression recognition and analysis
Wang, J., & Yin, L. (2007) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2006.10.011 · Cited by 80 · see citing works
Adaptation in Affective Video Games: A Literature Review
Bontchev, B. (2016) · Cybernetics and Information Technologies · doi:10.1515/cait-2016-0032 · Cited by 79 · see citing works
Nicholls, M. E. R., Ellis, B. E., Clement, J. G., & Yoshino, M. (2004) · Proceedings of the Royal Society B Biological Sciences · doi:10.1098/rspb.2003.2660 · Cited by 79 · see citing works
The spontaneous smile in dynamic motion
Tarantili, V. V., Halazonetis, D. J., & Spyropoulos, M. N. (2005) · American Journal of Orthodontics and Dentofacial Orthopedics · doi:10.1016/j.ajodo.2004.03.042 · Cited by 77 · see citing works
Automated recognition of pain in cats
Feighelstein, M., Shimshoni, I., Finka, L. R., Luna, S. P. L., Mills, D. S., & Zamansky, A. (2022) · Scientific Reports · doi:10.1038/s41598-022-13348-1 · Cited by 79 · see citing works
The neuropsychiatry and neuropsychology of lipoid proteinosis.
Thornton, H., Nel, D., Thornton, D., van Honk, J., Baker, G., & Stein, D. J. (2008) · The Journal of Neuropsychiatry and Clinical Neurosciences · doi:10.1176/jnp.2008.20.1.86 · Cited by 77 · see citing works
Raised eyebrows as gestural triggers in humour: The case of sarcasm and hyper-understanding
Université de Lille 3, F., Tabacaru, S., Lemmens, M., & Université de Lille 3, F. (2014) · European Journal of Humour Research · doi:10.7592/ejhr2014.2.2.tabacaru · Cited by 67 · see citing works
Automatic recognition of eye blinking in spontaneously occurring behavior
Moriyama, T., Takeo Kanade, A., Cohn, J. F., Xiao, J., Ambadar, Z., Gao, J., & Imamura, H. (2003) · doi:10.1109/icpr.2002.1047404 · Cited by 67 · see citing works
Eye-State Action Unit Detection by Gabor Wavelets
Tian, Y., Takeo Kanade, A., & Cohn, J. F. (2000) · Lecture notes in computer science · doi:10.1007/3-540-40063-x_19 · Cited by 66 · see citing works
An integrated telemedicine platform for the assessment of affective physiological states
Katsis, C. D., Ganiatsas, G., & Fotiadis, D. I. (2006) · Diagnostic Pathology · doi:10.1186/1746-1596-1-16 · Cited by 65 · see citing works
A weighted probabilistic approach to face recognition from multiple images and video sequences
Zhang, Y., & Martı́nez, A. M. (2005) · Image and Vision Computing · doi:10.1016/j.imavis.2005.08.004 · Cited by 61 · see citing works
Matching expression variant faces
Martı́nez, A. M. (2003) · Vision Research · doi:10.1016/s0042-6989(03)00079-8 · Cited by 61 · see citing works
Real World Real-time Automatic Recognition of Facial Expressions
Tian, Y., Brown, L. M., Hampapur, A., Pankanti, S., & Senior, A. (2003) · Cited by 60 · see citing works
Multimodal coordination of facial action, head rotation, and eye motion during spontaneous smiles
Cohn, J. F., Reed, L. I., Moriyama, T., Xiao, J., Schmidt, K. L., & Ambadar, Z. (2004) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/afgr.2004.1301520 · Cited by 60 · see citing works
Static posed and evoked facial expressions of emotions in schizophrenia
Kohler, C. G., Martin, E. A., Stolar, N., Barrett, F. S., Verma, R., Brensinger, C., Bilker, W., Gur, R. E., & Gur, R. C. (2008) · Schizophrenia Research · doi:10.1016/j.schres.2008.05.010 · Cited by 59 · see citing works
Let players evaluate serious games. Design and validation of the Serious Games Evaluation Scale
Fokides, E., Atsikpasi, P., Kaimara, P., & Deliyannis, I. (2019) · ICGA Journal · doi:10.3233/icg-190111 · Cited by 62 · see citing works
Children's and Adults' Knowledge of the Distinction Between Enjoyment and Nonenjoyment Smiles
Gosselin, P., Perron, M., Legault, M., & Campanella, P. (2002) · Journal of Nonverbal Behavior · doi:10.1023/a:1015613504532 · Cited by 59 · see citing works
Quantifying Micro-expressions with Constraint Local Model and Local Binary Pattern
Yan, W., Wang, S., Chen, Y.-H., Zhao, G., & Fu, X. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-16178-5_20 · Cited by 58 · see citing works
A psycho-ethological approach to social signal processing
Méhu, M., & Scherer, K. R. (2012) · Cognitive Processing · doi:10.1007/s10339-012-0435-2 · Cited by 56 · see citing works
Hetland, A., Vittersø, J., Wie, S. O. B., Kjelstrup, E., Mittner, M., & Dahl, T. I. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.00971 · Cited by 57 · see citing works
Computer Expression Recognition Toolbox
Bartlett, M. S., Littlewort, G. C., Wu, T., & Movellan, J. R. (2008) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/afgr.2008.4813406 · Cited by 56 · see citing works
Self-citationCohn, J. F., & Sayette, M. A. (2010) · Behavior Research Methods · doi:10.3758/brm.42.4.1079 · Cited by 55 · see citing works
Schwaninger, A., Wallraven, C., Cunningham, D. W., & Chiller-Glaus, S. D. (2006) · Progress in brain research · doi:10.1016/s0079-6123(06)56018-2 · Cited by 56 · see citing works
Holmes, M., Latham, A., Crockett, K. A., & O’Shea, J. D. (2017) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2017.2754497 · Cited by 54 · see citing works
Fei, Z., Yang, E., Yu, L., Li, X., Zhou, H., & Zhou, W. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.10.038 · Cited by 53 · see citing works
Boneh-Shitrit, T., Feighelstein, M., Bremhorst, A., Amir, S., Distelfeld, T., Dassa, Y., Yaroshetsky, S., Riemer, S., Shimshoni, I., Mills, D. S., et al. (2022) · Scientific Reports · doi:10.1038/s41598-022-27079-w · Cited by 54 · see citing works
Le décodage de l'expression faciale des émotions au cours de l'enfance.
Gosselin, P. (2005) · Canadian Psychology/Psychologie canadienne · doi:10.1037/h0087016 · Cited by 56 · see citing works
Schiano, D. J., Ehrlich, S. M., Rahardja, K., & Sheridan, K. (2000) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/332040.332430 · Cited by 52 · see citing works
Facial expression recognition using distance and texture signature relevant features
Barman, A., & Dutta, P. (2019) · Applied Soft Computing · doi:10.1016/j.asoc.2019.01.011 · Cited by 48 · see citing works
Co-Clustering to Reveal Salient Facial Features for Expression Recognition
Khan, S., Chen, L., & Yan, H. (2017) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2017.2780838 · Cited by 48 · see citing works
Detection of Genuine and Posed Facial Expressions of Emotion: Databases and Methods
Jia, S., Wang, S., Hu, C., Webster, P. J., & Li, X. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.580287 · Cited by 48 · see citing works
Facial Electromyography: Responses of Children to Odor and Taste Stimuli
Armstrong, J. E., Hutchinson, I., Laing, D. G., & Jinks, A. L. (2007) · Chemical Senses · doi:10.1093/chemse/bjm029 · Cited by 48 · see citing works
Recognizing upper face action units for facial expression analysis
Tian, Y., Kanada, T., & Cohn, J. F. (2002) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2000.855832 · Cited by 48 · see citing works
- Authors
- Bartlett, M. S., Hager, J. C., Ekman, P., & Sejnowski, T. J.
- Affiliations
- Salk Institute for Biological Studies (US)
- University of California, San Francisco (US)
- Corresponding author
- Bartlett, Marian Stewart
- Venue
- Psychophysiology
- Publisher
- Wiley
- Published
- 1999-03
- Volume
- 36
- Issue
- 2
- Pages
- 253-263
- ISSN
- 0048-5772 (ISSN-L)
- 1469-8986
- OpenAlex
- W2007431552
- Cited by
- 424
- References
- 47
- Citations by year
- 2026: 7 · 2025: 8 · 2024: 13 · 2023: 11 · 2022: 34 · 2021: 23 · 2020: 16 · 2019: 23 · 2018: 14 · 2017: 20 · 2016: 14 · 2015: 17 · 2014: 19 · 2013: 12 · 2012: 12
- FWCI
- 19.99
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychophysiology (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
- Life Sciences › Neuroscience › Cognitive Neuroscience › Face Recognition and Perception
- Keywords
- facial expressions, facial action units, facial action coding system, computer image analysis, wrinkles, hybrid classification system, emotion measurement
- MeSH terms
- Adult, Cues, Facial Expression, Female, Humans, Image Processing, Computer-Assisted, Male, Middle Aged, Databases as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
