The DOI matches its Crossref record; 51 of the 51 references it cites were found in a registry or an index.
Spontaneous vs. posed facial behavior: automatic analysis of brow actions
Abstract
Past research on automatic facial expression analysis has focused mostly on the recognition of prototypic expressions of discrete emotions rather than on the analysis of dynamic changes over time, although the importance of temporal dynamics of facial expressions for interpretation of the observed facial behavior has been acknowledged for over 20 years. For instance, it has been shown that the temporal dynamics of spontaneous and volitional smiles are fundamentally different from each other. In this work, we argue that the same holds for the temporal dynamics of brow actions and show that velocity, duration, and order of occurrence of brow actions are highly relevant parameters for distinguishing posed from spontaneous brow actions. The proposed system for discrimination between volitional and spontaneous brow actions is based on automatic detection of Action Units (AUs) and their temporal segments (onset, apex, offset) produced by movements of the eyebrows. For each temporal segment of an activated AU, we compute a number of mid-level feature parameters including the maximal intensity, duration, and order of occurrence. We use Gentle Boost to select the most important of these parameters. The selected parameters are used further to train Relevance Vector Machines to determine per temporal segment of an activated AU whether the action was displayed spontaneously or volitionally. Finally, a probabilistic decision function determines the class (spontaneous or posed) for the entire brow action. When tested on 189 samples taken from three different sets of spontaneous and volitional facial data, we attain a 90.7% correct recognition rate.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
(1981) · Cited by 11,963
Friedman, J., Hastie, T., & Tibshirani, R. (2000) · The Annals of Statistics · doi:10.1214/aos/1016218223 · Cited by 6,884
Ekman, P., & Friesen, W. V. (2019) · PsycTESTS Dataset · doi:10.1037/t27734-000 · Cited by 6,019
Comprehensive database for facial expression analysis
Takeo Kanade, A., Cohn, J. F., & Tian, Y. (2002) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/afgr.2000.840611 · Cited by 2,644
Self-citationThin slices of expressive behavior as predictors of interpersonal consequences: A meta-analysis.
Ambady, N., & Rosenthal, R. (1992) · Psychological Bulletin · doi:10.1037/0033-2909.111.2.256 · Cited by 2,003
Ekman, P., & Rosenberg, E. L. (2005) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780195179644.001.0001 · Cited by 1,971
Li, S. Z., Jain, A. K., Deng, J., & service), S. (. (2011) · Springer eBooks · doi:10.1007/978-0-85729-932-1 · Cited by 1,867
Web-Based Database for Facial Expression Analysis
Pantić, M., Valstar, M., Rademaker, R. M., & Maat, L. (2005) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2005.1521424 · Cited by 1,124
Self-citationFelt, false, and miserable smiles
Ekman, P., & Friesen, W. V. (1982) · Journal of Nonverbal Behavior · doi:10.1007/bf00987191 · Cited by 1,074
Tipping, M. E. (1999) · Cited by 869
Rinn, W. E. (1984) · Psychological Bulletin · doi:10.1037/0033-2909.95.1.52 · Cited by 804
Toward an affect-sensitive multimodal human-computer interaction
Pantic, M., & Rothkrantz, L. J. M. (2003) · Proceedings of the IEEE · doi:10.1109/jproc.2003.817122 · Cited by 791
Deciphering the Enigmatic Face
Ambadar, Z., Schooler, J. W., & Cohn, J. F. (2005) · Psychological Science · doi:10.1111/j.0956-7976.2005.01548.x · Cited by 678
Self-citationThe Psychology of Facial Expression
Mandler, G. (1997) · Cambridge University Press eBooks · doi:10.1017/cbo9780511659911 · Cited by 656
Darwin, Deception, and Facial Expression
Ekman, P. (2003) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1280.010 · Cited by 651
Facial expression of pain: An evolutionary account
Williams, A. C. D. C. (2002) · Behavioral and Brain Sciences · doi:10.1017/s0140525x02000080 · Cited by 612
Pantic, M., & Patras, I. (2006) · IEEE Transactions on Systems, Man and Cybernetics, Part B (Cybernetics) · doi:10.1109/tsmcb.2005.859075 · Cited by 580
Rinn, W. E. (1984) · Psychological Bulletin · doi:10.1037//0033-2909.95.1.52 · Cited by 538
(2002) · Interlending & Document Supply · doi:10.1108/ilds.2002.12230dab.004 · Cited by 479
Active and dynamic information fusion for facial expression understanding from image sequences
Zhang, Y., & Ji, Q. (2005) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2005.93 · Cited by 460
Measuring facial expressions by computer image analysis
Bartlett, M. S., Hager, J. C., Ekman, P., & Sejnowski, T. J. (1999) · Psychophysiology · doi:10.1017/s0048577299971664 · Cited by 424
Tian, Y., Takeo Kanade, A., & Cohn, J. F. (2005) · Springer eBooks · doi:10.1007/0-387-27257-7_12 · Cited by 407
Self-citationFacial motion in the perception of faces and of emotional expression.
Bassili, J. N. (1978) · Journal of Experimental Psychology Human Perception & Performance · doi:10.1037//0096-1523.4.3.373 · Cited by 369
Facial motion in the perception of faces and of emotional expression.
Bassili, J. N. (1978) · Journal of Experimental Psychology: Human Perception and Performance · doi:10.1037/0096-1523.4.3.373 · Cited by 369
Dynamics of facial expression extracted automatically from video
Littlewort, G. C., Bartlett, M. S., Fasel, I., Susskind, J. M., & Movellan, J. R. (2006) · Image and Vision Computing · doi:10.1016/j.imavis.2005.09.011 · Cited by 356
Fully Automatic Facial Action Unit Detection and Temporal Analysis
Valstar, M., & Pantić, M. (2006) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2006.85 · Cited by 352
Self-citationThe Symmetry of Emotional and Deliberate Facial Actions
Ekman, P., Hager, J. C., & Friesen, W. V. (1981) · Psychophysiology · doi:10.1111/j.1469-8986.1981.tb02919.x · Cited by 346
THE TIMING OF FACIAL MOTION IN POSED AND SPONTANEOUS SMILES
Cohn, J. F., & Schmidt, K. L. (2004) · International Journal of Wavelets Multiresolution and Information Processing · doi:10.1142/s021969130400041x · Cited by 308
Self-citationFully Automatic Facial Action Recognition in Spontaneous Behavior
Bartlett, M. S., Littlewort, G. C., Frank, M. G., Lainscsek, C. S. M., Fasel, I., & Movellan, J. R. (2006) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fgr.2006.55 · Cited by 295
Real-Time Inference of Complex Mental States from Facial Expressions and Head Gestures
El Kaliouby, R., & Robinson, P. (2005) · 2004 Conference on Computer Vision and Pattern Recognition Workshop · doi:10.1109/cvpr.2004.427 · Cited by 282
Fully Automatic Facial Feature Point Detection Using Gabor Feature Based Boosted Classifiers
Vukadinovic, D., & Pantić, M. (2006) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/icsmc.2005.1571392 · Cited by 264
Takeo Kanade, A. (2005) · Lecture notes in computer science · doi:10.1007/11564386_1 · Cited by 226
Schmidt, K. L., Ambadar, Z., Cohn, J. F., & Reed, L. I. (2006) · Journal of Nonverbal Behavior · doi:10.1007/s10919-005-0003-x · Cited by 223
Self-citationDifferentiating emotion elicited and deliberate emotional facial expressions
Heß, U., & Kleck, R. E. (1990) · European Journal of Social Psychology · doi:10.1002/ejsp.2420200502 · Cited by 202
Robust full‐motion recovery of head by dynamic templates and re‐registration techniques
Xiao, J., Moriyama, T., Takeo Kanade, A., & Cohn, J. F. (2003) · International Journal of Imaging Systems and Technology · doi:10.1002/ima.10048 · Cited by 184
Self-citationMachine learning methods for fully automatic recognition of facial expressions and facial actions
Bartlett, M. S., Littlewort, G. C., Lainscsek, C. S. M., Fasel, I., & Movellan, J. R. (2005) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/icsmc.2004.1398364 · Cited by 171
Human computing and machine understanding of human behavior
Pantić, M., Pentland, A., Nijholt, A., & Huang, T. S. (2006) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/1180995.1181044 · Cited by 157
Particle filtering with factorized likelihoods for tracking facial features
Patras, I., & Pantić, M. (2004) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/afgr.2004.1301515 · Cited by 100
Automatic analysis and recognition of brow actions and head motion in spontaneous facial behavior
Cohn, J. F., Reed, L. I., Ambadar, Z., Xiao, J., & Moriyama, T. (2005) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/icsmc.2004.1398367 · Cited by 91
Self-citationAn automated face reader for fatigue detection
Gu, H., & Ji, Q. (2004) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/afgr.2004.1301517 · Cited by 87
Fogel, A., Hsu, H., Shapiro, A. F., Nelson‐Goens, G. C., & Secrist, C. (2006) · Developmental Psychology · doi:10.1037/0012-1649.42.3.459 · Cited by 84
A dynamic systems approach to infant facial action
Messinger, D. S., Fogel, A., & Dickson, K. L. (1997) · Cambridge University Press eBooks · doi:10.1017/cbo9780511659911.011 · Cited by 71
7th International Conference on Automatic Face and Gesture Recognition (FGR06)
Gralewski, L., Campbell, N., & Penton‐Voak, I. S. (2006) · The Institute of Electrical and Electronics Engineers · Cited by 69
The influence of mental fatigue on facial EMG activity during a simulated workday
Veldhuizen, I. J. T., Gaillard, A. W. K., & Vries, J. D. (2003) · Biological Psychology · doi:10.1016/s0301-0511(03)00025-5 · Cited by 62
Latent semantic analysis of facial action codes for automatic facial expression recognition
Fasel, B., Monay, F., & Gática-Pérez, D. (2004) · doi:10.1145/1026711.1026742 · Cited by 55
Rosenberg, E. L., Ekman, P., & Blumenthal, J. A. (1998) · Health Psychology · doi:10.1037//0278-6133.17.4.376 · Cited by 48
The components of conversational facial expressions
Cunningham, D. W., Kleiner, M., Bülthoff, H., & Wallraven, C. (2004) · Symposium on Applied Perception in Graphics and Visualization (APGV) · doi:10.1145/1012551.1012578 · Cited by 44
Case-based reasoning for user-profiled recognition of emotions from face images
Pantić, M., & Rothkrantz, L. J. M. (2005) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2004.1394211 · Cited by 23
Using a Tensor Framework for the Analysis of Facial Dynamics
Gralewski, L., Campbell, N., & Penton‐Voak, I. S. (2006) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fgr.2006.108 · Cited by 21
Patras, I., & Pantić, M. (2006) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/icsmc.2005.1571287 · Cited by 14
Differentiating Emotion Elicited and Deliberate Emotional Facial Expressions
Hess, U. S., & Kleck, R. E. (1998) · doi:10.1093/oso/9780195104462.003.0013 · Cited by 2
Research works printed but not linked: 10
- Duchenne de Bologne G. B., 1862, Jules Renouard Libraire
- Ekman P., 2002, Salt Lake City
- 10.5555/1032641.1033044 · DOI
- Flecha-Garcia M. L., 2001, Proc. Theoretical & Applied Linguistics -- Postgraduate Conf.
- Gu H., 2004, Proc. Int'l Conf. Face & Gesture Recognition, 111--116
- 10.5555/795661.796155 · DOI
- Kendon A., 1973, Comparative Ecology and Behavior of Primates, Michael
- Lucas B., 1981, Proc. Int'l Joint Conf. on Artificial Intelligence, 674--679
- 10.5555/1949767.1949787 · DOI
- Tipping M. E., 2000, Proc. Int'l Conf. Adv. Neural Inf. Processing Systems, 332--338
Cited by
Sorted by citations
A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions
Zeng, Z., Pantic, M., Roisman, G. I., & Huang, T. S. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.52 · Cited by 2,797
AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild
Mollahosseini, A., Hasani, B., & Mahoor, M. H. (2019) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2017.2740923 · Cited by 2,338
Social signal processing: Survey of an emerging domain
Vinciarelli, A., Pantic, M., & Bourlard, H. (2009) · Image and Vision Computing · doi:10.1016/j.imavis.2008.11.007 · Cited by 842
Scherer, K. R., Schuller, B., & Elkins, A. (2017) · Cambridge University Press eBooks · doi:10.1017/9781316676202 · Cited by 768
Automatic Analysis of Facial Affect: A Survey of Registration, Representation, and Recognition
Sarıyanidi, E., Güneş, H., & Cavallaro, A. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2014.2366127 · Cited by 702
Metadata
- Authors
- Valstar, M. F., Pantic, M., Ambadar, Z., & Cohn, J. F.
- Affiliations
- Imperial College London (GB)
- Imperial College London, UK and University of Twente, The Netherlands
- University of Pittsburgh (US)
- Venue
- Proceedings of the 8th international conference on Multimodal interfaces
- Publisher
- ACM
- Published
- 2006-11-02
- Pages
- 162-170
- OpenAlex
- W2131462465
- Cited by
- 168
- References
- 49
- Citations by year
- 2026: 1 · 2024: 3 · 2022: 4 · 2021: 8 · 2020: 5 · 2019: 8 · 2018: 9 · 2017: 15 · 2016: 13 · 2015: 14 · 2014: 13 · 2013: 14 · 2012: 6
- FWCI
- 23.32
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION (publishedVersion)
- University of Twente Research Information (publishedVersion)
- Spiral (Imperial College London) (submittedVersion)
- University of Twente Research Information (submittedVersion)
- Funders
- National Institute of Mental Health
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face and Expression Recognition
- Keywords
- facial action units, eyebrow movements, facial expression analysis, relevance vector machine, mid-level features
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
