Spontaneous vs. posed facial behavior: automatic analysis of brow actions
Abstract
Past research on automatic facial expression analysis has focused mostly on the recognition of prototypic expressions of discrete emotions rather than on the analysis of dynamic changes over time, although the importance of temporal dynamics of facial expressions for interpretation of the observed facial behavior has been acknowledged for over 20 years. For instance, it has been shown that the temporal dynamics of spontaneous and volitional smiles are fundamentally different from each other. In this work, we argue that the same holds for the temporal dynamics of brow actions and show that velocity, duration, and order of occurrence of brow actions are highly relevant parameters for distinguishing posed from spontaneous brow actions. The proposed system for discrimination between volitional and spontaneous brow actions is based on automatic detection of Action Units (AUs) and their temporal segments (onset, apex, offset) produced by movements of the eyebrows. For each temporal segment of an activated AU, we compute a number of mid-level feature parameters including the maximal intensity, duration, and order of occurrence. We use Gentle Boost to select the most important of these parameters. The selected parameters are used further to train Relevance Vector Machines to determine per temporal segment of an activated AU whether the action was displayed spontaneously or volitionally. Finally, a probabilistic decision function determines the class (spontaneous or posed) for the entire brow action. When tested on 189 samples taken from three different sets of spontaneous and volitional facial data, we attain a 90.7% correct recognition rate.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Cites
(1981) · Cited by 11,963 · see citing works
Friedman, J., Hastie, T., & Tibshirani, R. (2000) · The Annals of Statistics · doi:10.1214/aos/1016218223 · Cited by 6,884 · see citing works
Ekman, P., & Friesen, W. V. (2019) · PsycTESTS Dataset · doi:10.1037/t27734-000 · Cited by 6,019 · see citing works
Comprehensive database for facial expression analysis
Takeo Kanade, A., Cohn, J. F., & Tian, Y. (2002) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/afgr.2000.840611 · Cited by 2,612 · see citing works
Self-citationEkman, P., & Rosenberg, E. L. (2005) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780195179644.001.0001 · Cited by 1,975 · see citing works
Thin slices of expressive behavior as predictors of interpersonal consequences: A meta-analysis.
Ambady, N., & Rosenthal, R. (1992) · Psychological Bulletin · doi:10.1037/0033-2909.111.2.256 · Cited by 1,925 · see citing works
Li, S. Z., Jain, A. K., Deng, J., & service), S. (. (2011) · Springer eBooks · doi:10.1007/978-0-85729-932-1 · Cited by 1,849 · see citing works
Web-Based Database for Facial Expression Analysis
Pantić, M., Valstar, M., Rademaker, R. M., & Maat, L. (2005) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2005.1521424 · Cited by 1,099 · see citing works
Self-citationFelt, false, and miserable smiles
Ekman, P., & Friesen, W. V. (1982) · Journal of Nonverbal Behavior · doi:10.1007/bf00987191 · Cited by 1,030 · see citing works
Tipping, M. E. (1999) · Cited by 869 · see citing works
Toward an affect-sensitive multimodal human-computer interaction
Pantic, M., & Rothkrantz, L. J. M. (2003) · Proceedings of the IEEE · doi:10.1109/jproc.2003.817122 · Cited by 791 · see citing works
Rinn, W. E. (1984) · Psychological Bulletin · doi:10.1037/0033-2909.95.1.52 · Cited by 681 · see citing works
The Psychology of Facial Expression
Mandler, G. (1997) · Cambridge University Press eBooks · doi:10.1017/cbo9780511659911 · Cited by 656 · see citing works
Deciphering the Enigmatic Face
Ambadar, Z., Schooler, J. W., & Cohn, J. F. (2005) · Psychological Science · doi:10.1111/j.0956-7976.2005.01548.x · Cited by 649 · see citing works
Self-citationDarwin, Deception, and Facial Expression
Ekman, P. (2003) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1280.010 · Cited by 627 · see citing works
Facial expression of pain: An evolutionary account
Williams, A. C. D. C. (2002) · Behavioral and Brain Sciences · doi:10.1017/s0140525x02000080 · Cited by 585 · see citing works
Pantic, M., & Patras, I. (2006) · IEEE Transactions on Systems, Man and Cybernetics, Part B (Cybernetics) · doi:10.1109/tsmcb.2005.859075 · Cited by 580 · see citing works
Rinn, W. E. (1984) · Psychological Bulletin · doi:10.1037//0033-2909.95.1.52 · Cited by 538 · see citing works
(2002) · Interlending & Document Supply · doi:10.1108/ilds.2002.12230dab.004 · Cited by 479 · see citing works
Active and dynamic information fusion for facial expression understanding from image sequences
Zhang, Y., & Ji, Q. (2005) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2005.93 · Cited by 460 · see citing works
Measuring facial expressions by computer image analysis
Bartlett, M. S., Hager, J. C., Ekman, P., & Sejnowski, T. J. (1999) · Psychophysiology · doi:10.1017/s0048577299971664 · Cited by 424 · see citing works
Tian, Y., Takeo Kanade, A., & Cohn, J. F. (2005) · Springer eBooks · doi:10.1007/0-387-27257-7_12 · Cited by 407 · see citing works
Self-citationFacial motion in the perception of faces and of emotional expression.
Bassili, J. N. (1978) · Journal of Experimental Psychology Human Perception & Performance · doi:10.1037//0096-1523.4.3.373 · Cited by 369 · see citing works
Facial motion in the perception of faces and of emotional expression.
Bassili, J. N. (1978) · Journal of Experimental Psychology: Human Perception and Performance · doi:10.1037/0096-1523.4.3.373 · Cited by 369 · see citing works
Dynamics of facial expression extracted automatically from video
Littlewort, G. C., Bartlett, M. S., Fasel, I., Susskind, J. M., & Movellan, J. R. (2006) · Image and Vision Computing · doi:10.1016/j.imavis.2005.09.011 · Cited by 356 · see citing works
Fully Automatic Facial Action Unit Detection and Temporal Analysis
Valstar, M., & Pantić, M. (2006) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2006.85 · Cited by 349 · see citing works
Self-citationThe Symmetry of Emotional and Deliberate Facial Actions
Ekman, P., Hager, J. C., & Friesen, W. V. (1981) · Psychophysiology · doi:10.1111/j.1469-8986.1981.tb02919.x · Cited by 341 · see citing works
THE TIMING OF FACIAL MOTION IN POSED AND SPONTANEOUS SMILES
Cohn, J. F., & Schmidt, K. L. (2004) · International Journal of Wavelets Multiresolution and Information Processing · doi:10.1142/s021969130400041x · Cited by 301 · see citing works
Self-citationFully Automatic Facial Action Recognition in Spontaneous Behavior
Bartlett, M. S., Littlewort, G. C., Frank, M. G., Lainscsek, C. S. M., Fasel, I., & Movellan, J. R. (2006) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fgr.2006.55 · Cited by 295 · see citing works
Real-Time Inference of Complex Mental States from Facial Expressions and Head Gestures
El Kaliouby, R., & Robinson, P. (2005) · 2004 Conference on Computer Vision and Pattern Recognition Workshop · doi:10.1109/cvpr.2004.427 · Cited by 282 · see citing works
Fully Automatic Facial Feature Point Detection Using Gabor Feature Based Boosted Classifiers
Vukadinovic, D., & Pantić, M. (2006) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/icsmc.2005.1571392 · Cited by 264 · see citing works
Takeo Kanade, A. (2005) · Lecture notes in computer science · doi:10.1007/11564386_1 · Cited by 226 · see citing works
Schmidt, K. L., Ambadar, Z., Cohn, J. F., & Reed, L. I. (2006) · Journal of Nonverbal Behavior · doi:10.1007/s10919-005-0003-x · Cited by 219 · see citing works
Self-citationDifferentiating emotion elicited and deliberate emotional facial expressions
Hess, U. S., & Kleck, R. E. (1990) · European Journal of Social Psychology · doi:10.1002/ejsp.2420200502 · Cited by 205 · see citing works
Robust full‐motion recovery of head by dynamic templates and re‐registration techniques
Xiao, J., Moriyama, T., Takeo Kanade, A., & Cohn, J. F. (2003) · International Journal of Imaging Systems and Technology · doi:10.1002/ima.10048 · Cited by 180 · see citing works
Self-citationMachine learning methods for fully automatic recognition of facial expressions and facial actions
Bartlett, M. S., Littlewort, G. C., Lainscsek, C. S. M., Fasel, I., & Movellan, J. R. (2005) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/icsmc.2004.1398364 · Cited by 168 · see citing works
Human computing and machine understanding of human behavior
Pantić, M., Pentland, A., Nijholt, A., & Huang, T. S. (2006) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/1180995.1181044 · Cited by 153 · see citing works
Particle filtering with factorized likelihoods for tracking facial features
Patras, I., & Pantić, M. (2004) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/afgr.2004.1301515 · Cited by 102 · see citing works
Automatic analysis and recognition of brow actions and head motion in spontaneous facial behavior
Cohn, J. F., Reed, L. I., Ambadar, Z., Xiao, J., & Moriyama, T. (2005) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/icsmc.2004.1398367 · Cited by 88 · see citing works
Self-citationAn automated face reader for fatigue detection
Gu, H., & Ji, Q. (2004) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/afgr.2004.1301517 · Cited by 87 · see citing works
Fogel, A., Hsu, H., Shapiro, A. F., Nelson‐Goens, G. C., & Secrist, C. (2006) · Developmental Psychology · doi:10.1037/0012-1649.42.3.459 · Cited by 84 · see citing works
A dynamic systems approach to infant facial action
Messinger, D. S., Fogel, A., & Dickson, K. L. (1997) · Cambridge University Press eBooks · doi:10.1017/cbo9780511659911.011 · Cited by 71 · see citing works
7th International Conference on Automatic Face and Gesture Recognition (FGR06)
Gralewski, L., Campbell, N., & Penton‐Voak, I. S. (2006) · The Institute of Electrical and Electronics Engineers · Cited by 69 · see citing works
The influence of mental fatigue on facial EMG activity during a simulated workday
Veldhuizen, I. J. T., Gaillard, A. W. K., & Vries, J. D. (2003) · Biological Psychology · doi:10.1016/s0301-0511(03)00025-5 · Cited by 62 · see citing works
Latent semantic analysis of facial action codes for automatic facial expression recognition
Fasel, B., Monay, F., & Gática-Pérez, D. (2004) · doi:10.1145/1026711.1026742 · Cited by 52 · see citing works
Rosenberg, E. L., Ekman, P., & Blumenthal, J. A. (1998) · Health Psychology · doi:10.1037//0278-6133.17.4.376 · Cited by 45 · see citing works
The components of conversational facial expressions
Cunningham, D. W., Kleiner, M., Bülthoff, H., & Wallraven, C. (2004) · Symposium on Applied Perception in Graphics and Visualization (APGV) · doi:10.1145/1012551.1012578 · Cited by 44 · see citing works
Case-based reasoning for user-profiled recognition of emotions from face images
Pantić, M., & Rothkrantz, L. J. M. (2005) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2004.1394211 · Cited by 23 · see citing works
Using a Tensor Framework for the Analysis of Facial Dynamics
Gralewski, L., Campbell, N., & Penton‐Voak, I. S. (2006) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fgr.2006.108 · Cited by 21 · see citing works
Patras, I., & Pantić, M. (2006) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/icsmc.2005.1571287 · Cited by 14 · see citing works
Differentiating Emotion Elicited and Deliberate Emotional Facial Expressions
Hess, U. S., & Kleck, R. E. (1998) · doi:10.1093/oso/9780195104462.003.0013 · Cited by 2 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions
Zeng, Z., Pantic, M., Roisman, G. I., & Huang, T. S. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.52 · Cited by 2,797 · see citing works
AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild
Mollahosseini, A., Hasani, B., & Mahoor, M. H. (2019) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2017.2740923 · Cited by 2,338 · see citing works
Social signal processing: Survey of an emerging domain
Vinciarelli, A., Pantic, M., & Bourlard, H. (2009) · Image and Vision Computing · doi:10.1016/j.imavis.2008.11.007 · Cited by 842 · see citing works
Scherer, K. R., Schuller, B., & Elkins, A. (2017) · Cambridge University Press eBooks · doi:10.1017/9781316676202 · Cited by 766 · see citing works
Automatic Analysis of Facial Affect: A Survey of Registration, Representation, and Recognition
Sarıyanidi, E., Güneş, H., & Cavallaro, A. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2014.2366127 · Cited by 689 · see citing works
Corneanu, C. A., Simon, M. O., Cohn, J. F., & Guerrero, S. E. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2515606 · Cited by 527 · see citing works
Self-citationDetecting depression from facial actions and vocal prosody
Cohn, J. F., Kruez, T. S., Matthews, I., Yang, Y., Nguyen, M. H., Padilla, M. T., Zhou, F., & De la Torre, F. (2009) · 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops · doi:10.1109/acii.2009.5349358 · Cited by 494 · see citing works
Self-citationThe painful face – Pain expression recognition using active appearance models
Ashraf, A. B., Lucey, S., Cohn, J. F., Chen, T., Ambadar, Z., Prkachin, K. M., & Solomon, P. E. (2009) · Image and Vision Computing · doi:10.1016/j.imavis.2009.05.007 · Cited by 355 · see citing works
Self-citationHuman Computing and Machine Understanding of Human Behavior: A Survey
Pantić, M., Pentland, A., Nijholt, A., & Huang, T. S. (2007) · Springer eBooks · doi:10.1007/978-3-540-72348-6_3 · Cited by 318 · see citing works
Fully Automatic Recognition of the Temporal Phases of Facial Actions
Valstar, M., & Pantić, M. (2011) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2011.2163710 · Cited by 304 · see citing works
Self-citationMachine Analysis of Facial Expressions
Pantic, M., & Stewart, M. (2007) · Face Recognition · doi:10.5772/4847 · Cited by 248 · see citing works
Learning Multiscale Active Facial Patches for Expression Analysis
Zhong, L., Liu, Q., Yang, P., Huang, J., & Metaxas, D. (2014) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2354351 · Cited by 217 · see citing works
How to distinguish posed from spontaneous smiles using geometric features
Valstar, M., Güneş, H., & Pantić, M. (2007) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/1322192.1322202 · Cited by 201 · see citing works
Self-citationLocal Gabor Binary Patterns from Three Orthogonal Planes for Automatic Facial Expression Recognition
Almaev, T. R., & Valstar, M. (2013) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii.2013.65 · Cited by 203 · see citing works
Self-citationDupré, D., Krumhuber, E. G., Küster, D., & McKeown, G. J. (2020) · PLoS ONE · doi:10.1371/journal.pone.0231968 · Cited by 189 · see citing works
Facial Expression Recognition Using Computer Vision: A Systematic Review
Canedo, D., & Neves, A. J. R. (2019) · Applied Sciences · doi:10.3390/app9214678 · Cited by 192 · see citing works
Valstar, M., & Pantić, M. (2007) · Springer eBooks · doi:10.1007/978-3-540-75773-3_13 · Cited by 158 · see citing works
Self-citationHuman computing and machine understanding of human behavior
Pantić, M., Pentland, A., Nijholt, A., & Huang, T. S. (2006) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/1180995.1181044 · Cited by 153 · see citing works
A Dynamic Appearance Descriptor Approach to Facial Actions Temporal Modeling
Jiang, B., Valstar, M., Martínez, B., & Pantić, M. (2013) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2013.2249063 · Cited by 146 · see citing works
Self-citationA survey of affect recognition methods: audio, visual and spontaneous expressions
Zeng, Z., Pantic, M., Roisman, G. I., & Huang, T. S. (2007) · Proceedings of the 9th international conference on Multimodal interfaces · doi:10.1145/1322192.1322216 · Cited by 144 · see citing works
A Unified Probabilistic Framework for Spontaneous Facial Action Modeling and Understanding
Tong, Y., Chen, J., & Ji, Q. (2008) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.293 · Cited by 139 · see citing works
Sangineto, E., Zen, G., Ricci, E., & Sebe, N. (2014) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2647868.2654916 · Cited by 127 · see citing works
A comprehensive survey on automatic facial action unit analysis
Zhi, R., Liu, M., & Zhang, D. (2019) · The Visual Computer · doi:10.1007/s00371-019-01707-5 · Cited by 127 · see citing works
Multi-view facial expression recognition
Hu, Y., Zeng, Z., Yin, L., Wei, X., Zhou, X., & Huang, T. S. (2008) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/afgr.2008.4813445 · Cited by 126 · see citing works
Real-time eye blink detection with GPU-based SIFT tracking
Lalonde, M., Byrns, D., Gagnon, L., Teasdale, N., & Laurendeau, D. (2007) · Canadian Conference on Computer and Robot Vision · doi:10.1109/crv.2007.54 · Cited by 123 · see citing works
Comparison of Deliberate and Spontaneous Facial Movement in Smiles and Eyebrow Raises
Schmidt, K. L., Bhattacharya, S., & Denlinger, R. (2009) · Journal of Nonverbal Behavior · doi:10.1007/s10919-008-0058-6 · Cited by 117 · see citing works
Multi-modal emotion analysis from facial expressions and electroencephalogram
Huang, X., Kortelainen, J., Zhao, G., Li, X., Moilanen, A., Seppänen, T., & Pietikäinen, M. (2016) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2015.09.015 · Cited by 117 · see citing works
Pfister, T., Li, X., Zhao, G., & Pietikäinen, M. (2011) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2011.6130343 · Cited by 110 · see citing works
Ashraf, A. B., Lucey, S., Cohn, J. F., Chen, T., Ambadar, Z., Prkachin, K., Solomon, P., & Theobald, B. J. (2007) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/1322192.1322197 · Cited by 106 · see citing works
Self-citationDibeklioğlu, H., Salah, A. A., & Gevers, T. (2015) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2015.2394777 · Cited by 104 · see citing works
Foundations of human computing
Cohn, J. F. (2006) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/1180995.1181043 · Cited by 102 · see citing works
Self-citationMachine analysis of facial behaviour: naturalistic and dynamic behaviour
Pantić, M. (2009) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2009.0135 · Cited by 103 · see citing works
Wang, W. (2010) · IGI Global eBooks · doi:10.4018/978-1-61520-919-4 · Cited by 135 · see citing works
Extended DISFA Dataset: Investigating Posed and Spontaneous Facial Expressions
Mavadati, M., Sanger, P. B., & Mahoor, M. H. (2016) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2016.182 · Cited by 99 · see citing works
Attentional processing of other’s facial display of pain: An eye tracking study
Vervoort, T., Trost, Z., Prkachin, K. M., & Mueller, S. C. (2013) · Pain · doi:10.1016/j.pain.2013.02.017 · Cited by 95 · see citing works
A Review of Dynamic Datasets for Facial Expression Research
Krumhuber, E. G., Skora, L. I., Küster, D., & Fou, L. (2016) · Emotion Review · doi:10.1177/1754073916670022 · Cited by 96 · see citing works
Learning Personalized Models for Facial Expression Analysis and Gesture Recognition
Zen, G., Porzi, L., Sangineto, E., Ricci, E., & Sebe, N. (2016) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2016.2523421 · Cited by 89 · see citing works
Automated Face Analysis for Affective Computing
Calvo, R. A., D’Mello, S. K., Gratch, J. S., Kappas, A., Cohn, J. F., & la Torre, F. D. (2014) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780199942237.013.020 · Cited by 82 · see citing works
Self-citationAnalyses of a Multimodal Spontaneous Facial Expression Database
Wang, S., Liu, Z., Wang, Z., Wu, G., Shen, P., He, S., & Wang, X. (2012) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2012.32 · Cited by 75 · see citing works
Does “lie to me” lie to you? An evaluation of facial clues to high-stakes deception
Su, L., & Levine, M. D. (2016) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2016.01.009 · Cited by 76 · see citing works
SchiNet: Automatic Estimation of Symptoms of Schizophrenia from Facial Behaviour Analysis
Bishay, M., Palasek, P., Priebe, S., & Patras, I. (2019) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2019.2907628 · Cited by 76 · see citing works
Spontaneous facial expression recognition: A robust metric learning approach
Wan, S., & Aggarwal, J. K. (2013) · Pattern Recognition · doi:10.1016/j.patcog.2013.11.025 · Cited by 74 · see citing works
Social Signal Processing: The Research Agenda
Pantić, M., Cowie, R., D’Errico, F., Heylen, D., Méhu, M., Pélachaud, C., Poggi, I., Schroeder, M., & Vinciarelli, A. (2011) · Springer eBooks · doi:10.1007/978-0-85729-997-0_26 · Cited by 74 · see citing works
Foundations of Human Computing: Facial Expression and Emotion
Cohn, J. F. (2007) · Springer eBooks · doi:10.1007/978-3-540-72348-6_1 · Cited by 73 · see citing works
Self-citation“I Didn’t Know I Looked Angry”: Characterizing Observed Emotion and Reported Affect at Work
Kaur, H., McDuff, D., Williams, A. C., Teevan, J. B., & Iqbal, S. T. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517453 · Cited by 72 · see citing works
Pain monitoring: A dynamic and context-sensitive system
Hammal, Z., & Miriam Kunz, A. (2011) · Pattern Recognition · doi:10.1016/j.patcog.2011.09.014 · Cited by 66 · see citing works
Automatic Facial Expression Analysis
Valstar, M. (2014) · Springer eBooks · doi:10.1007/978-81-322-1934-7_8 · Cited by 62 · see citing works
Self-citationContinual Learning for Affective Robotics: Why, What and How?
Churamani, N., Kalkan, S., & Güneş, H. (2020) · IEEE RO-MAN · doi:10.1109/ro-man47096.2020.9223564 · Cited by 62 · see citing works
A genetic programming-based feature selection and fusion for facial expression recognition
Ghazouani, H. (2021) · Applied Soft Computing · doi:10.1016/j.asoc.2021.107173 · Cited by 62 · see citing works
Koelstra, S., & Pantić, M. (2008) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/afgr.2008.4813361 · Cited by 60 · see citing works
The role of movement kinematics in facial emotion expression production and recognition.
Sowden, S., Schuster, B. A., Keating, C. T., Fraser, D. S., & Cook, J. L. (2021) · Emotion · doi:10.1037/emo0000835 · Cited by 61 · see citing works
Audio-Visual Spontaneous Emotion Recognition
Zeng, Z., Hu, Y., Roisman, G. I., Wen, Z., Fu, Y., & Huang, T. S. (2007) · Springer eBooks · doi:10.1007/978-3-540-72348-6_4 · Cited by 51 · see citing works
Unsupervised Domain Adaptation for Personalized Facial Emotion Recognition
Zen, G., Sangineto, E., Ricci, E., & Sebe, N. (2014) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/2663204.2663247 · Cited by 51 · see citing works
Detection of Genuine and Posed Facial Expressions of Emotion: Databases and Methods
Jia, S., Wang, S., Hu, C., Webster, P. J., & Li, X. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.580287 · Cited by 53 · see citing works
Crossmodal and incremental perception of audiovisual cues to emotional speech
Barkhuysen, P. N., Krahmer, E., & Swerts, M. G. J. (2010) · Language and Speech · doi:10.1177/0023830909348993 · Cited by 45 · see citing works
Detecting Genuine and Deliberate Displays of Surprise in Static and Dynamic Faces
Zloteanu, M., Krumhuber, E. G., & Richardson, D. C. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.01184 · Cited by 45 · see citing works
Spatiotemporal analysis of RGB-D-T facial images for multimodal pain level recognition
Irani, R., Nasrollahi, K., Simon, M. O., Corneanu, C. A., Escalera, S., Bahnsen, C., Lundtoft, D. H., Moeslund, T. B., Pedersen, T. L., Klitgaard, M.-L., et al. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2015.7301341 · Cited by 42 · see citing works
Multimodal user’s affective state analysis in naturalistic interaction
Caridakis, G., Karpouzis, K. C., Wallace, M., Kessous, L., & Amir, N. (2009) · Journal on Multimodal User Interfaces · doi:10.1007/s12193-009-0030-8 · Cited by 38 · see citing works
Emotion Recognition Based on Multimodal Information
Zeng, Z., Pantić, M., & Huang, T. S. (2008) · Springer eBooks · doi:10.1007/978-1-84800-306-4_14 · Cited by 33 · see citing works
Recognition of Emotional States in Natural Human-Computer Interaction
Cowie, R., Douglas‐Cowie, E., Karpouzis, K. C., Caridakis, G., Wallace, M., & Kollias, S. (2008) · Springer eBooks · doi:10.1007/978-3-540-78345-9_6 · Cited by 33 · see citing works
Decision-Level Fusion for Audio-Visual Laughter Detection
Reuderink, B., Poel, M., Truong, K. P., Poppe, R. W., & Pantić, M. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-85853-9_13 · Cited by 32 · see citing works
DynEmo: A video database of natural facial expressions of emotions.
Tcherkassof, A., Dupré, D., Meillon, B., Mandran, N., Dubois, M., & Adam, J.-M. (2013) · The International journal of Multimedia & Its Applications · doi:10.5121/ijma.2013.5505 · Cited by 34 · see citing works
Recognizing Facial Expressions Automatically from Video
Shan, C., & Braspenning, R. (2009) · Springer eBooks · doi:10.1007/978-0-387-93808-0_18 · Cited by 30 · see citing works
Recognizing spontaneous micro-expression from eye region
Duan, X., Dai, Q., Wang, X., Wang, Y., & Hua, Z. (2016) · Neurocomputing · doi:10.1016/j.neucom.2016.03.090 · Cited by 30 · see citing works
Posed and spontaneous expression recognition through modeling their spatial patterns
Wang, S., Wu, C., He, M., Wang, J., & Ji, Q. (2015) · Machine Vision and Applications · doi:10.1007/s00138-015-0657-2 · Cited by 28 · see citing works
Smile or smirk? Automatic detection of spontaneous asymmetric smiles to understand viewer experience
Sénéchal, T., Turcot, J., & el Kaliouby, R. (2013) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2013.6553776 · Cited by 28 · see citing works
Bartlett, M. S., & Whitehill, J. (2011) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780199559053.013.0025 · Cited by 27 · see citing works
Making Tactile Textures with Predefined Affective Properties
Elkharraz, G., Thumfart, S., Akay, D., Eitzinger, C., & Henson, B. W. (2013) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2013.21 · Cited by 27 · see citing works
An Android Head for Social-Emotional Intervention for Children with Autism Spectrum Conditions
Adams, A., & Robinson, P. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-24571-8_19 · Cited by 26 · see citing works
Improving Pain Recognition Through Better Utilisation of Temporal Information.
Lucey, P., Howlett, J. M., Cohn, J., Lucey, S., Sridharan, S., & Ambadar, Z. (2008) · PubMed · Cited by 25 · see citing works
Self-citationHailpern, J. M., Karahalios, K. G., Halle, J. W., DeThorne, L. S., & Coletto, M. (2009) · ACM Transactions on Accessible Computing · doi:10.1145/1530064.1530066 · Cited by 24 · see citing works
Spontaneous versus posed smile recognition using discriminative local spatial-temporal descriptors
Wu, P., Liu, H., & Zhang, X. (2014) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2014.6853795 · Cited by 23 · see citing works
Geometry vs. Appearance for Discriminating between Posed and Spontaneous Emotions
Zhang, L., Tjondronegoro, D., & Chandran, V. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-24965-5_49 · Cited by 21 · see citing works
A Novel Dynamic Model Capturing Spatial and Temporal Patterns for Facial Expression Analysis
Wang, S., Zheng, Z., Yin, S. W., Yang, J., & Ji, Q. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2911937 · Cited by 21 · see citing works
Hammal, Z., Cohn, J. F., Heike, C. L., & Speltz, M. L. (2015) · Frontiers in ICT · doi:10.3389/fict.2015.00021 · Cited by 21 · see citing works
Self-citationDiscrimination between smiling faces: Human observers vs. automated face analysis
Líbano, M. D., Calvo, M. G., Fernández-Martín, A., & Recio, G. (2018) · Acta Psychologica · doi:10.1016/j.actpsy.2018.04.019 · Cited by 20 · see citing works
Spontaneous Pain Expression Recognition in Video Sequences
Hammal, Z., Miriam Kunz, A., Arguin, M., & Gosselin, F. (2008) · Electronic workshops in computing · doi:10.14236/ewic/vocs2008.17 · Cited by 20 · see citing works
Machine Understanding of Human Behavior
Pantić, M., Pentland, A., Nijholt, A., & Huang, T. S. (2007) · University of Twente Research Information · Cited by 19 · see citing works
Guérin-Dugué, A., Roy, R., Kristensen, E., Rivet, B., Vercueil, L., & Tcherkassof, A. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.01190 · Cited by 20 · see citing works
Implicit image tagging via facial information
Jiao, J., & Pantić, M. (2010) · International Workshop on Social Signal Processing (SSPW) · doi:10.1145/1878116.1878133 · Cited by 18 · see citing works
Cultural factors in the regression of non-verbal communication perception
Sheerman-Chase, T., Ong, E.-J., & Bowden, R. (2011) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2011.6130393 · Cited by 18 · see citing works
Posed and spontaneous facial expression differentiation using deep Boltzmann machines
Gan, Q., Wu, C., Wang, S., & Ji, Q. (2015) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii.2015.7344637 · Cited by 18 · see citing works
Hierarchical fusion of visual and physiological signals for emotion recognition
Fang, Y., Rong, R., & Huang, J. (2021) · Multidimensional Systems and Signal Processing · doi:10.1007/s11045-021-00774-z · Cited by 17 · see citing works
A Video-Based Facial Motion Tracking and Expression Recognition System
Yu, J., & Wang, Z. (2016) · Multimedia Tools and Applications · doi:10.1007/s11042-016-3883-3 · Cited by 16 · see citing works
Deception Detection and Opinion Spam
Rosso, P., & Cagnina, L. (2017) · Socio-affective computing · doi:10.1007/978-3-319-55394-8_8 · Cited by 16 · see citing works
Introduction to Emotion Recognition
Konar, A., Halder, A., & Chakraborty, A. (2015) · Wiley eBooks · doi:10.1002/9781118910566.ch1 · Cited by 16 · see citing works
Differentiating Between Posed and Spontaneous Expressions with Latent Regression Bayesian Network
Gan, Q., Nie, S., Wang, S., & Ji, Q. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.11225 · Cited by 15 · see citing works
Dynamics of facial actions for assessing smile genuineness
Kawulok, M., Nalepa, J., Kawulok, J., & Bogdan Smolka, B. (2021) · PLoS ONE · doi:10.1371/journal.pone.0244647 · Cited by 15 · see citing works
Timing is everything: A spatio-temporal approach to the analysis of facial actions
Valstar, M. (2008) · Spiral (Imperial College London) · doi:10.25560/1394 · Cited by 14 · see citing works
Self-citationCapturing global spatial patterns for distinguishing posed and spontaneous expressions
Wang, S., Wu, C., & Ji, Q. (2016) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2015.08.007 · Cited by 13 · see citing works
Automatic detection of deceit in verbal communication
Mihalcea, R. F., Pérez‐Rosas, V., & Burzo, M. G. (2013) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/2522848.2522888 · Cited by 13 · see citing works
Towards multimodal deception detection -- step 1
Mihalcea, R. F., & Burzo, M. G. (2012) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/2388676.2388714 · Cited by 13 · see citing works
Tavakolian, M., Cruces, C. G. B., & Hadid, A. (2019) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2019.8756540 · Cited by 12 · see citing works
Affective Human-Humanoid Interaction Through Cognitive Architecture
Infantino, I. (2012) · InTech eBooks · doi:10.5772/25794 · Cited by 12 · see citing works
In Search of Truth: Analysis of Smile Intensity Dynamics to Detect Deception
Kawulok, M., Nalepa, J., Nurzynska, K., & Smolka, B. (2016) · Ibero-American Conference on AI · doi:10.1007/978-3-319-47955-2_27 · Cited by 11 · see citing works
Recognition of facial expression using Haar-like feature extraction method
Satiyan, M., & Nagarajan, R. (2010) · International Conference on Intelligent and Advanced Systems · doi:10.1109/icias.2010.5716228 · Cited by 12 · see citing works
Distinguishing Posed and Spontaneous Smiles by Facial Dynamics
Mandal, B., Lee, D., & Ouarti, N. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-54407-6_37 · Cited by 11 · see citing works
RealSmileNet: A Deep End-to-End Network for Spontaneous and Posed Smile Recognition
Yang, Y., Hossain, M. Z., Gedeon, T., & Rahman, S. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-69541-5_2 · Cited by 11 · see citing works
Human Emotion Recognition from Body Language of the Head using Soft Computing Techniques
Zhao, Y. (2012) · uO Research (University of Ottawa) · doi:10.20381/ruor-6160 · Cited by 11 · see citing works
EVALUATING SUBJECTIVE ASPECTS OF HCI ON AN EXAMPLE OF A NON-TASK ORIENTED CONVERSATIONAL SYSTEM
Dybała, P., Ptaszynski, M. E., Rzepka, R., & Araki, K. (2010) · International Journal of Artificial Intelligence Tools · doi:10.1142/s0218213010000431 · Cited by 10 · see citing works
- Authors
- Valstar, M. F., Pantic, M., Ambadar, Z., & Cohn, J. F.
- Affiliations
- Imperial College London (GB)
- Imperial College London, UK and University of Twente, The Netherlands
- University of Pittsburgh (US)
- Venue
- Proceedings of the 8th international conference on Multimodal interfaces
- Publisher
- ACM
- Published
- 2006-11-02
- Pages
- 162-170
- OpenAlex
- W2131462465
- Cited by
- 168
- References
- 49
- Citations by year
- 2026: 1 · 2024: 3 · 2022: 4 · 2021: 8 · 2020: 5 · 2019: 8 · 2018: 9 · 2017: 15 · 2016: 13 · 2015: 14 · 2014: 13 · 2013: 14 · 2012: 6
- FWCI
- 23.32
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION (publishedVersion)
- University of Twente Research Information (publishedVersion)
- Spiral (Imperial College London) (submittedVersion)
- University of Twente Research Information (submittedVersion)
- Funders
- National Institute of Mental Health
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face and Expression Recognition
- Keywords
- facial action units, eyebrow movements, facial expression analysis, relevance vector machine, mid-level features
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review