The DOI matches its Crossref record; 179 of the 179 references it cites were found in a registry or an index.
Automatic Analysis of Facial Affect: A Survey of Registration, Representation, and Recognition
Open accessAbstract
Automatic affect analysis has attracted great interest in various contexts including the recognition of action units and basic or non-basic emotions. In spite of major efforts, there are several open questions on what the important cues to interpret facial expressions are and how to encode them. In this paper, we review the progress across a range of affect recognition applications to shed light on these fundamental questions. We analyse the state-of-the-art solutions by decomposing their pipelines into fundamental components, namely face registration, representation, dimensionality reduction and recognition. We discuss the role of these components and highlight the models and new trends that are followed in their design. Moreover, we provide a comprehensive analysis of facial representations by uncovering their advantages and limitations; we elaborate on the type of information they encode and discuss how they deal with the key challenges of illumination variations, registration errors, head-pose variations, occlusions, and identity bias. This survey allows us to identify open issues and to define future directions for designing real-world affect recognition systems.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Distinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589
Histograms of Oriented Gradients for Human Detection
Dalal, N., & Triggs, B. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.177 · Cited by 33,584
Book Review: Pattern Recognition and Machine Learning
Nasrabadi, N. M. (2007) · Journal of Electronic Imaging · doi:10.1117/1.2819119 · Cited by 22,066
Rapid object detection using a boosted cascade of simple features
Viola, P., & Jones, M. (2005) · Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001 · doi:10.1109/cvpr.2001.990517 · Cited by 18,300
Turk, M., & Pentland, A. (1991) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.1991.3.1.71 · Cited by 13,733
Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex
Hubel, D. H., & Wiesel, T. N. (1962) · The Journal of Physiology · doi:10.1113/jphysiol.1962.sp006837 · Cited by 13,680
The expression of the emotions in man and animals.
Darwin, C. (1872) · J. Murray eBooks · doi:10.1037/10001-000 · Cited by 11,657
An Introduction To Compressive Sampling
Candes, E. J., & Wakin, M. B. (2008) · IEEE Signal Processing Magazine · doi:10.1109/msp.2007.914731 · Cited by 10,017
Pattern Recognition and Machine Learning
(2006) · doi:10.1007/978-0-387-45528-0 · Cited by 9,824
Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories
Lazebnik, S., Schmid, C., & Ponce, J. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.68 · Cited by 7,916
Friedman, J., Hastie, T., & Tibshirani, R. (2000) · The Annals of Statistics · doi:10.1214/aos/1016218223 · Cited by 6,884
Ekman, P., & Friesen, W. V. (2019) · PsycTESTS Dataset · doi:10.1037/t27734-000 · Cited by 6,019
Face Description with Local Binary Patterns: Application to Face Recognition
Ahonen, T., Hadid, A., & Pietikainen, M. (2006) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2006.244 · Cited by 5,584
Cootes, T. F., Edwards, G. J., & Taylor, C. J. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.927467 · Cited by 5,409
Lucey, P., Cohn, J. F., Kanade, T., Saragih, J., Ambadar, Z., & Matthews, I. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops · doi:10.1109/cvprw.2010.5543262 · Cited by 4,303
Lucas-Kanade 20 Years On: A Unifying Framework
Baker, S., & Matthews, I. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000011205.11775.fd · Cited by 3,155
Face recognition by elastic bunch graph matching
Wiskott, L., Fellous, J.-M., Kuiger, N., & von der Malsburg, C. (1997) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.598235 · Cited by 2,883
Dynamic Texture Recognition Using Local Binary Patterns with an Application to Facial Expressions
Zhao, G., & Pietikainen, M. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.1110 · Cited by 2,846
A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions
Zeng, Z., Pantic, M., Roisman, G. I., & Huang, T. S. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.52 · Cited by 2,797
Vlfeat: an open and portable library of computer vision algorithms
Vedaldi, A., & Fulkerson, B. (2010) · Proceedings of the 18th ACM international conference on Multimedia · doi:10.1145/1873951.1874249 · Cited by 2,664
Non-negative matrix factorization with sparseness constraints
Hoyer, P. O. (2004) · arXiv (Cornell University) · doi:10.48550/arxiv.cs/0408058 · Cited by 2,591
Comprehensive database for facial expression analysis
Kanade, T., Cohn, J. F., & Tian, Y. (2002) · Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580) · doi:10.1109/afgr.2000.840611 · Cited by 2,591
Facial expression recognition based on Local Binary Patterns: A comprehensive study
Shan, C., Gong, S., & McOwan, P. W. (2009) · Image and Vision Computing · doi:10.1016/j.imavis.2008.08.005 · Cited by 2,300
Image analysis via the general theory of moments*
Teague, M. R. (1980) · Journal of the Optical Society of America · doi:10.1364/josa.70.000920 · Cited by 2,276
Convolutional networks and applications in vision
LeCun, Y., Kavukcuoglu, K., & Farabet, C. (2010) · Proceedings of 2010 IEEE International Symposium on Circuits and Systems · doi:10.1109/iscas.2010.5537907 · Cited by 2,213
Coding facial expressions with Gabor wavelets
Lyons, M., Akamatsu, S., Kamachi, M., & Gyoba, J. (2002) · Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition · doi:10.1109/afgr.1998.670949 · Cited by 2,210
Face detection, pose estimation, and landmark localization in the wild
Zhu, X., & Ramanan, D. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248014 · Cited by 2,120
Thin slices of expressive behavior as predictors of interpersonal consequences: A meta-analysis.
Ambady, N., & Rosenthal, R. (1992) · Psychological Bulletin · doi:10.1037//0033-2909.111.2.256 · Cited by 1,922
Supervised Descent Method and Its Applications to Face Alignment
Xiong, X., & De la Torre, F. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.75 · Cited by 1,896
Review of shape representation and description techniques
Zhang, D., & Lu, G. (2004) · Pattern Recognition · doi:10.1016/j.patcog.2003.07.008 · Cited by 1,867
Distortion invariant object recognition in the dynamic link architecture
Lades, M., Vorbruggen, J. C., Buhmann, J., Lange, J., von der Malsburg, C., Wurtz, R. P., & Konen, W. (1993) · IEEE Transactions on Computers · doi:10.1109/12.210173 · Cited by 1,805
Recognizing action units for facial expression analysis
Tian, Y.-I., Kanade, T., & Cohn, J. F. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.908962 · Cited by 1,745
Robust Object Recognition with Cortex-Like Mechanisms
Serre, T., Wolf, L., Bileschi, S., Riesenhuber, M., & Poggio, T. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.56 · Cited by 1,667
A Theoretical Analysis of Feature Pooling in Visual Recognition
Boureau, Y.-L., Ponce, J., & LeCun, Y. (2010) · Cited by 1,661
Gross, R., Matthews, I., Cohn, J., Kanade, T., & Baker, S. (2010) · Image and Vision Computing · doi:10.1016/j.imavis.2009.08.002 · Cited by 1,621
SIFT Flow: Dense Correspondence across Scenes and Its Applications
Liu, C., Yuen, J., & Torralba, A. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.147 · Cited by 1,582
Recognizing emotion from facial expressions: psychological and neurological mechanisms
Adolphs, R. (2002) · Behavioral and Cognitive Neuroscience Reviews · doi:10.1177/1534582302001001003 · Cited by 1,432
Dictionaries for Sparse Representation Modeling
Rubinstein, R., Bruckstein, A. M., & Elad, M. (2010) · Proceedings of the IEEE · doi:10.1109/jproc.2010.2040551 · Cited by 1,306
A 3D Facial Expression Database For Facial Behavior Research
Yin, L., Wei, X., Sun, Y., Wang, J., & Rosato, M. J. (2006) · 7th International Conference on Automatic Face and Gesture Recognition (FGR06) · doi:10.1109/fgr.2006.6 · Cited by 1,171 · cited 1×
“from naturalistic environments with spontaneous affective behaviour [118] as existing datasets [122], [138], [176] contain exaggeratedly posed affective behaviour.”
BackgroundWeb-Based Database for Facial Expression Analysis
Pantic, M., Valstar, M., Rademaker, R., & Maat, L. (2005) · 2005 IEEE International Conference on Multimedia and Expo · doi:10.1109/icme.2005.1521424 · Cited by 1,039
Blur Insensitive Texture Classification Using Local Phase Quantization
Ojansivu, V., & Heikkilä, J. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-69905-7_27 · Cited by 1,027
Multilinear Analysis of Image Ensembles: TensorFaces
Vasilescu, M. A. O., & Terzopoulos, D. (2002) · Lecture Notes in Computer Science · doi:10.1007/3-540-47969-4_30 · Cited by 836
Bosphorus Database for 3D Face Analysis
Savran, A., Alyüz, N., Dibeklioğlu, H., Çeliktutan, O., Gökberk, B., Sankur, B., & Akarun, L. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-89991-4_6 · Cited by 812
DISFA: A Spontaneous Facial Action Intensity Database
Mavadati, S. M., Mahoor, M. H., Bartlett, K., Trinh, P., & Cohn, J. F. (2013) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2013.4 · Cited by 793
Facing Imbalanced Data--Recommendations for the Use of Performance Metrics
Jeni, L. A., Cohn, J. F., & De La Torre, F. (2013) · 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction · doi:10.1109/acii.2013.47 · Cited by 790
Oxford Handbook of Face Perception
(2011) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780199559053.001.0001 · Cited by 760
The Expression Of The Emotions In Man And Animals
Darwin, C. (1998) · doi:10.1093/oso/9780195112719.001.0001 · Cited by 732
McKeown, G., Valstar, M., Cowie, R., Pantic, M., & Schroder, M. (2012) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2011.20 · Cited by 665
Ambadar, Z., Schooler, J. W., & Cohn, J. F. (2005) · Psychological Science · doi:10.1111/j.0956-7976.2005.01548.x · Cited by 656
The Expression Of The Emotions In Man And Animals
Darwin, C. (1998) · The Expression Of The Emotions In Man And Animals · doi:10.1093/oso/9780195112719.002.0002 · Cited by 650
Convolutional Learning of Spatio-temporal Features
Taylor, G. W., Fergus, R., LeCun, Y., & Bregler, C. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15567-3_11 · Cited by 629
AVEC 2013: the continuous audio/visual emotion and depression recognition challenge
Valstar, M., Schuller, B., Smith, K., Eyben, F., Jiang, B., Bilakhia, S., Schnieder, S., Cowie, R., & Pantic, M. (2013) · Proceedings of the 3rd ACM international workshop on Audio/visual emotion challenge · doi:10.1145/2512530.2512533 · Cited by 581
Pantic, M., & Patras, I. (2006) · IEEE Transactions on Systems, Man and Cybernetics, Part B (Cybernetics) · doi:10.1109/tsmcb.2005.859075 · Cited by 580
(2017) · The SAGE Encyclopedia of Communication Research Methods · doi:10.4135/9781483381411.n178 · Cited by 559
Why is Real-World Visual Object Recognition Hard?
Pinto, N., Cox, D. D., & DiCarlo, J. J. (2008) · PLoS Computational Biology · doi:10.1371/journal.pcbi.0040027 · Cited by 537
A Social Interactional Theory of Emotions
WESTERMEYER, J. (1979) · American Journal of Psychiatry · doi:10.1176/ajp.136.6.870 · Cited by 534
A principal component analysis of facial expressions
Calder, A. J., Burton, A. M., Miller, P., Young, A. W., & Akamatsu, S. (2001) · Vision Research · doi:10.1016/s0042-6989(01)00002-5 · Cited by 471
Automatic Recognition of Facial Actions in Spontaneous Expressions
Bartlett, M. S., Littlewort, G. C., Frank, M. G., Lainscsek, C., Fasel, I. R., & Movellan, J. R. (2006) · Journal of Multimedia · doi:10.4304/jmm.1.6.22-35 · Cited by 468
Emotional speech: Towards a new generation of databases
Douglas-Cowie, E., Campbell, N., Cowie, R., & Roach, P. (2003) · Speech Communication · doi:10.1016/s0167-6393(02)00070-5 · Cited by 461
A face feature space in the macaque temporal lobe
Freiwald, W. A., Tsao, D. Y., & Livingstone, M. S. (2009) · Nature Neuroscience · doi:10.1038/nn.2363 · Cited by 440
Introducing the Geneva Multimodal expression corpus for experimental research on emotion perception.
Bänziger, T., Mortillaro, M., & Scherer, K. R. (2012) · Emotion · doi:10.1037/a0025827 · Cited by 439
The first facial expression recognition and analysis challenge
Valstar, M. F., Jiang, B., Mehu, M., Pantic, M., & Scherer, K. (2011) · Face and Gesture 2011 · doi:10.1109/fg.2011.5771374 · Cited by 385
Facial Action Unit Recognition by Exploiting Their Dynamic and Semantic Relationships
Tong, Y., Liao, W., & Ji, Q. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.1094 · Cited by 378
Learning active facial patches for expression analysis
Zhong, L., Liu, Q., Yang, P., Liu, B., Huang, J., & Metaxas, D. N. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247974 · Cited by 369
Face alignment through subspace constrained mean-shifts
Saragih, J. M., Lucey, S., & Cohn, J. F. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459377 · Cited by 368
Gunes, H., & Schuller, B. (2013) · Image and Vision Computing · doi:10.1016/j.imavis.2012.06.016 · Cited by 360
Static and dynamic 3D facial expression recognition: A comprehensive survey
Sandbach, G., Zafeiriou, S., Pantic, M., & Yin, L. (2012) · Image and Vision Computing · doi:10.1016/j.imavis.2012.06.005 · Cited by 357 · cited 2×
“from naturalistic environments with spontaneous affective behaviour [118] as existing datasets [122], [138], [176] contain exaggeratedly posed affective behaviour.”
BackgroundLocal binary patterns for multi-view facial expression recognition
Moore, S., & Bowden, R. (2011) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2010.12.001 · Cited by 351
Invariance properties of Gabor filter-based features-overview and applications
Kamarainen, J.-K., Kyrki, V., & Kalviainen, H. (2006) · IEEE Transactions on Image Processing · doi:10.1109/tip.2005.864174 · Cited by 333
Selective Transfer Machine for Personalized Facial Action Unit Detection
Chu, W.-S., De la Torre, F., & Cohn, J. F. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.451 · Cited by 307
Facial point detection using boosted regression and graph models
Valstar, M., Martinez, B., Binefa, X., & Pantic, M. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5539996 · Cited by 307
Fully Automatic Recognition of the Temporal Phases of Facial Actions
Valstar, M. F., & Pantic, M. (2012) · IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) · doi:10.1109/tsmcb.2011.2163710 · Cited by 304
THE TIMING OF FACIAL MOTION IN POSED AND SPONTANEOUS SMILES
COHN, J. F., & SCHMIDT, K. L. (2004) · International Journal of Wavelets, Multiresolution and Information Processing · doi:10.1142/s021969130400041x · Cited by 298
Ask the locals: Multi-way local pooling for image recognition
Boureau, Y.-L., Le Roux, N., Bach, F., Ponce, J., & LeCun, Y. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126555 · Cited by 277
Generic vs. person specific active appearance models
Gross, R., Matthews, I., & Baker, S. (2005) · Image and Vision Computing · doi:10.1016/j.imavis.2005.07.009 · Cited by 272
AVEC 2011–The First International Audio/Visual Emotion Challenge
Schuller, B., Valstar, M., Eyben, F., McKeown, G., Cowie, R., & Pantic, M. (2011) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-24571-8_53 · Cited by 271
Automatically Detecting Pain in Video Through Facial Action Units
Lucey, P., Cohn, J. F., Matthews, I., Lucey, S., Sridharan, S., Howlett, J., & Prkachin, K. M. (2011) · IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) · doi:10.1109/tsmcb.2010.2082525 · Cited by 267 · cited 1×
“AUs and temporal segments are well-analysed in psychology and their recognition enables the analysis of sophisticated emotional states such as pain [82] and helps distinguishing between genuine and posed behaviour [151].”
BackgroundFully Automatic Facial Feature Point Detection Using Gabor Feature Based Boosted Classifiers
Vukadinovic, D., & Pantic, M. (2006) · 2005 IEEE International Conference on Systems, Man and Cybernetics · doi:10.1109/icsmc.2005.1571392 · Cited by 263
Zhi, R., Flierl, M., Ruan, Q., & Kleijn, W. B. (2011) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2010.2044788 · Cited by 260
Facial Expression Recognition Using Facial Movement Features
Zhang, L., & Tjondronegoro, D. (2011) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2011.13 · Cited by 253
Towards robust and effective shape modeling: Sparse shape composition
Zhang, S., Zhan, Y., Dewan, M., Huang, J., Metaxas, D. N., & Zhou, X. S. (2012) · Medical Image Analysis · doi:10.1016/j.media.2011.08.004 · Cited by 249 · cited 1×
“[184] or manifold learning [6], [39] is an interesting future direction.”
Continuous Pain Intensity Estimation from Facial Expressions
Kaltwang, S., Rudovic, O., & Pantic, M. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33191-6_36 · Cited by 245 · cited 3×
“The above-listed linear transformations are often used with representations that model the whole face [59], [99], [156].”
BackgroundA Dynamic Texture-Based Approach to Recognition of Facial Actions and Their Temporal Models
Koelstra, S., Pantic, M., & Patras, I. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.50 · Cited by 242
On deep generative models with applications to recognition
Ranzato, M., Susskind, J., Mnih, V., & Hinton, G. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995710 · Cited by 237
Action unit detection using sparse appearance descriptors in space-time video volumes
Jiang, B., Valstar, M. F., & Pantic, M. (2011) · Face and Gesture 2011 · doi:10.1109/fg.2011.5771416 · Cited by 233
AVEC 2012: the continuous audio/visual emotion challenge
Schuller, B., Valster, M., Eyben, F., Cowie, R., & Pantic, M. (2012) · Proceedings of the 14th ACM international conference on Multimodal interaction · doi:10.1145/2388676.2388776 · Cited by 226
Emotion recognition using PHOG and LPQ features
Dhall, A., Asthana, A., Goecke, R., & Gedeon, T. (2011) · Face and Gesture 2011 · doi:10.1109/fg.2011.5771366 · Cited by 219
Automatic coding of facial expressions displayed during posed and genuine pain
Littlewort, G. C., Bartlett, M. S., & Lee, K. (2009) · Image and Vision Computing · doi:10.1016/j.imavis.2008.12.010 · Cited by 216
Disentangling Factors of Variation for Facial Expression Recognition
Rifai, S., Bengio, Y., Courville, A., Vincent, P., & Mirza, M. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33783-3_58 · Cited by 215 · cited 3×
“max-pooling [112]) improve robustness to registration errors further (see 1118 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL.”
BackgroundHow to distinguish posed from spontaneous smiles using geometric features
Valstar, M. F., Gunes, H., & Pantic, M. (2007) · Proceedings of the 9th international conference on Multimodal interfaces · doi:10.1145/1322192.1322202 · Cited by 203
Self-citationTied Factor Analysis for Face Recognition across Large Pose Differences
Prince, S. J. D., Warrell, J., Elder, J. H., & Felisberti, F. M. (2008) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.48 · Cited by 193
Automatic detection of pain intensity
Hammal, Z., & Cohn, J. F. (2012) · Proceedings of the 14th ACM international conference on Multimodal interaction · doi:10.1145/2388676.2388688 · Cited by 185
A comparative study of face landmarking techniques
Çeliktutan, O., Ulukaya, S., & Sankur, B. (2013) · EURASIP Journal on Image and Video Processing · doi:10.1186/1687-5281-2013-13 · Cited by 175
Spontaneous vs. posed facial behavior: automatic analysis of brow actions
Valstar, M. F., Pantic, M., Ambadar, Z., & Cohn, J. F. (2006) · Proceedings of the 8th international conference on Multimodal interfaces · doi:10.1145/1180995.1181031 · Cited by 168
Lucey, P., Cohn, J. F., Prkachin, K. M., Solomon, P. E., Chew, S., & Matthews, I. (2012) · Image and Vision Computing · doi:10.1016/j.imavis.2011.12.003 · Cited by 166
Learning Invariant Feature Hierarchies
LeCun, Y. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33863-2_51 · Cited by 164
Investigating Spontaneous Facial Action Recognition through AAM Representations of the Face
Lucey, S., Bilal, A., & F., J. (2007) · Face Recognition · doi:10.5772/4841 · Cited by 161
Valstar, M., & Pantić, M. (2007) · Springer eBooks · doi:10.1007/978-3-540-75773-3_13 · Cited by 160
Robust FFT-Based Scale-Invariant Image Registration with Image Gradients
Tzimiropoulos, G., Argyriou, V., Zafeiriou, S. P., & Stathaki, T. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.107 · Cited by 159
The Belfast Induced Natural Emotion Database
Sneddon, I., McRorie, M., McKeown, G. J., & Hanratty, J. (2011) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2011.26 · Cited by 154
A Dynamic Appearance Descriptor Approach to Facial Actions Temporal Modeling
Jiang, B., Valstar, M., Martínez, B., & Pantic, M. (2014) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2013.2249063 · Cited by 154
Annotation and processing of continuous emotional attributes: Challenges and opportunities
Metallinou, A., & Narayanan, S. S. (2013) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2013.6553804 · Cited by 153
A Unified Probabilistic Framework for Spontaneous Facial Action Modeling and Understanding
Tong, Y., Chen, J., & Ji, Q. (2008) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.293 · Cited by 149
Using Biologically Inspired Features for Face Processing
Meyers, E. M., & Wolf, L. (2007) · International Journal of Computer Vision · doi:10.1007/s11263-007-0058-8 · Cited by 143
Facial Action Recognition Combining Heterogeneous Features via Multikernel Learning
Sénéchal, T., Rapp, V., Salam, H., Séguier, R., Bailly, K., & Prévost, L. (2012) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2012.2193567 · Cited by 138
3D facial expression recognition using SIFT descriptors of automatically detected keypoints
Berretti, S., Ben Amor, B., Daoudi, M., & del Bimbo, A. (2011) · The Visual Computer · doi:10.1007/s00371-011-0611-x · Cited by 138
Boosting Coded Dynamic Features for Facial Action Units and Facial Expression Recognition
Yang, P., Liu, Q., & Metaxas, D. (2007) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2007.383059 · Cited by 137
Liao, S., Fan, W., Chung, A. C. S., & Yeung, D. (2006) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2006.312418 · Cited by 136
Multiple Classifier Systems for the Classification of Audio-Visual Emotional States
Glodek, M., Tschechne, S., Layher, G., Schels, M., Brosch, T., Scherer, S. J., Kächele, M., Schmidt, M., Heiko Neumann, B., Palm, G., et al. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-24571-8_47 · Cited by 134
Recognition of 3D facial expression dynamics
Sandbach, G., Zafeiriou, S. P., Pantić, M., & Rueckert, D. (2012) · Image and Vision Computing · doi:10.1016/j.imavis.2012.01.006 · Cited by 134
Robust continuous prediction of human emotions using multiscale dynamic cues
Nicolle, J., Rapp, V., Bailly, K., Prévost, L., & Chétouani, M. (2012) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/2388676.2388783 · Cited by 133
Ekman, P. (2003) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1280.002 · Cited by 132
Boosted multi-resolution spatiotemporal descriptors for facial expression recognition
Zhao, G., & Pietikäinen, M. (2009) · Pattern Recognition Letters · doi:10.1016/j.patrec.2009.03.018 · Cited by 132
Exploiting Unrelated Tasks in Multi-Task Learning
Romera‐Paredes, B., Argyriou, A. A., Bianchi‐Berthouze, N., & Pontil, M. (2012) · UCL Discovery (University College London) · Cited by 132
A framework for automated measurement of the intensity of non-posed Facial Action Units
Mahoor, M. H., Cadavid, S., Messinger, D. S., & Cohn, J. F. (2009) · 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2009.5204259 · Cited by 130
Regression-based intensity estimation of facial action units
Savran, A., Sankur, B., & Taha Bilge, M. (2012) · Image and Vision Computing · doi:10.1016/j.imavis.2011.11.008 · Cited by 129
Chang, Y., Hu, C., & Turk, M. (2004) · IEEE International SOI Conference (SOI) · doi:10.1109/amfg.2003.1240820 · Cited by 127
Tang, H., & Huang, T. S. (2008) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/afgr.2008.4813304 · Cited by 127
Facial expression recognition using Gabor motion energy filters
Wu, T., Bartlett, M. S., & Movellan, J. R. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops · doi:10.1109/cvprw.2010.5543267 · Cited by 126 · cited 7×
“[79], [141], [167]).”
BackgroundInfluentialCapturing Global Semantic Relationships for Facial Action Unit Recognition
Wang, Z., Li, Y., Wang, S., & Ji, Q. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.410 · Cited by 126
Dimensional affect recognition using Continuous Conditional Random Fields
Baltrušaitis, T., Banda, N., & Robinson, P. (2013) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2013.6553785 · Cited by 124
Machine assessment of neonatal facial expressions of acute pain
Brahnam, S., Chuang, C.-F., Sexton, R. S., & Shih, F. Y. (2006) · Decision Support Systems · doi:10.1016/j.dss.2006.02.004 · Cited by 118
Comparative evaluation of 3D vs. 2D modality for automatic detection of facial action units
Savran, A., Sankur, B., & Taha Bilge, M. (2012) · Pattern Recognition · doi:10.1016/j.patcog.2011.07.022 · Cited by 111
Output-associative RVM regression for dimensional and continuous emotion prediction
Nicolaou, M. A., Güneş, H., & Pantić, M. (2012) · Image and Vision Computing · doi:10.1016/j.imavis.2011.12.005 · Cited by 107
Rudovic, O. O., Pavlović, V., & Pantić, M. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6247983 · Cited by 104
Shape Priors using Manifold Learning Techniques
Étyngier, P., Ségonne, F., & Keriven, R. (2007) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2007.4409040 · Cited by 101
Exploring Bag of Words Architectures in the Facial Expression Domain
Sikka, K., Wu, T., Susskind, J., & Bartlett, M. S. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33868-7_25 · Cited by 101
Local features based facial expression recognition with face registration errors
Gritti, T., Shan, C., Jeanne, V., & Braspenning, R. (2008) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/afgr.2008.4813379 · Cited by 101
Boosting encoded dynamic features for facial expression recognition
Yang, P., Liu, Q., & Metaxas, D. (2008) · Pattern Recognition Letters · doi:10.1016/j.patrec.2008.03.014 · Cited by 100
Particle filtering with factorized likelihoods for tracking facial features
Patras, I., & Pantić, M. (2004) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/afgr.2004.1301515 · Cited by 100
Perception and Emotion: How We Recognize Facial Expressions
Adolphs, R. (2006) · Current Directions in Psychological Science · doi:10.1111/j.1467-8721.2006.00440.x · Cited by 99
Facial expression recognition using emotion avatar image
Yang, S., & Bhanu, B. (2011) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2011.5771364 · Cited by 98
Oxford Handbook of Face Perception
Calder, A., Rhodes, G., Johnson, M. H., & Haxby, J. (2011) · UWA Profiles and Research Repository (UWA) · Cited by 93
Continuous AU intensity estimation using localized, sparse facial feature space
Jeni, L. A., Girard, J. M., Cohn, J. F., & De La Torre, F. (2013) · 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG) · doi:10.1109/fg.2013.6553808 · Cited by 93
Appearance Manifold of Facial Expression
Shan, C., Gong, S., & McOwan, P. W. (2005) · Lecture notes in computer science · doi:10.1007/11573425_22 · Cited by 88
Action unit detection with segment-based SVMs
Simon, T., Nguyen, M. H., la Torre, F. D., & Cohn, J. F. (2010) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2010.5539998 · Cited by 86
Sieving Regression Forest Votes for Facial Feature Detection in the Wild
Yang, H., & Patras, I. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.243 · Cited by 79
AVEC 2012: the continuous audio/visual emotion challenge - an introduction
Schuller, B. W., Valstar, M., Cowie, R., & Pantić, M. (2012) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/2388676.2388758 · Cited by 75
Facial action unit recognition with sparse representation
Mahoor, M. H., Zhou, M., Veon, K. L., Mavadati, S. M., & Cohn, J. F. (2011) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2011.5771420 · Cited by 74
Sparse Representation for accurate classification of corrupted and occluded facial expressions
Cotter, S. F. (2010) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2010.5494903 · Cited by 73
Facial Action Unit Event Detection by Cascade of Tasks
Ding, X., Chu, W.-S., de la Torre, F. N., Cohn, J. F., & Wang, Q. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.298 · Cited by 73
Sparse representations for facial expressions recognition via l1 optimization
Zafeiriou, S. P., & Petrou, M. (2010) · 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2010.5543148 · Cited by 73
In the Pursuit of Effective Affective Computing: The Relationship Between Features and Registration
Chew, S. W., Lucey, P., Lucey, S., Saragih, J., Cohn, J. F., Matthews, I., & Sridharan, S. (2012) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2012.2194485 · Cited by 70
Effect of illumination on automatic expression recognition: A novel 3D relightable facial database
Stratou, G., Ghosh, A., Debevec, P. E., & Morency, L. (2011) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2011.5771467 · Cited by 69
Salient feature and reliable classifier selection for facial expression classification
Kyperountas, M., Tefas, A., & Pitas, I. (2009) · Pattern Recognition · doi:10.1016/j.patcog.2009.07.007 · Cited by 65
Mixed emotions: Holistic and analytic perception of facial expressions
Butler, S., Tanaka, J. W., Kaiser, M. D., & Grand, R. L. (2010) · Journal of Vision · doi:10.1167/9.8.496 · Cited by 65
Data-Free Prior Model for Facial Action Unit Recognition
Li, Y., Chen, J., Zhao, Y.-P., & Ji, Q. (2013) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2013.5 · Cited by 63
Towards a dynamic expression recognition system under facial occlusion
Huang, X., Zhao, G., Zheng, W., & Pietikäinen, M. (2012) · Pattern Recognition Letters · doi:10.1016/j.patrec.2012.07.015 · Cited by 63
Zhu, Y., la Torre, F. D., Cohn, J. F., & Zhang, Y. (2011) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2011.10 · Cited by 63
Savran, A., Cao, H., Shah, M., Nenkova, A., & Verma, R. (2012) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/2388676.2388781 · Cited by 62
Markov Random Field Structures for Facial Action Unit Intensity Estimation
Sandbach, G., Zafeiriou, S., & Pantic, M. (2013) · 2013 IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2013.101 · Cited by 60
Parts and Wholes in Expression Recognition
White, M. J. (2000) · Cognition & Emotion · doi:10.1080/026999300378987 · Cited by 60
Long, F., Wu, T., Movellan, J. R., Bartlett, M. S., & Littlewort, G. C. (2012) · Neurocomputing · doi:10.1016/j.neucom.2012.04.017 · Cited by 54
A Comprehensive Empirical Study on Linear Subspace Methods for Facial Expression Analysis
Shan, C., Gong, S., & McOwan, P. W. (2006) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2006.13 · Cited by 52
Feature level analysis for 3D facial expression recognition
Sha, T., Song, M., Bu, J., Chen, C., & Tao, D. (2011) · Neurocomputing · doi:10.1016/j.neucom.2011.01.008 · Cited by 51
Subclass discriminant Nonnegative Matrix Factorization for facial image analysis
Nikitidis, S., Tefas, A., Nikolaidis, N., & Pitas, I. (2012) · Pattern Recognition · doi:10.1016/j.patcog.2012.04.030 · Cited by 39
Active Image Labeling and Its Application to Facial Action Labeling
Zhang, L., Tong, Y., & Ji, Q. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-88688-4_52 · Cited by 38
HOG-Based Decision Tree for Facial Expression Classification
Orrite, C., Gañán, A., & Rogez, G. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-02172-5_24 · Cited by 38
Local Zernike Moment Representation for Facial Affect Recognition
Sarıyanidi, E., Güneş, H., Gökmen, M., & Cavallaro, A. (2013) · British Machine Vision Conference (BMVC) · doi:10.5244/c.27.108 · Cited by 38
Self-citationLocal Zernike Moments: A new representation for face recognition
Sarıyanidi, E., Dağlı, V., Tek, S. C., Tunç, B., & Gökmen, M. (2012) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2012.6466927 · Cited by 34
Intensity Rank Estimation of Facial Expressions Based on a Single Image
Chang, K.-Y., Chen, C., & Hung, Y. (2013) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc.2013.538 · Cited by 32
A Psychologically-Inspired Match-Score Fusion Model for Video-Based Facial Expression Recognition
Cruz, A. C., Bhanu, B., & Yang, S. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-24571-8_45 · Cited by 31
Continuous Emotion Recognition Using Gabor Energy Filters
Dahmane, M., & Meunier, J. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-24571-8_46 · Cited by 29
Fast-FACS: A Computer-Assisted System to Increase Speed and Reliability of Manual FACS Coding
la Torre, F. D., Simon, T., Ambadar, Z., & Cohn, J. F. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-24600-5_9 · Cited by 29
Action unit recognition transfer across datasets
Wu, T., Butko, N. J., Ruvolo, P., Whitehill, J., Bartlett, M. S., & Movellan, J. R. (2011) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2011.5771369 · Cited by 29
Emotion recognition based on a novel triangular facial feature extraction method
Huang, K., Huang, S., & Kuo, Y.-H. (2010) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2010.5596374 · Cited by 29
Neth, D., & Martínez, A. M. (2010) · Vision Research · doi:10.1016/j.visres.2010.05.024 · Cited by 28
The motion in emotion — A CERT based approach to the FERA emotion challenge
Littlewort, G. C., Whitehill, J., Wu, T., Butko, N. J., Ruvolo, P., Movellan, J. R., & Bartlett, M. S. (2011) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2011.5771370 · Cited by 28
Crowdsourcing micro-level multimedia annotations: the challenges of evaluation and interface
Park, S., Mohammadi, G., Artstein, R., & Morency, L. (2012) · doi:10.1145/2390803.2390816 · Cited by 26
Dynamic Probabilistic CCA for Analysis of Affective Behaviour
Nicolaou, M. A., Pavlović, V., & Pantić, M. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33786-4_8 · Cited by 24
When Holistic Processing Is Not Enough: Local Features Save the Day
Zhang, L., & Cottrell, G. W. (2004) · eScholarship (California Digital Library) · Cited by 20
Learning spatial weighting for facial expression analysis via constrained quadratic programming
Liao, C., Chuang, H.-J., Duan, C.-H., & Lai, S. (2013) · Pattern Recognition · doi:10.1016/j.patcog.2013.03.017 · Cited by 16
Dynamic soft encoded patterns for facial event analysis
Yang, P., Liu, Q., & Metaxas, D. (2011) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2010.11.015 · Cited by 16
Martínez, A. M. (2011) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2011.5981690 · Cited by 13
Is All Face Processing Holistic? The View From UCSD
(2005) · Psychology Press eBooks · doi:10.4324/9781410612328-14 · Cited by 13
Class dependent factor analysis and its application to face recognition
Tunç, B., Dağlı, V., & Gökmen, M. (2012) · Pattern Recognition · doi:10.1016/j.patcog.2012.05.017 · Cited by 11
Intensity measurement of spontaneous facial actions: Evaluation of different image representations
Zaker, N., Mahoor, M. H., Mattson, W. I., Messinger, D. S., & Cohn, J. F. (2012) · 2012 IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL) · doi:10.1109/devlrn.2012.6400846 · Cited by 10
Similarity Features for Facial Event Analysis
Yang, P., Liu, Q., & Metaxas, D. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-88682-2_52 · Cited by 8
Registration of 3D facial surfaces using covariance matrix pyramids
Kaiser, M. S., Kwolek, B., Staub, C., & Rigoll, G. (2010) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2010.5509629 · Cited by 5
Cited by
Openface 2.0: facial behavior analysis toolkit
Baltrusaitis, T., Zadeh, A., Lim, Y. C., & Morency, L.-P. (2018) · 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018) · doi:10.1109/fg.2018.00019 · Cited by 1,861
Deep Facial Expression Recognition: A Survey
Li, S., & Deng, W. (2022) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2020.2981446 · Cited by 1,753
OpenFace: An open source facial behavior analysis toolkit
Baltrusaitis, T., Robinson, P., & Morency, L.-P. (2016) · 2016 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv.2016.7477553 · Cited by 1,747
Facial Expression Analysis under Partial Occlusion: A Survey
Zhang, L., Verma, B., Tjondronegoro, D., & Chandran, V. (2019) · ACM Computing Surveys · doi:10.1145/3158369 · Cited by 1,258
Deep Learning for Emotion Recognition on Small Datasets using Transfer Learning
Ng, H.-W., Nguyen, V. D., Vonikakis, V., & Winkler, S. (2015) · Proceedings of the 2015 ACM on International Conference on Multimodal Interaction · doi:10.1145/2818346.2830593 · Cited by 657
Metadata
- Authors
- Sariyanidi, E., Gunes, H., & Cavallaro, A.
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2015-06-01
- Volume
- 37
- Issue
- 6
- Pages
- 1113-1133
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- OpenAlex
- W1965947362
- Cited by
- 685
- References
- 179
- FWCI
- 71.85
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.