The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression
Abstract
In 2000, the Cohn-Kanade (CK) database was released for the purpose of promoting research into automatically detecting individual facial expressions. Since then, the CK database has become one of the most widely used test-beds for algorithm development and evaluation. During this period, three limitations have become apparent: 1) While AU codes are well validated, emotion labels are not, as they refer to what was requested rather than what was actually performed, 2) The lack of a common performance metric against which to evaluate new algorithms, and 3) Standard protocols for common databases have not emerged. As a consequence, the CK database has been used for both AU and emotion detection (even though labels for the latter have not been validated), comparison with benchmark algorithms is missing, and use of random subsets of the original database makes meta-analyses difficult. To address these and other concerns, we present the Extended Cohn-Kanade (CK+) database. The number of sequences is increased by 22% and the number of subjects by 27%. The target expression for each sequence is fully FACS coded and emotion labels have been revised and validated. In addition to this, non-posed sequences for several types of smiles and their associated metadata have been added. We present baseline results using Active Appearance Models (AAMs) and a linear support vector machine (SVM) classifier using a leave-one-out subject cross-validation for both AU and emotion detection for the posed data. The emotion and AU labels, along with the extended image data and tracked landmarks will be made available July 2010.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Cites
LIBSVM: A library for support vector machines
Chang, C.-C., & Lin, C.-J. (2011) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/1961189.1961199 · Cited by 40,918 · see citing works
Statistical Methods for Rates and Proportions
Fleiss, J. L., Levin, B., & Paik, M. C. (2003) · Wiley Series in Probability and Statistics · doi:10.1002/0471445428 · Cited by 18,646 · see citing works
Libsvm : A library for support vector machines
Ferrari, V. (2008) · Medical Entomology and Zoology · Cited by 10,124 · see citing works
Cootes, T. F., Edwards, G. J., & Taylor, C. J. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.927467 · Cited by 5,409 · see citing works
A Practical Guide to Support Vector Classication
Hsu, C., Chang, C.-C., & Lin, C. (2008) · Cited by 5,070 · see citing works
A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions
Zeng, Z., Pantic, M., Roisman, G. I., & Huang, T. S. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.52 · Cited by 2,797 · see citing works
Comprehensive database for facial expression analysis
Takeo Kanade, A., Cohn, J. F., & Tian, Y. (2002) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/afgr.2000.840611 · Cited by 2,612 · see citing works
Self-citationStatistical Methods for Rates and Proportions.
Plewis, I., & Fleiss, J. L. (1982) · Journal of the Royal Statistical Society Series A (General) · doi:10.2307/2982107 · Cited by 1,753 · see citing works
Active Appearance Models Revisited
Matthews, I., & Baker, S. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029666.37597.d3 · Cited by 1,538 · see citing works
Self-citationSocial signal processing: Survey of an emerging domain
Vinciarelli, A., Pantic, M., & Bourlard, H. (2009) · Image and Vision Computing · doi:10.1016/j.imavis.2008.11.007 · Cited by 842 · see citing works
The Karolinska Directed Emotional Faces: A validation study
Goeleven, E., De Raedt, R., Leyman, L., & Verschuere, B. (2008) · Cognition & Emotion · doi:10.1080/02699930701626582 · Cited by 793 · see citing works
Scherer, K. R., Schuller, B., & Elkins, A. (2017) · Cambridge University Press eBooks · doi:10.1017/9781316676202 · Cited by 766 · see citing works
A high-resolution 3D dynamic facial expression database
Yin, L., Chen, X., Sun, Y., Worm, T., & Reale, M. (2008) · 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition · doi:10.1109/afgr.2008.4813324 · Cited by 488 · see citing works
Automatic Recognition of Facial Actions in Spontaneous Expressions
Bartlett, M. S., Littlewort, G. C., Frank, M. G., Lainscsek, C., Fasel, I. R., & Movellan, J. R. (2006) · Journal of Multimedia · doi:10.4304/jmm.1.6.22-35 · Cited by 468 · see citing works
Tian, Y., Takeo Kanade, A., & Cohn, J. F. (2005) · Springer eBooks · doi:10.1007/0-387-27257-7_12 · Cited by 407 · see citing works
Self-citationFacial Action Unit Recognition by Exploiting Their Dynamic and Semantic Relationships
Tong, Y., Liao, W., & Ji, Q. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.1094 · Cited by 377 · see citing works
Dynamics of facial expression extracted automatically from video
Littlewort, G. C., Bartlett, M. S., Fasel, I., Susskind, J. M., & Movellan, J. R. (2006) · Image and Vision Computing · doi:10.1016/j.imavis.2005.09.011 · Cited by 356 · see citing works
Ambadar, Z., Cohn, J. F., & Reed, L. I. (2008) · Journal of Nonverbal Behavior · doi:10.1007/s10919-008-0059-5 · Cited by 319 · see citing works
Self-citationToward Practical Smile Detection
Whitehill, J., Littlewort, G. C., Fasel, I., Bartlett, M. S., & Movellan, J. R. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.42 · Cited by 310 · see citing works
Fully Automatic Facial Action Recognition in Spontaneous Behavior
Bartlett, M. S., Littlewort, G. C., Frank, M. G., Lainscsek, C. S. M., Fasel, I., & Movellan, J. R. (2006) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fgr.2006.55 · Cited by 295 · see citing works
(2009) · Springer eBooks · doi:10.1007/978-0-387-73003-5_2171 · Cited by 269 · see citing works
A Dynamic Texture-Based Approach to Recognition of Facial Actions and Their Temporal Models
Koelstra, S., Pantic, M., & Patras, I. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.50 · Cited by 242 · see citing works
Takeo Kanade, A. (2005) · Lecture notes in computer science · doi:10.1007/11564386_1 · Cited by 226 · see citing works
Confidence Intervals for the Area Under the ROC Curve
Cortes, C., & Mohri, M. (2004) · neural information processing systems · Cited by 158 · see citing works
Automatically detecting pain using facial actions
Lucey, P., Cohn, J. F., Lucey, S., Matthews, I., Sridharan, S., & Prkachin, K. M. (2009) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii.2009.5349321 · Cited by 126 · see citing works
Self-citationAshraf, A. B., Lucey, S., Cohn, J. F., Chen, T., Ambadar, Z., Prkachin, K., Solomon, P., & Theobald, B. J. (2007) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/1322192.1322197 · Cited by 106 · see citing works
Self-citationAutomated Facial Expression Recognition System
Ryan, A. H., Cohn, J. F., Lucey, S., Saragih, J., Lucey, P., la Torre, F. D., & Rossi, A. (2009) · IEEE International Carnahan Conference on Security Technology (ICCST) · doi:10.1109/ccst.2009.5335546 · Cited by 106 · see citing works
Self-citationApplication-independent evaluation of speaker detection
Brümmer, N., & du Preez, J. (2006) · Computer Speech and Language · doi:10.1016/j.csl.2005.08.001 · Cited by 71 · see citing works
Ashraf, A. B., Lucey, S., & Chen, T. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.75 · Cited by 33 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild
Mollahosseini, A., Hasani, B., & Mahoor, M. H. (2019) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2017.2740923 · Cited by 2,338 · see citing works
Deep Facial Expression Recognition: A Survey
Li, S., & Deng, W. (2020) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2020.2981446 · Cited by 2,043 · see citing works
OpenFace: An open source facial behavior analysis toolkit
Baltrušaitis, T., Robinson, P., & Morency, L. (2016) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2016.7477553 · Cited by 1,794 · see citing works
Deep Hashing Network for Unsupervised Domain Adaptation
Venkateswara, H., Eusebio, J., Chakraborty, S., & Panchanathan, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.572 · Cited by 1,743 · see citing works
Reliable Crowdsourcing and Deep Locality-Preserving Learning for Expression Recognition in the Wild
Li, S., Deng, W., & Du, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.277 · Cited by 1,470 · see citing works
Facial Expression Analysis under Partial Occlusion
Zhang, L., Verma, B., Tjondronegoro, D., & Chandran, V. (2018) · ACM Computing Surveys · doi:10.1145/3158369 · Cited by 1,291 · see citing works
Going deeper in facial expression recognition using deep neural networks
Mollahosseini, A., Chan, D., & Mahoor, M. H. (2016) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2016.7477450 · Cited by 1,132 · see citing works
CASME II: An Improved Spontaneous Micro-Expression Database and the Baseline Evaluation
Yan, W.-J., Li, X., Wang, S.-J., Zhao, G., Liu, Y.-J., Chen, Y.-H., & Fu, X. (2014) · PLoS ONE · doi:10.1371/journal.pone.0086041 · Cited by 993 · see citing works
Occlusion Aware Facial Expression Recognition Using CNN With Attention Mechanism
Li, Y., Zeng, J., Shan, S., & Chen, X. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2886767 · Cited by 916 · see citing works
Facing Imbalanced Data--Recommendations for the Use of Performance Metrics
Jeni, L. A., Cohn, J. F., & de la Torre, F. N. (2013) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii.2013.47 · Cited by 828 · see citing works
Self-citationLopes, A. T., de Aguiar, E., De Souza, A. F., & Oliveira-Santos, T. (2017) · Pattern Recognition · doi:10.1016/j.patcog.2016.07.026 · Cited by 799 · see citing works
DISFA: A Spontaneous Facial Action Intensity Database
Mavadati, S. M., Mahoor, M. H., Bartlett, K., Trinh, P., & Cohn, J. F. (2013) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2013.4 · Cited by 793 · see citing works
Self-citationJoint Fine-Tuning in Deep Neural Networks for Facial Expression Recognition
Jung, H., Lee, S., Yim, J., Park, S., & Kim, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.341 · Cited by 781 · see citing works
Suppressing Uncertainties for Large-Scale Facial Expression Recognition
Wang, K., Peng, X., Yang, J., Lu, S., & Qiao, Y. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00693 · Cited by 772 · see citing works
A Brief Review of Facial Emotion Recognition Based on Visual Information
Ko, B. C. (2018) · Sensors · doi:10.3390/s18020401 · Cited by 747 · see citing works
The Faces of Engagement: Automatic Recognition of Student Engagementfrom Facial Expressions
Whitehill, J., Serpell, Z. N., Lin, Y.-C., Foster, A., & Movellan, J. R. (2014) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2014.2316163 · Cited by 694 · see citing works
Automatic Analysis of Facial Affect: A Survey of Registration, Representation, and Recognition
Sarıyanidi, E., Güneş, H., & Cavallaro, A. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2014.2366127 · Cited by 689 · see citing works
Li, S., & Deng, W. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2868382 · Cited by 670 · see citing works
Facial Expression Recognition via a Boosted Deep Belief Network
Liu, P., Han, S., Meng, Z., & Tong, Y. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.233 · Cited by 667 · see citing works
Collecting Large, Richly Annotated Facial-Expression Databases from Movies
Dhall, A., Goecke, R., Lucey, S., & Gedeon, T. (2012) · IEEE Multimedia · doi:10.1109/mmul.2012.26 · Cited by 658 · see citing works
A systematic review on affective computing: emotion models, databases, and recent advances
Wang, Y., Song, W., Tao, W., Liotta, A., Yang, D., Li, X., Gao, S., Sun, Y., Ge, W., Zhang, W., et al. (2022) · Information Fusion · doi:10.1016/j.inffus.2022.03.009 · Cited by 657 · see citing works
Detecting Affect States Using VGG16, ResNet50 and SE-ResNet50 Networks
Theckedath, D., & Sedamkar, R. R. (2020) · SN Computer Science · doi:10.1007/s42979-020-0114-9 · Cited by 653 · see citing works
Benitez-Quiroz, C. F., Srinivasan, R., & Martı́nez, A. M. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.600 · Cited by 616 · see citing works
Image based Static Facial Expression Recognition with Multiple Deep Network Learning
Yu, Z., & Zhang, C. (2015) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/2818346.2830595 · Cited by 602 · see citing works
Spatial–Temporal Recurrent Neural Network for Emotion Recognition
Zhang, T., Zheng, W., Cui, Z., Zong, Y., & Li, Y. (2019) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2017.2788081 · Cited by 595 · see citing works
The computer expression recognition toolbox (CERT)
Littlewort, G. C., Whitehill, J., Wu, T., Fasel, I., Frank, M. G., Movellan, J. R., & Bartlett, M. S. (2011) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2011.5771414 · Cited by 550 · see citing works
ASCERTAIN: Emotion and Personality Recognition Using Commercial Sensors
Subramanian, R., Wache, J., Abadi, M. K., Vieriu, R. L., WINKLER, S. S., & Sebe, N. (2016) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2016.2625250 · Cited by 536 · see citing works
Corneanu, C. A., Simon, M. O., Cohn, J. F., & Guerrero, S. E. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2515606 · Cited by 527 · see citing works
Self-citationLocal Directional Number Pattern for Face Analysis: Face and Expression Recognition
Rivera, A. R., Castillo, J. R., & Chae, O. (2012) · IEEE Transactions on Image Processing · doi:10.1109/tip.2012.2235848 · Cited by 512 · see citing works
Khare, S. K., Blanes‐Vidal, V., Nadimi, E. S., & Acharya, U. R. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.102019 · Cited by 508 · see citing works
Facial expression recognition via learning deep sparse autoencoders
Zeng, N., Zhang, H., Song, B., Liu, W., Li, Y., & Dobaie, A. M. (2017) · Neurocomputing · doi:10.1016/j.neucom.2017.08.043 · Cited by 488 · see citing works
Facial Expression Recognition Based on Deep Evolutional Spatial-Temporal Networks
Zhang, K., Huang, Y., Du, Y., & Wang, L. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2689999 · Cited by 486 · see citing works
Facial Expression Recognition by De-expression Residue Learning
Yang, H., Çiftçi, U. A., & Yin, L. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00231 · Cited by 461 · see citing works
What Else Does Your Biometric Data Reveal? A Survey on Soft Biometrics
Dantcheva, A., Elia, P., & Ross, A. A. (2015) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2015.2480381 · Cited by 431 · see citing works
Facial emotion recognition using convolutional neural networks (FERC)
Mehendale, N. D. (2020) · SN Applied Sciences · doi:10.1007/s42452-020-2234-1 · Cited by 428 · see citing works
Facial Landmark Detection: A Literature Survey
Wu, Y., & Ji, Q. (2018) · International Journal of Computer Vision · doi:10.1007/s11263-018-1097-z · Cited by 421 · see citing works
FaceNet2ExpNet: Regularizing a Deep Face Recognition Net for Expression Recognition
Ding, H., Zhou, S. K., & Chellappa, R. (2017) · 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017) · doi:10.1109/fg.2017.23 · Cited by 416 · see citing works
Cross-dataset learning and person-specific normalisation for automatic Action Unit detection
Baltrušaitis, T., Mahmoud, M., & Robinson, P. (2015) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2015.7284869 · Cited by 415 · see citing works
Learning Expressionlets on Spatio-temporal Manifold for Dynamic Facial Expression Recognition
Liu, M., Shan, S., Wang, R., & Chen, X. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.226 · Cited by 392 · see citing works
Zhao, Z., Liu, Q., & Wang, S. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3093397 · Cited by 389 · see citing works
Deep Affect Prediction in-the-Wild: Aff-Wild Database and Challenge, Deep Architectures, and Beyond
Kollias, D., Tzirakis, P., Nicolaou, M. A., Papaioannou, A., Zhao, G., Schuller, B. W., Kotsia, I., & Zafeiriou, S. P. (2019) · International Journal of Computer Vision · doi:10.1007/s11263-019-01158-4 · Cited by 388 · see citing works
Facial Expression Recognition With Visual Transformers and Attentional Selective Fusion
Ma, F., Sun, B., & Shutao Li, A. (2021) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2021.3122146 · Cited by 370 · see citing works
Attention mechanism-based CNN for facial expression recognition
Li, J., Jin, K., Zhou, D., Kubota, N., & Ju, Z. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.06.014 · Cited by 359 · see citing works
A survey on facial emotion recognition techniques: A state-of-the-art literature review
Canal, F. Z., Müller, T. R., Matias, J. C., Scotton, G. G., de Sá, A. R., Pozzebon, E., & Sobieranski, A. C. (2021) · Information Sciences · doi:10.1016/j.ins.2021.10.005 · Cited by 358 · see citing works
Automatic Analysis of Facial Actions: A Survey
Martínez, B., Valstar, M., Jiang, B., & Pantić, M. (2017) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2017.2731763 · Cited by 354 · see citing works
Facial Expression Recognition with Inconsistently Annotated Datasets
Zeng, J., Shan, S., & Chen, X. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01261-8_14 · Cited by 354 · see citing works
Facial emotion recognition using deep learning: review and insights
Mellouk, W., & Handouzi, W. (2020) · Procedia Computer Science · doi:10.1016/j.procs.2020.07.101 · Cited by 354 · see citing works
Research Review for Broad Learning System: Algorithms, Theory, and Applications
Gong, X., Zhang, T., Chen, C. L. P., & Liu, Z. (2021) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2021.3061094 · Cited by 354 · see citing works
A Micro-GA Embedded PSO Feature Selection Approach to Intelligent Facial Emotion Recognition
Mistry, K., Zhang, L., Neoh, S. C., Lim, C. P., & Fielding, B. (2016) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2016.2549639 · Cited by 350 · see citing works
Deep Region and Multi-label Learning for Facial Action Unit Detection
Zhao, K., Chu, W.-S., & Zhang, H. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.369 · Cited by 346 · see citing works
Facial Expression Recognition in the Wild via Deep Attentive Center Loss
Farzaneh, A. H., & Qi, X. (2021) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv48630.2021.00245 · Cited by 346 · see citing works
MEAD: A Large-Scale Audio-Visual Dataset for Emotional Talking-Face Generation
Wang, K., Wu, Q., Song, L., Yang, Z., Wu, W., Qian, C., He, R., Qiao, Y., & Loy, C. C. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58589-1_42 · Cited by 344 · see citing works
Island Loss for Learning Discriminative Features in Facial Expression Recognition
Cai, J., Meng, Z., Khan, A. S., Li, Z., O’Reilly, J. D., & Tong, Y. (2018) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2018.00051 · Cited by 337 · see citing works
Identity-Aware Convolutional Neural Network for Facial Expression Recognition
Meng, Z., Liu, P., Cai, J., Han, S., & Tong, Y. (2017) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2017.140 · Cited by 335 · see citing works
Deeply Learning Deformable Facial Action Parts Model for Dynamic Expression Analysis
Liu, M., Li, S., Shan, S., Wang, R., & Chen, X. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-16817-3_10 · Cited by 332 · see citing works
Driver Drowsiness Detection via a Hierarchical Temporal Deep Belief Network
Weng, C.-H., Lai, Y.-H., & Lai, S. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-54526-4_9 · Cited by 327 · see citing works
Context-Aware Emotion Recognition Networks
Lee, J., Kim, S., Kim, S., Park, J., & Sohn, K. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.01024 · Cited by 323 · see citing works
Peak-Piloted Deep Network for Facial Expression Recognition
Zhao, X., Liang, X., Liu, L., Li, T., Han, Y., Vasconcelos, N. M., & Yan, S. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46475-6_27 · Cited by 317 · see citing works
Video-based Facial Micro-Expression Analysis: A Survey of Datasets, Features and Algorithms
Ben, X., Ren, Y., Zhang, J., Wang, S., Kpalma, K., Meng, W., & Liu, Y. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3067464 · Cited by 316 · see citing works
Selective Transfer Machine for Personalized Facial Action Unit Detection
Chu, W.-S., la Torre, F. D., & Cohn, J. F. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.451 · Cited by 309 · see citing works
Self-citationKim, J., Kim, B., Roy, P. P., & Jeong, D.-M. (2019) · IEEE Access · doi:10.1109/access.2019.2907327 · Cited by 309 · see citing works
From Facial Expression Recognition to Interpersonal Relation Prediction
Zhang, Z., Luo, P., Loy, C. C., & Tang, X. (2017) · International Journal of Computer Vision · doi:10.1007/s11263-017-1055-1 · Cited by 305 · see citing works
Emotion Recognition for Human-Robot Interaction: Recent Advances and Future Perspectives
Spezialetti, M., Placidi, G., & Rossi, S. (2020) · Frontiers in Robotics and AI · doi:10.3389/frobt.2020.532279 · Cited by 302 · see citing works
Deep Pain: Exploiting Long Short-Term Memory Networks for Facial Expression Classification
Rodriguez, P., Cucurull, G., Gonzalez, J., Gonfaus, J. M., Nasrollahi, K., Moeslund, T. B., & Roca, F. X. (2022) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2017.2662199 · Cited by 297 · see citing works
Aff-Wild: Valence and Arousal ‘In-the-Wild’ Challenge
Zafeiriou, S. P., Kollias, D., Nicolaou, M. A., Papaioannou, A., Zhao, G., & Kotsia, I. (2017) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.248 · Cited by 297 · see citing works
Ghimire, D., & Lee, J. (2013) · Sensors · doi:10.3390/s130607714 · Cited by 295 · see citing works
CAS(ME): A Database for Spontaneous Macro-Expression and Micro-Expression Spotting and Recognition
Qu, F., Wang, S., Yan, W., Li, H., Wu, S., & Fu, X. (2017) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2017.2654440 · Cited by 294 · see citing works
Do Deep Neural Networks Learn Facial Action Units When Doing Expression Recognition?
Khorrami, P., Paine, T. L., & Huang, T. S. (2015) · 2015 IEEE International Conference on Computer Vision Workshop (ICCVW) · doi:10.1109/iccvw.2015.12 · Cited by 285 · see citing works
Hassouneh, A., Mutawa, A. M., & Murugappan, M. (2020) · Informatics in Medicine Unlocked · doi:10.1016/j.imu.2020.100372 · Cited by 285 · see citing works
Facial Expression Recognition Using Enhanced Deep 3D Convolutional Neural Networks
Hasani, B., & Mahoor, M. H. (2017) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.282 · Cited by 283 · see citing works
The future of service: The power of emotion in human-robot interaction
Chuah, S. H., & Yu, J. (2021) · Journal of Retailing and Consumer Services · doi:10.1016/j.jretconser.2021.102551 · Cited by 279 · see citing works
Mano, L. Y., Faiçal, B. S., Nakamura, L. H. V., Gomes, P. H., Libralon, G. L., Meneguette, R. I., Filho, G. P. R., Giancristofaro, G. T., Pessin, G., Krishnamachari, B., et al. (2016) · Computer Communications · doi:10.1016/j.comcom.2016.03.010 · Cited by 248 · see citing works
Multimodal Emotion Recognition with Deep Learning: Advancements, challenges, and future directions
Vijayaraghavan, G., Mala, T., Dhanasekaran, P., & Uma, E. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.102218 · Cited by 246 · see citing works
Deep learning for human affect recognition: Insights and new developments
Rouast, P. V., Adam, M. T. P., & Chiong, R. (2021) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2018.2890471 · Cited by 243 · see citing works
EMOCA: Emotion Driven Monocular Face Capture and Animation
Daněček, R., Black, M. J., & Bolkart, T. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01967 · Cited by 243 · see citing works
Label Distribution Learning on Auxiliary Label Space Graphs for Facial Expression Recognition
Chen, S., Wang, J., Chen, Y., Shi, Z., Geng, X., & Rui, Y. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01400 · Cited by 239 · see citing works
Towards an intelligent framework for multimodal affective data analysis
Poria, S., Cambria, E., Hussain, A., & Huang, G.-B. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.10.005 · Cited by 236 · see citing works
Video-based emotion recognition in the wild using deep transfer learning and score fusion
Kaya, H., Gürpınar, F., & Salah, A. A. (2017) · Image and Vision Computing · doi:10.1016/j.imavis.2017.01.012 · Cited by 235 · see citing works
McDuff, D., el Kaliouby, R., Senechal, T., Amr, M., Cohn, J. F., & Picard, R. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2013.130 · Cited by 234 · see citing works
Self-citationDeep-Emotion: Facial Expression Recognition Using Attentional Convolutional Network
Minaee, S., Minaei, M., & Abdolrashidi, A. (2021) · Sensors · doi:10.3390/s21093046 · Cited by 232 · see citing works
AU-aware Deep Networks for facial expression recognition
Liu, M., Li, S., Shan, S., & Chen, X. (2013) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2013.6553734 · Cited by 229 · see citing works
Facial Emotion Detection Using Deep Learning
Jaiswal, A., Raju, A. K., & Deb, S. (2020) · International Conference for Emerging Technology (INCET) · doi:10.1109/incet49848.2020.9154121 · Cited by 226 · see citing works
Facial Expression Recognition in Video with Multiple Feature Fusion
Chen, J., Chen, Z., Chi, Z., & Fu, H. C. (2016) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2016.2593719 · Cited by 225 · see citing works
Feature Decomposition and Reconstruction Learning for Effective Facial Expression Recognition
Ruan, D., Yan, Y., Lai, S., Chai, Z., Shen, C., & Wang, H. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00757 · Cited by 224 · see citing works
AU-inspired Deep Networks for Facial Expression Feature Learning
Liu, M., Li, S., Shan, S., & Chen, X. (2015) · Neurocomputing · doi:10.1016/j.neucom.2015.02.011 · Cited by 222 · see citing works
Adaptively Learning Facial Expression Representation via C-F Labels and Distillation
Li, H., Wang, N., Ding, X., Yang, X., & Gao, X. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3049955 · Cited by 221 · see citing works
Max-Margin Early Event Detectors
Hoai, M., & la Torre, F. D. (2013) · International Journal of Computer Vision · doi:10.1007/s11263-013-0683-3 · Cited by 220 · see citing works
BAUM-1: A Spontaneous Audio-Visual Face Database of Affective and Mental States
Zhalehpour, S., Onder, O., Akhtar, Z., & Erdem, Ç. E. (2016) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2016.2553038 · Cited by 220 · see citing works
Xie, S., & Hu, H. (2018) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2018.2844085 · Cited by 216 · see citing works
Frame Attention Networks for Facial Expression Recognition in Videos
Meng, D., Peng, X., Wang, K., & Qiao, Y. (2019) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2019.8803603 · Cited by 216 · see citing works
Privacy–Enhancing Face Biometrics: A Comprehensive Survey
Meden, B., Rot, P., Terhörst, P., Damer, N., Kuijper, A., Scheirer, W. J., Ross, A. A., Peer, P., & Štruc, V. (2021) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2021.3096024 · Cited by 214 · see citing works
DAiSEE: Towards User Engagement Recognition in the WildPreprint
Gupta, A., D'Cunha, A., Awasthi, K., & Balasubramanian, V. N. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1609.01885 · Cited by 212 · see citing works
A Review on Automatic Facial Expression Recognition Systems Assisted by Multimodal Sensor Data
Samadiani, N., Huang, G., Cai, B., Luo, W., Chi, C., Xiang, Y., & He, J. (2019) · Sensors · doi:10.3390/s19081863 · Cited by 211 · see citing works
Expression, Affect, Action Unit Recognition: Aff-Wild2, Multi-Task Learning and ArcFace
Kollias, D., & Zafeiriou, S. (2019) · arXiv · doi:10.48550/arxiv.1910.04855 · Cited by 210 · see citing works
Vocal and Facial Biomarkers of Depression based on Motor Incoordination and Timing
Williamson, J. R., Quatieri, T. F., Helfer, B. S., Ciccarelli, G., & Mehta, D. D. (2014) · Proceedings of the 4th International Workshop on Audio/Visual Emotion Challenge · doi:10.1145/2661806.2661809 · Cited by 209 · see citing works
Adaptive Deep Metric Learning for Identity-Aware Facial Expression Recognition
Liu, X., Kumar, B. V. K. V., You, J., & Jia, P. (2017) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.79 · Cited by 209 · see citing works
Xie, S., Hu, H., & Wu, Y. (2019) · Pattern Recognition · doi:10.1016/j.patcog.2019.03.019 · Cited by 207 · see citing works
Cross-Subject Multimodal Emotion Recognition Based on Hybrid Fusion
Çimtay, Y., Ekmekcioǧlu, E., & Özhan, Ş. Ç. (2020) · IEEE Access · doi:10.1109/access.2020.3023871 · Cited by 207 · see citing works
Yang, B., Cao, J., Ni, R., & Zhang, Y. (2017) · IEEE Access · doi:10.1109/access.2017.2784096 · Cited by 206 · see citing works
Former-DFER: Dynamic Facial Expression Recognition Transformer
Zhao, Z., & Liu, Q. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3475292 · Cited by 205 · see citing works
- Authors
- Lucey, P., Cohn, J. F., Kanade, T., Saragih, J., Ambadar, Z., & Matthews, I.
- Affiliations
- University of Pittsburgh (US)
- Carnegie Mellon University (US)
- Walt Disney (United States) (US)
- Venue
- 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops
- Publisher
- IEEE
- Published
- 2010-06
- Pages
- 94-101
- ISSN
- 2160-7508 (ISSN-L)
- OpenAlex
- W2103943262
- Cited by
- 4,303
- References
- 29
- Citations by year
- 2026: 139 · 2025: 286 · 2024: 350 · 2023: 462 · 2022: 440 · 2021: 464 · 2020: 411 · 2019: 397 · 2018: 332 · 2017: 268 · 2016: 214 · 2015: 199 · 2014: 139 · 2013: 110 · 2012: 53
- FWCI
- 121.79
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (publishedVersion)
- Funders
- National Institutes of Health
- National Institute of Mental Health
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face and Expression Recognition
- Keywords
- facial expression recognition, facial action units, emotion recognition, facial landmark tracking, active appearance models, support vector classifier, benchmark dataset
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
