The DOI matches its Crossref record; 40 of the 40 references it cites were found in a registry or an index.
Toward Practical Smile Detection
Abstract
Machine learning approaches have produced some of the highest reported performances for facial expression recognition. However, to date, nearly all automatic facial expression recognition research has focused on optimizing performance on a few databases that were collected under controlled lighting conditions on a relatively small number of subjects. This paper explores whether current machine learning methods can be used to develop an expression recognition system that operates reliably in more realistic conditions. We explore the necessary characteristics of the training data set, image registration, feature representation, and machine learning algorithms. A new database, GENKI, is presented which contains pictures, photographed by the subjects themselves, from thousands of different people in many different real-world imaging conditions. Results suggest that human-level expression recognition accuracy in real-life illumination conditions is achievable with machine learning technology. However, the data sets currently used in the automatic expression recognition literature to evaluate progress may be overly constrained and could potentially lead research into locally optimal algorithmic solutions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Elements of Statistical Learning
Hastie, T., Friedman, J., & Tibshirani, R. (2001) · Springer Series in Statistics · doi:10.1007/978-0-387-21606-5 · Cited by 24,162
A decision-theoretic generalization of on-line learning and an application to boosting
Freund, Y., & Schapire, R. E. (1997) · Journal of Computer and System Sciences · doi:10.1006/jcss.1997.1504 · Cited by 20,466
Object recognition from local scale-invariant features
Lowe, D. G. (1999) · Proceedings of the Seventh IEEE International Conference on Computer Vision · doi:10.1109/iccv.1999.790410 · Cited by 16,472
Robust Real-Time Face Detection
Viola, P., & Jones, M. J. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000013087.49260.fb · Cited by 14,097
Friedman, J., Hastie, T., & Tibshirani, R. (2000) · The Annals of Statistics · doi:10.1214/aos/1016218223 · Cited by 6,884
A comparative study of texture measures with classification based on featured distributions
Ojala, T., Pietikäinen, M., & Harwood, D. (1996) · Pattern Recognition · doi:10.1016/0031-3203(95)00067-4 · Cited by 6,882
Ekman, P. (1976) · Medical Entomology and Zoology · Cited by 4,345
Dynamic Texture Recognition Using Local Binary Patterns with an Application to Facial Expressions
Zhao, G., & Pietikainen, M. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.1110 · Cited by 2,846
A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions
Zeng, Z., Pantic, M., Roisman, G. I., & Huang, T. S. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.52 · Cited by 2,797
Comprehensive database for facial expression analysis
Kanade, T., Cohn, J. F., & Tian, Y. (2002) · Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580) · doi:10.1109/afgr.2000.840611 · Cited by 2,591
A desicion-theoretic generalization of on-line learning and an application to boosting
Freund, Y., & Schapire, R. E. (1995) · Lecture Notes in Computer Science · doi:10.1007/3-540-59119-2_166 · Cited by 2,252
Automatic facial expression analysis: a survey
Fasel, B., & Luettin, J. (2003) · Pattern Recognition · doi:10.1016/s0031-3203(02)00052-3 · Cited by 1,810
The CMU pose, illumination, and expression database
Sim, T., Baker, S., & Bsat, M. (2003) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2003.1251154 · Cited by 1,783
Automatic analysis of facial expressions: the state of the art
Pantie, M., & Rothkrantz, L. J. M. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.895976 · Cited by 1,778
Recognizing action units for facial expression analysis
Tian, Y.-I., Kanade, T., & Cohn, J. F. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.908962 · Cited by 1,745
Web-Based Database for Facial Expression Analysis
Pantic, M., Valstar, M., Rademaker, R., & Maat, L. (2005) · 2005 IEEE International Conference on Multimedia and Expo · doi:10.1109/icme.2005.1521424 · Cited by 1,039
Donato, G., Bartlett, M. S., Hager, J. C., Ekman, P., & Sejnowski, T. J. (1999) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.799905 · Cited by 999
Self-citationFacial expression recognition from video sequences: temporal and static modeling
Cohen, I., Sebe, N., Garg, A., Chen, L. S., & Huang, T. S. (2003) · Computer Vision and Image Understanding · doi:10.1016/s1077-3142(03)00081-x · Cited by 874
Kotsia, I., & Pitas, I. (2007) · IEEE Transactions on Image Processing · doi:10.1109/tip.2006.884954 · Cited by 687
Automatic Recognition of Facial Actions in Spontaneous Expressions
Bartlett, M. S., Littlewort, G. C., Frank, M. G., Lainscsek, C., Fasel, I. R., & Movellan, J. R. (2006) · Journal of Multimedia · doi:10.4304/jmm.1.6.22-35 · Cited by 468
Self-citationActive and dynamic information fusion for facial expression understanding from image sequences
Zhang, Y., & Ji, Q. (2005) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2005.93 · Cited by 461
Facial Action Recognition for Facial Expression Analysis From Static Face Images
Pantic, M., & Rothkrantz, L. J. M. (2004) · IEEE Transactions on Systems, Man and Cybernetics, Part B (Cybernetics) · doi:10.1109/tsmcb.2004.825931 · Cited by 417
Facial Action Unit Recognition by Exploiting Their Dynamic and Semantic Relationships
Tong, Y., Liao, W., & Ji, Q. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.1094 · Cited by 378
Learning object detection from a small number of examples: the importance of good features
Levi, K., & Weiss, Y. (2004) · Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004. · doi:10.1109/cvpr.2004.1315144 · Cited by 377
Dynamics of facial expression extracted automatically from video
Littlewort, G., Bartlett, M. S., Fasel, I., Susskind, J., & Movellan, J. (2006) · Image and Vision Computing · doi:10.1016/j.imavis.2005.09.011 · Cited by 358
Self-citationTian, Y.-L., Kanade, T., & Cohn, J. F. (2005) · Handbook of Face Recognition · doi:10.1007/0-387-27257-7_12 · Cited by 347
THE TIMING OF FACIAL MOTION IN POSED AND SPONTANEOUS SMILES
COHN, J. F., & SCHMIDT, K. L. (2004) · International Journal of Wavelets, Multiresolution and Information Processing · doi:10.1142/s021969130400041x · Cited by 298
Fully Automatic Facial Action Recognition in Spontaneous Behavior
Bartlett, M. S., Littlewort, G., Frank, M., Lainscsek, C., Fasel, I., & Movellan, J. (2006) · 7th International Conference on Automatic Face and Gesture Recognition (FGR06) · doi:10.1109/fgr.2006.55 · Cited by 290
Self-citationA video database of moving faces and people
O'Toole, A. J., Ha, J., Snow, S. L., Hurst, D. R., Pappas, M. R., Ayyad, J. H., & Abdi, H. (2005) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2005.90 · Cited by 212
A generative framework for real time object detection and classification
Fasel, I., Fortenberry, B., & Movellan, J. (2005) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2004.07.014 · Cited by 208
Self-citationLocal Binary Patterns as an Image Preprocessing for Face Authentication
Heusch, G., Rodriguez, Y., & Marcel, S. (2006) · 7th International Conference on Automatic Face and Gesture Recognition (FGR06) · doi:10.1109/fgr.2006.72 · Cited by 190
Confidence Intervals for the Area Under the ROC Curve
Cortes, C., & Mohri, M. (2004) · neural information processing systems · Cited by 158
Real time facial expression recognition with AdaBoost
Wang, Y., Ai, H., Wu, B., & Huang, C. (2004) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2004.1334680 · Cited by 142
Fully automatic upper facial action recognition
Kapoor, A., Qi, Y., & Picard, R. W. (2004) · IEEE International SOI Conference (SOI) · doi:10.1109/amfg.2003.1240843 · Cited by 137
Boosting Coded Dynamic Features for Facial Action Units and Facial Expression Recognition
Yang, P., Liu, Q., & Metaxas, D. (2007) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2007.383059 · Cited by 129
Capturing subtle facial motions in 3D face tracking
Wen, Z., & Huang (2003) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2003.1238646 · Cited by 97
Recognizing lower face action units for facial expression analysis
Tian, Y., Takeo Kanade, A., & Cohn, J. F. (2002) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/afgr.2000.840678 · Cited by 94
Robust Real-Time Face Pose and Facial Expression Recovery
Zhu, Z., & Ji, Q. (2006) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2006.259 · Cited by 60
Learning From a Small Number of Training Examples by Exploiting Object Categories
Levi, K., Fink, M., & Weiss, Y. (2005) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvpr.2004.382 · Cited by 28
Towards authentic emotion recognition
Sebe, N., Sun, Y., Bakker, E. M., Lew, M. S., Cohen, I., & Huang, T. S. (2005) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/icsmc.2004.1398369 · Cited by 27
Cited by
Lucey, P., Cohn, J. F., Kanade, T., Saragih, J., Ambadar, Z., & Matthews, I. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops · doi:10.1109/cvprw.2010.5543262 · Cited by 4,303
DISFA: A Spontaneous Facial Action Intensity Database
Mavadati, S. M., Mahoor, M. H., Bartlett, K., Trinh, P., & Cohn, J. F. (2013) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2013.4 · Cited by 793
The Faces of Engagement: Automatic Recognition of Student Engagementfrom Facial Expressions
Whitehill, J., Serpell, Z., Lin, Y.-C., Foster, A., & Movellan, J. R. (2014) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2014.2316163 · Cited by 677
Self-citationFacial Expression Recognition via a Boosted Deep Belief Network
Liu, P., Han, S., Meng, Z., & Tong, Y. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.233 · Cited by 650
Painful data: The UNBC-McMaster shoulder pain expression archive database
Lucey, P., Cohn, J. F., Prkachin, K. M., Solomon, P. E., & Matthews, I. (2011) · Face and Gesture 2011 · doi:10.1109/fg.2011.5771462 · Cited by 599
Metadata
- Authors
- Whitehill, J., Littlewort, G., Fasel, I., Bartlett, M., & Movellan, J.
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2009-11
- Volume
- 31
- Issue
- 11
- Pages
- 2106-2111
- ISSN
- 0162-8828 (ISSN-L)
- OpenAlex
- W2157653492
- Cited by
- 308
- References
- 40
- FWCI
- 27.32
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.