The DOI matches its Crossref record; 40 of the 40 references it cites were found in a registry or an index.
Toward Practical Smile Detection
Abstract
Machine learning approaches have produced some of the highest reported performances for facial expression recognition. However, to date, nearly all automatic facial expression recognition research has focused on optimizing performance on a few databases that were collected under controlled lighting conditions on a relatively small number of subjects. This paper explores whether current machine learning methods can be used to develop an expression recognition system that operates reliably in more realistic conditions. We explore the necessary characteristics of the training data set, image registration, feature representation, and machine learning algorithms. A new database, GENKI, is presented which contains pictures, photographed by the subjects themselves, from thousands of different people in many different real-world imaging conditions. Results suggest that human-level expression recognition accuracy in real-life illumination conditions is achievable with machine learning technology. However, the data sets currently used in the automatic expression recognition literature to evaluate progress may be overly constrained and could potentially lead research into locally optimal algorithmic solutions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Elements of Statistical Learning
Hastie, T., Friedman, J., & Tibshirani, R. (2001) · Springer Series in Statistics · doi:10.1007/978-0-387-21606-5 · Cited by 24,162
A decision-theoretic generalization of on-line learning and an application to boosting
Freund, Y., & Schapire, R. E. (1997) · Journal of Computer and System Sciences · doi:10.1006/jcss.1997.1504 · Cited by 20,466
Object recognition from local scale-invariant features
Lowe, D. G. (1999) · Proceedings of the Seventh IEEE International Conference on Computer Vision · doi:10.1109/iccv.1999.790410 · Cited by 16,472
Robust Real-Time Face Detection
Viola, P., & Jones, M. J. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000013087.49260.fb · Cited by 14,097
Friedman, J., Hastie, T., & Tibshirani, R. (2000) · The Annals of Statistics · doi:10.1214/aos/1016218223 · Cited by 6,884
A comparative study of texture measures with classification based on featured distributions
Ojala, T., Pietikäinen, M., & Harwood, D. (1996) · Pattern Recognition · doi:10.1016/0031-3203(95)00067-4 · Cited by 6,882
Ekman, P. (1976) · Medical Entomology and Zoology · Cited by 4,345
Dynamic Texture Recognition Using Local Binary Patterns with an Application to Facial Expressions
Zhao, G., & Pietikainen, M. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.1110 · Cited by 2,846
A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions
Zeng, Z., Pantic, M., Roisman, G. I., & Huang, T. S. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.52 · Cited by 2,797
Comprehensive database for facial expression analysis
Kanade, T., Cohn, J. F., & Tian, Y. (2002) · Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580) · doi:10.1109/afgr.2000.840611 · Cited by 2,591
Cited by
Lucey, P., Cohn, J. F., Kanade, T., Saragih, J., Ambadar, Z., & Matthews, I. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops · doi:10.1109/cvprw.2010.5543262 · Cited by 4,303
DISFA: A Spontaneous Facial Action Intensity Database
Mavadati, S. M., Mahoor, M. H., Bartlett, K., Trinh, P., & Cohn, J. F. (2013) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2013.4 · Cited by 793
The Faces of Engagement: Automatic Recognition of Student Engagementfrom Facial Expressions
Whitehill, J., Serpell, Z., Lin, Y.-C., Foster, A., & Movellan, J. R. (2014) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2014.2316163 · Cited by 677
Self-citationFacial Expression Recognition via a Boosted Deep Belief Network
Liu, P., Han, S., Meng, Z., & Tong, Y. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.233 · Cited by 650
Painful data: The UNBC-McMaster shoulder pain expression archive database
Lucey, P., Cohn, J. F., Prkachin, K. M., Solomon, P. E., & Matthews, I. (2011) · Face and Gesture 2011 · doi:10.1109/fg.2011.5771462 · Cited by 599
The computer expression recognition toolbox (CERT)
Littlewort, G., Whitehill, J., Wu, T., Fasel, I., Frank, M., Movellan, J., & Bartlett, M. (2011) · Face and Gesture 2011 · doi:10.1109/fg.2011.5771414 · Cited by 539
Self-citationStatic facial expression analysis in tough conditions: Data, evaluation protocol and benchmark
Dhall, A., Goecke, R., Lucey, S., & Gedeon, T. (2011) · 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops) · doi:10.1109/iccvw.2011.6130508 · Cited by 541
A Deep Neural Network-Driven Feature Learning Method for Multi-view Facial Expression Recognition
Zhang, T., Zheng, W., Cui, Z., Zong, Y., Yan, J., & Yan, K. (2016) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2016.2598092 · Cited by 314
CAS(ME): A Database for Spontaneous Macro-Expression and Micro-Expression Spotting and Recognition
Qu, F., Wang, S.-J., Yan, W.-J., Li, H., Wu, S., & Fu, X. (2018) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2017.2654440 · Cited by 293
Automatically Detecting Pain in Video Through Facial Action Units
Lucey, P., Cohn, J. F., Matthews, I., Lucey, S., Sridharan, S., Howlett, J., & Prkachin, K. M. (2011) · IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) · doi:10.1109/tsmcb.2010.2082525 · Cited by 267
Nonverbal social withdrawal in depression: Evidence from manual and automatic analyses
Girard, J. M., Cohn, J. F., Mahoor, M. H., Mavadati, S. M., Hammal, Z., & Rosenwald, D. P. (2014) · Image and Vision Computing · doi:10.1016/j.imavis.2013.12.007 · Cited by 255
McDuff, D., el Kaliouby, R., Senechal, T., Amr, M., Cohn, J. F., & Picard, R. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2013.130 · Cited by 234
Facial Expression Recognition: A Survey
Huang, Y., Chen, F., Lv, S., & Wang, X. (2019) · Symmetry · doi:10.3390/sym11101189 · Cited by 195
Emotion recognition in the wild challenge 2013
Dhall, A., Goecke, R., Joshi, J., Wagner, M., & Gedeon, T. (2013) · Proceedings of the 15th ACM on International conference on multimodal interaction · doi:10.1145/2522848.2531739 · Cited by 178
Automatic behavior descriptors for psychological disorder analysis
Scherer, S., Stratou, G., Mahmoud, M., Boberg, J., Gratch, J., Rizzo, A., & Morency, L.-P. (2013) · 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG) · doi:10.1109/fg.2013.6553789 · Cited by 147
Regression-based intensity estimation of facial action units
Savran, A., Sankur, B., & Taha Bilge, M. (2012) · Image and Vision Computing · doi:10.1016/j.imavis.2011.11.008 · Cited by 129
Facial expression recognition using Gabor motion energy filters
Wu, T., Bartlett, M. S., & Movellan, J. R. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops · doi:10.1109/cvprw.2010.5543267 · Cited by 126
“In fact some of applications, like digital cameras with automatic smile detection, have found their way to daily life consumer electronics [20].”
BackgroundSelf-citationComparative evaluation of 3D vs. 2D modality for automatic detection of facial action units
Savran, A., Sankur, B., & Taha Bilge, M. (2012) · Pattern Recognition · doi:10.1016/j.patcog.2011.07.022 · Cited by 111
The Machine Knows What You Are Hiding: An Automatic Micro-expression Recognition System
Wu, Q., Shen, X., & Fu, X. (2011) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-24571-8_16 · Cited by 104
The more the merrier: Analysing the affect of a group of people in images
Dhall, A., Joshi, J., Sikka, K., Goecke, R., & Sebe, N. (2015) · 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG) · doi:10.1109/fg.2015.7163151 · Cited by 93
Automatic measurement of ad preferences from facial responses gathered over the Internet
McDuff, D., El Kaliouby, R., Senechal, T., Demirdjian, D., & Picard, R. (2014) · Image and Vision Computing · doi:10.1016/j.imavis.2014.01.004 · Cited by 52
Estimating smile intensity: A better way
Girard, J. M., Cohn, J. F., & De la Torre, F. (2015) · Pattern Recognition Letters · doi:10.1016/j.patrec.2014.10.004 · Cited by 44
Fast and Robust Smile Intensity Estimation by Cascaded Support Vector Machines
Shimada, K., Noguchi, Y., & Kuria, T. (2013) · International Journal of Computer Theory and Engineering · doi:10.7763/ijcte.2013.v5.640 · Cited by 9
Metadata
- Authors
- Whitehill, J., Littlewort, G., Fasel, I., Bartlett, M., & Movellan, J.
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2009-11
- Volume
- 31
- Issue
- 11
- Pages
- 2106-2111
- ISSN
- 0162-8828 (ISSN-L)
- OpenAlex
- W2157653492
- Cited by
- 308
- References
- 40
- FWCI
- 27.32
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.