The DOI matches its Crossref record; 27 of the 27 references it cites were found in a registry or an index.
The computer expression recognition toolbox (CERT)
Abstract
We present the Computer Expression Recognition Toolbox (CERT), a software tool for fully automatic real-time facial expression recognition, and officially release it for free academic use. CERT can automatically code the intensity of 19 different facial actions from the Facial Action Unit Coding System (FACS) and 6 different prototypical facial expressions. It also estimates the locations of 10 facial features as well as the 3-D orientation (yaw, pitch, roll) of the head. On a database of posed facial expressions, Extended Cohn-Kanade (CK+[1]), CERT achieves an average recognition performance (probability of correctness on a two-alternative forced choice (2AFC) task between one positive and one negative example) of 90.1% when analyzing facial actions. On a spontaneous facial expression dataset, CERT achieves an accuracy of nearly 80%. In a standard dual core laptop, CERT can process 320 × 240 video images in real time at approximately 10 frames per second.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Robust Real-Time Face Detection
Viola, P., & Jones, M. J. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000013087.49260.fb · Cited by 14,097
Friedman, J., Hastie, T., & Tibshirani, R. (2000) · The Annals of Statistics · doi:10.1214/aos/1016218223 · Cited by 6,884
Ekman, P., & Friesen, W. V. (2019) · PsycTESTS Dataset · doi:10.1037/t27734-000 · Cited by 6,019
Lucey, P., Cohn, J. F., Kanade, T., Saragih, J., Ambadar, Z., & Matthews, I. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops · doi:10.1109/cvprw.2010.5543262 · Cited by 4,303
Comprehensive database for facial expression analysis
Kanade, T., Cohn, J. F., & Tian, Y. (2002) · Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580) · doi:10.1109/afgr.2000.840611 · Cited by 2,591
Web-Based Database for Facial Expression Analysis
Pantic, M., Valstar, M., Rademaker, R., & Maat, L. (2005) · 2005 IEEE International Conference on Multimedia and Expo · doi:10.1109/icme.2005.1521424 · Cited by 1,039
Donato, G., Bartlett, M. S., Hager, J. C., Ekman, P., & Sejnowski, T. J. (1999) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.799905 · Cited by 999
Self-citationAutomatic prediction of frustration
Kapoor, A., Burleson, W., & Picard, R. W. (2007) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2007.02.003 · Cited by 592
Keltner, D., Ekman, P., Gonzaga, & Beer, J. (2002) · Handbook of Affective Sciences · doi:10.1093/oso/9780195126013.003.0022 · Cited by 513
Automatic Recognition of Facial Actions in Spontaneous Expressions
Bartlett, M. S., Littlewort, G. C., Frank, M. G., Lainscsek, C., Fasel, I. R., & Movellan, J. R. (2006) · Journal of Multimedia · doi:10.4304/jmm.1.6.22-35 · Cited by 468
Self-citation
Research works printed but not linked: 3
- frank, 2010, The M3 database of spontaneous emotion expression (University of Buffalo), Prep
- pierce, 0, Perceptual and motor learning in the recognition and production of dynamic facial expressions, All Hands Meeting of the Temporal Dynamics of Learning Center UCSD
- 10.1037/0022-3514.72.6.1429 · doi:10.1037/0022-3514.72.6.1429 · Paper page
Cited by
The Faces of Engagement: Automatic Recognition of Student Engagementfrom Facial Expressions
Whitehill, J., Serpell, Z., Lin, Y.-C., Foster, A., & Movellan, J. R. (2014) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2014.2316163 · Cited by 677
Self-citationEmotional Mimicry as Social Regulation
Hess, U., & Fischer, A. (2013) · Personality and Social Psychology Review · doi:10.1177/1088868312472607 · Cited by 596
Corneanu, C. A., Simon, M. O., Cohn, J. F., & Guerrero, S. E. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2515606 · Cited by 527
AVEC 2016: Depression, Mood, and Emotion Recognition Workshop and Challenge
Valstar, M., Gratch, J., Schuller, B., Ringeval, F., Lalanne, D., Torres Torres, M., Scherer, S., Stratou, G., Cowie, R., & Pantic, M. (2016) · Proceedings of the 6th International Workshop on Audio/Visual Emotion Challenge · doi:10.1145/2988257.2988258 · Cited by 503
Facial emotion recognition using convolutional neural networks (FERC)
Mehendale, N. (2020) · SN Applied Sciences · doi:10.1007/s42452-020-2234-1 · Cited by 413
Learning Expressionlets on Spatio-temporal Manifold for Dynamic Facial Expression Recognition
Liu, M., Shan, S., Wang, R., & Chen, X. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.226 · Cited by 392
Gunes, H., & Schuller, B. (2013) · Image and Vision Computing · doi:10.1016/j.imavis.2012.06.016 · Cited by 360
Advanced, Analytic, Automated (AAA) Measurement of Engagement During Learning
D'Mello, S., Dieterle, E., & Duckworth, A. (2017) · Educational Psychologist · doi:10.1080/00461520.2017.1281747 · Cited by 350
Engagement detection in online learning: a review
Dewan, M. A. A., Murshed, M., & Lin, F. (2019) · Smart Learning Environments · doi:10.1186/s40561-018-0080-z · Cited by 350
Selective Transfer Machine for Personalized Facial Action Unit Detection
Chu, W.-S., De la Torre, F., & Cohn, J. F. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.451 · Cited by 307
Chen, X., Xie, H., & Hwang, G.-J. (2020) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2020.100005 · Cited by 303
Meta-Analysis of the First Facial Expression Recognition Challenge
Valstar, M. F., Mehu, M., Jiang, B., Pantic, M., & Scherer, K. (2012) · IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) · doi:10.1109/tsmcb.2012.2200675 · Cited by 297
Facial expression analysis with AFFDEX and FACET: A validation study
Stöckli, S., Schulte-Mecklenbeck, M., Borer, S., & Samson, A. C. (2018) · Behavior Research Methods · doi:10.3758/s13428-017-0996-1 · Cited by 294
MACH: my automated conversation coach
Hoque, M. (., Courgeon, M., Martin, J.-C., Mutlu, B., & Picard, R. W. (2013) · Proceedings of the 2013 ACM international joint conference on Pervasive and ubiquitous computing · doi:10.1145/2493432.2493502 · Cited by 290
Kim, D. H., Baddar, W. J., Jang, J., & Ro, Y. M. (2019) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2017.2695999 · Cited by 257
Deep learning for depression recognition with audiovisual cues: A review
He, L., Niu, M., Tiwari, P., Marttinen, P., Su, R., Jiang, J., Guo, C., Wang, H., Ding, S., Wang, Z., et al. (2022) · Information Fusion · doi:10.1016/j.inffus.2021.10.012 · Cited by 253
(2013) · Cited by 236
A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face
Lian, H., Lu, C., Li, S., Zhao, Y., Tang, C., & Zong, Y. (2023) · Entropy · doi:10.3390/e25101440 · Cited by 224
“At present, commonly utilized libraries consist of OKAO, Computer Expression Recognition Toolbox (CERT) [74], OpenFace [75], and FACET.”
MethodVocal and Facial Biomarkers of Depression based on Motor Incoordination and Timing
Williamson, J. R., Quatieri, T. F., Helfer, B. S., Ciccarelli, G., & Mehta, D. D. (2014) · Proceedings of the 4th International Workshop on Audio/Visual Emotion Challenge · doi:10.1145/2661806.2661809 · Cited by 209
Lab And/Or Field? Measuring Personality Processes and Their Social Consequences
Wrzus, C., & Mehl, M. R. (2015) · European Journal of Personality · doi:10.1002/per.1986 · Cited by 200
Emotional Responses to Bureaucratic Red Tape
Hattke, F., Hensel, D., & Kalucza, J. (2020) · Public Administration Review · doi:10.1111/puar.13116 · Cited by 179
Automatic Detection of Learning-Centered Affective States in the Wild
Bosch, N., D'Mello, S., Baker, R., Ocumpaugh, J., Shute, V., Ventura, M., Wang, L., & Zhao, W. (2015) · Proceedings of the 20th International Conference on Intelligent User Interfaces · doi:10.1145/2678025.2701397 · Cited by 178
Recognizing Emotions in Video Using Multimodal DNN Feature Fusion
Williams, J., Kleinegesse, S., Comanescu, R., & Radu, O. (2018) · Proceedings of Grand Challenge and Workshop on Human Multimodal Language (Challenge-HML) · doi:10.18653/v1/w18-3302 · Cited by 132
A multimodal dataset for various forms of distracted driving
Taamneh, S., Tsiamyrtzis, P., Dcosta, M., Buddharaju, P., Khatri, A., Manser, M., Ferris, T., Wunderlich, R., & Pavlidis, I. (2017) · Scientific Data · doi:10.1038/sdata.2017.110 · Cited by 129
Trevisan, D. A., Bowering, M., & Birmingham, E. (2016) · Molecular Autism · doi:10.1186/s13229-016-0108-6 · Cited by 107
“While FACET is in need of an extensive validation study, its academic predecessor, the Computer Expression Recognition Toolbox (CERT), from which FACET’s technology was derived, has demonstrated strong psychometric properties, as evidenced by its ability to correctly detect emotions from standardized facial expression…”
BackgroundFaceTube: predicting personality from facial expressions of emotion in online conversational video
Biel, J.-I., Teijeiro-Mosquera, L., & Gatica-Perez, D. (2012) · Proceedings of the 14th ACM international conference on Multimodal interaction · doi:10.1145/2388676.2388689 · Cited by 104
The more the merrier: Analysing the affect of a group of people in images
Dhall, A., Joshi, J., Sikka, K., Goecke, R., & Sebe, N. (2015) · 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG) · doi:10.1109/fg.2015.7163151 · Cited by 93
Synchronized affect in shared experiences strengthens social connection
Cheong, J. H., Molani, Z., Sadhukha, S., & Chang, L. J. (2023) · Communications Biology · doi:10.1038/s42003-023-05461-2 · Cited by 75
Spontaneous facial expression in unscripted social interactions can be measured automatically
Girard, J. M., Cohn, J. F., Jeni, L. A., Sayette, M. A., & De la Torre, F. (2015) · Behavior Research Methods · doi:10.3758/s13428-014-0536-1 · Cited by 69
Hi YouTube!: personality impressions and verbal content in social video
Biel, J.-I., Tsiminaki, V., Dines, J., & Gatica-Perez, D. (2013) · Proceedings of the 15th ACM on International conference on multimodal interaction · doi:10.1145/2522848.2522877 · Cited by 66
MindLink-Eumpy: An Open-Source Python Toolbox for Multimodal Emotion Recognition
Li, R., Liang, Y., Liu, X., Wang, B., Huang, W., Cai, Z., Ye, Y., Qiu, L., & Pan, J. (2021) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2021.621493 · Cited by 36
Multimodal modeling of collaborative problem-solving facets in triads
Stewart, A. E. B., Keirn, Z., & D’Mello, S. K. (2021) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-021-09290-y · Cited by 30
Klimecki, O. M., Vuilleumier, P., & Sander, D. (2016) · PLoS ONE · doi:10.1371/journal.pone.0151028 · Cited by 26
“The Computer Expression Recognition Toolbox [42] was used to quantify the intensity of smiles and frowns, indexing positive and negative affect, respectively.”
Pearlstein, S. L., Taylor, C. T., & Stein, M. B. (2019) · Clinical Psychological Science · doi:10.1177/2167702619825857 · Cited by 17
Metadata
- Authors
- Littlewort, G., Whitehill, J., Wu, T., Fasel, I., Frank, M., Movellan, J., & Bartlett, M.
- Venue
- Face and Gesture 2011
- Publisher
- IEEE
- Published
- 2011-03
- Pages
- 298-305
- OpenAlex
- W1968600824
- Cited by
- 539
- References
- 27
- FWCI
- 57.75
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.