The DOI matches its Crossref record; 39 of the 41 references it cites were found in a registry or an index.
Recognizing action units for facial expression analysis
Open accessAbstract
Most automatic expression analysis systems attempt to recognize a small set of prototypic expressions, such as happiness, anger, surprise, and fear. Such prototypic expressions, however, occur rather infrequently. Human emotions and intentions are more often communicated by changes in one or a few discrete facial features. In this paper, we develop an Automatic Face Analysis (AFA) system to analyze facial expressions based on both permanent facial features (brows, eyes, mouth) and transient facial features (deepening of facial furrows) in a nearly frontal-view face image sequence. The AFA system recognizes fine-grained changes in facial expression into action units (AUs) of the Facial Action Coding System (FACS), instead of a few prototypic expressions. Multistate face and facial component models are proposed for tracking and modeling the various facial features, including lips, eyes, brows, cheeks, and furrows. During tracking, detailed parametric descriptions of the facial features are extracted. With these parameters as the inputs, a group of action units (neutral expression, six upper face AUs and 10 lower face AUs) are recognized whether they occur alone or in combinations. The system has achieved average recognition rates of 96.4 percent (95.4 percent if neutral expressions are excluded) for upper face AUs and 96.7 percent (95.6 percent with neutral expressions excluded) for lower face AUs. The generalizability of the system has been tested by using independent image databases collected and FACS-coded for ground-truth by different research teams.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Based on the observation that some AU are better detected using geometrical features and others using appearance ones, it was suggested that a combination of both might increase recognition performance [127], [139], [183].”
From Survey on RGB, 3D, Thermal, and Multimodal Approaches for Facial Expression Recognition: History, Trends, and Affect-Related Applications (2016) · cites this paper as [139]
In the citing paper
“In the first part of the decade static representations were the primary choice in both RGB [99], [139], 3D [118], [120], [125], [131], [133], [219] and thermal [111].”
From Survey on RGB, 3D, Thermal, and Multimodal Approaches for Facial Expression Recognition: History, Trends, and Affect-Related Applications (2016) · cites this paper as [139]
In the citing paper
“In 2000, the CK dataset was published marking the beginning of modern AFER [139].”
From Survey on RGB, 3D, Thermal, and Multimodal Approaches for Facial Expression Recognition: History, Trends, and Affect-Related Applications (2016) · cites this paper as [139]
In the citing paper
“Others focused on appearance representations like Gabor filters [99], optical flow and LBPs [97] or combinations between the two [139].”
From Survey on RGB, 3D, Thermal, and Multimodal Approaches for Facial Expression Recognition: History, Trends, and Affect-Related Applications (2016) · cites this paper as [139]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Computational Approach to Edge Detection
Canny, J. (1986) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.1986.4767851 · Cited by 30,004
Statistical Methods for Rates and Proportions
Fleiss, J. L., Levin, B., & Paik, M. C. (2003) · Wiley Series in Probability and Statistics · doi:10.1002/0471445428 · Cited by 18,646
The expression of the emotions in man and animals.
Darwin, C. D. (1872) · J. Murray eBooks · doi:10.1037/10001-000 · Cited by 12,120
(1981) · Cited by 11,963
Face recognition using eigenfaces
Turk, M., & Pentland, A. (2002) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.1991.139758 · Cited by 5,262
Ekman, P. (1976) · Medical Entomology and Zoology · Cited by 4,345
Neural network-based face detection
Rowley, H. A., Baluja, S., & Kanade, T. (1998) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.655647 · Cited by 3,554
Ekman, P. (1993) · American Psychologist · doi:10.1037/0003-066x.48.4.384 · Cited by 3,441
Comprehensive database for facial expression analysis
Takeo Kanade, A., Cohn, J. F., & Tian, Y. (2002) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/afgr.2000.840611 · Cited by 2,644
Self-citationApplication of the Karhunen-Loeve procedure for the characterization of human faces
Kirby, M. J. L., & Sirovich, L. (1990) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.41390 · Cited by 2,459
Face recognition: features versus templates
Brunelli, R., & Poggio, T. (1993) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.254061 · Cited by 2,452
Statistical Methods for Rates and Proportions.
Plewis, I., & Fleiss, J. L. (1982) · Journal of the Royal Statistical Society Series A (General) · doi:10.2307/2982107 · Cited by 1,753
Feature extraction from faces using deformable templates
Yuille, A., Hallinan, P. W., & Cohen, D. S. (1992) · International Journal of Computer Vision · doi:10.1007/bf00127169 · Cited by 1,373
The Expression of the Emotions in Man and Animals
Darwin, C. D. (2013) · Cambridge University Press eBooks · doi:10.1017/cbo9781139833813 · Cited by 1,351
Donato, G., Bartlett, M. S., Hager, J. C., Ekman, P., & B, T. J. S. A. (1999) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.799905 · Cited by 999
Coding, analysis, interpretation, and recognition of facial expressions
Essa, I. A., & Pentland, A. (1997) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.598232 · Cited by 917
Age Classification from Facial Images
Kwon, Y. H., & Lobo, N. D. V. (1999) · Computer Vision and Image Understanding · doi:10.1006/cviu.1997.0549 · Cited by 550
Handbook of methods in nonverbal behavior research
Scherer, K. R., & Ekman, P. (1982) · Cited by 521
Recognizing Facial Expressions in Image Sequences Using Local Parameterized Models of Image Motion
Black, M. J., & Yacoob, Y. (1997) · International Journal of Computer Vision · doi:10.1023/a:1007977618277 · Cited by 514
Recognizing human facial expressions from long image sequences using optical flow
Yacoob, Y., & Davis, L. S. (1996) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.506414 · Cited by 463
Black, M. J., & Yacoob, Y. (2002) · Proceedings of IEEE International Conference on Computer Vision · doi:10.1109/iccv.1995.466915 · Cited by 457
Measuring facial expressions by computer image analysis
Bartlett, M. S., Hager, J. C., Ekman, P., & Sejnowski, T. J. (1999) · Psychophysiology · doi:10.1017/s0048577299971664 · Cited by 424
Recognition of Facial Expression from Optical Flow
Mase, K. (1991) · IEICE Transactions on Information and Systems · Cited by 404
Parameterized Modeling and Recognition of Activities
Yacoob, Y., & Black, M. J. (1999) · Computer Vision and Image Understanding · doi:10.1006/cviu.1998.0726 · Cited by 315
Locating and extracting the eye in human face images
Lam, K., & Yan, H. (1996) · Pattern Recognition · doi:10.1016/0031-3203(95)00119-0 · Cited by 313
Detection, tracking, and classification of action units in facial expression
Lien, J. J.-J., Takeo Kanade, A., Cohn, J. F., & Li, C.-C. (2000) · Robotics and Autonomous Systems · doi:10.1016/s0921-8890(99)00103-7 · Cited by 312
Self-citationAge classification from facial images
Kwon, Y. H., & Lobo, D. V. (1994) · doi:10.1109/cvpr.1994.323894 · Cited by 291
Cohn, J. F., Zlochower, A. J., Lien, J. J.-J., & Takeo Kanade, A. (1999) · Psychophysiology · doi:10.1017/s0048577299971184 · Cited by 287
Self-citationHuman expression recognition from motion using a radial basis function network architecture
Rosenblum, M., Yacoob, Y., & Davis, L. S. (1996) · IEEE Transactions on Neural Networks · doi:10.1109/72.536309 · Cited by 284
Facial expressions in Hollywood's protrayal of emotion.
Carroll, J. M., & Russell, J. A. (1997) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.72.1.164 · Cited by 184
Dual-state parametric eye tracking
Tian, Y., Kanade, T., & Cohn, J. F. (2002) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/afgr.2000.840620 · Cited by 161
Self-citationZHANG, Z. (1999) · International Journal of Pattern Recognition and Artificial Intelligence · doi:10.1142/s0218001499000495 · Cited by 146
Facial expressions in Hollywood's protrayal of emotion.
Carroll, J. M., & Russell, J. A. (1997) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.72.1.164 · Cited by 140
Parameterized modeling and recognition of activities
Yacoob, Y., & Black, M. J. (2002) · International Conference on Computer Vision · doi:10.1109/iccv.1998.710709 · Cited by 119
Analysis of facial images using physical and anatomical models
Terzopoulos, D., & Waters, K. (2002) · International Conference on Computer Vision · doi:10.1109/iccv.1990.139628 · Cited by 108
Robust Lip Tracking by Combining Shape, Color and Motion
Tian, Y., Kanade, T., & Cohn, J. F. (2000) · Cited by 40
Self-citationAudio visual interaction in multimedia
Chen, T., & Rao, R. (1995) · IEEE Circuits and Devices Magazine · doi:10.1109/101.476612 · Cited by 10
Naemura, M., Fukuda, A., Mizutani, Y., Izumi, Y., Yamaguchi, K., & Ninomiya, Y. (1998) · Systems and Computers in Japan · doi:10.1002/(sici)1520-684x(19980615)29:6 · Cited by 1
Audio-visual interaction in multimedia
Rao, R. R. (1998) · SMARTech Repository (Georgia Institute of Technology) · Cited by 1
Flagged first, then by citations
2 of the 41 references couldn’t be matched to a known paper.
Research works printed but not linked: 5
- eibl-eihesfeldt, 1989, Human Ethology
- ekman, 1978, The Facial Action Coding System A Technique for The Measurement of Facial Movement
- suwa, 1978, A Preliminary Note on Pattern Recognition of Human Emotional Expression, Proc Int'l Joint Conf Pattern Recognition, 408
- 2000, Facial Expression Coding Project
- 10.1109/34.506414 · DOI
Cited by
Sorted by citations
AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild
Mollahosseini, A., Hasani, B., & Mahoor, M. H. (2019) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2017.2740923 · Cited by 2,338
Facial expression recognition based on Local Binary Patterns: A comprehensive study
Shan, C., Gong, S., & McOwan, P. W. (2009) · Image and Vision Computing · doi:10.1016/j.imavis.2008.08.005 · Cited by 2,300
Deep Facial Expression Recognition: A Survey
Li, S., & Deng, W. (2020) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2020.2981446 · Cited by 2,175
Mitra, S., & Acharya, T. (2007) · IEEE Transactions on Systems Man and Cybernetics Part C (Applications and Reviews) · doi:10.1109/tsmcc.2007.893280 · Cited by 2,048
Automatic facial expression analysis: a survey
Fasel, B., & Luettin, J. (2003) · Pattern Recognition · doi:10.1016/s0031-3203(02)00052-3 · Cited by 1,810
Metadata
- Authors
- Tian, Y.-I., Kanade, T., & Cohn, J. F.
- Affiliations
- Carnegie Mellon University (US)
- University of Pittsburgh (US)
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2001
- Volume
- 23
- Issue
- 2
- Pages
- 97-115
- ISSN
- 0162-8828 (ISSN-L)
- PMC
- PMC4157835
- OpenAlex
- W2106390385
- Cited by
- 1,745
- References
- 41
- Citations by year
- 2026: 34 · 2025: 56 · 2024: 71 · 2023: 70 · 2022: 106 · 2021: 100 · 2020: 78 · 2019: 94 · 2018: 63 · 2017: 70 · 2016: 80 · 2015: 72 · 2014: 85 · 2013: 80 · 2012: 67
- FWCI
- 52.28
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- IEEE Transactions on Pattern Analysis and Machine Intelligence (publishedVersion)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- Funders
- University of Pittsburgh
- National Institute of Mental Health (R01MH051435)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face and Expression Recognition
- Keywords
- facial expression analysis, facial action units, facial action coding system, facial feature tracking, facial expression recognition, neutral expression
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
