The DOI matches its Crossref record; 68 of the 68 references it cites were found in a registry or an index.
Automated Facial Action Coding System for dynamic analysis of facial expressions in neuropsychiatric disorders
Open accessAbstract
Facial expression is widely used to evaluate emotional impairment in neuropsychiatric disorders. Ekman and Friesen’s Facial Action Coding System (FACS) encodes movements of individual facial muscles from distinct momentary changes in facial appearance. Unlike facial expression ratings based on categorization of expressions into prototypical emotions (happiness, sadness, anger, fear, disgust, etc.), FACS can encode ambiguous and subtle expressions, and therefore is potentially more suitable for analyzing the small differences in facial affect. However, FACS rating requires extensive training, and is time consuming and subjective thus prone to bias. To overcome these limitations, we developed an automated FACS based on advanced computer science technology. The system automatically tracks faces in a video, extracts geometric and texture features, and produces temporal profiles of each facial muscle movement. These profiles are quantified to compute frequencies of single and combined Action Units (AUs) in videos, which can facilitate statistical study of large populations in disorders affecting facial expression. We derived quantitative measures of flat and inappropriate facial affect automatically from temporal AU profiles. Applicability of the automated FACS was illustrated in a pilot study, by applying it to data of videos from eight schizophrenia patients and controls. We created temporal AU profiles that provided rich information on the dynamics of facial muscle movements for each subject. The quantitative measures of flatness and inappropriateness showed clear differences between patients and the controls, highlighting their potential in automatic and objective quantification of symptom severity.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Rapid object detection using a boosted cascade of simple features
Viola, P., & Jones, M. (2005) · Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001 · doi:10.1109/cvpr.2001.990517 · Cited by 18,300
Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond
Schölkopf, B., & Smola, A. J. (2001) · The MIT Press eBooks · doi:10.7551/mitpress/4175.001.0001 · Cited by 11,703
Active Shape Models-Their Training and Application
Cootes, T. F., Taylor, C. J., Cooper, D. H., & Graham, J. (1995) · Computer Vision and Image Understanding · doi:10.1006/cviu.1995.1004 · Cited by 7,190
Friedman, J., Hastie, T., & Tibshirani, R. (2000) · The Annals of Statistics · doi:10.1214/aos/1016218223 · Cited by 6,884
Cootes, T. F., Edwards, G. J., & Taylor, C. J. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.927467 · Cited by 5,409
Detecting faces in images: a survey
Yang, M.-H., Kriegman, D. J., & Ahuja, N. (2002) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.982883 · Cited by 3,427
An extended set of Haar-like features for rapid object detection
Lienhart, R., & Maydt, J. (2003) · Proceedings. International Conference on Image Processing · doi:10.1109/icip.2002.1038171 · Cited by 2,977
Statistical shape analysis: clustering, learning, and testing
Srivastava, A., Joshi, S. H., Mio, W., & Liu, X. (2005) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2005.86 · Cited by 2,727
Scale for the Assessment of Negative Symptoms (SANS)*
(1989) · British Journal of Psychiatry · doi:10.1192/s0007125000291502 · Cited by 2,701
Comprehensive database for facial expression analysis
Kanade, T., Cohn, J. F., & Tian, Y. (2002) · Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580) · doi:10.1109/afgr.2000.840611 · Cited by 2,591
Scale for Assessment of Positive Symptoms (SAPS) 1984
Andreasen, N. C. (1995) · Open Scholarship Institutional Repository (Washington University in St. Louis) · Cited by 2,423
Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2005.848998 · Cited by 2,225
Pictorial Structures for Object Recognition
Felzenszwalb, P. F., & Huttenlocher, D. P. (2005) · International Journal of Computer Vision · doi:10.1023/b:visi.0000042934.15159.49 · Cited by 2,180
(2011) · Springer eBooks · doi:10.1007/978-0-85729-932-1 · Cited by 1,819
Automatic facial expression analysis: a survey
Fasel, B., & Luettin, J. (2003) · Pattern Recognition · doi:10.1016/s0031-3203(02)00052-3 · Cited by 1,810
Automatic analysis of facial expressions: the state of the art
Pantie, M., & Rothkrantz, L. J. M. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.895976 · Cited by 1,778
Recognizing action units for facial expression analysis
Tian, Y.-I., Kanade, T., & Cohn, J. F. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.908962 · Cited by 1,745
Ekman, P. (1975) · Cited by 1,565
The Scale for the Assessment of Negative Symptoms (SANS): Conceptual and Theoretical Foundations
Andreasen, N. C. (1989) · British Journal of Psychiatry · doi:10.1192/s0007125000291496 · Cited by 1,530
Dementia Praecox oder Gruppe der Schizophrenien
Bleuler, E. (1911) · Cited by 1,429
Eibl‐Eibesfeldt, I. (1970) · Max Planck Digital Library · Cited by 1,053
Web-Based Database for Facial Expression Analysis
Pantic, M., Valstar, M., Rademaker, R., & Maat, L. (2005) · 2005 IEEE International Conference on Multimedia and Expo · doi:10.1109/icme.2005.1521424 · Cited by 1,039
Facial expression recognition from video sequences: temporal and static modeling
Cohen, I., Sebe, N., Garg, A., Chen, L. S., & Huang, T. S. (2003) · Computer Vision and Image Understanding · doi:10.1016/s1077-3142(03)00081-x · Cited by 874
Innate and universal facial expressions: Evidence from developmental and cross-cultural research.
Izard, C. E. (1994) · Psychological Bulletin · doi:10.1037//0033-2909.115.2.288 · Cited by 856
Srivastava, A., Kurtek, S., & Klassen, E. (2014) · Computer Vision · doi:10.1007/978-0-387-31439-6_778 · Cited by 848
(2005) · Springer eBooks · doi:10.1007/b138828 · Cited by 789
Kotsia, I., & Pitas, I. (2007) · IEEE Transactions on Image Processing · doi:10.1109/tip.2006.884954 · Cited by 687
Gur, R. C., Sara, R., Hagendoorn, M., Marom, O., Hughett, P., Macy, L., Turner, T., Bajcsy, R., Posner, A., & Gur, R. E. (2002) · Journal of Neuroscience Methods · doi:10.1016/s0165-0270(02)00006-7 · Cited by 647
Self-citationThe consistency of facial expressions of pain: a comparison across modalities
Prkachin, K. M. (1992) · Pain · doi:10.1016/0304-3959(92)90213-u · Cited by 533
Automatic Recognition of Facial Actions in Spontaneous Expressions
Bartlett, M. S., Littlewort, G. C., Frank, M. G., Lainscsek, C., Fasel, I. R., & Movellan, J. R. (2006) · Journal of Multimedia · doi:10.4304/jmm.1.6.22-35 · Cited by 468
Active and dynamic information fusion for facial expression understanding from image sequences
Zhang, Y., & Ji, Q. (2005) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2005.93 · Cited by 461
A Bayesian tutorial for data assimilation
Wikle, C. K., & Berliner, L. M. (2007) · Physica D: Nonlinear Phenomena · doi:10.1016/j.physd.2006.09.017 · Cited by 453
Expert system for automatic analysis of facial expressions
Pantic, M., & Rothkrantz, L. J. M. (2000) · Image and Vision Computing · doi:10.1016/s0262-8856(00)00034-2 · Cited by 415
Facial Action Unit Recognition by Exploiting Their Dynamic and Semantic Relationships
Tong, Y., Liao, W., & Ji, Q. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.1094 · Cited by 378
Dynamics of facial expression extracted automatically from video
Littlewort, G., Bartlett, M. S., Fasel, I., Susskind, J., & Movellan, J. (2006) · Image and Vision Computing · doi:10.1016/j.imavis.2005.09.011 · Cited by 358
Fully Automatic Facial Action Unit Detection and Temporal Analysis
Valstar, M., & Pantic, M. (2006) · 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'06) · doi:10.1109/cvprw.2006.85 · Cited by 346
Automatic feature localisation with constrained local models
Cristinacce, D., & Cootes, T. (2008) · Pattern Recognition · doi:10.1016/j.patcog.2008.01.024 · Cited by 320
Flat affect in schizophrenia does not reflect diminished subjective experience of emotion.
Kring, A. M., Kerr, S. L., Smith, D. A., & Neale, J. M. (1993) · Journal of Abnormal Psychology · doi:10.1037/0021-843x.102.4.507 · Cited by 319
Detection, tracking, and classification of action units in facial expression
Lien, J. J.-J., Kanade, T., Cohn, J. F., & Li, C.-C. (2000) · Robotics and Autonomous Systems · doi:10.1016/s0921-8890(99)00103-7 · Cited by 309
Fame-a flexible appearance modeling environment
Stegmann, M. B., Ersboll, B. K., & Larsen, R. (2003) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2003.817780 · Cited by 303
Differences in facial expressions of four universal emotions
Kohler, C. G., Turner, T., Stolar, N. M., Bilker, W. B., Brensinger, C. M., Gur, R. E., & Gur, R. C. (2004) · Psychiatry Research · doi:10.1016/j.psychres.2004.07.003 · Cited by 276
Self-citationGeneric vs. person specific active appearance models
Gross, R., Matthews, I., & Baker, S. (2005) · Image and Vision Computing · doi:10.1016/j.imavis.2005.07.009 · Cited by 272
Machine Analysis of Facial Expressions
Pantic, M., & Stewart, M. (2007) · Face Recognition · doi:10.5772/4847 · Cited by 248
A Dynamic Texture-Based Approach to Recognition of Facial Actions and Their Temporal Models
Koelstra, S., Pantic, M., & Patras, I. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.50 · Cited by 242
Flat Affect in Schizophrenia: Relation to Emotion Processing and Neurocognitive Measures
Gur, R. E., Kohler, C. G., Ragland, J. D., Siegel, S. J., Lesko, K., Bilker, W. B., & Gur, R. C. (2006) · Schizophrenia Bulletin · doi:10.1093/schbul/sbj041 · Cited by 240
Self-citationChildhood precursors of schizophrenia: facial expressions of emotion
(1993) · American Journal of Psychiatry · doi:10.1176/ajp.150.11.1654 · Cited by 210
Genuine, suppressed and faked facial behavior during exacerbation of chronic low back pain
Craig, K. D., Hyde, S. A., & Patrick, C. J. (1991) · Pain · doi:10.1016/0304-3959(91)90071-5 · Cited by 183
Gender differences in the clinical expression of schizophrenia
Shtasel, D. L., Gur, R. E., Gallacher, F., Heimberg, C., & Gur, R. C. (1992) · Schizophrenia Research · doi:10.1016/0920-9964(92)90016-x · Cited by 176
The Facial Expression Coding System (FACES): Development, validation, and utility.
Kring, A. M., & Sloan, D. M. (2007) · Psychological Assessment · doi:10.1037/1040-3590.19.2.210 · Cited by 165
Components and recognition of facial expression in the communication of emotion by actors.
Gosselin, P., Kirouac, G., & Doré, F. Y. (1995) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.68.1.83 · Cited by 163
Investigating Spontaneous Facial Action Recognition through AAM Representations of the Face
Lucey, S., Bilal, A., & F., J. (2007) · Face Recognition · doi:10.5772/4841 · Cited by 161
Cohen, I., Sebe, N., Gozman, F. G., Cirelo, M. C., & Huang, T. S. (2003) · 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings. · doi:10.1109/cvpr.2003.1211408 · Cited by 155
A Unified Probabilistic Framework for Spontaneous Facial Action Modeling and Understanding
Tong, Y., Chen, J., & Ji, Q. (2008) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.293 · Cited by 141
Prkachin, K. M., & Mercer, S. R. (1989) · Pain · doi:10.1016/0304-3959(89)90038-9 · Cited by 131
Boosting Coded Dynamic Features for Facial Action Units and Facial Expression Recognition
Yang, P., Liu, Q., & Metaxas, D. (2007) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2007.383059 · Cited by 129
Genuine, suppressed and faked facial expressions of pain in children
Larochette, A.-C., Chambers, C. T., & Craig, K. D. (2006) · Pain · doi:10.1016/j.pain.2006.06.013 · Cited by 121
Dementia praecox oder die Gruppe der Schizophrenien: Eugen Bleuler
BLEULER, M., & Bleuler, R. (1986) · The British Journal of Psychiatry · doi:10.1192/bjp.149.5.661 · Cited by 117
Lints-Martindale, A. C., Hadjistavropoulos, T., Barber, B., & Gibson, S. J. (2007) · Pain Medicine · doi:10.1111/j.1526-4637.2007.00358.x · Cited by 104
Impact of depression on response to comedy: A dynamic facial coding analysis.
Reed, L. I., Sayette, M. A., & Cohn, J. F. (2007) · Journal of Abnormal Psychology · doi:10.1037/0021-843x.116.4.804 · Cited by 103
Automated video-based facial expression analysis of neuropsychiatric disorders
Wang, P., Barrett, F. S., Martin, E. A., Milonova, M., Gur, R. E., Gur, R. C., Kohler, C. G., & Verma, R. (2007) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2007.09.030 · Cited by 90
Self-citationRecognition of asymmetric facial action unit activities and intensities
Fasel, B., & Luettin, J. (2000) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2000.905664 · Cited by 84
Differences between children and adults in the recognition of enjoyment smiles.
Giudice, M. D., & Colle, L. (2007) · Developmental Psychology · doi:10.1037/0012-1649.43.3.796 · Cited by 68
Pain Reactivity and Somatization in Kindergarten-Age Children
Rocha, E. M. (2002) · Journal of Pediatric Psychology · doi:10.1093/jpepsy/28.1.47 · Cited by 66
Static posed and evoked facial expressions of emotions in schizophrenia
Kohler, C. G., Martin, E. A., Stolar, N., Barrett, F. S., Verma, R., Brensinger, C., Bilker, W., Gur, R. E., & Gur, R. C. (2008) · Schizophrenia Research · doi:10.1016/j.schres.2008.05.010 · Cited by 59
Self-citationDepression in schizophrenia: I. association with neuropsychological deficits
Kohler, C. G., Gur, R. C., Swanson, C. L., Petty, R. G., & Gur, R. E. (1998) · Biological Psychiatry · doi:10.1016/s0006-3223(97)00033-4 · Cited by 49
Self-citationComputerized measurement of facial expression of emotions in schizophrenia
Alvino, C. V., Kohler, C. G., Barrett, F. S., Gur, R. E., Gur, R. C., & Verma, R. (2007) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2007.03.002 · Cited by 44
Self-citationQuantification of facial expressions using high-dimensional shape transformations
Verma, R., Davatzikos, C. A., Loughead, J., Indersmitten, T., Hu, R., Kohler, C. G., Gur, R. E., & Gur, R. C. (2004) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2004.05.016 · Cited by 34
Self-citationSymptom and demographic profiles in first-episode schizophrenia
Gelber, E. I., Kohler, C. G., Bilker, W. B., Gur, R. C., Brensinger, C. M., Siegel, S. J., & Gur, R. E. (2003) · Schizophrenia Research · doi:10.1016/s0920-9964(03)00083-5 · Cited by 27
Self-citation
Cited by
Compound facial expressions of emotion
Du, S., Tao, Y., & Martinez, A. M. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1322355111 · Cited by 806
A Brief Review of Facial Emotion Recognition Based on Visual Information
Ko, B. (2018) · Sensors · doi:10.3390/s18020401 · Cited by 724
A systematic review on affective computing: emotion models, databases, and recent advances
Wang, Y., Song, W., Tao, W., Liotta, A., Yang, D., Li, X., Gao, S., Sun, Y., Ge, W., Zhang, W., et al. (2022) · Information Fusion · doi:10.1016/j.inffus.2022.03.009 · Cited by 615
AVEC 2013: the continuous audio/visual emotion and depression recognition challenge
Valstar, M., Schuller, B., Smith, K., Eyben, F., Jiang, B., Bilakhia, S., Schnieder, S., Cowie, R., & Pantic, M. (2013) · Proceedings of the 3rd ACM international workshop on Audio/visual emotion challenge · doi:10.1145/2512530.2512533 · Cited by 581
AVEC 2014: 3D Dimensional Affect and Depression Recognition Challenge
Valstar, M., Schuller, B., Smith, K., Almaev, T., Eyben, F., Krajewski, J., Cowie, R., & Pantic, M. (2014) · Proceedings of the 4th International Workshop on Audio/Visual Emotion Challenge · doi:10.1145/2661806.2661807 · Cited by 472
Metadata
- Authors
- Hamm, J., Kohler, C. G., Gur, R. C., & Verma, R.
- Venue
- Journal of Neuroscience Methods
- Publisher
- Elsevier BV
- Published
- 2011-09
- Volume
- 200
- Issue
- 2
- Pages
- 237-256
- ISSN
- 0165-0270 (ISSN-L)
- OpenAlex
- W2006217447
- Cited by
- 288
- References
- 68
- FWCI
- 15.02
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.