Perception of emotional expressions in different representations using facial feature points
Abstract
Facial expression recognition is an enabling technology for affective computing. Many existing facial expression analysis systems rely on automatically tracked facial feature points. Although psychologists have studied emotion perception from manually specified or marker-based point-light displays, no formal study exists on the amount of emotional information conveyed through automatically tracked feature points. We assess the utility of automatically extracted feature points in conveying emotions for posed and naturalistic data and present results from an experiment that compared human raters' judgements of emotional expressions between actual video clips and three automatically generated representations of them. The implications for optimal face representation and creation of realistic animations are discussed.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Afzal et al.'s experiments confirmed that human judgement is still the best, with the highest recognition rates, followed by stick-figure models, point-light displays, and then XFace animations.
“Their experiences confirmed that the human judgement is still the best one, with the highest recognition rates, followed by stick-figure models, point-light displays, and then XFace animations.”
From Survey on AI-Based Multimodal Methods for Emotion Detection (2019) · cites this paper as [48]
Afzal et al. performed studies on differences between human raters' judgments of emotional expressions and three automatically generated representations in video clips.
“Afzal et al. [48] performed studies differences between human raters' judgments of emotional expressions and three automatically generated representations in video clips.”
From Survey on AI-Based Multimodal Methods for Emotion Detection (2019) · cites this paper as [48]
The three automatically generated representations were point-based display, stick figure models, and 3D XFace Animation.
“First, the point-based display was created from the output of a commercial automatic face-tracker [49] on a black background.Second one, stick figure models, which connecting the automatically tracked feature-points using straight lines and sketching eyes using typical shape.The last one, 3D XFace Animation was a 3D animated facial expressions and displays created using XFace open source toolkit.”
From Survey on AI-Based Multimodal Methods for Emotion Detection (2019) · cites this paper as [48]
Cites
The expression of the emotions in man and animals.
Darwin, C. D. (1872) · J. Murray eBooks · doi:10.1037/10001-000 · Cited by 12,120 · see citing works
Measuring nominal scale agreement among many raters.
Fleiss, J. L. (1971) · Psychological Bulletin · doi:10.1037/h0031619 · Cited by 8,733 · see citing works
The Expression of the Emotions in Man and Animals
Darwin, C. (1956) · The Journal of Nervous and Mental Disease · doi:10.1097/00005053-195601000-00016 · Cited by 8,096 · see citing works
Visual perception of biological motion and a model for its analysis
Johansson, G. (1973) · Perception & Psychophysics · doi:10.3758/bf03212378 · Cited by 4,547 · see citing works
Baron-Cohen, S., & Wheelwright, S. (2004) · Journal of Autism and Developmental Disorders · doi:10.1023/b:jadd.0000022607.19833.00 · Cited by 4,093 · see citing works
A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions
Zeng, Z., Pantic, M., Roisman, G. I., & Huang, T. S. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.52 · Cited by 2,797 · see citing works
Comprehensive database for facial expression analysis
Takeo Kanade, A., Cohn, J. F., & Tian, Y. (2002) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/afgr.2000.840611 · Cited by 2,612 · see citing works
Automatic facial expression analysis: a survey
Fasel, B., & Luettin, J. (2003) · Pattern Recognition · doi:10.1016/s0031-3203(02)00052-3 · Cited by 1,810 · see citing works
(1972) · Elsevier eBooks · doi:10.1016/c2013-0-02458-9 · Cited by 1,455 · see citing works
Pakštas, A., Forchheimer, R., & Pandžić, I. S. (2002) · Wiley eBooks · doi:10.1002/0470854626 · Cited by 422 · see citing works
Facial motion in the perception of faces and of emotional expression.
Bassili, J. N. (1978) · Journal of Experimental Psychology Human Perception & Performance · doi:10.1037//0096-1523.4.3.373 · Cited by 369 · see citing works
Facial motion in the perception of faces and of emotional expression.
Bassili, J. N. (1978) · Journal of Experimental Psychology: Human Perception and Performance · doi:10.1037/0096-1523.4.3.373 · Cited by 369 · see citing works
MPEG-4 Facial Animation: The Standard,Implementation and Applications
Pakštas, A., Forchheimer, R., & Pandzic, I. S. (2002) · John Wiley & Sons, Inc. eBooks · Cited by 346 · see citing works
Mind Reading: The Interactive Guide to Emotions
Junek, W. (2007) · PubMed Central · Cited by 314 · see citing works
THE TIMING OF FACIAL MOTION IN POSED AND SPONTANEOUS SMILES
Cohn, J. F., & Schmidt, K. L. (2004) · International Journal of Wavelets Multiresolution and Information Processing · doi:10.1142/s021969130400041x · Cited by 301 · see citing works
Thomas, S. M., & Jordan, T. R. (2001) · Behavior Research Methods, Instruments, & Computers · doi:10.3758/bf03195347 · Cited by 10 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Survey on AI-Based Multimodal Methods for Emotion Detection
Marechal, C., Mikołajewski, D., Tyburek, K., Prokopowicz, P., Bougueroua, L., Ancourt, C., & Węgrzyn-Wolska, K. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-16272-6_11 · Cited by 137 · see citing works
Intelligent facial emotion recognition and semantic-based topic detection for a humanoid robot
Zhang, L., Jiang, M. X., Farid, D. M., & Hossain, M. A. (2013) · Expert Systems with Applications · doi:10.1016/j.eswa.2013.03.016 · Cited by 119 · see citing works
Miller, E. J., Foo, Y. Z., Mewton, P., & Dawel, A. (2023) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2023.100283 · Cited by 48 · see citing works
Graph-Based Facial Affect Analysis: A Review
Liu, Y. M., Zhang, X., Li, Y., Zhou, J., Li, X., & Zhao, G. (2022) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2022.3215918 · Cited by 40 · see citing works
High-Performance Modelling and Simulation for Big Data Applications
Kołodziej, J., & González–Vélez, H. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-16272-6 · Cited by 37 · see citing works
Communication and Automatic Interpretation of Affect from Facial Expressions
Salah, A. A., Sebe, N., & Gevers, T. (2011) · IGI Global eBooks · doi:10.4018/978-1-61692-892-6.ch008 · Cited by 30 · see citing works
Unimodal approaches for emotion recognition: A systematic review
Tomar, P. S., Mathur, K., & Suman, U. (2022) · Cognitive Systems Research · doi:10.1016/j.cogsys.2022.10.012 · Cited by 25 · see citing works
Loob, C., Rasti, P., Lüsi, I., Jacques, J. C. S., Baró, X., Escalera, S., Sapiński, T., Kamińska, D., & Anbarjafari, G. (2017) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2017.106 · Cited by 18 · see citing works
Intelligent Facial Action and emotion recognition for humanoid robots
Li, Z., Hossain, A., & Jiang, M. X. (2014) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2014.6889647 · Cited by 17 · see citing works
Constructive Learning for Human-Robot Interaction
Singh, A., Karanam, S., & Kumar, D. (2013) · IEEE Potentials · doi:10.1109/mpot.2012.2189443 · Cited by 14 · see citing works
Sensing Affective States Using Facial Expression Analysis
Samara, A., Galway, L., Bond, R., & Wang, H. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-48746-5_35 · Cited by 11 · see citing works
Synthesizing expressions using facial feature point tracking
Baltrušaitis, T., Riek, L. D., & Robinson, P. (2010) · International Workshop on Affective Interaction in Natural Environments (AFFINE) · doi:10.1145/1877826.1877835 · Cited by 9 · see citing works
Self-citationBarbulescu, A., Ronfard, R. P., Bailly, G., Gagneré, G., & Cakmak, H. (2014) · Motion in Games · doi:10.1145/2668064.2668084 · Cited by 7 · see citing works
Emotional Reactions to Point-Light Display Animations
Venesvirta, H., Surakka, V., Gizatdinova, Y., Lylykangas, J., Špakov, O., Verho, J. A., Vetek, A., & Lekkala, J. O. (2015) · Interacting with Computers · doi:10.1093/iwc/iwv028 · Cited by 4 · see citing works
Generating Facial Expression Data
Vaitonyté, J., Blomsma, P. A., Alimardani, M., & Louwerse, M. M. (2019) · ACM International Conference on Intelligent Virtual Agents (IVA) · doi:10.1145/3308532.3329443 · Cited by 4 · see citing works
Facial Feature Location Based on Improved ASM
Qin, X. W., Zhang, H. Y., & Dong, X. G. (2012) · Advanced materials research · doi:10.4028/www.scientific.net/amr.546-547.1398 · Cited by 1 · see citing works
Perception of Emotions Using Constructive Learningthrough Speech
Sruthi, K. S., Singh, A., Kumar, N. S., & Reddy, A. (2012) · International Journal of Computer and Electrical Engineering · doi:10.7763/ijcee.2012.v4.504 · Cited by 1 · see citing works
KARATE WITH CONSTRUCTIVE LEARNING
Karanam, S., Singh, A., & Kumar, D. (2012) · ICTACT Journal on Image and Video Processing · doi:10.21917/ijivp.2012.0054 · Cited by 1 · see citing works
Facial Expression Feature Selection Based on Rough Set
Li, D., Tian, Y., Wan, C., & Liu, S. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-34062-8_21 · Cited by 1 · see citing works
Douse, L. E. (2013) · University of Bedfordshire Repository (University of Bedfordshire) · Cited by 1 · see citing works
Afzal, S. (2021) · CL Technical Reports · doi:10.48456/tr-793
Self-citationHCIViewer - A Tool for Human-Computer Interaction Practitioners
Samara, A., Galway, L., Bond, R., & Wang, H. (2017) · Electronic workshops in computing · doi:10.14236/ewic/hci2017.23
Fabregat, M. B. (2017) · Tampere University Institutional Repository (Tampere University)
Affective and cognitive state modelling within human-computer interaction
Samara, A. (2018) · Ulster University Research Portal (Ulster University)
BEET: Behavior Engagement Emotion Trigger – A AI-based model to maximize audience engagement
Badawy, W. M. (2021) · Menoufia Journal of Electronic Engineering Research · doi:10.21608/mjeer.2021.193089
- Authors
- Afzal, S., Sezgin, T. M., Gao, Y., & Robinson, P.
- Affiliations
- University of Cambridge (GB)
- Koç University (TR)
- Venue
- 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops
- Publisher
- IEEE
- Published
- 2009-09
- Pages
- 1-6
- ISSN
- 2156-8103 (ISSN-L)
- OpenAlex
- W2114231266
- Cited by
- 25
- References
- 16
- Citations by year
- 2023: 1 · 2022: 2 · 2021: 2 · 2019: 3 · 2018: 1 · 2017: 3 · 2016: 1 · 2015: 1 · 2014: 2 · 2013: 3 · 2012: 4
- FWCI
- 3.46
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Conference on Affective Computing and Intelligent Interaction and workshops (publishedVersion)
- Funders
- Gates Cambridge Trust
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
- Life Sciences › Neuroscience › Cognitive Neuroscience › Face Recognition and Perception
- Keywords
- facial expression recognition, emotion perception, facial feature points, point-light displays, face representation, affective computing, facial animation, human raters
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review