The DOI matches its Crossref record; 32 of the 34 references it cites were found in a registry or an index.
Automatic Facial Expression Recognition Using Facial Animation Parameters and Multistream HMMs
Abstract
The performance of an automatic facial expression recognition system can be significantly improved by modeling the reliability of different streams of facial expression information utilizing multistream hidden Markov models (HMMs). In this paper, we present an automatic multistream HMM facial expression recognition system and analyze its performance. The proposed system utilizes facial animation parameters (FAPs), supported by the MPEG-4 standard, as features for facial expression classification. Specifically, the FAPs describing the movement of the outer-lip contours and eyebrows are used as observations. Experiments are first performed employing single-stream HMMs under several different scenarios, utilizing outer-lip and eyebrow FAPs individually and jointly. A multistream HMM approach is proposed for introducing facial expression and FAP group dependent stream reliability weights. The stream weights are determined based on the facial expression recognition results obtained when FAP streams are utilized individually. The proposed multistream HMM facial expression system, which utilizes stream reliability weights, achieves relative reduction of the facial expression recognition error of 44% compared to the single-stream HMM system.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Is is rarely used to parametrize FEs for recognition purposes [136], [137].”
From Survey on RGB, 3D, Thermal, and Multimodal Approaches for Facial Expression Recognition: History, Trends, and Affect-Related Applications (2016) · cites this paper as [137]
In the citing paper
“Most of these early works used geometric representations, like vectors for describing the motion of the face [134], active contours for describing the shape of the mouth and eyebrows [137], or deformable 2D mesh models [116].”
From Survey on RGB, 3D, Thermal, and Multimodal Approaches for Facial Expression Recognition: History, Trends, and Affect-Related Applications (2016) · cites this paper as [137]
Citations
Cites
Kass, M., Witkin, A., & Terzopoulos, D. (1988) · International Journal of Computer Vision · doi:10.1007/bf00133570 · Cited by 17,330
Fundamentals of speech recognition
Rabiner, L. R., & Juang, B.-H. F. (1993) · Cited by 7,680
Ekman, P., & Friesen, W. V. (2019) · PsycTESTS Dataset · doi:10.1037/t27734-000 · Cited by 6,019
Neural network-based face detection
Rowley, H. A., Baluja, S., & Kanade, T. (1998) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.655647 · Cited by 3,554
Comprehensive database for facial expression analysis
Takeo Kanade, A., Cohn, J. F., & Tian, Y. (2002) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/afgr.2000.840611 · Cited by 2,644
Automatic facial expression analysis: a survey
Fasel, B., & Luettin, J. (2003) · Pattern Recognition · doi:10.1016/s0031-3203(02)00052-3 · Cited by 1,810
Example-based learning for view-based human face detection
Sung, K.-K., & Poggio, T. (1998) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.655648 · Cited by 1,784
Automatic analysis of facial expressions: the state of the art
Pantie, M., & Rothkrantz, L. J. M. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.895976 · Cited by 1,778 · cited 1×
“Several automatic facial expression and facial action recognition systems [29]–[ 33 ] utilize FACS as facial features.”
MethodFeature extraction from faces using deformable templates
Yuille, A., Hallinan, P. W., & Cohen, D. S. (1992) · International Journal of Computer Vision · doi:10.1007/bf00127169 · Cited by 1,373
Donato, G., Bartlett, M. S., Hager, J. C., Ekman, P., & B, T. J. S. A. (1999) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.799905 · Cited by 999
Zhang, Z., Lyons, M., Schuster, M., & Akamatsu, S. (2002) · Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition · doi:10.1109/afgr.1998.670990 · Cited by 613
Bartlett, M. S., Littlewort, G. C., Fasel, I., & Movellan, J. R. (2003) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2003.10057 · Cited by 580
Pakštas, A., Forchheimer, R., & Pandžić, I. S. (2002) · Wiley eBooks · doi:10.1002/0470854626 · Cited by 422
Recognition of Facial Expression from Optical Flow
Mase, K. (1991) · IEICE Transactions on Information and Systems · Cited by 404
Audio-visual speech recognition
Neti, C., Potamianos, G., Luettin, J., Matthews, I. A., Glotin, H., Vergyri, D., Sison, J. A. G., & Mashari, A. (2000) · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · Cited by 272
A mew ASR approach based on independent processing and recombination of partial frequency bands
Bourlard, H., & Dupont, S. (2002) · International Conference on Spoken Language Processing (ICSLP) · doi:10.1109/icslp.1996.607145 · Cited by 250
EMPATH: Face, Emotion, and Gender Recognition Using Holons
Cottrell, G. W., & Metcalfe, J. (1990) · neural information processing systems · Cited by 231
Automated facial expression recognition based on FACS action units
Lien, J. J., Kanade, T., Cohn, J. F., & Li, C.-C. (2002) · Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition · doi:10.1109/afgr.1998.670980 · Cited by 226
Representing Face Images for Emotion Classification
Padgett, C. W., & Cottrell, G. W. (1996) · neural information processing systems · Cited by 171
LAFTER: a real-time face and lips tracker with facial expression recognition
Oliver, N., Pentland, A., & Bérard, F. (2000) · Pattern Recognition · doi:10.1016/s0031-3203(99)00113-2 · Cited by 137
Fully automatic upper facial action recognition
Kapoor, A., Qi, Y., & Picard, R. W. (2004) · IEEE International SOI Conference (SOI) · doi:10.1109/amfg.2003.1240843 · Cited by 135
Dynamics of Facial Expression Extracted Automatically from Video
Littlewort, G., Bartlett, M. S., Fasel, I., Susskind, J., & Movellan, J. (2005) · 2004 Conference on Computer Vision and Pattern Recognition Workshop · doi:10.1109/cvpr.2004.327 · Cited by 126
Facial animation parameters extraction and expression recognition using Hidden Markov Models
Pardàs, M., & Bonafonte, A. (2002) · Signal Processing: Image Communication · doi:10.1016/s0923-5965(02)00078-4 · Cited by 81
Emotion Recognition and Synthesis Based on MPEG‐4 FAPs
Tsapatsoulis, N., Raouzaiou, A. T., Kollias, S., Cowie, R., & Douglas‐Cowie, E. (2002) · doi:10.1002/0470854626.ch9 · Cited by 63
Real-time lip tracking and bimodal continuous speech recognition
Chan, M. T., Zhang, Y., & Huang, T. S. (2002) · IEEE Workshop on Multimedia Signal Processing (MMSP) · doi:10.1109/mmsp.1998.738914 · Cited by 60
Audio-Visual Speech Recognition Using MPEG-4 Compliant Visual Features
Aleksic, P. S., Williams, J. J., Wu, Z., & Katsaggelos, A. K. (2002) · EURASIP Journal on Advances in Signal Processing · doi:10.1155/s1110865702206162 · Cited by 57
Self-citationEmotion recognition based on MPEG-4 Facial Animation Parameters
Pardàs, M., Bonafonte, A., & Landabaso, J. L. (2002) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2002.5745440 · Cited by 34
Identifying Emotion in Static Face Images
Padgett, C. W., & Cottrell, G. W. (1995) · Cited by 22
A recombination strategy for multi-band speech recognition based on mutual information criterion
Okawa, S., Nakajima, T., & Shirai, K. (1999) · European Conference on Speech Communication and Technology (Eurospeech) · doi:10.21437/eurospeech.1999-155 · Cited by 17
Extracting Facial Motion Parameters by Tracking Feature Points
Otsuka, T., & Ohya, J. (1999) · Lecture notes in computer science · doi:10.1007/3-540-48962-2_30 · Cited by 16
Emotion recognition based on MPEG-4 facial animation parameters
Pardas, Bonafonte, A., & Landabaso (2002) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2002.1004701 · Cited by 14
HMM recognition of expressions in unrestrained video intervals
Landabaso, J. L., Pardis, M., & Bonafonte, A. (2003) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2003.1199141 · Cited by 7
Research works printed but not linked: 6
- cohn, 1997, automated face coding: a computer-vision based method of facial expression analysis, 7th Eur Conf Facial Expression Measurement and Meaning, 329
- aleksic, 2005, automatic facial expression recognition using facial animation parameters and multi-stream hmms, 6th E Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS 05)
- suwa, 1978, a preliminary note on pattern recognition of human emotional expression, Proc 4th Int Joint Conf on Pattern Recognition, 408
- senior, 1999, face and feature finding for a face recognition system, Proc Int Conf Audio and Video Based Biometric Person Authentication, 154
- aleksic, 2003, an audio-visual person identification and verification system using faps as visual features, Proc Works Multimedia User Authentication, 80
- zhao, 1996, classifying facial emotions by back propagation neural networks with fuzzy inputs, Proc Int Conf Neural Inform Process, 1, 454
Cited by
Sorted by citations
Dynamic Texture Recognition Using Local Binary Patterns with an Application to Facial Expressions
Zhao, G., & Pietikainen, M. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.1110 · Cited by 2,846
Corneanu, C. A., Simon, M. O., Cohn, J. F., & Guerrero, S. E. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2515606 · Cited by 527
Facial expression recognition using radial encoding of local Gabor features and classifier synthesis
Gu, W., Xiang, C., Venkatesh, Y. V., Huang, D., & Lin, H. T. (2011) · Pattern Recognition · doi:10.1016/j.patcog.2011.05.006 · Cited by 312
Face Expression Recognition and Analysis: The State of the Art
Bettadapura, V. (2012) · arXiv · doi:10.48550/arxiv.1203.6722 · Cited by 336
Facial Expression Recognition Using Facial Movement Features
Zhang, L., & Tjondronegoro, D. (2011) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2011.13 · Cited by 253
Metadata
- Authors
- Aleksic, P. S., & Katsaggelos, A. K.
- Affiliations
- Northwestern University (US)
- ORCID
- Katsaggelos, A.K.
- Venue
- IEEE Transactions on Information Forensics and Security
- Published
- 2006-03
- Volume
- 1
- Issue
- 1
- Pages
- 3-11
- ISSN
- 1556-6013 (ISSN-L)
- OpenAlex
- W2098044144
- Cited by
- 235
- References
- 34
- Citations by year
- 2026: 1 · 2025: 4 · 2024: 4 · 2023: 3 · 2022: 11 · 2021: 1 · 2020: 14 · 2019: 8 · 2018: 7 · 2017: 15 · 2016: 11 · 2015: 20 · 2014: 19 · 2013: 19 · 2012: 12
- FWCI
- 24.02
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Transactions on Information Forensics and Security (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face and Expression Recognition
- Keywords
- facial expression recognition, facial animation parameters, eyebrow movements, MPEG-4, error reduction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
