The DOI matches its Crossref record; 32 of the 34 references it cites were found in a registry or an index.
Automatic Facial Expression Recognition Using Facial Animation Parameters and Multistream HMMs
Abstract
The performance of an automatic facial expression recognition system can be significantly improved by modeling the reliability of different streams of facial expression information utilizing multistream hidden Markov models (HMMs). In this paper, we present an automatic multistream HMM facial expression recognition system and analyze its performance. The proposed system utilizes facial animation parameters (FAPs), supported by the MPEG-4 standard, as features for facial expression classification. Specifically, the FAPs describing the movement of the outer-lip contours and eyebrows are used as observations. Experiments are first performed employing single-stream HMMs under several different scenarios, utilizing outer-lip and eyebrow FAPs individually and jointly. A multistream HMM approach is proposed for introducing facial expression and FAP group dependent stream reliability weights. The stream weights are determined based on the facial expression recognition results obtained when FAP streams are utilized individually. The proposed multistream HMM facial expression system, which utilizes stream reliability weights, achieves relative reduction of the facial expression recognition error of 44% compared to the single-stream HMM system.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Is is rarely used to parametrize FEs for recognition purposes [136], [137].”
From Survey on RGB, 3D, Thermal, and Multimodal Approaches for Facial Expression Recognition: History, Trends, and Affect-Related Applications (2016) · cites this paper as [137]
In the citing paper
“Most of these early works used geometric representations, like vectors for describing the motion of the face [134], active contours for describing the shape of the mouth and eyebrows [137], or deformable 2D mesh models [116].”
From Survey on RGB, 3D, Thermal, and Multimodal Approaches for Facial Expression Recognition: History, Trends, and Affect-Related Applications (2016) · cites this paper as [137]
Citations
Cites
Kass, M., Witkin, A., & Terzopoulos, D. (1988) · International Journal of Computer Vision · doi:10.1007/bf00133570 · Cited by 17,330
Fundamentals of speech recognition
Rabiner, L. R., & Juang, B.-H. F. (1993) · Cited by 7,680
Ekman, P., & Friesen, W. V. (2019) · PsycTESTS Dataset · doi:10.1037/t27734-000 · Cited by 6,019
Neural network-based face detection
Rowley, H. A., Baluja, S., & Kanade, T. (1998) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.655647 · Cited by 3,554
Comprehensive database for facial expression analysis
Takeo Kanade, A., Cohn, J. F., & Tian, Y. (2002) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/afgr.2000.840611 · Cited by 2,644
Automatic facial expression analysis: a survey
Fasel, B., & Luettin, J. (2003) · Pattern Recognition · doi:10.1016/s0031-3203(02)00052-3 · Cited by 1,810
Example-based learning for view-based human face detection
Sung, K.-K., & Poggio, T. (1998) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.655648 · Cited by 1,784
Automatic analysis of facial expressions: the state of the art
Pantie, M., & Rothkrantz, L. J. M. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.895976 · Cited by 1,778 · cited 1×
“Several automatic facial expression and facial action recognition systems [29]–[ 33 ] utilize FACS as facial features.”
MethodFeature extraction from faces using deformable templates
Yuille, A., Hallinan, P. W., & Cohen, D. S. (1992) · International Journal of Computer Vision · doi:10.1007/bf00127169 · Cited by 1,373
Donato, G., Bartlett, M. S., Hager, J. C., Ekman, P., & B, T. J. S. A. (1999) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.799905 · Cited by 999
Research works printed but not linked: 6
- cohn, 1997, automated face coding: a computer-vision based method of facial expression analysis, 7th Eur Conf Facial Expression Measurement and Meaning, 329
- aleksic, 2005, automatic facial expression recognition using facial animation parameters and multi-stream hmms, 6th E Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS 05)
- suwa, 1978, a preliminary note on pattern recognition of human emotional expression, Proc 4th Int Joint Conf on Pattern Recognition, 408
- senior, 1999, face and feature finding for a face recognition system, Proc Int Conf Audio and Video Based Biometric Person Authentication, 154
- aleksic, 2003, an audio-visual person identification and verification system using faps as visual features, Proc Works Multimedia User Authentication, 80
- zhao, 1996, classifying facial emotions by back propagation neural networks with fuzzy inputs, Proc Int Conf Neural Inform Process, 1, 454
Cited by
Sorted by citations
Items 201–234 of 234
Motion Detection Applied To Microtectonics Modellig
Guerra, M. A. J., & Estrela, V. V. (2014) · International Journal on Computational Science & Applications · doi:10.5121/ijcsa.2014.4604 · Cited by 1
A local feature-based facial expression recognition system from depth video
Uddin, M. Z. (2014) · Elsevier eBooks · doi:10.1016/b978-0-12-802045-6.00026-0 · Cited by 1
Neto, A. F., Silva, A. P., & Yehia, H. C. (2018) · Revista de Estudos da Linguagem · doi:10.17851/2237-2083.27.1.191-212 · Cited by 1
Face Image Processing and Analysis
Shih, F. Y. (2010) · doi:10.1002/9780470590416.ch10 · Cited by 1
A Subject‐Dependent Facial Expression Recognition System
Chang, C., & Huang, Y. (2015) · Wiley eBooks · doi:10.1002/9781118910566.ch4 · Cited by 1
Face Analysis Using Image Sequences
Pietikäinen, M., Hadid, A., Zhao, G., & Ahonen, T. (2011) · Computational imaging and vision · doi:10.1007/978-0-85729-748-8_11 · Cited by 1
Detection of Genuine and Posed Facial Expressions of Emotion: A ReviewPreprint
Jia, S., Wang, S., Hu, C., Webster, P. J., & Li, X. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.11353 · Cited by 1
Lang, L., & Hu, Z. (2011) · Journal of Signal and Information Processing · doi:10.4236/jsip.2011.24038
A FACE RECOGNITION SCHEME BASED ON EMBEDDED HIDDEN MARKOV MODEL AND SELECTIVE ENSEMBLE STRATEGY
Gao, X., Jin-xiu, L., & Xiao, B. (2009) · International Journal of Image and Graphics · doi:10.1142/s0219467809003472
Facial Expression Generated Automatically Using Triangular Segmentation
Sheu, J., Hsieh, T. S., & Shou, H.-N. (2013) · Applied Mechanics and Materials · doi:10.4028/www.scientific.net/amm.479-480.834
Facial Action Units Recognition: A Comparative Study
Popa, M. C., Rothkrantz, L. J. M., Wiggers, P., Braspenning, R. A. C., & Shan, C. (2011) · Data Archiving and Networked Services (DANS)
Face animation parameter extraction in real movie capturing environment
Ashourian, M., Hosseini, M., Mousavi, S., & Monadjemi, A. (2009) · International Conference on Innovative Technologies in Intelligent Systems and Industrial Applications (CITISIA) · doi:10.1109/citisia.2009.5224213
Interface Framework toDriveanIntelligent WheelchairUsing FacialExpressions
Faria, P., Valgôde, E., Reis, L. P., & Frias, R. (2007)
Video-based facial expression analysis
Li, Z. (2010) · Rutgers University Community Repository (Rutgers University) · doi:10.7282/t3bc3z9r
Semi Random Patches Sampling Based on Spatio-temporal Information for Facial Expression Recognition
Xia, H. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-23989-7_5
Visual Speech and Gesture Coding Using the MPEG-4 Face and Body Animation Standard
Petajan, E. D. (2009) · IGI Global eBooks · doi:10.4018/978-1-60566-186-5.ch004
Abounasr, N., & Pourghassem, H. (2013) · Iranian Conference on Machine Vision and Image Processing · doi:10.1109/iranianmvip.2013.6780027
Facial expression recognition based on compressive sensing and pyramid processing
Eleyan, A., & Ashir, A. M. (2017) · Pamukkale University Journal of Engineering Sciences · doi:10.5505/pajes.2016.43402
HAND POSTURE RECOGNITION FOR COMPLEX DECISION MAKING
Butalia, A., Ingale, M., Patil, S., Bangali, M., & Dusane, B. (2012) · CiiT international journal of data mining and knowledge engineering
Modelling facial action units using partial differential equations
Ismail, N. B. (2015) · Bradford Scholars (University of Bradford)
Facial expression recognition based on selective feature extraction and rule matching
Zhou, G., Zhan, Y., & Zhang, J. (2006) · Image Processing & Communications
Facial Expression Recognition on The Classroom Environments
Setiawan, W., Wihardi, Y., Junateti, E., & Faikar, N. (2020) · Proceedings of the Proceedings of the 7th Mathematics, Science, and Computer Science Education International Seminar, MSCEIS 2019, 12 October 2019, Bandung, West Java, Indonesia · doi:10.4108/eai.12-10-2019.2296519
Facial Expression Recognition System Using a Neural Network and Simulation on FPGA
Zarbakhsh, P., Golilarz, N. A., & Ozgunalp, U. (2020) · IEEE International Conference on Signal and Image Processing (ICSIP) · doi:10.1109/icsip49896.2020.9339442
Real Time Emotion Analysis Usingbrainwavesin Comparison to Facial Expression
Devi, S. K., Rajalakshmi, D. M., Saranya, S., Jeevanandan, B., & Ramya, A. (2020) · IOP Conference Series Materials Science and Engineering · doi:10.1088/1757-899x/994/1/012039
Jointly Optimizing Expressional and Residual Models for 3D Facial Expression Removal
Zheng, Q., Wang, Y., Hu, Z., Zhang, X., Wu, Z., & Pan, G. (2022) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3533312
Facial Sentimental Analysis in Hospitals
Kamala, M. M. (2024) · International Journal for Research in Applied Science and Engineering Technology · doi:10.22214/ijraset.2024.59189
Optimizing Viola-Jones for Advanced Face Detection:A Comprehensive Study
KUMAR, U. S. (2024) · IJARCCE · doi:10.17148/ijarcce.2024.13446
Sustainable multimodal human assessment and emotion recognition system
Lalitha, Y. S., Gayatri, Y., Inti, L. B., Sheetal, S. R., Shirisha, P., Shnain, A. H., Joshi, R., & Dutt, A. (2025) · AIP conference proceedings · doi:10.1063/5.0266409
Zhang, D., Ma, Q., Shen, Z., Ma, W., Yan, Y., & Kong, S. (2026) · Multimedia Systems · doi:10.1007/s00530-026-02640-z
Recognition of human activities and expressions in video sequences using shape context descriptor
Kholgade, N. (2009) · RIT Scholar Works (Rochester Institute of Technology)
Emotional Behavior Decision Model Based on Linear Dynamic System for Intelligent Service Robots
Ahn, H.-S. (2007) · Journal of Control Automation and Systems Engineering · doi:10.5302/j.icros.2007.13.8.760
A Deep Learning Perspective on the Origin of Facial Expressions
Breuer, R., & Kimmel, R. (2017) · arXiv · doi:10.48550/arxiv.1705.01842 · Cited by 92
Metadata
- Authors
- Aleksic, P. S., & Katsaggelos, A. K.
- Affiliations
- Northwestern University (US)
- ORCID
- Katsaggelos, A.K.
- Venue
- IEEE Transactions on Information Forensics and Security
- Published
- 2006-03
- Volume
- 1
- Issue
- 1
- Pages
- 3-11
- ISSN
- 1556-6013 (ISSN-L)
- OpenAlex
- W2098044144
- Cited by
- 235
- References
- 34
- Citations by year
- 2026: 1 · 2025: 4 · 2024: 4 · 2023: 3 · 2022: 11 · 2021: 1 · 2020: 14 · 2019: 8 · 2018: 7 · 2017: 15 · 2016: 11 · 2015: 20 · 2014: 19 · 2013: 19 · 2012: 12
- FWCI
- 24.02
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Transactions on Information Forensics and Security (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face and Expression Recognition
- Keywords
- facial expression recognition, facial animation parameters, eyebrow movements, MPEG-4, error reduction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review