An expert system for multiple emotional classification of facial expressions
Abstract
This paper discusses the Integrated System for Facial Expression Recognition (ISFER), which performs facial expression analysis from a still dual facial view image. The system consists of three major parts: a facial data generator, a facial data evaluator and a facial data analyser. While the facial data generator applies fairly conventional techniques for facial feature extraction, the rest of the system represents a novel way of performing a reliable identification of 30 different face actions and a multiple classification of expressions into the six basic emotion categories. An expert system has been utilised to convert low level face geometry into high level face actions, and then this into highest level weighted emotion labels. The system evaluation results demonstrated rather high concurrent validity with human coding of facial expressions using FACS and formal instructions in emotion signals.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Wang et al.'s method assumes the initial location of the FFPs in the first frame of an input image sequence is known.
“The initial location of the FFPs in the first frame of an input image sequence is assumed to be known.”
From Automatic analysis of facial expressions: the state of the art (2000) · cites this paper as [95]
The Wang et al. method achieved an average recognition rate of 95 percent.
“The average recognition rate was 95 percent.”
From Automatic analysis of facial expressions: the state of the art (2000) · cites this paper as [95]
The Wang et al. method had an average process time of 2.5s/frame.
“The average process time was 2.5s/frame.”
From Automatic analysis of facial expressions: the state of the art (2000) · cites this paper as [95]
Cites
Kass, M., Witkin, A., & Terzopoulos, D. (1988) · International Journal of Computer Vision · doi:10.1007/bf00133570 · Cited by 17,165 · see citing works
Russell, J. A. (1994) · Psychological Bulletin · doi:10.1037/0033-2909.115.1.102 · Cited by 1,640 · see citing works
(1972) · Elsevier eBooks · doi:10.1016/c2013-0-02458-9 · Cited by 1,455 · see citing works
Strong evidence for universals in facial expressions: A reply to Russell's mistaken critique.
Ekman, P. (1994) · Psychological Bulletin · doi:10.1037//0033-2909.115.2.268 · Cited by 1,130 · see citing works
A Fast algorithm for active contours and curvature estimation
Williams, D. J., & Shah, M. (1992) · CVGIP: Image Understanding · doi:10.1016/1049-9660(92)90003-l · Cited by 1,101 · see citing works
Coding, analysis, interpretation, and recognition of facial expressions
Essa, I. A., & Pentland, A. (1997) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.598232 · Cited by 906 · see citing works
Zhang, Z., Lyons, M., Schuster, M., & Akamatsu, S. (2002) · Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition · doi:10.1109/afgr.1998.670990 · Cited by 613 · see citing works
Recognizing Facial Expressions in Image Sequences Using Local Parameterized Models of Image Motion
Black, M. J., & Yacoob, Y. (1997) · International Journal of Computer Vision · doi:10.1023/a:1007977618277 · Cited by 514 · see citing works
Facial Action Coding System (FACS)
Ekman, P. (2002) · Medical Entomology and Zoology · Cited by 394 · see citing works
Facial motion in the perception of faces and of emotional expression.
Bassili, J. N. (1978) · Journal of Experimental Psychology: Human Perception and Performance · doi:10.1037/0096-1523.4.3.373 · Cited by 369 · see citing works
Facial motion in the perception of faces and of emotional expression.
Bassili, J. N. (1978) · Journal of Experimental Psychology Human Perception & Performance · doi:10.1037//0096-1523.4.3.373 · Cited by 369 · see citing works
Feature-point tracking by optical flow discriminates subtle differences in facial expression
Cohn, J. F., Zlochower, A. J., Lien, J.-J. J., & Takeo Kanade, A. (2002) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/afgr.1998.670981 · Cited by 181 · see citing works
Machine identification of human faces
Harmon, L. D., Khan, M. K., Lasch, R., & Ramig, P. F. (1981) · Pattern Recognition · doi:10.1016/0031-3203(81)90008-x · Cited by 146 · see citing works
Facial interaction between animated 3D face robot and human beings
Kobayashi, H., & Hara, F. (2002) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/icsmc.1997.633250 · Cited by 113 · see citing works
Online facial expression recognition based on personalized galleries
Hong, H., Neven, H., & von der Malsburg, C. (2002) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/afgr.1998.670974 · Cited by 93 · see citing works
Spotting segments displaying facial expression from image sequences using HMM
Otsuka, T., & Ohya, J. (2002) · Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition · doi:10.1109/afgr.1998.670988 · Cited by 69 · see citing works
Thalmann, N. M., Kalra, P. K., & Escher, M. (1998) · Proceedings of the IEEE · doi:10.1109/5.664277 · Cited by 57 · see citing works
Kearney, G. D., & McKenzie, S. (1993) · Cognitive Science · doi:10.1207/s15516709cog1704_5 · Cited by 36 · see citing works
Kearney, G. D., & McKenzie, S. (1993) · Cognitive Science · doi:10.1016/0364-0213(93)90005-s · Cited by 36 · see citing works
Image Feature Location in Multi-Resolution Images Using a Hierarchy of Multilayer Perceptrons
Vincent, J. M., Myers, D. J., & Hutchinson, R. (1992) · Springer eBooks · doi:10.1007/978-94-011-2360-0_3 · Cited by 17 · see citing works
Ekman: Emotion in the Human Face
Sigman, S. J. (1983) · ScholarlyCommons (University of Pennsylvania) · Cited by 6 · see citing works
Machine Interpretation of Emotion
Keamey, G. D., & McKenzie, S. (1993) · Cited by 3 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Automatic analysis of facial expressions: the state of the art
Pantie, M., & Rothkrantz, L. J. M. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.895976 · Cited by 1,778 · see citing works
Self-citation“How do you know that I don’t understand?” A look at the future of intelligent tutoring systems
Sarrafzadeh, A., Alexander, S., Dadgostar, F., Fan, C., & Bigdeli, A. (2007) · Computers in Human Behavior · doi:10.1016/j.chb.2007.07.008 · Cited by 143 · see citing works
Classification of Facial Expressions Using K-Nearest Neighbor Classifier
Sohail, A. S. M., & Bhattacharya, P. (2007) · Springer eBooks · doi:10.1007/978-3-540-71457-6_51 · Cited by 57 · see citing works
Learning partially-observed hidden conditional random fields for facial expression recognition
Chang, K.-Y., Liu, T.-L., & Lai, S. (2009) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2009.5206612 · Cited by 53 · see citing works
Review of automated emotion-based quantification of facial expression in Parkinson’s patients
Sonawane, B., & Sharma, P. (2020) · The Visual Computer · doi:10.1007/s00371-020-01859-9 · Cited by 50 · see citing works
Pao, T.-L., Chen, Y.-T., Yeh, J.-H., Cheng, Y.-M., & Lin, Y.-Y. (2007) · Springer eBooks · doi:10.1007/978-3-540-74171-8_101 · Cited by 31 · see citing works
Learning finite Beta-Liouville mixture models via variational bayes for proportional data clustering
Fan, W., & Bouguila, N. (2013) · International Joint Conference on Artificial Intelligence · Cited by 31 · see citing works
Wavelets-based facial expression recognition using a bank of support vector machines
Kazmi, S. B., Qurat‐ul‐Ain, & Jaffar, M. A. (2011) · Soft Computing · doi:10.1007/s00500-011-0721-4 · Cited by 28 · see citing works
(2000) · Cited by 27 · see citing works
Self-citationShergill, G., Sarrafzadeh, A., Diegel, O., & Shekar, A. (2008) · Tuwhera (Auckland University of Technology) · Cited by 23 · see citing works
Wavelets Based Facial Expression Recognition Using a Bank of Neural Networks
Kazmi, S. B., Ain, Q., & Jaffar, M. A. (2010) · International Conference on Future Information Technology · doi:10.1109/futuretech.2010.5482717 · Cited by 18 · see citing works
Facial Expression Recognition Using Artificial Neural Networks
Deepthi, S. (2013) · IOSR Journal of Computer Engineering · doi:10.9790/0661-0840106 · Cited by 18 · see citing works
Facial recognition system for driver vigilance mlonitoring
Dikkers, H. J., Spaans, M. A., Daten, D., Novak, M., & Rothkrantz, L. M. (2005) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/icsmc.2004.1400934 · Cited by 17 · see citing works
Self-citationFacial expression recognition using temporal POEM features
Cruz, E. A. S., Jung, C. R., & Franco, C. H. E. (2017) · Pattern Recognition Letters · doi:10.1016/j.patrec.2017.08.008 · Cited by 17 · see citing works
Facial Expression as an Implicit Customers' Feedback and the Challenges
Kasiran, Z., & Yahya, S. (2007) · International Conference on Computer Graphics, Imaging and Visualization (CGIV) · doi:10.1109/cgiv.2007.40 · Cited by 16 · see citing works
Customer Satisfaction Recognition Based on Facial Expression and Machine Learning Techniques
Bouzakraoui, M. S., Sadiq, A., & Alaoui, A. Y. (2020) · Advances in Science Technology and Engineering Systems Journal · doi:10.25046/aj050470 · Cited by 14 · see citing works
Extracting subtle facial expression for emotional analysis
Cowell, J., & Ayesh, A. (2005) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/icsmc.2004.1398378 · Cited by 14 · see citing works
Hierarchical multistream recognition of facial expressions
Chibelushi, C. C., & Bourel, F. (2004) · IEE Proceedings - Vision Image and Signal Processing · doi:10.1049/ip-vis:20040748 · Cited by 12 · see citing works
Emotional analysis of facial expressions
Ayesh, A., & Cowell, J. (2005) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/icsmc.2004.1398377 · Cited by 11 · see citing works
Classifying facial expressions using point-based analytic face model and Support Vector Machines
Sohail, A. S. M., & Bhattacharya, P. (2007) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/icsmc.2007.4413713 · Cited by 10 · see citing works
Schacter, D., Wang, C., Nejat, G., & Benhabib, B. (2013) · Advanced Robotics · doi:10.1080/01691864.2013.755278 · Cited by 9 · see citing works
Facial Feature Model for Emotion Recognition Using Fuzzy Reasoning
Contreras, R., Starostenko, O., Alarcon‐Aquino, V., & Flores-Pulido, L. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15992-3_2 · Cited by 9 · see citing works
Facial Emotion Recognition with a Neural Network Approach
Widanagamaachchi, W. (2013) · Cited by 9 · see citing works
Researching affective computing techniques for intelligent tutoring systems
Ivanova, M. (2013) · International Conference on Interactive Collaborative Learning · doi:10.1109/icl.2013.6644661 · Cited by 8 · see citing works
Fusion of face recognition and facial expression detection for authentication: a proposed model
Yin, D. B. M., Omar, S., Talip, B. A., Muklas, A., Norain, N. ‘. M., & Othman, A. T. (2017) · International Conference on Ubiquitous Information Management and Communication · doi:10.1145/3022227.3022247 · Cited by 11 · see citing works
Facial Expression Recognition as an Implicit Customers' Feedback
Kasiran, Z., Yahya, S., & Ibrahim, Z. (2008) · InTech eBooks · doi:10.5772/5930 · Cited by 7 · see citing works
Computationally Efficient Invariant Facial Expression Recognition
Hussain, M. H., Khan, S. A., Ullah, F., Riaz, N., Nazir, M., Zulfikar, S., & Bhutto, A. (2014) · Cited by 7 · see citing works
Facial expression analysis in e-learning systems - the problems and feasibility
Loh, M. P., Wong, Y.-P., & Wong, C.-O. (2005) · IEEE International Conference on Advanced Learning Technologies (ICALT) · doi:10.1109/icalt.2005.150 · Cited by 5 · see citing works
Automated Real Time Emotion Recognition Using Facial Expression Analysis
Fratesi, A. (2016) · doi:10.22215/etd/2016-11318 · Cited by 5 · see citing works
Texture analysis based facial expression recognition using a bank of Bayesian classifiers
Kazmi, S. B., Qurat‐ul‐Ain, Ishtiaq, M., & Jaffar, M. A. (2010) · International Conference on Information and Emerging Technologies · doi:10.1109/iciet.2010.5625676 · Cited by 4 · see citing works
Surendran, N., & Xie, S. Q. (2009) · International Journal of Intelligent Systems Technologies and Applications · doi:10.1504/ijista.2009.027112 · Cited by 4 · see citing works
Facial expression identification using 3D geometric features from Microsoft Kinect device
Han, D., Al‐Jawad, N., & Du, H. (2016) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.2223029 · Cited by 4 · see citing works
Improved Two-Phase Framework for Facial Emotion Recognition
Yoon, H., Park, S., Lee, Y., Han, M., & Jang, J. (2015) · ETRI Journal · doi:10.4218/etrij.15.0114.0523 · Cited by 3 · see citing works
Emotion Tracking for Remote Conferencing Applications using Neural Networks.
Redfern, S., & Smith, P. A. (2010) · Cited by 3 · see citing works
Poszukiwanie twarzy i rozpoznawanie emocji w scenach zmiennych
Wosiak, S., Buda, K., & Wiliński, A. (2011) · Metody Informatyki Stosowanej · Cited by 2 · see citing works
E-Learning with Affective Tutoring Systems
Sarrafzadeh, A., Shanbehzadeh, J., & Overmyer, S. P. (2010) · IGI Global eBooks · doi:10.4018/978-1-61692-008-1.ch007 · Cited by 2 · see citing works
Human facial expression mining based on cloud model
Wang, S., Chi, H., Feng, X., & Yin, J. (2009) · IEEE International Conference on Granular Computing (GrC) · doi:10.1109/grc.2009.5255059 · Cited by 2 · see citing works
A New Motion Based Fully Automatic Facial Expression Recognition System
Saha, C., Ahmed, W., & Mitra, S. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-27387-2_19 · Cited by 1 · see citing works
Activity recognition with Hidden Markov models using active learning
Alemdar, H. O., van Kasteren, T. L. M., & Ersoy, C. (2011) · Signal Processing and Communications Applications Conference (SIU) · doi:10.1109/siu.2011.5929862 · Cited by 1 · see citing works
Enhancing Emotion Recognition in Facial Images with Deep Learning
Pawar, M., Bhandari, P., Bhandari, M., Kanherkar, R., & Kale, S. (2024) · International Conference for Emerging Technology (INCET) · doi:10.1109/incet61516.2024.10593490 · Cited by 1 · see citing works
Deep Learning Model to Analyze Customer’s Satisfaction
Bouzakraoui, M. S., Abdelalim, S., & Abdessamad, Y. A. (2020) · International Journal of Engineering and Advanced Technology · doi:10.35940/ijeat.c6610.049420 · Cited by 1 · see citing works
Smith, P., & Redfern, S. (2011) · ARAN (University of Galway Research Repository) (Ollscoil na Gaillimhe – University of Galway) · doi:10.13025/18844
Classification of facial expressions by sparse coding
Erdoğmuş, N., & Dugelay, J. (2011) · Signal Processing and Communications Applications Conference (SIU) · doi:10.1109/siu.2011.5929861
An Implementation of Face Emotion Identification System using Active Contour Model and PCA
Rathore, N., & Raman, C. V. (2015) · IJSRD : international journal for scientific research and development
Data-Driven Facial Expression Analysis from Live Video
Tay, W. K. (2017) · doi:10.26686/wgtn.17064395
Content Based Retrieval of Emotions in Face Images.
Nileema, J., Chandra, P., Bhagvati, C., Pujari, A. K., & Deekshatulu, B. L. (2004) · Indian Conference on Computer Vision, Graphics and Image Processing
- Authors
- Pantic, M., & Rothkrantz, L. J. M.
- Affiliations
- Delft University of Technology (NL)
- ORCID
- Rothkrantz, L.J.M.
- Venue
- Proceedings 11th International Conference on Tools with Artificial Intelligence
- Publisher
- IEEE Comput. Soc
- Published
- 2003-01-20
- Pages
- 113-120
- ISSN
- 1082-3409 (ISSN-L)
- OpenAlex
- W1922666161
- Cited by
- 48
- References
- 20
- Citations by year
- 2024: 1 · 2020: 3 · 2017: 4 · 2016: 2 · 2015: 2 · 2014: 1 · 2013: 5 · 2012: 1
- FWCI
- 5.50
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE International Conference on Tools with Artificial Intelligence (ICTAI) (publishedVersion)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face and Expression Recognition
- Keywords
- facial expression recognition, emotion classification, expert systems, facial action units, basic emotions, facial feature extraction, FACS, concurrent validity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
