The DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
Facial emotion recognition using multi-modal information
Abstract
Facial emotion recognition will become vitally important in future multi-cultural visual communication systems, for emotion translation between cultures, which may be considered analogous to speech translation. However so far the recognition of facial emotions is mainly addressed by computer vision researchers, based on facial display. Also detection of vocal expressions of emotions can be found in research work done by acoustic researchers. Most of these research paradigms are devoted purely to visual or purely to auditory human emotion detection. However we found that it is very interesting to consider both these auditory and visual information together, for processing, since we hope this kind of multi-modal information processing will become a datum of information processing in future multimedia era. By several intensive subjective evaluation studies we found that human beings recognise anger, happiness, surprise and dislike by their visual appearance, compared to voice only detection. When the audio track of each emotion clip is dubbed with a different type of auditory emotional expression, still anger, happiness and surprise were video dominant. However the dislike emotion gave mixed responses to different speakers. In both studies we found that sadness and fear emotions were audio dominant. As a conclusion, we propose a method of facial emotion detection by using a hybrid approach, which uses multi-modal information for facial emotion recognition.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The method proposed by Cohn et al. deals neither with image sequences containing several facial actions in a row, nor with inaccurate facial data, nor with facial action intensity (yet the concept of the method makes it possible).”
From Automatic analysis of facial expressions: the state of the art (2000) · cites this paper as [10]
In the citing paper
“The accuracy of the classification was 92 percent for the eyebrow region, 88 percent for the eye region and 83 percent for the nose/ mouth region.”
From Automatic analysis of facial expressions: the state of the art (2000) · cites this paper as [10]
Citations
Cites
Russell, J. A. (1994) · Psychological Bulletin · doi:10.1037/0033-2909.115.1.102 · Cited by 1,640
Strong evidence for universals in facial expressions: A reply to Russell's mistaken critique.
Ekman, P. (1994) · Psychological Bulletin · doi:10.1037//0033-2909.115.2.268 · Cited by 1,130
Facial expression recognition using a dynamic model and motion energy
Essa, I. A., & Pentland, A. (2002) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.1995.466916 · Cited by 253
Coren, S., & Russell, J. A. (1992) · Cognition & Emotion · doi:10.1080/02699939208409690 · Cited by 51
Emotion Enhanced Face to Face Meetings Using the Concept of Virtual Space Teleconferencing
Silva, L. C. D., Miyasato, T., & Kishino, F. (1996) · IEICE Transactions on Information and Systems · Cited by 11
Self-citationFacial Expression Recognition from Image Sequence Using Hidden Markov Model
Sakaguchi, T., Ohya, J., & Kishino, F. (1995) · The Journal of The Institute of Image Information and Television Engineers · Cited by 11
Sakaguchi, T., Ohya, J., & Kishino, F. (1995) · The Journal of The Institute of Image Information and Television Engineers · doi:10.3169/itej1978.49.1060 · Cited by 8
Emotion enhanced multimedia meetings using the concept of virtual space teleconferencing
Silva, L. C. D., Miyasato, T., & Kishino, F. (2002) · IEEE International Conference on Multimedia Computing and Systems (ICMCS) · doi:10.1109/mmcs.1996.534951 · Cited by 6
Self-citationDegree of Human Perception of Facial Emotions Based on Audio and Video Information
Silva, L. C. D., & Miyasato, T. (1996) · 電子情報通信学会技術研究報告. IE, 画像工学 · Cited by 2
Self-citation
Research works printed but not linked: 6
- fridlund, 1994, Human facial expression An evalutionary view
- ekman, 1976, Pictures of Facial Effect
- 10.1037/0033-2909.115.2.268 · DOI · Paper page
- mase, 1991, Recognition of facial expressions from optiacl flow, IEICE Transactions, e74, 3474
- kobayashi, 1995, Monitoring of facial expressions, JSICE Journal of the Society of Instrument and Control Eng s, 34, 248
- yacoob, 1994, Recognizing human facial expression
Cited by
Sorted by citations
IEMOCAP: Interactive emotional dyadic motion capture database
Busso, C., Bulut, M., Lee, C.-C., Kazemzadeh, A., Mower, E., Kim, S., Chang, J. N., Lee, S., & Narayanan, S. S. (2008) · Language Resources and Evaluation · doi:10.1007/s10579-008-9076-6 · Cited by 3,869
Toward machine emotional intelligence: analysis of affective physiological state
Picard, R. W., Vyzas, E., & Healey, J. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.954607 · Cited by 2,334
Survey on speech emotion recognition: Features, classification schemes, and databases
Ayadi, M. M. H. E., Kamel, M. S., & Karray, F. (2010) · Pattern Recognition · doi:10.1016/j.patcog.2010.09.020 · Cited by 2,139
Automatic analysis of facial expressions: the state of the art
Pantie, M., & Rothkrantz, L. J. M. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.895976 · Cited by 1,778
Efficient Low-rank Multimodal Fusion With Modality-Specific Factors
Liu, Z., Shen, Y., Lakshminarasimhan, V. B., Liang, P. P., Bagher Zadeh, A., & Morency, L.-P. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1209 · Cited by 1,170
Metadata
- Authors
- De Silva, L. C., Miyasato, T., & Nakatsu, R.
- Affiliations
- Advanced Telecommunications Research Institute International (JP)
- ORCID
- Nakatsu, R.
- Venue
- Proceedings of ICICS, 1997 International Conference on Information, Communications and Signal Processing. Theme: Trends in Information Systems Engineering and Wireless Multimedia Communications (Cat. No.97TH8237)
- Publisher
- IEEE
- Published
- 2002-11-22
- Volume
- 1
- Pages
- 397-401
- OpenAlex
- W2154739180
- Cited by
- 319
- References
- 9
- Citations by year
- 2026: 2 · 2025: 10 · 2024: 16 · 2023: 16 · 2022: 39 · 2021: 16 · 2020: 18 · 2019: 11 · 2018: 25 · 2017: 11 · 2016: 18 · 2015: 15 · 2014: 8 · 2013: 8 · 2012: 12
- FWCI
- 13.69
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Conference on Information, Communications & Signal Processing (ICICS) (publishedVersion)
- National University of Singapore (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
- Keywords
- facial emotion recognition, multimodal emotion recognition, vocal emotion expression
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
