Facial emotion recognition using multi-modal information
Abstract
Facial emotion recognition will become vitally important in future multi-cultural visual communication systems, for emotion translation between cultures, which may be considered analogous to speech translation. However so far the recognition of facial emotions is mainly addressed by computer vision researchers, based on facial display. Also detection of vocal expressions of emotions can be found in research work done by acoustic researchers. Most of these research paradigms are devoted purely to visual or purely to auditory human emotion detection. However we found that it is very interesting to consider both these auditory and visual information together, for processing, since we hope this kind of multi-modal information processing will become a datum of information processing in future multimedia era. By several intensive subjective evaluation studies we found that human beings recognise anger, happiness, surprise and dislike by their visual appearance, compared to voice only detection. When the audio track of each emotion clip is dubbed with a different type of auditory emotional expression, still anger, happiness and surprise were video dominant. However the dislike emotion gave mixed responses to different speakers. In both studies we found that sadness and fear emotions were audio dominant. As a conclusion, we propose a method of facial emotion detection by using a hybrid approach, which uses multi-modal information for facial emotion recognition.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The method proposed by Cohn et al. deals neither with image sequences containing several facial actions in a row, nor with inaccurate facial data, nor with facial action intensity.
“The method proposed by Cohn et al. deals neither with image sequences containing several facial actions in a row, nor with inaccurate facial data, nor with facial action intensity (yet the concept of the method makes it possible).”
From Automatic analysis of facial expressions: the state of the art (2000) · cites this paper as [10]
The Cohn et al. method achieved classification accuracy of 92 percent for the eyebrow region, 88 percent for the eye region and 83 percent for the nose/mouth region.
“The accuracy of the classification was 92 percent for the eyebrow region, 88 percent for the eye region and 83 percent for the nose/ mouth region.”
From Automatic analysis of facial expressions: the state of the art (2000) · cites this paper as [10]
Cites
Russell, J. A. (1994) · Psychological Bulletin · doi:10.1037/0033-2909.115.1.102 · Cited by 1,640 · see citing works
Strong evidence for universals in facial expressions: A reply to Russell's mistaken critique.
Ekman, P. (1994) · Psychological Bulletin · doi:10.1037//0033-2909.115.2.268 · Cited by 1,130 · see citing works
Facial expression recognition using a dynamic model and motion energy
Essa, I. A., & Pentland, A. (2002) · IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.1995.466916 · Cited by 251 · see citing works
Coren, S., & Russell, J. A. (1992) · Cognition & Emotion · doi:10.1080/02699939208409690 · Cited by 49 · see citing works
Emotion Enhanced Face to Face Meetings Using the Concept of Virtual Space Teleconferencing
Silva, L. C. D., Miyasato, T., & Kishino, F. (1996) · IEICE Transactions on Information and Systems · Cited by 11 · see citing works
Self-citationFacial Expression Recognition from Image Sequence Using Hidden Markov Model
Sakaguchi, T., Ohya, J., & Kishino, F. (1995) · The Journal of The Institute of Image Information and Television Engineers · Cited by 11 · see citing works
Sakaguchi, T., Ohya, J., & Kishino, F. (1995) · The Journal of The Institute of Image Information and Television Engineers · doi:10.3169/itej1978.49.1060 · Cited by 8 · see citing works
Emotion enhanced multimedia meetings using the concept of virtual space teleconferencing
Silva, L. C. D., Miyasato, T., & Kishino, F. (2002) · IEEE International Conference on Multimedia Computing and Systems (ICMCS) · doi:10.1109/mmcs.1996.534951 · Cited by 6 · see citing works
Self-citationDegree of Human Perception of Facial Emotions Based on Audio and Video Information
Silva, L. C. D., & Miyasato, T. (1996) · 電子情報通信学会技術研究報告. IE, 画像工学 · Cited by 2 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
IEMOCAP: Interactive emotional dyadic motion capture database
Busso, C., Bulut, M., Lee, C.-C., Kazemzadeh, A., Mower, E., Kim, S., Chang, J. N., Lee, S., & Narayanan, S. S. (2008) · Language Resources and Evaluation · doi:10.1007/s10579-008-9076-6 · Cited by 3,869 · see citing works
Toward machine emotional intelligence: analysis of affective physiological state
Picard, R. W., Vyzas, E., & Healey, J. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.954607 · Cited by 2,334 · see citing works
Survey on speech emotion recognition: Features, classification schemes, and databases
Ayadi, M. M. H. E., Kamel, M. S., & Karray, F. (2010) · Pattern Recognition · doi:10.1016/j.patcog.2010.09.020 · Cited by 2,089 · see citing works
Automatic analysis of facial expressions: the state of the art
Pantie, M., & Rothkrantz, L. J. M. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.895976 · Cited by 1,778 · see citing works
Efficient Low-rank Multimodal Fusion With Modality-Specific Factors
Liu, Z., Shen, Y., Lakshminarasimhan, V. B., Liang, P. P., Bagher Zadeh, A., & Morency, L.-P. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1209 · Cited by 1,170 · see citing works
CREMA-D: Crowd-Sourced Emotional Multimodal Actors Dataset
Cao, H., Cooper, D. G., Keutmann, M. K., Gur, R. C., Nenkova, A., & Verma, R. (2014) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2014.2336244 · Cited by 954 · see citing works
Context-Dependent Sentiment Analysis in User-Generated Videos
Poria, S., Cambria, E., Hazarika, D., Majumder, N., Zadeh, A., & Morency, L.-P. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1081 · Cited by 939 · see citing works
Multi-sensor fusion in body sensor networks: State-of-the-art and research challenges
Gravina, R., Alinia, P., Ghasemzadeh, H., & Fortino, G. (2017) · Information Fusion · doi:10.1016/j.inffus.2016.09.005 · Cited by 899 · see citing works
Analysis of emotion recognition using facial expressions, speech and multimodal information
Busso, C., Deng, Z., Yildirim, S., Bulut, M., Lee, C. M., Kazemzadeh, A., Lee, S., Neumann, U., & Narayanan, S. (2004) · Proceedings of the 6th international conference on Multimodal interfaces · doi:10.1145/1027933.1027968 · Cited by 840 · see citing works
Gandhi, A., Adhvaryu, K., Poria, S., Cambria, E., & Hussain, A. (2023) · Information Fusion · doi:10.1016/j.inffus.2022.09.025 · Cited by 783 · see citing works
Kort, B. W., Reilly, R., & Picard, R. W. (2002) · IEEE International Conference on Advanced Learning Technologies (ICALT) · doi:10.1109/icalt.2001.943850 · Cited by 751 · see citing works
Convolutional MKL Based Multimodal Emotion Recognition and Sentiment Analysis
Poria, S., Chaturvedi, I., Cambria, E., & Hussain, A. (2016) · Proceedings · doi:10.1109/icdm.2016.0055 · Cited by 624 · see citing works
Facial expression recognition from video sequences: temporal and static modeling
Cohen, I., Sebe, N., Garg, A., Chen, L. S., & Huang, T. S. (2003) · Computer Vision and Image Understanding · doi:10.1016/s1077-3142(03)00081-x · Cited by 874 · see citing works
Toward an affect-sensitive multimodal human-computer interaction
Pantic, M., & Rothkrantz, L. J. M. (2003) · Proceedings of the IEEE · doi:10.1109/jproc.2003.817122 · Cited by 791 · see citing works
Corneanu, C. A., Simon, M. O., Cohn, J. F., & Guerrero, S. E. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2515606 · Cited by 527 · see citing works
Affective Learning — A Manifesto
Picard, R. W., Papert, S. A., Bender, W. R., Blumberg, B. M., Breazeal, C., Cavallo, D. P., Machover, T., Resnick, M., Roy, D. K., & Strohecker, C. (2004) · BT Technology Journal · doi:10.1023/b:bttj.0000047603.37042.33 · Cited by 511 · see citing works
Multimodal sentiment analysis using hierarchical fusion with context modeling
Majumder, N., Hazarika, D., Gelbukh, A. F., Cambria, E., & Poria, S. (2018) · Knowledge-Based Systems · doi:10.1016/j.knosys.2018.07.041 · Cited by 385 · see citing works
Toward Emotion Recognition in Car-Racing Drivers: A Biosignal Processing Approach
Katsis, C. D., Katertsidis, N. S., Ganiatsas, G., & Fotiadis, D. I. (2008) · IEEE Transactions on Systems Man and Cybernetics - Part A Systems and Humans · doi:10.1109/tsmca.2008.918624 · Cited by 332 · see citing works
State of the art of smart homes
Silva, L. C. D., Morikawa, C., & Petra, I. (2012) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2012.05.002 · Cited by 298 · see citing works
Kessous, L., Castellano, G., & Caridakis, G. (2010) · Journal on Multimodal User Interfaces · doi:10.1007/s12193-009-0025-5 · Cited by 256 · see citing works
Multimodal Sentiment Analysis: Addressing Key Issues and Setting Up the Baselines
Poria, S., Majumder, N., Hazarika, D., Cambria, E., Gelbukh, A. F., & Hussain, A. (2018) · IEEE Intelligent Systems · doi:10.1109/mis.2018.2882362 · Cited by 254 · see citing works
(2013) · Cited by 236 · see citing works
Automated identification of media bias in news articles: an interdisciplinary literature review
Hamborg, F., Donnay, K., & Gipp, B. (2018) · International Journal on Digital Libraries · doi:10.1007/s00799-018-0261-y · Cited by 217 · see citing works
Recognizing Human Emotional State From Audiovisual Signals
Wang, Y., & Guan, L. (2008) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2008.921734 · Cited by 213 · see citing works
Poria, S., Peng, H., Hussain, A., Howard, N., & Cambria, E. (2017) · Neurocomputing · doi:10.1016/j.neucom.2016.09.117 · Cited by 214 · see citing works
Gesture-Based Affective Computing on Motion Capture Data
Kapur, A., Kapur, A., Virji‐Babul, N., Tzanetakis, G., & Driessen, P. F. (2005) · Lecture notes in computer science · doi:10.1007/11573548_1 · Cited by 207 · see citing works
Recognizing Human Emotional State From Audiovisual Signals*
Wang, Y., & Guan, L. (2008) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2008.927665 · Cited by 202 · see citing works
Multimodal human emotion/expression recognition
Chen, L. S., Huang, T. S., Miyasato, T., & Nakatsu, R. (2002) · Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition · doi:10.1109/afgr.1998.670976 · Cited by 183 · see citing works
Self-citationFear-type emotion recognition for future audio-based surveillance systems
Clavel, C., Vasilescu, I., Devillers, L., Richard, G., & Ehrette, T. (2008) · Speech Communication · doi:10.1016/j.specom.2008.03.012 · Cited by 180 · see citing works
Emotion Recognition from Facial Expressions using Multilevel HMM
Cohen, I. L., Garg, A., & Huang, T. S. (2000) · Cited by 169 · see citing works
A survey of speech emotion recognition in natural environment
Fahad, M. S., Ranjan, A., Yadav, J., & Deepak, A. (2020) · Digital Signal Processing · doi:10.1016/j.dsp.2020.102951 · Cited by 162 · see citing works
Is there an advantage for recognizing multi-modal emotional stimuli?
Paulmann, S., & Pell, M. D. (2011) · Motivation and Emotion · doi:10.1007/s11031-011-9206-0 · Cited by 148 · see citing works
Computational approaches for emotion detection in text
Binali, H., Wu, C., & Potdar, V. M. (2010) · IEEE International Conference on Digital Ecosystems and Technologies · doi:10.1109/dest.2010.5610650 · Cited by 146 · see citing works
“How do you know that I don’t understand?” A look at the future of intelligent tutoring systems
Sarrafzadeh, A., Alexander, S., Dadgostar, F., Fan, C., & Bigdeli, A. (2007) · Computers in Human Behavior · doi:10.1016/j.chb.2007.07.008 · Cited by 143 · see citing works
Emotion recognition using a Cauchy Naive Bayes classifier
Sebe, N., Lew, M. S., Cohen, I., Garg, A., & Huang, T. S. (2003) · doi:10.1109/icpr.2002.1044578 · Cited by 140 · see citing works
Audiovisual Information Fusion in Human–Computer Interfaces and Intelligent Environments: A Survey
Shivappa, S. T., Trivedi, M. M., & Rao, B. D. (2010) · Proceedings of the IEEE · doi:10.1109/jproc.2010.2057231 · Cited by 127 · see citing works
Interpreting ambiguous emotional expressions
Mower, E., Metallinou, A., Lee, C.-C., Kazemzadeh, A., Busso, C., Lee, S., & Narayanan, S. S. (2009) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii.2009.5349500 · Cited by 127 · see citing works
Multiple kernel learning for emotion recognition in the wild
Sikka, K., Dykstra, K., Sathyanarayana, S., Littlewort, G., & Bartlett, M. (2013) · Proceedings of the 15th ACM on International conference on multimodal interaction · doi:10.1145/2522848.2531741 · Cited by 119 · see citing works
A Multimodal Emotion Detection System during Human–Robot Interaction
Alonso-Martín, F., Malfáz, M., Sequeira, J. S., Gorostiza, J. F., & Salichs, M. A. (2013) · Sensors · doi:10.3390/s131115549 · Cited by 117 · see citing works
Evaluating affective interactions: Alternatives to asking what users feel
Picard, R. W., & Daily, S. B. (2005) · Cited by 107 · see citing works
A neural network approach for human emotion recognition in speech
Bhatti, M. W., Wang, Y., & Guan, L. (2004) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas.2004.1329238 · Cited by 109 · see citing works
Towards a Learning Companion that Recognizes Affect
Kapoor, A., Mota, S., & Picard, R. W. (2001) · Cited by 107 · see citing works
Khan, M. M., Ward, R. D., & Ingleby, M. (2009) · ACM Transactions on Applied Perception · doi:10.1145/1462055.1462061 · Cited by 104 · see citing works
Careful what you share in six seconds
Rafiq, R. I., Hosseinmardi, H., Han, R. Y., Lv, Q., Mishra, S., & Mattson, S. A. (2015) · IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM) · doi:10.1145/2808797.2809381 · Cited by 92 · see citing works
Stappen, L., Baird, A., Schumann, L., & Schuller, B. W. (2021) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2021.3097002 · Cited by 92 · see citing works
Facial expression recognition from video sequences
Cohen, I., Sebe, N., Garg, A., Lew, M. S., & Huang, T. S. (2003) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2002.1035527 · Cited by 90 · see citing works
Emotion Detection from Speech Signals using Voting Mechanism on Classified Frames
Zamil, A. A. A., Hasan, S., Baki, S. M. J., Adam, J. M., & Zaman, I. (2019) · International Conference on Robotics, Electrical and Signal Processing Techniques (ICREST) · doi:10.1109/icrest.2019.8644168 · Cited by 87 · see citing works
Agent based affective tutoring systems: A pilot study
Mao, X., & Li, Z. (2010) · Computers & Education · doi:10.1016/j.compedu.2010.01.005 · Cited by 83 · see citing works
Efficient Low-rank Multimodal Fusion with Modality-Specific FactorsPreprint
Liu, Z., Shen, Y., Lakshminarasimhan, V., Liang, P. P., Zadeh, A., & Morency, L. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.00064 · Cited by 80 · see citing works
Recognising Emotions from Keyboard Stroke Pattern
Khanna, P., & Sasikumar, M. (2010) · International Journal of Computer Applications · doi:10.5120/1614-2170 · Cited by 78 · see citing works
Approaches, Applications, and Challenges in Physiological Emotion Recognition—A Tutorial Overview
Can, Y. S., Mahesh, B., & André, E. (2023) · Proceedings of the IEEE · doi:10.1109/jproc.2023.3286445 · Cited by 76 · see citing works
An integrated telemedicine platform for the assessment of affective physiological states
Katsis, C. D., Ganiatsas, G., & Fotiadis, D. I. (2006) · Diagnostic Pathology · doi:10.1186/1746-1596-1-16 · Cited by 65 · see citing works
Saha, S., Ghosh, M., Ghosh, S., Sen, S., Singh, P. K., Geem, Z. W., & Sarkar, R. (2020) · Applied Sciences · doi:10.3390/app10082816 · Cited by 63 · see citing works
WikiArt Emotions: An Annotated Dataset of Emotions Evoked by Art
Mohammad, S. M., & Kiritchenko, S. (2018) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/33oybkwsfa8r · Cited by 63 · see citing works
PRATIT: a CNN-based emotion recognition system using histogram equalization and data augmentation
Mungra, D., Agrawal, A., Sharma, P., Tanwar, S., & Obaidat, M. S. (2019) · Multimedia Tools and Applications · doi:10.1007/s11042-019-08397-0 · Cited by 62 · see citing works
HEU Emotion: a large-scale database for multimodal emotion recognition in the wild
Chen, J., Wang, C., Wang, K., Yin, C., Zhao, C., Xu, T., Zhang, X., Huang, Z., Liu, M., & Yang, T. (2021) · Neural Computing and Applications · doi:10.1007/s00521-020-05616-w · Cited by 56 · see citing works
Kort, B. W., Reilly, R. B., & Picard, R. W. (2001) · Cited by 56 · see citing works
Multimodal Emotion Recognition Based on Cascaded Multichannel and Hierarchical Fusion
Liu, X., Xu, Z., & Huang, K. (2023) · Computational Intelligence and Neuroscience · doi:10.1155/2023/9645611 · Cited by 55 · see citing works
Recognizing Human Emotion from Audiovisual Informaiton
Wang, Y., & Guan, L. (2006) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2005.1415607 · Cited by 55 · see citing works
Emotion recognition from audiovisual information
Chen, L. S., Tao, H., Huang, T. S., Miyasato, T., & Nakatsu, R. (2002) · IEEE International Workshop on Multimedia Signal Processing (MMSP) · doi:10.1109/mmsp.1998.738917 · Cited by 54 · see citing works
Self-citationGoncalves, L., & Busso, C. (2022) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2022.3216993 · Cited by 53 · see citing works
Automatic emotion detection model from facial expression
Dagar, D., Hudait, A., Tripathy, H. K., & Das, M. N. (2016) · IEEE International Conference on Advanced Communications, Control and Computing Technologies · doi:10.1109/icaccct.2016.7831605 · Cited by 52 · see citing works
Yang, E., & Dorneich, M. C. (2017) · International Journal of Social Robotics · doi:10.1007/s12369-017-0407-x · Cited by 52 · see citing works
Comparative Analytical Survey on Cognitive Agents with Emotional Intelligence
Zall, R., & Kangavari, M. R. (2022) · Cognitive Computation · doi:10.1007/s12559-022-10007-5 · Cited by 52 · see citing works
A comprehensive survey on deep learning-based approaches for multimodal sentiment analysis
Ghorbanali, A., & Sohrabi, M. K. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-023-10555-8 · Cited by 51 · see citing works
Real-time facial feature extraction and emotion recognition
Silva, L. C. D., & Hui, S. C. (2004) · International Conference on Information, Communications & Signal Processing (ICICS) · doi:10.1109/icics.2003.1292676 · Cited by 51 · see citing works
Emotional Prosody Processing in Schizophrenic Patients: A Selective Review and Meta-Analysis
Lin, Y., Ding, H., & Zhang, Y. (2018) · Journal of Clinical Medicine · doi:10.3390/jcm7100363 · Cited by 50 · see citing works
A Novel Exploitative and Explorative GWO-SVM Algorithm for Smart Emotion Recognition
Yan, X., Lin, Z., Lin, Z., & Vucetic, B. S. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3235356 · Cited by 47 · see citing works
Machines Outperform Laypersons in Recognizing Emotions Elicited by Autobiographical Recollection
Janssen, J. H., Tacken, P., de Vries, G.-J., van den Broek, E. L., Westerink, J. H. D. M., Haselager, P., & IJsselsteijn, W. A. (2013) · Human-Computer Interaction · doi:10.1080/07370024.2012.755421 · Cited by 48 · see citing works
Facial Expression Recognition: A Fully Integrated Approach
Valenti, R., Sebe, N., & Gevers, T. (2007) · doi:10.1109/iciapw.2007.25 · Cited by 44 · see citing works
On assisting a visual-facial affect recognition system with keyboard-stroke pattern information
Σταθοπούλου, Ι., Alepis, E., Tsihrintzis, G. A., & Virvou, M. K. (2009) · Knowledge-Based Systems · doi:10.1016/j.knosys.2009.11.007 · Cited by 43 · see citing works
The Multimodal Driver Monitoring Database: A Naturalistic Corpus to Study Driver Attention
Jha, S., Marzban, M. F., Hu, T., Mahmoud, M. H., Al‐Dhahir, N., & Busso, C. (2021) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2021.3095462 · Cited by 44 · see citing works
Asghar, M. Z., Subhan, F., Imran, M., Kundi, F. M., Shamshirband, S., Mosavi, A., Csiba, P., & Várkonyi-Kóczy, A. R. (2019) · Preprints.org · doi:10.20944/preprints201908.0019.v1 · Cited by 43 · see citing works
Tsihrintzis, G. A., Virvou, M. K., Alepis, E., & Σταθοπούλου, Ι. (2008) · International Conference on Information Technology: New Generations · doi:10.1109/itng.2008.152 · Cited by 40 · see citing works
Multimodal information fusion for selected multimedia applications
Guan, L., Wang, Y., Zhang, R., Tie, Y., Bulzacki, A., & Ibrahim, M. (2010) · International Journal of Multimedia Intelligence and Security · doi:10.1504/ijmis.2010.035969 · Cited by 34 · see citing works
The Design and Analysis of a Real-Time, Continuous Arousal Monitor
Grundlehner, B., Brown, L., Penders, J., & Gyselinckx, B. (2009) · Wearable and Implantable Body Sensor Networks · doi:10.1109/bsn.2009.21 · Cited by 39 · see citing works
Multi-Attention Multimodal Sentiment Analysis
Kim, T., & Lee, B. (2020) · International Conference on Multimedia Retrieval · doi:10.1145/3372278.3390698 · Cited by 39 · see citing works
Robust Computer Vision: Theory and Applications
Sebe, N., & Lew, M. S. (2003) · Cited by 38 · see citing works
RAMAS: Russian Multimodal Corpus of Dyadic Interaction for Affective Computing
Perepelkina, O. S., Kazimirova, E., & Konstantinova, M. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-99579-3_52 · Cited by 37 · see citing works
Using physiological signal analysis to design affective VR games
Li, Y., Elmaghraby, A. S., El-Baz, A. S., & Sokhadze, E. M. (2015) · IEEE International Symposium on Signal Processing and Information Technology (ISSPIT) · doi:10.1109/isspit.2015.7394401 · Cited by 37 · see citing works
Survey on Feature Extraction and Applications of Biosignals
Supratak, A., Wu, C., Dong, H., Sun, K., & Guo, Y. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-50478-0_8 · Cited by 36 · see citing works
Song, K.-S., Nho, Y., Seo, J.-H., & Kwon, D. (2018) · International Conference on Ubiquitous Robots (UR) · doi:10.1109/urai.2018.8441795 · Cited by 35 · see citing works
Modality-based Factorization for Multimodal Fusion
Barezi, E. J., & Fung, P. (2019) · Rare & Special e-Zone (The Hong Kong University of Science and Technology) · doi:10.18653/v1/w19-4331 · Cited by 35 · see citing works
Audio-lingual and Visual-facial Emotion Recognition: Towards a Bi-modal Interaction System
Alepis, E., Σταθοπούλου, Ι., Virvou, M. K., Tsihrintzis, G. A., & Kabassi, K. (2010) · IEEE International Conference on Tools with Artificial Intelligence (ICTAI) · doi:10.1109/ictai.2010.111 · Cited by 33 · see citing works
Soman, K., Sathiya, A., & Suganthi, N. (2014) · International Conference on Information Communication and Embedded Systems · doi:10.1109/icices.2014.7034000 · Cited by 33 · see citing works
Face mask design to mitigate facial expression occlusion
Genç, Ç., Colley, A., Löchtefeld, M., & Häkkilä, J. (2020) · International Symposium on Wearable Computers · doi:10.1145/3410531.3414303 · Cited by 33 · see citing works
Harnessing emotions for depression detection
Prabhu, S. M., Mittal, H., Varagani, R., Jha, S., & Singh, S. (2021) · Pattern Analysis and Applications · doi:10.1007/s10044-021-01020-9 · Cited by 39 · see citing works
Emotion Classification from Speech and Text in Videos Using a Multimodal Approach
Caschera, M. C., Grifoni, P., & Ferri, F. (2022) · Multimodal Technologies and Interaction · doi:10.3390/mti6040028 · Cited by 34 · see citing works
Donnier, S., Kovács, G., Oña, L. S., Bräuer, J., & Amici, F. (2020) · Scientific Reports · doi:10.1038/s41598-020-78275-5 · Cited by 32 · see citing works
The Effect of Time Delay on Emotion, Arousal, and Satisfaction in Human-Robot Interaction
Yang, E., & Dorneich, M. C. (2015) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1541931215591094 · Cited by 33 · see citing works
Affective computation on EEG correlates of emotion from musical and vocal stimuli
Khosrowabadi, R., Wahab, A., Ang, K. K., & Baniasad, M. H. (2009) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2009.5178748 · Cited by 31 · see citing works
Building a three-level multimodal emotion recognition framework
Garcia-Garcia, J. M., Lozano, M. D., Penichet, V. M. R., & Law, E. L. (2022) · Multimedia Tools and Applications · doi:10.1007/s11042-022-13254-8 · Cited by 32 · see citing works
"Emotions That Influence Purchase Decisions And Their Electronic Processing "
Consoli, D. (2009) · Annales Universitatis Apulensis Series Oeconomica · doi:10.29302/oeconomica.2009.11.2.45 · Cited by 30 · see citing works
Unobtrusive Multimodal Emotion Detection in Adaptive Interfaces: Speech and Facial Expressions
Truong, K. P., van Leeuwen, D. A., & Neerincx, M. A. (2007) · Lecture notes in computer science · doi:10.1007/978-3-540-73216-7_40 · Cited by 30 · see citing works
Jassim, W. A., Paramesran, R., & Harte, N. (2017) · IET Signal Processing · doi:10.1049/iet-spr.2016.0336 · Cited by 29 · see citing works
A New Information Fusion Method for Bimodal Robotic Emotion Recognition
Han, M.-J., Hsu, J.-H., Song, K., & Chang, F. (2008) · Journal of Computers · doi:10.4304/jcp.3.7.39-47 · Cited by 29 · see citing works
Analysis and detection of labeled cyberbullying instances in Vine, a video-based social network
Rafiq, R. I., Hosseinmardi, H., Mattson, S. A., Han, R. Y., Lv, Q., & Mishra, S. (2016) · Social Network Analysis and Mining · doi:10.1007/s13278-016-0398-x · Cited by 29 · see citing works
Emotion recognition based on a novel triangular facial feature extraction method
Huang, K., Huang, S., & Kuo, Y.-H. (2010) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2010.5596374 · Cited by 28 · see citing works
Step-wise emotion recognition using concatenated-HMM
Özkan, D., Scherer, S. J., & Morency, L. (2012) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/2388676.2388780 · Cited by 27 · see citing works
EEG-based emotion recognition using an improved radial basis function neural network
Zhang, J., Zhou, Y., & Liu, Y. (2020) · Journal of Ambient Intelligence and Humanized Computing · doi:10.1007/s12652-020-02049-0 · Cited by 26 · see citing works
- Authors
- De Silva, L. C., Miyasato, T., & Nakatsu, R.
- Affiliations
- Advanced Telecommunications Research Institute International (JP)
- ORCID
- Nakatsu, R.
- Venue
- Proceedings of ICICS, 1997 International Conference on Information, Communications and Signal Processing. Theme: Trends in Information Systems Engineering and Wireless Multimedia Communications (Cat. No.97TH8237)
- Publisher
- IEEE
- Published
- 2002-11-22
- Volume
- 1
- Pages
- 397-401
- OpenAlex
- W2154739180
- Cited by
- 319
- References
- 9
- Citations by year
- 2026: 2 · 2025: 10 · 2024: 16 · 2023: 16 · 2022: 39 · 2021: 16 · 2020: 18 · 2019: 11 · 2018: 25 · 2017: 11 · 2016: 18 · 2015: 15 · 2014: 8 · 2013: 8 · 2012: 12
- FWCI
- 13.69
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Conference on Information, Communications & Signal Processing (ICICS) (publishedVersion)
- National University of Singapore (submittedVersion)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
- Keywords
- facial emotion recognition, multimodal emotion recognition, vocal emotion expression
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
