Error Weighted Semi-Coupled Hidden Markov Model for Audio-Visual Emotion Recognition
The DOI matches its Crossref record; 75 of the 80 references it cites were found in a registry or an index.
Abstract
This paper presents an approach to the automatic recognition of human emotions from audio-visual bimodal signals using an error weighted semi-coupled hidden Markov model (EWSC-HMM). The proposed approach combines an SC-HMM with a state-based bimodal alignment strategy and a Bayesian classifier weighting scheme to obtain the optimal emotion recognition result based on audio-visual bimodal fusion. The state-based bimodal alignment strategy in SC-HMM is proposed to align the temporal relation between audio and visual streams. The Bayesian classifier weighting scheme is then adopted to explore the contributions of the SC-HMM-based classifiers for different audio-visual feature pairs in order to obtain the emotion recognition output. For performance evaluation, two databases are considered: the MHMC posed database and the SEMAINE naturalistic database. Experimental results show that the proposed approach not only outperforms other fusion-based bimodal emotion recognition methods for posed expressions but also provides satisfactory results for naturalistic expressions.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Emotions precipitate physiological changes that directly impact speech production and, thus, significantly affect speech intonation [36].”
From A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face (2023) · cites this paper as [36]
In the citing paper
“According to previous research findings, compared with neutral speech, the first formant frequency of happy and angry speech is slightly higher, while the first formant frequency of sad speech is significantly lower [36,42].”
From A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face (2023) · cites this paper as [36]
Citations
Cites
The Measurement of Observer Agreement for Categorical Data
Landis, J. R., & Koch, G. G. (1977) · Biometrics · doi:10.2307/2529310 · Cited by 81,148
Rapid object detection using a boosted cascade of simple features
Viola, P., & Jones, M. (2005) · Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001 · doi:10.1109/cvpr.2001.990517 · Cited by 18,300
Measuring nominal scale agreement among many raters.
Fleiss, J. L. (1971) · Psychological Bulletin · doi:10.1037/h0031619 · Cited by 8,733 · cited 1×
“Fleiss’ kappa can work for any number of raters giving categorical ratings to a fixed number of items [46].”
BackgroundPraat: Doing Phonetics by Computer
(2011) · Ear & Hearing · doi:10.1097/aud.0b013e31821473f7 · Cited by 7,863
Picard, R. W. (1997) · MIT Press · doi:10.7551/mitpress/1140.001.0001 · Cited by 7,382
Cootes, T. F., Edwards, G. J., & Taylor, C. J. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.927467 · Cited by 5,409
Handbook of Affective Sciences
(2002) · doi:10.1093/oso/9780195126013.001.0001 · Cited by 5,073
ELIZA—a computer program for the study of natural language communication between man and machine
Weizenbaum, J. (1966) · Communications of the ACM · doi:10.1145/365153.365168 · Cited by 4,452 · cited 2×
“scenario [40], [41] was developed from the ELIZA concept introduced by Weizenbaum [42]; it can be used to build a naturalistic emotion database through machine agent-human conversational elements [41].”
MethodEkman, P. (1993) · American Psychologist · doi:10.1037/0003-066x.48.4.384 · Cited by 3,441
Looking at pictures: Affective, facial, visceral, and behavioral reactions
LANG, P. J., GREENWALD, M. K., BRADLEY, M. M., & HAMM, A. O. (1993) · Psychophysiology · doi:10.1111/j.1469-8986.1993.tb03352.x · Cited by 3,190
Research works printed but not linked: 7
- brand, 1997, Coupled Hidden Markov Models for Modeling Interacting Processes MIT Media Lab Perceptual Computing/Learning and Common Sense Technical Report
- lucey, 2007, Face recognition, 275
- ivanov, 2005, Technical Report CBCL paper 258 Massachusetts Institute of Technology
- 1998, International Organization for Standardization Facial animation parameters Text for ISO/IEC FDIS 144962 Visual ISO/IET JTC1/SC29/WG11 N2502
- 10.1037/0033-2909.111.2.256 · DOI · Paper page
- ekman, 1978, The Facial Action Coding System A Technique for The Measurement of Facial Movement
- douglas-cowie, 2008, The sensitive artifical listener: An induction technique for generating emotionally coloured conversation, Proc Program Workshop Corpora for Research on Emotion and Affect
Cited by
A review of affective computing: From unimodal analysis to multimodal fusion
Poria, S., Cambria, E., Bajpai, R., & Hussain, A. (2017) · Information Fusion · doi:10.1016/j.inffus.2017.02.003 · Cited by 1,644
Akçay, M. B., & Oğuz, K. (2020) · Speech Communication · doi:10.1016/j.specom.2019.12.001 · Cited by 721
A systematic review on affective computing: emotion models, databases, and recent advances
Wang, Y., Song, W., Tao, W., Liotta, A., Yang, D., Li, X., Gao, S., Sun, Y., Ge, W., Zhang, W., et al. (2022) · Information Fusion · doi:10.1016/j.inffus.2022.03.009 · Cited by 615
A Review and Meta-Analysis of Multimodal Affect Detection Systems
D'mello, S. K., & Kory, J. (2015) · ACM Computing Surveys · doi:10.1145/2682899 · Cited by 628
Learning Affective Features With a Hybrid Deep Model for Audio–Visual Emotion Recognition
Zhang, S., Zhang, S., Huang, T., Gao, W., & Tian, Q. (2018) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2017.2719043 · Cited by 346
“Lin et al. [8] employ an error weighted semi-coupled Hidden Markov Models (HMM) to fuse audio and visual streams for emotion recognition.”
BackgroundA Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face
Lian, H., Lu, C., Li, S., Zhao, Y., Tang, C., & Zong, Y. (2023) · Entropy · doi:10.3390/e25101440 · Cited by 224
“(1) Prosodic features : Emotions precipitate physiological changes that directly impact speech production and, thus, significantly affect speech intonation [36].”
BackgroundSurvey on audiovisual emotion recognition: databases, features, and data fusion strategies
Wu, C.-H., Lin, J.-C., & Wei, W.-L. (2014) · APSIPA Transactions on Signal and Information Processing · doi:10.1017/atsip.2014.11 · Cited by 182
Self-citationMultimodal sentimental analysis for social media applications: A comprehensive review
Chandrasekaran, G., Nguyen, T. N., & Hemanth D., J. (2021) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1415 · Cited by 159
Speech Emotion Recognition Using Deep Neural Network Considering Verbal and Nonverbal Speech Sounds
Huang, K.-Y., Wu, C.-H., Hong, Q.-B., Su, M.-H., & Chen, Y.-H. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8682283 · Cited by 116
Self-citationA multimodal hierarchical approach to speech emotion recognition from audio and text
Singh, P., Srivastava, R., Rana, K. P. S., & Kumar, V. (2021) · Knowledge-Based Systems · doi:10.1016/j.knosys.2021.107316 · Cited by 131
D'Mello, S., & Kory, J. (2012) · Proceedings of the 14th ACM international conference on Multimodal interaction · doi:10.1145/2388676.2388686 · Cited by 97
Speech Emotion Recognition Considering Nonverbal Vocalization in Affective Conversations
Hsu, J.-H., Su, M.-H., Wu, C., & Chen, Y. (2021) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2021.3076364 · Cited by 84
Self-citationŠumak, B., Brdnik, S., & Pušnik, M. (2021) · Sensors · doi:10.3390/s22010020 · Cited by 100
Audio-visual emotion recognition using multi-directional regression and Ridgelet transform
Hossain, M. S., & Muhammad, G. (2015) · Journal on Multimodal User Interfaces · doi:10.1007/s12193-015-0207-2 · Cited by 69
Huang, K.-Y., Wu, C., Su, M.-H., & Kuo, Y.-T. (2018) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2018.2803178 · Cited by 73
Self-citationExploring the Contextual Factors Affecting Multimodal Emotion Recognition in Videos
Bhattacharya, P., Gupta, R. K., & Yang, Y. (2021) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2021.3071503 · Cited by 71
Wu, C., Lin, J., & Wei, W.-L. (2013) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2013.2269314 · Cited by 58
Self-citationExploiting the potentialities of features for speech emotion recognition
Li, D., Zhou, Y., Wang, Z., & Gao, D. (2020) · Information Sciences · doi:10.1016/j.ins.2020.09.047 · Cited by 63
Temporal Bayesian Fusion for Affect Sensing: Combining Video, Audio, and Lexical Modalities
Savran, A., Cao, H., Nenkova, A., & Verma, R. (2015) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2362101 · Cited by 53
“Unlike prior temporal fusion studies [9], [26]–[28], our approach does not aim at learning temporal relationships between different modalities.”
BackgroundMethodInfluentialEmotion recognition based on group phase locking value using convolutional neural network
Cui, G., Li, X., & Touyama, H. (2023) · Scientific Reports · doi:10.1038/s41598-023-30458-6 · Cited by 58
Multivariate analysis of car-following behavior data using a coupled hidden Markov model
Zou, Y., Zhu, T., Xie, Y., Zhang, Y., & Zhang, Y. (2022) · Transportation Research Part C Emerging Technologies · doi:10.1016/j.trc.2022.103914 · Cited by 48
Speaking Effect Removal on Emotion Recognition From Facial Expressions Based on Eigenface Conversion
Wu, C.-H., Wei, W.-L., Lin, J.-C., & Lee, W.-Y. (2013) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2013.2272917 · Cited by 45
Self-citationExploring latent semantic information for textual emotion recognition in blog articles
Kang, X., Ren, F., & Wu, Y. (2017) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2017.7510421 · Cited by 42
Emotion recognition from audio-visual data using rule based decision level fusion
Sahoo, S., & Routray, A. (2016) · IEEE Students’ Technology Symposium · doi:10.1109/techsym.2016.7872646 · Cited by 43
Review on Stimuli Presentation for Affect Analysis Based on EEG
Sarma, P., & Barma, S. (2020) · IEEE Access · doi:10.1109/access.2020.2980893 · Cited by 43
Fusion-ConvBERT: Parallel Convolution and BERT Fusion for Speech Emotion Recognition
Lee, S., Han, D. K., & Ko, H. (2020) · Sensors · doi:10.3390/s20226688 · Cited by 42
Hsu, J.-H., & Wu, C. (2023) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2023.3258900 · Cited by 47
Self-citationCombining speech-based and linguistic classifiers to recognize emotion in user spoken utterances
Griol, D., Molina, J. M., & Callejas, Z. (2017) · Neurocomputing · doi:10.1016/j.neucom.2017.01.120 · Cited by 41
Multimodal modelling of human emotion using sound, image and text fusion
Hosseini, S. S., Yamaghani, M. R., & Arabani, S. P. (2023) · Signal Image and Video Processing · doi:10.1007/s11760-023-02707-8 · Cited by 41
Survey of deep emotion recognition in dynamic data using facial, speech and textual cues
Zhang, T., & Tan, Z. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-023-17944-9 · Cited by 46
Zhang, H., Huang, H., & Han, H. (2020) · IEEE Access · doi:10.1109/access.2020.3047395 · Cited by 34
Huang, I.-S., Lu, Y., Shafiq, M., Laghari, A. A., & Yadav, R. (2021) · Mobile Information Systems · doi:10.1155/2021/3561829 · Cited by 32
Multimodal Affect Recognition: Current Approaches and Challenges
Osman, H. A., & Falk, T. H. (2017) · InTech eBooks · doi:10.5772/65683 · Cited by 44
Multimodal Framework for Analyzing the Affect of a Group of People
Huang, X., Dhall, A., Goecke, R., Pietikäinen, M., & Zhao, G. (2018) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2018.2818015 · Cited by 35
Facial Expression Recognition in the Presence of Speech Using Blind Lexical Compensation
Mariooryad, S., & Busso, C. (2015) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2015.2490070 · Cited by 30
Hear Me, See Me, Understand Me: Audio-Visual Autism Behavior Recognition
Deng, S., Kosloski, E. E., Patel, S. D., Barnett, Z. A., Nan, Y., Kaplan, A., Aarukapalli, S., Doan, W. T., Wang, M., Singh, H., et al. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2024.3521838 · Cited by 46
Basu, A., Dasgupta, A., Thyagharajan, A., Routray, A., Guha, R., & Mitra, P. (2018) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2018.2828845 · Cited by 26
FF-SKPCCA: Kernel probabilistic canonical correlation analysis
Sarvestani, R. R., & Boostani, R. (2016) · Applied Intelligence · doi:10.1007/s10489-016-0823-x · Cited by 25
A Visual–Audio-Based Emotion Recognition System Integrating Dimensional Analysis
Tian, J., & She, Y. (2022) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2022.3200060 · Cited by 24
Locally weighted linear regression for cross-lingual valence-arousal prediction of affective words
Wang, J., Yu, L.-C., Lai, K.-H. R., & Zhang, X. (2016) · Neurocomputing · doi:10.1016/j.neucom.2016.02.057 · Cited by 25
Inferring Emotional Tags From Social Images With User Demographics
Wu, B., Jia, J., Yang, Y., Zhao, P., Tang, J., & Tian, Q. (2017) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2017.2655881 · Cited by 22
D’Mello, S. K., Dowell, N., & Graesser, A. C. (2013) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2013.19 · Cited by 22
Speech Emotion Recognition using Convolutional Neural Network with Audio Word-based Embedding
Huang, K.-Y., Wu, C., Hong, Q.-B., Su, M.-H., & Zeng, Y.-R. (2018) · International Symposium on Chinese Spoken Language Processing (ISCSLP) · doi:10.1109/iscslp.2018.8706610 · Cited by 20
Self-citationMultimodal Emotion Recognition Based on Facial Expressions, Speech, and Body Gestures
Yan, J., Li, P., Du, C., Zhu, K., Zhou, X., Liu, Y., & Wei, J. (2024) · Electronics · doi:10.3390/electronics13183756 · Cited by 26
Fourier model based features for analysis and classification of out-of-breath speech
Deb, S., & Dandapat, S. (2017) · Speech Communication · doi:10.1016/j.specom.2017.04.002 · Cited by 19
Multimodal-multisensor affect detection
D’Mello, S. K., Bosch, N., & Chen, H. (2018) · ACM eBooks · doi:10.1145/3107990.3107998 · Cited by 27
Exploiting Psychological Factors for Interaction Style Recognition in Spoken Conversation
Wei, W.-L., Wu, C.-H., Lin, J.-C., & Li, H. (2014) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2014.2300339 · Cited by 20
Self-citationEmotion classification using residual sinusoidal peak amplitude
Deb, S., & Dandapat, S. (2016) · International Conference on Signal Processing and Communications (SPCOM) · doi:10.1109/spcom.2016.7746697 · Cited by 18
Speech emotion recognition using autoencoder bottleneck features and LSTM
Huang, K.-Y., Wu, C., Yang, T., Su, M.-H., & Chou, J. (2016) · International Conference on Orange Technologies · doi:10.1109/icot.2016.8278965 · Cited by 18
Self-citationHong, Q.-B., Wu, C., Su, M.-H., & Chang, C.-C. (2020) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2019.2909873 · Cited by 16
Self-citationMao, Q., Zhu, Q., Rao, Q., Jia, H., & Luo, S. (2019) · IEEE Access · doi:10.1109/access.2019.2916211 · Cited by 17
Introduction to Emotion Recognition
Konar, A., Halder, A., & Chakraborty, A. (2015) · Wiley eBooks · doi:10.1002/9781118910566.ch1 · Cited by 21
Lin, J., Wu, C., & Wei, W.-L. (2013) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2013.6638305 · Cited by 15
Self-citationCoherent Deep-Net Fusion To Classify Shots In Concert Videos
Lin, J., Wei, W.-L., Liu, T.-L., Yang, Y., Wang, H., Tyan, H.-R., & Liao, H.-Y. M. (2018) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2018.2820904 · Cited by 15
Self-citationWu, J., Chuang, H.-P., Hsu, M.-H., & Chen, W. (2013) · Applied and Environmental Microbiology · doi:10.1128/aem.02450-13 · Cited by 15
Mood detection from daily conversational speech using denoising autoencoder and LSTM
Huang, K.-Y., Wu, C., Su, M.-H., & Fu, H.-C. (2017) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2017.7953133 · Cited by 15
Self-citationExploration of Phase Information for Speech Emotion Classification
Deb, S., & Dandapat, S. (2017) · National Conference on Communications · doi:10.1109/ncc.2017.8077114 · Cited by 14
Inferring Emotion Tags from Object Images Using Convolutional Neural Network
Manzoor, A., Ahmad, W., Ehatisham-ul-Haq, M., Hannan, A., Khan, M. A., Ashraf, M. U., Alghamdi, A. M., & Alfakeeh, A. S. (2020) · Applied Sciences · doi:10.3390/app10155333 · Cited by 18
Action Unit Models of Facial Expression of Emotion in the Presence of Speech
Shah, M., Cooper, D. G., Cao, H., Gur, R. C., Nenkova, A., & Verma, R. (2013) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii.2013.15 · Cited by 16
A Cantonese Audio-Visual Emotional Speech (CAVES) dataset
Chong, C. S., Davis, C., & Kim, J. (2023) · Behavior Research Methods · doi:10.3758/s13428-023-02270-7 · Cited by 12
Efficient audio-visual emotion recognition approach
Alasiry, A., Al-Hussain, M. M., Alouane, M. T.-H., & Hadj-Alouane, N. B. (2025) · Multimedia Tools and Applications · doi:10.1007/s11042-024-20572-6 · Cited by 13
SwimSense: Monitoring Swimming Motion Using Body Sensor Networks
Wang, J., Wang, Z., Gao, F., & Guo, M. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-45940-0_5 · Cited by 10
Emotion recognition of conversational affective speech using temporal course modeling
Lin, J., Wu, C., & Wei, W.-L. (2013) · Interspeech (USB) · doi:10.21437/interspeech.2013-354 · Cited by 11
Self-citationLiang, S. (2023) · IET Software · doi:10.1049/2023/5566781 · Cited by 8
BFI-based speaker personality perception using acoustic-prosodic features
Liu, C.-J., Wu, C., & Chiu, Y. (2013) · Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) · doi:10.1109/apsipa.2013.6694234 · Cited by 8
Self-citationChiba, Y., Ito, M., Nose, T., & Ito, A. (2014) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/w14-4310 · Cited by 12
A locally weighted method to improve linear regression for lexical-based valence-arousal prediction
Wang, J., Yu, L.-C., Lai, K.-H. R., & Zhang, X. (2015) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii.2015.7344604 · Cited by 9
Deep-net fusion to classify shots in concert videos
Wei, W.-L., Lin, J., Liu, T.-L., Yang, Y., Wang, H., Tyan, H.-R., & Liao, H.-Y. M. (2017) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2017.7952383 · Cited by 7
Self-citationData collection of elicited facial expressions and speech responses for mood disorder detection
Huang, K.-Y., Wu, C., Kuo, Y.-T., Yen, H.-H., Jang, F.-L., & Chiu, Y. (2015) · International Conference on Orange Technologies · doi:10.1109/icot.2015.7498502 · Cited by 7
Self-citationProgressively Improving Supervised Emotion Classification Through Active Learning
Kang, X., Wu, Y., & Ren, F. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-03014-8_4 · Cited by 7
Halder, A., Chakraborty, A., Konar, A., & Nagar, A. K. (2013) · IEEE International Conference on Fuzzy Systems (FUZZ-IEEE) · doi:10.1109/fuzz-ieee.2013.6622543 · Cited by 6
Lin, J., Wei, W.-L., Wu, C., & Wang, H. (2014) · Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) · doi:10.1109/apsipa.2014.7041621 · Cited by 6
Self-citationA Novel Feature Extraction Model for Large-Scale Workload Prediction in Cloud Environment
Shishira, S. R., & Kandasamy, A. (2021) · SN Computer Science · doi:10.1007/s42979-021-00730-5 · Cited by 6
Arivazhagan, S., & Russel, N. S. (2019) · International Journal of Imaging Systems and Technology · doi:10.1002/ima.22338 · Cited by 6
Speaker-Independent Multimodal Sentiment Analysis for Big Data
Cambria, E., Poria, S., & Hussain, A. (2019) · Springer eBooks · doi:10.1007/978-3-319-97598-6_2 · Cited by 7
Deep Emotion Recognition using Facial, Speech and Textual Cues: A SurveyPreprint
Zhang, T., & Tan, Z. (2021) · doi:10.36227/techrxiv.15184302.v2 · Cited by 9
PUMAVE-D: panjab university multilingual audio and video facial expression dataset
Singh, L., Aggarwal, N., & Singh, S. (2022) · Multimedia Tools and Applications · doi:10.1007/s11042-022-14102-5 · Cited by 6
Integration of audio-visual information for multi-speaker multimedia speaker recognition
Yang, J., Chen, F., Cheng, Y., & Lin, P. (2023) · Digital Signal Processing · doi:10.1016/j.dsp.2023.104315 · Cited by 5
Understanding Blog author's emotions with hierarchical Bayesian models
Kang, X., & Ren, F. (2016) · IEEE International Conference on Networking, Sensing and Control (ICNSC) · doi:10.1109/icnsc.2016.7479037 · Cited by 5
Deep Analysis for Speech Emotion Recognization
Bhattacharya, S., Borah, S., Mishra, B. K., & Das, N. (2022) · International Conference on Computer Science, Engineering and Applications (ICCSEA) · doi:10.1109/iccsea54677.2022.9936080 · Cited by 7
Mood disorder identification using deep bottleneck features of elicited speech
Huang, K.-Y., Wu, C., Su, M.-H., & Chou, C.-H. (2017) · Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) · doi:10.1109/apsipa.2017.8282296 · Cited by 6
Self-citationEmotion recognition from multi-modal information
Wu, C., Lin, J., Wei, W.-L., & Cheng, K.-C. (2013) · Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) · doi:10.1109/apsipa.2013.6694347 · Cited by 6
Self-citationEmotion detection on phone calls during emergency using ensemble model with hyper parameter tuning
selvan, A. K., Nimmi, K., Janet, B., & Sivakumaran, N. (2022) · International Journal of Information Technology · doi:10.1007/s41870-022-01091-9 · Cited by 8
Huang, K.-Y., Lin, J.-K., Chiu, Y., & Wu, C. (2015) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2015.7178980 · Cited by 4
Self-citationExploring Macroscopic Fluctuation of Facial Expression for Mood Disorder Classification
Hong, Q.-B., Wu, C., Su, M.-H., & Huang, K.-Y. (2018) · Asian Conference on Affective Computing and Intelligent Interaction (ACII Asia) · doi:10.1109/aciiasia.2018.8470337 · Cited by 4
Self-citation3D skeletal movement-enhanced emotion recognition networks
Shi, J., Liu, C., Ishi, C. T., & Ishiguro, H. (2021) · APSIPA Transactions on Signal and Information Processing · doi:10.1017/atsip.2021.11 · Cited by 3
A probabilistic fusion strategy for audiovisual emotion recognition of sparse and noisy data
Lin, J., Wu, C., & Wei, W.-L. (2013) · International Conference on Orange Technologies · doi:10.1109/icot.2013.6521212 · Cited by 3
Self-citationEvent-Based Sports Videos Classification Using HMM Framework
Wilson, S., Mohan, C. K., & Murthy, K. S. (2014) · Advances in computer vision and pattern recognition · doi:10.1007/978-3-319-09396-3_11 · Cited by 3
Deep Emotion Recognition using Facial, Speech and Textual Cues: A SurveyPreprint
Zhang, T., & Tan, Z. (2021) · doi:10.36227/techrxiv.15184302 · Cited by 4
Mansouri, B. Z., Ghaffary, H. R., & Harimi, A. (2022) · Research Square · doi:10.21203/rs.3.rs-2369713/v1 · Cited by 4
Ong, J. H., Leung, F. Y. N., & Liu, F. (2023) · Language Resources and Evaluation · doi:10.1007/s10579-023-09698-5 · Cited by 4
Zhong, Q., & Shao, X.-H. (2024) · Intelligent Data Analysis · doi:10.3233/ida-230305 · Cited by 3
Action unit reconstruction of occluded facial expression
Wu, C., Lin, J., & Wei, W.-L. (2014) · International Conference on Orange Technologies · doi:10.1109/icot.2014.6956628 · Cited by 2
Self-citationAudio-Visual Emotion Recognition Using Neural Networks Learned with Hints
Lu, K., & Zhang, X. (2013) · International Conference on Image and Graphics · doi:10.1109/icig.2013.109 · Cited by 2
Recognizing human actions by fusing spatial-temporal HOG and key point histogram
Hsieh, C., Hung, M., & Huang, W.-Y. (2016) · Journal of the Chinese Institute of Engineers · doi:10.1080/02533839.2015.1134282 · Cited by 2
Interaction style detection based on Fused Cross-Correlation Model in spoken conversation
Wei, W.-L., Wu, C., Lin, J., & Li, H. Z. (2013) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2013.6639323 · Cited by 3
Self-citationSteady state coupled hidden Markov model
Kroumbi, H., & Nasroallah, A. (2025) · Communications in Statistics - Simulation and Computation · doi:10.1080/03610918.2025.2523855
Chen, S.-H., Lee, Y.-S., Hsu, Y., Wu, C., & Wang, J. (2018) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2018.8462680 · Cited by 1
Self-citationModeling User’s State During Dialog Turn Using HMM For Multi-modal Spoken Dialog System
Chiba, Y., Ito, A., & Ito, M. (2015) · Cited by 2
Metadata
- Authors
- Lin, J.-C., Wu, C.-H., & Wei, W.-L.
- Affiliations
- National Cheng Kung University (TW)
- Venue
- IEEE Transactions on Multimedia
- Published
- 2012-02
- Volume
- 14
- Issue
- 1
- Pages
- 142-156
- ISSN
- 1520-9210 (ISSN-L)
- 1941-0077
- OpenAlex
- W2038821742
- Cited by
- 121
- References
- 80
- Citations by year
- 2025: 2 · 2024: 5 · 2023: 11 · 2022: 7 · 2021: 11 · 2020: 9 · 2019: 7 · 2018: 12 · 2017: 13 · 2016: 10 · 2015: 10 · 2014: 9 · 2013: 14 · 2012: 1
- FWCI
- 9.88
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Transactions on Multimedia (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Keywords
- audio-visual emotion recognition, bimodal fusion
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.