The DOI matches its Crossref record; 32 of the 33 references it cites were found in a registry or an index.
K-Means Clustering-Based Kernel Canonical Correlation Analysis for Multimodal Emotion Recognition in Human–Robot Interaction
Abstract
In this article,K-meansclustering-based Kernel canonical correlation analysis algorithm is proposed for multimodal emotion recognition in human–robot interaction (HRI). The multimodal features (gray pixels; time and frequency domain) extracted from facial expression and speech are fused based on Kernel canonical correlation analysis.K-means clustering is used to select features from multiple modalities and reduce dimensionality. The proposed approach can improve the heterogenicity among different modalities and make multiple modalities complementary to promote multimodal emotion recognition. Experiments on two datasets, namely SAVEE and eNTERFACE‘05, are conducted to evaluate the accuracy of the proposed method. The results show that the proposed method produces good recognition rates that are higher than the ones produced by the methods withoutK-means clustering; more specifically, they are 2.77% higher in SAVEE and 4.7% higher in eNTERFACE‘05.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Davies, D. L., & Bouldin, D. W. (1979) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.1979.4766909 · Cited by 9,145
Survey on speech emotion recognition: Features, classification schemes, and databases
El Ayadi, M., Kamel, M. S., & Karray, F. (2011) · Pattern Recognition · doi:10.1016/j.patcog.2010.09.020 · Cited by 2,073
EmotionMeter: A Multimodal Framework for Recognizing Human Emotions
Zheng, W.-L., Liu, W., Lu, Y., Lu, B.-L., & Cichocki, A. (2019) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2018.2797176 · Cited by 1,276
Toward detecting emotions in spoken dialogs
Lee, C. M., & Narayanan, S. S. (2005) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/tsa.2004.838534 · Cited by 988
Muda, L., Begam, M. H., & Elamvazuthi, I. (2010) · arXiv (Cornell University) · doi:10.48550/arxiv.1003.4083 · Cited by 803
A Review and Meta-Analysis of Multimodal Affect Detection Systems
D'mello, S. K., & Kory, J. (2015) · ACM Computing Surveys · doi:10.1145/2682899 · Cited by 628
Spatial–Temporal Recurrent Neural Network for Emotion Recognition
Zhang, T., Zheng, W., Cui, Z., Zong, Y., & Li, Y. (2019) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2017.2788081 · Cited by 595
pyAudioAnalysis: An Open-Source Python Library for Audio Signal Analysis
Giannakopoulos, T. (2015) · PLOS ONE · doi:10.1371/journal.pone.0144610 · Cited by 454
Learning Affective Features With a Hybrid Deep Model for Audio–Visual Emotion Recognition
Zhang, S., Zhang, S., Huang, T., Gao, W., & Tian, Q. (2018) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2017.2719043 · Cited by 346 · cited 1×
“In the emotion recognition of multimodal information, emotion feature extraction and multimodal fusion are the most important steps [7], [8].”
BackgroundQin, J., Fu, W., Gao, H., & Zheng, W. X. (2017) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2016.2526683 · Cited by 333
Research works printed but not linked: 1
- 10.1109/TCSVT.2017.2719043 · DOI · Paper page
Cited by
Wu, Y., Mi, Q., & Gao, T. (2025) · Biomimetics · doi:10.3390/biomimetics10070418 · Cited by 182
“The feature extraction stage involves extracting relevant features from different modalities, such as speech, visual, and text data [22].”
Emotion recognition based on brain-like multimodal hierarchical perception
Zhu, X., Huang, Y., Wang, X., & Wang, R. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-17347-w · Cited by 142
Toward Explainable Affective Computing: A Review
Cortiñas-Lorenzo, K., & Lacey, G. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3270027 · Cited by 110
“Unsupervised methods such as clustering can not only help improve predictive performance by identifying the most relevant discriminant features [62], [63] but can also ease feature selection for semantic interpretation.”
MethodLearning facial expression and body gesture visual information for video emotion recognition
Wei, J., Hu, G., Yang, X., Tuan, L. A., & Dong, Y. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.121419 · Cited by 73
Prakash, J., V, Vijay, S. A. A., Ganeshkumar, P., & Karthikeyan, N. K. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.107977 · Cited by 50
Metadata
- Authors
- Chen, L., Wang, K., Li, M., Wu, M., Pedrycz, W., & Hirota, K.
- Affiliations
- China University of Geosciences (CN)
- University of Alberta (CA)
- Tokyo Institute of Technology (JP)
- Venue
- IEEE Transactions on Industrial Electronics
- Published
- 2023-01
- Volume
- 70
- Issue
- 1
- Pages
- 1016-1024
- ISSN
- 0278-0046 (ISSN-L)
- 1557-9948
- OpenAlex
- W4213102761
- Cited by
- 113
- References
- 33
- Citations by year
- 2026: 18 · 2025: 37 · 2024: 30 · 2023: 27
- FWCI
- 21.88
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Transactions on Industrial Electronics (publishedVersion)
- Funders
- National Natural Science Foundation of China (61973286, 61603356, 61773353)
- Higher Education Discipline Innovation Project (B17040)
- Fundamental Research Funds for the Central Universities
- China University of Geosciences, Wuhan (2021063)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face and Expression Recognition
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Keywords
- multimodal emotion recognition, human-robot interaction, kernel canonical correlation analysis, K-means clustering, feature selection, dimensionality reduction, facial expressions, speech emotion recognition, multimodal fusion, SAVEE
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: K-Means Clustering-Based Kernel Canonical Correlation Analysis for Multimodal Emotion Recognition in Human–Robot Interaction. https://app.cochat.ai/papers/10.1109/tie.2022.3150097
- The paper
- Chen, L., Wang, K., Li, M., Wu, M., Pedrycz, W., & Hirota, K. (2023). K-Means Clustering-Based Kernel Canonical Correlation Analysis for Multimodal Emotion Recognition in Human–Robot Interaction. IEEE Transactions on Industrial Electronics. https://doi.org/10.1109/tie.2022.3150097
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.