Emotion Recognition Based on Weighted Fusion Strategy of Multichannel Physiological Signals
Open accessAbstract
Emotion recognition is an important pattern recognition problem that has inspired researchers for several areas. Various data from humans for emotion recognition have been developed, including visual, audio, and physiological signals data. This paper proposes a decision-level weight fusion strategy for emotion recognition in multichannel physiological signals. Firstly, we selected four kinds of physiological signals, including Electroencephalography (EEG), Electrocardiogram (ECG), Respiration Amplitude (RA), and Galvanic Skin Response (GSR). And various analysis domains have been used in physiological emotion features extraction. Secondly, we adopt feedback strategy for weight definition, according to recognition rate of each emotion of each physiological signal based on Support Vector Machine (SVM) classifier independently. Finally, we introduce weight in decision level by linear fusing weight matrix with classification result of each SVM classifier. The experiments on the MAHNOB-HCI database show the highest accuracy. The results also provide evidence and suggest a way for further developing a more specialized emotion recognition system based on multichannel data using weight fusion strategy.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Özerdem and Polat achieved average overall accuracy of 77.14% using MLPNN and 72.92% using kNN on EEG signals.
“Özerdem and Polat used EEG signal -resultant average overall accuracy was 77.14% (using MLPNN) and 72.92% (using kNN) [56].”
From Survey on AI-Based Multimodal Methods for Emotion Detection (2019) · cites this paper as [56]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Supervised Machine Learning: A Review of Classification Techniques
Kotsiantis, S. B. (2007) · Informatica · Cited by 4,192 · see citing works
The emotion probe. Studies of motivation and attention.
Lang, P. J. (1995) · American Psychologist · doi:10.1037/0003-066x.50.5.372 · Cited by 2,846 · see citing works
The emotion probe: Studies of motivation and attention.
Lang, P. J. (1995) · American Psychologist · doi:10.1037//0003-066x.50.5.372 · Cited by 2,752 · see citing works
Zheng, W.-L., & Lu, B.-L. (2015) · IEEE Transactions on Autonomous Mental Development · doi:10.1109/tamd.2015.2431497 · Cited by 2,477 · see citing works
Universals and cultural differences in the judgments of facial expressions of emotion.
Ekman, P., Friesen, W. V., O'Sullivan, M., Chan, A., Diacoyanni-Tarlatzis, I., Heider, K., Krause, R., LeCompte, W. A., Pitcairn, T., Ricci-Bitti, P. E., et al. (1987) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.53.4.712 · Cited by 1,763 · see citing works
A Multimodal Database for Affect Recognition and Implicit Tagging
Soleymani, M., Lichtenauer, J., Pun, T., & Pantic, M. (2012) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2011.25 · Cited by 1,664 · see citing works
Machine learning: a review of classification and combining techniques
Kotsiantis, S. B., Zaharakis, I. D., & Pintelas, P. E. (2006) · Artificial Intelligence Review · doi:10.1007/s10462-007-9052-3 · Cited by 1,582 · see citing works
Emotion recognition based on physiological changes in music listening
Kim, J., & André, E. (2008) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.26 · Cited by 1,079 · see citing works
EEG-Based Emotion Recognition in Music Listening
Lin, Y., Wang, C., Jung, T., Wu, T.-L., Jeng, S., Duann, J., & Chen, J. (2010) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2010.2048568 · Cited by 1,022 · see citing works
Automatic prediction of frustration
Kapoor, A., Burleson, W., & Picard, R. W. (2007) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2007.02.003 · Cited by 592 · see citing works
ReliefF-Based EEG Sensor Selection Methods for Emotion Recognition
Zhang, J., Chen, M., Zhao, S., Hu, S., Shi, Z., & Cao, Y. (2016) · Sensors · doi:10.3390/s16101558 · Cited by 177 · see citing works
Emotion Modelling for Social Robotics Applications: A Review
Cavallo, F., Semeraro, F., Fiorini, L., Magyar, G., Sinčák, P., & Dario, P. (2018) · Journal of Bionic Engineering · doi:10.1007/s42235-018-0015-y · Cited by 139 · see citing works
Balters, S., & Steinert, M. (2015) · Journal of Intelligent Manufacturing · doi:10.1007/s10845-015-1145-2 · Cited by 114 · see citing works
Using physiological signals for emotion recognition
Szwoch, W. (2013) · Conference on Human System Interactions · doi:10.1109/hsi.2013.6577880 · Cited by 109 · see citing works
Emotion unfolded by motion: a role for parietal lobe in decoding dynamic facial expressions
Sarkheil, P., Goebel, R., Schneider, F., & Mathiak, K. (2013) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nss092 · Cited by 97 · see citing works
Khezri, M., Firoozabadi, M., & Sharafat, A. R. (2015) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2015.07.006 · Cited by 89 · see citing works
Analysis of physiological for emotion recognition with the IRS model
Li, C., Xu, C., & Feng, Z. (2015) · Neurocomputing · doi:10.1016/j.neucom.2015.07.112 · Cited by 78 · see citing works
Ellen R. Grass Lecture: Extraordinary EEG
Tatum, W. O. (2014) · The Neurodiagnostic Journal · doi:10.1080/21646821.2014.11079932 · Cited by 71 · see citing works
Sun, B., Li, L., Zuo, T., Chen, Y., Zhou, G., & Wu, X. (2014) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/2663204.2666272 · Cited by 71 · see citing works
Feature-level fusion of multimodal physiological signals for emotion recognition
Chen, J., Hu, B., Li-xin, X., Moore, P. T., & Su, Y. (2015) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm.2015.7359713 · Cited by 66 · see citing works
Hand exoskeleton for rehabilitation therapies with integrated optical force sensor
Díez, J. A., Blanco, A., Catalan, J. M., et al. (2018) · Advances in Mechanical Engineering · doi:10.1177/1687814017753881 · Cited by 65 · see citing works
Partial least squares regression on grassmannian manifold for emotion recognition
Liu, M., Wang, R., Huang, Z., Shan, S., & Chen, X. (2013) · Proceedings of the 15th ACM on International conference on multimodal interaction · doi:10.1145/2522848.2531738 · Cited by 62 · see citing works
Physiological emotion analysis using support vector regression
Chang, C., Chung, P., Zheng, J.-Y., & Chung, P. (2013) · Neurocomputing · doi:10.1016/j.neucom.2013.02.041 · Cited by 49 · see citing works
Short-term analysis of heart rate variability for emotion recognition via a wearable ECG device
Guo, H. W., Huang, Y. S., Chien, J. C., & Shieh, J. S. (2015) · International Conference on Intelligent Informatics and Biomedical Sciences (ICIIBMS) · doi:10.1109/iciibms.2015.7439542 · Cited by 43 · see citing works
Wiem, M. B. H., & Lachiri, Z. (2016) · International Conference on Control, Engineering & Information Technology · doi:10.1109/ceit.2016.7929117 · Cited by 17 · see citing works
Localization of emergency acoustic sources by micro aerial vehicles
Basiri, M., Schill, F., Lima, P. U., et al. (2017) · Journal of Field Robotics · doi:10.1002/rob.21733 · Cited by 15 · see citing works
Emotion Recognition of GSR Based on an Improved Quantum Neural Network
Zhang, Q., Lai, X., & Liu, G. (2016) · International Conference on Intelligent Human-Machine Systems and Cybernetics · doi:10.1109/ihmsc.2016.66 · Cited by 15 · see citing works
Interactive Tutor Robot for Collaborative e‐Learning System
Tojo, T., Ono, O., Noh, N. B. M., et al. (2018) · Electrical Engineering in Japan · doi:10.1002/eej.23073 · Cited by 6 · see citing works
Flagged first, then by citations
Research works printed but not linked: 8
- [5]You L. Z. Zhang S. D. D Zhu L. Bed-chair integration-new developing trend of helpage assistive robot Proceedings of the 5th International Conference on Machinery, Materials Science and Engineering Applications 2016 371 376
- [16]Chen J. Chen Z. Chi Z. Fu H. Emotion recognition in the wild with feature fusion and multiple kernel learning Proceedinds of the 16th Interntional Conference on multimodal interaction 2014 508 513
- [17]Liu M. Combining multiple kernel methods on iemannian manifold for emotion recognition in the wild Proceedings of the 16th International Conference on multimodal interaction 2014 494 501
- [18]Meudt S. Zharkov D. Kächele M. Schwenker F. Multi classifier systems and forward backward feature selection algorithms to classify emotional coloured speech Proceedings of the 2013 15th ACM International Conference on Multimodal Interaction, ICMI 2013 December 2013 aus 551 556 2-s2.0-84892570615
- [24]Park M.-S. Oh H.-S. Jeong H. Sohn J.-H. EEG-based emotion recogntion during emotionally evocative films Proceedings of the 2013 International Winter Workshop on Brain-Computer Interface, BCI 2013 February 2013 kor 56 57 2-s2.0-84877692903
- [31]Ktata S. Ouni K. Ellouze N. ECG signal maxima detection using wavelet transform Proceedings of the International Symposium on Industrial Electronics 2006, ISIE 2006 July 2006 can 700 703 2-s2.0-53649106678
- [32]Das P. Khasnobish A. Tibarewala D. N. Emotion recognition employing ECG and GSR signals as markers of ANS Proceedings of the 2016 Conference on Advances in Signal Processing, CASP 2016 June 2016 ind 37 42 2-s2.0-85004097427
- [33]Ahujaet N. D. GSR and HRA: its application in clinical diagnosis Proceedings of the 16th IEEE Symposium Computer-Based Medical Systems 2003 279 283
Cited by
Sorted by citations
A comprehensive survey on multimodal medical signals fusion for smart healthcare systems
Muhammad, G., Alshehri, F., Karray, F., Saddik, A. E., Alsulaiman, M., & Falk, T. H. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.06.007 · Cited by 287 · see citing works
Survey on AI-Based Multimodal Methods for Emotion Detection
Marechal, C., Mikołajewski, D., Tyburek, K., Prokopowicz, P., Bougueroua, L., Ancourt, C., & Węgrzyn-Wolska, K. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-16272-6_11 · Cited by 137 · see citing works
- Authors
- Wei, W., Jia, Q., Feng, Y., & Chen, G.
- Affiliations
- Beijing University of Posts and Telecommunications (CN)
- Corresponding author
- Wei, Wei
- Venue
- Computational Intelligence and Neuroscience
- Publisher
- Wiley
- Published
- 2018-07-05
- Volume
- 2018
- Pages
- 1-9
- ISSN
- 1687-5265 (ISSN-L)
- 1687-5273
- PMC
- PMC6057426
- OpenAlex
- W2880583584
- Cited by
- 82
- References
- 28
- Citations by year
- 2026: 3 · 2025: 5 · 2024: 11 · 2023: 9 · 2022: 16 · 2021: 7 · 2020: 5 · 2019: 8 · 2018: 8 · 2017: 4 · 2016: 6
- FWCI
- 9.16
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Computational Intelligence and Neuroscience (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- National Natural Science Foundation of China (61573066, 61327806)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face and Expression Recognition
- Health Sciences › Medicine › Cardiology and Cardiovascular Medicine › Heart Rate Variability and Autonomic Control
- Keywords
- emotion recognition, decision fusion, weighted fusion, EEG, ECG, skin conductance, feature extraction, support vector machine
- MeSH terms
- Brain, Electrodiagnosis, Emotions, Galvanic Skin Response, Heart, Humans, Pattern Recognition, Automated, Respiration, Signal Processing, Computer-Assisted, Support Vector Machine
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
