Cited by 30Emotion and Mood RecognitionHeart Rate Variability and Autonomic ControlFace and Expression Recognition

Multimodal emotion recognition by combining physiological signals and facial expressions: A preliminary study

Kortelainen, J., Tiinanen, S., Xiaohua Huang, Xiaobai Li, Laukka, S., Pietikainen, M., Seppanen, T.·2012·2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society

Abstract

Lately, multimodal approaches for automatic emotion recognition have gained significant scientific interest. In this paper, emotion recognition by combining physiological signals and facial expressions was studied. Heart rate variability parameters, respiration frequency, and facial expressions were used to classify person's emotions while watching pictures with emotional content. Three classes were used for both valence and arousal. The preliminary results show that, over the proposed channels, detecting arousal seem to be easier compared to valence. While the classification performance of 54.5% was attained with arousal, only 38.0% of the samples were classified correctly in terms of valence. In future, additional modalities as well as feature selection will be utilized to improve the results.

Integrity signals

Registry

Crossref · 10.1109/embc.2012.6347175

Cites

12 of 12 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Combining heart rate variability, respiration frequency, and facial expressions yielded relatively low accuracy of 54.5%.

    “Accuracy for combining heart rate variability, respiration frequency, and facial expressions was relatively low: 54.5% [59].”

    From Survey on AI-Based Multimodal Methods for Emotion Detection (2019) · cites this paper as [59]

Cites

Flagged first, then by citations

1 of the 13 references couldn’t be matched to a known paper.

Cited by

Sorted by citations

Authors
Kortelainen, J., Tiinanen, S., Huang, X., Li, X., Laukka, S., Pietikainen, M., & Seppanen, T.
Affiliations
University of Oulu (FI)
Venue
2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society
Publisher
IEEE
Published
2012-08
Volume
2012
Pages
5238-5241
ISSN
2375-7477 (ISSN-L)
OpenAlex
W2154129660
Cited by
30
References
13
Citations by year
2026: 1 · 2024: 2 · 2022: 3 · 2021: 5 · 2020: 2 · 2019: 3 · 2017: 5 · 2016: 2 · 2015: 2 · 2014: 2 · 2013: 3
FWCI
4.33
Type
conference paper
Language
en
Open access
closed
Locations
  • Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) (publishedVersion)
  • PubMed (publishedVersion)
Topics
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Emotion and Mood Recognition
  • Health Sciences › Medicine › Cardiology and Cardiovascular Medicine › Heart Rate Variability and Autonomic Control
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face and Expression Recognition
Keywords
multimodal emotion recognition, physiological signals, facial expressions, heart rate variability, respiratory rate, valence, arousal, emotion classification, emotional pictures
MeSH terms
Algorithms, Electrocardiography, Emotions, Facial Expression, Heart Rate, Humans, Image Interpretation, Computer-Assisted, Pattern Recognition, Automated, Pilot Projects, Psychometrics, Sensitivity and Specificity, Reproducibility of Results

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review