Continuous emotion detection using EEG signals and facial expressions
Open accessThe DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index.
Abstract
Emotions play an important role in how we select and consume multimedia. Recent advances on affect detection are focused on detecting emotions continuously. In this paper, for the first time, we continuously detect valence from electroencephalogram (EEG) signals and facial expressions in response to videos. Multiple annotators provided valence levels continuously by watching the frontal facial videos of participants who watched short emotional videos. Power spectral features from EEG signals as well as facial fiducial points are used as features to detect valence levels for each frame continuously. We study the correlation between features from EEG and facial expressions with continuous valence. We have also verified our model's performance for the emotional highlight detection using emotion recognition from EEG signals. Finally the results of multimodal fusion between facial expression and EEG signals are presented. Having such models we will be able to detect spontaneous and subtle affective responses over time and use them for video highlight detection.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
DEAP: A Database for Emotion Analysis ;Using Physiological Signals
Koelstra, S., Muhl, C., Soleymani, M., Lee, J.-S., Yazdani, A., Ebrahimi, T., Pun, T., Nijholt, A., & Patras, I. (2012) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2011.15 · Cited by 5,093
Self-citationWhat are emotions? And how can they be measured?
Scherer, K. R. (2005) · Social Science Information · doi:10.1177/0539018405058216 · Cited by 4,264 · cited 1×
“Different emotional representation models, including discrete and dimensional have been proposed by psychologists [4].”
BackgroundEvidence for a three-factor theory of emotions
Russell, J. A., & Mehrabian, A. (1977) · Journal of Research in Personality · doi:10.1016/0092-6566(77)90037-x · Cited by 1,940 · cited 1×
“Arousal ranges from calm to excited/activated, and valence ranges from unpleasant to pleasant [5].”
BackgroundA Multimodal Database for Affect Recognition and Implicit Tagging
Soleymani, M., Lichtenauer, J., Pun, T., & Pantic, M. (2012) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2011.25 · Cited by 1,664
Self-citationPrefrontal Brain Asymmetry: A Biological Substrate of the Behavioral Approach and Inhibition Systems
Sutton, S. K., & Davidson, R. J. (1997) · Psychological Science · doi:10.1111/j.1467-9280.1997.tb00413.x · Cited by 1,263 · cited 1×
“In addition to power spectral features, the difference between the spectral power of all the possible symmetrical pairs on the right and left hemisphere was extracted to measure the possible asymmetry in the brain activities due to the valence of emotional stimuli [17].”
MethodMcKeown, G., Valstar, M., Cowie, R., Pantic, M., & Schroder, M. (2012) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2011.20 · Cited by 665
Affective neuroscience and psychophysiology: Toward a synthesis
Davidson, R. J. (2003) · Psychophysiology · doi:10.1111/1469-8986.00067 · Cited by 535 · cited 1×
“The spectral power of EEG signals in different bands was found to be correlated with emotions [16].”
BackgroundNicolaou, M. A., Gunes, H., & Pantic, M. (2011) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2011.9 · Cited by 450 · cited 1×
“LSTM-RNN have shown to achieve top performances in emotion recognition studies for audio-visual modalities [8, 6].”
MethodGunes, H., & Schuller, B. (2013) · Image and Vision Computing · doi:10.1016/j.imavis.2012.06.016 · Cited by 360 · cited 3×
“For a comprehensive review of continuous emotion detection, we refer the reader to [6].”
BackgroundMethod
Cited by
Zheng, W.-L., & Lu, B.-L. (2015) · IEEE Transactions on Autonomous Mental Development · doi:10.1109/tamd.2015.2431497 · Cited by 2,595
Deep learning in bioinformatics
Min, S., Lee, B., & Yoon, S. (2016) · Briefings in Bioinformatics · doi:10.1093/bib/bbw068 · Cited by 1,465
Applications of deep learning and reinforcement learning to biological data
Mahmud, M., Kaiser, M. S., Hussain, A., & Vassanelli, S. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2790388 · Cited by 765
Analysis of EEG Signals and Facial Expressions for Continuous Emotion Detection
Soleymani, M., Asghari-Esfeden, S., Fu, Y., & Pantic, M. (2016) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2015.2436926 · Cited by 552
Self-citationEEG Based Emotion Recognition by Combining Functional Connectivity Network and Local Activations
Li, P., Liu, H., Si, Y., Li, C., Li, F., Zhu, X., Huang, X., Zeng, Y., Yao, D., Zhang, Y., et al. (2019) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2019.2897651 · Cited by 396
Metadata
- Authors
- Soleymani, M., Asghari-Esfeden, S., Pantic, M., & Fu, Y.
- Venue
- 2014 IEEE International Conference on Multimedia and Expo (ICME)
- Publisher
- IEEE
- Published
- 2014-07
- Pages
- 1-6
- OpenAlex
- W1967993123
- Cited by
- 95
- References
- 10
- FWCI
- 11.48
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Continuous emotion detection using EEG signals and facial expressions. https://app.cochat.ai/papers/10.1109/icme.2014.6890301
- The paper
- Soleymani, M., Asghari-Esfeden, S., Pantic, M., & Fu, Y. (2014). Continuous emotion detection using EEG signals and facial expressions. 2014 IEEE International Conference on Multimedia and Expo (ICME). https://doi.org/10.1109/icme.2014.6890301
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.