The DOI matches its Crossref record; 44 of the 44 references it cites were found in a registry or an index.
Analysis of EEG Signals and Facial Expressions for Continuous Emotion Detection
Open accessAbstract
Emotions are time varying affective phenomena that are elicited as a result of stimuli. Videos and movies in particular are made to elicit emotions in their audiences. Detecting the viewers' emotions instantaneously can be used to find the emotional traces of videos. In this paper, we present our approach in instantaneously detecting the emotions of video viewers' emotions from electroencephalogram (EEG) signals and facial expressions. A set of emotion inducing videos were shown to participants while their facial expressions and physiological responses were recorded. The expressed valence (negative to positive emotions) in the videos of participants' faces were annotated by five annotators. The stimuli videos were also continuously annotated on valence and arousal dimensions. Long-short-term-memory recurrent neural networks (LSTM-RNN) and continuous conditional random fields (CCRF) were utilized in detecting emotions automatically and continuously. We found the results from facial expressions to be superior to the results from EEG signals. We analyzed the effect of the contamination of facial muscle activities on EEG signals and found that most of the emotionally valuable content in EEG features are as a result of this contamination. However, our statistical analysis showed that EEG signals still carry complementary information in presence of facial expressions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Investigating causal relations by econometric models and cross-spectral methods
Granger, C. W. J. (1969) · Econometrica · doi:10.2307/1912791 · Cited by 22,989
Measuring emotion: The self-assessment manikin and the semantic differential
Bradley, M. M., & Lang, P. J. (1994) · Journal of Behavior Therapy and Experimental Psychiatry · doi:10.1016/0005-7916(94)90063-9 · Cited by 9,538
DEAP: A Database for Emotion Analysis ;Using Physiological Signals
Koelstra, S., Muhl, C., Soleymani, M., Lee, J.-S., Yazdani, A., Ebrahimi, T., Pun, T., Nijholt, A., & Patras, I. (2012) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2011.15 · Cited by 5,093
Self-citationWhat are emotions? And how can they be measured?
Scherer, K. R. (2005) · Social Science Information · doi:10.1177/0539018405058216 · Cited by 4,264
Evidence for a three-factor theory of emotions
Russell, J. A., & Mehrabian, A. (1977) · Journal of Research in Personality · doi:10.1016/0092-6566(77)90037-x · Cited by 1,940 · cited 1×
“Valence ranges from unpleasant to pleasant; arousal ranges from calm to activated and can describe emotions intensity; and dominance ranges from not in control to dominant [8].”
BackgroundA Multimodal Database for Affect Recognition and Implicit Tagging
Soleymani, M., Lichtenauer, J., Pun, T., & Pantic, M. (2012) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2011.25 · Cited by 1,664
Self-citationCulture and the categorization of emotions.
Russell, J. A. (1991) · Psychological Bulletin · doi:10.1037/0033-2909.110.3.426 · Cited by 1,344
Schaefer, A., Nils, F., Sanchez, X., & Philippot, P. (2010) · Cognition & Emotion · doi:10.1080/02699930903274322 · Cited by 875
Emotion Elicitation Using Films
Rottenberg, J., Ray, R. D., & Gross, J. J. (2007) · Handbook of Emotion Elicitation and Assessment · doi:10.1093/oso/9780195169157.003.0002 · Cited by 838
Cited by
Emotions Recognition Using EEG Signals: A Survey
Alarcao, S. M., & Fonseca, M. J. (2019) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2017.2714671 · Cited by 1,201
“[121] 2016 x x x x x x x x 57.”
Katsigiannis, S., & Ramzan, N. (2018) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2017.2688239 · Cited by 1,135
Emotion recognition using multi-modal data and machine learning techniques: A tutorial and review
Zhang, J., Yin, Z., Chen, P., & Nichele, S. (2020) · Information Fusion · doi:10.1016/j.inffus.2020.01.011 · Cited by 806
A systematic review on affective computing: emotion models, databases, and recent advances
Wang, Y., Song, W., Tao, W., Liotta, A., Yang, D., Li, X., Gao, S., Sun, Y., Ge, W., Zhang, W., et al. (2022) · Information Fusion · doi:10.1016/j.inffus.2022.03.009 · Cited by 615
A survey of sentiment analysis in social media
Yue, L., Chen, W., Li, X., Zuo, W., & Yin, M. (2019) · Knowledge and Information Systems · doi:10.1007/s10115-018-1236-4 · Cited by 577
EEG Based Emotion Recognition by Combining Functional Connectivity Network and Local Activations
Li, P., Liu, H., Si, Y., Li, C., Li, F., Zhu, X., Huang, X., Zeng, Y., Yao, D., Zhang, Y., et al. (2019) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2019.2897651 · Cited by 396
Real-Time Movie-Induced Discrete Emotion Recognition from EEG Signals
Liu, Y.-J., Yu, M., Zhao, G., Song, J., Ge, Y., & Shi, Y. (2018) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2017.2660485 · Cited by 356
A Micro-GA Embedded PSO Feature Selection Approach to Intelligent Facial Emotion Recognition
Mistry, K., Zhang, L., Neoh, S. C., Lim, C. P., & Fielding, B. (2017) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2016.2549639 · Cited by 354
Multichannel EEG-Based Emotion Recognition via Group Sparse Canonical Correlation Analysis
Zheng, W. (2017) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2016.2587290 · Cited by 303
Al-Nafjan, A., Hosny, M., Al-Ohali, Y., & Al-Wabil, A. (2017) · Applied Sciences · doi:10.3390/app7121239 · Cited by 300
SAE+LSTM: A New Framework for Emotion Recognition From Multi-Channel EEG
Xing, X., Li, Z., Xu, T., Shu, L., Hu, B., & Xu, X. (2019) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2019.00037 · Cited by 269
Emotion detection from text and speech: a survey
Sailunaz, K., Dhaliwal, M., Rokne, J., & Alhajj, R. (2018) · Social Network Analysis and Mining · doi:10.1007/s13278-018-0505-2 · Cited by 260
Data augmentation for EEG-based emotion recognition with deep convolutional neural networks
Wang, F., Zhong, S.-H., Peng, J., Jiang, J., & Liu, Y. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-73600-6_8 · Cited by 256
Context Based Emotion Recognition Using EMOTIC Dataset
Kosti, R., Alvarez, J., Recasens, A., & Lapedriza, A. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2916866 · Cited by 211
A Review of Recurrent Neural Network-Based Methods in Computational Physiology
Mao, S., & Sejdić, E. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3145365 · Cited by 128
A multimodal emotion recognition method based on facial expressions and electroencephalography
Tan, Y., Sun, Z., Duan, F., Solé-Casals, J., & Caiafa, C. F. (2021) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2021.103029 · Cited by 88
Saffaryazdi, N., Wasim, S. T., Dileep, K., Nia, A. F., Nanayakkara, S., Broadbent, E., & Billinghurst, M. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.864047 · Cited by 69
Strength modelling for real-worldautomatic continuous affect recognition from audiovisual signals
Han, J., Zhang, Z., Cummins, N., Ringeval, F., & Schuller, B. (2017) · Image and Vision Computing · doi:10.1016/j.imavis.2016.11.020 · Cited by 53
Towards a Personalized Learning Experience Using Reinforcement Learning
Shawky, D., & Badawi, A. (2019) · Studies in Computational Intelligence · doi:10.1007/978-3-030-02357-7_8 · Cited by 45
Human Emotion Recognition With Relational Region-Level Analysis
Li, W., Dong, X., & Wang, Y. (2023) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2021.3064918 · Cited by 43
“…interaction [2], [3], health-care [4], [5], digital entertainment [6], [7], etc. Traditional human emotion recognition methods only have faces [8], [9], [10], [11], [12] or/and bodies [13], [14] as inputs, while ignoring the fact that in practice, faces and bodies never appear in isolation.”
BackgroundMultimodal Attention Network for Continuous-Time Emotion Recognition Using Video and EEG Signals
Choi, D. Y., Kim, D.-H., & Song, B. C. (2020) · IEEE Access · doi:10.1109/access.2020.3036877 · Cited by 39
“In [6], PSD features were extracted from EEG signals, and the emotion states were estimated by applying a continuous-time conditional random field (CRF) and LSTM to PSD features.”
BackgroundMethodInfluentialArtificial Intelligence Techniques for Distance Education: A Systematic Literature Review
Aljarrah, A., Ababneh, M., Karagozlu, D., & Ozdamli, F. (2021) · TEM Journal · doi:10.18421/tem104-18 · Cited by 23
Emotion Recognition from Physiological Signals using Multi-Hypergraph Neural Networks
Zhu, J., Zhao, X., Hu, H., & Gao, Y. (2019) · 2019 IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2019.00111 · Cited by 23
“Most of physiological signals employ minimally invasive sensors which are more reliable since we do not have to rely on emotional self-reports and the participants prioritize completely free body movement [9].”
BackgroundHierarchical fusion of visual and physiological signals for emotion recognition
Fang, Y., Rong, R., & Huang, J. (2021) · Multidimensional Systems and Signal Processing · doi:10.1007/s11045-021-00774-z · Cited by 17
“Some researchers treat emotion recognition as continuous problem (Soleymani et al. 2015; Sen and Sert 2018; Li et al. 2019).”
Background
Metadata
- Authors
- Soleymani, M., Asghari-Esfeden, S., Fu, Y., & Pantic, M.
- Venue
- IEEE Transactions on Affective Computing
- Published
- 2016-01-01
- Volume
- 7
- Issue
- 1
- Pages
- 17-28
- ISSN
- 1949-3045 (ISSN-L)
- OpenAlex
- W2278113816
- Cited by
- 552
- References
- 44
- FWCI
- 51.36
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.