The DOI matches its Crossref record; 44 of the 44 references it cites were found in a registry or an index.
Analysis of EEG Signals and Facial Expressions for Continuous Emotion Detection
Open accessAbstract
Emotions are time varying affective phenomena that are elicited as a result of stimuli. Videos and movies in particular are made to elicit emotions in their audiences. Detecting the viewers' emotions instantaneously can be used to find the emotional traces of videos. In this paper, we present our approach in instantaneously detecting the emotions of video viewers' emotions from electroencephalogram (EEG) signals and facial expressions. A set of emotion inducing videos were shown to participants while their facial expressions and physiological responses were recorded. The expressed valence (negative to positive emotions) in the videos of participants' faces were annotated by five annotators. The stimuli videos were also continuously annotated on valence and arousal dimensions. Long-short-term-memory recurrent neural networks (LSTM-RNN) and continuous conditional random fields (CCRF) were utilized in detecting emotions automatically and continuously. We found the results from facial expressions to be superior to the results from EEG signals. We analyzed the effect of the contamination of facial muscle activities on EEG signals and found that most of the emotionally valuable content in EEG features are as a result of this contamination. However, our statistical analysis showed that EEG signals still carry complementary information in presence of facial expressions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Investigating causal relations by econometric models and cross-spectral methods
Granger, C. W. J. (1969) · Econometrica · doi:10.2307/1912791 · Cited by 22,989
Measuring emotion: The self-assessment manikin and the semantic differential
Bradley, M. M., & Lang, P. J. (1994) · Journal of Behavior Therapy and Experimental Psychiatry · doi:10.1016/0005-7916(94)90063-9 · Cited by 9,538
DEAP: A Database for Emotion Analysis ;Using Physiological Signals
Koelstra, S., Muhl, C., Soleymani, M., Lee, J.-S., Yazdani, A., Ebrahimi, T., Pun, T., Nijholt, A., & Patras, I. (2012) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2011.15 · Cited by 5,093
Self-citationWhat are emotions? And how can they be measured?
Scherer, K. R. (2005) · Social Science Information · doi:10.1177/0539018405058216 · Cited by 4,264
Evidence for a three-factor theory of emotions
Russell, J. A., & Mehrabian, A. (1977) · Journal of Research in Personality · doi:10.1016/0092-6566(77)90037-x · Cited by 1,940 · cited 1×
“Valence ranges from unpleasant to pleasant; arousal ranges from calm to activated and can describe emotions intensity; and dominance ranges from not in control to dominant [8].”
BackgroundA Multimodal Database for Affect Recognition and Implicit Tagging
Soleymani, M., Lichtenauer, J., Pun, T., & Pantic, M. (2012) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2011.25 · Cited by 1,664
Self-citationCulture and the categorization of emotions.
Russell, J. A. (1991) · Psychological Bulletin · doi:10.1037/0033-2909.110.3.426 · Cited by 1,344
Schaefer, A., Nils, F., Sanchez, X., & Philippot, P. (2010) · Cognition & Emotion · doi:10.1080/02699930903274322 · Cited by 875
Emotion Elicitation Using Films
Rottenberg, J., Ray, R. D., & Gross, J. J. (2007) · Handbook of Emotion Elicitation and Assessment · doi:10.1093/oso/9780195169157.003.0002 · Cited by 838
Multimodal Emotion Recognition in Response to Videos
Soleymani, M., Pantic, M., & Pun, T. (2012) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2011.37 · Cited by 722
Self-citationEMG contamination of EEG: spectral and topographical characteristics
Goncharova, I. I., McFarland, D. J., Vaughan, T. M., & Wolpaw, J. R. (2003) · Clinical Neurophysiology · doi:10.1016/s1388-2457(03)00093-2 · Cited by 696
McKeown, G., Valstar, M., Cowie, R., Pantic, M., & Schroder, M. (2012) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2011.20 · Cited by 665
FEELTRACE: an instrument for recording perceived emotion in real time
Cowie, R., Douglas‐Cowie, E., Savvidou, S. M., & McMahon, E. R. (2000) · Research Portal (Queen's University Belfast) · Cited by 539
Affective neuroscience and psychophysiology: Toward a synthesis
Davidson, R. J. (2003) · Psychophysiology · doi:10.1111/1469-8986.00067 · Cited by 535
Nicolaou, M. A., Gunes, H., & Pantic, M. (2011) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2011.9 · Cited by 450
Gunes, H., & Schuller, B. (2013) · Image and Vision Computing · doi:10.1016/j.imavis.2012.06.016 · Cited by 360
Wöllmer, M., Eyben, F., Reiter, S., Schuller, B., Cox, C., Douglas-Cowie, E., & Cowie, R. (2008) · Interspeech 2008 · doi:10.21437/interspeech.2008-192 · Cited by 328
LIBLINEAR: A Library for Large Linear Classification
FanRong-En, ChangKai-Wei, HsiehCho-Jui, WangXiang-Rui, & LinChih-Jen (2008) · Journal of Machine Learning Research · doi:10.5555/1390681.1442794 · Cited by 328
Fusion of facial expressions and EEG for implicit affective tagging
Koelstra, S., & Patras, I. (2013) · Image and Vision Computing · doi:10.1016/j.imavis.2012.10.002 · Cited by 228
AVEC 2012: the continuous audio/visual emotion challenge
Schuller, B., Valster, M., Eyben, F., Cowie, R., & Pantic, M. (2012) · Proceedings of the 14th ACM international conference on Multimodal interaction · doi:10.1145/2388676.2388776 · Cited by 226
Self-citationHabituation of phasic and tonic components of the orienting reflex
Barry, R. J., & Sokolov, E. N. (1993) · International Journal of Psychophysiology · doi:10.1016/0167-8760(93)90093-5 · Cited by 167
Misery loves company: Mood-congruent emotional responding to music.
Hunter, P. G., Schellenberg, E. G., & Griffith, A. T. (2011) · Emotion · doi:10.1037/a0023749 · Cited by 139
Robust continuous prediction of human emotions using multiscale dynamic cues
Nicolle, J., Rapp, V., Bailly, K., Prévost, L., & Chétouani, M. (2012) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/2388676.2388783 · Cited by 133
Crowdsourcing Facial Responses to Online Videos
McDuff, D., el Kaliouby, R., & Picard, R. W. (2012) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2012.19 · Cited by 128
Dimensional affect recognition using Continuous Conditional Random Fields
Baltrušaitis, T., Banda, N., & Robinson, P. (2013) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2013.6553785 · Cited by 124
Continuous emotion detection using EEG signals and facial expressions
Soleymani, M., Asghari-Esfeden, S., Pantić, M., & Fu, Y. (2014) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2014.6890301 · Cited by 108
Self-citationOutput-associative RVM regression for dimensional and continuous emotion prediction
Nicolaou, M. A., Güneş, H., & Pantić, M. (2012) · Image and Vision Computing · doi:10.1016/j.imavis.2011.12.005 · Cited by 107
Joho, H., Staiano, J., Sebe, N., & Jose, J. (2010) · Multimedia tools and applications · doi:10.1007/s11042-010-0632-x · Cited by 104
A supervised approach to movie emotion tracking
Malandrakis, N., Potamianos, A., Evangelopoulos, G., & Zlatintsi, A. (2011) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2011.5946961 · Cited by 93
Continuous emotion detection in response to music videos
Soleymani, M., Koelstra, S., Patras, I., & Pun, T. (2011) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2011.5771352 · Cited by 86
Self-citationAFFECTIVE CHARACTERIZATION OF MOVIE SCENES BASED ON CONTENT ANALYSIS AND PHYSIOLOGICAL CHANGES
Soleymani, M., Chanel, G., Kierkels, J. J. M., & Pun, T. (2009) · International Journal of Semantic Computing · doi:10.1142/s1793351x09000744 · Cited by 74
Self-citationImplicit human-centered tagging [Social Sciences
Pantić, M., & Vinciarelli, A. (2009) · IEEE Signal Processing Magazine · doi:10.1109/msp.2009.934186 · Cited by 72
Predicting online media effectiveness based on smile responses gathered over the Internet
McDuff, D., el Kaliouby, R., Demirdjian, D., & Picard, R. W. (2013) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2013.6553750 · Cited by 72
Corpus Development for Affective Video Indexing
Soleymani, M., Larson, M., Pun, T., & Hanjalić, A. (2014) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2014.2305573 · Cited by 68
Self-citationAffective Benchmarking of Movies Based on the Physiological Responses of a Real Audience
Fleureau, J., Guillotel, P., & Orlac, I. (2013) · 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction · doi:10.1109/acii.2013.19 · Cited by 61
Analysis and Compensation of the Reaction Lag of Evaluators in Continuous Emotional Annotations
Mariooryad, S., & Busso, C. (2013) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii.2013.21 · Cited by 60
Predicting audience responses to movie content from electro-dermal activity signals
Silveira, F., Eriksson, B., Sheth, A. N., & Sheppard, A. (2013) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/2493432.2493508 · Cited by 56
Queries and tags in affect-based multimedia retrieval
Kierkels, J. J. M., Soleymani, M., & Pun, T. (2009) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2009.5202772 · Cited by 49
Self-citationCorpus Development for Affective Video Indexing
Soleymani, M., Larson, M., Pun, T., & Hanjalić, A. (2014) · Archive ouverte UNIGE (University of Geneva) · Cited by 49
Self-citationLearning a sparse codebook of facial and body microexpressions for emotion recognition
Song, Y., Morency, L., & Davis, R. (2013) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/2522848.2522851 · Cited by 37
Crowdsourcing affective responses for predicting media effectiveness
McDuff, D. (2014) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 35
Your reactions suggest you liked the movie
Bao, X., Fan, S., Varshavsky, A., Li, K., & Choudhury, R. R. (2013) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/2493432.2493440 · Cited by 35
Highlight Detection in Movie Scenes Through Inter-users, Physiological Linkage
Chênes, C., Chanel, G., Soleymani, M., & Pun, T. (2012) · Computer communications and networks · doi:10.1007/978-1-4471-4555-4_10 · Cited by 29
Self-citation
Cited by
Emotions Recognition Using EEG Signals: A Survey
Alarcao, S. M., & Fonseca, M. J. (2019) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2017.2714671 · Cited by 1,201
Katsigiannis, S., & Ramzan, N. (2018) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2017.2688239 · Cited by 1,135
Emotion recognition using multi-modal data and machine learning techniques: A tutorial and review
Zhang, J., Yin, Z., Chen, P., & Nichele, S. (2020) · Information Fusion · doi:10.1016/j.inffus.2020.01.011 · Cited by 806
A systematic review on affective computing: emotion models, databases, and recent advances
Wang, Y., Song, W., Tao, W., Liotta, A., Yang, D., Li, X., Gao, S., Sun, Y., Ge, W., Zhang, W., et al. (2022) · Information Fusion · doi:10.1016/j.inffus.2022.03.009 · Cited by 615
A survey of sentiment analysis in social media
Yue, L., Chen, W., Li, X., Zuo, W., & Yin, M. (2019) · Knowledge and Information Systems · doi:10.1007/s10115-018-1236-4 · Cited by 577
Metadata
- Authors
- Soleymani, M., Asghari-Esfeden, S., Fu, Y., & Pantic, M.
- Venue
- IEEE Transactions on Affective Computing
- Published
- 2016-01-01
- Volume
- 7
- Issue
- 1
- Pages
- 17-28
- ISSN
- 1949-3045 (ISSN-L)
- OpenAlex
- W2278113816
- Cited by
- 552
- References
- 44
- FWCI
- 51.36
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.