The DOI matches its Crossref record; 56 of the 56 references it cites were found in a registry or an index.
Multimodal Emotion Recognition in Response to Videos
Open accessAbstract
This paper presents a user-independent emotion recognition method with the goal of recovering affective tags for videos using electroencephalogram (EEG), pupillary response and gaze distance. We first selected 20 video clips with extrinsic emotional content from movies and online resources. Then, EEG responses and eye gaze data were recorded from 24 participants while watching emotional video clips. Ground truth was defined based on the median arousal and valence scores given to clips in a preliminary study using an online questionnaire. Based on the participants' responses, three classes for each dimension were defined. The arousal classes were calm, medium aroused, and activated and the valence classes were unpleasant, neutral, and pleasant. One of the three affective labels of either valence or arousal was determined by classification of bodily responses. A one-participant-out cross validation was employed to investigate the classification performance in a user-independent approach. The best classification accuracies of 68.5 percent for three labels of valence and 76.4 percent for three labels of arousal were obtained using a modality fusion strategy and a support vector machine. The results over a population of 24 participants demonstrate that user-independent emotion recognition can outperform individual self-reports for arousal assessments and do not underperform for valence assessments.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LIBSVM: A library for support vector machines
Chang, C.-C., & Lin, C.-J. (2011) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/1961189.1961199 · Cited by 40,918
Development and validation of brief measures of positive and negative affect: The PANAS scales.
Watson, D., Clark, L. A., & Tellegen, A. (1988) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.54.6.1063 · Cited by 32,433
Welch, P. (1967) · IEEE Transactions on Audio and Electroacoustics · doi:10.1109/tau.1967.1161901 · Cited by 12,277
Libsvm : A library for support vector machines
Ferrari, V. (2008) · Medical Entomology and Zoology · Cited by 10,122
The Cognitive Structure of Emotions
Ortony, A., Clore, G. L., & Collins, A. (1988) · doi:10.1017/cbo9780511571299 · Cited by 6,208
(2005) · CTIT technical reports series · Cited by 5,307
What are emotions? And how can they be measured?
Scherer, K. R. (2005) · Social Science Information · doi:10.1177/0539018405058216 · Cited by 4,264
Combining Pattern Classifiers: Methods and Algorithms
Bagui, S. C. (2005) · Technometrics · doi:10.1198/tech.2005.s320 · Cited by 3,202
Looking at pictures: Affective, facial, visceral, and behavioral reactions
LANG, P. J., GREENWALD, M. K., BRADLEY, M. M., & HAMM, A. O. (1993) · Psychophysiology · doi:10.1111/j.1469-8986.1993.tb03352.x · Cited by 3,190
Combining Pattern Classifiers: Methods and Algorithms
Kuncheva, L. I. (2004) · Wiley eBooks · doi:10.1002/0471660264 · Cited by 3,005
Emotion elicitation using films
Gross, J. J., & Levenson, R. W. (1995) · Cognition & Emotion · doi:10.1080/02699939508408966 · Cited by 2,179
Subcortical and cortical brain activity during the feeling of self-generated emotions
Damasio, A. R., Grabowski, T. J., Bechara, A., Damasio, H., Ponto, L. L. B., Parvizi, J., & Hichwa, R. D. (2000) · Nature Neuroscience · doi:10.1038/79871 · Cited by 2,117
Evidence for a three-factor theory of emotions
Russell, J. A., & Mehrabian, A. (1977) · Journal of Research in Personality · doi:10.1016/0092-6566(77)90037-x · Cited by 1,940 · cited 1×
“The most famous such space, which is used in the present study and originates from cognitive theory, is the 2D valence-arousal or pleasure-arousal space [33].”
BackgroundThe pupil as a measure of emotional arousal and autonomic activation
Bradley, M. M., Miccoli, L., Escrig, M. A., & Lang, P. J. (2008) · Psychophysiology · doi:10.1111/j.1469-8986.2008.00654.x · Cited by 1,919
Probability Estimates for Multi-class Classification by Pairwise Coupling
Wu, T., Lin, C., & Weng, R. C. (2004) · Journal of Machine Learning Research · doi:10.5555/1005332.1016791 · Cited by 1,468
Culture and the categorization of emotions.
Russell, J. A. (1991) · Psychological Bulletin · doi:10.1037/0033-2909.110.3.426 · Cited by 1,344
Prefrontal Brain Asymmetry: A Biological Substrate of the Behavioral Approach and Inhibition Systems
Sutton, S. K., & Davidson, R. J. (1997) · Psychological Science · doi:10.1111/j.1467-9280.1997.tb00413.x · Cited by 1,263
Generalizing motion edits with Gaussian processes
Ikemoto, L., Arikan, O., & Forsyth, D. (2009) · ACM Transactions on Graphics · doi:10.1145/1477926.1477927 · Cited by 1,214
Platt, J. C. (2000) · Advances in Large-Margin Classifiers · doi:10.7551/mitpress/1113.003.0008 · Cited by 1,156
Emotion recognition based on physiological changes in music listening
Kim, J., & Andre, E. (2008) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.26 · Cited by 1,082
Probability Estimates for Multi-class Classification by Pairwise Coupling
Wu, T., Lin, C., Weng, R. C., & Singer, Y. (2003) · Cited by 989
Schaefer, A., Nils, F., Sanchez, X., & Philippot, P. (2010) · Cognition & Emotion · doi:10.1080/02699930903274322 · Cited by 875
Emotion Elicitation Using Films
Rottenberg, J., Ray, R. D., & Gross, J. J. (2007) · Handbook of Emotion Elicitation and Assessment · doi:10.1093/oso/9780195169157.003.0002 · Cited by 838
Pupil size variation as an indication of affective processing
Partala, T., & Surakka, V. (2003) · International Journal of Human-Computer Studies · doi:10.1016/s1071-5819(03)00017-x · Cited by 836
A systems approach to appraisal mechanisms in emotion
Sander, D., Grandjean, D., & Scherer, K. R. (2005) · Neural Networks · doi:10.1016/j.neunet.2005.03.001 · Cited by 700
Affective video content representation and modeling
Hanjalic, A., & Xu, L.-Q. (2005) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2004.840618 · Cited by 594
Affective neuroscience and psychophysiology: Toward a synthesis
Davidson, R. J. (2003) · Psychophysiology · doi:10.1111/1469-8986.00067 · Cited by 535
Dissociable neural systems for recognizing emotions
Adolphs, R., Tranel, D., & Damasio, A. R. (2003) · Brain and Cognition · doi:10.1016/s0278-2626(03)00009-5 · Cited by 485
Using Noninvasive Wearable Computers to Recognize Human Emotions from Physiological Signals
Lisetti, C. L., & Nasoz, F. (2004) · EURASIP Journal on Advances in Signal Processing · doi:10.1155/s1110865704406192 · Cited by 432
Studying the emotion-antecedent appraisal process: An expert system approach
Scherer, K. R. (1993) · Cognition & Emotion · doi:10.1080/02699939308409192 · Cited by 415
Emotion Assessment From Physiological Signals for Adaptation of Game Difficulty
Chanel, G., Rebetez, C., Bétrancourt, M., & Pun, T. (2011) · IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans · doi:10.1109/tsmca.2011.2116000 · Cited by 398
Self-citationShort-term emotion assessment in a recall paradigm
Chanel, G., Kierkels, J. J. M., Soleymani, M., & Pun, T. (2009) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2009.03.005 · Cited by 367
Self-citationAn Overview of Classifier Fusion Methods
Ruta, D., & Gabryś, B. (2000) · Bournemouth University Research Online (Bournemouth University) · Cited by 364
Aftanas, L. I., Reva, N. V., Varlamov, A. A., Pavlov, S. V., & Makhnev, V. P. (2004) · Neuroscience and Behavioral Physiology · doi:10.1023/b:neab.0000038139.39812.eb · Cited by 247
Wearable and automotive systems for affect recognition from physiology
Healey, J., & Picard, R. W. (2000) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 210
Koelstra, S., Yazdani, A., Soleymani, M., Mühl, C., Lee, J.-S., Nijholt, A., Pun, T., Ebrahimi, T., & Patras, I. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15314-3_9 · Cited by 186
Self-citationClark, K. (1968) · Cited by 176
Kolodyazhniy, V., Kreibig, S. D., Gross, J. J., Roth, W. T., & Wilhelm, F. H. (2011) · Psychophysiology · doi:10.1111/j.1469-8986.2010.01170.x · Cited by 124
Photorealistic models for pupil light reflex and iridal pattern deformation
Pamplona, V. F., Oliveira, M. M., & Baranoski, G. V. G. (2009) · ACM Transactions on Graphics · doi:10.1145/1559755.1559763 · Cited by 115
Evaluating affective interactions: Alternatives to asking what users feel
Picard, R. W., & Daily, S. B. (2005) · Cited by 107
Joho, H., Staiano, J., Sebe, N., & Jose, J. (2010) · Multimedia tools and applications · doi:10.1007/s11042-010-0632-x · Cited by 104
Using affective parameters in a content-based recommender system for images
Tkalčič, M., Burnik, U., & Košir, A. (2010) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-010-9079-z · Cited by 98
Wirth, W., & Schramm, H. (2005) · Zurich Open Repository and Archive (University of Zurich) · doi:10.5167/uzh-101679 · Cited by 91
Hippus of the pupil: Periods of slow oscillations of unknown origin
Bouma, H., & Baghuis, L. C. J. (1971) · Vision Research · doi:10.1016/0042-6989(71)90016-2 · Cited by 86
A Bayesian framework for video affective representation
Soleymani, M., Kierkels, J. J. M., Chanel, G., & Pun, T. (2009) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii.2009.5349563 · Cited by 83
Self-citationIrie, G., Satou, T., Kojima, A., Yamasaki, T., & Aizawa, K. (2010) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2010.2051871 · Cited by 79
Arapakis, I., Konstas, I., & Jose, J. M. (2009) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/1631272.1631336 · Cited by 78
AFFECTIVE CHARACTERIZATION OF MOVIE SCENES BASED ON CONTENT ANALYSIS AND PHYSIOLOGICAL CHANGES
Soleymani, M., Chanel, G., Kierkels, J. J. M., & Pun, T. (2009) · International Journal of Semantic Computing · doi:10.1142/s1793351x09000744 · Cited by 74
Self-citationImplicit human-centered tagging [Social Sciences
Pantić, M., & Vinciarelli, A. (2009) · IEEE Signal Processing Magazine · doi:10.1109/msp.2009.934186 · Cited by 72
Verbal rate, eyeblink, and content in structured psychiatric interviews.
Kanfer, F. H. (1960) · Journal of Abnormal & Social Psychology · doi:10.1037/h0038933 · Cited by 71
Emotion-based music recommendation by affinity discovery from film music
Shan, M.-K., Kuo, F.-F., Chiang, M.-F., & Lee, S.-Y. (2008) · Expert Systems with Applications · doi:10.1016/j.eswa.2008.09.042 · Cited by 66
Queries and tags in affect-based multimedia retrieval
Kierkels, J. J. M., Soleymani, M., & Pun, T. (2009) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2009.5202772 · Cited by 49
Self-citationRecognition of multiple drivers’ emotional state
Wang, J., & Gong, Y. (2008) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2008.4761904 · Cited by 36
Cost-Effective Solution to Synchronized Audio-Visual Capture Using Multiple Sensors
Lichtenauer, J. F., Valstar, M., Shen, J., & Pantić, M. (2009) · IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS) · doi:10.1109/avss.2009.92 · Cited by 24
A collaborative personalized affective video retrieval system
Soleymani, M., Davis, J., & Pun, T. (2009) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii.2009.5349526 · Cited by 19
Self-citationMonitoring and Processing of the Pupil Diameter Signal for Affective Assessment of a Computer User
Gao, Y., Barreto, A. B., & Adjouadi, M. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-02574-7_6 · Cited by 9
Cited by
Multimodal Machine Learning: A Survey and Taxonomy
Baltrusaitis, T., Ahuja, C., & Morency, L.-P. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2798607 · Cited by 5,018
A review of affective computing: From unimodal analysis to multimodal fusion
Poria, S., Cambria, E., Bajpai, R., & Hussain, A. (2017) · Information Fusion · doi:10.1016/j.inffus.2017.02.003 · Cited by 1,644
EmotionMeter: A Multimodal Framework for Recognizing Human Emotions
Zheng, W.-L., Liu, W., Lu, Y., Lu, B.-L., & Cichocki, A. (2019) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2018.2797176 · Cited by 1,276
Emotions Recognition Using EEG Signals: A Survey
Alarcao, S. M., & Fonseca, M. J. (2019) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2017.2714671 · Cited by 1,201
Multi-sensor fusion in body sensor networks: State-of-the-art and research challenges
Gravina, R., Alinia, P., Ghasemzadeh, H., & Fortino, G. (2017) · Information Fusion · doi:10.1016/j.inffus.2016.09.005 · Cited by 899
Metadata
- Authors
- Soleymani, M., Pantic, M., & Pun, T.
- Venue
- IEEE Transactions on Affective Computing
- Published
- 2012-04
- Volume
- 3
- Issue
- 2
- Pages
- 211-223
- ISSN
- 1949-3045 (ISSN-L)
- OpenAlex
- W2164699598
- Cited by
- 722
- References
- 56
- FWCI
- 23.33
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
