Inference of personality traits and affect schedule by analysis of spontaneous reactions to affective videos
Open accessAbstract
This paper presents a method for inferring the Positive and Negative Affect Schedule (PANAS) and the BigFive personality traits of 35 participants through the analysis of their implicit responses to 16 emotional videos. The employed modalities to record the implicit responses are (i) EEG, (ii) peripheral physiological signals (ECG, GSR), and (iii) facial landmark trajectories. The predictions of personality traits/PANAS are done using linear regression models that are trained independently on each modality. The main findings of this study are that: (i) PANAS and personality traits of individuals can be predicted based on the users' implicit responses to affective video content, (ii) ECG+GSR signals yield 70%±8% F1-score on the distinction between extroverts/introverts, (iii) EEG signals yield 69%±6% F1-score on the distinction between creative/non creative people, and finally (iv) for the prediction of agreeableness, emotional stability, and baseline affective states we achieved significantly higher than chance-level results.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The dominant approach for personality detection is to use multimodality by combining acoustic with visual cues or physiological with visual cues.
“While there are works on detecting personality and mood from FEs only [251], [252] the dominant approach is to use multimodality either by combining acoustic with visual cues [251], [257] or physiological with visual cues [258].”
From Survey on RGB, 3D, Thermal, and Multimodal Approaches for Facial Expression Recognition: History, Trends, and Affect-Related Applications (2016) · cites this paper as [258]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Histograms of Oriented Gradients for Human Detection
Dalal, N., & Triggs, B. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.177 · Cited by 33,584 · see citing works
Development and validation of brief measures of positive and negative affect: The PANAS scales.
Watson, D., Clark, L. A., & Tellegen, A. (1988) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.54.6.1063 · Cited by 32,433 · see citing works
Measuring emotion: The self-assessment manikin and the semantic differential
Bradley, M. M., & Lang, P. J. (1994) · Journal of Behavior Therapy and Experimental Psychiatry · doi:10.1016/0005-7916(94)90063-9 · Cited by 9,538 · see citing works
An Introduction to the Five‐Factor Model and Its Applications
McCrae, R. R., & John, O. P. (1992) · Journal of Personality · doi:10.1111/j.1467-6494.1992.tb00970.x · Cited by 6,835 · see citing works
Constants across cultures in the face and emotion.
Ekman, P., & Friesen, W. V. (1971) · Journal of Personality and Social Psychology · doi:10.1037/h0030377 · Cited by 5,509 · see citing works
DEAP: A Database for Emotion Analysis ;Using Physiological Signals
Koelstra, S., Muhl, C., Soleymani, M., Lee, J.-S., Yazdani, A., Ebrahimi, T., Pun, T., Nijholt, A., & Patras, I. (2012) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2011.15 · Cited by 5,093 · see citing works
Self-citationThe PANAS-X: Manual for the Positive and Negative Affect Schedule - Expanded Form
Watson, D., & Clark, L. A. (1994) · doi:10.17077/48vt-m4t2 · Cited by 3,697 · see citing works
A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions
Zeng, Z., Pantic, M., Roisman, G. I., & Huang, T. S. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.52 · Cited by 2,797 · see citing works
Supervised Descent Method and Its Applications to Face Alignment
Xiong, X., & la Torre, F. D. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.75 · Cited by 1,917 · see citing works
A Multimodal Database for Affect Recognition and Implicit Tagging
Soleymani, M., Lichtenauer, J., Pun, T., & Pantic, M. (2012) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2011.25 · Cited by 1,664 · see citing works
Emotion recognition based on physiological changes in music listening
Kim, J., & André, E. (2008) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.26 · Cited by 1,079 · see citing works
Using linguistic cues for the automatic recognition of personality in conversation and text
Mairesse, F., Walker, M. A., Mehl, M. R., & Moore, R. K. (2007) · Journal of Artificial Intelligence Research · doi:10.1613/jair.2349 · Cited by 1,046 · see citing works
Conditional Likelihood Maximisation: A Unifying Framework for Mutual Information Feature Selection
Gavin Brown, B., Pocock, A., Zhao, M., & Luján, M. (2012) · Research Explorer (The University of Manchester) · Cited by 1,031 · see citing works
Robust Face Landmark Estimation under Occlusion
Burgos-Artizzu, X. P., Perona, P., & Dollár, P. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.191 · Cited by 786 · see citing works
A Survey of Personality Computing
Vinciarelli, A., & Mohammadi, G. (2014) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2014.2330816 · Cited by 542 · see citing works
Using Noninvasive Wearable Computers to Recognize Human Emotions from Physiological Signals
Lisetti, C., & Nasoz, F. (2004) · EURASIP Journal on Advances in Signal Processing · doi:10.1155/s1110865704406192 · Cited by 432 · see citing works
Emotion recognition from physiological signals using wireless sensors for presence technologies
Nasoz, F., Alvarez, K., Lisetti, C., & Finkelstein, N. (2004) · Cognition Technology & Work · doi:10.1007/s10111-003-0143-x · Cited by 369 · see citing works
DECAF: MEG-Based Multimodal Database for Decoding Affective Physiological Responses
Abadi, M. K., Subramanian, R., Kia, S. M., Avesani, P., Patras, I., & Sebe, N. (2015) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2015.2392932 · Cited by 367 · see citing works
Self-citationAffective understanding in film
Wang, H. L., & Cheong, L. (2006) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2006.873781 · Cited by 267 · see citing works
Fusion of facial expressions and EEG for implicit affective tagging
Koelstra, S., & Patras, I. (2012) · Image and Vision Computing · doi:10.1016/j.imavis.2012.10.002 · Cited by 228 · see citing works
Self-citationMULTIMODAL EMOTION RECOGNITION
Sebe, N., Cohen, I., & Huang, T. S. (2005) · Handbook of Pattern Recognition and Computer Vision · doi:10.1142/9789812775320_0021 · Cited by 201 · see citing works
Self-citationPersonality and the EEG: Arousal and emotional arousability
Stenberg, G. (1992) · Personality and Individual Differences · doi:10.1016/0191-8869(92)90025-k · Cited by 142 · see citing works
Personality modulates the effects of emotional arousal and valence on brain activation
Kehoe, E. G., Toomey, J. M., Balsters, J. H., & Bokde, A. L. W. (2011) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsr059 · Cited by 117 · see citing works
Joho, H., Staiano, J., Sebe, N., & Jose, J. (2010) · Multimedia tools and applications · doi:10.1007/s11042-010-0632-x · Cited by 104 · see citing works
Self-citationClarifying the relation between neuroticism and positive emotions
Ng, S. H. (2009) · Personality and Individual Differences · doi:10.1016/j.paid.2009.01.049 · Cited by 72 · see citing works
Stough, C. K. K., Donaldson, C., Scarlata, B., & Ciorciari, J. (2001) · International Journal of Psychophysiology · doi:10.1016/s0167-8760(00)00176-8 · Cited by 50 · see citing works
Perugini, M. L. L., & Blas, L. D. (2002) · BOA (University of Milano-Bicocca) · Cited by 45 · see citing works
Don't ask me what i'm like, just watch and listen
Srivastava, R., Feng, J., Roy, S., Yan, S., & Sim, T. (2012) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2393347.2393397 · Cited by 21 · see citing works
Gilbert, B. O. (1991) · Personality and Individual Differences · doi:10.1016/0191-8869(91)90208-s · Cited by 19 · see citing works
Face Sketch Landmarks Localization in the Wild
Yang, H., Zou, C., & Patras, I. (2014) · IEEE Signal Processing Letters · doi:10.1109/lsp.2014.2333544 · Cited by 18 · see citing works
Self-citation
Flagged first, then by citations
3 of the 33 references couldn’t be matched to a known paper.
Research works printed but not linked: 3
- F. Nasoz, K. Alvarez, C. L. Lisetti, and N. Finkelstein. Emotion Recognition from Physiological Signals for Presence Technologies. Int. J of Cog., Tech., and Work, 6(1), 2003.
- W. Ng. Personality and Individual Differences, 47(1):69-72, 2009.
- A. Strehl and J. Ghosh. Cluster ensembles-a knowledge reuse framework for combining multiple partitions. The Journal of Machine Learning Research, 3:583-617, 2003.
Cited by
Sorted by citations
AMIGOS: A Dataset for Affect, Personality and Mood Research on Individuals and Groups
Miranda-Correa, J. A., Abadi, M. K., Sebe, N., & Patras, I. (2018) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2018.2884461 · Cited by 740 · see citing works
Self-citationASCERTAIN: Emotion and Personality Recognition Using Commercial Sensors
Subramanian, R., Wache, J., Abadi, M. K., Vieriu, R. L., WINKLER, S. S., & Sebe, N. (2016) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2016.2625250 · Cited by 536 · see citing works
Self-citationCorneanu, C. A., Simon, M. O., Cohn, J. F., & Guerrero, S. E. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2515606 · Cited by 527 · see citing works
ChaLearn LAP 2016: First Round Challenge on First Impressions - Dataset and Results
Ponce-López, V., Chen, B., Oliu, M., Corneanu, C. A., Clapés, A., Guyon, I., Baró, X., Escalante, H. J., & Escalera, S. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-49409-8_32 · Cited by 215 · see citing works
Detecting Personality Traits Using Eye-Tracking Data
Berkovsky, S., Taib, R., Koprinska, I., Wang, E., Zeng, Y., Li, J., & Kleitman, S. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300451 · Cited by 131 · see citing works
Nonlinear Regression via Deep Negative Correlation Learning
Zhang, L., Shi, Z., Cheng, M., Liu, Y., Bian, J.-W., Zhou, J. T., Guoyan Zheng, A., & Zeng, Z. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2943860 · Cited by 129 · see citing works
Multimodal Human-Human-Robot Interactions (MHHRI) Dataset for Studying Personality and Engagement
Çeliktutan, O., Skordos, E., & Güneş, H. (2017) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2017.2737019 · Cited by 119 · see citing works
First Impressions: A Survey on Vision-Based Apparent Personality Trait Analysis
Jacques, J. C. S., Güçlütürk, Y., Pérez, M., Guclu, U., Andújar, C., Baró, X., Escalante, H. J., Guyon, I., van Gerven, M. A. J., van Lier, R., et al. (2019) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2019.2930058 · Cited by 91 · see citing works
Personalized Emotion Recognition by Personality-Aware High-Order Learning of Physiological Signals
Zhao, S., Gholaminejad, A., Ding, G., Gao, Y., Han, J., & Keutzer, K. (2019) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3233184 · Cited by 81 · see citing works
A survey of multidisciplinary domains contributing to affective computing
Arya, R., Singh, J., & Kumar, A. P. (2021) · Computer Science Review · doi:10.1016/j.cosrev.2021.100399 · Cited by 77 · see citing works
Personality-Aware Personalized Emotion Recognition from Physiological Signals
Zhao, S., Ding, G., Han, J., & Gao, Y. (2018) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/230 · Cited by 54 · see citing works
Implicit User-centric Personality Recognition Based on Physiological Responses to Emotional Videos
Wache, J., Subramanian, R., Abadi, M. K., Vieriu, R.-L., Sebe, N., & WINKLER, S. S. (2015) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/2818346.2820736 · Cited by 39 · see citing works
Self-citationMiranda-Correa, J. A., & Patras, I. (2018) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2018.00060 · Cited by 37 · see citing works
Self-citationMultimodal analysis of personality traits on videos of self-presentation and induced behavior
Giritlioğlu, D., Mandira, B., Yilmaz, S. F., Ertenli, C. U., Akgür, B. F., Kınıklıoğlu, M., Kurt, A. G., Mutlu, E., Gürel, Ş. C., & Dibeklioğlu, H. (2020) · Journal on Multimodal User Interfaces · doi:10.1007/s12193-020-00347-7 · Cited by 40 · see citing works
Jointly Predicting Job Performance, Personality, Cognitive Ability, Affect, and Well-Being
Robles-Granda, P., Lin, S., Wu, X., Martinez, G. J., Mattingly, S. M., Moskal, E., Striegel, A., Chawla, N. V., D’Mello, S. K., Gregg, J. M., et al. (2021) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2021.3061877 · Cited by 30 · see citing works
First Impressions: A Survey on Vision-Based Apparent Personality Trait Analysis
Jacques, J. C. S., Güçlütürk, Y., Pérez, M., Güçlü, U., Andújar, C., Baró, X., Escalante, H. J., Guyon, I., van Gerven, M. A. J., van Lier, R., et al. (2018) · RECERCAT (Consorci de Serveis Universitaris de Catalunya) · doi:10.48550/arxiv.1804.08046 · Cited by 26 · see citing works
Patil, V. K., Pawar, V. R., Randive, S., Bankar, R. R., Yende, D., & Patil, A. K. (2023) · Journal of Electrical Systems and Information Technology · doi:10.1186/s43067-023-00085-2 · Cited by 24 · see citing works
Puviani, L., Rama, S., & Vitetta, G. M. (2018) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2018.2887385 · Cited by 18 · see citing works
Tian, Z., Huang, D., Zhou, S., Zhao, Z.-D., & Jiang, D. (2021) · Royal Society Open Science · doi:10.1098/rsos.201976 · Cited by 18 · see citing works
A Media-Guided Attentive Graphical Network for Personality Recognition Using Physiology
Yang, H.-C., & Lee, C.-C. (2021) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2021.3090040 · Cited by 17 · see citing works
Self-citationTaib, R., Berkovsky, S., Koprinska, I., Wang, E., Zeng, Y., & Li, J. (2020) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3357459 · Cited by 16 · see citing works
Study on determining the Big-Five personality traits of an individual based on facial expressions
Gavrilescu, M. (2015) · E-Health and Bioengineering Conference (EHB) · doi:10.1109/ehb.2015.7391604 · Cited by 13 · see citing works
Boljanić, T., Miljković, N., Lazarević, L. B., Knežević, G., & Milašinović, G. (2021) · Annals of Noninvasive Electrocardiology · doi:10.1111/anec.12919 · Cited by 10 · see citing works
A Survey on Big Five Personality Traits Prediction Using Tensorflow
Nilugonda, M., & Madhavi, K. (2020) · E3S Web of Conferences · doi:10.1051/e3sconf/202018401053 · Cited by 8 · see citing works
EEG Signals based Personality and Mood Recognition using Neural Network Approach
Arya, R., & Singh, J. (2022) · International Conference on Intelligent Engineering and Management (ICIEM) · doi:10.1109/iciem54221.2022.9853089 · Cited by 6 · see citing works
Implicit Human-computer Interaction
Wache, J. (2015) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/2818346.2823311 · Cited by 6 · see citing works
Self-citationA Siamese Content-Attentive Graph Convolutional Network for Personality Recognition Using Physiology
Yang, H.-C., & Lee, C.-C. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9054226 · Cited by 6 · see citing works
Self-citationRieck, C., Penava, P., & Buettner, R. (2025) · IEEE Access · doi:10.1109/access.2025.3586005 · Cited by 5 · see citing works
Singh, J., & Arya, R. (2023) · Education and Information Technologies · doi:10.1007/s10639-023-11619-6 · Cited by 6 · see citing works
Detecting five-pattern personality traits using eye movement features for observing emotional faces
Yu, Y., Lu, Q., Wu, X., Wang, Z., Zhang, C., Wu, X., & Yan, C. (2024) · Frontiers in Psychology · doi:10.3389/fpsyg.2024.1397340 · Cited by 3 · see citing works
Big-Five Personality Traits Based on Four Main Methods
Hima, P., & Shanmugam, M. (2019) · Communications in computer and information science · doi:10.1007/978-981-13-9187-3_65 · Cited by 2 · see citing works
The Role of Face Embeddings in Classification of Personality Traits from Portrait Images
Sreevidya, P., Veni, S., & Murthy, O. V. R. (2021) · IEEE Region 10 Conference (TENCON) · doi:10.1109/tencon54134.2021.9707263 · Cited by 1 · see citing works
Guo, A., Ma, J., & Wang, K. I. (2019) · IEEE Intl Conf on Dependable, Autonomic and Secure Computing, Intl Conf on Pervasive Intelligence and Computing, Intl Conf on Cloud and Big Data Computing, Intl Conf on Cyber Science and Technology Congress (DASC/PiCom/CBDCom/CyberSciTech) · doi:10.1109/dasc/picom/cbdcom/cyberscitech.2019.00043 · Cited by 1 · see citing works
Robust Regression via Deep Negative Correlation LearningPreprint
Zhang, L., Shi, Z., Cheng, M., Liu, Y., Bian, J.-W., Zhou, J. T., Guoyan Zheng, A., & Zeng, Z. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.09066 · Cited by 1 · see citing works
Personality recognition using composite audio-video features on custom CNN architecture
Eng, Z. J. (2020) · Cited by 1 · see citing works
Gökduman, A., Dumlu, B., Müezzinoğlu, D., Koçak, Z. B., & Nergız, H. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5461785
- Authors
- Abadi, M. K., Correa, J. A. M., Wache, J., Yang, H., Patras, I., & Sebe, N.
- Affiliations
- Telecom Italia (Italy) (IT)
- University of Trento (IT)
- Queen Mary University of London (GB)
- Venue
- 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG)
- Publisher
- IEEE
- Published
- 2015-05
- Pages
- 1-8
- ISSN
- 1541-5058 (ISSN-L)
- OpenAlex
- W1583278551
- Cited by
- 36
- References
- 33
- Citations by year
- 2025: 2 · 2024: 1 · 2023: 2 · 2022: 1 · 2021: 6 · 2020: 5 · 2019: 7 · 2018: 5 · 2017: 1 · 2016: 3 · 2015: 3
- FWCI
- 5.34
- Type
- conference paper
- Language
- en
- Open access
- green
- ieeexplore.ieee.org
- Licence
- other-oa
- Locations
- International Conference on Automatic Face and Gesture Recognition (publishedVersion)
- Iris (University of Trento) (submittedVersion, other-oa)
- Funders
- Queen Mary University of London
- Consejo Nacional de Ciencia y Tecnología
- Ministero dell’Istruzione, dell’Università e della Ricerca
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Human-Computer Interaction › Gaze Tracking and Assistive Technology
- Social Sciences › Psychology › Clinical Psychology › Personality Traits and Psychology
- Keywords
- PANAS, Big Five personality traits, EEG, electrocardiogram, skin conductance, extraversion-introversion, agreeableness, emotional stability, multimodal prediction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
