Hi YouTube!: personality impressions and verbal content in social video
Open accessAbstract
Despite the evidence that social video conveys rich human personality information, research investigating the automatic prediction of personality impressions in vlogging has shown that, amongst the Big-Five traits, automatic nonverbal behavioral cues are useful to predict mainly the Extraversion trait. This finding, also reported in other conversational settings, indicates that personality information may be coded in other behavioral dimensions like the verbal channel, which has been less studied in multimodal interaction research. In this paper, we address the task of predicting personality impressions from vloggers based on what they say in their YouTube videos. First, we use manual transcripts of vlogs and verbal content analysis techniques to understand the ability of verbal content for the prediction of crowdsourced Big-Five personality impressions. Second, we explore the feasibility of a fully-automatic framework in which transcripts are obtained using automatic speech recognition (ASR). Our results show that the analysis of error-free verbal content is useful to predict four of the Big-Five traits, three of them better than using nonverbal cues, and that the errors caused by the ASR system decrease the performance significantly.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The dominant approach for personality detection is to use multimodality by combining acoustic with visual cues or physiological with visual cues.
“While there are works on detecting personality and mood from FEs only [251], [252] the dominant approach is to use multimodality either by combining acoustic with visual cues [251], [257] or physiological with visual cues [258].”
From Survey on RGB, 3D, Thermal, and Multimodal Approaches for Facial Expression Recognition: History, Trends, and Affect-Related Applications (2016) · cites this paper as [257]
While many studies have showed improvements of prediction when combined with physiological or acoustic cues, facial expressions remain marginal in the study of personality trait prediction.
“Practically though, while many studies have showed improvements of prediction when combined with physiological or acoustic cues, FEs remain marginal in the study of personality trait prediction [251], [257], [259], [260].”
From Survey on RGB, 3D, Thermal, and Multimodal Approaches for Facial Expression Recognition: History, Trends, and Affect-Related Applications (2016) · cites this paper as [257]
Cites
An Introduction to the Five‐Factor Model and Its Applications
McCrae, R. R., & John, O. P. (1992) · Journal of Personality · doi:10.1111/j.1467-6494.1992.tb00970.x · Cited by 6,835 · see citing works
Linguistic styles: language use as an individual difference
Pennebaker, J. W., & King, L. A. (1999) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.77.6.1296 · Cited by 1,961 · see citing works
Deep Neural Networks for Acoustic Modeling in Speech Recognition
Hinton, G. E., Deng, L., Yu, D., Dahl, G. E., Mohamed, A., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T. N., et al. (2012) · Cited by 1,839 · see citing works
Using linguistic cues for the automatic recognition of personality in conversation and text
Mairesse, F., Walker, M. A., Mehl, M. R., & Moore, R. K. (2007) · Journal of Artificial Intelligence Research · doi:10.1613/jair.2349 · Cited by 1,046 · see citing works
A room with a cue: Personality judgments based on offices and bedrooms.
Gosling, S. D., Ko, S. J., Mannarelli, T., & Morris, M. E. (2002) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.82.3.379 · Cited by 886 · see citing works
Mehl, M. R., Gosling, S. D., & Pennebaker, J. W. (2006) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.90.5.862 · Cited by 764 · see citing works
Personality in 100,000 Words: A large-scale analysis of personality and word use among bloggers
Yarkoni, T. (2010) · Journal of Research in Personality · doi:10.1016/j.jrp.2010.04.001 · Cited by 637 · see citing works
The computer expression recognition toolbox (CERT)
Littlewort, G. C., Whitehill, J., Wu, T., Fasel, I., Frank, M. G., Movellan, J. R., & Bartlett, M. S. (2011) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2011.5771414 · Cited by 552 · see citing works
Oberlander, J., & Nowson, S. (2006) · International Conference on Computational Linguistics and Annual Meeting of the Association for Computational Linguistics (COLING/ACL) · doi:10.3115/1273073.1273154 · Cited by 211 · see citing works
The YouTube Lens: Crowdsourced Personality Impressions and Audiovisual Analysis of Vlogs
Biel, J.-I., & Gatica-Perez, D. (2013) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2012.2225032 · Cited by 209 · see citing works
Self-citationText messaging, personality, and the social context
Holtgraves, T. (2011) · Journal of Research in Personality · doi:10.1016/j.jrp.2010.11.015 · Cited by 166 · see citing works
What Are They Blogging About? Personality, Topic and Motivation in Blogs
Gill, A. J., Nowson, S., & Oberlander, J. (2009) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v3i1.13949 · Cited by 144 · see citing works
Large Scale Personality Classification of Bloggers
Iacobelli, F., Gill, A. J., Nowson, S., & Oberlander, J. (2011) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-24571-8_71 · Cited by 136 · see citing works
Transcribing Meetings With the AMIDA Systems
Hain, T., Burget, L., Dines, J., et al. (2011) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2011.2163395 · Cited by 121 · see citing works
Self-citationFaceTube: predicting personality from facial expressions of emotion in online conversational video
Biel, J.-I., Teijeiro-Mosquera, L., & Gatica-Perez, D. (2012) · Proceedings of the 14th ACM international conference on Multimodal interaction · doi:10.1145/2388676.2388689 · Cited by 104 · see citing works
Self-citationPlease, tell me about yourself: automatic personality assessment using short self-presentations
Batrinca, L. M., Mana, N., Lepri, B., Pianesi, F., & Sebe, N. (2011) · Proceedings of the 13th international conference on multimodal interfaces · doi:10.1145/2070481.2070528 · Cited by 96 · see citing works
Mohammadi, G., Vinciarelli, A., & Mortillaro, M. (2010) · International Workshop on Social Signal Processing (SSPW) · doi:10.1145/1878116.1878123 · Cited by 89 · see citing works
Birds of a feather: How personality influences blog writing and reading
Li, J. J., & Chignell, M. (2010) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2010.04.001 · Cited by 86 · see citing works
Identifying more bloggers Towards large scale personality classification of personal weblogs
Nowson, S., & Oberlander, J. (2007) · Cited by 65 · see citing works
Modeling the Personality of Participants During Group Interactions
Lepri, B., Mana, N., Cappelletti, A., Pianesi, F., & Zancanaro, M. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-02247-0_13 · Cited by 46 · see citing works
Assessing the impact of language style on emergent leadership perception from ubiquitous audio
Sánchez-Cortés, D., Motlíček, P., & Gática-Pérez, D. (2012) · International Conference on Mobile and Ubiquitous Multimedia (MUM) · doi:10.1145/2406367.2406408 · Cited by 11 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Corneanu, C. A., Simon, M. O., Cohn, J. F., & Guerrero, S. E. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2515606 · Cited by 527 · see citing works
- Authors
- Biel, J.-I., Tsiminaki, V., Dines, J., & Gatica-Perez, D.
- Affiliations
- École Polytechnique Fédérale de Lausanne (CH)
- Idiap Research Institute (CH)
- Idiap Research Institute / Ecole Polytechnique Fédérale de Lausanne (EPFL), Martigny, Switzerland
- Venue
- Proceedings of the 15th ACM on International conference on multimodal interaction
- Publisher
- ACM
- Published
- 2013-12-09
- Pages
- 119-126
- OpenAlex
- W2001658220
- Cited by
- 66
- References
- 21
- Citations by year
- 2025: 2 · 2024: 4 · 2023: 2 · 2022: 4 · 2021: 11 · 2020: 7 · 2019: 5 · 2018: 8 · 2017: 9 · 2016: 4 · 2015: 3 · 2014: 7
- FWCI
- 6.35
- Type
- conference paper
- Language
- en
- Open access
- green
- infoscience.epfl.ch
- Locations
- INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION (publishedVersion)
- Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion)
- Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion, cc-by-nc-nd)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Emotion and Mood Recognition
- Social Sciences › Psychology › Clinical Psychology › Personality Traits and Psychology
- Physical Sciences › Computer Science › Artificial Intelligence › Authorship Attribution and Profiling
- Keywords
- personality prediction, Big Five personality traits, vlogs, YouTube, speech recognition, speech transcripts, nonverbal cues, multimodal interaction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
