The DOI matches its Crossref record; 55 of the 60 references it cites were found in a registry or an index.
The YouTube Lens: Crowdsourced Personality Impressions and Audiovisual Analysis of Vlogs
Open accessAbstract
Despite an increasing interest in understanding human perception in social media through the automatic analysis of users' personality, existing attempts have explored user profiles and text blog data only. We approach the study of personality impressions in social media from the novel perspective of crowdsourced impressions, social attention, and audiovisual behavioral analysis on slices of conversational vlogs extracted from YouTube. Conversational vlogs are a unique case study to understand users in social media, as vloggers implicitly or explicitly share information about themselves that words, either written or spoken cannot convey. In addition, research in vlogs may become a fertile ground for the study of video interactions, as conversational video expands to innovative applications. In this work, we first investigate the feasibility of crowdsourcing personality impressions from vlogging as a way to obtain judgements from a variate audience that consumes social media video. Then, we explore how these personality impressions mediate the online video watching experience and relate to measures of attention in YouTube. Finally, we investigate on the use of automatic nonverbal cues as a suitable lens through which impressions are made, and we address the task of automatic prediction of vloggers' personality impressions using nonverbal cues and machine learning techniques. Our study, conducted on a dataset of 442 YouTube vlogs and 2210 annotations collected in Amazon's Mechanical Turk, provides new findings regarding the suitability of collecting personality impressions from crowdsourcing, the types of personality impressions that emerge through vlogging, their association with social attention, and the level of utilization of nonverbal cues in this particular setting. In addition, it constitutes a first attempt to address the task of automatic vlogger personality impression prediction using nonverbal cues, with promising results.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Intraclass correlations: Uses in assessing rater reliability.
Shrout, P. E., & Fleiss, J. L. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.2.420 · Cited by 24,532
Robust Real-Time Face Detection
Viola, P., & Jones, M. J. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000013087.49260.fb · Cited by 14,097
A tutorial on support vector regression
Smola, A. J., & Schölkopf, B. (2004) · Statistics and Computing · doi:10.1023/b:stco.0000035301.49549.88 · Cited by 13,723
Journal of personality and social psychology
Armstrong, J. (1988) · International Journal of Forecasting · doi:10.1016/0169-2070(88)90126-4 · Cited by 10,544
Amazon's Mechanical Turk: A New Source of Inexpensive, Yet High-Quality, Data?
Buhrmester, M., Kwang, T., & Gosling, S. D. (2011) · Perspectives on Psychological Science · doi:10.1177/1745691610393980 · Cited by 9,996
A very brief measure of the Big-Five personality domains
Gosling, S. D., Rentfrow, P. J., & Swann, W. B. (2003) · Journal of Research in Personality · doi:10.1016/s0092-6566(03)00046-1 · Cited by 8,307
An Introduction to the Five‐Factor Model and Its Applications
McCrae, R. R., & John, O. P. (1992) · Journal of Personality · doi:10.1111/j.1467-6494.1992.tb00970.x · Cited by 6,835
Journal of Statistical Software
de Leeuw, J. (2009) · Wiley Interdisciplinary Reviews Computational Statistics · doi:10.1002/wics.10 · Cited by 3,608
Robust real-time object detection.
Viola, P. A. (2001) · Cited by 2,340
Nonverbal communication in human interaction
Knapp, M. L. (1972) · Cited by 2,183
Flagged first, then by citations
5 of the 60 references couldn’t be matched to a known paper.
Research works printed but not linked: 9
- 2011, Thanks YouTube Community for Two BIG Gifts on Our Sixth Birthday!
- huberman, 2009, Social attention in the age of the web, Working Together or Apart Promoting the Next Generation of Digital Scholarship, 62
- 0, Skyecandy
- 0, Oovoo
- 2011, Shortform Video Platform VYou Reels in $3M
- 0, Videogenie
- 10.1037/0022-3514.81.1.116 · DOI · Paper page
- scherer, 1979, Social markers in speech, 147
- mason, 2010, A guide to conducting behavioral research on Amazon's mechanical turk, Social Science Research Network Working Paper Series
Cited by
Sorted by citations
Scherer, K. R., Schuller, B., & Elkins, A. (2017) · Cambridge University Press eBooks · doi:10.1017/9781316676202 · Cited by 768
“The main difference with respect to “standard” SSP approaches is the adoption of social network inspired features (e.g., number of times a video has been watched, on-line ratings, etc.) typically available in social media (Biel and Gatica-Perez, 2012; Salvagnini et al., 2012).”
A Survey of Personality Computing
Vinciarelli, A., & Mohammadi, G. (2014) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2014.2330816 · Cited by 540
Corneanu, C. A., Simon, M. O., Cohn, J. F., & Guerrero, S. E. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2515606 · Cited by 527
“Visual cues can refer to eye gaze [260], [261], frowning, head orientation, mouth fidgeting [260], primary FEs [252], [253] or characteristics of”
BackgroundComputational personality recognition in social media
Farnadi, G., Sitaraman, G., Sushmita, S., Celli, F., Kosiński, M., Stillwell, D. J., Davalos, S. V., Moens, M., & Cock, M. D. (2016) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-016-9171-0 · Cited by 224
Lab And/Or Field? Measuring Personality Processes and Their Social Consequences
Wrzus, C., & Mehl, M. R. (2015) · European Journal of Personality · doi:10.1002/per.1986 · Cited by 200
“Biel and Gatica-Perez (2013) recently explored crowdsourcing personality perceptions via Amazon’s Mechanical Turk.”
BackgroundHire me: Computational Inference of Hirability in Employment Interviews Based on Nonverbal Behavior
Nguyen, L. S., Frauendorfer, D., Mast, M. S., & Gática-Pérez, D. (2014) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2014.2307169 · Cited by 186
“machine learning algorithms (including hidden Markov models [44], probabilistic graphical models [44], support vector machines [10], [8], or topic models [26]) are trained and used”
BackgroundBuettner, R. (2016) · Electronic Markets · doi:10.1007/s12525-016-0228-z · Cited by 135
“…online social networks such as Facebook (Kao and Craigie 2014; Kern et al. 2014), Twitter (Gou et al. 2014; Mohammad and Kiritchenko 2015), YouTube (Biel and Gatica-Perez 2013; Aran et al. 2014), MySpace (Muscanell and Guadagno 2012; Balmaceda et al. 2014), Renren (Yu and Wu 2010; Wang et al.…”
BackgroundHuman and Computer Personality Prediction From Digital Footprints
Hinds, J. M., & Joinson, A. (2019) · Current Directions in Psychological Science · doi:10.1177/0963721419827849 · Cited by 146
Accuracy of judging personality
Back, M., & Nestler, S. (2016) · Cambridge University Press eBooks · doi:10.1017/cbo9781316181959.005 · Cited by 87
Automatic Personality and Interaction Style Recognition from Facebook Profile Pictures
Celli, F., Bruni, E., & Lepri, B. (2014) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2647868.2654977 · Cited by 110
User Profiling through Deep Multimodal Fusion
Farnadi, G., Tang, J., Cock, M. D., & Moens, M. (2018) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3159652.3159691 · Cited by 112
“In [3], Biel and Gatica-Perez predict the person-”
BackgroundLeveraging unstructured call log data for customer churn prediction
Vo, N. N. Y., Liu, S., Li, X., & Xu, G. (2020) · Knowledge-Based Systems · doi:10.1016/j.knosys.2020.106586 · Cited by 128
FaceTube: predicting personality from facial expressions of emotion in online conversational video
Biel, J.-I., Teijeiro-Mosquera, L., & Gatica-Perez, D. (2012) · Proceedings of the 14th ACM international conference on Multimodal interaction · doi:10.1145/2388676.2388689 · Cited by 104
“Moreover, in an attempt to address the personality prediction task from automatically extracted nonverbal cues [4], the same authors found that nonverbal cues from audio and visual activity patterns seemed useful mainly to predict Extraversion [4].”
BackgroundMethodResultInfluentialSelf-citationARBEE: Towards Automated Recognition of Bodily Expression of Emotion in the Wild
Luo, Y., Ye, J., Adams, R. B., Li, J., Newman, M. G., & Wang, J. Z. (2019) · International Journal of Computer Vision · doi:10.1007/s11263-019-01215-y · Cited by 106
First Impressions: A Survey on Vision-Based Apparent Personality Trait Analysis
Jacques, J. C. S., Güçlütürk, Y., Pérez, M., Guclu, U., Andújar, C., Baró, X., Escalante, H. J., Guyon, I., van Gerven, M. A. J., van Lier, R., et al. (2019) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2019.2930058 · Cited by 91
“Results show that extraversion is the trait showing the largest activity cue utilization (reinforced in Table 2), which is related to evidences found in the literature that extraversion is typically easier to judge [54], [55].”
BackgroundMethodResultInfluentialThe Workshop on Computational Personality Recognition 2014
Celli, F., Lepri, B., Biel, J.-I., Gática-Pérez, D., Riccardi, G., & Pianesi, F. (2014) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2647868.2647870 · Cited by 90
“We provided two gold-standard labelled datasets: Transcriptions of Youtube Video Blogs (Youtube dataset) [1] and Mobile Phones interactions (Mobile dataset) [9].”
MethodSelf-citationWhat your Facebook Profile Picture Reveals about your Personality
Segalin, C., Celli, F., Polonio, L., Kosiński, M., Stillwell, D. J., Sebe, N., Cristani, M., & Lepri, B. (2017) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3123266.3123331 · Cited by 106
“ection, where for each trait we retained only those features that resulted statistically significant. This approach is simple but very effective and often used in the personality computing literature [1, 3, 60]. As for the classifier, we use a logistic regressor, where the decision rule for minimizing the error is to…”
BackgroundMethodInfluentialAran, O., & Gática-Pérez, D. (2013) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/2522848.2522859 · Cited by 87
“In [5] and [6], the authors investigated a social media domain, vlogs, for the automatic analysis of personality using external observers’ attributes.”
BackgroundResultAerts, G., Smits, T., & Verlegh, P. W. J. (2017) · Computers in Human Behavior · doi:10.1016/j.chb.2017.06.023 · Cited by 82
“This method has been used before viaMTurk to get large samples of data into a content analysis (Biel & Gatica-Perez, 2013).”
MethodGüçlütürk, Y., Güçlü, U., van Gerven, M. A. J., & van Lier, R. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-49409-8_28 · Cited by 102
“In a series of studies using YouTube video blogs (vlogs) [1,3,2,29], researchers showed that this is indeed the case.”
BackgroundKim, Y., & Kim, J. H. (2018) · Information Processing & Management · doi:10.1016/j.ipm.2018.07.005 · Cited by 80
Predicting Personality Traits using Multimodal Information
Alam, F., & Riccardi, G. (2014) · doi:10.1145/2659522.2659531 · Cited by 78
“In this study, we investigate the usefulness of di.erent feature sets using a Youtube dataset released in the Workshop on Computational Personality Recognition (Shared Task) 2014 (WCPR14).”
BackgroundMethodInfluentialYamaguchi, K., Berg, T. L., & Ortiz, L. E. (2014) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2647868.2654958 · Cited by 78
“Some very recent work has looked into visual influence on popularity or behavior [2, 6, 1, 7], or in the reverse direction, to categorize visual contents using social information [8].”
BackgroundUsing Instagram Picture Features to Predict Users’ Personality
Ferwerda, B., Schedl, M., & Tkalčič, M. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-27671-7_71 · Cited by 80
“With the increasing connectedness of people, recent research has started to focus on personality acquisition from online behavior trails, such as, video blogs (vlogs) [3, 4], Facebook behavior (e.”
BackgroundHirability in the Wild: Analysis of Online Conversational Video Resumes
Nguyen, L. S., & Gática-Pérez, D. (2016) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2016.2557058 · Cited by 76
“, video blogs) in terms of setting [10].”
BackgroundMethodResultInfluentialMultimodal First Impression Analysis with Deep Residual Networks
Güçlütürk, Y., Guclu, U., Baró, X., Escalante, H. J., Guyon, I., Escalera, S., van Gerven, M. A. J., & van Lier, R. (2017) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2017.2751469 · Cited by 82
“In a series of studies using YouTube video blogs (vlogs) [5], [41], [42], [43], researchers showed that this is indeed the case.”
BackgroundTwiSty: A Multilingual Twitter Stylometry Corpus for Gender and Personality Profiling
Verhoeven, B., Daelemans, W. M. P., & Plank, B. (2016) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/3yne5b7ve4p2 · Cited by 74
Kim, J. H., & Kim, Y. (2018) · Information Processing & Management · doi:10.1016/j.ipm.2018.10.018 · Cited by 75
Çeliktutan, O., & Güneş, H. (2015) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2015.2513401 · Cited by 73
“…extracted from appearance information such as eigenfaces and Histogram of Gradient (HoG), while geometric features were extracted based on the spatial locations of the fiducial facial points (e.g., pairwise distance between points, the spatial relationship between each point and the mean face).”
BackgroundMethodWhat Your Face Vlogs About: Expressions of Emotion and Big-Five Traits Impressions in YouTube
Teijeiro-Mosquera, L., Biel, J.-I., Alba‐Castro, J. L., & Gática-Pérez, D. (2014) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2014.2370044 · Cited by 69
“The cue sets are first used in a correlation analysis to assess the relevance of each facial expression of emotion with respect to Big-Five impressions obtained from crowd-observers watching vlogs, and also as features for automatic personality impression prediction.”
BackgroundMethodInfluentialSelf-citationHi YouTube!: personality impressions and verbal content in social video
Biel, J.-I., Tsiminaki, V., Dines, J., & Gatica-Perez, D. (2013) · Proceedings of the 15th ACM on International conference on multimodal interaction · doi:10.1145/2522848.2522877 · Cited by 66
Self-citationFeature Analysis for Computational Personality Recognition Using YouTube Personality Data set
Sarkar, C., Bhatia, S., Agarwal, A. K., & Li, J. (2014) · doi:10.1145/2659522.2659528 · Cited by 67
Knowledge of words: An interpretable approach for personality recognition from social media
Han, S., Huang, H., & Tang, Y. (2020) · Knowledge-Based Systems · doi:10.1016/j.knosys.2020.105550 · Cited by 67
Streamers: the new wave of digital entrepreneurship? Extant corpus and research agenda
Törhönen, M., Giertz, J. N., Weiger, W. H., & Hamari, J. (2020) · Electronic Commerce Research and Applications · doi:10.1016/j.elerap.2020.101027 · Cited by 63
“…primarily highlighted the development of professional strategies and practices that could further the occupational elements of this activity, such as incorporation strategies related to collaborations, cross platform use and algorithms (Arnett et al., 2019; Bishop, 2019; Koch et al., 2018).”
BackgroundInfluentialUnderstanding the Role of Community in Online Dating
Masden, C. A., & Edwards, W. K. (2015) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702417 · Cited by 69
Corpus Development for Affective Video Indexing
Soleymani, M., Larson, M., Pun, T., & Hanjalić, A. (2014) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2014.2305573 · Cited by 66
“Research involving such corpora has been treated elsewhere in the literature [18], [19], [20].”
BackgroundWhat your Facebook Profile Picture Reveals about your Personality
Segalin, C., Celli, F., Polonio, L., Kosiński, M., Stillwell, D. J., Sebe, N., Cristani, M., & Lepri, B. (2017) · CaltechAUTHORS (California Institute of Technology) · Cited by 56
Senthil Kumar, T. (2021) · Journal of Information Technology and Digital World · doi:10.36548/jitdw.2021.1.004 · Cited by 54
“2021 the participants present within the learning activities [8][9][10].”
BackgroundBroadcasting Oneself: Visual Discovery of Vlogging Styles
Aran, O., Biel, J.-I., & Gática-Pérez, D. (2013) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2013.2284893 · Cited by 53
“Previous work in the study of conversational vlogs has identified the correlates between certain individual behavioral cues of vloggers and the impressions that people make about them, and have also addressed the prediction of vlogger impressions using supervised methods [4], [5].”
BackgroundSelf-citationUnderstanding Emoji Interpretation through User Personality and Message Context
Völkel, S. T., Buschek, D., Pranjic, J., & Hußmann, H. (2019) · International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI) · doi:10.1145/3338286.3340114 · Cited by 51
“Ex-traverts were also found to gesture more often and more expan-sively [8, 42].”
BackgroundAutomated Screening of Job Candidate Based on Multimodal Video Processing
Gorbova, J., Lüsi, I., Litvin, A., & Anbarjafari, G. (2017) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.214 · Cited by 51
“The first impression database contains 10000 video-clips taken from more than 3000 different HD YouTube videos, where people are mostly sitting and speaking in English in front of the camera [8, 12].”
BackgroundOn the Effect of Observed Subject Biases in Apparent Personality Analysis From Audio-Visual Signals
Principi, R. D. P., Palmero, C., Jacques, J. C. S., & Escalera, S. (2019) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2019.2956030 · Cited by 49
“Despite the great advances of previous work [18], [19], [20] to push the research on the topic, their studies are limited to analyses performed on relatively small datasets (i.”
BackgroundCorpus Development for Affective Video Indexing
Soleymani, M., Larson, M., Pun, T., & Hanjalić, A. (2014) · Archive ouverte UNIGE (University of Geneva) · Cited by 49
Social Profiling: A Review, Taxonomy, and Challenges
Bilal, M., Gani, A., Lali, M. I. U., Marjani, M., & Malik, N. (2019) · Cyberpsychology Behavior and Social Networking · doi:10.1089/cyber.2018.0670 · Cited by 48
“The data collected from Facebook 80–84 and YouTube videos 85–87 have been used by researchers in making user personality predictions.”
Deep Personality Trait Recognition: A Survey
Zhao, X., Tang, Z., & Zhang, S. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.839619 · Cited by 55
Güçlütürk, Y., Güçlü, U., Gerven, M. A. J. V., Lier, R. J. V., Hua, G., & Jégou, H. (2016) · Radboud Repository (Radboud University) · Cited by 47
Training on the job: behavioral analysis of job interviews in hospitality
Muralidhar, S., Nguyen, L. S., Frauendorfer, D., Odobez, J., Mast, M. S., & Gática-Pérez, D. (2016) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/2993148.2993191 · Cited by 44
“Overall Visual Motion We utilize a modified version of motion energy images, called Weighted Motion Energy Images (WMEI) [6] to capture the total amount of visual movement displayed by the applicant and interviewer during the job interview.”
BackgroundInfluentialPersonality Profiling of Fictional Characters using Sense-Level Links between Lexical Resources
Flek, L., & Gurevych, I. (2015) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d15-1208 · Cited by 47
“More recent work in this area focuses on the personality prediction in social networks (Kosinski et al., 2013; Kosinski et al., 2014) and multimodal personality prediction (Biel and Gatica-Perez, 2013; Aran and Gatica-Perez, 2013).”
BackgroundA Multivariate Regression Approach to Personality Impression Recognition of Vloggers
Farnadi, G., Sushmita, S., Sitaraman, G., Ton, N., Cock, M. D., & Davalos, S. V. (2014) · doi:10.1145/2659522.2659526 · Cited by 37
“In particular, we measure the performance of 5 multivariate regression techniques [18] on a YouTube personality dataset [4].”
BackgroundMethodResultInfluentialBeyan, C., Zunino, A., Shahid, M., & Murino, V. (2019) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2019.2944614 · Cited by 37
Multimodal analysis of personality traits on videos of self-presentation and induced behavior
Giritlioğlu, D., Mandira, B., Yilmaz, S. F., Ertenli, C. U., Akgür, B. F., Kınıklıoğlu, M., Kurt, A. G., Mutlu, E., Gürel, Ş. C., & Dibeklioğlu, H. (2020) · Journal on Multimodal User Interfaces · doi:10.1007/s12193-020-00347-7 · Cited by 42
Multimodal Personality Recognition in Collaborative Goal-Oriented Tasks
Batrinca, L., Mana, N., Lepri, B., Sebe, N., & Pianesi, F. (2016) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2016.2522763 · Cited by 37
“As we have shown in Section II, several works have started to target the computational recognition of personality in different scenarios [18]–[21] and from different sources of data [31]–[35].”
BackgroundInfluentialAutomatic multimodal assessment of soft skills in social interactions: a review
Rasipuram, S., & Jayagopi, D. B. (2020) · Multimedia Tools and Applications · doi:10.1007/s11042-019-08561-6 · Cited by 36
“The dataset used in [12] was collected in the wild.”
MethodMultimodal analysis of body communication cues in employment interviews
Nguyen, L. S., Marcos-Ramiro, A., Marrón-Romera, M., & Gática-Pérez, D. (2013) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/2522848.2522860 · Cited by 36
“These include small group interactions [25], video blogs [7], or human-computer interaction [6].”
BackgroundMethodPersonality perception of robot avatar tele-operators
Bremner, P., Celiktutan, O., & Gunes, H. (2016) · 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1109/hri.2016.7451745 · Cited by 36
“To facilitate this we aim to apply automated personality classifiers to our data (such as [29], [37]), which can extract and identify useful cues automatically.”
MethodČereković, A., Aran, O., & Gática-Pérez, D. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-11839-0_1 · Cited by 34
“The visual activity of the subject is extracted by using weighted motion energy images (wMEI), which is a binary image that describes the spatial motion distribution in the video sequence [3].”
BackgroundMethodInfluentialText-based automatic personality prediction: a bibliographic review
Feizi-Derakhshi, A.-R., Feizi‐Derakhshi, M., Ramezani, M., Nikzad-Khasmakhi, N., Asgari-Chenaghlu, M., Akan, T., Ranjbar-Khadivi, M., Zafarani-Moattar, E., & Jahanbakhsh-Nagadeh, Z. (2022) · Journal of Computational Social Science · doi:10.1007/s42001-022-00178-4 · Cited by 38
“As mentioned, YouTube dataset is a multimodal, video, speech, and transcript to text [82, 83].”
MethodAnalyzing Interpersonal Empathy via Collective Impressions
Kumano, S., Otsuka, K., Mikami, D., Matsuda, M., & Yamato, J. (2015) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2015.2417561 · Cited by 32
“…contagion” between a pair of participants, and defines perceived empathy (perceived counter-empathy) as ”the impression of an observer who judges that a pair of participants engaged in a multi-party conversation is in an empathic (counter-empathic) state at that moment, where the pair are…”
BackgroundInferring mood in ubiquitous conversational video
Sánchez-Cortés, D., Biel, J.-I., Kumano, S., Yamato, J., Otsuka, K., & Gática-Pérez, D. (2013) · International Conference on Mobile and Ubiquitous Multimedia (MUM) · doi:10.1145/2541831.2541864 · Cited by 31
“[6] J.-I. Biel and D. Gatica-Perez.”
MethodInfluentialSelf-citationAutomatic assessment of communication skill in interview-based interactions
Rasipuram, S., & Jayagopi, D. B. (2018) · Multimedia Tools and Applications · doi:10.1007/s11042-018-5654-9 · Cited by 28
“Non-verbal audio cues have shown to be significant for both the contexts of interface-based interviews where no human is involved [4] and for face-to-face when a conversational partner is involved [9].”
BackgroundMethodInfluentialTrustable Co-Label Learning From Multiple Noisy Annotators
Li, S., Liu, T., Tan, J., Zeng, D., & Ge, S. (2021) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2021.3137752 · Cited by 30
INNOVATIVE PERSONALITY-BASED DIGITAL SERVICES
Buettner, R. (2016) · doi:10.13140/rg.2.1.4417.6889 · Cited by 27
First Impressions: A Survey on Vision-Based Apparent Personality Trait Analysis
Jacques, J. C. S., Güçlütürk, Y., Pérez, M., Güçlü, U., Andújar, C., Baró, X., Escalante, H. J., Guyon, I., van Gerven, M. A. J., van Lier, R., et al. (2018) · RECERCAT (Consorci de Serveis Universitaris de Catalunya) · doi:10.48550/arxiv.1804.08046 · Cited by 26
“Results show that extraversion is the trait showing the largest activity cue utilization (reinforced in Table 2), which is related to evidences found in the literature that extraversion is typically easier to judge [54], [55].”
BackgroundMethodResultInfluentialShen, Z., Elibol, A., & Chong, N. Y. (2021) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2021.103874 · Cited by 27
Mori, K., & Haruno, M. (2020) · Journal of Personality · doi:10.1111/jopy.12578 · Cited by 25
“Thus, this study extended previous research that reported personality predictions based on digital footprints including SNS (Biel & Gatica-Perez, 2013; de Montjoye et al., 2013; Guntuku et al., 2017; Kosinski et al., 2013; Quercia et al., 2011) in terms of both the range of personality traits and…”
BackgroundResultAlam, F., & Riccardi, G. (2014) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2014.6853738 · Cited by 25
“There have been significant studies on predicting personality traits from social media by analyzing text, audio and video: Twitter [14], Facebook [22,23], blog [24] and video-blog [19].”
BackgroundAutomated Prediction of Extraversion During Human–Humanoid Interaction
Anzalone, S. M., Varni, G., Ivaldi, S., & Chétouani, M. (2017) · International Journal of Social Robotics · doi:10.1007/s12369-017-0399-6 · Cited by 25
“In [11], the authors proposed the recognition of “personality states” from the analysis of nonverbal social signals in 442 Youtube video blogs.”
BackgroundLiu, Z., Rehman, A., Wu, M., Cao, W., & Hao, M. (2020) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2020.3025108 · Cited by 27
“Contrastively, accuracy decreases when fewer features are used [18], [19], hence, the search for efficient tradeoffs.”
BackgroundA Hybrid Approach for Emotion Detection in Support of Affective Interaction
Gievska, S., Koroveshovski, K., & Chavdarova, T. (2014) · IEEE ... International Conference on Data Mining workshops · doi:10.1109/icdmw.2014.130 · Cited by 25
“The set of audio-visual features provided with the dataset of 404 vlogs made available by the IDIAP Research Institute [35] was extended with: 1) coursegrain affective features, the six Ekman’s emotional categories detected by our hybrid method and represented by their 10 http://conceptnet5.media.mit.edu/ valences; an…”
MethodRomeo, M., García, D. H., Han, T., Cangelosi, A., & Jokinen, K. (2021) · Advanced Robotics · doi:10.1080/01691864.2021.1974941 · Cited by 26
“The VLOG dataset [17] consists of videos between 1 and 6min long from 469 different vloggers.”
BackgroundMethodРюмина, Е. В., Markitantov, M., Ryumin, D., & Karpov, A. A. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.122441 · Cited by 23
Sánchez-Cortés, D., Kumano, S., Otsuka, K., & Gática-Pérez, D. (2015) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/2641577 · Cited by 23
“The extraction of these nonverbal cues in social video is challenging due to the variety of content available, but has nevertheless been addressed to build computational models of vlogger personality [Biel and Gatica-Perez 2013].”
BackgroundLoud and Trendy: Crowdsourcing Impressions of Social Ambiance in Popular Indoor Urban Places
Santani, D., & Gática-Pérez, D. (2014) · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · doi:10.48550/arxiv.1403.1168 · Cited by 22
“ACM 978-1-4503-3459-4/15/10 ...$15.00. http://dx.doi.org/10.1145/2733373.2806277.”
InfluentialInvestigation of Small Group Social Interactions Using Deep Visual Activity-Based Nonverbal Features
Beyan, C., Shahid, M., & Murino, V. (2018) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3240508.3240685 · Cited by 22
“[10, 11], by using broadcasted radio clips in [44], by investigating smartphone usage of people in [14] but this review considers small group meetings only.”
BackgroundAutomatic personality prediction from audiovisual data using random forest regression
Aydin, B., Kındıroğlu, A. A., Aran, O., & Akarun, L. (2016) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2016.7899604 · Cited by 22
“used audio features such as speaking activity and prosodic cues, and visual features like looking activity, pose, and overall visual activity [20], [21].”
Method“What’s So Awesome with YouTube”: Learning Music with Social Media Celebrities
Marone, V., & Rodriguez, R. C. (2019) · Online Journal of Communication and Media Technologies · doi:10.29333/ojcmt/5955 · Cited by 31
“…these outlets have also provided opportunities to grassroots authors to become emergent social media celebrities or “micro-celebrities” (Ashraf, 2009; Biel & Gatica-Perez, 2013; Marwick, 2015) who may become famous thanks to a single video that “goes viral” (Khan & Vong, 2014), i.e., that is…”
BackgroundAI-based personality prediction for human well-being from text data: a systematic review
Singh, S., & Singh, W. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-17282-w · Cited by 23
“The cross-lingual embeddings were obtained by training the CNN model on combined MyPersonality [71] and Vloggers [72] dataset with input as bilingual word embeddings to represent text.”
Multi-domain and multi-task prediction of extraversion and leadership from meeting videos
Kındıroğlu, A. A., Akarun, L., & Aran, O. (2017) · EURASIP Journal on Image and Video Processing · doi:10.1186/s13640-017-0224-z · Cited by 19
“More details about the dataset and the annotations can be found in [8].”
BackgroundIntegrating Vision and Language for First-Impression Personality Analysis
Gorbova, J., Avots, E., Lüsi, I., Fishel, M., Escalera, S., & Anbarjafari, G. (2018) · IEEE Multimedia · doi:10.1109/mmul.2018.023121162 · Cited by 19
Hide and Seek: Choices of Virtual Backgrounds in Video Chats and Their Effects on Perception
Hwang, A. H., Wang, C. Y., Yang, Y.-Y., & Won, A. S. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3476044 · Cited by 18
Santani, D., & Gática-Pérez, D. (2015) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2733373.2806277 · Cited by 18
“ACM 978-1-4503-3459-4/15/10 ...$15.00. http://dx.doi.org/10.1145/2733373.2806277.”
InfluentialStressful first impressions in job interviews
Finnerty, A. N., Muralidhar, S., Nguyen, L. S., Pianesi, F., & Gática-Pérez, D. (2016) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/2993148.2993198 · Cited by 19
“This acts as an indicator of kinetic expressiveness [9].”
BackgroundA Face-to-Face Neural Conversation Model
Chu, H., Li, D., & Fidler, S. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00743 · Cited by 19
“We pre-train our micro-gesture module on the FirstImpression dataset [3], which contains close-up talking videos that allows high precision tracking of micro-gesture.”
MethodRadisavljević, D., Rzepka, R., & Araki, K. (2023) · Applied Sciences · doi:10.3390/app13074506 · Cited by 27
Nowson, S., & Gill, A. J. (2014) · doi:10.1145/2659522.2659530 · Cited by 17
“One of the most important individual di.erences is personality, arguably one of the central tenets of the notion of self.”
In Your Eyes: Modality Disentangling for Personality Analysis in Short Video
Sun, X., Liu, B., Ai, L., Liu, D., Meng, Q., & Cao, J. (2022) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2022.3161708 · Cited by 17
“Biel and Gatica-Perez [30] analyzed the speech, rhythm, gaze, motion, and other features contained in the video and found that the prediction result of audio-visual combined information was significantly better than pictures or audio alone.”
BackgroundFirst Impressions - Predicting User Personality from Twitter Profile Images
Dhall, A., & Hoey, J. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46843-3_10 · Cited by 17
Multichannel Attention Network for Analyzing Visual Behavior in Public Speaking
Sharma, R. A., Guha, T., & Sharma, G. (2018) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2018.00058 · Cited by 18
“al interactions. Computational effort in this domain, though relatively recent, is as diverse as analyzing couples’ interaction [2], to personality analysis [3], to video blogging style analysis [4], [5]. Public speaking is an important aspect of human communication and interaction. A good speaker is articulate, has c…”
BackgroundSafari, F., & Chalechale, A. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-023-10603-3 · Cited by 20
Stronger Together: Personality, Intelligence and the Assessment of Career Potential
Leutner, F., & Chamorro‐Premuzic, T. (2018) · Journal of Intelligence · doi:10.3390/jintelligence6040049 · Cited by 19
“Non-verbal behavioral cues work best for predicting Extraversion [101,102], whilst the remaining Big Five traits are better predicted by verbal content [103].”
BackgroundSmith, K. A., Dennis, M., Masthoff, J., & Tintarev, N. (2019) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-019-09219-6 · Cited by 17
“Biel and Gatica-Perez [12] use features from video blogs such as speaking time, speaking speed, how much the person looks at the camera.”
BackgroundMining Crowdsourced First Impressions in Online Social Video
Biel, J.-I., & Gática-Pérez, D. (2014) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2014.2346471 · Cited by 16
“We use a dataset consisting of 442 YouTube vlogs that was previously used in [8].”
InfluentialSelf-citationPredicting Learning and Engagement in Tutorial Dialogue: A Personality-Based Model
Vail, A., Grafsgaard, J. F., Wiggins, J. B., Lester, J. C., & Boyer, K. (2014) · International Conference on Multimodal Interaction · doi:10.1145/2663204.2663276 · Cited by 15
“In an expanded follow-up survey, the Extraversion trait was found to be most accurately predicted by facial indicators of emotion among all five traits [4].”
BackgroundThe Impact of Affective Verbal Content on Predicting Personality Impressions in YouTube Videos
Gievska, S., & Koroveshovski, K. (2014) · doi:10.1145/2659522.2659529 · Cited by 15
“…advances in intelligent technologies employed on the massive quantities of information generated through new means of communication raise high hopes for the potential of social computing for predictive personality modeling (Bachrach et al. 2013; Biel et al. 2013; Markovikj et al., 2013).”
BackgroundMethodInfluentialCrowdsourcing Thousands of Specialized Labels: A Bayesian Active Training Approach
Servajean, M., Joly, A., Shasha, D. E., Champ, J., & Pacitti, E. (2017) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2017.2653763 · Cited by 16
“For instance, with Amazon Mechanical Turk, each image is presented to a set of users whose task is to provide relevant tags [3], [11].”
Multimodal Mixture Density Boosting Network for Personality Mining
Vo, N. N. Y., Liu, S., He, X.-Z. '., & Xu, G. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-93034-3_51 · Cited by 15
On Text-based Personality Computing: Challenges and Future Directions
Fang, Q., Giachanou, A., Bagheri, A., Boeschoten, L., van Kesteren, E., Kamalabad, M. S., & Oberski, D. L. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.691 · Cited by 14
Facing Employers and Customers
Muralidhar, S., Siegfried, R., Odobez, J., & Gática-Pérez, D. (2018) · International Conference on Mobile and Ubiquitous Multimedia (MUM) · doi:10.1145/3282894.3282925 · Cited by 14
Automatic Assessment of Human Personality Traits: A Step Towards Intelligent Human-Robot Interaction
Zafar, Z., Paplu, S. H., & Berns, K. (2018) · IEEE-RAS International Conference on Humanoid Robots (Humanoids) · doi:10.1109/humanoids.2018.8624975 · Cited by 14
“Interestingly, studies have shown that the outcomes of both measures are quite similar [13].”
BackgroundStappen, L., Baird, A., Lienhart, M., Bätz, A., & Schuller, B. W. (2022) · Frontiers in Computer Science · doi:10.3389/fcomp.2022.773154 · Cited by 22
“, 2013; Tan and Zhang, 2019), and focusing on both long (Biel and Gatica-Perez, 2013) and short form video sharing (Cheng et al.”
Background
Metadata
- Authors
- Biel, J.-I., & Gatica-Perez, D.
- Affiliations
- École Polytechnique Fédérale de Lausanne (CH)
- Venue
- IEEE Transactions on Multimedia
- Published
- 2013-01
- Volume
- 15
- Issue
- 1
- Pages
- 41-55
- ISSN
- 1520-9210 (ISSN-L)
- 1941-0077
- OpenAlex
- W2028422222
- Cited by
- 209
- References
- 60
- Citations by year
- 2026: 6 · 2025: 10 · 2024: 7 · 2023: 9 · 2022: 16 · 2021: 20 · 2020: 16 · 2019: 20 · 2018: 18 · 2017: 24 · 2016: 23 · 2015: 13 · 2014: 20 · 2013: 6
- FWCI
- 17.14
- Type
- journal article
- Language
- en
- Open access
- green
- infoscience.epfl.ch
- Locations
- IEEE Transactions on Multimedia (publishedVersion)
- Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion)
- Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion, cc-by-nc-nd)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Computer Science Applications › Mobile Crowdsensing and Crowdsourcing
- Social Sciences › Business, Management and Accounting › Marketing › Digital Marketing and Social Media
- Keywords
- personality impressions, vlogs, YouTube, crowdsourcing, nonverbal cues, audiovisual analysis, social attention, personality prediction, conversational video, vloggers, social media perception
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
