Person-Centered Predictions of Psychological Constructs with Social Media Contextualized by Multimodal Sensing
Open accessThe DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
Abstract
Personalized predictions have shown promises in various disciplines but they are fundamentally constrained in their ability to generalize across individuals. These models are often trained on limited datasets which do not represent the fluidity of human functioning. In contrast, generalized models capture normative behaviors between individuals but lack precision in predicting individual outcomes. This paper aims to balance the tradeoff between one-for-each and one-for-all models by clustering individuals on mutable behaviors and conducting cluster-specific predictions of psychological constructs in a multimodal sensing dataset of 754 individuals. Specifically, we situate our modeling on social media that has exhibited capability in inferring psychosocial attributes. We hypothesize that complementing social media data with offline sensor data can help to personalize and improve predictions. We cluster individuals on physical behaviors captured via Bluetooth, wearables, and smartphone sensors. We build contextualized models predicting psychological constructs trained on each cluster's social media data and compare their performance against generalized models trained on all individuals' data. The comparison reveals no difference in predicting affect and a decline in predicting cognitive ability, but an improvement in predicting personality, anxiety, and sleep quality. We construe that our approach improves predicting psychological constructs sharing theoretical associations with physical behavior. We also find how social media language associates with offline behavioral contextualization. Our work bears implications in understanding the nuanced strengths and weaknesses of personalized predictions, and how the effectiveness may vary by multiple factors. This work reveals the importance of taking a critical stance on evaluating the effectiveness before investing efforts in personalization.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Psychological Meaning of Words: LIWC and Computerized Text Analysis Methods
Tausczik, Y. R., & Pennebaker, J. W. (2010) · Journal of Language and Social Psychology · doi:10.1177/0261927x09351676 · Cited by 5,774 · cited 4×
“) that is known to be associated with less complex language [124].”
BackgroundMethodInfluentialPrivate traits and attributes are predictable from digital records of human behavior
Kosinski, M., Stillwell, D., & Graepel, T. (2013) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1218772110 · Cited by 2,829
Henson, R. K., & Roberts, J. K. (2006) · Educational and Psychological Measurement · doi:10.1177/0013164405282485 · Cited by 2,454
Soto, C. J., & John, O. P. (2017) · Journal of Personality and Social Psychology · doi:10.1037/pspp0000096 · Cited by 2,075 · cited 2×
“The enrollment process consisted of an initial survey questionnaire related to demographics (age, gender, education, type of occupation, role in the company, and income), and survey questionnaires of self-reported psychological constructs including: 1) Cognitive Ability, as assessed by the Shipley scales of Abstractio…”
MethodResultPersonality, Gender, and Age in the Language of Social Media: The Open-Vocabulary Approach
Schwartz, H. A., Eichstaedt, J. C., Kern, M. L., Dziurzynski, L., Ramones, S. M., Agrawal, M., Shah, A., Kosinski, M., Stillwell, D., Seligman, M. E. P., et al. (2013) · PLoS ONE · doi:10.1371/journal.pone.0073791 · Cited by 1,700
Reconciling modern machine-learning practice and the classical bias–variance trade-off
Belkin, M., Hsu, D., Ma, S., & Mandal, S. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1903070116 · Cited by 1,595
Brown, T. A., Chorpita, B. F., & Barlow, D. H. (1998) · Journal of Abnormal Psychology · doi:10.1037/0021-843x.107.2.179 · Cited by 1,400
Diurnal and seasonal mood vary with work, sleep, and daylength across diverse cultures
Golder, S. A., & Macy, M. W. (2011) · Science · doi:10.1126/science.1202775 · Cited by 1,155
Variable-Centered, Person-Centered, and Person-Specific Approaches: Where Theory Meets the Method
Howard, M. C., & Hoffman, M. E. (2018) · Organizational Research Methods · doi:10.1177/1094428117744021 · Cited by 1,121
Wang, R., Chen, F., Chen, Z., Li, T., Harari, G., Tignor, S., Zhou, X., Ben-Zeev, D., & Campbell, A. T. (2014) · Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing · doi:10.1145/2632048.2632054 · Cited by 1,116 · cited 5×
“The ubiquity and widespread use of smartphones and wearables allow researchers to capture longitudinal and dense human behavior at scale [130, 131].”
BackgroundMethodInfluentialDoi, Y., Minowa, M., Uchiyama, M., Okawa, M., Kim, K., Shibui, K., & Kamei, Y. (2000) · Psychiatry Research · doi:10.1016/s0165-1781(00)00232-8 · Cited by 992 · cited 1×
“The enrollment process consisted of an initial survey questionnaire related to demographics (age, gender, education, type of occupation, role in the company, and income), and survey questionnaires of self-reported psychological constructs including: 1) Cognitive Ability, as assessed by the Shipley scales of Abstractio…”
MethodMitigating bias in algorithmic hiring: evaluating claims and practices
Raghavan, M., Barocas, S., Kleinberg, J., & Levy, K. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372828 · Cited by 845
Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries
Olteanu, A., Castillo, C., Diaz, F., & Kıcıman, E. (2019) · Frontiers in Big Data · doi:10.3389/fdata.2019.00013 · Cited by 763 · cited 1×
“For instance, social media use varies across demographics and is skewed towards young adults [49], and the way social media use is distributed within the participants may not be representative of the population or social media landscape [81].”
BackgroundSocial Data: Biases, Methodological Pitfalls, and Ethical Boundaries
Olteanu, A., Castillo, C., Diaz, F., & Kiciman, E. (2016) · doi:10.2139/ssrn.2886526 · Cited by 763
Harnessing Smartphone-Based Digital Phenotyping to Enhance Behavioral and Mental Health
Onnela, J.-P., & Rauch, S. L. (2016) · Neuropsychopharmacology · doi:10.1038/npp.2016.7 · Cited by 726 · cited 1×
“Indeed, personalized modeling methods are gaining attention in the disciplines of social science, psychology, and health, including precision medicine and digital phenotyping [54, 56, 63, 82, 87].”
MethodCross cultural evidence for the fundamental features of extraversion
Lucas, R. E., Diener, E., Grob, A., Suh, E. M., & Shao, L. (2000) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.79.3.452 · Cited by 713
Canzian, L., & Musolesi, M. (2015) · Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing · doi:10.1145/2750858.2805845 · Cited by 477 · cited 1×
“For instance, significant research has focused on building models to predict depression from individuals’ social media behavior [13, 29, 67].”
BackgroundComorbidity clusters in autism spectrum disorders: an electronic health record time-series analysis
Doshi-Velez, F., Ge, Y., & Kohane, I. (2014) · Pediatrics · doi:10.1542/peds.2013-0819 · Cited by 412 · cited 1×
“Our work draws motivation from clustering based approaches previously adopted in digital phenotyping research with electronic health records to identify comorbidity of symptoms [33] and to compare and summarize clinical models [45].”
Background50 Years of Test (Un)fairness: Lessons for Machine Learning
Hutchinson, B., & Mitchell, M. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287600 · Cited by 297 · cited 2×
“Demographic attribute-based stratified modeling has been identified by the Fairness, Accountability, Transparency literature to reinforce stereotypes and existing societal biases, and even exacerbate them [53, 58, 96].”
BackgroundZinkhan, M., Berger, K., Hense, S., Nagel, M., Obst, A., Koch, B., Penzel, T., Fietze, I., Ahrens, W., Young, P., et al. (2014) · Sleep Medicine · doi:10.1016/j.sleep.2014.04.015 · Cited by 232
How age and gender affect smartphone usage
Andone, I., Błaszkiewicz, K., Eibes, M., Trendafilov, B., Montag, C., & Markowetz, A. (2016) · Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct · doi:10.1145/2968219.2971451 · Cited by 220 · cited 1×
“For instance, older adults are known to be more sedentary [12], and factors like age and gender have been shown to explain daily smartphone usage [4].”
BackgroundPREDICTING INDIVIDUAL WELL-BEING THROUGH THE LANGUAGE OF SOCIAL MEDIA
SCHWARTZ, H. A., SAP, M., KERN, M. L., EICHSTAEDT, J. C., KAPELNER, A., AGRAWAL, M., BLANCO, E., DZIURZYNSKI, L., PARK, G., STILLWELL, D., et al. (2016) · Biocomputing 2016 · doi:10.1142/9789814749411_0047 · Cited by 148 · cited 2×
“Social media is low-cost, large-scale, non-intrusive to collect, and can comprehensively reveal naturalistic patterns of mood, behavior, cognition, psychological states and social milieu [46, 66, 113].”
BackgroundKang, S.-G., Kang, J. M., Ko, K.-P., Park, S.-C., Mariani, S., & Weng, J. (2017) · Journal of Psychosomatic Research · doi:10.1016/j.jpsychores.2017.03.009 · Cited by 146
Modeling Stress with Social Media Around Incidents of Gun Violence on College Campuses
Saha, K., & De Choudhury, M. (2017) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3134727 · Cited by 124 · cited 1×
“Drawing on prior work [27, 29, 106, 107, 112], we obtain a variety of features from the Facebook data of the participants, and summarize them below.”
MethodSelf-citationWang, W., Harari, G. M., Wang, R., Müller, S. R., Mirjafari, S., Masaba, K., & Campbell, A. T. (2018) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3264951 · Cited by 100
Chung, C.-F., Wang, Q., Schroeder, J., Cole, A., Zia, J., Fogarty, J., & Munson, S. A. (2019) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3314394 · Cited by 93
Rooksby, J., Morrison, A., & Murray-Rust, D. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300655 · Cited by 87 · cited 1×
“…data which require forthcoming discussions among ubiquitous computing researchers, ethicists, and practitioners to understand and respect the individual perspectives on such use of data, which can start from those who choose to participate (or not participate) in multimodal sensing studies [100].”
BackgroundDevelopment and initial implementation of the Dynamic Assessment Treatment Algorithm (DATA)
Fernandez, K. C., Fisher, A. J., & Chi, C. (2017) · PLOS ONE · doi:10.1371/journal.pone.0178806 · Cited by 68
Clinical Data in Context: Towards Sensemaking Tools for Interpreting Personal Health Data
Raj, S., Lee, J. M., Garrity, A., & Newman, M. W. (2019) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3314409 · Cited by 67
Cited by
What is Human-Centered about Human-Centered AI? A Map of the Research Landscape
Capel, T., & Brereton, M. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580959 · Cited by 228
“…within intensive care units [148], post-operative behavior [206], assessing cognitive health [321], identifying depression through social media [239], emotion recognition [63], pregnancy [209], type 1 diabetes management [19], nutrition [103], ftness instruction [104], physical rehabilitation…”
BackgroundInfluentialUnderstanding behaviours in context using mobile sensing
Harari, G. M., & Gosling, S. D. (2023) · Nature Reviews Psychology · doi:10.1038/s44159-023-00235-3 · Cited by 56
Wang, W., Xu, W., Chander, A., Nepal, S., Buck, B., Pakhomov, S., Cohen, T., Ben-Zeev, D., & Campbell, A. (2023) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3610890 · Cited by 19
“We adopt predefined categories in LIWC that have been used in existing pieces of literature that examine mental health from languages [64, 98].”
Method
Metadata
- Authors
- Saha, K., Grover, T., Mattingly, S. M., swain, V. D., Gupta, P., Martinez, G. J., Robles-Granda, P., Mark, G., Striegel, A., & De Choudhury, M.
- Venue
- Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
- Published
- 2021-03-19
- Volume
- 5
- Issue
- 1
- Pages
- 1-32
- ISSN
- 2474-9567 (ISSN-L)
- OpenAlex
- W3134199894
- Cited by
- 43
- References
- 29
- FWCI
- 5.90
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.