The DOI matches its Crossref record; 74 of the 74 references it cites were found in a registry or an index.
Discovering Shifts to Suicidal Ideation from Mental Health Content in Social Media
Open accessAbstract
History of mental illness is a major factor behind suicide risk and ideation. However research efforts toward characterizing and forecasting this risk is limited due to the paucity of information regarding suicide ideation, exacerbated by the stigma of mental illness. This paper fills gaps in the literature by developing a statistical methodology to infer which individuals could undergo transitions from mental health discourse to suicidal ideation. We utilize semi-anonymous support communities on Reddit as unobtrusive data sources to infer the likelihood of these shifts. We develop language and interactional measures for this purpose, as well as a propensity score matching based statistical approach. Our approach allows us to derive distinct markers of shifts to suicidal ideation. These markers can be modeled in a prediction framework to identify individuals likely to engage in suicidal ideation in the future. We discuss societal and ethical implications of this research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The central role of the propensity score in observational studies for causal effects
ROSENBAUM, P. R., & RUBIN, D. B. (1983) · Biometrika · doi:10.1093/biomet/70.1.41 · Cited by 31,241
Normalized cuts and image segmentation
Shi, J., & Malik, J. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.868688 · Cited by 15,502
Cognitive Therapy for Depression: (675542012-001)
(2012) · PsycEXTRA Dataset · doi:10.1037/e675542012-001 · Cited by 9,663
SOME PRACTICAL GUIDANCE FOR THE IMPLEMENTATION OF PROPENSITY SCORE MATCHING
Caliendo, M., & Kopeinig, S. (2008) · Journal of Economic Surveys · doi:10.1111/j.1467-6419.2007.00527.x · Cited by 6,238
Causal Inference for Statistics, Social, and Biomedical Sciences: An Introduction
Imbens, G. W., & Rubin, D. B. (2015) · doi:10.1017/cbo9781139025751 · Cited by 4,210
Assessment of suicidal intention: The Scale for Suicide Ideation.
Beck, A. T., Kovacs, M., & Weissman, A. (1979) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.47.2.343 · Cited by 3,419
Psychological autopsy studies of suicide: a systematic review.
CAVANAGH, J. T. O., CARSON, A. J., SHARPE, M., & LAWRIE, S. M. (2003) · Psychological Medicine · doi:10.1017/s0033291702006943 · Cited by 2,207 · cited 2×
“We also acknowledge that although our dataset is relatively larger than what has been studied in the psychology literature [11], the Reddit communities are help-seeking forums.”
Causal Inference Using Potential Outcomes: Design, Modeling, Decisions
Rubin, D. B. (2005) · Journal of the American Statistical Association · doi:10.1198/016214504000001880 · Cited by 2,173 · cited 1×
“To do so, we work within the potential outcomes framework [57, 34] for causal modeling, applying stratified propensity score matching to estimate causal effects [56].”
Baumeister, R. F. (1990) · Psychological Review · doi:10.1037/0033-295x.97.1.90 · Cited by 1,919 · cited 2×
“A central challenge in public health revolves around how to identify individuals who are at risk for taking their own lives [3, 8, 66].”
Predicting Depression via Social Media
De Choudhury, M., Gamon, M., Counts, S., & Horvitz, E. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v7i1.14432 · Cited by 1,579
Self-citation
Research works printed but not linked: 9
- APA American Psychiatric Association, 1980, American Psychiatric Association, and others
- Beck Aaron T, Cognitive therapy of depression
- 10.1037/0022-006X.47.2.343 · doi:10.1037/0022-006x.47.2.343 · Paper page
- Coppersmith Glen, 2014, International Conference on Weblogs and Social Media (ICWSM).
- Choudhury Munmun De, 2014, Proc. ICWSM. AAAI.
- Choudhury Munmun De, 2013, AAAI Conference on Weblogs and Social Media.
- Cheng Qijin, 2015, Paul WC Wong, and Paul SF Yip
- 10.1145/2806416.2806594 · DOI
- Park Minsu, 2013, Proceedings of ICWSM.
Cited by
Personal Sensing: Understanding Mental Health Using Ubiquitous Sensors and Machine Learning
Mohr, D. C., Zhang, M., & Schueller, S. M. (2017) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-032816-044949 · Cited by 802
Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries
Olteanu, A., Castillo, C., Diaz, F., & Kıcıman, E. (2019) · Frontiers in Big Data · doi:10.3389/fdata.2019.00013 · Cited by 763
Self-citationSocial Data: Biases, Methodological Pitfalls, and Ethical Boundaries
Olteanu, A., Castillo, C., Diaz, F., & Kiciman, E. (2016) · doi:10.2139/ssrn.2886526 · Cited by 763
Detecting depression and mental illness on social media: an integrative review
Guntuku, S. C., Yaden, D. B., Kern, M. L., Ungar, L. H., & Eichstaedt, J. C. (2017) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2017.07.005 · Cited by 722
Methods in predictive techniques for mental health status on social media: a critical review
Chancellor, S., & De Choudhury, M. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0233-7 · Cited by 557
Natural language processing of social media as screening for suicide risk
Coppersmith, G., Leary, R., Crutchley, P., & Fine, A. (2018) · Biomedical Informatics Insights · doi:10.1177/1178222618792860 · Cited by 506
Self-citationInstagram photos reveal predictive markers of depression
Reece, A. G., & Danforth, C. M. (2017) · EPJ Data Science · doi:10.1140/epjds/s13688-017-0110-z · Cited by 519
Detection of Depression-Related Posts in Reddit Social Media Forum
Tadesse, M. M., Lin, H., Xu, B., & Yang, L. (2019) · IEEE Access · doi:10.1109/access.2019.2909180 · Cited by 463
Forecasting the onset and course of mental illness with Twitter data
Reece, A. G., Reagan, A. J., Lix, K. L. M., Dodds, P. S., Danforth, C. M., & Langer, E. J. (2017) · Scientific Reports · doi:10.1038/s41598-017-12961-9 · Cited by 401
“In addition to physical ailments, predictive screening methods have successfully identified markers in social media data for a number of mental health issues, including addiction (6), depression (712), PostTraumatic Stress Disorder (PTSD) (13,14), and suicidal ideation (15).”
BackgroundThieme, A., Belgrave, D., & Doherty, G. (2020) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3398069 · Cited by 406
Big Data in Public Health: Terminology, Machine Learning, and Privacy
Mooney, S. J., & Pejaver, V. (2018) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-040617-014208 · Cited by 325
“Supervised learning has been used to predict hospital readmission, (32; 44) tuberculosis transmission, (87) serious injuries in motor vehicle crashes (61) and Reddit users shifting towards suicidal ideation, (28) among many other applications.”
Sensitive Self-disclosures, Responses, and Social Support on Instagram: The Case of #Depression
Andalibi, N., Ozturk, P., & Forte, A. (2017) · Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing · doi:10.1145/2998181.2998243 · Cited by 323
Depression and Self-Harm Risk Assessment in Online Forums
Yates, A., Cohan, A., & Goharian, N. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1322 · Cited by 323
Data ex Machina: Introduction to Big Data
Lazer, D., & Radford, J. (2017) · Annual Review of Sociology · doi:10.1146/annurev-soc-060116-053457 · Cited by 303
Mental-LLM: Leveraging Large Language Models for Mental Health Prediction via Online Text Data
Xu, X., Yao, B., Dong, Y., Gabriel, S., Yu, H., Hendler, J., Ghassemi, M., Dey, A. K., & Wang, D. (2024) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3643540 · Cited by 278
Suicidal Ideation Detection: A Review of Machine Learning Methods and Applications
Ji, S., Pan, S., Li, X., Cambria, E., Long, G., & Huang, Z. (2021) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2020.3021467 · Cited by 257
Beyond Screen Time: Identity Development in the Digital Age
Granic, I., Morita, H., & Scholten, H. (2020) · Psychological Inquiry · doi:10.1080/1047840x.2020.1820214 · Cited by 237
“For instance, public social media text applications (e.g. Facebook, Twitter) could be used to track contamination themes as they emerge naturalistically in young people’s everyday digital discourse (e.g. De Choudhury & De, 2014; De Choudhury et al., 2016).”
BackgroundMentalBERT: Publicly Available Pretrained Language Models for Mental Healthcare
Ji, S., Zhang, T., Ansari, L., Fu, J., Tiwari, P., & Cambria, E. (2022) · Proceedings of the Language Resources and Evaluation Conference · doi:10.63317/2d7umxifj8wz · Cited by 233
Ethical research protocols for social media health research
Benton, A., Coppersmith, G., & Dredze, M. (2017) · Proceedings of the First ACL Workshop on Ethics in Natural Language Processing · doi:10.18653/v1/w17-1612 · Cited by 219
Self-citationBendig, E., Erb, B., Schulze-Thuesing, L., & Baumeister, H. (2022) · Verhaltenstherapie · doi:10.1159/000501812 · Cited by 214
The Language of Social Support in Social Media and Its Effect on Suicidal Ideation Risk
De Choudhury, M., & Kiciman, E. (2017) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v11i1.14891 · Cited by 202
“Following the method developed in (De Choudhury et al. 2016), we constructed two user classes (Table 1).”
BackgroundMethodInfluentialA Taxonomy of Ethical Tensions in Inferring Mental Health States from Social Media
Chancellor, S., Birnbaum, M. L., Caine, E. D., Silenzio, V. M. B., & De Choudhury, M. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287587 · Cited by 214
“Research has examined other disorders, such as post-traumatic stress disorder [21], anxiety [81], schizophrenia [6, 60], eating disorders [13, 14, 25], and suicidal ideation [30, 29, 39].”
BackgroundEarly Detection of Depression: Social Network Analysis and Random Forest Techniques
Cacheda, F., Fernandez, D., Novoa, F. J., & Carneiro, V. (2019) · Journal of Medical Internet Research · doi:10.2196/12554 · Cited by 206
Social Support, Reciprocity, and Anonymity in Responses to Sexual Abuse Disclosures on Social Media
Andalibi, N., Haimson, O. L., Choudhury, M. D., & Forte, A. (2018) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3234942 · Cited by 200
Self-citationPocket Skills: A Conversational Mobile Web App To Support Dialectical Behavioral Therapy
Schroeder, J., Wilkes, C., Rowan, K., Toledo, A., Paradiso, A., Czerwinski, M., Mark, G., & Linehan, M. M. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173972 · Cited by 175
Gender and Cross-Cultural Differences in Social Media Disclosures of Mental Illness
De Choudhury, M., Sharma, S. S., Logar, T., Eekhout, W., & Nielsen, R. C. (2017) · Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing · doi:10.1145/2998181.2998220 · Cited by 178
Psychosocial Effects of the COVID-19 Pandemic: Large-scale Quasi-Experimental Study on Social Media
Saha, K., Torous, J., Caine, E. D., & De Choudhury, M. (2020) · Journal of Medical Internet Research · doi:10.2196/22600 · Cited by 166
Seabrook, E. M., Kern, M. L., Fulcher, B. D., & Rickard, N. S. (2018) · Journal of Medical Internet Research · doi:10.2196/jmir.9267 · Cited by 144
“Using the social media platform Reddit, De Choudhury et al [46] considered transitions from mental health subreddits only to also using a suicide support subreddit.”
MethodMultimodal mental health analysis in social media
Yazdavar, A. H., Mahdavinejad, M. S., Bajaj, G., Romine, W., Sheth, A., Monadjemi, A. H., Thirunarayan, K., Meddar, J. M., Myers, A., Pathak, J., et al. (2020) · PLOS ONE · doi:10.1371/journal.pone.0226248 · Cited by 133
“% Users Faces_Found_ in_Profile % Users Faces_Found_ in_Media [11,19) [19,23) [23,34) [34,46) [46,60) [11,19) [19,23) [23,34) [34,46) [46,60) Control 4.”
BackgroundA Social Media Based Index of Mental Well-Being in College Campuses
Bagroy, S., Kumaraguru, P., & De Choudhury, M. (2017) · Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025909 · Cited by 115
Self-citationFrom Treatment to Healing:Envisioning a Decolonial Digital Mental Health
Pendse, S. R., Nkemelu, D., Bidwell, N. J., Jadhav, S., Pathare, S., De Choudhury, M., & Kumar, N. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501982 · Cited by 115
Self-citationSuicide Risk Assessment with Multi-level Dual-Context Language and
Matero, M., Idnani, A., Son, Y., Giorgi, S., Vu, H., Zamani, M., Limbachiya, P., Guntuku, S. C., & Schwartz, H. A. (2019) · Proceedings of the Sixth Workshop on Computational Linguistics and Clinical Psychology · doi:10.18653/v1/w19-3005 · Cited by 102
“With deaths by suicide increasing substantially (Curtin et al., 2016), researchers are turning to automated analysis of user generated content to potentially provide methods for early detection of suicide risk severity (Copper-smith et al., 2018; De Choudhury et al., 2016; Shing et al., 2018).”
MethodGooding, P., & Kariotis, T. (2021) · JMIR Mental Health · doi:10.2196/24668 · Cited by 100
“In terms of explicit reference to ethics, 10 studies included a specific section on the ethical issues raised by their work [64,73,80,87,92,98,99,105,163,187] and 10 others included a broad re fe rence to key e th ica l i s sues [64,70,122,124,126,133,140,152,184].”
BackgroundInfluentialA Social Media Study on the Effects of Psychiatric Medication Use
Saha, K., Sugar, B., Torous, J., Abrahao, B., Kıcıman, E., & De Choudhury, M. (2019) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v13i01.3242 · Cited by 96
Self-citationA computational study of mental health awareness campaigns on social media
Saha, K., Torous, J., Ernala, S. K., Rizuto, C., Stafford, A., & De Choudhury, M. (2019) · Translational Behavioral Medicine · doi:10.1093/tbm/ibz028 · Cited by 93
Conway, M., Hu, M., & Chapman, W. W. (2019) · Yearbook of Medical Informatics · doi:10.1055/s-0039-1677918 · Cited by 91
Feedpal: Understanding Opportunities for Chatbots in Breastfeeding Education of Women in India
Yadav, D., Malik, P., Dabas, K., & Singh, P. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359272 · Cited by 90
Beyond self-reflection: introducing the concept of rumination in personal informatics
Eikey, E. V., Caldeira, C. M., Figueiredo, M. C., Chen, Y., Borelli, J. L., Mazmanian, M., & Zheng, K. (2021) · Personal and Ubiquitous Computing · doi:10.1007/s00779-021-01573-w · Cited by 82
Women (Still) Ask For Less: Gender Differences in Hourly Rate in an Online Labor Marketplace
Foong, E., Vincent, N., Hecht, B., & Gerber, E. M. (2018) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3274322 · Cited by 80
Benefits and Harms of Large Language Models in Digital Mental Health
De Choudhury, M., Pendse, S. R., & Kumar, N. (2023) · doi:10.31234/osf.io/y8ax9 · Cited by 78
Causal Factors of Effective Psychosocial Outcomes in Online Mental Health Communities
Saha, K., & Sharma, A. (2020) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v14i1.7326 · Cited by 72
Meaningless comparisons lead to false optimism in medical machine learning
DeMasi, O., Kording, K., & Recht, B. (2017) · PLoS ONE · doi:10.1371/journal.pone.0184604 · Cited by 63
“As an example, social media posts may be used to predict individuals’ depressive symptoms or future suicidal ideation [11,12].”
BackgroundPANDORA Talks: Personality and Demographics on Reddit
Gjurković, M., Karan, M., Vukojević, I., Bošnjak, M., & Snajder, J. (2021) · Proceedings of the Ninth International Workshop on Natural Language Processing for Social Media · doi:10.18653/v1/2021.socialnlp-1.12 · Cited by 58
Where is the Human?: Bridging the Gap Between AI and HCI
Inkpen, K., Chancellor, S., De Choudhury, M., Veale, M., & Baumer, E. P. S. (2019) · Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290607.3299002 · Cited by 54
Self-citationProbable depression and suicidal ideation in Hong Kong amid massive civil unrest
Hou, W. K., Hall, B. J., Liang, L., Li, T. W., Liu, H., & Galea, S. (2021) · Annals of Epidemiology · doi:10.1016/j.annepidem.2020.09.006 · Cited by 50
Hswen, Y., Naslund, J. A., Brownstein, J. S., & Hawkins, J. B. (2018) · JMIR Mental Health · doi:10.2196/11483 · Cited by 43
Son, Y., Clouston, S. A. P., Kotov, R., Eichstaedt, J. C., Bromet, E. J., Luft, B. J., & Schwartz, H. A. (2023) · Psychological Medicine · doi:10.1017/s0033291721002294 · Cited by 42
Yuan, Y., Zhang, J., Aledavood, T., Zhang, R., & Saha, K. (2026) · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790558 · Cited by 26
(2020) · Pacific Asia journal of the Association for Information Systems · doi:10.17705/1pais.12201 · Cited by 26
Metadata
- Authors
- De Choudhury, M., Kiciman, E., Dredze, M., Coppersmith, G., & Kumar, M.
- Venue
- Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2016-05-07
- Volume
- 2016
- Pages
- 2098-2110
- OpenAlex
- W2405042511
- Cited by
- 683
- References
- 74
- FWCI
- 115.26
- Type
- conference paper
- Language
- en
- Open access
- green
- Funders
- National Institutes of Health (1R01GM11269701)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.