A Taxonomy of Ethical Tensions in Inferring Mental Health States from Social Media
The DOI matches its Crossref record; 85 of the 86 references it cites were found in a registry or an index.
Abstract
Powered by machine learning techniques, social media provides an unobtrusive lens into individual behaviors, emotions, and psychological states. Recent research has successfully employed social media data to predict mental health states of individuals, ranging from the presence and severity of mental disorders like depression to the risk of suicide. These algorithmic inferences hold great potential in supporting early detection and treatment of mental disorders and in the design of interventions. At the same time, the outcomes of this research can pose great risks to individuals, such as issues of incorrect, opaque algorithmic predictions, involvement of bad or unaccountable actors, and potential biases from intentional or inadvertent misuse of insights. Amplifying these tensions, there are also divergent and sometimes inconsistent methodological gaps and under-explored ethics and privacy dimensions. This paper presents a taxonomy of these concerns and ethical challenges, drawing from existing literature, and poses questions to be resolved as this research gains traction. We identify three areas of tension: ethics committees and the gap of social media research; questions of validity, data, and machine learning; and implications of this research for key stakeholders. We conclude with calls to action to begin resolving these interdisciplinary dilemmas.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“if a user decides to 'drop out' from social media after their data has been collected, their health data may still be stored [34].”
From Operationalising ethics in artificial intelligence for healthcare: a framework for AI developers (2023) · cites this paper as [34]
In the citing paper
“One consideration is the dignity of developers, who may have no training in healthcare and could find emotional content traumatic [34].”
From Operationalising ethics in artificial intelligence for healthcare: a framework for AI developers (2023) · cites this paper as [34]
In the citing paper
“may not be practically feasible for social media or other large datasets encompassing millions of individuals [34].”
From Operationalising ethics in artificial intelligence for healthcare: a framework for AI developers (2023) · cites this paper as [34]
In the citing paper
“Social media data may be problematic on numerous counts: it may be an inaccurate representation of an individuals' mental state (given one's self-portrayal on social media can differ markedly from offline behaviours); it may include other users who have been 'tagged' (who themselves have a right to privacy); and it may be difficult to anonymise [32,34].”
From Operationalising ethics in artificial intelligence for healthcare: a framework for AI developers (2023) · cites this paper as [34]
In the citing paper
“users generally expect their privacy to be respected online [37], with most feeling their data should not be used for mental health research without their explicit consent [34].”
From Operationalising ethics in artificial intelligence for healthcare: a framework for AI developers (2023) · cites this paper as [34]
4 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Diagnostic and Statistical Manual of Mental Disorders
American Psychiatric Association (2013) · doi:10.1176/appi.books.9780890425596 · Cited by 116,369
boyd, D., & Crawford, K. (2012) · Information, Communication & Society · doi:10.1080/1369118x.2012.678878 · Cited by 5,479
Detecting influenza epidemics using search engine query data
Ginsberg, J., Mohebbi, M. H., Patel, R. S., Brammer, L., Smolinski, M. S., & Brilliant, L. (2009) · Nature · doi:10.1038/nature07634 · Cited by 4,295
Experimental evidence of massive-scale emotional contagion through social networks
Kramer, A. D. I., Guillory, J. E., & Hancock, J. T. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1320040111 · Cited by 3,687 · cited 1×
“This emerged in scrutinized experimental studies of Facebook data [52], where researchers manipulated the mood of millions of Facebook users without consent.”
BackgroundHow stigma interferes with mental health care.
Corrigan, P. (2004) · American Psychologist · doi:10.1037/0003-066x.59.7.614 · Cited by 3,648
How the machine “thinks”: Understanding opacity in machine learning algorithms
Burrell, J. (2016) · Big Data & Society · doi:10.1177/2053951715622512 · Cited by 3,196 · cited 1×
“However, deep learning has a key limitation – they do not produce intelligible feature sets for human understanding [10].”
BackgroundPrivacy as Contextual IntegrityPreprint
Nissenbaum, H. F. (2004) · SSRN Electronic Journal · Cited by 2,890
What Makes Clinical Research Ethical?
Emanuel, E. J. (2000) · JAMA · doi:10.1001/jama.283.20.2701 · Cited by 2,710 · cited 1×
“In clinical studies, this obligation is at the forefront of experimental design [32].”
BackgroundThe Parable of Google Flu: Traps in Big Data Analysis
Lazer, D., Kennedy, R., King, G., & Vespignani, A. (2014) · Science · doi:10.1126/science.1248506 · Cited by 2,523
Writing About Emotional Experiences as a Therapeutic Process
Pennebaker, J. W. (1997) · Psychological Science · doi:10.1111/j.1467-9280.1997.tb00403.x · Cited by 2,486 · cited 1×
“processing [11] and psychology [71] to represent text as cues of well-being, these studies described mood shifts around political”
Background
Research works printed but not linked: 7
- Joshua Barrie . 2014. People are freaking out over this new anti-suicide twitter app. (Nov . 2014 ). https:/www.businessinsider.com/people-freaking-out-over-samaritans-twitter-app-2014-11. Joshua Barrie. 2014. People are freaking out over this new anti-suicide twitter app. (Nov. 2014). https:/www.businessinsider.com/people-freaking-out-over-samaritans-twitter-app-2014-11.
- Katherine Carpenter and David Dittrich . 2011 . Bridging the distance: removing the technology buffer and seeking consistent ethical analysis in computer security research . In 1st International Digital Ethics Symposium. Loyola University Chicago Center for Digital Ethics and Policy. Katherine Carpenter and David Dittrich. 2011. Bridging the distance: removing the technology buffer and seeking consistent ethical analysis in computer security research. In 1st International Digital Ethics Symposium. Loyola University Chicago Center for Digital Ethics and Policy.
- Brent Hecht et al. 2018. It's time to do something: mitigating the negative impacts of computing through a change to the peer review process. (Mar. 2018). https://acm-fca.org/2018/03/29/negativeimpacts/. Brent Hecht et al. 2018. It's time to do something: mitigating the negative impacts of computing through a change to the peer review process. (Mar. 2018). https://acm-fca.org/2018/03/29/negativeimpacts/.
- Dan Muriello , Lizzy Donahue , Danny Ben-David , Umut Ozertem , and Reshef Shilon . 2018. Under the hood: suicide prevention tools powered by ai. (Feb . 2018 ). https://code.fb.com/ml-applications/under-the-hood-suicide-prevention-tools-powered-by-ai/. Dan Muriello, Lizzy Donahue, Danny Ben-David, Umut Ozertem, and Reshef Shilon. 2018. Under the hood: suicide prevention tools powered by ai. (Feb. 2018). https://code.fb.com/ml-applications/under-the-hood-suicide-prevention-tools-powered-by-ai/.
- 10.5555/3154569 · doi:10.5555/3154569
- Guy Rosen . 2017. Getting our community help in real time. (Nov . 2017 ). https://newsroom.fb.com/news/2017/11/getting-our-community-help-in-real-time/. Guy Rosen. 2017. Getting our community help in real time. (Nov. 2017). https://newsroom.fb.com/news/2017/11/getting-our-community-help-in-real-time/.
- James Vincent . 2017. Facebook is using ai to spot users with suicidal thoughts and send them help. (Nov . 2017 ). https:/www.theverge.com/2017/11/28/16709224/facebook-suicidal-thoughts-ai-help. James Vincent. 2017. Facebook is using ai to spot users with suicidal thoughts and send them help. (Nov. 2017). https:/www.theverge.com/2017/11/28/16709224/facebook-suicidal-thoughts-ai-help.
Cited by
Torous, J., Bucci, S., Bell, I. H., Kessing, L. V., Faurholt‐Jepsen, M., Whelan, P., Carvalho, A. F., Keshavan, M., Linardon, J., & Firth, J. (2021) · World Psychiatry · doi:10.1002/wps.20883 · Cited by 1,199
“Using social media for mental health work also remains a catalyst for ethical tensions, and a recent review offers a practical taxonomy of these tensions as well as guidance for navigating through these ongoing challenges 77 .”
BackgroundSocial Media and Mental Health: Benefits, Risks, and Opportunities for Research and Practice
Naslund, J. A., Bondre, A., Torous, J., & Aschbrenner, K. A. (2020) · Journal of Technology in Behavioral Science · doi:10.1007/s41347-020-00134-x · Cited by 746
“With research increasingly showing the valuable insights that social media data can yield about mental health states, greater attention to the ethical concerns with using individual data in this way is necessary (Chancellor et al. 2019).”
BackgroundInfluentialMethods in predictive techniques for mental health status on social media: a critical review
Chancellor, S., & De Choudhury, M. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0233-7 · Cited by 557
Self-citationLee, E. E., Torous, J., De Choudhury, M., Depp, C. A., Graham, S. A., Kim, H.-C., Paulus, M. P., Krystal, J. H., & Jeste, D. V. (2021) · Biological Psychiatry: Cognitive Neuroscience and Neuroimaging · doi:10.1016/j.bpsc.2021.02.001 · Cited by 429
Thieme, A., Belgrave, D., & Doherty, G. (2020) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3398069 · Cited by 406
“This selection criterion is consistent with the mental health literature and other systematic reviews on affective mental health [29, 170].”
BackgroundResultInfluential
Metadata
- Authors
- Chancellor, S., Birnbaum, M. L., Caine, E. D., Silenzio, V. M. B., & De Choudhury, M.
- Affiliations
- Georgia Institute of Technology (US)
- Northwell Health (US)
- University of Rochester (US)
- Georgia Tech, Atlanta, GA, US
- Venue
- Proceedings of the Conference on Fairness, Accountability, and Transparency
- Publisher
- ACM
- Published
- 2019-01-29
- Pages
- 79-88
- OpenAlex
- W2908543495
- Cited by
- 214
- References
- 86
- Citations by year
- 2026: 21 · 2025: 27 · 2024: 27 · 2023: 33 · 2022: 42 · 2021: 35 · 2020: 18 · 2019: 11
- FWCI
- 27.70
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion)
- Funders
- National Institutes of Health (R01GM112697)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Psychology › Social Psychology › Mental Health via Writing
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Keywords
- mental health prediction, social media, ethical conflicts, algorithmic bias, privacy, ethics committees, mental disorder detection, suicide risk prediction, stakeholder implications
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: A Taxonomy of Ethical Tensions in Inferring Mental Health States from Social Media. https://app.cochat.ai/papers/10.1145/3287560.3287587
- The paper
- Chancellor, S., Birnbaum, M. L., Caine, E. D., Silenzio, V. M. B., & De Choudhury, M. (2019). A Taxonomy of Ethical Tensions in Inferring Mental Health States from Social Media. Proceedings of the Conference on Fairness, Accountability, and Transparency. https://doi.org/10.1145/3287560.3287587
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.