CoChat

A Taxonomy of Ethical Tensions in Inferring Mental Health States from Social Media

Chancellor, Stevie, Birnbaum, Michael L., Caine, Eric D., Silenzio, Vincent M. B., De Choudhury, Munmun · 2019 · Proceedings of the Conference on Fairness, Accountability, and Transparency

The DOI matches its Crossref record; 85 of the 86 references it cites were found in a registry or an index.

Abstract

Powered by machine learning techniques, social media provides an unobtrusive lens into individual behaviors, emotions, and psychological states. Recent research has successfully employed social media data to predict mental health states of individuals, ranging from the presence and severity of mental disorders like depression to the risk of suicide. These algorithmic inferences hold great potential in supporting early detection and treatment of mental disorders and in the design of interventions. At the same time, the outcomes of this research can pose great risks to individuals, such as issues of incorrect, opaque algorithmic predictions, involvement of bad or unaccountable actors, and potential biases from intentional or inadvertent misuse of insights. Amplifying these tensions, there are also divergent and sometimes inconsistent methodological gaps and under-explored ethics and privacy dimensions. This paper presents a taxonomy of these concerns and ethical challenges, drawing from existing literature, and poses questions to be resolved as this research gains traction. We identify three areas of tension: ethics committees and the gap of social media research; questions of validity, data, and machine learning; and implications of this research for key stakeholders. We conclude with calls to action to begin resolving these interdisciplinary dilemmas.

Overview

Integrity signals

Registry

Crossref · 10.1145/3287560.3287587

Cites

85 of 85 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

4 more claims about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Citations

Cites

Show all 85 references

Research works printed but not linked: 7

  1. Joshua Barrie . 2014. People are freaking out over this new anti-suicide twitter app. (Nov . 2014 ). https:/www.businessinsider.com/people-freaking-out-over-samaritans-twitter-app-2014-11. Joshua Barrie. 2014. People are freaking out over this new anti-suicide twitter app. (Nov. 2014). https:/www.businessinsider.com/people-freaking-out-over-samaritans-twitter-app-2014-11.
  2. Katherine Carpenter and David Dittrich . 2011 . Bridging the distance: removing the technology buffer and seeking consistent ethical analysis in computer security research . In 1st International Digital Ethics Symposium. Loyola University Chicago Center for Digital Ethics and Policy. Katherine Carpenter and David Dittrich. 2011. Bridging the distance: removing the technology buffer and seeking consistent ethical analysis in computer security research. In 1st International Digital Ethics Symposium. Loyola University Chicago Center for Digital Ethics and Policy.
  3. Brent Hecht et al. 2018. It's time to do something: mitigating the negative impacts of computing through a change to the peer review process. (Mar. 2018). https://acm-fca.org/2018/03/29/negativeimpacts/. Brent Hecht et al. 2018. It's time to do something: mitigating the negative impacts of computing through a change to the peer review process. (Mar. 2018). https://acm-fca.org/2018/03/29/negativeimpacts/.
  4. Dan Muriello , Lizzy Donahue , Danny Ben-David , Umut Ozertem , and Reshef Shilon . 2018. Under the hood: suicide prevention tools powered by ai. (Feb . 2018 ). https://code.fb.com/ml-applications/under-the-hood-suicide-prevention-tools-powered-by-ai/. Dan Muriello, Lizzy Donahue, Danny Ben-David, Umut Ozertem, and Reshef Shilon. 2018. Under the hood: suicide prevention tools powered by ai. (Feb. 2018). https://code.fb.com/ml-applications/under-the-hood-suicide-prevention-tools-powered-by-ai/.
  5. 10.5555/3154569 · doi:10.5555/3154569
  6. Guy Rosen . 2017. Getting our community help in real time. (Nov . 2017 ). https://newsroom.fb.com/news/2017/11/getting-our-community-help-in-real-time/. Guy Rosen. 2017. Getting our community help in real time. (Nov. 2017). https://newsroom.fb.com/news/2017/11/getting-our-community-help-in-real-time/.
  7. James Vincent . 2017. Facebook is using ai to spot users with suicidal thoughts and send them help. (Nov . 2017 ). https:/www.theverge.com/2017/11/28/16709224/facebook-suicidal-thoughts-ai-help. James Vincent. 2017. Facebook is using ai to spot users with suicidal thoughts and send them help. (Nov. 2017). https:/www.theverge.com/2017/11/28/16709224/facebook-suicidal-thoughts-ai-help.

Cited by

See all 214

Metadata

Authors
Chancellor, S., Birnbaum, M. L., Caine, E. D., Silenzio, V. M. B., & De Choudhury, M.
Affiliations
  • Georgia Institute of Technology (US)
  • Northwell Health (US)
  • University of Rochester (US)
  • Georgia Tech, Atlanta, GA, US
Venue
Proceedings of the Conference on Fairness, Accountability, and Transparency
Publisher
ACM
Published
2019-01-29
Pages
79-88
OpenAlex
W2908543495
Cited by
214
References
86
Citations by year
2026: 21 · 2025: 27 · 2024: 27 · 2023: 33 · 2022: 42 · 2021: 35 · 2020: 18 · 2019: 11
FWCI
27.70
Type
conference paper
Language
en
Open access
closed
Locations
ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion)
Funders
National Institutes of Health (R01GM112697)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
  • Social Sciences › Psychology › Social Psychology › Mental Health via Writing
  • Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
Keywords
mental health prediction, social media, ethical conflicts, algorithmic bias, privacy, ethics committees, mental disorder detection, suicide risk prediction, stakeholder implications

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: A Taxonomy of Ethical Tensions in Inferring Mental Health States from Social Media. https://app.cochat.ai/papers/10.1145/3287560.3287587
The paper
Chancellor, S., Birnbaum, M. L., Caine, E. D., Silenzio, V. M. B., & De Choudhury, M. (2019). A Taxonomy of Ethical Tensions in Inferring Mental Health States from Social Media. Proceedings of the Conference on Fairness, Accountability, and Transparency. https://doi.org/10.1145/3287560.3287587

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.