CoChat
Sign Up
Cited by 46Mental Health via WritingExplainable Artificial Intelligence (XAI)Sentiment Analysis and Opinion Mining

Explainable depression symptom detection in social media

Open access
Bao, Eliseo, Pérez, Anxo, Parapar, Javier·2024·Health Information Science and Systems
DOI 10.1007/s13755-024-00303-9PMC PMC11379836
Open this paper in CoChat — free accountOpen PDF

Abstract

Users of social platforms often perceive these sites as supportive spaces to post about their mental health issues. Those conversations contain important traces about individuals’ health risks. Recently, researchers have exploited this online information to construct mental health detection models, which aim to identify users at risk on platforms like Twitter, Reddit or Facebook. Most of these models are focused on achieving good classification results, ignoring the explainability and interpretability of the decisions. Recent research has pointed out the importance of using clinical markers, such as the use of symptoms, to improve trust in the computational models by health professionals. In this paper, we introduce transformer-based architectures designed to detect and explain the appearance of depressive symptom markers in user-generated content from social media. We present two approaches: (i) train a model to classify, and another one to explain the classifier’s decision separately and (ii) unify the two tasks simultaneously within a single model. Additionally, for this latter manner, we also investigated the performance of recent conversational Large Language Models (LLMs) utilizing both in-context learning and finetuning. Our models provide natural language explanations, aligning with validated symptoms, thus enabling clinicians to interpret the decisions more effectively. We evaluate our approaches using recent symptom-focused datasets, using both offline metrics and expert-in-the-loop evaluations to assess the quality of our models’ explanations. Our findings demonstrate that it is possible to achieve good classification results while generating interpretable symptom-based explanations.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1007/s13755-024-00303-9

Cites

14 of 14 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Rationale-supervised training represents a middle ground where models learn to generate explanations through supervised examples, potentially inheriting the reasoning patterns of expert annotators.

    “Rationale-supervised training [67,69,90,98,110,123,124] represents a middle ground where models learn to generate explanations through supervised examples, potentially inheriting the reasoning patterns of expert annotators.”

    From Explainability of decoder-only clinical large language models: A scoping review (2026) · cites this paper as [69]

    The Introduction describes training on posts hand-labelled by domain experts, and RQ2 reports that more training examples improve the quality of generated explanations. The paper’s explanations are evidence fragments from posts; it does not establish that the models inherit expert annotators’ reasoning patterns or describe this approach as a middle ground. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

  • Lexical overlap metrics were employed in studies with available gold-standard rationales.

    “These were employed in studies with available gold-standard rationales [65,69,72,98,124]”

    From Explainability of decoder-only clinical large language models: A scoping review (2026) · cites this paper as [69]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 67

  1. [1]Prince M, Patel V, Saxena S, Maj M, Maselko J, Phillips MR, Rahman A. No health without mental health. Lancet. 2007;370(9590):859–77. 17804063 · DOI
  2. [4]Picardi A, Lega I, Tarsitani L, Caredda M, Matteucci G, Zerella MP, Miglio R, Gigantesco A, Cerbo M, Gaddini A, Spandonaro F, Biondi M. SET-DEP Group: a randomised controlled trial of the effectiveness of a program for early detection and treatment of depression in primary care. J Affect Disord. 2016;198:96–101. 27015158 · DOI
  3. [6]De Choudhury M, Gamon M, Counts S, Horvitz E. Predicting depression via social media. In: Proceedings of the international AAAI conference on web and social media. 2013. p. 128–37
  4. [7]Sadeque F, Xu D, Bethard S. Measuring the latency of depression detection in social media. In: Proceedings of the eleventh ACM international conference on web search and data mining. WSDM ’18. New York: Association for Computing Machinery; 2018. p. 495–503
  5. [8]Yates A, Cohan A, Goharian N. Depression and self-harm risk assessment in online forums. In: Proceedings of the 2017 conference on empirical methods in natural language processing. Copenhagen: Association for Computational Linguistics; 2017. p. 2968–78
  6. [10]Beheshti A, Moraveji-Hashemi V, Yakhchi S, Motahari-Nezhad H.R, Ghafari S.M, Yang J. Personality2vec: enabling the analysis of behavioral disorders in social networks. In: Proceedings of the 13th international conference on web search and data mining. WSDM ’20. New York: Association for Computing Machinery; 2020. p. 825–28
  7. [11]Bucur A.-M, Cosma A, Rosso P, Dinu LP. It’s just a matter of time: detecting depression with time-enriched multimodal transformers. In: European conference on information retrieval; 2023. Springer. p. 200–15
  8. [12]Aragon M, Lopez Monroy A.P, Gonzalez L, Losada D.E, Montes M. DisorBERT: a double domain adaptation model for detecting signs of mental disorders in social media. In: Proceedings of the 61st annual meeting of the association for computational linguistics (vol. 1: Long Papers). Toronto: Association for Computational Linguistics; 2023. p. 15305–18
  9. [13]Callahan A, Inckle K. Cybertherapy or psychobabble? A mixed methods study of online emotional support. Br J Guidance Counsell. 2012;40(3):261–78. · DOI
  10. [14]Ferraro G, Loo Gee B, Ji S, Salvador-Carulla L. Lightme: analysing language in internet support groups for mental health. Health Inf Sci Syst. 2020;8(1):34. 33088490 PMC7554289 · DOI
  11. [15]Ríssola EA, Losada DE, Crestani F. A survey of computational methods for online mental state assessment on social media. ACM Trans. Comput. Healthc. 2021;2(2):1–31. · DOI
  12. [16]Parapar J, Martín-Rodilla P, Losada D.E, Crestani F. eRisk 2023: depression, pathological gambling, and eating disorder challenges. In: Advances in information retrieval: 45th European conference on information retrieval, ECIR 2023, Dublin, Ireland, April 2–6, 2023, Proceedings, Part III. Springer; 2023. p. 585–92
  13. [17]Zirikly A, Atzil-Slonim D, Liakata M, Bedrick S, Desmet B, Ireland M, Lee A, MacAvaney S, Purver M, Resnik R, Yates A, editors. Proceedings of the eighth workshop on computational linguistics and clinical psychology. Seattle: Association for Computational Linguistics; 2022
  14. [18]Walsh CG, Chaudhry B, Dua P, Goodman KW, Kaplan B, Kavuluru R, Solomonides A, Subbian V. Stigma, biomarkers, and algorithmic bias: recommendations for precision behavioral health with artificial intelligence. JAMIA Open. 2020;3(1):9–15. 32607482 PMC7309258 · DOI · Paper page
  15. [19]Hauser TU, Skvortsova V, De Choudhury M, Koutsouleris N. The promise of a model-based psychiatry: building computational models of mental ill health. Lancet Digit. Health. 2022;4(11):816–28. PMC9627546 36229345 · DOI
  16. [20]Ernala SK, Birnbaum ML, Candan KA, Rizvi AF, Sterling WA, Kane JM, De Choudhury M. Methodological gaps in predicting mental health states from social media: triangulating diagnostic signals. In: Proceedings of the 2019 CHI conference on human factors in computing systems. CHI ’19. New York: Association for Computing Machinery; 2019. p. 1–16
  17. [23]Pérez A, Parapar J. Barreiro: Automatic depression score estimation with word embedding models. Artif Intell Med. 2022;132:102380. 36207086 · DOI
  18. [24]Nguyen T, Yates A, Zirikly A, Desmet B, Cohan A. Improving the generalizability of depression detection by leveraging clinical questionnaires. In: Proceedings of the 60th annual meeting of the association for computational linguistics, vol. 1 (Long Papers). Dublin: Association for Computational Linguistics; 2022. p. 8446–59
  19. [25]Zhang Z, Chen S, Wu M, Zhu K. Symptom identification for interpretable detection of multiple mental disorders on social media. In: Proceedings of the 2022 conference on empirical methods in natural language processing. Abu Dhabi: Association for Computational Linguistics; 2022. p. 9970–85
  20. [26]Zhang Z, Chen S, Wu M, Zhu KQ. Psychiatric scale guided risky post screening for early detection of depression. In: Raedt LD, editor. Proceedings of the thirty-first international joint conference on artificial intelligence, IJCAI-22. AI for Good; 2022. p. 5220–26
  21. [27]Pérez A, Warikoo N, Wang K, Parapar J, Gurevych I. Semantic similarity models for depression severity estimation. In: Bouamor H, Pino J, Bali K, editors. Proceedings of the 2023 conference on empirical methods in natural language processing. Singapore: Association for Computational Linguistics; 2023. p. 16104–18.
  22. [28]Smith KM, Renshaw PF, Bilello J. The diagnosis of depression: current and emerging methods. Compr Psychiatry. 2013;54(1):1–6. 22901834 PMC5502713 · DOI
  23. [35]Yang K, Zhang T, Kuang Z, Xie Q, Ananiadou S. Mentalllama: interpretable mental health analysis on social media with large language models. To be published at: WWW ’24: the ACM web conference 2023, Singapore May 13–17, 2024. 2023
  24. [36]Perez A, Parapar J, Barreiro A, Lopez-Larrosa S. Bdi-sen: a sentence dataset for clinical symptoms of depression. In: Proceedings of the 46th international ACM SIGIR conference on research and development in information retrieval. 2023.
  25. [37]Islam MR, Kabir MA, Ahmed A, Kamal ARM, Wang H, Ulhaq A. Depression detection from social network data using machine learning techniques. Health Inf Sci Syst. 2018;6(1):8 PMC6111060 30186594
  26. [38]Li Z, An Z, Cheng W, Zhou J, Zheng F, Hu B. MHA: a multimodal hierarchical attention model for depression detection in social media. Health Inf. Sci. Syst. 2023;11(1):6. 36660408 PMC9846704 · DOI
  27. [39]Guntuku SC, Yaden DB, Kern ML, Ungar LH, Eichstaedt JC. Detecting depression and mental illness on social media: an integrative review. Big data in the behavioural sciences. Curr Opin Behav Sci. 2017;18:43–49
  28. [40]Skaik R, Inkpen D. Using social media for mental health surveillance: a review. ACM Comput Surv 2020;53(6)
  29. [41]Crestani F, Losada DE, Parapar J. Early detection of mental health disorders by social media monitoring: the first five years of the ERisk project vol. 1018. 2022
  30. [42]Wang T, Brede M, Ianni A, Mentzakis E. Detecting and characterizing eating-disorder communities on social media. In: Proceedings of the tenth ACM international conference on web search and data mining. WSDM ’17. New York: Association for Computing Machinery; 2017. p. 91–100
  31. [43]Shuai H-H, Shen C-Y, Yang D-N, Lan Y-F, Lee W-C, Yu PS, Chen M-S. Mining online social data for detecting social network mental disorders. In: Proceedings of the 25th international conference on world wide web. WWW ’16. International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE. 2016. p. 275–85
  32. [44]Halfin A. Depression: the benefits of early and appropriate treatment. Am J Manag Care. 2007;13(4 Suppl):92–7. 18041868
  33. [45]Guidotti R, Ruggieri S. On the stability of interpretable models. In: 2019 international joint conference on neural networks (IJCNN). 2019. p. 1–8
  34. [48]Baehrens D, Schroeter T, Harmeling S, Kawanabe M, Hansen K, Müller K.-R. How to explain individual classification decisions. J Mach Learn Res. 2010;11:1803–31
  35. [49]Ribeiro MT, Singh S, Guestrin C. "Why should i trust you?" Explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. 2016. p. 1135–44
  36. [50]Lundberg SM, Lee S-I. A unified approach to interpreting model predictions. In: Advances in neural information processing systems, vol. 30. 2017
  37. [51]Lopez-Larrosa S, Sánchez-Souto V, Losada D.E, Parapar J, Barreiro Ha A.P, Cummings E.M. Using machine learning techniques to predict adolescents’ involvement in family conflict. Soc Sci Comput Rev. 2023;41(5):1581–607
  38. [52]Nordin N, Zainol Z, Mohd Noor M.H, Chan L.F. An explainable predictive model for suicide attempt risk using an ensemble learning and shapley additive explanations (SHAP) approach. Asian J Psychiatry 2023;79:103316 36395702
  39. [53]Ustun B, Rudin C. Supersparse linear integer models for optimized medical scoring systems. Mach Learn. 2016;102(3):349–91. · DOI
  40. [54]Lakkaraju H, Bach SH, Leskovec J. Interpretable decision sets: A joint framework for description and prediction. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. KDD ’16. New York: Association for Computing Machinery; 2016. p. 1675–84 PMC5108651 27853627
  41. [55]Lundberg SM, Lee S-I. A unified approach to interpreting model predictions. In: Proceedings of the 31st international conference on neural information processing systems. NIPS’17. Red Hook: Curran Associates Inc.; 2017. p. 4768–77
  42. [56]Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Lu, Polosukhin I. Attention is all you need. In: Guyon I, Luxburg U.V, Bengio S, Wallach H, Fergus R, Vishwanathan S, Garnett R, editors. Advances in neural information processing systems. 2017;30
  43. [57]Mullenbach J, Wiegreffe S, Duke J, Sun J, Eisenstein J. Explainable prediction of medical codes from clinical text. In: Proceedings of the 2018 conference of the North American chapter of the association for computational linguistics: human language technologies, vol. 1 (Long Papers). New Orleans: Association for Computational Linguistics; 2018. p. 1101–111
  44. [58]Song H, You J, Chung J-W, Park JC. Feature attention network: Interpretable depression detection from social media. In: Proceedings of the 32nd Pacific Asia conference on language, information and computation. Hong Kong: Association for Computational Linguistics; 2018
  45. [59]Amini H, Kosseim L. Towards explainability in using deep learning for the detection of anorexia in social media. In: Métais E, Meziane F, Horacek H, Cimiano P, editors. Natural language processing and information systems. Cham: Springer; 2020. p. 225–35.
  46. [60]Wiegreffe S, Pinter Y. Attention is not not explanation. In: Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP). Hong Kong: Association for Computational Linguistics; 2019. p. 11–20
  47. [61]Tutek M, Šnajder J. Toward practical usage of the attention mechanism as a tool for interpretability. IEEE Access. 2022;10:47011–30. · DOI
  48. [62]Jain S, Wallace BC. Attention is not explanation. In: Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, vol. 1 (Long and Short Papers). Minneapolis: Association for Computational Linguistics; 2019. p. 3543–56.
  49. [64]Camburu O-M, Rocktäschel T, Lukasiewicz T, Blunsom P. e-snli: natural language inference with natural language explanations. In: Bengio S, Wallach H, Larochelle H, Grauman K, Cesa-Bianchi N, Garnett R, editors. Advances in neural information processing systems, vol. 31
  50. [66]Kojima T, Gu S.S, Reid M, Matsuo Y, Iwasawa Y. Large language models are zero-shot reasoners. In: Koyejo S, Mohamed S, Agarwal A, Belgrave D, Cho K, Oh A, editors. Advances in neural information processing systems. 2022;35:22199–213.
  51. [67]Cambria E, Malandri L, Mercorio F, Mezzanzanica M, Nobani N. A survey on xai and natural language explanations. Inf. Process Manag. 2023;60(1):103111. · DOI · Paper page
  52. [69]Sutskever I, Vinyals O, Le QV. Sequence to sequence learning with neural networks. In: Proceedings of the 27th international conference on neural information processing systems-volume 2. NIPS’14. Cambridge: MIT Press; 2014. p. 3104–12
  53. [70]Devlin J, Chang M-W, Lee K, Toutanova K. BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, vol 1 (Long and Short Papers). Minneapolis: Association for Computational Linguistics; 2019. p. 4171–86
  54. [73]Erliksson KF. Arpteg A, Matskin M, Payberah AH. Cross-domain transfer of generative explanations using text-to-text models. In: Natural language processing and information systems: 26th international conference on applications of natural language to information systems. NLDB 2021, Saarbrücken, Germany, June 23–25, 2021, Proceedings. Berlin, Heidelberg: Springer; 2021. p. 76–89.
  55. [75]Ji S, Zhang T, Ansari L, Fu J, Tiwari P, Cambria E. MentalBERT: publicly available pretrained language models for mental healthcare. In: Proceedings of the thirteenth language resources and evaluation conference. Marseille: European Language Resources Association; 2022. p. 7184–90.
  56. [76]Brown T, Mann B, Ryder N, Subbiah M, Kaplan J.D, Dhariwal P, Neelakantan A, Shyam P, Sastry G, Askell A, Agarwal S, Herbert-Voss A, Krueger G, Henighan T, Child R, Ramesh A, Ziegler D, Wu J, Winter C, Hesse C, Chen M, Sigler E, Litwin M, Gray S, Chess B, Clark J, Berner C, McCandlish S, Radford A, Sutskever I, Amodei D. Language models are few-shot learners. In: Larochelle H, Ranzato M, Hadsell R, Balcan MF Lin H, editors. Advances in neural information processing systems, vol. 33. 2020. p. 1877–901.
  57. [79]Jackson-Koku G. Beck depression inventory. Occup Med. 2016;66(2):174–5. 26892598 · DOI
  58. [80]Naseem U, Dunn A.G, Kim J, Khushi M. Early identification of depression severity levels on reddit using ordinal classification. In: Proceedings of the ACM web conference 2022. 2022. p. 2563–72
  59. [83]Lin C-Y. ROUGE: a package for automatic evaluation of summaries. In: Text summarization branches out. Association for Computational Linguistics, Barcelona, Spain; 2004. p. 74–81
  60. [84]Papineni K, Roukos S, Ward T, Zhu W-J. Bleu: a method for automatic evaluation of machine translation. In: Proceedings of the 40th annual meeting of the association for computational linguistics. Association for Computational Linguistics, Philadelphia, Pennsylvania, USA; 2002. p. 311–18
  61. [86]MacAvaney S, Desmet B, Cohan A, Soldaini L, Yates A, Zirikly A, Goharian N. RSDD-time: temporal annotation of self-reported mental health diagnoses. In: Proceedings of the fifth workshop on computational linguistics and clinical psychology: from keyboard to clinic. New Orleans: Association for Computational Linguistics; 2018. p. 168–73
  62. [87]Mowery D, Smith H, Cheney T, Stoddard G, Coppersmith G, Bryan C, Conway M, et al. Understanding depressive symptoms and psychosocial stressors on twitter: a corpus-based study. J Med Internet Res. 2017;19(2):6895. PMC5350450 28246066 · DOI
  63. [88]Findings of the Association for Computational Linguistics. Harrigian K, Aguirre C, Dredze M. Do models of mental health based on social media data generalize? In: EMNLP; 2020. p. 3774–88.
  64. [92]Zhao Z, Wallace E, Feng S, Klein D, Singh S. Calibrate before use: improving few-shot performance of language models. In: International conference on machine learning. PMLR; 2021. p. 12697–706
  65. [93]Xun G, Jia X, Gopalakrishnan V, Zhang A. A survey on context learning. IEEE Trans Knowl Data Eng. 2017;29(1):38–56. · DOI
  66. [94]First JM, Shin H, Ranjit YS, Houston JB. Covid-19 stress and depression: examining social media, traditional media, and interpersonal communication. J Loss Trauma. 2021;26(2):101–15. · DOI
  67. [95]Lacoste A, Luccioni A, Schmidt V, Dandres T. Quantifying the carbon emissions of machine learning. arXiv:1910.09700. 2019 · arXiv

Cited by

Sorted by citations

Authors
Bao, E., Pérez, A., & Parapar, J.
Corresponding author
Bao, Eliseo
ORCID
Venue
Health Information Science and Systems
Publisher
Springer Science and Business Media LLC
Published
2024-09-06
Volume
12
Issue
1
Pages
47-47
ISSN
2047-2501 (ISSN-L)
DOI
10.1007/s13755-024-00303-9
PMC
PMC11379836
OpenAlex
W4402301607
Cited by
46
References
14
Citations by year
2026: 23 · 2025: 19 · 2024: 4
FWCI
4.02
Type
journal article
Language
en
Open access
Licence
Locations
  • Health Information Science and Systems (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • RUC (Universidade Da Coruña) (submittedVersion, cc-by)
  • arXiv (Cornell University)
Funders
  • Ministerio de Ciencia e Innovación (PLEC2021-00766, PID2022-137061OB-C21)
  • Consellería de Cultura, Educación e Ordenación Universitaria, Xunta de Galicia (2019-2022 ED431G/01, GPC ED431B 2022/33)
  • Universidade da Coruña
Topics
  • Social Sciences › Psychology › Social Psychology › Mental Health via Writing
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
Keywords
depression detection, depressive symptoms, explainable AI, social media, user-generated content, transformer models, natural language explanations, large language models, in-context learning, fine-tuning, clinical biomarkers

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review