Mental-LLM: Leveraging Large Language Models for Mental Health Prediction via Online Text Data
Open accessAbstract
Advances in large language models (LLMs) have empowered a variety of applications. However, there is still a significant gap in research when it comes to understanding and enhancing the capabilities of LLMs in the field of mental health. In this work, we present a comprehensive evaluation of multiple LLMs on various mental health prediction tasks via online text data, including Alpaca, Alpaca-LoRA, FLAN-T5, GPT-3.5, and GPT-4. We conduct a broad range of experiments, covering zero-shot prompting, few-shot prompting, and instruction fine-tuning. The results indicate a promising yet limited performance of LLMs with zero-shot and few-shot prompt designs for mental health tasks. More importantly, our experiments show that instruction finetuning can significantly boost the performance of LLMs for all tasks simultaneously. Our best-finetuned models, Mental-Alpaca and Mental-FLAN-T5, outperform the best prompt design of GPT-3.5 (25 and 15 times bigger) by 10.9% on balanced accuracy and the best of GPT-4 (250 and 150 times bigger) by 4.8%. They further perform on par with the state-of-the-art task-specific language model. We also conduct an exploratory case study on LLMs' capability on mental health reasoning tasks, illustrating the promising capability of certain models such as GPT-4. We summarize our findings into a set of action guidelines for potential methods to enhance LLMs' capability for mental health tasks. Meanwhile, we also emphasize the important limitations before achieving deployability in real-world mental health settings, such as known racial and gender bias. We highlight the important ethical risks accompanying this line of research.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515 · see citing works
Advances in Neural Information Processing Systems 19
(2007) · The MIT Press eBooks · doi:10.52202/085713 · Cited by 16,037 · see citing works
LLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602 · see citing works
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q., & Zhou, D. (2022) · arXiv · doi:10.48550/arxiv.2201.11903 · Cited by 4,263 · see citing works
Llama 2: Open Foundation and Fine-Tuned Chat Models
Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.09288 · Cited by 2,604 · see citing works
PaLM: Scaling Language Modeling with Pathways
Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.02311 · Cited by 2,590 · see citing works
LoRA: Low-Rank Adaptation of Large Language Models
Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., & Chen, W. (2021) · arXiv · doi:10.48550/arxiv.2106.09685 · Cited by 2,523 · see citing works
The Balanced Accuracy and Its Posterior Distribution
Brodersen, K. H., Ong, C. S., Stephan, K. E., & Buhmann, J. M. (2010) · 2010 20th International Conference on Pattern Recognition · doi:10.1109/icpr.2010.764 · Cited by 1,816 · cited 1× · see citing works
“It is noteworthy that despite the name similarity, Alpaca-LoRA is completely different from Alpaca.”
BackgroundToward expert-level medical question answering with large language models
Singhal, K., Tu, T., Gottweis, J., Sayres, R., Wulczyn, E., Amin, M., Hou, L., Clark, K., Pfohl, S. R., Cole-Lewis, H., et al. (2025) · Nature Medicine · doi:10.1038/s41591-024-03423-7 · Cited by 1,734 · see citing works
Predicting Depression via Social Media
Choudhury, M. D., Gamon, M., Counts, S., & Horvitz, E. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v7i1.14432 · Cited by 1,583 · see citing works
Sparks of Artificial General Intelligence: Early experiments with GPT-4
Bubeck, S., Chandrasekaran, V., Eldan, R., Gehrke, J., Horvitz, E., Kamar, E., Lee, P., Lee, Y. T., Li, Y., Lundberg, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.12712 · Cited by 1,526 · see citing works
Scaling Instruction-Finetuned Language Models
Chung, H. W., Hou, L., Longpre, S., Zoph, B., Tay, Y., Fedus, W., Li, Y., Wang, X., Dehghani, M., Brahma, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2210.11416 · Cited by 1,441 · see citing works
Self-citationFinetuned Language Models Are Zero-Shot Learners
Wei, J., Bosma, M., Zhao, V. Y., Guu, K., Yu, A. W., Lester, B., Du, N., Dai, A. M., & Le, Q. V. (2021) · arXiv · doi:10.48550/arxiv.2109.01652 · Cited by 1,101 · see citing works
Bias in data‐driven artificial intelligence systems—An introductory survey
Ntoutsi, E., Fafalios, P., Gadiraju, U., Iosifidis, V., Nejdl, W., Vidal, M., Ruggieri, S., Turini, F., Papadopoulos, S., Krasanakis, E., et al. (2020) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1356 · Cited by 1,043 · see citing works
Artificial intelligence for mental health and mental illnesses: an overview
Graham, S., Depp, C., Lee, E. E., Nebeker, C., Tu, X., Kim, H.-C., & Jeste, D. V. (2019) · Current Psychiatry Reports · doi:10.1007/s11920-019-1094-0 · Cited by 970 · see citing works
You Are What You Tweet: Analyzing Twitter for Public Health
Paul, M., & Dredze, M. H. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v5i1.14137 · Cited by 947 · see citing works
Mental Health Discourse on reddit: Self-Disclosure, Social Support, and Anonymity
De Choudhury, M., & De, S. (2014) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v8i1.14526 · Cited by 836 · see citing works
ChatGPT: Jack of all trades, master of none
Kocoń, J., Cichecki, I., Kaszyca, O., Kochanek, M., Szydło, D., Baran, J., Bielaniewicz, J., Gruza, M., Janz, A., Kanclerz, K., et al. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101861 · Cited by 758 · cited 1× · see citing works
“Some work explored the capability of LLMs for sentiment analysis and emotion reasoning [56, 81, 115].”
BackgroundDetecting depression and mental illness on social media: an integrative review
Guntuku, S. C., Yaden, D. B., Kern, M. L., Ungar, L. H., & Eichstaedt, J. C. (2017) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2017.07.005 · Cited by 722 · see citing works
Discovering Shifts to Suicidal Ideation from Mental Health Content in Social Media
Choudhury, M. D., Kıcıman, E., Dredze, M. H., Coppersmith, G., & Kumar, M. (2016) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858207 · Cited by 682 · see citing works
Ethical Machine Learning in Healthcare
Chen, I. Y., Pierson, E., Rose, S., Joshi, S., Ferryman, K., & Ghassemi, M. (2021) · Annual Review of Biomedical Data Science · doi:10.1146/annurev-biodatasci-092820-114757 · Cited by 645 · see citing works
Self-citationFacebook language predicts depression in medical records
Eichstaedt, J. C., Smith, R. J., Merchant, R. M., Ungar, L. H., Crutchley, P., Preoţiuc-Pietro, D., Asch, D. A., & Schwartz, H. A. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1802331115 · Cited by 643 · see citing works
Li, Y., Li, Z., Zhang, K., Dan, R., Jiang, S. B., & Zhang, Y. (2023) · Cureus · doi:10.7759/cureus.40895 · Cited by 616 · see citing works
Feeling bad on Facebook: depression disclosures by college students on a social networking site
Moreno, M. A., Jelenchick, L. A., Egan, K. G., COX, E. D., Young, H. N., Gannon, K., & Becker, T. (2011) · Depression and Anxiety · doi:10.1002/da.20805 · Cited by 607 · see citing works
An overview of the features of chatbots in mental health: A scoping review
Abd-alrazaq, A. A., Alajlani, M., Alalwan, A. A., Bewick, B. M., Gardner, P., & Househ, M. (2019) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2019.103978 · Cited by 605 · see citing works
A Review on Fairness in Machine Learning
Pessach, D., & Shmueli, E. (2023) · ACM Computing Surveys · doi:10.1145/3494672 · Cited by 598 · see citing works
Methods in predictive techniques for mental health status on social media: a critical review
Chancellor, S., & De Choudhury, M. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0233-7 · Cited by 557 · see citing works
Capabilities of GPT-4 on Medical Challenge Problems
Nori, H., King, N., McKinney, S. M., Carignan, D., & Horvitz, E. (2023) · arXiv · doi:10.48550/arxiv.2303.13375 · Cited by 509 · see citing works
Natural language processing of social media as screening for suicide risk
Coppersmith, G., Leary, R., Crutchley, P., & Fine, A. (2018) · Biomedical Informatics Insights · doi:10.1177/1178222618792860 · Cited by 506 · see citing works
Sharma, A., Lin, I. W., Miner, A. S., Atkins, D. C., & Althoff, T. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-022-00593-2 · Cited by 500 · see citing works
Social media as a measurement tool of depression in populations
De Choudhury, M., Counts, S., & Horvitz, E. (2013) · Proceedings of the 5th Annual ACM Web Science Conference · doi:10.1145/2464464.2464480 · Cited by 497 · see citing works
Persistent Anti-Muslim Bias in Large Language Models
Abid, A., Farooqi, M., & Zou, J. (2021) · Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3461702.3462624 · Cited by 476 · see citing works
Detection of Depression-Related Posts in Reddit Social Media Forum
Tadesse, M. M., Lin, H., Xu, B., & Yang, L. (2019) · IEEE Access · doi:10.1109/access.2019.2909180 · Cited by 466 · see citing works
Health system-scale language models are all-purpose prediction engines
Jiang, L. Y., Liu, X. C., Nejatian, N. P., Nasir-Moin, M., Wang, D., Abidin, A., Eaton, K., Riina, H. A., Laufer, I., Punjabi, P., et al. (2023) · Nature · doi:10.1038/s41586-023-06160-y · Cited by 460 · see citing works
Self-citationCan AI Help Reduce Disparities in General Medical and Mental Health Care?
(2019) · AMA Journal of Ethics · doi:10.1001/amajethics.2019.167 · Cited by 401 · see citing works
Least-to-Most Prompting Enables Complex Reasoning in Large Language Models
Zhou, D., Schärli, N., Hou, L., Wei, J., Scales, N., Wang, X., Schuurmans, D., Cui, C., Bousquet, O., Le, Q., et al. (2022) · arXiv · doi:10.48550/arxiv.2205.10625 · Cited by 380 · see citing works
Large language models are few-shot clinical information extractors
Agrawal, M., Hegselmann, S., Lang, H., Kim, Y., & Sontag, D. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.130 · Cited by 358 · see citing works
Recognizing Depression from Twitter Activity
Tsugawa, S., Kikuchi, Y., Kishino, F., Nakajima, K., Itoh, Y., & Ohsaki, H. (2015) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702280 · Cited by 349 · see citing works
From ADHD to SAD: Analyzing the Language of Mental Health on Twitter through Self-Reported Diagnoses
Coppersmith, G., Dredze, M. H., Harman, C., & Hollingshead, K. (2015) · Workshop on Computational Linguistics and Clinical Psychology (CLPsych) · doi:10.3115/v1/w15-1201 · Cited by 342 · see citing works
CLPsych 2015 Shared Task: Depression and PTSD on Twitter
Coppersmith, G., Dredze, M. H., Harman, C., Hollingshead, K., & Mitchell, M. (2015) · Workshop on Computational Linguistics and Clinical Psychology (CLPsych) · doi:10.3115/v1/w15-1204 · Cited by 329 · see citing works
Measuring post traumatic stress disorder in Twitter
Coppersmith, G., Harman, C., & Dredze, M. (2014) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v8i1.14574 · Cited by 316 · see citing works
Perceptions and opinions of patients about mental health chatbots: scoping review
Abd-Alrazaq, A. A., Alajlani, M., Ali, N., Denecke, K., Bewick, B. M., & Househ, M. (2021) · Journal of Medical Internet Research · doi:10.2196/17828 · Cited by 301 · see citing works
Wang, D., Churchill, E., Maes, P., Fan, X., Shneiderman, B., Shi, Y., & Wang, Q. (2020) · Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3334480.3381069 · Cited by 271 · see citing works
Self-citationTimmons, A. C., Duong, J. B., Simo Fiallo, N., Lee, T., Vo, H. P. Q., Ahle, M. W., Comer, J. S., Brewer, L. C., Frazier, S. L., & Chaspari, T. (2023) · Perspectives on Psychological Science · doi:10.1177/17456916221134490 · Cited by 263 · see citing works
Detection of Suicide Ideation in Social Media Forums Using Deep Learning
Tadesse, M. M., Lin, H., Xu, B., & Yang, L. (2019) · Algorithms · doi:10.3390/a13010007 · Cited by 250 · see citing works
A Mental Health Chatbot for Regulating Emotions (SERMO) - Concept and Usability Test
Denecke, K., Vaaheesan, S., & Arulnathan, A. (2020) · IEEE Transactions on Emerging Topics in Computing · doi:10.1109/tetc.2020.2974478 · Cited by 240 · see citing works
Large Language Models Can Self-Improve
Huang, J., Gu, S. S., Hou, L., Wu, Y., Wang, X., Yu, H., & Han, J. (2022) · arXiv · doi:10.48550/arxiv.2210.11610 · Cited by 230 · see citing works
Self-citationAffective and Content Analysis of Online Depression Communities
Nguyen, T., Phung, D. Q., Dao, B., Venkatesh, S., & Berk, M. (2014) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2014.2315623 · Cited by 217 · see citing works
Detecting Anxiety through Reddit
Shen, J. H., & Rudzicz, F. (2017) · Workshop on Computational Linguistics and Clinical Psychology (CLPsych) · doi:10.18653/v1/w17-3107 · Cited by 215 · see citing works
Ridout, B., & Campbell, A. (2018) · Journal of Medical Internet Research · doi:10.2196/12244 · Cited by 214 · see citing works
MentaLLaMA: Interpretable Mental Health Analysis on Social Media with Large Language Models
Yang, K., Zhang, T., Kuang, Z., Xie, Q., Huang, J., & Ananiadou, S. (2024) · International World Wide Web Conference (WWW) · doi:10.1145/3589334.3648137 · Cited by 214 · see citing works
Stress detection using natural language processing and machine learning over social interactions
Nijhawan, T., Attigeri, G. V., & Thalengala, A. (2022) · Journal Of Big Data · doi:10.1186/s40537-022-00575-6 · Cited by 197 · see citing works
Lee, Y., Yamashita, N., & Huang, Y. (2020) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3392836 · Cited by 195 · see citing works
Knowledge-aware Assessment of Severity of Suicide Risk for Early Intervention
Gaur, M., Alambo, A., Sain, J. P., Kursuncu, U., Thirunarayan, K., Kavuluru, R., Sheth, A., Welton, R., & Pathak, J. (2019) · International World Wide Web Conference (WWW) · doi:10.1145/3308558.3313698 · Cited by 193 · see citing works
Rumshisky, A., Ghassemi, M., Naumann, T., Peter Szolovits, B., Castro, V. M., McCoy, T. H., & Perlis, R. H. (2016) · Translational Psychiatry · doi:10.1038/tp.2015.182 · Cited by 190 · see citing works
Self-citationJo, E., Epstein, D. A., Jung, H., & Kim, Y.-H. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581503 · Cited by 174 · see citing works
Quantifying the Language of Schizophrenia in Social Media
Mitchell, M., Hollingshead, K., & Coppersmith, G. (2015) · Proceedings of the 2nd Workshop on Computational Linguistics and Clinical Psychology: From Linguistic Signal to Clinical Reality · doi:10.3115/v1/w15-1202 · Cited by 172 · see citing works
Electronic Workshops in Computing
(2013) · Electronic workshops in computing · doi:10.14236/ewic · Cited by 170 · see citing works
Birnbaum, M. L., Ernala, S. K., Rizvi, A. F., De Choudhury, M., & Kane, J. M. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.7956 · Cited by 168 · cited 1× · see citing works
“Since then, researchers have proposed a wide range of feature engineering methods and built machine-learning models for the prediction [13, 67, 69, 88, 103].”
BackgroundSharma, A. K., Lin, I. W., Miner, A. S., Atkins, D. C., & Althoff, T. (2021) · International World Wide Web Conference (WWW) · doi:10.1145/3442381.3450097 · Cited by 168 · see citing works
Mental Health Support and its Relationship to Linguistic Accommodation in Online Communities
Sharma, E., & De Choudhury, M. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174215 · Cited by 156 · see citing works
Estimating county health statistics with twitter
Culotta, A. (2014) · International Conference on Human Factors in Computing Systems · doi:10.1145/2556288.2557139 · Cited by 154 · see citing works
Can ChatGPT Understand Too? A Comparative Study on ChatGPT and Fine-tuned BERT
Zhong, Q., Ding, L., Liu, J., Du, B., & Tao, D. (2023) · arXiv · doi:10.48550/arxiv.2302.10198 · Cited by 146 · see citing works
Suicide Announcement on Facebook
Ruder, T. D., Hatch, G. M., Ampanozi, G., Thali, M. J., & Fischer, N. (2011) · Crisis · doi:10.1027/0227-5910/a000086 · Cited by 142 · see citing works
Towards a chatbot for digital counselling
Cameron, G., Cameron, D., Megaw, G., Bond, R., Mulvenna, M. D., O’Neill, S. M., Armour, C., & McTear, M. (2017) · Electronic workshops in computing · doi:10.14236/ewic/hci2017.24 · Cited by 138 · see citing works
Machine Classification and Analysis of Suicide-Related Communication on Twitter
Burnap, P., Colombo, W., & Scourfield, J. (2015) · ACM Conference on Hypertext and Social Media (HT) · doi:10.1145/2700171.2791023 · Cited by 136 · see citing works
Perception Differences between the Depressed and Non-Depressed Users in Twitter
Park, M., McDonald, D. W., & Cha, M. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v7i1.14425 · Cited by 135 · see citing works
Technology and mental health: The role of artificial intelligence
Lovejoy, C. A. (2019) · European Psychiatry · doi:10.1016/j.eurpsy.2018.08.004 · Cited by 129 · see citing works
The language of mental health problems in social media
Gkotsis, G., Oellrich, A., Hubbard, T., Dobson, R., Liakata, M., Velupillai, S. U., & Dutta, R. (2016) · Workshop on Computational Linguistics and Clinical Psychology (CLPsych) · doi:10.18653/v1/w16-0307 · Cited by 127 · see citing works
Xu, X., Chikersal, P., Doryab, A., Villalba, D. K., Dutcher, J. M., Tumminia, M. J., Althoff, T., Cohen, S., Creswell, K. G., Creswell, J. D., et al. (2019) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3351274 · Cited by 125 · see citing works
Self-citationExploring and Learning Suicidal Ideation Connotations on Social Media with Deep Learning
Sawhney, R., Manchanda, P., Mathur, P., et al. (2018) · Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis (WASSA) · doi:10.18653/v1/w18-6223 · Cited by 116 · see citing works
Machine learning models to detect anxiety and depression through social media: A scoping review
Ahmed, A., Aziz, S., Toro, C. T., Alzubaidi, M. S., Irshaidat, S., Serhan, H. A. A., Abd-Alrazaq, A. A., & Househ, M. S. (2022) · Computer Methods and Programs in Biomedicine Update · doi:10.1016/j.cmpbup.2022.100066 · Cited by 106 · see citing works
On integrating a language model into neural machine translation
Gülçehre, Ç., Fırat, O., Xu, K., Cho, K., & Bengio, Y. (2017) · Computer Speech & Language · doi:10.1016/j.csl.2017.01.014 · Cited by 105 · see citing works
Early Identification of Depression Severity Levels on Reddit Using Ordinal Classification
Naseem, U., Dunn, A. G., Kim, J., & Khushi, M. (2022) · International World Wide Web Conference (WWW) · doi:10.1145/3485447.3512128 · Cited by 104 · see citing works
GLOBEM: Cross-Dataset Generalization of Longitudinal Human Behavior Modeling
Xu, X., Liu, X., Zhang, H., Wang, W., Nepal, S., Sefidgar, Y., Seo, W., Kuehn, K. S., Huckins, J. F., Morris, M. E., et al. (2022) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3569485 · Cited by 96 · see citing works
Self-citationAssessing the Usability of a Chatbot for Mental Health Care
Cameron, G., Cameron, D., Megaw, G., Bond, R., Mulvenna, M. D., O’Neill, S. M., Armour, C., & McTear, M. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-17705-8_11 · Cited by 92 · see citing works
Livingston, J., Cianfrone, M., Korf-Uzan, K., & Coniglio, C. (2014) · Social Psychiatry and Psychiatric Epidemiology · doi:10.1007/s00127-013-0815-7 · Cited by 91 · see citing works
Social media-based interventions for adolescent and young adult mental health: A scoping review
Kruzan, K. P., Williams, K. D. A., Meyerhoff, J., Yoo, D. W., O’Dwyer, L., Choudhury, M. D., & Mohr, D. C. (2022) · Internet Interventions · doi:10.1016/j.invent.2022.100578 · Cited by 90 · see citing works
Monitoring Tweets for Depression to Detect At-risk Users
Jamil, Z., Inkpen, D. Z., Buddhitha, P., & White, K. (2017) · Workshop on Computational Linguistics and Clinical Psychology (CLPsych) · doi:10.18653/v1/w17-3104 · Cited by 90 · see citing works
Compressing Context to Enhance Inference Efficiency of Large Language Models
Li, Y., Dong, B., Lin, C., & Guerin, F. (2023) · arXiv · Cited by 88 · see citing works
Multi-Task Learning for Mental Health using Social Media TextPreprint
Benton, A., Mitchell, M., & Hovy, D. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.03538 · Cited by 84 · see citing works
Detection of Mental Health from Reddit via Deep Contextualized Representations
Jiang, Z., Levitan, S. I., Zomick, J., & Hirschberg, J. (2020) · International Workshop on Health Text Mining and Information Analysis (LOUHI) · doi:10.18653/v1/2020.louhi-1.16 · Cited by 80 · see citing works
Inferring Mood Instability on Social Media by Leveraging Ecological Momentary Assessments
Saha, K., Chan, L., De Barbaro, K., Abowd, G. D., & De Choudhury, M. (2017) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3130960 · Cited by 77 · see citing works
Deep Learning for Suicide and Depression Identification with Unsupervised Label Correction
Haque, A., Reddi, V., & Giallanza, T. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-86383-8_35 · Cited by 70 · see citing works
Leveraging Collaborative-Filtering for Personalized Behavior Modeling
Xu, X., Chikersal, P., Dutcher, J. M., Sefidgar, Y. S., Seo, W., Tumminia, M. J., Villalba, D. K., Cohen, S., Creswell, K. G., Creswell, J. D., et al. (2021) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3448107 · Cited by 65 · see citing works
Self-citationEvaluation of ChatGPT for NLP-based Mental Health ApplicationsPreprint
Lamichhane, B. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2303.15727 · Cited by 55 · see citing works
Harnessing Large Language Models for Cognitive Assistants in Factories
Freire, S. K., Foosherian, M., Wang, C., & Niforatos, E. (2023) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3571884.3604313 · Cited by 53 · see citing works
PMC-LLaMA: Towards Building Open-source Language Models for MedicinePreprint
Wu, C., Lin, W., Zhang, X., Zhang, Y., Wang, Y., & Xie, W. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2304.14454 · Cited by 48 · see citing works
Mauriello, M. L., Lincoln, E. T., Hon, G., Simon, D., Jurafsky, D., & Paredes, P. E. (2021) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3411763.3451799 · Cited by 38 · see citing works
Omar, R., Mangukiya, O., Kalnis, P., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2302.06466 · Cited by 36 · see citing works
Sampath, K., & Thenmozhi, D. (2022) · IFIP advances in information and communication technology · doi:10.1007/978-3-031-16364-7_11 · Cited by 34 · see citing works
Trustworthy Embodied Conversational Agents for Healthcare
Lupetti, M. L., Hagens, E., van der Maden, W., Steegers‐Theunissen, R. P. M., & Rousian, M. (2023) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3571884.3597128 · Cited by 33 · see citing works
Raise a Child in Large Language Model: Towards Effective and Generalizable Fine-tuningPreprint
Xu, R., Luo, F., Zhang, Z., Tan, C., Chang, B., Huang, S., & Huang, F. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.05687 · Cited by 33 · see citing works
Analyzing the Use of Large Language Models for Content Moderation with ChatGPT Examples
Franco, M., Gaggi, O., & Palazzi, C. E. (2023) · doi:10.1145/3599696.3612895 · Cited by 29 · see citing works
Classification of Breast Cancer in Mri with Multimodal Fusion
Morais, M., Calisto, F. M., Santiago, C., Aleluia, C., & Nascimento, J. C. (2023) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi53787.2023.10230686 · Cited by 29 · see citing works
Dreaddit: A Reddit Dataset for Stress Analysis in Social MediaPreprint
Turcan, E., & McKeown, K. R. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.00133 · Cited by 26 · see citing works
Accounting for Confirmation Bias in Crowdsourced Label Aggregation
Gemalmaz, M. A., & Yin, M. (2021) · doi:10.24963/ijcai.2021/238 · Cited by 19 · see citing works
Han, S., Mao, R., & Cambria, E. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2209.07494 · Cited by 19 · see citing works
Balagopalan, A., Madras, D., Yang, D., Hadfield-Menell, D., Hadfield, G. K., & Ghassemi, M. (2023) · Science Advances · doi:10.1126/sciadv.abq0701 · Cited by 14 · see citing works
Self-citationPredicting Depression and Anxiety on Reddit: a Multi-task Learning Approach
Sarkar, S., Alhamadani, A., Alkulaib, L., et al. (2022) · IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM) · doi:10.1109/asonam55673.2022.10068655 · Cited by 14 · see citing works
PsyHeal: An Approach to Remote Mental Health Monitoring System
Patel, V., Mishra, P., & Patni, J. C. (2018) · International Conference on Advances in Computing and Communication Engineering · doi:10.1109/icacce.2018.8441760 · Cited by 13 · see citing works
Understanding practices and needs of researchers in human state modeling by passive mobile sensing
Xu, X., Mankoff, J. C., & Dey, A. K. (2021) · CCF Transactions on Pervasive Computing and Interaction · doi:10.1007/s42486-021-00072-4 · Cited by 13 · see citing works
Self-citationGhosh, S., & Caliskan, A. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.10510 · Cited by 13 · see citing works
Otsuka, N., Kawanishi, Y., Doi, F., Takeda, T., Okumura, K., Yamauchi, T., Yada, S., Wakamiya, S., Aramaki, E., & Makinodan, M. (2023) · Psychiatry and Clinical Neurosciences · doi:10.1111/pcn.13580 · Cited by 10 · see citing works
In-Context Ability Transfer for Question Decomposition in Complex QAPreprint
Venktesh, V., Bhattacharya, S., & Anand, A. (2023) · arXiv.org · doi:10.48550/arxiv.2310.18371 · Cited by 6 · see citing works
Improving the Generalizability of Depression Detection by Leveraging Clinical QuestionnairesPreprint
Nguyen, T., Yates, A., Zirikly, A., et al. (2022) · UvA-DARE (University of Amsterdam) · doi:10.48550/arxiv.2204.10432 · Cited by 3 · see citing works
Is ChatGPT a General-Purpose Natural Language Processing Task Solver?
Qin, C., Zhang, A., Zhang, Z., Chen, J., Yasunaga, M., & Yang, D. (2023) · arXiv · doi:10.48550/arxiv.2302.06476
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805
Flagged first, then by citations
Research works printed but not linked: 16
- [10]Amin Mostafa M., Cambria Erik, and Schuller Björn W.. 2023. Will Affective Computing Emerge from Foundation Models and General AI? A First Evaluation on ChatGPT. http://arxiv.org/abs/2303.03186 · arXiv
- [29]Coppersmith Glen, Leary Ryan, Crutchley Patrick, and Fine Alex. 2018. Natural Language Processing of Social Media as Screening for Suicide Risk. Biomedical Informatics Insights 10 (Jan. 2018), 117822261879286. 10.1177/1178222618792860 PMC6111391 30158822 · DOI · Paper page
- [31]Dang Hai, Mecke Lukas, Lehmann Florian, Goller Sven, and Buschek Daniel. 2022. How to prompt? Opportunities and challenges of zero-and few-shot learning for human-AI interaction in creative applications of generative models. arXiv preprint arXiv:2209.01390 (2022). · arXiv
- [46]Guntuku Sharath Chandra, Buffone Anneke, Jaidka Kokil, Eichstaedt Johannes C, and Ungar Lyle H. 2019. Understanding and measuring psychological stress using social media. In Proceedings of the international AAAI conference on web and social media, Vol. 13. 214– 225.
- [58]Ji Shaoxiong, Zhang Tianlin, Ansari Luna, Fu Jie, Tiwari Prayag, and Cambria Erik. 2021. MentalBERT: Publicly Available Pretrained Language Models for Mental Healthcare. http://arxiv.org/abs/2110.15621 · arXiv
- [65]Kojima Takeshi, Gu Shixiang Shane, Reid Machel, Matsuo Yutaka, and Iwasawa Yusuke. 2022. Large Language Models are Zero-Shot Reasoners. In 36th Conference on Neural Information Processing Systems.
- [71]Liu Xin, McDuff Daniel, Kovacs Geza, Galatzer-Levy Isaac, Sunshine Jacob, Zhan Jiening, Poh Ming-Zher, Liao Shun, Achille Paolo Di, and Patel Shwetak. 2023. Large Language Models are Few-Shot Health Learners. In arXiv.
- [81]Nepal Subigya, Martinez Gonzalo J., Mirjafari Shayan, Saha Koustuv, Swain Vedant Das, Xu Xuhai, Audia Pino G., Choudhury Munmun De, Dey Anind K., Striegel Aaron, and Campbell Andrew T.. 2022. A Survey of Passive Sensing in the Workplace. arXiv:2201.03074 [ cs.HC] · arXiv
- [93]Posner K, Brent D, Lucas C, Gould M, Stanley B, Brown G, Fisher P, Zelazny J, Burke A, Oquendo MJNY, 2008. Columbia-suicide severity rating scale (C-SSRS). New York, NY: Columbia University Medical Center 10 (2008), 2008.
- [97]Raffel Colin, Shazeer Noam, Roberts Adam, Lee Katherine, Narang Sharan, Matena Michael, Zhou Yanqi, Li Wei, and Liu Peter J. 2020. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. Journal of Machine Learning Research (2020).
- [98]Regier Darrel A, Kuhl Emily A, and Kupfer David J. 2013. The DSM-5: Classification and criteria changes. World psychiatry 12, 2 (2013), 92– 98. 23737408 PMC3683251
- [100]Robinson Joshua and Wingate David. 2023. Leveraging Large Language Models for Multiple Choice Question Answering. In The Eleventh International Conference on Learning Representations. https://openreview.net/forum?id=yKbprarjc5B
- [104]Saifullah Shoffan, Fauziah Yuli, and Aribowo Agus Sasmito. 2021. Comparison of machine learning for sentiment analysis in detecting anxiety based on social media data. arXiv preprint arXiv:2101.06353 (2021). · arXiv
- [123]Wei Jason, Bosma Maarten, Zhao Vincent Y., Guu Kelvin, Yu Adams Wei, Lester Brian, Du Nan, Dai Andrew M., and Le Quoc V.. 2022. Finetuned Language Models Are Zero-Shot Learners. http://arxiv.org/abs/2109.01652 arXiv:2109.01652 [cs]. · arXiv
- [131]Xu Xuhai, Zhang Han, Sefidgar Yasaman, Ren Yiyi, Liu Xin, Seo Woosuk, Brown Jennifer, Kuehn Kevin, Merrill Mike, Nurius Paula, Patel Shwetak, Althoff Tim, Margaret E Morris Eve Riskin, Mankoff Jennifer, and Dey Anind K. 2022. GLOBEM Dataset: Multi-Year Datasets for Longitudinal Human Behavior Modeling Generalization. In Thirty-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track. 18.
- [132]Yang Kailai, Ji Shaoxiong, Zhang Tianlin, Xie Qianqian, and Ananiadou Sophia. 2023. On the Evaluations of ChatGPT and Emotion-enhanced Prompting for Mental Health Analysis. http://arxiv.org/abs/2304.03347 · arXiv
Cited by
Sorted by citations
A Comprehensive Review on Synergy of Multi-Modal Data and AI Technologies in Medical Diagnosis
Xu, X., Li, J., Zhu, Z., Zhao, L., Wang, H., Song, C., Chen, Y., Zhao, Q., Yang, J., & Pei, Y. (2024) · Bioengineering · doi:10.3390/bioengineering11030219 · Cited by 242 · see citing works
Self-citationA Review on Edge Large Language Models: Design, Execution, and Applications
Zheng, Y., Chen, Y., Qian, B., Shi, X., Shu, Y., & Chen, J. (2025) · ACM Computing Surveys · doi:10.1145/3719664 · Cited by 169 · see citing works
Yang, Z., Xu, X., Yao, B., Rogers, E., Zhang, S., Intille, S., Shara, N., Gao, G. G., & Wang, D. (2024) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3659625 · Cited by 120 · see citing works
Self-citationLarge Language Models in Mental Health Care: A Scoping Review
Hua, Y., Liu, F., Yang, K., Li, Z., Na, H., Sheu, Y.-H., Zhou, P., Moran, L. V., Ananiadou, S., Clifton, D. A., et al. (2025) · Current Treatment Options in Psychiatry · doi:10.1007/s40501-025-00363-y · Cited by 92 · see citing works
Limitations of the LLM-as-a-Judge Approach for Evaluating LLM Outputs in Expert Knowledge Tasks
Szymanski, A., Ziems, N., Eicher-Miller, H. A., Li, T. J.-J., Jiang, M., & Metoyer, R. A. (2025) · Proceedings of the 30th International Conference on Intelligent User Interfaces · doi:10.1145/3708359.3712091 · Cited by 77 · see citing works
Large Language Models in Mental Health Care: A Systematic Scoping Review (Preprint)
hua, Y., Liu, F., Yang, K., Li, Z., Sheu, Y.-H., Zhou, P., Moran, L. V., Ananiadou, S., Clifton, D. A., & Beam, A. (2024) · doi:10.2196/preprints.64088 · Cited by 46 · see citing works
Private Yet Social: How LLM Chatbots Support and Challenge Eating Disorder Recovery
Choi, R., Kim, T., Park, S., Kim, J. G., & Lee, S.-J. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713485 · Cited by 19 · see citing works
Dalal, S., Tilwani, D., Gaur, M., Jain, S., Shalin, V. L., & Sheth, A. P. (2025) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2024.3483577 · Cited by 19 · see citing works
Kumar, H., Yoo, S., Bernuy, A., Shi, J., Luo, H., Williams, J. J., Kuzminykh, A., Anderson, A., & Kornfield, R. (2024) · doi:10.2139/ssrn.4892998 · Cited by 9 · see citing works
Ang, B. H., Gollapalli, S. D., Du, M., & Ng, S.-K. (2025) · Journal of Medical Internet Research · doi:10.2196/59524 · Cited by 6 · see citing works
AI-simulated clinical consultations: Assessing the potential of ChatGPT to support medical training
Saggar, A., Dimitrova, V., Sarikaya, D., Hogg, D., & Darling, J. C. (2026) · Archives of Disease in Childhood · doi:10.1136/archdischild-2025-329846 · Cited by 4 · see citing works
- Authors
- Xu, X., Yao, B., Dong, Y., Gabriel, S., Yu, H., Hendler, J., Ghassemi, M., Dey, A. K., & Wang, D.
- Affiliations
- University of Washington (US)
- Rensselaer Polytechnic Institute (US)
- Stanford University (US)
- Massachusetts Institute of Technology (US)
- University of Massachusetts Lowell (US)
- Northeastern University (US)
- ORCID
- Venue
- Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
- Published
- 2024-03-06
- Volume
- 8
- Issue
- 1
- Pages
- 1-32
- ISSN
- 2474-9567 (ISSN-L)
- PMC
- PMC11806945
- OpenAlex
- W4392503764
- Cited by
- 278
- References
- 109
- Citations by year
- 2026: 86 · 2025: 121 · 2024: 69 · 2023: 2
- FWCI
- 130.94
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- DSpace@MIT (Massachusetts Institute of Technology) (submittedVersion, cc-by-nc)
- arXiv (Cornell University) (cc-by)
- Funders
- VW Foundation
- Quanta Computing
- NIH (1R01MD018424-01)
- Topics
- Social Sciences › Psychology › Social Psychology › Mental Health via Writing
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Keywords
- large language models, mental health prediction, instruction tuning, zero-shot prompting, few-shot prompting, GPT-4, GPT-3.5, alpaca, Flan-T5, racial and gender bias, ethical risks
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
