Cited by 33Digital Mental Health InterventionsSocial Robot Interaction and HRIEmotion and Mood Recognition

An “Integrative Survey on Mental Health Conversational Agents to Bridge Computer Science and Medical Perspectives”

Open access
Cho, Young, Rai, Sunny, Ungar, Lyle, Sedoc, João, Guntuku, Sharath·2023·Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing

Abstract

Mental health conversational agents (a.k.a. chatbots) are widely studied for their potential to offer accessible support to those experiencing mental health challenges. Previous surveys on the topic primarily consider papers published in either computer science or medicine, leading to a divide in understanding and hindering the sharing of beneficial knowledge between both domains. To bridge this gap, we conduct a comprehensive literature review using the PRISMA framework, reviewing 534 papers published in both computer science and medicine. Our systematic review reveals 136 key papers on building mental health-related conversational agents with diverse characteristics of modeling and experimental design techniques. We find that computer science papers focus on LLM techniques and evaluating response quality using automated metrics with little attention to the application while medical papers use rule-based conversational agents and outcome metrics to measure the health outcomes of participants. Based on our findings on transparency, ethics, and cultural heterogeneity in this review, we provide a few recommendations to help bridge the disciplinary divide and enable the cross-disciplinary development of mental health conversational agents.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.18653/v1/2023.emnlp-main.698

Cites

149 of 149 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

1 retracted

Flagged first, then by citations

Research works printed but not linked: 13

  1. Mohammad Rafayet Ali, Seyedeh Zahra Razavi, Raina Langevin, Abdullah Al Mamun, Benjamin Kane, Reza Rawassizadeh, Lenhart K Schubert, and Ehsan Hoque. 2020. A virtual conversational agent for teens with autism spectrum disorder: Experimental results and design lessons. In Proceedings of the 20th ACM International Conference on Intelligent Virtual Agents, pages 1-8.
  2. Avisha Das, Salih Selek, Alia R Warner, Xu Zuo, Yan Hu, Vipina Kuttichi Keloth, Jianfu Li, W Jim Zheng, and Hua Xu. 2022a. Conversational bots for psy- chotherapy: A study of generative transformer mod- els using domain-specific dialogues. In Proceedings of the 21st Workshop on Biomedical Language Pro- cessing, pages 285-297.
  3. Tessa Eagle, Conrad Blau, Sophie Bales, Noopur Desai, Victor Li, and Steve Whittaker. 2022. "i don't know what you mean byi am anxious'": A new method for evaluating conversational agent responses to stan- dardized mental health inputs for anxiety and depres- sion. ACM Transactions on Interactive Intelligent Systems (TiiS), 12(2):1-23.
  4. Minha Lee, Sander Ackermans, Nena Van As, Hanwen Chang, Enzo Lucas, and Wijnand IJsselsteijn. 2019. Caring for vincent: a chatbot for self-compassion. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, pages 1-13.
  5. Martin H Luerssen and Tim Hawke. 2018. Virtual agents as a service: Applications in healthcare. In Proceedings of the 18th International Conference on Intelligent Virtual Agents, pages 107-112.
  6. SoHyun Park, Anja Thieme, Jeongyun Han, Sungwoo Lee, Wonjong Rhee, and Bongwon Suh. 2021. "i wrote as if i were telling a story to someone i knew.": Designing chatbot interactions for expressive writing in mental health. In Designing Interactive Systems Conference 2021, pages 926-941.
  7. Courtney Potts, Raymond Bond, Maurice D Mulvenna, Edel Ennis, Andrea Bickerdike, Edward K Coughlan, Thomas Broderick, Con Burns, Michael F McTear, Lauri Kuosmanen, et al. 2021. Insights and lessons learned from trialling a mental health chatbot in the wild. In 2021 IEEE Symposium on Computers and Communications (ISCC), pages 1-6. IEEE.
  8. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9.
  9. Beatrice Rammstedt, Christoph J Kemper, Mira Céline Klein, Constanze Beierlein, and Anastassiya Koval- eva. 2013. A short scale for assessing the big five dimensions of personality: 10 item big five inventory (bfi-10). methods, data, analyses, 7(2):17.
  10. Hyeyoung Ryu, Soyeon Kim, Dain Kim, Sooan Han, Keeheon Lee, and Younah Kang. 2020. Simple and steady interactions win the healthy mentality: de- signing a chatbot service for the elderly. Proceed- ings of the ACM on Human-Computer Interaction, 4(CSCW2):1-25.
  11. Tulika Saha, Saichethan Reddy, Anindya Das, Sriparna Saha, and Pushpak Bhattacharyya. 2022b. A shoul- der to cry on: Towards a motivational virtual assistant for assuaging mental agony. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Hu- man Language Technologies, pages 2436-2449.
  12. Jessica Schroeder, Chelsey Wilkes, Kael Rowan, Ar- turo Toledo, Ann Paradiso, Mary Czerwinski, Gloria Mark, and Marsha M Linehan. 2018. Pocket skills: A conversational mobile web app to support dialec- tical behavioral therapy. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, pages 1-15.
  13. Ji Youn Shin and Jina Huh-Yoo. 2020. Designing ev- eryday conversational agents for managing health and wellness: A study of alexa skills reviews. In Pro- ceedings of the 14th EAI International Conference on Pervasive Computing Technologies for Healthcare, pages 50-61.

Cited by

Sorted by citations

Authors
Cho, Y., Rai, S., Ungar, L., Sedoc, J., & Guntuku, S.
Affiliations
  • University of Pennsylvania (US)
  • New York University (US)
Venue
Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing
Published
2023
Volume
2023
Pages
11346-11369
OpenAlex
W4389523925
Cited by
33
References
145
Citations by year
2026: 17 · 2025: 11 · 2024: 3 · 2023: 2
FWCI
16.28
Type
conference paper
Language
en
Open access
Licence
cc-by
Locations
  • Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • PubMed Central (acceptedVersion)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Funders
  • University of Pennsylvania
  • National Institutes of Health
  • National Institute on Minority Health and Health Disparities (R01MD018340)
Topics
  • Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
  • Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Emotion and Mood Recognition
Keywords
mental health chatbots, chatbots, interdisciplinary research, large language models, rule-based chatbots, response quality evaluation, health outcome measurement, transparency, ethics, cultural heterogeneity, mental health support

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review