Mental Health Chatbot for Young Adults With Depressive Symptoms During the COVID-19 Pandemic: Single-Blind, Three-Arm Randomized Controlled Trial
Open accessThe DOI matches its Crossref record; 83 of the 83 references it cites were found in a registry or an index.
Abstract
Background: Depression has a high prevalence among young adults, especially during the COVID-19 pandemic. However, mental health services remain scarce and underutilized worldwide. Mental health chatbots are a novel digital technology to provide fully automated interventions for depressive symptoms. Objective: The purpose of this study was to test the clinical effectiveness and nonclinical performance of a cognitive behavioral therapy (CBT)–based mental health chatbot (XiaoE) for young adults with depressive symptoms during the COVID-19 pandemic. Methods: In a single-blind, 3-arm randomized controlled trial, participants manifesting depressive symptoms recruited from a Chinese university were randomly assigned to a mental health chatbot (XiaoE; n=49), an e-book (n=49), or a general chatbot (Xiaoai; n=50) group in a ratio of 1:1:1. Participants received a 1-week intervention. The primary outcome was the reduction of depressive symptoms according to the 9-item Patient Health Questionnaire (PHQ-9) at 1 week later (T1) and 1 month later (T2). Both intention-to-treat and per-protocol analyses were conducted under analysis of covariance models adjusting for baseline data. Controlled multiple imputation and δ-based sensitivity analysis were performed for missing data. The secondary outcomes were the level of working alliance measured using the Working Alliance Questionnaire (WAQ), usability measured using the Usability Metric for User Experience-LITE (UMUX-LITE), and acceptability measured using the Acceptability Scale (AS). Results: Participants were on average 18.78 years old, and 37.2% (55/148) were female. The mean baseline PHQ-9 score was 10.02 (SD 3.18; range 2-19). Intention-to-treat analysis revealed lower PHQ-9 scores among participants in the XiaoE group compared with participants in the e-book group and Xiaoai group at both T1 (F2,136=17.011; P<.001; d=0.51) and T2 (F2,136=5.477; P=.005; d=0.31). Better working alliance (WAQ; F2,145=3.407; P=.04) and acceptability (AS; F2,145=4.322; P=.02) were discovered with XiaoE, while no significant difference among arms was found for usability (UMUX-LITE; F2,145=0.968; P=.38). Conclusions: A CBT-based chatbot is a feasible and engaging digital therapeutic approach that allows easy accessibility and self-guided mental health assistance for young adults with depressive symptoms. A systematic evaluation of nonclinical metrics for a mental health chatbot has been established in this study. In the future, focus on both clinical outcomes and nonclinical metrics is necessary to explore the mechanism by which mental health chatbots work on patients. Further evidence is required to confirm the long-term effectiveness of the mental health chatbot via trails replicated with a longer dose, as well as exploration of its stronger efficacy in comparison with other active controls. Trial Registration: Chinese Clinical Trial Registry ChiCTR2100052532; http://www.chictr.org.cn/showproj.aspx?proj=135744
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures

![Figure 2 Efficacy for the reduction of depression symptoms in participants. The image presents the mean change from baseline in the primary outcome measure (9-item Patient Health Questionnaire [PHQ-9]) and the between-group differences in participants with XiaoE versus those with e-book and Xiaoai at postintervention and at follow-up. Means and standard errors are displayed. (A) Intention-to-treat analysis. (B) Per-protocol analysis. *P<.05; **P<.01; ***P<.001.](https://figures.cochat.ai/paper-figures/89e37df5698847099952dc3fb4d7c7a5/mental-health-chatbot-for-young-adults-with-depressive-symptoms-during-the-covid-figure-2.jpg)


Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“For instance, Chen et al [50] reported a differential attrition rate of 34% and He et al [54] reported 24 %.”
From Generative AI Mental Health Chatbots as Therapeutic Tools: Systematic Review and Meta-Analysis of Their Role in Reducing Mental Health Issues (2025) · cites this paper as [54]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“At the same time, a growing body of research shows that interactions with chatbots can contribute to improving their users’ mental health and quality of life, especially if the specific needs of the user match with chatbot capabilities (22, 24, 27, 31, 34, 40, 49–56).”
From Waiting for a digital therapist: three challenges on the path to psychotherapy delivered by artificial intelligence (2023) · cites this paper as [51]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Second, engagement and adherence appear to decrease over time29,31.”
From Use of automated conversational agents in improving young population mental health: a scoping review (2024) · cites this paper as [31]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“one incorporated NLP with dynamic programming (DP) [54]”
From Generative AI Mental Health Chatbots as Therapeutic Tools: Systematic Review and Meta-Analysis of Their Role in Reducing Mental Health Issues (2025) · cites this paper as [54]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Not all studies reported on the feasibility and acceptability of their interventions. Those that did, however, found the intervention to be feasible [57,59] and acceptable [57-59,63], met with high satisfaction [19,62], and more accepted and engaging in the treatment group than in the control group [61].”
2 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007) · Behavior Research Methods · doi:10.3758/bf03193146 · Cited by 65,971 · cited 1×
“Sample size calculation was conducted with G* Power (version 3) [72].”
The PHQ-9: Validity of a brief depression severity measure
Kroenke, K., Spitzer, R. L., & Williams, J. B. W. (2001) · Journal of General Internal Medicine · doi:10.1046/j.1525-1497.2001.016009606.x · Cited by 44,910
(2003) · Journal of Machine Learning Research · doi:10.5555/944919.944937 · Cited by 32,337
The Psychological Impact of Quarantine and How to Reduce It: Rapid Review of the Evidence
Brooks, S. K., Webster, R. K., Smith, L. E., Woodland, L., Wessely, S., Greenberg, N., & Rubin, G. J. (2020) · doi:10.2139/ssrn.3532534 · Cited by 16,394
The psychological impact of quarantine and how to reduce it: Rapid review of the evidence
Brooks, S. K., Webster, R. K., Smith, L. E., Woodland, L., Wessely, S., Greenberg, N., & Rubin, G. J. (2020) · The Lancet · doi:10.1016/s0140-6736(20)30460-8 · Cited by 16,378 · cited 1×
“Limited accessibility and acceptability were more obvious with the rising risk of mental health problems [5,6] led by quarantine and social isolation during the COVID-19 pandemic [7], especially among adolescents [8].”
RUBIN, D. B. (1976) · Biometrika · doi:10.1093/biomet/63.3.581 · Cited by 9,925
Moher, D., Hopewell, S., Schulz, K. F., Montori, V., Gotzsche, P. C., Devereaux, P. J., Elbourne, D., Egger, M., & Altman, D. G. (2010) · BMJ · doi:10.1136/bmj.c869 · Cited by 8,883 · cited 1×
“Figure 1 shows the participant flow (CONSORT flow diagram) [82].”
Cohen, J. (1992) · Psychological Bulletin · doi:10.1037//0033-2909.112.1.155 · Cited by 8,771
The global landscape of AI ethics guidelines
Jobin, A., Ienca, M., & Vayena, E. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0088-2 · Cited by 6,220
Holmes, E. A., O'Connor, R. C., Perry, V. H., Tracey, I., Wessely, S., Arseneault, L., Ballard, C., Christensen, H., Cohen Silver, R., Everall, I., et al. (2020) · The Lancet Psychiatry · doi:10.1016/s2215-0366(20)30168-1 · Cited by 5,715 · cited 1×
“Limited accessibility and acceptability were more obvious with the rising risk of mental health problems [5,6] led by quarantine and social isolation during the COVID-19 pandemic [7], especially among adolescents [8].”
ELIZA—a computer program for the study of natural language communication between man and machine
Weizenbaum, J. (1966) · Communications of the ACM · doi:10.1145/365153.365168 · Cited by 4,452 · cited 1×
“The first chatbot, ELIZA, was applied in the field of psychology, and users could input text to simulate a conversation with a Rogerian psychotherapist [14].”
Malhi, G. S., & Mann, J. J. (2018) · The Lancet · doi:10.1016/s0140-6736(18)31948-2 · Cited by 4,099 · cited 1×
“Cognitive behavioral therapy (CBT) has been continuously developed and is currently recognized as a widespread and effective evidence-based psychotherapy for depression [30,31], serving as one of the crucial theoretical frameworks for chatbot interventions.”
How stigma interferes with mental health care.
Corrigan, P. (2004) · American Psychologist · doi:10.1037/0003-066x.59.7.614 · Cited by 3,648 · cited 1×
“…many limitations in traditional face-to-face psychotherapy and mental health services, including expensive treatment, geographical limitations, few experienced therapists, and delayed treatment [2], and stigma is considered as the most significant barrier to providing mental health services [3,4].”
Timely mental health care for the 2019 novel coronavirus outbreak is urgently needed
Xiang, Y.-T., Yang, Y., Li, W., Zhang, L., Zhang, Q., Cheung, T., & Ng, C. H. (2020) · The Lancet Psychiatry · doi:10.1016/s2215-0366(20)30046-8 · Cited by 3,598 · cited 1×
“Mobile health and digital medicine have rapidly become an important area of study [9] in response to the conundrum posed by the escalating demand for mental health assistance [10] and the severe shortage of traditional health care facilities [11].”
The empirical status of cognitive-behavioral therapy: A review of meta-analyses
Butler, A. C., Chapman, J. E., Forman, E. M., & Beck, A. T. (2006) · Clinical Psychology Review · doi:10.1016/j.cpr.2005.07.003 · Cited by 3,215 · cited 1×
“Cognitive behavioral therapy (CBT) has been continuously developed and is currently recognized as a widespread and effective evidence-based psychotherapy for depression [30,31], serving as one of the crucial theoretical frameworks for chatbot interventions.”
Clement, S., Schauman, O., Graham, T., Maggioni, F., Evans-Lacko, S., Bezborodovs, N., Morgan, C., Rüsch, N., Brown, J. S. L., & Thornicroft, G. (2015) · Psychological Medicine · doi:10.1017/s0033291714000129 · Cited by 3,152 · cited 1×
“…many limitations in traditional face-to-face psychotherapy and mental health services, including expensive treatment, geographical limitations, few experienced therapists, and delayed treatment [2], and stigma is considered as the most significant barrier to providing mental health services [3,4].”
Loades, M. E., Chatburn, E., Higson-Sweeney, N., Reynolds, S., Shafran, R., Brigden, A., Linney, C., McManus, M. N., Borwick, C., & Crawley, E. (2020) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1016/j.jaac.2020.05.009 · Cited by 2,906 · cited 1×
“Limited accessibility and acceptability were more obvious with the rising risk of mental health problems [5,6] led by quarantine and social isolation during the COVID-19 pandemic [7], especially among adolescents [8].”
Fitzpatrick, K. K., Darcy, A., & Vierhile, M. (2017) · JMIR Mental Health · doi:10.2196/mental.7785 · Cited by 2,730 · cited 4×
“…better effectiveness of XiaoE for depression in comparison with that of the 2 controls for 1 week, achieving a moderate effect size ( d =0.51), which was between the effect sizes of 2 previous studies [32,33] (Woebot: d =0.44; Tess: d =0.68) and remained robust in sensitivity analysis.”
InfluentialPrevalence of mental disorders in China: a cross-sectional epidemiological study
Huang, Y., Wang, Y., Wang, H., Liu, Z., Yu, X., Yan, J., Yu, Y., Kou, C., Xu, X., Lu, J., et al. (2019) · The Lancet Psychiatry · doi:10.1016/s2215-0366(18)30511-x · Cited by 2,335 · cited 1×
“In China, the figure has been reported to be 95 million [26], and the prevalence among college students was 28.4% [27], reaching 34% during the COVID-19 pandemic [28].”
InfluentialSelf-citationLatent Dirichlet allocation (LDA) and topic modeling: models, applications, a survey
Jelodar, H., Wang, Y., Yuan, C., Feng, X., Jiang, X., Li, Y., & Zhao, L. (2019) · Multimedia Tools and Applications · doi:10.1007/s11042-018-6894-4 · Cited by 1,961 · cited 1×
“Important issues to be addressed for chatbots in the future could be extracted from the perceptions and opinions of patients [58], and thematic analysis with a topic model is a qualitative research method to accurately capture and concisely present key information in texts [59].”
A Survey of the Usages of Deep Learning for Natural Language Processing
Otter, D. W., Medina, J. R., & Kalita, J. K. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2979670 · Cited by 1,836 · cited 2×
“The technology of XiaoE is rooted in natural language processing (NLP) and deep learning [61].”
Conversational agents in healthcare: a systematic review
Laranjo, L., Dunn, A. G., Tong, H. L., Kocaballi, A. B., Chen, J., Bashir, R., Surian, D., Gallego, B., Magrabi, F., Lau, A. Y. S., et al. (2018) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocy072 · Cited by 1,388 · cited 1×
“In addition, chatbots were designed to focus on interactive capabilities instead of single psychological education for facilitating the process of psychotherapy [19].”
Establishing and maintaining long-term human-computer relationships
Bickmore, T. W., & Picard, R. W. (2005) · ACM Transactions on Computer-Human Interaction · doi:10.1145/1067860.1067867 · Cited by 1,333 · cited 1×
“A mental health chatbot provides more accessibility than traditional face-to-face counseling and psychotherapy [15,16], through which users can feel accompanied and understood [17,18].”
Torous, J., Bucci, S., Bell, I. H., Kessing, L. V., Faurholt‐Jepsen, M., Whelan, P., Carvalho, A. F., Keshavan, M., Linardon, J., & Firth, J. (2021) · World Psychiatry · doi:10.1002/wps.20883 · Cited by 1,199
Wu, T., Jia, X., Shi, H., Niu, J., Yin, X., Xie, J., & Wang, X. (2021) · Journal of Affective Disorders · doi:10.1016/j.jad.2020.11.117 · Cited by 1,138 · cited 1×
“Limited accessibility and acceptability were more obvious with the rising risk of mental health problems [5,6] led by quarantine and social isolation during the COVID-19 pandemic [7], especially among adolescents [8].”
Inkster, B., Sarda, S., & Subramanian, V. (2018) · doi:10.2196/preprints.12106 · Cited by 1,063 · cited 1×
“…tested through randomized controlled trials, providing interventions for different mental health problems, with Woebot [32], Tess [33,34], Wysa [35], Vivibot [36], and XiaoNan [37] directly targeting depression and anxiety symptoms; Shim [38], SABORI [39], and Bella [40] directly targeting…”
Chatbots and Conversational Agents in Mental Health: A Review of the Psychiatric Landscape
Vaidyam, A. N., Wisniewski, H., Halamka, J. D., Kashavan, M. S., & Torous, J. B. (2019) · The Canadian Journal of Psychiatry · doi:10.1177/0706743719828977 · Cited by 1,048 · cited 1×
“A mental health chatbot provides more accessibility than traditional face-to-face counseling and psychotherapy [15,16], through which users can feel accompanied and understood [17,18].”
Rubin, D. B. (1976) · Biometrika · doi:10.2307/2335739 · Cited by 885
Karyotaki, E., Efthimiou, O., Miguel, C., Maas genannt Bermpohl, F., Furukawa, T. A., Cuijpers, P., Individual Patient Data Meta-Analyses for Depression (IPDMA-DE) Collaboration, Riper, H., Patel, V., Mira, A., et al. (2021) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2020.4364 · Cited by 866
Strategy for intention to treat analysis in randomised trials with missing outcome data
White, I. R., Horton, N. J., Carpenter, J., statistics, R. I. M. A. S., & Pocock, S. J. (2011) · BMJ · doi:10.1136/bmj.d40 · Cited by 849 · cited 1×
“The results of both the intention-to-treat (ITT) analysis [75] on the full analysis set (all enrolled participants) and the per-protocol (PP) analysis on the PP set (participants in the full analysis set without important protocol violations leading to exclusion) were reported by including all…”
Prevalence of depressive disorders and treatment in China: a cross-sectional epidemiological study
Lu, J., Xu, X., Huang, Y., Li, T., Ma, C., Xu, G., Yin, H., Xu, X., Ma, Y., Wang, L., et al. (2021) · The Lancet Psychiatry · doi:10.1016/s2215-0366(21)00251-0 · Cited by 844 · cited 1×
“However, the use of health services for depressive disorders in China has been rather limited, with the access rate of adequate treatment being less than 0.5% [29].”
InfluentialSelf-citationLattie, E. G., Adkins, E. C., Winquist, N., Stiles-Shields, C., Wafford, Q. E., & Graham, A. K. (2019) · Journal of Medical Internet Research · doi:10.2196/12869 · Cited by 843 · cited 2×
“Mobile health and digital medicine have rapidly become an important area of study [9] in response to the conundrum posed by the escalating demand for mental health assistance [10] and the severe shortage of traditional health care facilities [11].”
The Role of Common Factors in Psychotherapy Outcomes
Cuijpers, P., Reijnders, M., & Huibers, M. J. H. (2019) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-050718-095424 · Cited by 748 · cited 1×
“Working alliance (also known as “therapeutic alliance”) represents the cooperative and emotionally connected relationship between the client and the therapist, and is considered a common factor in psychotherapy outcomes [49] and a metric to assess the computer-patient relationship as well [50,51].”
Fulmer, R., Joerin, A., Gentile, B., Lakerink, L., & Rauws, M. (2018) · doi:10.2196/preprints.9782 · Cited by 715 · cited 4×
“…better effectiveness of XiaoE for depression in comparison with that of the 2 controls for 1 week, achieving a moderate effect size ( d =0.51), which was between the effect sizes of 2 previous studies [32,33] (Woebot: d =0.44; Tess: d =0.68) and remained robust in sensitivity analysis.”
InfluentialRasa: Open Source Language Understanding and Dialogue ManagementPreprint
Bocklisch, T., Faulkner, J., Pawlowski, N., & Nichol, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.05181 · Cited by 650 · cited 1×
“The whole chatbot dialogue system has been constructed through the open-source framework RASA [62], with content about mental health produced, discussed, and supervised by a psychologist panel led by several experienced clinical and counseling experts from schools and hospitals.”
Ho, A., Hancock, J., & Miner, A. S. (2018) · Journal of Communication · doi:10.1093/joc/jqy026 · Cited by 629 · cited 1×
“A mental health chatbot provides more accessibility than traditional face-to-face counseling and psychotherapy [15,16], through which users can feel accompanied and understood [17,18].”
An overview of the features of chatbots in mental health: A scoping review
Abd-alrazaq, A. A., Alajlani, M., Alalwan, A. A., Bewick, B. M., Gardner, P., & Househ, M. (2019) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2019.103978 · Cited by 605 · cited 1×
“Chatbots in mental health are nascent [22], and although chatbots have demonstrated feasibility to provide mental health treatment, more high-quality evidence regarding the effectiveness and acceptability of mental health chatbots is needed [23], particularly during the COVID-19 pandemic [24].”
McTear, M., Callejas, Z., & Griol, D. (2016) · doi:10.1007/978-3-319-32967-3 · Cited by 594
Abd-Alrazaq, A. A., Rababeh, A., Alajlani, M., Bewick, B. M., & Househ, M. (2020) · Journal of Medical Internet Research · doi:10.2196/16021 · Cited by 503 · cited 1×
“Chatbots in mental health are nascent [22], and although chatbots have demonstrated feasibility to provide mental health treatment, more high-quality evidence regarding the effectiveness and acceptability of mental health chatbots is needed [23], particularly during the COVID-19 pandemic [24].”
Establishing the computer-patient working alliance in automated health behavior change interventions
Bickmore, T., Gruber, A., & Picard, R. (2005) · Patient Education and Counseling · doi:10.1016/j.pec.2004.09.008 · Cited by 494 · cited 1×
“Working alliance (also known as “therapeutic alliance”) represents the cooperative and emotionally connected relationship between the client and the therapist, and is considered a common factor in psychotherapy outcomes [49] and a metric to assess the computer-patient relationship as well [50,51].”
Deng, J., Zhou, F., Hou, W., Silver, Z., Wong, C. Y., Chang, O., Drakos, A., Zuo, Q. K., & Huang, E. (2021) · Psychiatry Research · doi:10.1016/j.psychres.2021.113863 · Cited by 474 · cited 1×
“In China, the figure has been reported to be 95 million [26], and the prevalence among college students was 28.4% [27], reaching 34% during the COVID-19 pandemic [28].”
InfluentialThe impact of digital technology on psychological treatments and their dissemination
Fairburn, C. G., & Patel, V. (2017) · Behaviour Research and Therapy · doi:10.1016/j.brat.2016.08.012 · Cited by 448 · cited 1×
“Driven by digital technologies, such as computers, the internet, mobile devices, mobile software apps, and virtual reality (VR), treatment for mental health problems has undergone an unprecedented transformation [12].”
Ly, K. H., Ly, A.-M., & Andersson, G. (2017) · Internet Interventions · doi:10.1016/j.invent.2017.10.002 · Cited by 368
The Personalization of Conversational Agents in Health Care: Systematic Review
Kocaballi, A. B., Berkovsky, S., Quiroz, J. C., Laranjo, L., Tong, H. L., Rezazadegan, D., Briatore, A., & Coiera, E. (2019) · Journal of Medical Internet Research · doi:10.2196/15360 · Cited by 332 · cited 1×
“The utilization of user profiles or user models to support personalized and adaptive features, and assessments for personalization are still limited in mental health chatbots [89].”
Chatbots in the fight against the COVID-19 pandemic
Miner, A. S., Laranjo, L., & Kocaballi, A. B. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0280-0 · Cited by 322 · cited 1×
“Chatbots in mental health are nascent [22], and although chatbots have demonstrated feasibility to provide mental health treatment, more high-quality evidence regarding the effectiveness and acceptability of mental health chatbots is needed [23], particularly during the COVID-19 pandemic [24].”
Perceptions and opinions of patients about mental health chatbots: scoping review
Abd-Alrazaq, A. A., Alajlani, M., Ali, N., Denecke, K., Bewick, B. M., & Househ, M. (2021) · Journal of Medical Internet Research · doi:10.2196/17828 · Cited by 301 · cited 1×
“Important issues to be addressed for chatbots in the future could be extracted from the perceptions and opinions of patients [58], and thematic analysis with a topic model is a qualitative research method to accurately capture and concisely present key information in texts [59].”
Liu, H., Peng, H., Song, X., Xu, C., & Zhang, M. (2022) · Internet Interventions · doi:10.1016/j.invent.2022.100495 · Cited by 296 · cited 3×
“…of the relationship and working alliance in digital treatment, and several randomized controlled trials of mental health chatbots had employed the Working Alliance Inventory (WAI) as a measurement method of working alliance, with all of these demonstrating good measure effects [37,56,57].”
InfluentialPrevalence of depression among Chinese university students: a systematic review and meta-analysis
Gao, L., Xie, Y., Jia, C., & Wang, W. (2020) · Scientific Reports · doi:10.1038/s41598-020-72998-1 · Cited by 279 · cited 1×
“In China, the figure has been reported to be 95 million [26], and the prevalence among college students was 28.4% [27], reaching 34% during the COVID-19 pandemic [28].”
InfluentialConversational agents in the treatment of mental health problems: mixed-method systematic review
Gaffney, H., Mansell, W., & Tai, S. (2019) · JMIR Mental Health · doi:10.2196/14166 · Cited by 275 · cited 2×
“Based on the principles of CBT, multiturn dialogue [63] and personalized customization were taken as the main intervention forms by referring to the content and process of several mature CBT-based chatbots [20] and internet-delivered cognitive behavioral therapy (ICBT) apps [64].”
Kazdin, A. E. (2019) · Journal of Child Psychology and Psychiatry · doi:10.1111/jcpp.12937 · Cited by 274 · cited 1×
“A mental health chatbot provides more accessibility than traditional face-to-face counseling and psychotherapy [15,16], through which users can feel accompanied and understood [17,18].”
Morris, R. R., Kouddous, K., Kshirsagar, R., & Schueller, S. M. (2018) · Journal of Medical Internet Research · doi:10.2196/10148 · Cited by 273 · cited 1×
“The concept of empathic chatbots has been proposed [87], accompanied with system design and development [88], but there is no mature product present and effectiveness has not been tested yet.”
Greer, S., Ramo, D., Chang, Y.-J., Fu, M., Moskowitz, J., & Haritatos, J. (2019) · JMIR mHealth and uHealth · doi:10.2196/15018 · Cited by 265 · cited 1×
“…randomized controlled trials, providing interventions for different mental health problems, with Woebot [32], Tess [33,34], Wysa [35], Vivibot [36], and XiaoNan [37] directly targeting depression and anxiety symptoms; Shim [38], SABORI [39], and Bella [40] directly targeting stress,…”
Effectiveness of an Empathic Chatbot in Combating Adverse Effects of Social Exclusion on Mood
de Gennaro, M., Krumhuber, E. G., & Lucas, G. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.03061 · Cited by 265
Darcy, A., Daniels, J., Salinger, D., Wicks, P., & Robinson, A. (2021) · JMIR Formative Research · doi:10.2196/27868 · Cited by 260
Beatty, C., Malik, T., Meheli, S., & Sinha, C. (2022) · Frontiers in Digital Health · doi:10.3389/fdgth.2022.847991 · Cited by 224
Hauser-Ulrich, S., Künzli, H., Meier-Peterhans, D., & Kowatsch, T. (2020) · JMIR mHealth and uHealth · doi:10.2196/15806 · Cited by 210 · cited 1×
“…of the relationship and working alliance in digital treatment, and several randomized controlled trials of mental health chatbots had employed the Working Alliance Inventory (WAI) as a measurement method of working alliance, with all of these demonstrating good measure effects [37,56,57].”
Cognitive Bibliotherapy for Depression: A Meta-Analysis.
Gregory, R. J., Schwer Canning, S., Lee, T. W., & Wise, J. C. (2004) · Professional Psychology: Research and Practice · doi:10.1037/0735-7028.35.3.275 · Cited by 203 · cited 1×
“…evidence for the intervention itself was still limited, and the e-book intervention, as a self-help approach, only involved the concept of psychological education, and it was not equipped with a complete set of programs for psychological therapy [83] or designed for multiple or recurring sessions.”
Cro, S., Morris, T. P., Kenward, M. G., & Carpenter, J. R. (2020) · Statistics in Medicine · doi:10.1002/sim.8569 · Cited by 203 · cited 1×
“Using mi impute within Stata (version 15; StataCorp), we processed missing data via multiple imputation (MI) methods and performed further sensitivity analysis via δ -based methods [77].”
Klos, M. C., Escoredo, M., Joerin, A., Lemos, V. N., Rauws, M., & Bunge, E. L. (2021) · JMIR Formative Research · doi:10.2196/20678 · Cited by 200 · cited 1×
“…has been tested through randomized controlled trials, providing interventions for different mental health problems, with Woebot [32], Tess [33,34], Wysa [35], Vivibot [36], and XiaoNan [37] directly targeting depression and anxiety symptoms; Shim [38], SABORI [39], and Bella [40] directly…”
Discordance between reported intention-to-treat and per protocol analyses
Porta, N., Bonet, C., & Cobo, E. (2007) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2006.09.013 · Cited by 189 · cited 2×
“…(ITT) analysis [75] on the full analysis set (all enrolled participants) and the per-protocol (PP) analysis on the PP set (participants in the full analysis set without important protocol violations leading to exclusion) were reported by including all available observations in the analysis [76].”
Jang, S., Kim, J., Kim, S., Hong, J., Kim, S., & Kim, E. J. (2021) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2021.104440 · Cited by 176 · cited 1×
“…process and content, was assessed using a 5-point Likert scale (AS), referring to items used in previous studies on mental health chatbots [33,71] covering overall satisfaction, content satisfaction, emotional awareness, learning new knowledge, relevance to daily life, and promotion of the…”
Lim, S. M., Shiau, C. W. C., Cheng, L. J., & Lau, Y. (2022) · Behavior Therapy · doi:10.1016/j.beth.2021.09.007 · Cited by 163 · cited 1×
“Chatbots are intended to foster collaboration, integration, and co-development between psychological science and other fields [47].”
Technical metrics used to evaluate health care chatbots: scoping review
Abd-Alrazaq, A., Safi, Z., Alajlani, M., Warren, J., Househ, M., & Denecke, K. (2020) · Journal of Medical Internet Research · doi:10.2196/18301 · Cited by 150 · cited 1×
“As an alternative and useful precursor to clinical effectiveness, nonclinical metrics are just as important as clinical outcomes and may contribute to further exploring the mechanism by which the mental health chatbots work [48].”
Xu, C., Xu, Y., Xu, S., Zhang, Q., Liu, X., Shao, Y., Xu, X., Peng, L., & Li, M. (2020) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2020.00858 · Cited by 136 · cited 1×
“Mobile health and digital medicine have rapidly become an important area of study [9] in response to the conundrum posed by the escalating demand for mental health assistance [10] and the severe shortage of traditional health care facilities [11].”
Burton, C., Szentagotai Tatar, A., McKinstry, B., Matheson, C., Matu, S., Moldovan, R., Macnab, M., Farrow, E., David, D., Pagliari, C., et al. (2016) · Journal of Telemedicine and Telecare · doi:10.1177/1357633x15609793 · Cited by 125 · cited 1×
“…XiaoNan [37] directly targeting depression and anxiety symptoms; Shim [38], SABORI [39], and Bella [40] directly targeting stress, well-being, or quality of life; and MYLO [41-43] and Help4Mood [44] directly targeting general psychological distress such as problem solving and negative cognition.”
Gaffney, H., Mansell, W., Edwards, R., & Wright, J. (2014) · Behavioural and Cognitive Psychotherapy · doi:10.1017/s135246581300060x · Cited by 122 · cited 1×
“…XiaoNan [37] directly targeting depression and anxiety symptoms; Shim [38], SABORI [39], and Bella [40] directly targeting stress, well-being, or quality of life; and MYLO [41-43] and Help4Mood [44] directly targeting general psychological distress such as problem solving and negative cognition.”
Prochaska, J. J., Vogel, E. A., Chieng, A., Baiocchi, M., Maglalang, D. D., Pajarito, S., Weingardt, K. R., Darcy, A., & Robinson, A. (2021) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2021.108986 · Cited by 113 · cited 1×
“…of the relationship and working alliance in digital treatment, and several randomized controlled trials of mental health chatbots had employed the Working Alliance Inventory (WAI) as a measurement method of working alliance, with all of these demonstrating good measure effects [37,56,57].”
Suganuma, S., Sakamoto, D., & Shimoyama, H. (2018) · JMIR Mental Health · doi:10.2196/10454 · Cited by 112 · cited 1×
“…with Woebot [32], Tess [33,34], Wysa [35], Vivibot [36], and XiaoNan [37] directly targeting depression and anxiety symptoms; Shim [38], SABORI [39], and Bella [40] directly targeting stress, well-being, or quality of life; and MYLO [41-43] and Help4Mood [44] directly targeting general…”
Bird, T., Mansell, W., Wright, J., Gaffney, H., & Tai, S. (2018) · Behavioural and Cognitive Psychotherapy · doi:10.1017/s1352465817000820 · Cited by 109
Loveys, K., Sagar, M. A., Pickering, I., & Broadbent, E. (2021) · JMIR Mental Health · doi:10.2196/31586 · Cited by 101
Clarke, J., Proudfoot, J., Whitton, A., Birch, M.-R., Boyd, M., Parker, G., Manicavasagar, V., Hadzi-Pavlovic, D., & Fogarty, A. (2016) · JMIR Mental Health · doi:10.2196/mental.4656 · Cited by 86 · cited 1×
“…doubt regarding whether machines can provide emotional experiences, they typically respond better to agents that express emotions than those that do not [85], illustrating the importance of a positive therapeutic alliance in the internet environment in the absence of therapist support [86].”
Bonding With Bot: User Feedback on a Chatbot for Social Isolation
Dosovitsky, G., & Bunge, E. L. (2021) · Frontiers in Digital Health · doi:10.3389/fdgth.2021.735053 · Cited by 86 · cited 2×
“…of “relationship” reflected the establishment, development, and function of the relationship between XiaoE and participants, as Dosovitsky et al [52] found that individuals can form a positive bond with an AI chatbot owing to its personality traits, such as being caring, open to listening, and…”
(2022) · doi:10.2196/32630 · Cited by 83 · cited 1×
“Three recent studies [52-55] by Dosovitsky et al, Beatty et al, and Darcy et al had emphasized the viability and significance of the relationship and working alliance in digital treatment, and several randomized controlled trials of mental health chatbots had employed the Working Alliance Inventory…”
Bennion, M. R., Hardy, G. E., Moore, R. K., Kellett, S., & Millings, A. (2020) · Journal of Medical Internet Research · doi:10.2196/16794 · Cited by 68 · cited 1×
“…XiaoNan [37] directly targeting depression and anxiety symptoms; Shim [38], SABORI [39], and Bella [40] directly targeting stress, well-being, or quality of life; and MYLO [41-43] and Help4Mood [44] directly targeting general psychological distress such as problem solving and negative cognition.”
Enhancing Conversational Agents with Empathic Abilities
Casas, J., Spring, T., Daher, K., Mugellini, E., Khaled, O. A., & Cudré-Mauroux, P. (2021) · ACM International Conference on Intelligent Virtual Agents (IVA) · doi:10.1145/3472306.3478344 · Cited by 59 · cited 1×
“The concept of empathic chatbots has been proposed [87], accompanied with system design and development [88], but there is no mature product present and effectiveness has not been tested yet.”
Borsci, S., Buckle, P., & Walne, S. (2019) · Applied Ergonomics · doi:10.1016/j.apergo.2019.103007 · Cited by 58 · cited 1×
“The secondary outcomes were the scores of the Working Alliance Questionnaire (WAQ) [68], the Usability Metric for User Experience-LITE (UMUX-LITE) [69], and the Acceptability Scale (AS).”
Survey on psychotherapy chatbots
Xu, B., & Zhuang, Z. (2020) · Concurrency and Computation Practice and Experience · doi:10.1002/cpe.6170 · Cited by 58 · cited 1×
“However, studies have shown that mental health chatbots have some risks as well, such as “misunderstanding,” which may lead to ineffective or even harmful interventions, lack of crisis warning mechanisms, and lack of privacy protection [21].”
Tan, P.-T., Cro, S., Van Vogt, E., Szigeti, M., & Cornelius, V. R. (2021) · BMC Medical Research Methodology · doi:10.1186/s12874-021-01261-6 · Cited by 42 · cited 1×
“For a continuous outcome, δ , the offset parameter can represent the difference in the mean response between the observed and unobserved cases [80].”
Virtual Humans in Health-Related Interventions: A Meta-Analysis
Ma, T., Sharifi, H., & Chattopadhyay, D. (2019) · Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290607.3312853 · Cited by 37
Gardiner, P. M., Bickmore, T., Yinusa-Nyahkoon, L. S., Reichert, M., Julce, C., Sidduri, N., Martin-Howard, J., Woodhams, E. J., Aryan, J., Zhang, Z., et al. (2021) · Frontiers in Endocrinology · doi:10.3389/fendo.2020.571705 · Cited by 36 · cited 1×
“…and format of chatbots for mental health problems have evolved from script bots with only text communication to embodied conversational agents [45] with image and voice, and digital humans [40] and virtual humans [46], which discern and control emotional and facial expressions during…”
Model of Multi-turn Dialogue in Emotional Chatbot
Kao, C.-H., Chen, C.-C., & Tsai, Y.-T. (2019) · Conference on Technologies and Applications of Artificial Intelligence (TAAI) · doi:10.1109/taai48200.2019.8959855 · Cited by 31 · cited 1×
“Based on the principles of CBT, multiturn dialogue [63] and personalized customization were taken as the main intervention forms by referring to the content and process of several mature CBT-based chatbots [20] and internet-delivered cognitive behavioral therapy (ICBT) apps [64].”
Computerised therapy for psychiatric disorders
Christensen, H. (2007) · The Lancet · doi:10.1016/s0140-6736(07)61065-4 · Cited by 20 · cited 1×
“…many limitations in traditional face-to-face psychotherapy and mental health services, including expensive treatment, geographical limitations, few experienced therapists, and delayed treatment [2], and stigma is considered as the most significant barrier to providing mental health services [3,4].”
Abdulrahman, A., & Richards, D. (2019) · ACM International Conference on Intelligent Virtual Agents (IVA) · doi:10.1145/3308532.3329413 · Cited by 18 · cited 1×
“Working alliance (also known as “therapeutic alliance”) represents the cooperative and emotionally connected relationship between the client and the therapist, and is considered a common factor in psychotherapy outcomes [49] and a metric to assess the computer-patient relationship as well [50,51].”
Research works printed but not linked: 7
- 10.2196/mental.9782 · doi:10.2196/mental.9782 · Paper page
- 10.2196/12106 · doi:10.2196/12106 · Paper page
- Yi, X, 2021, Chinese Journal of Clinical Psychology, 29, 962
- BocklischTFaulknerJPawlowskiNRasa: Open Source Language Understanding and Dialogue ManagementarXiv20172022-11-11https://arxiv.org/abs/1712.05181
- Johnstone, M, 2007, I Had a Black Dog
- Xu, Z, 2011, Chinese Journal of Clinical Psychology, 449
- 10.1037/0033-2909.112.1.155 · doi:10.1037/0033-2909.112.1.155 · Paper page
Cited by
Li, H., Zhang, R., Lee, Y.-C., Kraut, R. E., & Mohr, D. C. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00979-5 · Cited by 598
“…process 17,26,29 ( n = 3) Accessibility 17,29 ( n = 2) Interaction mode 33 ( n = 1) Factors associated with negative user experience 17,21,26,28,29,33,41,46 ( n = 8) Content ( n = 4): (cid:129) Topic of content 17,28,29 ( n = 3) (cid:129) Format of content 17,46 ( n = 2) Impersonal 17,29 (…”
BackgroundMethodInfluentialRoles, Users, Benefits, and Limitations of Chatbots in Health Care: Rapid Review (Preprint)
Laymouna, M., Ma, Y., Lessard, D., Schuster, T., Engler, K., & Lebouché, B. (2024) · doi:10.2196/preprints.56930 · Cited by 302
Roles, Users, Benefits, and Limitations of Chatbots in Health Care: Rapid Review
Laymouna, M., Ma, Y., Lessard, D., Schuster, T., Engler, K., & Lebouché, B. (2024) · Journal of Medical Internet Research · doi:10.2196/56930 · Cited by 305
He, Y., Yang, L., Qian, C., Li, T., Su, Z., Zhang, Q., & Hou, X. (2023) · Journal of Medical Internet Research · doi:10.2196/43862 · Cited by 213
“…that CAIs can help users feel accompanied and understood [19,20]; several studies found that users establish therapeutic bonds with chatbots [21-24]; and other studies revealed some hazards associated with CAIs, including misunderstanding that may result in inefficient or even harmful…”
Self-citationAI Chatbots for Mental Health: A Scoping Review of Effectiveness, Feasibility, and Applications
Casu, M., Triscari, S., Battiato, S., Guarnera, L., & Caponnetto, P. (2024) · Applied Sciences · doi:10.3390/app14135889 · Cited by 186
“[61], Olano-Espinosa et al. [66], Fitzsimmon-Craft et al. [67], He et al. [58], Bennion et al. [71], Greer et al. [69], and Oh et al. [70] exhibited a low risk of bias in all domains.”
Influential
Metadata
- Authors
- He, Y., Yang, L., Zhu, X., Wu, B., Zhang, S., Qian, C., & Tian, T.
- Affiliations
- Tianjin University (CN)
- Tianjin Vocational Institute (CN)
- Corresponding author
- Yang, Li
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2022-11-21
- Volume
- 24
- Issue
- 11
- Pages
- e40719
- ISSN
- 1438-8871 (ISSN-L)
- PMC
- PMC9680932
- OpenAlex
- W4308687068
- Cited by
- 178
- References
- 81
- Citations by year
- 2026: 60 · 2025: 57 · 2024: 38 · 2023: 23
- FWCI
- 26.05
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Journal of Medical Internet Research (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Funders
- National Social Science Fund of China
- Topics
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Health Sciences › Medicine › Psychiatry and Mental health › Mental Health Treatment and Access
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Anxiety, Depression, Psychometrics, Treatment, Cognitive Processes
- Keywords
- mental health chatbots, depressive symptoms, young adults, COVID-19 pandemic, cognitive behavioral therapy, therapeutic relationship, acceptability, usability, digital mental health interventions, university students, PHQ-9, attrition, attrition bias
- MeSH terms
- COVID-19, Adolescent, Depression, Female, Humans, Male, Mental Health, Single-Blind Method, Young Adult, Pandemics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.