CoChat

Mental Health Chatbot for Young Adults With Depressive Symptoms During the COVID-19 Pandemic: Single-Blind, Three-Arm Randomized Controlled Trial

Open access
He, Yuhao, Yang, Li, Zhu, Xiaokun, Wu, Bin, Zhang, Shuo, Qian, Chunlian, Tian, Tian · 2022 · Journal of Medical Internet Research

The DOI matches its Crossref record; 83 of the 83 references it cites were found in a registry or an index.

Abstract

Background: Depression has a high prevalence among young adults, especially during the COVID-19 pandemic. However, mental health services remain scarce and underutilized worldwide. Mental health chatbots are a novel digital technology to provide fully automated interventions for depressive symptoms. Objective: The purpose of this study was to test the clinical effectiveness and nonclinical performance of a cognitive behavioral therapy (CBT)–based mental health chatbot (XiaoE) for young adults with depressive symptoms during the COVID-19 pandemic. Methods: In a single-blind, 3-arm randomized controlled trial, participants manifesting depressive symptoms recruited from a Chinese university were randomly assigned to a mental health chatbot (XiaoE; n=49), an e-book (n=49), or a general chatbot (Xiaoai; n=50) group in a ratio of 1:1:1. Participants received a 1-week intervention. The primary outcome was the reduction of depressive symptoms according to the 9-item Patient Health Questionnaire (PHQ-9) at 1 week later (T1) and 1 month later (T2). Both intention-to-treat and per-protocol analyses were conducted under analysis of covariance models adjusting for baseline data. Controlled multiple imputation and δ-based sensitivity analysis were performed for missing data. The secondary outcomes were the level of working alliance measured using the Working Alliance Questionnaire (WAQ), usability measured using the Usability Metric for User Experience-LITE (UMUX-LITE), and acceptability measured using the Acceptability Scale (AS). Results: Participants were on average 18.78 years old, and 37.2% (55/148) were female. The mean baseline PHQ-9 score was 10.02 (SD 3.18; range 2-19). Intention-to-treat analysis revealed lower PHQ-9 scores among participants in the XiaoE group compared with participants in the e-book group and Xiaoai group at both T1 (F2,136=17.011; P<.001; d=0.51) and T2 (F2,136=5.477; P=.005; d=0.31). Better working alliance (WAQ; F2,145=3.407; P=.04) and acceptability (AS; F2,145=4.322; P=.02) were discovered with XiaoE, while no significant difference among arms was found for usability (UMUX-LITE; F2,145=0.968; P=.38). Conclusions: A CBT-based chatbot is a feasible and engaging digital therapeutic approach that allows easy accessibility and self-guided mental health assistance for young adults with depressive symptoms. A systematic evaluation of nonclinical metrics for a mental health chatbot has been established in this study. In the future, focus on both clinical outcomes and nonclinical metrics is necessary to explore the mechanism by which mental health chatbots work on patients. Further evidence is required to confirm the long-term effectiveness of the mental health chatbot via trails replicated with a longer dose, as well as exploration of its stronger efficacy in comparison with other active controls. Trial Registration: Chinese Clinical Trial Registry ChiCTR2100052532; http://www.chictr.org.cn/showproj.aspx?proj=135744

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.2196/40719

Cites

83 of 83 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 Flow of participants (CONSORT). CSMHSS, College Students Mental Health Screening Scale.
Figure 1 Flow of participants (CONSORT). CSMHSS, College Students Mental Health Screening Scale.CC BY · PubMed Central
Figure 2 Efficacy for the reduction of depression symptoms in participants. The image presents the mean change from baseline in the primary outcome measure (9-item Patient Health Questionnaire [PHQ-9]) and the between-group differences in participants with XiaoE versus those with e-book and Xiaoai at postintervention and at follow-up. Means and standard errors are displayed. (A) Intention-to-treat analysis. (B) Per-protocol analysis. *P<.05; **P<.01; ***P<.001.
Figure 2 Efficacy for the reduction of depression symptoms in participants. The image presents the mean change from baseline in the primary outcome measure (9-item Patient Health Questionnaire [PHQ-9]) and the between-group differences in participants with XiaoE versus those with e-book and Xiaoai at postintervention and at follow-up. Means and standard errors are displayed. (A) Intention-to-treat analysis. (B) Per-protocol analysis. *P<.05; **P<.01; ***P<.001.CC BY · PubMed Central
Figure 3 Use and engagement with XiaoE. The image shows the frequency and duration of interaction with the chatbot and the trend of daily interactions and interactions for 12 time periods per day in the XiaoE group during the intervention. (A) Daily engagement. The x-axis represents each day of the 1-week intervention. (B) Engagement for 12 time periods. The x-axis represents each time period in 1 day.
Figure 3 Use and engagement with XiaoE. The image shows the frequency and duration of interaction with the chatbot and the trend of daily interactions and interactions for 12 time periods per day in the XiaoE group during the intervention. (A) Daily engagement. The x-axis represents each day of the 1-week intervention. (B) Engagement for 12 time periods. The x-axis represents each time period in 1 day.CC BY · PubMed Central
Figure 4 Chart of themes and perplexity. The image shows the perplexity under different number of themes for participant feedback of the 2 questions, "What was your best experience using XiaoE?" and "What was your worst experience using XiaoE?." (A) Themes-perplexity chart of “best experience.” (B) Themes-perplexity chart of “worst experience”.
Figure 4 Chart of themes and perplexity. The image shows the perplexity under different number of themes for participant feedback of the 2 questions, "What was your best experience using XiaoE?" and "What was your worst experience using XiaoE?." (A) Themes-perplexity chart of “best experience.” (B) Themes-perplexity chart of “worst experience”.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

2 more claims about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 7

  1. 10.2196/mental.9782 · doi:10.2196/mental.9782 · Paper page
  2. 10.2196/12106 · doi:10.2196/12106 · Paper page
  3. Yi, X, 2021, Chinese Journal of Clinical Psychology, 29, 962
  4. BocklischTFaulknerJPawlowskiNRasa: Open Source Language Understanding and Dialogue ManagementarXiv20172022-11-11https://arxiv.org/abs/1712.05181
  5. Johnstone, M, 2007, I Had a Black Dog
  6. Xu, Z, 2011, Chinese Journal of Clinical Psychology, 449
  7. 10.1037/0033-2909.112.1.155 · doi:10.1037/0033-2909.112.1.155 · Paper page

Cited by

See all 180

Metadata

Authors
He, Y., Yang, L., Zhu, X., Wu, B., Zhang, S., Qian, C., & Tian, T.
Affiliations
  • Tianjin University (CN)
  • Tianjin Vocational Institute (CN)
Corresponding author
Yang, Li
Venue
Journal of Medical Internet Research
Published
2022-11-21
Volume
24
Issue
11
Pages
e40719
ISSN
1438-8871 (ISSN-L)
OpenAlex
W4308687068
Cited by
178
References
81
Citations by year
2026: 60 · 2025: 57 · 2024: 38 · 2023: 23
FWCI
26.05
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • Journal of Medical Internet Research (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
Funders
National Social Science Fund of China
Topics
  • Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
  • Health Sciences › Medicine › Psychiatry and Mental health › Mental Health Treatment and Access
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Anxiety, Depression, Psychometrics, Treatment, Cognitive Processes
Keywords
mental health chatbots, depressive symptoms, young adults, COVID-19 pandemic, cognitive behavioral therapy, therapeutic relationship, acceptability, usability, digital mental health interventions, university students, PHQ-9, attrition, attrition bias
MeSH terms
COVID-19, Adolescent, Depression, Female, Humans, Male, Mental Health, Single-Blind Method, Young Adult, Pandemics

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.