CoChat

Chatbot-Delivered Cognitive Behavioral Therapy in Adolescents With Depression and Anxiety During the COVID-19 Pandemic: Feasibility and Acceptability Study

Open access
Nicol, Ginger, Wang, Ruoyun, Graham, Sharon, Dodd, Sherry, Garbutt, Jane · 2022 · JMIR Formative Research

The DOI matches its Crossref record; 55 of the 55 references it cites were found in a registry or an index; 13 of the 50 sources its claims cite were checked for support.

Abstract

Background: Symptoms of depression and anxiety, suicidal ideation, and self-harm have escalated among adolescents to crisis levels during the COVID-19 pandemic. As a result, primary care providers (PCPs) are often called on to provide first-line care for these youth. Digital health interventions can extend mental health specialty care, but few are evidence based. We evaluated the feasibility of delivering an evidence-based mobile health (mHealth) app with an embedded conversational agent to deliver cognitive behavioral therapy (CBT) to symptomatic adolescents presenting in primary care settings during the pandemic. Objective: In this 12-week pilot study, we evaluated the feasibility of delivering the app-based intervention to adolescents aged 13 to 17 years with moderate depressive symptoms who were treated in a practice-based research network (PBRN) of academically affiliated primary care clinics. We also obtained preliminary estimates of app acceptability, effectiveness, and usability. Methods: This small, pilot randomized controlled trial (RCT) evaluated depressive symptom severity in adolescents randomized to the app or to a wait list control condition. The primary end point was depression severity at 4-weeks, measured by the 9-item Patient Health Questionnaire (PHQ-9). Data on acceptability, feasibility, and usability were collected from adolescents and their parent or legal guardian. Qualitative interviews were conducted with 13 PCPs from 11 PBRN clinics to identify facilitators and barriers to incorporating mental health apps in treatment planning for adolescents with depression and anxiety. Results: The pilot randomized 18 participants to the app (n=10, 56%) or to a wait list control condition (n=8, 44%); 17 participants were included in the analysis, and 1 became ineligible upon chart review due to lack of eligibility based on documented diagnosis. The overall sample was predominantly female (15/17, 88%), White (15/17, 88%), and privately insured (15/17, 88%). Mean PHQ-9 scores at 4 weeks decreased by 3.3 points in the active treatment group (representing a shift in mean depression score from moderate to mild symptom severity categories) and 2 points in the wait list control group (no shift in symptom severity category). Teen- and parent-reported usability, feasibility, and acceptability of the app was high. PCPs reported preference for introducing mHealth interventions like the one in this study early in the course of care for individuals presenting with mild or moderate symptoms. Conclusions: In this small study, we demonstrated the feasibility, acceptability, usability, and safety of using a CBT-based chatbot for adolescents presenting with moderate depressive symptoms in a network of PBRN-based primary care clinics. This pilot study could not establish effectiveness, but our results suggest that further study in a larger pediatric population is warranted. Future study inclusive of rural, socioeconomically disadvantaged, and underrepresented communities is needed to establish generalizability of effectiveness and identify implementation-related adaptations needed to promote broader uptake in pediatric primary care. Trial Registration: ClinicalTrials.gov NCT04603053; https://clinicaltrials.gov/ct2/show/NCT04603053

Overview

CoChat Summary

## TL;DR

This 12-week pilot randomized trial found that a CBT-based chatbot delivered in primary care was feasible, acceptable, usable, and safe for adolescents with moderate depressive symptoms during the COVID-19 pandemic. The study could not establish effectiveness, but depressive symptoms improved more in the app group than in the wait-list control, and the authors recommend larger, more diverse studies.

## Key findings

  • The pilot randomized 18 participants to the app (n=10, 56%) or wait-list control (n=8, 44%); 17 were included in analysis after 1 was found ineligible on chart review.
  • At 4 weeks, mean PHQ-9 scores decreased by 3.3 points in the active treatment group, shifting from moderate to mild symptom severity, versus a 2-point decrease in the wait-list control group with no shift in severity category.
  • In the subgroup with depression (n=10), PHQ-9 scores decreased by 3 points in the intervention group (11.8 to 8.8) versus 1 point in the wait-list control group (15 to 14); remission at 4 weeks in those with baseline PHQ-9 >9 was 67% (2/3) vs 0% (0/5), and at 12 weeks 50% (1/2) vs 20% (1/5).
  • Teen-reported and parent-reported usability, feasibility, and acceptability were high; among adolescents, 80% agreed or completely agreed “I like using the app,” 70% agreed that using the app seemed possible, and mean usability was 21.4/25.

## Methods

  • Open-label, randomized pilot study conducted over 12 weeks in 11 primary care practices within the Washington University Pediatric and Adolescent Ambulatory Research Consortium; interviews were also conducted with 13 PCPs from 11 clinics.
  • Adolescents aged 13 to 17 years with a new diagnosis of depression and anxiety in the past 3 months were randomized 1:1 to the chatbot app or wait-list control; all received usual PCP care.
  • Primary outcome was PHQ-9 depression severity at 4 weeks; secondary outcomes included GAD-7 anxiety and MHSES self-efficacy, plus app usability/acceptability measures and qualitative PCP interviews analyzed with inductive coding.

## Limitations

  • The paper states that recruitment was challenged by the COVID-19 pandemic and reduced access to/utilization of primary care resources, especially early in the pandemic.
  • The authors note that conclusions about effectiveness are limited by the small sample size and the racial, geographic, and socioeconomic homogeneity of participants, who were mostly female, White, and privately insured.
  • The text also makes evident that the study was underpowered for between-group comparisons and that concurrent antidepressant use in some participants may have further limited detection of differences between groups.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.2196/40242

Cites

55 of 55 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Figure 1 Flowchart indicating the study procedures and schedule of events. AIM: Acceptability of Intervention Measure; FIM: Feasibility of Intervention Measure; GAD-7: 7-item Generalized Anxiety Disorder questionnaire; MHSES: Mental Health Self-Efficacy Scale; PHQ-9: 9-item Patient Health Questionnaire; SUS: System Usability Scale.
Figure 1 Flowchart indicating the study procedures and schedule of events. AIM: Acceptability of Intervention Measure; FIM: Feasibility of Intervention Measure; GAD-7: 7-item Generalized Anxiety Disorder questionnaire; MHSES: Mental Health Self-Efficacy Scale; PHQ-9: 9-item Patient Health Questionnaire; SUS: System Usability Scale.CC BY · PubMed Central
Figure 2 Participant Disposition.
Figure 2 Participant Disposition.CC BY · PubMed Central
Figure 3 Change in mean PHQ-9 and GAD-7 scores by treatment group over 12 weeks. GAD-7: 7-item Generalized Anxiety Disorder questionnaire; PHQ-9: 9-item Patient Health Questionnaire.
Figure 3 Change in mean PHQ-9 and GAD-7 scores by treatment group over 12 weeks. GAD-7: 7-item Generalized Anxiety Disorder questionnaire; PHQ-9: 9-item Patient Health Questionnaire.CC BY · PubMed Central

Verification

Claims the paper makes

5 Valid · 1 Not valid · 24 Not checked · 62 Own results and interpretations

27 more claims

Create a free account to see all 92 claims and to verify the 37 references still open.

See all claims — free account
  • It may state two separate claims as one. 33
  • Probably not a substantial claim. 38

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

4 more claims about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

1 of 1

Citations

Cites

Show all 55 references

Research works printed but not linked: 5

  1. Perou, R, 2013, MMWR Suppl, 62, 1
  2. Heron, M, 2019, Natl Vital Stat Rep, 68, 1
  3. Curtin, SC, 2020, Natl Vital Stat Rep, 69, 1
  4. O'ReillyKProposed guidelines aim for safe, effective mobile health appsAmerican Medical Association20182022-09-10https://www.ama-assn.org/practice-management/digital/proposed-guidelines-aim-safe-effective-mobile-health-apps
  5. Mossman, SA, 2017, Ann Clin Psychiatry, 29, 227

Cited by

See all 123

Metadata

Authors
Nicol, G., Wang, R., Graham, S., Dodd, S., & Garbutt, J.
Affiliations
Washington University in St. Louis (US)
Corresponding author
Nicol, Ginger
Venue
JMIR Formative Research
Published
2022-11-22
Volume
6
Issue
11
Pages
e40242
ISSN
2561-326X (ISSN-L)
OpenAlex
W4309659478
Cited by
123
References
55
Citations by year
2026: 36 · 2025: 31 · 2024: 38 · 2023: 18
FWCI
19.21
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • JMIR Formative Research (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Digital Commons@Becker (Washington University School of Medicine) (submittedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
Funders
  • Institute of Clinical and Translational Sciences (UL1 TR002345)
  • National Institutes of Health (UL1TR000445, TR000445, (NIH/NCATS UL1 TR000445)., UL1TR002345)
  • National Center for Advancing Translational Sciences (TR-000445, NIH/NCATS UL1 TR000445)
Topics
  • Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Anxiety, Depression, Psychometrics, Treatment, Cognitive Processes
  • Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
Keywords
adolescents, adolescent depression, adolescent anxiety, COVID-19 pandemic, mobile health apps, chatbots, primary care mental health, feasibility, acceptability, usability, digital mental health interventions, parent-reported outcomes, suicidal ideation, self-harm

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (qwen3.8-omni-flash and gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.