CoChat
Sign Up
Cited by 214Digital Mental Health InterventionsAnxiety, Depression, Psychometrics, Treatment, Cognitive ProcessesSocial Robot Interaction and HRI

Conversational Agent Interventions for Mental Health Problems: Systematic Review and Meta-analysis of Randomized Controlled Trials

Open access
He, Yuhao, Yang, Li, Qian, Chunlian, Li, Tong, Su, Zhengyuan, Zhang, Qiang, Hou, Xiangqing·2023·Journal of Medical Internet Research
DOI 10.2196/43862PMC PMC10182468

Published version of a preprint

Open this paper in CoChat — free accountOpen PDF

Abstract

Background: Mental health problems are a crucial global public health concern. Owing to their cost-effectiveness and accessibility, conversational agent interventions (CAIs) are promising in the field of mental health care. Objective: This study aims to present a thorough summary of the traits of CAIs available for a range of mental health problems, find evidence of efficacy, and analyze the statistically significant moderators of efficacy via a meta-analysis of randomized controlled trial. Methods: Web-based databases (Embase, MEDLINE, PsycINFO, CINAHL, Web of Science, and Cochrane) were systematically searched dated from the establishment of the database to October 30, 2021, and updated to May 1, 2022. Randomized controlled trials comparing CAIs with any other type of control condition in improving depressive symptoms, generalized anxiety symptoms, specific anxiety symptoms, quality of life or well-being, general distress, stress, mental disorder symptoms, psychosomatic disease symptoms, and positive and negative affect were considered eligible. This study followed the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. Data were extracted by 2 independent reviewers, checked by a third reviewer, and pooled using both random effect models and fixed effects models. Hedges g was chosen as the effect size. Results: Of the 6900 identified records, a total of 32 studies were included, involving 6089 participants. CAIs showed statistically significant short-term effects compared with control conditions in improving depressive symptoms (g=0.29, 95% CI 0.20-0.38), generalized anxiety symptoms (g=0.29, 95% CI 0.21-0.36), specific anxiety symptoms (g=0.47, 95% CI 0.07-0.86), quality of life or well-being (g=0.27, 95% CI 0.16-0.39), general distress (g=0.33, 95% CI 0.20-0.45), stress (g=0.24, 95% CI 0.08-0.41), mental disorder symptoms (g=0.36, 95% CI 0.17-0.54), psychosomatic disease symptoms (g=0.62, 95% CI 0.14-1.11), and negative affect (g=0.28, 95% CI 0.05-0.51). However, the long-term effects of CAIs for the most mental health outcomes were not statistically significant (g=−0.04 to 0.39). Personalization and empathic response were 2 critical facilitators of efficacy. The longer duration of interaction with conversational agents was associated with the larger pooled effect sizes. Conclusions: The findings show that CAIs are research-proven interventions that ought to be implemented more widely in mental health care. CAIs are effective and easily acceptable for those with mental health problems. The clinical application of this novel digital technology will conserve human health resources and optimize the allocation of mental health services. Trial Registration: PROSPERO CRD42022350130; https://tinyurl.com/mvhk6w9p

AI snapshot

Written from the paper’s full text.

TL;DR

This systematic review and meta-analysis of 32 RCTs (6089 participants) found that conversational agent interventions (CAIs) produced statistically significant short-term improvements across multiple mental health outcomes, with small-to-moderate effect sizes (g=0.24–0.62). Long-term effects were mostly not statistically significant, while personalization, empathic response, and longer daily interaction were associated with greater efficacy.

Key findings

  • Short-term CAIs were significantly better than controls for depressive symptoms (g=0.29, 95% CI 0.20–0.38), generalized anxiety symptoms (g=0.29, 95% CI 0.21–0.36), specific anxiety symptoms (g=0.47, 95% CI 0.07–0.86), quality of life/well-being (g=0.27, 95% CI 0.16–0.39), general distress (g=0.33, 95% CI 0.20–0.45), stress (g=0.24, 95% CI 0.08–0.41), mental disorder symptoms (g=0.36, 95% CI 0.17–0.54), psychosomatic disease symptoms (g=0.62, 95% CI 0.14–1.11), and negative affect (g=0.28, 95% CI 0.05–0.51).
  • Long-term effects were mostly not statistically significant (g=−0.04 to 0.39); significant long-term effects were reported for depressive symptoms (g=0.16, 95% CI 0.06–0.26), general distress (g=0.39, 95% CI 0.19–0.59), mental disorder symptoms (g=0.31, 95% CI 0.07–0.55), and psychosomatic disease symptoms (g=0.27, 95% CI 0.02–0.53), while stress long-term was g=0.09 (95% CI −0.12 to 0.29).
  • Personalization and empathic response were critical facilitators of efficacy; longer duration of interaction with conversational agents was associated with larger pooled effect sizes (depressive symptoms dose b=0.160, 95% CI 0.014–0.306, P=.04; generalized anxiety dose b=0.067, 95% CI 0.006–0.127, P=.04). Automatic reminders to engage were associated with smaller effects for depressive symptoms (P=.04).
  • Different CA types varied by outcome: chatbot showed the largest effect for generalized anxiety symptoms and quality of life, embodied CA for depressive symptoms and stress, CA in virtual reality for specific anxiety symptoms, and avatar for general distress.

Methods

  • Systematic review and meta-analysis of RCTs following PRISMA, registered in PROSPERO (CRD42022350130); searched Embase, MEDLINE, PsycINFO, CINAHL, Web of Science, and Cochrane from database establishment to October 30, 2021, updated to May 1, 2022.
  • Included 32 RCTs with 26 CAs and 6089 participants; mean age 36.32 (SD 13.44) years; 75.41% (2922/3875) women from reported data; CA types were chatbot (16 studies), embodied CA (6), CA in virtual reality (5), and avatar (5); controls were active, information/attentional, or passive.
  • Used Hedges g as effect size, random- and fixed-effects models, I² and Cochran Q for heterogeneity, subgroup analyses and univariate random-effects meta-regression, sensitivity analyses, and publication bias tests (funnel plot, Begg and Mazumdar, Egger, trim-and-fill).

Limitations

  • Stated: the broad search prioritized sensitivity over specificity, which may have resulted in significant heterogeneity; some included studies treated a specific mental health issue as a secondary or auxiliary outcome; broad participant and comparator inclusion may also have increased heterogeneity.
  • Stated: the most frequent measurement instruments per outcome were used and standardized mean differences applied, but different instruments for the same outcome are not exactly equivalent; a single measurement tool or additional subgroup analysis would be preferable if sufficient research exists. Meta-regression could only include participant characteristics at the study/trial level, which may not reflect individual participants and may lead to aggregate bias.
  • Evident from the text: heterogeneity was high for specific anxiety symptoms (I²=93.17%) and some other outcomes; risk of bias was high in 13 studies, some in 11, and low in 8; 17 studies had no follow-up, limiting long-term evidence, and the short-term specific anxiety effect did not remain significant in sensitivity analyses using the smallest effect size or low-risk-of-bias trials.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.2196/43862

Cites

90 of 90 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 76 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    CAIs showed a statistically significant short-term effect compared with control conditions in improving general distress (g=0.33, 95% CI 0.20-0.45).

    “general distress (g=0.33, 95% CI 0.20-0.45)”
    Quote in the paperSubstantial claim
  • Findingown data

    CAIs showed a statistically significant short-term effect compared with control conditions in improving negative affect (g=0.28, 95% CI 0.05-0.51).

    “negative affect (g=0.28, 95% CI 0.05-0.51)”
    Quote in the paperSubstantial claim
  • Findingown data

    CAIs showed a statistically significant short-term effect compared with control conditions in improving mental disorder symptoms (g=0.36, 95% CI 0.17-0.54).

    “mental disorder symptoms (g=0.36, 95% CI 0.17-0.54)”
    Quote in the paperSubstantial claim
  • Findingown data

    CAIs showed a statistically significant short-term effect compared with control conditions in improving depressive symptoms (g=0.29, 95% CI 0.20-0.38).

    “CAIs showed statistically significant short-term effects compared with control conditions in improving depressive symptoms (g=0.29, 95% CI 0.20-0.38)”
    • Also stated in: § Efficacy · plain
      “The pooled effect size for the 27 postintervention comparisons between CAIs and control conditions on depressive symptoms was g=0.29 (95% CI 0.20-0.38), with moderate heterogeneity (I2=42.90%, 95% CI 9.79%-63.86%; Figures 2 and 3; Table 1).”
    Quote in the paperSubstantial claim
  • Findingown data

    CAIs showed a statistically significant short-term effect compared with control conditions in improving generalized anxiety symptoms (g=0.29, 95% CI 0.21-0.36).

    “generalized anxiety symptoms (g=0.29, 95% CI 0.21-0.36)”
    Quote in the paperSubstantial claim
  • Classified as probably not a substantial claim. 31
  • May combine two claims. 37
  • The quote only partly states this claim. 7

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    18 studies supported personalization and tailoring, and 14 studies did not support personalization and tailoring.

    “A total of 18 studies supported personalization and tailoring, and 14 studies did not support personalization and tailoring.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    17 studies were based on cognitive behavioral therapy (CBT), and 15 studies were based on other theories.

    “In addition, 17 studies were based on cognitive behavioral therapy (CBT), and 15 studies were based on other theories such as patient-centered therapy, acceptance and commitment therapy, method of levels therapy, and problem-solving treatment.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    Eleven studies had an intervention length of 0 to 4 weeks, 10 had 5 to 8 weeks, and 11 had >9 weeks.

    “Eleven studies had an intervention length of 0 to 4 weeks, 10 studies had an intervention length of 5 to 8 weeks, and 11 studies had an intervention length of >9 weeks.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    75.41% (2922/3875) of the participants were women from the reported data.

    “and 75.41% (2922/3875) of the participants were women from the reported data.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    Seventeen studies had no follow-up, 7 had a follow-up length of 0 to 8 weeks, and 8 had a follow-up length of ≥9 weeks.

    “Seventeen studies had no follow-up, 7 studies had a follow-up length of 0 to 8 weeks, and 8 studies had a follow-up length of ≥9 weeks.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Nine studies directly aimed at depressive symptoms, 11 at generalized anxiety symptoms, 6 at specific anxiety symptoms, 2 at quality of life, 5 at general distress, 2 at stress, and 4 at other outcomes.

    “Nine studies directly aimed at depressive symptoms, 11 studies directly aimed at generalized anxiety symptoms, 6 studies directly aimed at specific anxiety symptoms, 2 studies directly aimed at quality of life, 5 studies directly aimed at general distress, 2 studies directly aimed at stress, and 4 studies directly aimed at other outcomes.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    Among the included studies, 9 were conducted in the United States; 6 in the United Kingdom; 3 in Sweden; 2 each in Japan, Korea, Switzerland, and Canada; and 1 each in Argentina, China, Ireland, the Netherlands, Germany, and New Zealand.

    “Among the included studies, 9 were conducted in the United States; 6 were conducted in the United Kingdom; 3 were conducted in Sweden; 2 each conducted in Japan, Korea, Switzerland, and Canada; and 1 each conducted in Argentina, China, Ireland, the Netherlands, Germany, and New Zealand.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Attrition rates between baseline and postintervention measures were reported in 31 studies and varied widely from no attrition to 82.98% (2214/2668) of participants.

    “Attrition rates between baseline and postintervention measures were reported in 31 studies and varied widely from no attrition to 82.98% (2214/2668) of participants.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The average age of participants was 36.32 (SD 13.44; range 21.40-71.47) years.

    “The average age was 36.32 (SD 13.44; range 21.40-71.47) years”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    Fourteen studies used passive controls, 8 used information or attentional controls, and 10 used active controls.

    “Fourteen studies used passive controls, 8 studies used information or attentional controls, and 10 studies used active controls.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Eight studies were assessed as having a low risk of bias, 11 studies had some risk of bias, and 13 studies had a high risk of bias.

    “Eight studies were assessed as having a low risk of bias, 11 studies had some risk of bias, and 13 studies had a high risk of bias (Figures S1 and S2 in Multimedia Appendix 1).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    A total of 16 studies with CAs were classified as chatbot, 6 as ECA, 5 as CA in VR, and 5 as avatar.

    “A total of 16 studies with CAs were classified as chatbot [28-43], 6 studies with CAs classified as ECA [44-49], 5 studies with CAs classified as CA in VR [50-54], and 5 studies with CAs classified as avatar [27,55-58].”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The 32 RCTs with 26 CAs included 6089 participants, with sample sizes ranging from 19 to 2668.

    “The 32 RCTs with 26 CAs were included, involving 6089 participants, with sample sizes ranging from 19 to 2668 (Table S2 in Multimedia Appendix 1).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The interaction reported in 16 studies ranged from 0.57 to 6.43 (mean 4.48, SD 2.11) minutes per day.

    “The ration of the interaction reported in 16 studies ranged from 0.57 to 6.43 (mean 4.48, SD 2.11) minutes per day.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    A total of 8 studies recruited clinical samples, 14 recruited symptomatic samples, and 10 recruited general samples.

    “A total of 8 studies recruited clinical samples, 14 recruited symptomatic samples, and 10 recruited general samples.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    15 studies provided automatic reminders to engage, and 17 studies did not provide automatic reminders to engage.

    “In total, 15 studies provided automatic reminders to engage, and 17 studies did not provide automatic reminders to engage.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    18 studies supported emotional and empathic responses, and 14 studies did not support emotional and empathic responses.

    “Moreover, 18 studies supported emotional and empathic responses, and 14 studies did not support emotional and empathic responses.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    Fifteen studies reported user experience, of which 8 performed thematic analysis on the feedback of participants.

    “Fifteen studies reported user experience, of which 8 performed thematic analysis on the feedback of participants.”
    Quote in the paperNot a substantial claim

  • The quote only partly states this claim.

    Findingown data

    Meta-regression analyses revealed a statistically significant effect of gender on the pooled effect size for generalized anxiety symptoms (b=0.448, 95% CI 0.024-0.873; P=.04).

    “Meta-regression analyses revealed statistically significant effects of gender (b=0.448, 95% CI 0.024-0.873; P=.04) and dosage (b=0.067, 95% CI 0.006-0.127; P=.04) on the pooled effect size (Table 2; Figure S6 in Multimedia Appendix 1).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Table 2

      Not checked in the free run.

    • cites Figure S6 in Multimedia Appendix 1

      Not checked in the free run.

  • The quote only partly states this claim.

    Findingown data

    Meta-regression analyses revealed a statistically significant effect of dosage on the pooled effect size for generalized anxiety symptoms (b=0.067, 95% CI 0.006-0.127; P=.04).

    “Meta-regression analyses revealed statistically significant effects of gender (b=0.448, 95% CI 0.024-0.873; P=.04) and dosage (b=0.067, 95% CI 0.006-0.127; P=.04) on the pooled effect size (Table 2; Figure S6 in Multimedia Appendix 1).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Table 2

      Not checked in the free run.

    • cites Figure S6 in Multimedia Appendix 1

      Not checked in the free run.

  • The quote only partly states this claim.

    Findingown data

    Meta-regression analyses of the effect of conversational agent interventions on general distress revealed no statistically significant results.

    “Meta-regression analyses revealed no statistically significant results (Table 2).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    For general distress, the pooled effect size across the 12 postintervention comparisons was g=0.33 (95% CI 0.20-0.45), with low heterogeneity (I2=6.93%, 95% CI 0.00%-43.77%).

    “The pooled effect size for the 12 postintervention comparisons was g=0.33 (95% CI 0.20-0.45), with low heterogeneity (I2=6.93%, 95% CI 0.00%-43.77%; Figure 2; Table 4; Figure S14 in Multimedia Appendix 1).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The pooled long-term effect of conversational agent interventions on quality of life or well-being was g=−0.04 (95% CI −0.19 to 0.11), with low heterogeneity (I2=24.17%, 95% CI 0.00%-64.28%).

    “The pooled effect size of long-term efficacy was g=−0.04 (95% CI −0.19 to 0.11), with low heterogeneity (I2=24.17%, 95% CI 0.00%-64.28%; Figure 2; Table 3; Figure S12 in Multimedia Appendix 1).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Significant effect sizes were observed in the symptomatic samples for depressive symptoms, generalized anxiety symptoms, and quality of life.

    “Significant effect sizes were observed in the nonsymptomatic (depressive symptoms, generalized anxiety symptoms, quality of life, and general distress), symptomatic (depressive symptoms, generalized anxiety symptoms, and quality of life), and clinical samples (depressive symptoms, specific anxiety symptoms, and general distress).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The pooled long-term effect of conversational agent interventions on stress was g=0.09, with low heterogeneity (I2=0.00%, 95% CI 0.00%-84.69%).

    “The pooled effect size of long-term efficacy was g=0.09, with low heterogeneity (I2=0.00%, 95% CI 0.00%-84.69%; Figure 2; Table 4; Figure S18 in Multimedia Appendix 1).”
    Quote in the paperNot a substantial claim

  • The quote only partly states this claim.

    Findingown data

    Meta-regression analyses of the effect of conversational agent interventions on stress revealed no statistically significant results.

    “Meta-regression analyses revealed no statistically significant results (Table 2).”
    Quote in the paperNot a substantial claim

  • The quote only partly states this claim.

    Findingown data

    Meta-regression analyses revealed a statistically significant effect of gender on the pooled effect size for specific anxiety symptoms (b=−3.414, 95% CI −6.118 to −0.710; P=.02).

    “Meta-regression analyses revealed statistically significant effects of age (b=0.084, 95% CI 0.014-0.154; P=.02) and gender (b=−3.414, 95% CI −6.118 to −0.710; P=.02) on the pooled effect size (Table 2; Figure S6 in Multimedia Appendix 1).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Table 2

      Not checked in the free run.

    • cites Figure S6 in Multimedia Appendix 1

      Not checked in the free run.

  • May combine two claims.

    Findingown data

    Significant effect sizes were observed in the clinical samples for depressive symptoms, specific anxiety symptoms, and general distress.

    “Significant effect sizes were observed in the nonsymptomatic (depressive symptoms, generalized anxiety symptoms, quality of life, and general distress), symptomatic (depressive symptoms, generalized anxiety symptoms, and quality of life), and clinical samples (depressive symptoms, specific anxiety symptoms, and general distress).”
    Quote in the paperSubstantial claim

  • The quote only partly states this claim.

    Findingown data

    Meta-regression analyses revealed a statistically significant effect of age on the pooled effect size for specific anxiety symptoms (b=0.084, 95% CI 0.014-0.154; P=.02).

    “Meta-regression analyses revealed statistically significant effects of age (b=0.084, 95% CI 0.014-0.154; P=.02) and gender (b=−3.414, 95% CI −6.118 to −0.710; P=.02) on the pooled effect size (Table 2; Figure S6 in Multimedia Appendix 1).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Table 2

      Not checked in the free run.

    • cites Figure S6 in Multimedia Appendix 1

      Not checked in the free run.

  • May combine two claims.

    Findingown data

    Significant effect sizes were observed in the nonsymptomatic samples for depressive symptoms, generalized anxiety symptoms, quality of life, and general distress.

    “Significant effect sizes were observed in the nonsymptomatic (depressive symptoms, generalized anxiety symptoms, quality of life, and general distress), symptomatic (depressive symptoms, generalized anxiety symptoms, and quality of life), and clinical samples (depressive symptoms, specific anxiety symptoms, and general distress).”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing), S4 (based). The quote only partly states this claim.

    Findingown data

    The pooled short-term effect of conversational agent interventions on stress remained significant across all sensitivity analyses.

    “It remained significant across all sensitivity analyses (Table 4).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingnone given

    Follow-up length is a moderator of long-term effects, whereas the other analyzed variables are moderators of short-term effects.

    “Follow-up length is a moderator of long-term effects, whereas the other variables are moderators of short-term effects.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The pooled long-term effect of conversational agent interventions on general distress was g=0.39, with low heterogeneity (I2=0.00%, 95% CI 0.00%-70.81%).

    “The pooled effect size of long-term efficacy was g=0.39, with low heterogeneity (I2=0.00%, 95% CI 0.00%-70.81%; Figure 2; Table 4; Figure S15 in Multimedia Appendix 1).”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    This study followed the PRISMA guidelines.

    “This study followed the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    CAIs were the new wave of digital mental health interventions to cope with the insufficient and inadequate mental health services.

    “Conversational agent interventions (CAIs) [12] were the new wave of digital mental health interventions to cope with the insufficient and inadequate mental health services [1,13].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [1] World mental health report: transforming mental health for allWorld Health Organization202206162022-07-20https://www.who.int/publications/i/item/9789240049338

      Not checked in the free run.

    • cites [13] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Digital mental health interventions have emerged as an important research area with evidence-based psychotherapies implemented on digital platforms.

    “Because of their increased acceptance and accessibility, digital mental health interventions have emerged as an important research area with evidence-based psychotherapies implemented on digital platforms [11].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [11] source

      Not checked in the free run.

  • May combine two claims.

    Interpretationown data

    Personalization and empathic response were 2 critical facilitators of efficacy.

    “Personalization and empathic response were 2 critical facilitators of efficacy.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    Mental health services are few and still underutilized globally.

    “However, mental health services are few and still underutilized globally [4-6].”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [4] source

      Not checked in the free run.

    • cites [5] source

      Not checked in the free run.

    • cites [6] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Stigma and discrimination are considered the most important barriers to providing mental health services.

    “and the resulting stigma and discrimination are also considered the most important barriers to providing mental health services [9,10].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [9] source

      Not checked in the free run.

    • cites [10] source

      Not checked in the free run.

  • May combine two claims.

    Interpretationown data

    CAIs are effective and easily acceptable for those with mental health problems.

    “CAIs are effective and easily acceptable for those with mental health problems.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    Traditional face-to-face mental health care had many limitations, such as expensive treatment, lack of experienced therapists, poor service quality, geographical constraints, and delayed treatment.

    “This was because traditional face-to-face mental health care still had many limitations, such as expensive treatment, lack of experienced therapists, poor service quality, geographical constraints, and delayed treatment [7,8]”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [7] source

      Not checked in the free run.

    • cites [8] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    Data were extracted by 2 independent reviewers and checked by a third reviewer.

    “Data were extracted by 2 independent reviewers, checked by a third reviewer, and pooled using both random effect models and fixed effects models.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductcitation

    The quality of the trials was independently evaluated by 2 authors using the Cochrane Collaboration tool.

    “The quality of the trials was independently evaluated by 2 authors using the Cochrane Collaboration tool [67] to assess the risk of bias.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [67] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The study protocol was registered in PROSPERO (CRD42022350130).

    “The study protocol was registered in the PROSPERO database (CRD42022350130).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    An increasing number of CAIs have emerged in recent years, solving and improving a larger range of mental health issues in addition to depression and anxiety.

    “An increasing number of CAIs have emerged in recent years, as the digital medical and mobile health fields have grown [26], solving and improving a larger range of mental health issues in addition to depression and anxiety.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [26] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    An analysis only including low-risk articles was performed to determine the pooled effect size after controlling for risk of bias.

    “We also performed an analysis only including low-risk articles to determine the pooled effect size after controlling for the risk of bias.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    Publication bias was examined using funnel plot, Begg and Mazumdar rank correlation test, Egger regression test, and Duval and Tweedie trim-and-fill procedure.

    “We applied different methods to examine publication bias (funnel plot, Begg and Mazumdar rank correlation test [74], Egger regression test [75], and Duval and Tweedie trim-and-fill procedure [76]).”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [74] Begg, CB, 1994, Biometrics source

      Not checked in the free run.

    • cites [75] source

      Not checked in the free run.

    • cites [76] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Premisecitation

    Substantial human health resources will be saved if the CAIs are proven effective and suitable for widespread implementation.

    “Substantial human health resources will be saved if the CAIs are proven effective and suitable for widespread implementation [14].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [14] source

      Not checked in the free run.

  • May combine two claims.

    Interpretationown data

    Chatbot showed the largest effect size for generalized anxiety symptoms and quality of life.

    “Chatbot showed the largest effect size for generalized anxiety symptoms and quality of life, ECA showed the largest effect size for depressive symptoms and stress, CA in VR showed the largest effect size for specific anxiety symptoms, and avatar showed the largest effect size for general distress.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    Conversational agent interventions were significantly better than control conditions in improving depressive symptoms, generalized anxiety symptoms, specific anxiety symptoms, quality of life or well-being, general distress, stress, mental disorder symptoms, psychosomatic disease symptoms, and negative affect, with effect sizes ranging from g=0.24 to 0.62, most of which remained robust after the 1

    “With effect sizes ranging from g=0.24 to 0.62, most of which remained robust even after performing various sensitivity analyses, they were significantly better than control conditions in improving depressive symptoms, generalized anxiety symptoms, specific anxiety symptoms, quality of life or well-being, general distress, stress, mental disorder symptoms, psychosomatic disease symptoms, and neg”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    The long-term effects of conversational agent interventions on depressive symptoms, general distress, stress, mental disorder symptoms, and psychosomatic disease symptoms were statistically significant (g=0.16-0.39).

    “The long-term effects of CAIs on depressive symptoms, general distress, stress, mental disorder symptoms, and psychosomatic disease symptoms were statistically significant (g=0.16-0.39).”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The measurement instruments that most frequently corresponded to each mental health outcome were chosen to extract the data.

    “we chose the measurement instruments that most frequently corresponded to each of the mental health outcome to extract the data.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Study design and conductcitation

    In meta-regression, participant characteristics can only be included as covariates at the study or trial level for analysis, which may not accurately reflect the level of individual participants, leading to aggregate bias.

    “in meta-regression, participant characteristics can only be included as covariates at the study or trial level for analysis, which may not accurately reflect the level of individual participants, leading to aggregate bias [83].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [83] source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    The stronger effect when no automatic reminders were included may be because digital interventions such as internet-based CBT largely consisted of linear, structured psychotherapy modules, whereas more freedom for participants was offered in the context of CAIs, where interference and repetition would instead have an adverse effect on interest.

    “This may be because digital interventions such as internet-based CBT largely consisted of linear, structured psychotherapy modules [81,82], whereas more freedom for participants was offered in the context of CAIs [12], where interference and repetition would instead have an adverse effect on interest.”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [81] source

      Not checked in the free run.

    • cites [82] source

      Not checked in the free run.

    • cites [12] source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    A thorough search was conducted, prioritizing sensitivity over specificity because of the lack of standardized language in this field.

    “First, a thorough search was conducted, prioritizing sensitivity over specificity because of the lack of standardized language in this field.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationreasoning

    The design of CAs needs to be succinct and empathic because, otherwise, the user's interest will quickly wane, which is not conducive to the establishment of a stable working alliance between CAs and participants.

    “We learned that the design of CAs needs to be succinct and empathic because, otherwise, the user’s interest will quickly wane, which is not conducive to the establishment of a stable working alliance between CAs and participants.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Study design and conductnone given

    Different measurement instruments for the same mental health outcome are not exactly equivalent.

    “it should be noted that different measurement instruments for the same mental health outcome are not exactly equivalent.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    CAs with advanced empathetic skills have improved user affinities and experiences.

    “CAs with advanced empathetic skills have improved user affinities and experiences [77,78].”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [77] source

      Not checked in the free run.

    • cites [78] source

      Not checked in the free run.

  • May combine two claims.

    Study design and conductreasoning

    The broadest range of participants and comparators was used, which may have resulted in significant heterogeneity.

    “we used the broadest range of participants and comparators, which may have resulted in significant heterogeneity.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    Previous research concluded that studies offering engagement reminders were consistently associated with larger effect sizes.

    “In contrast to previous research [72,80], which concluded that studies offering engagement reminders were consistently associated with larger effect sizes, our study showed a stronger effect on depressive symptoms when the intervention did not include automatic reminders to engage.”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [72] source

      Not checked in the free run.

    • cites [80] source

      Not checked in the free run.

60 of 60

  • The paper relies on this without support.

    CAIs are promising in the field of mental health care.

    “Owing to their cost-effectiveness and accessibility, conversational agent interventions (CAIs) are promising in the field of mental health care.”
1 of 1

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Generalized-anxiety gender meta-regression values differ between text and table

    Minor

    The text reports a gender coefficient of 0.448 with a 95% CI of 0.024 to 0.873; Table 2 gives 0.438 with a 95% CI of 0.029 to 0.847 for the same analysis.

  • Specific-anxiety age meta-regression P values differ

    Minor

    The text reports P=.02 for the age coefficient, whereas Table 2 reports P=.01 for the same coefficient and confidence interval.

2 of 2

Cites

Flagged first, then by citations

Research works printed but not linked: 1

  1. 10.1037/0003-066x.59.7.614 · DOI · Paper page

Cited by

Sorted by citations

Related

Authors
He, Y., Yang, L., Qian, C., Li, T., Su, Z., Zhang, Q., & Hou, X.
Affiliations
  • Tianjin University (CN)
  • Sun Yat-sen University (CN)
Corresponding author
He, Yuhao
ORCID
Venue
Journal of Medical Internet Research
Publisher
JMIR Publications Inc.
Published
2023-04-28
Volume
25
Pages
e43862
ISSN
1438-8871 (ISSN-L)
DOI
10.2196/43862
PMC
PMC10182468
OpenAlex
W4323900694
Semantic Scholar
e0b7889e1b8ed56c12d405039d0ad7dd998a8df5
Cited by
213
References
90
Citations by year
2026: 87 · 2025: 75 · 2024: 42 · 2023: 9
FWCI
7.23
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • Journal of Medical Internet Research (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • Tianjin University
  • National Social Science Fund of China (21BSH017)
Topics
  • Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Anxiety, Depression, Psychometrics, Treatment, Cognitive Processes
  • Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
Keywords
mental health, depression, anxiety, generalized anxiety disorder, quality of life, well-being, distress, stress, psychosomatic symptoms, positive and negative affect, digital mental health interventions, treatment efficacy, cost-effectiveness, access to mental health care, implementation barriers, implementation facilitators
MeSH terms
Anxiety, Humans, Mental Health, Quality of Life, Randomized Controlled Trials as Topic

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-07.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-07.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review