CoChat
Sign Up
Cited by 259Digital Mental Health InterventionsSubstance Abuse Treatment and OutcomesMobile Health and mHealth Applications

A Therapeutic Relational Agent for Reducing Problematic Substance Use (Woebot): Development and Usability Study

Open access
Prochaska, Judith J, Vogel, Erin A, Chieng, Amy, Kendra, Matthew, Baiocchi, Michael, Pajarito, Sarah, Robinson, Athena·2021·Journal of Medical Internet Research
DOI 10.2196/24850PMC PMC8074987

Published version of a preprint

Open this paper in CoChat — free account

Abstract

Background: Misuse of substances is common, can be serious and costly to society, and often goes untreated due to barriers to accessing care. Woebot is a mental health digital solution informed by cognitive behavioral therapy and built upon an artificial intelligence–driven platform to deliver tailored content to users. In a previous 2-week randomized controlled trial, Woebot alleviated depressive symptoms. Objective: This study aims to adapt Woebot for the treatment of substance use disorders (W-SUDs) and examine its feasibility, acceptability, and preliminary efficacy. Methods: American adults (aged 18-65 years) who screened positive for substance misuse without major health contraindications were recruited from online sources and flyers and enrolled between March 27 and May 6, 2020. In a single-group pre/postdesign, all participants received W-SUDs for 8 weeks. W-SUDs provided mood, craving, and pain tracking and modules (psychoeducational lessons and psychotherapeutic tools) using elements of dialectical behavior therapy and motivational interviewing. Paired samples t tests and McNemar nonparametric tests were used to examine within-subject changes from pre- to posttreatment on measures of substance use, confidence, cravings, mood, and pain. Results: The sample (N=101) had a mean age of 36.8 years (SD 10.0), and 75.2% (76/101) of the participants were female, 78.2% (79/101) were non-Hispanic White, and 72.3% (73/101) were employed. Participants’ W-SUDs use averaged 15.7 (SD 14.2) days, 12.1 (SD 8.3) modules, and 600.7 (SD 556.5) sent messages. About 94% (562/598) of all completed psychoeducational lessons were rated positively. From treatment start to end, in-app craving ratings were reduced by half (87/101, 86.1% reporting cravings in the app; odds ratio 0.48, 95% CI 0.32-0.73). Posttreatment assessment completion was 50.5% (51/101), with better retention among those who initially screened higher on substance misuse. From pre- to posttreatment, confidence to resist urges to use substances significantly increased (mean score change +16.9, SD 21.4; P<.001), whereas past month substance use occasions (mean change −9.3, SD 14.1; P<.001) and scores on the Alcohol Use Disorders Identification Test-Concise (mean change −1.3, SD 2.6; P<.001), 10-item Drug Abuse Screening Test (mean change −1.2, SD 2.0; P<.001), Patient Health Questionnaire-8 item (mean change 2.1, SD 5.2; P=.005), Generalized Anxiety Disorder-7 (mean change −2.3, SD 4.7; P=.001), and cravings scale (68.6% vs 47.1% moderate to extreme; P=.01) significantly decreased. Most participants would recommend W-SUDs to a friend (39/51, 76%) and reported receiving the service they desired (41/51, 80%). Fewer felt W-SUDs met most or all of their needs (22/51, 43%). Conclusions: W-SUDs was feasible to deliver, engaging, and acceptable and was associated with significant improvements in substance use, confidence, cravings, depression, and anxiety. Study attrition was high. Future research will evaluate W-SUDs in a randomized controlled trial with a more diverse sample and with the use of greater study retention strategies. Trial Registration: ClinicalTrials.gov NCT04096001; http://clinicaltrials.gov/ct2/show/NCT04096001.

AI snapshot

Written from the paper’s full text.

TL;DR

In a single-group 8-week study, an adapted Woebot conversational agent for substance use (W-SUDs) was feasible and acceptable and was associated with significant pre-post reductions in substance use, cravings, depression, and anxiety. However, posttreatment assessment completion was only 50.5%, limiting causal inference.

Key findings

  • Among 101 adults who screened positive for substance misuse, W-SUDs use averaged 15.7 days (SD 14.2), 12.1 modules (SD 8.3), and 600.7 sent messages (SD 556.5); 94.0% (562/598) of completed psychoeducational lessons were rated positively.
  • In-app craving ratings fell by about half from week 1 to weeks 8 and 9 (odds ratio 0.48, 95% CI 0.32-0.73; 86.1% reported cravings in the app).
  • Among the 51 participants who completed posttreatment assessments, confidence to resist urges increased (mean change +16.9, SD 21.4; P<.001), while past-month substance use occasions decreased (mean change −9.3, SD 14.1; P<.001), AUDIT-C decreased (mean change −1.3, SD 2.6; P<.001), DAST-10 decreased (mean change −1.2, SD 2.0; P<.001), PHQ-8 depression decreased (mean change 2.1, SD 5.2; P=.005), and GAD-7 anxiety decreased (mean change −2.3, SD 4.7; P=.001).
  • Most posttreatment completers would recommend W-SUDs to a friend (39/51, 76%) and reported receiving the service they desired (41/51, 80%), but fewer felt W-SUDs met most or all of their needs (22/51, 43%).

Methods

  • Single-group pre/post design; participants were recruited online and via flyers, enrolled between March 27 and May 6, 2020, and received W-SUDs for 8 weeks. Of 3597 screened, 175 were eligible, 141 consented, 128 completed baseline, and 101 registered with W-SUDs and initiated use.
  • W-SUDs was an automated text-based conversational agent adapted from Woebot, drawing on cognitive behavioral therapy, dialectical behavior therapy, motivational interviewing, and relapse prevention; it included mood, craving, and pain tracking plus over 50 psychoeducational lessons and psychotherapeutic tools.
  • Analyses included paired samples t tests and McNemar tests for pre-post changes, generalized estimating equation linear models for weekly in-app craving ratings, and bivariate correlations and t tests for secondary associations.

Limitations

  • The paper states the single-group design and short-term posttreatment outcomes limit the strength of inferences; the sample was predominantly female, non-Hispanic White, and employed full-time; nearly all participants (99/101, 98.0%) were sheltering in place during COVID-19; all outcomes were self-reported; acceptability measures were not open-ended or qualitative; few participants were misusing opioids due to exclusions; and attrition was high (posttreatment completion 50.5%, 51/101).
  • The text makes evident that without a control group, improvements cannot be attributed to W-SUDs, and the small posttreatment sample (n=51) limits precision and generalizability.
  • The paper also notes that non-Hispanic White participants reported higher acceptability on 2 of 4 measures than participants from other racial or ethnic groups, indicating a need for more diverse sampling and cultural tailoring.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.2196/24850

Cites

55 of 55 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 53 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    Participants used an average of 4.3 tools (SD 1.4; median 4; IQR 1).

    “Participants used an average of 4.3 tools (SD 1.4; median 4; IQR 1).”
    Quote in the paperNot a substantial claim
  • Findingown data

    Compared with week 1, craving ratings were significantly lower at weeks 4 through 9.

    “Compared with week 1, craving ratings were significantly lower at weeks 4 through 9.”
    Quote in the paperSubstantial claim
  • Findingown data

    Scores on the Patient Health Questionnaire-8 item significantly decreased (mean change 2.1, SD 5.2; P=.005).

    “Patient Health Questionnaire-8 item (mean change 2.1, SD 5.2; P=.005)”
    Quote in the paperSubstantial claim
  • Findingown data

    GAD-7 anxiety scores significantly reduced from pre- to posttreatment (P<.05).

    “significant reductions were observed from pre- to posttreatment in past month substance use occasions, AUDIT-C and DAST-10 scores (overall and among those in the clinical range at pretreatment), and PHQ-8 depression and GAD-7 anxiety scores (all P values<.05).”
    Quote in the paperSubstantial claim
  • Findingown data

    Lesson acceptability ratings were high across the board, with 94.0% (562/598) of completed lessons receiving thumbs up.

    “Lesson acceptability ratings were high across the board, with 94.0% (562/598) of completed lessons receiving thumbs up.”
    Quote in the paperNot a substantial claim
  • May combine two claims. 46
  • Classified as probably not a substantial claim. 27
  • The quote only partly states this claim. 1

  • May combine two claims.

    Findingown data

    Woebot data indicate that users do not use Woebot for crisis management; approximately 6.3% trigger the safety net protocol, with 27% of those confirming that it is indeed a crisis when Woebot asks to confirm (ie, the true positive rate).

    “Woebot data indicate that users do not use Woebot for crisis management; approximately 6.3% trigger the safety net protocol, with 27% of those confirming that it is indeed a crisis when Woebot asks to confirm (ie, the true positive rate).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    DAST-10 scores significantly reduced from pre- to posttreatment overall and among those in the clinical range at pretreatment (P<.05).

    “significant reductions were observed from pre- to posttreatment in past month substance use occasions, AUDIT-C and DAST-10 scores (overall and among those in the clinical range at pretreatment), and PHQ-8 depression and GAD-7 anxiety scores (all P values<.05).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Lesson completion rates were highest (>50%) for content concerning COVID-19, urge surfing, and SUD labels and lowest (<5%) for content concerning sleep and grief.

    “Lesson completion rates were highest (>50%) for content concerning COVID-19, urge surfing, and SUD labels and lowest (<5%) for content concerning sleep and grief.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    A total of 1571 mood ratings were entered into the W-SUDs app by 90 of the 101 (89.1%) participants, with each participant entering on average 17.5 mood ratings (SD 16.1; median 10; IQR 25.3) or 2.2 per week.

    “A total of 1571 mood ratings were entered into the W-SUDs app by 90 of the 101 (89.1%) participants, with each participant entering on average 17.5 mood ratings (SD 16.1; median 10; IQR 25.3) or 2.2 per week.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    Reports of pain intensity and pain interference with work did not change significantly from pre- to posttreatment.

    “Reports of pain intensity and pain interference with work did not change significantly from pre- to posttreatment.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    In paired sample t tests, confidence scores overall and in all 8 domains significantly increased from pre- to posttreatment (all P values<.05).

    “In paired sample t tests, confidence scores overall and in all 8 domains significantly increased from pre- to posttreatment (all P values<.05).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Pain ratings did not differ significantly by week and over the 9 weeks averaged 2.3 (SD 2.1), on a scale of 0 to 10.

    “In contrast, pain ratings did not differ significantly by week and over the 9 weeks averaged 2.3 (SD 2.1), on a scale of 0 to 10.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    AUDIT-C scores significantly reduced from pre- to posttreatment overall and among those in the clinical range at pretreatment (P<.05).

    “significant reductions were observed from pre- to posttreatment in past month substance use occasions, AUDIT-C and DAST-10 scores (overall and among those in the clinical range at pretreatment), and PHQ-8 depression and GAD-7 anxiety scores (all P values<.05).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Among the full sample (N=101), for the 8-week treatment period, participants' use of W-SUDs averaged 15.7 days (SD 14.2; median 10; IQR 20) or 2.0 times per week, with an average of 600.7 user sent messages (SD 556.5; median 360; IQR 763) or 75.1 messages per week and engagement on average with 12.1 modules (SD 8.3; median 9; IQR 12.5).

    “Among the full sample (N=101), for the 8-week treatment period, participants’ use of W-SUDs averaged 15.7 days (SD 14.2; median 10; IQR 20) or 2.0 times per week, with an average of 600.7 user sent messages (SD 556.5; median 360; IQR 763) or 75.1 messages per week and engagement on average with 12.1 modules (SD 8.3; median 9; IQR 12.5)”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    A total of 1399 craving and 1403 pain ratings were entered into the W-SUDs app by 87 of the 101 participants (86.1%), with each participant providing an average of 16.1 ratings (SD 14.8; median 9; IQR 21) for cravings and 16.1 ratings (SD 14.9; median 9; IQR 21) for pain.

    “A total of 1399 craving and 1403 pain ratings were entered into the W-SUDs app by 87 of the 101 participants (86.1%), with each participant providing an average of 16.1 ratings (SD 14.8; median 9; IQR 21) for cravings and 16.1 ratings (SD 14.9; median 9; IQR 21) for pain.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given

    Declines in depressive symptoms observed from pre- to posttreatment were greater among the participants in therapy.

    “Declines in depressive symptoms observed from pre- to posttreatment were greater among the participants in therapy.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The serious adverse event was deemed by the DSMB to be unrelated to study involvement.

    “was deemed by the DSMB to be unrelated to study involvement.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Scores for the 3 WAI-SR subscales differed significantly from each other in pairwise t test comparisons (all P values<.05), with the highest ratings on development of an affective bond to Woebot, followed by agreement on the tasks of treatment and then agreement on the goals of treatment.

    “Scores for the 3 WAI-SR subscales, with identical response options, differed significantly from each other in pairwise t test comparisons (all P values<.05), with the highest ratings on development of an affective bond to Woebot, followed by agreement on the tasks of treatment and then agreement on the goals of treatment.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    A greater decline in the DAST-10 score was not associated with number of use occasions (r=0.10), confidence (r=−0.12), or GAD-7 anxiety (r=0.21).

    “A greater decline in the DAST-10 score was associated with greater reductions in PHQ-8 depression (r=0.40; P<.01) but not with the number of use occasions (r=0.10), confidence (r=−0.12), or GAD-7 anxiety (r=0.21).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Study enrollment was double the initial recruitment goal, reflecting interest in W-SUDs.

    “Study strengths include study enrollment being double the initial recruitment goal, reflecting interest in W-SUDs.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Only about half of the sample completed the posttreatment assessment, with better retention among those screening higher on the CAGE-AID.

    “Only about half of the sample completed the posttreatment assessment, with better retention among those screening higher on the CAGE-AID.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The sample was predominately female and identified as non-Hispanic White, and the majority were employed full-time.

    “The sample was predominately female and identified as non-Hispanic White, and the majority were employed full-time.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The sample's baseline ratings of pain intensity and pain interference were low.

    “Measures of physical pain did not change with the use of W-SUDs as reported in pre- and posttreatment measures or within the app; however, the sample’s baseline ratings of pain intensity and pain interference were low.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Most participants used all facets of the W-SUDs app: tracked their mood, cravings, and pain; completed on average over 7 psychoeducational lessons; and used tools in the W-SUDs app.

    “Most participants used all facets of the W-SUDs app: tracked their mood, cravings, and pain; completed on average over 7 psychoeducational lessons; and used tools in the W-SUDs app.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Most participants reported lifetime psychiatric diagnoses, and approximately half of the participants endorsed current moderate-to-severe levels of depression or anxiety.

    “Most participants reported lifetime psychiatric diagnoses, and approximately half of the participants endorsed current moderate-to-severe levels of depression or anxiety.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Within-app lesson completion and content acceptability were high for the overall sample, although there was a wide range of use patterns.

    “Within-app lesson completion and content acceptability were high for the overall sample, although there was a wide range of use patterns.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingreasoning

    Few participants were misusing opioids, likely due to study exclusion designed to mitigate risk, namely, the requirement of engagement with medication-assisted treatment and no history of opioid overdose requiring Narcan (naloxone).

    “Few participants were misusing opioids, likely due to study exclusion designed to mitigate risk, namely, the requirement of engagement with medication-assisted treatment and no history of opioid overdose requiring Narcan (naloxone).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Misuse of substances can be serious and costly to society.

    “can be serious and costly to society”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [5] source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Treatment of depression and anxiety, the most common psychiatric comorbidities among patients with SUDs, may reduce craving and substance use and enhance overall outcomes.

    “Treatment of depression and anxiety, the most common psychiatric comorbidities among patients with SUDs, may reduce craving and substance use and enhance overall outcomes”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [4] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Misuse of substances is common.

    “Misuse of substances is common”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [3] Substance Abuse and Mental Health Services AdministrationKey substance use and mental health indicators in the United States: results from the 2018 National Survey on Drug Use and HealthCenter for Behavioral Health Statistics and Quality, Substance Abuse and Mental Health Services Administration20192021-02-17https://www.samhsa.gov/data/sites/default/files/cbhsq-reports/NSDUHNationalFindingsReport2018/NSDUHNationalFindingsReport2018.pdf

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    In 2018, less than 1 in 5 individuals with a SUD received addiction treatment.

    “In 2018, less than 1 in 5 individuals with a SUD received addiction treatment”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [3] Substance Abuse and Mental Health Services AdministrationKey substance use and mental health indicators in the United States: results from the 2018 National Survey on Drug Use and HealthCenter for Behavioral Health Statistics and Quality, Substance Abuse and Mental Health Services Administration20192021-02-17https://www.samhsa.gov/data/sites/default/files/cbhsq-reports/NSDUHNationalFindingsReport2018/NSDUHNationalFindingsReport2018.pdf

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Among a sample of adults (N=70) randomly assigned to Woebot or an information only control group, Woebot users had statistically and clinically significant reductions in depressive symptoms (F1,48=6.03; P=.02) after 2 weeks of use, whereas those in the control group did not.

    “Among a sample of adults (N=70) randomly assigned to Woebot or an information only control group, Woebot users had statistically and clinically significant reductions in depressive symptoms (F1,48=6.03; P=.02) after 2 weeks of use, whereas those in the control group did not.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [18] source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Although the efficacy of automated conversational agent digital therapeutics for SUDs is still untested, such products are commercially available, and few consumers are aware that the products lack evidence.

    “Although the efficacy of automated conversational agent digital therapeutics for SUDs is still untested, such products are commercially available, and few consumers are aware that the products lack evidence”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [29] source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Brief intervention can minimize potential dropout, a problem common to SUD treatment; therefore, W-SUDs was designed as an 8-week treatment.

    “Brief intervention can minimize potential dropout, a problem common to SUD treatment; [38] therefore, we designed W-SUDs as an 8-week treatment.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [38] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Premisecitation

    The scale potential of conversational agents is unconstrained, immediate, and available to users in an instant.

    “The scale potential of conversational agents is unconstrained, immediate, and available to users in an instant”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [12] source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Early responsiveness to SUD treatment is predictive of long-term outcomes, and brief addiction treatments are efficacious.

    “Early responsiveness to SUD treatment is predictive of long-term outcomes [36], and brief addiction treatments are efficacious [37].”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [36] source

      Not checked in the free run.

    • cites [37] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductcitation

    The DAST-10 has moderate test-retest reliability, sensitivity, and specificity.

    “The DAST-10 has moderate test-retest reliability, sensitivity, and specificity [40].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [40] source

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    Evidence for conversational agent interventions for addressing mental health problems is growing quickly and appears promising with regard to acceptability and efficacy.

    “Evidence for conversational agent interventions for addressing mental health problems is growing quickly and appears promising with regard to acceptability and efficacy”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [27] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    The AUDIT-C has been found to be a valid screening test for heavy drinking and/or active alcohol abuse or dependence.

    “The AUDIT-C has been found to be a valid screening test for heavy drinking and/or active alcohol abuse or dependence [39].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [39] source

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    CBT evidence-based, guided self-help treatments have ranged in length from 2 to 12 weeks, and the National Institutes for Clinical Excellence describes guided self-help as including 6 to 8 face-to-face sessions.

    “CBT evidence-based, guided self-help treatments have ranged in length from 2 to 12 weeks [31-34], and the National Institutes for Clinical Excellence describes guided self-help as including 6 to 8 face-to-face sessions [35].”
    Quote in the paperSubstantial claim0 of 5 sources checked
    • cites [31] source

      Not checked in the free run.

    • cites [32] source

      Not checked in the free run.

    • cites [33] CullyJATetenALA therapist's guide to brief cognitive behavioral therapyDepartment of Veterans Affairs, South Central Mental Illness Research, Education, and Clinical Center (MIRECC)20082021-02-17https://depts.washington.edu/dbpeds/therapists_guide_to_brief_cbtmanual.pdf

      Not checked in the free run.

    • cites [34] source

      Not checked in the free run.

    • cites [35] Appendix E: Glossary. National Institute for Health and Care Excellence; 2011National Institute for Health and Care Excellence20112021-02-17https://www.nice.org.uk/guidance/cg123

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims. The quote only partly states this claim.

    Interpretationreasoning

    Woebot's app-based platform and user-centered design philosophy make it a promising modality for SUD treatment delivery; it offers immediate, evidence-based tailored support in the peak moment of craving.

    “Woebot’s app-based platform and user-centered design philosophy make it a promising modality for SUD treatment delivery; it offers immediate, evidence-based tailored support in the peak moment of craving.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: Automated conversational agents can deliver a coach-like or sponsor-like experience without human implementation assistance for in-the-moment treatment delivery.
    • Rests on an assumption: Conversational agents can provide real-time support to address substance use urges, unlike traditional in-person frameworks of weekly visits.
    • Rests on an assumption: The efficacy of conversational agents for treating SUDs remains unknown.

  • May combine two claims.

    Premisecitation

    An informal poll of Woebot users (in July 2018) indicated that 63% had interest in content addressing SUDs; 22% reported binge use; 5% endorsed using nonprescription drugs.

    “An informal poll of Woebot users (in July 2018) indicated that 63% had interest in content addressing SUDs; 22% of surveyed users reported having 5 or more alcoholic drinks in a row within a couple of hours (ie, binge use) [28], and 5% endorsed using nonprescription drugs.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [28] NIAAA council approves definition of binge drinkingNIAAA Newsletter20042021-02-17https://pubs.niaaa.nih.gov/publications/Newsletter/winter2004/Newsletter_Number3.pdf

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The study procedures were approved by the Institutional Review Board of Stanford Medicine.

    “The study procedures were approved by the Institutional Review Board of Stanford Medicine.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Study design and conductcitation

    The CAGE-AID has demonstrated validity, with high internal consistency in screening for problematic drug and alcohol use; a cutoff point of 2+ on the CAGE-AID has a sensitivity of 70% and specificity of 85% for identifying individuals with SUDs.

    “The CAGE-AID has demonstrated validity, with high internal consistency in screening for problematic drug and alcohol use; a cutoff point of 2+ on the CAGE-AID has a sensitivity of 70% and specificity of 85% for identifying individuals with SUDs”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [30] Brown, RL, 1995, Wis Med J

      Not checked in the free run.

  • May combine two claims.

    Study design and conductown data

    Retention was lowest among those with a CAGE-AID score of 2 (7/26, 27%) and higher for those scoring 3 (22/38, 58%) or 4 (22/37, 59%).

    “Retention was lowest among those with a CAGE-AID score of 2 (7/26, 27%) and higher for those scoring 3 (22/38, 58%) or 4 (22/37, 59%).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Study design and conductown data

    Posttreatment survey completion was 50.5% (51/101), with better retention among those with a higher CAGE-AID score at screening (γ=0.37; P=.02).

    “Posttreatment survey completion was 50.5% (51/101), with better retention among those with a higher CAGE-AID score at screening (γ=0.37; P=.02).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Missing data on individual survey items was minimal.

    “Missing data on individual survey items was minimal.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    In a single instance, a participant's average score values were imputed when missing 1 item on the PHQ-8.

    “In a single instance, a participant’s average score values were imputed when missing 1 item on the PHQ-8.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Study design and conductown data

    Retention was unrelated to participant demographic characteristics, previous use of Woebot, psychiatric diagnoses, primary problematic substance, depressive symptoms, pain, cravings, confidence, substance use occasions, AUDIT-C scores, or DAST-10 scores (all P values>.102).

    “Retention was unrelated to participant demographic characteristics, previous use of Woebot, psychiatric diagnoses, primary problematic substance, depressive symptoms, pain, cravings, confidence, substance use occasions, AUDIT-C scores, or DAST-10 scores (all P values>.102).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    W-SUDs was associated with significant improvements in substance use, confidence, cravings, depression, and anxiety.

    “was associated with significant improvements in substance use, confidence, cravings, depression, and anxiety”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    W-SUDs was feasible to deliver, engaging, and acceptable.

    “W-SUDs was feasible to deliver, engaging, and acceptable”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductcitation

    Both the PHQ-8 and GAD-7 have good internal consistency and demonstrated convergent validity with measures of depression, stress, and anxiety.

    “Both the PHQ-8 and GAD-7 have good internal consistency and demonstrated convergent validity with measures of depression, stress, and anxiety.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [42] source

      Not checked in the free run.

    • cites [43] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductcitation

    The CSQ-8 items have 4-point rating scales with response descriptors that vary. Internal consistency exceeds 0.90, and the total sum score ranges from 8 to 32, with higher total scores indicating higher satisfaction.

    “The CSQ-8 items have 4-point rating scales with response descriptors that vary. Internal consistency exceeds 0.90, and the total sum score ranges from 8 to 32, with higher total scores indicating higher satisfaction.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [45] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Traditionally, working alliance has been characterized as the cooperation and collaboration in the therapeutic relationship between the patient and the therapist.

    “Traditionally, working alliance has been characterized as the cooperation and collaboration in the therapeutic relationship between the patient and the therapist [53-55].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [53] source

      Not checked in the free run.

    • cites [54] source

      Not checked in the free run.

    • cites [55] source

      Not checked in the free run.

  • May combine two claims.

    Interpretationown data

    W-SUDs, an automated conversational agent, was feasible to deliver, engaging, and acceptable and was associated with significant improvements pre- to posttreatment in self-reported measures of substance use, confidence, craving, depression, and anxiety and in-app measures of craving.

    “W-SUDs, an automated conversational agent, was feasible to deliver, engaging, and acceptable and was associated with significant improvements pre- to posttreatment in self-reported measures of substance use, confidence, craving, depression, and anxiety and in-app measures of craving.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Working alliance is proposed to mediate clinical outcomes in traditional therapeutic settings.

    “This study also examined working alliance, proposed to mediate clinical outcomes in traditional therapeutic settings [52].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [52] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    A systematic review concluded that conversational agents showed preliminary efficacy in reducing psychological distress among adults with mental health concerns compared with inactive control conditions.

    “Furthermore, although a systematic review concluded that conversational agents showed preliminary efficacy in reducing psychological distress among adults with mental health concerns compared with inactive control conditions [27], this is the first published study of a conversational agent adapted for substance use.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [27] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Alcohol sales in the United States increased during the COVID-19 pandemic.

    “Notably, however, alcohol sales in the United States increased during the COVID-19 pandemic [59].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [59] Rebalancing the ‘COVID-19 effect’ on alcohol sales2021-02-17https://nielseniq.com/global/en/insights/2020/rebalancing-the-covid-19-effect-on-alcohol-sales/

      Not checked in the free run.

  • May combine two claims.

    Study design and conductreasoning

    This study had a single-group design, and the outcomes were short term and limited to posttreatment, thus limiting the strength of inferences that can be drawn.

    “This study had a single-group design, and the outcomes were short term and limited to posttreatment, thus limiting the strength of inferences that can be drawn.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationnone given

    The delivery modality of W-SUDs offered easy, immediate, and stigma-free access to emotional support and substance use recovery information, particularly relevant during a time of global physical distancing and sheltering in place.

    “The delivery modality of W-SUDs offered easy, immediate, and stigma-free access to emotional support and substance use recovery information, particularly relevant during a time of global physical distancing and sheltering in place.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Electronic screening and brief intervention programs have been found to effectively reduce alcohol consumption and alcohol-related problems.

    “Electronic screening and brief intervention programs, which use mobile tools to screen for excessive alcohol use and deliver personalized feedback, have been found to effectively reduce alcohol consumption and alcohol-related problems [58].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [58] Preventing excessive alcohol consumption: electronic Screening and Brief Intervention (e-SBI)Community Preventive Services Task Force20122021-02-17https://safesupportivelearning.ed.gov/resources/preventing-excessive-alcohol-consumption%C2%A0electronic-screening-and-brief-intervention-e-sbi

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    By addressing problematic substance use, including but not limited to alcohol, the W-SUDs intervention supports and extends a growing body of literature on the use of automated conversational agents (or chatbots) and other mobile apps to support behavioral health.

    “By addressing problematic substance use, including but not limited to alcohol, the W-SUDs intervention supports and extends a growing body of literature on the use of automated conversational agents (or chatbots) and other mobile apps to support behavioral health.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Mobile apps were less common than web-based interventions, with weaker evidence of efficacy and some indication of causing harm.

    “Mobile apps were less common than web-based interventions, with weaker evidence of efficacy and some indication of causing harm (ie, inadvertently helping users increase, rather than decrease, their blood alcohol level while partying).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [6] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Study design and conductnone given

    The primary outcomes of substance use, cravings, confidence, mood, and program acceptability were standard measures with demonstrated validity and reliability.

    “The primary outcomes of substance use, cravings, confidence, mood, and program acceptability were standard measures with demonstrated validity and reliability.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    The therapeutic approach is acceptable, feasible, and safe.

    “The therapeutic approach is acceptable, feasible, and safe.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Study design and conductnone given

    The limitations were that all outcomes were self-reported, and acceptability measures were not open-ended or qualitative.

    “The limitations were that all were self-reported, and acceptability measures were not open-ended or qualitative.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    This study is the first empirical evaluation of an SUD-focused digital therapeutic delivered via a fully automated conversational agent.

    “This study is the first empirical evaluation of an SUD-focused digital therapeutic delivered via a fully automated conversational agent.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    The scalability and accessibility of an automated program coupled with the growing problem of substance use suggest the potential for an engaging and effective therapeutic to reduce the burden of SUDs.

    “The scalability and accessibility of an automated program coupled with the growing problem of substance use suggest the potential for an engaging and effective therapeutic to reduce the burden of SUDs.”
    Quote in the paperNot a substantial claim
63 of 63

  • The paper relies on this without support.

    Automated conversational agents can deliver a coach-like or sponsor-like experience without human implementation assistance for in-the-moment treatment delivery.

    “Automated conversational agents can deliver a coach-like or sponsor-like experience and yet do not require human implementation assistance for in-the-moment treatment delivery.”

  • The paper relies on this without support.

    Conversational agents can provide real-time support to address substance use urges, unlike traditional in-person frameworks of weekly visits.

    “Conversational agents can provide real-time support to address substance use urges, unlike traditional in-person frameworks of weekly visits.”

  • The paper relies on this without support.

    The efficacy of conversational agents for treating SUDs remains unknown.

    “However, the efficacy of conversational agents for treating SUDs remains unknown.”

  • The paper relies on this without support.

    Most digital SUD interventions have been delivered on a web platform, rather than via mobile apps.

    “To date, most digital SUD interventions have been delivered on a web platform, rather than via mobile apps.”
4 of 4

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Many studies (5/15, 33%) reported total number of messages exchanged with the chatbot; only 7% (1/15) reported exact proportion of user-initiated conversations (approximately 30%).

    “Many studies reported the total number of messages exchanged with the chatbot (5/15, 33%) [6,26,29,31,32]; however, only 7% (1/15) of studies reported the exact proportion of user-initiated conversations (approximately 30%) [6], which depicted the participants’ level of interest in having health-related conversations with the chatbot.”

    From Artificial Intelligence-Based Chatbots for Promoting Health Behavioral Changes: Systematic Review. (2023) · cites this paper as [31]

    The Results section (W-SUDs Use and Within-App User Feedback; page unavailable) reports an average of 600.7 user-sent messages, supporting this paper's inclusion among studies that reported message counts. Our copy does not report the proportion of user-initiated conversations or an approximately 30% figure; it also does not establish the citing review's 5/15 or 1/15 study counts. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • Prochaska et al reported a significant increase in confidence to resist urges to use substances (mean score change +16.9, SD 21.4; P<.001) and significant decreases in substance use occasions, AUDIT-C, DAST-10, PHQ-8, GAD-7, and cravings scale.

    “Prochaska et al [31] reported a significant increase in the confidence to resist urges to use substances (mean score change +16.9, SD 21.4; P<.001) and a significant decrease in the following: substance use occasions (mean change −9.3, SD 14.1; P<.001) and the scores of Alcohol Use Disorders Identification Test-Concise (mean change −1.3, SD 2.6; P<.001), 10-item Drug Abuse Screening Test (mean”

    From Artificial Intelligence-Based Chatbots for Promoting Health Behavioral Changes: Systematic Review. (2023) · cites this paper as [31]

    Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

    Experimental — accuracy not yet measured

  • Of the 15 studies, 11 (73%) reported feasibility of AI chatbots in terms of safety, messages exchanged, retention rate, and duration of engagement.

    “Out of the 15 studies, 11 (73%) reported the feasibility of AI chatbots in terms of (1) safety [22] (ie, no adverse events were reported), (2) messages exchanged with the chatbot [6,26,29,31,32], (3) retention rate [22,26], and (4) duration of engagement.”

    From Artificial Intelligence-Based Chatbots for Promoting Health Behavioral Changes: Systematic Review. (2023) · cites this paper as [31]

    The Conclusions describe the Woebot intervention as feasible and safe, and Methods > Data Analyses reports 50.5% posttreatment survey retention. However, Results > Serious Adverse Events says one serious adverse event was reported, although the safety board deemed it unrelated to study involvement. Thus the cited study supports the feasibility and retention points, but not the unqualified statement that no adverse events were reported; it cannot itself establish the citing review’s 11-of-15 count. (LLM, conf=0.89)

    Experimental — accuracy not yet measured

  • Of the 15 studies, only 1 (7%) aimed at reducing problematic substance use.

    “Out of the 15 studies, only 1 (7%) study [31] aimed at reducing problematic substance use.”

    From Artificial Intelligence-Based Chatbots for Promoting Health Behavioral Changes: Systematic Review. (2023) · cites this paper as [31]

  • In Woebot, CBT was combined with motivational interviewing and dialectical behavior therapy to provide emotional support and personalized psychoeducation to resist substance misuse.

    “Similarly, in Woebot [31], CBT was clubbed with motivational interviewing and dialectical behavior therapy to provide emotional support and personalized psychoeducation to resist substance misuse.”

    From Artificial Intelligence-Based Chatbots for Promoting Health Behavioral Changes: Systematic Review. (2023) · cites this paper as [31]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Cravings reduction has conflicting McNemar P values

    Major

    The Results describe the pre-to-post cravings reduction as having P<.001, whereas Table 3 gives .013 for the McNemar test of that change. Both indicate significance, but they are not the same reported result.

  • DAST-10 possible range differs between Methods and Table 1

    Minor

    The Methods say that omitting one item reduced the sample's possible DAST-10 range to 0-9, but Table 1 labels it 0-10.

2 of 2

Cites

Flagged first, then by citations

Research works printed but not linked: 2

  1. 2018, Global status report on alcohol and health
  2. Brown, RL, 1995, Wis Med J, 94, 135

Cited by

Sorted by citations

Related

Authors
Prochaska, J. J., Vogel, E. A., Chieng, A., Kendra, M., Baiocchi, M., Pajarito, S., & Robinson, A.
Affiliations
  • Stanford Medicine (US)
  • Woebot Health (United States) (US)
Corresponding author
Prochaska, Judith J
ORCID
Venue
Journal of Medical Internet Research
Publisher
JMIR Publications Inc.
Published
2021-03-23
Volume
23
Issue
3
Pages
e24850
ISSN
1438-8871 (ISSN-L)
DOI
10.2196/24850
PMC
PMC8074987
OpenAlex
W3126938010
Semantic Scholar
0a3e4f01c27cdb97219b6a6a9181e64aec036bcc
Cited by
257
References
55
Citations by year
2026: 44 · 2025: 62 · 2024: 61 · 2023: 54 · 2022: 27 · 2021: 9
FWCI
33.33
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • Journal of Medical Internet Research (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • eScholarship (California Digital Library) (submittedVersion)
Funders
National Institute on Drug Abuse (R44D048712, R44DA048712)
Topics
  • Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
  • Health Sciences › Medicine › Epidemiology › Substance Abuse Treatment and Outcomes
  • Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
Keywords
Woebot, chatbot therapy, substance use disorders, problematic substance use, artificial intelligence, cognitive behavioral therapy, dialectical behavior therapy, motivational interviewing, craving reduction, digital mental health interventions, treatment retention, usability, feasibility, psychoeducation, mood tracking, United States adults
MeSH terms
Adult, Alcoholism, Artificial Intelligence, Female, Humans, Mental Health, Cognitive Behavioral Therapy, Substance-Related Disorders

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-10.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-10.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review