Cited by 149Opioid Use Disorder TreatmentSubstance Abuse Treatment and OutcomesDigital Mental Health Interventions

Adding an Internet-delivered treatment to an efficacious treatment package for opioid dependence.

Open access
Christensen, Darren R., Landes, Reid D., Jackson, Lisa, Marsch, Lisa A., Mancino, Michael J., Chopra, Mohit P., Bickel, Warren K.·2014·Journal of Consulting and Clinical Psychology

Abstract

OBJECTIVE: To examine the benefit of adding an Internet-delivered behavior therapy to a buprenorphine medication program and voucher-based motivational incentives. METHOD: A block-randomized, unblinded, parallel, 12-week treatment trial was conducted with 170 opioid-dependent adult patients (mean age = 34.3 years; 54.1% male; 95.3% White). Participants received an Internet-based community reinforcement approach intervention plus contingency management (CRA+) and buprenorphine or contingency management alone (CM-alone) plus buprenorphine. The primary outcomes, measured over the course of treatment, were longest continuous abstinence, total abstinence, and days retained in treatment. RESULTS: Compared to those receiving CM-alone, CRA+ recipients exhibited, on average, 9.7 total days more of abstinence (95% confidence interval [CI = 2.3, 17.2]) and had a reduced hazard of dropping out of treatment (hazard ratio = 0.47; 95% CI [0.26, 0.85]). Prior treatment for opioid dependence significantly moderated the additional improvement of CRA+ for longest continuous days of abstinence. CONCLUSIONS: These results provide further evidence that an Internet-based CRA+ treatment is efficacious and adds clinical benefits to a contingency management/medication based program for opioid dependence.

Integrity signals

Registry

Crossref · 10.1037/a0037496

Cites

35 of 35 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

0 of 40 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    In opioid-dependent adults, adding internet-delivered CRA to contingency management plus buprenorphine increased total days of abstinence by an average of 9.7 days compared with contingency management alone (95% CI 2.3 to 17.2).

    “Compared to those receiving CM-alone, CRA+ recipients exhibited on average 9.7 total days more of abstinence, 95% CI: (2.3, 17.2)”
    Quote in the paperSubstantial claim
  • Findingown data

    In the same trial, participants receiving CRA+ had a lower hazard of dropping out of treatment than those receiving CM-alone (HR 0.47, 95% CI 0.26 to 0.85).

    “and had a reduced hazard of dropping out of treatment, Hazard Ratio (HR)=0.47; 95% CI: (0.26, 0.85).”
    Quote in the paperSubstantial claim
  • Findingown data

    A history of prior treatment for opioid dependence significantly moderated how much CRA+ improved longest continuous days of abstinence.

    “Prior treatment for opioid dependence significantly moderated the additional improvement of CRA+ for longest continuous days of abstinence.”
    Quote in the paperNot a substantial claim
  • Findingown data

    The odds ratio for completing the 12-week treatment was 2.30 favoring CRA+ (χ2[1]=5.57, p=.018).

    “The odds ratio for completing the 12-week treatment was 2.30 favoring CRA+ (χ2[1]=5.57, p=.018; see Table 2).”
    Quote in the paperSubstantial claim
  • Findingown data

    Among treatment-naïve participants, the hazard of dropping out for CM-alone was 1.15 times that for CRA+ (χ2[1]=0.13, p=.718).

    “For treatment-naïve participants, the hazard for CM-alone participants was 1.15 times (χ2[1] = 0.13, p=.718) that for CRA+ participants.”
    Quote in the paperSubstantial claim
  • May combine two claims. 21
  • Classified as probably not a substantial claim. 20
  • May combine two claims.

    Findingown data

    The observed differences in LCA were in the expected direction but were approximately one-third of the expected difference.

    “Our differences in LCA were in the right direction, but were approximately 1/3 of the difference we expected.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    At both mid- and end-trial, participants tended to improve from baseline on ASI subscales for opioids, drugs, alcohol, psychological issues, employment, and family issues (mid-trial all six p<.03; trial-end all six p<.01).

    “At both mid- and end-trial, participants tended to show improvement from baseline levels of the ASI subscales for opioids, drugs, alcohol, psychological issues, employment, and family issues (mid-trial: all six p<.03; trial-end: all six p<.01).”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    The legal issues and medications ASI subscales did not show a change from baseline at either time point (mid-trial both p>.05; end-trial both p>.16).

    “The subscales for legal issues and medications did not show a change from baseline values at either time point (mid-trial: both p>.05; end-trial both p>.16).”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Income had no statistical effect in any analysis of treatment outcomes.

    “We thus included income in analyses of treatment outcomes, but in all instances, income had no statistical effect; hence, we omit these results.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The CRA+ group had a lower median monthly income at baseline than the CM-alone group ($1,000 vs. $1,808; Wilcoxon-Mann-Whitney z=2.09, p=.037).

    “The CRA+ group had a lower median monthly income than those in the CM-alone group: $1,000 vs. $1,808 (Wilcoxon-Mann-Whitney z=2.09, p=.037).”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    For the cocaine ASI subscale, participants showed decreases from baseline at mid-trial (t[165]=2.03, p=.04), but did not maintain a statistical difference at trial-end (t[189]=0.33, p=.74).

    “For the cocaine subscale, participants showed decreases from baseline levels at mid-trial (t[165]=2.03, p=.04), but did not maintain a statistical difference at trial-end (t[189]=0.33, p=.74).”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    Among CRA+ participants with prior treatment, mean LCA and TA were 61.1 and 72.6 days, respectively, compared with CM-alone means of 46.0 and 54.8 days (LCA: t[74.6]=2.52, p=.014; TA: t[53.8]=3.70, p=.001).

    “For CRA+ participants having previously undergone treatment for opioids, their mean LCA and TA were 61.1 and 72.6 days, respectively, compared to their CM-alone counterparts’ means of 46.0 and 54.8 days, respectively (LCA: t[74.6]=2.52, p=.014; TA: t[53.8]= 3.70, p=.001).”
    • Also stated in: § DISCUSSION · plain
      “Further, for participants who experienced prior treatment, those in the CRA+ condition had longer continuous abstinence and longer total abstinence weeks than the CM-alone group.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Premisecitation

    The CRA plus CM package has been widely studied and shown to be efficacious for cocaine dependence.

    “The CRA plus CM treatment package has been extensively researched and shown to be a demonstrably efficacious treatment for cocaine dependence”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites Higgins, Bickel, & Hughes, 1994

      Not checked in the free run.

    • cites Higgins et al., 2003

      Not checked in the free run.

    • cites Van Horn & Frank, 1998

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Computer-based and internet-delivered methods are being used more often to deliver mental health and substance abuse treatment.

    “Recently, mental health and substance abuse treatments have been increasingly implemented using computer-based and internet-delivered approaches (Bickel, Marsch, Buchhalter, & Badger, 2008; Brooks, Ryder, Carise, & Kirby, 2010).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Bickel, Marsch, Buchhalter, & Badger, 2008

      Not checked in the free run.

    • cites Brooks, Ryder, Carise, & Kirby, 2010

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    In opioid-dependent patients, adding internet-based CRA to reduced standard treatment produced greater opioid abstinence over 12 months, both across all study weeks and across tested weeks, than standard treatment.

    “Marsch et al. (2013) found that participants assigned to the CRA condition reported statistically significantly higher rates of overall study and tested weeks of opioid abstinence over 12 months than the standard treatment group.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Marsch et al. (2013)

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    In that trial, CRA participants had a greater percentage of tested weeks of continuous opioid abstinence, but not of overall study weeks, than standard treatment.

    “CRA participants also had a statistically significant greater percentage of tested weeks of continuous abstinence (but not overall study weeks of continuous abstinence) from opioids compared to the standard treatment group.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Marsch et al. (2013)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Drugs of abuse are partly potent because their reinforcing effects are more overpowering and immediate than those of more pro-social behaviors.

    “Evidence from these literatures suggest that drugs of abuse are partly so potent due to their over-powering and more immediately reinforcing effects than more pro-social behaviors”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Higgins, et al., 1994

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation

    Internet-delivered systems are especially appropriate for behavioral treatments in which participants learn skills that can be trained to mastery through programmable methods.

    “These delivery systems are particularly suited to behavioral treatments where participants learn skills that can be trained to a level of mastery using programmable methods”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Bickel, et al., 2008

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    CRA and CRA plus CM are behavioral treatments that have shown efficacy in substance abuse treatment.

    “Behavioral treatments that show efficacy in the treatment of substance abuse are the Community Reinforcement Approach (CRA: Hunt & Azrin, 1973) and CRA plus Contingency Management (CM: Higgins et al., 1993).”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites Hunt & Azrin, 1973

      Not checked in the free run.

    • cites Higgins et al., 1993

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Marsch et al. found no difference between groups in drop-out but high attrition in both groups.

    “However, although Marsch et al. (2013) reported no differences in drop-out between groups, they did report high attrition rates across both groups.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Marsch et al. (2013)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    In cocaine users, an internet-based CRA program with contingency payments for completing modules improved CRA knowledge and increased selection of CRA-style coping responses compared with a yoked control.

    “Brooks et al. (2010) found participants in the CRA condition demonstrated statistically significant improvements in CRA knowledge and were significantly more likely to select CRA-style coping responses, relative to the yoked group.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Brooks et al. (2010)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    Treatment allocation used minimum likelihood allocation to balance groups on characteristics likely to affect outcomes.

    “Allocation to treatment groups was based on the “minimum likelihood allocation” method (Aickin, 1982), which balances treatment groups on patient characteristics likely to influence treatment outcomes.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Aickin, 1982

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    Using an internet-based counseling program is likely to reduce clinician training time and increase standardization of treatment, resulting in financial savings and treatment improvements for time-poor and resource-challenged counseling services.

    “The practical implications of using an internet-based counseling program are likely to be less clinician training time and greater standardization of treatment resulting in significant financial savings and treatment improvements to time-poor and resource challenged counseling services.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Study design and conductown data

    This trial randomly assigned 170 adults with opioid dependence, ages 20 to 63, to one of two 12-week treatment arms.

    “Of the 206 consented, 170 participants aged 20 to 63 were randomly assigned to one of two conditions for 12 weeks of treatment.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    No abstinence data were missing from clinic visits that participants attended.

    “There were no missing abstinence data from attended clinic visits.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Improved retention is consistent with previous contingency management effects.

    “Although improved retention is consistent with previous contingency management effects (Preston et al., 1999), we believe this is the first demonstration of improved retention in an internet delivered program of CRA+ for abstinence.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Preston et al., 1999

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    If baseline measures differed between groups, their impact on group comparisons was assessed by including them as covariates.

    “If any baseline measure was found to differ between the two groups, the impact of that measure on group comparisons was assessed by entering it into the analysis as a covariate.”
    Quote in the paperNot a substantial claim
  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductown data

    Assignment to treatment was random, unblinded, and in a 1:1 ratio to CRA+ or CM-alone.

    “Allocation to treatment groups was randomly assigned (un-blinded) by 1:1 allotment to either the CRA+ or CM-alone treatment conditions.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    People who previously entered substance abuse treatment have greater drug problems, higher psychiatric co-morbidities, and more life challenges than those without previous treatment experience.

    “research that suggests people who previously entered substance abuse treatment have greater drug problems, higher psychiatric co-morbidities, and suffer more life challenges than those without previous treatment experience (Claus, Mannen, & Schicht, 1999).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Claus, Mannen, & Schicht, 1999

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation

    Computer-delivered approaches have been found effective in treating a variety of substance use disorders.

    “Computer-delivered approaches have now been found to be effective in treating a variety of substance use disorders (Bickel, et al., 2008; Carroll et al., 2009).”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites Bickel, et al., 2008

      Not checked in the free run.

    • cites Carroll et al., 2009

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Opioid-dependent individuals typically report low incomes.

    “as opioid-dependent individuals typically report low incomes (Marsch, et al., 2005)”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Marsch, et al., 2005

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    To prevent compliance from confounding the test of CRA efficacy, the internet CRA program could be accessed only at the clinic.

    “Given that we wished to examine the efficacy of this program alone and not have these results confounded with issues of compliance, treatments were only accessible from the clinic site.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Interpretationown data

    CRA+ had lower drop-out rates and higher treatment completion than CM-alone, suggesting that CRA provides value additional to motivational incentives and buprenorphine.

    “CRA+ was found to have lower drop-out rates and higher rates of treatment completion than CM-alone suggesting that CRA provides value to opioid-dependent participants additional to motivational incentives and buprenorphine medication.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Study design and conductown data

    The sample size was designed to detect a difference in LCA of 3 weeks (9 visits) between the two groups and might have been under-powered to detect smaller differences.

    “Another was the sample size which was designed to detect a difference in LCA of 3 weeks (9 visits) between the two groups, and might have been under-powered to pick up smaller differences.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Interpretationown data

    The study demonstrated increased overall abstinence and retention efficacy for internet-delivered computerized CRA+ over motivational incentives and buprenorphine, and increased continuous abstinence for CRA+ individuals with prior treatment.

    “In summary, our study demonstrated increased overall abstinence and retention efficacy in an internet-delivered computerized CRA+ group over motivational incentives and buprenorphine, and increased continuous abstinence for those CRA+ individuals who engaged in prior treatment.”
    Quote in the paperSubstantial claim
  • Classified as probably not a substantial claim.

    Study design and conductnone given

    Missed visits were counted as positive results.

    “Missed visits were interpreted as positive results.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductcitation

    The Addiction Severity Index has been shown to be internally consistent and valid for measuring addiction severity.

    “Psychometric assessments have shown the ASI as an internally consistent and valid instrument for measuring addiction severity”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Butler et al., 2001

      Not checked in the free run.

    • cites Leonhard, Mulvey, Gastfriend, & Shwartz, 2000

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    Randomization was stratified by buprenorphine stabilization dose, distance from the clinic, prior opioid dependence treatment, and past-month cocaine use.

    “We stratified the randomization on four characteristics:”
    • Also stated in: § Procedure · plain
      “receipt of previous treatment for opioid dependence (yes, no), and (4) cocaine use in the past month (yes, no).”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    Potential confounders were examined with analysis of variance, and a known confounder was handled through stratified analyses.

    “We evaluated potential confounders in an analysis of variance and accounted for an established confounder with stratified analyses.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    The planned sample of 170 gave 80% power to detect a 3.0-week difference between arms in mean weeks of continuous abstinence at a two-sided .05 level, assuming a within-group SD of 7.0 weeks.

    “A sample size of N=170 was estimated to provide 0.80 power to detect a 3.0 week difference between treatment conditions in mean weeks of continuous abstinence using a .05 significance level on a two-sided t-test, while assuming a within-group standard deviation of 7.0 weeks based on reports from previous trials using CRA+.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Premisecitation

    A growing literature suggests technology-based interventions are more reliable, provide greater program fidelity, are easier to develop, reach larger populations, are more cost-effective, and provide similar client satisfaction than traditional face-to-face therapies.

    “Specifically, a growing literature suggests technology-based interventions are more reliable, provide greater program fidelity, are easier to develop, reach larger populations, are more cost-effective, and provide similar levels of client satisfaction than traditional face-to-face therapies (Craske et al., 2009; Griest, 2008; Munoz et al., 2009; Rotondi et al., 2010; Marsch et al., 2013).”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites Craske et al., 2009

      Not checked in the free run.

    • cites Griest, 2008

      Not checked in the free run.

    • cites Munoz et al., 2009

      Not checked in the free run.

36 of 36
  • The paper relies on this without support.

    The combination of CRA and CM for drug abstinence had not yet been tested with web-based delivery.

    “Further, no study to our knowledge has yet to trial the combination of CRA and CM for drug abstinence using web-based approaches.”
  • The paper relies on this without support.

    Prior CRA treatment had not been studied for its effect on clinical outcomes.

    “no study has examined the effect of previous CRA treatments on clinical outcomes.”
  • The paper relies on this without support.

    Increasing non-drug reinforcement and providing immediate abstinence incentives will outcompete drug use as the preferred behavior.

    “The increased non-drug sources of reinforcement and the immediately available incentives for abstinence are then expected to successfully compete with the powerful and immediately reinforcing effects of drug use as the preferred behavior.”
  • The paper relies on this without support.

    Participants in the trial had sufficient internet access to receive the internet-delivered CRA+ treatment.

    “This study examined the benefit of adding an internet-delivered therapy to motivational incentives and buprenorphine as treatments for opioid dependence.”
4 of 4

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Statistics recomputed

Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.

  • Consistentz = 2.09reported p = .037 · recomputed p = .037

    “z=2.09, p = .037”

1 of 1

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Self-check results appear here once an independent check has confirmed them.

Cites

Flagged first, then by citations

1 of the 36 references couldn’t be matched to a known paper.

Research works printed but not linked: 1

  1. Stitzer ML Higgins ST Bloom FE Kupfer DJ Behavioral treatment of drug and alcohol abuse Psychopharmacology, The Fourth Generation of Progress 1995 New York Raven Press 1807 1819

Cited by

Sorted by citations

Authors
Christensen, D. R., Landes, R. D., Jackson, L., Marsch, L. A., Mancino, M. J., Chopra, M. P., & Bickel, W. K.
Affiliations
  • University of Lethbridge (CA)
  • University of Arkansas for Medical Sciences (US)
  • Dartmouth College (US)
  • Boston University (US)
Venue
Journal of Consulting and Clinical Psychology
Published
2014
Volume
82
Issue
6
Pages
964-972
ISSN
  • 0022-006X (ISSN-L)
  • 1939-2117
OpenAlex
W1985718825
Cited by
149
References
36
Citations by year
2026: 5 · 2025: 14 · 2024: 9 · 2023: 24 · 2022: 19 · 2021: 24 · 2020: 24 · 2019: 10 · 2018: 6 · 2017: 6 · 2016: 5 · 2015: 2 · 2014: 1
FWCI
4.58
Type
journal article
Language
en
Open access
Locations
  • Journal of Consulting and Clinical Psychology (publishedVersion)
  • PubMed (publishedVersion)
  • PubMed Central (acceptedVersion)
Funders
  • National Institute on Drug Abuse (R01 DA 12997)
  • Wilbur Mills Endowment
  • National Center for Advancing Translational Science (UL1TR000039)
Topics
  • Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Opioid Use Disorder Treatment
  • Health Sciences › Medicine › Epidemiology › Substance Abuse Treatment and Outcomes
  • Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
Keywords
opioid use disorder, internet-delivered treatment, community reinforcement approach, contingency management, buprenorphine, abstinence, treatment retention
MeSH terms
Adult, Aged, Analgesics, Opioid, Buprenorphine, Combined Modality Therapy, Female, Humans, Male, Middle Aged, Motivation, Narcotic Antagonists, Opioid-Related Disorders, Reinforcement, Psychology, Residence Characteristics, Cognitive Behavioral Therapy, Treatment Outcome, Internet, Opiate Substitution Treatment

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-07.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-07.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review