CoChat
Sign Up
Cited by 35Autism Spectrum Disorder ResearchHealth Policy Implementation ScienceBehavioral and Psychological Studies

Training clinicians to deliver group CBT to manage anxiety in youth with ASD: Results of a multisite trial.

Open access
Reaven, Judy, Moody, Eric J., Grofer Klinger, Laura, Keefer, Amy, Duncan, Amie, O'Kelley, Sarah, Meyer, Allison, Hepburn, Susan +1 more·2018·Journal of Consulting and Clinical Psychology
DOI 10.1037/ccp0000285PMC PMC5841598
Open this paper in CoChat — free account

Abstract

OBJECTIVE: Few guidelines are available regarding optimal training models for practitioners delivering cognitive-behavioral therapy (CBT) for anxiety in youth with autism spectrum disorder (ASD). The present study systematically compared 3 instructional conditions for delivering the Facing Your Fears program (FYF) to children with ASD and anxiety. METHOD: Thirty-four clinicians (Mage = 34 years; 94% women, 88% Caucasian) and an intent-to-treat sample of 91 children with ASD and anxiety (Mage = 11; 84% male 53% Caucasian) met eligibility criteria across 4 sites. A 3-group parallel design via a Latin square procedure was used to randomize 9 teams of clinicians to 1 of 3 training conditions: Manual, Workshop, Workshop-Plus. The effectiveness of instructional condition was assessed via implementation (CBT knowledge, treatment fidelity) and treatment outcomes (reductions in anxiety as measured by the Anxiety Disorders Interview Schedule-Parent (ADIS-P). RESULTS: Clinicians in both Workshop conditions significantly increased CBT knowledge postworkshop, F(1, 18) = 19.8, p < .001. Excellent treatment fidelity was obtained across conditions (above 89%), although clinicians in the Workshop conditions obtained significantly higher fidelity ratings and delivered FYF with greater quality than the Manual condition. Children with ASD demonstrated significant reductions in anxiety symptoms for three of the four anxiety diagnoses, with no differences noted across instructional condition. Rates of improvement were lower than those obtained in a previous controlled trial. CONCLUSIONS: Results suggest that although there may be some advantage to participating in a Workshop, clinicians in all conditions could deliver FYF with excellent fidelity and yield positive treatment outcomes. Lack of a no-treatment comparison group limits interpretation of findings. (PsycINFO Database Record

Integrity signals

Registry

Crossref · 10.1037/ccp0000285

Cites

31 of 31 references verified0 citation errors · 0 not checkable

Statistics

19 tests recomputed · 0 with the reported and recomputed p on opposite sides of .05

Claims the paper makes

0 of 41 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    Anxiety symptom reductions did not differ by instructional condition.

    “Children with ASD demonstrated significant reductions in anxiety symptoms for three of the four anxiety diagnoses, with no differences noted across instructional condition.”
    Quote in the paperSubstantial claim
  • Findingown data

    There was a significant effect of time on SOC severity.

    “There was a significant effect for time, F(1, 47.73) = 21.48, p <.0001, ωp2 = .29.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The observed reductions in anxiety symptoms are attributable to FYF treatment rather than passage of time, repeated assessment, or other uncontrolled factors.
  • Findingown data

    Workshop-condition clinicians had significantly higher fidelity ratings than Manual-condition clinicians.

    “although clinicians in the Workshop conditions obtained significantly higher fidelity ratings and delivered FYF with greater quality than the Manual condition.”
    Quote in the paperSubstantial claim
  • Findingown data

    There was a significant effect of time on SpP severity.

    “There was a significant effect for time, F(1, 73.23) = 17.25, p < .0001, ωp2 = .18.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The observed reductions in anxiety symptoms are attributable to FYF treatment rather than passage of time, repeated assessment, or other uncontrolled factors.
  • Findingown data

    Clinicians in the Manual condition were rated significantly lower in quality of treatment delivery than clinicians in the Workshop and Workshop-Plus conditions.

    “Tukey HSD post hoc comparisons revealed that clinicians in the Manual condition were rated as significantly lower in quality of treatment delivery (M = 4.26, SD = .95) compared to clinicians in the Workshop (M = 4.67, SD = .57), p < .001, and Workshop-Plus (M = 4.69, SD = .57), p <.001 conditions.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The fidelity ratings by unblinded raters are not biased by knowledge of instructional condition.
  • May combine two claims. 33
  • Classified as probably not a substantial claim. 38

  • May combine two claims.

    Findingown data

    There was no Time by Condition interaction or main effect of Condition for SOC.

    “There was no Time by Condition interaction or main effect of Condition.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The observed reductions in anxiety symptoms are attributable to FYF treatment rather than passage of time, repeated assessment, or other uncontrolled factors.

  • May combine two claims.

    Findingown data

    All instructional conditions exceeded the minimum standard for treatment fidelity (80%): Manual 89.9%, Workshop 92.8%, Workshop-Plus 93.2%.

    “All instructional conditions exceeded the minimum standard for treatment fidelity (80%) – Manual – 89.9%; Workshop – 92.8%; and Workshop-Plus – 93.2%.”
    • Also stated in: § Implementation outcomes · plain
      “Furthermore, clinicians in all three instructional conditions obtained excellent treatment fidelity for presence of core FYF components, and above the minimum standard for acceptable treatment fidelity.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    There was no Time by Condition interaction or main effect of Condition for GAD.

    “There was a significant effect for Time, F(1, 59.37) = 76.02, p<.0001, ωp2 = .64, but no effect for the Time by Condition interaction or a main effect for Condition.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The observed reductions in anxiety symptoms are attributable to FYF treatment rather than passage of time, repeated assessment, or other uncontrolled factors.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    Nine teams of clinicians delivered the FYF intervention to 24 cohorts across three years, with eight cohorts assigned to each instructional condition.

    “Nine teams of clinicians delivered the FYF intervention to 24 cohorts across three years. Eight cohorts were assigned to each instructional condition.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The proportions of youth meeting criteria for a clinically meaningful outcome were Workshop-Plus 25.9%, Workshop 44.4%, and Manual 29.6%.

    “Workshop-Plus – 25.9%; Workshop – 44.4 %; and Manual – 29.6%.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The observed reductions in anxiety symptoms are attributable to FYF treatment rather than passage of time, repeated assessment, or other uncontrolled factors.
    • Rests on an assumption: The ADIS-P and CGIS-I assessments by unblinded CEs are not biased by knowledge of instructional condition.

  • May combine two claims.

    Findingown data

    Youth who met criteria for a clinically meaningful outcome ranged from approximately 26% to 44%, with no significant differences across instructional conditions.

    “Finally, youth who met criteria for a clinically meaningful outcome (CGIS-I of 1 or 2) ranged from a low of approximately 26% to a high of 44%, although differences across instructional conditions were not significant.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The observed reductions in anxiety symptoms are attributable to FYF treatment rather than passage of time, repeated assessment, or other uncontrolled factors.
    • Rests on an assumption: The ADIS-P and CGIS-I assessments by unblinded CEs are not biased by knowledge of instructional condition.
    • Rests on an assumption: The sample size was sufficient to detect meaningful differences between instructional conditions.

  • May combine two claims.

    Findingown data

    Youth across all instructional conditions demonstrated significant reductions in anxiety symptoms, but there was variability by treatment site.

    “Youth across all instructional conditions demonstrated significant reductions in anxiety symptoms, but there was also variability by treatment site.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The observed reductions in anxiety symptoms are attributable to FYF treatment rather than passage of time, repeated assessment, or other uncontrolled factors.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Anxiety in youth with ASD disrupts functioning in multiple settings and is linked to negative outcomes such as unemployment, substance abuse, additional psychiatric conditions, and increased financial burden.

    “Anxiety interferes with functioning across contexts and is associated with a variety of negative outcomes (e.g., unemployment, substance abuse, additional psychiatric conditions, and increased financial burden; Gosch, Flannery-Schroeder, Mauro & Compton, 2006; van Steensel, Dirksen, & Bogels, 2013).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Gosch, Flannery-Schroeder, Mauro & Compton, 2006

      Not checked in the free run.

    • cites van Steensel, Dirksen, & Bogels, 2013

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    About 14 randomized controlled trials exist, most of which found that anxiety symptoms improved after these treatments.

    “To date, approximately 14 randomized controlled trials have been completed, the majority of which have documented a positive impact on anxiety symptoms following participation in these interventions (e.g., Reaven, Blakeley-Smith, Culhane-Shelburne, & Hepburn, 2012; Wood et al. 2009; Storch et al. 2015; Weston, Hodgekins, & Langdon, 2016).”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites Reaven et al. 2012

      Not checked in the free run.

    • cites Wood et al. 2009

      Not checked in the free run.

    • cites Storch et al. 2015

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Failure to address these factors can lead to outright rejection or inconsistent delivery, which may reduce fidelity and weaken treatment effects.

    “If these variables are not adequately addressed, outright rejection of an intervention can occur, as well as haphazard and/or partial delivery of treatment programs which can lead to poor treatment fidelity, limiting the effectiveness of the treatment even when delivered (Elkins, McHugh, Santucci, & Barlow, 2011; Wood et al. 2015).”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites Elkins, McHugh, Santucci, & Barlow, 2011

      Not checked in the free run.

    • cites Wood et al. 2015

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Youth with ASD are at high risk for co-occurring psychiatric conditions, and anxiety disorders are among the most common comorbid conditions.

    “Over the last decade, a steadily growing literature has documented that youth with Autism Spectrum Disorder (ASD) are at high risk for developing co-occurring psychiatric conditions, and anxiety disorders are among the most common co-morbid conditions (Simonoff et al., 2008; van Steensel, Bogels & Perrin, 2011).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Simonoff et al., 2008

      Not checked in the free run.

    • cites van Steensel, Bogels & Perrin, 2011

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    Of 109 children consented, 97 met study criteria for participation.

    “There were 109 children who were consented to participate in the study across sites. One child did not complete eligibility measures, and another 11 did not meet eligibility criteria; thus, 97 children met study criteria for participation.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Study design and conductnone given

    Clinicians administering the ADOS-2 were research reliable.

    “Clinicians administering the ADOS-2 were research reliable.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    Eighty-four children completed FYF, but parents of four of these children did not complete post-treatment assessments.

    “Eighty-four children completed FYF, but parents of the four children did not complete post treatment assessments.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    Every FYF session was video-recorded, encrypted, and sent to the Colorado site for fidelity coding and Workshop-Plus consultation.

    “All FYF sessions were digitally video-recorded and sent as encrypted files to the Colorado site, for purposes of coding fidelity as well as providing consultation for clinicians in Workshop-Plus condition.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductcitation

    The treatment fidelity measure used in this study was adapted from the Yale Adherence and Competence Scale (YACSII) Guidelines and had been applied in previous studies.

    “A treatment fidelity measure, adapted from the Yale Adherence and Competence Scale (YACSII) Guidelines (Nuro et al., 2005) and applied in previous studies (Reaven et al., 2014), was used to assess the absence/presence of core treatment components on a session-by-session basis.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Nuro et al., 2005

      Not checked in the free run.

    • cites Reaven et al., 2014

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    For each site and condition, the top ASD and anxiety experience levels were averaged and used as a model covariate.

    “In order to manage this challenge, we averaged the highest level of ASD and anxiety experience at each site for each instructional condition to use as a covariate in the statistical models.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    Workshop-Plus had fewer Caucasian families than chance would predict.

    “In addition, fewer Caucasian families were assigned to the Workshop-Plus condition than would be expected by chance (Caucasian-standardized residual −2.6).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Study design and conductown data

    Interclass correlations for all ADIS-P subscales were high: GAD = .81; social phobia = .89; separation anxiety = .84; specific phobia = .86.

    “Interclass correlations (ICC) for all subscales was high: GAD = .81; social phobia =.89; separation anxiety =.84; specific phobia =.86.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    A new intervention's perceived complexity or cultural insensitivity can hinder its adoption.

    “Perceived complexity, as well as cultural insensitivity of the new intervention can also present barriers to eventual adoption of EBPs and are critical to consider when moving an intervention from university to community settings (Ratto et al. 2016).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Ratto et al. 2016

      Not checked in the free run.

  • May combine two claims.

    Interpretationown data

    The findings suggest that while Workshop participation may offer some benefit, clinicians in every condition could deliver FYF with excellent fidelity and produce positive outcomes.

    “Results suggest that although there may be some advantage to participating in a Workshop, clinicians in all conditions could deliver FYF with excellent fidelity and yield positive treatment outcomes.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    In this sample, SCARED domain internal consistency was adequate, ranging from .72 to .88 for parent and child reports.

    “Within the present sample, internal consistency of the domains of the SCARED is adequate and ranged from .72 to .88 for both the parent and child report.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    The Latin square ensured that every training condition appeared once per site and once per year.

    “The Latin square was constructed so that each training condition ultimately occurred only once in each site, and once per year.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    Six children dropped out prior to beginning FYF, and another seven dropped out prior to completion.

    “Six children dropped out of treatment prior to beginning FYF, and another 7 dropped out of treatment prior to completion.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    Workshop-Plus received more Asian-American and African-American families than chance would predict.

    “however, there were more Asian-American and African-American families assigned to the Workshop-Plus condition than would be expected by chance (Asian-American standardized residual 3.7; African-American standardized residual 2.0).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    No universal guidelines specify who may deliver mental health treatments for youth with ASD or how such practitioners should be trained and supervised.

    “There are added challenges in the implementation of interventions to address the mental health symptoms of youth with ASD, as there are no universally established guidelines detailing who can deliver the treatments, nor are there guidelines for how to train and supervise these practitioners (Wood et al. 2015).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Wood et al. 2015

      Not checked in the free run.

  • May combine two claims.

    Study design and conductown data

    The three conditions did not differ before treatment on child age, ethnicity, gender, overall anxiety, number of anxiety diagnoses, or IQ.

    “There were no pre-treatment differences between the three conditions on age, ethnicity, gender, overall anxiety symptoms, number of anxiety diagnoses, or IQ”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    A subsample of 29% of CSRs was coded by a second coder to ensure reliability.

    “A subsample of 29% of CSRs was coded by a second coder to ensure reliability of the scores.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductown data

    The child participant sample was analyzed using an intent-to-treat approach, including all children who met eligibility and began treatment even if they dropped out.

    “The child participant sample was analyzed using an intent-to-treat (ITT) approach. All children who met eligibility criteria and began treatment were included in the analyses, even if they dropped out prior to finishing the study.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Clinicians find manuals helpful because they standardize how an intervention is delivered, define allowable adaptations, and offer advice for common difficulties.

    “Manuals are particularly useful to clinicians as they standardize intervention delivery, specify acceptable adaptations, and provide suggestions for trouble-shooting common problems that may arise during the course of the treatment (Kasari & Smith 2013).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Kasari & Smith 2013

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The ADIS-P has convergent validity with other anxiety measures.

    “as well as convergent validity with other anxiety measures (Lyneham & Rapee, 2005; Wood, Piancentini, Bergman, McCracken, & Barrios, 2002).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Lyneham & Rapee, 2005

      Not checked in the free run.

    • cites Wood, Piancentini, Bergman, McCracken, & Barrios, 2002

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The study followed the requirements of the Colorado Multiple IRB and each site's IRB.

    “This study was completed in compliance with the Colorado Multiple Institutional Review Board (University of Colorado Anschutz Medical Campus) and the IRB of each of the four sites.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    All clinicians, children, and caregivers gave informed consent or assent before data collection.

    “Informed consent and assent was obtained for all clinician, child, and caregiver participants prior to collecting data.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    In the same study, significant reductions in child problem behaviors followed treatment participation.

    “Results indicated that the therapists delivered the treatment with good fidelity, and significant reductions in child problem behaviors were apparent following treatment participation.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Brookman-Frazee, Drahota & Stadnick, 2012

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    ADIS-P administration training required clinicians to score above 80% reliability on clinical diagnoses and CSRs on three videotaped and three live administrations.

    “Training criteria included scoring above 80% reliability on clinical diagnoses and CSRs on three videotaped administrations of the ADIS-P and three live administrations (serving as the interviewer).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Partial omega squared was chosen as the effect size measure because it is less biased than partial eta squared and can be generalized to numerous research designs.

    “This measure is preferable because it is less biased than ηp2 (partial eta squared) and can be generalized to numerous research designs (Lakens, 2013).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Lakens, 2013

      Not checked in the free run.

  • May combine two claims.

    Study design and conductown data

    At baseline, clinicians did not differ by gender, ethnicity, education level, CBT experience, or ASD experience.

    “There were no baseline clinician differences for gender, ethnicity, highest level of education, years of CBT experience, or years working with ASD.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductown data

    Nine teams of clinicians were randomized to one of three training conditions using a Latin square procedure.

    “The study consisted of a 3-group parallel design where 9 teams (3 per year) of 3 clinicians were randomized via a Latin square procedure to receive one of three instructional conditions: 1) Manual, 2) Workshop, 3) Workshop-Plus (see descriptions below).”
    • Also stated in: § Method · plain
      “A three group parallel design via a Latin square procedure was used to randomize 9 teams of clinicians to one of three training conditions: Manual, Workshop, Workshop-Plus.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    A study by Brookman-Frazee et al. (2012) found that community mental health therapists delivered an EBP targeting challenging behaviors in school-aged youth with ASD with good fidelity.

    “Results indicated that the therapists delivered the treatment with good fidelity, and significant reductions in child problem behaviors were apparent following treatment participation.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Brookman-Frazee, Drahota & Stadnick, 2012

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    The FYF manual is user-friendly, as clinicians are able to pick it up and effectively deliver the program and yield significant reductions in anxiety symptoms.

    “Overall, these results are encouraging and may indicate that the FYF manual is indeed user-friendly, as clinicians are able to “pick it up” and not only effectively deliver the program, but yield significant reductions in anxiety symptoms.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The observed reductions in anxiety symptoms are attributable to FYF treatment rather than passage of time, repeated assessment, or other uncontrolled factors.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationown data

    The greater experience in the Manual condition may have washed out expected differences in fidelity.

    “potentially washing out expected differences.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationnone given

    The FYF manual was comprised of core CBT content that may have been familiar to clinicians employed in university clinics.

    “Additionally, the FYF manual was comprised of core CBT content that may have been familiar to clinicians employed in university clinics.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    Clinicians across instructional conditions could deliver FYF with excellent adherence to protocol and with excellent quality.

    “Results indicated that clinicians across instructional conditions could deliver FYF with excellent adherence to protocol and with excellent quality.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    Challenges in capturing the entirety of each group session on video may have limited the usefulness of consultation.

    “Challenges in capturing the entirety of each group session on video (e.g., filming all participants engaged in exposure practice), may have also limited the usefulness of consultation, particularly if the supervisor was unable to view portions of the session.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning

    The clinicians in this study were well educated and presented with good experience, which may have contributed to lack of differences across instructional conditions.

    “In fact, for the most part, the clinicians in this study were well educated and presented with good experience, which may have contributed to lack of differences across instructional conditions.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    Clinicians in this study worked in university-affiliated clinics and may be more skilled in working with specialty populations and diagnosis-specific interventions than practitioners in other community settings.

    “The clinicians all worked in clinics that were affiliated with Universities and may be more skilled in working with specialty populations and diagnosis-specific interventions than practitioners working in other community settings.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    These same expectations and supports may not occur in the real world.

    “These same expectations and supports may not occur in the “real world”.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Fidelity ratings were completed by two of the study authors who were not blinded to instructional condition.

    “Fidelity ratings were completed by two of the study authors who were not blinded to instructional condition”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationnone given

    It is plausible that there were subtle effects that the study was unable to detect given its sample size.

    “Moreover, it is plausible that there were subtle effects that we unable to detect given our sample size.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.

    Interpretationreasoning

    Procedural knowledge may have been high to begin with, and the details provided in the manual regarding application to youth with ASD may have been sufficiently clear to support good adherence to protocol, rendering ongoing supervision less necessary.

    “Thus, procedural knowledge may have been high to begin with, and the details provided in the manual regarding the application to youth with ASD may have been sufficiently clear to support good adherence to protocol, rendering ongoing supervision less necessary.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Study design and conductown data

    CBT Knowledge tests were not consistently given to clinicians in the Manual condition, making it impossible to know the extent to which participation in the Workshop conditions would have improved CBT knowledge above and beyond just reading the manual.

    “CBT Knowledge tests were not consistently given to clinicians in the Manual condition, making it impossible to know the extent to which participation in the Workshop conditions would have improved CBT knowledge above and beyond just reading the manual.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning

    Greater experience in working with anxiety disorders among Manual condition practitioners may have enhanced the effectiveness of clinicians within this condition, potentially accounting for the unexpectedly positive treatment outcome for youth in this condition.

    “As noted above, greater experience in working with anxiety disorders as demonstrated by practitioners assigned to the Manual condition may have enhanced the effectiveness of clinicians within this condition, potentially accounting for the unexpectedly positive treatment outcome for youth in this condition.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    Ongoing consultation may not have substantially enhanced clinician ability to deliver FYF, and attending a Workshop may be sufficient in some circumstances.

    “Perhaps the most parsimonious explanation is that ongoing consultation may not have substantially enhanced clinician ability to deliver FYF, and that attending a Workshop may be sufficient in some circumstances.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductown data

    Eighty children completed the FYF treatment and post-intervention ADIS-Ps.

    “Eighty children completed the FYF treatment and post-intervention ADIS-Ps.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Lack of a no-treatment comparison group represents a notable limitation when interpreting the results.

    “Lack of a no-treatment comparison group also represents a notable limitation when interpreting the results.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Study design and conductown data

    At the time of writing, there were no follow-up data on implementation outcomes or maintenance of treatment outcomes, so statements regarding long-term sustainability of FYF cannot be made.

    “Finally, at the writing of this paper, there were no follow up data on implementation outcomes or maintenance of treatment outcomes, so statements regarding the long term sustainability of FYF cannot be made.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    The CEs who completed ADIS-P interviews at each site did not participate in the treatment but were not blinded to instructional condition.

    “and although the CEs who completed ADIS-P interview at each site did not participate in the treatment, they too, were not blinded to instructional condition.”
    Quote in the paperNot a substantial claim
57 of 57

  • The paper relies on this without support.

    Many efficacy studies of CBT for anxiety in youth with ASD are conducted in academic settings.

    “As is frequently the case in the early design and development of novel interventions, many of these studies focus on efficacy, and are conducted in academic settings.”

  • The paper relies on this without support.

    Implementation of mental health treatments for youth with ASD in new sites is poorly understood.

    “For treatments targeting the mental health symptoms of youth with ASD, little is known about the implementation (e.g., feasibility, effectiveness) of these interventions in other sites.”

  • The paper relies on this without support.

    Research on implementing evidence-based mental health interventions for youth with ASD in community settings is still very limited.

    “The implementation of evidence based mental health interventions for children and adolescents with ASD in “real world” settings is in its infancy, with only a handful of studies that have begun to examine the implementation of EBPs targeting mental health symptoms in community settings.”

  • The paper relies on this without support.

    No studies have yet examined successful anxiety treatment for youth with ASD in community settings.

    “Information about successful treatment of anxiety in youth with ASD in community settings has not yet been studied.”

  • The paper relies on this without support.

    No known study has systematically compared different training strategies for manualized EBPs that treat anxiety in youth with ASD.

    “There are no known studies that have systematically compared differential instructional strategies for manualized EBPs for treating anxiety in youth with ASD.”

  • The paper relies on this without support.

    The observed reductions in anxiety symptoms are attributable to FYF treatment rather than passage of time, repeated assessment, or other uncontrolled factors.

    “Lack of a no-treatment comparison group also represents a notable limitation when interpreting the results.”

  • The paper relies on this without support.

    The two non-identical versions of the CBT knowledge test are equivalent in difficulty and content, so pre-post changes reflect knowledge gain.

    “Two similar but non-identical versions of the assessment were administered prior to and following the workshop.”

  • The paper relies on this without support.

    The fidelity ratings by unblinded raters are not biased by knowledge of instructional condition.

    “Fidelity ratings were completed by two of the study authors who were not blinded to instructional condition”

  • The paper relies on this without support.

    The ADIS-P and CGIS-I assessments by unblinded CEs are not biased by knowledge of instructional condition.

    “and although the CEs who completed ADIS-P interview at each site did not participate in the treatment, they too, were not blinded to instructional condition.”

  • The paper relies on this without support.

    The sample size was sufficient to detect meaningful differences between instructional conditions.

    “Moreover, it is plausible that there were subtle effects that we unable to detect given our sample size.”
10 of 10

Statistics that don’t add up

Each reported p-value recomputed from its test statistic and degrees of freedom.

  • Inconsistent p-value: χ2(2) = 1.63, p = .45

    Minor

    The paper reports p = .45 here; recomputing from the test statistic and its degrees of freedom gives p = .443. The check cannot tell whether the statistic or the p-value is the misprint.

    Reported p = .45 · recomputed p = .443

  • Inconsistent p-value: χ2(2) = 5.1, p = .78

    Minor

    The paper reports p = .78 here; recomputing from the test statistic and its degrees of freedom gives p = .078. The check cannot tell whether the statistic or the p-value is the misprint.

    Reported p = .78 · recomputed p = .078

2 of 2

Statistics recomputed

Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.

  • Doesn’t add upχ²(2) = 5.1reported p = .78 · recomputed p = .078

    “χ2(2) = 5.1, p = .78”

  • Doesn’t add upχ²(2) = 1.63reported p = .45 · recomputed p = .443

    “χ2(2) = 1.63, p = .45”

  • ConsistentF(1, 18) = 19.8reported p < .001 · recomputed p < .001

    “F(1,18)=19.8, p < .001”

  • ConsistentF(2, 31) = 5.34reported p = .01 · recomputed p = .010

    “F(2, 31) = 5.34, p = .01”

  • ConsistentF(3, 30) = 3.99reported p = .02 · recomputed p = .017

    “F(3, 30) = 3.99, p = .02”

  • ConsistentF(1, 21) = 0.002reported p = .96 · recomputed p = .965

    “F(1, 21) = 0.002, p = .96”

  • ConsistentF(2, 22) = 0.7reported p = .51 · recomputed p = .507

    “F(2, 22) = .70, p = .51”

  • ConsistentF(1, 18) = 19.8reported p < .001 · recomputed p < .001

    “F(1, 18) = 19.8, p < .001”

  • Consistentχ²(2) = 11.94reported p = .003 · recomputed p = .003

    “X2(2) = 11.94, p = .003”

  • ConsistentF(2, 1374) = 50.55reported p < .0001 · recomputed p < .001

    “F(2, 1374)=50.55, p < .0001”

  • ConsistentF(1, 59.37) = 76.02reported p < .0001 · recomputed p < .001

    “F(1, 59.37) = 76.02, p < .0001”

  • ConsistentF(1, 47.73) = 21.48reported p < .0001 · recomputed p < .001

    “F(1, 47.73) = 21.48, p < .0001”

  • ConsistentF(1, 73.23) = 17.25reported p < .0001 · recomputed p < .001

    “F(1, 73.23) = 17.25, p < .0001”

  • ConsistentF(2, 72.95) = 3.02reported p = .06 · recomputed p = .055

    “F(2, 72.95) = 3.02, p = .06”

  • ConsistentF(1, 21.95) = 3.57reported p = .072 · recomputed p = .072

    “F(1, 21.95) = 3.57, p = .072”

  • Consistentχ²(2) = 2.94reported p = .23 · recomputed p = .230

    “χ2(2) = 2.94, p = .23”

  • Consistentχ²(2) = 3.46reported p = .18 · recomputed p = .177

    “χ2(2) = 3.46, p = .18”

  • Consistentχ²(2) = 0.07reported p = .97 · recomputed p = .966

    “χ2(2) = 0.07, p = .97”

  • Consistentχ²(3) = 11.68reported p = .009 · recomputed p = .009

    “χ2(3) = 11.68, p = .009”

19 of 19

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Reported interaction degrees of freedom do not match the design

    Minor

    A time-by-condition interaction with two time points and two training conditions has one numerator degree of freedom, not the reported two.

  • Workshop outcome percentages exceed 100%

    Minor

    The Workshop percentages for clinically meaningful improvement, somewhat improved, and no real change add to 101.5%. These mutually exclusive outcomes cannot exceed 100%, even allowing for rounding.

2 of 2

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Related

Authors
Reaven, J., Moody, E. J., Grofer Klinger, L., Keefer, A., Duncan, A., O'Kelley, S., Meyer, A., Hepburn, S., & Blakeley-Smith, A.
Affiliations
  • University of Colorado Anschutz (US)
  • Kennedy Krieger Institute (US)
  • Cincinnati Children's Hospital Medical Center (US)
  • Colorado State University (US)
ORCID
Venue
Journal of Consulting and Clinical Psychology
Publisher
American Psychological Association (APA)
Published
2018-03
Volume
86
Issue
3
Pages
205-217
ISSN
  • 0022-006X (ISSN-L)
  • 1939-2117
DOI
10.1037/ccp0000285
PMC
PMC5841598
OpenAlex
W2789765526
Cited by
35
References
31
Citations by year
2026: 1 · 2025: 2 · 2024: 6 · 2023: 3 · 2022: 6 · 2021: 8 · 2020: 3 · 2019: 3 · 2018: 3
FWCI
2.53
Type
journal article
Language
en
Open access
Licence
Locations
  • Journal of Consulting and Clinical Psychology (publishedVersion, other-oa)
  • PubMed (publishedVersion)
  • Europe PMC (PubMed Central) (acceptedVersion)
  • PubMed Central (acceptedVersion)
Funders
  • Health Resources and Services Administration (T73MC11044)
  • Administration on Intellectual and Developmental Disabilities (90DD0632)
  • National Institutes of Health (R33MH089291-03)
Topics
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Autism Spectrum Disorder Research
  • Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
  • Social Sciences › Psychology › Developmental and Educational Psychology › Behavioral and Psychological Studies
Keywords
autism spectrum disorder, anxiety, cognitive behavioral therapy, group therapy, clinician training, intervention fidelity, training workshop, children, anxiety symptoms
MeSH terms
Autism Spectrum Disorder, Adolescent, Anxiety Disorders, Child, Education, Continuing, Female, Humans, Male, Psychotherapy, Group, Cognitive Behavioral Therapy, Disease Management

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read under definition version 3 (extract-v3) on 2026-09-28.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-09-28.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review