Treatment adherence, competence, and outcome in individual and family therapy for adolescent behavior problems

“Cohens’ d effect size for this association (van Lier, Muthe´n, van der Sar, & Crijnen, 2004) was .44; according to Cohen (1988) , d .20 is a small effect, .50 is medium, and .80 large.”

“Both the CBT and MDFT items demonstrated good-to-excellent interrater reliability for adherence but only fair-to-poor reliability for competence as measured by the intraclass correlation coefficient (ICC[1,2]; Shrout & Fleiss, 1979 ).”

“Missing data were addressed with full information maximum likelihood estimation, which produces unbiased parameter estimates under the assumption that data are missing at random (Schafer & Graham, 2002).”

“For this subgroup, the featured elements of manualized treatments designed to target drug use and externalizing behaviors specifically may need to be moderated in favor of auxiliary interventions that directly target internalizing problems: changing unrealistic negative thoughts, interpersonal problem-solving skills,…”

“Another study limitation is that a full examination of treatment dropout effects on fidelity–outcome relations (see Hedeker & Gibbons, 1997) was beyond the scope of this study.”

“For example, Hogue et al. (2008) found no relationship between competence and intervention outcomes.”

“Furthermore, findings from a few of the studies reviewed (e.g., Barber et al, 2006, 2008; Hogue et al., 2008; Piper et al., 1991) suggest that, at least in some contexts, the relation between adherence or competence and outcome may be nonlinear (e.g., quadratic).”

“Moreover, conceptualizations of fidelity vary in the role of clinician competence, with some suggesting that it is a component of fidelity (e.g., Hogue et al., 2008) and others suggesting that it may reflect a related but distinct construct (e.g., Barber, Sharpless, Klosterman, & McCarthy, 2007).”

“Several studies have found that therapist adherence to the treatment protocol is an important predictor of clinical outcomes, especially when studies are conducted in community settings (Henggeler et al., 1999; Hogue et al., 2008; Huey et al., 2000; Schoenwald et al., 2000).”

“However, others view a tighter focus on fidelity as essential to ensure greater validity of the intervention evaluation, as well as a more transparent intervention process (Hogue et al., 2008; Holliday, Audrey, Moore, Parry-Langdon, & Campbell, 2009; Moskowitz, 1993).”

“Bridge, & Birmaher, 2000) and one model of family therapy other than MST (Hogue et al., 2008).”

“The main mechanism explored to date has been treatment adherence and/ or therapist competence (Hogue et al., 2008).”

“Research relating implementation fidelity and outcomes has shown variable results (Barber et al., 2006; Hogue et al., 2008; Huey, Henggeler, Brondino, & Pickrel, 2000), and little attention has been paid to fidelity over time (Zvoch, 2009).”

“Hogue et al. (2008) examined the impact of adherence in multidimensional family therapy for adolescents with externalizing behaviors and found a linear relationship between adherence and outcomes for externalizing problems, but moderate levels of adherence were associated with the stronger outcomes…”

“…has shown strong psychometric properties in studies of treatment adherence (Diamond et al. 2007; Hogue et al. 1998, 2008a), therapist competence (Hogue et al. 2008a), and fidelity-outcome links (Hogue et al. 2006, 2008b) with samples including drug-using, conduct-disordered, and depressed teens.”

“Alternatively, defining quality in terms of maintaining fidelity to EBTs may contribute to more favorable outcomes (e.g., Hogue et al., 2008; Schoenwald, Sheidow, & Chapman, 2009), though such an approach may increase costs for the agency through the need to monitor fidelity.”

“In the two studies in which it was investigated, an interaction between adherence and competence was not found in the prediction of outcomes [11, 12].”

“As with previous findings on fidelity (e.g., Hogue et al., 2008), relationships between process variables of this nature and clinical change may be nonlinear, and they may be moderated by client characteristics or clinical presentation.”

“Hogue et al. (2008) found a similar relationship when examining CBT and family therapy interventions for adolescent problem behaviors.”

“…be another important variable in treatment integrity (Barber et al., 2006; Carroll et al., 2000), but “few studies have examined whether competence predicts outcome, and here too the findings are inconsistent, with competence showing moderate effects in some studies” (Hogue et al., 2008, p. 545).”

“Implementer Competence Treatment studies pioneered research on interventionist competence (e.g., Barberet al., 1996; Creed & Kendall, 2005; Davidson, Scott, Schmidt, Tata, Thornton et al., 2004; Hogue et al., 2008).”

“Few studies have examined whether therapist competence predicts treatment outcome, and results are often nonsignificant (e.g., Barber et al., 2006), potentially due to the difficulty of reliably rating competence (Hogue et al., 2008).”

“Overly rigid implementation can sometimes reduce the positive effects of a practice because special educators usually need to individualize and personalize instruction to meet the unique needs of their students (Harn, Parisi, & Stool - miller, in press; Hogue et al., 2008).”

“…mechanisms of action in attachmentbased models may be scarce due to methodological challenges involved in measuring treatment integrity and establishing and isolating integrity–outcome associations (see Hogue et al., 2008; Nock, 2007; Perepletchikova & Kazdin, 2005; Perepletchikova et al., 2007).”

“Varying levels of adherence help predict changes in youth behavior problems and internalizing symptoms: High levels are associated with reduced delinquency, drug use, and other externalizing symptoms in FFT, MDFT, MST, and BSFT, while intermediate levels are linked to improvements in youth internalizing disorders in M…”

“In general, findings indicate that higher rates of adherence have predicted better clinical outcomes (Hogue et al., 2008; Huey, Henggeler, Brondino, & Pickrel, 2000).”

“tion Science: Introduction to Special Issue”

“Howell, J. C. (2003). Diffusing research into practice using the comprehensive strategy for serious, violent, and chronic juvenile offenders.”

“The service component of the 12-step model emphasizes helping other people with SUDs which also provides significant psychological and social benefits (Moos, 2008).”

“, coach family interactions; improve communication and attachment) and that predict long-term gains in family functioning as well as adolescent symptoms (Hogue et al., 2006; Hogue et al., 2008; Hogue, Liddle, Dauber, & Samuolis, 2004).”

“…approaches to fidelity assessment exist that rely on various informants (e.g., observers, practitioners [clinicians, teachers], supervisors; e.g., Hogue et al., 2008; McLeod, Sutherland et al., 2022), but the different fidelity informants often do not agree in their judgments (Hogue et al.,…”

“Provider integrity to the treatment model and competence in the delivery of evidence-based practices, which are potential formative outcomes of supervision, have been linked to psychosocial treatment outcomes (Hogue et al., 2008; Schoenwald et al., 2009a, 2004).”

“That said, the ICC for competence is consistent with previous studies (e.g., Barber, Crits-Christoph, & Luborsky, 1996; Hogue et al., 2008), and interrater reliability tends to be lower for competence than adherence (e.g., Carroll et al., 2000; Hogue et al., 2008).”

“, coach in-session family interactions; target caregivers for change) that predict long-term improvement in family functioning as well as adolescent symptoms (Hogue, Dauber, Samuolis, & Liddle, 2006; Hogue et al., 2008).”

“Moreover, even reliable measures of competence do not reliably predict outcomes, a counterintuitive but persistent finding (Barber et al. 2007; Hogue et al. 2008a, b).”

“This, in part, is attributable to empirical findings linking therapist adherence and desirable clinical outcomes (e.g., Hogue et al., 2008; Sexton & Turner, 2010).”

“However, other researchers have found nonlinear relationships between intervention fidelity and outcome, such that extremely high fidelity and extremely low fidelity, relative to moderate levels of fidelity, were linked with poorer outcomes (Barber et al., 2006; Hogue et al., 2008).”

“Similar to the adherence-outcome literature, competence-outcome findings in studies of CBT have been inconsistent, with some reporting positive findings, at least for some measures (eg, Kuyken & Tsivrikos, 2009 ; Shaw et al, 1999 ; Strunk, Brotman, DeRubeis, & Hollon, 2010 ) and others reporting null findings (eg, Hog…”

“Previous research has found that the proportion of variance explained by the therapist does vary across scores on scales designed to capture different therapeutic approaches (see Hill, O’Grady, & Elkin, 1992; Hogue et al., 2008).”

“However, only three of these studies evaluated five or more sessions per patient (Huppert, Barlow, Gorman, Shear, & Woods, 2006; Ogrodniczuk & Piper, 1999; Shaw et al., 1999), and only 11 studies used seven or more patients per therapist (Barber, Crits-Christoph, & Luborsky, 1996; Barber et al., 2006, 2008; Carroll, N…”

“This is because treatment integrity is an indicator of training effectiveness (Crits-Christoph et al., 1998; Rakovshik & McManus, 2010; Sholomskas et al., 2005) and a predictor of clinical outcomes (Hogue et al., 2008; Schoenwald, Sheidow, & Letourneau, 2004).”

“Most studies opted to measure a multitude of outcomes at once (Butler et al., 2013; Carlson & de Mamani, 2010; Gutermann et al., 2015; Hogue et al., 2008b; Martino et al., 2008; Paivio et al., 2004; Papas et al., 2011) while others chose only one or two outcomes, but opted for several di ff erent…”

“…meaningful child- and parent-level outcomes (e.g., Forgatch, Patterson, & DeGarmo, 2005; Henggeler, Melton, Brondino, Scherer, & Hanley, 1997; Hogue et al., 2008; Ogden, Hagen, Askeland, & Christensen, 2009; Smith, Dishion, Shaw, & Wilson, 2013), few studies have evaluated fidelity to…”

“Since treatment delivery is linked with clinical outcomes (Hogue et al., 2008), it is important to understand if differences in treatment delivery across settings do exist.”

“mented, but high fidelity may indicate that not enough flexibility exists to respond to client needs [16, 17].”

“Greater adherence has been linked with better outcomes in youth psychotherapy (e.g. Hogue et al., 2008; Schoenwald, Carter, Chapman, & Sheidow, 2008).”

“…integrity indicates to what extent an intervention encourages learners to engage in appropriate behaviors, and empirical studies have reported that there is a positive relation between an intervention ’ s treatment integrity scores and participants ’ learning outcomes (Hogue et al., 2008).”

“Other studies have shown adherence-outcome relationships only when therapeutic alliance was low (8,9) or when therapeutic alliance was controlled (10).”

“Adherence to MDFT was measured using the Multidimensional Family Therapy Intervention Inventory (MII; Rowe, Dakof, & Liddle, 2007), an extension of a well-validated adherence measure, the Therapist Behavior Rating Scale (Hogue, Liddle, & Rowe, 1996; Hogue et al., 2006, 2008).”

“Therapists’ competence is considered to be another important variable in psychotherapy (Baldwin & Imel, 2013; Barber et al., 2006; Carroll et al., 2000), although most findings are inconsistent but still show moderate effects in some studies (Hogue et al., 2008; Webb, DeRubeis, & Barber, 2010).”

“These integrity components may also reflect the degree to which training and supervision impact therapist behavior (Proctor et al., 2011; Schoenwald et al., 2011), and thus predict EBT effectiveness (e.g., Hogue et al., 2008; Robbins et al., 2011).”

“Objective measures of fi delity are also more likely to be linked to outcomes than self-report data (Hansen, Graham, Wolkenstein, & Rohrbach, 1991; Hogue et al., 2008; Knoche, Sheridan, Edwards, & Osborn, 2010; Lillehoj, Griffi n, & Spoth, 2004; Trepka, Rees, Shapiro, Hardy, & Barkham, 2004).”

“The goal was to identify whether the scaling (1-7) was appropriate to capture the range of skills in implementing these interventions and to identify challenges in coding.”

“…critical methodological decisions regarding measurement of treatment processes and outcomes, as well as selection of potential covariates that may be confounds (e.g., client and therapist characteristics, relationship factors, etc.; Beauchaine, Webster-Stratton, & Reid, 2005; Hogue et al., 2008).”

“However, emerging research (Hogue et al., 2008; McMaster et al., 2010) suggests that very high levels of implementation fidelity may actually decrease an intervention’s effectiveness.”

“…fidelity may lead to greater treatment gains (e.g. Durlak & DuPre, 2008; Forgatch, Patterson, & DeGarmo, 2005), others have found that extremely high fidelity and extremely low fidelity may lead to poorer intervention outcomes compared to moderate fidelity (Barber et al., 2006; Hogue et al., 2008).”

“The TBRS has been found to be reliable in a number of previous studies (Diamond et al., 2007; Hogue et al., 2008).”

“…(Breitenstein et al. 2010; Cross and West 2011; Dumas et al. 2001; Durlak and DuPre 2008; Forgatch and DeGarmo 2011; Hepner et al. 2011; Hogue et al. 2008; Schoenwald et al., Electronic supplementary material The online version of this article (doi:10.1007/s11121-014-0488-9) contains…”

“Adherence measures provide a comprehensive overview of the content of treatments by listing their critical domains and practice elements (Hogue et al., 2008).”

“Additionally, it builds upon research that has examined the relationship between therapist competency and treatment outcome for clients [36,69,70].”

“Manuals have been shown to lead to better outcomes and even improve the therapeutic relationship (Hogue et al., 2008; Loeb et al., 2005).”

“Allerdings sind diese Zusammenhänge nur für einige Studien eindeutig [DeRubeis und Feeley, 1990; Hogue et al., 2008; Kingdon et al., 1996; Kuyken et al., 2009; Trepka et al., 2004].”

“Two studies (Chinchilla, 2007; Hogue et al., 2008) aimed to review five sessions of an intervention (two from the beginning and three sessions from later in therapy).”

“Adherence refers to the extent to which interventionists’ behaviors conform to the intervention protocol (Hogue et al., 2008).”

“Higher therapist adherence ratings predict subsequent improvements in depression,[18] especially early in treatment,[3] and substance abuse.[19, 20] However, there are contradictory findings as well.”

“…between quality of alliance and intervention outcome in adolescents (e.g., Diamond et al., 2006; Eltz & Shrik, 1995; Hintikka, Laukkanen, Marttunen, & Lehtonen, 2006; Hogue, Dauber, Cecero, Stambaugh, & Liddle, 2006; Hogue et al., 2008; Kazdin, Marciano, & Whitley, 2005; Tetzlaff et al., 2005).”

“…children has not been reported, the interrater reliability of the BiCACS was comparable with the reliability at the item (e.g., Barber, Mercer, Krakauer, & Calvo, 1996; Hogue et al., 2008) and subscale (e.g., Carroll et al., 2000) level for treatment integrity measures in the psychotherapy field.”

“Adherence is the extent to which practitioners’ behaviors conform to the intervention protocol (Hogue et al. 2008), whereas competence refers to the skillfulness of delivering the intervention (Forgatch et al.”

“…shown to demonstrate good psychometric properties (e.g., Barber, Mercer, Krakauer, & Calvo, 1996; Carroll et al., 2000; Henggeler & Bourdin, 1992; Hogue et al., 2008; Madson, Campbell, Barrett, Brondino, & Melchert, 2005; Moyers, Martin, Manuel, Hendrickson, & Miller, 2005; Pierson et al., 2007).”

“Fidelity to EBPs has been shown to predict patient outcomes for a number of mental health disorders [44-49], and has been associated with PTSD symptom outcomes in our preliminary research on CPT.”

“Other research has found that although greater adherence predicts greater improvement for some outcomes, there is a curvilinear relationship between integrity and outcome for others, with intermediate levels of adherence predicting better outcomes than low or high levels of adherence (Hogue et al., 2008).”

“…adherence are problematic because high levels of treatment adherence have been associated with positive client outcomes (Guydish et al., 2014; Hogue et al., 2008; Sexton & Turner, 2010), client therapeutic skill acquisition (Webb et al., 2012), therapeutic alliance (Brauhardt et al., 2014;…”

“Table 5 shows that 3 studies assessed treatment integrity in terms of adherence and competence (Hogue et al., 2008; Robbins et al., 2011), 2 studies assessed treatment integrity only as competence (Eames et al.”

“…reductions of externalizing behavior problems, and this appeared not only true for PMTO (Forgatch and DeGarmo 2011; Forgatch et al. 2005; Hukkelberg and Ogden 2013) but also for other interventions aimed at externalizing behavior problems in children (e.g., Hogue et al. 2008; Huey et al. 2000).”