Multidimensional Implementation Evaluation of a Residential Treatment Program for Adolescent Substance Abuse
The DOI matches its Crossref record; 41 of the 42 references it cites were found in a registry or an index.
Abstract
The authors applied contemporary methods from the evaluation literature to measure implementation in a residential treatment program for adolescent substance abuse. A logic model containing two main components was measured. Program structure (adherence to the intended framework of service delivery) was measured using data from daily activity logs completed by program staff. Treatment process, conceptualized as therapeutic milieu, was measured using an adapted version of a scale used to measure implementation in therapeutic communities. In addition, variability in implementation was measured using statistical process control (SPC) procedures. Adolescents completed, on average, 50% of the weekly prescribed services. The milieu of the program was rated by the adolescents as highly therapeutic. Moreover, preliminary psychometrics suggest therapeutic milieu can be measured reliably in adolescents. These two main variables were implemented with consistency across adolescents. These findings are discussed along with implications for evaluation work in similar fields.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“A study measuring fidelity to an exercise programme for women with hip fractures used direct observation by the designers of the intervention to monitor the intervention that was actually being delivered, and then provided feedback to the exercise trainers [21]. In this way, deviations from the intended content of the programme were addressed and corrected, and high fidelity was achieved.”
From A conceptual framework for implementation fidelity (2007) · cites this paper as [21]
In the citing paper
“With the exception of a few studies that do measure quality of delivery or participant responsiveness [8,20,38], most implementation fidelity research focuses solely on a fidelity score determined almost exclusively by adherence [3,6-8,21,22,27-29,38,39].”
From A conceptual framework for implementation fidelity (2007) · cites this paper as [21]
Citations
Cites
Dusenbury, L. (2003) · Health Education Research · doi:10.1093/her/18.2.237 · Cited by 1,671
Dane, A. V., & Schneider, B. H. (1998) · Clinical Psychology Review · doi:10.1016/s0272-7358(97)00043-3 · Cited by 1,557 · cited 1×
“Recent calls for accountability and better understanding of intervention processes (Dane & Schneider, 1998; Gresham, Gansle, Noell, Cohen, & Rosenblum, 1993; Moncher & Prinz, 1991; Waltz, Addis, Koerner, & Jacobson, 1993) have been answered by a growing emphasis on implementation evaluation…”
BackgroundHandbook of Practical Program Evaluation
(2015) · Wiley eBooks · doi:10.1002/9781119171386 · Cited by 1,353
Handbook of Applied Social Research Methods
Bickman, L., & Rog, D. J. (1998) · Cited by 1,169
Handbook of Practical Program Evaluation
Wholey, J. S., Hatry, H. P., & Newcomer, K. E. (2010) · Cited by 960
Fidelity Criteria: Development, Measurement, and Validation
MOWBRAY, C. (2003) · The American Journal of Evaluation · doi:10.1016/s1098-2140(03)00057-2 · Cited by 884
Testing the integrity of a psychotherapy protocol: Assessment of adherence and competence
Waltz, J., Addis, M. E., Koerner, K., & Jacobson, N. S. (1993) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.61.4.620 · Cited by 873
Treatment fidelity in outcome studies
Moncher, F. J., & Prinz, R. J. (1991) · Clinical Psychology Review · doi:10.1016/0272-7358(91)90103-2 · Cited by 797 · cited 2×
“…for accountability and better understanding of intervention processes (Dane & Schneider, 1998; Gresham, Gansle, Noell, Cohen, & Rosenblum, 1993; Moncher & Prinz, 1991; Waltz, Addis, Koerner, & Jacobson, 1993) have been answered by a growing emphasis on implementation evaluation (Dusenbury,…”
BackgroundThe Therapeutic Community: Theory, Model, and Method
Leon, G. D. (2000) · Medical Entomology and Zoology · Cited by 516
Examining the Role of Implementation Quality in School-Based Prevention Using the PATHS Curriculum
Kam, C.-M., Greenberg, M. T., & Walls, C. T. (2003) · Prevention Science · doi:10.1023/a:1021786811186 · Cited by 515
The Therapeutic Community: Theory, Model, and Method
De Leon, G. (2000) · Springer Publishing Company eBooks · doi:10.1891/9780826116673 · Cited by 452
Yeaton, W. H., & Sechrest, L. (1981) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.49.2.156 · Cited by 404
An Evaluation of Drug Treatments for Adolescents in 4 US Cities
Hser, Y.-I., Grella, C. E., Hubbard, R. L., Hsieh, S.-C., Fletcher, B. W., Brown, B. S., & Anglin, M. D. (2001) · Archives of General Psychiatry · doi:10.1001/archpsyc.58.7.689 · Cited by 341 · cited 4×
“This latter example represents a role for SPC that falls more in line with the traditional use of SPC to monitor product output in industrial settings ( Hoyer & Ellis, 1996 ).”
BackgroundResultInfluentialCommunity clinic treatment of depressed youth: Benchmarking usual care against CBT clinical trials.
Weersing, V. R., & Weisz, J. R. (2002) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.70.2.299 · Cited by 301
Treatment Integrity of School-Based Behavioral Intervention Studies: 1980–1990
Gresham, F. M., Gansle, K. A., Noell, G. H., Cohen, S., & Rosenblum, S. (1993) · School Psychology Review · doi:10.1080/02796015.1993.12085651 · Cited by 241
Why Program Implementation is Important
Durlak, J. A. (1998) · Journal of Prevention & Intervention in the Community · doi:10.1300/j005v17n02_02 · Cited by 236
On the Importance of Being Earnest about Treatment Integrity
Schlosser, R. W. (2002) · Augmentative and Alternative Communication · doi:10.1080/714043395 · Cited by 151
Design Sensitivity: Statistical Power for Applied Experimental Research
Lipsey, M. W., & Hurley, S. (2009) · Sage eBooks · doi:10.4135/9781483348858.n2 · Cited by 135
Patient histories, retention, and outcome models for younger and older adults in DATOS
Grella, C. E., Hser, Y., Joshi, V., & Anglin, M. D. (1999) · Drug and Alcohol Dependence · doi:10.1016/s0376-8716(99)00082-4 · Cited by 107
Adolescents in Therapeutic Communities: One-Year Posttreatment Outcomes
Jainchill, N., Hawke, J., De Leon, G., & Yagelka, J. (2000) · Journal of Psychoactive Drugs · doi:10.1080/02791072.2000.10400214 · Cited by 90
Hogue, A., Liddle, H. A., Rowe, C., Turner, R. M., Dakof, G. A., & LaPann, K. (1998) · Journal of Counseling Psychology · doi:10.1037/0022-0167.45.1.104 · Cited by 89
Self-citationTherapeutic Communities for Addictions: Readings in Theory, Research and Practice
Leon, G. D. (1986) · Cited by 86
Assessing program fidelity in substance abuse health services research
Orwin, R. G. (2000) · Addiction · doi:10.1046/j.1360-0443.95.11s3.3.x · Cited by 65
Clarifying the Nature of Therapeutic Community Treatment
Melnick, G., & Leon, G. D. (1999) · Journal of Substance Abuse Treatment · doi:10.1016/s0740-5472(98)00036-1 · Cited by 62
Effects of a rational-emotive school mental health program on children's emotional adjustment
Digiuseppe, R., & Kassinove, H. (1976) · Journal of Community Psychology · doi:10.1002/1520-6629(197610)4:4<382::aid-jcop2290040410>3.0.co;2-2 · Cited by 59
Strength of theory and theory of strength
Scott, A. G., & SECHREST, L. B. (1989) · Evaluation and Program Planning · doi:10.1016/0149-7189(89)90049-9 · Cited by 59
Ward Treatment Milieu and Posthospital Functioning
Klass, D. B. (1977) · Archives of General Psychiatry · doi:10.1001/archpsyc.1977.01770210061005 · Cited by 49
Program Strength and Fidelity in Evaluation
Summerfelt, W. T. (2003) · Applied Developmental Science · doi:10.1207/s1532480xads0702_2 · Cited by 47
Recruiting High-Risk Families into Family-Based Prevention and Prevention Research.
Hogue, A., Johnson‐Leckrone, J., & Liddle, H. A. (1999) · Journal of Mental Health Counseling · Cited by 40
Self-citationAccountability and Performance in Charter Schools in Florida: A Theory-Based Evaluation
Crew, R. E., & Anderson, M. R. (2003) · American Journal of Evaluation · doi:10.1016/s1098-2140(03)00030-4 · Cited by 36
Formative Evaluation for Implementation: Evaluating Educational Technology Applications and Lessons
Weston, T. (2003) · American Journal of Evaluation · doi:10.1177/109821400402500104 · Cited by 36
TREATMENT ENVIRONMENT AND PATIENT IMPROVEMENT
Alden, L. E. (1978) · The Journal of Nervous and Mental Disease · doi:10.1097/00005053-197805000-00003 · Cited by 33
APPLYING STATISTICAL PROCESS CONTROL TO CLINICAL DATA: AN ILLUSTRATION
Pfadt, A., Cohen, I. L., Sudhalter, V., Romanczyk, R. G., & Wheeler, D. J. (1992) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.1992.25-551 · Cited by 30
Patient and Staff Perceptions of a Psychiatric Ward Environment
Main, S., McBride, A. B., & AUSTIN, J. K. (1991) · Issues in Mental Health Nursing · doi:10.3109/01612849109040510 · Cited by 28
Forsetlund, L., Talseth, K. O., Bradley, P., Nordheim, L. V., & Bjørndal, A. (2003) · Evaluation Review · doi:10.1177/0193841x02250528 · Cited by 25
Monitoring change in community-oriented treatment programs
Bliss, F., Moos, R. H., & Bromet, E. J. (1976) · Journal of Community Psychology · doi:10.1002/1520-6629(197610)4:4<315::aid-jcop2290040402>3.0.co;2-w · Cited by 25
Using the Ward Atmosphere Scale to Help Change the Treatment Environment
Verinis, J. S., & Flaherty, J. A. (1978) · Psychiatric Services · doi:10.1176/ps.29.4.238 · Cited by 18
Green, R. S. (1999) · Evaluation and Program Planning · doi:10.1016/s0149-7189(99)00004-x · Cited by 16
Dewa, C. S., Horgan, S. A., Russell, M., & Keates, J. (2001) · Evaluation and Program Planning · doi:10.1016/s0149-7189(01)00023-4 · Cited by 12
Statistical process control and direct care staff performance
Dey, M. L., Sluyter, G. V., & Keating, J. E. (1994) · The Journal of Behavioral Health Services & Research · doi:10.1007/bf02521327 · Cited by 12
Friedman, A. S., Glickman, N. W., & Kovach, J. A. (1986) · The American Journal of Drug and Alcohol Abuse · doi:10.3109/00952998609083741 · Cited by 9
Research works printed but not linked: 8
- 10.1177/109821400302400204 · doi:10.1177/109821400302400204
- Hogue, A., 1999, Journal of Mental Health Counseling, 21, 337
- Hoyer, R. W., 1996, Quality Progress, 29, 65
- Minitab, Inc., 2000, Minitab reference manual release 9
- 10.1177/109821400302400303 · doi:10.1177/109821400302400303 · Paper page
- 10.1080/aac.18.1.36.44 · doi:10.1080/aac.18.1.36.44 · Paper page
- Verinis, J. S., 1978, Hospital and Community Psychiatry, 29, 238
- 10.1037/0022-006X.70.2.299 · DOI · Paper page
Cited by
A conceptual framework for implementation fidelity
Carroll, C., Patterson, M., Wood, S., Booth, A., Rick, J., & Balain, S. (2007) · Implementation Science · doi:10.1186/1748-5908-2-40 · Cited by 2,559
Escartí, A., Goig, R. L., & Wright, P. M. (2017) · Journal of Teaching in Physical Education · doi:10.1123/jtpe.2016-0200 · Cited by 115
Hogue, A., Ozechowski, T. J., Robbins, M. S., & Waldron, H. B. (2013) · Clinical Psychology: Science and Practice · doi:10.1111/cpsp.12023 · Cited by 62
Self-citationDevelopment of an audiotape review system for supporting adherence to an evidence-based treatment.
Sheidow, A. J., Donohue, B. C., Hill, H. H., Henggeler, S. W., & Ford, J. D. (2008) · Professional Psychology: Research and Practice · doi:10.1037/0735-7028.39.5.553 · Cited by 34
Rezze, B. D., Law, M., Eva, K. W., Pollock, N., & Gorter, J. W. (2013) · Disability and Rehabilitation · doi:10.3109/09638288.2013.775358 · Cited by 20
Metadata
- Authors
- Faw, L., Hogue, A., & Liddle, H. A.
- Affiliations
- National Clinical Assessment Authority,
- Columbia University (US)
- University of Miami (US)
- Venue
- American Journal of Evaluation
- Publisher
- SAGE Publications
- Published
- 2005-03
- Volume
- 26
- Issue
- 1
- Pages
- 77-94
- ISSN
- 1098-2140 (ISSN-L)
- 1557-0878
- OpenAlex
- W2117593804
- Semantic Scholar
- 18217d7edf6505b514c36da7c31b0abe7d55bef1
- Cited by
- 13
- References
- 42
- Citations by year
- 2023: 1 · 2020: 1 · 2018: 1 · 2017: 1 · 2015: 1 · 2013: 3 · 2012: 1
- FWCI
- 1.06
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
- Locations
- American Journal of Evaluation (publishedVersion)
- Topics
- Social Sciences › Public Administration › Evaluation and Performance Assessment
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Health Sciences › Medicine › Epidemiology › Substance Abuse Treatment and Outcomes
- Keywords
- implementation evaluation, residential treatment, adolescent substance abuse, logic model, therapeutic environment, therapeutic communities, statistical process control, program adherence, treatment process
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.