Measurement of fidelity of implementation of evidence‐based practices: Case example of the IPS Fidelity Scale.
The DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.
Abstract
[Clin Psychol Sci Prac 18: 126–141, 2011] Advances in the scientific study and dissemination of evidence-based practices (EBPs) require psychometrically valid fidelity scales, which measure adherence to program models. We examined the literature on the Individual Placement and Support Fidelity Scale to illustrate the strengths and limitations of this methodology. We found that this scale had excellent psychometric properties. Nine of 10 studies assessing its predictive validity found positive associations with employment outcomes. Its use in quality improvement was supported by positive reports from seven multisite projects. Although not yet evaluated as an accreditation tool, three states have adopted the scale for reimbursement purposes. Development of valid fidelity scales is necessary for the field to advance. Technical and logistical challenges impede the replication of these findings for other EBPs.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Wampold, B. E., Mondin, G. W., Moody, M., Stich, F., Benson, K., & Ahn, H.-N. (1997) · Psychological Bulletin · doi:10.1037/0033-2909.122.3.203 · Cited by 1,091
Specifying and reporting complex behaviour change interventions: the need for a scientific method
Michie, S., Fixsen, D., Grimshaw, J. M., & Eccles, M. P. (2009) · Implementation Science · doi:10.1186/1748-5908-4-40 · Cited by 976
Burns, T., Catty, J., Becker, T., Drake, R. E., Fioritti, A., Knapp, M., Lauber, C., Rössler, W., Tomov, T., van Busschbach, J., et al. (2007) · The Lancet · doi:10.1016/s0140-6736(07)61516-5 · Cited by 613
Self-citationPerepletchikova, F., Treat, T. A., & Kazdin, A. E. (2007) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.75.6.829 · Cited by 533
Henggeler, S. W., Pickrel, S. G., & Brondino, M. J. (1999) · Mental Health Services Research · doi:10.1023/a:1022373813261 · Cited by 449
Drake, R. E., O'Neal, E. L., & Wallach, M. A. (2008) · Journal of Substance Abuse Treatment · doi:10.1016/j.jsat.2007.01.011 · Cited by 447
Self-citationMeasurement of Fidelity in Psychiatric Rehabilitation
Bond, G. R., Evans, L., Salyers, M. P., Williams, J., & Kim, H.-W. (2000) · Mental Health Services Research · doi:10.1023/a:1010153020697 · Cited by 302
Self-citationAarons, G. A., Wells, R. S., Zagursky, K., Fettes, D. L., & Palinkas, L. A. (2009) · American Journal of Public Health · doi:10.2105/ajph.2009.161711 · Cited by 299
The fidelity‐adaptation debate: Implications for the implementation of public sector social programs
Blakely, C. H., Mayer, J. P., Gottschalk, R. G., Schmitt, N., Davidson, W. S., Roitman, D. B., & Emshoff, J. G. (1987) · American Journal of Community Psychology · doi:10.1007/bf00922697 · Cited by 297
The Hartford Study of Supported Employment for Persons With Severe Mental Illness.
Mueser, K. T., Clark, R. E., Haines, M., Drake, R. E., McHugo, G. J., Bond, G. R., Essock, S. M., Becker, D. R., Wolfe, R., & Swain, K. (2004) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.72.3.479 · Cited by 289
Self-citation
Research works printed but not linked: 100
- 2002, Supported employment implementation resource kit: User’s guide
- Becker, 2010, Validation of the Supported Employment Fidelity Scale
- Becker, 2003, A working life for people with severe mental illness, 10.1093/acprof:oso/9780195131215.001.0001 · DOI · Paper page
- Becker, 2001, Fidelity of supported employment programs and employment outcomes, Psychiatric Services, 52, 834, 10.1176/appi.ps.52.6.834 · DOI · Paper page
- Becker, 2008, Supported Employment Fidelity Scale
- Becker, 2008, Evidence-based supported employment fidelity review manual
- Becker, 2006, What predicts supported employment program outcomes?, Community Mental Health Journal, 42, 303, 10.1007/s10597-006-9037-8 · DOI
- Bell, 2007, An RCT of neurocognitive enhancement therapy with supported employment: Employment outcomes at 24 months, Schizophrenia Bulletin, 33, 420
- Bellack, 2004, Social skills training for schizophrenia: A step-by-step guide
- Bhar, 2009, Treatment integrity of studies that compare short-term psychodynamic psychotherapy with cognitive-behavior therapy, Clinical Psychology: Science and Practice, 16, 370, 10.1111/j.1468-2850.2009.01176.x · DOI · Paper page
- Bjorklund, 2009, Washington state’s initiative to disseminate and implement high-fidelity ACT teams, Psychiatric Services, 60, 24, 10.1176/appi.ps.60.1.24 · DOI
- Bond, 1998, Principles of the individual placement and support model: Empirical support, Psychiatric Rehabilitation Journal, 22, 11, 10.1037/h0095271 · DOI · Paper page
- Bond, 2004, Supported employment: Evidence for an evidence-based practice, Psychiatric Rehabilitation Journal, 27, 345, 10.2975/27.2004.345.359 · DOI · Paper page
- Bond, 2007, Modest implementation efforts, modest fidelity, and modest outcomes, Psychiatric Services, 58, 334, 10.1176/appi.ps.58.3.334 · doi:10.1176/appi.ps.58.3.334
- Bond, 1997, A fidelity scale for the individual placement and support model of supported employment, Rehabilitation Counseling Bulletin, 40, 265
- Bond, 2002, A scale to measure quality of supported employment for persons with severe mental illness, Journal of Vocational Rehabilitation, 17, 1, 10.3233/JVR-2002-00164 · DOI · Paper page
- Bond, 2008, Predictors of competitive employment among patients with schizophrenia, Current Opinion in Psychiatry, 21, 362, 10.1097/YCO.0b013e328300eb0e · DOI · Paper page
- Bond, 2008a, An update on randomized controlled trials of evidence-based supported employment, Psychiatric Rehabilitation Journal, 31, 280, 10.2975/31.4.2008.280.290 · DOI · Paper page
- Bond, 2008b, Fidelity of supported employment: Lessons learned from the National Evidence-Based Practices Project, Psychiatric Rehabilitation Journal, 31, 300, 10.2975/31.4.2008.300.305 · DOI · Paper page
- Bond, 2004, Prediction of outcome from the Dartmouth ACT Fidelity Scale, CNS Spectrums, 9, 937, 10.1017/S1092852900009792 · DOI · Paper page
- Bond, 2001, Dimensions of supported employment: Factor structure of the IPS Fidelity Scale, Journal of Mental Health, 10, 383, 10.1080/09638230120041146 · DOI
- Boyce, 2008, Factors influencing the delivery of evidence-based supported employment in England, Psychiatric Rehabilitation Journal, 31, 360, 10.2975/31.4.2008.360.366 · DOI
- Brekke, 1988, What do we really know about community support programs? Strategies for better monitoring, Hospital and Community Psychiatry, 39, 946
- Bremer, 2008, Pay for performance in behavioral health, Psychiatric Services, 59, 1419, 10.1176/appi.ps.59.12.1419 · DOI · Paper page
- Burns, 2007, Use of intensive case management to reduce time in hospital in people with severe mental illness: Systematic review and meta-regression, British Medical Journal, 335, 336, 10.1136/bmj.39251.599259.55 · DOI · Paper page
- Chandler, 2011, Fidelity and outcomes in six integrated dual disorders treatment programs, Community Mental Health Journal, 47, 82, 10.1007/s10597-009-9245-0 · doi:10.1007/s10597-009-9245-0
- Cook, 2005, Integration of psychiatric and vocational services: A multisite randomized, controlled trial of supported employment, American Journal of Psychiatry, 162, 1948, 10.1176/appi.ajp.162.10.1948 · DOI · Paper page
- Corbière, 2010, A Pan-Canadian evaluation of supported employment programs dedicated to people with severe mental disorders, Community Mental Health Journal, 46, 44, 10.1007/s10597-009-9207-6 · DOI
- Crowther, 2001, Helping people with severe mental illness to obtain work: Systematic review, British Medical Journal, 322, 204, 10.1136/bmj.322.7280.204 · DOI · Paper page
- Cunningham, 2006, The struggle to provide community-based care to low-income people with serious mental illness, Health Affairs, 25, 694, 10.1377/hlthaff.25.3.694 · DOI · Paper page
- Drake, 2006, Best practices: The Johnson & Johnson-Dartmouth Community Mental Health Program: Disseminating evidence-based practice, Psychiatric Services, 57, 302, 10.1176/appi.ps.57.3.302 · DOI
- Drake, 2009, Implementing evidence-based practices for people with schizophrenia, Schizophrenia Bulletin, 35, 704, 10.1093/schbul/sbp041 · DOI · Paper page
- Drake, 2006, Explaining the variance within supported employment programs: Comment on “What predicts supported employment outcomes?”, Community Mental Health Journal, 42, 315, 10.1007/s10597-006-9038-7 · DOI
- Drake, 2009, The science-to-service gap in real-world schizophrenia treatment: The 95% problem, Schizophrenia Bulletin, 35, 677, 10.1093/schbul/sbp047 · DOI · Paper page
- Drake, 2001, Implementing evidence-based practices in routine mental health service settings, Psychiatric Services, 52, 179, 10.1176/appi.ps.52.2.179 · DOI · Paper page
- Drake, 1999, A randomized clinical trial of supported employment for inner-city patients with severe mental illness, Archives of General Psychiatry, 56, 627, 10.1001/archpsyc.56.7.627 · DOI · Paper page
- Drake, 1996, The New Hampshire study of supported employment for people with severe mental illness: Vocational outcomes, Journal of Consulting and Clinical Psychology, 64, 391, 10.1037/0022-006X.64.2.391 · doi:10.1037/0022-006x.64.2.391 · Paper page
- 2005, Evidence-based mental health practice: A textbook
- van Erp, 2007, A multisite study of implementing supported employment in the Netherlands, Psychiatric Services, 58, 1421, 10.1176/appi.ps.58.11.1421 · DOI · Paper page
- Evans, 2008, Expert ratings on the critical ingredients of supported employment for people with severe mental illness, Psychiatric Rehabilitation Journal, 31, 318, 10.2975/31.4.2008.318.331 · DOI
- Falloon, 2005, The Clinical Strategies Implementation Scale to measure implementation of treatment in mental health services, Psychiatric Services, 56, 1584, 10.1176/appi.ps.56.12.1584 · DOI
- Finnerty, 2009, The State Health Authority Yardstick (SHAY), Community Mental Health Journal, 45, 228, 10.1007/s10597-009-9181-z · DOI
- Fixsen, 2005, Implementation research: A synthesis of the literature
- Frey, 2008, The Mental Health Treatment Study, Psychiatric Rehabilitation Journal, 31, 306, 10.2975/31.4.2008.306.312 · DOI
- Gawande, 2009, The checklist manifesto: How to get things right
- Glied, 2009, Better but not best: Recent trends in the well-being of the mentally ill, Health Affairs, 28, 637, 10.1377/hlthaff.28.3.637 · DOI · Paper page
- Gold, 2006, Randomized trial of supported employment integrated with assertive community treatment for rural adults with severe mental illness, Schizophrenia Bulletin, 32, 378, 10.1093/schbul/sbi056 · DOI · Paper page
- Gowdy, 2003, Practices differentiating high-performing from low-performing supported employment programs, Psychiatric Rehabilitation Journal, 26, 232, 10.2975/26.2003.232.239 · DOI · Paper page
- Gowdy, 2004, Organizational factors differentiating high-performing from low-performing supported employment programs, Psychiatric Rehabilitation Journal, 28, 150, 10.2975/28.2004.150.156 · DOI · Paper page
- Henry, 2009, September, Outcomes of supported employment services for adults with serious mental illness in Massachusetts: Findings from the Services for Education and Employment Technical Assistance Project (SEE-TAP)
- Hepburn, 2007, Extending the welcome mat: Opening DOoRS to employment for individuals with co-occurring mental health and substance use disorders
- Institute of Medicine, 2001, Crossing the quality chasm: A new health system for the 21st century
- Jerrell, 1999, Impact of robustness of program implementation on outcomes of clients in dual diagnosis programs, Psychiatric Services, 50, 109, 10.1176/ps.50.1.109 · DOI · Paper page
- Johnson, 2009, Service users’ perceptions of the effective ingredients in supported employment, Journal of Mental Health, 18, 121, 10.1080/09638230701879151 · DOI
- Kreyenbuhl, 2010, The Schizophrenia Patient Outcomes Research Team (PORT): Updated treatment recommendations 2009, Schizophrenia Bulletin, 36, 94, 10.1093/schbul/sbp130 · DOI · Paper page
- Kukla, 2009, The working alliance and employment outcomes for people with severe mental illness enrolled in vocational programs, Rehabilitation Psychology, 54, 157, 10.1037/a0015596 · DOI · Paper page
- Latimer, 1999, Economic impacts of assertive community treatment: A review of the literature, Canadian Journal of Psychiatry, 44, 443, 10.1177/070674379904400504 · DOI · Paper page
- Latimer, 2010, An effective intervention delivered at sub-therapeutic dose becomes an ineffective intervention, British Journal of Psychiatry, 196, 341, 10.1192/bjp.bp.109.073338 · DOI
- Leff, 2005, Effects of job development and job support on competitive employment of persons with severe mental illness, Psychiatric Services, 56, 1237, 10.1176/appi.ps.56.10.1237 · DOI
- Marshall, 2008, Key factors for implementing supported employment, Psychiatric Services, 59, 886, 10.1176/appi.ps.59.8.886 · DOI · Paper page
- Marty, 2001, The experts speak: The critical ingredients of strengths model case management, Psychiatric Rehabilitation Journal, 24, 214, 10.1037/h0095090 · DOI
- McDonnell, 1989, An analysis of the procedural components of supported employment programs associated with employment outcomes, Journal of Applied Behavior Analysis, 22, 417, 10.1901/jaba.1989.22-417 · DOI
- McGovern, 2007, Assessing the dual diagnosis capability of addiction treatment services: The Dual Diagnosis Capability in Addiction Treatment (DDCAT) Index, Journal of Dual Diagnosis, 3, 111, 10.1300/J374v03n02_13 · DOI
- McGrew, 2007, IPS fidelity survey of 17 supported employment programs in Indiana: Final report to SECT Center
- McGrew, 2008, Final report to SECT Center: IPS fidelity survey of supported employment programs in Indiana
- McGrew, 2009, Testing alternative fidelity assessment methods
- McGrew, 1995, Critical ingredients of assertive community treatment: Judgments of the experts, Journal of Mental Health Administration, 22, 113, 10.1007/BF02518752 · DOI · Paper page
- McGrew, 1994, Measuring the fidelity of implementation of a mental health program model, Journal of Consulting and Clinical Psychology, 62, 670, 10.1037/0022-006X.62.4.670 · doi:10.1037/0022-006x.62.4.670 · Paper page
- McGurk, 2008, Cognitive remediation and vocational rehabilitation, Psychiatric Rehabilitation Journal, 31, 350, 10.2975/31.4.2008.350.359 · DOI · Paper page
- McHugo, 1999, Fidelity to assertive community treatment and client outcomes in the New Hampshire Dual Disorders Study, Psychiatric Services, 50, 818, 10.1176/ps.50.6.818 · DOI · Paper page
- McHugo, 2007, Fidelity outcomes in the National Implementing Evidence-Based Practices Project, Psychiatric Services, 58, 1279, 10.1176/appi.ps.58.10.1279 · doi:10.1176/appi.ps.58.10.1279 · Paper page
- Miller, 2004, The Texas Medication Algorithm Project: Clinical results for schizophrenia, Schizophrenia Bulletin, 30, 627, 10.1093/oxfordjournals.schbul.a007111 · DOI · Paper page
- Morrissey, 2009, Searching for fidelity: Defining critical ingredients for forensic assertive community treatment
- Mowbray, 2003, Fidelity criteria: Development, measurement, and validation, American Journal of Evaluation, 24, 315, 10.1177/109821400302400303 · doi:10.1177/109821400302400303 · Paper page
- National Alliance on Mental Illness, 2006, Grading the states: A report on America’s health care system for serious mental illness
- New Freedom Commission on Mental Health, 2003, Achieving the promise: Transforming mental health care in America
- Ogden, 2009, Implementing and evaluating evidence-based treatments of conduct problems in children and youth in Norway, Research on Social Work Practice, 19, 582, 10.1177/1049731509335530 · DOI
- Oxman, 2006, A fidelity measure for integrated management of depression in primary care, Medical Care, 44, 1030, 10.1097/01.mlr.0000233683.82254.63 · DOI · Paper page
- Pearson, 2008, The impact of pay-for-performance on health care quality in Massachusetts, 2001-2003, Health Affairs, 27, 1167, 10.1377/hlthaff.27.4.1167 · DOI
- Petersen, 2006, Does pay-for-performance improve the quality of health care?, Annals of Internal Medicine, 145, 265, 10.7326/0003-4819-145-4-200608150-00006 · DOI · Paper page
- Rapp, 2009, Strengths-based case management fidelity
- Rapp, 2008, Evidence-based practice implementation strategies: Results of a qualitative study, Community Mental Health Journal, 44, 213, 10.1007/s10597-007-9109-4 · DOI
- Rapp, 2010, Evidence-based practice implementation in Kansas, Community Mental Health Journal, 46, 461, 10.1007/s10597-010-9311-7 · DOI
- Resnick, 2009, Measuring fidelity in the national dissemination of supported employment in the Veterans Health Administration
- Resnick, 2009, Scaling up the dissemination of evidence-based mental health practice to large systems and long-term time frames, Psychiatric Services, 60, 682, 10.1176/appi.ps.60.5.682 · doi:10.1176/appi.ps.60.5.682
- Salyers , M. P. McGuire , A. B. Rollins , A. L. Bond , G. R. Mueser , K. T. Macy , V. 2010 Integrating assertive community treatment and illness management and recovery for consumers with severe mental illness Community Mental Health Journal 46 319 329 · DOI
- Salyers, 2007, The role of technical assistance centers in implementing evidence-based practices: Lessons learned, American Journal of Psychiatric Rehabilitation, 10, 85, 10.1080/15487760701345968 · DOI · Paper page
- Substance Abuse and Mental Health Services Administration., 2008, The next generation of national reporting of evidence-based practices: Recommendations by an expert panel
- Swanson, 2008, Supported employment: A practical guide for practitioners and supervisors
- Taylor, 2007, Exploratory study of consumer perceptions of two employment models for people with severe mental illness
- Taylor, 2010, Employment specialists’ competencies as predictors of employment outcomes
- Taylor, 2009, A scale to evaluate quality of medication management: Development and psychometric properties, Administration and Policy in Mental Health and Mental Health Services Research, 36, 247, 10.1007/s10488-009-0209-z · DOI
- Teague, 1998, Program fidelity in assertive community treatment: Development and use of a measure, American Journal of Orthopsychiatry, 68, 216, 10.1037/h0080331 · DOI · Paper page
- Torrey, 2005, Promoting the implementation of practices that are supported by research: The National Implementing Evidence-Based Practice Project, Child and Adolescent Psychiatric Clinics of North America, 14, 297, 10.1016/j.chc.2004.05.004 · DOI
- Tsai, 2008, A comparison of electronic medical records to conventional paper records in community mental health centers, International Journal for Quality in Health Care, 20, 136, 10.1093/intqhc/mzm064 · DOI
- Turner, 1978, The NIMH community support program: Pilot approach to a needed social reform, Schizophrenia Bulletin, 4, 319, 10.1093/schbul/4.3.319 · DOI · Paper page
- Twamley, 2003, Vocational rehabilitation in schizophrenia and other psychotic disorders: A literature review and meta-analysis of randomized controlled trials, Journal of Nervous and Mental Disease, 191, 515, 10.1097/01.nmd.0000082213.42509.69 · DOI · Paper page
- U.S. Surgeon General, 2000, Mental health: A report of the Surgeon General-executive summary
- Waghorn, 2007, Challenges to implementing evidence-based supported employment in Australia, Journal of Vocational Rehabilitation, 27, 29, 10.3233/JVR-2007-00382 · DOI · Paper page
- Wilson, 2009, Psychometric properties of the Dual-Disorder Treatment Fidelity Scale: Inter-rater reliability and concurrent validity, Community Mental Health Journal, 45, 171, 10.1007/s10597-008-9167-2 · DOI
Cited by
BOND, G. R., DRAKE, R. E., & BECKER, D. R. (2012) · World Psychiatry · doi:10.1016/j.wpsyc.2012.01.005 · Cited by 359
Self-citationMaking the Case for IPS Supported Employment
Bond, G. R., & Drake, R. E. (2014) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-012-0444-6 · Cited by 189
Self-citationvon Thiele Schwarz, U., Aarons, G. A., & Hasson, H. (2019) · BMC Health Services Research · doi:10.1186/s12913-019-4668-y · Cited by 134
Ottomanelli, L., Goetz, L. L., Suris, A., McGeough, C., Sinnott, P. L., Toscano, R., Barnett, S. D., Cipher, D. J., Lind, L. M., Dixon, T. M., et al. (2012) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/j.apmr.2012.01.002 · Cited by 103
O'Neill, L., Latchford, G., McCracken, L. M., & Graham, C. D. (2019) · Journal of Contextual Behavioral Science · doi:10.1016/j.jcbs.2019.08.008 · Cited by 87
“For example, the most widely used measure of psychological flexibility, the Acceptance and Action Questionnaire II (Bond et al., 2011) includes items such as “I’m afraid of my feelings” and “Emotions cause problems in my life” that appear to lack a granular focus on the context or function of behaviours.”
Method
Metadata
- Authors
- Bond, G. R., Becker, D. R., & Drake, R. E.
- Venue
- Clinical Psychology: Science and Practice
- Published
- 2011-06
- Volume
- 18
- Issue
- 2
- Pages
- 126-141
- ISSN
- 0969-5893 (ISSN-L)
- 1468-2850
- OpenAlex
- W1891987747
- Cited by
- 127
- References
- 19
- FWCI
- 22.53
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- doi.wiley.com/10.1002/tdm_license_1 (tdm, 1553 d)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Measurement of fidelity of implementation of evidence‐based practices: Case example of the IPS Fidelity Scale. https://app.cochat.ai/papers/10.1111/j.1468-2850.2011.01244.x
- The paper
- Bond, G. R., Becker, D. R., & Drake, R. E. (2011). Measurement of fidelity of implementation of evidence‐based practices: Case example of the IPS Fidelity Scale. Clinical Psychology: Science and Practice. https://doi.org/10.1111/j.1468-2850.2011.01244.x
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.