Development of a tool to assess fidelity to a psycho-educational intervention
Open accessAbstract
AIM: This paper is a description of a method to develop and conduct a customized psycho-educational intervention fidelity assessment as part of pilot work for an efficacy study. A tool designed to assess treatment fidelity to a psycho-education intervention for patients with end-stage renal disease and their surrogate decision makers, Sharing the Patient's Illness Representations to Increase Trust, is presented as an illustration. BACKGROUND: Despite the specificity and idiosyncrasy of individual interventions and the call to systematically evaluate treatment fidelity, how to accomplish this goal has not been clarified. Tools to adequately measure treatment fidelity are lacking. METHODS: We developed the Sharing the Patient's Illness Representations to Increase Trust Treatment Fidelity Assessment tool by identifying elements that were idiosyncratic to the intervention and those that could be adapted from existing tools. The tool has four components: overall adherence to the intervention content elements; pacing of the intervention delivery; overall dyad responsiveness; and, overall quality index of intervention delivery. The study was undertaken between 2006 and 2008. RESULTS: Inter-rater reliability ranged from 0.80 to 0.87 for the four components. The tool showed utility in training and monitoring, such as detecting unplanned content elements delivered and the use of proscribed communication behaviours. CONCLUSION: Psycho-educational interventions are one of the most common types of nursing interventions worldwide. Use of fidelity assessment tools customized to the individual interventions may enhance systematic evaluation of training and monitoring treatment fidelity.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Fidelity assessments are important in psychological intervention trials to assess whether an intervention is delivered as conceived and intended.
“Fidelity assessments are important in psychological intervention trials to assess whether an intervention is delivered as conceived and intended (Song 2010).”
From Psychological interventions for improving adherence to inhaled therapies in people with cystic fibrosis (2023) · cites this paper as Song 2010
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The Measurement of Observer Agreement for Categorical Data
Landis, J. R., & Koch, G. G. (1977) · Biometrics · doi:10.2307/2529310 · Cited by 81,148 · see citing works
Health Measurement Scales: A Practical Guide to Their Development and Use
Streiner, D. L., Norman, G. R., & Cairney, J. (1989) · Cited by 9,595 · see citing works
Motivational interviewing: preparing people for change.
Miller, W. R., & Rollnick, S. (2002) · Cited by 7,002 · see citing works
Bellg, A. J., Treatment Fidelity Workgroup of the NIH Behavior Change Consortium, Borrelli, B., Resnick, B., Hecht, J., Minicucci, D. S., Ory, M., Ogedegbe, G., Orwig, D., Ernst, D., et al. (2004) · Health Psychology · doi:10.1037/0278-6133.23.5.443 · Cited by 2,620 · see citing works
Health Measurement Scales: A Practical Guide To Their Development And Use
Streiner, D. L., & Norman, G. R. (2003) · doi:10.1093/oso/9780198528470.001.0001 · Cited by 1,460 · see citing works
Complex interventions: how “out of control” can a randomised controlled trial be?
Hawe, P., Shiell, A., & Riley, T. (2004) · BMJ · doi:10.1136/bmj.328.7455.1561 · Cited by 1,244 · see citing works
ROTER, D. L., & Larson, S. (2002) · Patient Education and Counseling · doi:10.1016/s0738-3991(02)00012-5 · Cited by 913 · see citing works
Testing the integrity of a psychotherapy protocol: Assessment of adherence and competence
Waltz, J., Addis, M. E., Koerner, K., & Jacobson, N. S. (1993) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.61.4.620 · Cited by 873 · see citing works
Treatment fidelity in outcome studies
Moncher, F. J., & Prinz, R. J. (1991) · Clinical Psychology Review · doi:10.1016/0272-7358(91)90103-2 · Cited by 797 · see citing works
Treatment integrity and therapeutic change: issues and research recommendations
Perepletchikova, F., & Kazdin, A. E. (2005) · Clinical Psychology: Science and Practice · doi:10.1093/clipsy.bpi045 · Cited by 638 · see citing works
Carroll, K. M. (2000) · Drug and Alcohol Dependence · doi:10.1016/s0376-8716(99)00049-6 · Cited by 386 · see citing works
Forgatch, M. S., Patterson, G. R., & DeGarmo, D. S. (2005) · Behavior Therapy · doi:10.1016/s0005-7894(05)80049-8 · Cited by 320 · see citing works
Promoting intervention fidelity
Dumas, J. E., Lynch, A. M., Laughlin, J. E., Smith, E. P., & Prinz, R. J. (2001) · American Journal of Preventive Medicine · doi:10.1016/s0749-3797(00)00272-5 · Cited by 277 · see citing works
Patient‐centered interventions
Lauver, D., Ward, S. E., Heidrich, S. M., Keller, M. L., Bowers, B. J., Brennan, P. F., Kirchhoff, K. T., & Wells, T. J. (2002) · Research in Nursing & Health · doi:10.1002/nur.10044 · Cited by 241 · see citing works
Measures of fidelity in motivational enhancement: A systematic review
Madson, M. B., & Campbell, T. C. (2006) · Journal of Substance Abuse Treatment · doi:10.1016/j.jsat.2006.03.010 · Cited by 227 · see citing works
Resnick, B. M., Bellg, A. J., Borrelli, B., Francesco, C. D., Breger, R. K. R., Hecht, J., Sharp, D. L., Levesque, C. S., Orwig, D., Ernst, D., et al. (2005) · Annals of Behavioral Medicine · doi:10.1207/s15324796abm2902s_8 · Cited by 215 · see citing works
Santacroce, S. J., Maccarelli, L. M., & Grey, M. J. (2004) · Nursing Research · doi:10.1097/00006199-200401000-00010 · Cited by 178 · see citing works
Manual for the Motivational Interviewing Skill Code (MISC)
Amrhein, P. C., Miller, W. R., Moyers, T. B., & Ernst, D. (2008) · Cited by 176 · see citing works
Song, M., Kirchhoff, K. T., Douglas, J. A., Ward, S. E., & Hammes, B. J. (2005) · Medical Care · doi:10.1097/01.mlr.0000178192.10283.b4 · Cited by 154 · see citing works
Self-citationTreatment Fidelity in Behavior Change Research: A Case Example
Resnick, B., Inguito, P., Orwig, D., Yahiro, J. Y., Hawkes, W., Werner, M., Zimmerman, S., & Magaziner, J. (2005) · Nursing Research · doi:10.1097/00006199-200503000-00010 · Cited by 136 · see citing works
Experts Practice What They Preach
ROTER, D. L., Larson, S., Fischer, G. S., Arnold, R. M., & Tulsky, J. A. (2000) · Archives of Internal Medicine · doi:10.1001/archinte.160.22.3477 · Cited by 127 · see citing works
An efficacious theory-based intervention for stepfamilies
Forgatch, M. S., DeGarmo, D. S., & Beldavs, Z. G. (2005) · Behavior Therapy · doi:10.1016/s0005-7894(05)80117-0 · Cited by 118 · see citing works
Beckstead, J. W. (2009) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2009.04.014 · Cited by 118 · see citing works
A Representational Approach to Patient Education
Donovan, H. S., & Ward, S. E. (2001) · Journal of Nursing Scholarship · doi:10.1111/j.1547-5069.2001.00211.x · Cited by 114 · see citing works
Song, M., Ward, S. E., Happ, M. B., Piraino, B., Donovan, H. S., Shields, A. M., & Connolly, M. C. (2009) · Research in Nursing & Health · doi:10.1002/nur.20320 · Cited by 104 · see citing works
Self-citationStein, K. F., Sargent, J. T., & Rafaels, N. M. (2006) · Nursing Research · doi:10.1097/00006199-200701000-00007 · Cited by 90 · see citing works
An update on the representational approach to patient education.
Donovan, H. S., Ward, S. E., Song, M., Heidrich, S. M., Gunnarsdottir, S., & Phillips, C. M. (2007) · Journal of Nursing Scholarship · doi:10.1111/j.1547-5069.2007.00178.x · Cited by 90 · see citing works
Self-citationSong, M., Donovan, H. S., Piraino, B., Choi, J., Bernardini, J. Y., Verosky, D., & Ward, S. E. (2009) · Applied Nursing Research · doi:10.1016/j.apnr.2008.05.002 · Cited by 85 · see citing works
Self-citationTraining, supervision and quality monitoring of the COMBINE Study behavioral interventions.
Miller, W. R., Moyers, T. B., Arciniega, L. T., Ernst, D., & Forcehimes, A. A. (2005) · Journal of Studies on Alcohol Supplement · doi:10.15288/jsas.2005.s15.188 · Cited by 82 · see citing works
Harshbarger, C., Simmons, G., Coelho, H., Sloop, K., & Collins, C. (2006) · AIDS Education and Prevention · doi:10.1521/aeap.2006.18.supp.184 · Cited by 72 · see citing works
Leeuw, M., Goossens, M. E. J. B., de Vet, H. C. W., & Vlaeyen, J. W. S. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2008.03.008 · Cited by 65 · see citing works
Leventhal, H., & Friedman, M. A. (2004) · Health Psychology · doi:10.1037/0278-6133.23.5.452 · Cited by 57 · see citing works
Assessing program fidelity in substance abuse health services research
Orwin, R. G. (2000) · Addiction · doi:10.1080/09652140020004250 · Cited by 53 · see citing works
Chan, E. J., O'Neill, I., McKenzie, M., Love, A. W., & Kissane, D. W. (2004) · Journal of Pain and Symptom Management · doi:10.1016/j.jpainsymman.2003.10.008 · Cited by 53 · see citing works
Assessing program fidelity in substance abuse health services research
Orwin, R. G. (2000) · Addiction · doi:10.1046/j.1360-0443.95.11s3.3.x · Cited by 46 · see citing works
A PRACTICAL INSTRUMENT TO DOCUMENT THE PROCESS OF MOTIVATIONAL INTERVIEWING
Tappin, D. M., McKay, C., McIntyre, D., Gilmour, W. H., Cowan, S., Crawford, F., Currie, F., & Lumsden, M. A. (2000) · Behavioural and Cognitive Psychotherapy · doi:10.1017/s1352465800000035 · Cited by 30 · see citing works
Hawe, P., Shiell, A., & Riley, T. (2008) · Journal of Advanced Nursing · doi:10.1111/j.1365-2648.2008.04686.x · Cited by 20 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Making Visible a Theory-Guided Advance Care Planning Intervention.
Song, M., & Ward, S. E. (2015) · Journal of Nursing Scholarship · doi:10.1111/jnu.12156 · Cited by 22 · see citing works
Self-citationDawson, S., Girling, C.-J., Cowap, L., & Clark-Carter, D. (2023) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013766.pub2 · Cited by 14 · see citing works
- Authors
- Song, M., Happ, M. B., & Sandelowski, M.
- Affiliations
- University of North Carolina at Chapel Hill (US)
- University of Pittsburgh (US)
- Venue
- Journal of Advanced Nursing
- Publisher
- Wiley
- Published
- 2010-02-09
- Volume
- 66
- Issue
- 3
- Pages
- 673-682
- ISSN
- 0309-2402 (ISSN-L)
- 1365-2648
- PMC
- PMC2861796
- OpenAlex
- W2061106145
- Cited by
- 70
- References
- 37
- Citations by year
- 2026: 4 · 2025: 7 · 2024: 4 · 2023: 9 · 2022: 2 · 2021: 10 · 2019: 7 · 2018: 4 · 2017: 2 · 2016: 5 · 2015: 4 · 2014: 5 · 2013: 2 · 2012: 2
- FWCI
- 10.52
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Journal of Advanced Nursing (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Funders
- National Institute of Nursing Research (R21NR009662, K24NR010244, R01NR011464)
- Topics
- Social Sciences › Education › Health Education and Validation
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Health Sciences › Health Professions › General Health Professions › Health Sciences Research and Education
- Keywords
- intervention fidelity, psychoeducational intervention, end-stage renal disease, surrogate decision makers, illness representations, trust, inter-rater reliability, nursing interventions
- MeSH terms
- Decision Making, Delivery of Health Care, Humans, Kidney Failure, Chronic, Patient Compliance, Patient Education as Topic, Psychotherapy, Terminal Care
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
