Measures of fidelity of delivery of, and engagement with, complex, face‐to‐face health behaviour change interventions: A systematic review of measure quality
Open accessThe DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index.
Abstract
Purpose: Understanding the effectiveness of complex, face‐to‐face health behaviour change interventions requires high‐quality measures to assess fidelity of delivery and engagement. This systematic review aimed to (1) identify the types of measures used to monitor fidelity of delivery of, and engagement with, complex, face‐to‐face health behaviour change interventions and (2) describe the reporting of psychometric and implementation qualities. Methods: Electronic databases were searched, systematic reviews and reference lists were hand‐searched, and 21 experts were contacted to identify articles. Studies that quantitatively measured fidelity of delivery of, and/or engagement with, a complex, face‐to‐face health behaviour change intervention for adults were included. Data on interventions, measures, and psychometric and implementation qualities were extracted and synthesized using narrative analysis. Results: Sixty‐six studies were included: 24 measured both fidelity of delivery and engagement, 20 measured fidelity of delivery, and 22 measured engagement. Measures of fidelity of delivery included observation ( n = 17; 38.6%), self‐report ( n = 15; 34%), quantitatively rated qualitative interviews ( n = 1; 2.3%), or multiple measures ( n = 11; 25%). Measures of engagement included self‐report ( n = 18; 39.1%), intervention records ( n = 11; 24%), or multiple measures ( n = 17; 37%). Fifty‐one studies (77%) reported at least one psychometric or implementation quality; 49 studies (74.2%) reported at least one psychometric quality, and 17 studies (25.8%) reported at least one implementation quality. Conclusion: Fewer than half of the reviewed studies measured both fidelity of delivery of, and engagement with complex, face‐to‐face health behaviour change interventions. More studies reported psychometric qualities than implementation qualities. Interpretation of intervention outcomes from fidelity of delivery and engagement measurements may be limited due to a lack of reporting of psychometric and implementation qualities. Statement of contribution: What is already known on this subject? Evidence of fidelity and engagement is needed to understand effectiveness of complex interventions Evidence of fidelity and engagement are rarely reported High‐quality measures are needed to measure fidelity and engagement What does this study add? Evidence that indicators of quality of measures are reported in some studies Evidence that psychometric qualities are reported more frequently than implementation qualities A recommendation for intervention evaluations to report indicators of quality of fidelity and engagement measures
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Durlak, J. A., & DuPre, E. P. (2008) · American Journal of Community Psychology · doi:10.1007/s10464-008-9165-0 · Cited by 5,304
The Delphi Technique: Making Sense of Consensus
Hsu, C.-C., & Sandford, B. A. (2007) · University of Massachusetts Amherst · doi:10.7275/pdz9-th90 · Cited by 4,500 · cited 1×
“This systematic method could be used for building a consensus (Hsu & Sandford, 2007) regarding which psychometric and implementation qualities are most important, and which qualities should be given the most weighting when developing and evaluating fidelity of delivery and engagement measures.”
MethodHow we design feasibility studies
Bowen, D. J., Kreuter, M., Spring, B., Cofta-Woerpel, L., Linnan, L., Weiner, D., Bakken, S., Kaplan, C. P., Squiers, L., Fabrizio, C., et al. (2009) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2009.02.002 · Cited by 4,021
Framework for design and evaluation of complex interventions to improve health
Campbell, M. (2000) · BMJ · doi:10.1136/bmj.321.7262.694 · Cited by 3,474 · cited 5×
“5%)(31) Baranowski & Stables (2000): (n = 2; 3.3%)(42,50) Saunders et al. (2005) (n = 1; 1.”
InfluentialContent Analysis in Mass Communication: Assessment and Reporting of Intercoder Reliability
Lombard, M., Snyder-Duch, J., & Bracken, C. C. (2002) · Human Communication Research · doi:10.1111/j.1468-2958.2002.tb00826.x · Cited by 3,068
Bellg, A. J., Treatment Fidelity Workgroup of the NIH Behavior Change Consortium, Borrelli, B., Resnick, B., Hecht, J., Minicucci, D. S., Ory, M., Ogedegbe, G., Orwig, D., Ernst, D., et al. (2004) · Health Psychology · doi:10.1037/0278-6133.23.5.443 · Cited by 2,620
A conceptual framework for implementation fidelity
Carroll, C., Patterson, M., Wood, S., Booth, A., Rick, J., & Balain, S. (2007) · Implementation Science · doi:10.1186/1748-5908-2-40 · Cited by 2,559
Popay, J., Roberts, H., Sowden, A., Petticrew, M., Arai, L., Rodgers, M., Britten, N., Roen, K., & Duffy, S. (2006) · Lancaster University · doi:10.13140/2.1.1018.4643 · Cited by 2,070 · cited 1×
“Due to the heterogeneity of studies, a descriptive rather than quantitative synthesis of data was conducted (Deeks, Higgins, & Altman, 2008; Popay et al., 2006).”
Analysing Data and Undertaking Meta‐Analyses
Deeks, J. J., Higgins, J. P., & Altman, D. G. (2008) · Cochrane Handbook for Systematic Reviews of Interventions · doi:10.1002/9780470712184.ch9 · Cited by 1,800
Process evaluation in randomised controlled trials of complex interventions
Oakley, A., Strange, V., Bonell, C., Allen, E., & Stephenson, J. (2006) · BMJ · doi:10.1136/bmj.332.7538.413 · Cited by 1,247 · cited 1×
“…than implementation qualities A recommendation for intervention evaluations to report indicators of quality of fidelity and engagement measures Most interventions aimed at changing health behaviours are complex in that they contain multiple components (Campbell et al., 2000; Oakley et al., 2006).”
Background
Cited by
The Handbook of Behavior Change
(2020) · Cambridge University Press eBooks · doi:10.1017/9781108677318 · Cited by 407
Short, C. E., DeSmet, A., Woods, C., Williams, S. L., Maher, C., Middelweerd, A., Müller, A. M., Wark, P. A., Vandelanotte, C., Poppe, L., et al. (2018) · Journal of Medical Internet Research · doi:10.2196/jmir.9397 · Cited by 400
“Some researchers have defined engagement solely as a psychological process relating to user perceptions and experience, whereas others consider engagement a purely behavioral construct, synonymous with intervention usage [4,7].”
BackgroundChanging Behavior Using the Theory of Planned Behavior
Ajzen, I., & Schmidt, P. (2020) · The Handbook of Behavior Change · doi:10.1017/9781108677318.002 · Cited by 266
Changing Behavior Using the Model of Action Phases
Keller, L., Gollwitzer, P. M., & Sheeran, P. (2020) · The Handbook of Behavior Change · doi:10.1017/9781108677318.006 · Cited by 147
Saban, M., Dagan, E., & Drach-Zahavy, A. (2021) · Journal of Emergency Nursing · doi:10.1016/j.jen.2020.09.009 · Cited by 24
Galea Holmes, M. N., Wileman, V., Hassan, S., Denning, J., Critchley, D., Norton, S., McCracken, L. M., & Godfrey, E. (2022) · British Journal of Health Psychology · doi:10.1111/bjhp.12583 · Cited by 13
Kuo, C. C., Sibeko, G., Akande, M., Allie, S., Tisaker, N., Stein, D. J., & Becker, S. J. (2021) · Addiction Science & Clinical Practice · doi:10.1186/s13722-021-00236-8 · Cited by 8
“To date, implementation science efforts have yielded a plethora of provider fidelity scales [37, 38], but a dearth of trainer fidelity measures, which are essential to the task-sharing model.”
Background
Metadata
- Authors
- Walton, H., Spector, A., Tombor, I., & Michie, S.
- Venue
- British Journal of Health Psychology
- Publisher
- Wiley
- Published
- 2017-08
- Volume
- 22
- Issue
- 4
- Pages
- 872-903
- ISSN
- 1359-107X (ISSN-L)
- 2044-8287
- OpenAlex
- W2740999512
- Cited by
- 238
- References
- 33
- FWCI
- 5.53
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.