A method for assessing fidelity of delivery of telephone behavioral support for smoking cessation
Abstract
OBJECTIVES: Behavioral support for smoking cessation is delivered through different modalities, often guided by treatment manuals. Recently developed methods for assessing fidelity of delivery have shown that face-to-face behavioral support is often not delivered as specified in the service treatment manual. This study aimed to extend this method to evaluate fidelity of telephone-delivered behavioral support. METHOD: A treatment manual and transcripts of 75 audio-recorded behavioral support sessions were obtained from the United Kingdom's national Quitline service and coded into component behavior change techniques (BCTs) using a taxonomy of 45 smoking cessation BCTs. Interrater reliability was assessed using percentage agreement. Fidelity was assessed by comparing the number of BCTs identified in the manual with those delivered in telephone sessions by 4 counselors. Fidelity was assessed according to session type, duration, counselor, and BCT. Differences between self-reported and actual BCT use were examined. RESULTS: Average coding reliability was high (81%). On average, 41.8% of manual-specified BCTs were delivered per session (SD = 16.2), with fidelity varying by counselor from 32% to 49%. Fidelity was highest in pre-quit sessions (46%) and for BCT "give options for additional support" (95%). Fidelity was lowest for quit-day sessions (35%) and BCT "set graded tasks" (0%). Session duration was positively correlated with fidelity (r = .585; p < .01). Significantly fewer BCTs were used than were reported as being used, t(15) = -5.52, p < .001. CONCLUSIONS: The content of telephone-delivered behavioral support can be reliably coded in terms of BCTs. This can be used to assess fidelity to treatment manuals and to in turn identify training needs. The observed low fidelity underlines the need to establish routine procedures for monitoring delivery of behavioral support.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
BCTTv1 may be used to inform the development of measures for assessing delivery of BCTs.
“BCTTv1 may also be used to inform the development of measures for assessing delivery of BCTs.31”
Cites
Developing and evaluating complex interventions: the new Medical Research Council guidance
Craig, P., Dieppe, P., Macintyre, S., Michie, S., Nazareth, I., & Petticrew, M. (2008) · BMJ · doi:10.1136/bmj.a1655 · Cited by 10,995 · see citing works
Self-citationWeighted kappa: nominal scale agreement provision for scaled disagreement or partial credit
Cohen, J. (1968) · Psychological Bulletin · doi:10.1037/h0026256 · Cited by 8,736 · see citing works
Michie, S., Richardson, M., Johnston, M., Abraham, C., Francis, J., Hardeman, W., Eccles, M. P., Cane, J., & Wood, C. E. (2013) · Annals of Behavioral Medicine · doi:10.1007/s12160-013-9486-6 · Cited by 7,629 · see citing works
Self-citationWhy Don't Physicians Follow Clinical Practice Guidelines?: A Framework for Improvement
Cabana, M. D., Rand, C. S., Powe, N. R., Wu, A. W., Wilson, M. H., Abboud, P.-A. C., & Rubin, H. R. (1999) · JAMA · doi:10.1001/jama.282.15.1458 · Cited by 6,677 · see citing works
Boutron, I., Moher, D., Altman, D. G., Schulz, K. F., Ravaud, P., & for the CONSORT Group* (2008) · Annals of Internal Medicine · doi:10.7326/0003-4819-148-4-200802190-00008 · Cited by 2,292 · see citing works
Michie, S., Ashford, S., Sniehotta, F. F., Dombrowski, S. U., Bishop, A., & French, D. P. (2011) · Psychology & Health · doi:10.1080/08870446.2010.540664 · Cited by 1,756 · see citing works
Self-citationDusenbury, L. (2003) · Health Education Research · doi:10.1093/her/18.2.237 · Cited by 1,671 · see citing works
Dane, A. V., & Schneider, B. H. (1998) · Clinical Psychology Review · doi:10.1016/s0272-7358(97)00043-3 · Cited by 1,557 · cited 4× · see citing works
“There are few examples of established, routine, procedures for monitoring fidelity of behaviour change interventions as delivered in the context of actual clinical practice (Moncher & Prinz, 1991; Dane & Schneider, 1998; Dusenbury et al. 2003).”
InfluentialCollins, L. M., Murphy, S. A., & Strecher, V. (2007) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2007.01.022 · Cited by 1,082 · see citing works
Specifying and reporting complex behaviour change interventions: the need for a scientific method
Michie, S., Fixsen, D., Grimshaw, J. M., & Eccles, M. P. (2009) · Implementation Science · doi:10.1186/1748-5908-4-40 · Cited by 976 · see citing works
Self-citationThe assessment, monitoring, and enhancement of treatment fidelity in public health clinical trials
Borrelli, B. (2011) · Journal of Public Health Dentistry · doi:10.1111/j.1752-7325.2011.00233.x · Cited by 970 · see citing works
Smoking cessation guidelines for health professionals: an update
West, R. (2000) · Thorax · doi:10.1136/thorax.55.12.987 · Cited by 969 · see citing works
Self-citationBorrelli, B., Sepinwall, D., Ernst, D., Bellg, A. J., Czajkowski, S., Breger, R., DeFrancesco, C., Levesque, C., Sharp, D. L., Ogedegbe, G., et al. (2005) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.73.5.852 · Cited by 803 · see citing works
Treatment fidelity in outcome studies
Moncher, F. J., & Prinz, R. J. (1991) · Clinical Psychology Review · doi:10.1016/0272-7358(91)90103-2 · Cited by 797 · see citing works
Grant, A., Treweek, S., Dreischulte, T., Foy, R., & Guthrie, B. (2013) · Trials · doi:10.1186/1745-6215-14-15 · Cited by 541 · see citing works
An implementation research agenda
Eccles, M. P., Armstrong, D., Baker, R., Cleary, K., Davies, H., Davies, S., Glasziou, P., Ilott, I., Kinmonth, A.-L., Leng, G., et al. (2009) · Implementation Science · doi:10.1186/1748-5908-4-18 · Cited by 481 · see citing works
Self-citationEvidence of Real-World Effectiveness of a Telephone Quitline for Smokers
Zhu, S., Anderson, C. M., Tedeschi, G. J., Rosbrook, B. P., Johnson, C., Byrd, M., & Gutiérrez-Terrell, E. (2002) · New England Journal of Medicine · doi:10.1056/nejmsa020660 · Cited by 462 · see citing works
Michie, S., Hyder, N., Walia, A., & West, R. (2011) · Addictive Behaviors · doi:10.1016/j.addbeh.2010.11.016 · Cited by 459 · see citing works
Self-citationZhu, S.-H., Stretch, V., Balabanis, M., Rosbrook, B., Sadler, G., & Pierce, J. P. (1996) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.64.1.202 · Cited by 302 · see citing works
Identification of behaviour change techniques to reduce excessive alcohol consumption
Michie, S., Whittington, C., Hamoudi, Z., Zarnani, F., Tober, G., & West, R. (2012) · Addiction · doi:10.1111/j.1360-0443.2012.03845.x · Cited by 288 · see citing works
Self-citationGroup behaviour therapy programmes for smoking cessation.
Stead, L., Carroll, A. J., & Lancaster, T. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd001007.pub3 · Cited by 281 · see citing works
Telephone counseling for smoking cessation: rationales and meta-analytic review of evidence
Lichtenstein, E., Glasgow, R. E., Lando, H. A., Ossip-Klein, D. J., & Boles, S. M. (1996) · Health Education Research · doi:10.1093/her/11.2.243 · Cited by 279 · see citing works
Online support for smoking cessation: a systematic review of the literature
Shahab, L., & McEwen, A. (2009) · Addiction · doi:10.1111/j.1360-0443.2009.02710.x · Cited by 275 · see citing works
Self-reported physician practices for children with asthma: are national guidelines followed?
Finkelstein, J., Lozano, P., Shulruff, R., Inui, T., Soumerai, S., Ng, M.-T., & Weiss, K. (2000) · Pediatrics · doi:10.1542/peds.106.s3.886 · Cited by 258 · see citing works
Curry, S. J., McBride, C., Grothaus, L. C., Louie, D., & Wagner, E. H. (1995) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.63.6.1005 · Cited by 217 · see citing works
Curry, S. J., McBride, C. M., Grothaus, L. C., Louie, D., & Wagner, E. H. (1995) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.63.6.1005 · Cited by 214 · see citing works
Telephone Counseling for Smoking Cessation: What's in a Call?
Zhu, S., Tedeschi, G. J., Anderson, C. M., & Pierce, J. P. (1996) · Journal of Counseling & Development · doi:10.1002/j.1556-6676.1996.tb02319.x · Cited by 213 · see citing works
Fidelity of delivery of a physical activity intervention: Predictors and consequences
Hardeman, W., Michie, S., Fanshawe, T., Prevost, A. T., Mcloughlin, K., & Kinmonth, A. L. (2008) · Psychology & Health · doi:10.1080/08870440701615948 · Cited by 195 · see citing works
Self-citationWest, R., Walia, A., Hyder, N., Shahab, L., & Michie, S. (2010) · Nicotine & Tobacco Research · doi:10.1093/ntr/ntq074 · Cited by 165 · see citing works
Self-citationImproving Health: Changing Behaviour. NHS Health Trainer Handbook
Michie, S., Rumsey, N., Fussell, A., Hardeman, W., Johnston, M., Newman, S. P., & Yardley, L. (2005) · PsycEXTRA Dataset · doi:10.1037/e442892008-001 · Cited by 133 · see citing works
Self-citationIdentifying Evidence-Based Competences Required to Deliver Behavioural Support for Smoking Cessation
Michie, S., Churchill, S., & West, R. (2011) · Annals of Behavioral Medicine · doi:10.1007/s12160-010-9235-z · Cited by 126 · see citing works
Self-citationThe English smoking treatment services: short-term outcomes
Judge, K., Bauld, L., Chesterman, J., & Ferguson, J. (2005) · Addiction · doi:10.1111/j.1360-0443.2005.01027.x · Cited by 126 · see citing works
Assessing fidelity of delivery of smoking cessation behavioural support in practice
Lorencatto, F., West, R., Christopherson, C., & Michie, S. (2013) · Implementation Science · doi:10.1186/1748-5908-8-40 · Cited by 107 · see citing works
Self-citationIncreasing the Demand for and Use of Effective Smoking-Cessation Treatments
Orleans, C. T. (2007) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2007.09.003 · Cited by 99 · see citing works
WHAT WORKS AND HOW? DESIGNING MORE EFFECTIVE INTERVENTIONS NEEDS ANSWERS TO BOTH QUESTIONS
Michie, S. (2008) · Addiction · doi:10.1111/j.1360-0443.2007.02112.x · Cited by 98 · see citing works
Self-citationSpecifying Evidence-Based Behavior Change Techniques to Aid Smoking Cessation in Pregnancy
Lorencatto, F., West, R., & Michie, S. (2012) · Nicotine & Tobacco Research · doi:10.1093/ntr/ntr324 · Cited by 85 · see citing works
Self-citationBenefits of Telephone Care Over Primary Care for Smoking Cessation
An, L. C., Zhu, S., Nelson, D. B., Arikian, N. J., Nugent, S. M., Partin, M. R., & Joseph, A. M. (2006) · Archives of Internal Medicine · doi:10.1001/archinte.166.5.536 · Cited by 76 · see citing works
Thyrian, J. R., Panagiotakos, D. B., Polychronopoulos, E., West, R., Zatoński, W. A., & John, U. (2008) · BMC Public Health · doi:10.1186/1471-2458-8-2 · Cited by 67 · see citing works
Self-citationClinical and public health significance of treatments to aid smoking cessation
West, R., & Stapleton, J. (2008) · European Respiratory Review · doi:10.1183/09059180.00011005 · Cited by 60 · see citing works
Self-citationLeventhal, H., & Friedman, M. A. (2004) · Health Psychology · doi:10.1037/0278-6133.23.5.452 · Cited by 57 · see citing works
Lorencatto, F., West, R., Seymour, N., & Michie, S. (2013) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0032106 · Cited by 45 · see citing works
Self-citationBrose, L. S., McEwen, A., & West, R. (2012) · Addiction · doi:10.1111/j.1360-0443.2012.03935.x · Cited by 37 · see citing works
Self-citationTober, G., Clyne, W., Finnegan, O., Farrin, A. J., & Ian Russell Team (2008) · Alcohol and Alcoholism · doi:10.1093/alcalc/agn064 · Cited by 34 · see citing works
Dewing, S., Mathews, C., Cloete, A., Schaay, N., Shah, M., Simbayi, L. C., & Louw, J. (2013) · AIDS and Behavior · doi:10.1007/s10461-013-0509-x · Cited by 32 · see citing works
Steinijans, V. W., Diletti, E., Bömches, B., Greis, C., & Solleder, P. (1997) · PubMed · Cited by 31 · see citing works
Asthma Care Practices, Perceptions, and Beliefs of Chicago-Area Asthma Specialists
Moy, J. N., Grant, E. N., Turner-Roan, K., Li, T., & Weiss, K. B. (1999) · CHEST Journal · doi:10.1378/chest.116.suppl_2.154s · Cited by 26 · see citing works
Local Stop Smoking Services: Service Delivery and Monitoring Guidance 2011/2012
Croghan, E. (2011) · PsycEXTRA Dataset · doi:10.1037/e611032011-001 · Cited by 26 · see citing works
Effectiveness of an Online Knowledge Training and Assessment Program for Stop Smoking Practitioners
Brose, L. S., West, R., Michie, S., Kenyon, J. A. M., & McEwen, A. (2012) · Nicotine & Tobacco Research · doi:10.1093/ntr/ntr286 · Cited by 24 · see citing works
Self-citationCollins, S. E., Eck, S., Kick, E., Schröter, M., Torchalla, I., & Batra, A. (2008) · Addictive Behaviors · doi:10.1016/j.addbeh.2008.12.008 · Cited by 14 · see citing works
McDermott, M. S., Thomson, H., West, R., Kenyon, J. A., & McEwen, A. (2012) · BMC Health Services Research · doi:10.1186/1472-6963-12-121 · Cited by 7 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Michie, S., Wood, C. E., Johnston, M., Abraham, C., Francis, J. J., & Hardeman, W. (2015) · Health Technology Assessment · doi:10.3310/hta19990 · Cited by 680 · see citing works
Self-citationMichie, S., West, R., Sheals, K., & Godinho, C. A. (2018) · Translational Behavioral Medicine · doi:10.1093/tbm/ibx019 · Cited by 377 · see citing works
Self-citationJohnson, B. T., Michie, S., & Snyder, L. B. (2014) · JAIDS Journal of Acquired Immune Deficiency Syndromes · doi:10.1097/qai.0000000000000235 · Cited by 40 · see citing works
Self-citation
- Authors
- Lorencatto, F., West, R., Bruguera, C., & Michie, S.
- Affiliations
- City, University of London (GB)
- University College London (GB)
- Venue
- Journal of Consulting and Clinical Psychology
- Published
- 2014-06
- Volume
- 82
- Issue
- 3
- Pages
- 482-491
- ISSN
- 0022-006X (ISSN-L)
- 1939-2117
- OpenAlex
- W2011414918
- Cited by
- 52
- References
- 51
- Citations by year
- 2025: 2 · 2024: 1 · 2023: 10 · 2022: 1 · 2021: 8 · 2020: 4 · 2019: 10 · 2018: 3 · 2017: 4 · 2016: 3 · 2015: 5 · 2014: 1
- FWCI
- 2.08
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Consulting and Clinical Psychology (publishedVersion)
- PubMed (publishedVersion)
- City Research Online (City University London) (acceptedVersion)
- UCL Discovery (University College London) (submittedVersion)
- Funders
- Department of Health (DOH T336/BSS/M)
- Topics
- Health Sciences › Medicine › Physiology › Smoking Behavior and Cessation
- Social Sciences › Psychology › Applied Psychology › Behavioral Health and Interventions
- Health Sciences › Medicine › Epidemiology › Substance Abuse Treatment and Outcomes
- Keywords
- smoking cessation, intervention fidelity, behaviour change techniques, quitlines, treatment manual, counselor training needs, session duration
- MeSH terms
- Behavior Therapy, United Kingdom, Humans, Tape Recording, Telephone, Reproducibility of Results, Smoking Cessation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review