Reliability of Identification of Behavior Change Techniques in Intervention Descriptions
Open accessAbstract
PURPOSE: The aim of this paper is to assess the frequency of identification as well as the inter-coder and test-retest reliability of identification of behavior change techniques (BCTs) in written intervention descriptions. METHODS: Forty trained coders applied the "Behavior Change Technique Taxonomy version 1" (BCTTv1) to 40 intervention descriptions published in protocols and repeated this 1 month later. RESULTS: Eighty of 93 defined BCTs were identified by at least one trained coder, and 22 BCTs were identified in 16 (40 %) or more of 40 descriptions. Good inter-coder reliability was observed across 80 BCTs identified in the protocols: 66 (80 %) achieved mean prevalence and bias-adjusted kappa (PABAK) scores of 0.70 or greater, and 59 (74 %) achieved mean scores of 0.80 or greater. There was good within-coder agreement between baseline and 1 month, demonstrating good test-retest reliability. CONCLUSIONS: BCTTv1 can be used by trained coders to identify BCTs in intervention descriptions reliably. However, some frequently occurring BCT definitions require further clarification.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
BCTTv1 is an extensive, integrated, hierarchical classification system for reliably specifying intervention components in terms of 93 BCTs, organised into 16 groupings.
“This is an extensive, integrated, hierarchical classification system for reliably specifying intervention components in terms of 93 BCTs, organised into 16 groupings [15, 16, 22–24].”
We used consensus about presence of BCTs as a criterion against which trained coder codings were judged and concurrent validity was assessed.
“We used this consensus about the presence of BCTs in the descriptions as a criterion against which trained coder codings were judged and concurrent validity was assessed.107”
Trained coders had all demonstrated competence in using BCTTv1 to reliably and validly specify the content of BCIs, achieving criteria of satisfactory performance on coding intervention descriptions following training in a formal assessment of coding competence.
“Coders had all demonstrated competence in using BCTTv1 to reliably and validly specify the content of BCIs, that is they achieved criteria of satisfactory performance on coding intervention descriptions following training, in a formal assessment of coding competence.107”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Michie, S., van Stralen, M. M., & West, R. (2011) · Implementation Science · doi:10.1186/1748-5908-6-42 · Cited by 13,697 · see citing works
Self-citationHoffmann, T. C., Glasziou, P. P., Boutron, I., Milne, R., Perera, R., Moher, D., Altman, D. G., Barbour, V., Macdonald, H., Johnston, M., et al. (2014) · BMJ · doi:10.1136/bmj.g1687 · Cited by 10,548 · see citing works
Self-citationMichie, S., Richardson, M., Johnston, M., Abraham, C., Francis, J., Hardeman, W., Eccles, M. P., Cane, J., & Wood, C. E. (2013) · Annals of Behavioral Medicine · doi:10.1007/s12160-013-9486-6 · Cited by 7,629 · see citing works
Self-citationAudit and feedback: effects on professional practice and healthcare outcomes
Ivers, N., Jamtvedt, G., Flottorp, S., Young, J. M., Odgaard-Jensen, J., French, S. D., O'Brien, M. A., Johansen, M., Grimshaw, J., & Oxman, A. D. (2012) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd000259.pub3 · Cited by 5,485 · see citing works
Moher, D., Schulz, K. F., & Altman, D. G. (2001) · BMC Medical Research Methodology · doi:10.1186/1471-2288-1-2 · Cited by 2,755 · see citing works
A taxonomy of behavior change techniques used in interventions
Abraham, C., & Michie, S. (2008) · Health Psychology · doi:10.1037/0278-6133.27.3.379 · Cited by 2,525 · see citing works
Self-citationBoutron, I., Moher, D., Altman, D. G., Schulz, K. F., Ravaud, P., & for the CONSORT Group* (2008) · Annals of Internal Medicine · doi:10.7326/0003-4819-148-4-200802190-00008 · Cited by 2,292 · see citing works
Effective techniques in healthy eating and physical activity interventions: A meta-regression.
Michie, S., Abraham, C., Whittington, C., McAteer, J., & Gupta, S. (2009) · Health Psychology · doi:10.1037/a0016136 · Cited by 2,229 · see citing works
Self-citationByrt, T., Bishop, J., & Carlin, J. B. (1993) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(93)90018-v · Cited by 2,179 · see citing works
Michie, S., Ashford, S., Sniehotta, F. F., Dombrowski, S. U., Bishop, A., & French, D. P. (2011) · Psychology & Health · doi:10.1080/08870446.2010.540664 · Cited by 1,756 · see citing works
Self-citationAlbarracín, D., Gillette, J. C., Earl, A. N., Glasman, L. R., Durantini, M. R., & Ho, M.-H. (2005) · Psychological Bulletin · doi:10.1037/0033-2909.131.6.856 · Cited by 792 · see citing works
Michie, S., Hyder, N., Walia, A., & West, R. (2011) · Addictive Behaviors · doi:10.1016/j.addbeh.2010.11.016 · Cited by 459 · see citing works
Self-citationIdentification of behaviour change techniques to reduce excessive alcohol consumption
Michie, S., Whittington, C., Hamoudi, Z., Zarnani, F., Tober, G., & West, R. (2012) · Addiction · doi:10.1111/j.1360-0443.2012.03845.x · Cited by 288 · see citing works
Self-citationAlbrecht, L., Archibald, M., Arseneau, D., & Scott, S. D. (2013) · Implementation Science · doi:10.1186/1748-5908-8-52 · Cited by 252 · see citing works
Applying the behaviour change technique (BCT) taxonomy v1: A study of coder training
Wood, C. E., Richardson, M., Johnston, M., Abraham, C., Francis, J., Hardeman, W., & Michie, S. (2015) · Translational Behavioral Medicine · doi:10.1007/s13142-014-0290-z · Cited by 100 · see citing works
Self-citationReporting of interventions in randomised trials: an audit of journal Instructions to Authors
Hoffmann, T., English, T., & Glasziou, P. (2014) · Trials · doi:10.1186/1745-6215-15-20 · Cited by 53 · see citing works
Flagged first, then by citations
Research works printed but not linked: 33
- Whitlock G, Lewington S, Sherliker P, et al.: Body-mass index and cause-specific mortality in 900 000 adults: collaborative analyses of 57 prospective studies. Lancet. 2009, 373:1083- 1096.
- Wing R, Jeffery R: Benefits of recruiting participants with friends and increasing social support for weight loss and maintenance. J Consult Clin Psychol. 1999, 67:132-138.
- Fisher J, Fisher W: Changing AIDS-risk behavior. Psychol Bull. 1992, 111:455-474.
- Denford, S., Abraham, C., Smith, J., Lloyd, J. J., White, M., Tarrant, M., Wyatt, K., Greaves, C., & Dean, S. (2015). Designing and evaluating behavior change interventions to promote health. In K.J. Reynolds & N.R. Branscombe (Eds.), The Psychology of Change: Life Contexts, Experiences, and Identities. New York: Psychology Press.
- Davidson KW, Goldstein M, Kaplan RM, et al.: Evidence-based behavioral medicine: what is it and how do we achieve it? Ann Behav Med. 2003, 26:161-171.
- Abraham C, Johnson BT, de Bruin, M. & Luszczynska A. Enhancing reporting of behavior change intervention evaluations. JAIDS. 2014 -in press.
- Glasgow R, Bull S, Gillette C, Klesges L, Dzewaltowski D: Behavior change intervention research in healthcare settings: a review of recent reports with emphasis on external validity. Am J Prev Med. 2002, 23:62-69.
- Bartholomew L, Parcel G, Kok G, Gottlieb N, Fernandez M (eds): Planning Health Promotion Programs: An Intervention Mapping Approach (3rd Ed.). San Francisco, CA: Jossey- Bass, 2011.
- Michie S. : Behaviour Change Techniques. In M. T. Gellman, RJ. (ed), Encyclopeadia of Behavioural Medicine. London: Springer, 2013, 182-187.
- Carver CS, Scheier MF: Control theory: A useful conceptual framework for personality- social, clinical, and health psychology. Psychological bulletin. 1982, 92:111.
- Michie S, Churchill S, West R: Identifying evidence-based competences required to deliver behavioural support for smoking cessation. Ann Behav Med. 2011, 41:59-70.
- Abraham C, Good A, Huedo-Medina TB, Warren MR, Johnson BT: Reliability and Utility of the SHARP Taxonomy of Behaviour Change Techniques. Psychology & Health. 2012, 27:1-2.
- Schulz R, Czaja SJ, McKay JR, Ory MG, Belle SH. Intervention Taxonomy (ITAX): Describing Essential Features of Interventions (HMC). American Journal of Health Behavior 2010, 34:811- 821.
- Michie S, Abraham C, Eccles MP, et al.: Strengthening evaluation and implementation by specifying components of behaviour change interventions: a study protocol. Implement Sci. 2011, 6:10.
- Cohen J: Weighted kappa: Nominal scale agreement provision for scaled disagreement or partial credit. Psychological bulletin. 1968, 70:213.
- Gwet KL: Handbook of Inter-Rater Reliability: The Definitive Guide to Measuring the Extent of Agreement Among Multiple Raters: Advanced Analytics Press, 2012.
- van Beurden, S. B., Greaves, C. J., Smith, J. R. & Abraham , C. (under review). Techniques for managing impulsive processes associated with unhealthy eating: A systematic review. Health Psychology.
- Durantini MR, Albarracin D, Mitchell AL, Earl AN, Gillette JC: Conceptualizing the Influence of Social Agents of Behavior Change: A Meta-Analysis of the Effectiveness of HIV- Prevention Interventionists for Different Groups. Psychol Bull. 2006, 132:212-248. 2010
- BMC Public HS McNamara et al. (2010). Development and Evaluation of a Psychosocial Intervention for Children and Teenagers Experiencing Diabetes (DEPICTED): a protocol for a cluster randomised controlled trial of the effectiveness of a communication skills training programme for healthcare professionals working with young people with type 1 diabetes. BMC Health Services Research, 10, 36. doi:10.1186/1472-6963-10-36 2010 · DOI
- BMC Public H Colagiuri et al. (2010). The Sydney Diabetes Prevention Program: A community-based translational study. BMC Public Health, 10, 328. doi:10.1186/1471-2458-10-328 2010 Imp. Science McAlister et al. (2010). The preventing recurrent vascular events and neurological worsening through intensive organized case- management (PREVENTION) trial protocol (clinicaltrials.gov identifier: NCT00931788). Implementation Science, 5, 27. doi:10.1186/1748-5908-5-27 2010 · DOI
- BMC Public HS Lau et al. (2010). Evaluation of a community pharmacy-based intervention for improving patient adherence to anti-hypertensives: a randomised controlled trial. BMC Health Services Research, 10, 34.doi:10.1186/1472-6963-10-34 2010 Imp. Science · DOI
- Rost, K., Marshall, D. (2010). Marketing depression care management to employers: design of a randomised controlled trial. Implementation Science, 5, 22. doi:10.1186/1748-5908-5-22 2010 Imp. Science Garner et al. (2010).The Reinforcing Therapist Performance (RTP) experiment: Study protocol for a cluster randomised trial. Implementation Science, doi:10.1186/1748-5908-5-5 2010 · DOI
- BMC Public H Johnston et al. (2010). The study protocol for a randomised controlled trial of a family-centred tobacco control program about environmental tobacco smoke (ETS) to reduce respiratory illness in Indigenous infants. BMC Public Health, 10, 114. doi:10.1186/1471-2458-10-114 2010 · DOI
- BMC Public H Taylor et al. (2010). Motivational interviewing for screening and feedback and encouraging lifestyle changes to reduce relative weight in 4-8 year old children: design of the MInT study.BMC Public Health, 10, 271. doi:10.1186/1471-2458-10-271 2010 · DOI
- BMC Public H Geimer et al. (2010) Use of a liquid nicotine product to promote smoking cessation, BMC Public Health, doi:10.1186/1471-2458-10- 155 2010 · DOI
- BMC Public H Gonseth et al. (2010). Smoking cessation, BMC Public Health, doi:10.1186/1471-2458-10-348 2010 · DOI
- BMC Public H Mann et al. (2010). Diabetes screening attendance, BMC Public Health, doi:10.1186/1471-2458-10-768 2010 · DOI
- BMC Public H Menza et al. (2010). Reduction in use of methamphetamine, doi:10.1186/1471-2458-10-774 BMC Public Health 2010 BMC Public H Rosenkranz et al. (2010). Health promotion and obesity prevention including physical activity (PA) /inactivity and healthy eating (encompassing a range of behaviours) among Junior Girl Scout, BMC Public Health, doi:10.1186/1471-2458-10-81 2010 · DOI
- BMC Public H Werkman et al. (2010). Physical activity and dietary intake (encompassing a range of behaviours), doi:10.1186/1471-2458-10-110 BMC Public Health 2009 BMC Public H Buis et al.(2009). Physical activity, doi:10.1186/1471-2458-9-331 BMC Public Health 2010 BMC Public H Bull et al. (2010). Physical activity (encompassing a range of behaviours), doi:10.1186/1471-2458-10-463 BMC Public Health 2010 BMC Public H Claesson et al. (2010). Eating/food intake and physical activity to manage weight gain among pregnant obese women. doi:10.1186/1471-2458-10-766 BMC Public Health, 2010 BMC Public H Storro et al. (2010). men and infants risk behaviours for allergic diseases (encompassing a range of behaviours). doi:10.1186/1471- 2458-10-443 BMC Public Health 2010 BMC Public H Ramos et al. (2010). Smoking cessation, doi:10.1186/1471-2458-10-89 BMC Public Health 2010 Imp. Science Leon et al. (2010). HIV screening,doi:10.1186/1748-5908-5-8 Implementation Science 2010 Imp. Science · DOI
- Pilling et al. (2010). Patients talking to their providers about hypertension (to change provider prescribing behaviour), doi:10.1186/1748-5908-5-23 Implementation Science 2010 · DOI
- BMC Public H Bilardi et al. (2010). General Practitioners (GPs) treatment of Chlamydia (encompassing a range of behaviours), doi:10.1186/1471- 2458-10-70 BMC Public Health, 2010 BMC Public H Brousseau et al. (2010). Administration of vaccinations, doi:10.1186/1471-2458-10-750 BMC Public Health 2010 Imp. Science Kauth et al. (2010). Therapists' skill development in, and application of, Cognitive Behavioural Therapy (CBT), doi:10.1186/1748- 5908-5-75 Implementation Science 2010 Imp. Science Kennedy et al. (2010). Target behaviour/s: creating, finding, and implementing self-care support for people with long-term health conditions (encompassing a range of behaviours) Nb. This is a service level intervention that targets the behaviours of general practitioners, nurses, practice managers, clerical and reception staff, doi:10.1186/1748-5908-5-7 Implementation Science 2010 Imp. Science · DOI
- McCluskey et al. (2010). Rehabilitation therapists delivery of the 'Out-and-About Implementation Program' (encompassing a range of behaviourus), doi:10.1186/1748-5908-5-59 Implementation Science, 2010 Imp. Science · DOI
- Ramsay et al. (2010). Primary care doctors' test requesting behaviour, doi:10.1186/1748-5908-5-71 Implementation Science, 2010 BMC Public HS Cantrell et al. (2010). Provision of smoking cessation assistance by community health centres, doi:10.1186/1472-6963-10-25 BMC Health Services Research · DOI
Cited by
Sorted by citations
Michie, S., Wood, C. E., Johnston, M., Abraham, C., Francis, J. J., & Hardeman, W. (2015) · Health Technology Assessment · doi:10.3310/hta19990 · Cited by 680 · see citing works
Self-citationMichie, S., Carey, R. N., Johnston, M., Rothman, A. J., de Bruin, M., Kelly, M. P., & Connell, L. E. (2018) · Annals of Behavioral Medicine · doi:10.1007/s12160-016-9816-6 · Cited by 475 · see citing works
Self-citationBehavior Change Techniques in Popular Alcohol Reduction Apps
Crane, D., Garnett, C., Brown, J., West, R., & Michie, S. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.4060 · Cited by 177 · see citing works
Self-citationBorek, A. J., Abraham, C., Greaves, C. J., Gillison, F., Tarrant, M., Morgan-Trimmer, S., McCabe, R., & Smith, J. R. (2019) · Health Psychology Review · doi:10.1080/17437199.2019.1625282 · Cited by 152 · see citing works
Self-citationKolehmainen, N., & Francis, J. J. (2012) · Implementation Science · doi:10.1186/1748-5908-7-100 · Cited by 24 · see citing works
Self-citation
- Authors
- Abraham, C., Wood, C. E., Johnston, M., Francis, J., Hardeman, W., Richardson, M., & Michie, S.
- Affiliations
- University of Exeter (GB)
- University College London (GB)
- University of Aberdeen (GB)
- City, University of London (GB)
- University of Cambridge (GB)
- Corresponding author
- Abraham, Charles
- ORCID
- Venue
- Annals of Behavioral Medicine
- Publisher
- Oxford University Press (OUP)
- Published
- 2015-08-20
- Volume
- 49
- Issue
- 6
- Pages
- 885-900
- ISSN
- 0742-5554 (ISSN-L)
- 0883-6612
- 1532-4796
- OpenAlex
- W1912038704
- Cited by
- 71
- References
- 16
- Citations by year
- 2026: 2 · 2025: 2 · 2024: 6 · 2023: 4 · 2022: 11 · 2021: 8 · 2020: 6 · 2019: 7 · 2018: 12 · 2017: 4 · 2016: 4 · 2015: 4 · 2012: 1
- FWCI
- 12.94
- Type
- journal article
- Language
- en
- Open access
- green
- ueaeprints.uea.ac.uk
- Locations
- Annals of Behavioral Medicine (publishedVersion)
- PubMed (publishedVersion)
- Figshare (publishedVersion)
- UEA Digital Repository (University of East Anglia) (submittedVersion)
- Apollo (University of Cambridge) (submittedVersion)
- UCL Discovery (University College London) (submittedVersion)
- Figshare (submittedVersion, cc-by)
- Open Research Exeter (University of Exeter) (submittedVersion)
- Funders
- Medical Research Council (MR/K023195/1)
- Topics
- Social Sciences › Psychology › Applied Psychology › Behavioral Health and Interventions
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- behaviour change techniques, intervention description, intercoder reliability, test-retest reliability, intervention protocol
- MeSH terms
- Behavior Therapy, Humans, Reproducibility of Results, Health Behavior
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
