Reducing the risk of bias in health behaviour change trials: Improving trial design, reporting or bias assessment criteria? A review and case study
Open accessThe DOI matches its Crossref record; 27 of the 27 references it cites were found in a registry or an index.
Abstract
OBJECTIVE: It has been suggested that randomised controlled trials (RCTs) of health behaviour change (HBC) interventions are less rigorously designed than - for example- drug trials. This study presents an approach to clarifying whether this is due to poor trial design, incomplete trial reporting and/or the inappropriateness of commonly applied risk of bias assessment criteria. DESIGN: First, a framework of key sources of bias and common strategies for reducing bias risk is developed based on a literature review. Second, we describe the design of a multi-site RCT evaluating the cost-effectiveness of an HIV-treatment adherence intervention (case study). The choices made by the multidisciplinary team trying to minimise the risk of bias are compared against the risk of bias framework. MAIN OUTCOME MEASURES: Implementation of common strategies for reducing the risk of bias in the case study; alternative or additional strategies applied; a justification for each deviation from the risk of bias framework. RESULTS: Most of the common strategies for reducing the risk of bias could be implemented. Alternative strategies were developed for minimising the risk of performance bias and contamination. Several additional, domain-specific risk of bias strategies were implemented. CONCLUSIONS: The literature provides useful guidance for reducing the risk of bias in HBC trials. Yet, the case study suggests that HBC trial designers may face specific challenges that require alternative/additional measures for reducing the risk of bias. Using the risk of bias justification table (RATIONALE) could lead to better-designed HBC trials, more comprehensive trial reports and the data necessary for evaluating the appropriateness of commonly applied risk of bias assessment criteria to HBC trials.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Interventions for enhancing medication adherence
Nieuwlaat, R., Wilczynski, N., Navarro, T., Hobson, N., Jeffery, R., Keepanasseril, A., Agoritsas, T., Mistry, N., Iorio, A., Jack, S., et al. (2014) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd000011.pub4 · Cited by 2,945
GRADE guidelines: 4. Rating the quality of evidence—study limitations (risk of bias)
Guyatt, G. H., Oxman, A. D., Vist, G., Kunz, R., Brozek, J., Alonso-Coello, P., Montori, V., Akl, E. A., Djulbegovic, B., & Falck-Ytter, Y. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.07.017 · Cited by 2,798
Wood, L., Egger, M., Gluud, L. L., Schulz, K. F., Jüni, P., Altman, D. G., Gluud, C., Martin, R. M., Wood, A. J. G., & Sterne, J. A. C. (2008) · BMJ · doi:10.1136/bmj.39465.451748.ad · Cited by 2,538 · cited 6×
“…when personnel and/or participants are not blinded to treatment assignment, and the content of interventions is transferrable between arms, the internal validity of trial results is particularly at risk (Guyatt et al., 2011; Higgins et al., 2011; Keogh-Brown et al., 2007; Wood et al., 2008).”
BackgroundInfluentialBoutron, I., Moher, D., Altman, D. G., Schulz, K. F., Ravaud, P., & for the CONSORT Group* (2008) · Annals of Internal Medicine · doi:10.7326/0003-4819-148-4-200802190-00008 · Cited by 2,292
What is meant by intention to treat analysis? Survey of published randomised controlled trials
Hollis, S., & Campbell, F. (1999) · BMJ · doi:10.1136/bmj.319.7211.670 · Cited by 2,160
Pharmaceutical industry sponsorship and research outcome and quality: systematic review
Lexchin, J., Bero, L. A., Djulbegovic, B., & Clark, O. (2003) · BMJ · doi:10.1136/bmj.326.7400.1167 · Cited by 2,147 · cited 1×
“About 34% of researchers admitted to questionable research practices (Fanelli, 2009) and there are several studies (indirectly) demonstrating the impact of such behaviours on trial outcomes (Lexchin et al., 2003; Vartanian et al., 2007).”
BackgroundIntention-to-treat concept: A review
Gupta, S. (2011) · Perspectives in Clinical Research · doi:10.4103/2229-3485.83221 · Cited by 1,921
Fanelli, D. (2009) · PLoS ONE · doi:10.1371/journal.pone.0005738 · Cited by 1,911 · cited 6×
“Scientific malpractice Fanelli (2009) discusses three behaviours that amount to scientific malpractice that can bias trial results: the fabrication of data, the wilful distortion of data or results, and questionable research practices that ‘lie somewhere on a continuum between scientific fraud,…”
BackgroundInfluentialEffects of soft drink consumption on nutrition and health: a systematic review and meta-analysis
Vartanian, L. R., Schwartz, M. B., & Brownell, K. D. (2007) · American Journal of Public Health · doi:10.2105/ajph.2005.083782 · Cited by 1,863
Interventions for enhancing medication adherence
Haynes, R. B., Ackloo, E., Sahota, N., McDonald, H. P., & Yao, X. (2008) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd000011.pub3 · Cited by 1,244
Chaudoir, S. R., Dugan, A. G., & Barr, C. H. (2013) · Implementation Science · doi:10.1186/1748-5908-8-22 · Cited by 800
In Pursuit of Balance: Randomization in Practice in Development Field Experiments
Bruhn, M., & McKenzie, D. (2009) · American Economic Journal: Applied Economics · doi:10.1257/app.1.4.200 · Cited by 788 · cited 2×
“Several simulation studies suggest that when using simple randomisation, the probability of significant baseline imbalances in prognostic covariates reduces considerably when sample sizes exceed 100–200 participants (Bruhn & McKenzie, 2009; Kernan et al., 1999).”
BackgroundGeneration of allocation sequences in randomised trials: chance, not choice
Schulz, K. F., & Grimes, D. A. (2002) · The Lancet · doi:10.1016/s0140-6736(02)07683-3 · Cited by 774
Interventions for promoting physical activity
Foster, C., Hillsdon, M., Thorogood, M., Kaur, A., & Wedatilake, T. (2005) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd003180.pub2 · Cited by 692
Savović, J., Jones, H., Altman, D., Harris, R., Jűni, P., Pildal, J., Als-Nielsen, B., Balk, E., Gluud, C., Gluud, L., et al. (2012) · Health Technology Assessment · doi:10.3310/hta16350 · Cited by 668 · cited 1×
“Such risk of bias scores have been found to explain heterogeneity in effect sizes across trials, especially in trials with subjective outcome measures (Savovic et al., 2012; Wood et al., 2008).”
BackgroundStratified Randomization for Clinical Trials
Kernan, W. N., Viscoli, C. M., Makuch, R. W., Brass, L. M., & Horwitz, R. I. (1999) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(98)00138-3 · Cited by 627 · cited 1×
“Several simulation studies suggest that when using simple randomisation, the probability of significant baseline imbalances in prognostic covariates reduces considerably when sample sizes exceed 100–200 participants (Bruhn & McKenzie, 2009; Kernan et al., 1999).”
BackgroundBlase, K. A., Van Dyke, M., Fixsen, D. L., & Bailey, F. W. (2012) · Handbook of Implementation Science for Psychology in Education · doi:10.1017/cbo9781139013949.004 · Cited by 395
Reporting methods of blinding in randomized trials assessing nonpharmacological treatments
Boutron, I., Guittet, L., Estellat, C., Moher, D., Hróbjartsson, A., & Ravaud, P. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040061 · Cited by 389
Puffer, S., Torgerson, D., & Watson, J. (2003) · BMJ · doi:10.1136/bmj.327.7418.785 · Cited by 351
McCambridge, J., & Kypri, K. (2011) · PLoS ONE · doi:10.1371/journal.pone.0023748 · Cited by 242 · cited 1×
“…questions about the processes/determinants targeted by the intervention – common practice in HBC trials but problematic to the intent of distraction (McCambridge & Kypri, 2011) and possibly leading to differential reactance effects in control and intervention arms (McCambridge, 2015; McCambridge,…”
BackgroundSelf-citationde Bruin, M. (2010) · Archives of Internal Medicine · doi:10.1001/archinternmed.2009.536 · Cited by 181
de Bruin, M., Viechtbauer, W., Hospers, H. J., Schaalma, H. P., & Kok, G. (2009) · Health Psychology · doi:10.1037/a0015989 · Cited by 152 · cited 1×
“Moreover, since data on ‘usual nursing practices’ can be a great source of information (e.g. for identifying best practices; see de Bruin et al., 2009; de Bruin, Viechtbauer, et al., 2010), this research objective was credible.”
BackgroundCommentary: Measuring the success of blinding in RCTs: don't, must, can't or needn't?
Sackett, D. L. (2007) · International Journal of Epidemiology · doi:10.1093/ije/dym088 · Cited by 150 · cited 4×
“Educational (or: behavioural) trials may be particularly sensitive to contamination due to the transferability of the active intervention content, especially when interventions are simple (Keogh-Brown et al., 2007) and when there is no blinding of treatment assignment (Sackett, 2007).”
BackgroundInfluentialMcCambridge, J., Kypri, K., & Elbourne, D. (2014) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2013.09.004 · Cited by 138
Self-citationDifferential attrition in health behaviour change trials: a systematic review and meta-analysis
Crutzen, R., Viechtbauer, W., Spigt, M., & Kotz, D. (2015) · Psychology & Health · doi:10.1080/08870446.2014.953526 · Cited by 132
Bishop, F. L., Fenge-Davies, A. L., Kirby, S., & Geraghty, A. W. A. (2015) · Psychology & Health · doi:10.1080/08870446.2014.953529 · Cited by 121
Randomized Trials Stopped Early for Benefit: A Systematic Review
Montori, V. M., Devereaux, P. J., Adhikari, N. K. J., Burns, K. E. A., Eggert, C. H., Briel, M., Lacchetti, C., Leung, T. W., Darling, E., Bryant, D. M., et al. (2005) · JAMA · doi:10.1001/jama.294.17.2203 · Cited by 26
Cited by
Effective Employee Engagement in the Workplace
Osborne, S., & Hammoud, M. S. (2017) · International Journal of Applied Management and Technology · doi:10.5590/ijamt.2017.16.1.04 · Cited by 381
McEwan, D., Harden, S. M., Zumbo, B. D., Sylvester, B. D., Kaulius, M., Ruissen, G. R., Dowd, A. J., & Beauchamp, M. R. (2016) · Health Psychology Review · doi:10.1080/17437199.2015.1104258 · Cited by 271
Differential attrition in health behaviour change trials: a systematic review and meta-analysis
Crutzen, R., Viechtbauer, W., Spigt, M., & Kotz, D. (2015) · Psychology & Health · doi:10.1080/08870446.2014.953526 · Cited by 132
de Bruin, M., Viechtbauer, W., Eisma, M. C., Hartmann-Boyce, J., West, R., Bull, E., Michie, S., & Johnston, M. (2016) · Systematic Reviews · doi:10.1186/s13643-016-0253-1 · Cited by 41
de Bruin, M. (2015) · Psychology & Health · doi:10.1080/08870446.2014.960653 · Cited by 36
Metadata
- Authors
- de Bruin, M., McCambridge, J., & Prins, J. M.
- Venue
- Psychology & Health
- Publisher
- Informa UK Limited
- Published
- 2014-10-21
- Volume
- 30
- Issue
- 1
- Pages
- 8-34
- ISSN
- 0887-0446 (ISSN-L)
- 1476-8321
- OpenAlex
- W2156814347
- Cited by
- 53
- References
- 27
- FWCI
- 7.05
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.