Evidence for risk of bias in cluster randomised trials: review of recent trials published in three general medical journals
Open accessAbstract
Objective To examine the prevalence of a risk of bias associated with the design and conduct of cluster randomised controlled trials among a sample of recently published studies. Design Retrospective review of cluster randomised trials published in the BMJ, Lancet , and New England Journal of Medicine from January 1997 to October 2002. Main outcome measures Prevalence of secure randomisation of clusters, identification of participants before randomisation (to avoid foreknowledge of allocation), differential recruitment between treatment arms, differential application of inclusion and exclusion criteria, and differential attrition. Results Of the 36 trials identified, 24 were published in the BMJ ,11 in the Lancet , and a single trial in the New England Journal of Medicine . At the cluster level, 15 (42%) trials provided evidence for secure allocation and 25 (69%) used stratified allocation. Few trials showed evidence of imbalance at the cluster level. However, some evidence of susceptibility to risk of bias at the individual level existed in 14 (39%) studies. Conclusions Some recently published cluster randomised trials may not have taken adequate precautions to guard against threats to the internal validity of their design.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Pragmatic enrolment approaches were also observed, underscoring a tension between feasibility and precision in SWD planning.
“Pragmatic enrolment approaches were also observed [e.g., Dorothy Thomas (40) and Eduard J. Sanders (41)], underscoring a tension between feasibility and precision in SWD planning (115).”
From Applications and implementation considerations for stepped-wedge designs in sub-Saharan Africa: a systematic review (2026) · cites this paper as [115]
The Introduction (pp. 2–3) says prospective recruitment is sometimes necessary in cluster trials and can introduce selection bias; it recommends identifying participants before randomisation where possible. This supports a practical enrolment trade-off, but the paper does not address statistical precision or stepped-wedge design (SWD) planning. (LLM, conf=0.81)
Experimental — accuracy not yet measuredImprovement is still needed in reporting allocation concealment, restricted randomisation procedures, blinding where feasible, and adverse events to strengthen transparency and replicability.
“Nonetheless, improvement is still needed in reporting allocation concealment, restricted randomisation procedures, blinding where feasible, and adverse events to strengthen transparency and replicability (115).”
From Applications and implementation considerations for stepped-wedge designs in sub-Saharan Africa: a systematic review (2026) · cites this paper as [115]
The Results (p. 4) say many trials did not clearly report who allocated clusters or how, and describe the use of stratified randomisation. “A new CONSORT statement?” (p. 6) calls for reporting whether recruiters were blind to allocation. The paper does not call for improved adverse-event reporting, and it does not specifically identify reporting of restricted randomisation procedures as a deficiency. (LLM, conf=0.87)
Experimental — accuracy not yet measuredOthers have assessed about 40% of cluster trials to be at risk of these types of biases; and sometimes this has been reported to be somewhat lower despite including many trials with post randomization recruitment.
“Others have assessed about 40% of cluster trials to be at risk of these types of biases [2,7,28]; and sometimes this has been reported to be somewhat lower despite including many trials with post randomization recruitment [10,13].”
From Cluster randomized trials of individual-level interventions were at high risk of bias (2021) · cites this paper as [28]
The Abstract (p. 2) reports susceptibility to individual-level bias in 14 of 36 cluster trials (39%), supporting the “about 40%” figure. The Results (p. 5) say 23 trials had not identified participants before randomisation, but the paper does not report a somewhat lower overall bias-risk figure for those trials; its finding of differential recruitment or consent in 7 of the 23 concerns only one type of risk. (LLM, conf=0.84)
Experimental — accuracy not yet measuredMethodological reviews have identified that many cluster trials are at risk of these identification and recruitment biases because they recruit participants with knowledge of allocated treatment and this often manifests in baseline imbalances.
“Methodological reviews have identified that many cluster trials are at risk of these identification and recruitment biases because they recruit participants with knowledge of allocated treatment and this often manifests in baseline imbalances [1,2,28].”
From Cluster randomized trials of individual-level interventions were at high risk of bias (2021) · cites this paper as [28]
Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
Experimental — accuracy not yet measuredSWD offers a pragmatic balance between rigorous evaluation and the ethical imperative to ensure eventual access to potentially beneficial interventions.
“In particular, SWD offers a pragmatic balance between rigorous evaluation and the ethical imperative to ensure eventual access to potentially beneficial interventions (115).”
From Applications and implementation considerations for stepped-wedge designs in sub-Saharan Africa: a systematic review (2026) · cites this paper as [115]
The Introduction (p. 2) describes cluster randomisation as a method of evaluation, and the Conclusion (p. 6) addresses its risk of bias. Neither discusses stepped-wedge designs (SWD), eventual access to interventions, or an ethical balance between access and evaluation. Searches for those concepts found no supporting passage in our copy. (LLM, conf=0.76)
Experimental — accuracy not yet measured
The variability in SWD design parameters illustrates SWD's flexibility for diverse implementation contexts and timelines.
“This variability illustrates SWD's flexibility for diverse implementation contexts and timelines (115).”
From Applications and implementation considerations for stepped-wedge designs in sub-Saharan Africa: a systematic review (2026) · cites this paper as [115]
The Results (pp. 4–5) review risks of bias in cluster-randomised trials, and the Conclusion (p. 6) addresses avoiding those biases. Searches for stepped-wedge design, design-parameter variability, and implementation timelines found no passage addressing the claimed flexibility of SWD. (LLM, conf=0.76)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
No citation lists for this paper
No citation lists for this paper yet — checked again daily.
References 16 · Cited by 351
Research works printed but not linked: 10
- 1 Shulz KF, Chalmers I, Grimes DA, Altman DG. Assessing the quality of randomization from reports of controlled trials in obstretrics and gynaecology journals. JAMA 1994;272:125-8.
- 2 Kjaergaard LL, Villumsen J, Cluud C. Reported methodologic quality and discrepancies between large and small randomized trials in meta-analyses. Ann Intern Med 2001;135:982-9.
- 3 Schulz KF. Subverting randomisation in controlled trials. JAMA 1995;274:1456-8.
- 4 Zelen M. Randomized consent designs for clinical trials: an update. Stat Med 1990;9:645-56.
- 6 Gotzsche PC, Olsen O. Is screening for breast cancer with mammography justified? Lancet 2000;355:129-34.
- 7 Alexander FE, Anderson TJ, Brown HK, Forrest APM, Hepburn W, Kirkpatrick AE, et al. 14 years of follow-up from the Edinburgh randomised trial of breast-cancer screening. Lancet 1999;353:1903-8.
- 8 Sanci LA, Coffey CMM, Veit FCM, Carr-Gregg M, Patton GC, Day N, et al. Evaluation of the effectiveness of an educational intervention for general practitioners in adolescent health care: randomised controlled trial. BMJ 2000;320:224-30.
- 9 Glazener CMA, Ramsay CR, Campbell MK, Booth P, Duffty P, Lloyd DJ, et al. Neonatal examination and screening trial (NEST): a randomised, controlled, switchback trial of alternative policies for low risk infants. BMJ 1999;318:627-32.
- 10 O'Connell DL, Henry D, Tomlins R. Randomised controlled trial of effect of feedback on general practitioners' prescribing in Australia. BMJ 1999;318:507-11.
- 14 Elbourne DR, Campbell MK. Extending the CONSORT statement to cluster randomized trials: for discussion. Stat Med 2001;20:489-96. (Accepted 10 July 2003)
- Authors
- Puffer, S., Torgerson, D., & Watson, J.
- Affiliations
- York University (CA)
- Corresponding author
- Puffer, Suezann
- ORCID
- Torgerson, David
- Venue
- BMJ
- Publisher
- BMJ
- Published
- 2003-10-02
- Volume
- 327
- Issue
- 7418
- Pages
- 785-789
- ISSN
- 0007-1447 (ISSN-L)
- 0959-8138
- 1468-5833
- PMC
- PMC214092
- OpenAlex
- W2127719902
- Cited by
- 351
- References
- 16
- Citations by year
- 2026: 5 · 2025: 8 · 2024: 8 · 2023: 14 · 2022: 11 · 2021: 12 · 2020: 14 · 2019: 22 · 2018: 12 · 2017: 8 · 2016: 19 · 2015: 22 · 2014: 15 · 2013: 25 · 2012: 23
- FWCI
- 12.63
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- www.bmj.com/company/legal-information/terms-conditions/legal-information/tdm-licencepolicy (tdm)
- Locations
- BMJ (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- White Rose Research Online (University of Leeds, The University of Sheffield, University of York) (acceptedVersion)
- Funders
- Medical Research Council
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Keywords
- risk of bias, allocation concealment, differential attrition, internal validity, general medical journals
- MeSH terms
- Humans, Informed Consent, Periodicals as Topic, Retrospective Studies, Bias, Cluster Analysis, Randomized Controlled Trials as Topic, Patient Selection
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
