CoChat
Sign Up
Cited by 439Meta-analysis and systematic reviewsStatistical Methods in Clinical TrialsAdvanced Causal Inference Techniques

Randomisation to protect against selection bias in healthcare trials

Open access
Odgaard-Jensen, Jan, Vist, Gunn E, Timmer, Antje, Kunz, Regina, Akl, Elie A, Schünemann, Holger, Briel, Matthias, Nordmann, Alain J +2 more·2011·Cochrane Database of Systematic Reviews
DOI 10.1002/14651858.mr000012.pub3PMC PMC7150228
Open this paper in CoChat — free accountOpen PDF

Abstract

BACKGROUND: Randomised trials use the play of chance to assign participants to comparison groups. The unpredictability of the process, if not subverted, should prevent systematic differences between comparison groups (selection bias). Differences due to chance will still occur and these are minimised by randomising a sufficiently large number of people. OBJECTIVES: To assess the effects of randomisation and concealment of allocation on the results of healthcare studies. SEARCH STRATEGY: We searched the Cochrane Methodology Register, MEDLINE, SciSearch and reference lists up to September 2009. In addition, we screened articles citing included studies (ISI Science Citation Index) and papers related to included studies (PubMed). SELECTION CRITERIA: Eligible study designs were cohorts of studies, systematic reviews or meta-analyses of healthcare interventions that compared random allocation versus non-random allocation or adequate versus inadequate/unclear concealment of allocation in randomised trials. Outcomes of interest were the magnitude and direction of estimates of effect and imbalances in prognostic factors. DATA COLLECTION AND ANALYSIS: We retrieved and assessed studies that appeared to meet the inclusion criteria independently. At least two review authors independently appraised methodological quality and extracted information. We prepared tabular summaries of the results for each comparison and assessed the results across studies qualitatively to identify common trends or discrepancies. MAIN RESULTS: A total of 18 studies (systematic reviews or meta-analyses) met our inclusion criteria. Ten compared random allocation versus non-random allocation and nine compared adequate versus inadequate or unclear concealment of allocation within controlled trials. All studies were at high risk of bias.For the comparison of randomised versus non-randomised studies, four comparisons yielded inconclusive results (differed between outcomes or different modes of analysis); three comparisons showed similar results for random and non-random allocation; two comparisons had larger estimates of effect in non-randomised studies than in randomised trials; and two comparisons had larger estimates of effect in randomised than in non-randomised studies.Five studies found larger estimates of effect in trials with inadequate concealment of allocation than in trials with adequate concealment. The four other studies did not find statistically significant differences. AUTHORS' CONCLUSIONS: The results of randomised and non-randomised studies sometimes differed. In some instances non-randomised studies yielded larger estimates of effect and in other instances randomised trials yielded larger estimates of effect. The results of controlled trials with adequate and inadequate/unclear concealment of allocation sometimes differed. When differences occurred, most often trials with inadequate or unclear allocation concealment yielded larger estimates of effects relative to controlled trials with adequate allocation concealment. However, it is not generally possible to predict the magnitude, or even the direction, of possible selection biases and consequent distortions of treatment effects from studies with non-random allocation or controlled trials with inadequate or unclear allocation concealment.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1002/14651858.mr000012.pub3

Cites

166 of 166 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

5 of the 171 references couldn’t be matched to a known paper.

Research works printed but not linked: 11

  1. Kjaergard 2001 {published data only} Kjaergard LL, Villumsen J, Gluud C. Reported methodologic quality and discrepancies between large and small randomized trials in meta-analysis. Annals of Internal Medicine 2001;135:982-9.
  2. Morrison 1997 {published data only} Morrison B, Lilford RJ, Earnst E. Methodological rigor and results of clinical trials of homeopathic remedies. Perfusion 2000;13:132-8.
  3. Kunz R, Oxman AD. Two systematic reviews -two different answers?. BMJ Electronic Letters 28 August 1999.
  4. Schulz KF, Chalmers I, Hayes RJ, Altman DG: Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 1995, 273:408-412.
  5. Moher D, Pham B, Jones A, Cook DJ, Jadad AR, Moher M, Tugwell P, Klassen TP: Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? Lancet 1998, 352:609-613.
  6. J ̈uni P, Altman DG, Egger M: Systematic reviews in health care: assessing the quality of controlled clinical trials. BMJ 2001, 323:42-46.
  7. Howick J: The Philosophy of Evidence-Based Medicine. Chichester: Wiley Blackwell & BMJ Books; 2011.
  8. Savović J, Jones HE, Altman DG, Harris R, J ̈uni P, Pildal J, Als-Nielsen B, Balk E, Gluud C, Gluud L, Ioannidis J, Schulz K, Beynon R, Welton N, Wood L, Moher D, Deeks J, Sterne J: Influence of reported study design characteristics on intervention effect estimates from randomized, controlled trials. Ann Inter Med 2012, 157:429-438.
  9. Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, Sch ̈unemann HJ, GRADE Working Group: GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ 2008, 336:924-926.
  10. OCEBM Levels of Evidence Working Group: Oxford Centre for Evidence-Based Medicine 2011 Levels of Evidence. [http://www.cebm.net/ index.aspx?o = 5653].
  11. Savović J, Jones HE, Altman DG, Harris R, J ̈uni P, Pildal J, Als-Nielsen B, Balk E, Gluud C, Gluud L, Ioannidis J, Schulz K, Beynon R, Welton N, Wood L, Moher D, Deeks J, Sterne J: Influence of reported study design characteristics on intervention effect estimates from randomized, controlled trials. Ann Inter Med 2012, 157:429-438.

Cited by

Sorted by citations

Related

Authors
Odgaard-Jensen, J., Vist, G. E., Timmer, A., Kunz, R., Akl, E. A., Schünemann, H., Briel, M., Nordmann, A. J., Pregno, S., & Oxman, A. D.
Affiliations
  • Norwegian Knowledge Centre for the Health Services; Global Health Unit; PO Box 7004, St. Olavs Plass Oslo Norway N-0130
  • Nasjonalt Kunnskapssenter for Helsetjenesten (NO)
  • Carl von Ossietzky Universität Oldenburg (DE)
  • University of Basel (CH)
  • American University of Beirut (LB)
  • McMaster University (CA)
  • University Hospital of Basel (CH)
  • University of Modena and Reggio Emilia (IT)
ORCID
Venue
Cochrane Database of Systematic Reviews
Publisher
Wiley
Published
2011-04-13
Volume
2015
Issue
4
Pages
MR000012-MR000012
ISSN
  • 1361-6137 (ISSN-L)
  • 1465-1858
DOI
10.1002/14651858.mr000012.pub3
PMC
PMC7150228
OpenAlex
W1919332000
Cited by
439
References
171
Citations by year
2026: 4 · 2025: 8 · 2024: 20 · 2023: 18 · 2022: 18 · 2021: 32 · 2020: 23 · 2019: 22 · 2018: 22 · 2017: 19 · 2016: 23 · 2015: 28 · 2014: 30 · 2013: 38 · 2012: 45
FWCI
10.15
Type
journal article
Language
en
Open access
Locations
  • Cochrane Database of Systematic Reviews (publishedVersion)
  • PubMed Central (publishedVersion)
  • PubMed (publishedVersion)
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
  • Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
Keywords
randomization, allocation concealment, selection bias, randomised trials, non-randomized studies, health interventions, methodological quality of trials
MeSH terms
Clinical Trials as Topic, Random Allocation, Selection Bias, Randomized Controlled Trials as Topic, Treatment Outcome, Controlled Clinical Trials as Topic

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review