CoChat

Randomisation to protect against selection bias in healthcare trials

Kunz, Regina, Vist, Gunn Elisabeth, Oxman, Andrew D · 2007 · Cochrane Database of Systematic Reviews
Cited by 185

The DOI matches its Crossref record; 96 of the 96 references it cites were found in a registry or an index.

Abstract

BACKGROUND: Randomised trials use the play of chance to assign participants to comparison groups. The unpredictability of the process, if not subverted, should prevent systematic differences between comparison groups (selection bias), provided that a sufficient number of people are randomised. OBJECTIVES: To assess the effects of randomisation and concealment of allocation on the results of healthcare trials. SEARCH STRATEGY: We searched the Cochrane Methodology Register, MEDLINE, SciSearch, reference lists up to August 2000 and used personal communication. SELECTION CRITERIA: Cohorts of trials, systematic reviews or meta-analyses of healthcare interventions that compared outcomes or prognostic factors for one of the following comparisons: randomised versus non-randomised trials, randomised trials with adequately versus inadequately concealed allocation, or high versus low quality trials where selection bias could not be separated from other sources of bias. DATA COLLECTION AND ANALYSIS: One of us went through all of the citations in the Cochrane Methodology Register and accumulated reference lists. Studies that appeared to meet the inclusion criteria were retrieved and assessed independently by two of the reviewers. The methodological quality of included studies was appraised and information extracted by one of us and checked by a second. Tabular summaries of the results were prepared for each comparison and the results across studies were assessed qualitatively to identify common trends or discrepancies. MAIN RESULTS: We identified 32 studies including over 3000 trials. Twenty-two studies compared randomised versus non-randomised trials, three compared adequately versus inadequately concealed allocation, and nine compared high versus low quality trials (some studies included more than one comparison). Five studies were of high methodological quality. In 15 of the 22 studies that compared randomised and non-randomised trials of the same intervention, important differences were found in the estimates of effect. Some of these differences were due to a poorer prognosis in the control groups in the non-randomised trials. The results of the other seven studies that compared randomised and non-randomised trials across different interventions are less clear. Comparisons of adequately and inadequately concealed allocation in randomised trials of the same intervention provided high quality evidence that concealment can be crucial in achieving similar treatment groups and, therefore, unbiased estimates of treatment effects. Studies with inadequate concealment tended to overestimate treatment effects. Comparisons of high and low quality trials of the same intervention have found important differences in estimates of effect, but it is not possible to determine the extent to which these differences can be attributed to randomisation or concealment of allocation. Omitting comparisons between randomised trials and non-randomised trials using historical controls did not substantially alter the results or conclusions of our review. AUTHORS' CONCLUSIONS: On average, non-randomised trials and randomised trials with inadequate concealment of allocation tend to result in larger estimates of effect than randomised trials with adequately concealed allocation. However, it is not generally possible to predict the magnitude, or even the direction, of possible selection biases and consequent distortions of treatment effects.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1002/14651858.mr000012.pub2

Cites

96 of 96 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

1 retracted

Research works printed but not linked: 21

  1. Ortiz, 1998, The efficacy of folic acid and folinic acid in reducing methotrexate gastrointestinal toxicity in rheumatoid arthritis. A meta-analysis of randomized controlled trials. J Rheumatol, J Rheumatol, 25, 36
  2. Stanton, 1997, Outcome, attrition and family-couples treatment for drug abuse, Psychological Bulletin, 122, 170, 10.1037/0033-2909.122.2.170 · DOI · Paper page
  3. Wortman, 1983, Synthesis of results in trials of coronary artery bypass graft surgery, 8, 536
  4. Stieb, 1990, The effectiveness and usefulness of Haemophilus influenzae Type b Vaccines: a systematic overview (meta-analysis), Can Med Assoc J, 142, 719
  5. Black, 1996, The effectiveness of surgery for stress incontinence in women: a systematic review, Br J Urol, 78, 497, 10.1046/j.1464-410X.1996.01422.x · DOI · Paper page
  6. Black, 1996, The effectiveness of surgery for stress incontinence in women: a systematic review, Br J Urol, 78, 497, 10.1046/j.1464-410X.1996.01422.x · DOI · Paper page
  7. Kasiske, 1993, Elective cyclosporine withdrawal after renal transplantation. A meta-analysis, JAMA, 269, 395, 10.1001/jama.1993.03500030093040 · doi:10.1001/jama.1993.03500030093040
  8. Shadish, 1992, Do family and marital psychotherapies change what people do? A meta-analysis of behavioural outcomes, 129
  9. Shadish, 1997, Experiments versus quasi-experiments: do they yield the same answer?, 147
  10. Watson, 2002, Techniques for pelvic surgery in subfertility, Cochrane Database of Systematic Reviews
  11. Britton A McKee M Black N McPherson K Sanderson C Bain C Three systematic reviews - not so different answers BMJ Electronic letters
  12. Dans, 2002, Applying the results to individual patients., 369
  13. Gubler, 1971, Alternating hemiplegia, a sign of pontine lesion, and documentation of the proof of the facial decussation, 9
  14. Guyatt, 2002, Therapy, 55
  15. Kleijnen, 1997, So what's so special about randomisation?, 93
  16. Kunz R Oxman AD Two systematic reviews - two different answers? BMJ Electronic letters
  17. Moore, 1995, Deadly Medicine
  18. 1948, Streptomycin treatment of pulmonary tuberculosis, BMJ, ii, 769
  19. Reeves, 1998, Comparison of effect size estimates derived from randomised and non-randomised studies, 73
  20. 1994, Identifying Health technologies that Work:Searching for Evidence, 41
  21. Valenstein, 1986, Great and Desparate Cures: The Rise and Decline of Psychosurgery and Other Radical Treatments for Mental Illness, 338

Cited by

See all 185

Metadata

Authors
Kunz, R., Vist, G. E., & Oxman, A. D.
Venue
Cochrane Database of Systematic Reviews
Published
2007-04-18
Issue
2
Pages
MR000012-MR000012
ISSN
1361-6137 (ISSN-L)
OpenAlex
W4245428103
Cited by
185
References
96
Type
reference entry
Language
en
Open access
closed

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.