Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Cited by 185

The DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.

Randomisation to protect against selection bias in healthcare trials

Kunz, Regina, Vist, Gunn Elisabeth, Oxman, Andrew D · 2007 · Cochrane Database of Systematic Reviews

Abstract

BACKGROUND: Randomised trials use the play of chance to assign participants to comparison groups. The unpredictability of the process, if not subverted, should prevent systematic differences between comparison groups (selection bias), provided that a sufficient number of people are randomised. OBJECTIVES: To assess the effects of randomisation and concealment of allocation on the results of healthcare trials. SEARCH STRATEGY: We searched the Cochrane Methodology Register, MEDLINE, SciSearch, reference lists up to August 2000 and used personal communication. SELECTION CRITERIA: Cohorts of trials, systematic reviews or meta-analyses of healthcare interventions that compared outcomes or prognostic factors for one of the following comparisons: randomised versus non-randomised trials, randomised trials with adequately versus inadequately concealed allocation, or high versus low quality trials where selection bias could not be separated from other sources of bias. DATA COLLECTION AND ANALYSIS: One of us went through all of the citations in the Cochrane Methodology Register and accumulated reference lists. Studies that appeared to meet the inclusion criteria were retrieved and assessed independently by two of the reviewers. The methodological quality of included studies was appraised and information extracted by one of us and checked by a second. Tabular summaries of the results were prepared for each comparison and the results across studies were assessed qualitatively to identify common trends or discrepancies. MAIN RESULTS: We identified 32 studies including over 3000 trials. Twenty-two studies compared randomised versus non-randomised trials, three compared adequately versus inadequately concealed allocation, and nine compared high versus low quality trials (some studies included more than one comparison). Five studies were of high methodological quality. In 15 of the 22 studies that compared randomised and non-randomised trials of the same intervention, important differences were found in the estimates of effect. Some of these differences were due to a poorer prognosis in the control groups in the non-randomised trials. The results of the other seven studies that compared randomised and non-randomised trials across different interventions are less clear. Comparisons of adequately and inadequately concealed allocation in randomised trials of the same intervention provided high quality evidence that concealment can be crucial in achieving similar treatment groups and, therefore, unbiased estimates of treatment effects. Studies with inadequate concealment tended to overestimate treatment effects. Comparisons of high and low quality trials of the same intervention have found important differences in estimates of effect, but it is not possible to determine the extent to which these differences can be attributed to randomisation or concealment of allocation. Omitting comparisons between randomised trials and non-randomised trials using historical controls did not substantially alter the results or conclusions of our review. AUTHORS' CONCLUSIONS: On average, non-randomised trials and randomised trials with inadequate concealment of allocation tend to result in larger estimates of effect than randomised trials with adequately concealed allocation. However, it is not generally possible to predict the magnitude, or even the direction, of possible selection biases and consequent distortions of treatment effects.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1002/14651858.mr000012.pub2

Cites

15 of 15 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 15 references

Research works printed but not linked: 96

  1. Aronson, 1996, Triiodothyronine augmentation in the treatment of refractory depression. A meta-analysis, Arch Gen Psychiatry, 53, 842, 10.1001/archpsyc.1996.01830090090013 · DOI · Paper page
  2. Bhansali, 1996, Chemotherapy for carcinoma of the esophagus: a comparison of evidence from meta-analyses of randomized trials and of historical control studies, Ann Oncol, 7, 355, 10.1093/oxfordjournals.annonc.a010601 · DOI
  3. Carroll, 1996, Randomization is important in studies with pain outcomes: systematic review of transcutaneous electrical nerve stimulation in acute postoperative pain. Br J Anaesth, Br J Anaesth, 77, 798, 10.1093/bja/77.6.798 · DOI · Paper page
  4. Chalmers, 1997, Evidence favoring the use of anticoagulants in the hospital phase of acute myocardial infarction, N Engl J Med, 297, 1091, 10.1056/NEJM197711172972004 · DOI · Paper page
  5. Colditz, 1989, How study design affects outcomes in comparisons of therapy. I: Medical, Stat Med, 8, 441, 10.1002/sim.4780080408 · DOI · Paper page
  6. Diehl, 1986, A comparison of randomized concurrent control groups with matched historical control groups: are historical controls valid?, J Clin Oncol, 4, 1114, 10.1200/JCO.1986.4.7.1114 · DOI
  7. Emerson, 1990, An empirical study of the possible relation of treatment differences to quality scores in controlled randomized clinical trials, Control Clin Trials, 11, 339, 10.1016/0197-2456(90)90175-2 · DOI · Paper page
  8. Forgie, 1998, Preoperative autologous donation decreases allogeneic transfusion but increases exposure to all red blood cell transfusion: results of a meta-analysis. International Study of Perioperative Transfusion (ISPOT) Investigators, Arch Intern Med, 158, 610, 10.1001/archinte.158.6.610 · DOI · Paper page
  9. Guyatt, 2000, Randomized trials versus observational studies in adolescent pregnancy prevention, J Clin Epidemiol, 53, 167, 10.1016/S0895-4356(99)00160-2 · DOI
  10. Imperiale, 1990, Do corticosteroids reduce mortality from alcoholic hepatitis? A meta analysis of the randomized trials. Ann Intern Med, Ann Intern Med, 113, 299, 10.7326/0003-4819-113-4-299 · DOI · Paper page
  11. Khan, 1996, The importance of quality of primary studies in producing unbiased systematic reviews. Arch Intern Med, Arch Intern Med, 156, 661, 10.1001/archinte.1996.00440060089011 · doi:10.1001/archinte.1996.00440060089011 · Paper page
  12. Miller, 1989, How study design affects outcomes in comparisons of therapy. II, Surgical Stat Med, 8, 455, 10.1002/sim.4780080409 · DOI
  13. Mullen, 1997, A meta-analysis of trials evaluating patient education and counseling for three groups of preventive health behaviors, Patient Educ Couns, 32, 157, 10.1016/S0738-3991(97)00037-2 · DOI
  14. Nurmohamed, 1992, Low molecular weight heparin versus standard heparin in general and orthopaedic surgery: a meta-analysis. Lancet, Lancet, 340, 152, 10.1016/0140-6736(92)93223-A · DOI · Paper page
  15. Ortiz, 1998, The efficacy of folic acid and folinic acid in reducing methotrexate gastrointestinal toxicity in rheumatoid arthritis. A meta-analysis of randomized controlled trials. J Rheumatol, J Rheumatol, 25, 36
  16. Ottenbacher, 1991, Epistemology and experimentation: an examination of quality factors in research design, Am J Occup Ther, 45, 917, 10.5014/ajot.45.10.917 · DOI
  17. Ottenbacher, 1992, Impact of random assignment on study outcome: an empirical examination, Control Clin Trials, 13, 50, 10.1016/0197-2456(92)90029-Y · DOI
  18. Ottenbacher, 1993, The results of clinical trials in stroke rehabilitation research [see comments], Arch Neurol, 50, 37, 10.1001/archneur.1993.00540010033014 · DOI · Paper page
  19. Potter, 1998, Routine protein energy supplementation in adults: systematic review, BMJ, 317, 495, 10.1136/bmj.317.7157.495 · DOI · Paper page
  20. Pyorala, 1995, A review and meta-analysis of hormonal treatment of cryptorchidism, J Clin Endocrinol Metab, 80, 2795
  21. Reimold, 1992, Assessment of the efficacy and safety of antiarrhythmic therapy for chronic atrial fibrillation: observations on the role of trial design and implications of drug related mortality., Am Heart J, 124, 924, 10.1016/0002-8703(92)90974-Z · DOI
  22. 1994, Worldwide collaborative observational study and meta analysis on allogenic leukocyte immunotherapy for recurrent spontaneous abortion, Am J Reprod Immunol, 32, 55, 10.1111/j.1600-0897.1994.tb01095.x · DOI · Paper page
  23. Sacks, 1982, Randomized versus historical controls for clinical trials, Am J Med, 72, 233, 10.1016/0002-9343(82)90815-4 · DOI · Paper page
  24. Stanton, 1997, Outcome, attrition and family-couples treatment for drug abuse, Psychological Bulletin, 122, 170, 10.1037/0033-2909.122.2.170 · DOI · Paper page
  25. Watson, 1994, A meta-analysis of the therapeutic role of oil soluble contrast media at hysterosalpingography: a surprising result?, Fertil Steril, 61, 470, 10.1016/S0015-0282(16)56578-9 · DOI
  26. Wortman, 1983, Synthesis of results in trials of coronary artery bypass graft surgery, 8, 536
  27. Colditz, 1994, Efficacy of BCG vaccine in the prevention of tuberculosis. Meta analysis of the published literature, JAMA, 271, 698, 10.1001/jama.1994.03510330076038 · DOI · Paper page
  28. Gilbert, 1977, Statistics and ethics in surgery and anesthesia, Science, 198, 684, 10.1126/science.333585 · DOI
  29. Hovell, 1982, The experimental evidence for weight-loss treatment of essential hypertension: a critical review, Am J Public Health, 72, 359, 10.2105/AJPH.72.4.359 · DOI
  30. Hutchinson, 1999, Clinical effectiveness of pneumococcal vaccine. Meta-analysis, Can Fam Physician, 45, 2381
  31. Kerlikowske, 1995, Efficacy of screening mammography. A meta-analysis, JAMA, 273, 149, 10.1001/jama.1995.03520260071035 · doi:10.1001/jama.1995.03520260071035 · Paper page
  32. Koes, 1994, The efficacy of back schools: a review of randomized clinical trials, J Clin Epidemiol, 47, 851, 10.1016/0895-4356(94)90188-0 · DOI
  33. Black, 1996, The effectiveness of surgery for stress incontinence in women: a systematic review, Br J Urol, 78, 497, 10.1046/j.1464-410X.1996.01422.x · DOI · Paper page
  34. Macarthur, 1995, Qualitative assessment of studies included in a meta-analysis: DES and the risk of pregnancy loss, J Clin Epidemiol, 48, 739, 10.1016/0895-4356(94)00190-2 · DOI
  35. Mehta, 1999, Coronary stent implantation in acute myocardial infarction, Am Heart J, 137, 603, 10.1016/S0002-8703(99)70212-7 · DOI
  36. Rozenberg, 1999, Efficacy of epidural steroids in low back pain and sciatica. A critical appraisal by a French Task Force of randomized trials. Critical Analysis Group of the French Society for Rheumatology, Rev Rhum Engl Ed, 66, 79
  37. Stieb, 1990, The effectiveness and usefulness of Haemophilus influenzae Type b Vaccines: a systematic overview (meta-analysis), Can Med Assoc J, 142, 719
  38. Abrutyn, 1991, Intrathecal therapy in tetanus. A meta-analysis, JAMA, 266, 2262, 10.1001/jama.1991.03470160094039 · DOI
  39. Black, 1996, The effectiveness of surgery for stress incontinence in women: a systematic review, Br J Urol, 78, 497, 10.1046/j.1464-410X.1996.01422.x · DOI · Paper page
  40. Black, 1996, The effectiveness of surgery for stress incontinence in women: a systematic review, Br J Urol, 78, 497, 10.1046/j.1464-410X.1996.01422.x · DOI · Paper page
  41. Brosseau, 2002, Low level laser therapy (Classes I, II and III) for treating rheumatoid arthritis, Cochrane Database of Systematic Reviews
  42. Callahan, 1991, Oral corticosteroid therapy for patients with stable chronic obstructive pulmonary disease. A meta-analysis, Ann Intern Med, 114, 216, 10.7326/0003-4819-114-3-216 · DOI · Paper page
  43. Camma, 1996, Interferon as treatment for acute hepatitis C. A meta-analysis, Dig Dis Sci, 41, 1248, 10.1007/BF02088245 · DOI
  44. Chinoy, 1999, Parker MJ. Fixed nail plates versus sliding hip systems for the treatment of trochanteric femoral fractures: a meta-analysis of 14 studies, Injury, 30, 157, 10.1016/S0020-1383(99)00074-1 · DOI
  45. Cook, 1998, Do spermicides containing nonoxynol-9 prevent sexually transmitted infections? A meta-analysis, Sex Transm Dis, 25, 144, 10.1097/00007435-199803000-00007 · DOI
  46. Fouque, 1992, Controlled low protein diets in chronic renal insufficiency: meta- analysis, BMJ, 304, 216, 10.1136/bmj.304.6821.216 · DOI · Paper page
  47. Gifford, 1995, A meta-analysis of infant outcomes after breech delivery, Obstet Gynecol, 85, 1047, 10.1016/0029-7844(95)00053-T · DOI
  48. Glasziou, 1993, Mackerras DE. Vitamin A supplementation in infectious diseases: a meta-analysis, BMJ, 306, 366, 10.1136/bmj.306.6874.366 · DOI · Paper page
  49. Grullon, 1997, The safety of early postpartum discharge: a review and critique, Obstet. Gynecol, 90, 860, 10.1016/S0029-7844(97)00405-5 · DOI
  50. Hart, 2000, Aspirin for the primary prevention of stroke and other major vascular events: meta-analysis and hypotheses, Arch Neurol, 57, 326, 10.1001/archneur.57.3.326 · DOI · Paper page
  51. Hommes, 1992, Subcutaneous heparin compared with continuous intravenous heparin administration in the initial treatment of deep vein thrombosis. A meta-analysis, Ann Intern Med, 116, 279, 10.7326/0003-4819-116-4-279 · DOI · Paper page
  52. Imperiale, 1991, A meta-analysis of low-dose aspirin for the prevention of pregnancy- induced hypertensive disease, JAMA, 266, 260, 10.1001/jama.1991.03470020086038 · DOI · Paper page
  53. Insua, 1994, Drug treatment of hypertension in the elderly: a meta-analysis, Ann Intern Med, 121, 355, 10.7326/0003-4819-121-5-199409010-00008 · DOI · Paper page
  54. Ioannidis, 2001, Accuracy and clinical effect of out-of-hospital electrocardiography in the diagnosis of acute cardiac ischemia: a meta-analysis, Ann Emerg Med, 37, 461, 10.1067/mem.2001.114904 · DOI
  55. Kasiske, 1993, Elective cyclosporine withdrawal after renal transplantation. A meta-analysis, JAMA, 269, 395, 10.1001/jama.1993.03500030093040 · doi:10.1001/jama.1993.03500030093040
  56. Kasiske, 1998, A meta-analysis of the effects of dietary protein restriction on the rate of decline in renal function, Am J Kidney Dis, 31, 954, 10.1053/ajkd.1998.v31.pm9631839 · DOI · Paper page
  57. McAlister, 1998, Perioperative allogeneic blood transfusion does not cause adverse sequelae in patients with cancer: a meta-analysis of unconfounded studies, Br J Surg, 85, 171, 10.1046/j.1365-2168.1998.00698.x · DOI
  58. Patel, 2002, Polysaccharide vaccines for preventing serogroup A meningococcal meningitis, Cochrane Database of Systematic Reviews
  59. Psaty, 1997, Health outcomes associated with antihypertensive therapies used as first-line agents. A systematic review and meta-analysis, JAMA, 277, 739, 10.1001/jama.1997.03540330061036 · DOI · Paper page
  60. Ramsey, 2000, Corticosteroid treatment for idiopathic facial nerve paralysis: a meta- analysis, Laryngoscope, 110, 335, 10.1097/00005537-200003000-00001 · DOI
  61. Shadish, 1992, Do family and marital psychotherapies change what people do? A meta-analysis of behavioural outcomes, 129
  62. Shadish, 1997, Experiments versus quasi-experiments: do they yield the same answer?, 147
  63. Srisurapanont, 2002, Opioid antagonists for alcohol dependence, Cochrane Database of Systematic Reviews, 10.1002/14651858.CD001867 · DOI · Paper page
  64. Tangkanakul, 2002, Local versus general anaesthesia for carotid endarterectomy, Cochrane Database of Systematic Reviews
  65. Nieuwenhoven, 2001, Relationship between methodological trial quality and the effects of selective digestive decontamination on pneumonia and mortality in critically ill patients, JAMA, 286, 335, 10.1001/jama.286.3.335 · DOI · Paper page
  66. Vandekerckhove, 2002, Oil-soluble versus water-soluble media for assessing tubal patency with hysterosalpingography or laparoscopy in subfertile women, Cochrane Database of Systematic Reviews, 10.1002/14651858.CD000092 · DOI
  67. Vandenbroucke-Grauls, 1991, Effect of selective decontamination of the digestive tract on respiratory tract infections and mortality in the intensive care unit, Lancet, 338, 859, 10.1016/0140-6736(91)91510-2 · DOI · Paper page
  68. Watson, 2002, Techniques for pelvic surgery in subfertility, Cochrane Database of Systematic Reviews
  69. Wells, 1994, Graduated compression stockings in the prevention of postoperative venous thromboembolism. A meta-analysis, Arch Intern Med, 154, 67, 10.1001/archinte.1994.00420010095011 · doi:10.1001/archinte.1994.00420010095011
  70. Austin, 1992, The history of malariotherapy for neurosyphilis, JAMA, 268, 516, 10.1001/jama.1992.03490040092031 · DOI
  71. Britton A McKee M Black N McPherson K Sanderson C Bain C Three systematic reviews - not so different answers BMJ Electronic letters
  72. Chalmers, 1997, Assembling comparison groups to assess the effects of health care, J R Soc Med, 90, 379, 10.1177/014107689709000706 · DOI · Paper page
  73. Cochrane, 1972, Effectiveness and Efficiency: Random Reflections on Health Services, 20
  74. Counsell, 1994, The miracle of DICE therapy for acute stroke: fact or fictional product of subgroup analysis?, BMJ, 309, 1677, 10.1136/bmj.309.6970.1677 · DOI
  75. Dans, 2002, Applying the results to individual patients., 369
  76. Dickersin, 1993, NIH clinical trials and publication bias, Online J Curr Clin Trials, Doc No 50, 4967
  77. Dickersin, 1997, How important is publication bias? A synthesis of available data, AIDS Educ Prev, 9, 15
  78. 1985, Failure of extracranial-intracranial arterial bypass to reduce the risk of ischemic stroke. Results of an international randomized trial, N Engl J Med, 313, 1191, 10.1056/NEJM198511073131904 · DOI · Paper page
  79. Echt, 1991, Mortality and morbidity in patients receiving encainide, flecainide, or placebo. The Cardiac Arrhythmia Supression Trial, N Engl J Med, 324, 781, 10.1056/NEJM199103213241201 · DOI · Paper page
  80. 1995, Which anticonvulsant for women with eclampsia?, Lancet, 345, 1455, 10.1016/S0140-6736(95)91034-4 · DOI · Paper page
  81. Egger, 1997, Bias in meta-analysis detected by a simple, graphical test, BMJ, 315, 629, 10.1136/bmj.315.7109.629 · DOI · Paper page
  82. Gubler, 1971, Alternating hemiplegia, a sign of pontine lesion, and documentation of the proof of the facial decussation, 9
  83. Guyatt, 2002, Therapy, 55
  84. Jüni, 2001, Assessing the quality of randomised controlled trials, 10.1002/9780470693926.ch5 · DOI · Paper page
  85. Kleijnen, 1997, So what's so special about randomisation?, 93
  86. Kownacki, 1999, Does Alcoholics Anonymous work? The results from a meta-analysis of controlled experiments, Subst Use Misuse, 34, 1897, 10.3109/10826089909039431 · DOI · Paper page
  87. Kunz R Oxman AD Two systematic reviews - two different answers? BMJ Electronic letters
  88. McKee, 1999, Methods in health services research. Interpreting the evidence: choosing between randomised and non-randomised studies, BMJ, 319, 312, 10.1136/bmj.319.7205.312 · DOI · Paper page
  89. Moore, 1995, Deadly Medicine
  90. 1948, Streptomycin treatment of pulmonary tuberculosis, BMJ, ii, 769
  91. Reeves, 1998, Comparison of effect size estimates derived from randomised and non-randomised studies, 73
  92. Teo, 1993, Effects of prophylactic antiarrhythmic drug therapy in acute myocardial infarction. An overview of results from randomized trials, JAMA, 270, 1589, 10.1001/jama.1993.03510130095038 · doi:10.1001/jama.1993.03510130095038 · Paper page
  93. 1994, Identifying Health technologies that Work:Searching for Evidence, 41
  94. Valenstein, 1986, Great and Desparate Cures: The Rise and Decline of Psychosurgery and Other Radical Treatments for Mental Illness, 338
  95. Weiss, 1998, Evaluation. Methods for Studying Programs and Policies, 229
  96. Kunz, 1998, The unpredictability paradox: review of empirical comparisons of randomised and non-randomised clinical trials, BMJ, 317, 1185, 10.1136/bmj.317.7167.1185 · DOI · Paper page

Cited by

See all 185

Metadata

Authors
Kunz, R., Vist, G. E., & Oxman, A. D.
Venue
Cochrane Database of Systematic Reviews
Published
2007-04-18
Issue
2
Pages
MR000012-MR000012
ISSN
1361-6137 (ISSN-L)
OpenAlex
W4245428103
Cited by
185
References
15
Type
reference entry
Language
en
Open access
closed

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Randomisation to protect against selection bias in healthcare trials. https://app.cochat.ai/papers/10.1002/14651858.mr000012.pub2
The paper
Kunz, R., Vist, G. E., & Oxman, A. D. (2007). Randomisation to protect against selection bias in healthcare trials. Cochrane Database of Systematic Reviews. https://doi.org/10.1002/14651858.mr000012.pub2

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.