CoChat
Sign Up
Cited by 939Meta-analysis and systematic reviewsStatistical Methods in Clinical TrialsStatistical Methods in Epidemiology

Much ado about nothing: a comparison of the performance of meta-analytical methods with rare events

Bradburn, Michael J., Deeks, Jonathan J., Berlin, Jesse A., Russell Localio, A.·2007·Statistics in Medicine
DOI 10.1002/sim.2528
Open this paper in CoChat — free account

Abstract

For rare outcomes, meta-analysis of randomized trials may be the only way to obtain reliable evidence of the effects of healthcare interventions. However, many methods of meta-analysis are based on large sample approximations, and may be unsuitable when events are rare. Through simulation, we evaluated the performance of 12 methods for pooling rare events, considering estimability, bias, coverage and statistical power. Simulations were based on data sets from three case studies with between five and 19 trials, using baseline event rates between 0.1 and 10 per cent and risk ratios of 1, 0.75, 0.5 and 0.2. We found that most of the commonly used meta-analytical methods were biased when data were sparse. The bias was greatest in inverse variance and DerSimonian and Laird odds ratio and risk difference methods, and the Mantel-Haenszel (MH) odds ratio method using a 0.5 zero-cell correction. Risk difference meta-analytical methods tended to show conservative confidence interval coverage and low statistical power at low event rates. At event rates below 1 per cent the Peto one-step odds ratio method was the least biased and most powerful method, and provided the best confidence interval coverage, provided there was no substantial imbalance between treatment and control group sizes within trials, and treatment effects were not exceptionally large. In other circumstances the MH OR without zero-cell corrections, logistic regression and the exact method performed similarly to each other, and were less biased than the Peto method.

Integrity signals

Registry

Crossref · 10.1002/sim.2528

Cites

22 of 22 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Certain common effect approaches have practical advantages over random-effects methods, such as being easier to implement, and the one-step 'Peto' method has desirable properties when binary events are rare.

    “Furthermore, certain common effect approaches have practical advantages over random-effects methods. For example, they are easier to implement, and the one-step ‘Peto’ method has been observed to have desirable properties when binary events are rare (Bradburn et al., 2006).”

    From A Re-Evaluation of Random-Effects Meta-Analysis (2009) · cites this paper as Bradburn et al., 2006

Cites

Flagged first, then by citations

1 of the 23 references couldn’t be matched to a known paper.

Cited by

Sorted by citations

Related

Authors
Bradburn, M. J., Deeks, J. J., Berlin, J. A., & Russell Localio, A.
Affiliations
  • University of Oxford (GB)
  • University of Pennsylvania (US)
Corresponding author
Deeks, Jonathan J.
ORCID
Venue
Statistics in Medicine
Publisher
Wiley
Published
2006-04-04
Volume
26
Issue
1
Pages
53-77
ISSN
  • 0277-6715 (ISSN-L)
  • 1097-0258
DOI
10.1002/sim.2528
OpenAlex
W2128076015
Cited by
938
References
23
Citations by year
2026: 37 · 2025: 28 · 2024: 42 · 2023: 36 · 2022: 58 · 2021: 66 · 2020: 59 · 2019: 58 · 2018: 55 · 2017: 49 · 2016: 62 · 2015: 36 · 2014: 65 · 2013: 45 · 2012: 53
FWCI
39.22
Type
journal article
Language
en
Open access
closed
Licence
Locations
  • Statistics in Medicine (publishedVersion)
  • PubMed (publishedVersion)
Funders
  • Cancer Research UK
  • U.K. Department of Health Senior Research Scientist in Evidence Synthesis award
  • Agency for Healthcare Research and Quality (AHRQ) Centers for Education and Research on Therapeutics (U18 HS10399)
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
Keywords
rare events, odds ratio, risk difference, Mantel-Haenszel estimator, confidence interval coverage, statistical power, estimation bias, data sparsity, exact algorithms, logistic regression
MeSH terms
Analysis of Variance, Mental Disorders, Biometry, Computer Simulation, Female, Humans, Infant Mortality, Infant, Newborn, Infant, Postmature, Pregnancy, Risk, Safety, Meta-Analysis as Topic, Confidence Intervals, Logistic Models, Odds Ratio, Randomized Controlled Trials as Topic, Databases, Factual, Drug-Related Side Effects and Adverse Reactions

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review