Meta-analyses of safety data: a comparison of exact versus asymptotic methods
The DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.
Abstract
The objectives of this study were to establish and describe a database of Cochrane and non-Cochrane meta-analyses of safety data and to determine under what conditions exact methods differ from asymptotic methods in meta-analyses of safety data. A sample of Cochrane ( n = 500) and non-Cochrane ( n = 200) systematic reviews was randomly selected and a database of safety meta-analyses established. Point estimates and confidence intervals for each meta-analysis were recalculated using exact methods and compared to the results of asymptotic methods. Cochrane reviews were nearly four times as likely as non-Cochrane reviews to contain meta-analyses of safety data (35% compared to 9%). More than 50% of safety meta-analyses contained an outcome with a rare event rate ( <5%) and 30% contained at least one study with no events in one arm of the study. For rare event meta-analyses, exact point estimates differed substantially from asymptotic estimates 46% of the time, compared to 17% for those without rare events. Exact confidence intervals differed substantially from asymptotic ones 67% of the time compared to only 19% for those without rare events. The magnitude of differences was also correlated with the number of studies and the summary statistic used to combine the data. Asymptotic methods will not always be a good approximation for exact methods in safety meta-analyses. Event rates and number of studies should be closely examined when choosing the statistical method for combining rare event data.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The number needed to treat: a clinically useful measure of treatment effect
Cook, R. J., & Sackett, D. L. (1995) · BMJ · doi:10.1136/bmj.310.6977.452 · Cited by 1,885
What to add to nothing? Use and avoidance of continuity corrections in meta‐analysis of sparse data
J. Sweeting, M., J. Sutton, A., & C. Lambert, P. (2004) · Statistics in Medicine · doi:10.1002/sim.1761 · Cited by 1,734
Much ado about nothing: a comparison of the performance of meta-analytical methods with rare events
Bradburn, M. J., Deeks, J. J., Berlin, J. A., & Russell Localio, A. (2007) · Statistics in Medicine · doi:10.1002/sim.2528 · Cited by 938
Deeks, J. J. (2002) · Statistics in Medicine · doi:10.1002/sim.1188 · Cited by 660
Bayesian approaches to random-effects meta-analysis: a comparative study
Smith, T. C., Spiegelhalter, D. J., & Thomas, A. (1995) · Statistics in Medicine · doi:10.1002/sim.4780142408 · Cited by 568
Completeness of Safety Reporting in Randomized Trials: An Evaluation of 7 Medical Areas
Ioannidis, J. P. A., & Lau, J. (2001) · JAMA · doi:10.1001/jama.285.4.437 · Cited by 3
Cited by
Practical guide to the meta-analysis of rare events
Efthimiou, O. (2018) · Evidence Based Mental Health · doi:10.1136/eb-2018-102911 · Cited by 231
Quality of reporting in systematic reviews of adverse events: systematic review
Zorzela, L., Golder, S., Liu, Y., Pilkington, K., Hartling, L., Joffe, A., Loke, Y., & Vohra, S. (2014) · BMJ · doi:10.1136/bmj.f7668 · Cited by 217
Self-citationKuss, O. (2015) · Statistics in Medicine · doi:10.1002/sim.6383 · Cited by 206
“[3] found that 30% of meta-analyses from a random sample of 500 Cochrane reviews contained at least one single-zero study.”
BackgroundEvans, R., White, J., Turley, R., Slater, T., Morgan, H., Strange, H., & Scourfield, J. (2017) · Children and Youth Services Review · doi:10.1016/j.childyouth.2017.09.020 · Cited by 180
A proposed framework to guide evidence synthesis practice for meta-analysis with zero-events studies
Xu, C., Furuya-Kanamori, L., Zorzela, L., Lin, L., & Vohra, S. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.02.012 · Cited by 117
Metadata
- Authors
- Vandermeer, B., Bialy, L., Hooton, N., Hartling, L., Klassen, T. P., Johnston, B. C., & Wiebe, N.
- Venue
- Statistical Methods in Medical Research
- Publisher
- SAGE Publications
- Published
- 2008-06-18
- Volume
- 18
- Issue
- 4
- Pages
- 421-432
- ISSN
- 0962-2802 (ISSN-L)
- 1477-0334
- OpenAlex
- W2009966581
- Cited by
- 24
- References
- 6
- FWCI
- 0.75
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Meta-analyses of safety data: a comparison of exact versus asymptotic methods. https://app.cochat.ai/papers/10.1177/0962280208092559
- The paper
- Vandermeer, B., Bialy, L., Hooton, N., Hartling, L., Klassen, T. P., Johnston, B. C., & Wiebe, N. (2009). Meta-analyses of safety data: a comparison of exact versus asymptotic methods. Statistical Methods in Medical Research. https://doi.org/10.1177/0962280208092559
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.