The DOI matches its Crossref record; 27 of the 27 references it cites were found in a registry or an index.
The Number of Patients and Events Required to Limit the Risk of Overestimation of Intervention Effects in Meta-Analysis—A Simulation Study
Open accessAbstract
BACKGROUND: Meta-analyses including a limited number of patients and events are prone to yield overestimated intervention effect estimates. While many assume bias is the cause of overestimation, theoretical considerations suggest that random error may be an equal or more frequent cause. The independent impact of random error on meta-analyzed intervention effects has not previously been explored. It has been suggested that surpassing the optimal information size (i.e., the required meta-analysis sample size) provides sufficient protection against overestimation due to random error, but this claim has not yet been validated. METHODS: We simulated a comprehensive array of meta-analysis scenarios where no intervention effect existed (i.e., relative risk reduction (RRR) = 0%) or where a small but possibly unimportant effect existed (RRR = 10%). We constructed different scenarios by varying the control group risk, the degree of heterogeneity, and the distribution of trial sample sizes. For each scenario, we calculated the probability of observing overestimates of RRR>20% and RRR>30% for each cumulative 500 patients and 50 events. We calculated the cumulative number of patients and events required to reduce the probability of overestimation of intervention effect to 10%, 5%, and 1%. We calculated the optimal information size for each of the simulated scenarios and explored whether meta-analyses that surpassed their optimal information size had sufficient protection against overestimation of intervention effects due to random error. RESULTS: The risk of overestimation of intervention effects was usually high when the number of patients and events was small and this risk decreased exponentially over time as the number of patients and events increased. The number of patients and events required to limit the risk of overestimation depended considerably on the underlying simulation settings. Surpassing the optimal information size generally provided sufficient protection against overestimation. CONCLUSIONS: Random errors are a frequent cause of overestimation of intervention effects in meta-analyses. Surpassing the optimal information size will provide sufficient protection against overestimation.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Meta-analysis in clinical trials
DerSimonian, R., & Laird, N. (1986) · Controlled Clinical Trials · doi:10.1016/0197-2456(86)90046-2 · Cited by 39,552 · cited 1×
“We simulated binary meta-analysis data sets using a DerSimonian-Laird random-effects model framework [3,18,19].”
MethodCochrane Handbook for Systematic Reviews of Interventions: Cochrane Book Series
(2008) · doi:10.1002/9780470712184 · Cited by 25,048
GRADE: an emerging consensus on rating quality of evidence and strength of recommendations
Guyatt, G. H., Oxman, A. D., Vist, G. E., Kunz, R., Falck-Ytter, Y., Alonso-Coello, P., & Schünemann, H. J. (2008) · BMJ · doi:10.1136/bmj.39489.470347.ad · Cited by 23,305
Self-citationUsers' guides to the medical literature
Guyatt, G. H. (1993) · JAMA · doi:10.1001/jama.270.17.2096 · Cited by 2,970
Self-citationGRADE guidelines 6. Rating the quality of evidence—imprecision
Guyatt, G. H., Oxman, A. D., Kunz, R., Brozek, J., Alonso-Coello, P., Rind, D., Devereaux, P. J., Montori, V. M., Freyschuss, B., Vist, G., et al. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.01.012 · Cited by 2,693
Self-citationWood, L., Egger, M., Gluud, L. L., Schulz, K. F., Jüni, P., Altman, D. G., Gluud, C., Martin, R. M., Wood, A. J. G., & Sterne, J. A. C. (2008) · BMJ · doi:10.1136/bmj.39465.451748.ad · Cited by 2,538
Self-citationTrial sequential analysis may establish when firm evidence is reached in cumulative meta-analysis
Wetterslev, J., Thorlund, K., Brok, J., & Gluud, C. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.03.013 · Cited by 1,881
Self-citationBrok, J., Thorlund, K., Gluud, C., & Wetterslev, J. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.10.007 · Cited by 1,113
Self-citationEstimating required information size by quantifying diversity in random-effects model meta-analyses
Wetterslev, J., Thorlund, K., Brok, J., & Gluud, C. (2009) · BMC Medical Research Methodology · doi:10.1186/1471-2288-9-86 · Cited by 938
Self-citationBrok, J., Thorlund, K., Wetterslev, J., & Gluud, C. (2009) · International Journal of Epidemiology · doi:10.1093/ije/dyn188 · Cited by 917
Self-citationCan trial sequential monitoring boundaries reduce spurious inferences from meta-analyses?
Thorlund, K., Devereaux, P. J., Wetterslev, J., Guyatt, G., Ioannidis, J. P. A., Thabane, L., Gluud, L.-L., Als-Nielsen, B., & Gluud, C. (2009) · International Journal of Epidemiology · doi:10.1093/ije/dyn179 · Cited by 833
Self-citationGuyatt, G. H., Haynes, R. B., Jaeschke, R. Z., Cook, D. J., Green, L., Naylor, C. D., Wilson, M. C., Richardson, W. S., & for the Evidence-Based Medicine Working Group (2000) · JAMA · doi:10.1001/jama.284.10.1290 · Cited by 748
Self-citationDevereaux, P. J., Beattie, W. S., Choi, P. T.-L., Badner, N. H., Guyatt, G. H., Villar, J. C., Cinà, C. S., Leslie, K., Jacka, M. J., Montori, V. M., et al. (2005) · BMJ · doi:10.1136/bmj.38503.623646.8f · Cited by 438
Self-citationOvercoming the limitations of current meta-analysis of randomised controlled trials
Pogue, J., & Yusuf, S. (1998) · The Lancet · doi:10.1016/s0140-6736(97)08461-4 · Cited by 428
Pogue, J. M., & Yusuf, S. (1997) · Controlled Clinical Trials · doi:10.1016/s0197-2456(97)00051-2 · Cited by 415
Meta-analysis of randomised controlled trials
Kelly, C. P. (1997) · The Lancet · doi:10.1016/s0140-6736(05)63834-2 · Cited by 382
Cumulative meta-analysis of clinical trials builds evidence for exemplary medical care
Lau, J., Schmid, C. H., & Chalmers, T. C. (1995) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(94)00106-z · Cited by 356
Sequential methods for random‐effects meta‐analysis
Higgins, J. P. T., Whitehead, A., & Simmonds, M. (2011) · Statistics in Medicine · doi:10.1002/sim.4088 · Cited by 258
Simple Heterogeneity Variance Estimation for Meta-Analysis
Sidik, K., & Jonkman, J. N. (2005) · Journal of the Royal Statistical Society Series C: Applied Statistics · doi:10.1111/j.1467-9876.2005.00489.x · Cited by 253
Evolution of treatment effects over time: Empirical insight from recursive cumulative metaanalyses
Ioannidis, J. P. A. (2001) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.021529998 · Cited by 192
Effect sizes in cumulative meta-analyses of mental health randomized trials evolved over time
Trikalinos, T. A., Churchill, R., Ferri, M., Leucht, S., Tuunainen, A., Wahlbeck, K., & Ioannidis, J. P. A. (2004) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2004.02.018 · Cited by 177
Pereira, T. V., & Ioannidis, J. P. A. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.12.012 · Cited by 160 · cited 1×
“As demonstrated in empirical studies, early large intervention effects are likely to dissipate and early statistically significant meta-analyses are likely to be false positives [4– 6,12,21].”
Updating meta-analyses leads to larger type I errors than publication bias
Borm, G. F., & Donders, A. R. T. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2008.08.010 · Cited by 112
The fading of reported effectiveness. A meta-analysis of randomised controlled trials
Gehr, B., Weiß, C., & Porzsolt, F. (2006) · BMC Medical Research Methodology · doi:10.1186/1471-2288-6-25 · Cited by 90
Uncertainty of the time of first significance in random effects cumulative meta-analysis
Berkey, C. S., Mosteller, F. C., Lau, J., & Antman, E. M. (1996) · Controlled Clinical Trials · doi:10.1016/s0197-2456(96)00014-1 · Cited by 80
Sequential meta-analysis: an efficient decision-making tool
van der Tweel, I., & Bollen, C. W. (2010) · Clinical Trials · doi:10.1177/1740774509360994 · Cited by 52
“Fading of Reported Effectiveness” Bias: Longitudinal Meta-Analysis of Randomized Controlled Trials
Gehr, B., Weiß, C., & Porzsolt, F. (2007) · Springer eBooks · doi:10.1007/978-0-387-33921-4_25 · Cited by 2
Cited by
Trial Sequential Analysis in systematic reviews with meta-analysis
Wetterslev, J., Jakobsen, J. C., & Gluud, C. (2017) · BMC Medical Research Methodology · doi:10.1186/s12874-017-0315-7 · Cited by 1,189
Self-citationPlasma Exchange and Glucocorticoids in Severe ANCA-Associated Vasculitis
Walsh, M., Merkel, P. A., Peh, C.-A., Szpirt, W. M., Puéchal, X., Fujimoto, S., Hawley, C. M., Khalidi, N., Floßmann, O., Wald, R., et al. (2020) · New England Journal of Medicine · doi:10.1056/nejmoa1803537 · Cited by 893
Self-citationWalsh, M., Srinathan, S. K., McAuley, D. F., Mrkobrada, M., Levine, O., Ribic, C., Molnar, A. O., Dattani, N. D., Burke, A., Guyatt, G., et al. (2014) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2013.10.019 · Cited by 762
Self-citationAkolekar, R., Beta, J., Picciarelli, G., Ogilvie, C., & D'Antonio, F. (2015) · Ultrasound in Obstetrics & Gynecology · doi:10.1002/uog.14636 · Cited by 711
Jakobsen, J. C., Wetterslev, J., Winkel, P., Lange, T., & Gluud, C. (2014) · BMC Medical Research Methodology · doi:10.1186/1471-2288-14-120 · Cited by 704
“Furthermore, meta-analyses with too small accrued information size (sparse data) have an increased risk of either overestimating or underestimating the effect size and variance [25,26,44-46].”
BackgroundSelf-citation
Metadata
- Authors
- Thorlund, K., Imberger, G., Walsh, M., Chu, R., Gluud, C., Wetterslev, J., Guyatt, G., Devereaux, P. J., & Thabane, L.
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2011-10-18
- Volume
- 6
- Issue
- 10
- Pages
- e25491
- ISSN
- 1932-6203 (ISSN-L)
- OpenAlex
- W2072380646
- Cited by
- 319
- References
- 27
- FWCI
- 22.96
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.