The DOI matches its Crossref record; 27 of the 27 references it cites were found in a registry or an index.
The Number of Patients and Events Required to Limit the Risk of Overestimation of Intervention Effects in Meta-Analysis—A Simulation Study
Open accessAbstract
BACKGROUND: Meta-analyses including a limited number of patients and events are prone to yield overestimated intervention effect estimates. While many assume bias is the cause of overestimation, theoretical considerations suggest that random error may be an equal or more frequent cause. The independent impact of random error on meta-analyzed intervention effects has not previously been explored. It has been suggested that surpassing the optimal information size (i.e., the required meta-analysis sample size) provides sufficient protection against overestimation due to random error, but this claim has not yet been validated. METHODS: We simulated a comprehensive array of meta-analysis scenarios where no intervention effect existed (i.e., relative risk reduction (RRR) = 0%) or where a small but possibly unimportant effect existed (RRR = 10%). We constructed different scenarios by varying the control group risk, the degree of heterogeneity, and the distribution of trial sample sizes. For each scenario, we calculated the probability of observing overestimates of RRR>20% and RRR>30% for each cumulative 500 patients and 50 events. We calculated the cumulative number of patients and events required to reduce the probability of overestimation of intervention effect to 10%, 5%, and 1%. We calculated the optimal information size for each of the simulated scenarios and explored whether meta-analyses that surpassed their optimal information size had sufficient protection against overestimation of intervention effects due to random error. RESULTS: The risk of overestimation of intervention effects was usually high when the number of patients and events was small and this risk decreased exponentially over time as the number of patients and events increased. The number of patients and events required to limit the risk of overestimation depended considerably on the underlying simulation settings. Surpassing the optimal information size generally provided sufficient protection against overestimation. CONCLUSIONS: Random errors are a frequent cause of overestimation of intervention effects in meta-analyses. Surpassing the optimal information size will provide sufficient protection against overestimation.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Meta-analysis in clinical trials
DerSimonian, R., & Laird, N. (1986) · Controlled Clinical Trials · doi:10.1016/0197-2456(86)90046-2 · Cited by 39,552 · cited 1×
“We simulated binary meta-analysis data sets using a DerSimonian-Laird random-effects model framework [3,18,19].”
MethodCochrane Handbook for Systematic Reviews of Interventions: Cochrane Book Series
(2008) · doi:10.1002/9780470712184 · Cited by 25,048
GRADE: an emerging consensus on rating quality of evidence and strength of recommendations
Guyatt, G. H., Oxman, A. D., Vist, G. E., Kunz, R., Falck-Ytter, Y., Alonso-Coello, P., & Schünemann, H. J. (2008) · BMJ · doi:10.1136/bmj.39489.470347.ad · Cited by 23,305
Self-citationUsers' guides to the medical literature
Guyatt, G. H. (1993) · JAMA · doi:10.1001/jama.270.17.2096 · Cited by 2,970
Self-citationGRADE guidelines 6. Rating the quality of evidence—imprecision
Guyatt, G. H., Oxman, A. D., Kunz, R., Brozek, J., Alonso-Coello, P., Rind, D., Devereaux, P. J., Montori, V. M., Freyschuss, B., Vist, G., et al. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.01.012 · Cited by 2,693 · cited 1×
“Extending this purpose, authors have previously considered information size requirements as a means of gaining control over the risk of overestimation [2,20].”
BackgroundSelf-citationWood, L., Egger, M., Gluud, L. L., Schulz, K. F., Jüni, P., Altman, D. G., Gluud, C., Martin, R. M., Wood, A. J. G., & Sterne, J. A. C. (2008) · BMJ · doi:10.1136/bmj.39465.451748.ad · Cited by 2,538
Self-citationTrial sequential analysis may establish when firm evidence is reached in cumulative meta-analysis
Wetterslev, J., Thorlund, K., Brok, J., & Gluud, C. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.03.013 · Cited by 1,881
Self-citationBrok, J., Thorlund, K., Gluud, C., & Wetterslev, J. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.10.007 · Cited by 1,113
Self-citationEstimating required information size by quantifying diversity in random-effects model meta-analyses
Wetterslev, J., Thorlund, K., Brok, J., & Gluud, C. (2009) · BMC Medical Research Methodology · doi:10.1186/1471-2288-9-86 · Cited by 938
Self-citationBrok, J., Thorlund, K., Wetterslev, J., & Gluud, C. (2009) · International Journal of Epidemiology · doi:10.1093/ije/dyn188 · Cited by 917
Self-citation
Cited by
Trial Sequential Analysis in systematic reviews with meta-analysis
Wetterslev, J., Jakobsen, J. C., & Gluud, C. (2017) · BMC Medical Research Methodology · doi:10.1186/s12874-017-0315-7 · Cited by 1,189
Self-citationPlasma Exchange and Glucocorticoids in Severe ANCA-Associated Vasculitis
Walsh, M., Merkel, P. A., Peh, C.-A., Szpirt, W. M., Puéchal, X., Fujimoto, S., Hawley, C. M., Khalidi, N., Floßmann, O., Wald, R., et al. (2020) · New England Journal of Medicine · doi:10.1056/nejmoa1803537 · Cited by 893
Self-citationWalsh, M., Srinathan, S. K., McAuley, D. F., Mrkobrada, M., Levine, O., Ribic, C., Molnar, A. O., Dattani, N. D., Burke, A., Guyatt, G., et al. (2014) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2013.10.019 · Cited by 762
Self-citationAkolekar, R., Beta, J., Picciarelli, G., Ogilvie, C., & D'Antonio, F. (2015) · Ultrasound in Obstetrics & Gynecology · doi:10.1002/uog.14636 · Cited by 711
Jakobsen, J. C., Wetterslev, J., Winkel, P., Lange, T., & Gluud, C. (2014) · BMC Medical Research Methodology · doi:10.1186/1471-2288-14-120 · Cited by 704
“Furthermore, meta-analyses with too small accrued information size (sparse data) have an increased risk of either overestimating or underestimating the effect size and variance [25,26,44-46].”
BackgroundSelf-citationVitamin D supplementation for prevention of mortality in adults
Bjelakovic, G., Gluud, L. L., Nikolova, D., Whitfield, K., Wetterslev, J., Simonetti, R. G., Bjelakovic, M., & Gluud, C. (2014) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd007470.pub3 · Cited by 695
Self-citationThe impact of study size on meta-analyses: examination of underpowered studies in Cochrane reviews
Turner, R. M., Bird, S. M., & Higgins, J. P. T. (2013) · PLoS ONE · doi:10.1371/journal.pone.0059202 · Cited by 672
Populations at risk for severe or complicated influenza illness: systematic review and meta-analysis
Mertz, D., Kim, T. H., Johnstone, J., Lam, P.-P., Science, M., Kuster, S. P., Fadel, S. A., Tran, D., Fernandez, E., Bhatnagar, N., et al. (2013) · BMJ · doi:10.1136/bmj.f5061 · Cited by 634
Fluoride toothpastes of different concentrations for preventing dental caries
Walsh, T., Worthington, H. V., Glenny, A.-M., Marinho, V. C., & Jeroncic, A. (2019) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd007868.pub3 · Cited by 620
Power analysis for random-effects meta-analysis
Jackson, D., & Turner, R. (2017) · Research Synthesis Methods · doi:10.1002/jrsm.1240 · Cited by 619
Cannabis-based medicines for chronic neuropathic pain in adults
Mücke, M., Phillips, T., Radbruch, L., Petzke, F., & Häuser, W. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012182.pub2 · Cited by 571
Gabapentin for chronic neuropathic pain in adults
Wiffen, P. J., Derry, S., Bell, R. F., Rice, A. S., Tölle, T. R., Phillips, T., & Moore, R. A. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd007938.pub4 · Cited by 561
Interventions for replacing missing teeth: different times for loading dental implants
Esposito, M., Grusovin, M. G., Maghaireh, H., & Worthington, H. V. (2013) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd003878.pub5 · Cited by 550
Tan, C.-C., Yu, J.-T., Wang, H.-F., Tan, M.-S., Meng, X.-F., Wang, C., Jiang, T., Zhu, X.-C., & Tan, L. (2014) · Journal of Alzheimer s Disease · doi:10.3233/jad-132690 · Cited by 484
Evolution of Heterogeneity (I2) Estimates and Their 95% Confidence Intervals in Large Meta-Analyses
Thorlund, K., Imberger, G., Johnston, B. C., Walsh, M., Awad, T., Thabane, L., Gluud, C., Devereaux, P. J., & Wetterslev, J. (2012) · PLoS ONE · doi:10.1371/journal.pone.0039471 · Cited by 423
Self-citationHemmingsen, B., Lund, S. S., Gluud, C., Vaag, A., Almdal, T., Hemmingsen, C., & Wetterslev, J. (2011) · BMJ · doi:10.1136/bmj.d6898 · Cited by 386
Self-citationJakobsen, J. C., Katakam, K. K., Schou, A., Hellmuth, S. G., Stallknecht, S. E., Leth-Møller, K., Iversen, M., Banke, M. B., Petersen, I. J., Klingenberg, S. L., et al. (2017) · BMC Psychiatry · doi:10.1186/s12888-016-1173-2 · Cited by 366
Self-citationPerioperative intravenous ketamine for acute postoperative pain in adults
Brinck, E., Tiippana, E., Heesen, M., Bell, R. F., Straube, S., Moore, R. A., & Kontinen, V. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012033.pub4 · Cited by 373
Pregabalin for neuropathic pain in adults
Derry, S., Bell, R. F., Straube, S., Wiffen, P. J., Aldington, D., & Moore, R. A. (2019) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd007076.pub3 · Cited by 364
Haase, N., Perner, A., Hennings, L. I., Siegemund, M., Lauridsen, B., Wetterslev, M., & Wetterslev, J. (2013) · BMJ · doi:10.1136/bmj.f839 · Cited by 310
Self-citationImberger, G., Thorlund, K., Gluud, C., & Wetterslev, J. (2016) · BMJ Open · doi:10.1136/bmjopen-2016-011890 · Cited by 277
Self-citationOpioids for cancer pain - an overview of Cochrane reviews
Wiffen, P. J., Wee, B., Derry, S., Bell, R. F., & Moore, R. A. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012592.pub2 · Cited by 271
Wang, H.-F., Yu, J.-T., Tang, S.-W., Jiang, T., Tan, C.-C., Meng, X.-F., Wang, C., Tan, M.-S., & Tan, L. (2015) · Journal of Neurology, Neurosurgery & Psychiatry · doi:10.1136/jnnp-2014-307659 · Cited by 269
Sample size and power considerations in network meta-analysis
Thorlund, K., & Mills, E. J. (2012) · Systematic Reviews · doi:10.1186/2046-4053-1-41 · Cited by 197
Self-citationBjelakovic, G., Nikolova, D., & Gluud, C. (2013) · PLoS ONE · doi:10.1371/journal.pone.0074558 · Cited by 183
Rogliani, P., Calzetta, L., Cavalli, F., Matera, M. G., & Cazzola, M. (2016) · Pulmonary Pharmacology & Therapeutics · doi:10.1016/j.pupt.2016.07.009 · Cited by 168
Castellini, G., Bruschettini, M., Gianola, S., Gluud, C., & Moja, L. (2018) · Systematic Reviews · doi:10.1186/s13643-018-0770-1 · Cited by 161
Self-citationPharmacological interventions for benzodiazepine discontinuation in chronic benzodiazepine users
Baandrup, L., Ebdrup, B. H., Rasmussen, J. Ø., Lindschou, J., Gluud, C., & Glenthøj, B. Y. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011481.pub2 · Cited by 128
Self-citationZhao, C., Lai, L., Zhang, L., Cai, Z., Ren, Z., Shi, C., Luo, W., & Yan, Y. (2021) · Journal of Psychosomatic Research · doi:10.1016/j.jpsychores.2020.110304 · Cited by 104
Calculating statistical power for meta-analysis using metapower
Griffin, J. W. (2021) · The Quantitative Methods for Psychology · doi:10.20982/tqmp.17.1.p024 · Cited by 84
“…meta-analysis is also critical because underpowered meta-analyses lack precision in estimating a summary effect size - especially when individual studies vary considerably - which can lead to conclusions that are incorrect (Thorlund et al., 2011; Pigott & Polanin, 2020; Jackson & Turner, 2017).”
BackgroundControl interventions in randomised trials among people with mental health disorders
Faltinsen, E., Todorovac, A., Staxen Bruun, L., Hróbjartsson, A., Gluud, C., Kongerslev, M. T., Simonsen, E., & Storebø, O. J. (2022) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000050.pub2 · Cited by 40
Self-citationMcGill, K., Sackley, C., Godwin, J., Gavaghan, D., Ali, M., Ballester, B. R., Brady, M. C., Brady, M. C., Ali, M., Ashburn, A., et al. (2022) · Journal of Stroke and Cerebrovascular Diseases · doi:10.1016/j.jstrokecerebrovasdis.2021.106229 · Cited by 14
Placebo, usual care and wait-list interventions for all mental health disorders
Faltinsen, E., Todorovac, A., Hróbjartsson, A., Gluud, C., Kongerslev, M. T., Simonsen, E., & Storebø, O. J. (2019) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000050 · Cited by 9
Self-citation
Metadata
- Authors
- Thorlund, K., Imberger, G., Walsh, M., Chu, R., Gluud, C., Wetterslev, J., Guyatt, G., Devereaux, P. J., & Thabane, L.
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2011-10-18
- Volume
- 6
- Issue
- 10
- Pages
- e25491
- ISSN
- 1932-6203 (ISSN-L)
- OpenAlex
- W2072380646
- Cited by
- 319
- References
- 27
- FWCI
- 22.96
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.