The Impact of Study Size on Meta-analyses: Examination of Underpowered Studies in Cochrane Reviews
Open accessThe DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Most meta-analyses include data from one or more small studies that, individually, do not have power to detect an intervention effect. The relative influence of adequately powered and underpowered studies in published meta-analyses has not previously been explored. We examine the distribution of power available in studies within meta-analyses published in Cochrane reviews, and investigate the impact of underpowered studies on meta-analysis results. METHODS AND FINDINGS: For 14,886 meta-analyses of binary outcomes from 1,991 Cochrane reviews, we calculated power per study within each meta-analysis. We defined adequate power as ≥50% power to detect a 30% relative risk reduction. In a subset of 1,107 meta-analyses including 5 or more studies with at least two adequately powered and at least one underpowered, results were compared with and without underpowered studies. In 10,492 (70%) of 14,886 meta-analyses, all included studies were underpowered; only 2,588 (17%) included at least two adequately powered studies. 34% of the meta-analyses themselves were adequately powered. The median of summary relative risks was 0.75 across all meta-analyses (inter-quartile range 0.55 to 0.89). In the subset examined, odds ratios in underpowered studies were 15% lower (95% CI 11% to 18%, P<0.0001) than in adequately powered studies, in meta-analyses of controlled pharmacological trials; and 12% lower (95% CI 7% to 17%, P<0.0001) in meta-analyses of controlled non-pharmacological trials. The standard error of the intervention effect increased by a median of 11% (inter-quartile range -1% to 35%) when underpowered studies were omitted; and between-study heterogeneity tended to decrease. CONCLUSIONS: When at least two adequately powered studies are available in meta-analyses reported by Cochrane reviews, underpowered studies often contribute little information, and could be left out if a rapid review of the evidence is required. However, underpowered studies made up the entirety of the evidence in most Cochrane reviews.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“we pooled results excluding studies with fewer than 50 participants in the experimental group, as a planned sensitivity analysis”
From Effectiveness of psychological interventions in prison to reduce recidivism: a systematic review and meta-analysis of randomised controlled trials (2021) · cites this paper as [66]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“there is a risk that untested approaches may introduce new forms of bias 18 .”
From Software support for environmental evidence synthesis (2018) · cites this paper as [18]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“On collating the evidence, some studies may appear underpowered for the outcome(s) of interest to us (Turner 2013), yet were adequately powered for the studies' primary measure.”
From Physical activity and exercise for chronic pain in adults: an overview of Cochrane Reviews (2017) · cites this paper as Turner 2013
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Although our number of effect sizes (n=37) is relatively large in terms of outcomes analyzed, it is small in terms of statistical analyses that combine empirical studies [100].”
From Generative AI Mental Health Chatbots as Therapeutic Tools: Systematic Review and Meta-Analysis of Their Role in Reducing Mental Health Issues (2025) · cites this paper as [100]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Meta-analysis in clinical trials
DerSimonian, R., & Laird, N. (1986) · Controlled Clinical Trials · doi:10.1016/0197-2456(86)90046-2 · Cited by 39,552 · cited 2×
“In each meta-analysis j, we fitted a random-effects model, using a method-of-moments estimate for the between study variance [17], and calculated the variance Vj of the combined intervention effect (on the log relative risk scale).”
MethodSterne, J. A. C., Sutton, A. J., Ioannidis, J. P. A., Terrin, N., Jones, D. R., Lau, J., Carpenter, J., Rucker, G., Harbord, R. M., Schmid, C. H., et al. (2011) · BMJ · doi:10.1136/bmj.d4002 · Cited by 7,709 · cited 1×
“[24], who published new guidelines.”
Self-citationKjaergard, L. L., Villumsen, J., & Gluud, C. (2001) · Annals of Internal Medicine · doi:10.7326/0003-4819-135-11-200112040-00010 · Cited by 2,283 · cited 1×
“Alternatively, small-study effects may arise from biases caused by methodological flaws arising more frequently in small studies [3], or may be due to true differences in the underlying effects between smaller and larger studies.”
BackgroundPublication and related bias in meta-analysis
Sterne, J. A. C., Gavaghan, D., & Egger, M. (2000) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(00)00242-0 · Cited by 2,282
Trial sequential analysis may establish when firm evidence is reached in cumulative meta-analysis
Wetterslev, J., Thorlund, K., Brok, J., & Gluud, C. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.03.013 · Cited by 1,881
Brok, J., Thorlund, K., Gluud, C., & Wetterslev, J. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.10.007 · Cited by 1,113 · cited 1×
“[20], who had, however, excluded both reviews with fewer than three trials and those in which all trials had a high risk of bias, in which metaanalytic size is likely to have been smaller.”
ResultTotal Plasma Homocysteine and Cardiovascular Risk Profile: The Hordaland Homocysteine Study
Nygård, O., Vollset, S. E., Refsum, H., Stensvold, I., Tverdal, A., Nordrehaug, J. E., Ueland, P. M., & Kvåle, G. (1995) · JAMA · doi:10.1001/jama.1995.03530190040032 · Cited by 1,042
The scandal of poor medical research.
Altman, D. G. (1994) · BMJ · doi:10.1136/bmj.308.6924.283 · Cited by 897 · cited 1×
“Researchers who choose to undertake a study that is capable of detecting only an unrealistically large effect may lack understanding of both scientific methods and ethics [10].”
BackgroundThe power of statistical tests in meta-analysis
Hedges, L. V., & Pigott, T. D. (2001) · Psychological Methods · doi:10.1037/1082-989x.6.3.203 · Cited by 716 · cited 1×
“where W is the cumulative standard normal distribution function and Ca=2~W {1 1{a=2 ð Þ[18].”
BackgroundMeta‐analysis of drug‐related deaths soon after release from prison
Merrall, E. L. C., Kariminia, A., Binswanger, I. A., Hobbs, M. S., Farrell, M., Marsden, J., Hutchinson, S. J., & Bird, S. M. (2010) · Addiction · doi:10.1111/j.1360-0443.2010.02990.x · Cited by 674 · cited 1×
“In teaching, Bird has long advocated that trials should not be started unless they could deliver at least 50% power in respect of a priori plausible, worthwhile effect sizes [6].”
BackgroundSelf-citation
Cited by
Physical activity and exercise for chronic pain in adults: an overview of Cochrane Reviews
Geneen, L. J., Moore, R. A., Clarke, C., Martin, D., Colvin, L. A., & Smith, B. H. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011279.pub3 · Cited by 1,222
Trial Sequential Analysis in systematic reviews with meta-analysis
Wetterslev, J., Jakobsen, J. C., & Gluud, C. (2017) · BMC Medical Research Methodology · doi:10.1186/s12874-017-0315-7 · Cited by 1,189
“Only 22% of the metaanalyses in The Cochrane Library have 80% power to conclude whether there is an intervention effect of 30% or not when the usual maximal risks of type I error (α) of 5% and type II error (β) of 20% are applied [1].”
BackgroundInfluentialYe, Q., Zou, B., Yeo, Y. H., Li, J., Huang, D. Q., Wu, Y., Yang, H., Liu, C., Kam, L. Y., Tan, X. X. E., et al. (2020) · The Lancet Gastroenterology & Hepatology · doi:10.1016/s2468-1253(20)30077-7 · Cited by 925
Physical activity and exercise for chronic pain in adults: an overview of Cochrane Reviews
Geneen, L. J., Moore, R. A., Clarke, C., Martin, D., Colvin, L. A., & Smith, B. H. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011279.pub2 · Cited by 719
Jakobsen, J. C., Wetterslev, J., Winkel, P., Lange, T., & Gluud, C. (2014) · BMC Medical Research Methodology · doi:10.1186/1471-2288-14-120 · Cited by 704
“Most review authors do not assess if an accrued information size is sufficient or not to detect or reject a given intervention effect, and this is problematic as nearly all meta-analyses in systematic reviews are underpowered (the meta-analyses do not have enough information to reject or accept the null hypothesis whe…”
BackgroundThe Power of Bias in Economics Research
Ioannidis, J. P. A., Stanley, T. D., & Doucouliagos, H. (2017) · The Economic Journal · doi:10.1111/ecoj.12461 · Cited by 634
“Among 14,886 meta-analyses in the Cochrane Database of Systematic Reviews of medical research, Turner et al. (2013) find that 86% do not contain a single study with adequate power (≥80”
BackgroundMethodPower analysis for random-effects meta-analysis
Jackson, D., & Turner, R. (2017) · Research Synthesis Methods · doi:10.1002/jrsm.1240 · Cited by 619
“We will compare the power of the individual studies and the power of the meta-analysis to which they contribute, because both types of power are important considerations, which impact on each other in practice.(5,6) The planning of future studies may be based on the power or precision of meta-analysis results.”
MethodSelf-citationCannabis-based medicines for chronic neuropathic pain in adults
Mücke, M., Phillips, T., Radbruch, L., Petzke, F., & Häuser, W. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012182.pub2 · Cited by 571
Gabapentin for chronic neuropathic pain in adults
Wiffen, P. J., Derry, S., Bell, R. F., Rice, A. S., Tölle, T. R., Phillips, T., & Moore, R. A. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd007938.pub4 · Cited by 561
P300 Development across the Lifespan: A Systematic Review and Meta-Analysis
van Dinteren, R., Arns, M., Jongsma, M. L. A., & Kessels, R. P. C. (2014) · PLoS ONE · doi:10.1371/journal.pone.0087347 · Cited by 478
Holst, L. B., Petersen, M. W., Haase, N., Perner, A., & Wetterslev, J. (2015) · BMJ · doi:10.1136/bmj.h1354 · Cited by 426
Iozzino, L., Ferrari, C., Large, M., Nielssen, O., & de Girolamo, G. (2015) · PLOS ONE · doi:10.1371/journal.pone.0128536 · Cited by 401
“The study size effect was evaluated through the computation of power per study as defined by Turner and colleagues [22] (i.”
Pregabalin for neuropathic pain in adults
Derry, S., Bell, R. F., Straube, S., Wiffen, P. J., Aldington, D., & Moore, R. A. (2019) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd007076.pub3 · Cited by 364
Perioperative intravenous ketamine for acute postoperative pain in adults
Brinck, E., Tiippana, E., Heesen, M., Bell, R. F., Straube, S., Moore, R. A., & Kontinen, V. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012033.pub4 · Cited by 373
Quansah, R., Armah, F. A., Essumang, D. K., Luginaah, I., Clarke, E., Marfoh, K., Cobbina, S. J., Nketiah-Amponsah, E., Namujju, P. B., Obiri, S., et al. (2015) · Environmental Health Perspectives · doi:10.1289/ehp.1307894 · Cited by 349
Opioids for cancer pain - an overview of Cochrane reviews
Wiffen, P. J., Wee, B., Derry, S., Bell, R. F., & Moore, R. A. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012592.pub2 · Cited by 271
Risk factors for child sexual abuse victimization: A meta-analytic review.
Assink, M., van der Put, C. E., Meeuwsen, M. W. C. M., de Jong, N. M., Oort, F. J., Stams, G. J. J. M., & Hoeve, M. (2019) · Psychological bulletin · doi:10.1037/bul0000188 · Cited by 322
Small studies are more heterogeneous than large ones: a meta-meta-analysis
IntHout, J., Ioannidis, J. P. A., Borm, G. F., & Goeman, J. J. (2015) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.03.017 · Cited by 314
“of studies with less than 50% power to detect a 30% relative risk reduction [2].”
BackgroundInfluentialLying takes time: A meta-analysis on reaction time measures of deception.
Suchotzki, K., Verschuere, B., Van Bockstaele, B., Ben-Shakhar, G., & Crombez, G. (2017) · Psychological Bulletin · doi:10.1037/bul0000087 · Cited by 279
Hemmingsen, B., Lund, S. S., Gluud, C., Vaag, A., Almdal, T. P., & Wetterslev, J. (2015) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd008143.pub4 · Cited by 291
Imberger, G., Thorlund, K., Gluud, C., & Wetterslev, J. (2016) · BMJ Open · doi:10.1136/bmjopen-2016-011890 · Cited by 277
Chaboyer, W. P., Thalib, L., Harbeck, E. L., Coyer, F. M., Blot, S., Bull, C. F., Nogueira, P. C., & Lin, F. F. (2018) · Critical Care Medicine · doi:10.1097/ccm.0000000000003366 · Cited by 272
Topical analgesics for acute and chronic pain in adults - an overview of Cochrane Reviews
Derry, S., Wiffen, P. J., Kalso, E. A., Bell, R. F., Aldington, D., Phillips, T., Gaskell, H., & Moore, R. A. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd008609.pub2 · Cited by 243
Trial sequential analysis: novel approach for meta-analysis
Kang, H. (2021) · Anesthesia and Pain Medicine · doi:10.17085/apm.21038 · Cited by 212
“leading to spurious positive results [9].”
Vitamin D supplementation for prevention of cancer in adults
Bjelakovic, G., Gluud, L. L., Nikolova, D., Whitfield, K., Krstic, G., Wetterslev, J., & Gluud, C. (2014) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd007469.pub2 · Cited by 212
Fong Yan, A., Cobley, S., Chan, C., Pappas, E., Nicholson, L. L., Ward, R. E., Murdoch, R. E., Gu, Y., Trevor, B. L., Vassallo, A. J., et al. (2018) · Sports Medicine · doi:10.1007/s40279-017-0853-5 · Cited by 197
“and particularly where they contained low sample sizes [75].”
BackgroundZimmermann, F. M., Omerovic, E., Fournier, S., Kelbæk, H., Johnson, N. P., Rothenbühler, M., Xaplanteris, P., Abdel-Wahab, M., Barbato, E., Høfsten, D. E., et al. (2019) · European Heart Journal · doi:10.1093/eurheartj/ehy812 · Cited by 209
“The power of our meta-analysis to detect a 30% relative risk reduction in the primary composite outcome was calculated as described by Turner et al.(19) Betweentrial heterogeneity was considered to be low if the between-trial variance s(2) was 0.”
MethodInfluentialMartin-Doyle, W., Leow, J. J., Orsola, A., Chang, S. L., & Bellmunt, J. (2015) · Journal of Clinical Oncology · doi:10.1200/jco.2014.57.6967 · Cited by 207
Tramadol for neuropathic pain in adults
Duehmke, R. M., Derry, S., Wiffen, P. J., Bell, R. F., Aldington, D., & Moore, R. A. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd003726.pub4 · Cited by 191
Hemmingsen, B., Lund, S. S., Gluud, C., Vaag, A., Almdal, T. P., & Wetterslev, J. (2013) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd008143.pub3 · Cited by 203
RetractedJakobsen, J. C., Gluud, C., Winkel, P., Lange, T., & Wetterslev, J. (2014) · BMC Medical Research Methodology · doi:10.1186/1471-2288-14-34 · Cited by 203
“The majority of randomised clinical trials have difficulties in obtaining the stipulated sample size [10,31,32].”
BackgroundFlexible endoscopic treatment for Zenker's diverticulum: a systematic review and meta-analysis
Ishaq, S., Hassan, C., Antonello, A., Tanner, K., Bellisario, C., Battaglia, G., Anderloni, A., Correale, L., Sharma, P., Baron, T. H., et al. (2016) · Gastrointestinal Endoscopy · doi:10.1016/j.gie.2016.01.039 · Cited by 202
Beaudry, G., Yu, R., Perry, A. E., & Fazel, S. (2021) · The Lancet Psychiatry · doi:10.1016/s2215-0366(21)00170-x · Cited by 196
Tran, A., Yates, J., Lau, A., Lampron, J., & Matar, M. (2018) · Journal of Trauma and Acute Care Surgery · doi:10.1097/ta.0000000000001816 · Cited by 171
Olbert, C. M., Nagendra, A., & Buck, B. (2018) · Journal of Abnormal Psychology · doi:10.1037/abn0000309 · Cited by 169
Gillespie, B. M., Harbeck, E., Rattray, M., Liang, R., Walker, R., Latimer, S. L., Thalib, L., Andersson, A. E., Griffin, B. R., Ware, R. S., et al. (2021) · International Journal of Surgery · doi:10.1016/j.ijsu.2021.106136 · Cited by 205
Hoben, M., Clarke, A., Huynh, K. T., Kobagi, N., Kent, A., Hu, H., Pereira, R. A. C., Xiong, T., Yu, K., Xiang, H., et al. (2017) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2017.05.003 · Cited by 166
“As Turner et al. (2013) highlight, underpowered studies contribute little additional information if at least two adequately powered studies are available.”
BackgroundKline, A. C., Cooper, A. A., Rytwinksi, N. K., & Feeny, N. C. (2018) · Clinical Psychology Review · doi:10.1016/j.cpr.2017.10.009 · Cited by 162
“A uthor M anuscript A uthor M anuscript A uthor M anuscript A uthor M anuscript et al., 2015; Kazdin & Bass, 1989), and meta-analytic research (e.g., Hedges & Pigott, 2001; Hedges & Pigott, 2004; Kraemer, Gardner, Brooks, & Yesavage, 1998; Shadish Jr. & Sweeney, 1991; Turner et al., 2013).”
BackgroundScurt, F. G., Ewert, L., Mertens, P. R., Haller, H., Schmidt, B. M. W., & Chatzikyrkou, C. (2019) · The Lancet · doi:10.1016/s0140-6736(18)32091-9 · Cited by 160
Pierce, M., Bird, S. M., Hickman, M., Marsden, J., Dunn, G., Jones, A., & Millar, T. (2016) · Addiction · doi:10.1111/add.13193 · Cited by 156
Self-citationFibrinogen concentrate in bleeding patients
Wikkelsø, A., Lunde, J., Johansen, M., Stensballe, J., Wetterslev, J., Møller, A. M., & Afshari, A. (2013) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd008864.pub2 · Cited by 150
Living systematic reviews: 3. Statistical methods for updating meta-analyses
Simmonds, M. C., Salanti, G., McKenzie, J. E., Elliott, J., Agoritsas, T., Hilton, J., Perron, C., Akl, E. A., Hodder, R. K., Pestridge, C., et al. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.08.008 · Cited by 163
Toli, A., Webb, T. L., & Hardy, G. E. (2016) · British Journal of Clinical Psychology · doi:10.1111/bjc.12086 · Cited by 137
Hoerger, M. P., Wayser, G. R., Schwing, G., Suzuki, A., & Perry, L. M. (2018) · Annals of Behavioral Medicine · doi:10.1093/abm/kay077 · Cited by 139
“As well, even in cancer-specific studies, those with small samples are at an elevated risk of bias and contribute limited information when two or more large studies exist [39].”
BackgroundOral nonsteroidal anti-inflammatory drugs (NSAIDs) for cancer pain in adults
Derry, S., Wiffen, P. J., Moore, A., McNicol, E. D., Bell, R. F., Carr, D. B., McIntyre, M., & Wee, B. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012638.pub2 · Cited by 150
Krumbein, H., Kümmel, L. S., Fragkou, P. C., Thölken, C., Hünerbein, B. L., Reiter, R., Papathanasiou, K. A., Renz, H. E., & Skevaki, C. L. (2022) · Reviews in Medical Virology · doi:10.1002/rmv.2365 · Cited by 132
“It is therefore advisable that future reviews should include studies with a higher minimum number of COVID‐19 patients tested for co‐ infections.(38) The prevalences of RVCI differ greatly among continents.”
BackgroundFinding the power to reduce publication bias
Stanley, T. D., Doucouliagos, H., & Ioannidis, J. P. A. (2017) · Statistics in Medicine · doi:10.1002/sim.7228 · Cited by 131
“When there are no adequately powered studies, a common occurrence in medical research [39],WAAP cannot be computed and reduces toWLS; hence, our combined estimator,WAAP-WLS.”
BackgroundPapalia, N., Spivak, B., Daffern, M., & Ogloff, J. R. P. (2019) · Clinical Psychology: Science and Practice · doi:10.1111/cpsp.12282 · Cited by 130
Maiorino, M. I., Chiodini, P., Bellastella, G., Giugliano, D., & Esposito, K. (2015) · Endocrine · doi:10.1007/s12020-015-0812-6 · Cited by 151
“Moreover, we included trials that assessed at least 30 women in order to limit heterogeneity, as studies with few participants are more likely to be biased [49, 50].”
MethodEffects of Mindfulness-Based Interventions on Alexithymia: A Systematic Review
Norman, H., Marzano, L., Coulson, M., & Oskis, A. (2019) · Evidence Based Mental Health · doi:10.1136/ebmental-2018-300029 · Cited by 128
Morphine for chronic neuropathic pain in adults
Cooper, T. E., Chen, J., Wiffen, P. J., Derry, S., Carr, D. B., Aldington, D. J., Cole, P., & Moore, R. A. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011669.pub2 · Cited by 148
Caldeira, D., Rodrigues, F. B., Barra, M., Santos, A. T., Abreu, D., Gonçalves, N., Pinto, F. J., Ferreira, J. J., & Costa, J. N. (2015) · Heart · doi:10.1136/heartjnl-2015-307489 · Cited by 134
Bright white light therapy in depression: A critical review of the evidence
Mårtensson, B., Pettersson, A., Berglund, L., & Ekselius, L. (2015) · Journal of Affective Disorders · doi:10.1016/j.jad.2015.04.013 · Cited by 120
Cemented versus cementless hemiarthroplasty for a displaced fracture of the femoral neck
Veldman, H. D., Heyligers, I. C., Grimm, B. P., & Boymans, T. A. E. J. (2017) · The Bone & Joint Journal · doi:10.1302/0301-620x.99b4.bjj-2016-0758.r1 · Cited by 119
Geense, W. W., van den Boogaard, M., van der Hoeven, J. G., Vermeulen, H., Hannink, G., & Zegers, M. (2019) · Critical Care Medicine · doi:10.1097/ccm.0000000000003974 · Cited by 132
“Some researchers argue for excluding small studies from meta-analyses (94).”
BackgroundOu‐Yang, M., Sun, Y., Liebowitz, M. C., Chen, C., Fang, M.-L. E., Dai, W., Chuang, T., & Chen, J. (2020) · PLoS ONE · doi:10.1371/journal.pone.0232238 · Cited by 143
“Given a meta-analysis with a small sample size may increase the risk for overestimation or underestimation of the effect size, results shoud be interpreted with caution[86, 88].”
BackgroundPrevalence of Left Atrial Thrombus in Anticoagulated Patients With Atrial Fibrillation
Lurie, A., Wang, J., Hinnegan, K. J., McIntyre, W. F., Belley‐Côté, E. P., Amit, G., Healey, J. S., Connolly, S. J., & Wong, J. A. (2021) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2021.04.036 · Cited by 133
Outcome of open total arch replacement in the modern era
Settepani, F., Cappai, A., Basciu, A., Barbone, A., & Tarelli, G. (2016) · Journal of Vascular Surgery · doi:10.1016/j.jvs.2015.10.061 · Cited by 114
“5 38 (43) 38 (43) ND 35 Kaku et al 2002-2011 107 70 88 (82) 12 (11) 14 (13) 47 Okita et al 2002-2012 423 69.”
BackgroundGlucocorticosteroids for sepsis: systematic review with meta-analysis and trial sequential analysis
Volbeda, M., Wetterslev, J., Gluud, C., Zijlstra, J. G., van der Horst, I. C. C., & Keus, F. (2015) · Intensive Care Medicine · doi:10.1007/s00134-015-3899-6 · Cited by 114
“O p p er t [5 7 ] 4 8 S ep ti c sh o ck H C (5 0 0 m g /5 0 m l) b o lu s 5 0 m g , fo ll o w ed b y co n ti n u o u s”
BackgroundInfluentialDevillier, P., Dreyfus, J.-F., Demoly, P., & Calderón, M. A. (2014) · BMC Medicine · doi:10.1186/1741-7015-12-71 · Cited by 115
“In fact, underpowered studies often suffer from publication bias and contribute little information to metaanalyses when two or more adequately powered large trials are available [24-26].”
BackgroundXue, J., Qin, Z., Cai, H., Zhang, C., Li, X., Xu, W., Wang, J., Xu, Z., Yu, B., Xu, T., et al. (2017) · Oncotarget · doi:10.18632/oncotarget.15056 · Cited by 130
Evrenoglou, T., Metelli, S., & Chaimani, A. (2021) · Springer eBooks · doi:10.1007/978-3-319-52677-5_287-1 · Cited by 115
Garattini, S., Jakobsen, J. C., Wetterslev, J., Bertelé, V., Banzi, R., Rath, A., Neugebauer, E. A. M., Laville, M., Masson, Y., Hivert, V., et al. (2016) · European Journal of Internal Medicine · doi:10.1016/j.ejim.2016.03.020 · Cited by 108
Quality of Life in Patients With Pancreatic Cancer and Their Caregivers: A Systematic Review
Bauer, M. R., Bright, E. E., MacDonald, J. J., Cleary, E. H., Hines, O. J., & Stanton, A. L. (2018) · Pancreas · doi:10.1097/mpa.0000000000001025 · Cited by 107
Lower vs Higher Fluid Volumes During Initial Management of Sepsis
Meyhoff, T. S., Møller, M. H., Hjortrup, P. B., Cronhjort, M., Perner, A., & Wetterslev, J. (2020) · Chest · doi:10.1016/j.chest.2019.11.050 · Cited by 107
Adams, N. S., & Vincent, B. (2019) · Transgender Health · doi:10.1089/trgh.2019.0009 · Cited by 120
“concern is the small-study effect, whereby small cohort studies of lower methodological rigor report different effects than larger ones.(167,168) To some degree, concerns of”
BackgroundCaldeira, D., Barra, M., Santos, A. T., Abreu, D., Pinto, F. J., Ferreira, J. J., & Costa, J. N. (2014) · Heart · doi:10.1136/heartjnl-2013-305288 · Cited by 112
Professionals underestimate patients' pain: a comprehensive review
Seers, T., Derry, S., Seers, K., & Moore, A. (2018) · Pain · doi:10.1097/j.pain.0000000000001165 · Cited by 127
“Small studies are highly susceptible to random chance effects,(40,43,59) and considerable biases have been observed in small studies.(10,11,16,24,44,49,60) We therefore”
BackgroundSondrup, N., Termannsen, A.-D., Eriksen, J. N., Hjorth, M. F., Færch, K., Klingenberg, L., & Quist, J. S. (2022) · Sleep Medicine Reviews · doi:10.1016/j.smrv.2022.101594 · Cited by 92
Fetuses with right aortic arch: a multicenter cohort study and meta‐analysis
D’Antonio, F., Khalil, A. A., Zidere, V., & Carvalho, J. S. (2015) · Ultrasound in Obstetrics and Gynecology · doi:10.1002/uog.15805 · Cited by 108
Candelaria, D., Randall, S., Ladak, L. A., & Gallagher, R. D. (2019) · Quality of Life Research · doi:10.1007/s11136-019-02338-y · Cited by 119
Imberger, G. L., Gluud, C. N., Boylan, J. F., & Wetterslev, J. (2015) · Anesthesia & Analgesia · doi:10.1213/ane.0000000000000892 · Cited by 108
Intracranial hemorrhage risk with the new oral anticoagulants: a systematic review and meta-analysis
Caldeira, D., Barra, M., Pinto, F. J., Ferreira, J. J., & Costa, J. N. (2014) · Journal of Neurology · doi:10.1007/s00415-014-7462-0 · Cited by 111
Lambert, S. D., Duncan, L. R., Kapellas, S., Bruson, A.-M., Myrand, M., Santa Mina, D., Culos-Reed, N., & Lambrou, A. (2016) · Annals of Behavioral Medicine · doi:10.1007/s12160-016-9819-3 · Cited by 98
Wassenaar, T. M., Williamson, W., Johansen-Berg, H., Dawes, H., Roberts, N., Foster, C., & Sexton, C. E. (2020) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/s12966-020-00959-y · Cited by 97
“Finally, small sample sizes of primary studies may present with larger effect sizes and thereby affect meta-analyses results [52, 53].”
BackgroundRapid reviews may produce different results to systematic reviews: a meta-epidemiological study
Marshall, I. J., Marshall, R., Wallace, B. C., Brassey, J., & Thomas, J. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2018.12.015 · Cited by 95
“We chose these methods in particular, as they are commonly used in existing rapid syntheses [13] or proposed and evaluated in the evidence synthesis literature [18,22].”
Oral paracetamol (acetaminophen) for cancer pain
Wiffen, P. J., Derry, S., Moore, R. A., McNicol, E. D., Bell, R. F., Carr, D. B., McIntyre, M., & Wee, B. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012637.pub2 · Cited by 74
Blood-Brain Barrier Crossing Renin-Angiotensin Drugs and Cognition in the Elderly: A Meta-Analysis
Ho, J. K., Moriarty, F., Manly, J. J., Larson, E. B., Evans, D. A., Rajan, K. B., Hudak, E. M., Hassan, L., Liu, E., Sato, N., et al. (2021) · Hypertension · doi:10.1161/hypertensionaha.121.17049 · Cited by 106
Vitamin D supplementation for chronic liver diseases in adults
Bjelakovic, G., Nikolova, D., Bjelakovic, M., & Gluud, C. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011564.pub2 · Cited by 84
Shaohua, L., & Shorey, S. (2021) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2021.103871 · Cited by 91
Interventions to Mitigate COVID-19 Misinformation: A Systematic Review and Meta-Analysis
Janmohamed, K., Walter, N., Nyhan, K., Khoshnood, K., Tucker, J. D., Sangngam, N., Altice, F. L., Ding, Q., Wong, A., Schwitzky, Z. M., et al. (2021) · Journal of Health Communication · doi:10.1080/10810730.2021.2021460 · Cited by 104
Kang, Y., O’Haire, S., Franchini, F., IJzerman, M. J., Zalcberg, J. R., Macrae, F., Canfell, K., & Steinberg, J. (2022) · Scientific Reports · doi:10.1038/s41598-022-23319-1 · Cited by 98
Efficacy of Serious Games in Healthcare Professions Education: A Systematic Review and Meta-analysis
Maheu‐Cadotte, M., Cossette, S., Dubé, V., Fontaine, G., Lavallée, A., Lavoie, P., Mailhot, T., & Deschênes, M. (2020) · Simulation in Healthcare The Journal of the Society for Simulation in Healthcare · doi:10.1097/sih.0000000000000512 · Cited by 112
Empirical evaluation of which trial characteristics are associated with treatment effect estimates
Dechartres, A., Trinquart, L., Faber, T., & Ravaud, P. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2016.04.005 · Cited by 85
Mojtahedi, M. F., Sepidarkish, M., Almukhtar, M., Eslami, Y., Mohammadianamiri, F., Moghadam, K. B., Rouholamin, S., Razavi, M., Tadi, M. J., Naghibi, A. F., et al. (2023) · Journal of Hospital Infection · doi:10.1016/j.jhin.2023.05.019 · Cited by 116
The Necessity of Randomized Clinical Trials
Jakobsen, J. (2013) · British Journal of Medicine and Medical Research · doi:10.9734/bjmmr/2013/3208 · Cited by 84
“1463 Randomized clinical trials or systematic reviews with low methodological quality (high risks of systematic errors due to bias and design errors) and insufficient sample sizes (high risks of random error) [81-85] should not be used to guide decision makers and clinicians about which intervention to choose.”
BackgroundGresham, G. K., Meinert, J. L., Gresham, A. G., & Meinert, C. L. (2020) · JAMA Network Open · doi:10.1001/jamanetworkopen.2020.14682 · Cited by 94
“2010-2014 NIH/US government 36 (20-69) 149 (64-480) 70 (33-206) 48 (25-124) Industry 42 (24-90) 251 (101-513) 60 (30-140) 66 (30-204) Other 33 (18-64) 74 (40-162) 62 (30-153) 59 (28-130) Total 39 (20-77) 120 (50-325) 62 (30-152) 60 (29-150)”
The study of followers in leadership research: A systematic and critical review
Oc, B., Chintakananda, K., Bashshur, M. R., & Day, D. V. (2023) · The Leadership Quarterly · doi:10.1016/j.leaqua.2022.101674 · Cited by 117
Wolf, H., Huusom, L. D., Henriksen, T. B., Hegaard, H. K., Brok, J. S., & Pinborg, A. B. (2020) · BJOG An International Journal of Obstetrics & Gynaecology · doi:10.1111/1471-0528.16238 · Cited by 101
Lunny, C., Ramasubbu, C., Puil, L. J., Liu, T. W. B., Gerrish, S., Salzwedel, D. M., Mintzes, B. J., & Wright, J. M. (2021) · PLoS ONE · doi:10.1371/journal.pone.0250356 · Cited by 96
“Evidence shows biased results from poorly designed and reported studies can mislead decision-making in healthcare at all levels [62-64].”
BackgroundMyers, J. A., Witzel, B. S., Powell, S. R., Li, H., Pigott, T. D., Xin, Y. P., & Hughes, E. M. (2022) · Review of Educational Research · doi:10.3102/00346543211070049 · Cited by 60
“Second, although our study included a larger sample size than the previous meta-analyses examining WPS performance among elementary school students with MD, the sample we used remains relatively small for empirically summarizing literature (Turner et al., 2013).”
ResultBarbateskovic, M., Schjørring, O. L., Krauss, S. R., Jakobsen, J. C., Meyhoff, C. S., Dahl, R. M., Rasmussen, B. S., Perner, A., & Wetterslev, J. (2019) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012631.pub2 · Cited by 90
de Jonge, S. W., Atema, J. J., Solomkin, J. S., & Boermeester, M. A. (2017) · British journal of surgery · doi:10.1002/bjs.10445 · Cited by 91
Yu, Q., Wong, K.-K., Lei, O.-K., Nie, J., Shi, Q., Zou, L., & Kong, Z. (2022) · Sports Medicine - Open · doi:10.1186/s40798-022-00529-5 · Cited by 80
“The issue of effective sample size and power of network meta-analysis may be a hindrance to resolving uncertainty by synthetizing all available evidence and exploring causes of between-study heterogeneity [194].”
BackgroundNgan, H. Y., Chong, Y. Y., & Chien, W. T. (2021) · Diabetic Medicine · doi:10.1111/dme.14525 · Cited by 77
Rice, A. S. C., Fisher, E., Finn, D. P., Finnerup, N. B., Gilron, I., Haroutounian, S., Krane, E. J., Rice, A. S. C., Rowbotham, M. C., Wallace, M. S., et al. (2020) · Pain · doi:10.1097/j.pain.0000000000001941 · Cited by 94
Anxiety linked to COVID-19: A systematic review comparing anxiety rates in different populations
Saeed, H., Eslami, A., Nassif, N. T., Simpson, A. M., & Lal, S. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph19042189 · Cited by 76
“The initial search on COVID-19 yielded many results to confirm that only the most vital papers were included; any studies with sample sizes less than 1000 were excluded [22].”
ResultMeta-analysis of virtual reality exposure therapy for social anxiety disorder
Morina, N., Kampmann, I., Emmelkamp, P., Barbui, C., & Hoppen, T. H. (2023) · Psychological Medicine · doi:10.1017/s0033291721001690 · Cited by 75
Risperidone for children and adolescents with autism spectrum disorder: a systematic review
Maneeton, N., Maneeton, B., Putthisri, S., Woottiluk, P., Narkpongphun, A., & Srisurapanont, M. (2018) · Neuropsychiatric Disease and Treatment · doi:10.2147/ndt.s151802 · Cited by 85
“First, this review included a small RCT(31) into quantitative synthesis which may not have the power to determine the effect of intervention.(39) In addition, small RCTs potentially illustrate a greater effect than larger RCTs.”
Kline, A. C., Cooper, A. A., Rytwinski, N. K., & Feeny, N. C. (2020) · Behavior Therapy · doi:10.1016/j.beth.2020.04.015 · Cited by 77
Metadata
- Authors
- Turner, R. M., Bird, S. M., & Higgins, J. P. T.
- Affiliations
- MRC Biostatistics Unit (GB)
- University of Bristol (GB)
- Corresponding author
- Turner, Rebecca M.
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2013-03-27
- Volume
- 8
- Issue
- 3
- Pages
- e59202
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC3609745
- OpenAlex
- W2015824469
- Cited by
- 672
- References
- 25
- Citations by year
- 2026: 46 · 2025: 51 · 2024: 51 · 2023: 59 · 2022: 70 · 2021: 70 · 2020: 59 · 2019: 49 · 2018: 56 · 2017: 60 · 2016: 45 · 2015: 31 · 2014: 20 · 2013: 4
- FWCI
- 36.78
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, other-oa)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- UCL Discovery (University College London) (submittedVersion)
- Bristol Research (University of Bristol) (submittedVersion)
- Funders
- Medical Research Council (U105260794)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Keywords
- underpowered studies, study size, statistical power, meta-analysis, binary outcomes, relative risk, standard errors, study heterogeneity, rapid review
- MeSH terms
- Humans, Review Literature as Topic, Meta-Analysis as Topic, Odds Ratio, Sample Size, Risk Reduction Behavior
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
