The Impact of Study Size on Meta-analyses: Examination of Underpowered Studies in Cochrane Reviews
Open accessThe DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Most meta-analyses include data from one or more small studies that, individually, do not have power to detect an intervention effect. The relative influence of adequately powered and underpowered studies in published meta-analyses has not previously been explored. We examine the distribution of power available in studies within meta-analyses published in Cochrane reviews, and investigate the impact of underpowered studies on meta-analysis results. METHODS AND FINDINGS: For 14,886 meta-analyses of binary outcomes from 1,991 Cochrane reviews, we calculated power per study within each meta-analysis. We defined adequate power as ≥50% power to detect a 30% relative risk reduction. In a subset of 1,107 meta-analyses including 5 or more studies with at least two adequately powered and at least one underpowered, results were compared with and without underpowered studies. In 10,492 (70%) of 14,886 meta-analyses, all included studies were underpowered; only 2,588 (17%) included at least two adequately powered studies. 34% of the meta-analyses themselves were adequately powered. The median of summary relative risks was 0.75 across all meta-analyses (inter-quartile range 0.55 to 0.89). In the subset examined, odds ratios in underpowered studies were 15% lower (95% CI 11% to 18%, P<0.0001) than in adequately powered studies, in meta-analyses of controlled pharmacological trials; and 12% lower (95% CI 7% to 17%, P<0.0001) in meta-analyses of controlled non-pharmacological trials. The standard error of the intervention effect increased by a median of 11% (inter-quartile range -1% to 35%) when underpowered studies were omitted; and between-study heterogeneity tended to decrease. CONCLUSIONS: When at least two adequately powered studies are available in meta-analyses reported by Cochrane reviews, underpowered studies often contribute little information, and could be left out if a rapid review of the evidence is required. However, underpowered studies made up the entirety of the evidence in most Cochrane reviews.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“we pooled results excluding studies with fewer than 50 participants in the experimental group, as a planned sensitivity analysis”
From Effectiveness of psychological interventions in prison to reduce recidivism: a systematic review and meta-analysis of randomised controlled trials (2021) · cites this paper as [66]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“there is a risk that untested approaches may introduce new forms of bias 18 .”
From Software support for environmental evidence synthesis (2018) · cites this paper as [18]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“On collating the evidence, some studies may appear underpowered for the outcome(s) of interest to us (Turner 2013), yet were adequately powered for the studies' primary measure.”
From Physical activity and exercise for chronic pain in adults: an overview of Cochrane Reviews (2017) · cites this paper as Turner 2013
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Although our number of effect sizes (n=37) is relatively large in terms of outcomes analyzed, it is small in terms of statistical analyses that combine empirical studies [100].”
From Generative AI Mental Health Chatbots as Therapeutic Tools: Systematic Review and Meta-Analysis of Their Role in Reducing Mental Health Issues (2025) · cites this paper as [100]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Meta-analysis in clinical trials
DerSimonian, R., & Laird, N. (1986) · Controlled Clinical Trials · doi:10.1016/0197-2456(86)90046-2 · Cited by 39,552 · cited 2×
“In each meta-analysis j, we fitted a random-effects model, using a method-of-moments estimate for the between study variance [17], and calculated the variance Vj of the combined intervention effect (on the log relative risk scale).”
MethodSterne, J. A. C., Sutton, A. J., Ioannidis, J. P. A., Terrin, N., Jones, D. R., Lau, J., Carpenter, J., Rucker, G., Harbord, R. M., Schmid, C. H., et al. (2011) · BMJ · doi:10.1136/bmj.d4002 · Cited by 7,709 · cited 1×
“[24], who published new guidelines.”
Self-citationKjaergard, L. L., Villumsen, J., & Gluud, C. (2001) · Annals of Internal Medicine · doi:10.7326/0003-4819-135-11-200112040-00010 · Cited by 2,283 · cited 1×
“Alternatively, small-study effects may arise from biases caused by methodological flaws arising more frequently in small studies [3], or may be due to true differences in the underlying effects between smaller and larger studies.”
BackgroundPublication and related bias in meta-analysis
Sterne, J. A. C., Gavaghan, D., & Egger, M. (2000) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(00)00242-0 · Cited by 2,282
Trial sequential analysis may establish when firm evidence is reached in cumulative meta-analysis
Wetterslev, J., Thorlund, K., Brok, J., & Gluud, C. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.03.013 · Cited by 1,881
Brok, J., Thorlund, K., Gluud, C., & Wetterslev, J. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.10.007 · Cited by 1,113 · cited 1×
“[20], who had, however, excluded both reviews with fewer than three trials and those in which all trials had a high risk of bias, in which metaanalytic size is likely to have been smaller.”
ResultTotal Plasma Homocysteine and Cardiovascular Risk Profile: The Hordaland Homocysteine Study
Nygård, O., Vollset, S. E., Refsum, H., Stensvold, I., Tverdal, A., Nordrehaug, J. E., Ueland, P. M., & Kvåle, G. (1995) · JAMA · doi:10.1001/jama.1995.03530190040032 · Cited by 1,042
The scandal of poor medical research.
Altman, D. G. (1994) · BMJ · doi:10.1136/bmj.308.6924.283 · Cited by 897 · cited 1×
“Researchers who choose to undertake a study that is capable of detecting only an unrealistically large effect may lack understanding of both scientific methods and ethics [10].”
BackgroundThe power of statistical tests in meta-analysis
Hedges, L. V., & Pigott, T. D. (2001) · Psychological Methods · doi:10.1037/1082-989x.6.3.203 · Cited by 716 · cited 1×
“where W is the cumulative standard normal distribution function and Ca=2~W {1 1{a=2 ð Þ[18].”
BackgroundMeta‐analysis of drug‐related deaths soon after release from prison
Merrall, E. L. C., Kariminia, A., Binswanger, I. A., Hobbs, M. S., Farrell, M., Marsden, J., Hutchinson, S. J., & Bird, S. M. (2010) · Addiction · doi:10.1111/j.1360-0443.2010.02990.x · Cited by 674 · cited 1×
“In teaching, Bird has long advocated that trials should not be started unless they could deliver at least 50% power in respect of a priori plausible, worthwhile effect sizes [6].”
BackgroundSelf-citation
Cited by
Items 401–500 of 672
Alhwoaimel, N. A., Alenazi, A. M., Alhowimel, A., Alqahtani, B. A., & Alshehri, M. M. (2023) · Topics in Stroke Rehabilitation · doi:10.1080/10749357.2023.2261701 · Cited by 12
Hu, S., Jing, Y., Fu, Y., & Ye, X. (2024) · Gene · doi:10.1016/j.gene.2024.148314 · Cited by 10
Zhou, Y., Rele, S., & Elsewaisy, O. (2024) · The Knee · doi:10.1016/j.knee.2024.02.009 · Cited by 10
Dominguez, S. K., & Lee, C. W. (2019) · Journal of EMDR Practice and Research · doi:10.1891/1933-3196.13.4.247 · Cited by 10
Interventions to mitigate COVID-19 misinformation: protocol for a scoping review
Kumar, N., Walter, N., Nyhan, K., Khoshnood, K., Tucker, J. D., Bauch, C. T., Ding, Q., Jones-Jang, S. M., Choudhury, M. D., Schwartz, J. L., et al. (2022) · Systematic Reviews · doi:10.1186/s13643-022-01917-4 · Cited by 10
“Case reports and case series often contain relatively limited evidence [26].”
BackgroundAn evidence-based approach to red blood cell transfusions in asymptomatically anaemic patients
Chan, A., & de Gara, C. (2015) · Annals of The Royal College of Surgeons of England · doi:10.1308/rcsann.2015.0047 · Cited by 12
“00001, I(2)≥93%) and small sample sizes of individual studies (which impact on power to detect clinical differences) call into question the value of this type of ‘level 1 evidence’.(14) Overall, these studies recommend Hb 70–80g/l as the transfusion trigger for patients without CAD.”
BackgroundSteinleitlinie: Rück- und Ausblick
Türk, C. (2016) · Der Urologe · doi:10.1007/s00120-016-0230-2 · Cited by 10
“Eine kritische Betrachtung der in eine Metaanalyse eingeschlossenen Studien ist daher essentiell [14, 15].”
BackgroundMandal, R. K., Khan, M. A., Hussain, A., Akhter, N., Jawed, A., Dar, S. A., Wahid, M., Panda, A. K., Lohani, M., Mishra, B. N., et al. (2018) · Bioscience Reports · doi:10.1042/bsr20181052 · Cited by 11
Koster, T. M., Wetterslev, J., Gluud, C. N., Jakobsen, J. C., Kaufmann, T., Eck, R. J., Koster, G., Hiemstra, B., van der Horst, I. C. C., & Keus, E. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.05.011 · Cited by 12
“This is why a meta-analysis ought to be regarded as an interim analysis on its way toward the required (sufficient) information size [19].”
BackgroundThrombolysis for acute upper extremity deep vein thrombosis
Feinberg, J. B., Nielsen, E. E., & Jakobsen, J. C. (2016) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012175 · Cited by 11
Bishoge, O. K., Omary, M., Liheluka, E., Mshana, J. M., Nguyamu, M., Mbatia, Y. J., Njee, R. M., & Kafuye, M. Y. (2023) · Journal of Water Sanitation and Hygiene for Development · doi:10.2166/washdev.2023.222 · Cited by 17
NIPPV vs CPAP: Lessons from meta-analyses
Deguise, M., & Lemyre, B. (2025) · Seminars in Perinatology · doi:10.1016/j.semperi.2025.152062 · Cited by 10
When a meta-analysis can be really useful?
Basile, C., Villaschi, A., & Maggioni, A. P. (2025) · International Journal of Cardiology · doi:10.1016/j.ijcard.2025.133423 · Cited by 10
Elhannani, A., Martel, M.-E., Collet, A., Chepy, A., Sanges, S., HACHULLA, E., Dubucquoi, S., Launay, D., & Sobanski, V. (2025) · Rheumatology · doi:10.1093/rheumatology/keaf392 · Cited by 9
Rojas, J., Srikumaran, U., & McFarland, E. G. (2020) · Shoulder & Elbow · doi:10.1177/1758573219896794 · Cited by 9
“Trial sequential analysis is a cumulative random-effect meta-analysis methodology(24,25) to control for the risk of random errors in metaanalyses.(26) This methodology enables the estimation of the required meta-analysis sample size–optimal information size (OIS) to obtain sufficient statistical power and adjusts the…”
MethodXiao, Q., Chen, J., Zeng, S., Cai, H., & Zhu, G. (2022) · Medicine · doi:10.1097/md.0000000000031247 · Cited by 9
“They can rectify the enhanced risk of type I and type II errors from multiple updates of the meta-analysis, which are mainly due to data merging.[31] The application of TSA minimizes false-positive results resulting from random errors when the sample size of meta-analysis does not reach a sufficient sample size.”
Backgrounddos Santos, M. A. L., Couto, G. R., Sabey, M. J. S., BORGES, D. D. P. R., & Takeshita, W. M. (2021) · Dentomaxillofacial Radiology · doi:10.1259/dmfr.20210149 · Cited by 9
Caldeira, D., Gonçalves, N., Ferreira, J. J., Pinto, F. J., & Costa, J. N. (2015) · American Journal of Cardiovascular Drugs · doi:10.1007/s40256-015-0132-5 · Cited by 11
Chagas Disease Prevalence in Colombia: A Meta-Analysis and Systematic Review
Llau, A. F., Tejada, C. E., & Ahmed, N. U. (2018) · Vector-Borne and Zoonotic Diseases · doi:10.1089/vbz.2018.2308 · Cited by 9
“To account for this potential issue, some researchers argue for excluding small studies from meta-analyses (Turner et al. 2013), while others suggest reviewers should include a separate analysis excluding all smaller studies (Nu¨esch et al. 2010).”
Brauer, J. R., Day, J. C., & Hammond, B. M. (2019) · Sociological Methods & Research · doi:10.1177/0049124119826158 · Cited by 9
Systematic Reviews and Meta‐Analysis
Crombie, I. (2021) · Wiley eBooks · doi:10.1002/9781394322183.ch5 · Cited by 9
Vasopressor and inotrope treatment for septic shock: An umbrella review of reviews
Einav, S., Helviz, Y., Ippolito, M., & Cortegiani, A. (2021) · Journal of Critical Care · doi:10.1016/j.jcrc.2021.05.017 · Cited by 10
Kaddoura, R., Al-Tamimi, H., Abushanab, D., Hayat, S. A., & Papasavvas, T. (2024) · International Journal of Cardiology Cardiovascular Risk and Prevention · doi:10.1016/j.ijcrp.2024.200255 · Cited by 14
“Reporting or publication bias was evaluated using a visual inspection of the funnel plot [14 – 16].”
Is Discrimination Related to Markers of Systemic Inflammation? A Systematic Review and Meta-analysis
Cardenas, M. N., Antenucci, N. M., Sheeran, P., & Muscatell, K. A. (2025) · Psychosomatic Medicine · doi:10.1097/psy.0000000000001429 · Cited by 9
Asakawa-Haas, K. D., Spiegel, D., Bossert, L., Garić, A., Schwartz, K., Voracek, M., & Tran, U. S. (2026) · Communications Psychology · doi:10.1038/s44271-026-00414-x · Cited by 10
“About 70% of all meta-analyses might be solely based on studies too underpowered to detect a small effect 49 .”
The Cochrane Hepato-Biliary Group as a Resource Example of Evidence-Based Medicine for All
Nikolova, D., Klingenberg, S. L., & Gluud, C. N. (2013) · Acta Médica Portuguesa · doi:10.20344/amp.3996 · Cited by 8
“The cochrane hepato-biliary group as a resource of evidence-based medicine, Acta Med Port 2013 Mar-Apr;26(2):81-82”
BackgroundMethodInfluentialLe, V. (2020) · Teachers College Record The Voice of Scholarship in Education · doi:10.1177/016146812012200304 · Cited by 8
Bacorro, W., Baldivia, K., Yu, K. K. L., Mariano, J. Z., Gonzalez, G., & Ortin, T. S. (2022) · Gynecologic Oncology · doi:10.1016/j.ygyno.2022.06.018 · Cited by 8
Marsman, M. S., Wetterslev, J., Vriens, P. W. H. E., Bleys, R. L. A. W., Jahrome, A. K., Moll, F. L., Keus, F., Reijnen, M. M. P. J., & Koning, G. G. (2023) · Surgery Open Science · doi:10.1016/j.sopen.2023.05.003 · Cited by 10
Meta-Analyses Do Not Establish Improved Mortality With Ivermectin Use in COVID-19
Rothrock, S. G., Weber, K. D., Giordano, P., & Barneck, M. D. (2022) · American Journal of Therapeutics · doi:10.1097/mjt.0000000000001477 · Cited by 8
Mong, M. S. A., Lai, M. Y. C., Cheng, L. J., & Lau, Y. (2022) · Neuromodulation Technology at the Neural Interface · doi:10.1016/j.neurom.2022.11.003 · Cited by 8
Haloperidol for delirium in critically ill patients – protocol for a systematic review
Barbateskovic, M., Kraus, S. R., Collet, M. O., Mathiesen, O., Jakobsen, J. C., Perner, A., & Wetterslev, J. (2018) · Acta Anaesthesiologica Scandinavica · doi:10.1111/aas.13088 · Cited by 8
Pharmacological therapies for fibromyalgia in adults - an overview of Cochrane Reviews
Häuser, W., Welsch, P., Klose, P., Derry, S., Straube, S., Wiffen, P. J., & Moore, A. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013151 · Cited by 9
Marsman, M. S., Wetterslev, J., Vriens, P. W. H. E., Bleys, R. L. A. W., Jahrome, A. K., Moll, F. L., Keus, F., & Koning, G. G. (2020) · BMJ Open · doi:10.1136/bmjopen-2019-030503 · Cited by 8
Yang, S., Fu, Q., Wu, J., Zhang, Q., Deng, H., Chen, S., Yang, H., Yan, M., & Zhang, L. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e39058 · Cited by 9
“TSA thus offers a more transparent assumption-based analysis compared to traditional meta-analyses that use unadjusted confidence intervals and unadjusted statistical significance thresholds [49,52].”
InfluentialA Best-Evidence Synthesis and Meta-analysis on Effective Reading Programs in Spanish
Tirado, J. L. A., Martín, F. D. F., Torres, M. H., Jiménez‐Fernández, G., Calet, N., Defior, S., Neitzel, A. J., & Slavin, R. E. (2024) · Review of Educational Research · doi:10.3102/00346543241297668 · Cited by 12
“Although our sample is relatively large in terms of outcomes analyzed, it is small in terms of statistical analyses that combine empirical studies (Turner et al., 2013).”
Lim, X., Ang, M. Q., Foo, A., Chng, K. K. Y., & Koh, S. S. L. (2024) · International Journal of Gynecology & Obstetrics · doi:10.1002/ijgo.16019 · Cited by 9
“49 Furthermore, only one study is available that reports anxiety as an outcome, and it does not have a positive significance when expressive writing is carried out for post-partum anxiety.”
Glycaemic control for patients with severe acute brain injury: Protocol for a systematic review
Fenger, A. W., Olsen, M. H., Fabritius, M. L., Riberholt, C. G., & Møller, K. (2022) · Acta Anaesthesiologica Scandinavica · doi:10.1111/aas.14166 · Cited by 9
Prone position for acute respiratory distress syndrome and the hazards of meta-analysis
Poole, D., Pisa, A., & Fumagalli, R. (2024) · Pulmonology · doi:10.1016/j.pulmoe.2022.12.005 · Cited by 9
Ramos‐Bossini, A. J. L., Gutiérrez, P. M. J., & Santiago, F. R. (2024) · La radiologia medica · doi:10.1007/s11547-024-01809-8 · Cited by 9
“Regarding the limitations, there is a relatively low number of RCTs, which limits the statistical power of the meta-analysis [46], high heterogeneity in study design, inclusion criteria, and outcome measures. addition, the cutoff points chosen for short, medium, and long-term follow-up times lack…”
Meta-analysis and public policy: Reconciling the evidence on deworming
Croke, K. G., Hicks, J. H., Hsu, E., Kremer, M. R., Maertens, R., Miguel, E. A., & Więcek, W. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2308733121 · Cited by 7
Aaqillah‐Amr, M. A., Hidir, A., Ikhwanuddin, M., Fatan, N. A., Muhammad-Rahimi, R., Ning-Shahira, S., Siankhe, R. G. N., Vandenberg, G. W., Tanga, C. M., & Yossa, R. (2024) · Journal of Insects as Food and Feed · doi:10.1163/23524588-00001096 · Cited by 12
“…and environmental benefits through waste utilization but also in this field to further validate and solidify these findings, considering the lack of degree of freedom in the current study as the scarcity of data hampers a comprehensive understanding of these dimensions (Turner et al ., 2013).”
Yogev, A., Arnold, J. I., Nelson, H., Rosenblat, M. A., Clarke, D. C., Guenette, J. A., Sporer, B. C., & Koehle, M. S. (2024) · Frontiers in Sports and Active Living · doi:10.3389/fspor.2024.1406987 · Cited by 10
Privacy-preserving multicenter differential protein abundance analysis with FedProt
Burankova, Y. P., Abele, M., Bakhtiari, M., von Toerne, C., Barth, T. K., Schweizer, L., Giesbertz, P., Schmidt, J. R., Kalkhof, S., Müller‐Deile, J., et al. (2025) · Nature Computational Science · doi:10.1038/s43588-025-00832-7 · Cited by 9
“Meta-analysis performance improves with an increase in sample sizes and number of studies 17 or with the availability of raw data for combined reanalysis 18 , which is challenging in proteomics.”
Value and limitations of clinical practice guidelines in neonatology
Polin, R. A., & Lorenz, J. M. (2015) · Seminars in Fetal and Neonatal Medicine · doi:10.1016/j.siny.2015.09.004 · Cited by 8
de Ribera, O. S., Trajtenberg, N., & Christensen, L. S. (2020) · Child Abuse & Neglect · doi:10.1016/j.chiabu.2020.104463 · Cited by 11
Antibiotics for adults with acute cholecystitis or acute cholangitis or both
Markotić, F., Grgić, S., Poropat, G., Fox, A. M., Nikolova, D., Vukojević, K., Jakobsen, J. C., & Gluud, C. N. (2020) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013646 · Cited by 11
Stress ulcer prophylaxis in adult intensive care unit patients – a protocol for a systematic review
Barbateskovic, M., Marker, S., Jakobsen, J. C., Krag, M., Granholm, A., Anthon, C. T., Perner, A., Wetterslev, J., & Møller, M. H. (2018) · Acta Anaesthesiologica Scandinavica · doi:10.1111/aas.13109 · Cited by 7
Powell‐Jackson, T., Davey, C., Masset, E., Krishnaratne, S., Hayes, R., Hanson, K., & Hargreaves, J. (2018) · Health Policy and Planning · doi:10.1093/heapol/czy028 · Cited by 8
“The evidence, beyond just trials, suggests that most empirical research in both economics and health is underpowered (Ioannidis et al. 2017; Turner et al. 2013).”
BackgroundMandal, R. K., Wahid, M., Jawed, A., Dar, S. A., Panda, A. K., Akhter, N., Lohani, M., Mishra, B. N., Khan, S. A., Areeshi, M. Y., et al. (2019) · Microbial Pathogenesis · doi:10.1016/j.micpath.2018.12.050 · Cited by 7
Al-Khani, A. M., Kalou, Y., & Haider, K. H. (2023) · Heart Lung and Circulation · doi:10.1016/j.hlc.2023.07.001 · Cited by 7
Chorti, A. G., Bontinis, V., Bontinis, A., Alifieris, C. E., Chatziantoniou, G., Karlafti, E., Michalopoulos, A., & Paramythiotis, D. (2023) · Minerva Surgery · doi:10.23736/s2724-5691.23.10024-4 · Cited by 6
Fu, Q., Zhu, S., Chen, J., Liu, Y.-Q., Liang, C., Song, L., Zhuang, J., Tan, X., Liu, L., Luo, L., et al. (2025) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2025.105097 · Cited by 6
Axelsson, E. (2025) · BMJ Open · doi:10.1136/bmjopen-2025-102996 · Cited by 7
Zhang, T., Hui, B. P. H., Ducasse, D., Li, Y., Wang, Y., Hu, J., Tang, W. C. S., & Ke, Y. (2025) · Psychotherapy and Psychosomatics · doi:10.1159/000548398 · Cited by 6
Juul, S., Riberholt, C. G., Olsen, M. H., Milan, J. B., Hafliðadóttir, S. H., Svanholm, J. H., Pedersen, E. B., Lew, C. C. H., Asante, M. A., Ribeiro, J. P., et al. (2025) · BMC Medical Research Methodology · doi:10.1186/s12874-025-02716-w · Cited by 8
“Often, meta-analyses of randomised clinical trials include an insufficient number of participants to obtain adequate statistical power for reliable testing for even large anticipated intervention effects making it difficult to discern true negative results from false negative results [6].”
Ellis, L. B., Tighe, J., Bowden, S. J., Kechagias, K. S., Paraskevaidi, M., Garg, A., Paraskevaidis, E., Arbyn, M., Veroniki, A. A., Kalliala, I. E. J., et al. (2026) · European Journal of Cancer · doi:10.1016/j.ejca.2026.116823 · Cited by 8
Ellekjær, K. L., Perner, A., Sivapalan, P., & Møller, M. H. (2022) · Acta Anaesthesiologica Scandinavica · doi:10.1111/aas.14076 · Cited by 8
“In brief, TSA estimates the required information size needed to detect or reject an a priori pre-specified realistic intervention effect in a metaanalysis and widens the CIs (TSA adjusted CI) in cases where data are too sparse to draw conclusions.(29,33,34) Moreover, we used TSA to test for futility by applying futili…”
MethodTanner, A. E., Goldenberg, T., Erausquin, J. T., Mertus, S., Jibriel, M. S. E., & Barrington, C. (2022) · BMJ Open · doi:10.1136/bmjopen-2022-061691 · Cited by 6
Small studies in systematic reviews: To include or not to include?
Alayli, A. E., Thomas, P. J., Jdiaa, S. S., Mansour, R. A., Gautam, A. S., Phadnis, M. A., Mikati, I. K. E., & Mustafa, R. A. (2023) · F1000Research · doi:10.12688/f1000research.127100.1 · Cited by 6
Association of Glycoprotein IIIa PlA1/A2 Polymorphism with Risk of Stroke: Updated Meta-Analysis
Coadǎ, C. A., Lupu, M., Florea, I., Constanzo, S. D., Coluccelli, S., & Şimon, I. (2024) · Current Issues in Molecular Biology · doi:10.3390/cimb46060321 · Cited by 6
“Moreover, many meta-analyses themselves do not estimate their power [63].”
BackgroundChen, H., Wang, Q., Zhu, J., Zhu, Y., Yang, F., Hui, J., Tang, X., & Zhang, T. (2025) · Journal of Affective Disorders · doi:10.1016/j.jad.2025.01.029 · Cited by 8
Association between diabetes mellitus and tinnitus: A meta-analysis
Luo, S., Wen, J., Bao, Q., Ou, H., Yi, S., & Peng, P. (2025) · Biomolecules and Biomedicine · doi:10.17305/bb.2024.11634 · Cited by 6
“Smaller studies are more susceptible to confounding and random error, which can bias meta-analytic findings [30].”
InfluentialCastellini, G., Gianola, S., Banzi, R., Corbetta, D., Gatti, R., Sirtori, V., Gluud, C. N., & Moja, L. (2014) · Trials · doi:10.1186/1745-6215-15-512 · Cited by 10
Moussa, F. A. H., & Brownlee, I. A. (2023) · Food Hydrocolloids for Health · doi:10.1016/j.fhfh.2023.100146 · Cited by 7
Ferreira, V. L., Mainka, F. F., Wiens, A., & Pontarolo, R. (2023) · Clinical Drug Investigation · doi:10.1007/s40261-023-01301-7 · Cited by 6
“Publications evaluating 5 patients were also excluded to reduce bias [18, 23].”
BackgroundA Mendelian Randomization Study of Plasma Homocysteine and Multiple Myeloma
Yang, X., Li, X., Hu, Z., Teng, Z., & Hu, D. (2016) · Scientific Reports · doi:10.1038/srep25204 · Cited by 6
Sethi, N. J., Safi, S., Feinberg, J. B., Nielsen, E. E., Gluud, C. N., & Jakobsen, J. C. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0470-2 · Cited by 7
“The required information size takes into account the anticipated intervention effect, the variance of the anticipated difference in intervention effects, the acceptable risk of falsely rejecting the null hypothesis (alpha), the acceptable risk of falsely confirming the null hypothesis (beta), and the variance of the i…”
BackgroundMandal, R. K., & Mittal, R. D. (2017) · Indian Journal of Clinical Biochemistry · doi:10.1007/s12291-017-0669-y · Cited by 6
Lower vs. higher fluid volumes in sepsis—protocol for a systematic review with meta‐analysis
Meyhoff, T. S., Møller, M. H., Hjortrup, P. B., Cronhjort, M., Perner, A., & Wetterslev, J. (2017) · Acta Anaesthesiologica Scandinavica · doi:10.1111/aas.12952 · Cited by 6
“The required information size(52) will take into account the event proportion in the control group, the assumption of a plausible RRR, and the heterogeneity variance of the meta-analysis.(52,53) We will use conservative estimations of the anticipated intervention effect estimates to reduce the risk of random error.”
MethodMandal, R. K., Dar, S. A., Jawed, A., Wahid, M., Lohani, M., Panda, A. K., Mishra, B. N., Akhter, N., Areeshi, M. Y., & Haque, S. (2017) · Oncotarget · doi:10.18632/oncotarget.23547 · Cited by 6
Gu, Y., Shi, J., Qiu, S., Qiao, Y., Zhang, X., Cheng, Y., & Liu, Y. (2018) · BMC Cancer · doi:10.1186/s12885-018-4941-1 · Cited by 6
Cochrane in CORR®: Preventing Occupational Stress in Healthcare Workers
McCarthy, C., & Bhandari, M. (2019) · Clinical Orthopaedics and Related Research · doi:10.1097/corr.0000000000000735 · Cited by 8
“We agree with the finding that most of the studies included in this review were of smaller sample sizes, only 17 containing more than 110 participants, and an increase in sample size would produce morereliable results [13, 17].”
ResultStress ulcer prophylaxis in critically ill children: Protocol for a systematic review
Jensen, M. M., Marker, S., Do, H. Q., Perner, A., & Møller, M. H. (2019) · Acta Anaesthesiologica Scandinavica · doi:10.1111/aas.13361 · Cited by 6
McGill, K., McGarry, J., Sackley, C. M., Godwin, J., Nicoll, A., & Brady, M. (2020) · Clinical Rehabilitation · doi:10.1177/0269215520930796 · Cited by 9
“Trials may end with small samples sizes which, through lack of statistical power, increase the risk of results being falsely identified as positive (type I error) or negative (type II error) (4-6) and an inability to determine how well a treatment has worked (7-10).”
BackgroundHolper, L. (2020) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2020.00633 · Cited by 7
“These aspects are even more problematic considering small sample sizes, because the reduction in statistical power consequently requires larger sample sizes in binary outcomes to be adequately powered (55).”
Alhobeira, H. A. (2020) · Bioinformation · doi:10.6026/97320630016789 · Cited by 6
Beta-blockers for suspected or diagnosed acute myocardial infarction
Nielsen, E. E., Feinberg, J. B., Safi, S., Sethi, N. J., Gluud, C. N., & Jakobsen, J. C. (2016) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012484 · Cited by 6
Interventions to mitigate vaping misinformation: protocol for a scoping review
Kumar, N., Hampsher, S., Walter, N., Nyhan, K., Ding, Q., Tracy, K., Jones-Jang, S. M., Nyhan, B. J., Choudhury, M. D., & Forastiere, L. (2022) · Systematic Reviews · doi:10.1186/s13643-022-02094-0 · Cited by 8
Survival Times of Lung, Breast, Cervical, and Prostate Cancer Patients in Africa
Atoyebi, S. B. (2023) · Journal of Cancer and Tumor International · doi:10.9734/jcti/2023/v13i1226 · Cited by 9
“Stepping Up”: A Decade of Relationship Violence Prevention
Carter‐Snell, C., & Warthe, D. G. (2023) · Social Sciences · doi:10.3390/socsci12090501 · Cited by 7
Trial sequential analysis: A simple guide for judging the conclusiveness of the effect
Mheissen, S. (2024) · Journal of Orthodontics · doi:10.1177/14653125241245140 · Cited by 6
“However, meta-analysis may spuri-ously overestimate the effect size of the treatment due to systematic and random errors (Schulz et al., 1995) and may have low statistical power, due to a low number of participants or a low number of included trials (Thorlund et al., 2011; Turner et al., 2013).”
Tomasoni, M., Borsetto, D., Canzi, P., Vural, A., Testa, G., Piazza, C., Pinelli, L., & de Zinis, L. O. R. (2024) · Otolaryngology · doi:10.1002/ohn.974 · Cited by 5
Prosty, C. J., Bortolussi‐Courval, É., Dubé, L.-R., Lee, T. C., & McDonald, E. G. (2024) · CMI Communications · doi:10.1016/j.cmicom.2024.105041 · Cited by 5
Johnson, B., Jung, K. E., MacKenzie, M., Bah, A., Jetté, N., Mohamed, N., & Blank, L. J. (2025) · Epilepsia · doi:10.1111/epi.18277 · Cited by 5
Riluzole for Amyotrophic Lateral Sclerosis Treatment
Jafari, Z., Mahood, Q., & Hamson, A. (2023) · Canadian Journal of Health Technologies · doi:10.51731/cjht.2023.680 · Cited by 7
Melão, B. V. L. A., de Amorim, L. G. C. R., Sanches, M. R., Gomes, G. V., Gewehr, D. M., Moreira, L. H. D. O., Uzeda, T. S., Lobo, M. D. M., & Bechara, G. R. (2024) · International braz j urol · doi:10.1590/s1677-5538.ibju.2024.0134 · Cited by 5
“Additionally, cohorts or case series with fewer than ten patients were excluded from the pooled analysis due to the high possibility of selection bias (13).”
BackgroundChackochan, A., Rashid, M., Reghunath, S. R., Poojari, P. G., Thunga, G., Nagri, S. K., Guddattu, V., Shenoy, R. P., Shenoy, V. P., & Acharya, L. D. (2025) · Therapeutic Advances in Endocrinology and Metabolism · doi:10.1177/20420188251319476 · Cited by 11
“65,66 This could be a possible explanation for the lack of significance for the impact of vitamin D supplementation on kidney function and other related parameters.”
Wang, F., Qin, Z., Si, S., Tang, J., Xu, L., Xu, H., Li, R., Han, P., & Yang, H. (2017) · Oncotarget · doi:10.18632/oncotarget.19023 · Cited by 5
Pędziwiatr, M., Mizera, M., Wysocki, M., Małczak, P., Stefura, T., Gajewska, N., Torbicz, G., Droś, J., Kacprzyk, A., Major, P., et al. (2019) · Clinical Nutrition · doi:10.1016/j.clnu.2019.05.024 · Cited by 5
Discussion on the Meeting on ‘Signs and Sizes:Understanding and Replicating Statistical Findings’
L. Hutton, J., Diggle, P. J., Bird, S. M., Hennig, C. M., Longford, N., Mathur, M. B., VanderWeele, T. J., Ioannidis, J. P. A., Chai, C. P., Dowe, D. L., et al. (2019) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/rssa.12544 · Cited by 8
“And no-one should embark on a randomized controlled trial that does not have at least 50% power to give the right answer (Turner et al., 2013) .... Christian Hennig (University of Bologna) I enjoyed reading the paper by Rice, Bonnett and Krakauer, and I agree that it gives a new view on twosided…”
BackgroundSelf-citationThe 1959 meeting in Vienna on controlled clinical trials – A methodological landmark
Bird, S. M. (2015) · Journal of the Royal Society of Medicine · doi:10.1177/0141076815595794 · Cited by 5
“for doctor.(3,4,12) Presciently, George Pickering also noted (p.”
BackgroundSelf-citationBeta-blockers for heart failure
Safi, S., Korang, S. K., Nielsen, E. E., Sethi, N. J., Feinberg, J. B., Gluud, C. N., & Jakobsen, J. C. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012897 · Cited by 5
RetractedCassese, S., Ndrepepa, G., Byrne, R. A., Laugwitz, K., Schunkert, H., Fusaro, M., Alfonso, F., & Kastrati, A. (2019) · American Heart Journal · doi:10.1016/j.ahj.2019.08.020 · Cited by 5
Rezazadegan, M., Mirjalili, F. S., Clark, C. C. T., & Rouhani, M. H. (2020) · British Journal Of Nutrition · doi:10.1017/s0007114520003268 · Cited by 7
“Indeed, the power of a meta-analysis strongly depends on number of included studies(58).”
BackgroundJenkins, D., Willett, W. C., Srichaikul, K., Khan, T. A., & Sievenpiper, J. L. (2024) · The Lancet Diabetes & Endocrinology · doi:10.1016/s2213-8587(24)00188-8 · Cited by 6
Occupational Therapy for Mental Health Conditions and Substance Use Disorders
Jafari, Z., & Grobelna, A. (2023) · Canadian Journal of Health Technologies · doi:10.51731/cjht.2023.617 · Cited by 5
Duan, Z., Wang, X., Zhang, Z.-Y., Wang, X., Zhang, Y., & Du, X. (2025) · Sleep Medicine · doi:10.1016/j.sleep.2025.106870 · Cited by 5
Willoughby, B., Smit, K., Wilson, I. M., Gilchrist, G., Hopkins, C., Tanyos, A., Saunders, M., Diress, G., & Laslett, A. (2025) · Trauma Violence & Abuse · doi:10.1177/15248380251383937 · Cited by 6
Chen, J., Dillon-Rossiter, K., Grigsby‐Duffy, L., Morava, A., Nović, A., Salmani, B., Smith, S. T., Prapavessis, H., & Gardiner, P. A. (2025) · Journal of Alzheimer s Disease · doi:10.1177/13872877251394751 · Cited by 4
“As meta-analyses generally require a minimum of fi ve studies addressing the same speci fi c research question to ensure suf fi cient statistical power and comparability of data, with 10 or more studies being the ideal threshold, 30 a meta-analysis was not feasible.”
Metadata
- Authors
- Turner, R. M., Bird, S. M., & Higgins, J. P. T.
- Affiliations
- MRC Biostatistics Unit (GB)
- University of Bristol (GB)
- Corresponding author
- Turner, Rebecca M.
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2013-03-27
- Volume
- 8
- Issue
- 3
- Pages
- e59202
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC3609745
- OpenAlex
- W2015824469
- Cited by
- 672
- References
- 25
- Citations by year
- 2026: 46 · 2025: 51 · 2024: 51 · 2023: 59 · 2022: 70 · 2021: 70 · 2020: 59 · 2019: 49 · 2018: 56 · 2017: 60 · 2016: 45 · 2015: 31 · 2014: 20 · 2013: 4
- FWCI
- 36.78
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, other-oa)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- UCL Discovery (University College London) (submittedVersion)
- Bristol Research (University of Bristol) (submittedVersion)
- Funders
- Medical Research Council (U105260794)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Keywords
- underpowered studies, study size, statistical power, meta-analysis, binary outcomes, relative risk, standard errors, study heterogeneity, rapid review
- MeSH terms
- Humans, Review Literature as Topic, Meta-Analysis as Topic, Odds Ratio, Sample Size, Risk Reduction Behavior
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
