The Impact of Study Size on Meta-analyses: Examination of Underpowered Studies in Cochrane Reviews
Open accessThe DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Most meta-analyses include data from one or more small studies that, individually, do not have power to detect an intervention effect. The relative influence of adequately powered and underpowered studies in published meta-analyses has not previously been explored. We examine the distribution of power available in studies within meta-analyses published in Cochrane reviews, and investigate the impact of underpowered studies on meta-analysis results. METHODS AND FINDINGS: For 14,886 meta-analyses of binary outcomes from 1,991 Cochrane reviews, we calculated power per study within each meta-analysis. We defined adequate power as ≥50% power to detect a 30% relative risk reduction. In a subset of 1,107 meta-analyses including 5 or more studies with at least two adequately powered and at least one underpowered, results were compared with and without underpowered studies. In 10,492 (70%) of 14,886 meta-analyses, all included studies were underpowered; only 2,588 (17%) included at least two adequately powered studies. 34% of the meta-analyses themselves were adequately powered. The median of summary relative risks was 0.75 across all meta-analyses (inter-quartile range 0.55 to 0.89). In the subset examined, odds ratios in underpowered studies were 15% lower (95% CI 11% to 18%, P<0.0001) than in adequately powered studies, in meta-analyses of controlled pharmacological trials; and 12% lower (95% CI 7% to 17%, P<0.0001) in meta-analyses of controlled non-pharmacological trials. The standard error of the intervention effect increased by a median of 11% (inter-quartile range -1% to 35%) when underpowered studies were omitted; and between-study heterogeneity tended to decrease. CONCLUSIONS: When at least two adequately powered studies are available in meta-analyses reported by Cochrane reviews, underpowered studies often contribute little information, and could be left out if a rapid review of the evidence is required. However, underpowered studies made up the entirety of the evidence in most Cochrane reviews.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“we pooled results excluding studies with fewer than 50 participants in the experimental group, as a planned sensitivity analysis”
From Effectiveness of psychological interventions in prison to reduce recidivism: a systematic review and meta-analysis of randomised controlled trials (2021) · cites this paper as [66]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“there is a risk that untested approaches may introduce new forms of bias 18 .”
From Software support for environmental evidence synthesis (2018) · cites this paper as [18]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“On collating the evidence, some studies may appear underpowered for the outcome(s) of interest to us (Turner 2013), yet were adequately powered for the studies' primary measure.”
From Physical activity and exercise for chronic pain in adults: an overview of Cochrane Reviews (2017) · cites this paper as Turner 2013
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Although our number of effect sizes (n=37) is relatively large in terms of outcomes analyzed, it is small in terms of statistical analyses that combine empirical studies [100].”
From Generative AI Mental Health Chatbots as Therapeutic Tools: Systematic Review and Meta-Analysis of Their Role in Reducing Mental Health Issues (2025) · cites this paper as [100]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Meta-analysis in clinical trials
DerSimonian, R., & Laird, N. (1986) · Controlled Clinical Trials · doi:10.1016/0197-2456(86)90046-2 · Cited by 39,552 · cited 2×
“In each meta-analysis j, we fitted a random-effects model, using a method-of-moments estimate for the between study variance [17], and calculated the variance Vj of the combined intervention effect (on the log relative risk scale).”
MethodSterne, J. A. C., Sutton, A. J., Ioannidis, J. P. A., Terrin, N., Jones, D. R., Lau, J., Carpenter, J., Rucker, G., Harbord, R. M., Schmid, C. H., et al. (2011) · BMJ · doi:10.1136/bmj.d4002 · Cited by 7,709 · cited 1×
“[24], who published new guidelines.”
Self-citationKjaergard, L. L., Villumsen, J., & Gluud, C. (2001) · Annals of Internal Medicine · doi:10.7326/0003-4819-135-11-200112040-00010 · Cited by 2,283 · cited 1×
“Alternatively, small-study effects may arise from biases caused by methodological flaws arising more frequently in small studies [3], or may be due to true differences in the underlying effects between smaller and larger studies.”
BackgroundPublication and related bias in meta-analysis
Sterne, J. A. C., Gavaghan, D., & Egger, M. (2000) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(00)00242-0 · Cited by 2,282
Trial sequential analysis may establish when firm evidence is reached in cumulative meta-analysis
Wetterslev, J., Thorlund, K., Brok, J., & Gluud, C. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.03.013 · Cited by 1,881
Brok, J., Thorlund, K., Gluud, C., & Wetterslev, J. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.10.007 · Cited by 1,113 · cited 1×
“[20], who had, however, excluded both reviews with fewer than three trials and those in which all trials had a high risk of bias, in which metaanalytic size is likely to have been smaller.”
ResultTotal Plasma Homocysteine and Cardiovascular Risk Profile: The Hordaland Homocysteine Study
Nygård, O., Vollset, S. E., Refsum, H., Stensvold, I., Tverdal, A., Nordrehaug, J. E., Ueland, P. M., & Kvåle, G. (1995) · JAMA · doi:10.1001/jama.1995.03530190040032 · Cited by 1,042
The scandal of poor medical research.
Altman, D. G. (1994) · BMJ · doi:10.1136/bmj.308.6924.283 · Cited by 897 · cited 1×
“Researchers who choose to undertake a study that is capable of detecting only an unrealistically large effect may lack understanding of both scientific methods and ethics [10].”
BackgroundThe power of statistical tests in meta-analysis
Hedges, L. V., & Pigott, T. D. (2001) · Psychological Methods · doi:10.1037/1082-989x.6.3.203 · Cited by 716 · cited 1×
“where W is the cumulative standard normal distribution function and Ca=2~W {1 1{a=2 ð Þ[18].”
BackgroundMeta‐analysis of drug‐related deaths soon after release from prison
Merrall, E. L. C., Kariminia, A., Binswanger, I. A., Hobbs, M. S., Farrell, M., Marsden, J., Hutchinson, S. J., & Bird, S. M. (2010) · Addiction · doi:10.1111/j.1360-0443.2010.02990.x · Cited by 674 · cited 1×
“In teaching, Bird has long advocated that trials should not be started unless they could deliver at least 50% power in respect of a priori plausible, worthwhile effect sizes [6].”
BackgroundSelf-citation
Cited by
Items 301–400 of 672
Aluminium adjuvants used in vaccines
Djurisic, S., Jakobsen, J. C., Petersen, S. B., Kenfelt, M., Klingenberg, S. L., & Gluud, C. N. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013086 · Cited by 26
Vesterling, C., & Koglin, U. (2020) · Journal of Psychosomatic Research · doi:10.1016/j.jpsychores.2020.109932 · Cited by 20
The assessment of replicability using the sum of p -values
Held, L., Pawel, S., & Micheloud, C. (2024) · Royal Society Open Science · doi:10.1098/rsos.240149 · Cited by 18
“Most original studies are designed to have 80% power to detect the assumed true effect size at significance level α , but the power can be considerably lower in reality [35,36].”
BackgroundRead, D. J., & Wainger, L. (2022) · Conservation Biology · doi:10.1111/cobi.14009 · Cited by 21
“, 2015), their inclusion likely widens coefficient standard errors and random-effect variances (Turner et al., 2013).”
BackgroundAntibiotics for secondary prevention of coronary heart disease
Sethi, N. J., Safi, S., Korang, S. K., Hróbjartsson, A., Skoog, M., Gluud, C. N., & Jakobsen, J. C. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd003610.pub3 · Cited by 19
Abdelnoor, M., Sandven, I., Limalanathan, S., & Eritsland, J. (2014) · Vascular Health and Risk Management · doi:10.2147/vhrm.s67154 · Cited by 21
Cassese, S., Ndrepepa, G., Byrne, R. A., Kufner, S., Lahmann, A. L., Mankerious, N., Xhepa, E., Laugwitz, K., Schunkert, H., Fusaro, M., et al. (2018) · EuroIntervention · doi:10.4244/eij-d-18-00024 · Cited by 19
Wride, J., & Bannigan, K. (2019) · Scandinavian Journal of Pain · doi:10.1515/sjpain-2018-0347 · Cited by 22
“This is a weakness seen in many studies across all fields [47].”
Kirkham, A. M., Bailey, A. J.-M., Shorr, R., Lalu, M. M., Fergusson, D. A., & Allan, D. (2022) · Cytotherapy · doi:10.1016/j.jcyt.2022.10.003 · Cited by 23
“This heterogeneity and lack of sufficient sample size meant that the authors’ meta-analyses lacked adequate statistical power to provide definitive conclusions regarding the efficacy of MSCs as a therapeutic intervention for COVID-19 [76,77].”
Bakker-Bons, M., Hijmering, R. M. J., Soer, R., & Wolff, A. P. (2025) · BJA Open · doi:10.1016/j.bjao.2025.100487 · Cited by 25
“76 However, the inclusion of studies with a high sample size and the use of Cohen ’ s d should lead to a decrease in sample bias, 76,77 thus decreasing the effect of a low sample size.”
Meta-analysis on the Association Between Thyroid Hormone Disorders and Arterial Stiffness
Bernardi, S., Grillo, A., Antonello, R. M., Cola, M. F., Dobrinja, C., Fabris, B., & Giudici, F. (2022) · Journal of the Endocrine Society · doi:10.1210/jendso/bvac016 · Cited by 22
“Small research studies are generally a concern, because they are unstable and thus prone to being underpowered considered individually, and they tend to show more treatment effects than larger studies [62].”
ResultWhat has preclinical systematic review ever done for us?
Russell, A., Sutherland, B. A., Landowski, L. M., Macleod, M. R., & Howells, D. W. (2022) · BMJ Open Science · doi:10.1136/bmjos-2021-100219 · Cited by 18
Gillespie, B. M., Thalib, L., Harbeck, E., Tobiano, G., Kang, E., Tobiano, S., Tong, M. Y. T., Clark, J. M., Patel, B. M., & Chaboyer, W. P. (2023) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2023.104471 · Cited by 25
Bansal, S. K., Gupta, G., & Rajender, S. (2016) · Mutation Research/Reviews in Mutation Research · doi:10.1016/j.mrrev.2016.04.007 · Cited by 18
“The studies using small sample size and other outlier studies are likely to affect the results of meta-analysis [31].”
BackgroundMethodAluminium adjuvants used in vaccines versus placebo or no intervention
Djurisic, S., Jakobsen, J. C., Petersen, S. B., Kenfelt, M., & Gluud, C. N. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012805 · Cited by 19
Hiemstra, B., Koster, G., Wetterslev, J., Gluud, C. N., Jakobsen, J. C., Scheeren, T., Keus, F., & van der Horst, I. C. C. (2018) · Acta Anaesthesiologica Scandinavica · doi:10.1111/aas.13294 · Cited by 18
Interventions to Mitigate Vaping Misinformation: A Meta-Analysis
Janmohamed, K., Walter, N., Sangngam, N., Hampsher, S., Nyhan, K., Choudhury, M. D., & Kumar, N. (2022) · Journal of Health Communication · doi:10.1080/10810730.2022.2044941 · Cited by 21
Chaabna, K., Chaabane, S., Jithesh, A., Doraiswamy, S., Mamtani, R., & Cheema, S. (2022) · Frontiers in Public Health · doi:10.3389/fpubh.2022.1009703 · Cited by 20
“To be included in the meta-analysis, there was a minimum study sample size of 25(34).”
BackgroundKiernan, M., & Baiocchi, M. (2022) · American Journal of Epidemiology · doi:10.1093/aje/kwac019 · Cited by 22
Tan, S. W., Wu, A., Cheng, L. J., Wong, S. H., Lau, Y., & Lau, S. T. (2022) · Dysphagia · doi:10.1007/s00455-022-10424-6 · Cited by 21
“However, the extreme treatment effect in this review could be attributed to the result of being underpowered through the use of only three studies with small sample sizes [77].”
BackgroundJeuland, M., Orgill‐Meyer, J., Morgan, S., Hudner, D., Pucilowski, M., Wyatt, A., Shafei, M., Cajka, J., & Albert, J. (2023) · Water Resources Research · doi:10.1029/2022wr033897 · Cited by 17
Scalamogna, M., Kuna, C., Voll, F., Aytekin, A., Lahu, S., Kessler, T., Kufner, S., Rheude, T. A., Sager, H. B., Xhepa, E., et al. (2023) · Clinical Research in Cardiology · doi:10.1007/s00392-023-02324-y · Cited by 17
“We calculated the power of our meta-analysis to detect a 50% relative risk difference for main outcomes with modified-balloon conditional on the observed precision of the pooled estimate [12].”
MethodPongpanit, K., Dayan, N., Janaudis‐Ferreira, T., Roig, M., Spahija, J., & Bertagnolli, M. (2024) · European Journal of Preventive Cardiology · doi:10.1093/eurjpc/zwae165 · Cited by 20
Xiao, F., Yan, K., Tang, M., Ji, X., Hu, L., Yang, L., & Zhou, W. (2022) · Expert Review of Molecular Diagnostics · doi:10.1080/14737159.2022.2123704 · Cited by 18
“This may result from methodological flaws [40] or true differences in the underlying effects between small sample and larger sample studies [41].”
BackgroundYang, X., Wang, L., Zhi, H., Li, X. A., & Wei, P. (2016) · Medicine · doi:10.1097/md.0000000000002846 · Cited by 18
Cassese, S., Belle, L., Ndrepepa, G., Bosson, J., Fusaro, M., Lønborg, J., Ahtarovski, K. A., Kelbæk, H., & Fusaro, M. (2018) · Canadian Journal of Cardiology · doi:10.1016/j.cjca.2018.07.480 · Cited by 17
Riberholt, C. G., Olsen, M. H., Milan, J. B., Hafliðadóttir, S. H., Svanholm, J. H., Pedersen, E. B., Lew, C. C. H., Asante, M. A., Ribeiro, J. P., Wagner, V., et al. (2024) · BMC Medical Research Methodology · doi:10.1186/s12874-024-02318-y · Cited by 18
Mandal, R. K., Khan, M. A., Hussain, A., Dar, S. A., Aloufi, S., Jawed, A., Wahid, M., Panda, A. K., Lohani, M., Akhter, N., et al. (2019) · Infection and Drug Resistance · doi:10.2147/idr.s188980 · Cited by 16
Wan, J. S. H., & Liew, E. C. Y. (2020) · Journal of Plant Pathology · doi:10.1007/s42161-020-00519-0 · Cited by 17
Bellal, S., Feghali, R. E., Mehta, A., Namachivayam, A., & Benedicenti, S. (2021) · Lasers in Medical Science · doi:10.1007/s10103-021-03391-1 · Cited by 16
Westphal, A., Kriston, L., Hölzel, L. P., Härter, M., & von Wolff, A. (2014) · Journal of Public Health Research · doi:10.4081/jphr.2014.177 · Cited by 15
Marker, S., Perner, A., Wetterslev, J., Barbateskovic, M., Jakobsen, J. C., Krag, M., Granholm, A., Anthon, C. T., & Møller, M. H. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0509-4 · Cited by 17
“Statistically significant meta-analyses with few participants have low reliability, and the interventional effect is often overrated [45, 53].”
BackgroundWeighted corrected covered area (wCCA): A measure of informational overlap among reviews
Ying, X., Bougioukas, K. I., Pieper, D., & Mayo‐Wilson, E. (2025) · Research Synthesis Methods · doi:10.1017/rsm.2025.19 · Cited by 17
“Thus, large studies are given greater weight than small studies—without being dominant—reflecting their typical influence on meta-analyses and SR conclusions.”
Oltra‐Cucarella, J., Pérez-Elvira, R., Espert, R., & McCormick, A. S. (2016) · Neuropsychology · doi:10.1037/neu0000283 · Cited by 22
“Turner, Bird, and Higgins (2013) found that in 70% of a Cochrane meta-analyses review all studies were underpowered, whereas 66% of the meta-analyses themselves were underpowered.”
Buggeskov, K. B., Grønlykke, L., Risom, E. C., Wei, M. L., & Wetterslev, J. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011098.pub2 · Cited by 16
Alushi, B., Lauten, A., Cassese, S., Colleran, R., Schüpke, S., Rai, H., Schunkert, H., Meier, B., Landmesser, U., & Kastrati, A. (2018) · Clinical Research in Cardiology · doi:10.1007/s00392-018-1246-y · Cited by 15
The power of meta-analysis: a challenge for evidence-based medicine
Berchialla, P., Chiffi, D., Valente, G., & Voutilainen, A. (2020) · European Journal for Philosophy of Science · doi:10.1007/s13194-020-00321-w · Cited by 17
Antibiotics for hospital-acquired pneumonia in neonates and children
Korang, S. K., Nava, C., Mohana, S. P., Nygaard, U., & Jakobsen, J. C. (2021) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013864.pub2 · Cited by 21
Early aspirin discontinuation after coronary stenting: a systematic review and meta-analysis
Wiebe, J., Ndrepepa, G., Kufner, S., Lahmann, A. L., Xhepa, E., Kuna, C., Voll, F., Gosetti, R., Laugwitz, K., Joner, M., et al. (2021) · Journal of the American Heart Association · doi:10.1161/jaha.120.018304 · Cited by 15
McGill, K., Sackley, C., Godwin, J., Gavaghan, D., Ali, M., Ballester, B. R., Brady, M. C., Brady, M. C., Ali, M., Ashburn, A., et al. (2022) · Journal of Stroke and Cerebrovascular Diseases · doi:10.1016/j.jstrokecerebrovasdis.2021.106229 · Cited by 14
Ville, A., McRae, R., Nomchong, J., Reidlinger, D. P., Davidson, A. R., Staudacher, H. M., & Albarqouni, L. (2025) · Journal of Human Nutrition and Dietetics · doi:10.1111/jhn.70106 · Cited by 16
Marsman, M. S., Wetterslev, J., Jahrome, A. K., Gluud, C. N., Moll, F. L., Karimi, A., Keus, F., & Koning, G. G. (2019) · BMJ Open · doi:10.1136/bmjopen-2018-026419 · Cited by 15
Blood Purification Studies in the ICU: What Endpoints Should We Use?
Forni, L. G. (2022) · Blood Purification · doi:10.1159/000523761 · Cited by 15
Finkelstein, T., Zhang, Y., Vollenhoven, B. J., Rolnik, D. L., & Horta, F. (2023) · European Journal of Obstetrics & Gynecology and Reproductive Biology · doi:10.1016/j.ejogrb.2023.11.004 · Cited by 15
Trial sequential analysis methodology for interpreting meta-analytical findings
Greco, A., & Capodanno, D. (2024) · European Journal of Internal Medicine · doi:10.1016/j.ejim.2023.12.029 · Cited by 14
How much does the typical ecological meta‐analysis overestimate the true mean effect size?
Fox, J. W. (2022) · Ecology and Evolution · doi:10.1002/ece3.9521 · Cited by 16
“Meta- meta- analysis can be used to compare the statistical properties of different meta- analyses, such as Type I error rate, heterogeneity, and statistical power (Cafri et al., 2010; Senior et al., 2016; Turner et al., 2013).”
MethodBabiloni, A. H., Bouferguene, Y., Exposto, F. G., Beauregard, R., Lavigne, G. J., Moana-Filho, E. J., & Arbour, C. (2023) · Pain · doi:10.1097/j.pain.0000000000002949 · Cited by 18
Nielsen, B., Henneberg, S. W., Schmiegelow, K., Friis, S. M., & Rømsing, J. (2015) · Acta Anaesthesiologica Scandinavica · doi:10.1111/aas.12529 · Cited by 15
“A ‘Risk of bias’ table was completed for each included study, assigning ‘low risk of bias’, ‘unclear risk of bias’, or ‘high risk of bias’ for random sequence generation, allocation concealment, blinding of outcome assessment, incomplete outcome data, selective outcome reporting, and other biases including study size…”
MethodQureshi, A. I., Ishfaq, M. F., Rahman, H., & Thomas, A. P. (2016) · Journal of Stroke and Cerebrovascular Diseases · doi:10.1016/j.jstrokecerebrovasdis.2016.03.053 · Cited by 14
Areeshi, M. Y., Mandal, R. K., Dar, S. A., Jawed, A., Wahid, M., Lohani, M., Panda, A. K., Mishra, B. N., Akhter, N., & Haque, S. (2017) · Scientific Reports · doi:10.1038/s41598-017-17308-y · Cited by 14
Meta-Analysis and Public Policy: Reconciling the Evidence on Deworming
Croke, K. G., Hicks, J. H., Hsu, E., Kremer, M. R., Maertens, R., Miguel, E. A., & Więcek, W. (2016) · National Bureau of Economic Research · doi:10.3386/w22382 · Cited by 15
Bruyère, O., Alokail, M. S., Al‐Daghri, N. M., Reginster, J.-Y., & Sabico, S. (2025) · Maturitas · doi:10.1016/j.maturitas.2025.108779 · Cited by 18
Impulsivity as a trait in domestic dogs (Canis familiaris): A systematic review and meta-analysis.
Barela, J., Worth, Y., & Stevens, J. R. (2023) · Journal of comparative psychology · doi:10.1037/com0000352 · Cited by 14
Chang, T., Shih, M., Wu, Y.-L., Wu, T.-T., Yang, J.-T., & Wu, C. (2024) · British Journal of Anaesthesia · doi:10.1016/j.bja.2024.08.008 · Cited by 14
“37 In 2024, two updated systematic reviews were published, revealing findings similar to those of the present study regarding the effects of balanced crystalloids on DGF. 38,39 However, significant differences exist between these reviews and our study.”
Post-traumatic stress disorder in peacekeepers: a systematic literature review and meta-analysis
Carmona, L., Camilo, C., Carvalho, V. S., & Chambel, M. J. (2024) · European Journal of Psychotraumatology · doi:10.1080/20008066.2024.2413735 · Cited by 21
“Small sample sizes in several studies were noted, potentially limiting the reliability of the findings and heightening the impact of publication bias (e.g., Turner et al., 2013).”
Milne‐Ives, M., Burns, L., Swancutt, D. R., Calitri, R., Ananthakrishnan, A., Davis, H., Pinkney, J. H., Tarrant, M., & Meinert, E. (2024) · International Journal of Obesity · doi:10.1038/s41366-024-01669-2 · Cited by 17
Lee, W., Pulbrook, M., Sheehan, C., Kochovska, S., Chang, S., Hosie, A., Lobb, E. A., Parker, D., Draper, B., Agar, M. R., et al. (2020) · Journal of Pain and Symptom Management · doi:10.1016/j.jpainsymman.2020.07.011 · Cited by 15
Oliveira, C. M., Henderson, L. M., & Hayiou‐Thomas, M. E. (2023) · Cognitive Science · doi:10.1111/cogs.13310 · Cited by 15
Label‐invariant models for the analysis of meta‐epidemiological data
Rhodes, K. M., Mawdsley, D., Turner, R., Jones, H. E., Savović, J., & Higgins, J. P. T. (2017) · Statistics in Medicine · doi:10.1002/sim.7491 · Cited by 13
“Empirical studies have investigated the impact of small studies on meta‐analysis results.(8,17,18) There is empirical evidence to suggest that estimates of intervention effect are exaggerated in smaller studies, but little attention has been paid to small study effects on heterogeneity.”
BackgroundSelf-citationSubstance use and substance use disorder, in relation to COVID-19: protocol for a scoping review
Kumar, N., Janmohamed, K., Nyhan, K., Martins, S. S., Cerdá, M., Hasin, D. S., Scott, J., Pates, R. M., Ghandour, L. A., Wazaify, M., et al. (2021) · Systematic Reviews · doi:10.1186/s13643-021-01605-9 · Cited by 17
Ndlovu, S., Naqshband, M. S., Masunda, S., Ndlovu, K., Chettiar, K., & Anugraha, A. (2023) · The Bone & Joint Journal · doi:10.1302/0301-620x.105b1.bjj-2022-0934.r1 · Cited by 12
Raj, S., Bola, H., & York, T. J. (2023) · Journal of Orthopaedics · doi:10.1016/j.jor.2023.04.007 · Cited by 12
Atanassova, D. V., Madariaga, V. I., Oosterman, J. M., & Brazil, I. A. (2024) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2024.105786 · Cited by 17
Effects of mental fatigue on Olympic ball sports performance: a systematic review
Pan, X., Soh, K. G., Jaafar, W. M. W., Soh, K. L., Li, M., Liu, H., & Liu, C. (2025) · Acta Psychologica · doi:10.1016/j.actpsy.2025.105109 · Cited by 13
Lakbar, I., Poole, D., Delamarre, L., Chanques, G., Pensier, J., Monet, C., Belafia, F., Capdevila, M., JONG, A. D., & Jaber, S. (2025) · Intensive Care Medicine · doi:10.1007/s00134-025-08143-1 · Cited by 18
Sorin, M., Okde, R., Goulet, M., Ghaleb, L., Pang, R., Prosty, C. J., Brailovski, E., & Assouline, S. E. (2025) · The Lancet Haematology · doi:10.1016/s2352-3026(25)00318-7 · Cited by 14
Jakobsen, J. C., Wetterslev, J., & Gluud, C. N. (2019) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2018-111154 · Cited by 13
Peñarrocha-Oltra, S., Soto‐Peñaloza, R., Alonso‐Arroyo, A., Vidal‐Infer, A., & Pascual‐Segarra, J. (2022) · Acta Ophthalmologica · doi:10.1111/aos.15160 · Cited by 13
“However, not every published review has a proper design and execution or may not have enough accrued data (e.g. few randomized participants or studies) to obtain sufficient statistical power to allow reliable assessment of large anticipated intervention effects (Turner et al. 2013).”
BackgroundPayne, T., Moran, B., Loadsman, J. A., Marschner, I. C., McCulloch, T. J., & Sanders, R. (2023) · British Journal of Anaesthesia · doi:10.1016/j.bja.2023.01.011 · Cited by 14
Sexual health and COVID-19: protocol for a scoping review
Kumar, N., Janmohamed, K., Nyhan, K., Forastiere, L., Zhang, W., Kågesten, A., Uhlich, M., de Velde, S. M. V., Francis, J. M., Erausquin, J. T., et al. (2021) · Systematic Reviews · doi:10.1186/s13643-021-01591-y · Cited by 12
Willingness to pay for cleaning up beach litter: A meta-analysis
Mutuku, J., Yanotti, M., Tinch, D., & MacDonald, D. H. (2022) · Marine Pollution Bulletin · doi:10.1016/j.marpolbul.2022.114220 · Cited by 15
Taboła, R., Cirocchi, R., Fingerhut, A. L., Arezzo, A., Randolph, J., Grassi, V., Binda, G. A., D’Andrea, V., Abraha, I., Popivanov, G., et al. (2017) · Techniques in Coloproctology · doi:10.1007/s10151-017-1583-2 · Cited by 14
“that underpowered studies form the data repository of most of the Cochrane systematic reviews [41].”
BackgroundYan, H., Hong, Y., & Cai, Y. (2019) · Bioscience Reports · doi:10.1042/bsr20191197 · Cited by 14
The quest for the holy volume therapy
ROBERTIS, E. D., Afshari, A., & Longrois, D. (2016) · European Journal of Anaesthesiology · doi:10.1097/eja.0000000000000441 · Cited by 12
Drug-eluting stents versus bare-metal stents for acute coronary syndrome
Feinberg, J. B., Nielsen, E. E., Greenhalgh, J., Hounsome, J. C., Sethi, N. J., Safi, S., Gluud, C. N., & Jakobsen, J. C. (2016) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012481 · Cited by 13
Jafari, Z., Fitzpatrick, E. M., Schramm, D., Rouillon, I., & Koravand, A. (2024) · American Journal of Audiology · doi:10.1044/2024_aja-23-00272 · Cited by 11
Underpowered studies in muscle metabolism research: Determinants and considerations
Houtvast, D. C. J., Betz, M. W., Hooren, B. V., Vanbelle, S., Verdijk, L. B., van Loon, L. J. C., & Trommelen, J. (2024) · Clinical Nutrition ESPEN · doi:10.1016/j.clnesp.2024.10.152 · Cited by 13
Sex Differences in the Prevalence of Chronic Pain in Mid-Life: A Systematic Review and Meta-Analysis
Borra, C., Pawson, J., Rich, N., & Hardy, R. (2025) · Biomedicines · doi:10.3390/biomedicines13102523 · Cited by 13
“Random-effect meta-analysis accounts for between-study heterogeneity and differences in sample sizes by assigning smaller weight to studies with small sample sizes, meaning that such studies contribute little information to the meta-analysis [74].”
Cannabis-based medicines for chronic neuropathic pain in adults
Ateş, G., Welsch, P., Klose, P., Phillips, T., Lambers, B., Häuser, W., & Radbruch, L. (2026) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012182.pub3 · Cited by 20
Surgical resection after endoscopic resection in patients with T1 colorectal cancer: a meta-analysis
Tian, Y., Rong, L., & Ma, Y. (2020) · International Journal of Colorectal Disease · doi:10.1007/s00384-020-03752-2 · Cited by 13
Moore, A., Fisher, E., Häuser, W., Bell, R. F., Perrot, S., Bidonde, J., Makri, S., & Straube, S. (2021) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013151.pub2 · Cited by 12
Buck, B., Gagen, E. C., Halverson, T. F., Nagendra, A., Ludwig, K., & Fortney, J. C. (2021) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2021.12.053 · Cited by 12
Riberholt, C. G., Olsen, M. H., Milan, J. B., & Gluud, C. N. (2022) · Systematic Reviews · doi:10.1186/s13643-022-01987-4 · Cited by 12
Bower-Cargill, C., Yarandi, N., & Petróczi, A. (2022) · Phytomedicine Plus · doi:10.1016/j.phyplu.2022.100326 · Cited by 11
Søndergaard, M. H., Thavarajah, J. J., Henson, H. C., & Wejse, C. M. (2023) · HIV Medicine · doi:10.1111/hiv.13537 · Cited by 13
“This is referred to as the small-study effect [23, 24].”
BackgroundÅsberg, A. N., Heuch, I., & Hagen, K. (2016) · Musculoskeletal Care · doi:10.1002/msc.1156 · Cited by 11
Barbateskovic, M., Schjørring, O. L., Jakobsen, J. C., Meyhoff, C. S., Dahl, R. M., Rasmussen, B. S., Perner, A., & Wetterslev, J. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012631 · Cited by 11
Kiabi, F. H., Soleimani, A., & Habibi, M. R. (2019) · Brazilian Journal of Cardiovascular Surgery · doi:10.21470/1678-9741-2018-0263 · Cited by 11
Multiverse to inform neurological research: an example using recovery outcome of neglect
Moore, M. J., & Demeyere, N. (2021) · Journal of Neurology · doi:10.1007/s00415-021-10612-8 · Cited by 13
The Fragility Index of randomized controlled trials in pediatric anesthesiology
Hayes, J., Zuercher, M., Gai, N., Chowdhury, A. R., & Aoyama, K. (2023) · Canadian Journal of Anesthesia/Journal canadien d anesthésie · doi:10.1007/s12630-023-02513-3 · Cited by 11
“The publication of small underpowered trials with significant results may contribute to publication bias and mislead clinicians about the true benefit or risk of an intervention and contribute to poorer patient outcomes.(20) An analysis of trials published in high impact medical journals in 2005–2006 found that only 3…”
BackgroundInfluentialKhan, S. A., Dar, S. A., Mandal, R. K., Jawed, A., Wahid, M., Panda, A. K., Lohani, M., Mishra, B. N., Akhter, N., & Haque, S. (2018) · Frontiers in Physiology · doi:10.3389/fphys.2018.01793 · Cited by 11
Encalada, S., Hunt, C., Duszynski, B. S., Salmasi, V., Scholten, P., Zhao, Z., Rappard, G., Rivers, W. E., Vu, T.-N., Lobel, S. M., et al. (2025) · Interventional Pain Medicine · doi:10.1016/j.inpm.2025.100569 · Cited by 15
“The review was restricted to studies with larger cohorts to focus on adequately powered studies, given that when enough such studies exist, including smaller or inadequately powered studies contributes little additional helpful information [17].”
Budhathoki, M., Phuyal, S., Pandey, S., & Thomsen, M. (2025) · Reviews in Aquaculture · doi:10.1111/raq.70069 · Cited by 15
Lorenzen, U. S., Buggeskov, K. B., Nielsen, E. E., Sethi, N. J., Carranza, C. L., Gluud, C. N., & Jakobsen, J. C. (2019) · Systematic Reviews · doi:10.1186/s13643-019-1155-9 · Cited by 12
“The required information size takes into account the event proportion in the control group, the assumption of a plausible relative risk (RR) reduction and the heterogeneity of the metaanalysis [69, 73, 74].”
BackgroundCarrasco, J., Daïen, V., Eldem, B. M., Spoorendonk, J. A., & Yoon, J. (2021) · Ophthalmology and Therapy · doi:10.1007/s40123-021-00350-5 · Cited by 10
“There was an attempt to reduce publication bias [40] and uncertainty associated with small sample sizes [41] by excluding studies that reported \ 40 patients/eyes from the meta-analysis.”
Priyanka, Sharma, M., & Sharma, S. (2021) · International Journal of Mycobacteriology · doi:10.4103/ijmy.ijmy_108_21 · Cited by 10
Babić, A., Peričić, T. P., Pieper, D., & Puljak, L. (2022) · Research Synthesis Methods · doi:10.1002/jrsm.1556 · Cited by 12
“Similar approach can be taken with meta-analyses, but it has been shown that few meta-analyses ever reach a sufficient sample size.(17) It has been suggested that trial sequential analysis (TSA) may establish when firm evidence is reached in meta-analyses.”
ResultKaddoura, R., Ibrahim, M. I. M., Al‐Badriyeh, D., Omar, A. S., Alkindi, F., & Arabi, A. (2022) · PLoS ONE · doi:10.1371/journal.pone.0263270 · Cited by 14
“I 2 values of more than 50% were considered highly heterogeneous [73–75].”
Millar, J. E., Boyle, A. J., Drake, T. M., Adams, C. E., Glass, A. W., Blackwood, B., McNamee, J. J., & McAuley, D. F. (2022) · European Respiratory Review · doi:10.1183/16000617.0030-2022 · Cited by 10
“The combined effect is that the pooled estimates may not provide increased precision over evaluation of the studies individually, particularly when considering subgroups ( e.g. those with ARDS) [48].”
Gillespie, B. M., Harbeck, E., Sandy‐Hodgetts, K., Rattray, M., Thalib, L., Patel, B. M., Andersson, A. E., Walker, R., Latimer, S. L., & Chaboyer, W. P. (2023) · Journal of Wound Care · doi:10.12968/jowc.2023.32.sup8a.s31 · Cited by 14
Metadata
- Authors
- Turner, R. M., Bird, S. M., & Higgins, J. P. T.
- Affiliations
- MRC Biostatistics Unit (GB)
- University of Bristol (GB)
- Corresponding author
- Turner, Rebecca M.
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2013-03-27
- Volume
- 8
- Issue
- 3
- Pages
- e59202
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC3609745
- OpenAlex
- W2015824469
- Cited by
- 672
- References
- 25
- Citations by year
- 2026: 46 · 2025: 51 · 2024: 51 · 2023: 59 · 2022: 70 · 2021: 70 · 2020: 59 · 2019: 49 · 2018: 56 · 2017: 60 · 2016: 45 · 2015: 31 · 2014: 20 · 2013: 4
- FWCI
- 36.78
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, other-oa)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- UCL Discovery (University College London) (submittedVersion)
- Bristol Research (University of Bristol) (submittedVersion)
- Funders
- Medical Research Council (U105260794)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Keywords
- underpowered studies, study size, statistical power, meta-analysis, binary outcomes, relative risk, standard errors, study heterogeneity, rapid review
- MeSH terms
- Humans, Review Literature as Topic, Meta-Analysis as Topic, Odds Ratio, Sample Size, Risk Reduction Behavior
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
