The Impact of Study Size on Meta-analyses: Examination of Underpowered Studies in Cochrane Reviews
Open accessThe DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Most meta-analyses include data from one or more small studies that, individually, do not have power to detect an intervention effect. The relative influence of adequately powered and underpowered studies in published meta-analyses has not previously been explored. We examine the distribution of power available in studies within meta-analyses published in Cochrane reviews, and investigate the impact of underpowered studies on meta-analysis results. METHODS AND FINDINGS: For 14,886 meta-analyses of binary outcomes from 1,991 Cochrane reviews, we calculated power per study within each meta-analysis. We defined adequate power as ≥50% power to detect a 30% relative risk reduction. In a subset of 1,107 meta-analyses including 5 or more studies with at least two adequately powered and at least one underpowered, results were compared with and without underpowered studies. In 10,492 (70%) of 14,886 meta-analyses, all included studies were underpowered; only 2,588 (17%) included at least two adequately powered studies. 34% of the meta-analyses themselves were adequately powered. The median of summary relative risks was 0.75 across all meta-analyses (inter-quartile range 0.55 to 0.89). In the subset examined, odds ratios in underpowered studies were 15% lower (95% CI 11% to 18%, P<0.0001) than in adequately powered studies, in meta-analyses of controlled pharmacological trials; and 12% lower (95% CI 7% to 17%, P<0.0001) in meta-analyses of controlled non-pharmacological trials. The standard error of the intervention effect increased by a median of 11% (inter-quartile range -1% to 35%) when underpowered studies were omitted; and between-study heterogeneity tended to decrease. CONCLUSIONS: When at least two adequately powered studies are available in meta-analyses reported by Cochrane reviews, underpowered studies often contribute little information, and could be left out if a rapid review of the evidence is required. However, underpowered studies made up the entirety of the evidence in most Cochrane reviews.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“we pooled results excluding studies with fewer than 50 participants in the experimental group, as a planned sensitivity analysis”
From Effectiveness of psychological interventions in prison to reduce recidivism: a systematic review and meta-analysis of randomised controlled trials (2021) · cites this paper as [66]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“there is a risk that untested approaches may introduce new forms of bias 18 .”
From Software support for environmental evidence synthesis (2018) · cites this paper as [18]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“On collating the evidence, some studies may appear underpowered for the outcome(s) of interest to us (Turner 2013), yet were adequately powered for the studies' primary measure.”
From Physical activity and exercise for chronic pain in adults: an overview of Cochrane Reviews (2017) · cites this paper as Turner 2013
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Although our number of effect sizes (n=37) is relatively large in terms of outcomes analyzed, it is small in terms of statistical analyses that combine empirical studies [100].”
From Generative AI Mental Health Chatbots as Therapeutic Tools: Systematic Review and Meta-Analysis of Their Role in Reducing Mental Health Issues (2025) · cites this paper as [100]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Meta-analysis in clinical trials
DerSimonian, R., & Laird, N. (1986) · Controlled Clinical Trials · doi:10.1016/0197-2456(86)90046-2 · Cited by 39,552 · cited 2×
“In each meta-analysis j, we fitted a random-effects model, using a method-of-moments estimate for the between study variance [17], and calculated the variance Vj of the combined intervention effect (on the log relative risk scale).”
MethodSterne, J. A. C., Sutton, A. J., Ioannidis, J. P. A., Terrin, N., Jones, D. R., Lau, J., Carpenter, J., Rucker, G., Harbord, R. M., Schmid, C. H., et al. (2011) · BMJ · doi:10.1136/bmj.d4002 · Cited by 7,709 · cited 1×
“[24], who published new guidelines.”
Self-citationKjaergard, L. L., Villumsen, J., & Gluud, C. (2001) · Annals of Internal Medicine · doi:10.7326/0003-4819-135-11-200112040-00010 · Cited by 2,283 · cited 1×
“Alternatively, small-study effects may arise from biases caused by methodological flaws arising more frequently in small studies [3], or may be due to true differences in the underlying effects between smaller and larger studies.”
BackgroundPublication and related bias in meta-analysis
Sterne, J. A. C., Gavaghan, D., & Egger, M. (2000) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(00)00242-0 · Cited by 2,282
Trial sequential analysis may establish when firm evidence is reached in cumulative meta-analysis
Wetterslev, J., Thorlund, K., Brok, J., & Gluud, C. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.03.013 · Cited by 1,881
Brok, J., Thorlund, K., Gluud, C., & Wetterslev, J. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.10.007 · Cited by 1,113 · cited 1×
“[20], who had, however, excluded both reviews with fewer than three trials and those in which all trials had a high risk of bias, in which metaanalytic size is likely to have been smaller.”
ResultTotal Plasma Homocysteine and Cardiovascular Risk Profile: The Hordaland Homocysteine Study
Nygård, O., Vollset, S. E., Refsum, H., Stensvold, I., Tverdal, A., Nordrehaug, J. E., Ueland, P. M., & Kvåle, G. (1995) · JAMA · doi:10.1001/jama.1995.03530190040032 · Cited by 1,042
The scandal of poor medical research.
Altman, D. G. (1994) · BMJ · doi:10.1136/bmj.308.6924.283 · Cited by 897 · cited 1×
“Researchers who choose to undertake a study that is capable of detecting only an unrealistically large effect may lack understanding of both scientific methods and ethics [10].”
BackgroundThe power of statistical tests in meta-analysis
Hedges, L. V., & Pigott, T. D. (2001) · Psychological Methods · doi:10.1037/1082-989x.6.3.203 · Cited by 716 · cited 1×
“where W is the cumulative standard normal distribution function and Ca=2~W {1 1{a=2 ð Þ[18].”
BackgroundMeta‐analysis of drug‐related deaths soon after release from prison
Merrall, E. L. C., Kariminia, A., Binswanger, I. A., Hobbs, M. S., Farrell, M., Marsden, J., Hutchinson, S. J., & Bird, S. M. (2010) · Addiction · doi:10.1111/j.1360-0443.2010.02990.x · Cited by 674 · cited 1×
“In teaching, Bird has long advocated that trials should not be started unless they could deliver at least 50% power in respect of a priori plausible, worthwhile effect sizes [6].”
BackgroundSelf-citation
Cited by
Items 101–200 of 672
Chua, T. C., Yang, T. X., Gill, A. J., & Samra, J. S. (2015) · Annals of Surgical Oncology · doi:10.1245/s10434-015-4826-3 · Cited by 74
“The selection criteria were: all studies [20 patients reporting enucleation of pancreatic lesions.(8) Studies that included nonpancreatic neuroendocrine tumors were included if data on complications were presented and/or the results of the neuroendocrine subjects were analyzed separately and clearly reported.”
BackgroundMasarwa, R., Paret, G., Perlman, A., Reif, S. S., Raccah, B. H., & Matok, I. (2017) · Critical Care · doi:10.1186/s13054-016-1589-6 · Cited by 76
“Trial sequential analysis Meta-analyses may result in type I errors owing to an increased risk of random error when sparse data are analyzed [24].”
Kokje, V. B. C., Hamming, J. F., & Lindeman, J. H. N. (2015) · European Journal of Vascular and Endovascular Surgery · doi:10.1016/j.ejvs.2015.08.010 · Cited by 86
Wee, S. K., Hughes, A., Warner, M. B., & Burridge, J. H. (2014) · Neurorehabilitation and neural repair · doi:10.1177/1545968314521011 · Cited by 82
Kindred, R., Bates, G. W., & McBride, N. L. (2022) · Journal of Anxiety Disorders · doi:10.1016/j.janxdis.2022.102640 · Cited by 87
Carvalho, L. D. F., Sette, C. P., & Ferrari, B. (2018) · Cyberpsychology Journal of Psychosocial Research on Cyberspace · doi:10.5817/cp2018-3-5 · Cited by 83
Anitha, S., Givens, D. I., Botha, R., Kane‐Potaka, J., Sulaiman, N. S., Tsusaka, T. W., Subramaniam, K., Rajendran, A., Parasannanavar, D. J., & Bhandari, R. K. (2021) · Sustainability · doi:10.3390/su13168677 · Cited by 88
“These limitations suggest the requirement of more studies on this topic, especially considering the importance of finger millet as a sustainable dietary option [44].”
BackgroundSimmonds, M. C., Elliott, J., Synnot, A., & Turner, T. (2021) · Methods in molecular biology · doi:10.1007/978-1-0716-1566-9_7 · Cited by 78
Panizzutti, B., Skvarc, D., Lin, S. C., Croce, S., Meehan, A., Bortolasci, C. C., Marx, W., Walker, A. J., Hasebe, K., Kavanagh, B. E., et al. (2023) · International Journal of Molecular Sciences · doi:10.3390/ijms24065250 · Cited by 74
Boone, J., Vandekerckhove, K., Coomans, I., Prieur, F., & Bourgois, J. G. (2016) · European Journal of Applied Physiology · doi:10.1007/s00421-016-3468-x · Cited by 68
“Additionally, it has been shown that underpowered studies can substantially effect the results of systematic reviews (Illi et al. 2012; Turner et al. 2013b) and, therefore, the results should be considered with care.”
BackgroundPrasitlumkum, N., Cheungpasitporn, W., Chokesuwattanaskul, A., Thangjui, S., Thongprayoon, C., Bathini, T. J., Vallabhajosyula, S., Kanitsoraphan, C., Leesutipornchai, T., & Chokesuwattanaskul, R. (2020) · Archives of cardiovascular diseases · doi:10.1016/j.acvd.2020.05.015 · Cited by 78
Tramadol with or without paracetamol (acetaminophen) for cancer pain
Wiffen, P. J., Derry, S., & Moore, R. A. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012508.pub2 · Cited by 79
Lavazza, C., Galli, M., Abenavoli, A., & Maggiani, A. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-045106 · Cited by 67
Paediatric, maternal, and congenital mpox: a systematic review and meta-analysis
Clemente, N. S., Coles, C., Paixão, E. S., Brickley, E. B., Whittaker, E., Alfvén, T., Rulisa, S., Higuita, N. I. A., Torpiano, P., Agravat, P., et al. (2024) · The Lancet Global Health · doi:10.1016/s2214-109x(23)00607-1 · Cited by 66
“103 The ability to collect high-quality data in endemic regions is hampered by armed conflict and flooding 104 affecting the highly endemic areas of Kasaï Oriental, North Kivu, and South Kivu in eastern DR Congo for over 30 years.”
Caldeira, D., Barra, M., Ferreira, A., Rocha, A., Augusto, A., Pinto, F. J., Costa, J. N., & Ferreira, J. J. (2015) · Alimentary Pharmacology & Therapeutics · doi:10.1111/apt.13412 · Cited by 74
Kelley, G. A., & Kelley, K. S. (2017) · BMC Cancer · doi:10.1186/s12885-017-3687-5 · Cited by 66
“For example, a post-hoc analysis of Cochrane meta-analyses found that only 22% of the included studies demonstrated at least 80% power to detect a relative risk reduction of 30% [90].”
MethodMcGill, K., Sackley, C. M., Godwin, J., McGarry, J., & Brady, M. (2020) · Trials · doi:10.1186/s13063-019-3991-2 · Cited by 73
Carcel, C., Woodward, M., Balicki, G., Koroneos, G. L., de Sousa, D. A., Cordonnier, C., Lukaszyk, C., Thompson, K., Wang, X., Davies, L., et al. (2019) · International Journal of Stroke · doi:10.1177/1747493019851292 · Cited by 64
Maia, C. R. M., Cortese, S., Caye, A., Deakin, T. K., Polanczyk, G. V., Polanczyk, C. A., & Rohde, L. A. P. (2017) · Journal of Attention Disorders · doi:10.1177/1087054714559643 · Cited by 65
Increased risks for random errors are common in outcomes graded as high certainty of evidence
Gartlehner, G., Nußbaumer-Streit, B., Wagner, G., Patel, S. V., Evans, T. S., Dobrescu, A. I., & Gluud, C. N. (2018) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2018.10.009 · Cited by 63
Abraham, A., Chaabna, K., Sheikh, J. I., Mamtani, R., Jithesh, A., Khawaja, S., & Cheema, S. (2024) · Scientific Reports · doi:10.1038/s41598-024-52923-6 · Cited by 86
“To be included in the MA, the minimum study sample size was 25 57 .”
BackgroundCaldeira, D., Nunes-Ferreira, A., Rodrigues, R. O., Vicente, E., Pinto, F. J., & Ferreira, J. J. (2018) · Archives of Gerontology and Geriatrics · doi:10.1016/j.archger.2018.12.013 · Cited by 64
Does Mass Deworming Affect Child Nutrition? Meta-Analysis, Cost-Effectiveness, and Statistical Power
Croke, K. G., Hicks, J. H., Hsu, E., Kremer, M. R., & Miguel, E. A. (2016) · World Bank, Washington, DC eBooks · doi:10.1596/1813-9450-7921 · Cited by 68
Meta-Analysis Study: The Relationship Between Reflective Thinking And Learning Achievement
Chamdani, M., Yusuf, F. A., Salimi, M., & Fajari, L. E. W. (2022) · Journal on Efficiency and Responsibility in Education and Science · doi:10.7160/eriesj.2022.150305 · Cited by 84
“The data can be expressed by various measures that are calculated or searched in advance by formulas in various mathematical equations, which are closely related to the research objectives of the meta-analysis carried out (Pereira et al., 2019; Turner, Bird and Higgins, 2013).”
MethodTowards best practice in physical and physiological employment standards
Petersen, S. R., Andérson, G. S., Tipton, M., Docherty, D., Graham, T. E., Sharkey, B. J., & Taylor, N. A. S. (2016) · Applied Physiology, Nutrition, and Metabolism · doi:10.1139/apnm-2016-0003 · Cited by 71
“Now consider this; when performing a metaanalysis, it is standard practice to exclude under-powered studies (small sample sizes), yet Turner et al. (2013) have shown that “underpowered studies made up the entirety of the evidence in most Cochrane reviews”.”
BackgroundRaghuraman, S., Stuttard, N., & Hunt, N. C. (2020) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.2486 · Cited by 70
Khan, S. A., Mandal, R. K., Elasbali, A. M., Dar, S. A., Jawed, A., Wahid, M., Mahto, H., Lohani, M., Mishra, B. N., Akhter, N., et al. (2018) · Bioscience Reports · doi:10.1042/bsr20180845 · Cited by 61
Zhang, Q., Zhang, R., Xiong, Y., Sui, Y., Tong, C., & Lin, F.-H. (2025) · Journal of Medical Internet Research · doi:10.2196/78238 · Cited by 56
“Although our number of effect sizes (n=37) is relatively large in terms of outcomes analyzed, it is small in terms of statistical analyses that combine empirical studies [100].”
Can systematic reviews with sparse data be trusted?
Afshari, A., Wetterslev, J., & Smith, A. F. (2016) · Anaesthesia · doi:10.1111/anae.13730 · Cited by 60
“detect or reject a 10% relative risk reduction [29].”
BackgroundAcupuncture for the Relief of Chronic Pain: A Synthesis of Systematic Reviews
Paley, C. A., & Johnson, M. I. (2019) · Medicina · doi:10.3390/medicina56010006 · Cited by 69
“[203] examined the distribution of statistical power within meta-analyses published as part of Cochrane reviews and argued that the results of meta-analyses that contain at least two adequately powered studies are not influenced to any significant degree when underpowered studies are omitted.”
BackgroundSubstance, use in relation to COVID-19: A scoping review
Kumar, N., Janmohamed, K., Nyhan, K., Martins, S. S., Cerdá, M., Hasin, D. S., Scott, J., Frimpong, A. S., Pates, R. M., Ghandour, L. A., et al. (2021) · Addictive Behaviors · doi:10.1016/j.addbeh.2021.107213 · Cited by 60
“Commentaries, correspondences, case reports, case series, editorials, and opinion pieces were excluded, as they often contain relatively limited evidence (Turner et al., 2013).”
Factors Associated with Dental Root Caries: A Systematic Review
Zhang, J., Sardana, D., Wong, M. C. M., Leung, K., & Lo, E. C. M. (2019) · JDR Clinical & Translational Research · doi:10.1177/2380084419849045 · Cited by 72
Barbateskovic, M., Krauss, S. R., Collet, M. O., Larsen, L. K., Jakobsen, J. C., Perner, A., & Wetterslev, J. (2019) · BMJ Open · doi:10.1136/bmjopen-2018-024562 · Cited by 54
Dakin, C. A., Beaulieu, K., Hopkins, M., Gibbons, C., Finlayson, G., & Stubbs, R. J. (2022) · Obesity Reviews · doi:10.1111/obr.13515 · Cited by 68
Yan, X., Li, K., Lei, Z., Luo, J., Wang, Q., & Wei, S. (2023) · International Journal of Infectious Diseases · doi:10.1016/j.ijid.2023.08.021 · Cited by 58
Goh, K. K., Wu, T., Chen, C.-H., & Lu, M. (2021) · Journal of Psychopharmacology · doi:10.1177/0269881120965937 · Cited by 58
“The pooled results were only considered when at least two adequately powered studies were available to avoid underpowered results (Turner et al., 2013).”
BackgroundWeller, C. M., Wilbrink, L. A., Houwing‐Duistermaat, J. J., Koelewijn, S. C., Vijfhuizen, L. S., Haan, J., Ferrari, M. D., Terwindt, G. M., van den Maagdenberg, A. M. J. M., & de Vries, B. (2014) · Cephalalgia · doi:10.1177/0333102414557839 · Cited by 57
“Notably, a Cochrane review demonstrated that underpowered studies tend to have a large influence on the outcomes of meta-analyses, if not at least two well-powered studies were included (33).”
Lamberink, H. J., Otte, W. M., Sinke, M. R. T., Lakens, D., Glasziou, P., Tijdink, J. K., & Vinkers, C. H. (2018) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2018.06.014 · Cited by 64
Frost, S. A., Hou, Y. C., Lombardo, L., Metcalfe, L., Lynch, J. M., Hunt, L., Alexandrou, E., Brennan, K. J., Sanchez, D., Åneman, A., et al. (2018) · BMC Infectious Diseases · doi:10.1186/s12879-018-3521-y · Cited by 59
Reporting of health equity considerations in cluster and individually randomized trials
Petkovic, J., Jull, J., Yoganathan, M., Dewidar, O., Baird, S., Grimshaw, J., Johansson, K. A., Kristjansson, E., McGowan, J., Moher, D., et al. (2020) · Trials · doi:10.1186/s13063-020-4223-5 · Cited by 58
Kamath, V., Sutherland, E. R., & Chaney, G.-A. (2020) · Journal of the International Neuropsychological Society · doi:10.1017/s1355617719001115 · Cited by 51
“10); Effect sizes in double brackets are presented for descriptive purposes due to the small number of studies included in the analysis (Turner et al., 2013).”
BackgroundSoftware support for environmental evidence synthesis
Westgate, M. J., Haddaway, N. R., Cheng, S. H., McIntosh, E. J., Marshall, C., & Lindenmayer, D. B. (2018) · Nature Ecology & Evolution · doi:10.1038/s41559-018-0502-x · Cited by 51
Searching for a Consensus Among Inflammatory Bowel Disease Studies: A Systematic Meta-Analysis
Abdel-Rahman, L. I. H., & Morgan, X. C. (2022) · Inflammatory Bowel Diseases · doi:10.1093/ibd/izac194 · Cited by 58
Bhargava, R., Kumar, P., & Arora, Y. (2015) · Eye & Contact Lens Science & Clinical Practice · doi:10.1097/icl.0000000000000179 · Cited by 59
“This could be explained by the fact that 6 of the 7 randomized controlled trials were underpowered (n1⁄4272, overall) as compared with the adequately powered trial (n1⁄4518), showing significant improvement in the symptom score in 3 months.(14,19,20) Ocular surface inflammation is known to decrease aqueous tear produc…”
ResultFischer, S., Hyder, S., & Walker, A. (2020) · Australian Journal of Management · doi:10.1177/0312896219899450 · Cited by 76
“Although the number of studies exploring the relationships in this article is small, meta-analyses of this nature are considered efficacious and worth exploring for the purpose of new theory development (Turner et al., 2013; Valentine et al., 2010).”
BackgroundCabanas‐Valdés, R., Serra-Llobet, P., Rodríguez-Rubio, P. R., López-de–Celis, C., Llauró-Fores, M., & Sanz, J. C. (2020) · Clinical Rehabilitation · doi:10.1177/0269215520932196 · Cited by 57
Central venous catheter (CVC) removal for patients of all ages with candidaemia
Janum, S., & Afshari, A. (2016) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011195.pub2 · Cited by 58
Flap reconstruction for sacrectomy defects: A systematic review and meta-analysis
Asaad, M., Rajesh, A., Wahood, W., Vyas, K. S., Houdek, M. T., Rose, P. S., & Moran, S. L. (2019) · Journal of Plastic Reconstructive & Aesthetic Surgery · doi:10.1016/j.bjps.2019.09.049 · Cited by 49
Evrenoglou, T., Metelli, S., & Chaimani, A. (2022) · Springer eBooks · doi:10.1007/978-3-319-52636-2_287 · Cited by 54
Tsertsvadze, A., Grove, A., Freeman, K., Court, R. A., Johnson, S. A., Connock, M. J., Clarke, A., & Sutcliffe, P. A. (2014) · PLoS ONE · doi:10.1371/journal.pone.0099804 · Cited by 56
Härkänen, M., Voutilainen, A., Turunen, E., & Vehviläinen-Julkunen, K. M. (2016) · Nurse Education Today · doi:10.1016/j.nedt.2016.03.017 · Cited by 66
Ning, G., Kan, S.-L., Chen, L., Shangguan, L., Feng, S., & Zhou, Y. (2016) · Scientific Reports · doi:10.1038/srep23726 · Cited by 50
Yang, G. S., Kim, H. J., Griffith, K. A., Zhu, S., Dorsey, S. G., & Renn, C. L. (2016) · Cancer Nursing · doi:10.1097/ncc.0000000000000409 · Cited by 53
Cooper, A. A., Kline, A. C., Baier, A. L., & Feeny, N. C. (2018) · Behavior Modification · doi:10.1177/0145445518792251 · Cited by 55
“As previously noted, psychotherapy studies are often small and under-powered, which may distort results of meta-analyses (e.g., Turner, Bird, & Higgins, 2013).”
ResultCabanas‐Valdés, R., Sanz, J. C., Urrútia, G., Serra-Llobet, P., Pérez‐Bellmunt, A., & Germán-Romero, A. (2019) · Topics in Stroke Rehabilitation · doi:10.1080/10749357.2019.1654242 · Cited by 54
“3, considered as small, medium, large and very large, respectively.(27) Heterogeneity across studies was tested using the I2 test, I2 score >50% indicated significant heterogeneity.”
MethodResultKlitgaard, T., Schjørring, O. L., Nielsen, F. M., Meyhoff, C. S., Perner, A., Wetterslev, J., Rasmussen, B. S., & Barbateskovic, M. (2023) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012631.pub3 · Cited by 63
Many meta-analyses of rare events in the Cochrane Database of Systematic Reviews were underpowered
Jia, P., Lin, L., Kwong, J. S., & Xu, C. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.11.017 · Cited by 48
“21) for an RRR of 30% for safety outcomes (adverse events) [22].”
BackgroundWhen may systematic reviews and meta-analyses be considered reliable?
Afshari, A., & Wetterslev, J. (2015) · European Journal of Anaesthesiology · doi:10.1097/eja.0000000000000186 · Cited by 49
Diagnostic accuracy of imaging devices in glaucoma: A meta-analysis
Fallon, M., Valero, O., Pazos, M., & Antón, A. (2017) · Survey of Ophthalmology · doi:10.1016/j.survophthal.2017.01.001 · Cited by 47
Guidelines for conducting and publishing systematic reviews in Psychology
Carvalho, L. D. F., Pianowski, G., & dos Santos, M. A. (2019) · Estudos de Psicologia (Campinas) · doi:10.1590/1982-0275201936e180144 · Cited by 56
Cabanas‐Valdés, R., Sanz, J. C., Serra-Llobet, P., Kait, J. A., González-Rueda, V., & Rodríguez-Rubio, P. R. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18094424 · Cited by 56
“Funnel plots were used to illustrate the risk of publication bias [25].”
MethodVitamin D supplementation for chronic liver diseases in adults
Bjelaković, M., Nikolova, D., Bjelaković, G., & Gluud, C. N. (2021) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011564.pub3 · Cited by 55
Oral paracetamol (acetaminophen) for cancer pain
Wiffen, P. J., Derry, S., Moore, R. A., McNicol, E. D., Bell, R. F., Carr, D. B., McIntyre, M., & Wee, B. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012637 · Cited by 45
A meta-analysis of the impact of rail stations on property values: Applying a transit planning lens
Rennert, L. (2022) · Transportation Research Part A Policy and Practice · doi:10.1016/j.tra.2022.06.013 · Cited by 46
Statin and Cancer Mortality and Survival: An Umbrella Systematic Review and Meta-Analysis
Jeong, G. H., Lee, K. H., Kim, J. Y., Eisenhut, M., Kronbichler, A., van der Vliet, H. J., Shin, J. I., & Gamerith, G. (2020) · Journal of Clinical Medicine · doi:10.3390/jcm9020326 · Cited by 47
“However, underpowered‐studies supported by the small number of studies have limitations to be validated [45].”
BackgroundViral and atypical respiratory co‐infections in COVID‐19: a systematic review and meta‐analysis
Davis, B., Rothrock, A. N., Swetland, S. B., Andris, H., Davis, P. J., & Rothrock, S. G. (2020) · Journal of the American College of Emergency Physicians Open · doi:10.1002/emp2.12128 · Cited by 50
Tran, A., Heuser, J., Ramsay, T., McIsaac, D. I., & Martel, G. (2020) · Canadian Journal of Anesthesia/Journal canadien d anesthésie · doi:10.1007/s12630-020-01857-4 · Cited by 50
“Nevertheless, given the sample sizes in most studies and lack of a priori power calculations in all included studies, any conclusions drawn would be at risk of type II error.(42) Our findings are consistent with the existing literature describing visual”
BackgroundResultDelagneau, G., Twilhaar, E. S., Testa, R., van Veen, S., & Anderson, P. J. (2023) · Child Development · doi:10.1111/cdev.13885 · Cited by 46
“Although we used an approach that is more robust in cases of small numbers of studies and performed sensitivity analyses, this may have increased heterogeneity in our analyses and resulted in an overestimation of some of our effect sizes (Turner et al., 2013).”
MethodWang, S., Li, M., Ren, J., Tao, J., Fang, M., & Kong, L. (2025) · BMC Public Health · doi:10.1186/s12889-025-24905-4 · Cited by 58
“Exclusion criteria included: 1) studies focused on specific subgroups not representative of the general population (e.g., adolescents with specific diseases or young athletes); 2) studies with fewer than 50 participants [20–22].”
Comparing Treatment Modalities For End-Stage Renal Disease: A Meta-Analysis
Queeley, G. L., & Campbell, E. S. (2014) · Value in Health · doi:10.1016/j.jval.2014.03.1690 · Cited by 43
Autograft versus allograft in anterior cruciate ligament reconstruction
Kan, S.-L., Yuan, Z.-F., Ning, G., Yang, B., Li, H., Sun, J.-C., & Feng, S. (2016) · Medicine · doi:10.1097/md.0000000000004936 · Cited by 49
Drug-eluting stents versus bare-metal stents for acute coronary syndrome
Feinberg, J. B., Nielsen, E. E., Greenhalgh, J., Hounsome, J. C., Sethi, N. J., Safi, S., Gluud, C. N., & Jakobsen, J. C. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012481.pub2 · Cited by 53
Gresham, G. K., Ehrhardt, S., Meinert, J. L., Appel, L. J., & Meinert, C. L. (2017) · Clinical Trials · doi:10.1177/1740774517727742 · Cited by 45
Moro, M. G., Souto, M. L. S., Franco, G. C. N., Holzhausen, M., & Pannuti, C. M. (2018) · Journal of Periodontal Research · doi:10.1111/jre.12525 · Cited by 48
Lurasidone in the Treatment of Bipolar Depression: Systematic Review of Systematic Reviews
Fornaro, M., Berardis, D. D., Perna, G., Solmi, M., Veronese, N., Orsolini, L., Buonaguro, E. F., Iasevoli, F., Köhler, C. A., Carvalho, A. F., et al. (2017) · BioMed Research International · doi:10.1155/2017/3084859 · Cited by 46
“This is compelling, especially when novel compounds of potential priority interest for the clinical practice like lurasidone, even in the treatment of BD, may need to await for additional RCTs to allow reliable meta-analytic pooling [37, 38].”
BackgroundMeta-analysis on risk of bleeding with apixaban in patients with renal impairment
Pathak, R., Pandit, A., Karmacharya, P., Aryal, M. R., Ghimire, S., Poudel, D. R., & Shamoun, F. E. (2015) · The American Journal of Cardiology · doi:10.1016/j.amjcard.2014.10.042 · Cited by 41
Cassese, S., Byrne, R. A., Jüni, P., Wykrzykowska, J. J., Puricel, S., Ndrepepa, G., Schunkert, H., Fusaro, M., Cook, S., Kimura, T., et al. (2018) · EuroIntervention · doi:10.4244/eij-d-17-00492 · Cited by 41
Simmonds‐Buckley, M., Kellett, S., & Waller, G. (2019) · Behavior Therapy · doi:10.1016/j.beth.2019.01.003 · Cited by 56
Beta-blockers for suspected or diagnosed acute myocardial infarction
Safi, S., Sethi, N. J., Nielsen, E. E., Feinberg, J. B., Gluud, C. N., & Jakobsen, J. C. (2019) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012484.pub2 · Cited by 49
The Wages of Mothers' Labor: A Meta‐Regression Analysis
Leonard, M. D. L., & Stanley, T. D. (2020) · Journal of Marriage and the Family · doi:10.1111/jomf.12693 · Cited by 48
Xu, C., Fan, S., Tian, Y., Liu, F., Furuya-Kanamori, L., Clark, J., Zhang, C., Li, S., Lin, L., Chu, H., et al. (2025) · BMJ · doi:10.1136/bmj-2024-082068 · Cited by 39
García‐Perdomo, H. A., Jiménez-Mejías, E., & López‐Ramos, H. (2015) · International braz j urol · doi:10.1590/s1677-5538.ibju.2014.0198 · Cited by 40
“These symptoms are associated with higher morbidity in patients and increase the cost for healthcare system (3-5).”
BackgroundTerkawi, A. S., Mavridis, D., Flood, P. D., Wetterslev, J., Terkawi, R. S., Abdulhak, A. A. B., Nunemaker, M. S., & Tiouririne, M. (2016) · Anesthesiology · doi:10.1097/aln.0000000000001039 · Cited by 39
Li, Y., Sun, W., Sun, X., Sun, J., Yang, D., Jia, B.-L., & Yuan, B. (2020) · Frontiers of Nursing · doi:10.2478/fon-2020-0001 · Cited by 47
Abbott, S., Smith, E., Tighe, B., & Lycett, D. (2020) · Journal of Human Nutrition and Dietetics · doi:10.1111/jhn.12853 · Cited by 47
Exercisers’ Affective and Enjoyment Responses: A Meta-Analytic and Meta-Regression Review
Tavares, V. D. D. O., Schuch, F. B., Tempest, G. D., Parfitt, G., Neto, L. D. O., Coelho, N. L. G., & Hackett, D. A. (2021) · Perceptual and Motor Skills · doi:10.1177/00315125211024212 · Cited by 49
Du, K., Qing, H., Zheng, M., Wang, X., & Zhang, L. (2020) · Annals of Allergy Asthma & Immunology · doi:10.1016/j.anai.2020.06.038 · Cited by 42
Scott‐Sheldon, L. A. J., Carey, K. B., Kaiser, T. S., Knight, J. M., & Carey, M. P. (2016) · Health Psychology · doi:10.1037/hea0000357 · Cited by 44
The Social Information Processing Model in Child Physical Abuse and Neglect: A Meta-analytic Review
Camilo, C., Garrido, M. V., & Calheiros, M. M. (2020) · Child Abuse & Neglect · doi:10.1016/j.chiabu.2020.104666 · Cited by 37
Shi, J., Liu, Y., Liu, Y., Li, Y., Qiu, S., Bai, Y., Gu, Y., Luo, J., Cui, H., Li, Y., et al. (2018) · Gene · doi:10.1016/j.gene.2018.06.097 · Cited by 42
Longitudinal Job Crafting Research: A Meta-Analysis
Silapurem, L., Slemp, G. R., & Jarden, A. (2024) · International Journal of Applied Positive Psychology · doi:10.1007/s41042-024-00159-0 · Cited by 45
“Although only two studies are needed to conduct a meta-analysis (Valentine et al., 2010), this may have increased the possibility that our results for such analyses are influenced by second-order sampling error (Schmidt & Oh, 2013), or by extreme or inflated effect sizes (Turner et al., 2013).”
Nissen, K. G. (2016) · Psycho-Oncology · doi:10.1002/pon.4057 · Cited by 44
“A study investigating the impact of underpowered studies on meta-analysis results has shown that when at least two adequately powered studies are available in a meta-analysis, underpowered studies can be left out, because they contribute with heterogeneity and little information [41].”
BackgroundWang, N., Qian, P. C., Kumar, S., Yan, T. D., & Phan, K. H. (2016) · International Journal of Cardiology · doi:10.1016/j.ijcard.2016.02.083 · Cited by 38
“TSAs are performed to explore the risk of random error as a result of sparse data and repetitive testing in order to increase the robustness of the meta-analyses and distinguish the current sample size from the required information size [14,15].”
BackgroundSvendsen, M. T., Andersen, F., Hansen, J., Johannessen, H., & Andersen, K. E. (2016) · Journal of Dermatological Treatment · doi:10.1080/09546634.2016.1178375 · Cited by 36
“Metanalysis was not planned, due to the limited number of studies and few study participants (40), heterogeneity of outcome measures, and heterogeneity in study design.”
BackgroundHagger, M. S. (2022) · International Review of Sport and Exercise Psychology · doi:10.1080/1750984x.2021.1966824 · Cited by 44
“Meta-analyses based on studies with low statistical power may also lead to biased estimates of the averaged effect across studies (Turner et al., 2013).”
BackgroundKamath, V., Chaney, G.-A. S., DeRight, J., & Onyike, C. U. (2019) · Psychological Medicine · doi:10.1017/s0033291718003604 · Cited by 35
Brain activation elicited by acute stress: An ALE meta-analysis
Qiu, Y., Fan, Z., Zhong, M., Yang, J., Wu, K., Hu, H., Zhang, R., Guo, Y. Q., Lee, T. M. C., & Huang, R. (2021) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2021.11.020 · Cited by 40
Caldeira, D., Canastro, M., Barra, M., Ferreira, A., Costa, J. N., Pinto, F. J., & Ferreira, J. J. (2015) · JAMA Ophthalmology · doi:10.1001/jamaophthalmol.2015.0985 · Cited by 39
French, C., Glassford, N. J., Gantner, D. C., Higgins, A. M., Cooper, D. J., Nichol, A., Skrifvars, M. B., Imberger, G. L., Presneill, J., Bailey, M., et al. (2016) · Annals of Surgery · doi:10.1097/sla.0000000000001746 · Cited by 36
Zhang, D., Wang, X., Yang, Z.-X., Gan, J.-X., Pan, J.-B., & Yin, L.-N. (2017) · International Journal of Surgery · doi:10.1016/j.ijsu.2017.06.001 · Cited by 42
Retracted
Metadata
- Authors
- Turner, R. M., Bird, S. M., & Higgins, J. P. T.
- Affiliations
- MRC Biostatistics Unit (GB)
- University of Bristol (GB)
- Corresponding author
- Turner, Rebecca M.
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2013-03-27
- Volume
- 8
- Issue
- 3
- Pages
- e59202
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC3609745
- OpenAlex
- W2015824469
- Cited by
- 672
- References
- 25
- Citations by year
- 2026: 46 · 2025: 51 · 2024: 51 · 2023: 59 · 2022: 70 · 2021: 70 · 2020: 59 · 2019: 49 · 2018: 56 · 2017: 60 · 2016: 45 · 2015: 31 · 2014: 20 · 2013: 4
- FWCI
- 36.78
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, other-oa)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- UCL Discovery (University College London) (submittedVersion)
- Bristol Research (University of Bristol) (submittedVersion)
- Funders
- Medical Research Council (U105260794)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Keywords
- underpowered studies, study size, statistical power, meta-analysis, binary outcomes, relative risk, standard errors, study heterogeneity, rapid review
- MeSH terms
- Humans, Review Literature as Topic, Meta-Analysis as Topic, Odds Ratio, Sample Size, Risk Reduction Behavior
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
