The Impact of Study Size on Meta-analyses: Examination of Underpowered Studies in Cochrane Reviews
Open accessThe DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Most meta-analyses include data from one or more small studies that, individually, do not have power to detect an intervention effect. The relative influence of adequately powered and underpowered studies in published meta-analyses has not previously been explored. We examine the distribution of power available in studies within meta-analyses published in Cochrane reviews, and investigate the impact of underpowered studies on meta-analysis results. METHODS AND FINDINGS: For 14,886 meta-analyses of binary outcomes from 1,991 Cochrane reviews, we calculated power per study within each meta-analysis. We defined adequate power as ≥50% power to detect a 30% relative risk reduction. In a subset of 1,107 meta-analyses including 5 or more studies with at least two adequately powered and at least one underpowered, results were compared with and without underpowered studies. In 10,492 (70%) of 14,886 meta-analyses, all included studies were underpowered; only 2,588 (17%) included at least two adequately powered studies. 34% of the meta-analyses themselves were adequately powered. The median of summary relative risks was 0.75 across all meta-analyses (inter-quartile range 0.55 to 0.89). In the subset examined, odds ratios in underpowered studies were 15% lower (95% CI 11% to 18%, P<0.0001) than in adequately powered studies, in meta-analyses of controlled pharmacological trials; and 12% lower (95% CI 7% to 17%, P<0.0001) in meta-analyses of controlled non-pharmacological trials. The standard error of the intervention effect increased by a median of 11% (inter-quartile range -1% to 35%) when underpowered studies were omitted; and between-study heterogeneity tended to decrease. CONCLUSIONS: When at least two adequately powered studies are available in meta-analyses reported by Cochrane reviews, underpowered studies often contribute little information, and could be left out if a rapid review of the evidence is required. However, underpowered studies made up the entirety of the evidence in most Cochrane reviews.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“we pooled results excluding studies with fewer than 50 participants in the experimental group, as a planned sensitivity analysis”
From Effectiveness of psychological interventions in prison to reduce recidivism: a systematic review and meta-analysis of randomised controlled trials (2021) · cites this paper as [66]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“there is a risk that untested approaches may introduce new forms of bias 18 .”
From Software support for environmental evidence synthesis (2018) · cites this paper as [18]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“On collating the evidence, some studies may appear underpowered for the outcome(s) of interest to us (Turner 2013), yet were adequately powered for the studies' primary measure.”
From Physical activity and exercise for chronic pain in adults: an overview of Cochrane Reviews (2017) · cites this paper as Turner 2013
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Although our number of effect sizes (n=37) is relatively large in terms of outcomes analyzed, it is small in terms of statistical analyses that combine empirical studies [100].”
From Generative AI Mental Health Chatbots as Therapeutic Tools: Systematic Review and Meta-Analysis of Their Role in Reducing Mental Health Issues (2025) · cites this paper as [100]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Meta-analysis in clinical trials
DerSimonian, R., & Laird, N. (1986) · Controlled Clinical Trials · doi:10.1016/0197-2456(86)90046-2 · Cited by 39,552 · cited 2×
“In each meta-analysis j, we fitted a random-effects model, using a method-of-moments estimate for the between study variance [17], and calculated the variance Vj of the combined intervention effect (on the log relative risk scale).”
MethodSterne, J. A. C., Sutton, A. J., Ioannidis, J. P. A., Terrin, N., Jones, D. R., Lau, J., Carpenter, J., Rucker, G., Harbord, R. M., Schmid, C. H., et al. (2011) · BMJ · doi:10.1136/bmj.d4002 · Cited by 7,709 · cited 1×
“[24], who published new guidelines.”
Self-citationKjaergard, L. L., Villumsen, J., & Gluud, C. (2001) · Annals of Internal Medicine · doi:10.7326/0003-4819-135-11-200112040-00010 · Cited by 2,283 · cited 1×
“Alternatively, small-study effects may arise from biases caused by methodological flaws arising more frequently in small studies [3], or may be due to true differences in the underlying effects between smaller and larger studies.”
BackgroundPublication and related bias in meta-analysis
Sterne, J. A. C., Gavaghan, D., & Egger, M. (2000) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(00)00242-0 · Cited by 2,282
Trial sequential analysis may establish when firm evidence is reached in cumulative meta-analysis
Wetterslev, J., Thorlund, K., Brok, J., & Gluud, C. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.03.013 · Cited by 1,881
Brok, J., Thorlund, K., Gluud, C., & Wetterslev, J. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.10.007 · Cited by 1,113 · cited 1×
“[20], who had, however, excluded both reviews with fewer than three trials and those in which all trials had a high risk of bias, in which metaanalytic size is likely to have been smaller.”
ResultTotal Plasma Homocysteine and Cardiovascular Risk Profile: The Hordaland Homocysteine Study
Nygård, O., Vollset, S. E., Refsum, H., Stensvold, I., Tverdal, A., Nordrehaug, J. E., Ueland, P. M., & Kvåle, G. (1995) · JAMA · doi:10.1001/jama.1995.03530190040032 · Cited by 1,042
The scandal of poor medical research.
Altman, D. G. (1994) · BMJ · doi:10.1136/bmj.308.6924.283 · Cited by 897 · cited 1×
“Researchers who choose to undertake a study that is capable of detecting only an unrealistically large effect may lack understanding of both scientific methods and ethics [10].”
BackgroundThe power of statistical tests in meta-analysis
Hedges, L. V., & Pigott, T. D. (2001) · Psychological Methods · doi:10.1037/1082-989x.6.3.203 · Cited by 716 · cited 1×
“where W is the cumulative standard normal distribution function and Ca=2~W {1 1{a=2 ð Þ[18].”
BackgroundMeta‐analysis of drug‐related deaths soon after release from prison
Merrall, E. L. C., Kariminia, A., Binswanger, I. A., Hobbs, M. S., Farrell, M., Marsden, J., Hutchinson, S. J., & Bird, S. M. (2010) · Addiction · doi:10.1111/j.1360-0443.2010.02990.x · Cited by 674 · cited 1×
“In teaching, Bird has long advocated that trials should not be started unless they could deliver at least 50% power in respect of a priori plausible, worthwhile effect sizes [6].”
BackgroundSelf-citation
Cited by
Items 201–300 of 672
Power estimations for non-primary outcomes in randomised clinical trials
Jakobsen, J. C., Ovesen, C., Winkel, P., Hilden, J., Gluud, C. N., & Wetterslev, J. (2019) · BMJ Open · doi:10.1136/bmjopen-2018-027092 · Cited by 37
Prevention of generalized tonic–clonic seizures in refractory focal epilepsy: A meta‐analysis
Hemery, C., Ryvlin, P., & Rheims, S. (2014) · Epilepsia · doi:10.1111/epi.12765 · Cited by 35
Shi, J., Liu, S., Guo, Y., Liu, S., Xu, J., Pan, L., Hu, Y., Liu, Y., & Cheng, Y. (2021) · BMC Cardiovascular Disorders · doi:10.1186/s12872-021-02192-2 · Cited by 34
Enhancing aquaculture disease resistance: Antimicrobial peptides and gene editing
Wang, J., & Cheng, Y. (2023) · Reviews in Aquaculture · doi:10.1111/raq.12845 · Cited by 36
Stamp, K. D., Prasun, M. A., Lee, C. S., Jaarsma, T., Piano, M. R., & Albert, N. M. (2018) · Heart & Lung · doi:10.1016/j.hrtlng.2018.01.003 · Cited by 40
Herbst, S. R., Pitchika, V., Herbst, C. S., Kosan, E., & Schwendicke, F. (2025) · International Endodontic Journal · doi:10.1111/iej.14256 · Cited by 39
“Limited or unclear power is a common phenomenon in medicine (and endodontics); the majority of Cochrane reviews has been found to include (and synthesize) under-powered studies (Turner et al., 2013).”
Neto, E. A. P., Lewthwaite, H., Boyle, T., Johnston, K., Bennett, H., & Williams, M. (2021) · PeerJ · doi:10.7717/peerj.11554 · Cited by 37
Beets, M. W., Weaver, R. G., Ioannidis, J. P. A., Pfledderer, C. D., Jones, A., von Klinggraeff, L., & Armstrong, B. (2023) · Systematic Reviews · doi:10.1186/s13643-023-02184-7 · Cited by 35
“Despite this, the vast majority of clinical research is conducted on “small” samples (~ 100 participants or fewer) [6] and over two-thirds of all metaanalyses in the Cochrane reviews are based solely on underpowered trials [7].”
BackgroundTargeting Next Generations to Change the Common Practice of Underpowered Research
Crutzen, R., & Peters, G.-J. (2017) · Frontiers in Psychology · doi:10.3389/fpsyg.2017.01184 · Cited by 39
“The biases due to underpowered studies are not limited to primary research, but can also distort meta-analytic evidence (Turner et al., 2013; Nuijten et al., 2015).”
BackgroundInfluentialExtending a Gamified Mobile App with a Public Display to Encourage Walking
Altmeyer, M., Lessel, P., Sander, T., & Krüger, A. (2018) · International Academic Mindtrek Conference (Mindtrek) · doi:10.1145/3275116.3275135 · Cited by 34
“We computed the post-hoc power of our study and found it to be 0.56 for the step count tests, which is still acceptable according to [48], but also shows that validating our findings with more participants is an important next step.”
ResultInfluentialAlhwoaimel, N. A., Turk, R., Warner, M. B., Verheyden, G., Thijs, L., Wee, S. K., & Hughes, A. (2018) · Neurorehabilitation · doi:10.3233/nre-182446 · Cited by 31
“The effect size was categorized as; 0.2, 0.5, 0.8, and 1.3, considered as small, medium, large and very large, respectively (Turner et al. 2013).”
InfluentialStundner, O., Poeran, J., Ladenhauf, H. N., Berger, M. M., Levy, S. B., Zubizarreta, N. J., Mazumdar, M., Bekeris, J., Liu, J., Galatz, L. M., et al. (2019) · Regional Anesthesia & Pain Medicine · doi:10.1136/rapm-2018-100145 · Cited by 31
Kaddoura, R., Omar, A. S., Ibrahim, M. I. M., Alkhulaifi, A. M., Lorusso, R., Elsherbini, H., Soliman, O. I. I., & Çalişkan, K. (2021) · Journal of Cardiothoracic and Vascular Anesthesia · doi:10.1053/j.jvca.2021.01.019 · Cited by 37
Cassidy, L., Hill, L., Fitzsimons, D., & McGaughey, J. (2021) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2020.103806 · Cited by 30
Mathematics Interventions for Adolescents with Mathematics Difficulties: A Meta–Analysis
Myers, J. A., Brownell, M. T., Griffin, C. C., Hughes, E. M., Witzel, B. S., Gage, N. A., Peyton, D. J., Acosta, K., & Wang, J. (2021) · Learning Disabilities Research and Practice · doi:10.1111/ldrp.12244 · Cited by 42
“Although our sample is relatively large compared to previous meta-analyses, it is small in terms of statistical analyses that combine empirical studies (Turner et al., 2013).”
ResultBérczi, B., Gerencsér, G., Farkas, N., Hegyi, P., Veres, G., Bajor, J., Czopf, L., Alizadeh, H., Rakonczay, Z., Vigh, É., et al. (2017) · Scientific Reports · doi:10.1038/s41598-017-14375-z · Cited by 33
Systematic reviews do not (yet) represent the ‘gold standard’ of evidence: A position paper
Moore, A., Fisher, E., & Eccleston, C. (2022) · European Journal of Pain · doi:10.1002/ejp.1905 · Cited by 38
Tillman, A., Muthalib, M., Hendy, A. M., Johnson, L. G., Rantalainen, T., Kidgell, D. J., Enticott, P. G., & Teo, W. (2015) · Frontiers in Aging Neuroscience · doi:10.3389/fnagi.2015.00040 · Cited by 31
“Lastly, it could be that the small number of studies involved in this meta-analysis and underpowered studies used were not enough to detect a significant difference between groups (Turner et al., 2013).”
BackgroundGarcia-Alamino, J. M., Bankhead, C. R., Heneghan, C. J., Pidduck, N., & Perera, R. (2017) · BMJ Open · doi:10.1136/bmjopen-2017-015888 · Cited by 35
“Brok et al(3) demonstrated, in a subset of Cochrane reviews, that many meta-analyses have false-positive results due to insufficient information, and Turner et al showed that most meta-analysis do not have sufficient power to identify even moderate effects.(7) 8 Sample size calculation and the OIS are influenced by se…”
BackgroundMoreau, D., Stonyer, J., McKay, N. S., & Waldie, K. E. (2018) · Brain Research · doi:10.1016/j.brainres.2018.01.014 · Cited by 34
Caldeira, D., Gonçalves, N., Pinto, F. J., Costa, J. N., & Ferreira, J. J. (2015) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.3791 · Cited by 31
Hildebrand, M., Wibbelink, C. J. M., & Verschuère, B. (2018) · International Journal of Law and Psychiatry · doi:10.1016/j.ijlp.2018.02.013 · Cited by 36
Genetic Polymorphisms and Diversity in Nonalcoholic Fatty Liver Disease (NAFLD): A Mini Review
Sulaiman, S. A., Dorairaj, V., & Adrus, M. N. H. (2022) · Biomedicines · doi:10.3390/biomedicines11010106 · Cited by 34
“Underpowered studies tend to show extreme effects compared to large studies; though, this effect varied across meta-analyses [207].”
ResultLadanie, A., Speich, B., Briel, M., Sclafani, F., Bucher, H. C., Agarwal, A., Ioannidis, J. P. A., Pereira, T., Kasenda, B., & Hemkens, L. G. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.05.033 · Cited by 32
Barker, T. G., O’Higgins, A., Fonagy, P., & Gardner, F. E. M. (2024) · Journal of Affective Disorders · doi:10.1016/j.jad.2023.12.080 · Cited by 32
Chaabane, S., Chaabna, K., Khawaja, S., Aboughanem, J., Mittal, D., Mamtani, R., & Cheema, S. (2024) · Scientific Reports · doi:10.1038/s41598-024-53818-2 · Cited by 40
Ghai, S., Ghai, I., & Narciss, S. (2024) · BMC Musculoskeletal Disorders · doi:10.1186/s12891-024-07571-2 · Cited by 35
“This variance in the number of studies might have influenced the observed difference in magnitude [155].”
BackgroundLi, C., Ngorsuraches, S., Chou, C., Chen, L., & Qian, J. (2021) · Critical Reviews in Oncology/Hematology · doi:10.1016/j.critrevonc.2021.103346 · Cited by 31
Retrospective median power, false positive meta‐analysis and large‐scale replication
Stanley, T. D., Doucouliagos, H., & Ioannidis, J. P. A. (2021) · Research Synthesis Methods · doi:10.1002/jrsm.1529 · Cited by 34
“It has been observed that 70% of meta-analyses in the Cochrane database do not contain any study with 50% or higher power.(89) To address this widespread problem of underpowered studies along with their potentially negative small-study effects, Turner, Bird and STANLEY ET AL.”
BackgroundCarayannopoulos, K. L., Pidutti, A., Upadhyaya, Y., Alshamsi, F. E., Basmaji, J., Granholm, A., Alhazzani, W. A., & Lewis, K. A. (2022) · Critical Care Medicine · doi:10.1097/ccm.0000000000005726 · Cited by 31
“If the cumulative z-curve does not cross any of the threshold boundaries, we conclude that the quantity of evidence is insufficient to firmly accept or refute the effect size of interest (14, 15).”
BackgroundSharafi, H., Bakouni, H., McAnulty, C., Drouin, S., Coronado‐Montoya, S., Bahremand, A., Bach, P., Ezard, N., Foll, B. L., Schütz, C. G., et al. (2023) · Addiction · doi:10.1111/add.16347 · Cited by 37
Kanu, J. S., Qiu, S., Cheng, Y., Li, R., Kou, C., Gu, Y., Bai, Y., Shi, J., Li, Y., Liu, Y., et al. (2018) · Lipids in Health and Disease · doi:10.1186/s12944-018-0767-8 · Cited by 30
“Trial sequential analysis (TSA) Traditional meta-analysis may result in type I and type II errors owing to dispersed data and repeated significance testing [55, 56].”
Lei, L. Y., Raj, S. R., & Sheldon, R. S. (2020) · Heart Rhythm · doi:10.1016/j.hrthm.2020.02.033 · Cited by 30
Statistical power in clinical trials of interventions for mood, anxiety, and psychotic disorders
de Vries, Y. A., Schoevers, R. A., Higgins, J. P. T., Munafo, M. R., & Bastiaansen, J. A. (2022) · Psychological Medicine · doi:10.1017/s0033291722001362 · Cited by 28
“These findings are fairly consistent with previous work by Turner et al. (2013), who found a median power of 0.14 for binary outcomes, and indicate that statistical power for the (usually) continuous primary outcome in psychiatric trials is actually somewhat better than suggested by previous work, although still inade…”
InfluentialSelf-citationIdentification of Soluble Mediators in IgG-Mediated Anaphylaxis via Fcγ Receptor: A Meta-Analysis
Kow, A. S. F., Chik, A., Soo, K.-M., Khoo, L. W., Abas, F., & Tham, C. L. (2019) · Frontiers in Immunology · doi:10.3389/fimmu.2019.00190 · Cited by 28
Assessing the Evidential Value of Mental Fatigue and Exercise Research
Holgado, D., Mesquida, C., & Román-Caballero, R. (2023) · Sports Medicine · doi:10.1007/s40279-023-01926-w · Cited by 33
“Meta-analyses may minimize some of the shortcomings of low-power studies, but they cannot provide a realistic picture of the literature as a whole from a set of low-powered studies [115].”
BackgroundChan, Y.-H., Chang, H., Lu, M., & Goh, K. K. (2023) · Psychiatry Research · doi:10.1016/j.psychres.2023.115621 · Cited by 33
“To avoid under-powered results, we included pooled results only when at least two adequately powered studies were available (Turner et al., 2013).”
MethodHow to perform and write a trial sequential analysis
Clephas, P. R. D., Kranke, P., & Heesen, M. A. (2022) · Anaesthesia · doi:10.1111/anae.15811 · Cited by 28
Wang, N., Qian, P. C., Yan, T. D., & Phan, K. H. (2016) · International Journal of Cardiology · doi:10.1016/j.ijcard.2016.01.004 · Cited by 28
“One validated methodology to overcome this limitation is trial sequential analyses, which are performed to explore the risk of random error as a result of sparse data and repetitive testing in order to increase the robustness of the metaanalyses and distinguish the current sample size from the required information siz…”
MethodHolst, L. B. (2016) · PubMed · Cited by 29
Barbateskovic, M., Larsen, L. K., Collet, M. O., Jakobsen, J. C., Perner, A., & Wetterslev, J. (2016) · Systematic Reviews · doi:10.1186/s13643-016-0391-5 · Cited by 31
“The required information size takes into account the event proportion in the control group, the assumption of a plausible relative risk (RR) reduction, and the heterogeneity variance [93] of the meta-analysis [92].”
BackgroundSingendonk, M. M. J., Brink, A. J., Steutel, N. F., van Etten‐Jamaludin, F. S., van Wijk, M. P., Benninga, M. A., & Tabbers, M. M. (2017) · PEDIATRICS · doi:10.1542/peds.2016-4166 · Cited by 28
Hunter, J., Arentz, S., Goldenberg, J. Z., Yang, G., Beardsley, J., Mertz, D., & Leeder, S. (2020) · Integrative Medicine Research · doi:10.1016/j.imr.2020.100457 · Cited by 28
Carayannopoulos, K. L., Alshamsi, F. E., Chaudhuri, D., Spatafora, L., Piticaru, J., Campbell, K., Alhazzani, W. A., & Lewis, K. A. (2024) · Critical Care Medicine · doi:10.1097/ccm.0000000000006251 · Cited by 31
Krogh, J., Speyer, H., Gluud, C. N., & Nordentoft, M. (2015) · Systematic Reviews · doi:10.1186/s13643-015-0030-6 · Cited by 33
“cient power [30], and if there is no significant effect of the intervention, it is also interesting to know whether this”
BackgroundTay, J. H. T., Jiang, Y., Hong, J., He, H. G., & Wang, W. (2020) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2020.103779 · Cited by 28
Srisurapanont, M., Suradom, C., Suttajit, S., Kongsaengdao, S., & Maneeton, B. (2024) · General Hospital Psychiatry · doi:10.1016/j.genhosppsych.2024.02.008 · Cited by 27
Fisher, E., Eccleston, C., Degenhardt, L. J., Finn, D. P., Finnerup, N. B., Gilron, I., Haroutounian, S., Krane, E. J., Rice, A. S. C., Rowbotham, M. C., et al. (2019) · PAIN Reports · doi:10.1097/pr9.0000000000000741 · Cited by 30
Maneeton, B., Maneeton, N., Woottiluk, P., & Likhitsathian, S. (2020) · Current Neuropharmacology · doi:10.2174/1570159x18666200221113134 · Cited by 28
“Firstly, although a meta-analysis can be conducted with at least two studies and a systematic review can be performed without any included study [47], the limited numbers of included RCTs with a small sample size in this review may decrease the power to estimate the effect of intervention [48].”
Williams, A. C. D. C., Fisher, E., Hearn, L., & Eccleston, C. (2021) · Pain · doi:10.1097/j.pain.0000000000002273 · Cited by 25
“This is true even when many (small) studies are pooled with the rigor of Cochrane methodology [18,27].”
BackgroundChoong, S. Y. X., Tan, X. Y. J., Cheng, L. J., & Lau, Y. (2021) · Behavioral Sleep Medicine · doi:10.1080/15402002.2021.1931221 · Cited by 26
“Hence, the results generated may present extreme treatment effects due to insufficient power (Turner et al., 2013).”
van Zwet, E. W., & Goodman, S. N. (2022) · Statistics in Medicine · doi:10.1002/sim.9406 · Cited by 31
Vukadinović, D., Abdin, A., Emrich, I. E., Schulze, P. C., von Haehling, S., & Böhm, M. (2023) · Clinical Research in Cardiology · doi:10.1007/s00392-023-02207-2 · Cited by 29
“We calculated the power of our meta-analysis for observed risk ratio reduction (RRR) for composite of recurrent HF hospitalizations and CVD and odds/hazard ratio reduction (ORR/HRR) for CVD in the main and both sensitivity analyses [18].”
MethodEarly maladaptive schemas and addictive behaviours: A systematic review and meta‐analysis
Sakulsriprasert, C., Thawornwutichat, R., Phukao, D., & Guadamuz, T. E. (2023) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.2882 · Cited by 32
Systematic Literature Review Langchain Proposed
Asyrofi, R., Dewi, M. R., Luthfi, M. I., & Wibowo, P. A. (2023) · International Electronics Symposium (IES) · doi:10.1109/ies59143.2023.10242497 · Cited by 28
“[3] Manual systematic reviews have similar reproducibility and transparency issues.”
BackgroundThe potential of international large-scale assessments for meta-analyses in education
Scherer, R., Siddiq, F., & Nilsen, T. (2024) · Large-scale Assessments in Education · doi:10.1186/s40536-024-00191-1 · Cited by 27
“Large-sample studies may well increase precision and reduce sampling error, yet may also influence the effect size estimate and its variance components substantially due to large weights in the meta-analytic data set (Turner et al., 2013).”
BackgroundLidocaine dose-response effect on postoperative cognitive deficit: meta-analysis and meta-regression
Habibi, M. R., Habibi, V., Habibi, A., & Soleimani, A. (2018) · Expert Review of Clinical Pharmacology · doi:10.1080/17512433.2018.1425614 · Cited by 25
“argued that when at least two adequately powered studies are available in meta-analysis, underpowered studies often contribute little information, so the result is reliable [42].”
BackgroundMarsman, M. S., Wetterslev, J., Jahrome, A. K., Gluud, C. N., Moll, F. L., Keus, F., & Koning, G. G. (2021) · Systematic Reviews · doi:10.1186/s13643-021-01692-8 · Cited by 28
Sohmaran, C., & Shorey, S. (2019) · Journal of Advanced Nursing · doi:10.1111/jan.14166 · Cited by 24
Oc, B., & Chintakananda, K. (2024) · The Leadership Quarterly · doi:10.1016/j.leaqua.2024.101842 · Cited by 31
Goh, K. K., Chen, C.-H., Wu, T., Chiu, Y.-H., & Lu, M. (2022) · Frontiers in Pharmacology · doi:10.3389/fphar.2022.944437 · Cited by 26
“Pooled results were only considered when at least two adequately powered studies were available to avoid underpowered results (Turner et al., 2013).”
BackgroundResultYoldi‐Negrete, M., Gill, L.-N., Olivares, S., Lauzière, A., Désilets, M., & Tourjman, V. (2022) · Journal of Affective Disorders · doi:10.1016/j.jad.2022.08.005 · Cited by 27
Magalhães, E., & Camilo, C. (2023) · The European Journal of Psychology Applied to Legal Context · doi:10.5093/ejpalc2023a10 · Cited by 32
Baradari, A. G., Habibi, M. R., Habibi, V., & Nouraei, S. M. (2016) · Expert Review of Clinical Pharmacology · doi:10.1080/17512433.2017.1266252 · Cited by 24
“argued that when at least two adequately powered studies are available in meta-analyses, underpowered studies often contribute little information [34]; our results can be accounted for the neuroprotective effect of lidocaine.”
BackgroundConway, A., Conway, Z. J., Soalheira, K., & Sutherland, J. (2017) · European Journal of Anaesthesiology · doi:10.1097/eja.0000000000000691 · Cited by 24
Kinesio taping for rotator cuff disease
Gianola, S., Iannicelli, V., Fascio, E., Andreano, A., LI, L. C., Valsecchi, M. G., Moja, L., & Castellini, G. (2021) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012720.pub2 · Cited by 34
Yet another problem with systematic reviews: a living review update
Uttley, L., Weng, Y., & Falzon, L. (2024) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2024.111608 · Cited by 27
Mondello, S., Amrein, K., Czeiter, E., Citerio, G., Diaz-Arrastia, R. R., Gao, G., Lagares, A., Manley, G. T., Menon, D. K., Newcombe, V., et al. (2025) · Journal of Neurotrauma · doi:10.1089/neu.2024.0620 · Cited by 25
“35,36 To avoid double counting, in the case of publications reporting results from the same cohort or studies with overlapping samples, we selected the most complete article and those presenting complementary results (for instance, if they reported measurement of different biomarkers or outcomes).”
Rygård, S. L., Jonsson, A. B., Madsen, M. B., Perner, A., Holst, L. B., Johansson, P. I., & Wetterslev, J. (2018) · Intensive Care Medicine · doi:10.1007/s00134-018-5069-0 · Cited by 25
Alho, I., Mirka, J., Laura, J., Muotka, J. S., & Lappalainen, R. I. (2020) · Journal of Contextual Behavioral Science · doi:10.1016/j.jcbs.2020.12.003 · Cited by 27
Apfelbach, C. S., & Guzmán, M. (2021) · Journal of Voice · doi:10.1016/j.jvoice.2021.09.031 · Cited by 23
Atwal, A., Snowsill, T., Cabrera-Dandy, M., Krum, T., Newton, C. L., Evans, D. G., Crosbie, E. J., & Ryan, N. A. J. (2022) · International Journal of Cancer · doi:10.1002/ijc.34165 · Cited by 26
Myers, J. A., Hughes, E. M., Witzel, B. S., Anderson, R. D., & Owens, J. (2022) · Journal of Research on Educational Effectiveness · doi:10.1080/19345747.2022.2080131 · Cited by 34
“First, one obvious limitation of the present synthesis is the sample size was too small in the context of meta-analytic studies (Turner et al., 2013).”
BackgroundFraley, R. C., Chong, J. Y., Baacke, K. A., Greco, A. J., Guan, H., & Vazire, S. (2022) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459221120217 · Cited by 28
“Fraley and Vazire (2014) proposed one way of doing so, called the N-pact factor (NF).”
BackgroundResultBakouni, H., Sharafi, H., Bahremand, A., Drouin, S., Ziegler, D., Bach, P., Foll, B. L., Schütz, C. G., Tardelli, V. S., Ezard, N., et al. (2023) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2023.111018 · Cited by 26
Kim, M. S., Kim, J. Y., Song, Y. S., Hong, S., Won, H., Kim, W. J., Kwon, Y., Ha, J., Fiedorowicz, J. G., Solmi, M., et al. (2023) · Obesity Reviews · doi:10.1111/obr.13670 · Cited by 27
“31,32 We calculated I 2 and Cochran's Q test p -values to estimate within-subgroup heterogeneity and between-subgroup heterogeneity.”
MethodNeural bases of reading fluency: A systematic review and meta-analysis
Lee, M. M., & Stoodley, C. J. (2024) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2024.108947 · Cited by 28
Wang, G., Zhang, L., Lou, S., Chen, Y., Cao, Y., Wang, R., Zhang, L., & Tang, P. (2016) · Medicine · doi:10.1097/md.0000000000002927 · Cited by 24
“Type I errors may appear in meta-analyses owing to an increased risk of random error, although a few data were analyzed.(18) TSA was used to this meta-analysis in order to assess the risk of type I errors (program version 0.”
MethodMorotti, A., Sollaku, I., Catalani, S., Franceschini, F., Cavazzana, I., Fredi, M., Sala, E., & Palma, G. D. (2020) · Rheumatology · doi:10.1093/rheumatology/keaa444 · Cited by 25
Veroniki, A. A., Straus, S. E., Ashoor, H. M., Hamid, J. S., Hemmelgarn, B. R., Holroyd-Leduc, J. M., Majumdar, S. R., McAuley, G., & Tricco, A. C. (2016) · BMJ Open · doi:10.1136/bmjopen-2015-010251 · Cited by 23
“the studies usually included in a meta-analysis are underpowered themselves.(28) The aim of this study is to examine the comparative effectiveness and safety of cognitive enhancers versus”
MethodMijnster, T., Boersma, G. J., Meijer, E., & Lancel, M. (2022) · Journal of Personalized Medicine · doi:10.3390/jpm12071070 · Cited by 28
Ding, X., Ma, Y., Ma, Y., Chen, C., Zhang, X., Duan, Q., & Ma, Y. (2023) · International Journal of Surgery · doi:10.1097/js9.0000000000000630 · Cited by 26
“Additionally, we excluded studies that included patients with morbid obesity and studies with a sample size of less than 20 in any study arm due to the potential for sampling error caused by small sample sizes, which may affect our results and pose a high risk of bias [23,24] .”
BackgroundLai, P.-C., Lai, C., Lai, E. C., & Huang, Y. (2023) · Critical Care Medicine · doi:10.1097/ccm.0000000000006161 · Cited by 29
“Generally, large sample sizes enhance study power by providing clear effect size estimates and minimizing chance results, a principle that is equally applicable to (network) meta-analysis (45).”
BackgroundLucien, A., Francis, H., Wu, W., Woldhuis, T., & Gandy, M. (2024) · Multiple Sclerosis and Related Disorders · doi:10.1016/j.msard.2024.105858 · Cited by 28
Saywell, N. L., Taylor, N., Rodgers, E., Skinner, L. C., & Boocock, M. (2016) · Clinical Rehabilitation · doi:10.1177/0269215516631384 · Cited by 24
Koushede, V., Brixval, C. S., Thygesen, L. C., Axelsen, S. F., Winkel, P., Lindschou, J., Gluud, C. N., & Due, P. (2017) · PLoS ONE · doi:10.1371/journal.pone.0176819 · Cited by 33
Riberholt, C. G., Wagner, V., Lindschou, J., Gluud, C. N., Mehlsen, J., & Møller, K. (2020) · PLoS ONE · doi:10.1371/journal.pone.0237136 · Cited by 25
“required information size for a dichotomous outcome takes into account the event proportion in the control group, the assumption of a plausible risk ratio (RR) reduction, and the heterogeneity of the meta-analysis [49,50].”
BackgroundMikačić, I., & Bosnar, D. (2016) · Drug Safety · doi:10.1007/s40264-016-0408-y · Cited by 22
Ferreira, V. L., Leonart, L. P., Tonin, F. S., Borba, H. H. L., & Pontarolo, R. (2018) · Clinical Drug Investigation · doi:10.1007/s40261-018-0624-6 · Cited by 21
Accumulation Bias in meta-analysis: the need to consider time in error control
ter Schure, J., & Grünwald, P. (2019) · F1000Research · doi:10.12688/f1000research.19375.1 · Cited by 40
“Nevertheless, it is often argued that many meta-analyses are underpowered (Turner et al., 2013; Davey et al., 2011) and that this should be taken into account in evaluating significance in meta-analyses.”
Belloni, S., Caruso, R., Cattani, D., Mandelli, G., Donizzetti, D., Mazzoleni, B., & Tedeschi, M. (2022) · Worldviews on Evidence-Based Nursing · doi:10.1111/wvn.12574 · Cited by 26
Ong, Q.-E. O., Ong, J. W., Ang, M. Q., Vehviläinen-Julkunen, K. M., & He, H.-G. (2023) · Patient Education and Counseling · doi:10.1016/j.pec.2023.107678 · Cited by 30
Clari, M., Albanesi, B., Comoretto, R. I., Conti, A., Renzi, E., Luciani, M., Ausili, D., Massimi, A., & Dimonte, V. (2024) · Eurosurveillance · doi:10.2807/1560-7917.es.2024.29.9.2300276 · Cited by 24
Caldeira, D., Alves, M., David, C., Costa, J. N., Ferreira, J. J., & Pinto, F. J. (2019) · Primary care diabetes · doi:10.1016/j.pcd.2019.11.004 · Cited by 24
Robinson, R., Walker, K. F., White, V. A., Bugg, G. J., Snell, K. I. E., & Jones, N. W. (2020) · European Journal of Obstetrics & Gynecology and Reproductive Biology · doi:10.1016/j.ejogrb.2020.01.019 · Cited by 22
Vukadinović, D., Schirmer, S. H., Vukadinović, A. N., Ukena, C., Scheller, B., Mahfoud, F., & Böhm, M. (2018) · Clinical Research in Cardiology · doi:10.1007/s00392-018-1334-z · Cited by 22
A Critical Analysis of Intestinal Enteric Neuron Loss and Constipation in Parkinson’s Disease
O’Day, C., Finkelstein, D. I., Diwakarla, S., & McQuade, R. M. (2022) · Journal of Parkinson s Disease · doi:10.3233/jpd-223262 · Cited by 24
Corporate social responsibility and performance in the workplace: a meta-analysis
Kim, J. Y., & Keane, A. (2023) · International Journal for Educational and Vocational Guidance · doi:10.1007/s10775-023-09581-3 · Cited by 29
“It has been posited that, when compared with older studies, recent primary studies may be more adequately powered in terms of extracting results for secondary meta-analysis (Turner et al., 2013).”
Heesen, M. A., Weibel, S., Klimek, M., Rossaint, R., Arends, L. R., & Kranke, P. (2017) · Anaesthesia · doi:10.1111/anae.14033 · Cited by 21
Thrombolysis for acute upper extremity deep vein thrombosis
Feinberg, J. B., Nielsen, E. E., & Jakobsen, J. C. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012175.pub2 · Cited by 26
Metadata
- Authors
- Turner, R. M., Bird, S. M., & Higgins, J. P. T.
- Affiliations
- MRC Biostatistics Unit (GB)
- University of Bristol (GB)
- Corresponding author
- Turner, Rebecca M.
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2013-03-27
- Volume
- 8
- Issue
- 3
- Pages
- e59202
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC3609745
- OpenAlex
- W2015824469
- Cited by
- 672
- References
- 25
- Citations by year
- 2026: 46 · 2025: 51 · 2024: 51 · 2023: 59 · 2022: 70 · 2021: 70 · 2020: 59 · 2019: 49 · 2018: 56 · 2017: 60 · 2016: 45 · 2015: 31 · 2014: 20 · 2013: 4
- FWCI
- 36.78
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, other-oa)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- UCL Discovery (University College London) (submittedVersion)
- Bristol Research (University of Bristol) (submittedVersion)
- Funders
- Medical Research Council (U105260794)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Keywords
- underpowered studies, study size, statistical power, meta-analysis, binary outcomes, relative risk, standard errors, study heterogeneity, rapid review
- MeSH terms
- Humans, Review Literature as Topic, Meta-Analysis as Topic, Odds Ratio, Sample Size, Risk Reduction Behavior
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
