Power analysis for random-effects meta-analysis
Open accessAbstract
One of the reasons for the popularity of meta-analysis is the notion that these analyses will possess more power to detect effects than individual studies. This is inevitably the case under a fixed-effect model. However, the inclusion of the between-study variance in the random-effects model, and the need to estimate this parameter, can have unfortunate implications for this power. We develop methods for assessing the power of random-effects meta-analyses, and the average power of the individual studies that contribute to meta-analyses, so that these powers can be compared. In addition to deriving new analytical results and methods, we apply our methods to 1991 meta-analyses taken from the Cochrane Database of Systematic Reviews to retrospectively calculate their powers. We find that, in practice, 5 or more studies are needed to reasonably consistently achieve powers from random-effects meta-analyses that are greater than the studies that contribute to them. Not only is statistical inference under the random-effects model challenging when there are very few studies but also less worthwhile in such cases. The assumption that meta-analysis will result in an increase in power is challenged by our findings.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The number of research literature for all modalities was greater than 5, meeting the requirements of random-effect analysis.
“number of research literature for all modalities was greater than 5, which meets the requirements of random-effect analysis138.”
From AI-assisted multi-modal information for the screening of depression: a systematic review and meta-analysis (2025) · cites this paper as [138]
The empirical investigation (page not supplied) finds that random-effects meta-analyses with fewer than five studies are quite likely to have less power than their constituent studies, and that power generally improves as the number of studies increases. It does not establish more than five studies as a formal requirement for random-effects analysis or show that this count alone makes an analysis appropriate. (LLM, conf=0.87)
Experimental — accuracy not yet measuredMeta-analyses using fixed-effects models have increased power to detect effects compared to individual studies, but this is not necessarily true for meta-analyses using random-effects models due to allowing for between-study variance.
“Meta-analyses using fixed-effects models have an increased power to detect effects compared to individual studies, but this is not necessarily true for meta-analyses using random-effects models due to allowing for between-study variance (Jackson and Turner, 2017).”
From Guided internet-based cognitive-behavioral therapy for patients with chronic pain: A meta-analytic review (2022) · cites this paper as Jackson and Turner, 2017
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Meta-analysis in clinical trials
DerSimonian, R., & Laird, N. (1986) · Controlled Clinical Trials · doi:10.1016/0197-2456(86)90046-2 · Cited by 39,552 · see citing works
Quantifying heterogeneity in a meta‐analysis
Higgins, J. P. T., & Thompson, S. G. (2002) · Statistics in Medicine · doi:10.1002/sim.1186 · Cited by 36,763 · see citing works
Cochrane Handbook for Systematic Reviews of Interventions: Cochrane Book Series
(2008) · doi:10.1002/9780470712184 · Cited by 25,048 · see citing works
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009) · doi:10.1002/9780470743386 · Cited by 14,225 · see citing works
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2021) · doi:10.1002/9781119558378 · Cited by 9,799 · see citing works
Interpretation of random effects meta-analyses
Riley, R. D., Higgins, J. P. T., & Deeks, J. J. (2011) · BMJ · doi:10.1136/bmj.d549 · Cited by 2,884 · see citing works
A Re-Evaluation of Random-Effects Meta-Analysis
Higgins, J. P. T., Thompson, S. G., & Spiegelhalter, D. J. (2009) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/j.1467-985x.2008.00552.x · Cited by 2,569 · see citing works
The Abuse of Power: The Pervasive Fallacy of Power Calculations for Data Analysis
Hoenig, J. M., & Heisey, D. M. (2001) · The American Statistician · doi:10.1198/000313001300339897 · Cited by 2,046 · see citing works
Trial sequential analysis may establish when firm evidence is reached in cumulative meta-analysis
Wetterslev, J., Thorlund, K., Brok, J., & Gluud, C. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.03.013 · Cited by 1,881 · see citing works
Methods to estimate the between‐study variance and its uncertainty in meta‐analysis
Veroniki, A. A., Jackson, D., Viechtbauer, W., Bender, R., Bowden, J., Knapp, G., Kuss, O., Higgins, J. P., Langan, D., & Salanti, G. (2016) · Research Synthesis Methods · doi:10.1002/jrsm.1164 · Cited by 1,685 · see citing works
Self-citationHow Many Studies Do You Need?: A Primer on Statistical Power for Meta-Analysis
Valentine, J. C., Pigott, T. D., & Rothstein, H. R. (2010) · Journal of Educational and Behavioral Statistics · doi:10.3102/1076998609346961 · Cited by 1,294 · see citing works
The Interpretation of Random-Effects Meta-Analysis in Decision Models
Ades, A. E., Lu, G., & Higgins, J. P. T. (2005) · Medical Decision Making · doi:10.1177/0272989x05282643 · Cited by 861 · see citing works
A refined method for the meta-analysis of controlled clinical trials with binary outcome
Hartung, J., & Knapp, G. (2001) · Statistics in Medicine · doi:10.1002/sim.1009 · Cited by 775 · see citing works
Stijnen, T., Hamza, T. H., & Özdemir, P. (2010) · Statistics in Medicine · doi:10.1002/sim.4040 · Cited by 757 · see citing works
The power of statistical tests in meta-analysis
Hedges, L. V., & Pigott, T. D. (2001) · Psychological Methods · doi:10.1037/1082-989x.6.3.203 · Cited by 716 · cited 4× · see citing works
“Others have previously discussed power calculations under the random-effects model.13,14 Hedges and Pigott (2001)15 also discuss how the unknown between-study variance complicates power analysis in the random-effects setting.”
BackgroundMethodInfluentialThe impact of study size on meta-analyses: examination of underpowered studies in Cochrane reviews
Turner, R. M., Bird, S. M., & Higgins, J. P. T. (2013) · PLoS ONE · doi:10.1371/journal.pone.0059202 · Cited by 672 · see citing works
Self-citationUsing and Understanding Medical Statistics
Matthews, D. E., Farewell, V. T., & Koss, N. (1987) · Plastic & Reconstructive Surgery · doi:10.1097/00006534-198706000-00033 · Cited by 528 · see citing works
How meta-analysis increases statistical power.
Cohn, L. D., & Becker, B. J. (2003) · Psychological Methods · doi:10.1037/1082-989x.8.3.243 · Cited by 500 · see citing works
Davey, J., Turner, R. M., Clarke, M. J., & Higgins, J. P. (2011) · BMC Medical Research Methodology · doi:10.1186/1471-2288-11-160 · Cited by 404 · see citing works
Self-citationThorlund, K., Imberger, G. L., Walsh, M. W., Chu, R., Gluud, C. N., Wetterslev, J., Guyatt, G. H., Devereaux, P. J., & Thabane, L. (2011) · PLoS ONE · doi:10.1371/journal.pone.0025491 · Cited by 320 · see citing works
Misunderstandings about Q and ‘Cochran's Q test' in meta‐analysis
Hoaglin, D. C. (2015) · Statistics in Medicine · doi:10.1002/sim.6632 · Cited by 313 · see citing works
Sequential methods for random‐effects meta‐analysis
Higgins, J. P. T., Whitehead, A., & Simmonds, M. C. (2010) · Statistics in Medicine · doi:10.1002/sim.4088 · Cited by 258 · see citing works
The exact distribution of Cochran's heterogeneity statistic in one‐way random effects meta‐analysis
Biggerstaff, B. J., & Jackson, D. (2008) · Statistics in Medicine · doi:10.1002/sim.3428 · Cited by 130 · see citing works
Self-citationA general framework for the use of logistic regression models in meta-analysis
Simmonds, M. C., & Higgins, J. P. T. (2014) · Statistical Methods in Medical Research · doi:10.1177/0962280214534409 · Cited by 107 · see citing works
Jackson, D. (2013) · Research Synthesis Methods · doi:10.1002/jrsm.1081 · Cited by 83 · see citing works
Self-citationConfidence intervals for a random‐effects meta‐analysis based on Bartlett‐type corrections
Noma, H. (2011) · Statistics in Medicine · doi:10.1002/sim.4350 · Cited by 82 · see citing works
The use of meta-analytic statistical significance testing
Polanin, J. R., & Pigott, T. D. (2015) · Research Synthesis Methods · doi:10.1002/jrsm.1124 · Cited by 78 · see citing works
Planning future studies based on the conditional power of a meta‐analysis
Roloff, V., Higgins, J. P. T., & Sutton, A. J. (2012) · Statistics in Medicine · doi:10.1002/sim.5524 · Cited by 68 · see citing works
Low‐event‐rate meta‐analyses of clinical trials: implementing good practices
Shuster, J. J., & Walker, M. A. (2016) · Statistics in Medicine · doi:10.1002/sim.6844 · Cited by 45 · see citing works
Planning future studies based on the precision of network meta‐analysis results
Nikolakopoulou, A., Mavridis, D., & Salanti, G. (2015) · Statistics in Medicine · doi:10.1002/sim.6608 · Cited by 44 · see citing works
Nikolakopoulou, A., Mavridis, D., & Salanti, G. (2014) · Biometrical Journal · doi:10.1002/bimj.201300216 · Cited by 42 · see citing works
A re‐evaluation of the ‘quantile approximation method’ for random effects meta‐analysis
Jackson, D., & Bowden, J. (2008) · Statistics in Medicine · doi:10.1002/sim.3487 · Cited by 36 · see citing works
Self-citationJACKKNIFE BIAS REDUCTION FOR POLYCHOTOMOUS LOGISTIC REGRESSION
Bull, S. B., Greenwood, C. M. T., & Hauck, W. W. (1997) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19970315)16:5 · Cited by 22 · see citing works
Using and Understanding Medical Statistics
Matthews, D. E., & Farewell, V. T. (2015) · KARGER eBooks · doi:10.1159/978-3-318-05459-0 · Cited by 21 · see citing works
Sidik, K., & Jonkman, J. N. (2003) · Statistics in Medicine · doi:10.1002/sim.1729 · Cited by 7 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Fact-checking: a meta-analysis of what works and for whom
Walter, N., Cohen, J., Holbert, R. L., & Morag, Y. (2020) · Political Communication · doi:10.1080/10584609.2019.1668894 · Cited by 690 · see citing works
Hillmayr, D., Ziernwald, L., Reinhold, F., Hofer, S. I., & Reiss, K. M. (2020) · Computers & Education · doi:10.1016/j.compedu.2020.103897 · Cited by 574 · see citing works
“The small number of studies in most subgroups and the use of a random-effects model resulted in reduced statistical power, which may partly explain non-significant effects and between-levels variance in the moderator analyses (Jackson & Turner, 2017).”
BackgroundResultWatson, J. A., Ryan, C. G., Cooper, L., Ellington, D., Whittle, R., Lavender, M., Dixon, J., Atkinson, G., Cooper, K., & Martin, D. J. (2019) · The Journal of Pain · doi:10.1016/j.jpain.2019.02.011 · Cited by 428 · see citing works
A predictive index for health status using species-level gut microbiome profiling
Gupta, V. K., Kim, M., Bakshi, U., Cunningham, K. Y., Davis, J. M., Lazaridis, K. N., Nelson, H., Chia, N., & Sung, J. (2020) · Nature Communications · doi:10.1038/s41467-020-18476-8 · Cited by 338 · see citing works
Rogers, A. H., & Farris, S. G. (2022) · European Journal of Pain · doi:10.1002/ejp.1994 · Cited by 280 · see citing works
“…that at least two studies must be present to conduct a meta- analysis (Valentine et al., 2010), but given the random- effects nature of the analysis as well as the significance observed within and between- study heterogeneity, we used the threshold of five studies (Jackson & Turner, 2017).”
MethodAntecedents and Effects of Parasocial Relationships: A Meta-Analysis
Tukachinsky, R., Walter, N., & Saucier, C. J. (2020) · Journal of Communication · doi:10.1093/joc/jqaa034 · Cited by 212 · see citing works
Loneliness among people with substance use problems: A narrative systematic review
Ingram, I., Kelly, P. J., Deane, F. P., Baker, A. L., Goh, M. C. W., Raftery, D. K., & Dingle, G. A. (2020) · Drug and Alcohol Review · doi:10.1111/dar.13064 · Cited by 206 · see citing works
Pilkington, P. D., Bishop, A., & Younan, R. (2021) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.2533 · Cited by 204 · see citing works
A meta-analysis of cytokine concentrations in eating disorders
Dalton, B., Bartholdy, S., Robinson, L., Solmi, M., Ibrahim, M. A. A., Breen, G., Schmidt, U., & Himmerich, H. (2018) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2018.06.002 · Cited by 197 · see citing works
Giusti, E. M., Lacerenza, M., Manzoni, G. M., & Castelnuovo, G. (2021) · Pain · doi:10.1097/j.pain.0000000000001999 · Cited by 188 · see citing works
Godard, R., & Holtzman, S. (2023) · Journal of Computer-Mediated Communication · doi:10.1093/jcmc/zmad055 · Cited by 171 · see citing works
A meta-analysis on verbal working memory in children and adolescents with ADHD
Ramos, A. A., Hamdan, A. C., & Machado, L. (2020) · The Clinical Neuropsychologist · doi:10.1080/13854046.2019.1604998 · Cited by 125 · see citing works
Davies, T. B., Tran, D. L., Hogan, C. M., Haff, G. G., & Latella, C. (2021) · Sports Medicine · doi:10.1007/s40279-020-01408-3 · Cited by 82 · see citing works
Calculating statistical power for meta-analysis using metapower
Griffin, J. W. (2021) · The Quantitative Methods for Psychology · doi:10.20982/tqmp.17.1.p024 · Cited by 84 · see citing works
Validity of data extraction in evidence synthesis practice of adverse events: reproducibility study
Xu, C., Yu, T., Furuya-Kanamori, L., Lin, L., Zorzela, L., Zhou, X., Dai, H., Loke, Y., & Vohra, S. (2022) · BMJ · doi:10.1136/bmj-2021-069155 · Cited by 82 · see citing works
Barakat, S., Maguire, S., Smith, K. E., Mason, T. B., Crosby, R. D., & Touyz, S. (2019) · International Journal of Eating Disorders · doi:10.1002/eat.23131 · Cited by 77 · see citing works
Ashtary-Larky, D., Bagheri, R., Abbasnezhad, A., Tinsley, G. M., Alipour, M., & Wong, A. (2020) · British Journal of Nutrition · doi:10.1017/s000711452000224x · Cited by 72 · see citing works
Human performance in detecting deepfakes: A systematic review and meta-analysis of 56 papers
Diel, A., Lalgi, T., Schröter, I. C., MacDorman, K. F., Teufel, M., & Bäuerle, A. (2024) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2024.100538 · Cited by 64 · see citing works
Kohn, T. P., Kohn, J. R., & Ramasamy, R. (2018) · Journal of Urology · doi:10.1016/j.juro.2017.11.045 · Cited by 56 · see citing works
Xu, C., Zhou, X., Zorzela, L., Ju, K., Furuya-Kanamori, L., Lin, L., Lu, C., Musa, O. A. H., & Vohra, S. (2021) · BMC Medicine · doi:10.1186/s12916-021-02008-2 · Cited by 43 · see citing works
Barnicot, K., Michael, C., Trione, E., Lang, S., Saunders, T., Sharp, M., & Crawford, M. J. (2020) · Clinical Psychology Review · doi:10.1016/j.cpr.2020.101929 · Cited by 45 · see citing works
Meissle, M., Naranjo, S. E., & Romeis, J. (2022) · Environmental Evidence · doi:10.1186/s13750-022-00272-0 · Cited by 41 · see citing works
“As a minimum requirement, an individual meta-analysis was only conducted when at least 5 records were available for a given taxon and comparison type [30] and when those records derived from 3 different articles to ensure a certain level of data independence.”
Terpstra, J. A., van der Vaart, R., van Beugen, S., van Eersel, R. A., Gkika, I., Erdős, D., Schmidt, J., Radstake, C., Kloppenburg, M., van Middendorp, H., et al. (2022) · Internet Interventions · doi:10.1016/j.invent.2022.100587 · Cited by 38 · see citing works
Fawson, S., Moon, Z., Novogrudsky, K., Moxham, F., Forster, K., Tribe, I., Moss-Morris, R., Johnson, C., & Hughes, L. D. (2024) · Health Psychology Review · doi:10.1080/17437199.2023.2261518 · Cited by 40 · see citing works
Changes in Disordered Eating Behaviors over 10 or More Years: A Meta-Analysis
Romano, K. A., Heron, K. E., Amerson, R., Howard, L. M., MacIntyre, R. I., & Mason, T. B. (2020) · International Journal of Eating Disorders · doi:10.1002/eat.23288 · Cited by 36 · see citing works
Unpacking stigma: meta-analyses of correlates and moderators of personal stigma in psychosis
Eliasson, E. T., McNamee, L., Swanson, L., Lawrie, S. M., & Schwannauer, M. (2021) · Clinical Psychology Review · doi:10.1016/j.cpr.2021.102077 · Cited by 33 · see citing works
Doma, K., Singh, U., Boullosa, D., & Connor, J. D. (2021) · Applied Physiology Nutrition and Metabolism · doi:10.1139/apnm-2021-0110 · Cited by 34 · see citing works
Turner, D. T., Riedel, E., Kobeissi, L. H., Karyotaki, E., Garcia-Moreno, C., Say, L., & Cuijpers, P. (2020) · Journal of Global Health · doi:10.7189/jogh.10.010409 · Cited by 29 · see citing works
Ruiz‐Íñiguez, R., Santed Germán, M. Á., Burgos‐Julián, F. A., Díaz‐Silveira, C., & Carralero Montero, A. (2020) · Early Intervention in Psychiatry · doi:10.1111/eip.12849 · Cited by 29 · see citing works
Wang, L., Wang, C., Li, C., Murai, T., Bai, Y., Song, Z., Zhang, S., Zhang, Q., Huang, Y., Bi, X., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01933-3 · Cited by 33 · see citing works
Yu, C., Cheng, M., An, X., Chueh, T., Wu, J., Wang, K., & Hung, T. (2025) · Scandinavian Journal of Medicine & Science in Sports · doi:10.1111/sms.70055 · Cited by 26 · see citing works
- Authors
- Jackson, D., & Turner, R.
- Affiliations
- MRC Biostatistics Unit (GB)
- Corresponding author
- Jackson, Dan
- Venue
- Research Synthesis Methods
- Publisher
- Wiley
- Published
- 2017-04-04
- Volume
- 8
- Issue
- 3
- Pages
- 290-302
- ISSN
- 1759-2879 (ISSN-L)
- 1759-2887
- PMC
- PMC5590730
- OpenAlex
- W2746334579
- Cited by
- 619
- References
- 35
- Citations by year
- 2026: 66 · 2025: 80 · 2024: 74 · 2023: 89 · 2022: 92 · 2021: 96 · 2020: 57 · 2019: 43 · 2018: 16 · 2017: 5
- FWCI
- 29.83
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Research Synthesis Methods (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- UCL Discovery (University College London) (submittedVersion)
- Apollo (University of Cambridge) (submittedVersion, cc-by)
- Apollo (cc-by)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Keywords
- power analysis, random-effects meta-analysis, study heterogeneity, fixed effects model, sample size, Cochrane Database of Systematic Reviews, statistical power
- MeSH terms
- Humans, Meta-Analysis as Topic, Models, Statistical, Bias, Databases, Factual
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
