How Many Studies Do You Need?: A Primer on Statistical Power for Meta-Analysis
The DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index.
Abstract
In this article, the authors outline methods for using fixed and random effects power analysis in the context of meta-analysis. Like statistical power analysis for primary studies, power analysis for meta-analysis can be done either prospectively or retrospectively and requires assumptions about parameters that are unknown. The authors provide some suggestions for thinking about these parameters, in particular for the random effects variance component. The authors also show how the typically uninformative retrospective power analysis can be made more informative. The authors then discuss the value of confidence intervals, show how they could be used in addition to or instead of retrospective power analysis, and also demonstrate that confidence intervals can convey information more effectively in some situations than power analyses alone. Finally, the authors take up the question “How many studies do you need to do a meta-analysis?” and show that, given the need for a conclusion, the answer is “two studies,” because all other synthesis techniques are less transparent and/or are less likely to be valid. For systematic reviewers who choose not to conduct a quantitative synthesis, the authors provide suggestions for both highlighting the current limitations in the research base and for displaying the characteristics and results of studies that were found to meet inclusion criteria.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“the small number of effect sizes included in the moderator analyses can be seen as a limitation as such analyses may be considered under-powered (Valentine et al., 2010).”
From Taking stock: A meta-analysis of the effects of foreign language enjoyment (2022) · cites this paper as Valentine et al., 2010
In the citing paper
“Thus, the meta-analysis was well powered to detect moderate-to-large effects but underpowered for small effects, a common limitation in meta-analyses with relatively few studies (Valentine et al., 2010).”
From Gamification on Mathematics Engagement and Motivation in Secondary School and Higher Education: A Systematic Review and Meta-Analysis (2026) · cites this paper as Valentine et al., 2010
In the citing paper
“An a priori power analysis with a calculator (Quintana & Tiebel, 2019) was conducted based on equations from Valentine et al. (2010) using conservative values from Durlak et al. (2011)”
From The state of evidence for social and emotional learning: A contemporary meta-analysis of universal school-based SEL interventions (2023) · cites this paper as Valentine et al., 2010
In the citing paper
“Fourth, our random-effects meta-analyses were performed with two or more studies, following guidance (e.g.Cochrane Consumers and Communication Review Group, 2016;Valentine et al., 2010)”
From A Systematic Review and Meta-Analysis Exploring Effects of Third-Wave Psychological Therapies on Hearing-Related Distress, Depression, Anxiety, and Quality of Life in People With Audiological Problems. (2022) · cites this paper as Valentine et al., 2010
In the citing paper
“For anxiety all comparisons had a high level of imprecision, and for treatment dropout data, only the comparison between AI-self-guided eCBT and psychoeducation had a low level of imprecision ( Figure 4 ) (54, 57, 58).”
From Examining the role of AI technology in online mental healthcare: opportunities, challenges, and implications, a mixed-methods review (2024) · cites this paper as [57]
6 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Statistical Power Analysis for the Behavioral Sciences
Muller, K. (1989) · Technometrics · doi:10.1080/00401706.1989.10488618 · Cited by 99,167
Statistical power analysis for the behavioral sciences
(1990) · Computers, Environment and Urban Systems · doi:10.1016/0198-9715(90)90050-4 · Cited by 65,269
Measuring inconsistency in meta-analyses
Higgins, J. P. T., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003) · BMJ · doi:10.1136/bmj.327.7414.557 · Cited by 63,139 · cited 2×
“Based on an analysis of I2 values from more than 500 meta-analyses in medicine, Higgins et al. (2003) suggest that an I2 value of 25% represents a small degree of heterogeneity, while 50% represents a moderate degree, and 75% represents a large degree of heterogeneity.”
MethodStatistical Power Analysis for the Behavioral Sciences (2nd ed.).
Lachenbruch, P. A., & Cohen, J. (1989) · Journal of the American Statistical Association · doi:10.2307/2290095 · Cited by 37,395 · cited 2×
“Assume that, based on Cohen (1988), the reviewers have decided that ‘‘sufficient’’ power means that they have an 80% chance of correctly rejecting a false null hypothesis and further assume that all other values are estimated to be the same as they were in the previous examples (i.e., typical…”
BackgroundQuantifying heterogeneity in a meta‐analysis
Higgins, J. P. T., & Thompson, S. G. (2002) · Statistics in Medicine · doi:10.1002/sim.1186 · Cited by 36,763 · cited 1×
“When past experience in the area is not sufficient to guide the decision, we suggest that prospective meta-analysts use the work of Higgins and colleagues (Higgins & Thompson, 2002; Higgins, Thompson, Deeks, & Altman, 2003) to guide their selection of values of t2.”
MethodScoping studies: towards a methodological framework
Arksey, H., & O'Malley, L. (2005) · International Journal of Social Research Methodology · doi:10.1080/1364557032000119616 · Cited by 36,742 · cited 1×
“Scoping reviews are increasingly recommended as a means of determining the feasibility and relevance of a full-scale systematic review (Arksey & O’Malley, 2005).”
BackgroundStatistical Methods for Meta-Analysis
Wasserman, S., Hedges, L. V., & Olkin, I. (1988) · Journal of Educational Statistics · doi:10.2307/1164953 · Cited by 10,326
Statistical Methods for Psychology.
Gudgeon, A. C., & Howell, D. C. (1994) · The Statistician · doi:10.2307/2348956 · Cited by 8,278
Saeverot, H. (2022) · Indirect Education · doi:10.4324/9781003193463-8 · Cited by 3,059
Fixed-and random-effects models in meta-analysis.
Hedges, L. V., & Vevea, J. L. (1998) · Psychological Methods · doi:10.1037/1082-989x.3.4.486 · Cited by 2,977 · cited 1×
“Fixed effects analyses allow for inferences only to studies that are highly similar to the ones included in the meta-analysis, while random effects analyses allow for inferences to a wider range of studies (Hedges & Vevea, 1998).”
Method
Research works printed but not linked: 3
- Kish L., 1965, Survey sampling
- Levin J. R., 2000, Educational Researcher, 29, 34
- Robinson D. H., 1997, Educational Researcher, 26, 21
Cited by
A Global Perspective on Child Sexual Abuse: Meta-Analysis of Prevalence Around the World
Stoltenborgh, M., van IJzendoorn, M. H., Euser, E. M., & Bakermans-Kranenburg, M. J. (2011) · Child Maltreatment · doi:10.1177/1077559511403920 · Cited by 2,010
Lakens, D. (2022) · Collabra Psychology · doi:10.1525/collabra.33267 · Cited by 1,962
On the costs and benefits of emotional labor: A meta-analysis of three decades of research.
Hülsheger, U. R., & Schewe, A. F. (2011) · Journal of Occupational Health Psychology · doi:10.1037/a0022876 · Cited by 1,269
Trial Sequential Analysis in systematic reviews with meta-analysis
Wetterslev, J., Jakobsen, J. C., & Gluud, C. (2017) · BMC Medical Research Methodology · doi:10.1186/s12874-017-0315-7 · Cited by 1,189
van Laer, T., de Ruyter, K., Visconti, L. M., & Wetzels, M. (2014) · Journal of Consumer Research · doi:10.1086/673383 · Cited by 1,129
“This number is insufficient to provide meaningful insight into the extent to which a factor may have an influence (Valentine, Pigott, and Rothstein 2010); therefore, our model does not include medium.”
Metadata
- Authors
- Valentine, J. C., Pigott, T. D., & Rothstein, H. R.
- Affiliations
- University of Louisville (US)
- Loyola University-Chicago
- Baruch College (US)
- Venue
- Journal of Educational and Behavioral Statistics
- Published
- 2010-04
- Volume
- 35
- Issue
- 2
- Pages
- 215-247
- ISSN
- 1076-9986 (ISSN-L)
- 1935-1054
- OpenAlex
- W2171853361
- Cited by
- 1,294
- References
- 36
- Citations by year
- 2026: 93 · 2025: 139 · 2024: 113 · 2023: 118 · 2022: 142 · 2021: 133 · 2020: 121 · 2019: 113 · 2018: 100 · 2017: 63 · 2016: 57 · 2015: 22 · 2014: 23 · 2013: 24 · 2012: 15
- FWCI
- 13.65
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
- Locations
- Journal of Educational and Behavioral Statistics (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Keywords
- meta-analysis, statistical power analysis, fixed effects model, random effects models, confidence intervals, systematic reviews, qualitative evidence synthesis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: How Many Studies Do You Need?: A Primer on Statistical Power for Meta-Analysis. https://app.cochat.ai/papers/10.3102/1076998609346961
- The paper
- Valentine, J. C., Pigott, T. D., & Rothstein, H. R. (2010). How Many Studies Do You Need?: A Primer on Statistical Power for Meta-Analysis. Journal of Educational and Behavioral Statistics. https://doi.org/10.3102/1076998609346961
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.