Power analysis software for educational researchers
Abstract
Given the importance of statistical power analysis in quantitative research and the repeated emphasis on it by American Educational Research Association/American Psychological Association journals, the authors examined the reporting practice of power analysis by the quantitative studies published in 12 education/psychology journals between 2005 and 2010. It was surprising to uncover that less than 2% of the studies conducted prospective power analysis. Another 3.5% computed observed power, a practice not endorsed by the literature on power analysis. In this article, the authors clarify these 2 types of power analysis and discuss functionalities of 8 programs/packages (G*Power 3.1.3, PASS 11, SAS/STAT 9.3, Stata 12, SPSS 19, SPSS/Sample Power 3.0.1, Optimal Design Software 2.01, and MLPowSim 1.0 BETA) to encourage proper and planned power analysis. On the basis of their review, the authors recommend 2 programs (SPSS/Sample Power and G*Power) for general purpose univariate/multivariate analyses, and 1 (Optimal Design Software) for hierarchical/multilevel modeling and meta-analysis. Recommendations are also made for reporting power analysis results and exploring additional software. The article concludes with an examination of the role of statistical power in research and viable alternatives to hypothesis testing.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Statistical programmes to help perform a priori sample size calculations exist for a variety of experimental designs and statistical analyses, both freeware and commercial software.
“Statistical programmes to help perform a priori sample size calculations exist for a variety of experimental designs and statistical analyses, both freeware (web-based applets and functions in R) and commercial software [38–40].”
From Reporting animal research: Explanation and elaboration for the ARRIVE guidelines 2.0 (2020) · cites this paper as [39]
Cites
Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007) · Behavior Research Methods · doi:10.3758/bf03193146 · Cited by 65,971 · see citing works
Statistical methods in psychology journals: Guidelines and explanations.
Wilkinson, L., & Task Force on Statistical Inference (1999) · American Psychologist · doi:10.1037/0003-066x.54.8.594 · Cited by 2,791 · see citing works
The Abuse of Power: The Pervasive Fallacy of Power Calculations for Data Analysis
Hoenig, J. M., & Heisey, D. M. (2001) · The American Statistician · doi:10.1198/000313001300339897 · Cited by 2,046 · see citing works
The statistical power of abnormal-social psychological research: A review.
Cohen, J. (1962) · The Journal of Abnormal and Social Psychology · doi:10.1037/h0045186 · Cited by 1,824 · see citing works
Do studies of statistical power have an effect on the power of studies?
Sedlmeier, P., & Gigerenzer, G. (1989) · Psychological Bulletin · doi:10.1037/0033-2909.105.2.309 · Cited by 748 · see citing works
Standard Errors and Sample Sizes for Two-Level Research
Snijders, T. A. B., & Bosker, R. J. (1993) · Journal of Educational Statistics · doi:10.3102/10769986018003237 · Cited by 487 · cited 1× · see citing works
“The free software PinT (stands for Power in Two-level designs) considers budget constraints while estimating sample sizes for two-level HLM models (Bosker, Snijders, & Guldemond, 1996; Snijders & Bosker, 1993).”
MethodA survey of the statistical power of research in behavioral ecology and animal behavior
Jennions, M. D. (2003) · Behavioral Ecology · doi:10.1093/beheco/14.3.438 · Cited by 278 · see citing works
A Note on Misconceptions Concerning Prospective and Retrospective Power
Zumbo, B. D., & Hubley, A. M. (1998) · Journal of the Royal Statistical Society: Series D (The Statistician) · doi:10.1111/1467-9884.00139 · Cited by 121 · see citing works
Finch, S., Cumming, G., & Thomason, N. (2001) · Educational and Psychological Measurement · doi:10.1177/0013164401612001 · Cited by 52 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Reporting animal research: Explanation and elaboration for the ARRIVE guidelines 2.0
Percie du Sert, N., Ahluwalia, A., Alam, S., Avey, M. T., Baker, M., Browne, W. J., Clark, A., Cuthill, I. C., Dirnagl, U., Emerson, M., et al. (2020) · PLOS Biology · doi:10.1371/journal.pbio.3000411 · Cited by 2,677 · see citing works
Does ChatGPT enhance student learning? A systematic review and meta-analysis of experimental studies
Deng, R., Jiang, M., Yu, X., Lu, Y., & Liu, S. (2025) · Computers & Education · doi:10.1016/j.compedu.2024.105224 · Cited by 531 · see citing works
Single-case experimental research designs.
Kazdin, A. E. (2015) · Methodological issues and strategies in clinical research (4th ed.). · doi:10.1037/14805-029 · Cited by 39 · see citing works
- Authors
- Peng, C.-Y. J., Long, H., & Abaci, S.
- Affiliations
- Indiana University Bloomington (US)
- Indiana University–Bloomington
- Venue
- The Journal of Experimental Education
- Publisher
- Informa UK Limited
- Published
- 2012-01
- Volume
- 80
- Issue
- 2
- Pages
- 113-136
- ISSN
- 0022-0973 (ISSN-L)
- 1940-0683
- OpenAlex
- W2025455895
- Cited by
- 59
- References
- 9
- Citations by year
- 2026: 3 · 2025: 3 · 2024: 6 · 2023: 3 · 2022: 2 · 2021: 25 · 2020: 3 · 2019: 2 · 2017: 1 · 2016: 4 · 2015: 7
- FWCI
- 10.84
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- The Journal of Experimental Education (publishedVersion)
- IUScholarWorks (Indiana University) (submittedVersion)
- Topics
- Social Sciences › Education › Educational Research and Methods
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Keywords
- statistical power analysis, G*Power, PASS, multilevel modeling, meta-analysis, reporting practices, educational research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
