Sample Size in Psychological Research over the Past 30 Years
The DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
Abstract
The American Psychological Association (APA) Task Force on Statistical Inference was formed in 1996 in response to a growing body of research demonstrating methodological issues that threatened the credibility of psychological research, and made recommendations to address them. One issue was the small, even dramatically inadequate, size of samples used in studies published by leading journals. The present study assessed the progress made since the Task Force's final report in 1999. Sample sizes reported in four leading APA journals in 1955, 1977, 1995, and 2006 were compared using nonparametric statistics, while data from the last two waves were fit to a hierarchical generalized linear growth model for more in-depth analysis. Overall, results indicate that the recommendations for increasing sample sizes have not been integrated in core psychological research, although results slightly vary by field. This and other implications are discussed in the context of current methodological critique and practice.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Third, our sample is 50 times larger than the average sample of studies published in high-impact psychology journals (30, 31).”
From Cognitive and noncognitive predictors of success (2019) · cites this paper as [31]
Citations
Cites
Statistical Power Analysis for the Behavioral Sciences
Muller, K. (1989) · Technometrics · doi:10.1080/00401706.1989.10488618 · Cited by 99,167
Statistical power analysis for the behavioral sciences
(1990) · Computers, Environment and Urban Systems · doi:10.1016/0198-9715(90)90050-4 · Cited by 65,269
Statistical Power Analysis for the Behavioral Sciences (2nd ed.).
Lachenbruch, P. A., & Cohen, J. (1989) · Journal of the American Statistical Association · doi:10.2307/2290095 · Cited by 37,395
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702
Answering the Call for a Standard Reliability Measure for Coding Data
Hayes, A. F., & Krippendorff, K. (2007) · Communication Methods and Measures · doi:10.1080/19312450709336664 · Cited by 4,225 · cited 1×
“Agreement was measured with Krippendorff’s alpha, α K , using the KALPHA SPSS (now PASW) macro (Hayes & Krippendorff, 2007), treating sample size as a ratio level measurement and performing 1,000 bootstrapping samples.”
Statistical methods in psychology journals: Guidelines and explanations.
Wilkinson, L., & Task Force on Statistical Inference (1999) · American Psychologist · doi:10.1037/0003-066x.54.8.594 · Cited by 2,791
The statistical power of abnormal-social psychological research: A review.
Cohen, J. (1962) · The Journal of Abnormal and Social Psychology · doi:10.1037/h0045186 · Cited by 1,824 · cited 5×
“(1981) parallel the surveys of observed power reported in the literature (Cohen, 1962; Sedlmeier & Gigerenzer, 1989; Rossi, 1990; Maddock & Ros-si, 2001; Maxwell, 2004), and even overlap in specific journals (i.e., Journal of Abnormal Psychology ).”
InfluentialNonparametric statistics for non-statisticians: a step-by-step approach
(2010) · Choice Reviews Online · doi:10.5860/choice.47-3859 · Cited by 1,021
Nonparametric Statistics for Non‐Statisticians: A Step‐by‐Step Approach
Corder, G. W., & Foreman, D. I. (2009) · Wiley eBooks · doi:10.1002/9781118165881 · Cited by 954
Research works printed but not linked: 4
- American Psychological Association, 2001, Publication manual
- Cochrane R., 1974, Bulletin of the British Psychological Society, 27, 117
- 10.1037/0278-6133.20.1.76 · doi:10.1037/0278-6133.20.1.76 · Paper page
- Raudenbush S. W., 2004, HLM 6: Hierarchical linear and nonlinear modeling
Cited by
Effect size guidelines for individual differences researchers
Gignac, G. E., & Szodorai, E. T. (2016) · Personality and Individual Differences · doi:10.1016/j.paid.2016.06.069 · Cited by 2,860
The association between adolescent well-being and digital technology use
Orben, A., & Przybylski, A. K. (2019) · Nature Human Behaviour · doi:10.1038/s41562-018-0506-1 · Cited by 1,500
de Winter, J. C. F., Gosling, S. D., & Potter, J. (2016) · Psychological Methods · doi:10.1037/met0000079 · Cited by 1,201
“Although sample sizes in psychological research vary widely, we used N = 200 because this is in line with typical sample sizes used in applied and personality psychology (Fraley & Vazire, 2014; Marszalek et al., 2011).”
BackgroundMethodThe rules of the game called psychological science
Bakker, M., van Dijk, A., & Wicherts, J. M. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612459060 · Cited by 800
Replicability, Robustness, and Reproducibility in Psychological Science
Nosek, B. A., Hardwicke, T. E., Moshontz, H., Allard, A., Corker, K. S., Dreber, A., Fidler, F., Hilgard, J., Kline Struhl, M., Nuijten, M. B., et al. (2022) · Annual Review of Psychology · doi:10.1146/annurev-psych-020821-114157 · Cited by 770
Metadata
- Authors
- Marszalek, J. M., Barber, C., Kohlhart, J., & Cooper, B. H.
- Affiliations
- University of Missouri–Kansas City (US)
- Emporia State University (US)
- Corresponding author
- Marszalek, Jacob M.
- Venue
- Perceptual and Motor Skills
- Publisher
- SAGE Publications
- Published
- 2011-04
- Volume
- 112
- Issue
- 2
- Pages
- 331-348
- ISSN
- 0031-5125 (ISSN-L)
- 1558-688X
- OpenAlex
- W2126053474
- Cited by
- 167
- References
- 29
- Citations by year
- 2026: 6 · 2025: 9 · 2024: 10 · 2023: 11 · 2022: 9 · 2021: 10 · 2020: 17 · 2019: 21 · 2018: 11 · 2017: 15 · 2016: 19 · 2015: 12 · 2014: 8 · 2013: 6 · 2012: 3
- FWCI
- 7.03
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
- Locations
- Perceptual and Motor Skills (publishedVersion)
- PubMed (publishedVersion)
- MOspace Institutional Repository (University of Missouri) (submittedVersion)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › General Psychology › Academic and Historical Perspectives in Psychology
- Keywords
- sample size, statistical power, methodological critique, research credibility, APA journals, methodological recommendations
- MeSH terms
- Data Interpretation, Statistical, Forecasting, Humans, Periodicals as Topic, Psychology, Publishing, Research, Reproducibility of Results, Bias, Linear Models, Sample Size, Statistics, Nonparametric, Editorial Policies
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Sample Size in Psychological Research over the Past 30 Years. https://app.cochat.ai/papers/10.2466/03.11.pms.112.2.331-348
- The paper
- Marszalek, J. M., Barber, C., Kohlhart, J., & Cooper, B. H. (2011). Sample Size in Psychological Research over the Past 30 Years. Perceptual and Motor Skills. https://doi.org/10.2466/03.11.pms.112.2.331-348
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.