The DOI matches its Crossref record; 97 of the 97 references it cites were found in a registry or an index.
Psychology, Science, and Knowledge Construction: Broadening Perspectives from the Replication Crisis
Abstract
Psychology advances knowledge by testing statistical hypotheses using empirical observations and data. The expectation is that most statistically significant findings can be replicated in new data and in new laboratories, but in practice many findings have replicated less often than expected, leading to claims of a replication crisis. We review recent methodological literature on questionable research practices, meta-analysis, and power analysis to explain the apparently high rates of failure to replicate. Psychologists can improve research practices to advance knowledge in ways that improve replicability. We recommend that researchers adopt open science conventions of preregi-stration and full disclosure and that replication efforts be based on multiple studies rather than on a single replication attempt. We call for more sophisticated power analyses, careful consideration of the various influences on effect sizes, and more complete disclosure of nonsignificant as well as statistically significant findings.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482
Statistical Power Analysis for the Behavioral Sciences
Muller, K. (1989) · Technometrics · doi:10.1080/00401706.1989.10488618 · Cited by 99,167
Bootstrap methods: another look at the jackknife
Efron, B. (1979) · The Annals of Statistics · doi:10.1214/aos/1176344552 · Cited by 17,602
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009) · doi:10.1002/9780470743386 · Cited by 14,225
David, F. N., & Tukey, J. W. (1977) · Biometrics · doi:10.2307/2529486 · Cited by 12,708
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
Estimating the reproducibility of psychological science
Open Science Collaboration (2015) · Science · doi:10.1126/science.aac4716 · Cited by 8,515
Power failure: why small sample size undermines the reliability of neuroscience
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3475 · Cited by 7,641 · cited 1×
“Some have recommended u s e on l y increased sample sizes (Button et al. 2013, Francis 2012a, Maxwell et al. 2015, Perugini 2014), and some have even suggested specific target attention to measurement error (Asendorpf et al. 2013) and to the quality of peer review (Coyne 2016, Nosek & Bar-Anan F o r…”
BackgroundSimmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · cited 3×
“…of reported analyses, and replication of Y o r U empirical findings go far beyond the boundaries of the field of psychology (Dickersin & Rennie 2012, Ioannidis 2005), although our treatment in this article focuses on our own field. i d e by e w state of our science (Coyne 2016, Schmidt & Oh 2016),…”
Research works printed but not linked: 9
- 10.4135/9780857020994.n16 · doi:10.4135/9780857020994.n16
- Box JF, 1978, R. A. Fisher: The Life of a Scientist
- Cronbach LJ, 1972, The Dependability of Behavioral Measurements
- Fisher RA, 1958, Centen. Rev. Arts Sci., 2, 261
- 10.1037/pspi0000013 · doi:10.1037/pspi0000013
- Kruschke JK, 2017, Psychon. Bull. Rev., 24, 1, 10.3758/s13423-016-1113-7 · doi:10.3758/s13423-016-1113-7
- Mill JS, 2008, A System of Logic
- 10.1037/0033-2909.113.3.553 · doi:10.1037/0033-2909.113.3.553 · Paper page
- Schmidt FL, 2016, Arch. Sci. Psychol., 4, 32
Cited by
Siddaway, A. P., Wood, A. M., & Hedges, L. V. (2019) · Annual Review of Psychology · doi:10.1146/annurev-psych-010418-102803 · Cited by 2,550
Jusup, M., Holme, P., Kanazawa, K., Takayasu, M., Romić, I., Wang, Z., Geček, S., Lipić, T., Podobnik, B., Wang, L., et al. (2022) · Physics Reports · doi:10.1016/j.physrep.2021.10.005 · Cited by 643
Goretzko, D., Pham, T. T. H., & Bühner, M. (2021) · Current Psychology · doi:10.1007/s12144-019-00300-2 · Cited by 621
Yarkoni, T. (2022) · Behavioral and Brain Sciences · doi:10.1017/s0140525x20001685 · Cited by 701
Small Effects: The Indispensable Foundation for a Cumulative Psychological Science
Götz, F. M., Gosling, S. D., & Rentfrow, P. J. (2022) · Perspectives on Psychological Science · doi:10.1177/1745691620984483 · Cited by 538
Metadata
- Authors
- Shrout, P. E., & Rodgers, J. L.
- Venue
- Annual Review of Psychology
- Publisher
- Annual Reviews
- Published
- 2018-01-04
- Volume
- 69
- Issue
- 1
- Pages
- 487-510
- ISSN
- 0066-4308 (ISSN-L)
- 1545-2085
- OpenAlex
- W2573945754
- Cited by
- 725
- References
- 97
- FWCI
- 15.74
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Psychology, Science, and Knowledge Construction: Broadening Perspectives from the Replication Crisis. https://app.cochat.ai/papers/10.1146/annurev-psych-122216-011845
- The paper
- Shrout, P. E., & Rodgers, J. L. (2018). Psychology, Science, and Knowledge Construction: Broadening Perspectives from the Replication Crisis. Annual Review of Psychology. https://doi.org/10.1146/annurev-psych-122216-011845
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.