Psychology’s Replication Crisis and the Grant Culture: Righting the Ship
Open accessThe DOI matches its Crossref record; 7 of the 7 references it cites were found in a registry or an index.
Abstract
The past several years have been a time for soul searching in psychology, as we have gradually come to grips with the reality that some of our cherished findings are less robust than we had assumed. Nevertheless, the replication crisis highlights the operation of psychological science at its best, as it reflects our growing humility. At the same time, institutional variables, especially the growing emphasis on external funding as an expectation or de facto requirement for faculty tenure and promotion, pose largely unappreciated hazards for psychological science, including (a) incentives for engaging in questionable research practices, (b) a single-minded focus on programmatic research, (c) intellectual hyperspecialization, (d) disincentives for conducting direct replications, (e) stifling of creativity and intellectual risk taking, (f) researchers promising more than they can deliver, and (g) diminished time for thinking deeply. Preregistration should assist with (a), but will do little about (b) through (g). Psychology is beginning to right the ship, but it will need to confront the increasingly deleterious impact of the grant culture on scientific inquiry.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The case for motivated reasoning.
Kunda, Z. (1990) · Psychological Bulletin · doi:10.1037/0033-2909.108.3.480 · Cited by 9,022
Estimating the reproducibility of psychological science
Open Science Collaboration (2015) · Science · doi:10.1126/science.aac4716 · Cited by 8,515
Confirmation Bias: A Ubiquitous Phenomenon in Many Guises
Nickerson, R. S. (1998) · Review of General Psychology · doi:10.1037/1089-2680.2.2.175 · Cited by 7,267
Arkes, H. R., & Blumer, C. (1985) · Organizational Behavior and Human Decision Processes · doi:10.1016/0749-5978(85)90049-4 · Cited by 2,573 · cited 1×
“Research on sunk costs (Arkes & Blumer, 1985) and effort justification (Axsom & Cooper, 1985) further suggests that once individuals have invested a great deal of time and effort in a project, they will feel the need to persist in it even when doing so is no longer fruitful.”
BackgroundNosek (2012) · doi:10.1177/1745691612459058 · Cited by 1,301
How to Make More Published Research True
Ioannidis, J. P. A. (2014) · PLoS Medicine · doi:10.1371/journal.pmed.1001747 · Cited by 760 · cited 2×
“…from institutional incentives for prioritizing scholarly quality and replicabil ity over financial success to more radical proposals, such as penalizing scholars who have a lengthy track record of grant funding without a commensurate record of high quality published research (Ioannidis, 2014).”
BackgroundUnder what conditions does theory obstruct research progress?
Greenwald, A. G., Pratkanis, A. R., Leippe, M. R., & Baumgardner, M. H. (1986) · Psychological Review · doi:10.1037/0033-295x.93.2.216 · Cited by 574
Cited by
Yarkoni, T. (2022) · Behavioral and Brain Sciences · doi:10.1017/s0140525x20001685 · Cited by 701
The ABCD study: understanding the development of risk for mental and physical health outcomes
Karcher, N. R., & Barch, D. M. (2021) · Neuropsychopharmacology · doi:10.1038/s41386-020-0736-6 · Cited by 351
“The importance of replication is increasingly highlighted as a part of robust research practices [71, 77].”
BackgroundTackett, J. L., Lilienfeld, S. O., Patrick, C. J., Johnson, S. L., Krueger, R. F., Miller, J. D., Oltmanns, T. F., & Shrout, P. E. (2017) · Perspectives on Psychological Science · doi:10.1177/1745691617690042 · Cited by 251
Self-citationWhat Is the Function of Confirmation Bias?
Peters, U. (2022) · Erkenntnis · doi:10.1007/s10670-020-00252-1 · Cited by 236
Beauchaine, T. P., Bell, Z., Knapton, E., McDonough‐Caplan, H., Shader, T., & Zisner, A. (2019) · Psychophysiology · doi:10.1111/psyp.13329 · Cited by 144
Metadata
- Authors
- Lilienfeld, S. O.
- Venue
- Perspectives on Psychological Science
- Publisher
- SAGE Publications
- Published
- 2017-07
- Volume
- 12
- Issue
- 4
- Pages
- 660-664
- ISSN
- 1745-6916 (ISSN-L)
- 1745-6924
- OpenAlex
- W2738348569
- Cited by
- 187
- References
- 7
- FWCI
- 27.86
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.