The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Is the Replicability Crisis Overblown? Three Arguments Examined
Abstract
We discuss three arguments voiced by scientists who view the current outpouring of concern about replicability as overblown. The first idea is that the adoption of a low alpha level (e.g., 5%) puts reasonable bounds on the rate at which errors can enter the published literature, making false-positive effects rare enough to be considered a minor issue. This, we point out, rests on statistical misunderstanding: The alpha level imposes no limit on the rate at which errors may arise in the literature (Ioannidis, 2005b). Second, some argue that whereas direct replication attempts are uncommon, conceptual replication attempts are common—providing an even better test of the validity of a phenomenon. We contend that performing conceptual rather than direct replication attempts interacts insidiously with publication bias, opening the door to literatures that appear to confirm the reality of phenomena that in fact do not exist. Finally, we discuss the argument that errors will eventually be pruned out of the literature if the field would just show a bit of patience. We contend that there are no plausible concrete scenarios to back up such forecasts and that what is needed is not patience, but rather systematic reforms in scientific practice.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“If about 10% of all postulated hypotheses are correct, similar to what is expected for experimental psychology in general (21), we would expect, in the absence of bias, that only about 2% of the investigated effects would turn out to be statistically significant.”
From Statistical and Methodological Considerations for the Interpretation of Intranasal Oxytocin Studies (2016) · cites this paper as [21]
In the citing paper
“Although it is often argued that conceptual replications assess both the validity and generality of previous studies, and therefore should be considered more effective than direct replications, this is not necessarily true (21).”
From Statistical and Methodological Considerations for the Interpretation of Intranasal Oxytocin Studies (2016) · cites this paper as [21]
In the citing paper
“If the value of R for IN-OT studies is approximately 0.10, comparable to what has suggested for experimental psychology (21) and exploratory epidemiology (16), the PPV is no higher than 24%, given a statistical power of 16%.”
From Statistical and Methodological Considerations for the Interpretation of Intranasal Oxytocin Studies (2016) · cites this paper as [21]
In the citing paper
“From a statistical point of view this problem might not be any more serious for this field than other domains of psychology (20, 21, 31) or neuroscience in general (14).”
From Statistical and Methodological Considerations for the Interpretation of Intranasal Oxytocin Studies (2016) · cites this paper as [21]
In the citing paper
“The only way to effectively separate true from false hypotheses is through repeated investigation, including both direct and conceptual replication [14,15].”
From The natural selection of bad science (2016) · cites this paper as [14]
3 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · cited 8×
“Thanks to the work of Ioannidis (2005b) and other statisticians, the essential problems with this view are already familiar to many.”
BackgroundInfluentialRaise standards for preclinical cancer research
Begley, C. G., & Ellis, L. M. (2012) · Nature · doi:10.1038/483531a · Cited by 2,979 · cited 3×
“Writing in Nature, Begley and Ellis (2012) described the firsthand experience of the Amgen Corporation over a 10-year period in attempting to build drug development programs upon 53 “landmark” published studies in preclinical (basic) cancer research.”
BackgroundHARKing: Hypothesizing after the results are known
Kerr, N. L. (1998) · Personality and Social Psychology Review · doi:10.1207/s15327957pspr0203_4 · Cited by 1,989
Contradicted and Initially Stronger Effects in Highly Cited Clinical Research
Ioannidis, J. P. A. (2005) · JAMA · doi:10.1001/jama.294.2.218 · Cited by 1,465
Wagenmakers, E.-J., Wetzels, R., Borsboom, D., & van der Maas, H. L. J. (2011) · Journal of Personality and Social Psychology · doi:10.1037/a0022790 · Cited by 977
The rules of the game called psychological science
Bakker, M., van Dijk, A., & Wicherts, J. M. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612459060 · Cited by 800 · cited 1×
“The key element would seem to be that a pseudoresult appears both exciting and easy—the kind of result that would tempt hundreds of researchers to undertake small low-powered conceptual replication attempts (see Bakker et al., 2012; Ioannidis, 2005b).”
BackgroundWhy Most Published Research Findings Are False
Ioannidis, J. P. A. (2019) · CHANCE · doi:10.1080/09332480.2019.1579573 · Cited by 463
Sterling, T. D., Rosenbaum, W. L., & Weinkam, J. J. (1995) · The American Statistician · doi:10.1080/00031305.1995.10476125 · Cited by 453
Cited by
Using Bayes to get the most out of non-significant results
Dienes, Z. (2014) · Frontiers in Psychology · doi:10.3389/fpsyg.2014.00781 · Cited by 1,998
P-curve: A key to the file-drawer.
Simonsohn, U., Nelson, L. D., & Simmons, J. P. (2014) · Journal of Experimental Psychology: General · doi:10.1037/a0033242 · Cited by 1,500
Pashler, H., & Wagenmakers, E. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612465253 · Cited by 1,321
Self-citationHow to get statistically significant effects in any ERP experiment (and why you shouldn't)
Luck, S. J., & Gaspelin, N. (2017) · Psychophysiology · doi:10.1111/psyp.12639 · Cited by 1,195
A Multilab Preregistered Replication of the Ego-Depletion Effect
Hagger, M. S., Chatzisarantis, N. L. D., Alberts, H., Anggono, C. O., Batailler, C., Birt, A. R., Brand, R., Brandt, M. J., Brewer, G., Bruyneel, S., et al. (2016) · Perspectives on Psychological Science · doi:10.1177/1745691616652873 · Cited by 1,079
Metadata
- Authors
- Pashler, H., & Harris, C. R.
- Affiliations
- University of California San Diego (US)
- ORCID
- Pashler, Harold
- Venue
- Perspectives on Psychological Science
- Publisher
- SAGE Publications
- Published
- 2012-11
- Volume
- 7
- Issue
- 6
- Pages
- 531-536
- ISSN
- 1745-6916 (ISSN-L)
- 1745-6924
- OpenAlex
- W2167083402
- Cited by
- 697
- References
- 21
- Citations by year
- 2026: 33 · 2025: 22 · 2024: 23 · 2023: 27 · 2022: 42 · 2021: 74 · 2020: 67 · 2019: 61 · 2018: 63 · 2017: 75 · 2016: 59 · 2015: 60 · 2014: 54 · 2013: 32 · 2012: 4
- FWCI
- 53.90
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
- Locations
- Perspectives on Psychological Science (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Keywords
- replication crisis, false positives, significance level, publication bias, conceptual replication, direct replication, statistical power, reproducibility
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Is the Replicability Crisis Overblown? Three Arguments Examined. https://app.cochat.ai/papers/10.1177/1745691612463401
- The paper
- Pashler, H., & Harris, C. R. (2012). Is the Replicability Crisis Overblown? Three Arguments Examined. Perspectives on Psychological Science. https://doi.org/10.1177/1745691612463401
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.