The DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index.
The Value of Direct Replication
Abstract
Reproducibility is the cornerstone of science. If an effect is reliable, any competent researcher should be able to obtain it when using the same procedures with adequate statistical power. Two of the articles in this special section question the value of direct replication by other laboratories. In this commentary, I discuss the problematic implications of some of their assumptions and argue that direct replication by multiple laboratories is the only way to verify the reliability of an effect.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“almost none of the findings in the literature underwent direct replication attempts”
From Is there a Publication Bias in Behavioural Intranasal Oxytocin Research on Humans? Opening the File Drawer of One Laboratory (2016) · cites this paper as [26]
Citations
Cites
The weirdest people in the world?
Henrich, J., Heine, S. J., & Norenzayan, A. (2010) · Behavioral and Brain Sciences · doi:10.1017/s0140525x0999152x · Cited by 12,641
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
Belief in the law of small numbers.
Tversky, A., & Kahneman, D. (1971) · Psychological Bulletin · doi:10.1037/h0031322 · Cited by 3,246
Measuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling
John, L. K., Loewenstein, G., & Prelec, D. (2012) · Psychological Science · doi:10.1177/0956797611430953 · Cited by 2,239
Evaluating amazon’s mechanical turk as a tool for experimental behavioral research
Crump, M. J. C., McDonnell, J. V., & Gureckis, T. M. (2013) · PLoS ONE · doi:10.1371/journal.pone.0057410 · Cited by 1,617
Pashler, H., & Wagenmakers, E. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612465253 · Cited by 1,321
Recommendations for Increasing Replicability in Psychology
Asendorpf, J. B., Conner, M. T., Fruyt, F. D., Houwer, J. D., Denissen, J. J. A., Fiedler, K., Fiedler, S., Funder, D. C., Kliegl, R. M., Nosek, B. A., et al. (2013) · European Journal of Personality · doi:10.1002/per.1919 · Cited by 876
The Replication Recipe: What makes for a convincing replication?
Brandt, M. J., IJzerman, H., Dijksterhuis, A., Farach, F. J., Geller, J., Giner‐Sorolla, R., Grange, J. A., Perugini, M., Spies, J. R., & Veer, A. V. '. (2013) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2013.10.005 · Cited by 758
“Positive” results increase down the Hierarchy of the Sciences
Fanelli, D. (2010) · PLoS ONE · doi:10.1371/journal.pone.0010068 · Cited by 734
Is the Replicability Crisis Overblown? Three Arguments Examined
Pashler, H., & Harris, C. R. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612463401 · Cited by 697
Germine, L., Nakayama, K., Duchaine, B. C., Chabris, C. F., Chatterjee, G., & Wilmer, J. B. (2012) · Psychonomic Bulletin & Review · doi:10.3758/s13423-012-0296-9 · Cited by 669
The alleged crisis and the illusion of exact replication
Stroebe, W., & Strack, F. (2014) · Perspectives on Psychological Science · doi:10.1177/1745691613514450 · Cited by 597
Priming, Replication, and the Hardest Science
Cesario, J. (2014) · Perspectives on Psychological Science · doi:10.1177/1745691613513470 · Cited by 320
Giner-Sorolla, R. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612457576 · Cited by 274
Life after P-HackingPreprint
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2013) · SSRN Electronic Journal · doi:10.2139/ssrn.2205186 · Cited by 210
The importance of negative results in psychological research.
Smart, R. (1964) · The Canadian Psychologist · doi:10.1037/h0083036 · Cited by 156
Psychology’s Woes and a Partial Cure: The Value of Replication
Roediger, H. L., III (2012) · APS observer · Cited by 101
An Intersensory Interaction Account of Priming Effects—and Their Absence
Klatzky, R. L., & Creswell, J. D. (2014) · Perspectives on Psychological Science · doi:10.1177/1745691613513468 · Cited by 36
Research works printed but not linked: 1
- Roediger H. L., 2012, Observer, 25, 27
Cited by
Sorted by citations
Is psychology suffering from a replication crisis? What does “failure to replicate” really mean?
Maxwell, S. E., Lau, M. Y., & Howard, G. S. (2015) · American Psychologist · doi:10.1037/a0039400 · Cited by 1,102
Constraints on Generality (COG): A Proposed Addition to All Empirical Papers
Simons, D. J., Shoda, Y., & Lindsay, D. S. (2017) · Perspectives on Psychological Science · doi:10.1177/1745691617708630 · Cited by 986
Self-citationReplicability, Robustness, and Reproducibility in Psychological Science
Nosek, B. A., Hardwicke, T. E., Moshontz, H., Allard, A., Corker, K. S., Dreber, A., Fidler, F., Hilgard, J., Kline Struhl, M., Nuijten, M. B., et al. (2022) · Annual Review of Psychology · doi:10.1146/annurev-psych-020821-114157 · Cited by 770
Nelson, L. D., Simmons, J., & Simonsohn, U. (2018) · Annual Review of Psychology · doi:10.1146/annurev-psych-122216-011836 · Cited by 548
Ferguson, C. J. (2015) · Perspectives on Psychological Science · doi:10.1177/1745691615592234 · Cited by 524
Metadata
- Authors
- Simons, D. J.
- Affiliations
- University of Illinois Urbana-Champaign (US)
- Corresponding author
- Simons, Daniel J.
- ORCID
- Simons, Daniel J.
- Venue
- Perspectives on Psychological Science
- Publisher
- SAGE Publications
- Published
- 2014-01
- Volume
- 9
- Issue
- 1
- Pages
- 76-80
- ISSN
- 1745-6916 (ISSN-L)
- 1745-6924
- OpenAlex
- W2077240377
- Cited by
- 630
- References
- 18
- Citations by year
- 2026: 18 · 2025: 36 · 2024: 29 · 2023: 44 · 2022: 51 · 2021: 69 · 2020: 91 · 2019: 75 · 2018: 67 · 2017: 47 · 2016: 36 · 2015: 48 · 2014: 18
- FWCI
- 48.59
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Perspectives on Psychological Science (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Social Sciences › Psychology › General Psychology › Academic and Historical Perspectives in Psychology
- Keywords
- direct replication, reproducibility, statistical power, replication study, research credibility
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
