The DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index.
The Value of Direct Replication
Abstract
Reproducibility is the cornerstone of science. If an effect is reliable, any competent researcher should be able to obtain it when using the same procedures with adequate statistical power. Two of the articles in this special section question the value of direct replication by other laboratories. In this commentary, I discuss the problematic implications of some of their assumptions and argue that direct replication by multiple laboratories is the only way to verify the reliability of an effect.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“almost none of the findings in the literature underwent direct replication attempts”
From Is there a Publication Bias in Behavioural Intranasal Oxytocin Research on Humans? Opening the File Drawer of One Laboratory (2016) · cites this paper as [26]
Citations
Cites
The weirdest people in the world?
Henrich, J., Heine, S. J., & Norenzayan, A. (2010) · Behavioral and Brain Sciences · doi:10.1017/s0140525x0999152x · Cited by 12,641
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
Belief in the law of small numbers.
Tversky, A., & Kahneman, D. (1971) · Psychological Bulletin · doi:10.1037/h0031322 · Cited by 3,246
Measuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling
John, L. K., Loewenstein, G., & Prelec, D. (2012) · Psychological Science · doi:10.1177/0956797611430953 · Cited by 2,239
Evaluating amazon’s mechanical turk as a tool for experimental behavioral research
Crump, M. J. C., McDonnell, J. V., & Gureckis, T. M. (2013) · PLoS ONE · doi:10.1371/journal.pone.0057410 · Cited by 1,617
Pashler, H., & Wagenmakers, E. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612465253 · Cited by 1,321
Recommendations for Increasing Replicability in Psychology
Asendorpf, J. B., Conner, M. T., Fruyt, F. D., Houwer, J. D., Denissen, J. J. A., Fiedler, K., Fiedler, S., Funder, D. C., Kliegl, R. M., Nosek, B. A., et al. (2013) · European Journal of Personality · doi:10.1002/per.1919 · Cited by 876
The Replication Recipe: What makes for a convincing replication?
Brandt, M. J., IJzerman, H., Dijksterhuis, A., Farach, F. J., Geller, J., Giner‐Sorolla, R., Grange, J. A., Perugini, M., Spies, J. R., & Veer, A. V. '. (2013) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2013.10.005 · Cited by 758
“Positive” results increase down the Hierarchy of the Sciences
Fanelli, D. (2010) · PLoS ONE · doi:10.1371/journal.pone.0010068 · Cited by 734
Is the Replicability Crisis Overblown? Three Arguments Examined
Pashler, H., & Harris, C. R. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612463401 · Cited by 697
Research works printed but not linked: 1
- Roediger H. L., 2012, Observer, 25, 27
Cited by
Sorted by citations
Items 601–634 of 634
Method and Interpretation in the Humanities
Grimm, S. R., Peels, R., & van Woudenberg, R. (2025) · doi:10.1093/9780197749647.003.0005
Grimm, S. R., Peels, R., & van Woudenberg, R. (2025) · doi:10.1093/9780197749647.003.0009
The Humanities and Political Aims
Grimm, S. R., Peels, R., & van Woudenberg, R. (2025) · doi:10.1093/9780197749647.003.0007
Replications in Judgment and Decision Making: A Critical Review and Future Pathways
Sobków, A., Jurewicz, K., & Traczyk, J. (2025) · Integrated science · doi:10.1007/978-3-031-88415-3_2
Kammermann, B. (2025) · Queensland University of Technology · doi:10.5204/thesis.eprints.258886
URGE: The easiest way to create a great ReadMe file for your dataset
Torsoe, D.-E., Dogan, A., & Kelil, A. A. (2025) · Septentrio Conference Series · doi:10.7557/5.8194
The Invisible Influence on Canadian Police Decision-Making
Chitpin, S. (2025) · Emerald Group Publishing Limited eBooks · doi:10.1108/978-1-83662-694-720251004
The flow and flux of feeling: relations between attention to emotion and emotion reactivity dynamics
Robinson, M. D., Asad, M. R., Fereidouni, H., Irvin, R. L., & Klein, R. J. (2025) · Cognition & Emotion · doi:10.1080/02699931.2025.2568556
Implications of the Replication Crisis
Schweinberger, M. (2025) · Cambridge University Press eBooks · doi:10.1017/9781009415682.010
Benefits of Open and High-Powered Research Outweigh CostsPreprint
LeBel, E. P., Campbell, L., & Loving, T. J. (2016) · SSRN Electronic Journal · doi:10.2139/ssrn.2616384
In the Beast Mode: Predator Self-Identifications as a Model of Disagreeable Functioning
Robinson, M. D., Irvin, R. L., & Asad, M. R. (2026) · Assessment · doi:10.1177/10731911251407880
Scharf, F., & Müller, D. (2026) · International Journal of Psychophysiology · doi:10.1016/j.ijpsycho.2026.113330
Ackerman, S. L., & Moore, M. T. (2026) · International Journal of Cognitive Behavioral Therapy · doi:10.1007/s41811-026-00277-4
Arnéguy, E. (2020) · theses.fr (ABES) · doi:10.70675/9de4269dzb68az472dza9f9z344d2af7b15e
Implications d'un essentialisme stratégique dans le paradigme de la menace du stéréotype
Nativel, A. (2021) · theses.fr (ABES) · doi:10.70675/2056e402zec31z4ef9z8c28z725574513d90
Three Essays on Consumer Social Cognition in a Technology-Rich World
Linares, C. (2022) · theses.fr (ABES) · doi:10.70675/4898d96az4b9bz443fz8877z1201e0ad2d7c
Psychology’s RenaissancePreprint
Simonsohn, U., Nelson, L. D., & Simmons, J. P. (2022) · OSF Preprints (OSF Preprints)
Nieuwenhuys, A., Furley, P., & Bijlstra, G. (2023) · OSF Preprints (OSF Preprints)
Verschuère, B., Kleinberg, B., & Warmelink, L. (2023) · OSF Preprints (OSF Preprints)
Priming Resistance to Persuasion decreases adherence to Conspiracy TheoriesPreprint
Varet, F., Monaco, G. L., Adam‐Troian, J., Bonetto, É., & Girandola, F. (2018) · OSF Preprints (OSF Preprints)
White, J. B., Bellman, S., Beal, V., Michelon, A. J., & Sharp, B. M. (2026) · International Journal of Market Research · doi:10.1177/14707853261459215
On the Goals and Limitations of Psychological Science: Some Thoughts in Memory of Daniel Kahneman
Keren, G. B. (2026) · Perspectives on Psychological Science · doi:10.1177/17456916261417988
The Unresolved Puzzle of Psychological Theories
de Boeck, P. A. L., Pek, J. C. X., Andersen, B. L., & Turner, B. M. (2026) · Springer eBooks · doi:10.1007/978-3-032-01453-5_7
Longitudinal models of skill development: conceptual and statistical considerations
Duncan, R. J., Ernst, J. R., Anderson, K. L., Purpura, D. J., & Schmitt, S. A. (2026) · Journal of Experimental Child Psychology · doi:10.1016/j.jecp.2026.106600
Morgulev, E., Raphaeli, M., & Goldschmied, N. P. (2026) · Basketball Studies · doi:10.1016/j.basket.2026.100006
The cost of reproducibility in artificial intelligence
Snelleman, T., Hoos, H. H., & Gundersen, O. E. (2026) · Patterns · doi:10.1016/j.patter.2026.101640
A Replication of Rensink, O'Regan, and Clark (1997): Change Blindness in an Online Flicker Paradigm
Ouattara, M. M., Alexandre, E., Gagnon, M., Joanette, F., Lamarre, É., Guirreh, S. M., Adon, O. A., Bellout, A., Billings, B., Bisson, G., et al. (2026) · The Quantitative Methods for Psychology · doi:10.20982/tqmp.22.2.p056
Evaluation of replication success: A statistical perspective and extended analysis of the SCORE data
van Aert, R. C. M., van Assen, M. A. L. M., & López-Nicolás, R. (2026) · PLoS ONE · doi:10.1371/journal.pone.0358126
Matarese, V. (2026) · Elsevier eBooks · doi:10.1016/b978-0-443-40622-5.00015-3
Vieth, H. E. (2015) · The University of Queensland · doi:10.14264/uql.2015.601
Metadata
- Authors
- Simons, D. J.
- Affiliations
- University of Illinois Urbana-Champaign (US)
- Corresponding author
- Simons, Daniel J.
- ORCID
- Simons, Daniel J.
- Venue
- Perspectives on Psychological Science
- Publisher
- SAGE Publications
- Published
- 2014-01
- Volume
- 9
- Issue
- 1
- Pages
- 76-80
- ISSN
- 1745-6916 (ISSN-L)
- 1745-6924
- OpenAlex
- W2077240377
- Cited by
- 630
- References
- 18
- Citations by year
- 2026: 18 · 2025: 36 · 2024: 29 · 2023: 44 · 2022: 51 · 2021: 69 · 2020: 91 · 2019: 75 · 2018: 67 · 2017: 47 · 2016: 36 · 2015: 48 · 2014: 18
- FWCI
- 48.59
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Perspectives on Psychological Science (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Social Sciences › Psychology › General Psychology › Academic and Historical Perspectives in Psychology
- Keywords
- direct replication, reproducibility, statistical power, replication study, research credibility
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
