The DOI matches its Crossref record; 256 of the 256 references it cites were found in a registry or an index.
Replicability, Robustness, and Reproducibility in Psychological Science
Open accessPublished version of a preprint
Abstract
Replication—an important, uncommon, and misunderstood practice—is gaining appreciation in psychology. Achieving replicability is important for making research progress. If findings are not replicable, then prediction and theory development are stifled. If findings are replicable, then interrogation of their meaning and validity can advance knowledge. Assessing replicability can be productive for generating and testing hypotheses by actively confronting current understandings to identify weaknesses and spur innovation. For psychology, the 2010s might be characterized as a decade of active confrontation. Systematic and multi-site replication projects assessed current understandings and observed surprising failures to replicate many published findings. Replication efforts highlighted sociocultural challenges such as disincentives to conduct replications and a tendency to frame replication as a personal attack rather than a healthy scientific practice, and they raised awareness that replication contributes to self-correction. Nevertheless, innovation in doing and understanding replication and its cousins, reproducibility and robustness, has positioned psychology to improve research practices and accelerate progress.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“By using a common approach to analyze data from multiple informants, time points, and developmental domains, we can address concerns about the robustness of scientific findings (20,21).”
From A Comprehensive Multilevel Analysis of the Bucharest Early Intervention Project: Causal Effects on Recovery From Early Severe Deprivation (2023) · cites this paper as [21]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This means that even when updates address and mitigate detected biases, they also introduce the potential for “process reproducibility failure” (98) in the generated data and impede reproducibility, critical for scientific rigor (15, 100, 101).”
From Perils and opportunities in using large language models in psychological research (2024) · cites this paper as [98]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In psychology, across 307 systematic replications and multisite replications, 64% reported statistically significant evidence in the same direction and effect sizes 68% as large as the original experiments”
From Investigating the replicability of preclinical cancer biology (2021) · cites this paper as Nosek et al., 2021
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Research into replicability in other disciplines has also found that findings with stronger initial evidence (such as larger effect sizes and/or smaller p-values) is more likely to replicate”
From Investigating the replicability of preclinical cancer biology (2021) · cites this paper as Nosek et al., 2021
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Independent verification is central to the scientific method (Nosek et al., 2022).”
From Read the Paper, Write the Code: Agentic Reproduction of Social-Science Results (2026) · cites this paper as Nosek et al., 2022
The cited source says what the claim attributes to it.
In the citing paper
“A substantial proportion of replications in recent largescale efforts have failed to support the original finding (Nosek et al., 2022).”
From Many Labs 4: Failure to Replicate Mortality Salience Effect With and Without Original Author Involvement (2022) · cites this paper as Nosek et al., 2022
In the citing paper
“However, particular challenges arise when applying these principles to LLMs, particularly proprietary ones, whose black-box nature impedes the reproducibility of findings that pertain to them.”
From Perils and opportunities in using large language models in psychological research (2024) · cites this paper as [98]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Valente, T. W. (2003) · Genetics in Medicine · doi:10.1097/01.gim.0000061743.67794.c4 · Cited by 20,559
The FAIR Guiding Principles for scientific data management and stewardship
Wilkinson, M. D., Dumontier, M., Aalbersberg, I. J., Appleton, G., Axton, M., Baak, A., Blomberg, N., Boiten, J.-W., da Silva Santos, L. B., Bourne, P. E., et al. (2016) · Scientific Data · doi:10.1038/sdata.2016.18 · Cited by 18,423
(2002) · Cited by 13,211
Construct validity in psychological tests.
Cronbach, L. J., & Meehl, P. E. (1955) · Psychological Bulletin · doi:10.1037/h0040957 · Cited by 11,195
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068
The case for motivated reasoning.
Kunda, Z. (1990) · Psychological Bulletin · doi:10.1037/0033-2909.108.3.480 · Cited by 9,022
Cohen, J. (1992) · Psychological Bulletin · doi:10.1037//0033-2909.112.1.155 · Cited by 8,771
Estimating the reproducibility of psychological science
Open Science Collaboration (2015) · Science · doi:10.1126/science.aac4716 · Cited by 8,515
Power failure: why small sample size undermines the reliability of neuroscience
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3475 · Cited by 7,641
Self-citation
Research works printed but not linked: 17
- Artner R, 2020, Psychol. Methods.
- Cent. Open Sci, 2020, Center for Open Science
- Cohen J., 1973, Am. Educ. Res. J., 10, 225
- 10.1037/0033-2909.112.1.155 · DOI · Paper page
- Crisp RJ, 2014, Soc. Psychol., 45, 303
- Epskamp S, 2018, Statistical Software
- Errington TM, 2021, eLife
- Ferguson MJ, 2014, Soc. Psychol., 45, 301
- Gelman A, 2013, The garden of forking paths: why multiple comparisons can be a problem, even when there is no “fishing expedition” or “p-hacking” and the research hypothesis was posited ahead of time
- Marcus A, 2020, WIRED
- Meyer MN, 2014, Slate Magazine
- Mischel W., 2008, APS Observer
- Nosek B., 2019, Center for Open Science
- Nosek B., 2019, Center for Open Science
- Schwarz N, 2014, Soc. Psychol., 45, 305
- Sterling TD., 1959, J. Am. Stat. Assoc., 54, 30
- Yong E, 2012, National Geo-graphic
Cited by
Sorted by citations
Items 601–700 of 772
Prevalence and Consequences of Anomalous Citation Styles in Ethiopian Academic JournalsPreprint
Asefa, A., & Debella, H. J. (2024) · Research Square · doi:10.21203/rs.3.rs-4387184/v1
Saldarini, F., Yamasaki, K., & Miyata, H. (2024) · Mindfulness · doi:10.1007/s12671-024-02376-9
Researchers’ Perceptions of the State of Social and Personality PsychologyPreprint
Schiavone, S. R., Vazire, S., & Rhemtulla, M. (2024) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/ja4u2
Self-citationA note on the discussion regarding terrorism and land use in agriculture
Mußhoff, O., Ölkers, T., & Kirchner, E. (2024) · Land Use Policy · doi:10.1016/j.landusepol.2024.107236
Camacho, N. L., & Gaffrey, M. S. (2024) · doi:10.31234/osf.io/sq93e
Reflecting on Past Theoretical Contributions in Psychological Science: A New Initiative
Grossmann, I. (2024) · Psychological Inquiry · doi:10.1080/1047840x.2024.2370740
Preregistration in psychology: The need to consider auxiliary assumptionsPreprint
Quinton, T. S. (2024) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/keza8
Search term validation in agricultural economics: conceptual background and applicationPreprint
Völker, R., Hirschauer, N., Lind, F., & Grüner, S. (2024) · doi:10.31219/osf.io/v68r7
Kreis, H., Wildner, R., & Kuß, A. (2024) · Springer eBooks · doi:10.1007/978-3-658-44456-3_2
Frank, M., Grüning, D. J., Pronizius, E., Korbmacher, M., Elsherif, M. M., Bergmann, L., Azevedo, F., Kramer, S. J. S., Straub, N., & Herderich, A. (2024) · doi:10.31219/osf.io/7gbvp
Enhancing Replication Research in Developmental Coordination Disorder
Ruddock, S., Petridis, L., & Rahimi‐Golkhandan, S. (2024) · Current Developmental Disorders Reports · doi:10.1007/s40474-024-00310-1
Why are replication rates so low?
Vu, P. (2024) · Journal of Econometrics · doi:10.1016/j.jeconom.2024.105868
Cliff, D. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4967488
Ensuring Robustness in Scientific Research, Split-root assays as an example casePreprint
Salvatore, L., Pierik, R., Kajala, K., & ten Tusscher, K. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.10.31.621251
Atshan, S., Calvo-Hernandez, C., Gulden, T., & Frank, A. (2024) · Springer proceedings in complexity · doi:10.1007/978-3-031-64193-0_20
The Promise and Pitfalls of Studying the Neurophysiological Correlates of Automatic Imitation
Darda, K. M., & Ramsey, R. (2024) · Springer eBooks · doi:10.1007/978-3-031-62634-0_6
Aggression - The Dark Side of Social ComparisonPreprint
Buechner, V. L., & Maier, M. (2024) · doi:10.31234/osf.io/62za7
Ghazzawi, R., Bender, M., Wiegand, A., & Miletich, M. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.5066730
Wanat, T. (2024) · Ruch Prawniczy Ekonomiczny i Socjologiczny · doi:10.14746/rpeis.2024.86.4.14
Conducting Multi-nation Research
Martos, T., Rosta‐Filep, O., León, G. A., Orbán, K., & Kanth, D. B. (2025) · Cambridge University Press eBooks · doi:10.1017/9781009345774.003
Ip, M. H. C., & Feldman, G. (2025) · Personality and Social Psychology Bulletin · doi:10.1177/01461672251327169
Open and Reproducible Special Education Research
Kalandadze, T. (2025) · Springer eBooks · doi:10.1007/978-3-031-40858-8_24-1
Preregistration in psychology: The need to consider auxiliary assumptionsPreprint
Quinton, T. S. (2024) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/keza8_v1
Bonheur : ce que dit la science
Dance, A. (2025) · Cerveau & Psycho · doi:10.3917/cerpsy.176.0060
Horsham, Z., Haydock-Symonds, A. N., Imada, H., Tai, H. C., Lau, W. L., Shum, T. L., Zeng, Y., Chow, H. T. K., & Feldman, G. (2025) · Royal Society Open Science · doi:10.1098/rsos.250441 · Cited by 1
Larivée, S. J., & Sénéchal, C. (2025) · Revue française de criminologie et de droit pénal. · doi:10.3917/rfcdp.023.0003
Challenges and actions towards a more comprehensive developmental science
Tantaros, S. (2025) · European Journal of Developmental Psychology · doi:10.1080/17405629.2025.2498756
Replication using confirmatory latent class analysis: A school climate example
Romero, L. S., O’Malley, M. D., & Carter, D. B. (2025) · Journal of School Psychology · doi:10.1016/j.jsp.2025.101455 · Cited by 2
Ethical Standard in Psychological Research: A Review
Hu, J. (2023) · Psychomachina · doi:10.59388/pm00343 · Cited by 3
Revisiting the Replication Crisis and the Untrustworthiness of Empirical Evidence
Spanos, A. (2025) · Stats · doi:10.3390/stats8020041
Reproducibility in lie detection research: A case study of the cue called complications
Neequaye, D. A. (2025) · Legal and Criminological Psychology · doi:10.1111/lcrp.12315
Frenken, F., Evert, S., Schneider, G., & Neumann, S. (2025) · ICAME journal · doi:10.2478/icame-2025-0003
Bajraktari, R., Booth, F., Munafo, M. R., & Beazley‐Long, N. (2025) · F1000Research · doi:10.12688/f1000research.161819.1
Stability and Change in Gratitude Expressions for the Fulfilment of Human Needs Categories
Nguyen, S. P., & Gordon, C. L. (2025) · Journal of Happiness Studies · doi:10.1007/s10902-025-00895-3
Theoretical and Practical Neuroscience Tools for Neurosurgery
Beffara, B., Veyrie, M., Desmurget, M., Mottolèse, C., Szathmári, A., Rocco, F. D., Cristofori, I., & Beuriat, P.-A. (2025) · Advances and technical standards in neurosurgery · doi:10.1007/978-3-031-90762-3_11 · Cited by 1
Reproductive Physiology and Maternal-Fetal Food Safety
Maqsood, S., Arshad, M. T., Khalid, W., & Bibi, A. (2025) · Springer eBooks · doi:10.1007/978-3-031-94582-3_6
From drowning to debunking: the cognitive science of religion's role in unraveling religious beliefs
McKay, R., Levy, N. L., van Elk, M., & Ross, R. M. (2025) · Religion Brain & Behavior · doi:10.1080/2153599x.2025.2474391
Anikin, A. (2025) · Psychonomic Bulletin & Review · doi:10.3758/s13423-025-02740-3
Doerr, E. M., Muffato, V., Borella, E., Carbone, E., Capodieci, A., Carretti, B., & Meneghetti, C. (2025) · Cognitive Processing · doi:10.1007/s10339-025-01291-8
A hybrid approach for pattern recognition and interpretation in age-related false memory
Golilarz, N. A., Hossain, E., Rahimi, S., & Karimi, H. (2025) · Frontiers in Psychology · doi:10.3389/fpsyg.2025.1579259
Kotseruba, I., & Tsotsos, J. K. (2025) · Oxford University Press eBooks · doi:10.1093/oso/9780192844835.002.0005
Kotseruba, I., & Tsotsos, J. K. (2025) · Oxford University Press eBooks · doi:10.1093/oso/9780192844835.002.0007
Kotseruba, I., & Tsotsos, J. K. (2025) · Oxford University Press eBooks · doi:10.1093/oso/9780192844835.003.0007
Kotseruba, I., & Tsotsos, J. K. (2025) · Oxford University Press eBooks · doi:10.1093/oso/9780192844835.003.0001
Taxonomies of Cognitive Architectures
Kotseruba, I., & Tsotsos, J. K. (2025) · Oxford University Press eBooks · doi:10.1093/oso/9780192844835.003.0004
Practical Applications of Cognitive Architectures
Kotseruba, I., & Tsotsos, J. K. (2025) · Oxford University Press eBooks · doi:10.1093/oso/9780192844835.003.0010
Cognitive Architectures, AI, and Cognitive Science
Kotseruba, I., & Tsotsos, J. K. (2025) · Oxford University Press eBooks · doi:10.1093/oso/9780192844835.003.0003
Kotseruba, I., & Tsotsos, J. K. (2025) · Oxford University Press eBooks · doi:10.1093/oso/9780192844835.002.0004
Kotseruba, I., & Tsotsos, J. K. (2025) · Oxford University Press eBooks · doi:10.1093/oso/9780192844835.003.0008
Challenges of the Past and Opportunities Ahead
Kotseruba, I., & Tsotsos, J. K. (2025) · Oxford University Press eBooks · doi:10.1093/oso/9780192844835.003.0013
Kotseruba, I., & Tsotsos, J. K. (2025) · Oxford University Press eBooks · doi:10.1093/oso/9780192844835.003.0006
Cognitive Architectures in the Deep Learning Era
Kotseruba, I., & Tsotsos, J. K. (2025) · Oxford University Press eBooks · doi:10.1093/oso/9780192844835.003.0012
Kotseruba, I., & Tsotsos, J. K. (2025) · Oxford University Press eBooks · doi:10.1093/oso/9780192844835.003.0009
What Are Cognitive Architectures?
Kotseruba, I., & Tsotsos, J. K. (2025) · Oxford University Press eBooks · doi:10.1093/oso/9780192844835.003.0002
Evaluating Cognitive Architectures
Kotseruba, I., & Tsotsos, J. K. (2025) · Oxford University Press eBooks · doi:10.1093/oso/9780192844835.003.0011
Miotti, S. (2025) · IGI Global eBooks · doi:10.4018/979-8-3373-4445-4.ch002
Research integrity, thick and thin
Vincent, M., & Hooper, M. (2025) · Synthese · doi:10.1007/s11229-025-05157-6 · Cited by 2
Hamilton, D. G., McKenzie, J. E., Nguyen, P., Rethlefsen, M. L., McDonald, S. J., Brennan, S., Fidler, F. M., Higgins, J. P. T., Kanukula, R., Karunananthan, S., et al. (2025) · medRxiv · doi:10.1101/2025.07.17.25331623
Self-citationTransparency and reproducibility in invasion science
Mologni, F., & Pither, J. (2025) · NeoBiota · doi:10.3897/neobiota.100.143736
Divisive negative discourse biases social experience: a live experiment at a massive public event
Ponferrada, J., Inchauspe, J., Zimmerman, F., Garbulsky, G., Navajas, J., & García, A. M. (2025) · Humanities and Social Sciences Communications · doi:10.1057/s41599-025-05652-8
Ensuring robustness in scientific research, split-root assays as an example case
Salvatore, L., Pierik, R., Kajala, K., & ten Tusscher, K. (2025) · Quantitative Plant Biology · doi:10.1017/qpb.2025.10004
Tomat, N., Kindermann, N., & Ohlhorst, J. (2025) · Analysis · doi:10.1093/analys/anaf032
Lam, M. I., Tong, L. K., Xu, Q., & Cheong, P. L. (2025) · Journal of Affective Disorders · doi:10.1016/j.jad.2025.120098
319Societal Implications of COVID-19 Misinformation in Uganda
Buwule, R. S. (2025) · De Gruyter eBooks · doi:10.1515/9783111176345-018
Chandrashekar, S. P., & Feldman, G. (2025) · Judgment and Decision Making · doi:10.1017/jdm.2025.10011
Reveilhac, M., & Schneider, G. (2025) · International Journal of Corpus Linguistics · doi:10.1075/ijcl.24132.rev
Li, M., & Feldman, G. (2025) · Royal Society Open Science · doi:10.1098/rsos.250979
Replicability and transparency in physical therapy research: Time to wake upPreprint
Jabouille, F., Farajzadeh, A., Exertier, A. G., Martine, J., Sadek, N., Klinger, L. H., Iliut, C. M., Kapsa, V., & Boisgontier, M. P. (2025) · medRxiv · doi:10.1101/2025.09.23.25336497 · Cited by 1
The Conundrums in Behavioral Animal Research and Some Findings with Ketamine
Ventura-Aquino, E., & Ågmo, A. (2025) · Neuromethods · doi:10.1007/978-1-0716-4603-8_4
Laverty, C., & Richards, C. (2025) · Journal of Intellectual Disability Research · doi:10.1111/jir.70055
Generative AI can influence climate beliefs and actions
Sabherwal, A. C. (2025) · Nature Climate Change · doi:10.1038/s41558-025-02438-3
Revisiting the Role of Gender and Beauty in Entrepreneurship: A Large-Scale Conceptual Replication
Deng, W. (2025) · Entrepreneurship Theory and Practice · doi:10.1177/10422587251382870
The need to consider auxiliary assumptions in preregistration practices
Quinton, T. S., & Trafimow, D. (2025) · Frontiers in Psychology · doi:10.3389/fpsyg.2025.1633990
Attentional Bias and Tobacco Smoking Frequency: A Reflection on Bartlett et al. (2022)
Copeland, A. (2025) · Journal of Trial and Error · doi:10.36850/f7a7-4c8d
Walleczek, J., von Stillfried, N., Schmidt, S., Wittmann, M., Kirmse, K. A., Moll, J., & Kekecs, Z. (2025) · PLoS ONE · doi:10.1371/journal.pone.0335330 · Cited by 1
Porreca, A., & Simonelli, A. (2025) · Child Abuse & Neglect · doi:10.1016/j.chiabu.2025.107786
Schoenherr, J. R. (2025) · Elsevier eBooks · doi:10.1016/b978-0-323-90509-1.00037-0
Willard, D. R., & Foster, J. D. (2025) · Personality and Individual Differences · doi:10.1016/j.paid.2025.113572
Null Hypothesis Statistical Testing (NHST)
Graham, M. E., & Slaney, K. L. (2025) · Springer eBooks · doi:10.1007/978-3-031-70581-6_9-1
Kowalski, J. (2025) · Przegląd Psychologiczny · doi:10.31648/przegldpsychologiczny.11470
Comparing expert assessments of research quality between the Global North and East Africa
Singh, J., Mody, F., Schmidt, M., Shari, S., Wambua, J., Wilkinson, D. P., & Forscher, P. S. (2025) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2024.0275
Robust and replicable effects of ageing on resting state brain electrophysiology measured with MEGPreprint
Quinn, A. J., Pitt, J., Kohl, O., Gohil, C., van Es, M. W. J., Nobre, A. C., & Woolrich, M. W. (2025) · eLife · doi:10.7554/elife.108829.1
Commentary on Publishing Systematic Reviews as Registered Reports
Cook, B. G., & Therrien, W. J. (2025) · Behavioral Disorders · doi:10.1177/01987429251400223
Hardwicke, T. E., & Nosek, B. A. (2020) · OSF Preprints (OSF Preprints) · doi:10.17605/osf.io/vg9zy
Self-citationBest, S., Thursky, K., Buzza, M., Klaic, M., Peters, S., Guccione, L., Trainer, A. H., & Francis, J. (2024) · Figshare · doi:10.6084/m9.figshare.25414898
Best, S., Thursky, K., Buzza, M., Klaic, M., Peters, S., Guccione, L., Trainer, A. H., & Francis, J. (2024) · Figshare · doi:10.6084/m9.figshare.25414972
Rubin, M. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4673378
Fechner, H. L., Reinhard†, M.-A., Ende, L., & Reinhardt, N. (2025) · Society and Animals · doi:10.1163/15685306-bja10250
Umanath, S., Coane, J. H., Huff, M. J., Cimenian, T., & Chang, K. (2024) · Figshare · doi:10.6084/m9.figshare.26559137
Flynn, M. K., Berkout, O. V., Malnar, C. S., Fraire, J. V., & Alde, E. N. (2025) · The Psychological Record · doi:10.1007/s40732-025-00666-9
Probabilistic functionalism as a limiting condition for robustness
Forsgren, M., Mandl, B., & Rehbinder, G. K. (2025) · Scientific Reports · doi:10.1038/s41598-025-32580-z
What should replicate in intelligence research? Setting the bar for a cumulative science
Iliescu, D., & Greiff, S. (2025) · Intelligence · doi:10.1016/j.intell.2025.101992
Brasil, M. L. D. A., Figueira, G. L., Pires, P. O., Zakhour, S., Nardi, A. E., & Sardinha, A. (2025) · Sexuality Research and Social Policy · doi:10.1007/s13178-025-01266-6
Robust tests should be the default, not the backup
Höfler, M. (2026) · Peer Community Journal · doi:10.24072/pcjournal.670 · Cited by 1
Lewis, R. J., Lembrechts, J. J., Walker, P., Li, C., & Engen, S. (2025) · Biodiversity Information Science and Standards · doi:10.3897/biss.10.183778
On the Complexity of a Seemingly ‘Simple Attitude Similarity–Attraction Relation’
Singh, R. (2026) · Psychology and Developing Societies · doi:10.1177/09713336251399447
Göbel, K., & Kluge, A. (2026) · Methods in Psychology · doi:10.1016/j.metip.2026.100232
The Tides of Change: Recent Trends and Developments in Close Relationships Research
Cavallo, J. V., & Perlman, D. (2026) · Cambridge University Press eBooks · doi:10.1017/9781009381949.002
Metadata
- Authors
- Nosek, B. A., Hardwicke, T. E., Moshontz, H., Allard, A., Corker, K. S., Dreber, A., Fidler, F., Hilgard, J., Kline Struhl, M., Nuijten, M. B., Rohrer, J. M., Romero, F., Scheel, A. M., Scherer, L. D., Schönbrodt, F. D., & Vazire, S.
- Affiliations
- Center for Open Science (US)
- University of Amsterdam (NL)
- University of Wisconsin–Madison (US)
- University of California, Davis (US)
- Grand Valley State University (US)
- Stockholm School of Economics (SE)
- The University of Melbourne (AU)
- Illinois State University (US)
- Center for Open Science, Charlottesville, Virginia 22903, USA
- Tilburg University (NL)
- Leipzig University (DE)
- University of Groningen (NL)
- Eindhoven University of Technology (NL)
- University of Colorado Anschutz (US)
- Ludwig-Maximilians-Universität München (DE)
- ORCID
- Venue
- Annual Review of Psychology
- Publisher
- Annual Reviews
- Published
- 2022-01-04
- Volume
- 73
- Issue
- 1
- Pages
- 719-748
- ISSN
- 0066-4308 (ISSN-L)
- 1545-2085
- OpenAlex
- W3161588210
- Cited by
- 770
- References
- 253
- Citations by year
- 2026: 109 · 2025: 192 · 2024: 180 · 2023: 177 · 2022: 92
- FWCI
- 19.02
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Annual Review of Psychology (publishedVersion)
- PubMed (publishedVersion)
- Research portal (Tilburg University) (submittedVersion, cc-by)
- Data Archiving and Networked Services (DANS) (submittedVersion, cc-by)
- HAL (submittedVersion)
- Minerva Access (University of Melbourne) (submittedVersion)
- University of Groningen research database (University of Groningen / Centre for Information Technology) (submittedVersion, other-oa)
- OSF Preprints (OSF Preprints) (submittedVersion)
- PsyArXiv (OSF Preprints) (submittedVersion)
- PsyArXiv (OSF Preprints) (submittedVersion, cc-by)
- Funders
- John Templeton Foundation
- Patient-Centered Outcomes Research Institute
- Arnold Ventures
- European Commission (HORIZON2020, 841188)
- University of Melbourne
- Nederlandse Organisatie voor Wetenschappelijk Onderzoek
- Templeton World Charity Foundation
- Templeton Religion Trust
- National Institutes of Health
- Agency for Healthcare Research and Quality
- Australian Research Council (FT150100297)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › General Psychology › Academic and Historical Perspectives in Psychology
- Keywords
- replication, replicability, reproducibility, robustness, research practices, replication study, research incentives
- MeSH terms
- Humans, Research Design, Reproducibility of Results
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
