The DOI matches its Crossref record; 256 of the 256 references it cites were found in a registry or an index.
Replicability, Robustness, and Reproducibility in Psychological Science
Open accessPublished version of a preprint
Abstract
Replication—an important, uncommon, and misunderstood practice—is gaining appreciation in psychology. Achieving replicability is important for making research progress. If findings are not replicable, then prediction and theory development are stifled. If findings are replicable, then interrogation of their meaning and validity can advance knowledge. Assessing replicability can be productive for generating and testing hypotheses by actively confronting current understandings to identify weaknesses and spur innovation. For psychology, the 2010s might be characterized as a decade of active confrontation. Systematic and multi-site replication projects assessed current understandings and observed surprising failures to replicate many published findings. Replication efforts highlighted sociocultural challenges such as disincentives to conduct replications and a tendency to frame replication as a personal attack rather than a healthy scientific practice, and they raised awareness that replication contributes to self-correction. Nevertheless, innovation in doing and understanding replication and its cousins, reproducibility and robustness, has positioned psychology to improve research practices and accelerate progress.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“By using a common approach to analyze data from multiple informants, time points, and developmental domains, we can address concerns about the robustness of scientific findings (20,21).”
From A Comprehensive Multilevel Analysis of the Bucharest Early Intervention Project: Causal Effects on Recovery From Early Severe Deprivation (2023) · cites this paper as [21]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This means that even when updates address and mitigate detected biases, they also introduce the potential for “process reproducibility failure” (98) in the generated data and impede reproducibility, critical for scientific rigor (15, 100, 101).”
From Perils and opportunities in using large language models in psychological research (2024) · cites this paper as [98]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In psychology, across 307 systematic replications and multisite replications, 64% reported statistically significant evidence in the same direction and effect sizes 68% as large as the original experiments”
From Investigating the replicability of preclinical cancer biology (2021) · cites this paper as Nosek et al., 2021
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Research into replicability in other disciplines has also found that findings with stronger initial evidence (such as larger effect sizes and/or smaller p-values) is more likely to replicate”
From Investigating the replicability of preclinical cancer biology (2021) · cites this paper as Nosek et al., 2021
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Independent verification is central to the scientific method (Nosek et al., 2022).”
From Read the Paper, Write the Code: Agentic Reproduction of Social-Science Results (2026) · cites this paper as Nosek et al., 2022
The cited source says what the claim attributes to it.
In the citing paper
“A substantial proportion of replications in recent largescale efforts have failed to support the original finding (Nosek et al., 2022).”
From Many Labs 4: Failure to Replicate Mortality Salience Effect With and Without Original Author Involvement (2022) · cites this paper as Nosek et al., 2022
In the citing paper
“However, particular challenges arise when applying these principles to LLMs, particularly proprietary ones, whose black-box nature impedes the reproducibility of findings that pertain to them.”
From Perils and opportunities in using large language models in psychological research (2024) · cites this paper as [98]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Valente, T. W. (2003) · Genetics in Medicine · doi:10.1097/01.gim.0000061743.67794.c4 · Cited by 20,559
The FAIR Guiding Principles for scientific data management and stewardship
Wilkinson, M. D., Dumontier, M., Aalbersberg, I. J., Appleton, G., Axton, M., Baak, A., Blomberg, N., Boiten, J.-W., da Silva Santos, L. B., Bourne, P. E., et al. (2016) · Scientific Data · doi:10.1038/sdata.2016.18 · Cited by 18,423
(2002) · Cited by 13,211
Construct validity in psychological tests.
Cronbach, L. J., & Meehl, P. E. (1955) · Psychological Bulletin · doi:10.1037/h0040957 · Cited by 11,195
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068
The case for motivated reasoning.
Kunda, Z. (1990) · Psychological Bulletin · doi:10.1037/0033-2909.108.3.480 · Cited by 9,022
Cohen, J. (1992) · Psychological Bulletin · doi:10.1037//0033-2909.112.1.155 · Cited by 8,771
Estimating the reproducibility of psychological science
Open Science Collaboration (2015) · Science · doi:10.1126/science.aac4716 · Cited by 8,515
Power failure: why small sample size undermines the reliability of neuroscience
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3475 · Cited by 7,641
Self-citation
Research works printed but not linked: 17
- Artner R, 2020, Psychol. Methods.
- Cent. Open Sci, 2020, Center for Open Science
- Cohen J., 1973, Am. Educ. Res. J., 10, 225
- 10.1037/0033-2909.112.1.155 · DOI · Paper page
- Crisp RJ, 2014, Soc. Psychol., 45, 303
- Epskamp S, 2018, Statistical Software
- Errington TM, 2021, eLife
- Ferguson MJ, 2014, Soc. Psychol., 45, 301
- Gelman A, 2013, The garden of forking paths: why multiple comparisons can be a problem, even when there is no “fishing expedition” or “p-hacking” and the research hypothesis was posited ahead of time
- Marcus A, 2020, WIRED
- Meyer MN, 2014, Slate Magazine
- Mischel W., 2008, APS Observer
- Nosek B., 2019, Center for Open Science
- Nosek B., 2019, Center for Open Science
- Schwarz N, 2014, Soc. Psychol., 45, 305
- Sterling TD., 1959, J. Am. Stat. Assoc., 54, 30
- Yong E, 2012, National Geo-graphic
Cited by
Sorted by citations
Items 301–400 of 772
Introduction to structural causal models in science studies
Klebel, T., & Traag, V. (2025) · Quantitative Science Studies · doi:10.1162/qss.a.405 · Cited by 4
Hamilton, D. G., McKenzie, J. E., Nguyen, P., Rethlefsen, M. L., McDonald, S. J., Brennan, S. E., Fidler, F., Higgins, J. P. T., Kanukula, R., Karunananthan, S., et al. (2026) · Research Synthesis Methods · doi:10.1017/rsm.2025.10064 · Cited by 5
Self-citationInterpretations of reproducibility crisis in medical education research: a qualitative study
Ahmady, S., Kohan, N., Hamidi, H., Vahednasiri, A., & Hooshmandja, M. (2026) · Scientific Reports · doi:10.1038/s41598-025-34640-w · Cited by 4
Responsible assessment of what research? Beware of epistemic diversity!
Ulpts, S. (2024) · Meta-Psychology · doi:10.15626/mp.2023.3797 · Cited by 4
Wong, C. L., & Feldman, G. (2025) · Royal Society Open Science · doi:10.1098/rsos.240687 · Cited by 5
Baruch, Y., Guttormsen, D. S. A., Gyoshev, S. B., Pavkov, T., & Plesca, M. (2025) · Human Resource Management Journal · doi:10.1111/1748-8583.12602 · Cited by 4
Berg, M., Suchotzki, K., Zimmermann, J., Merz, C. J., Szota, K., Brandt, H., Hartwigsen, G., Lincoln, T. M., Mokros, A., Gade, M., et al. (2025) · European Psychologist · doi:10.1027/1016-9040/a000555 · Cited by 4
A Registered Report on Selective Reporting in Single-Case Experimental Research
Travers, J. C., Tincani, M., Dowdy, A., Forbes, H. J., & Johnson, J. V. (2025) · Remedial and Special Education · doi:10.1177/07419325251350697 · Cited by 5
Skouloudis, A., Botetzagias, I., Malesios, C., & Koutroumpinis, P. (2025) · Smart Cities · doi:10.3390/smartcities8050177 · Cited by 4
Direct Replication in Psychological Assessment Research
Allen, M. S., Iliescu, D., & Greiff, S. (2023) · European Journal of Psychological Assessment · doi:10.1027/1015-5759/a000755 · Cited by 4
Can a replication revolution resolve the duplication crisis in systematic reviews?
Karunananthan, S., Grimshaw, J., Maxwell, L. J., Nguyen, P., Page, M. J., Pardo, J. P., Petkovic, J., Vachon, B., Welch, V. A., & Tugwell, P. (2023) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2022-112125 · Cited by 4
Time after time: Factors influencing children’s comprehension of Before and After
Wagner, L., & Holt, R. F. (2023) · Journal of Child Language · doi:10.1017/s0305000923000612 · Cited by 4
Ethnography vs. zombie methodologies
Downey, G. (2023) · American Ethnologist · doi:10.1111/amet.13235 · Cited by 5
Namuth, A., Brown, M., Macchione, A. L., & Sacco, D. F. (2023) · ETHICS IN PROGRESS · doi:10.14746/eip.2023.2.4 · Cited by 4
Perreault, G., & Dienlin, T. (2024) · Journalism & Mass Communication Quarterly · doi:10.1177/10776990241262346 · Cited by 4
When “replicability” is more than just “reliability”: The Hubble constant controversy
Matarese, V., & McCoy, C. D. (2024) · Studies in History and Philosophy of Science Part A · doi:10.1016/j.shpsa.2024.07.005 · Cited by 4
Reporting Standards for Bayesian Network Modelling
Barons, M. J., Hanea, A. M., Mascaro, S., & Woodberry, O. (2025) · Entropy · doi:10.3390/e27010069 · Cited by 4
Bortolini, T., Newson, M., Tovar-Moll, F., Latgé-Tovar, S., Whitehouse, H., Moll, J., & Fischer, R. D. (2025) · Evolution and Human Behavior · doi:10.1016/j.evolhumbehav.2025.106666 · Cited by 8
Reproducibility, replicability, robustness, and generalizability in corpus linguistics
Flanagan, J. (2025) · International Journal of Corpus Linguistics · doi:10.1075/ijcl.24113.fla · Cited by 6
Roth, J., Duan, Y., Mahner, F. P., Kaniuth, P., Wallis, T. S. A., & Hebart, M. N. (2025) · Communications Psychology · doi:10.1038/s44271-025-00236-3 · Cited by 4
Reubenson, A., Ng, L., Lawton, V., Nahon, I., Terry, R., Baldwin, C. E., Blackford, J., Bond, A., Corrigan, R., Dalton, M., et al. (2025) · PLoS ONE · doi:10.1371/journal.pone.0321397 · Cited by 5
Riesthuis, P., Rassin, E., Bücken, C. A., Booker, A., Chin, J. M., Goldfarb, D., Deferme, D., & Otgaar, H. (2025) · The International Journal of Evidence & Proof · doi:10.1177/13657127251357630 · Cited by 4
The Computational Evolution of Cognitive Architectures
Kotseruba, I., & Tsotsos, J. K. (2025) · doi:10.1093/oso/9780192844835.001.0001 · Cited by 4
Leslie-Evans, M., Bartschi, J. G., & Allen, M. S. (2022) · Personality and Individual Differences · doi:10.1016/j.paid.2022.111753 · Cited by 4
Transparency and reproducibility in the journal of contextual behavioral science: An audit study
Lear, M. K., Spata, A., Tittler, M. V., Fishbein, J. N., Arch, J. J., & Luoma, J. B. (2023) · Journal of Contextual Behavioral Science · doi:10.1016/j.jcbs.2023.03.017 · Cited by 5
Evading open science: The black box of student data collection
Ludwig, T., Altenmüller, M. S., Schramm, L. F. F., & Twardawski, M. (2023) · Social Psychological Bulletin · doi:10.32872/spb.9411 · Cited by 4
Nozaki, Y., & Kobayashi, R. (2023) · Journal of Cross-Cultural Psychology · doi:10.1177/00220221231212176 · Cited by 4
Quantifying replicability of multiple studies in a meta-analysis
Xiao, M., Chu, H., Hodges, J. S., & Lin, L. (2024) · The Annals of Applied Statistics · doi:10.1214/23-aoas1806 · Cited by 4
Let’s Give Together: Can Collaborative Giving Boost Generosity?
Proulx, J. D. E., Aknin, L. B., & Barasch, A. (2022) · Nonprofit and Voluntary Sector Quarterly · doi:10.1177/08997640221074699 · Cited by 10
Continuity and credibility in the Cognitive Science of Religion
Ross, R. M., & McKay, R. (2021) · Religion Brain & Behavior · doi:10.1080/2153599x.2021.1987308 · Cited by 4
Synergy Between the Credibility Revolution and Human Development in AfricaPreprint
Adetula, A., Forscher, P. S., Basnight-Brown, D. M., Azouaghe, S., Ouherrou, N., Charyate, A. C. A., Hansen, N., Adetula, G. A., & IJzerman, H. (2021) · doi:10.31730/osf.io/e57bq · Cited by 4
The Hypository: Registering Hypotheses for Cumulative SciencePreprint
Moeller, J., Langener, A. M., Lafit, G., Karhulahti, V., Bastiaansen, J. A., & Bergmann, C. (2022) · doi:10.31234/osf.io/5qgj7 · Cited by 4
Using preregistration as a tool for transparent fNIRS study designPreprint
Schroeder, P. A., Artemenko, C., Kosie, J. E., Cockx, H., Stute, K., Pereira, J., Klein, F., & Mehler, D. M. A. (2022) · doi:10.31222/osf.io/zfawx · Cited by 4
How to Open Science: Debugging Reproducibility within the Educational Data Mining ConferencePreprint
Haim, A., Shaw, S. T., & Heffernan, N. T. (2023) · doi:10.35542/osf.io/4hn8q · Cited by 4
Diagnosis and therapy of psychology by means of systematicity theoryPreprint
Witte, E. H. (2023) · doi:10.31219/osf.io/dy4kn · Cited by 4
Introduction to causality in science studiesPreprint
Klebel, T., & Traag, V. (2024) · doi:10.31235/osf.io/4bw9e · Cited by 4
Hardwicke, T. E., Thibault, R. T., Clarke, B., Moodie, N., Crüwell, S., Schiavone, S. R., Handcock, S. A., Nghiem, K. A., Mody, F., Eerola, T., et al. (2024) · doi:10.31234/osf.io/t2zs9 · Cited by 4
Self-citationInvesting in Data Quality for High-Impact Entrepreneurship Research
MAULA, M. V. J., Mickiewicz, T. M., Silvio Vismara, A., & Wiklund, J. (2026) · Entrepreneurship Theory and Practice · doi:10.1177/10422587261435916 · Cited by 4
Research into evidence-based psychological interventions needs a stronger focus on replicability
Niemeyer, H., Knaevelsrud, C., van Aert, R. C. M., & Ehring, T. (2023) · Clinical Psychology in Europe · doi:10.32872/cpe.9997 · Cited by 3
Reproducible brain PET data analysis: easier said than done
Naseri, M., Ramakrishnapillai, S., & Carmichael, O. T. (2024) · Frontiers in Neuroinformatics · doi:10.3389/fninf.2024.1420315 · Cited by 4
Li, H. (2025) · Current Psychology · doi:10.1007/s12144-025-07492-w · Cited by 3
Hoffmann, J., Twardawski, M., Höhs, J. M., Gast, A., Pohl, S., & Sengewald, M.-A. D.-P. (2025) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459251328273 · Cited by 5
Kałowski, P., Obarska, M., Romaneczko, J., Walczak, D., & Banasik‐Jemielniak, N. (2025) · International Journal of Bilingualism · doi:10.1177/13670069251358991 · Cited by 3
Predicting replication outcomes and generalizability of results
Dreber, A. (2025) · European Review of Agricultural Economics · doi:10.1093/erae/jbaf035 · Cited by 3
Self-citationBig Five personality traits and voting: A systematic review, meta-analysis, and mega-analysis
Stahlmann, A. G., Hopwood, C. J., Orth, U., Smillie, L. D., & Bleidorn, W. (2025) · European Journal of Personality · doi:10.1177/08902070251383955 · Cited by 3
Delios, A., Hu, T., Yu, S., Zhou, N., Ahsan, F. M., Bahl, M., Bai, T., Basu, M., Bathula, H., Batsakis, G. K., et al. (2025) · Journal of International Business Studies · doi:10.1057/s41267-025-00808-9 · Cited by 3
The emerging submission crisis in behavioral science
Spitzer, M. (2026) · Trends in Neuroscience and Education · doi:10.1016/j.tine.2026.100276 · Cited by 5
Chandrashekar, S. P., Viganola, D., Dreber, A., Johannesson, M., Pfeiffer, T., Siegel, A., & Feldman, G. (2026) · Royal Society Open Science · doi:10.1098/rsos.250377 · Cited by 3
Self-citationPsychophysiological recovery from viewing nature and urban settings: A multisite replication
den Berg, A. E. V., Dijkstra, K., Meuwese, D., Beute, F., Darcy, P. M., Dewitte, S., Gatersleben, B., Gidlow, C. J., Hagerhall, C. M. P., Hipp, J. A., et al. (2026) · Journal of Environmental Psychology · doi:10.1016/j.jenvp.2026.102956 · Cited by 3
Porreca, A., de Carli, P., Filippi, B., Bakermans-Kranenburg, M., van IJzendoorn, M. H., & Simonelli, A. (2024) · Development and Psychopathology · doi:10.1017/s0954579424000026 · Cited by 2
Costigan, S., Ruscio, J., & Crawford, J. T. (2024) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459241227865 · Cited by 3
Neural Diversity and Decisions
Thoelen, G., & Zak, P. J. (2024) · Adaptive Human Behavior and Physiology · doi:10.1007/s40750-024-00237-2 · Cited by 3
Evans, T. C., Carlson, J. S., Zuberer, A., Fry, R., Agnoli, S., Britton, J. C., Degutis, J. M., & Esterman, M. S. (2024) · Motivation and Emotion · doi:10.1007/s11031-024-10076-z · Cited by 3
Dudda, L. A., Kozula, M., Ross‐Hellauer, T., Kormann, E., Spijker, R., DeVito, N., Gopalakrishna, G., den Eynden, V. V., Onghena, P., Naudet, F., et al. (2024) · Open Research Europe · doi:10.12688/openreseurope.16567.2 · Cited by 4
Open science needs a standardized data format: Suggestions for the field of psychoneuroendocrinology
Meier, M., Vinkers, C. H., Pruessner, J. C., & Sep, M. S. C. (2024) · Psychoneuroendocrinology · doi:10.1016/j.psyneuen.2024.107170 · Cited by 3
Lam, M. I., Bai, W., Feng, Y., Zhang, Q., Zhang, Y., Jackson, T., Rao, S.-Y., Ho, T.-I., Su, Z., Cheung, T. C. C., et al. (2024) · Journal of Psychosomatic Research · doi:10.1016/j.jpsychores.2024.111910 · Cited by 3
Kaufman, M. J., Hudson, J. I., Kanayama, G., Muse, S., Schnabel, J., Sokoll, R., & Pope, H. G. (2024) · The American Journal of Drug and Alcohol Abuse · doi:10.1080/00952990.2024.2403582 · Cited by 3
Chan, C. W., & Feldman, G. (2024) · Cognition & Emotion · doi:10.1080/02699931.2024.2434156 · Cited by 3
Servitization and firm performance: a p-curve analysis
Latan, H., Jabbour, A. B. L. D. S., Jabbour, C. J. C., Ali, M., & Filho, M. G. (2024) · Journal of Business and Industrial Marketing · doi:10.1108/jbim-07-2023-0392 · Cited by 3
Replication: A Critique and Overview for Kinesiology
Martin, J. J., & Yukhymenko–Lescroart, M. A. (2025) · Kinesiology Review · doi:10.1123/kr.2024-0057 · Cited by 3
Ethical Considerations in Designing and Testing Robots for Children in Educational Settings
Arriaga, P., Neto, I., Correia, F., & Soares, A. (2025) · Studies in computational intelligence · doi:10.1007/978-3-031-82915-4_8 · Cited by 5
Hitsuwari, J., & Nomura, M. (2025) · Perceptual and Motor Skills · doi:10.1177/00315125251330926 · Cited by 4
Emotional Intelligence and Business Ethics: Feeling Confident in Doing the Right Thing
Carver, S. J., & Woodley, H. J. R. (2025) · Journal of Business Ethics · doi:10.1007/s10551-025-06044-3 · Cited by 5
Measurement science meets the reproducibility challenge
Possolo, A. M. (2023) · Metrologia · doi:10.1088/1681-7575/acdef7 · Cited by 3
Programming an n-Back task in Qualtrics using HTML and JavaScript
Bradley-Garcia, M., & Bolton, V. (2023) · The Quantitative Methods for Psychology · doi:10.20982/tqmp.19.2.p136 · Cited by 3
Accelerating addiction research via Open Science and Team Science
Dora, J., Piccirillo, M. L., Foster, K. T., & King, K. M. (2023) · The Psychology of learning and motivation/The psychology of learning and motivation · doi:10.1016/bs.plm.2023.06.004 · Cited by 3
Dudda, L. A., Kozula, M., Ross‐Hellauer, T., Kormann, E., Spijker, R., DeVito, N., Gopalakrishna, G., den Eynden, V. V., Onghena, P., Naudet, F., et al. (2023) · Open Research Europe · doi:10.12688/openreseurope.16567.1 · Cited by 3
Growth From Uncertainty: Understanding the Replication ‘Crisis’ in Infant Cognition
Lavelle, J. S. (2023) · Philosophy of Science · doi:10.1017/psa.2023.157 · Cited by 3
Cognitive Metascience: A New Approach to the Study of Theories
Miłkowski, M. (2023) · Przegląd Psychologiczny · doi:10.31648/przegldpsychologiczny.9682 · Cited by 3
Effects of Preferred versus Nonpreferred Music on Bench Press Performance
Hutchinson, J., Murphy, J., Lucia, B. D., O’Neill, E., Curtis, D., Mellano, K. T., Pelton, L. M., & Coker, N. A. (2024) · Communications in Kinesiology · doi:10.51224/cik.2024.60 · Cited by 3
Sacco, D. F., Namuth, A., Macchione, A. L., & Brown, M. (2024) · Journal of Academic Ethics · doi:10.1007/s10805-024-09505-y · Cited by 3
Testing the efficacy of the cognitive interview to road traffic accident investigations
Maulina, D., Irwanda, D. Y., Guritnaningsih, Otgaar, H., Nurfajriah, D. S., Armas, L. O. M. A., & Dewi, A. I. (2024) · Applied Cognitive Psychology · doi:10.1002/acp.4177 · Cited by 3
Denissen, J. J. A. (2024) · Personality Science · doi:10.1177/27000710241237944 · Cited by 5
Enriquez, D. (2024) · Frontiers in Sociology · doi:10.3389/fsoc.2024.1157514 · Cited by 3
Systemic Contextualisation: Guiding Assumptions and Framing
Cooksey, R. (2024) · Springer eBooks · doi:10.1007/978-981-97-3462-7_4 · Cited by 3
Methodological issues in measuring psychosocial safety climate: a systematic review protocol
Amoadu, M., Obeng, P., Cobbinah, G., Salu, P. K., Ofori, G. O., & Ansah, E. W. (2024) · BMJ Open · doi:10.1136/bmjopen-2024-087315 · Cited by 3
Grant, S., & Khatua, S. (2024) · Journal of Development Effectiveness · doi:10.1080/19439342.2024.2388102 · Cited by 4
Two replications of Wiechert et al.’s (2023) online Think/No-Think study in undergraduate students
Wessel, I., Lehmann, R., & Wiechert, S. (2024) · Memory · doi:10.1080/09658211.2024.2412025 · Cited by 3
How to do research in classroom settings
Goudeau, S., Easterbrook, M. J., & Fayant, M. (2024) · Nature Human Behaviour · doi:10.1038/s41562-024-02027-y · Cited by 3
Is false belief understanding stable from infancy to childhood? We don’t know yet
Poulin‐Dubois, D., & Goldman, E. J. (2023) · Cognitive Development · doi:10.1016/j.cogdev.2023.101309 · Cited by 3
Response to Commentaries on Sakaluk (2020)
Sakaluk, J. K. (2021) · Archives of Sexual Behavior · doi:10.1007/s10508-021-02020-w · Cited by 2
Psychosomatic Medicine: Looking Forward
Segerstrom, S. C. (2022) · Psychosomatic Medicine · doi:10.1097/psy.0000000000001067 · Cited by 3
Bollmann, S., Renton, A., Dao, T. T., Johnstone, T., Civier, O., Sullivan, R., White, D., Lyons, P., Slade, B., Abbott, D. F., et al. (2023) · Research Square · doi:10.21203/rs.3.rs-2649734/v1 · Cited by 3
The DECIDE Framework: Describing Ethical Choices in Digital-Behavioral-Data ExplorationsPreprint
Shaw, H., Brown, O., Hinds, J. M., Nightingale, S. J., Towse, J. N., & Ellis, D. A. (2023) · doi:10.31234/osf.io/c8ert · Cited by 3
(FULL GUIDE) A Guide for Social Science Journal Editors on Easing into Open SciencePreprint
Silverstein, P., Elman, C., Montoya, A. K., McGillivray, B. C., Pennington, C. R., Harrison, C. H., Steltenpohl, C. N., Röer, J. P., Corker, K. S., Charron, L., et al. (2023) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/hstcx · Cited by 3
Self-citationSchroeders, U., Morgenstern, M., Jankowsky, K., & Gnambs, T. (2023) · doi:10.31234/osf.io/nw8k7 · Cited by 3
Investigating Issues with Reproducibility in Journal Articles Published in Psychological SciencePreprint
Homewood, C. (2023) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/6d5nq · Cited by 3
Dannlowski, U., Winter, N. R., Meinert, S. L., Grotegerd, D., Kraus, A., Flinkenflügel, K., Leehr, E. J., Böhnlein, J., Borgers, T., Fisch, L., et al. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4854882 · Cited by 3
AITurk: Using ChatGPT for Social Science ResearchPreprint
Qin, X., Huang, M., & Ding, J. (2024) · doi:10.31234/osf.io/xkd23 · Cited by 4
Cumming, G. (2025) · Springer eBooks · doi:10.1007/978-3-662-69359-9_704 · Cited by 3
Online Platforms Provide Access to Valid Eastern SamplesPreprint
Wang, L., Gao, R., Jung, M. H., Hung, I. W., & Nelson, L. D. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5488129 · Cited by 2
Lecuona, O. Z., Martínez-Barajas, O., Gimeno-Martín, A., Martín, A. H., Carrillo-Molina, C., Cantero, R. A., Rodríguez‐Carvajal, R., & de Rivas, S. (2024) · Journal of Homosexuality · doi:10.1080/00918369.2024.2364891 · Cited by 2
Developing Surveys on Questionable Research Practices: Four Challenging Design Problems
Berggren, C., Gerdin, B., & Karabağ, S. F. (2024) · Journal of Academic Ethics · doi:10.1007/s10805-024-09565-0 · Cited by 4
Naming Ability in the Chronic Phase of Moderate–Severe Traumatic Brain Injury
McCurdy, R. A., Covington, N. V., & Duff, M. C. (2024) · American Journal of Speech-Language Pathology · doi:10.1044/2024_ajslp-23-00249 · Cited by 3
Curated Bibliographic Data: The Case of the Estonian National Bibliography
Kruusmaa, K., Tinits, P., & Nemvalts, L. (2025) · Journal of Open Humanities Data · doi:10.5334/johd.280 · Cited by 2
Multi-voxel pattern analysis for developmental cognitive neuroscientists
Moreira, J. F. G., & Silvers, J. A. (2025) · Developmental Cognitive Neuroscience · doi:10.1016/j.dcn.2025.101555 · Cited by 2
Prevalence of Registered Reports in experimental psychology journals: A bibliometric study
Gorman, D. M., & Hubbard, D. E. (2025) · Quantitative Science Studies · doi:10.1162/qss_a_00364 · Cited by 2
Spencer, S. D., Borgogna, N. C., Owen, T., Rast, C. E., Johnson, D. A., Cheng, J. S.-C., Castillo, A. A., Storch, E. A., Cervin, M., & Hollon, S. D. (2025) · Personalized Medicine in Psychiatry · doi:10.1016/j.pmip.2025.100154 · Cited by 2
Ohana, M. (2025) · Journal of Business Research · doi:10.1016/j.jbusres.2025.115556 · Cited by 6
Digital Interventions for Anxiety and Depressive Symptoms in Adolescence: Systematic Review
Kuberka, P., Johnston, M., Shafran, R., Pike, K. C., & Yardley, L. (2025) · Journal of Adolescent Health · doi:10.1016/j.jadohealth.2025.05.021 · Cited by 5
Metadata
- Authors
- Nosek, B. A., Hardwicke, T. E., Moshontz, H., Allard, A., Corker, K. S., Dreber, A., Fidler, F., Hilgard, J., Kline Struhl, M., Nuijten, M. B., Rohrer, J. M., Romero, F., Scheel, A. M., Scherer, L. D., Schönbrodt, F. D., & Vazire, S.
- Affiliations
- Center for Open Science (US)
- University of Amsterdam (NL)
- University of Wisconsin–Madison (US)
- University of California, Davis (US)
- Grand Valley State University (US)
- Stockholm School of Economics (SE)
- The University of Melbourne (AU)
- Illinois State University (US)
- Center for Open Science, Charlottesville, Virginia 22903, USA
- Tilburg University (NL)
- Leipzig University (DE)
- University of Groningen (NL)
- Eindhoven University of Technology (NL)
- University of Colorado Anschutz (US)
- Ludwig-Maximilians-Universität München (DE)
- ORCID
- Venue
- Annual Review of Psychology
- Publisher
- Annual Reviews
- Published
- 2022-01-04
- Volume
- 73
- Issue
- 1
- Pages
- 719-748
- ISSN
- 0066-4308 (ISSN-L)
- 1545-2085
- OpenAlex
- W3161588210
- Cited by
- 770
- References
- 253
- Citations by year
- 2026: 109 · 2025: 192 · 2024: 180 · 2023: 177 · 2022: 92
- FWCI
- 19.02
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Annual Review of Psychology (publishedVersion)
- PubMed (publishedVersion)
- Research portal (Tilburg University) (submittedVersion, cc-by)
- Data Archiving and Networked Services (DANS) (submittedVersion, cc-by)
- HAL (submittedVersion)
- Minerva Access (University of Melbourne) (submittedVersion)
- University of Groningen research database (University of Groningen / Centre for Information Technology) (submittedVersion, other-oa)
- OSF Preprints (OSF Preprints) (submittedVersion)
- PsyArXiv (OSF Preprints) (submittedVersion)
- PsyArXiv (OSF Preprints) (submittedVersion, cc-by)
- Funders
- John Templeton Foundation
- Patient-Centered Outcomes Research Institute
- Arnold Ventures
- European Commission (HORIZON2020, 841188)
- University of Melbourne
- Nederlandse Organisatie voor Wetenschappelijk Onderzoek
- Templeton World Charity Foundation
- Templeton Religion Trust
- National Institutes of Health
- Agency for Healthcare Research and Quality
- Australian Research Council (FT150100297)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › General Psychology › Academic and Historical Perspectives in Psychology
- Keywords
- replication, replicability, reproducibility, robustness, research practices, replication study, research incentives
- MeSH terms
- Humans, Research Design, Reproducibility of Results
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
