The DOI matches its Crossref record; 208 of the 208 references it cites were found in a registry or an index.
The replication crisis has led to positive structural, procedural, and community changes
Open accessAbstract
The emergence of large-scale replication projects yielding successful rates substantially lower than expected caused the behavioural, cognitive, and social sciences to experience a so-called ‘replication crisis’. In this Perspective, we reframe this ‘crisis’ through the lens of a credibility revolution, focusing on positive structural, procedural and community-driven changes. Second, we outline a path to expand ongoing advances and improvements. The credibility revolution has been an impetus to several substantive changes which will have a positive, long-term impact on our research environment.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Figures

Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The weirdest people in the world?
Henrich, J., Heine, S. J., & Norenzayan, A. (2010) · Behavioral and Brain Sciences · doi:10.1017/s0140525x0999152x · Cited by 12,641
The PRISMA 2020 Statement: An Updated Guideline for Reporting Systematic Reviews
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., et al. (2021) · International Journal of Surgery · doi:10.1016/j.ijsu.2021.105906 · Cited by 11,748
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2021) · doi:10.1002/9781119558378 · Cited by 9,799
Estimating the reproducibility of psychological science
Open Science Collaboration (2015) · Science · doi:10.1126/science.aac4716 · Cited by 8,515
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
A manifesto for reproducible science
Munafò, M. R., Nosek, B. A., Bishop, D. V. M., Button, K. S., Chambers, C. D., Percie du Sert, N., Simonsohn, U., Wagenmakers, E.-J., Ware, J. J., & Ioannidis, J. P. A. (2017) · Nature Human Behaviour · doi:10.1038/s41562-016-0021 · Cited by 3,397 · cited 1×
“After several notable scandals and controversies in 2011 [2–4], skepticism regarding claims in psychological science increased, and inspired the development of projects examining the replicability and reproducibility of past findings [5].”
BackgroundRedefine statistical significance
Benjamin, D. J., Berger, J. O., Johannesson, M., Nosek, B. A., Wagenmakers, E.-J., Berk, R., Bollen, K. A., Brembs, B., Brown, L., Camerer, C., et al. (2017) · Nature Human Behaviour · doi:10.1038/s41562-017-0189-z · Cited by 2,772
Conditions for intuitive expertise: A failure to disagree.
Kahneman, D., & Klein, G. (2009) · American Psychologist · doi:10.1037/a0016755 · Cited by 2,443 · cited 1×
“…collaborations ’ of this kind are arguably one of the most important developments in procedures to advance research because they allow for a consensus-based resolution of scienti fi c debates and they facilitate more ef fi cient knowledge production and self-correction by reducing bias 3,167 .”
BackgroundMeasuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling
John, L. K., Loewenstein, G., & Prelec, D. (2012) · Psychological Science · doi:10.1177/0956797611430953 · Cited by 2,239 · cited 1×
“Researchers continue to employ a variety of QRPs, which are signi fi cant contributors to low replicability 202,204,207 – 209 , and a lack of transparency in reporting can mask and exacerbate these issues 14 .”
BackgroundHARKing: Hypothesizing after the results are known
Kerr, N. L. (1998) · Personality and Social Psychology Review · doi:10.1207/s15327957pspr0203_4 · Cited by 1,989 · cited 1×
“Researchers continue to employ a variety of QRPs, which are signi fi cant contributors to low replicability 202,204,207 – 209 , and a lack of transparency in reporting can mask and exacerbate these issues 14 .”
Background
Research works printed but not linked: 10
- Azevedo, F. et al. Introducing a framework for open and reproducible research training (FORRT). Preprint at https://osf.io/bnh7p/ (2019). This paper describes the importance of integrating open scholarship into higher education, its benefits and challenges, as well as about FORRT initiatives aiming to support educators in this endeavor.
- Pownall, M. et al. The impact of open and reproducible scholarship on students’ scientific literacy, engagement, and attitudes towards science: a review and synthesis of the evidence. Roy. Soc. Open Sci., 10, 221255 (2023). This review article describes the available (empirical) evidence of the impact (and importance) of integrating open scholarship into higher education, its benefits and challenges on three specific areas: students’ (a) scientific literacy; (b) engagement with science; and (c) attitudes towards science.
- Elife. https://elifesciences.org/for-the-press/b2329859/elife-ends-accept-reject-decisions-following-peer-review (2023).
- Higgins, J. P. et al. Cochrane Handbook for Systematic Reviews of Interventions (John Wiley & Sons, 2022).
- Humphreys, L., Lewis Jr, N. A., Sender, K. & Won, A. S. Integrating qualitative methods and open science: five principles for more trustworthy research. J. Commun. 71, 855–874 (2021).
- Bergmann, C. How to integrate open science into language acquisition research? Student workshop at BUCLD 43 (2018).
- Candela, A. G. Exploring the function of member checking. Qual. Rep. 24, 619–628 (2019).
- Checchi, D., De Fraja, G. & Verzillo, S. Incentives and careers in academia: theory and empirical analysis. Rev. Econ. Stat. 103, 786–802 (2021).
- Grove, L. The Effects of Funding Policies on Academic Research. Ph.D. thesis, University College London (2017).
- Devezer, B. & Buzbas, E. Minimum viable experiment to replicate (2021). Preprint at http://philsci-archive.pitt.edu/21475/.
Cited by
What we mean when we say semantic: Toward a multidisciplinary semantic glossary
Reilly, J. J., Shain, C., Borghesani, V., Kuhnke, P., Vigliocco, G., Peelle, J. E., Mahon, B. Z., Buxbaum, L. J., Majid, A., Brysbaert, M., et al. (2024) · Psychonomic Bulletin & Review · doi:10.3758/s13423-024-02556-7 · Cited by 123
“As such, limited construct specificity has been identified as one of the leading causes of the ongoing replication crisis in psychology and related fields (Korbmacher et al., 2023; Oberauer & Lewandowsky, 2019).”
König, L. M., Krukowski, R. A., Kuntsche, E. N., Busse, H., Gumbert, L., Gemesi, K., Neter, E., Mohamed, N. F., Ross, K. M., John-Akinola, Y. O., et al. (2023) · International Journal for Equity in Health · doi:10.1186/s12939-023-02055-6 · Cited by 62
“Since many journals require reporting checklists to be included upon submission, and preregistration of studies is becoming more common (even for study designs other than clinical trials), it is likely that these structural changes will lead to a relatively quick change in research practices [14].”
BackgroundSelf-citationA manifesto for a globally diverse, equitable, and inclusive open science
Ghai, S., Thériault, R., Forscher, P., Shoda, Y., Syed, M., Puthillam, A., Peng, H. C., Basnight-Brown, D., Majid, A., Azevedo, F., et al. (2025) · Communications Psychology · doi:10.1038/s44271-024-00179-1 · Cited by 32
“Increasingly, journals and funding agencies have encouraged and sought to normalise these research practices 12 , introducing a fundamental cultural shift in how psychological scientists conduct research.”
Self-citationHillman, C., Fontana, B. D., Amstislavskaya, T. G., Gorbunova, M. A., Altenhofen, S., Barthelson, K., Bastos, L. M., Borba, J. V., Bonan, C. D., Brennan, C. H., et al. (2025) · Lab Animal · doi:10.1038/s41684-025-01548-x · Cited by 40
Mayrhofer, R., Büchner, I. C., & Hevesi, J. (2024) · Frontiers in Psychology · doi:10.3389/fpsyg.2024.1390233 · Cited by 34
Metadata
- Authors
- Korbmacher, M., Azevedo, F., Pennington, C. R., Hartmann, H., Pownall, M., Schmidt, K., Elsherif, M., Breznau, N., Robertson, O., Kalandadze, T., Yu, S., Baker, B. J., O’Mahony, A., Olsnes, J. Ø. S., Shaw, J. J., Gjoneska, B., Yamada, Y., Röer, J. P., Murphy, J., Alzahawi, S., Grinschgl, S., Oliveira, C. M., Wingen, T., Yeung, S. K., Liu, M., König, L. M., Albayrak-Aydemir, N., Lecuona, O., Micheli, L., & Evans, T.
- Affiliations
- Oslo University Hospital (NO)
- Aston University (GB)
- Essen University Hospital (DE)
- University of Leeds (GB)
- Ashland University (US)
- University of Birmingham (GB)
- University of Bremen (DE)
- University of Oxford (GB)
- Østfold University of Applied Sciences (NO)
- Temple University (US)
- Cardiff University (GB)
- Norwegian University of Science and Technology (NO)
- De Montfort University (GB)
- Macedonian Academy of Sciences and Arts (MK)
- Kyushu University (JP)
- Witten/Herdecke University (DE)
- Technological University Dublin (IE)
- Stanford University (US)
- University of Graz (AT)
- University of York (GB)
- University of Bonn (DE)
- Chinese University of Hong Kong (HK)
- University of Cambridge (GB)
- University of Bayreuth (DE)
- Universidad Rey Juan Carlos (ES)
- University of Greenwich (GB)
- ORCID
- Korbmacher, Max
- Azevedo, Flavio
- Pennington, Charlotte R.
- Hartmann, Helena
- Pownall, Madeleine
- Schmidt, Kathleen
- Elsherif, Mahmoud
- Breznau, Nate
- Robertson, Olly
- Kalandadze, Tamara
- Yu, Shijun
- Baker, Bradley J.
- O’Mahony, Aoife
- Shaw, John J.
- Gjoneska, Biljana
- Yamada, Yuki
- Röer, Jan P.
- Murphy, Jennifer
- Alzahawi, Shilaan
- Grinschgl, Sandra
- Oliveira, Catia M.
- Wingen, Tobias
- Yeung, Siu Kit
- Liu, Meng
- König, Laura M.
- Albayrak-Aydemir, Nihan
- Lecuona, Oscar
- Micheli, Leticia
- Evans, Thomas
- Venue
- Communications Psychology
- Published
- 2023-07-25
- Volume
- 1
- Issue
- 1
- Pages
- 3-3
- ISSN
- 2731-9121 (ISSN-L)
- PMC
- PMC11290608
- OpenAlex
- W4385240232
- Cited by
- 137
- References
- 204
- Citations by year
- 2026: 36 · 2025: 64 · 2024: 27 · 2023: 10
- FWCI
- 5.80
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Communications Psychology (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Leiden Repository (Leiden University) (publishedVersion, cc-by)
- Edge Hill University Research Information Repository (Edge Hill University) (publishedVersion, cc-by)
- ORCA Online Research @Cardiff (Cardiff University) (acceptedVersion, other-oa)
- Open Research Online (The Open University) (acceptedVersion, other-oa)
- London School of Economics and Political Science Research Online (London School of Economics and Political Science) (acceptedVersion, other-oa)
- University of Birmingham Research Portal (University of Birmingham) (submittedVersion, other-oa)
- Leiden Repository (Leiden University) (submittedVersion, cc-by)
- NORA - Norwegian Open Research Archives (submittedVersion, cc-by)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Keywords
- replication crisis, credibility revolution, replication study, research credibility, research practices
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The replication crisis has led to positive structural, procedural, and community changes. https://app.cochat.ai/papers/10.1038/s44271-023-00003-2
- The paper
- Korbmacher, M., Azevedo, F., Pennington, C. R., Hartmann, H., Pownall, M., Schmidt, K., Elsherif, M., Breznau, N., Robertson, O., Kalandadze, T., Yu, S., Baker, B. J., O’Mahony, A., Olsnes, J. Ø. S., Shaw, J. J., Gjoneska, B., Yamada, Y., Röer, J. P., Murphy, J., . . . Evans, T. (2023). The replication crisis has led to positive structural, procedural, and community changes. Communications Psychology. https://doi.org/10.1038/s44271-023-00003-2
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.