Reproducibility failure in biomedical research: problems and solutions
Open accessAbstract
Reproducibility concerns in biomedical research have persisted for more than a decade, with large-scale assessments revealing significant challenges in replicating findings. Despite widespread acknowledgment of these issues, responses remain inconsistent, and proposed solutions often lack rigorous evaluation. This review examines the factors that contribute to irreproducibility in conducting, reporting, and reviewing research and assesses the effectiveness and desirability of interventions aimed at improving reproducibility. It highlights the need for balanced scientific reforms that strengthen reproducibility without stifling innovation or introducing unintended consequences. A critical appraisal of the role of meta-research is essential to ensure sustainable improvements in research quality.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Numerous reports that decry failures to replicate findings in behavioral and clinical neuroscience have triggered increased emphasis on the importance of replication.
“Numerous reports that decry failures to replicate findings in behavioral and clinical neuroscience have triggered increased emphasis on the importance of replication [1-5].”
From A feature and not a bug: Replication failures can lead to new biology (2026) · cites this paper as [5]
The passage labeled “acknowledgments / literature cited” (page unavailable) says low replication success raised concerns about a crisis and prompted journals, funders, and others to put reproducibility more prominently on their agendas. It discusses biomedical research broadly, however, and does not specifically establish this sequence for behavioral and clinical neuroscience. (LLM, conf=0.84)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The FAIR Guiding Principles for scientific data management and stewardship
Wilkinson, M. D., Dumontier, M., Aalbersberg, I. J., Appleton, G., Axton, M., Baak, A., Blomberg, N., Boiten, J.-W., da Silva Santos, L. B., Bourne, P. E., et al. (2016) · Scientific Data · doi:10.1038/sdata.2016.18 · Cited by 18,423 · see citing works
1,500 scientists lift the lid on reproducibility
Baker, M. (2016) · Nature · doi:10.1038/533452a · Cited by 4,209 · see citing works
Raise standards for preclinical cancer research
Begley, C. G., & Ellis, L. M. (2012) · Nature · doi:10.1038/483531a · Cited by 2,979 · see citing works
HARKing: Hypothesizing after the results are known
Kerr, N. L. (1998) · Personality and Social Psychology Review · doi:10.1207/s15327957pspr0203_4 · Cited by 1,989 · see citing works
The state of OA: a large-scale analysis of the prevalence and impact of Open Access articles
Piwowar, H., Priem, J., Larivière, V., Alperin, J. P., Matthias, L., Norlander, B., Farley, A., West, J., & Haustein, S. (2018) · PeerJ · doi:10.7717/peerj.4375 · Cited by 1,223 · see citing works
Boutron, I. (2010) · JAMA · doi:10.1001/jama.2010.651 · Cited by 842 · see citing works
Limitations of Animal Studies for Predicting Toxicity in Clinical Trials
Van Norman, G. A. (2019) · JACC: Basic to Translational Science · doi:10.1016/j.jacbts.2019.10.008 · Cited by 709 · see citing works
Reviewer bias in single- versus double-blind peer review
Tomkins, A., Zhang, M., & Heavlin, W. D. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1707323114 · Cited by 605 · see citing works
Publication Bias in Editorial Decision Making
Olson, C. M. (2002) · JAMA · doi:10.1001/jama.287.21.2825 · Cited by 395 · see citing works
Compliance with Results Reporting at ClinicalTrials.gov
Anderson, M. L., Chiswell, K., Peterson, E. D., Tasneem, A., Topping, J., & Califf, R. M. (2015) · New England Journal of Medicine · doi:10.1056/nejmsa1409364 · Cited by 380 · see citing works
Gender bias in scholarly peer review
Helmer, M., Schottdorf, M., Neef, A., & Battaglia, D. (2017) · eLife · doi:10.7554/elife.21718 · Cited by 380 · see citing works
Editors, T. P. M. (2006) · PLoS Medicine · doi:10.1371/journal.pmed.0030291 · Cited by 373 · see citing works
Publication bias: the elephant in the review
Dalton, J. E., Bolen, S. D., & Mascha, E. J. (2016) · Anesthesia & Analgesia · doi:10.1213/ane.0000000000001596 · Cited by 337 · see citing works
Soderberg, C. K., Errington, T. M., Schiavone, S. R., Bottesini, J., Thorn, F. S., Vazire, S., Esterling, K. M., & Nosek, B. A. (2021) · Nature Human Behaviour · doi:10.1038/s41562-021-01142-4 · Cited by 289 · see citing works
COMPare: a prospective cohort study correcting and monitoring 58 misreported trials in real time
Goldacre, B., Drysdale, H., Dale, A., Milosevic, I., Slade, E., Hartley, P., Marston, C., Powell-Smith, A., Heneghan, C., & Mahtani, K. R. (2019) · Trials · doi:10.1186/s13063-019-3173-2 · Cited by 233 · see citing works
on behalf of the IICARus Collaboration, Hair, K., Macleod, M. R., & Sena, E. S. (2019) · Research Integrity and Peer Review · doi:10.1186/s41073-019-0069-3 · Cited by 174 · see citing works
Reproducible and reusable research: are journal data sharing policies meeting the mark?
Vasilevsky, N. A., Minnier, J., Haendel, M. A., & Champieux, R. E. (2017) · PeerJ · doi:10.7717/peerj.3208 · Cited by 165 · see citing works
Assessment of transparency indicators across the biomedical literature: How open is open?
Serghiou, S., Contopoulos-Ioannidis, D. G., Boyack, K. W., Riedel, N., Wallach, J. D., & Ioannidis, J. P. A. (2021) · PLOS Biology · doi:10.1371/journal.pbio.3001107 · Cited by 139 · see citing works
Self-citationDigital open science—Teaching digital tools for reproducible and transparent research
Toelch, U., & Ostwald, D. (2018) · PLOS Biology · doi:10.1371/journal.pbio.2006022 · Cited by 80 · see citing works
Dudda, L., Kormann, E., Kozula, M., DeVito, N. J., Klebel, T., Dewi, A. P. M., Spijker, R., Stegeman, I., Van den Eynden, V., Ross-Hellauer, T., et al. (2025) · Royal Society Open Science · doi:10.1098/rsos.242057 · Cited by 28 · see citing works
Hunniford, V. T., Grudniewicz, A., Fergusson, D. A., Montroy, J., Grigor, E., Lansdell, C., Lalu, M. M., & On behalf of The Canadian Critical Care Translational Biology Group (2023) · eLife · doi:10.7554/elife.76300 · Cited by 26 · see citing works
DeVito, N. J., Morley, J., Smith, J. A., Drysdale, H., Goldacre, B., & Heneghan, C. (2024) · BMJ Medicine · doi:10.1136/bmjmed-2023-000738 · Cited by 18 · see citing works
Flagged first, then by citations
Research works printed but not linked: 4
- Vol. 63 (2012), pp. 185–198
- Vol. 62 (2011), pp. 141–155
- Vol. 57 (2006), pp. 1–18
- Vol. 55 (2004), pp. 209–222
Cited by
Sorted by citations
A feature and not a bug: Replication failures can lead to new biology
Marder, E. (2026) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2026.103268
- Authors
- Haven, T. L., & Ioannidis, J. P. A.
- Affiliations
- Tilburg University (NL)
- Stanford University (US)
- Venue
- Annual Review of Medicine
- Publisher
- Annual Reviews
- Published
- 2026-01-27
- Volume
- 77
- Issue
- 1
- Pages
- 537-549
- ISSN
- 0066-4219 (ISSN-L)
- 1545-326X
- OpenAlex
- W4414791581
- Cited by
- 9
- References
- 22
- Citations by year
- 2026: 9
- FWCI
- 1.52
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Annual Review of Medicine (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Information Systems and Management › Academic Publishing and Open Access
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Computer Science › Information Systems › Research Data Management Practices
- Keywords
- reproducibility, irreproducibility, research replicability, meta-research, reporting quality, peer review, scientific reform, biomedical research
- MeSH terms
- Humans, Research Design, Reproducibility of Results, Biomedical Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
