Is biomedical research self-correcting? Modelling insights on the persistence of spurious science
Open accessAbstract
The reality that volumes of published biomedical research are not reproducible is an increasingly recognized problem. Spurious results reduce trustworthiness of reported science, increasing research waste. While science should be self-correcting from a philosophical perspective, that in insolation yields no information on efforts required to nullify suspect findings or factors shaping how quickly science may be corrected. There is also a paucity of information on how perverse incentives in the publishing ecosystem favouring novel positive findings over null results shape the ability of published science to self-correct. Knowledge of factors shaping self-correction of science remain obscure, limiting our ability to mitigate harms. This modelling study introduces a simple model to capture dynamics of the publication ecosystem, exploring factors influencing research waste, trustworthiness, corrective effort and time to correction. Results from this work indicate that research waste and corrective effort are highly dependent on field-specific false positive rates and time delays to corrective results to spurious findings are propagated. The model also suggests conditions under which biomedical science is self-correcting and those under which publication of correctives alone cannot stem propagation of untrustworthy results. Finally, this work models a variety of potential mitigation strategies, including researcher- and publisher-driven interventions.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The self-correcting nature of science is at best torturously slow.
“It can be shown that the”
From Fraudulent studies are undermining the reliability of systematic reviews: on the prevalence of problematic images in preclinical depression studies (2025) · cites this paper as [56]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
2 retracted
Retracted Science and the Retraction Index
Fang, F. C., & Casadevall, A. (2011) · Infection and Immunity · doi:10.1128/iai.05661-11 · Cited by 405 · see citing works
RetractedKataoka, Y., Banno, M., Tsujimoto, Y., Ariie, T., Taito, S., Suzuki, T., Oide, S., & Furukawa, T. A. (2022) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2022.06.015 · Cited by 44 · see citing works
RetractedWhy Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505 · see citing works
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068 · cited 1× · see citing works
“This ‘ file-drawer ’ problems means that important and diligent negative findings are often not submitted for publication, while spurious findings are unduly favoured [27 – 30].”
BackgroundRaise standards for preclinical cancer research
Begley, C. G., & Ellis, L. M. (2012) · Nature · doi:10.1038/483531a · Cited by 2,979 · cited 1× · see citing works
“In fields as vital as cancer research, only 11% – 50% of preclinical results are deemed replicable [18,19].”
BackgroundAvoidable waste in the production and reporting of research evidence
Chalmers, I., & Glasziou, P. (2009) · The Lancet · doi:10.1016/s0140-6736(09)60329-9 · Cited by 2,208 · see citing works
Fanelli, D. (2009) · PLoS ONE · doi:10.1371/journal.pone.0005738 · Cited by 1,911 · cited 1× · see citing works
“Inappropriate statistical practices and manipulation underpin many spurious results [10 – 14], affecting up to 75% of biomedical publications [15 – 17].”
BackgroundIncreasing value and reducing waste in research design, conduct, and analysis
Ioannidis, J. P. A., Greenland, S., Hlatky, M. A., Khoury, M. J., Macleod, M. R., Moher, D., Schulz, K. F., & Tibshirani, R. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62227-8 · Cited by 1,597 · see citing works
Reducing waste from incomplete or unusable reports of biomedical research
Glasziou, P., Altman, D. G., Bossuyt, P., Boutron, I., Clarke, M., Julious, S., Michie, S., Moher, D., & Wager, E. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62228-x · Cited by 1,319 · see citing works
Negative results are disappearing from most disciplines and countries
Fanelli, D. (2012) · Scientometrics · doi:10.1007/s11192-011-0494-7 · Cited by 1,248 · see citing works
Biomedical research: increasing value, reducing waste.
Macleod, M. R., Michie, S., Roberts, I., Dirnagl, U., Chalmers, I., Ioannidis, J. P. A., Salman, R. A.-S., Chan, A.-W., & Glasziou, P. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62329-6 · Cited by 1,003 · see citing works
The natural selection of bad science
Smaldino, P. E., & McElreath, R. (2016) · Royal Society Open Science · doi:10.1098/rsos.160384 · Cited by 860 · see citing works
An investigation of the false discovery rate and the misinterpretation of <i>p</i> -values
Colquhoun, D. (2014) · Royal Society Open Science · doi:10.1098/rsos.140216 · Cited by 792 · see citing works
Replicability, Robustness, and Reproducibility in Psychological Science
Nosek, B. A., Hardwicke, T. E., Moshontz, H., Allard, A., Corker, K. S., Dreber, A., Fidler, F., Hilgard, J., Kline Struhl, M., Nuijten, M. B., et al. (2022) · Annual Review of Psychology · doi:10.1146/annurev-psych-020821-114157 · Cited by 770 · see citing works
Why Science Is Not Necessarily Self-Correcting
Ioannidis, J. P. A. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612464056 · Cited by 563 · see citing works
Evolution of Reporting <i>P</i> Values in the Biomedical Literature, 1990-2015
Chavalarias, D., Wallach, J. D., Li, A. H. T., & Ioannidis, J. P. A. (2016) · JAMA · doi:10.1001/jama.2016.1952 · Cited by 370 · see citing works
An Excess of Positive Results: Comparing the Standard Psychology Literature With Registered Reports
Scheel, A. M., Schijen, M., & Lakens, D. (2020) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459211007467 · Cited by 359 · see citing works
Glasziou, P., Sanders, S., & Hoffmann, T. (2020) · BMJ · doi:10.1136/bmj.m1847 · Cited by 330 · see citing works
Challenges for assessing replicability in preclinical cancer biology
Errington, T. M., Denis, A., Perfito, N., Iorns, E., & Nosek, B. A. (2021) · eLife · doi:10.7554/elife.67995 · Cited by 323 · see citing works
Lawrence, P. A. (2007) · Current Biology · doi:10.1016/j.cub.2007.06.014 · Cited by 296 · see citing works
Phenomena of Retraction: Reasons for Retraction and Citations to the Publications
Budd, J. M., Sievert, M., & Schultz, T. R. (1998) · JAMA · doi:10.1001/jama.280.3.296 · Cited by 230 · see citing works
Modelling science trustworthiness under publish or perish pressure
Grimes, D. R., Bauch, C. T., & Ioannidis, J. P. A. (2018) · Royal Society Open Science · doi:10.1098/rsos.171511 · Cited by 219 · see citing works
Self-citationCurrent Incentives for Scientists Lead to Underpowered Studies with Erroneous Conclusions
Higginson, A. D., & Munafò, M. R. (2016) · PLOS Biology · doi:10.1371/journal.pbio.2000995 · Cited by 192 · see citing works
Post retraction citations in context: a case study
Bar-Ilan, J., & Halevi, G. (2017) · Scientometrics · doi:10.1007/s11192-017-2242-0 · Cited by 191 · see citing works
Methodological quality of COVID-19 clinical research
Jung, R. G., Di Santo, P., Clifford, C., Prosperi-Porta, G., Skanes, S., Hung, A., Parlow, S., Visintini, S., Ramirez, F. D., Simard, T., et al. (2021) · Nature Communications · doi:10.1038/s41467-021-21220-5 · Cited by 189 · see citing works
Altman, D. G., & Simera, I. (2016) · Journal of the Royal Society of Medicine · doi:10.1177/0141076815625599 · Cited by 174 · see citing works
COVID-19-related medical research: a meta-research and critical appraisal
Raynaud, M., Zhang, H., Louis, K., Goutaudier, V., Wang, J., Dubourg, Q., Wei, Y., Demir, Z., Debiais, C., Aubert, O., et al. (2021) · BMC Medical Research Methodology · doi:10.1186/s12874-020-01190-w · Cited by 172 · see citing works
The carnage of substandard research during the COVID-19 pandemic: a call for quality
Bramstedt, K. A. (2020) · Journal of Medical Ethics · doi:10.1136/medethics-2020-106494 · Cited by 170 · see citing works
Alexander, P. E., Debono, V. B., Mammen, M. J., Iorio, A., Aryal, K., Deng, D., Brocard, E., & Alhazzani, W. A. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.04.016 · Cited by 157 · see citing works
Worsening file-drawer problem in the abstracts of natural, medical and social science databases
Pautasso, M. (2010) · Scientometrics · doi:10.1007/s11192-010-0233-5 · Cited by 136 · see citing works
An excess of positive results: Comparing the standard Psychology literature with Registered Reports
Scheel, A. M., Schijen, M., & Lakens, D. (2020) · doi:10.31234/osf.io/p6e9c · Cited by 136 · see citing works
Statistics and ethics in medical research. Misuse of statistics is unethical.
Altman, D. G. (1980) · BMJ · doi:10.1136/bmj.281.6249.1182 · Cited by 132 · see citing works
Bishop, D. (2019) · Quarterly Journal of Experimental Psychology · doi:10.1177/1747021819886519 · Cited by 109 · see citing works
The Mismeasure of Science: Citation Analysis
MacRoberts, M. H., & MacRoberts, B. R. (2018) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23970 · Cited by 101 · see citing works
de Vries, Y. A., Roest, A. M., de Jonge, P., Cuijpers, P. J., Munafo, M. R., & Bastiaansen, J. A. (2018) · Psychological Medicine · doi:10.1017/s0033291718001873 · Cited by 100 · see citing works
The (lack of) Impact of Retraction on Citation Networks
Madlock-Brown, C. R., & Eichmann, D. (2015) · Science and Engineering Ethics · doi:10.1007/s11948-014-9532-1 · Cited by 89 · see citing works
Altman, N., & Krzywinski, M. I. (2017) · Nature Methods · doi:10.1038/nmeth.4210 · Cited by 85 · see citing works
Flawed ivermectin preprint highlights challenges of COVID drug studies.
Reardon, S. (2021) · Nature · doi:10.1038/d41586-021-02081-w · Cited by 83 · see citing works
Where Are the Self-Correcting Mechanisms in Science?
Vazire, S., & Holcombe, A. O. (2022) · Review of General Psychology · doi:10.1177/10892680211033912 · Cited by 81 · see citing works
Does retraction after misconduct have an impact on citations? A pre–post study
Candal-Pedreira, C., Ruano-Ravina, A., Fernández, E., Ramos, J., Campos-Varela, I., & Pérez-Ríos, M. (2020) · BMJ Global Health · doi:10.1136/bmjgh-2020-003719 · Cited by 67 · see citing works
Krawczyk, M. W. (2015) · PLoS ONE · doi:10.1371/journal.pone.0127872 · Cited by 52 · see citing works
Science Is Not Always “Self-Correcting”
Cofnas, N. (2015) · Foundations of Science · doi:10.1007/s10699-015-9421-3 · Cited by 51 · see citing works
Correction of scientific literature: Too little, too late!
Besançon, L., Bik, E., Heathers, J., & Meyerowitz-Katz, G. (2022) · PLOS Biology · doi:10.1371/journal.pbio.3001572 · Cited by 46 · see citing works
Proposed mechanisms for homeopathy are physically impossible
Grimes, D. R. (2012) · Focus on Alternative and Complementary Therapies · doi:10.1111/j.2042-7166.2012.01162.x · Cited by 46 · see citing works
Self-citationHardwicke, T. E., Szűcs, D., Thibault, R. T., Crüwell, S., van den Akker, O. R., Nuijten, M. B., & Ioannidis, J. P. A. (2021) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459211040837 · Cited by 44 · see citing works
Fragility indices for only sufficiently likely modifications
Baer, B. R., Gaudino, M. F. L., Charlson, M. E., Fremes, S. E., & Wells, M. T. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2105254118 · Cited by 40 · see citing works
Yu, L., Miao, M., Liu, W., Zhang, B., & Zhang, P. (2020) · Accountability in Research · doi:10.1080/08989621.2020.1809386 · Cited by 37 · see citing works
Poor quality research and clinical practice during COVID-19
Dobler, C. C. (2020) · Breathe · doi:10.1183/20734735.0112-2020 · Cited by 34 · see citing works
Self-correction in science: The effect of retraction on the frequency of citations
Kühberger, A., Streit, D., & Scherndl, T. (2022) · PLoS ONE · doi:10.1371/journal.pone.0277814 · Cited by 33 · see citing works
Reporting all results efficiently: A RARE proposal to open up the file drawer
Laitin, D. D., Miguel, E. A., Alrababa’h, A., Bogdanoski, A., Grant, S., Hoeberling, K., Mo, C. H., Moore, D. A., Vazire, S., Weinstein, J. M., et al. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2106178118 · Cited by 30 · see citing works
Wagner, J. (2021) · International Journal of Social Research Methodology · doi:10.1080/13645579.2021.1922805 · Cited by 29 · see citing works
Singh, J. A., & Ravinetto, R. (2020) · Journal of Pharmaceutical Policy and Practice · doi:10.1186/s40545-020-00244-0 · Cited by 26 · see citing works
von Hippel, P. T. (2022) · Perspectives on Psychological Science · doi:10.1177/17456916211072525 · Cited by 25 · see citing works
Vorland, C. J., Brown, A. W., Ejima, K., Mayo‐Wilson, E., Valdez, D., & Allison, D. B. (2020) · European Journal of Clinical Investigation · doi:10.1111/eci.13190 · Cited by 24 · see citing works
Homeopathy can offer empirical insights on treatment effects in a null field
Sigurdson, M. K., Sainani, K. L., & Ioannidis, J. P. A. (2023) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2023.01.010 · Cited by 17 · see citing works
Grimes, D. R., & Bishop, D. (2017) · Child Development · doi:10.1111/cdev.13013 · Cited by 16 · see citing works
Self-citationThe darker side of quantitative academic performance metrics
Crous, C. J. (2019) · South African Journal of Science · doi:10.17159/sajs.2019/5785 · Cited by 14 · see citing works
Scientists should get credit for correcting the literature
Ribeiro, M. D., KALICHMAN, M. W., & Vasconcelos, S. M. R. (2022) · Nature Human Behaviour · doi:10.1038/s41562-022-01415-6 · Cited by 11 · see citing works
Grimes, D. R. (2022) · eLife · doi:10.7554/elife.79573 · Cited by 11 · see citing works
Self-citationAssociation between magnetic field exposure and miscarriage risk is not supported by the data
Grimes, D. R., & Heathers, J. A. J. (2021) · Scientific Reports · doi:10.1038/s41598-021-01391-3 · Cited by 8 · see citing works
Self-citationThe Citation of Retracted COVID-19 Papers is Common and Rarely CriticalPreprint
Meyerowitz‐Katz, G., Sekhar, P., Besançon, L., Turner, T., & McDonald, S. J. (2022) · medRxiv · doi:10.1101/2022.06.30.22277084 · Cited by 6 · see citing works
Is biomedical research self-correcting? Modeling insights on the persistence of spurious sciencePreprint
Grimes, D. R. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.07.17.549436 · Cited by 2 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
da Silva, J. A. T., & Nazarovets, S. (2025) · Naunyn-Schmiedeberg s Archives of Pharmacology · doi:10.1007/s00210-025-04651-5 · Cited by 19 · see citing works
Smith, G. D., & Ebrahim, S. B. J. (2024) · International Journal of Epidemiology · doi:10.1093/ije/dyae179 · Cited by 10 · see citing works
Towards replicability and sustainability in cancer research
Grimes, D. R. (2024) · BJC Reports · doi:10.1038/s44276-024-00090-6 · Cited by 8 · see citing works
Self-citationBerrío, J. P., & Kalliokoski, O. (2025) · FEBS Letters · doi:10.1002/1873-3468.70077 · Cited by 7 · see citing works
Methodological issues in visible LED therapy dermatological research and reporting
Grimes, D. R. (2025) · PLoS ONE · doi:10.1371/journal.pone.0332995 · Cited by 3 · see citing works
Self-citation“Cake causes herpes?” - promiscuous dichotomisation induces false positives
Grimes, D. R., & Heathers, J. A. J. (2025) · BMC Medical Research Methodology · doi:10.1186/s12874-025-02712-0 · Cited by 3 · see citing works
Self-citationA primer: peer review process for Advances in Physiology Education
Wilson, T. E., & Harrison‐Bernard, L. M. (2024) · AJP Advances in Physiology Education · doi:10.1152/advan.00127.2024 · Cited by 3 · see citing works
Grimes, D., & Gorski, D. H. (2025) · Journal of Medical Internet Research · doi:10.2196/64679 · Cited by 2 · see citing works
Self-citationWang, J. (2025) · Science Editing · doi:10.6087/kcse.360 · Cited by 1 · see citing works
Grimes, D. R. (2024) · Vaccine · doi:10.1016/j.vaccine.2024.126657 · Cited by 1 · see citing works
Self-citationGrimes, D. R. (2025) · eLife · doi:10.1101/2025.08.15.25333793
Self-citationGrimes, D. R. (2025) · eLife · doi:10.7554/elife.108693.1
Self-citationGrimes, D. R. (2025) · eLife · doi:10.7554/elife.108693
Self-citationFròsina, G. (2026) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.70366
Grimes, D. R. (2026) · eLife · doi:10.7554/elife.108693.2
Self-citationThe Illusion of the “Self-correcting” Nature of Science
Smith, G. D. (2026) · Epidemiology · doi:10.1097/ede.0000000000001987
Hydrolyzed Collagen for Skin Aging Meta‐Analyses Are Unreliable
Grimes, D. R., & Mahto, A. (2026) · Journal of Cosmetic Dermatology · doi:10.1111/jocd.71208
Self-citation
- Authors
- Grimes, D. R.
- Affiliations
- Trinity College Dublin (IE)
- Corresponding author
- Grimes, David Robert
- ORCID
- Grimes, David Robert
- Venue
- Royal Society Open Science
- Publisher
- The Royal Society
- Published
- 2024-01
- Volume
- 11
- Issue
- 1
- Pages
- 231056-231056
- ISSN
- 2054-5703 (ISSN-L)
- PMC
- PMC10827424
- OpenAlex
- W4391889715
- Cited by
- 18
- References
- 62
- Citations by year
- 2026: 4 · 2025: 9 · 2024: 5
- FWCI
- 6.72
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- Royal Society Open Science (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Wellcome Trust (214461/A/18/Z)
- Topics
- Social Sciences › Decision Sciences › Information Systems and Management › Academic Publishing and Open Access
- Social Sciences › Safety Research › Academic integrity and plagiarism
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › scientometrics and bibliometrics research
- Keywords
- spurious findings, research waste, reproducibility, publication incentives, false positive rate, mitigation measures, biomedical research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
