Protect us from poor-quality medical research
Open accessThe DOI matches its Crossref record; 47 of the 48 references it cites were found in a registry or an index.
Abstract
Much of the published medical research is apparently flawed, cannot be replicated and/or has limited or no utility. This article presents an overview of the current landscape of biomedical research, identifies problems associated with common study designs and considers potential solutions. Randomized clinical trials, observational studies, systematic reviews and meta-analyses are discussed in terms of their inherent limitations and potential ways of improving their conduct, analysis and reporting. The current emphasis on statistical significance needs to be replaced by sound design, transparency and willingness to share data with a clear commitment towards improving the quality and utility of clinical research.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The text we found for this paper looks like another document, so the self-check wasn’t run.
Citations
Cites
A manifesto for reproducible science
Munafò, M. R., Nosek, B. A., Bishop, D. V. M., Button, K. S., Chambers, C. D., Percie du Sert, N., Simonsohn, U., Wagenmakers, E.-J., Ware, J. J., & Ioannidis, J. P. A. (2017) · Nature Human Behaviour · doi:10.1038/s41562-016-0021 · Cited by 3,397
Self-citationChan, A.-W., Hróbjartsson, A., Haahr, M. T., Gøtzsche, P. C., & Altman, D. G. (2004) · JAMA · doi:10.1001/jama.291.20.2457 · Cited by 1,845
The Mass Production of Redundant, Misleading, and Conflicted Systematic Reviews and Meta‐analyses
IOANNIDIS, J. P. A. (2016) · The Milbank Quarterly · doi:10.1111/1468-0009.12210 · Cited by 1,483
Self-citationReducing waste from incomplete or unusable reports of biomedical research
Glasziou, P., Altman, D. G., Bossuyt, P., Boutron, I., Clarke, M., Julious, S., Michie, S., Moher, D., & Wager, E. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62228-x · Cited by 1,319
What does research reproducibility mean?
Goodman, S. N., Fanelli, D., & Ioannidis, J. P. A. (2016) · Science Translational Medicine · doi:10.1126/scitranslmed.aaf5027 · Cited by 1,293
Self-citationSavović, J., Jones, H. E., Altman, D. G., Harris, R. J., Jüni, P., Pildal, J., Als-Nielsen, B., Balk, E. M., Gluud, C., Gluud, L. L., et al. (2012) · Annals of Internal Medicine · doi:10.7326/0003-4819-157-6-201209180-00537 · Cited by 1,049
Self-citationBiomedical research: increasing value, reducing waste.
Macleod, M. R., Michie, S., Roberts, I., Dirnagl, U., Chalmers, I., Ioannidis, J. P. A., Salman, R. A.-S., Chan, A.-W., & Glasziou, P. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62329-6 · Cited by 1,003
Self-citationThe scandal of poor medical research.
Altman, D. G. (1994) · BMJ · doi:10.1136/bmj.308.6924.283 · Cited by 897
How to Make More Published Research True
Ioannidis, J. P. A. (2014) · PLoS Medicine · doi:10.1371/journal.pmed.1001747 · Cited by 760
Self-citationSavović, J., Jones, H., Altman, D., Harris, R., Jűni, P., Pildal, J., Als-Nielsen, B., Balk, E., Gluud, C., Gluud, L., et al. (2012) · Health Technology Assessment · doi:10.3310/hta16350 · Cited by 668
Self-citation
Research works printed but not linked: 1
- Clinical Practice Guidelines We Can Trust, 2011, Institute of Medicine (US) Committee on Standards for Developing Trustworthy Clinical Practice Guidelines
Cited by
Preventive psychiatry: a blueprint for improving the mental health of young people
Fusar‐Poli, P., Correll, C. U., Arango, C., Berk, M., Patel, V., & Ioannidis, J. P. A. (2021) · World Psychiatry · doi:10.1002/wps.20869 · Cited by 497
Self-citationGuidance to best tools and practices for systematic reviews
Kolaski, K., Logan, L. R., & Ioannidis, J. P. A. (2023) · Systematic Reviews · doi:10.1186/s13643-023-02255-9 · Cited by 236
“Once a research question is established, searching on registry sites and databases for existing systematic reviews addressing the same or a similar topic is necessary in order to avoid contributing to research waste [131].”
Self-citationWang, R., Li, W., Bordewijk, E. M., Legro, R. S., Zhang, H., Wu, X., Gao, J., Morin-Papunen, L., Homburg, R., König, T. E., et al. (2019) · Human Reproduction Update · doi:10.1093/humupd/dmz029 · Cited by 137
Self-citationKolaski, K., Logan, L. R., Goss, K. D., & Butler, C. (2021) · Developmental Medicine & Child Neurology · doi:10.1111/dmcn.14949 · Cited by 47
Rajan, S., Campagnolo, M., Callaghan, B. C., & Gibbons, C. H. (2018) · Clinical Autonomic Research · doi:10.1007/s10286-018-0540-0 · Cited by 47
Guidance to best tools and practices for systematic reviews
Kolaski, K., Logan, L. R., & Ioannidis, J. P. A. (2023) · JBI Evidence Synthesis · doi:10.11124/jbies-23-00139 · Cited by 63
Self-citationStocking, K., Wilkinson, J., Lensen, S. F., Brison, D. R., Roberts, S. A., & Vail, A. (2019) · Human Reproduction · doi:10.1093/humrep/dez017 · Cited by 46
Rafael, Z. B. (2020) · Human Reproduction · doi:10.1093/humrep/deaa134 · Cited by 28
Guidance to best tools and practices for systematic reviews1
Kolaski, K., Logan, L. R., & Ioannidis, J. P. A. (2023) · Journal of Pediatric Rehabilitation Medicine · doi:10.3233/prm-230019 · Cited by 48
“Oncearesearch question is established, searching on registry sites and databases for existing systematic reviews addressing the same or a similar topic is necessary in order to avoid contributing to research waste [131].”
Self-citationGuidance to best tools and practices for systematic reviews
Kolaski, K., Logan, L. R., & Ioannidis, J. P. A. (2023) · British Journal of Pharmacology · doi:10.1111/bph.16100 · Cited by 33
Self-citationGuidance to best tools and practices for systematic reviews
Kolaski, K., Logan, L. R., & Ioannidis, J. P. A. (2023) · BMC Infectious Diseases · doi:10.1186/s12879-023-08304-x · Cited by 27
“Once a research question is established, searching on registry sites and databases for existing systematic reviews addressing the same or a similar topic is necessary in order to avoid contributing to research waste [131].”
Self-citationConcerning the “repression” of ω -3 polyunsaturated fatty acids by adepts of evidence-based medicine
Torshin, I. Y., Громова, О. А., & Kobalava, Z. D. (2019) · FARMAKOEKONOMIKA Modern Pharmacoeconomics and Pharmacoepidemiology · doi:10.17749/2070-4909.2019.12.2.91-114 · Cited by 22
Guidance to Best Tools and Practices for Systematic Reviews
Kolaski, K., Logan, L. R., & Ioannidis, J. P. A. (2023) · JBJS Reviews · doi:10.2106/jbjs.rvw.23.00077 · Cited by 28
Self-citationIVF, from the past to the future: the inheritance of the Capri Workshop Group
The Annual Capri Workshop Group, Albertini, D. F., Crosignani, P., Dumoulin, J. C. M., Evers, J. L. H., Léridon, H., Mastenbroek, S., Painter, R. C., Pinborg, A., Somigliana, E., et al. (2020) · Human Reproduction Open · doi:10.1093/hropen/hoaa040 · Cited by 18
Self-citationMapping the characteristics of meta-analyses of pharmacy services: a systematic review
Bonetti, A. D. F., Rocca, A. M. D., Lucchetta, R. C., Tonin, F. S., Fernández-Llimós, F., & Pontarolo, R. (2020) · International Journal of Clinical Pharmacy · doi:10.1007/s11096-020-01058-5 · Cited by 17
Should we still offer elective freezing of all embryos in all IVF cycles?
Rafael, Z. B. (2020) · Human Reproduction · doi:10.1093/humrep/deaa204 · Cited by 15
LITERAS: Biomedical literature review and citation retrieval agents
Gorenshtein, A., Shihada, K., Sorka, M., Aran, D., & Shelly, S. (2025) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2025.110363 · Cited by 18
Jemioło, P., Storman, D., & Orzechowski, P. (2022) · Journal of Clinical Medicine · doi:10.3390/jcm11072054 · Cited by 12
Methodological quality and risk of bias of meta-analyses of pharmacy services: A systematic review
Bonetti, A. D. F., Tonin, F. S., Rocca, A. M. D., Lucchetta, R. C., Fernández-Llimós, F., & Pontarolo, R. (2020) · Research in Social and Administrative Pharmacy · doi:10.1016/j.sapharm.2020.12.011 · Cited by 15
Tarakji, Z., Kanaan, A., Saadi, S. M., Firwana, M., Allababidi, A. K., Abusalih, M. E., Basmaci, R., Rajjo, T. I., Wang, Z., Murad, M. H., et al. (2024) · BMC Medical Research Methodology · doi:10.1186/s12874-024-02372-6 · Cited by 11
“Nevertheless, assessing study quality is a time-consuming step in systematic reviews that can be associated with subjectivity and inconsistency [5].”
BackgroundJournal clubs in Australian medical schools: prevalence, application, and educators’ opinions
Ianno, D., Mirowska-Allen, K. L., Kunz, S., & O’Brien, R. C. (2020) · Journal of Educational Evaluation for Health Professions · doi:10.3352/jeehp.2020.17.9 · Cited by 10
Kemper, J. M., Wang, H. T. Y., Ong, A. G. J., Mol, B. W., & Rolnik, D. L. (2019) · European Journal of Obstetrics & Gynecology and Reproductive Biology · doi:10.1016/j.ejogrb.2019.12.022 · Cited by 7
Endometrial injury before IVF: light at the end of the tunnel or false hope?
Venetis, C. A. (2020) · Human Reproduction · doi:10.1093/humrep/deaa295 · Cited by 8
Sports Metaresearch: An Emerging Discipline of Sport Science and Medicine
Warmenhoven, J., Menaspà, P., Borg, D. N., Vazire, S., White, N. M., Sainani, K. L., Nimphius, S., Coutts, A. J., & Impellizzeri, F. M. (2025) · Sports Medicine · doi:10.1007/s40279-025-02181-x · Cited by 11
“…having a real problem to fix, appropriate anchoring of the question within the context of prior evidence, etc.) and multiplicity issues (e.g. assessing the primary endpoint in more than one way) [87], many of which are highly likely to be present in sport research, but have been largely unexplored.”
Network meta-analyses in reproductive medicine: challenges and opportunities
Wang, R., Seidler, A. L., Askie, L. M., Norman, R. J., Bhattacharya, S., van Wely, M., & Mol, B. W. (2020) · Human Reproduction · doi:10.1093/humrep/deaa126 · Cited by 6
Self-citationGuidance to best tools and practices for systematic reviews
Kolaski, K., Logan, L. R., & Ioannidis, J. P. A. (2023) · Acta Anaesthesiologica Scandinavica · doi:10.1111/aas.14295 · Cited by 11
Self-citationQuality of evidence on pre-eclampsia in the last three decades: An analysis of published literature
Wang, H. T. Y., Ong, A. G. J., Kemper, J. M., Mol, B. W., & Rolnik, D. L. (2019) · Pregnancy Hypertension · doi:10.1016/j.preghy.2019.09.005 · Cited by 6
Muche, T., Hayen, A. D., & Dawson, A. J. (2024) · Health Promotion International · doi:10.1093/heapro/daae199 · Cited by 6
Torshin, I. Y., Громова, О. А., & Кобалава, Ж. Д. (2019) · Effective Pharmacotherapy · doi:10.33978/2307-3586-2019-15-9-26-34 · Cited by 4
Rajkumar, R. V. (2022) · Galore International Journal of Applied Sciences and Humanities · doi:10.52403/gijash.20220701 · Cited by 4
A Dolce & Gabbana model in every ART clinic?
Barratt, C. L. R. (2018) · Molecular Human Reproduction · doi:10.1093/molehr/gay030 · Cited by 2
Self-citationPublish or perish: time for a rethink?Preprint
Saxena, S. G., Godfrey, T. E., & Yannessa, J. F. (2023) · Qeios · doi:10.32388/fpzjkp · Cited by 3
Echeverri, D., Sarria, M. A., & Palma, G. (2025) · Memórias do Instituto Oswaldo Cruz · doi:10.1590/0074-02760240144 · Cited by 3
“In recent years, there has been growing concern about the dissemination of low-quality research in scientific literature and predatory journals, (41,42) high-lighting how manuscripts with insufficient peer review process or questionable methods or data, can gain rapid visibility, influencing…”
Warmenhoven, J., Shrier, I., Bullock, G. S., Holcombe, A. O., Nimphius, S., Menaspà, P., Coutts, A. J., & Impellizzeri, F. M. (2026) · Sports Medicine · doi:10.1007/s40279-026-02394-8 · Cited by 2
Trkulja, V., & Hrabač, P. (2020) · Croatian Medical Journal · doi:10.3325/cmj.2020.61.469 · Cited by 1
On the Indispensability for Standardization of the Basic Examination of Human Semen
Björndahl, L. (2021) · Springer eBooks · doi:10.1007/978-3-030-66292-9_41 · Cited by 1
Actuación de los clérigos y religiosos en la independencia de Colombia
Piedrahita, S. D. (2011) · Boletín de historia y antigüedades/Boletin de historia y antiguedades · Cited by 1
Rajkumar, R. V. (2023) · International Journal of Research and Review · doi:10.52403/ijrr.20230250
(2018) · Human Reproduction · doi:10.1093/humrep/dey085 · Cited by 1
Not many shoulders of giants to stand on these days
Albertini, D. F. (2018) · Journal of Assisted Reproduction and Genetics · doi:10.1007/s10815-018-1201-x
Can any meta-analysis help us manage polycystic ovary syndrome?
Nader, S. (2018) · Fertility and Sterility · doi:10.1016/j.fertnstert.2018.05.021
Human Reproduction’s greatest hits
Evers, J. L. H. (2018) · Human Reproduction · doi:10.1093/humrep/dey308
Self-citationDonor Insemination: Past, Present and Future Perspectives
Barratt, C. L. R., Agnew, R., & Heighton, E. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108762571.016
Self-citationDas endometriale Scratching: neue Daten zur Effektivität?
(2021) · Gynäkologische Endokrinologie · doi:10.1007/s10304-021-00394-7
On the pasta theory of reproductive research
(2018) · Human Reproduction · doi:10.1093/humrep/dey223
Evidence in reproductive medicine
Revelli, A., Ruffa, A., & Gennarelli, G. (2022) · Reproductive and Developmental Medicine · doi:10.1097/rd9.0000000000000028
Updates in Management of SARS-CoV-2 Infection
Flisiak, R. (2022) · MDPI eBooks · doi:10.3390/books978-3-0365-5390-0
Birds of a Feather: Advancing US Active Duty, Reservist, and Veteran Medical Research, Together
Babyar, J. C. (2022) · Public Integrity · doi:10.1080/10999922.2022.2124606
“Issues with data omission, data inclusion, trial design, reliance on specific statistical criteria, inappropriate conclusions, inconsistencies, reproducibility, usefulness, ethics, and more recently, meta-analysis redundancy (ESHRE Capri Workshop Group, 2018) lessen the quality of the research.”
BackgroundIndrayan, A., Vishwakarma, G., Malhotra, R., Gupta, P., Sachdev, H., Karande, S., Asthana, S., & Labani, S. (2022) · Journal of Postgraduate Medicine · doi:10.4103/jpgm.jpgm_460_22 · Cited by 1
Flynn, M. F. (2023) · International Surgery Journal · doi:10.18203/2349-2902.isj20230284
Why Do Researchers Falsify Data or Manipulate Study Findings?
Crombie, I. (2021) · Wiley eBooks · doi:10.1002/9781394322183.ch7
Diagnostics and Therapeutics in Ophthalmology
Arnold‐Vangsted, A., & Subhi, Y. (2025) · Journal of Personalized Medicine · doi:10.3390/jpm15010028
Padayachy, K., Fatima, I., Kahere, M., du Rose, A., & Pohlman, K. A. (2025) · BMC Complementary Medicine and Therapies · doi:10.1186/s12906-025-05044-0 · Cited by 1
“A significant advantage of using Masters dissertations was the standardization of outcome measurements across studies and institutions, allowing for the pooling of data—a feature often lacking in published literature—which strengthens the findings [69].”
A Retrospective Reappraisal of Labetalol’s Hemodynamic Effects
Rinker, M. M., Valadez, T., Gimbar, R. P., & Posen, A. K. (2025) · Journal of Pharmacy Practice · doi:10.1177/08971900251370870
Hegazi, M., Saadi, S. M., Allababidi, A. K., Farah, M. H., Abusalih, M. E., Basha, A. S., Hasan, B., Rajjo, T. I., Wang, Z., & Murad, M. H. (2025) · JBI Evidence Synthesis · doi:10.11124/jbies-25-00225
Gallagher, S., Attinger, S. A., Kerridge, I., Norman, R. J., & Lipworth, W. L. (2026) · Accountability in Research · doi:10.1080/08989621.2025.2611383
“There have also been calls for more responsible approach to research in corporatised clinics (Dondorp and de Wert 2011; ESHRE Capri Workshop Group 2018; European Society of Human Reproduction and Embryology 2022; Human Fertilisation and Embryology Authority 2019; Mastenbroek et al. 2021; van de…”
The importance of good medical practice as a basis of evidence in the decision-making process
Giannoudis, P., Rodham, P. L., & Pitsilos, C. (2026) · Elsevier eBooks · doi:10.1016/b978-0-443-34264-6.00038-1
Muhanga, M. (2025) · Springer eBooks · doi:10.1007/978-981-97-0821-5_168-1
Muhanga, M. (2026) · Springer eBooks · doi:10.1007/978-981-92-1290-3_168
Metadata
- Authors
- ESHRE Capri Workshop Group, Ioannidis, J. P. A., Bhattacharya, S., Evers, J. L. H., der Veen, F. V., Somigliana, E., Barratt, C. L. R., Bontempi, G., Baird, D. T., Crosignani, P., Devroey, P., Diedrich, K., Farquharson, R. G., Fraser, L. R., Geraedts, J. P. M., Gianaroli, L., La Vecchia, C., Magli, C., Negri, E., Sunde, A., Tapanainen, J. S., Tarlatzis, B. C., Steirteghem, A. V., & Veiga, A.
- Affiliations
- Stanford University (US)
- University of Aberdeen (GB)
- Maastricht University Medical Centre (NL)
- Academic Medical Centre, University of Amsterdam, Reproduction Medicine, Amsterdam, The Netherlands
- University of Dundee (GB)
- Université Libre de Bruxelles (BE)
- Fondazione IRCCS Ca' Granda Ospedale Maggiore Policlinico (IT)
- Vrije Universiteit Brussel (BE)
- Liverpool Women's Hospital (GB)
- King's College London (GB)
- University of Milan (IT)
- Department of Clinical Sciences and Community Health, Università degli Studi di Milano, Milan, Italy
- Reproductive Medicine, Sahlgrenska University Hospital, Gothenburg, Sweden
- University of Helsinki (FI)
- Centre for Reproductive Medicine, Universitair Ziekenhuis Vrije Universiteit Brussel, Belgium
- Reproductive Medicine Service, Dexeus Woemn’s Health, Barcelona, Spain
- ORCID
- Venue
- Human Reproduction
- Publisher
- Oxford University Press (OUP)
- Published
- 2018-03-29
- Volume
- 33
- Issue
- 5
- Pages
- 770-776
- ISSN
- 0268-1161 (ISSN-L)
- 1460-2350
- OpenAlex
- W2794626044
- Cited by
- 59
- References
- 48
- Citations by year
- 2026: 4 · 2025: 8 · 2024: 2 · 2023: 10 · 2022: 6 · 2021: 5 · 2020: 10 · 2019: 6 · 2018: 7
- FWCI
- 5.34
- Type
- journal article
- Language
- en
- Open access
- Licence
- Locations
- Human Reproduction (publishedVersion)
- PubMed (publishedVersion)
- Research Publications (Maastricht University) (publishedVersion, other-oa)
- Discovery Research Portal (University of Dundee) (publishedVersion, other-oa)
- Archivio istituzionale della ricerca (Alma Mater Studiorum Università di Bologna) (submittedVersion)
- Dépôt institutionnel de l'Université libre de Bruxelles (Université Libre de Bruxelles) (submittedVersion)
- ORCA Online Research @Cardiff (Cardiff University) (submittedVersion)
- Research Publications (Maastricht University) (submittedVersion, other-oa)
- Funders
- European Society of Human Reproduction and Embryology
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Health Sciences › Nursing › Research and Theory › Methodologies in Health Research and Practice
- Keywords
- research quality, reproducibility, randomized controlled trials, observational studies, systematic reviews, meta-analysis, statistical significance, research design, transparency, data sharing, clinical research quality
- MeSH terms
- Humans, Research Design, Information Dissemination, Biomedical Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.