Protect us from poor-quality medical research
Open accessThe DOI matches its Crossref record; 47 of the 48 references it cites were found in a registry or an index.
Abstract
Much of the published medical research is apparently flawed, cannot be replicated and/or has limited or no utility. This article presents an overview of the current landscape of biomedical research, identifies problems associated with common study designs and considers potential solutions. Randomized clinical trials, observational studies, systematic reviews and meta-analyses are discussed in terms of their inherent limitations and potential ways of improving their conduct, analysis and reporting. The current emphasis on statistical significance needs to be replaced by sound design, transparency and willingness to share data with a clear commitment towards improving the quality and utility of clinical research.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The text we found for this paper looks like another document, so the self-check wasn’t run.
Citations
Cites
A manifesto for reproducible science
Munafò, M. R., Nosek, B. A., Bishop, D. V. M., Button, K. S., Chambers, C. D., Percie du Sert, N., Simonsohn, U., Wagenmakers, E.-J., Ware, J. J., & Ioannidis, J. P. A. (2017) · Nature Human Behaviour · doi:10.1038/s41562-016-0021 · Cited by 3,397
Self-citationChan, A.-W., Hróbjartsson, A., Haahr, M. T., Gøtzsche, P. C., & Altman, D. G. (2004) · JAMA · doi:10.1001/jama.291.20.2457 · Cited by 1,845
The Mass Production of Redundant, Misleading, and Conflicted Systematic Reviews and Meta‐analyses
IOANNIDIS, J. P. A. (2016) · The Milbank Quarterly · doi:10.1111/1468-0009.12210 · Cited by 1,483
Self-citationReducing waste from incomplete or unusable reports of biomedical research
Glasziou, P., Altman, D. G., Bossuyt, P., Boutron, I., Clarke, M., Julious, S., Michie, S., Moher, D., & Wager, E. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62228-x · Cited by 1,319
What does research reproducibility mean?
Goodman, S. N., Fanelli, D., & Ioannidis, J. P. A. (2016) · Science Translational Medicine · doi:10.1126/scitranslmed.aaf5027 · Cited by 1,293
Self-citationSavović, J., Jones, H. E., Altman, D. G., Harris, R. J., Jüni, P., Pildal, J., Als-Nielsen, B., Balk, E. M., Gluud, C., Gluud, L. L., et al. (2012) · Annals of Internal Medicine · doi:10.7326/0003-4819-157-6-201209180-00537 · Cited by 1,049
Self-citationBiomedical research: increasing value, reducing waste.
Macleod, M. R., Michie, S., Roberts, I., Dirnagl, U., Chalmers, I., Ioannidis, J. P. A., Salman, R. A.-S., Chan, A.-W., & Glasziou, P. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62329-6 · Cited by 1,003
Self-citationThe scandal of poor medical research.
Altman, D. G. (1994) · BMJ · doi:10.1136/bmj.308.6924.283 · Cited by 897
How to Make More Published Research True
Ioannidis, J. P. A. (2014) · PLoS Medicine · doi:10.1371/journal.pmed.1001747 · Cited by 760
Self-citationSavović, J., Jones, H., Altman, D., Harris, R., Jűni, P., Pildal, J., Als-Nielsen, B., Balk, E., Gluud, C., Gluud, L., et al. (2012) · Health Technology Assessment · doi:10.3310/hta16350 · Cited by 668
Self-citation
Research works printed but not linked: 1
- Clinical Practice Guidelines We Can Trust, 2011, Institute of Medicine (US) Committee on Standards for Developing Trustworthy Clinical Practice Guidelines
Cited by
Preventive psychiatry: a blueprint for improving the mental health of young people
Fusar‐Poli, P., Correll, C. U., Arango, C., Berk, M., Patel, V., & Ioannidis, J. P. A. (2021) · World Psychiatry · doi:10.1002/wps.20869 · Cited by 497
Self-citationGuidance to best tools and practices for systematic reviews
Kolaski, K., Logan, L. R., & Ioannidis, J. P. A. (2023) · Systematic Reviews · doi:10.1186/s13643-023-02255-9 · Cited by 236
Self-citationWang, R., Li, W., Bordewijk, E. M., Legro, R. S., Zhang, H., Wu, X., Gao, J., Morin-Papunen, L., Homburg, R., König, T. E., et al. (2019) · Human Reproduction Update · doi:10.1093/humupd/dmz029 · Cited by 137
Self-citationKolaski, K., Logan, L. R., Goss, K. D., & Butler, C. (2021) · Developmental Medicine & Child Neurology · doi:10.1111/dmcn.14949 · Cited by 47
Rajan, S., Campagnolo, M., Callaghan, B. C., & Gibbons, C. H. (2018) · Clinical Autonomic Research · doi:10.1007/s10286-018-0540-0 · Cited by 47
Metadata
- Authors
- ESHRE Capri Workshop Group, Ioannidis, J. P. A., Bhattacharya, S., Evers, J. L. H., der Veen, F. V., Somigliana, E., Barratt, C. L. R., Bontempi, G., Baird, D. T., Crosignani, P., Devroey, P., Diedrich, K., Farquharson, R. G., Fraser, L. R., Geraedts, J. P. M., Gianaroli, L., La Vecchia, C., Magli, C., Negri, E., Sunde, A., Tapanainen, J. S., Tarlatzis, B. C., Steirteghem, A. V., & Veiga, A.
- Affiliations
- Stanford University (US)
- University of Aberdeen (GB)
- Maastricht University Medical Centre (NL)
- Academic Medical Centre, University of Amsterdam, Reproduction Medicine, Amsterdam, The Netherlands
- University of Dundee (GB)
- Université Libre de Bruxelles (BE)
- Fondazione IRCCS Ca' Granda Ospedale Maggiore Policlinico (IT)
- Vrije Universiteit Brussel (BE)
- Liverpool Women's Hospital (GB)
- King's College London (GB)
- University of Milan (IT)
- Department of Clinical Sciences and Community Health, Università degli Studi di Milano, Milan, Italy
- Reproductive Medicine, Sahlgrenska University Hospital, Gothenburg, Sweden
- University of Helsinki (FI)
- Centre for Reproductive Medicine, Universitair Ziekenhuis Vrije Universiteit Brussel, Belgium
- Reproductive Medicine Service, Dexeus Woemn’s Health, Barcelona, Spain
- ORCID
- Venue
- Human Reproduction
- Publisher
- Oxford University Press (OUP)
- Published
- 2018-03-29
- Volume
- 33
- Issue
- 5
- Pages
- 770-776
- ISSN
- 0268-1161 (ISSN-L)
- 1460-2350
- OpenAlex
- W2794626044
- Cited by
- 59
- References
- 48
- Citations by year
- 2026: 4 · 2025: 8 · 2024: 2 · 2023: 10 · 2022: 6 · 2021: 5 · 2020: 10 · 2019: 6 · 2018: 7
- FWCI
- 5.34
- Type
- journal article
- Language
- en
- Open access
- Licence
- Locations
- Human Reproduction (publishedVersion)
- PubMed (publishedVersion)
- Research Publications (Maastricht University) (publishedVersion, other-oa)
- Discovery Research Portal (University of Dundee) (publishedVersion, other-oa)
- Archivio istituzionale della ricerca (Alma Mater Studiorum Università di Bologna) (submittedVersion)
- Dépôt institutionnel de l'Université libre de Bruxelles (Université Libre de Bruxelles) (submittedVersion)
- ORCA Online Research @Cardiff (Cardiff University) (submittedVersion)
- Research Publications (Maastricht University) (submittedVersion, other-oa)
- Funders
- European Society of Human Reproduction and Embryology
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Health Sciences › Nursing › Research and Theory › Methodologies in Health Research and Practice
- Keywords
- research quality, reproducibility, randomized controlled trials, observational studies, systematic reviews, meta-analysis, statistical significance, research design, transparency, data sharing, clinical research quality
- MeSH terms
- Humans, Research Design, Information Dissemination, Biomedical Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Protect us from poor-quality medical research. https://app.cochat.ai/papers/10.1093/humrep/dey056
- The paper
- ESHRE Capri Workshop Group, Ioannidis, J. P. A., Bhattacharya, S., Evers, J. L. H., der Veen, F. V., Somigliana, E., Barratt, C. L. R., Bontempi, G., Baird, D. T., Crosignani, P., Devroey, P., Diedrich, K., Farquharson, R. G., Fraser, L. R., Geraedts, J. P. M., Gianaroli, L., La Vecchia, C., Magli, C., Negri, E., . . . Veiga, A. (2018). Protect us from poor-quality medical research. Human Reproduction. https://doi.org/10.1093/humrep/dey056
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.