Why Most Clinical Research Is Not Useful
Open accessThe DOI matches its Crossref record; 56 of the 56 references it cites were found in a registry or an index.
Abstract
John Ioannidis argues that problem base, context placement, information gain, pragmatism, patient centeredness, value for money, feasibility, and transparency define useful clinical research. He suggests most clinical research is not useful and reform is overdue.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Neglecting the actual development phase can be problematic, as a poor problem definition, insufficient attention to existing evidence and context needs, a missing model underlying the intervention, and an unsound selection of hypothesized change mechanisms can lead to inefficient treatments (30–32).”
From Developing a mechanism-based therapy for acute psychiatric inpatients with psychotic symptoms: an Intervention Mapping approach (2023) · cites this paper as [30]
Experimental — accuracy not yet measuredIn the citing paper
“Systematic review guidelines typically advocate that all relevant studies must be included for the conclusions of that review to be valid 22”
From Software support for environmental evidence synthesis (2018) · cites this paper as [22]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
Self-citationWhy Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · cited 1×
“I have previously written about why most published research is false [2] and how to make more of it true Summary Points”
BackgroundSelf-citationTunis (2003) · doi:10.1001/jama.290.12.1624 · Cited by 1,724 · cited 1×
“Pragmatism has long been advocated in clinical research [15], but it is rare.”
BackgroundSurrogate End Points in Clinical Trials: Are We Being Misled?
Fleming, T. R., & DeMets, D. L. (1996) · Annals of Internal Medicine · doi:10.7326/0003-4819-125-7-199610010-00011 · Cited by 1,651 · cited 1×
“Nonvalidated surrogate outcomes lacking clinical insight [13] and composite outcomes that combine outcomes of very different clinical portent [14] are often utilized so that authors can claim that clinical studies are well powered.”
BackgroundIncreasing value and reducing waste in research design, conduct, and analysis
Ioannidis, J. P. A., Greenland, S., Hlatky, M. A., Khoury, M. J., Macleod, M. R., Moher, D., Schulz, K. F., & Tibshirani, R. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62227-8 · Cited by 1,597
Self-citationReducing waste from incomplete or unusable reports of biomedical research
Glasziou, P., Altman, D. G., Bossuyt, P., Boutron, I., Clarke, M., Julious, S., Michie, S., Moher, D., & Wager, E. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62228-x · Cited by 1,319
Biomedical research: increasing value, reducing waste.
Macleod, M. R., Michie, S., Roberts, I., Dirnagl, U., Chalmers, I., Ioannidis, J. P. A., Salman, R. A.-S., Chan, A.-W., & Glasziou, P. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62329-6 · Cited by 1,003
Self-citationMoynihan, R., G⊘tzsche, P. C., Heath, I., & Henry, D. (2002) · BMJ · doi:10.1136/bmj.324.7342.886 · Cited by 900
Increasing value and reducing waste: addressing inaccessible research
Chan, A.-W., Song, F., Vickers, A., Jefferson, T., Dickersin, K., Gøtzsche, P. C., Krumholz, H. M., Ghersi, D., & van der Worp, H. B. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62296-5 · Cited by 887
How to Make More Published Research True
Ioannidis, J. P. A. (2014) · PLoS Medicine · doi:10.1371/journal.pmed.1001747 · Cited by 760 · cited 1×
“Joint efforts by multiple stakeholders may yield solutions that are more likely to be more widely adopted and thus successful [3].”
BackgroundSelf-citation
Cited by
CARE guidelines for case reports: explanation and elaboration document
Riley, D. S., Barber, M. S., Kienle, G. S., Aronson, J. K., von Schoen-Angerer, T., Tugwell, P., Kiene, H., Helfand, M., Altman, D. G., Sox, H., et al. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.04.026 · Cited by 2,287
“The evidence that hypotheses are not false is associatedwith the prior probability of an association between variablesdfor example, a diagnostic single nucleotide polymorphism test and a disease [64,65].”
BackgroundResearch co-design in health: a rapid overview of reviews
Slattery, P., Saeri, A. K., & Bragge, P. (2020) · Health Research Policy and Systems · doi:10.1186/s12961-020-0528-9 · Cited by 1,313
“The resulting gap between research and enduser needs is underscored by Ioannidis, who points out that “practicing doctors and other health care professionals [are] familiar with how little of what they find in medical journals is useful” ([3], p.”
BackgroundInfluentialMachine Learning Approaches for Clinical Psychology and Psychiatry
Dwyer, D. B., Falkai, P., & Koutsouleris, N. (2018) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-032816-045037 · Cited by 966
Seyhan, A. A. (2019) · Translational Medicine Communications · doi:10.1186/s41231-019-0050-7 · Cited by 713
Lederer, D. J., Bell, S. C., Branson, R. D., Chalmers, J. D., Marshall, R., Maslove, D. M., Ost, D. E., Punjabi, N. M., Schatz, M., Smyth, A. R., et al. (2019) · Annals of the American Thoracic Society · doi:10.1513/annalsats.201808-564ps · Cited by 715
On doing better science: From thrill of discovery to policy implications
Antonakis, J. (2017) · The Leadership Quarterly · doi:10.1016/j.leaqua.2017.01.006 · Cited by 475
“…and procedures so that other researchers are able to faithfully reproduce it (see Atwater, et al., 2014; Cortina, et al., 2016; Goodman, Fanelli, & Ioannidis, 2016; Nosek et al., 2015); computational steps taken to arrive at analyzed datasets and results, as well as programing and needed computer…”
BackgroundInfluentialWhy rankings of biomedical image analysis competitions should be interpreted with care
Maier-Hein, L., Eisenmann, M., Reinke, A., Onogur, S., Stankovic, M., Scholz, P., Arbel, T., Bogunovic, H., Bradley, A. P., Carass, A., et al. (2018) · Nature Communications · doi:10.1038/s41467-018-07619-7 · Cited by 380
Bleijenberg, N., de Man-van Ginkel, J. M., Trappenburg, J. C. A., Ettema, R. G. A., Sino, C. G., Heim, N., Hafsteindóttir, T. B., Richards, D. A., & Schuurmans, M. J. (2018) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2017.12.001 · Cited by 334
“This is even more important, since ‘value for money’ is an important consideration, especially when resources are limited (Ioannidis, 2016).”
BackgroundBarry, E., Roberts, S., Oke, J., Vijayaraghavan, S., Normansell, R., & Greenhalgh, T. (2017) · BMJ · doi:10.1136/bmj.i6538 · Cited by 330
Colloidal capsules: nano- and microcapsules with colloidal particle shells
Bollhorst, T., Rezwan, K., & Maas, M. (2017) · Chemical Society Reviews · doi:10.1039/c6cs00632a · Cited by 324
Weibel, S., Rücker, G., Eberhart, L. H., Pace, N. L., Hartl, H. M., Jordan, O. L., Mayer, D., Riemer, M., Schaefer, M. S., Raj, D., et al. (2020) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012859.pub2 · Cited by 168
No raw data, no science: another possible source of the reproducibility crisis
Miyakawa, T. (2020) · Molecular Brain · doi:10.1186/s13041-020-0552-2 · Cited by 268
Velupillai, S., Suominen, H., Liakata, M., Roberts, A., Shah, A. D., Morley, K., Osborn, D., Hayes, J., Stewart, R., Downs, J., et al. (2018) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2018.10.005 · Cited by 247
“For clinical NLP method development to become more integral in clinical outcomes research, there is a need to develop evaluation workbenches that can be used by clinicians to better understand the underlying parts of an NLP system and its impact on outcomes.”
A scoping review describes methods used to identify, prioritize and display gaps in health research
Nyanchoka, L., Tudur-Smith, C., Thu, V. N., Iversen, V., Tricco, A. C., & Porcher, R. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.01.005 · Cited by 198
Madden, M., & Speed, E. (2017) · Frontiers in Sociology · doi:10.3389/fsoc.2017.00007 · Cited by 204
What is science’s crisis really about?
Saltelli, A., & Funtowicz, S. (2017) · Futures · doi:10.1016/j.futures.2017.05.010 · Cited by 187
“The crisis leaves no field untouched, from preclinical medical research (Begley & Ellis, 2012; Begley, 2013) to clinical research (Ioannidis, 2016).”
BackgroundKelter, R. (2020) · BMC Medical Research Methodology · doi:10.1186/s12874-020-00980-6 · Cited by 167
Where is the evidence?: a call to action for learning analytics
Ferguson, R., & Clow, D. (2017) · Proceedings of the Seventh International Learning Analytics & Knowledge Conference · doi:10.1145/3027385.3027396 · Cited by 162
“Access, Ethics, Evidence, Evidence Hub, Generalisability, Learning Analytics Cycle, Reliability, Validity”
Editorial: Enhancing quantitative theory-testing entrepreneurship research
Anderson, B. S., Wennberg, K. J., & McMullen, J. S. (2019) · Journal of Business Venturing · doi:10.1016/j.jbusvent.2019.02.001 · Cited by 184
“We discuss the limitations of null hypothesis testing not necessarily to dissuade its use, although we believe that alternative approaches such as Bayesian inference and other methods offer untapped promise for entrepreneurship research (Hedström and Wennberg, 2017; Johnson et al., 2015).”
MethodRoutinely collected data for randomized trials: promises, barriers, and implications
Mc Cord, K. A., Al-Shahi Salman, R., Treweek, S., Gardner, H., Strech, D., Whiteley, W., Ioannidis, J. P. A., & Hemkens, L. G. (2018) · Trials · doi:10.1186/s13063-017-2394-5 · Cited by 148
“Various limitations of RCTs are used as arguments to support observational “real-world” RCD studies [9, 10].”
MethodSelf-citationHow to Implement Evidence-Based Healthcare
Nguyen-Buckley, C., & Steadman, R. (2018) · Anesthesia & Analgesia · doi:10.1213/ane.0000000000002842 · Cited by 141
Peters, S., Guccione, L., Francis, J., Best, S., Tavender, E. J., Curran, J., Davies, K., Rowe, S., Palmer, V. J., & Klaic, M. (2024) · Implementation Science · doi:10.1186/s13012-024-01394-4 · Cited by 158
“One contributor to this waste is that research often focusses on questions and outcomes that are of limited importance to end-users [4, 5].”
BackgroundHow to survive the medical misinformation mess
Ioannidis, J. P. A., Stuart, M. E., Brownlee, S., & Strite, S. A. (2017) · European Journal of Clinical Investigation · doi:10.1111/eci.12834 · Cited by 133
“Usefulness of clinical research [15] requires existence of a real problem to address, proper context placement, sufficient information gain, patient-centeredness, pragmatism, reasonable value-for-money, non-futility, and transparency.”
BackgroundSelf-citationNordhaus, W. D., & Moffat, A. (2017) · National Bureau of Economic Research · doi:10.3386/w23646 · Cited by 119
“There is a growing body of evidence on the difficulty of replicating studies, and this is clearly essential to areas of interest as centrally important as climate change damages (see particularly Ioannidis 2016).”
Sturmberg, J. P., Picard, M. A., Aron, D. C., Bennett, J. M., Bircher, J., DeHaven, M. J., Gijzel, S. M. W., Heng, H. H., Marcum, J. A., Martin, C. M., et al. (2019) · Frontiers in Medicine · doi:10.3389/fmed.2019.00059 · Cited by 118
Haber, N., Smith, E. R., Moscoe, E., Andrews, K., Audy, R., Bell, W., Brennan, A. T., Breskin, A., Kane, J. C., Karra, M., et al. (2018) · PLoS ONE · doi:10.1371/journal.pone.0196346 · Cited by 97
“A host of factors can render studies inaccurate [15] or not useful in practice [16].”
BackgroundKelter, R. (2020) · BMC Medical Research Methodology · doi:10.1186/s12874-020-00968-2 · Cited by 106
Hernandez‐Boussard, T., Monda, K. L., Crespo, B. C., & Riskin, D. J. (2019) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz119 · Cited by 89
“While randomized clinical trials (RCT) provide a foundation for clinical evidence, individual trials to assess treatment for a broad condition, such as hypertension or high cholesterol, may be less applicable to typical patients with multiple comorbidities.(2,3) Trials are often performed in highly controlled environm…”
BackgroundGan, D. Z. Q., McGillivray, L., Larsen, M. E., Christensen, H. M., & Torok, M. H. (2022) · Digital Health · doi:10.1177/20552076221098268 · Cited by 95
“54,55 Collaboration with end-users should therefore constitute an essential part of the design, development and evaluation of engagement strategies for DMHIs.”
BackgroundGraph Theoretic Analysis of Resting State Functional MR Imaging
Medaglia, J. D. (2017) · Neuroimaging Clinics of North America · doi:10.1016/j.nic.2017.06.008 · Cited by 90
“The features that distinguish useful clinical research include problem base, context placement, information gain, pragmatism, patient centeredness, value for money, feasibility, and transparency [90].”
BackgroundInfluentialAcknowledging and Overcoming Nonreproducibility in Basic and Preclinical Research
Ioannidis, J. P. A. (2017) · JAMA · doi:10.1001/jama.2017.0549 · Cited by 78
Self-citationTranslational machine learning for psychiatric neuroimaging
Walter, M., Alizadeh, S., Jamalabadi, H., Lueken, U., Dannlowski, U., Walter, H., Olbrich, S., Čolić, L., Kambeitz, J. P., Koutsouleris, N., et al. (2018) · Progress in Neuro-Psychopharmacology and Biological Psychiatry · doi:10.1016/j.pnpbp.2018.09.014 · Cited by 91
“This is especially the case if the aim is to predict outcomes at a single-subject level (Eklund et al., 2016; Eklund et al., 2012; Whelan & Garavan, 2014; Abi- Dargham & Horga, 2016; Dwyer et al., 2018a; Ioannidis, 2016).”
Recent advances in psychoradiology
Luo, L., You, W., DelBello, M. P., Gong, Q., & Li, F. Y. (2022) · Physics in Medicine and Biology · doi:10.1088/1361-6560/ac9d1e · Cited by 78
“Another important issue is that statistically significantfindings are often not clinicallymeaningful (Ioannidis 2016).”
BackgroundAnalytical Nanoscience and Nanotechnology: Where we are and where we are heading
Soriano, M. L., Zougagh, M., Valcárcel, M., & Ríos, Á. (2017) · Talanta · doi:10.1016/j.talanta.2017.09.012 · Cited by 84
Implementation science in maternity care: a scoping review
Dadich, A., Piper, A., & Coates, D. (2021) · Implementation Science · doi:10.1186/s13012-021-01083-6 · Cited by 78
“This is because those who deliver, manage and receive healthcare, negotiate multiple forms and sources of evidence, which complement and compete with each other [2, 11, 12] within a complex system of institutional logics [11, 13].”
BackgroundPsychedelic Research and the Need for Transparency: Polishing Alice’s Looking Glass
Petranker, R., Anderson, T., & Farb, N. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.01681 · Cited by 69
“Describing all the ways in which long-term research goals can be undermined by short-term Questionable Research Practices (QRPs) is outside the scope of this paper; we recommend the work of others (Ioannidis, 2005, 2012, 2014, 2016; Munafò et al., 2017) as a primer on the greatest concerns.”
BackgroundAvoidable waste of research related to outcome planning and reporting in clinical trials
Yordanov, Y., Dechartres, A., Atal, I., Tran, V.-T., Boutron, I., Créquit, P., & Ravaud, P. (2018) · BMC Medicine · doi:10.1186/s12916-018-1083-x · Cited by 76
Annual Research Review: Translational machine learning for child and adolescent psychiatry
Dwyer, D., & Koutsouleris, N. (2022) · Journal of Child Psychology and Psychiatry · doi:10.1111/jcpp.13545 · Cited by 76
Dirnagl, U., Duda, G. N., Grainger, D. W., Reinke, P., & Roubenoff, R. (2022) · Advanced Drug Delivery Reviews · doi:10.1016/j.addr.2022.114118 · Cited by 72
“Despite obvious ethical implications involving human quality of life, clinical research also has questionable quality, reliability and replicability challenges [24,33,104].”
BackgroundThe challenges with the validation of research antibodies
Voskuil, J. L. A. (2017) · F1000Research · doi:10.12688/f1000research.10851.1 · Cited by 74
Shin, H. S., Shin, H., & Shudo, Y. (2021) · Frontiers in Bioengineering and Biotechnology · doi:10.3389/fbioe.2021.673683 · Cited by 72
“…Decelerate Reliable Clinical Translations in MI Studies A rapid technological advancement has dramatically improved our understanding of human heart diseases and therapeutic development; however, the translation of these findings has not been keeping up with this trend (Ioannidis, 2005, 2016).”
BackgroundIncreasing the Impact of Materials in and beyond Bio-Nano Science
Björnmalm, M., Faria, M., & Caruso, F. (2016) · Journal of the American Chemical Society · doi:10.1021/jacs.6b08673 · Cited by 67
Pragmatic trials and implementation science: grounds for divorce?
Pawson, R. (2019) · BMC Medical Research Methodology · doi:10.1186/s12874-019-0814-9 · Cited by 65
Protect us from poor-quality medical research
ESHRE Capri Workshop Group, Ioannidis, J. P. A., Bhattacharya, S., Evers, J. L. H., der Veen, F. V., Somigliana, E., Barratt, C. L. R., Bontempi, G., Baird, D. T., Crosignani, P., et al. (2018) · Human Reproduction · doi:10.1093/humrep/dey056 · Cited by 59
Self-citationBrucar, L. R., Feczko, E., Fair, D. A., & Zilverstand, A. (2022) · Biological Psychiatry · doi:10.1016/j.biopsych.2022.12.020 · Cited by 63
Evrard, Y. A., Srivastava, A., Randjelovic, J., Doroshow, J. H., Dean, D. A., Morris, J. S., & Chuang, J. H. (2020) · Cancer Research · doi:10.1158/0008-5472.can-19-3101 · Cited by 62
Dental Research Waste in Design, Analysis, and Reporting: A Scoping Review
Pandis, N., Fleming, P. S., Katsaros, C., & Ioannidis, J. P. A. (2021) · Journal of Dental Research · doi:10.1177/0022034520962751 · Cited by 58
Self-citationMapping the multicausality of Alzheimer’s disease through group model building
Uleman, J. F., Melis, R. J. F., Quax, R., van der Zee, E. A., Thijssen, D. H. J., Dresler, M., van de Rest, O., van der Velpen, I. F., Adams, H. H. H., Schmand, B. A., et al. (2020) · GeroScience · doi:10.1007/s11357-020-00228-7 · Cited by 68
“This increased understanding may increase the prior probability of such scenarios and limit the number of failed empirical intervention studies (Ioannidis 2005, 2016).”
From ‘publish or perish’ to ‘publish for purpose’
Elbanna, S., & Child, J. (2023) · European Management Review · doi:10.1111/emre.12618 · Cited by 77
Kearney, A., Williamson, P. R., Young, B., Bagley, H., Gamble, C. L., Denegri, S., Muir, D., Simon, N. A., Thomas, S., Elliot, J., et al. (2017) · Health Expectations · doi:10.1111/hex.12583 · Cited by 65
Nudging clinicians: A systematic scoping review of the literature
Lamprell, K., Tran, Y., Arnolda, G., & Braithwaite, J. (2020) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.13401 · Cited by 61
Ioannidis, J. P. A. (2018) · Social Science & Medicine · doi:10.1016/j.socscimed.2018.04.029 · Cited by 58
Self-citationSystematic overview of hepatitis C infection in the Middle East and North Africa
Chaabna, K., Cheema, S., Abraham, A., Alrouh, H., Lowenfels, A. B., Maisonneuve, P., & Mamtani, R. (2018) · World Journal of Gastroenterology · doi:10.3748/wjg.v24.i27.3038 · Cited by 58
The AIMe registry for artificial intelligence in biomedical research
Matschinske, J., Alcaraz, N., Benis, A., Golebiewski, M., Grimm, D. G., Heumos, L., Kacprowski, T., Lazareva, O., List, M., Louadi, Z., et al. (2021) · Nature Methods · doi:10.1038/s41592-021-01241-0 · Cited by 58
Gleadhill, C., Alderman, K. B., Davidson, S. R. E., Kamper, S. J., Tutty, A., Robson, E. K., da Silva, P. V., Donald, B., Dooley, K., Manvell, J., et al. (2022) · BMC Health Services Research · doi:10.1186/s12913-022-08741-5 · Cited by 77
Software support for environmental evidence synthesis
Westgate, M. J., Haddaway, N. R., Cheng, S. H., McIntosh, E. J., Marshall, C., & Lindenmayer, D. B. (2018) · Nature Ecology & Evolution · doi:10.1038/s41559-018-0502-x · Cited by 51
Büttner, F., Toomey, E., McClean, S., Roe, M., & Delahunt, E. (2020) · British Journal of Sports Medicine · doi:10.1136/bjsports-2019-101863 · Cited by 47
Clinical Trials Overview: From Explanatory to Pragmatic Clinical Trials
Le‐Rademacher, J. G., Gunn, H., Yao, X., & Schaid, D. J. (2023) · Mayo Clinic Proceedings · doi:10.1016/j.mayocp.2023.04.013 · Cited by 61
Sand, A., Hagberg, E., & Lohmander, A. (2022) · Journal of Speech Language and Hearing Research · doi:10.1044/2021_jslhr-21-00367 · Cited by 54
Outcome reporting bias in randomized-controlled trials investigating antipsychotic drugs
Lancee, M., Lemmens, C. M. C., Kahn, R. S., Vinkers, C. H., & Luykx, J. J. (2017) · Translational Psychiatry · doi:10.1038/tp.2017.203 · Cited by 51
Oliver, D., Spada, G., Colling, C., Broadbent, M., Baldwin, H., Patel, R., Stewart, R., Stahl, D., Dobson, R., McGuire, P., et al. (2021) · Schizophrenia Research · doi:10.1016/j.schres.2020.05.007 · Cited by 49
“…research is the scientific study of methods translating research findings into practical, useful outcomes; it seeks to understand and work within real-world conditions, rather than trying to control for them (Ioannidis, 2016; Peters et al., 2014; Rapport et al., 2018).”
Smit, L. C., Schuurmans, M. J., Blom, J. W., Fabbricotti, I. N., Jansen, A. P. D., Kempen, G. I. J. M., Koopmans, R. T. C. M., Looman, W., Melis, R. J. F., Metzelthin, S., et al. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.12.013 · Cited by 54
“Examining all components in-depth and fitting them to the context may lead to optimally designed interventions that contribute to improved patient outcomes [59].”
BackgroundMethodTowards robotic laboratory automation Plug & Play: The “LAPP” framework
Wolf, Á., Wolton, D., Trapl, J., Janda, J., Romeder-Finger, S., Gatternig, T., Farcet, J., Galambos, P., & Széll, K. (2022) · SLAS TECHNOLOGY · doi:10.1016/j.slast.2021.11.003 · Cited by 58
“The LAPP sequence (1) Generate map, (2) Detect barcode, (3) Detect fiducial pose, (4) Upload barcode ID to the scheduler, (5) Request device information from the cloud database, (6) Instanciate digital twin from the cloud database, (7) Keep digital twin updated, (8a) Device control, (8b) Robot control, (9) Robotic man…”
MethodHung, L., Phinney, A., Chaudhury, H., Rodney, P., Tabamo, J., & Bohl, D. (2018) · International Journal of Qualitative Methods · doi:10.1177/1609406918769444 · Cited by 48
“Ioannidis (2016) asked why most clinical research is not useful and found clinical research does not always address real practice problems and rarely reflect patient priorities.”
BackgroundCunningham‐Erves, J., Barajas, C. P., Mayo‐Gamble, T. L., McAfee, C. R., Hull, P. C., Sanderson, M., Canedo, J., Beard, K. R., & Wilkins, C. H. (2020) · BMC Public Health · doi:10.1186/s12889-020-08939-4 · Cited by 52
On the objectivity, reliability, and validity of deep learning enabled bioimage analyses
Segebarth, D., Griebel, M., Stein, N., von Collenberg, C. R., Martin, C., Fiedler, D., Comeras, L. B., Sah, A., Schoeffler, V., Lüffe, T., et al. (2020) · eLife · doi:10.7554/elife.59780 · Cited by 59
“…to the discussion about a reproducibility crisis in the fields of biology, medicine and artificial intelligence (Siebert et al., 2015; Baker, 2016; Ioannidis, 2016; Hutson, 2018; Fanelli, 2018; Chen et al., 2019), these limitations indicate that DL could aggravate this crisis by adding even more…”
BackgroundInnovation crisis in the pharmaceutical industry? A survey
Laermann-Nguyen, U., & Backfisch, M. (2021) · SN Business & Economics · doi:10.1007/s43546-021-00163-5 · Cited by 54
Porche, M. V., Folk, J. B., Tolou-Shams, M., & Fortuna, L. R. (2022) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2022.867460 · Cited by 45
“Strong arguments decrying the overwhelming lack of useful clinical research highlights limited patient-centeredness (16).”
BackgroundKronbichler, A., Effenberger, M., Eisenhut, M., Lee, K. H., & Shin, J. I. (2020) · Autoimmunity Reviews · doi:10.1016/j.autrev.2020.102570 · Cited by 51
“Furthermore, it has been claimed that most published research findings are false and not useful [16,17] (though this should also be interpreted with caution) and numerously published meta-analyses are also misleading [18].”
Kent, S., Becker, F., Feenstra, T., Tran‐Duy, A., Schlackow, I., Tew, M., Zhang, P., Ye, W., Lizheng, S., Herman, W. H., et al. (2019) · PharmacoEconomics · doi:10.1007/s40273-019-00825-1 · Cited by 52
Hijacked evidence-based medicine: stay the course and throw the pirates overboard
Ioannidis, J. P. A. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.02.001 · Cited by 42
“But we need clinicians and even more than clinicians we need patients and healthy people to tell us what the real problems are, what matters to them [18].”
BackgroundSelf-citationRabeyron, T. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.562992 · Cited by 55
Increasing value and reducing waste in stroke research
Berge, E., Salman, R. A.-S., van der Worp, H. B., Stapf, C., Sandercock, P., Sprigg, N., Macleod, M. R., Kelly, P. J., Nederkoorn, P. J., Ford, G. A., et al. (2017) · The Lancet Neurology · doi:10.1016/s1474-4422(17)30078-9 · Cited by 41
Morris, A. H., Stagg, B. C., Lanspa, M. J., Orme, J. F., Clemmer, T. P., Weaver, L. K., Thomas, F., Grissom, C. K., Hirshberg, E. L., East, T. D., et al. (2021) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaa294 · Cited by 44
Evidence-Based Pragmatic Psychiatry—A Call to Action
Paulus, M. P. (2017) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2017.2439 · Cited by 44
Yadav, U. N., Lloyd, J. E., Baral, K. P., Bhatta, N., Mehata, S., & Harris, M. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-045175 · Cited by 43
Evidence-based medicine and big genomic data
Ioannidis, J. P. A., & Khoury, M. J. (2018) · Human Molecular Genetics · doi:10.1093/hmg/ddy065 · Cited by 47
Self-citationL’évaluation et l’efficacité des psychothérapies psychanalytiques et de la psychanalyse
Rabeyron, T. (2020) · L Évolution Psychiatrique · doi:10.1016/j.evopsy.2020.07.003 · Cited by 48
Fontaine, G., Maheu‐Cadotte, M., Lavallée, A., Mailhot, T., Rouleau, G., Bouix‐Picasso, J., & Bourbonnais, A. (2019) · JMIR Public Health and Surveillance · doi:10.2196/14447 · Cited by 38
“Indeed, critics point that health scientists often fail in either addressing an important health problem, generating new knowledge, or in producing rigorous research [39].”
Some Myths in Dentin Bonding: An Evidence-Based Perspective
Reis, A., Loguércio, A. D., Favoreto, M. W., & Chibinski, A. C. R. (2023) · Journal of Dental Research · doi:10.1177/00220345221146714 · Cited by 42
Scientific rigor and credibility in the nutrition research landscape
Kroeger, C. M., Garza, C., Lynch, C. J., Myers, E. F., Rowe, S. B., Schneeman, B. O., Sharma, A. M., & Allison, D. B. (2018) · American Journal of Clinical Nutrition · doi:10.1093/ajcn/nqx067 · Cited by 45
Caru, M., Lévesque, A., Rao, P., Dandekar, S. C., Terry, C., Brown, V. I., McGregor, L. M., & Schmitz, K. H. (2022) · Critical Reviews in Oncology/Hematology · doi:10.1016/j.critrevonc.2022.103620 · Cited by 42
Co-creating health: more than a dream
Richards, T., Snow, R., & Schroter, S. (2016) · BMJ · doi:10.1136/bmj.i4550 · Cited by 45
Challenges and enablers of the embedded researcher model
Coates, D., & Mickan, S. M. (2020) · Journal of Health Organization and Management · doi:10.1108/jhom-02-2020-0043 · Cited by 38
Emmenegger, M., De Cecco, E., Lamparter, D., Jacquat, R. P. B., Riou, J., Menges, D., Ballouz, T., Ebner, D., Schneider, M. M., Morales, I. C., et al. (2023) · iScience · doi:10.1016/j.isci.2023.105928 · Cited by 39
“Limitations inherent to our pragmatic study design 79 relying on ‘further use’ of bio-specimens were addressed where appropriate, e.g. by adjusting the age and sex distribution of the study collective to the adult population of the canton of Zurich.”
Failing to encourage physical activity with wearable technology: what next?
Ellis, D. A., & Piwek, L. (2018) · Journal of the Royal Society of Medicine · doi:10.1177/0141076818788856 · Cited by 38
Olszewski, D., Lu, A., Stillman, C., Warren, K., Kitroser, C., Pascual, A., Ukirde, D., Butler, K. R. B., & Traynor, P. (2023) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3576915.3623130 · Cited by 85
Practical guide for identifying unmet clinical needs for biomarkers.
Monaghan, P. J., Robinson, S. M., Rajdl, D., Bossuyt, P. M., Sandberg, S., John, A. S., O’Kane, M., Lennartz, L., Röddiger, R., Lord, S. J., et al. (2018) · PubMed Central · Cited by 44
Protein Biomarkers for Early Detection of Pancreatic Ductal Adenocarcinoma: Progress and Challenges
Root, A., Allen, P. J., Tempst, P., & Yu, K. H. (2018) · Cancers · doi:10.3390/cancers10030067 · Cited by 36
Lanspa, M. J., Gong, M. N., Schoenfeld, D., Lee, K. T., Grissom, C. K., Hou, P. C., Neto, A. S., Brown, S. M., Iwashyna, T. J., Yealy, D. M., et al. (2018) · Annals of the American Thoracic Society · doi:10.1513/annalsats.201807-459oc · Cited by 43
Azar, M., Riehm, K. E., Saadat, N., Sanchez, T., Chiovitti, M. J., Qi, L., Rice, D. B., Levis, B., Fedoruk, C., Levis, A. W., et al. (2019) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2018.8009 · Cited by 38
Blockchain for Science and Knowledge Creation
Bartling, S. H. (2018) · Springer eBooks · doi:10.1007/978-3-662-57611-3_10 · Cited by 35
Riley, S., Brooks, J., Goodman, S., Cahill, S., Branney, P., Treharne, G. J., & Sullivan, C. (2019) · Qualitative Research in Psychology · doi:10.1080/14780887.2019.1605275 · Cited by 31
How Routinely Collected Data for Randomized Trials Provide Long-term Randomized Real-World Evidence
Hemkens, L. G. (2018) · JAMA Network Open · doi:10.1001/jamanetworkopen.2018.6014 · Cited by 33
Redundant trials can be prevented, if the EU clinical trial regulation is applied duly
Kim, D., & Hasford, J. (2020) · BMC Medical Ethics · doi:10.1186/s12910-020-00536-9 · Cited by 33
“Apart from obvious ethical concerns, studies where the use of placebo is unjustified can represent a significant source of waste as they do not generate knowledge regarding comparative benefits and risks of medical interventions [90].”
BackgroundHow to do and report survey studies robustly: A helpful mnemonic SURVEY
Latour, J. M., & Tume, L. N. (2021) · Nursing in Critical Care · doi:10.1111/nicc.12669 · Cited by 33
The ultimate power play in research - partnering with patients, partnering with power
Richards, D. P., Bowden, J., Gee, P. O., Haagaard, A., Kothari, A. R., McKinnon, A., Primeau, C. A., Tricco, A. C., Wang, E., Woolley, K. L., et al. (2025) · Research Involvement and Engagement · doi:10.1186/s40900-025-00745-9 · Cited by 31
“This reflects a longstanding recognition of the real-world challenges in health research, such as limitations in study designs, difficulty in translating findings into practice, and misalignment with patient priorities [23, 24].”
Saesen, R., Lejeune, S., Quaglio, G., Lacombe, D., & Huys, I. (2020) · Frontiers in Pharmacology · doi:10.3389/fphar.2020.00043 · Cited by 30
Opportunities and barriers for pragmatic embedded trials: Triumphs and tribulations
Ramsberg, J., & Platt, R. G. (2017) · Learning Health Systems · doi:10.1002/lrh2.10044 · Cited by 29
“While traditional RCTs have strong internal validity, they are expensive and often have limited generalizability.(4) Observational studies, while less expensive and often applicable to broad populations, are prone to selection bias and confounding.”
BackgroundClinical Research and Regulatory Affairs: Skills and Tools in Pharmacy Education
Herdeiro, M. T., Gouveia, N., & Roque, F. (2020) · Advances in medical education, research, and ethics (AMERE) book series · doi:10.4018/978-1-7998-4486-0.ch008 · Cited by 28
Metadata
- Authors
- Ioannidis, J. P. A.
- Affiliations
- Stanford Medicine (US)
- Corresponding author
- Ioannidis, John P. A.
- Venue
- PLOS Medicine
- Publisher
- Public Library of Science (PLoS)
- Published
- 2016-06-21
- Volume
- 13
- Issue
- 6
- Pages
- e1002049
- ISSN
- 1549-1277 (ISSN-L)
- 1549-1676
- PMC
- PMC4915619
- OpenAlex
- W2469474955
- Cited by
- 655
- References
- 56
- Citations by year
- 2026: 68 · 2025: 55 · 2024: 48 · 2023: 60 · 2022: 55 · 2021: 77 · 2020: 81 · 2019: 59 · 2018: 69 · 2017: 65 · 2016: 18
- FWCI
- 69.75
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS Medicine (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Funders
- Patient-Centered Outcomes Research Institute
- Laura and John Arnold Foundation
- National Institutes of Health
- Topics
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Health and Medical Research Impacts
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Health Professions › General Health Professions › Healthcare cost, quality, practices
- Keywords
- research value, research reform, problem-based research, information gain, pragmatism, patient-oriented research, value for money, feasibility, research transparency
- MeSH terms
- Humans, Biomedical Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.