Why Most Clinical Research Is Not Useful
Open accessThe DOI matches its Crossref record; 56 of the 56 references it cites were found in a registry or an index.
Abstract
John Ioannidis argues that problem base, context placement, information gain, pragmatism, patient centeredness, value for money, feasibility, and transparency define useful clinical research. He suggests most clinical research is not useful and reform is overdue.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Neglecting the actual development phase can be problematic, as a poor problem definition, insufficient attention to existing evidence and context needs, a missing model underlying the intervention, and an unsound selection of hypothesized change mechanisms can lead to inefficient treatments (30–32).”
From Developing a mechanism-based therapy for acute psychiatric inpatients with psychotic symptoms: an Intervention Mapping approach (2023) · cites this paper as [30]
Experimental — accuracy not yet measuredIn the citing paper
“Systematic review guidelines typically advocate that all relevant studies must be included for the conclusions of that review to be valid 22”
From Software support for environmental evidence synthesis (2018) · cites this paper as [22]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
Self-citationWhy Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · cited 1×
“I have previously written about why most published research is false [2] and how to make more of it true Summary Points”
BackgroundSelf-citationTunis (2003) · doi:10.1001/jama.290.12.1624 · Cited by 1,724 · cited 1×
“Pragmatism has long been advocated in clinical research [15], but it is rare.”
BackgroundSurrogate End Points in Clinical Trials: Are We Being Misled?
Fleming, T. R., & DeMets, D. L. (1996) · Annals of Internal Medicine · doi:10.7326/0003-4819-125-7-199610010-00011 · Cited by 1,651 · cited 1×
“Nonvalidated surrogate outcomes lacking clinical insight [13] and composite outcomes that combine outcomes of very different clinical portent [14] are often utilized so that authors can claim that clinical studies are well powered.”
BackgroundIncreasing value and reducing waste in research design, conduct, and analysis
Ioannidis, J. P. A., Greenland, S., Hlatky, M. A., Khoury, M. J., Macleod, M. R., Moher, D., Schulz, K. F., & Tibshirani, R. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62227-8 · Cited by 1,597
Self-citationReducing waste from incomplete or unusable reports of biomedical research
Glasziou, P., Altman, D. G., Bossuyt, P., Boutron, I., Clarke, M., Julious, S., Michie, S., Moher, D., & Wager, E. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62228-x · Cited by 1,319
Biomedical research: increasing value, reducing waste.
Macleod, M. R., Michie, S., Roberts, I., Dirnagl, U., Chalmers, I., Ioannidis, J. P. A., Salman, R. A.-S., Chan, A.-W., & Glasziou, P. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62329-6 · Cited by 1,003
Self-citationMoynihan, R., G⊘tzsche, P. C., Heath, I., & Henry, D. (2002) · BMJ · doi:10.1136/bmj.324.7342.886 · Cited by 900
Increasing value and reducing waste: addressing inaccessible research
Chan, A.-W., Song, F., Vickers, A., Jefferson, T., Dickersin, K., Gøtzsche, P. C., Krumholz, H. M., Ghersi, D., & van der Worp, H. B. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62296-5 · Cited by 887
How to Make More Published Research True
Ioannidis, J. P. A. (2014) · PLoS Medicine · doi:10.1371/journal.pmed.1001747 · Cited by 760 · cited 1×
“Joint efforts by multiple stakeholders may yield solutions that are more likely to be more widely adopted and thus successful [3].”
BackgroundSelf-citation
Cited by
Items 101–200 of 667
Nordhaus, W. D., & Moffat, A. (2017) · SSRN Electronic Journal · doi:10.2139/ssrn.3009427 · Cited by 28
Co-design for stroke intervention development: Results of a scoping review
Singh, H., Benn, N., Fung, A., Kokorelias, K. M., Martyniuk, J., Nelson, M. L. A., Colquhoun, H. L., Cameron, J. I., Munce, S., Saragosa, M., et al. (2024) · PLoS ONE · doi:10.1371/journal.pone.0297162 · Cited by 33
Evidence maps: a tool to guide research agenda setting
Alahdab, F., & Murad, M. H. (2019) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2018-111137 · Cited by 29
What Is Post-normal Science? A Personal Encounter
Saltelli, A. (2022) · Foundations of Science · doi:10.1007/s10699-023-09932-x · Cited by 26
The perilous path from publication to practice
Chekroud, A. M., & Koutsouleris, N. (2017) · Molecular Psychiatry · doi:10.1038/mp.2017.227 · Cited by 30
Rietkerk, W., Smit, M. F., Wynia, K., Slaets, J. P. J., Zuidema, S. U., & Gerritsen, D. L. (2019) · BMC Geriatrics · doi:10.1186/s12877-018-1025-7 · Cited by 30
der Elst, K. V., Verschueren, P., Stouten, V., Pazmiño, S., Groef, A. D., Cock, D. D., Joly, J., Moons, P., & Westhovens, R. (2019) · Arthritis Care & Research · doi:10.1002/acr.23900 · Cited by 32
“This diversity among patients entails the need for an alternative way of using PROs, if we want to study questions that are meaningful to patients with early RA (12).”
BackgroundArtificial intelligence and psychotherapy: A counterpoint
Richards, D. (2024) · Counselling and Psychotherapy Research · doi:10.1002/capr.12758 · Cited by 50
“The approach has faced criticism for its limited replicability and where reported results may not be clinically meaningful (Dwyer et al., 2018; Ioannidis, 2016 ).”
Makridakis, S., Kirkham, R., Wakefield, A. B., Papadaki, M., Kirkham, J., & Long, L. (2018) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2017.11.003 · Cited by 33
O’Neil, J., McEwen, D., Bel, M. J. D., Jo, D., Thevenot, O., MacKiddie, O. S., & Brosseau, L. (2018) · Clinical Rehabilitation · doi:10.1177/0269215518763714 · Cited by 26
Kelter, R. (2021) · BMC Medical Research Methodology · doi:10.1186/s12874-021-01341-7 · Cited by 33
“Background Hypothesis testing is among the most widely established statistical methods in the biomedical sciences [1, 2].”
BackgroundPrabu Raja, G., Bhat, S. N., Fernández‐de‐las‐Peñas, C., Gangavelli, R., Davis, F., Shankar, R., & Anupama Prabhu, B. (2021) · Trials · doi:10.1186/s13063-021-05533-w · Cited by 26
“Ioannidis JP.”
BackgroundMethodInfluentialThe Social Value Misconception in Clinical Research
Earl, J., Dawson, L., & Rid, A. (2024) · The American Journal of Bioethics · doi:10.1080/15265161.2024.2371119 · Cited by 27
“The clinical research enterprise also can develop and implement tools that would improve predictions and communication about expected social value of research studies and programs (Binik and Hey 2019; Chalmers et al. 2014; Ioannidis 2016; Minelli and Baio 2015).”
Research integrity in clinical trials: innocent errors and spin versus scientific misconduct
Núñez‐Núñez, M., Andrews, J. C., Fawzy, M. M., Bueno‐Cavanillas, A., & Khan, K. S. (2022) · Current Opinion in Obstetrics & Gynecology · doi:10.1097/gco.0000000000000807 · Cited by 24
Clausen, M. B., Bandholm, T., Rathleff, M. S., Christensen, K. B., Zebis, M. K., Graven‐Nielsen, T., Hölmich, P., & Thorborg, K. (2018) · Trials · doi:10.1186/s13063-018-2509-7 · Cited by 27
“Table 6 Features to consider in appraising whether clinical research is useful (retrieved from Ioannides [60])”
BackgroundCamillo, F. D., Grimaldi, D. A., Cattarinussi, G., Giorgio, A. D., Locatelli, C., Khuntia, A. T., Enrico, P., Brambilla, P., Koutsouleris, N., & Sambataro, F. (2024) · Psychiatry and Clinical Neurosciences · doi:10.1111/pcn.13736 · Cited by 32
“13 Large sample sizes may yield statistical signi fi cance, but this does not necessarily translate into meaningful differences at the individual level, posing a challenge in translating results at the group level.”
Ming, Z., Moalin, M., & Haenen, G. R. M. M. (2019) · International Journal of Molecular Sciences · doi:10.3390/ijms20092333 · Cited by 24
“In addition, critical examination indicates that many controlled clinical trials, the fundamental of evidence based medicine, may actually give false results [7,18].”
BackgroundThe cooperative revolution reaches clinical psychology and psychotherapy: An example from Germany
Margraf, J., Hoyer, J., Fydrich, T., In‐Albon, T., Lincoln, T. M., Lutz, W., Schlarb, A. A., Schöttke, H., Willutzki, U., & Velten, J. (2021) · Clinical Psychology in Europe · doi:10.32872/cpe.4459 · Cited by 25
“616) we have a situation, where “most published research findings are false” (Ioannidis, 2005) and “most clinical research is not useful” (Ioannidis, 2016).”
BackgroundVelázquez‐Campoy, A., Claro, B., Abián, O., Höring, J., Bourlon, L., Claveria‐Gimeno, R., Ennifar, E., England, P., Chaires, J. B., Wu, D., et al. (2021) · European Biophysics Journal · doi:10.1007/s00249-021-01523-7 · Cited by 23
“The lack of reproducibility is a matter of severe concern among researchers from academia and industry (Baker 2016; Begley and Ellis 2012; Ioannidis 2016; Mobley et al. 2013; Prinz et al. 2011), but there is still an ongoing debate on the criteria that should be used to Special Issue: COST Action CA15126, MOBIEU: Betw…”
BackgroundCo-design of Guidance for Patient and Public Involvement in Psychedelic Research
Close, J. B., Bornemann, J., Piggin, M., Jayacodi, S., Luan, L. X., Carhart-Harris, R. L., & Spriggs, M. J. (2021) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2021.727496 · Cited by 27
“Neglecting to recognize the reality of how data will be used downstream can lead to impractical “blue sky” research which ultimately fails to get adopted (11).”
BackgroundGan, D. Z. Q., McGillivray, L., Larsen, M. E., Bloomfield, T., & Torok, M. H. (2023) · JMIR Formative Research · doi:10.2196/45234 · Cited by 26
von Niederhäusern, B., Guyatt, G. H., Briel, M., & Pauli‐Magnus, C. (2018) · PLoS Medicine · doi:10.1371/journal.pmed.1002580 · Cited by 28
Oliveira, L., Portugal, L. C. L., Pereira, M. G., Chase, H. W., Bertocci, M. A., Stiffler, R. S., Greenberg, T., Bebko, G. M., Lockovich, J., Aslam, H. A., et al. (2019) · Biological Psychiatry Cognitive Neuroscience and Neuroimaging · doi:10.1016/j.bpsc.2019.04.005 · Cited by 27
“As highlighted previously (30,31), of 49 clinical studies with more than 1000 citations, findings from only 44% of these studies were replicated.”
BackgroundResultTournier, T., Hendriks, A. H. C., Jahoda, A. J., Hastings, R. P., & Embregts, P. J. C. M. (2020) · Journal of Policy and Practice in Intellectual Disabilities · doi:10.1111/jppi.12333 · Cited by 26
“However, aspects of evidence-based interventions may not always be transferable (Gabbay & Le May, 2004; Ioannidis, 2016) and the social and cultural context may have a significant influence on the use and development of care interventions.”
BackgroundByrne, M., McSharry, J., Meade, O., Lavoie, K., & Bacon, S. (2020) · Trials · doi:10.1186/s13063-020-04235-z · Cited by 25
Diet and Endometriosis: An Umbrella Review
Neri, L. D. C. L., Quintiero, F., Fiorini, S., Guglielmetti, M., Ferraro, O. E., Tagliabue, A., Gardella, B., & Ferraris, C. (2025) · Foods · doi:10.3390/foods14122087 · Cited by 30
“The criteria used to classify evidence (Ioannidis’s criteria) were as follows [12]: The heterogeneity among studies using the I2 index; 2.”
Assessing the usefulness of randomised trials in obstetrics and gynaecology
Hooft, J. V. ’., van Dijk, C. E., Axfors, C., Alfirević, Ž., Oudijk, M. A., Khan, K. S., Mol, B. W., Bossuyt, P. M., & Ioannidis, J. P. A. (2023) · BJOG An International Journal of Obstetrics & Gynaecology · doi:10.1111/1471-0528.17411 · Cited by 25
Self-citationAla‐Korpela, M., & Smith, G. D. (2016) · International Journal of Epidemiology · doi:10.1093/ije/dyw305 · Cited by 24
“Ioannidis has recently reflected upon biomedical innovations and scientific peer review.(89,101) He makes the point that ’stealth research’ is prone to create ambiguity about what evidence can be trusted in a mix of (possibly) ground-breaking ideas, aggressive corporate announcements and mass media hype.”
BackgroundInfluentialXu, H. G., Ullman, A. J., Rickard, C. M., & Johnston, A. N. B. (2023) · International Emergency Nursing · doi:10.1016/j.ienj.2023.101366 · Cited by 23
Malterud, K., Aamland, A., & Iden, K. R. (2018) · BMC Family Practice · doi:10.1186/s12875-018-0778-6 · Cited by 22
“to substantial shifts in clinical practice [2].”
BackgroundDanell, J. A. B., Danell, R., & Vuolanto, P. H. (2020) · Journal of Scientometric Research · doi:10.5530/jscires.9.3.34 · Cited by 21
“Although there is an ongoing debate on the value of clinical studies,[49-51] there is no doubt that clinical support is crucial to have new therapies accepted in medical practice.”
BackgroundSystematic reviews of therapeutic interventions frequently consider patient-important outcomes
Ameur, H., Ravaud, P., Fayard, F., Riveros, C., & Dechartres, A. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.01.006 · Cited by 23
Nutzenbewertung digitaler Gesundheitsanwendungen – Herausforderungen und Möglichkeiten
Hemkens, L. G. (2021) · Bundesgesundheitsblatt - Gesundheitsforschung - Gesundheitsschutz · doi:10.1007/s00103-021-03413-x · Cited by 24
“Ein Großteil aller RCTs ist in der Tat ungeeignet, die Fragen zu beantworten, die fürEntscheidungsträger,Klinikerund Patienten relevant sind, und kann daher in vielerlei Hinsicht als nutzlos angesehenwerden[21].”
BackgroundSensor Reproducibility Analysis: Challenges and Potential Solutions
Sekhar, P. K., Billey, W., Begay, M., Thomas, B., Woody, C., & Soundappan, T. (2024) · ECS Sensors Plus · doi:10.1149/2754-2726/ad9936 · Cited by 23
Hensher, M., Kish, K., Farley, J., Quilley, S., & Zywert, K. (2020) · Global Sustainability · doi:10.1017/sus.2020.21 · Cited by 20
“The wisdom of this reliance has been under question for several years in the light of the emerging ‘crisis of reproducibility’ affecting medicine and many other scientific disciplines (Ioannidis, 2005, 2016; Munafò et al., 2017).”
BackgroundWhat Really Matters for Cancer Care – Health Systems Strengthening or Technological Innovation?
Aggarwal, A. K., Lievens, Y. N., Sullivan, R. J., & Nolte, E. (2022) · Clinical Oncology · doi:10.1016/j.clon.2022.02.012 · Cited by 22
Chauvin, A., Truchot, J., Bafeta, A., Pateron, D., Plaisance, P. I., & Yordanov, Y. (2017) · Internal and Emergency Medicine · doi:10.1007/s11739-017-1770-1 · Cited by 24
“Even if poor reporting does not mean poor methods [31], adequate reporting allows readers to assess the strength and weakness of studies and improve the replication of interventions in daily practice [32, 33].”
BackgroundCOLLECT , a collaborative database for pregnancy and placental research studies worldwide
Myers, J., Myatt, L., Roberts, J., & Redman, C. W. G. (2018) · BJOG An International Journal of Obstetrics & Gynaecology · doi:10.1111/1471-0528.15393 · Cited by 20
Amirmahani, F., Ebrahimi, N., Molaei, F., Faghihkhorasani, F., Goharrizi, K. J., Mirtaghi, S. M., Boroujeni, M. B., & Hamblin, M. R. (2021) · Annals of the New York Academy of Sciences · doi:10.1111/nyas.14544 · Cited by 21
“13,14 In the current era of astronomical costs, 15 high attrition levels, and the long time necessary for new drugs to be approved, 16,17 and in light of limited resources, translational research requires a renewed perspective.”
BackgroundDetermining the Value-Efficiency of Respiratory Care
Chatburn, R. L., Ford, R. M., & Kauffman, G. W. (2021) · Respiratory Care · doi:10.4187/respcare.09100 · Cited by 20
“15 After decades of effort, the AARC has only 7 evidence-based clinical practice guidelines on only 5 topics: management of tracheostomy, airway clearance therapies, inhaled nitric oxide, care of the ventilator circuit, and discontinuing ventilatory support.”
Lacombe, D., Bogaerts, J., TOMBAL, B. F., Maignen, F. M., Osipenko, L., Sullivan, R. J., & Golfinopoulos, V. (2019) · Molecular Oncology · doi:10.1002/1878-0261.12431 · Cited by 21
Laustsen, C. E., Petersson, P., Westergren, A., & Haak, M. (2021) · Health Research Policy and Systems · doi:10.1186/s12961-021-00763-5 · Cited by 27
“Even when published, some research might not be useful in practice if the result lacks relevance for practice, for patients, and is difficult to implement [7].”
BackgroundChoosing Wisely in clinical practice: Embracing critical thinking, striving for safer care
Furlan, L., Francesco, P. D., Costantino, G., & Montano, N. (2022) · Journal of Internal Medicine · doi:10.1111/joim.13472 · Cited by 23
Machine learning for metabolomics research in drug discovery
Martinelli, D. D. (2023) · Intelligence-Based Medicine · doi:10.1016/j.ibmed.2023.100101 · Cited by 24
The Promotion of Avoidable Ignorance in the British Journal of Social Work
Gambrill, E. D. (2018) · Research on Social Work Practice · doi:10.1177/1049731518763953 · Cited by 23
Reimagining Global Oncology Clinical Trials for the Postpandemic Era: A Call to Arms
Saini, K. S., Heras, B. D. L., Plummer, R., Moreno, V., Romano, M., de Castro, J., Aftimos, P. G., Fredriksson, J., Bhattacharyya, G. S., Olivo, M. S., et al. (2020) · JCO Global Oncology · doi:10.1200/go.20.00346 · Cited by 20
“The process of developing new anticancer therapeutics has been considered by some to be expensive,(1) time consuming,(2) bureaucratic,(3) and, to some extent, inefficient.(4) The coronavirus disease 2019 (COVID-19) pandemic has significantly affected clinical oncology studies(5,6) and underlined the need to embrace an…”
BackgroundDiagnostic Biomarkers: Are We Moving from Discovery to Clinical Application?
Parker, L. A., Chilet‐Rosell, E., Hernández‐Aguado, I., Pastor‐Valero, M., Gea, S., & Lumbreras, B. (2018) · Clinical Chemistry · doi:10.1373/clinchem.2018.292854 · Cited by 22
Taylor, S. P., & Kowalkowski, M. A. (2020) · Journal of General Internal Medicine · doi:10.1007/s11606-020-06220-3 · Cited by 22
Lin, T., Chueh, T., & Hung, T. (2023) · Sports Medicine - Open · doi:10.1186/s40798-023-00640-1 · Cited by 19
Madden, M., & Stark, J. F. (2019) · Journal of Tissue Viability · doi:10.1016/j.jtv.2019.03.003 · Cited by 19
“Ioannidis and others have identified the failure to take account of patient perspectives when designing research outcomes as a common flaw in most clinical research [85].”
BackgroundGarg, S., Huifu, H., Kaul, S. C., & Wadhwa, R. (2018) · BMC Research Notes · doi:10.1186/s13104-018-3512-5 · Cited by 19
Chinapaw, M. J. M., Klaufus, L. H., Oyeyemi, A. L., Draper, C. E., Palmeira, A. L., Silva, M. N., Belle, S. V., Pawlowski, C. S., Schipperijn, J., & Altenburg, T. M. (2024) · BMJ Open · doi:10.1136/bmjopen-2024-084657 · Cited by 20
Costa, D. L. D., Corlett, S. A., & Dodds, L. J. (2019) · International Journal of Pharmacy Practice · doi:10.1111/ijpp.12587 · Cited by 18
Designing clinically useful psychopharmacological trials: challenges and ways forward
Chevance, A., Ravaud, P., Cornelius, V. R., Mayo‐Wilson, E., & Furukawa, T. A. (2022) · The Lancet Psychiatry · doi:10.1016/s2215-0366(22)00041-4 · Cited by 24
Johnson, M. I., Paley, C. A., Wittkopf, P. G., Mulvey, M. R., & Jones, G. (2022) · Medicina · doi:10.3390/medicina58060803 · Cited by 19
“Ioannidis raised concern about the proliferation of the publication of useless and meaningless research associated with inadequate sample sizes, “Most clinical research therefore fails to be useful not because of its findings but because of its design” [639] p.”
BackgroundPesce, F., & Cheungpasitporn, W. (2025) · Renal Failure · doi:10.1080/0886022x.2025.2581933 · Cited by 21
Preserving equipoise and performing randomised trials for COVID-19 social distancing interventions
Cristea, I. A., Naudet, F., & Ioannidis, J. P. A. (2020) · Epidemiology and Psychiatric Sciences · doi:10.1017/s2045796020000992 · Cited by 17
“We should acknowledge that most studies, both RCTs and others, indeed have limited usefulness (Ioannidis, 2016).”
BackgroundSelf-citationShee, A. W., Donaldson, A., Clapham, R. P., Aitken, J. C., Ananthapavan, J., Ugalde, A., Versace, V. L., & Namara, K. M. (2024) · Health Research Policy and Systems · doi:10.1186/s12961-024-01163-1 · Cited by 21
“Their involvement can help ensure healthcare delivery and health interventions are pragmatic, patient-centred and feasible [6].”
BackgroundNIH behavioral and social sciences research support: 1980–2016.
Kaplan, R. M., Johnson, S. B., & Kobor, P. C. (2017) · American Psychologist · doi:10.1037/amp0000222 · Cited by 17
“Most major clinical research programs in these areas have failed to produce successful therapeutics (Ioannidis, 2016).”
Stavropoulou, C., Somai, M. M., & Ioannidis, J. P. A. (2019) · PLoS ONE · doi:10.1371/journal.pone.0211460 · Cited by 16
Self-citationScorza, P., Monk, C. E., Lee, S., Feng, T., Berry, O. O., & Werner, E. A. (2020) · American Journal of Obstetrics & Gynecology MFM · doi:10.1016/j.ajogmf.2020.100230 · Cited by 18
Needs and Challenges for Radiation Oncology in the Era of Precision Medicine
Quon, H., McNutt, T., Lee, J., Bowers, M., Jiang, W., Lakshminarayanan, P., Cheng, Z., Han, P., Hui, X., Shah, V., et al. (2018) · International Journal of Radiation Oncology*Biology*Physics · doi:10.1016/j.ijrobp.2018.11.017 · Cited by 17
Proposing a new approach to funding behavioural interventions using iterative methods
Kwaśnicka, D., ten Hoor, G. A., Hekler, E. B., Hagger, M. S., & Kok, G. (2021) · Psychology and Health · doi:10.1080/08870446.2021.1945061 · Cited by 16
“However, many funders adopt a model primarily focused on supporting large-scale efficacy testing of behavioural interventions may lead to biases in the evidence base (Ioannidis, 2016), particularly when underlying assumptions have not been adequately tested a priori.”
BackgroundOptimizing Expectations via Mobile Apps: A New Approach for Examining and Enhancing Placebo Effects
Gruszka, P., Burger, C., & Jensen, M. P. (2019) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2019.00365 · Cited by 15
“Thus, effects from the lab can be extended to the natural surroundings of patients, thereby increasing the potential usefulness of interventions (64).”
BackgroundInfluentialThe application of fNIRS in studies on occupational workload: a systematic review
Gemmerich, R., Müller, O., & Schaller, A. (2025) · Frontiers in Public Health · doi:10.3389/fpubh.2025.1560605 · Cited by 18
Meta-research in pharmacy: Time for a look in the mirror
Harpe, S. E. (2021) · Research in Social and Administrative Pharmacy · doi:10.1016/j.sapharm.2021.04.006 · Cited by 15
Lusambili, A. M., Khaemba, P., Agoi, F., Oguna, M., Nakstad, B., Scorgie, F., Filippi, V., Hess, J. J., Roos, N., Chersich, M., et al. (2023) · Frontiers in Public Health · doi:10.3389/fpubh.2023.1146048 · Cited by 15
“Many health interventions tested in clinical settings are not as effective when translated into real-life practice (12).”
Flavonol Technology: From the Compounds’ Chemistry to Clinical Research
Przybylski, T., Czerniel, J., Dobrosielski, J., & Stawny, M. (2025) · Molecules · doi:10.3390/molecules30153113 · Cited by 15
“As a result, systematic errors may occur, which make it impossible to reproduce the process reliably and lead to incorrect conclusions [150,154].”
Subjecting the ankle-brachial index to timely scrutiny: is it time to say goodbye to the ABI?
Trevethan, R. (2017) · Scandinavian Journal of Clinical and Laboratory Investigation · doi:10.1080/00365513.2017.1416665 · Cited by 15
“Furthermore, sometimes unimpressive results appear to have been interpreted to appeal to positive publication bias and, on close inspection, some of the research appears to attract the scathing criticisms leveled at research by Ionnides [10,11].”
ResultMoore, J. E., Liu, B., Khan, S., Harris, C., Ewusie, J. E., Hamid, J. S., & Straus, S. E. (2019) · BMC Geriatrics · doi:10.1186/s12877-019-1124-0 · Cited by 21
Saesen, R., Lacombe, D. A., & Huys, I. (2021) · European Journal of Cancer · doi:10.1016/j.ejca.2021.04.012 · Cited by 15
Zhang, N., Tu, J., Lin, Y., Li, J., Zou, X., Wang, Y., Li, H., Wei, X., Wang, Y., Shi, G., et al. (2021) · Journal of Pain Research · doi:10.2147/jpr.s319195 · Cited by 19
BUSINESS MODEL CONFIGURATIONS IN DIGITAL HEALTHCARE—A GERMAN CASE STUDY ABOUT DIGITAL TRANSFORMATION
GEHDE, K. M., RAUSCH, F., & Leker, J. (2022) · International Journal of Innovation Management · doi:10.1142/s1363919622400187 · Cited by 18
The Melbourne Statement on Practice Research in Social Work: Practice Meets Research
Joubert, L., Webber, M., Uggerhøj, L., Julkunen, I., Yliruka, L., Hampson, R., Simpson, G. K., Sim, T., Manguy, A.-M., & Austin, M. J. (2023) · Research on Social Work Practice · doi:10.1177/10497315221139835 · Cited by 15
“…of health and mental health, this move has been driven by a growing understanding of the limited relevance and potential waste of much of the health research that is produced, due to a lack of integration among clients, practitioners, and researchers in undertaking such work (Ioannidis, 2016).”
BackgroundCata, J. P., & Sessler, D. I. (2024) · Anesthesiology · doi:10.1097/aln.0000000000004823 · Cited by 15
Smartphones within Psychological Science
Ellis, D. A. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108671408 · Cited by 15
Should we still offer elective freezing of all embryos in all IVF cycles?
Rafael, Z. B. (2020) · Human Reproduction · doi:10.1093/humrep/deaa204 · Cited by 15
The Application of Evidence-Based Medicine in Individualized Medicine
de Vliet, P. V., Sprenger, T., Kampers, L. F. C., Makalowski, J., Schirrmacher, V., Stücker, W., & Gool, S. V. (2023) · Biomedicines · doi:10.3390/biomedicines11071793 · Cited by 16
“According to Ioannidis [36], the usefulness of clinical research requires (1) the existence of a real problem to address; (2) proper context placement; (3) sufficient information gain; (4) patient-centeredness; (5) pragmatism; (6) reasonable value-for-money; (7) non-futility; and (8) transparency.”
BackgroundDrug information center in India: Overview, challenges, and future prospects
Patil, A. P. P. A. U., Padhy, B., Prasanthi, S., & Rohilla, R. (2018) · International Journal of Pharmaceutical Investigation · doi:10.4103/jphi.jphi_103_17 · Cited by 14
Bolland, M. J., Avenell, A., & Grey, A. B. (2018) · BMC Medical Research Methodology · doi:10.1186/s12874-018-0556-0 · Cited by 16
Analytical reliability of simple, rapid, minuturizated, direct analytical processes: A call to arms
Soriano, M. L., Zougagh, M., Ríos, Á., & Valcárcel, M. (2019) · TrAC Trends in Analytical Chemistry · doi:10.1016/j.trac.2019.03.005 · Cited by 14
Theoretical Psychiatry as a Link Between Academic and Clinical Psychiatry
Jakovljević, M., & Jakovljević, I. (2019) · Advances in experimental medicine and biology · doi:10.1007/978-981-32-9721-0_19 · Cited by 18
Mao, Y., Hu, G.-J., Meng, Q., Li, X., Sun, X., Zhou, J., Zhang, T., Liu, H., Wang, C., & Du, X. (2021) · Journal of Ethnopharmacology · doi:10.1016/j.jep.2021.113927 · Cited by 14
The Case of Dr. Oz: Ethics, Evidence, and Does Professional Self-Regulation Work?
Tilburt, J. C., Allyse, M. A., & Hafferty, F. W. (2017) · The AMA Journal of Ethic · doi:10.1001/journalofethics.2017.19.2.msoc1-1702 · Cited by 13
“We settle for incomplete, selectively published data in journals heavily subsidized by pharmaceutical companies and for outcomes that don’t give firm answers [21].”
BackgroundConquest of COVID‐19. Publish it to death?
Gale, R. P. (2020) · British Journal of Haematology · doi:10.1111/bjh.16905 · Cited by 13
“Given the baseline estimate 85% of clinical research is not useful or wrong, we may be pushing this estimate to 90 or 95% this year.(2,3) One explanation of this publication deluge is the opportunity the SARS-CoV-2 pandemic offers authors and journals.”
BackgroundHeterogeneity in clinical research data quality monitoring: A national survey
Houston, L., Yu, P., Martin, A. M., & Probst, Y. (2020) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2020.103491 · Cited by 20
“Introduction Clinical research is vital in the discovery of new treatments, prevention, diagnosis/screening and reducing the risk of disease in humans[1].”
Kampman, J. M., Weiland, N. H. S., Hollmann, M. W., Repping, S., & Hermanides, J. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.05.003 · Cited by 16
Kelter, R. (2023) · Biometrical Journal · doi:10.1002/bimj.202200095 · Cited by 21
“Such as the replication crisis in medical science puts the finger on a weak spot in applied empirical research (Ioannidis, 2016; Wagenmakers & Pashler, 2012; Wasserstein & Lazar, 2016; Wasserstein et al., 2019), the lack of neutral comparison studies which aim at providing guidelines as described…”
BackgroundReplicability of Machine Learning Models in the Social Sciences: A Case Study in Variable Selection
Vijayakumar, R., & Cheung, M. W. ‐. (2018) · Zeitschrift für Psychologie · doi:10.1027/2151-2604/a000344 · Cited by 15
“Concerns about researchers’ overemphasis on the p-value (Goodman, 1992), p-hacking and data-fudging have emboldened cries for changing the methods of analysis in the field (Ioannidis, 2005, 2016).”
MethodThe nature of work-related problems: messy, co-produced and wicked
Fergusson, L. C. (2019) · Journal of Work-Applied Management · doi:10.1108/jwam-08-2019-0024 · Cited by 17
Rocca, E., Anjum, R. L., & Mumford, S. D. (2020) · Boston studies in the philosophy of science · doi:10.1007/978-3-030-29179-2_2 · Cited by 15
“The problem of improving knowledge translation has gained increasing attention in the medical literature over the last years (Ioannidis 2016, Edwards and Hugman 1997).”
BackgroundNetwork Analyses and Nervous System Disorders
Medaglia, J. D., & Bassett, D. S. (2016) · Oxford University Press eBooks · doi:10.1093/acrefore/9780190264086.013.121 · Cited by 17
“Independent of mechanism, an essential question confronts clinical network neuroscientists: how can network analysis provide benefits in the clinic? The clinical imperative requires robust techniques that inform clinical diagnostics, prediction, and treatments [167].”
BackgroundRe-Evaluating Evidence for Best Practice in Paediatric Speech-Language Pathology
Dodd, B. (2021) · Folia Phoniatrica et Logopaedica · doi:10.1159/000505265 · Cited by 12
A Decade of Efforts to Add Value to Child Health Research Practices
Sammy, A., Baba, A., Klassen, T. P., Moher, D., & Offringa, M. (2023) · The Journal of Pediatrics · doi:10.1016/j.jpeds.2023.113840 · Cited by 13
Research Note: Patient and public involvement in research
LI, L. C., Richards, D. P., & Primeau, C. A. (2025) · Journal of physiotherapy · doi:10.1016/j.jphys.2025.03.001 · Cited by 14
Shaw, S. R., D'Intino, J., & Lysenko, E. (2019) · Canadian Journal of School Psychology · doi:10.1177/0829573519843027 · Cited by 12
Broes, S., Saesen, R., Lacombe, D. A., & Huys, I. (2020) · Clinical and Translational Science · doi:10.1111/cts.12863 · Cited by 12
Tyler, N., Wright, N., Gregoriou, K., & Waring, J. J. (2021) · BMC Health Services Research · doi:10.1186/s12913-021-07136-2 · Cited by 12
“questions and outcomes of limited relevance to clinicians, patients and other end-users [22].”
BackgroundSmith, J. A., Braithwaite, J., O'Brien, T. A., Smith, S., Tyrrell, V. J., Mould, E. V. A., Long, J. C., & Rapport, F. (2022) · Qualitative Health Research · doi:10.1177/10497323221120501 · Cited by 13
“Unfortunately, health research frequently addresses questions of limited relevance to clinicians leading to research ‘waste’ (Ioannidis, 2016).”
BackgroundSearch strategies (filters) to identify systematic reviews in MEDLINE and Embase
Liquitay, C. M. E., Garegnani, L., Garrote, V., Solà, I., & Franco, J. V. A. (2023) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000054.pub2 · Cited by 24
Metadata
- Authors
- Ioannidis, J. P. A.
- Affiliations
- Stanford Medicine (US)
- Corresponding author
- Ioannidis, John P. A.
- Venue
- PLOS Medicine
- Publisher
- Public Library of Science (PLoS)
- Published
- 2016-06-21
- Volume
- 13
- Issue
- 6
- Pages
- e1002049
- ISSN
- 1549-1277 (ISSN-L)
- 1549-1676
- PMC
- PMC4915619
- OpenAlex
- W2469474955
- Cited by
- 655
- References
- 56
- Citations by year
- 2026: 68 · 2025: 55 · 2024: 48 · 2023: 60 · 2022: 55 · 2021: 77 · 2020: 81 · 2019: 59 · 2018: 69 · 2017: 65 · 2016: 18
- FWCI
- 69.75
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS Medicine (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Funders
- Patient-Centered Outcomes Research Institute
- Laura and John Arnold Foundation
- National Institutes of Health
- Topics
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Health and Medical Research Impacts
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Health Professions › General Health Professions › Healthcare cost, quality, practices
- Keywords
- research value, research reform, problem-based research, information gain, pragmatism, patient-oriented research, value for money, feasibility, research transparency
- MeSH terms
- Humans, Biomedical Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.