How to Make More Published Research True
Open accessThe DOI matches its Crossref record; 75 of the 75 references it cites were found in a registry or an index.
Abstract
Currently, many published research findings are false or exaggerated, and an estimated 85% of research resources are wasted.N To make more published research true, practices that have improved credibility and efficiency in specific fields may be transplanted to others which would benefit from them-possibilities include the adoption of large-scale collaborative research; replication culture; registration; sharing; reproducibility practices; better statistical methods; standardization of definitions and analyses; more appropriate (usually more stringent) statistical thresholds; and improvement in study design standards, peer review, reporting and dissemination of research, and training of the scientific workforce. N Selection of interventions to improve research practices requires rigorous examination and experimental testing whenever feasible.N Optimal interventions need to understand and harness the motives of various stakeholders who operate in scientific research and who differ on the extent to which they are interested in promoting publishable, fundable, translatable, or profitable results.N Modifications need to be made in the reward system for science, affecting the exchange rates for currencies (e.g., publications and grants) and purchased academic goods (e.g., promotion and other academic or administrative power) and introducing currencies that are better aligned with translatable and reproducible research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Accordingly, there is growing recognition that data availability is a critical feature of an efficient, progressive, and ultimately self-correcting scientific ecosystem that generates credible findings [1–3].”
From Data availability, reusability, and analytic reproducibility: evaluating the impact of a mandatory open data policy at the journal Cognition (2018) · cites this paper as [1]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Some journals have heeded the growing calls for adoption of transparent and reproducible research practices and introduced new policies to implement change [1,2].”
From Data availability, reusability, and analytic reproducibility: evaluating the impact of a mandatory open data policy at the journal Cognition (2018) · cites this paper as [1]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, protocol registration is applicable to and recommended for all studies and is supported by various platforms, such as the Open Science Framework [3, 29, 35–37].”
From Assessing transparency practices in dental randomized controlled trials (2024) · cites this paper as [3]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Well-planned, well-executed, and well-reported RCTs, which embrace transparent and open science practices, enhance the likelihood that results are accurate, reliable, and applicable [2, 3].”
From Assessing transparency practices in dental randomized controlled trials (2024) · cites this paper as [3]
Experimental — accuracy not yet measuredIn the citing paper
“Consider that the stakeholders who fund and support science may have incentives that are not always aligned with those of active researchers [82].”
From The natural selection of bad science (2016) · cites this paper as [82]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
Self-citationConfirmation Bias: A Ubiquitous Phenomenon in Many Guises
Nickerson, R. S. (1998) · Review of General Psychology · doi:10.1037/1089-2680.2.2.175 · Cited by 7,267 · cited 2×
“powerful in scientific processes [8,9].”
BackgroundSimmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · cited 2×
“Many new proposed associations and/or effects are false or grossly exaggerated [2,3], and translation of knowledge into useful applications is often slow and potentially inefficient [4].”
BackgroundSelf-citationClinical Practice Guidelines We Can Trust
Committee on Standards for Developing Trustworthy Clinical Practice Guidelines, Board on Health Care Services, & Institute of Medicine (2011) · National Academies Press eBooks · doi:10.17226/13058 · Cited by 3,081
Raise standards for preclinical cancer research
Begley, C. G., & Ellis, L. M. (2012) · Nature · doi:10.1038/483531a · Cited by 2,979 · cited 1×
“Pharmaceutical industry teams have championed reproducibility in preclinical research [56,57] because they depend on pre-clinical academic investigations accurately pinpointing useful drug targets.”
BackgroundBelieve it or not: how much can we rely on published data on potential drug targets?
Prinz, F., Schlange, T., & Asadullah, K. (2011) · Nature Reviews Drug Discovery · doi:10.1038/nrd3439-c1 · Cited by 1,888 · cited 1×
“Pharmaceutical industry teams have championed reproducibility in preclinical research [56,57] because they depend on pre-clinical academic investigations accurately pinpointing useful drug targets.”
BackgroundEvidence based medicine: a movement in crisis?
Greenhalgh, T., Howick, J., Maskrey, N., & for the Evidence Based Medicine Renaissance Group (2014) · BMJ · doi:10.1136/bmj.g3725 · Cited by 1,766
Scientific method: Statistical errors
Nuzzo, R. (2014) · Nature · doi:10.1038/506150a · Cited by 1,536
Why Most Discovered True Associations Are Inflated
Ioannidis, J. P. A. (2008) · Epidemiology · doi:10.1097/ede.0b013e31818131e7 · Cited by 1,495 · cited 1×
“Many new proposed associations and/or effects are false or grossly exaggerated [2,3], and translation of knowledge into useful applications is often slow and potentially inefficient [4].”
BackgroundSelf-citationReproducible research in computational science
Peng, R. D. (2011) · Science · doi:10.1126/science.1213847 · Cited by 1,449 · cited 1×
“Conversely, the industry is defensive about data sharing from clinical trials [30], which occurs at a point in the product development when re-analyses may correctly or incorrectly [58] invalidate evidence supporting drugs in which it has already invested heavily.”
BackgroundHow to increase value and reduce waste when research priorities are set
Chalmers, I., Bracken, M. B., Djulbegovic, B., Garattini, S., Grant, J., Gülmezoglu, A. M., Howells, D. W., Ioannidis, J. P. A., & Oliver, S. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62229-1 · Cited by 1,436
Self-citationReplicating genotype–phenotype associations
NCI-NHGRI Working Group on Replication in Association Studies, Chanock, S. J., Manolio, T., Boehnke, M., Boerwinkle, E., Hunter, D. J., Thomas, G., Hirschhorn, J. N., Abecasis, G., Altshuler, D., et al. (2007) · Nature · doi:10.1038/447655a · Cited by 1,352
Nepotism and sexism in peer-review
Wennerås, C., & Wold, A. (1997) · Nature · doi:10.1038/387341a0 · Cited by 1,345 · cited 1×
“Non-meritocratic practices, including nepotism, sexism, and unwarranted conservatism, are probably widespread [7].”
BackgroundReducing waste from incomplete or unusable reports of biomedical research
Glasziou, P., Altman, D. G., Bossuyt, P., Boutron, I., Clarke, M., Julious, S., Michie, S., Moher, D., & Wager, E. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62228-x · Cited by 1,319
Nosek (2012) · doi:10.1177/1745691612459058 · Cited by 1,301
Policy: NIH plans to enhance reproducibility
Collins, F. S., & Tabak, L. A. (2014) · Nature · doi:10.1038/505612a · Cited by 1,247
A call for transparent reporting to optimize the predictive value of preclinical research
Landis, S. C., Amara, S. G., Asadullah, K., Austin, C. P., Blumenstein, R., Bradley, E. W., Crystal, R. G., Darnell, R. B., Ferrante, R. J., Fillit, H., et al. (2012) · Nature · doi:10.1038/nature11556 · Cited by 1,196 · cited 1×
“There is increasing interest in proposing checklists for the conduct of studies to be approved [46,47], making it vital to ensure both that checklist items are indeed essential and that claims of adherence to them are verifiable.”
BackgroundDwan, K., Gamble, C., Williamson, P. R., Kirkham, J. J., & the Reporting Bias Group (2013) · PLoS ONE · doi:10.1371/journal.pone.0066844 · Cited by 1,127 · cited 1×
“results [23]) has enhanced transparency in clinical trials research and has allowed probing of selective reporting biases [24,25], even if not fully remedying them.”
BackgroundA catalogue of reporting guidelines for health research
Simera, I., Moher, D., Hoey, J., Schulz, K. F., & Altman, D. G. (2010) · European Journal of Clinical Investigation · doi:10.1111/j.1365-2362.2009.02234.x · Cited by 1,047 · cited 1×
“, as catalogued by the EQUATOR initiative [48]) and multiple ideas about how to change peer review (by whom, how, and when) and dissemination of information [25,49–51].”
BackgroundBiomedical research: increasing value, reducing waste.
Macleod, M. R., Michie, S., Roberts, I., Dirnagl, U., Chalmers, I., Ioannidis, J. P. A., Salman, R. A.-S., Chan, A.-W., & Glasziou, P. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62329-6 · Cited by 1,003
Self-citationBohannon, J. (2013) · Science · doi:10.1126/science.342.6154.60 · Cited by 934
Increasing value and reducing waste: addressing inaccessible research
Chan, A.-W., Song, F., Vickers, A., Jefferson, T., Dickersin, K., Gøtzsche, P. C., Krumholz, H. M., Ghersi, D., & van der Worp, H. B. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62296-5 · Cited by 887
The ClinicalTrials.gov Results Database — Update and Key Issues
Zarin, D. A., Tse, T., Williams, R. J., Califf, R. M., & Ide, N. C. (2011) · New England Journal of Medicine · doi:10.1056/nejmsa1012065 · Cited by 838 · cited 1×
“results [23]) has enhanced transparency in clinical trials research and has allowed probing of selective reporting biases [24,25], even if not fully remedying them.”
BackgroundRevised standards for statistical evidence
Johnson, V. E. (2013) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1313476110 · Cited by 808 · cited 1×
“Adoption of more appropriate statistical methods [38], standardized definitions and analyses and more stringent thresholds for claiming discoveries or ‘‘successes’’ [39] may decrease false-positive rates in fields that have to-date been too lenient (like epidemiology [40], psychology [41,42], or economics [43]).”
Method“Positive” results increase down the Hierarchy of the Sciences
Fanelli, D. (2010) · PLoS ONE · doi:10.1371/journal.pone.0010068 · Cited by 734 · cited 1×
“It may lead them to higher credibility, more akin to that of fields that have traditionally been more rigorous in this regard, like the physical sciences [44].”
BackgroundIs the Replicability Crisis Overblown? Three Arguments Examined
Pashler, H., & Harris, C. R. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612463401 · Cited by 697 · cited 1×
“Adoption of more appropriate statistical methods [38], standardized definitions and analyses and more stringent thresholds for claiming discoveries or ‘‘successes’’ [39] may decrease false-positive rates in fields that have to-date been too lenient (like epidemiology [40], psychology [41,42], or economics [43]).”
MethodIncreasing value and reducing waste in biomedical research regulation and management
Salman, R. A.-S., Beller, E., Kagan, J., Hemminki, E., Phillips, R. S., Savulescu, J., Macleod, M., Wisely, J., & Chalmers, I. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62297-7 · Cited by 534
Confirmation Bias in a Simulated Research Environment: An Experimental Study of Scientific Inference
Mynatt, C. R., Doherty, M. E., & Tweney, R. D. (1977) · Quarterly Journal of Experimental Psychology · doi:10.1080/00335557743000053 · Cited by 513 · cited 2×
“powerful in scientific processes [8,9].”
BackgroundWhy Current Publication Practices May Distort Science
Young, N. S., Ioannidis, J. P. A., & Al-Ubaydli, O. (2008) · PLoS Medicine · doi:10.1371/journal.pmed.0050201 · Cited by 502 · cited 1×
“The exact exchange rate of currencies and the price of academic goods [59] may vary across institutional microenvironments, scientific disciplines and circumstances, and are also affected by each microenvironment’s fairness or unfairness (e.”
BackgroundSelf-citationJørgensen, A. W., Hilden, J., & Gøtzsche, P. C. (2006) · BMJ · doi:10.1136/bmj.38973.444699.0b · Cited by 444
Scientific Utopia: I. Opening Scientific Communication
Nosek, B. A., & Bar-Anan, Y. (2012) · Psychological Inquiry · doi:10.1080/1047840x.2012.692215 · Cited by 368
The False-positive to False-negative Ratio in Epidemiologic Studies
Ioannidis, J. P. A., Tarone, R., & McLaughlin, J. K. (2011) · Epidemiology · doi:10.1097/ede.0b013e31821b506e · Cited by 321
Self-citationReproducible Epidemiologic Research
Peng, R. D., Dominici, F., & Zeger, S. L. (2006) · American Journal of Epidemiology · doi:10.1093/aje/kwj093 · Cited by 292 · cited 1×
“Sharing of data, protocols, materials, and software has been promoted in several -omics fields, creating a substrate for reproducible data practices [28–31].”
BackgroundStodden, V., Guo, P., & Ma, Z. (2013) · PLoS ONE · doi:10.1371/journal.pone.0067111 · Cited by 257 · cited 1×
“Sharing of data, protocols, materials, and software has been promoted in several -omics fields, creating a substrate for reproducible data practices [28–31].”
BackgroundOverlapping meta-analyses on the same topic: survey of published studies
Siontis, K. C., Hernandez-Boussard, T., & Ioannidis, J. P. A. (2013) · BMJ · doi:10.1136/bmj.f4501 · Cited by 248 · cited 1×
“Otherwise, some questions are not addressed at all or are addressed by single studies that are never replicated, while others are subjected to multiple unnecessary replications or even redundant metaanalyses combining them [21].”
BackgroundSelf-citationDeming, Data and Observational Studies
Young, S. S., & Karr, A. (2011) · Significance · doi:10.1111/j.1740-9713.2011.00506.x · Cited by 235
The Journal's Policy on Cost-Effectiveness Analyses
Kassirer, J. P., & Angell, M. (1994) · New England Journal of Medicine · doi:10.1056/nejm199409083311009 · Cited by 233 · cited 1×
“Dissociation of some research types from specific conflicted sponsors or authors has been proposed (not without debate) for designs as diverse as cost-effectiveness analyses [34], meta-analyses [35,36], and guidelines [37].”
BackgroundIssues in the Registration of Clinical Trials
Zarin, D. A., Ide, N. C., Tse, T., Harlan, W. R., West, J. C., & Lindberg, D. A. B. (2007) · JAMA · doi:10.1001/jama.297.19.2112 · Cited by 222 · cited 1×
“Registration of randomized trials [22] (and, more recently, registration of their”
BackgroundLife cycle of translational research for medical interventions
Contopoulos-Ioannidis, D. G., Alexiou, G. A., Gouvias, T. C., & Ioannidis, J. P. A. (2008) · Science · doi:10.1126/science.1160622 · Cited by 207 · cited 1×
“Many new proposed associations and/or effects are false or grossly exaggerated [2,3], and translation of knowledge into useful applications is often slow and potentially inefficient [4].”
BackgroundSelf-citationHagen, N. T. (2008) · PLoS ONE · doi:10.1371/journal.pone.0004021 · Cited by 171 · cited 1×
“Adjustments for multi-authorship are readily feasible [71,72].”
Disclosure of Researcher Contributions: A Study of Original Research Articles in The Lancet
Yank, V., & Rennie, D. (1999) · Annals of Internal Medicine · doi:10.7326/0003-4819-130-8-199904200-00013 · Cited by 170
Nicholson, J. M., & Ioannidis, J. P. A. (2012) · Nature · doi:10.1038/492034a · Cited by 169 · cited 3×
“For example, there is evidence that grant reviewers typically have only modest CVs and most of the top influential scientists don’t review grant applications and don’t get funded by government funds, even in the United States [6], which arguably has the strongest scientific impact at the moment than any other country…”
BackgroundInfluentialSelf-citationIoannidis, J. P. A. (2011) · Nature · doi:10.1038/477529a · Cited by 163 · cited 1×
“, protocol-based, manuscript-based, post-publication) [5,12,13] or on how research funds should be allocated [14,15].”
BackgroundSelf-citationAn invitation to reproducible computational research
Donoho, D. L. (2010) · Biostatistics · doi:10.1093/biostatistics/kxq028 · Cited by 162 · cited 1×
“Sharing of data, protocols, materials, and software has been promoted in several -omics fields, creating a substrate for reproducible data practices [28–31].”
BackgroundRedundant surgical publications: Tip of the iceberg?
Schein, M., & Paladugu, R. (2001) · Surgery · doi:10.1067/msy.2001.114549 · Cited by 155
Clinical Trial Registration: Looking Back and Moving Ahead
Laine, C., Horton, R., DeAngelis, C. D., Drazen, J. M., Frizelle, F. A., Godlee, F., Haug, C., Hébert, P. C., Kotzin, S., Marusic, A., et al. (2007) · JAMA · doi:10.1001/jama.298.1.jed70037 · Cited by 135
The Power of Meta-Analysis in Genome-Wide Association Studies
Panagiotou, O. A., Willer, C. J., Hirschhorn, J. N., & Ioannidis, J. P. A. (2013) · Annual Review of Genomics and Human Genetics · doi:10.1146/annurev-genom-091212-153520 · Cited by 124 · cited 1×
“These techniques have helped transform genetic epidemiology from a spurious field [17] to a highly credible one [18].”
BackgroundSelf-citationMaking Prospective Registration of Observational Research a Reality
Dal-Ré, R., Ioannidis, J. P., Bracken, M. B., Buffler, P. A., Chan, A.-W., Franco, E. L., La Vecchia, C., & Weiderpass, E. (2014) · Science Translational Medicine · doi:10.1126/scitranslmed.3007513 · Cited by 123 · cited 1×
“Registration is currently proposed for many other types of research, including both human observational studies [26] and nonhuman studies [27].”
BackgroundSelf-citationTransforming Epidemiology for 21st Century Medicine and Public Health
Khoury, M. J., Lam, T. K., Ioannidis, J. P. A., Hartge, P., Spitz, M. R., Buring, J. E., Chanock, S. J., Croyle, R. T., Goddard, K. A., Ginsburg, G. S., et al. (2013) · Cancer Epidemiology, Biomarkers & Prevention · doi:10.1158/1055-9965.epi-13-0146 · Cited by 119 · cited 1×
“Such practices could be applied to other fields of observational research and beyond [19].”
BackgroundSelf-citationWHAT'S TO KNOW ABOUT THE CREDIBILITY OF EMPIRICAL ECONOMICS?
Ioannidis, J., & Doucouliagos, C. (2013) · Journal of Economic Surveys · doi:10.1111/joes.12032 · Cited by 118 · cited 1×
“Adoption of more appropriate statistical methods [38], standardized definitions and analyses and more stringent thresholds for claiming discoveries or ‘‘successes’’ [39] may decrease false-positive rates in fields that have to-date been too lenient (like epidemiology [40], psychology [41,42], or economics [43]).”
MethodSelf-citationStamatakis, E., Weiler, R., & Ioannidis, J. P. A. (2013) · European Journal of Clinical Investigation · doi:10.1111/eci.12074 · Cited by 113 · cited 2×
“For healthcare and clinical practice, while evidence-based medicine has grown stronger over time, some argue that it is currently in crisis [10] and ‘‘evidencebased’’ terminology has been usurped to promote expert-based beliefs and industry agendas [11].”
BackgroundSelf-citationContent area experts as authors: helpful or harmful for systematic reviews and meta-analyses?
Gotzsche, P. C., & Ioannidis, J. P. A. (2012) · BMJ · doi:10.1136/bmj.e7031 · Cited by 105
Self-citationHopewell, S., Collins, G. S., Boutron, I., Yu, L.-M., Cook, J., Shanyinde, M., Wharton, R., Shamseer, L., & Altman, D. G. (2014) · BMJ · doi:10.1136/bmj.g4145 · Cited by 103 · cited 1×
“It can also be superficial, lead to only modest improvements of the reviewed work, and allow for the acceptance of blatantly wrong papers [68,69].”
BackgroundReproducibility: The risks of the replication drive
Bissell, M. (2013) · Nature · doi:10.1038/503333a · Cited by 99
Assessing Value in Biomedical Research: The PQRST of Appraisal and Reward
Ioannidis, J. P. A., & Khoury, M. J. (2014) · JAMA · doi:10.1001/jama.2014.6932 · Cited by 92 · cited 1×
“The reward system may be systematically modified [62].”
BackgroundSelf-citationWhy animal research needs to improve
Macleod, M. R. (2011) · Nature · doi:10.1038/477511a · Cited by 83 · cited 2×
“For example, for animal studies of interventions, this would include randomization and blinding of investigators [27].”
BackgroundDanthi, N., Wu, C. O., Shi, P., & Lauer, M. (2014) · Circulation Research · doi:10.1161/circresaha.114.302656 · Cited by 82
China tops Europe in R&D intensity
Noorden, R. V. (2014) · Nature · doi:10.1038/505144a · Cited by 74
Raw data from clinical trials: within reach?
Doshi, P., Goodman, S. N., & Ioannidis, J. P. A. (2013) · Trends in Pharmacological Sciences · doi:10.1016/j.tips.2013.10.006 · Cited by 67 · cited 1×
“Promotion of data sharing in clinical trials may similarly improve the credibility of clinical research [32].”
BackgroundSelf-citationProfit (p)-Index: The Degree to Which Authors Profit from Co-Authors
Aziz, N. A., & Rozing, M. P. (2013) · PLoS ONE · doi:10.1371/journal.pone.0059814 · Cited by 59 · cited 1×
“Adjustments for multi-authorship are readily feasible [71,72].”
Study of Neurontin: Titrate to Effect, Profile of Safety (STEPS) Trial
Krumholz, S. D., Egilman, D. S., & Ross, J. S. (2011) · Archives of Internal Medicine · doi:10.1001/archinternmed.2011.241 · Cited by 59 · cited 1×
“Corporate influence may subvert science for the purposes of advertising, with papers in influential journals, prestigious society meetings, and a professorate system of opinion leaders becoming branches of their marketing department [11,54].”
BackgroundBiospecimens, biomarkers, and burgeoning data: the imperative for more rigorous research standards
Poste, G. H. (2012) · Trends in Molecular Medicine · doi:10.1016/j.molmed.2012.09.003 · Cited by 51
Sea Change in Open Science and Data Sharing: Leadership by Industry
Krumholz, H. M., Gross, C. P., Blount, K. L., Ritchie, J. D., Hodshon, B., Lehman, R., & Ross, J. S. (2014) · Circulation: Cardiovascular Quality and Outcomes · doi:10.1161/circoutcomes.114.001166 · Cited by 45 · cited 1×
“) is not incentivized or requested, with some notable exceptions [65–67].”
BackgroundA list of highly influential biomedical researchers, 1996–2011
Boyack, K. W., Klavans, R., Sorensen, A. A., & Ioannidis, J. P. A. (2013) · European Journal of Clinical Investigation · doi:10.1111/eci.12171 · Cited by 42
Self-citationDeming, data and observational studies: A process out of control and needing fixing
Young, S. S., & Karr, A. F. (2013) · Quality Engineering · Cited by 41
Monforton, C. A. (2005) · American Journal of Public Health · doi:10.2105/ajph.2005.064410 · Cited by 37
Knowledge integration at the center of genomic medicine
Khoury, M. J., GWINN, M. L., Dotson, W. D., & Schully, S. D. (2012) · Genetics in Medicine · doi:10.1038/gim.2012.43 · Cited by 32 · cited 1×
“As we design, test, and implement interventions on research practices, we need to understand who is affected by and shaping research [5,52,53].”
MethodChristakis, D., & Zimmerman, F. J. (2013) · JAMA · doi:10.1001/jama.2013.281337 · Cited by 31
Rennie, D., & Flanagin, A. (2014) · JAMA · doi:10.1001/jama.2014.1362 · Cited by 30 · cited 1×
“, protocol-based, manuscript-based, post-publication) [5,12,13] or on how research funds should be allocated [14,15].”
BackgroundScientific research in the age of omics: the good, the bad, and the sloppy
Witten, D., & Tibshirani, R. J. (2012) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2012-000972 · Cited by 21 · cited 1×
“Conversely, interesting post-publication review efforts such as PubMed Commons [61] have so far not fulfilled their potential as progressive vehicles for evaluating research, probably because there is currently no reward for such post-publication peer review.”
BackgroundIs there a glass ceiling for highly cited scientists at the top of research universities?
Ioannidis, J. P. A. (2010) · The FASEB Journal · doi:10.1096/fj.10-162974 · Cited by 18
Self-citationRewarding high-power replication research
Wagenmakers, E., & Forstmann, B. U. (2013) · Cortex · doi:10.1016/j.cortex.2013.09.010 · Cited by 16 · cited 1×
“Value is given to sound ideas and results that are replicated and reproducible [74] rather than publication per se.”
BackgroundCancer-gene data sharing boosted
Hayden, E. C. (2014) · Nature · doi:10.1038/510198a · Cited by 5 · cited 1×
“) is not incentivized or requested, with some notable exceptions [65–67].”
BackgroundData sharing will pay dividends
(2014) · Nature · doi:10.1038/505131a · Cited by 3
Research works printed but not linked: 1
- R Al-Shahi Salman, 2014, Increasing value and reducing waste in biomedical research regulation and management, Lancet, 383, 176, 10.1016/s0140-6736(13)62297-7 · DOI · Paper page
Cited by
A manifesto for reproducible science
Munafò, M. R., Nosek, B. A., Bishop, D. V. M., Button, K. S., Chambers, C. D., Percie du Sert, N., Simonsohn, U., Wagenmakers, E.-J., Ware, J. J., & Ioannidis, J. P. A. (2017) · Nature Human Behaviour · doi:10.1038/s41562-016-0021 · Cited by 3,397
“Equally, these proposals are not an exhaustive list; there are many other nascent and maturing ideas for making research practices more efficient and reliable 73 .”
Self-citationLefaucheur, J.-P., Aleman, A., Baeken, C., Benninger, D. H., Brunelin, J., Di Lazzaro, V., Filipović, S. R., Grefkes, C., Hasan, A., Hummel, F. C., et al. (2020) · Clinical Neurophysiology · doi:10.1016/j.clinph.2019.11.002 · Cited by 2,382
The Mass Production of Redundant, Misleading, and Conflicted Systematic Reviews and Meta‐analyses
IOANNIDIS, J. P. A. (2016) · The Milbank Quarterly · doi:10.1111/1468-0009.12210 · Cited by 1,483
Self-citationGagnier, J. J., Lai, J., Mokkink, L. B., & Terwee, C. B. (2021) · Quality of Life Research · doi:10.1007/s11136-021-02822-4 · Cited by 881
The natural selection of bad science
Smaldino, P. E., & McElreath, R. (2016) · Royal Society Open Science · doi:10.1098/rsos.160384 · Cited by 860
“ampbell, 1976; Wilson, Hayes, Biglan & Embry, 2014). We are hardly the first to consider a need to alter the incentives for career success in science (Nosek & Lakens, 2014; Nosek et al., 2012; Ioannidis, 2014a; Vankov et al., 2014; Fischer, Ritchie & Hanspach, 2012; Brembs, Button & Munafo, 2013; Begley…”
BackgroundInfluential
Metadata
- Authors
- Ioannidis, J. P. A.
- Affiliations
- Stanford Medicine (US)
- Corresponding author
- Ioannidis, John P. A.
- Venue
- PLoS Medicine
- Publisher
- Public Library of Science (PLoS)
- Published
- 2014-10-21
- Volume
- 11
- Issue
- 10
- Pages
- e1001747
- ISSN
- 1549-1277 (ISSN-L)
- 1549-1676
- PMC
- PMC4204808
- OpenAlex
- W2077181076
- Cited by
- 760
- References
- 75
- Citations by year
- 2026: 26 · 2025: 31 · 2024: 37 · 2023: 50 · 2022: 51 · 2021: 44 · 2020: 51 · 2019: 68 · 2018: 93 · 2017: 108 · 2016: 111 · 2015: 80 · 2014: 10
- FWCI
- 192.35
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS Medicine (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Laura and John Arnold Foundation
- Topics
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Social Sciences › Education › Academic integrity and plagiarism
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- research integrity, reproducibility, research credibility, research registration, statistical thresholds, peer review, research reporting, research incentives, publication incentives, research waste
- MeSH terms
- Publications, Publishing, Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.