The Mass Production of Redundant, Misleading, and Conflicted Systematic Reviews and Meta‐analyses
Open accessAbstract
Policy Points : Currently, there is massive production of unnecessary, misleading, and conflicted systematic reviews and meta‐analyses. Instead of promoting evidence‐based medicine and health care, these instruments often serve mostly as easily produced publishable units or marketing tools. Suboptimal systematic reviews and meta‐analyses can be harmful given the major prestige and influence these types of studies have acquired. The publication of systematic reviews and meta‐analyses should be realigned to remove biases and vested interests and to integrate them better with the primary production of evidence. Context: Currently, most systematic reviews and meta‐analyses are done retrospectively with fragmented published information. This article aims to explore the growth of published systematic reviews and meta‐analyses and to estimate how often they are redundant, misleading, or serving conflicted interests. Methods: Data included information from PubMed surveys and from empirical evaluations of meta‐analyses. Findings: Publication of systematic reviews and meta‐analyses has increased rapidly. In the period January 1, 1986, to December 4, 2015, PubMed tags 266,782 items as “systematic reviews” and 58,611 as “meta‐analyses.” Annual publications between 1991 and 2014 increased 2,728% for systematic reviews and 2,635% for meta‐analyses versus only 153% for all PubMed‐indexed items. Currently, probably more systematic reviews of trials than new randomized trials are published annually. Most topics addressed by meta‐analyses of randomized trials have overlapping, redundant meta‐analyses; same‐topic meta‐analyses may exceed 20 sometimes. Some fields produce massive numbers of meta‐analyses; for example, 185 meta‐analyses of antidepressants for depression were published between 2007 and 2014. These meta‐analyses are often produced either by industry employees or by authors with industry ties and results are aligned with sponsor interests. China has rapidly become the most prolific producer of English‐language, PubMed‐indexed meta‐analyses. The most massive presence of Chinese meta‐analyses is on genetic associations (63% of global production in 2014), where almost all results are misleading since they combine fragmented information from mostly abandoned era of candidate genes. Furthermore, many contracting companies working on evidence synthesis receive industry contracts to produce meta‐analyses, many of which probably remain unpublished. Many other meta‐analyses have serious flaws. Of the remaining, most have weak or insufficient evidence to inform decision making. Few systematic reviews and meta‐analyses are both non‐misleading and useful. Conclusions: The production of systematic reviews and meta‐analyses has reached epidemic proportions. Possibly, the large majority of produced systematic reviews and meta‐analyses are unnecessary, misleading, and/or conflicted.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Meta-analysis is indifferent to the quality of the original evidence.
“is indifferent to the quality of the original evidence (14)”
From Empirical audit and review and an assessment of evidentiary value in research on the psychological consequences of scarcity (2021) · cites this paper as [14]
The Conclusion (page unavailable in our copy) says systematic reviews cannot be expected to correct deficiencies in primary evidence, supporting the concern behind the claim. But it also says they can reveal how unreliable published evidence is, so describing meta-analysis as simply indifferent to evidence quality overstates the paper's position. (LLM, conf=0.82)
Experimental — accuracy not yet measuredA priori registration of SR protocols could boost transparency, minimize outcome reporting bias, and reduce improper spending of research funds due overlaps between SRs.
“Registered reviews offer better quality and conduct and reporting completeness compared to non-registered ones, and a priori registration of SR protocols could boost transparency, minimize outcome reporting bias, and reduce improper spending of research funds due overlaps between SRs [32–34].”
From Reporting quality of abstracts of systematic reviews/meta-analyses: An appraisal of Arab Journal of Urology across 12 years: the PRISMA-Abstracts checklist (2023) · cites this paper as [34]
The section “Flawed Meta-analyses and Correct but Noninformative Meta-analyses” (page unavailable) says that preregistering full protocols has been proposed to improve transparency. It does not say that registering systematic-review protocols reduces outcome-reporting bias or prevents spending on overlapping reviews; it also cautions that preregistration does not guarantee sound reporting. (LLM, conf=0.85)
Experimental — accuracy not yet measuredRegistered reviews offer better quality and conduct and reporting completeness compared to non-registered ones.
“Registered reviews offer better quality and conduct and reporting completeness compared to non-registered ones, and a priori registration of SR protocols could boost transparency, minimize outcome reporting bias, and reduce improper spending of research funds due overlaps between SRs [32–34].”
From Reporting quality of abstracts of systematic reviews/meta-analyses: An appraisal of Arab Journal of Urology across 12 years: the PRISMA-Abstracts checklist (2023) · cites this paper as [34]
In “Flawed Meta-analyses and Correct but Noninformative Meta-analyses” (page unavailable), the paper says preregistration has been proposed to improve transparency and perhaps quality. It does not establish that registered reviews have better conduct or more complete reporting than non-registered reviews; it cautions that registration does not guarantee sound conduct or reporting. (LLM, conf=0.85)
Experimental — accuracy not yet measuredMany systematic reviews continue to be poorly conducted and reported.
“However, many SRs continue to be poorly conducted and reported [10, 11]”
From How to properly use the PRISMA Statement (2021) · cites this paper as [11]
The number of published systematic reviews has increased exponentially in recent years.
“The number of published systematic reviews has increased exponentially in recent years [18, 19].”
From Machine learning computational tools to assist the performance of systematic reviews: A mapping review. (2022) · cites this paper as [18]
Formal statistical meta-analysis methods have been developed to put the results of different studies on the same scale and combine them using various statistical models.
“Formal statistical meta-analysis methods have been developed to put the results of different studies on the same scale and combine them using various statistical models 5-8 .”
From Meta-analysis and the science of research synthesis (2018) · cites this paper as [5]
Methodology was formalized and developed in the following two decades in multiple fields.
“Methodology was formalized and developed in the following two decades in multiple fields 5-8 .”
From Meta-analysis and the science of research synthesis (2018) · cites this paper as [5]
Identifying and interpreting evidence from the growing number of sometimes redundant, misleading, or conflicting syntheses is an onerous task.
“Identifying and interpreting evidence from the growing number of sometimes redundant, misleading, or conflicting syntheses is an onerous task”
From Reporting guideline for overviews of reviews of healthcare interventions: development of the PRIOR statement. (2022) · cites this paper as [4]
There is mass production of reviews that are often unnecessary, misleading and conflicted with most having weak or insufficient evidence to inform decision making.
“It has been argued recently that there is mass production of reviews that are often unnecessary, misleading and conflicted with most having weak or insufficient evidence to inform decision making [66].”
From What kind of systematic review should I conduct? A proposed typology and guidance for systematic reviewers in the medical and health sciences (2018) · cites this paper as [66]
The strengths of this review include a comprehensive search, detailed prespecification of methods, robust analytic techniques, the fact that none of the authors developed or taught any included MBPs, and the synthesis of a substantial amount of evidence; these strengths overcome most limitations highlighted in an extensive critique of existing healthcare reviews.
“The strengths of this review include a comprehensive search, detailed prespecification of methods, robust analytic techniques, the fact that none of us have developed or taught any of the included MBPs, and the synthesis of a substantial amount of evidence; these strengths overcome most of the limitations highlighted in an extensive critique of existing healthcare reviews [299].”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000100 · Cited by 28,164 · see citing works
Self-citationShamseer, L., Moher, D., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P., Stewart, L. A., & the PRISMA-P Group (2015) · BMJ · doi:10.1136/bmj.g7647 · Cited by 13,120 · see citing works
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505 · see citing works
Self-citationLiberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-4-200908180-00136 · Cited by 10,349 · see citing works
Self-citationHutton, B., Salanti, G., Caldwell, D. M., Chaimani, A., Schmid, C. H., Cameron, C., Ioannidis, J. P. A., Straus, S., Thorlund, K., Jansen, J. P., et al. (2015) · Annals of Internal Medicine · doi:10.7326/m14-2385 · Cited by 8,269 · see citing works
Self-citationWhy Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · see citing works
Self-citationThe Sociology of Science: Theoretical and Empirical Investigations
Kaplan, N. M., Merton, R. Κ., & Storer, N. W. (1974) · Technology and Culture · doi:10.2307/3102264 · Cited by 4,056 · see citing works
Raise standards for preclinical cancer research
Begley, C. G., & Ellis, L. M. (2012) · Nature · doi:10.1038/483531a · Cited by 2,979 · see citing works
Selective publication of antidepressant trials and its influence on apparent efficacy
Turner, E. H., Matthews, A. M., Linardatos, E., Tell, R. A., & Rosenthal, R. (2008) · New England Journal of Medicine · doi:10.1056/nejmsa065779 · Cited by 2,435 · see citing works
Believe it or not: how much can we rely on published data on potential drug targets?
Prinz, F., Schlange, T., & Asadullah, K. (2011) · Nature Reviews Drug Discovery · doi:10.1038/nrd3439-c1 · Cited by 1,888 · see citing works
Evidence based medicine: a movement in crisis?
Greenhalgh, T., Howick, J., Maskrey, N., & for the Evidence Based Medicine Renaissance Group (2014) · BMJ · doi:10.1136/bmj.g3725 · Cited by 1,787 · see citing works
Cipriani, A., Furukawa, T. A., Salanti, G., Geddes, J. R., Higgins, J. P., Churchill, R., Watanabe, N., Nakagawa, A., Omori, I. M., McGuire, H., et al. (2009) · The Lancet · doi:10.1016/s0140-6736(09)60046-5 · Cited by 1,654 · see citing works
Increasing value and reducing waste in research design, conduct, and analysis
Ioannidis, J. P. A., Greenland, S., Hlatky, M. A., Khoury, M. J., Macleod, M. R., Moher, D., Schulz, K. F., & Tibshirani, R. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62227-8 · Cited by 1,597 · see citing works
Self-citationAntman, E. M. (1992) · JAMA · doi:10.1001/jama.268.2.240 · Cited by 1,535 · see citing works
The nuts and bolts of PROSPERO: an international prospective register of systematic reviews
Booth, A., Clarke, M., Dooley, G., Ghersi, D., Moher, D., Petticrew, M., & Stewart, L. (2012) · Systematic Reviews · doi:10.1186/2046-4053-1-2 · Cited by 1,433 · see citing works
Evaluating the quality of evidence from a network meta-analysis
Salanti, G., Del Giovane, C., Chaimani, A., Caldwell, D. M., & Higgins, J. P. T. (2014) · PLoS ONE · doi:10.1371/journal.pone.0099682 · Cited by 1,402 · see citing works
Seventy-five trials and eleven systematic reviews a day: how will we ever keep up?
Bastian, H., Glasziou, P., & Chalmers, I. (2010) · PLoS Medicine · doi:10.1371/journal.pmed.1000326 · Cited by 1,188 · see citing works
Cumulative Meta-Analysis of Therapeutic Trials for Myocardial Infarction
Lau, J., Antman, E. M., Jimenez-Silva, J., Kupelnick, B., Mosteller, F., & Chalmers, T. C. (1992) · New England Journal of Medicine · doi:10.1056/nejm199207233270406 · Cited by 1,140 · see citing works
Biomedical research: increasing value, reducing waste.
Macleod, M. R., Michie, S., Roberts, I., Dirnagl, U., Chalmers, I., Ioannidis, J. P. A., Salman, R. A.-S., Chan, A.-W., & Glasziou, P. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62329-6 · Cited by 1,003 · see citing works
Self-citationFinding What Works in Health Care: Standards for Systematic Reviews
Eden, J., Levit, L. A., Berg, A. O., & Morton, S. C. (2013) · Cited by 943 · see citing works
Epidemiology and reporting characteristics of systematic reviews.
Moher, D., Tetzlaff, J., Tricco, A. C., Sampson, M., & Altman, D. G. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040078 · Cited by 891 · see citing works
The Medical Review Article: State of the Science
MULROW, C. D. (1987) · Annals of Internal Medicine · doi:10.7326/0003-4819-106-3-485 · Cited by 856 · see citing works
Page, M. J., Shamseer, L., Altman, D. G., Tetzlaff, J., Sampson, M., Tricco, A. C., Catalá-López, F., Li, L., Reid, E. K., Sarkis-Onofre, R., et al. (2016) · PLOS Medicine · doi:10.1371/journal.pmed.1002028 · Cited by 768 · see citing works
How to Make More Published Research True
Ioannidis, J. P. A. (2014) · PLoS Medicine · doi:10.1371/journal.pmed.1001747 · Cited by 760 · see citing works
Self-citationStress Ulcer Prophylaxis in Critically III Patients
Cook, D. J. (1996) · JAMA · doi:10.1001/jama.1996.03530280060038 · Cited by 603 · see citing works
Publication bias in reports of animal stroke studies leads to major overstatement of efficacy
Sena, E. S., van der Worp, H. B., Bath, P. M. W., Howells, D. W., & Macleod, M. R. (2010) · PLoS Biology · doi:10.1371/journal.pbio.1000344 · Cited by 600 · see citing works
CHALMERS, I. (1993) · Annals of the New York Academy of Sciences · doi:10.1111/j.1749-6632.1993.tb26345.x · Cited by 561 · see citing works
Jadad, A. R., Cook, D. J., Jones, A., Klassen, T. P., Tugwell, P., Moher, M., & Moher, D. (1998) · JAMA · doi:10.1001/jama.280.3.278 · Cited by 534 · see citing works
Owens, D. K., Lohr, K. N., Atkins, D., Treadwell, J. R., Reston, J. T., Bass, E. B., Chang, S., & Helfand, M. (2010) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.03.009 · Cited by 513 · see citing works
Relative Citation Impact of Various Study Designs in the Health Sciences
Patsopoulos, N. A. (2005) · JAMA · doi:10.1001/jama.293.19.2362 · Cited by 498 · see citing works
Jefferson, T., Jones, M., Doshi, P., Spencer, E. A., Onakpoya, I., & Heneghan, C. J. (2014) · BMJ · doi:10.1136/bmj.g2545 · Cited by 449 · see citing works
Jørgensen, A. W., Hilden, J., & Gøtzsche, P. C. (2006) · BMJ · doi:10.1136/bmj.38973.444699.0b · Cited by 439 · see citing works
A guide to interpreting discordant systematic reviews.
Jadad, A., Cook, D. J., & Browman, G. P. (1997) · Europe PMC (PubMed Central) · Cited by 393 · see citing works
Sterne, J. A. C., Jüni, P., Schulz, K. F., Altman, D. G., Bartlett, C., & Egger, M. (2002) · Statistics in Medicine · doi:10.1002/sim.1184 · Cited by 382 · see citing works
Publication of NIH funded trials registered in ClinicalTrials.gov: cross sectional analysis
Ross, J. S., Tse, T., Zarin, D. A., Xu, H., Zhou, L., & Krumholz, H. M. (2012) · BMJ · doi:10.1136/bmj.d7292 · Cited by 371 · see citing works
Gartlehner, G., Hansen, R. A., Morgan, L. C., Thaler, K., Lux, L., Van Noord, M., Mager, U., Thieda, P., Gaynes, B. N., Wilkins, T., et al. (2011) · Annals of Internal Medicine · doi:10.7326/0003-4819-155-11-201112060-00009 · Cited by 355 · see citing works
Noury, J. C. L., Nardo, J. M., Healy, D., Jureidini, J. N., Raven, M. K., Tufănaru, C., & Abi‐Jaoude, E. (2015) · BMJ · doi:10.1136/bmj.h4320 · Cited by 329 · see citing works
The False-positive to False-negative Ratio in Epidemiologic Studies
Ioannidis, J. P. A., Tarone, R., & McLaughlin, J. K. (2011) · Epidemiology · doi:10.1097/ede.0b013e31821b506e · Cited by 321 · see citing works
Self-citationReproducible Research Practices and Transparency across the Biomedical Literature
Iqbal, S. A., Wallach, J. D., Khoury, M. J., Schully, S. D., & Ioannidis, J. P. A. (2016) · PLOS Biology · doi:10.1371/journal.pbio.1002333 · Cited by 310 · see citing works
Self-citationEvaluation of excess significance bias in animal studies of neurological diseases
Tsilidis, K. K., Panagiotou, O. A., Sena, E. S., Aretouli, E., Evangelou, E., Howells, D. W., Salman, R. A.-S., Macleod, M. R., & Ioannidis, J. P. A. (2013) · PLoS Biology · doi:10.1371/journal.pbio.1001609 · Cited by 295 · see citing works
Self-citationThe need for randomization in animal trials: an overview of systematic reviews
Hirst, J. A., Howick, J., Aronson, J. K., Roberts, N., Perera, R., Koshiaris, C., & Heneghan, C. (2014) · PLoS ONE · doi:10.1371/journal.pone.0098856 · Cited by 294 · see citing works
META-ANALYSIS IN THE DESIGN AND MONITORING OF CLINICAL TRIALS
DerSimonian, R. (1996) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19960630)15:12 · Cited by 257 · see citing works
Systematic reviews and meta-analyses on treatment of asthma: critical evaluation
Jadad, A. R. (2000) · BMJ · doi:10.1136/bmj.320.7234.537 · Cited by 257 · see citing works
Effectiveness of antidepressants: an evidence myth constructed from a thousand randomized trials?
Ioannidis, J. P. (2008) · Philosophy, Ethics, and Humanities in Medicine · doi:10.1186/1747-5341-3-14 · Cited by 253 · see citing works
Self-citationOverlapping meta-analyses on the same topic: survey of published studies
Siontis, K. C., Hernandez‐Boussard, T., & Ioannidis, J. P. A. (2013) · BMJ · doi:10.1136/bmj.f4501 · Cited by 249 · see citing works
Self-citationYank, V., Rennie, D., & Bero, L. A. (2007) · BMJ · doi:10.1136/bmj.39376.447211.be · Cited by 235 · see citing works
Meta‐research: The art of getting it wrong
Ioannidis, J. P. A. (2010) · Research Synthesis Methods · doi:10.1002/jrsm.19 · Cited by 231 · see citing works
Self-citationEmpirical Evidence of Bias in the Design of Experimental Stroke Studies
Crossley, N. A., Sena, E. S., Göhler, J., Horn, J., van der Worp, H. B., Bath, P. M., Macleod, M. R., & Dirnagl, U. (2008) · Stroke · doi:10.1161/strokeaha.107.498725 · Cited by 231 · see citing works
How evidence‐based medicine is failing due to biased trials and selective publication
Every‐Palmer, S., & Howick, J. (2014) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.12147 · Cited by 228 · see citing works
Quality of reporting in systematic reviews of adverse events: systematic review
Zorzela, L., Golder, S., Liu, Y., Pilkington, K., Hartling, L., Joffe, A., Loke, Y., & Vohra, S. (2014) · BMJ · doi:10.1136/bmj.f7668 · Cited by 217 · see citing works
Cuatrecasas, P. (2006) · Journal of Clinical Investigation · doi:10.1172/jci29999 · Cited by 209 · see citing works
Jadad, A. R., & McQuay, H. J. (1996) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(95)00062-3 · Cited by 205 · see citing works
Ensuring the integrity of clinical practice guidelines: a tool for protecting patients
Lenzer, J., Hoffman, J. R., Furberg, C. D., & Ioannidis, J. P. A. (2013) · BMJ · doi:10.1136/bmj.f5535 · Cited by 191 · see citing works
Self-citationMeta-Analyses in Orthopaedic Surgery
Bhandari, M., Morrow, F., Kulkarni, A. V., & Tornetta, P. (2001) · Journal of Bone and Joint Surgery · doi:10.2106/00004623-200101000-00003 · Cited by 190 · see citing works
PROSPERO at one year: an evaluation of its utility
Booth, A., Clarke, M. J., Dooley, G., Ghersi, D., Moher, D., Petticrew, M. P., & Stewart, L. A. (2013) · Systematic Reviews · doi:10.1186/2046-4053-2-4 · Cited by 187 · see citing works
Reanalyses of Randomized Clinical Trial Data
Ebrahim, S., Sohani, Z. N., Montoya, L., Agarwal, A., Thorlund, K., Mills, E. J., & Ioannidis, J. P. A. (2014) · JAMA · doi:10.1001/jama.2014.9646 · Cited by 186 · see citing works
Self-citationWhy we can't trust clinical guidelines
Lenzer, J. (2013) · BMJ · doi:10.1136/bmj.f3830 · Cited by 157 · see citing works
Krauth, D., Woodruff, T. J., & Bero, L. A. (2013) · Environmental Health Perspectives · doi:10.1289/ehp.1206389 · Cited by 147 · see citing works
Williamson, P. R., & Clarke, M. J. (2012) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.ed000041 · Cited by 143 · see citing works
A systematic evaluation of the quality of meta-analyses in the critical care literature
Delaney, A. P., Bagshaw, S. M., Ferland, A., Manns, B., Laupland, K. B., & Doig, C. J. (2005) · Critical Care · doi:10.1186/cc3803 · Cited by 142 · see citing works
Woodruff, T. J., Sutton, P., & The Navigation Guide Work Group (2011) · Health Affairs · doi:10.1377/hlthaff.2010.1219 · Cited by 134 · see citing works
The automation of systematic reviews
Tsafnat, G., Dunn, A., Glasziou, P., & Coiera, E. (2013) · BMJ · doi:10.1136/bmj.f139 · Cited by 131 · see citing works
Evaluating Meta-analyses in the General Surgical Literature: A Critical Appraisal
Dixon, E., Hameed, M., Sutherland, F., Cook, D. J., & Doig, C. (2005) · Annals of Surgery · doi:10.1097/01.sla.0000154258.30305.df · Cited by 129 · see citing works
Discussion Sections in Reports of Controlled Trials Published in General Medical Journals
Clarke, M. J. (2002) · JAMA · doi:10.1001/jama.287.21.2799 · Cited by 123 · see citing works
Bibliometrics: Is your most cited work your best?
Ioannidis, J. P. A., Boyack, K. W., Small, H. G., et al. (2014) · Nature · doi:10.1038/514561a · Cited by 123 · see citing works
Self-citationThe Power of Meta-Analysis in Genome-Wide Association Studies
Panagiotou, O. A., Willer, C. J., Hirschhorn, J. N., & Ioannidis, J. P. A. (2013) · Annual Review of Genomics and Human Genetics · doi:10.1146/annurev-genom-091212-153520 · Cited by 122 · see citing works
Self-citationPublication of Trials Funded by the National Heart, Lung, and Blood Institute
Gordon, D., Taddei‐Peters, W. C., Mascette, A. M., Antman, M., Kaufmann, P. G., & Lauer, M. S. (2013) · New England Journal of Medicine · doi:10.1056/nejmsa1300237 · Cited by 120 · see citing works
Ushering in a New Era of Open Science Through Data Sharing
Ross, J. S., & Krumholz, H. M. (2013) · JAMA · doi:10.1001/jama.2013.1299 · Cited by 118 · see citing works
How to make more published research true
Ioannidis, J. P. A. (2015) · Acimed · Cited by 115 · see citing works
Self-citationShea, B., Moher, D., Graham, I. D., Pham, B., & Tugwell, P. (2002) · Evaluation & the Health Professions · doi:10.1177/0163278702025001008 · Cited by 109 · see citing works
Ebrahim, S., Bance, S., Athale, A., Malachowski, C., & Ioannidis, J. P. A. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.08.021 · Cited by 107 · see citing works
Self-citationContent area experts as authors: helpful or harmful for systematic reviews and meta-analyses?
Gøtzsche, P. C., & Ioannidis, J. P. A. (2012) · BMJ · doi:10.1136/bmj.e7031 · Cited by 106 · see citing works
Self-citationThe Proposed Rule for U.S. Clinical Trial Registration and Results Submission
Zarin, D. A., Tse, T., & Sheehan, J. (2014) · New England Journal of Medicine · doi:10.1056/nejmsr1414226 · Cited by 102 · see citing works
Cochrane and non-Cochrane systematic reviews in leading orthodontic journals: a quality paradigm?
Fleming, P. S., Seehra, J., Polychronopoulou, A., Fedorowicz, Z., & Pandis, N. (2013) · The European Journal of Orthodontics · doi:10.1093/ejo/cjs016 · Cited by 95 · see citing works
Systematic Reviews of Animal Studies; Missing Link in Translational Research?
van Luijk, J. A. K. R., Bakker, B., Rovers, M. M., Ritskes-Hoitinga, M. J., de Vries, R. B. M., & Leenaars, M. (2014) · PLoS ONE · doi:10.1371/journal.pone.0089981 · Cited by 89 · see citing works
Financial Conflicts of Interest and Conclusions About Neuraminidase Inhibitors for Influenza
Dunn, A. G., Arachi, D., Hudgins, J. D., Tsafnat, G., Coiera, E. W., & Bourgeois, F. T. (2014) · Annals of Internal Medicine · doi:10.7326/m14-0933 · Cited by 83 · see citing works
Twenty Years of Meta-Analyses in Orthopaedic Surgery: Has Quality Kept Up with Quantity?
Dijkman, B. G., Abouali, J., Dr. Bauke Kooistra, O. S., Conter, H. J., Poolman, R. W., Kulkarni, A. V., Tornetta, P., & Bhandari, M. (2010) · Journal of Bone and Joint Surgery · doi:10.2106/jbjs.i.00251 · Cited by 82 · see citing works
Jørgensen, A. W., Maric, K. L., Tendal, B., Faurschou, A., & Gøtzsche, P. C. (2008) · BMC Medical Research Methodology · doi:10.1186/1471-2288-8-60 · Cited by 80 · see citing works
Levin, A. R. (2001) · Annals of Internal Medicine · doi:10.7326/0003-4819-135-4-200108210-00035 · Cited by 74 · see citing works
Reviewing the reviews. How strong is the evidence? How clear are the conclusions?
Ezzo, J., Bausell, B., Moerman, D. E., Berman, B., & Hadhazy, V. (2001) · PubMed · Cited by 72 · see citing works
Mallett, S. V. (2003) · BMJ Evidence-Based Medicine · doi:10.1136/ebm.8.4.100 · Cited by 71 · see citing works
Survey of systematic reviews in dentistry
Bader, J. D., & Ismail, A. I. (2004) · The Journal of the American Dental Association · doi:10.14219/jada.archive.2004.0212 · Cited by 71 · see citing works
An international survey indicated that unpublished systematic reviews exist
Tricco, A. C., Pham, B., Brehaut, J., Tetroe, J., Cappelli, M., Hopewell, S., Lavis, J. N., Berlin, J. A., & Moher, D. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2008.09.014 · Cited by 64 · see citing works
Primer: strengths and weaknesses of meta-analysis
Finckh, A., & Tramèr, M. R. (2008) · Nature Clinical Practice Rheumatology · doi:10.1038/ncprheum0732 · Cited by 62 · see citing works
The Geometric Increase in Meta-Analyses from China in the Genomic Era
Ioannidis, J. P. A., Chang, C. Q., Lam, T. K., Schully, S. D., & Khoury, M. J. (2013) · PLoS ONE · doi:10.1371/journal.pone.0065602 · Cited by 62 · see citing works
Self-citationWee, B. L., Hadley, G., & Derry, S. (2008) · BMC Palliative Care · doi:10.1186/1472-684x-7-13 · Cited by 61 · see citing works
Multisystem failure: the story of anti-influenza drugs
Jefferson, T., & Doshi, P. (2014) · BMJ · doi:10.1136/bmj.g2263 · Cited by 60 · see citing works
Sox, H. C., & Rennie, D. (2008) · Annals of Internal Medicine · doi:10.7326/0003-4819-149-4-200808190-00012 · Cited by 57 · see citing works
Following 411 Cochrane Protocols to Completion: A Retrospective Cohort Study
Tricco, A. C., Brehaut, J., Chen, M. H., & Moher, D. (2008) · PLoS ONE · doi:10.1371/journal.pone.0003684 · Cited by 52 · see citing works
Fugh‐Berman, A., McDonald, C. P., Bell, A. M., Bethards, E. C., & Scialli, A. R. (2011) · PLoS Medicine · doi:10.1371/journal.pmed.1000425 · Cited by 51 · see citing works
Ezzo, J., Bausell, B., Moerman, D. E., Berman, B., & Hadhazy, V. (2001) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462301107014 · Cited by 48 · see citing works
The COMET initiative database: progress and activities update (2014)
Gargon, E., Williamson, P. R., Altman, D. G., Blazeby, J. M., & Clarke, M. J. (2015) · Trials · doi:10.1186/s13063-015-1038-x · Cited by 46 · see citing works
Sea Change in Open Science and Data Sharing
Krumholz, H. M., Gross, C. P., Blount, K. L., Ritchie, J. D., Hodshon, B., Lehman, R., & Ross, J. S. (2014) · Circulation Cardiovascular Quality and Outcomes · doi:10.1161/circoutcomes.114.001166 · Cited by 44 · see citing works
Hartog, C. S., Skupin, H., Natanson, C., Sun, J., & Reinhart, K. (2012) · Intensive Care Medicine · doi:10.1007/s00134-012-2614-0 · Cited by 42 · see citing works
Why data sharing should be the expected norm
Krumholz, H. M. (2015) · BMJ · doi:10.1136/bmj.h599 · Cited by 38 · see citing works
Labour's health policy is having paradoxical effect in Iberian countries
Badía, J. G. (1999) · BMJ · doi:10.1136/bmj.318.7181.466 · Cited by 37 · see citing works
Susan Mallett, D., & Mike Clarke, D. (2003) · ACP Journal Club · doi:10.7326/acpjc-2003-139-1-a11 · Cited by 37 · see citing works
Vamvakas, E. C. (2007) · Vox Sanguinis · doi:10.1111/j.1423-0410.2007.00959.x · Cited by 36 · see citing works
Serghiou, S., Patel, C. J., Tan, Y. Y., Koay, P., & Ioannidis, J. P. A. (2015) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.09.004 · Cited by 35 · see citing works
Self-citationHigh reprint orders in medical journals and pharmaceutical industry funding: case-control study
Handel, A. E., Patel, S. V., Pakpoor, J., et al. (2012) · BMJ · doi:10.1136/bmj.e4212 · Cited by 34 · see citing works
Ioannidis, J. P. A., Horbar, J. D., Ovelman, C., Brosseau, Y., Thorlund, K., Buus‐Frank, M. E., Mills, E. J., & Soll, R. (2015) · BMJ · doi:10.1136/bmj.h72 · Cited by 33 · see citing works
Self-citationDissemination Bias in Systematic Reviews of Animal Research: A Systematic Review
Mueller, K. F., Briel, M., Strech, D., Meerpohl, J. J., Lang, B., Motschall, E., Gloy, V., Lamontagne, F., & Bassler, D. (2014) · PLoS ONE · doi:10.1371/journal.pone.0116016 · Cited by 31 · see citing works
Tatsioni, A., Siontis, G. C. M., & Ioannidis, J. P. A. (2010) · Journal of General Internal Medicine · doi:10.1007/s11606-010-1360-7 · Cited by 27 · see citing works
Self-citationDevelopment of a combined database for meta‐epidemiological research
Savović, J., Harris, R., Wood, L., Beynon, R., Altman, D. G., Als-Nielsen, B. E. T., Balk, E. M., Deeks, J. J., Gluud, L. L., Gluud, C. N., et al. (2010) · Research Synthesis Methods · doi:10.1002/jrsm.18 · Cited by 25 · see citing works
Self-citationMost meta-analyses of drug interventions have narrow scopes and many focus on specific agents
Haidich, A., Pilalas, D., Contopoulos‐Ioannidis, D. G., & Ioannidis, J. P. A. (2013) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2012.10.014 · Cited by 24 · see citing works
Self-citationTrust but Verify: Trial Registration and Determining Fidelity to the Protocol
Zarin, D. A., & Tse, T. (2013) · Annals of Internal Medicine · doi:10.7326/0003-4819-159-1-201307020-00011 · Cited by 23 · see citing works
Turner, E. H., Moreno, S. G., & Sutton, A. J. (2009) · The Lancet · doi:10.1016/s0140-6736(09)60975-2 · Cited by 23 · see citing works
Lessons Learned from a Prospective Meta‐analysis
Margitić, S. E., Morgan, T. M., Sager, M. A., & Furberg, C. D. (1995) · Journal of the American Geriatrics Society · doi:10.1111/j.1532-5415.1995.tb05820.x · Cited by 19 · see citing works
Ioannidis, J. P. A. (2009) · The Lancet · doi:10.1016/s0140-6736(09)60974-0 · Cited by 19 · see citing works
Self-citationMills, E. J., Druyts, E., Thorlund, Humphreys, Lion, & Cooper, C. L. (2013) · Clinical Epidemiology · doi:10.2147/clep.s44273 · Cited by 17 · see citing works
Evaluation of meta‐analyses in the otolaryngological literature
Rudmik, L. R., Walen, S. G., Dixon, E., & Dort, J. C. (2008) · Otolaryngology · doi:10.1016/j.otohns.2008.03.020 · Cited by 16 · see citing works
Trusting researchers to use open trial registers such as PROSPERO responsibly
Booth, A., & Stewart, L. A. (2013) · BMJ · doi:10.1136/bmj.f5870 · Cited by 15 · see citing works
The methodology for developing a prospective meta-analysis in the family planning community
Turok, D. K., Espey, E. L., Edelman, A. B., Lotke, P. S., Lathrop, E., Teal, S. B., Jacobson, J. C., Simonsen, S. E., & Schulz, K. F. (2011) · Trials · doi:10.1186/1745-6215-12-104 · Cited by 14 · see citing works
The case for duplication of meta-analyses and systematic reviews
Krumholz, H. M. (2013) · BMJ · doi:10.1136/bmj.f5506 · Cited by 9 · see citing works
Antidepressants and suicide. The history of a controversy--and where it stands today.
(2007) · PubMed · Cited by 1 · see citing works
Flagged first, then by citations
Research works printed but not linked: 5
- [11]11. Olkin I. Meta-analysis: current issues in research synthesis. Stat Med. 1996;15(12):1253-1257; discussion 1259-1262.
- [14]14. Iqbal SA, Wallach JD, Khoury MJ, Schully SD, Ioannidis JP. Re- producible research practices and transparency across the biomed- ical literature. PLoS Biol. 2016;14(1):e1002333.
- [40]40. Ebrahim S, Montoya L, el Din MK, et al. Secondary publications from randomized controlled trials: a meta-epidemiological study. J Clin Epidemiol. In press.
- [45]45. Horrobin DF. Beyond conflict of interest. Non-financial con- flicts of interest are more serious than financial conflicts. BMJ. 1999;318(7181):466.
- [89]89. Moher D, Tetzlaff J, Tricco AC, Sampson M, Altman DG. Epi- demiology and reporting characteristics of systematic reviews. PLoS Med. 2007;4(3):e78.
Cited by
Sorted by citations
SANRA—a scale for the quality assessment of narrative review articles
Baethge, C., Goldbeck-Wood, S., & Mertens, S. (2019) · Research Integrity and Peer Review · doi:10.1186/s41073-019-0064-8 · Cited by 3,469 · see citing works
A communal catalogue reveals Earth's multiscale microbial diversity
Thompson, L. R., Sanders, J. G., McDonald, D., Amir, A., Ladau, J., Locey, K. J., Prill, R. J., Tripathi, A., Gibbons, S. M., Ackermann, G., et al. (2017) · Nature · doi:10.1038/nature24621 · Cited by 2,684 · see citing works
Sanderson, E., Glymour, M. M., Holmes, M. V., Kang, H., Morrison, J., Munafò, M. R., Palmer, T., Schooling, C. M., Wallace, C., Zhao, Q., et al. (2022) · Nature Reviews Methods Primers · doi:10.1038/s43586-021-00092-5 · Cited by 2,229 · see citing works
Meta-analysis and the science of research synthesis
Gurevitch, J., Koricheva, J., Nakagawa, S., & Stewart, G. (2018) · Nature · doi:10.1038/nature25753 · Cited by 1,709 · see citing works
Time to challenge the spurious hierarchy of systematic over narrative reviews?
Greenhalgh, T., Thorne, S., & Malterud, K. (2018) · European Journal of Clinical Investigation · doi:10.1111/eci.12931 · Cited by 1,267 · see citing works
Munn, Z., Stern, C., Aromataris, E., Lockwood, C., & Jordan, Z. (2018) · BMC Medical Research Methodology · doi:10.1186/s12874-017-0468-4 · Cited by 1,164 · see citing works
How to properly use the PRISMA Statement
Sarkis-Onofre, R., Catalá-López, F., Aromataris, E., & Lockwood, C. (2021) · Systematic Reviews · doi:10.1186/s13643-021-01671-z · Cited by 1,104 · see citing works
Hartwig, F. P., Davies, N. M., Hemani, G., & Davey Smith, G. (2016) · International Journal of Epidemiology · doi:10.1093/ije/dyx028 · Cited by 860 · see citing works
Gates, M., Gates, A., Pieper, D., Fernandes, R. M., Tricco, A. C., Moher, D., Brennan, S. E., Li, T., Pollock, M., Lunny, C., et al. (2022) · BMJ · doi:10.1136/bmj-2022-070849 · Cited by 856 · see citing works
Page, M. J., & Moher, D. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0663-8 · Cited by 769 · see citing works
Salameh, J.-P., Bossuyt, P. M., McGrath, T. A., Thombs, B. D., Hyde, C. J., Macaskill, P., Deeks, J. J., Leeflang, M., Korevaar, D. A., Whiting, P., et al. (2020) · BMJ · doi:10.1136/bmj.m2632 · Cited by 719 · see citing works
Meta-evaluation of meta-analysis: ten appraisal questions for biologists
Nakagawa, S., Noble, D. W. A., Senior, A. M., & Lagisz, M. (2017) · BMC Biology · doi:10.1186/s12915-017-0357-7 · Cited by 579 · see citing works
The gut microbiota: a key factor in the therapeutic effects of (poly)phenols
Espín, J. C., González-Sarrías, A., & Tomás-Barberán, F. A. (2017) · Biochemical Pharmacology · doi:10.1016/j.bcp.2017.04.033 · Cited by 603 · see citing works
What causes psychosis? An umbrella review of risk and protective factors
Radua, J., Ramella‐Cravaro, V., Ioannidis, J. P. A., Reichenberg, A., Phiphopthatsanee, N., Amir, T., Yenn Thoo, H., Oliver, D., Davies, C., Morgan, C., et al. (2018) · World Psychiatry · doi:10.1002/wps.20490 · Cited by 553 · see citing works
Self-citationDietary fiber and health outcomes: an umbrella review of systematic reviews and meta-analyses
Veronese, N., Solmi, M., Caruso, M. G., Giannelli, G., Osella, A. R., Evangelou, E., Maggi, S., Fontana, L., Stubbs, B., & Tzoulaki, I. (2018) · The American Journal of Clinical Nutrition · doi:10.1093/ajcn/nqx082 · Cited by 536 · see citing works
Registration of systematic reviews in PROSPERO: 30,000 records and counting
Page, M. J., Shamseer, L., & Tricco, A. C. (2018) · Systematic Reviews · doi:10.1186/s13643-018-0699-4 · Cited by 509 · see citing works
Preventive psychiatry: a blueprint for improving the mental health of young people
Fusar‐Poli, P., Correll, C. U., Arango, C., Berk, M., Patel, V., & Ioannidis, J. P. A. (2021) · World Psychiatry · doi:10.1002/wps.20869 · Cited by 497 · see citing works
Self-citationPrevention of Psychosis: Advances in Detection, Prognosis, and Intervention
Fusar-Poli, P., Salazar de Pablo, G., Correll, C. U., Meyer-Lindenberg, A., Millan, M. J., Borgwardt, S., Galderisi, S., Bechdolf, A., Pfennig, A., Kessing, L. V., et al. (2020) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2019.4779 · Cited by 476 · see citing works
The neurocognitive profile of attention-deficit/hyperactivity disorder: a review of meta-analyses
Pievsky, M. A., & McGrath, R. E. (2018) · Archives of Clinical Neuropsychology · doi:10.1093/arclin/acx055 · Cited by 401 · see citing works
The impact of digital technology use on adolescent well-being
Dienlin, T., & Johannes, N. (2020) · Dialogues in Clinical Neuroscience · doi:10.31887/dcns.2020.22.2/tdienlin · Cited by 393 · see citing works
Statistical Inference as Severe Testing: How to Get Beyond the Statistics Wars
Mayo, D. G. (2018) · doi:10.1017/9781107286184 · Cited by 376 · see citing works
Hennessy, E. A., & Johnson, B. T. (2020) · Research Synthesis Methods · doi:10.1002/jrsm.1390 · Cited by 348 · see citing works
Galante, J., Friedrich, C., Dawson, A. F., Modrego-Alarcón, M., Gebbing, P., Delgado-Suárez, I., Gupta, R., Dean, L., Dalgleish, T., White, I. R., et al. (2021) · PLOS Medicine · doi:10.1371/journal.pmed.1003481 · Cited by 319 · see citing works
Hoffmann, F., Allers, K., Rombey, T., Helbach, J., Hoffmann, A., Mathes, T., & Pieper, D. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.05.022 · Cited by 278 · see citing works
Soysal, P., Hurst, C., Demurtas, J., Firth, J., Howden, R., Yang, L., Tully, M. A., Koyanagi, A., Ilie, P. C., López-Sánchez, G. F., et al. (2021) · Journal of Sport and Health Science · doi:10.1016/j.jshs.2020.06.009 · Cited by 283 · see citing works
Lunny, C., Pieper, D., Thabet, P., & Kanji, S. (2021) · BMC Medical Research Methodology · doi:10.1186/s12874-021-01269-y · Cited by 252 · see citing works
McKeown, S., & Mir, Z. M. (2021) · Systematic Reviews · doi:10.1186/s13643-021-01583-y · Cited by 251 · see citing works
Lack of evidence to favor specific preventive interventions in psychosis: a network meta‐analysis
Davies, C., Cipriani, A., Ioannidis, J. P. A., Radua, J., Stahl, D., Provenzani, U., McGuire, P., & Fusar‐Poli, P. (2018) · World Psychiatry · doi:10.1002/wps.20526 · Cited by 229 · see citing works
Self-citationError rates of human reviewers during abstract screening in systematic reviews
Wang, Z., Nayfeh, T., Tetzlaff, J., O’Blenis, P., & Murad, M. H. (2020) · PLOS ONE · doi:10.1371/journal.pone.0227742 · Cited by 217 · see citing works
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T., Mulrow, C. D., Shamseer, L., & Moher, D. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.11.010 · Cited by 211 · see citing works
Umbrella reviews: what they are and why we need them
Papatheodorou, S. (2019) · European Journal of Epidemiology · doi:10.1007/s10654-019-00505-6 · Cited by 201 · see citing works
Scientific citations favor positive results: a systematic review and meta-analysis.
Duyx, B., Urlings, M. J. E., Swaen, G. M. H., Bouter, L. M., & Zeegers, M. P. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.06.002 · Cited by 197 · see citing works
A review of recent publication trends from top publishing countries
Fontelo, P., & Liu, F. (2018) · Systematic Reviews · doi:10.1186/s13643-018-0819-1 · Cited by 186 · see citing works
Evidence map of crop diversification strategies at the global scale
Beillouin, D., Ben-Ari, T., & Makowski, D. (2019) · Environmental Research Letters · doi:10.1088/1748-9326/ab4449 · Cited by 192 · see citing works
Dinu, M., Pagliai, G., Angelino, D., Rosi, A., Dall'Asta, M., Bresciani, L., Ferraris, C., Guglielmetti, M., Godos, J., Del Bo', C., et al. (2020) · Advances in Nutrition · doi:10.1093/advances/nmaa006 · Cited by 164 · see citing works
Selecting and implementing overview methods: implications from five exemplar overviews
Pollock, A., Campbell, P., Brunton, G., Hunt, H., & Estcourt, L. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0534-3 · Cited by 134 · see citing works
Overviews of systematic reviews: great promise, greater challenge
McKenzie, J. E., & Brennan, S. E. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0582-8 · Cited by 134 · see citing works
Practical recommendations to conduct a neuroimaging meta-analysis for neuropsychiatric disorders
Tahmasian, M., Sepehry, A. A., Samea, F., Khodadadifar, T., Soltaninejad, Z., Javaheripour, N., Khazaie, H., Zarei, M., Eickhoff, S. B., & Eickhoff, C. R. (2019) · Human Brain Mapping · doi:10.1002/hbm.24746 · Cited by 126 · see citing works
Pollock, M., Fernandes, R. M., Newton, A. S., Scott, S. D., & Hartling, L. (2019) · Systematic Reviews · doi:10.1186/s13643-018-0768-8 · Cited by 132 · see citing works
A guide to prospective meta-analysis
Seidler, A. L., Hunter, K. E., Cheyne, S., Ghersi, D., Berlin, J. A., & Askie, L. (2019) · BMJ · doi:10.1136/bmj.l5342 · Cited by 125 · see citing works
Low-carbohydrate diets for overweight and obesity: A systematic review of the systematic reviews
Churuangsuk, C., Kherouf, M., Combet, E., & Lean, M. (2018) · Obesity Reviews · doi:10.1111/obr.12744 · Cited by 118 · see citing works
Reproducible research practices are underused in systematic reviews of biomedical interventions
Page, M. J., Altman, D. G., Shamseer, L., McKenzie, J. E., Ahmadzai, N., Wolfe, D., Yazdi, F., Catalá-López, F., Tricco, A. C., & Moher, D. (2018) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.10.017 · Cited by 115 · see citing works
Systematic reviews: guidance relevant for studies of older people
Shenkin, S. D., Harrison, J. K., Wilkinson, T., Dodds, R. M., & Ioannidis, J. P. A. (2017) · Age and Ageing · doi:10.1093/ageing/afx105 · Cited by 109 · see citing works
Self-citationHennessy, E. A., Johnson, B. T., & Keenan, C. (2019) · Applied Psychology: Health and Well-Being · doi:10.1111/aphw.12169 · Cited by 91 · see citing works
“For such evidence to be put to maximal use, meta-reviews are important and even necessary (Bastian, Glasziou, & Chalmers, 2010; Ioannidis, 2016).”
BackgroundCierco Jimenez, R., Lee, T., Rosillo, N., Cordova, R., Cree, I. A., Gonzalez, A., & Indave Ruiz, B. I. (2022) · BMC Medical Research Methodology · doi:10.1186/s12874-022-01805-4 · Cited by 85 · see citing works
Page, M. J., Altman, D. G., McKenzie, J. E., Shamseer, L., Ahmadzai, N., Wolfe, D., Yazdi, F., Catalá-López, F., Tricco, A. C., & Moher, D. (2018) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.11.022 · Cited by 82 · see citing works
Meta-analyses indexed in PsycINFO had a better completeness of reporting when they mention PRISMA
Leclercq, V., Beaudart, C., Ajamieh, S., Rabenda, V., Tirelli, E., & Bruyère, O. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.06.014 · Cited by 69 · see citing works
Abbott, R., Bethel, A., Rogers, M., Whear, R., Orr, N., Shaw, L., Stein, K., & Thompson Coon, J. (2022) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2021-111710 · Cited by 56 · see citing works
Effect of hand hygiene on infectious diseases in the office workplace: a systematic review
Zivich, P. N., Gancz, A. S., & Aiello, A. E. (2018) · American Journal of Infection Control · doi:10.1016/j.ajic.2017.10.006 · Cited by 58 · see citing works
O’Donnell, M., Dev, A. S., Antonoplis, S., Baum, S. M., Benedetti, A. H., Brown, N. D., Carrillo, B., Choi, A. L., Connor, P., Donnelly, K., et al. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2103313118 · Cited by 56 · see citing works
Protocol registration improves reporting quality of systematic reviews in dentistry
dos Santos, M. B. F., Agostini, B. A., Bassani, R., Pereira, G. K. R., & Sarkis-Onofre, R. (2020) · BMC Medical Research Methodology · doi:10.1186/s12874-020-00939-7 · Cited by 61 · see citing works
Protect us from poor-quality medical research
ESHRE Capri Workshop Group, Ioannidis, J. P. A., Bhattacharya, S., Evers, J. L. H., der Veen, F. V., Somigliana, E., Barratt, C. L. R., Bontempi, G., Baird, D. T., Crosignani, P., et al. (2018) · Human Reproduction · doi:10.1093/humrep/dey056 · Cited by 59 · see citing works
Self-citationSystematic reviews in dentistry: Current status, epidemiological and reporting characteristics
Bassani, R., Pereira, G. K. R., Page, M. J., Tricco, A. C., Moher, D., & Sarkis-Onofre, R. (2019) · Journal of Dentistry · doi:10.1016/j.jdent.2019.01.014 · Cited by 53 · see citing works
Integrity of randomized controlled trials: challenges and solutions
Li, W., van Wely, M., Gurrin, L., & Mol, B. W. (2020) · Fertility and Sterility · doi:10.1016/j.fertnstert.2020.04.018 · Cited by 49 · see citing works
Gómez-García, F., Ruano, J., Gay-Mimbrera, J., Aguilar-Luque, M., Sanz-Cabanillas, J. L., Alcalde-Mellado, P., Maestre-López, B., Carmona-Fernández, P. J., González-Padilla, M., García-Nieto, A. V., et al. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.08.015 · Cited by 37 · see citing works
Solmi, M., Correll, C. U., Carvalho, A. F., & Ioannidis, J. P. A. (2018) · Epidemiology and Psychiatric Sciences · doi:10.1017/s204579601800032x · Cited by 44 · see citing works
Self-citationQuestions and answers about conducting systematic reviews in sport and exercise psychology
Gunnell, K., Poitras, V. J., & Tod, D. (2020) · International Review of Sport and Exercise Psychology · doi:10.1080/1750984x.2019.1695141 · Cited by 45 · see citing works
Schuit, E., Li, A. H., & Ioannidis, J. P. A. (2019) · International Journal of Epidemiology · doi:10.1093/ije/dyy239 · Cited by 27 · see citing works
Self-citationAutomation of Systematic Reviews with Large Language Models
Cao, C., Arora, R., Cento, P., Budak, A., Manta, K., Farahani, E., Cecere, M., Selemon, A., Sang, J., Gong, L. X., et al. (2025) · medRxiv · doi:10.1101/2025.06.13.25329541 · Cited by 33 · see citing works
Enhancing reproducibility: Failures from Reproducibility Initiatives underline core challenges
Mullane, K., & Williams, M. (2017) · Biochemical Pharmacology · doi:10.1016/j.bcp.2017.04.008 · Cited by 28 · see citing works
Discriminating between empirical studies and nonempirical works using automated text classification
Langlois, A., Nie, J., Thomas, J., Hong, Q. N., & Pluye, P. (2018) · Research Synthesis Methods · doi:10.1002/jrsm.1317 · Cited by 26 · see citing works
Chalmers, H., Brown, J., & Koryakina, A. (2024) · Applied Linguistics Review · doi:10.1515/applirev-2022-0190 · Cited by 26 · see citing works
Lagisz, M., Yang, Y., Young, S., & Nakagawa, S. (2025) · Research Synthesis Methods · doi:10.1017/rsm.2024.6 · Cited by 24 · see citing works
Increase value and reduce waste in research on psychological therapies
Cristea, I. A., & Naudet, F. (2019) · Behaviour Research and Therapy · doi:10.1016/j.brat.2019.103479 · Cited by 22 · see citing works
Re-Evaluating Evidence for Best Practice in Paediatric Speech-Language Pathology
Dodd, B. (2021) · Folia Phoniatrica et Logopaedica · doi:10.1159/000505265 · Cited by 12 · see citing works
Gutierrez-Arias, R., Pieper, D., Lunny, C., Torres-Castro, R., Aguilera-Eguía, R., & Seron, P. (2023) · BMJ Open · doi:10.1136/bmjopen-2022-069906 · Cited by 7 · see citing works
El Ansari, W., AlRumaihi, K., El-Ansari, K., Arafa, M., Elbardisi, H., Majzoub, A., Shamsodini, A., & Al Ansari, A. (2023) · Arab Journal of Urology · doi:10.1080/2090598x.2022.2113127 · Cited by 6 · see citing works
Predictors of Higher Quality of Systematic Reviews Addressing Nutrition and Cancer Prevention
Storman, D., Koperny, M., Zając, J., Polak, M., Weglarz, P., Bochenek-Cibor, J., Swierz, M. J., Staskiewicz, W., Gorecka, M., Skuza, A., et al. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph19010506 · Cited by 6 · see citing works
AIM review tool: artificial intelligence for smarter systematic review screening
Mena, S., Rituerto-González, E., Coutts, F., von Trott, J., Jacobs, G. R., Bryant, L., Moles, L., Sirbu, N., Promet, L., Oliver, D., et al. (2026) · npj Artificial Intelligence · doi:10.1038/s44387-026-00080-8
Singh, A., Podder, S., Khairnar, M., & Thakkar, T. (2026) · European Journal of Dental Education · doi:10.1111/eje.70264
- Authors
- IOANNIDIS, J. P. A.
- Affiliations
- Stanford Medicine (US)
- Corresponding author
- IOANNIDIS, JOHN P.A.
- ORCID
- IOANNIDIS, JOHN P.A.
- Venue
- The Milbank Quarterly
- Publisher
- Wiley
- Published
- 2016-09
- Volume
- 94
- Issue
- 3
- Pages
- 485-514
- ISSN
- 0887-378X (ISSN-L)
- 1468-0009
- PMC
- PMC5020151
- OpenAlex
- W2520714999
- Cited by
- 1,483
- References
- 116
- Citations by year
- 2026: 122 · 2025: 150 · 2024: 125 · 2023: 165 · 2022: 184 · 2021: 191 · 2020: 166 · 2019: 141 · 2018: 142 · 2017: 81 · 2016: 15 · 2015: 1
- FWCI
- 43.35
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Milbank Quarterly (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Laura and John Arnold Foundation
- Stanford Bio-X
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Life Sciences › Pharmacology, Toxicology and Pharmaceutics › Pharmacology › Pharmaceutical industry and healthcare
- Social Sciences › Decision Sciences › Information Systems and Management › Academic Publishing and Open Access
- Keywords
- systematic reviews, meta-analysis, publication growth, redundancy, conflicts of interest, industry funding, research bias, evidence synthesis, candidate genes, evidence-based medicine, research integrity, contracting companies
- MeSH terms
- Antidepressive Agents, Genetics, Deception, Quality Control, Review Literature as Topic, Meta-Analysis as Topic, Bias, Randomized Controlled Trials as Topic, Evidence-Based Medicine
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
