Epidemiology and Reporting Characteristics of Systematic Reviews of Biomedical Research: A Cross-Sectional Study
Open accessThe DOI matches its Crossref record; 41 of the 41 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Systematic reviews (SRs) can help decision makers interpret the deluge of published biomedical literature. However, a SR may be of limited use if the methods used to conduct the SR are flawed, and reporting of the SR is incomplete. To our knowledge, since 2004 there has been no cross-sectional study of the prevalence, focus, and completeness of reporting of SRs across different specialties. Therefore, the aim of our study was to investigate the epidemiological and reporting characteristics of a more recent cross-section of SRs. METHODS AND FINDINGS: We searched MEDLINE to identify potentially eligible SRs indexed during the month of February 2014. Citations were screened using prespecified eligibility criteria. Epidemiological and reporting characteristics of a random sample of 300 SRs were extracted by one reviewer, with a 10% sample extracted in duplicate. We compared characteristics of Cochrane versus non-Cochrane reviews, and the 2014 sample of SRs versus a 2004 sample of SRs. We identified 682 SRs, suggesting that more than 8,000 SRs are being indexed in MEDLINE annually, corresponding to a 3-fold increase over the last decade. The majority of SRs addressed a therapeutic question and were conducted by authors based in China, the UK, or the US; they included a median of 15 studies involving 2,072 participants. Meta-analysis was performed in 63% of SRs, mostly using standard pairwise methods. Study risk of bias/quality assessment was performed in 70% of SRs but was rarely incorporated into the analysis (16%). Few SRs (7%) searched sources of unpublished data, and the risk of publication bias was considered in less than half of SRs. Reporting quality was highly variable; at least a third of SRs did not report use of a SR protocol, eligibility criteria relating to publication status, years of coverage of the search, a full Boolean search logic for at least one database, methods for data extraction, methods for study risk of bias assessment, a primary outcome, an abstract conclusion that incorporated study limitations, or the funding source of the SR. Cochrane SRs, which accounted for 15% of the sample, had more complete reporting than all other types of SRs. Reporting has generally improved since 2004, but remains suboptimal for many characteristics. CONCLUSIONS: An increasing number of SRs are being published, and many are poorly conducted and reported. Strategies are needed to help reduce this avoidable waste in research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures




Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Many systematic review reports include narrative summaries of the characteristics and risk of bias across all included studies.36”
From PRISMA 2020 explanation and elaboration: updated guidance and exemplars for reporting systematic reviews (2021) · cites this paper as [36]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Even for those studies which explicitly reference PRISMA, there is only slight, statistically non-significant evidence of improved reporting, as found by Page et al. [15].”
From PRISMA-S: an extension to the PRISMA Statement for Reporting Literature Searches in Systematic Reviews (2021) · cites this paper as [15]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The studies included in a systematic review will generally vary with respect to design, study populations, and risk of bias [1].”
From COSMOS-E: Guidance on conducting systematic reviews and meta-analyses of observational studies of etiology (2019) · cites this paper as [1]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“in a survey of 300 systematic reviews, 64% of the reviews included observational studies [7].”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [7]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Although few published evaluations exist, evidence suggests that the endorsement of PRISMA by journals is associated with more complete reporting of systematic reviews than in journals not endorsing the reporting guideline.50 51 52”
From Reporting guideline for overviews of reviews of healthcare interventions: development of the PRIOR statement. (2022) · cites this paper as [51]
Experimental — accuracy not yet measured
23 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & for the PRISMA Group (2009) · BMJ · doi:10.1136/bmj.b2535 · Cited by 83,823
Self-citationPreferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & The PRISMA Group (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000097 · Cited by 64,042
Self-citationPreferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.
(2009) · PubMed Central · Cited by 44,859
Self-citationThe Cochrane Collaboration's tool for assessing risk of bias in randomised trials
Higgins, J. P. T., Altman, D. G., Gotzsche, P. C., Juni, P., Moher, D., Oxman, A. D., Savovic, J., Schulz, K. F., Weeks, L., Sterne, J. A. C., et al. (2011) · BMJ · doi:10.1136/bmj.d5928 · Cited by 33,817
Self-citationLiberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000100 · Cited by 28,164
Self-citationPRISMA-P Group, Moher, D., Shamseer, L., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P., & Stewart, L. A. (2015) · Systematic Reviews · doi:10.1186/2046-4053-4-1 · Cited by 26,223
Self-citationDermyshi, E., Wang, Y., Yan, C., Hong, W., Qiu, G., Gong, X., & Zhang, T. (2017) · Neonatology · doi:10.1159/000454668 · Cited by 24,451
Meta-analysis of Observational Studies in Epidemiology<SUBTITLE>A Proposal for Reporting</SUBTITLE>
Stroup, D. F. (2000) · JAMA · doi:10.1001/jama.283.15.2008 · Cited by 20,623
Wang, G., Ma, R., Qiao, G., Wada, K., Aizawa, Y., & Satoh, T. (2015) · Integrative Medicine International · doi:10.1159/000368408 · Cited by 18,657
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gotzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · BMJ · doi:10.1136/bmj.b2700 · Cited by 17,648
Self-citation
Research works printed but not linked: 2
- Agency for Healthcare Research and Quality. Methods guide for effectiveness and comparative effectiveness reviews. AHRQ Publication No. 10(14)-EHC063-EF. 2014 Jan [cited 21 Apr 2016]. Available: https://www.effectivehealthcare.ahrq.gov/ehc/products/60/318/CER-Methods-Guide-140109.pdf. 21433403
- Nordic Cochrane Centre. Review Manager (RevMan), version 5.1. Copenhagen: Nordic Cochrane Centre; 2011.
Cited by
Items 201–300 of 793
Feng, S., Tang, M., Huang, G., Wang, J., He, S., Liu, D., & Gu, L. (2023) · Frontiers in Neurology · doi:10.3389/fneur.2022.1056156 · Cited by 32
Sociodemographic and individual predictors of residential solar water heater adoption behaviour
Alipour, M., Zare, S. G., Taghikhah, F. R., & Hafezi, R. (2023) · Energy Research & Social Science · doi:10.1016/j.erss.2023.103155 · Cited by 36
Decoding semi-automated title-abstract screening: findings from a convenience sample of reviews
Gates, A., Gates, M., DaRosa, D., Elliott, S. A., Pillay, J., Rahman, S., Vandermeer, B., & Hartling, L. (2020) · Systematic Reviews · doi:10.1186/s13643-020-01528-x · Cited by 31
A framework for prospective, adaptive meta-analysis (FAME) of aggregate data from randomised trials.
Tierney, J. F., Fisher, D. J., Vale, C. L., Burdett, S., Rydzewska, L. H., Rogozińska, E., Godolphin, P. J., White, I. R., & Parmar, M. K. B. (2021) · PLOS Medicine · doi:10.1371/journal.pmed.1003629 · Cited by 31
“Often unpublished and ongoing trials are overlooked [2,3], meaning results may not be interpreted in the context of all the potential evidence, and updating is considered separately [4].”
BackgroundThe use of systematic reviews to justify anaesthesiology trials: A meta‐epidemiological study
Engelking, A., Ćavar, M., & Puljak, L. (2018) · European Journal of Pain · doi:10.1002/ejp.1280 · Cited by 31
Chen, Y., Hua, F., Mei, Y., Thiruvenkatachari, B., Riley, P., & He, H. (2019) · Journal of Evidence Based Dental Practice · doi:10.1016/j.jebdp.2019.03.001 · Cited by 31
“C) then verified the data for these items from all articles.(22)”
MethodGao, Y., Cai, Y., Yang, K., Liu, M., Shi, S., Chen, J., Sun, Y., Song, F., Zhang, J., & Tian, J. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.11.012 · Cited by 31
“Recent estimates suggest that more than 8,000 new S Rs were published in Medline annually, which is equivalent to a threefold increa s over the past decade [7, 8].”
BackgroundThe Nordic Nutrition Recommendations 2022 – structure and rationale of qualified systematic reviews
Arnesen, E., Christensen, J. J., Andersen, R., Eneroth, H., Erkkola, M., Høyer, A., Lemming, E. W., Meltzer, H. M., Halldorsson, T. I., Þórsdóttir, I., et al. (2020) · Food & Nutrition Research · doi:10.29219/fnr.v64.4403 · Cited by 32
“With more than 8,000 SRs published yearly (26), SR as a method is an evolving field (27).”
BackgroundThe Pandora’s Box of Evidence Synthesis and the case for a living Evidence Synthesis Taxonomy
Munn, Z., Pollock, D., Barker, T. H., Stone, J., Stern, C., Aromataris, E., Schünemann, H. J., Clyne, B., Khalil, H., Mustafa, R. A., et al. (2023) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2022-112065 · Cited by 27
Self-citationMinnucci, S., Innocenti, T., Salvioli, S., Giagio, S., Yousif, M. S., Riganelli, F., Carletti, C., Feller, D., Brindisino, F., Faletra, A., et al. (2023) · Journal of Orthopaedic and Sports Physical Therapy · doi:10.2519/jospt.2023.11708 · Cited by 36
Systematic Reviews in Health Research
Egger, M., Higgins, J. P. T., & Smith, G. D. (2022) · Wiley eBooks · doi:10.1002/9781119099369.ch1 · Cited by 43
Wang, Y., Chen, N., Guo, K., Li, Y., Fenfen, E., Yang, C., Shang, X., Li, X., & Yang, K. (2022) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2022.11.004 · Cited by 31
Evolving Trends in Physiotherapy Research Publications between 1995 and 2015
Jesus, T. S., Gianola, S., Castellini, G., Colquhoun, H. L., & Brooks, D. (2019) · Physiotherapy Canada · doi:10.3138/ptc-2018-0065 · Cited by 32
Lunny, C., Reid, E. K., Neelakant, T., Chen, A., Zhang, J. H., Shinger, G., Stevens, A., Tasnim, S., Sadeghipouya, S., Adams, S., et al. (2022) · Research Synthesis Methods · doi:10.1002/jrsm.1542 · Cited by 32
“The former(37) is a broad overview focusing on patients with any types of pain, which subsumes the more specific populations in the broad overview focusing on patients with pain from any type of clinical condition.(39) Both of these overviews subsume the scope of the narrow overview investigating spinal manipulation i…”
BackgroundSelf-citationYuan, Y., Lü, L., Xie, C., Lu, Y., Liang, J., Qi, Y., Dong, B., Chen, Q., Fang, Q., & Tian, L. (2022) · Journal of Affective Disorders · doi:10.1016/j.jad.2022.04.152 · Cited by 29
Guidance to best tools and practices for systematic reviews
Kolaski, K., Logan, L. R., & Ioannidis, J. P. A. (2023) · British Journal of Pharmacology · doi:10.1111/bph.16100 · Cited by 33
“As a result, information that may be relevant to the critical appraisal of non-Cochrane syntheses is often removed or is rele-gated to online-only supplements that may not be readily or fully accessible (Page et al., 2016).”
BackgroundMethodInfluentialKamioka, H., Tsutani, K., Origasa, H., Yoshizaki, T., Kitayuguchi, J., Shimada, M., Tang, W., & Takano‐Ohmuro, H. (2017) · Nutrition Research · doi:10.1016/j.nutres.2017.02.008 · Cited by 31
Luhnen, M., Prediger, B., Neugebauer, E. A. M., & Mathes, T. (2019) · Research Synthesis Methods · doi:10.1002/jrsm.1342 · Cited by 30
Steffen, R. J., Erber, W. N., & Schmitt, H. J. (2021) · Ticks and Tick-borne Diseases · doi:10.1016/j.ttbdis.2021.101779 · Cited by 29
Mayo-Wilson, E., Ng, S. M., Chuck, R. S. H., & Li, T. (2017) · BMC Ophthalmology · doi:10.1186/s12886-017-0561-9 · Cited by 28
“Our findings are thus consistent with evidence that most systematic reviews published to date have been neither conducted nor reported following best practices [54, 55].”
ResultRunjić, E., Behmen, D., Pieper, D., Mathes, T., Tricco, A., Moher, D., & Puljak, L. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.03.006 · Cited by 21
Self-citationA review of the reporting of web searching to identify studies for Cochrane systematic reviews
Briscoe, S. (2018) · Research Synthesis Methods · doi:10.1002/jrsm.1275 · Cited by 27
Boscolo, A., Spiezia, L., CASSAI, A. D., Pasin, L., Pesenti, E., Zatta, M., Zampirollo, S., Andreatta, G., Sella, N., Pettenuzzo, T., et al. (2020) · Journal of Critical Care · doi:10.1016/j.jcrc.2020.09.034 · Cited by 29
Trends and predictors of biomedical research quality, 1990-2015: a meta-research study
Catillon, M. (2019) · BMJ Open · doi:10.1136/bmjopen-2019-030342 · Cited by 27
Torp-Pedersen, C. T., Goette, A., Nielsen, P. B., Potpara, T., Fauchier, L., Camm, A. J., Arbelo, E., Boriani, G., Skjoeth, F., Rumsfeld, J. S., et al. (2019) · EP Europace · doi:10.1093/europace/euz210 · Cited by 27
Protogerou, C., McHugh, R. K., & Johnson, B. T. (2020) · Health Psychology Review · doi:10.1080/17437199.2019.1707104 · Cited by 30
“Numerous evidence syntheses are published, with some estimates suggesting that about 22 systematic reviews are published every day in the broad biomedical research field (Page et al., 2016).”
BackgroundBarker, T. H., Dias, M. M., Stern, C., Porritt, K. A., Wiechula, R., Aromataris, E., Brennan, S., Schünemann, H. J., & Munn, Z. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.10.017 · Cited by 30
A suggested data structure for transparent and repeatable reporting of bibliographic searching
Haddaway, N., Rethlefsen, M. L., Davies, M. C., Glanville, J., McGowan, B. S., Nyhan, K., & Young, S. (2022) · Campbell Systematic Reviews · doi:10.1002/cl2.1288 · Cited by 39
“This applies to all aspects of review methods, but is particularly evident in search methods (de Kock et al., 2021; Page et al., 2016; Sargeant et al., 2021).”
BackgroundAssociation between sense of coherence and dental caries: systematic review and meta-analysis
Torres, T. A. P., Corradi‐Dias, L., Oliveira, P. D., Martins, C. C., Paiva, S. M., Pordeus, I. A., & Abreu, L. G. (2019) · Health Promotion International · doi:10.1093/heapro/daz038 · Cited by 29
Cochrane Rehabilitation Methodology Committee: an international survey of priorities for future work
Levack, W., Meyer, T., Négrini, S., & Malmivaara, A. O. V. (2017) · European Journal of Physical and Rehabilitation Medicine · doi:10.23736/s1973-9087.17.04958-9 · Cited by 27
The methodological and reporting characteristics of Campbell reviews: A systematic review
Wang, X., Welch, V. A., Li, M., Yao, L., Littell, J. H., Li, H., Yang, N., Wang, J., Shamseer, L., Chen, Y., et al. (2021) · Campbell Systematic Reviews · doi:10.1002/cl2.1134 · Cited by 27
“1 (MECCIR, 2017) and additional review characteristics used in a similar methodological study of reporting quality (Page et al., 2016).”
BackgroundMethodInfluentialSelf-citationProspective approaches to accumulating evidence
Thomas, J., Askie, L. M., Berlin, J. A., Elliott, J. H., Ghersi, D., Simmonds, M., Takwoingi, Y., Tierney, J. F., & Higgins, J. P. (2019) · Cochrane Handbook for Systematic Reviews of Interventions · doi:10.1002/9781119536604.ch22 · Cited by 26
Chapman, S. J., Drake, T. M., Bolton, W. S., Barnard, J., & Bhangu, A. A. (2016) · British journal of surgery · doi:10.1002/bjs.10423 · Cited by 27
Patient-important outcomes in systematic reviews: Poor quality of evidence
Yordanov, Y., Dechartres, A., & Ravaud, P. (2018) · PLoS ONE · doi:10.1371/journal.pone.0195460 · Cited by 28
Hasuike, A., Ueno, D., Nagashima, H., Kubota, T., Tsukune, N., Watanabe, N., & Sato, S. (2019) · Journal of Periodontal Research · doi:10.1111/jre.12638 · Cited by 26
Korevaar, D. A., Salameh, J., Vali, Y., Cohen, J. F., McInnes, M. D. F., Spijker, R., & Bossuyt, P. M. (2020) · Research Synthesis Methods · doi:10.1002/jrsm.1389 · Cited by 27
“Data were analyzed for non-Cochrane and Cochrane systematic reviews separately as we expected considerable differences in searching practices between the two groups, as has been found for systematic reviews of therapeutic studies.(21) We did not attempt a statistical comparison between nonCochrane and Cochrane systema…”
MethodResultQureshi, A. I., Lodhi, A., Akhtar, I. N., Ma, X., Kherani, D., Kwok, C. S., Ford, D. E., Hanley, D. F., Hassan, A. E., Nguyen, T. N., et al. (2023) · International Journal of Stroke · doi:10.1177/17474930231184369 · Cited by 27
“43 However, assessing the causal relationship between IAT and various outcomes is limited by both bias and confounding.”
BackgroundKnowledge Synthesis in Evidence-Based Medicine
Garritty, C., Stevens, A., Hamel, C., Golfam, M., Hutton, B., & Wolfe, D. M. (2019) · Seminars in Nuclear Medicine · doi:10.1053/j.semnuclmed.2018.11.006 · Cited by 30
Toolkit of methodological resources to conduct systematic reviews
Figuls, M. R. I., García, L. M., Solà, I., Alonso‐Coello, P., Bonfill, X., & Zamora, J. (2020) · F1000Research · doi:10.12688/f1000research.22032.2 · Cited by 28
dos Reis‐Prado, A. H., Abreu, L. G., Tavares, W. L. F., Peixoto, I. F. D. C., Viana, A. C. D., de Oliveira, E. M. C., Bastos, J. V., Sobrinho, A. P. R., & Benetti, F. (2021) · Clinical Oral Investigations · doi:10.1007/s00784-020-03690-x · Cited by 25
Accumulation Bias in meta-analysis: the need to consider time in error control
ter Schure, J., & Grünwald, P. (2019) · F1000Research · doi:10.12688/f1000research.19375.1 · Cited by 40
“…16 and 15 studies per meta-analysis were reported to be the median numbers in Medline metaanalyses from 2004 and 2014 (Moher et al., 2007a; Page et al., 2016), while 3 studies per meta-analysis were reported in Cochrane meta-analyses from 2008 (Cochrane Database of Systematic Reviews…”
Do not make clinical decisions based on abstracts of healthcare research: A systematic review
Nascimento, D. P., Ostelo, R., van Tulder, M. W., Gonzalez, G. Z., Araújo, A. C., Vanin, A. A., & Costa, L. O. P. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.03.030 · Cited by 30
“[12 , 18 , 19] However, there is no consensus reported in literature on what measures should be taken.”
BackgroundCantrell, A. J., Booth, A., & Chambers, D. (2022) · Health Information & Libraries Journal · doi:10.1111/hir.12428 · Cited by 27
Guidance to Best Tools and Practices for Systematic Reviews
Kolaski, K., Logan, L. R., & Ioannidis, J. P. A. (2023) · JBJS Reviews · doi:10.2106/jbjs.rvw.23.00077 · Cited by 28
Guidance to best tools and practices for systematic reviews
Kolaski, K., Logan, L. R., & Ioannidis, J. P. A. (2023) · BMC Infectious Diseases · doi:10.1186/s12879-023-08304-x · Cited by 27
“As a result, information that may be relevant to the critical appraisal of non-Cochrane reviews is often removed or is relegated to online-only supplements that may not be readily or fully accessible [28].”
InfluentialSaldanha, I. J., Lindsley, K. B., Lum, F., Dickersin, K., & Li, T. (2019) · JAMA Ophthalmology · doi:10.1001/jamaophthalmol.2019.1063 · Cited by 26
Quality of reporting in systematic reviews published in dermatology journals
Croitoru, D. O., Huang, Y. T., Kurdina, A., Chan, A., & Drucker, A. M. (2019) · British Journal of Dermatology · doi:10.1111/bjd.18528 · Cited by 24
McGrath, T. A., Bowdridge, J., Prager, R. T., Frank, R. A., Treanor, L. M., Sharifabadi, A. D., Salameh, J., Leeflang, M., Korevaar, D. A., Bossuyt, P. M., et al. (2020) · Clinical Chemistry · doi:10.1093/clinchem/hvaa093 · Cited by 26
Predatory publishing dilutes and distorts evidence in systematic reviews
Hayden, J. A. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.01.013 · Cited by 24
de Kock, S., Stirk, L., Ross, J., Duffy, S., Noake, C., & Misso, K. (2020) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462320002135 · Cited by 25
Pieper, D., Hellbrecht, I., Zhao, L., Baur, C., Pick, G., Schneider, S., Harder, T., Young, K., Tricco, A. C., Westhaver, E., et al. (2022) · Systematic Reviews · doi:10.1186/s13643-022-02051-x · Cited by 23
“In two recent samples of SRs in dentistry the proportion of industry-funded SRs was 2% [87], while it was 3% in a general sample of SRs [88].”
BackgroundSelf-citationWang, D., Cui, W. J., Hou, Z. H., & Gao, Y. (2023) · Frontiers in Aging Neuroscience · doi:10.3389/fnagi.2023.1215495 · Cited by 29
“The surface under the cumulative ranking curve (SUCRA), a simple numerical statistic summarizing the cumulative ranking probability plots for each intervention, was provided as an estimated probability that was used to rate the exercise intervention (Page et al., 2016).”
MethodCatalá-López, F., Ridao-López, M., Hurtado, I., Núñez-Beltrán, A., Gènova-Maleras, R., Alonso‐Arroyo, A., Tobías, A., Aleixandre‐Benavent, R., Catalá, M. A., & Tabarés‐Seisdedos, R. (2019) · Systematic Reviews · doi:10.1186/s13643-019-1061-1 · Cited by 26
Self-citationClinical decision making in the era of evidence-based dentistry
Elangovan, S., Guzmán‐Armstrong, S., Marshall, T. A., & Johnsen, D. C. (2018) · The Journal of the American Dental Association · doi:10.1016/j.adaj.2018.06.001 · Cited by 23
Ivaldi, D., Burgos, M. A., Oltra, G. V., Liquitay, C. M. E., & Garegnani, L. (2024) · Cochrane Evidence Synthesis and Methods · doi:10.1002/cesm.12074 · Cited by 29
“…firstly because of their more complete reporting due to their high methodological standards and the rigorous editorial process they have to fulfill [27], and second, because those reviews used to follow the Methodological Expectations of Cochrane Intervention Reviews (MECIR) recommendations, which…”
Methodological quality of systematic reviews on treatments for depression: a cross-sectional study
Chung, V. C., Wu, X. Y., Feng, Y., Ho, R. S. T., Wong, S. Y. S., & Threapleton, D. E. (2017) · Epidemiology and Psychiatric Sciences · doi:10.1017/s2045796017000208 · Cited by 22
“A lower proportion of our sample reported PRISMA-like flow diagrams and treatment harms when compared to a sample of SRs from MEDLINE indexed in February 2014 (Page et al. 2016).”
ResultRuano, J., Gómez-García, F. J., Gay-Mimbrera, J., Aguilar-Luque, M., Fernández-Rueda, J. L., Fernández-Chaichio, J., Alcalde‐Mellado, P., Carmona-Fernández, P. J., Sanz-Cabanillas, J. L., Viguera-Guerra, I., et al. (2018) · Systematic Reviews · doi:10.1186/s13643-018-0709-6 · Cited by 22
Methodological Quality Assessment of Meta-analyses in Endodontics
Kattan, S. A., Lee, S., Kohli, M. R., Setzer, F. C., & Karabucak, B. (2017) · Journal of Endodontics · doi:10.1016/j.joen.2017.07.019 · Cited by 23
“We also found that the quality of Cochrane systematic reviews was better than that of non-Cochrane reviews selected in this study, which is consistent with the findings of the previous studies (47, 48).”
ResultTsujimoto, Y., Tsutsumi, Y., Kataoka, Y., Banno, M., & Furukawa, T. A. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.08.030 · Cited by 21
Cornito, C. M. (2021) · Innovation journal of social sciences and economic review. · doi:10.36923/ijsser.v3i3.122 · Cited by 22
Xun, Y., Guo, Q., Ren, M., Liu, Y., Sun, Y., Wu, S., Lan, H., Zhang, J., Liu, H., Wang, J., et al. (2023) · Frontiers in Public Health · doi:10.3389/fpubh.2023.998588 · Cited by 22
Catalá-López, F., Alonso‐Arroyo, A., Page, M. J., Hutton, B., Ridao-López, M., Tabarés‐Seisdedos, R., Aleixandre‐Benavent, R., & Moher, D. (2019) · BMJ Open · doi:10.1136/bmjopen-2018-022769 · Cited by 21
Self-citationThird-wave interventions for eating disorders in adolescence – systematic review with meta-analysis
Buerger, A., Vloet, T. D., Haber, L., & Geissler, J. M. (2021) · Borderline Personality Disorder and Emotion Dysregulation · doi:10.1186/s40479-021-00158-6 · Cited by 22
“Risk of bias across studies Due to the small number of included studies, we did not perform an analysis of publication bias [56].”
BackgroundNascimento, D. P., Gonzalez, G. Z., Araújo, A. C., & Costa, L. O. P. (2019) · European Spine Journal · doi:10.1007/s00586-019-06206-8 · Cited by 23
Liang, J., Lü, L., Wang, Y., Jia, R., Qu, X.-Y., Li, J., Li, J., Qian, S., Qian, Y., Wang, S., et al. (2019) · Ageing Research Reviews · doi:10.1016/j.arr.2019.100965 · Cited by 25
Introducing Systematic Reviews
Tod, D. (2019) · Springer eBooks · doi:10.1007/978-3-030-12263-8_1 · Cited by 21
Opheim, E., Andersen, P. N., Jakobsen, M., Aasen, B., & Kvaal, K. S. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.01558 · Cited by 20
Feldmann, J., Puhan, M. A., & Mütsch, M. (2019) · BMJ Open · doi:10.1136/bmjopen-2018-024587 · Cited by 20
“To get consumer feedback 1 (25) 0 0 0 1 (2)”
InfluentialSystematic Review: A Method at Risk for Being Corrupted
Bero, L. A. (2016) · American Journal of Public Health · doi:10.2105/ajph.2016.303518 · Cited by 24
O’Mara, M., Greene, D., Watson, H. J., Shafran, R., Kenworthy, I., Cresswell, C., & Egan, S. J. (2022) · Journal of Behavior Therapy and Experimental Psychiatry · doi:10.1016/j.jbtep.2022.101777 · Cited by 20
Kocakabak, C., van den Hoogen, A., Rothfus, M. A., Campbell-Yeo, M. L., Kostenzer, J., Axelin, A., Schofield, P., & Latour, J. M. (2024) · Pediatric Research · doi:10.1038/s41390-024-03293-2 · Cited by 20
Alabousi, M., Alabousi, A., McGrath, T. A., Cobey, K. D., Budhram, B., Frank, R. A., Nguyen, F., Salameh, J., Sharifabadi, A. D., & McInnes, M. D. F. (2018) · European Radiology · doi:10.1007/s00330-018-5567-z · Cited by 22
Šarić, L., Došenović, S., Saldanha, I. J., Kadić, A. J., & Puljak, L. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.09.011 · Cited by 20
Interventions for tic disorders: An updated overview of systematic reviews and meta analyses
Yang, C., Cheng, X., Zhang, Q., Yu, D., Li, J., & Zhang, L. (2020) · Psychiatry Research · doi:10.1016/j.psychres.2020.112905 · Cited by 20
Reviews in environmental health: How systematic are they?
Sutton, P., Chartres, N., Rayasam, S. D. G., Daniels, N., Lam, J., Maghrbi, E., & Woodruff, T. J. (2021) · Environment International · doi:10.1016/j.envint.2021.106473 · Cited by 21
Rainkie, D. C., Abedini, Z. S., & Abdelkader, N. N. (2020) · PLoS ONE · doi:10.1371/journal.pone.0243091 · Cited by 20
“A cross sectional study of therapeutic non-Cochrane SR/MAs in 2016 found that approximately 2 of every 3 published SR/MAs did not use a protocol or did not use PRISMA as a reporting guideline [7].”
BackgroundWang, Y., Yao, M., Wang, J., Liu, H., Zhang, X., Zhao, L., Hu, X., Guan, H., & Lyu, Z. (2022) · Frontiers in Endocrinology · doi:10.3389/fendo.2022.818537 · Cited by 19
Kermansaravi, M., Vitiello, A., Valizadeh, R., Shahmiri, S. S., & Musella, M. (2023) · International Journal of Surgery · doi:10.1097/js9.0000000000000629 · Cited by 20
“The number of articles being below 10 made meta-regression and funnel plots inap-plicable to publication bias assessments [30] .”
BackgroundUnpublished systematic reviews and financial support: a meta-epidemiological study
Tsujimoto, H., Tsujimoto, Y., & Kataoka, Y. (2017) · BMC Research Notes · doi:10.1186/s13104-017-3043-5 · Cited by 19
“However, the impact on assessment of non-publication of high-quality SRs was probably minimal, because the quality of reporting of published SRs in 2014 was still low despite the fact that reporting guidelines recommend protocol registration [10, 11].”
Rombey, T., Lochner, V., Puljak, L., Könsgen, N., Mathes, T., & Pieper, D. (2020) · Research Synthesis Methods · doi:10.1002/jrsm.1409 · Cited by 20
“of studies included in update (n = 100) 19 (9-28)”
BackgroundDevelopment and application of an outcome-centric approach for conducting overviews of reviews
Kho, M. E., Poitras, V. J., Janssen, I., Chaput, J., Saunders, T. J., Giangregorio, L., Tomasone, J. R., Ross‐White, A., & Ross, R. (2020) · Applied Physiology, Nutrition, and Metabolism · doi:10.1139/apnm-2020-0564 · Cited by 18
“In the medical literature, approximately 75 new randomized controlled trials and 11 systematic reviews were published per day in 2010 (Bastian et al. 2010); the number of systematic reviews doubled to approximately 22 per day by 2016 (Page et al. 2016).”
BackgroundSynnot, A., Bragge, P., Lunny, C., Menon, D. K., Clavisi, O., Pattuwage, L., Volovici, V., Mondello, S., Cnossen, M. C., Donoghue, E., et al. (2018) · PLoS ONE · doi:10.1371/journal.pone.0198676 · Cited by 18
“A further complication is that many systematic reviews are poorly conducted and reported [10], with unnecessary duplication of topics, and conflicting or misleading results common [9, 11].”
BackgroundResultInfluentialSun, X., Zhou, X., Zhang, Y., & Liu, H. (2019) · Journal of Nursing Scholarship · doi:10.1111/jnu.12462 · Cited by 17
“However, some articles (Jadad et al., 1998; Jadad & McQuay, 1996; Mrkobrada et al., 2008; Page et al., 2016; Shea, Moher, Graham, Ba, & Tugwell, 2002) found that the quality of reporting was generally poor as a result of reporting and methodological deficiencies.”
BackgroundDivya, D., Savita, S., & Kaur, S. (2025) · International Journal of Pharmaceutical and Healthcare Marketing · doi:10.1108/ijphm-05-2024-0043 · Cited by 24
Completeness of Systematic Reviews in Nursing Literature Based on PRISMA Reporting Guidelines
Carter‐Templeton, H., Oermann, M. H., Owens, J. K., Vance, B. S., Mastorovich, M. L., Quazi, M. A., Wrigley, J., Walter, S. M., Carpenter, R. D., & Thurman, F. (2025) · Advances in Nursing Science · doi:10.1097/ans.0000000000000567 · Cited by 21
on behalf of the Project Transform Team, Turner, T., developers, I., Green, S., Tovey, D. I., McDonald, S. J., Soares‐Weiser, K., Pestridge, C., & Elliott, J. H. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0542-3 · Cited by 20
“[I would] Hate to see Cochrane only having paid staff, but as part of the team it is essential (Author, Editor) The respondents highlighted the key role of volunteer Cochrane authors, both in producing the bulk of Cochrane Reviews and in producing a wide variety of reviews and also in maintaining the connection betwee…”
BackgroundInfluentialChaabna, K., Cheema, S., Abraham, A., & Mamtani, R. (2020) · Journal of Evidence-Based Medicine · doi:10.1111/jebm.12394 · Cited by 17
Babić, A., Peričić, T. P., Pieper, D., & Puljak, L. (2020) · Research Synthesis Methods · doi:10.1002/jrsm.1451 · Cited by 17
“We believe that this work is relevant for the wider research community, beyond Cochrane, because the pioneering work of Cochrane in this field can be used as a case study to further improve guidelines for stabilizing an SR and help reduce research waste associated with redundant SRs.”
BackgroundInfluentialA new method for testing reproducibility in systematic reviews was developed, but needs more testing
Pieper, D., Heß, S., & Faggion, C. M. (2021) · BMC Medical Research Methodology · doi:10.1186/s12874-021-01342-6 · Cited by 21
Mualem, W., Durrani, S., Lakomkin, N., Gompel, J. J. V., Quiñones‐Hinojosa, A., & Bydon, M. (2022) · World Neurosurgery · doi:10.1016/j.wneu.2022.07.036 · Cited by 18
Summary of findings tables for communicating key findings of systematic reviews
Conway, A., Clarke, M. J., Treweek, S. P., Schünemann, H. J., Santesso, N. A., Morgan, R. L., Darragh, M., Maguire, L. K., & Devane, D. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000044 · Cited by 20
Head-to-head trials of antibiotics for bronchiectasis
Kaehne, A., Milan, S. J., Felix, L. M., Sheridan, E., Marsden, P., & Spencer, S. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012590.pub2 · Cited by 17
Paquette, M., Alotaibi, A., Nieuwlaat, R., Santesso, N. A., & Mbuagbaw, L. C. E. (2019) · Systematic Reviews · doi:10.1186/s13643-019-1152-z · Cited by 19
Systematic reviews: let's keep them trustworthy
García-Doval, I., van Zuuren, E. J., Bath-Hextall, F. J., & Ingram, J. R. (2017) · British Journal of Dermatology · doi:10.1111/bjd.15826 · Cited by 17
Assessment of Duplicate Publication of Chinese-Sponsored Randomized Clinical Trials
Jia, Y., Huang, D., Wen, J., Qureshı, R., Wang, Y., Rosman, L. M., Chen, Q., Robinson, K. A., Gagnier, J. J., Ehrhardt, S., et al. (2020) · JAMA Network Open · doi:10.1001/jamanetworkopen.2020.27104 · Cited by 16
Riberholt, C. G., Olsen, M. H., Milan, J. B., Hafliðadóttir, S. H., Svanholm, J. H., Pedersen, E. B., Lew, C. C. H., Asante, M. A., Ribeiro, J. P., Wagner, V., et al. (2024) · BMC Medical Research Methodology · doi:10.1186/s12874-024-02318-y · Cited by 18
Lee, J. Y., Wright, C. A., Zheng, X., Todaro, R. D., Golinkoff, R. M., & Hirsh‐Pasek, K. (2024) · Teachers and Teaching · doi:10.1080/13540602.2024.2404081 · Cited by 20
“It enhances the credibility of study findings by ensuring thorough reporting (Leclercq et al., 2019; Page et al., 2016).”
Bodnaruc, A. M., Khan, H., Shaver, N., Bennett, A., Wong, Y. L., Gracey, C., Ly, V., Shea, B., Little, J., Brouwers, M. C., et al. (2024) · American Journal of Clinical Nutrition · doi:10.1016/j.ajcnut.2024.10.013 · Cited by 16
“As the publication rate of SRs continues to increase [43,69], the improvement in SR reporting transparency and reproducibility seems to fall behind, with poorly conducted and reported SRs continuing to be published [70 – 72].”
Self-citationHabtewold, T. D., Alemu, S. M., Mohammed, S. H., Endalamaw, A., Akibu, M., Teferra, A. A., Tura, A. K., Asefa, N. G., & Tegegne, B. S. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.01.011 · Cited by 16
Yu, W., Huang, Y., Zhang, X., Luo, H., Chen, W., Jiang, Y., & Cheng, Y. Z. (2021) · BMC Neurology · doi:10.1186/s12883-021-02303-8 · Cited by 18
Metadata
- Authors
- Page, M. J., Shamseer, L., Altman, D. G., Tetzlaff, J., Sampson, M., Tricco, A. C., Catalá-López, F., Li, L., Reid, E. K., Sarkis-Onofre, R., & Moher, D.
- Affiliations
- University of Bristol (GB)
- University of Ottawa (CA)
- University of Oxford (GB)
- Ottawa Hospital Research Institute (CA)
- Children's Hospital of Eastern Ontario (CA)
- St. Michael's Hospital (CA)
- Universitat de València (ES)
- Lanzhou University (CN)
- Vancouver General Hospital (CA)
- Corresponding author
- Moher, David
- ORCID
- Venue
- PLOS Medicine
- Publisher
- Public Library of Science (PLoS)
- Published
- 2016-05-24
- Volume
- 13
- Issue
- 5
- Pages
- e1002028
- ISSN
- 1549-1277 (ISSN-L)
- 1549-1676
- PMC
- PMC4878797
- OpenAlex
- W2396846174
- Semantic Scholar
- 363d54e2baf229adfd585079bfd7c18ae8feed26
- Cited by
- 768
- References
- 41
- Citations by year
- 2026: 25 · 2025: 43 · 2024: 85 · 2023: 73 · 2022: 85 · 2021: 146 · 2020: 95 · 2019: 83 · 2018: 58 · 2017: 60 · 2016: 15
- FWCI
- 24.71
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS Medicine (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Explore Bristol Research (publishedVersion, other-oa)
- Monash University Research Portal (Monash University) (acceptedVersion, other-oa)
- Lanzhou University Institutional Repository (submittedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Bristol Research (University of Bristol) (submittedVersion, other-oa)
- Funders
- National Institute for Health and Care Research
- Coordenação de Aperfeiçoamento de Pessoal de Nível Superior
- Governo Brasil
- Generalitat Valenciana (PROMETEOII/2015/021)
- Canadian Institutes of Health Research (1088535)
- Medical Research Council (MR/J00488X/1)
- National Health and Medical Research Council (1088535)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Health Sciences › Nursing › Issues, ethics and legal aspects › Pharmaceutical industry and healthcare
- Keywords
- systematic reviews, reporting quality, meta-analysis, risk of bias assessment, publication bias, research funding disclosure, biomedical literature, research quality improvement, research integrity, research transparency, research accountability, research reproducibility, research credibility, research trustworthiness
- MeSH terms
- Cross-Sectional Studies, Epidemiologic Methods, Humans, Research Design, Review Literature as Topic, Biomedical Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.