Epidemiology and Reporting Characteristics of Systematic Reviews of Biomedical Research: A Cross-Sectional Study
Open accessThe DOI matches its Crossref record; 41 of the 41 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Systematic reviews (SRs) can help decision makers interpret the deluge of published biomedical literature. However, a SR may be of limited use if the methods used to conduct the SR are flawed, and reporting of the SR is incomplete. To our knowledge, since 2004 there has been no cross-sectional study of the prevalence, focus, and completeness of reporting of SRs across different specialties. Therefore, the aim of our study was to investigate the epidemiological and reporting characteristics of a more recent cross-section of SRs. METHODS AND FINDINGS: We searched MEDLINE to identify potentially eligible SRs indexed during the month of February 2014. Citations were screened using prespecified eligibility criteria. Epidemiological and reporting characteristics of a random sample of 300 SRs were extracted by one reviewer, with a 10% sample extracted in duplicate. We compared characteristics of Cochrane versus non-Cochrane reviews, and the 2014 sample of SRs versus a 2004 sample of SRs. We identified 682 SRs, suggesting that more than 8,000 SRs are being indexed in MEDLINE annually, corresponding to a 3-fold increase over the last decade. The majority of SRs addressed a therapeutic question and were conducted by authors based in China, the UK, or the US; they included a median of 15 studies involving 2,072 participants. Meta-analysis was performed in 63% of SRs, mostly using standard pairwise methods. Study risk of bias/quality assessment was performed in 70% of SRs but was rarely incorporated into the analysis (16%). Few SRs (7%) searched sources of unpublished data, and the risk of publication bias was considered in less than half of SRs. Reporting quality was highly variable; at least a third of SRs did not report use of a SR protocol, eligibility criteria relating to publication status, years of coverage of the search, a full Boolean search logic for at least one database, methods for data extraction, methods for study risk of bias assessment, a primary outcome, an abstract conclusion that incorporated study limitations, or the funding source of the SR. Cochrane SRs, which accounted for 15% of the sample, had more complete reporting than all other types of SRs. Reporting has generally improved since 2004, but remains suboptimal for many characteristics. CONCLUSIONS: An increasing number of SRs are being published, and many are poorly conducted and reported. Strategies are needed to help reduce this avoidable waste in research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures




Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Many systematic review reports include narrative summaries of the characteristics and risk of bias across all included studies.36”
From PRISMA 2020 explanation and elaboration: updated guidance and exemplars for reporting systematic reviews (2021) · cites this paper as [36]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Even for those studies which explicitly reference PRISMA, there is only slight, statistically non-significant evidence of improved reporting, as found by Page et al. [15].”
From PRISMA-S: an extension to the PRISMA Statement for Reporting Literature Searches in Systematic Reviews (2021) · cites this paper as [15]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The studies included in a systematic review will generally vary with respect to design, study populations, and risk of bias [1].”
From COSMOS-E: Guidance on conducting systematic reviews and meta-analyses of observational studies of etiology (2019) · cites this paper as [1]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“in a survey of 300 systematic reviews, 64% of the reviews included observational studies [7].”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [7]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Although few published evaluations exist, evidence suggests that the endorsement of PRISMA by journals is associated with more complete reporting of systematic reviews than in journals not endorsing the reporting guideline.50 51 52”
From Reporting guideline for overviews of reviews of healthcare interventions: development of the PRIOR statement. (2022) · cites this paper as [51]
Experimental — accuracy not yet measured
23 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & for the PRISMA Group (2009) · BMJ · doi:10.1136/bmj.b2535 · Cited by 83,823
Self-citationPreferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & The PRISMA Group (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000097 · Cited by 64,042
Self-citationPreferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.
(2009) · PubMed Central · Cited by 44,859
Self-citationThe Cochrane Collaboration's tool for assessing risk of bias in randomised trials
Higgins, J. P. T., Altman, D. G., Gotzsche, P. C., Juni, P., Moher, D., Oxman, A. D., Savovic, J., Schulz, K. F., Weeks, L., Sterne, J. A. C., et al. (2011) · BMJ · doi:10.1136/bmj.d5928 · Cited by 33,817
Self-citationLiberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000100 · Cited by 28,164
Self-citationPRISMA-P Group, Moher, D., Shamseer, L., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P., & Stewart, L. A. (2015) · Systematic Reviews · doi:10.1186/2046-4053-4-1 · Cited by 26,223
Self-citationDermyshi, E., Wang, Y., Yan, C., Hong, W., Qiu, G., Gong, X., & Zhang, T. (2017) · Neonatology · doi:10.1159/000454668 · Cited by 24,451
Meta-analysis of Observational Studies in Epidemiology<SUBTITLE>A Proposal for Reporting</SUBTITLE>
Stroup, D. F. (2000) · JAMA · doi:10.1001/jama.283.15.2008 · Cited by 20,623
Wang, G., Ma, R., Qiao, G., Wada, K., Aizawa, Y., & Satoh, T. (2015) · Integrative Medicine International · doi:10.1159/000368408 · Cited by 18,657
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gotzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · BMJ · doi:10.1136/bmj.b2700 · Cited by 17,648
Self-citation
Research works printed but not linked: 2
- Agency for Healthcare Research and Quality. Methods guide for effectiveness and comparative effectiveness reviews. AHRQ Publication No. 10(14)-EHC063-EF. 2014 Jan [cited 21 Apr 2016]. Available: https://www.effectivehealthcare.ahrq.gov/ehc/products/60/318/CER-Methods-Guide-140109.pdf. 21433403
- Nordic Cochrane Centre. Review Manager (RevMan), version 5.1. Copenhagen: Nordic Cochrane Centre; 2011.
Cited by
Items 401–500 of 793
Systematic Reviews and Meta‐Analysis
Crombie, I. (2021) · Wiley eBooks · doi:10.1002/9781394322183.ch5 · Cited by 9
Vasopressor and inotrope treatment for septic shock: An umbrella review of reviews
Einav, S., Helviz, Y., Ippolito, M., & Cortegiani, A. (2021) · Journal of Critical Care · doi:10.1016/j.jcrc.2021.05.017 · Cited by 10
Moher, D., & Ravaud, P. (2016) · BMC Medicine · doi:10.1186/s12916-016-0707-2 · Cited by 8
Self-citationWu, B., Liu, W., Wang, S., Li, H., & Yin, M. (2025) · Complementary Therapies in Medicine · doi:10.1016/j.ctim.2025.103188 · Cited by 8
Zhao, H., Zhang, L., Yang, J., Guo, W., Sun, C., Shi, R., & Wang, Z. (2024) · Frontiers in Neurology · doi:10.3389/fneur.2024.1432256 · Cited by 9
Gattie, M., Lieven, E., & Kluk, K. (2024) · Journal of Fluency Disorders · doi:10.1016/j.jfludis.2024.106085 · Cited by 11
Lenton, E., & Fuller, K. (2019) · Journal of the Canadian Health Libraries Association / Journal de l Association de bilbiothèques de la santé du Canada · doi:10.29173/jchla29391 · Cited by 8
“Recent estimates indicate a three-fold increase in the number of published systematic reviews over the last decade [1], and that nearly half of all published scoping reviews have been published within the last six years alone [2].”
BackgroundIverson, S., Seta, M. D., Lefebvre, C., Ritchie, A., Traditi, L., & Baliozian, K. (2021) · Journal of the Medical Library Association JMLA · doi:10.5195/jmla.2021.1301 · Cited by 9
Cheng, J., Liu, B., Wang, B., Long, X., Li, Z., Chen, R., Wu, R., & Xu, L. (2021) · BMC Cancer · doi:10.1186/s12885-021-08685-9 · Cited by 8
Toolkit of methodological resources to conduct systematic reviews
Figuls, M. R. I., García, L. M., Solà, I., Alonso‐Coello, P., Bonfill, X., & Zamora, J. (2020) · F1000Research · doi:10.12688/f1000research.22032.1 · Cited by 9
Catalá-López, F., Alonso‐Arroyo, A., Page, M. J., Castelló‐Cogollos, L., Hutton, B., Ridao-López, M., Tabarés‐Seisdedos, R., Aleixandre‐Benavent, R., & Moher, D. (2023) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2023.02.001 · Cited by 8
Self-citationUse of inactive Cochrane reviews in academia: A citation analysis
Hoffmeyer, B., Fonnes, S., Andresen, K., & Rosenberg, J. (2023) · Scientometrics · doi:10.1007/s11192-023-04691-9 · Cited by 8
Replication of systematic reviews: is it to the benefit or detriment of methodological quality?
Chapelle, C., Ollier, É., Bonjean, P., Locher, C., Zufferey, P. J., Cucherat, M., & Laporte, S. (2023) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2023.08.012 · Cited by 8
Korevaar, E., Turner, S., Forbes, A. B., Karahalios, A., Taljaard, M., & McKenzie, J. E. (2024) · BMC Medical Research Methodology · doi:10.1186/s12874-024-02147-z · Cited by 8
“The WT method relies on large-sample approximations, which are not generally met in the context of meta-analysis due to few included studies [43, 44].”
Meta-analyses of diagnostic test accuracy could not be reproduced
Stegeman, I., & Leeflang, M. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.06.033 · Cited by 8
Nascimento, D. P., Gonzalez, G. Z., Araújo, A. C., Moseley, A. M., Maher, C., & Costa, L. O. P. (2020) · Brazilian Journal of Physical Therapy · doi:10.1016/j.bjpt.2020.10.002 · Cited by 10
“These differences may be due to the rigorous editorial processes and higher number of words permitted in the abstracts of Cochrane reviews.(40) Reporting quality of abstracts could be improved by changing journal editorial policies regarding the abstract word count, use of the PRISMA-A checklist, and by providing trai…”
BackgroundTai Chi for Chronic Obstructive Pulmonary Disease (COPD): An Overview of Systematic Reviews
Yang, L., Zhong, D., Zhang, Y., Li, Y., Liu, T., Zheng, Y., Wang, W., Li, J., Guan, L., & Jin, R. (2021) · International Journal of General Medicine · doi:10.2147/ijgm.s308955 · Cited by 10
Taekwondo kicks at the center of biomechanical research
Yılmaz, E. A., & Ateş, O. (2025) · Kinesiology · doi:10.26582/k.57.2.13 · Cited by 7
“Systematic reviews are distinguished from traditional literature reviews by their methodological rigour, transparency, and reproduc-Yılmaz ibility, which are essential for minimizing bias and enhancing the reliability of conclusions drawn from the evidence (Page, et al., 2016; Pieper, et al., 2022).”
Jayant, K., Cotter, T. G., Reccia, I., Virdis, F., Podda, M., Machairas, N., Arasaradnam, R. P., di Sabato, D., LaMattina, J. C., Barth, R. N., et al. (2023) · Journal of Clinical Medicine · doi:10.3390/jcm12185795 · Cited by 8
“Our search strategy was based on recommendations from the Cochrane Handbook for Systematic Reviews of Intervention and reported according to the guidelines of Meta-analysis of Observational Studies in Epidemiology [11,12].”
MethodWang, D., Valtonen, A. M., Thiel, T. P. H., Stenroth, L., Gao, Y., & Kulmala, J. (2024) · Journal of Orthopaedic and Sports Physical Therapy · doi:10.2519/jospt.2024.12720 · Cited by 7
Wandscher, K., Helbach, J., Pieper, D., & Hoffmann, F. (2025) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2025.111713 · Cited by 8
Rethlefsen, M. L., Schroter, S., Bouter, L. M., Kirkham, J. J., Moher, D., Ayala, A. P., Blanco, D., Brigham, T., Nardini, H. K. G., Kirtley, S., et al. (2025) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2024-113527 · Cited by 11
Self-citationImplementing a Three-Tiered Service Model for Knowledge Syntheses at an Academic Teaching Hospital
Yang, L., Orchanian‐Cheff, A., Anderson, M. A., Farrell, A., & Tripp, T. (2020) · Journal of the Canadian Health Libraries Association / Journal de l Association de bilbiothèques de la santé du Canada · doi:10.29173/jchla29436 · Cited by 7
Assessing the roles and challenges of librarians in dental systematic and scoping reviews
Schvaneveldt, N., & Stellrecht, E. M. (2021) · Journal of the Medical Library Association JMLA · doi:10.5195/jmla.2021.1031 · Cited by 10
“One study found that at least a third of the analyzed systematic reviews across all disciplines that had been indexed in February 2014 did not include a full report of a search strategy for at least one database, meaning these reviews were not reproducible [4], a finding that was also found in dentistry [5].”
BackgroundA comparison of systematic reviews and guideline-based systematic reviews in medical studies
Schniedermann, A. (2021) · Scientometrics · doi:10.1007/s11192-021-04199-0 · Cited by 8
“Usually, medical researchers and method developers 1 3 investigate the “epidemiology of [..] systematic reviews” by focusing on characteristics such language, inclusion criteria or whether a protocol was pre-registered or published (Page et al., 2016).”
BackgroundGadde, P., Pasupuleti, M. K., Penmetsa, G. S., Haribabu, N., Indukuri, S. L. D., & Akkaloori, A. (2022) · Journal of International Society of Preventive and Community Dentistry · doi:10.4103/jispcd.jispcd_127_22 · Cited by 7
“This is considered to be a meager contribution from India when compared with the bulk of SRs published every year.[9] Saltaji et al.”
BackgroundA Note on Cherry-Picking in Meta-Analyses
Yoneoka, D., & Rieck, B. (2023) · Entropy · doi:10.3390/e25040691 · Cited by 13
“, (2016) examined the reporting completeness of Biomedical Research meta-analyses and found that only 16% of the included reviews had a publicly accessible protocol published before the review was conducted [14].”
BackgroundEfficient evidence selection for systematic reviews in traditional Chinese medicine
Li, Y., Huang, Z., Luan, Z., Xu, S., Zhang, Y., Wu, L., Wu, D., Han, D., & Liu, Y. (2025) · BMC Medical Research Methodology · doi:10.1186/s12874-024-02430-z · Cited by 9
Mathes, T., Willms, G., Polus, S., Stegbauer, C., Messer, M., Klingler, C., Ehrenreich, H., Niebuhr, D., Marckmann, G., Gerhardus, A., et al. (2018) · Systematic Reviews · doi:10.1186/s13643-018-0743-4 · Cited by 7
Valderrama, Á., Jiménez‐Contreras, E., Valderrama, P., Escabias, M., & Baca, P. (2019) · Accountability in Research · doi:10.1080/08989621.2019.1672541 · Cited by 9
“Recent years have witnessed an increasing number of SR published (Page et al. 2016), and the field of Dentistry has kept the pace (Muniz et al. 2018).”
BackgroundPremji, Z., Hayden, A., & Rutherford, S. (2021) · Evidence Based Library and Information Practice · doi:10.18438/eblip29895 · Cited by 9
“They specifically advocate for education focused on strategies to identify bias in a SR, as well as strategies to minimize these biases, which will help to improve the quality of published systematic reviews, and, subsequently, help to “reduce this avoidable waste in research” (Page et al., 2016).”
InfluentialLucena, J. R. P. G., & Cruz, O. G. D. (2023) · International Journal of Integrated Engineering · doi:10.30880/ijie.2023.15.02.017 · Cited by 7
Synnot, A., Hawkins, M., Merner, B., Summers, M. P., Filippini, G., Osborne, R. H., Shapland, S. D. P., Cherry, C. L., Stuckey, R., Milne, C. A., et al. (2018) · Health Science Reports · doi:10.1002/hsr2.24 · Cited by 7
Li, L., Deng, K., Busse, J. W., Zhou, X., Xu, C., Liu, Z., Ren, Y., Zou, K., & Sun, X. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.03.017 · Cited by 7
Self-citationLiang, J., Zhang, S.-X., Chen, Y.-C., Tan, K., Zhang, J., Zhao, Y., Kakaer, A., & Chen, Y.-J. (2021) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2021.07.051 · Cited by 10
Kim, M. M., Pound, L. D., Steffensen, I., & Curtin, G. M. (2021) · Harm Reduction Journal · doi:10.1186/s12954-021-00570-9 · Cited by 9
Ayala, A. P., Sikora, L., Kirtley, S., & Labelle, P. (2019) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/gkzf2 · Cited by 7
El Ansari, W., AlRumaihi, K., El-Ansari, K., Arafa, M., Elbardisi, H., Majzoub, A., Shamsodini, A., & Al Ansari, A. (2023) · Arab Journal of Urology · doi:10.1080/2090598x.2022.2113127 · Cited by 6
“Published SRs are about > 8000 per year, with financial implications related to unclear abstracts [40,45].”
BackgroundHead-to-head trials of antibiotics for non-cystic fibrosis bronchiectasis
Kaehne, A., Milan, S. J., Felix, L. M., Spencer, S., Sheridan, E., & Marsden, P. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012590 · Cited by 6
Simon‐Tillaux, N., Gerard, A.-L., Rajendrabose, D., Tubach, F., & Dechartres, A. (2022) · Scientific Reports · doi:10.1038/s41598-022-24447-4 · Cited by 7
Liquitay, C. M. E., Vergara‐Merino, L., Verdejo, C., Kirmayr, M., Schuller-Martínez, B., Eva Madrid, A., Meza, N., Bracchiglione, J., & Franco, J. V. A. (2023) · Health Information & Libraries Journal · doi:10.1111/hir.12505 · Cited by 6
Williamson, M., Dong, S., D'Souza, R. D., Brignardello‐Petersen, R., & Ronzoni, S. (2024) · Acta Obstetricia Et Gynecologica Scandinavica · doi:10.1111/aogs.14903 · Cited by 7
“45 Where evidence from RCTs may be lacking or of questionable certainty, it becomes reasonable for recommendations to be made based on the meta-analysis of observational studies.”
BackgroundSalameh, J., Moher, D., McGrath, T. A., Frank, R. A., Sharifabadi, A. D., Islam, N., Lam, E. W. ‐., Adamo, R. G., Dawit, H., Al-Ghita, M. K., et al. (2024) · The Journal of Applied Laboratory Medicine · doi:10.1093/jalm/jfae117 · Cited by 6
Self-citationLindsay, C., Blancquaert, I. R., & Rousseau, F. (2025) · European Journal of Human Genetics · doi:10.1038/s41431-025-01861-6 · Cited by 7
Financial conflicts of interest and outcomes and quality of systematic reviews
Nejstgaard, C. H., Lundh, A., Rasmussen, K., Gøtzsche, P. C., & Hróbjartsson, A. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000047 · Cited by 8
A descriptive analysis of non-Cochrane child-relevant systematic reviews published in 2014
Gates, M., Elliott, S. A., Johnson, C., Thomson, D., Williams, K. J., Fernandes, R. M., & Hartling, L. (2018) · BMC Medical Research Methodology · doi:10.1186/s12874-018-0562-2 · Cited by 6
“(2016) [9] found that many authors misinterpreted the PRISMA guidelines.”
BackgroundInfluentialMcKeown, S., Mir, Z. M., Ritonja, J. A., & Soleas, E. (2021) · Journal of the Canadian Health Libraries Association / Journal de l Association de bilbiothèques de la santé du Canada · doi:10.29173/jchla29571 · Cited by 6
“The production of systematic reviews has been increasing steadily for decades, but many have been conducted and reported poorly, impeding their intended usefulness and value [4, 5].”
BackgroundJamal, H., Yaghmoor, R. B., Abed, H., Young, A., & Ashley, P. F. (2022) · European Journal of Dentistry · doi:10.1055/s-0042-1757582 · Cited by 9
“The systematic review was developed according to the PICO scheme ( ► Table 1 ) 26 and was conducted according to the Preferred Reporting Items for Systematic Review and Meta-analyses (PRISMA) guidelines 27 : Population: all studies examining extracted human teeth, caries-free dentine, healthy…”
MethodBriscoe, S., Abbott, R. A., & Meléndez‐Torres, G. J. (2022) · Health Information & Libraries Journal · doi:10.1111/hir.12468 · Cited by 10
Kugler, C. M., Perleth, M., Mathes, T., Gooßen, K., & Pieper, D. (2023) · Systematic Reviews · doi:10.1186/s13643-023-02204-6 · Cited by 6
Reproducibility of Published Meta-Analyses on Clinical-Psychological Interventions
López‐Nicolás, R., Lakens, D., López-López, J. A., Rubio‐Aparicio, M., Sandoval‐Lentisco, A., López‐Ibáñez, C., Blázquez-Rincón, D., & Sánchez‐Meca, J. (2024) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459231202929 · Cited by 6
“Research Question/Aims Process Reproducibility: -Medicine: Koffel & Rethlefsen, 2016; Maggio et al., 2011; Page et al., 2016 -Psychology: Lopez-Nicolas et al., 2021; Muffins et al., 2014; Polanin et al., 2020 -Health Sciences: Nguyen et al., 2022 Outcome reproducibility: -Databases consulted -Exact…”
Munce, S., Wong, E., Luong, D., Rao, J., Cunningham, J., Bailey, K., John, T., Barber, C. E. H., Batthish, M., Chambers, K., et al. (2024) · BMJ Open · doi:10.1136/bmjopen-2023-080822 · Cited by 7
Qureshi, A. I., Lodhi, A., Maqsood, H., Ma, X., Hubert, G. J., Gomez, C. R., Kwok, C. S., Ford, D. E., Hanley, D. F., Mehr, D. R., et al. (2024) · Journal of the American Heart Association · doi:10.1161/jaha.123.031906 · Cited by 6
“Although observational studies are included in 64% of systematic reviews, 42–44 we acknowledge the limitations posed by observational studies in determining the direct effect of physician transfer on various outcomes due to both bias and confounding.”
BackgroundPachanov, A., Muente, C., Hirt, J., & Pieper, D. (2024) · Research Synthesis Methods · doi:10.1002/jrsm.1763 · Cited by 6
A Systematic Literature Review of Apology and Request Strategies in the EFL Context
Kulsawang, P., & Ambele, E. A. (2024) · IJELTAL (Indonesian Journal of English Language Teaching and Applied Linguistics) · doi:10.21093/ijeltal.v9i1.1601 · Cited by 10
Pradhan, R., Garnick, K., Barkondaj, B., Jordan, H. S., Ash, A. S., & Yu, H. (2018) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2018.05.024 · Cited by 6
“Bolstered by this ease of electronic literature searches, however, although publications of systematic reviews and meta-analyses (SRMAs) have increased exponentially over the last decade, there has not been comparable improvement in methodological rigor or reporting standards [2].”
BackgroundMeulemeester, A. D., Schietse, B., Vermeeren, B., Ghesquière, E., Declève, G., Buysse, H. E. C., Discart, I., Alewaeters, K., Durieux, N., Peleman, R. A., et al. (2018) · Health Information & Libraries Journal · doi:10.1111/hir.12237 · Cited by 6
“It is estimated that over the last decade, there has been a threefold increase in published systematic reviews in biomedical research (Page et al., 2016).”
Esam, H., Kanukula, R., Dhurjati, R., Aerram, R., Chevireddy, S., Bhaumik, S., Atkins, E., Huffman, M. D., Rodgers, A., & Salam, A. (2021) · Journal of Clinical Hypertension · doi:10.1111/jch.14216 · Cited by 6
Revisiones rápidas para la síntesis de la evidencia
Catalá-López, F., Stevens, A., Garritty, C., & Hutton, B. (2017) · Medicina Clínica · doi:10.1016/j.medcli.2016.12.016 · Cited by 9
Self-citationLyu, Z., Huang, Z., Liu, F., & Hou, Z. (2020) · Gastroenterology Research and Practice · doi:10.1155/2020/3868057 · Cited by 6
Gooßen, K., Rombey, T., Kugler, C. M., Santis, K. K. D., & Pieper, D. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.02.006 · Cited by 6
Li, Y., Li, R., Li, X., Liu, L., Zhu, J., & Li, D. (2021) · Medicine · doi:10.1097/md.0000000000025615 · Cited by 9
Chen, Y., Li, L., Zhang, Q., Liu, H., Huang, Y., Lin, S., Yin, G., & Xie, Q. (2021) · Medicine · doi:10.1097/md.0000000000027950 · Cited by 6
Self-citationRapid reviews for evidence synthesis
Catalá-López, F., Stevens, A., Garritty, C., & Hutton, B. (2017) · Medicina Clínica (English Edition) · doi:10.1016/j.medcle.2016.12.068 · Cited by 8
Self-citationGodolphin, P. J., Rogozińska, E., Fisher, D. J., Vale, C. L., & Tierney, J. F. (2022) · Nature Medicine · doi:10.1038/s41591-021-01675-1 · Cited by 7
Littell, J. H., & Gorman, D. M. (2022) · Systematic Reviews · doi:10.1186/s13643-022-01998-1 · Cited by 6
“org), although the quality of reporting of biomedical reviews has improved over time [1].”
BackgroundDiong, J. H. L., Bye, E., Djajadikarta, Z. J., Butler, A. A., Gandevia, S. C., & Héroux, M. E. (2023) · PLoS ONE · doi:10.1371/journal.pone.0283753 · Cited by 5
Yamamoto, N., Taito, S., Miura, T., Ariie, T., Tomita, Y., Ogihara, H., Shiratsuchi, D., Yorifuji, T., & Tsujimoto, Y. (2023) · Journal of Clinical Medicine · doi:10.3390/jcm12227031 · Cited by 6
Methodological quality of umbrella reviews in endodontics: A cross‐sectional study
Nagendrababu, V., Gopinath, V. K., Narasimhan, S., Acharya, A. B., Dummer, P. M. H., & Faggion, C. M. (2024) · International Endodontic Journal · doi:10.1111/iej.14114 · Cited by 7
Several methods for assessing research waste in reviews with a systematic search: a scoping review
Rosengaard, L. O., Andersen, M. Z., Rosenberg, J., & Fonnes, S. (2024) · PeerJ · doi:10.7717/peerj.18466 · Cited by 7
“…and Meta-Analyses (PRISMA) guideline ( Pussegoda et al., 2017a ; Hacke & Nunan, 2019 ; Pussegoda et al., 2017b ; Whear et al., 2022 ; Fisher et al., 2022 ; Page et al., 2016 ) or Quality of Reporting of Meta-analyses (QUOROM) ( Hacke & Nunan, 2019 ; Pussegoda et al., 2017b ; Reddy et al., 2023 ).”
InfluentialMeta-epidemiology and reporting characteristics of mapping reviews: a scoping review
Li, Y., Ghogomu, E., Khalil, H., Hui, X., E, F., Campbell, F., Li, X., Gaarder, M., Nduku, P. M., White, H., et al. (2025) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2024.111648 · Cited by 2
Innovative solutions are needed to overcome implementation barriers to using reporting guidelines
Collins, G. S. (2025) · BMJ · doi:10.1136/bmj.r718 · Cited by 9
Software engineering principles address current problems in the systematic review ecosystem
Bashir, R., & Dunn, A. G. (2018) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2018.12.014 · Cited by 6
Yu, J., Yang, Z., Zhang, Y., Cui, Y., Tang, J., Hirst, A., & Li, Y. (2021) · Asian Journal of Surgery · doi:10.1016/j.asjsur.2021.10.021 · Cited by 6
Decision-making by the NICE Interventional Procedures Advisory Committee
Carroll, C., Dickson, R., Boland, A., Houten, R., & Walton, M. (2019) · British journal of surgery · doi:10.1002/bjs.11334 · Cited by 5
Eskrootchi, R., Shahraki-Mohammadi, A., Panahi, S., & Zahedi, R. (2020) · Evidence Based Library and Information Practice · doi:10.18438/eblip29609 · Cited by 6
“This is consistent with the results of Page et al. (2016) and Opheim et al. (2019), where the full search strategy in at least one database was presented by one-third and less than one-third of the systematic review studies examined.”
InfluentialIntroduction to Systematic Reviews
Li, T., Saldanha, I. J., & Robinson, K. A. (2021) · Springer eBooks · doi:10.1007/978-3-319-52677-5_194-1 · Cited by 5
Publication practices of sub-Saharan African Cochrane authors: a bibliometric study
Mbuagbaw, L. C. E., Schoonees, A., Oliver, J., Arikpo, D., Durão, S., Effa, E., Hohlfeld, A., Kredo, T., Wiysonge, C. S., & Young, T. (2021) · BMJ Open · doi:10.1136/bmjopen-2021-051839 · Cited by 6
The importance of meta-research in dentistry
Sarkis‐Onofre, R., Sofi‐Mahmudi, A., Puljak, L., & Moraes, R. R. (2023) · Evidence-Based Dentistry · doi:10.1038/s41432-023-00880-w · Cited by 5
Self-citationTikhonova, E., & Raitskaya, L. (2025) · Journal of language and Education · doi:10.17323/jle.2025.28633 · Cited by 6
Tobias, J. H., Samaras, K., Ivell, R., Davies, T. F., Sjöholm, Å., Day-Haynes, I., & Holly, J. M. P. (2024) · Frontiers in Endocrinology · doi:10.3389/fendo.2024.1400583 · Cited by 5
“Evaluations of systematic reviews and meta-analyses published across biomedical research revealed many to be poorly conducted and conclusions drawn either redundant or misleading (2, 3).”
BackgroundSumsuzzman, D. M., Kim, Y., Baek, S., & Hong, Y. (2024) · Journal of Lifestyle Medicine · doi:10.15280/jlm.2024.14.2.57 · Cited by 4
Sasia, F. S., Berrío, S. P. R., Ascúa, R. A., & Ramirez-Barrera, A. (2024) · DYNA · doi:10.15446/dyna.v91n233.113699 · Cited by 4
Goldkuhle, M., Hirsch, C., Iannizzi, C., Zorger, A.-M., Bender, R., van Dalen, E. C., Hemkens, L. G., Monsef, I., Kreuzberger, N., & Skoetz, N. (2024) · BMC Medical Research Methodology · doi:10.1186/s12874-024-02401-4 · Cited by 6
Reporting Guidelines for Imaging Research
McInnes, M. D. F., Lim, C. S., van der Pol, C. B., Salameh, J., McGrath, T. A., & Frank, R. A. (2018) · Seminars in Nuclear Medicine · doi:10.1053/j.semnuclmed.2018.11.008 · Cited by 7
Jesus, T. S. (2022) · The International Journal of Health Planning and Management · doi:10.1002/hpm.3510 · Cited by 4
Oliván, J. A. S., Cuenca, G. M., & Avilés, R. A. (2023) · Ibersid revista de sistemas de información y documentación · doi:10.54886/ibersid.v17i1.4892 · Cited by 5
Santos, V. S., Fandim, J. V., Silva, F. G., Hatakeyama, B. A., Fioratti, I., Costa, L. O. P., Saragiotto, B. T., & Yamato, T. P. (2023) · Musculoskeletal Science and Practice · doi:10.1016/j.msksp.2023.102902 · Cited by 4
Preparing a Systematic Review for the American Journal of Ophthalmology: Updated Guidance
Virgili, G., & Parrish, R. K., II (2017) · American Journal of Ophthalmology · doi:10.1016/j.ajo.2017.08.014 · Cited by 4
“T HE PRODUCTION OF SYSTEMATIC REVIEWS (SRS) HAS increased exponentially in recent years.(1) With this rise in popularity, researchers issued warnings regarding the quality and reliability of a large fraction of such reviews,(2,3) leading to indiscriminate accusations of poor conduct, irrelevance, and even fabrication.”
BackgroundNature and reporting characteristics of UK health technology assessment systematic reviews
Carroll, C., & Kaltenthaler, E. (2018) · BMC Medical Research Methodology · doi:10.1186/s12874-018-0498-6 · Cited by 4
“This is important because the results of poorlyconducted, reported or flawed systematic reviews include ‘misleading conclusions’ , which can have major implications for decision-making [1].”
BackgroundMethodInfluentialInconsistent Reporting Between Meta-analysis Protocol and Publication – A Cross-Sectional Study
Delgado, A. F., & Delgado, A. F. (2017) · Anticancer Research · doi:10.21873/anticanres.11928 · Cited by 4
Ayorinde, A., Williams, I., Mannion, R., Song, F., Skrybant, M., Lilford, R. J., & Chen, Y. (2020) · Health Services and Delivery Research · doi:10.3310/hsdr08330 · Cited by 6
“Type of review, n (%) Association (n = 100, 50%) 5 (5) 95 (95)”
Cristea, I. A., Georgescu, R., & Ioannidis, J. P. A. (2021) · Clinical Psychological Science · doi:10.1177/21677026211049366 · Cited by 5
“In addition, the quality of reporting and analysis might also have improved with time (Page et al., 2016; Wen et al., 2008).”
BackgroundSurian, D., Bourgeois, F. T., & Dunn, A. G. (2021) · BMC Medical Research Methodology · doi:10.1186/s12874-021-01485-6 · Cited by 4
Banno, M., Tsujimoto, Y., Kohmura, K., Dohi, E., Taito, S., Someko, H., & Kataoka, Y. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph191912261 · Cited by 5
“Moreover, poor reporting of the results of primary outcomes and adverse events in RTCs and SRs might hinder the application of the article’s results to patient treatments [3,4].”
BackgroundPrevalence and methodological quality of systematic reviews in Korean medical journals
Kim, S., Han, M. A., Jung, J. H., Hwang, E. C., Kim, H. R., Yoon, S. E., Kim, S.-H., Kim, P., & Kim, S.-Y. (2023) · Epidemiology and Health · doi:10.4178/epih.e2023017 · Cited by 4
“A previous study conducted in 2014 reported that more than 8,000 SRs were indexed in MEDLINE annually, with a three-fold increase over the last decade [1].”
BackgroundCochrane review abstracts are getting longer, but this has no large impact on the reporting quality
Helbach, J., Wandscher, K., Pieper, D., & Hoffmann, F. (2023) · Journal of Evidence-Based Medicine · doi:10.1111/jebm.12545 · Cited by 7
Whaley, P., Wattam, S., Scott, A. M., Vidler, J. E., & Evidence-Based Toxicology Collaboration (2023) · Evidence-Based Toxicology · doi:10.1080/2833373x.2023.2259938 · Cited by 4
Wang, L., Rouse, B., Marks‐Anglin, A., Duan, R., Shi, Q., Quach, K., Chen, Y., Cameron, C., Schmid, C. H., & Li, T. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.06.010 · Cited by 4
Kim, T., Tian, Z., Liao, X., & Robinson, N. (2021) · European Journal of Integrative Medicine · doi:10.1016/j.eujim.2021.101605 · Cited by 4
Metadata
- Authors
- Page, M. J., Shamseer, L., Altman, D. G., Tetzlaff, J., Sampson, M., Tricco, A. C., Catalá-López, F., Li, L., Reid, E. K., Sarkis-Onofre, R., & Moher, D.
- Affiliations
- University of Bristol (GB)
- University of Ottawa (CA)
- University of Oxford (GB)
- Ottawa Hospital Research Institute (CA)
- Children's Hospital of Eastern Ontario (CA)
- St. Michael's Hospital (CA)
- Universitat de València (ES)
- Lanzhou University (CN)
- Vancouver General Hospital (CA)
- Corresponding author
- Moher, David
- ORCID
- Venue
- PLOS Medicine
- Publisher
- Public Library of Science (PLoS)
- Published
- 2016-05-24
- Volume
- 13
- Issue
- 5
- Pages
- e1002028
- ISSN
- 1549-1277 (ISSN-L)
- 1549-1676
- PMC
- PMC4878797
- OpenAlex
- W2396846174
- Semantic Scholar
- 363d54e2baf229adfd585079bfd7c18ae8feed26
- Cited by
- 768
- References
- 41
- Citations by year
- 2026: 25 · 2025: 43 · 2024: 85 · 2023: 73 · 2022: 85 · 2021: 146 · 2020: 95 · 2019: 83 · 2018: 58 · 2017: 60 · 2016: 15
- FWCI
- 24.71
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS Medicine (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Explore Bristol Research (publishedVersion, other-oa)
- Monash University Research Portal (Monash University) (acceptedVersion, other-oa)
- Lanzhou University Institutional Repository (submittedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Bristol Research (University of Bristol) (submittedVersion, other-oa)
- Funders
- National Institute for Health and Care Research
- Coordenação de Aperfeiçoamento de Pessoal de Nível Superior
- Governo Brasil
- Generalitat Valenciana (PROMETEOII/2015/021)
- Canadian Institutes of Health Research (1088535)
- Medical Research Council (MR/J00488X/1)
- National Health and Medical Research Council (1088535)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Health Sciences › Nursing › Issues, ethics and legal aspects › Pharmaceutical industry and healthcare
- Keywords
- systematic reviews, reporting quality, meta-analysis, risk of bias assessment, publication bias, research funding disclosure, biomedical literature, research quality improvement, research integrity, research transparency, research accountability, research reproducibility, research credibility, research trustworthiness
- MeSH terms
- Cross-Sectional Studies, Epidemiologic Methods, Humans, Research Design, Review Literature as Topic, Biomedical Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.