Systematic review adherence to methodological or reporting quality
Open accessThe DOI matches its Crossref record; 84 of the 85 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Guidelines for assessing methodological and reporting quality of systematic reviews (SRs) were developed to contribute to implementing evidence-based health care and the reduction of research waste. As SRs assessing a cohort of SRs is becoming more prevalent in the literature and with the increased uptake of SR evidence for decision-making, methodological quality and standard of reporting of SRs is of interest. The objective of this study is to evaluate SR adherence to the Quality of Reporting of Meta-analyses (QUOROM) and PRISMA reporting guidelines and the A Measurement Tool to Assess Systematic Reviews (AMSTAR) and Overview Quality Assessment Questionnaire (OQAQ) quality assessment tools as evaluated in methodological overviews. METHODS: The Cochrane Library, MEDLINE®, and EMBASE® databases were searched from January 1990 to October 2014. Title and abstract screening and full-text screening were conducted independently by two reviewers. Reports assessing the quality or reporting of a cohort of SRs of interventions using PRISMA, QUOROM, OQAQ, or AMSTAR were included. All results are reported as frequencies and percentages of reports and SRs respectively. RESULTS: Of the 20,765 independent records retrieved from electronic searching, 1189 reports were reviewed for eligibility at full text, of which 56 reports (5371 SRs in total) evaluating the PRISMA, QUOROM, AMSTAR, and/or OQAQ tools were included. Notable items include the following: of the SRs using PRISMA, over 85% (1532/1741) provided a rationale for the review and less than 6% (102/1741) provided protocol information. For reports using QUOROM, only 9% (40/449) of SRs provided a trial flow diagram. However, 90% (402/449) described the explicit clinical problem and review rationale in the introduction section. Of reports using AMSTAR, 30% (534/1794) used duplicate study selection and data extraction. Conversely, 80% (1439/1794) of SRs provided study characteristics of included studies. In terms of OQAQ, 37% (499/1367) of the SRs assessed risk of bias (validity) in the included studies, while 80% (1112/1387) reported the criteria for study selection. CONCLUSIONS: Although reporting guidelines and quality assessment tools exist, reporting and methodological quality of SRs are inconsistent. Mechanisms to improve adherence to established reporting guidelines and methodological assessment tools are needed to improve the quality of SRs.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Without transparent reporting, even well-designed reviews will fail to show their methodological strengths, undermining their utility in decision-making contexts [17].”
From ROSES reporting standards for systematic evidence syntheses: pro forma, flow-diagram and descriptive summary of the plan and conduct of environmental systematic reviews and systematic maps (2018) · cites this paper as [17]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Nevertheless, substandard reviews remain more numerous (see [15,16]) with flaws in planning and design (e.g.protocol either missing or lacks crucial details), conduct (e.g.non-comprehensive search) and/or reporting (e.g.poor clarity or comprehensiveness in the write-up) [17,18].”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000100 · Cited by 28,164 · cited 2×
“SRs and meta-analyses were defined based on the guidelines provided by the Cochrane Collaboration and the PRISMA Statement [12, 16].”
Self-citationPRISMA-P Group, Moher, D., Shamseer, L., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P., & Stewart, L. A. (2015) · Systematic Reviews · doi:10.1186/2046-4053-4-1 · Cited by 26,223 · cited 3×
“Review protocols allow researchers to plan and anticipate potential issues, assess validity of methods, and prevent arbitrary decisionmaking [78, 79].”
Self-citationShamseer, L., Moher, D., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P., Stewart, L. A., & the PRISMA-P Group (2015) · BMJ · doi:10.1136/bmj.g7647 · Cited by 13,120 · cited 3×
“Review protocols allow researchers to plan and anticipate potential issues, assess validity of methods, and prevent arbitrary decisionmaking [78, 79].”
Self-citationLiberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-4-200908180-00136 · Cited by 10,349
Self-citationMoher, D., Cook, D. J., Eastwood, S., Olkin, I., Rennie, D., & Stroup, D. F. (1999) · The Lancet · doi:10.1016/s0140-6736(99)04149-5 · Cited by 4,826
Self-citationDevelopment of AMSTAR: A measurement tool to assess the methodological quality of systematic reviews
Shea, B. J., Grimshaw, J. M., Wells, G. A., Boers, M., Andersson, N., Hamel, C., Porter, A. C., Tugwell, P., Moher, D., & Bouter, L. M. (2007) · BMC Medical Research Methodology · doi:10.1186/1471-2288-7-10 · Cited by 4,374
Self-citationROBIS: A new tool to assess risk of bias in systematic reviews was developed
Whiting, P., Savović, J., Higgins, J. P. T., Caldwell, D. M., Reeves, B. C., Shea, B., Davies, P., Kleijnen, J., & Churchill, R. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.06.005 · Cited by 2,272
Avoidable waste in the production and reporting of research evidence
Chalmers, I., & Glasziou, P. (2009) · The Lancet · doi:10.1016/s0140-6736(09)60329-9 · Cited by 2,208
Systematic Reviews: Synthesis of Best Evidence for Clinical Decisions
Cook, D. J., Mulrow, C. D., & Haynes, R. B. (1997) · Annals of Internal Medicine · doi:10.7326/0003-4819-126-5-199703010-00006 · Cited by 2,184 · cited 2×
“Unfortunately, poorly conducted or reported SRs may be associated with bias, limiting their usefulness [5].”
Shea, B. J., Hamel, C., Wells, G. A., Bouter, L. M., Kristjansson, E., Grimshaw, J., Henry, D. A., & Boers, M. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2008.10.009 · Cited by 1,755 · cited 1×
“As such, OQAQ differs from AMSTAR, which was published and validated more recently [80, 81].”
Meta-Analyses of Randomized Controlled Trials
Sacks, H. S., Berrier, J., Reitman, D., Ancona-Berk, V. A., & Chalmers, T. C. (1987) · New England Journal of Medicine · doi:10.1056/nejm198702193160806 · Cited by 1,250 · cited 1×
“With the rise of evidence-based medicine, criteria for assessing quality began to emerge, such as Mulrow [6] and Sacks [7].”
Methodology in conducting a systematic review of systematic reviews of healthcare interventions
Smith, V., Devane, D., Begley, C. M., & Clarke, M. (2011) · BMC Medical Research Methodology · doi:10.1186/1471-2288-11-15 · Cited by 1,227 · cited 1×
“We adopted the term overview to mean a summary of evidence from more than one SR at a variety of different levels, including the combination of different interventions, different outcomes, different conditions, problems or populations, or the provision of a summary of evidence on the adverse events of an intervention…”
Epidemiology and reporting characteristics of systematic reviews.
Moher, D., Tetzlaff, J., Tricco, A. C., Sampson, M., & Altman, D. G. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040078 · Cited by 891 · cited 5×
“Our study, like others, shows that reporting of review protocols is poorly reported [2, 24].”
InfluentialSelf-citationPanic, N., Leoncini, E., de Belvis, G., Ricciardi, W., & Boccia, S. (2013) · PLoS ONE · doi:10.1371/journal.pone.0083138 · Cited by 883 · cited 1×
“One small study found that reporting and methodological quality (adherence to both AMSTAR and PRISMA) significantly increased after journal endorsement of the PRISMA guidelines [25].”
The Medical Review Article: State of the Science
MULROW, C. D. (1987) · Annals of Internal Medicine · doi:10.7326/0003-4819-106-3-485 · Cited by 856 · cited 1×
“With the rise of evidence-based medicine, criteria for assessing quality began to emerge, such as Mulrow [6] and Sacks [7].”
Validation of an index of the quality of review articles
Oxman, A. D., & Guyatt, G. H. (1991) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(91)90160-b · Cited by 786 · cited 2×
“In 1991, Oxman and Guyatt developed the Overview Quality Assessment Questionnaire (OQAQ) [8], a validated tool to assess methodological quality for SRs of intervention studies.”
Page, M. J., Shamseer, L., Altman, D. G., Tetzlaff, J., Sampson, M., Tricco, A. C., Catalá-López, F., Li, L., Reid, E. K., Sarkis-Onofre, R., et al. (2016) · PLOS Medicine · doi:10.1371/journal.pmed.1002028 · Cited by 768 · cited 1×
“SR evidence has become increasingly important in clinical decision-making and for informing clinical guidelines and health policy [2, 3].”
Self-citationAn evidence-based practice guideline for the peer review of electronic search strategies
Sampson, M., McGowan, J., Cogo, E., Grimshaw, J., Moher, D., & Lefebvre, C. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2008.10.012 · Cited by 608 · cited 1×
“The search strategy was peer-reviewed prior to execution [20].”
Self-citationAvoidable Waste in the Production and Reporting of Research Evidence
Chalmers, I., & Glasziou, P. (2009) · Obstetrics & Gynecology · doi:10.1097/aog.0b013e3181c3020d · Cited by 600 · cited 1×
“Poorly conducted SRs can lead to inaccurate estimates of treatment effectiveness, misleading conclusions, and reduced applicability, all of which are a waste of limited resources [4].”
Kung, J. (2010) · The Open Dentistry Journal · doi:10.2174/1874210601004020084 · Cited by 329 · cited 1×
“In 2010, a revised tool (RAMSTAR) was developed to provide a quantitative scoring method to assess quality [10].”
Page, M. J., McKenzie, J. E., Kirkham, J., Dwan, K., Kramer, S., Green, S., & Forbes, A. (2014) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000035.pub2 · Cited by 281 · cited 1×
“In addition, evidence is emerging that biases within SRs could influence results and quality of overviews [86].”
Systematic reviews and meta-analyses on treatment of asthma: critical evaluation
Jadad, A. R. (2000) · BMJ · doi:10.1136/bmj.320.7234.537 · Cited by 263
Stevens, A., Shamseer, L., Weinstein, E., Yazdi, F., Turner, L., Thielman, J., Altman, D. G., Hirst, A., Hoey, J., Palepu, A., et al. (2014) · BMJ · doi:10.1136/bmj.g3804 · Cited by 242 · cited 2×
“Further, in thinking about publication and then journal endorsement as potential interventions, we would agree with previously published work that journal endorsement might serve as a “stronger” intervention [87].”
Self-citationMoseley, A. M., Elkins, M. R., Herbert, R. D., Maher, C. G., & Sherrington, C. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2008.09.018 · Cited by 202
The Physical Environment and Physical Activity
Gebel, K., Bauman, A. E., & Petticrew, M. (2007) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2007.01.020 · Cited by 195
Pieper, D., Mathes, T., Neugebauer, E., & Eikermann, M. (2013) · Journal of the American College of Surgeons · doi:10.1016/j.jamcollsurg.2012.12.049 · Cited by 160
Biondi-Zoccai, G. G. L., Lotrionte, M., Abbate, A., Testa, L., Remigi, E., Burzotta, F., Valgimigli, M., Romagnoli, E., Crea, F., & Agostoni, P. (2006) · BMJ · doi:10.1136/bmj.38693.516782.7c · Cited by 149
Tunis, A. S., McInnes, M. D. F., Hanna, R., & Esmail, K. (2013) · Radiology · doi:10.1148/radiol.13130273 · Cited by 146
Systematic reviews published in higher impact clinical journals were of higher quality
Fleming, P. S., Koletsi, D., Seehra, J., & Pandis, N. (2014) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2014.01.002 · Cited by 118
Delaney, A., Bagshaw, S. M., Ferland, A., Laupland, K., Manns, B., & Doig, C. (2007) · Critical Care Medicine · doi:10.1097/01.ccm.0000253394.15628.fd · Cited by 112
Reporting and Methodological Quality of Systematic Reviews in the Orthopaedic Literature
Gagnier, J. J., & Kellam, P. J. (2013) · Journal of Bone and Joint Surgery · doi:10.2106/jbjs.l.00597 · Cited by 112
Shea, B., Moher, D., Graham, I., Pham, B., & Tugwell, P. (2002) · Evaluation & the Health Professions · doi:10.1177/0163278702025001008 · Cited by 110
Self-citationA PRISMA assessment of the reporting quality of systematic reviews in orthodontics
Fleming, P. S., Seehra, J., Polychronopoulou, A., Fedorowicz, Z., & Pandis, N. (2013) · The Angle Orthodontist · doi:10.2319/032612-251.1 · Cited by 105
Cochrane and non-Cochrane systematic reviews in leading orthodontic journals: a quality paradigm?
Fleming, P. S., Seehra, J., Polychronopoulou, A., Fedorowicz, Z., & Pandis, N. (2013) · The European Journal of Orthodontics · doi:10.1093/ejo/cjs016 · Cited by 95
Assessment of therapeutic safety in systematic reviews: literature review
Ernst, E., & Pittler, M. H. (2001) · BMJ · doi:10.1136/bmj.323.7312.546 · Cited by 94 · cited 1×
“They are powerful tools used to assess treatment effectiveness which can subsequently improve patient care [1].”
Ma, I. W. Y., Khan, N. A., Kang, A., Zalunardo, N., & Palepu, A. (2007) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2006.11.009 · Cited by 89 · cited 1×
“Other studies have also found that methodological and reporting quality is suboptimal [2, 84, 85].”
The reporting quality of meta-analyses improves: a random sampling study
Wen, J., Ren, Y., Wang, L., Li, Y., Liu, Y., Zhou, M., Liu, P., Ye, L., Li, Y., & Tian, W. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.10.008 · Cited by 84 · cited 1×
“Additional reports eligible for inclusion were identified by members of the research team prior to the start of the project [2, 21, 22].”
Evaluating the quality of systematic reviews in the emergency medicine literature
Kelly, K. D., Travers, A. H., Dorgan, M., Slater, L., & Rowe, B. H. (2001) · Annals of Emergency Medicine · doi:10.1067/mem.2001.115881 · Cited by 84
Moher, D., Soeken, K., Sampson, M., Ben-Porat, L., & Berman, B. (2002) · BMC Pediatrics · doi:10.1186/1471-2431-2-3 · Cited by 81
Self-citationSeo, H.-J., & Kim, K. U. (2012) · BMC Medical Research Methodology · doi:10.1186/1471-2288-12-129 · Cited by 80
Quality of systematic reviews in pediatric oncology – A systematic review
Lundh, A., Knijnenburg, S. L., Jørgensen, A. W., van Dalen, E. C., & Kremer, L. C. M. (2009) · Cancer Treatment Reviews · doi:10.1016/j.ctrv.2009.08.010 · Cited by 78
An analysis of quality of systematic reviews on pharmacist health interventions
Melchiors, A. C., Correr, C. J., Venson, R., & Pontarolo, R. (2011) · International Journal of Clinical Pharmacy · doi:10.1007/s11096-011-9592-0 · Cited by 76
Examining the Evidence in Anesthesia Literature: A Critical Appraisal of Systematic Reviews
T-L. Choi, P., Halpern, S. H., Malik, N., Jadad, A. R., Tramèr, M. R., & Walder, B. (2001) · Anesthesia & Analgesia · doi:10.1097/00000539-200103000-00029 · Cited by 75
Pussegoda, K., Turner, L., Garritty, C., Mayhew, A., Skidmore, B., Stevens, A., Boutron, I., Sarkis-Onofre, R., Bjerre, L. M., Hróbjartsson, A., et al. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0507-6 · Cited by 75 · cited 7×
“and October 2014 with a primary objective to assess the quality of methodology, reporting, or other quality characteristics of SRs was included [15].”
InfluentialSelf-citationCollier, A., Heilig, L., Schilling, L., Williams, H., & Dellavalle, R. P. (2006) · British Journal of Dermatology · doi:10.1111/j.1365-2133.2006.07496.x · Cited by 73
Cochrane Systematic Reviews of Chinese Herbal Medicines: An Overview
Hu, J., Zhang, J., Zhao, W., Zhang, Y., Zhang, L., & Shang, H. (2011) · PLoS ONE · doi:10.1371/journal.pone.0028696 · Cited by 71
Evaluation of methodology and quality characteristics of systematic reviews in orthodontics
Papageorgiou, S., Papadopoulos, M., & Athanasiou, A. (2011) · Orthodontics & Craniofacial Research · doi:10.1111/j.1601-6343.2011.01522.x · Cited by 68
Sequeira-Byron, P., Fedorowicz, Z., Jagannath, V. A., & Sharif, M. O. (2011) · Journal of Applied Oral Science · doi:10.1590/s1678-77572011000500002 · Cited by 64 · cited 1×
“One small study has also shown the need for better adherence to AMSTAR [82].”
Anttila, H., Samuelsson, K., Salminen, A., & Brandt, Å. (2012) · Technology and Disability · doi:10.3233/tad-2012-0332 · Cited by 63
Ma, B., Guo, J., Qi, G., Li, H., Peng, J., Zhang, Y., Ding, Y., & Yang, K. (2011) · PLoS ONE · doi:10.1371/journal.pone.0020185 · Cited by 56 · cited 1×
“Additional reports eligible for inclusion were identified by members of the research team prior to the start of the project [2, 21, 22].”
MacDonald, S. L., Canfield, S. E., Fesperman, S. F., & Dahm, P. (2010) · Journal of Urology · doi:10.1016/j.juro.2010.03.127 · Cited by 55
Quality of reviews on sugar-sweetened beverages and health outcomes: a systematic review
Weed, D. L., Althuis, M. D., & Mink, P. J. (2011) · American Journal of Clinical Nutrition · doi:10.3945/ajcn.111.015875 · Cited by 54
Braga, L. H., Pemberton, J., DeMaria, J., & Lorenzo, A. J. (2011) · Journal of Urology · doi:10.1016/j.juro.2011.03.044 · Cited by 50
Brugha, T. S., Matthews, R., Morgan, Z., Hill, T., Alonso, J., & Jones, D. R. (2012) · British Journal of Psychiatry · doi:10.1192/bjp.bp.111.098103 · Cited by 50 · cited 1×
“Other studies have also found that methodological and reporting quality is suboptimal [2, 84, 85].”
Does updating improve the methodological and reporting quality of systematic reviews?
Shea, B., Boers, M., Grimshaw, J. M., Hamel, C., & Bouter, L. M. (2006) · BMC Medical Research Methodology · doi:10.1186/1471-2288-6-27 · Cited by 49
Junhua, Z., Hongcai, S., Xiumei, G., Boli, Z., Yaozu, X., Hongbo, C., & Ming, R. (2007) · The Journal of Alternative and Complementary Medicine · doi:10.1089/acm.2007.7195 · Cited by 48
Reporting and methodologic quality of Cochrane Neonatal review group systematic reviews
Faleh, K. A., & Al-Omran, M. A. (2009) · BMC Pediatrics · doi:10.1186/1471-2431-9-38 · Cited by 48
Windsor, B., Popovich, I., Jordan, V., Showell, M., Shea, B., & Farquhar, C. (2012) · Human Reproduction · doi:10.1093/humrep/des342 · Cited by 46
Methodological quality of systematic reviews on influenza vaccination
Remschmidt, C., Wichmann, O., & Harder, T. (2014) · Vaccine · doi:10.1016/j.vaccine.2014.01.060 · Cited by 44
Shea, B., Bouter, L. M., Grimshaw, J., Francis, D. P., Ortiz, Z., Wells, G. A., Tugwell, P., & Boers, M. (2006) · PubMed · Cited by 43
Lawson, M. L., Pham, B., Klassen, T. P., & Moher, D. (2005) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2004.08.022 · Cited by 40
Self-citationQigong for healthcare: an overview of systematic reviews
Lee, M. S., Oh, B., & Ernst, E. (2011) · JRSM Short Reports · doi:10.1258/shorts.2010.010091 · Cited by 38
Analysis of reporting of systematic reviews in physical therapy published in Portuguese
Padula, R. S., Pires, R. S., Alouche, S. R., Chiavegato, L. D., Lópes, A. D., & Costa, L. O. P. (2012) · Brazilian Journal of Physical Therapy · doi:10.1590/s1413-35552012005000040 · Cited by 36
Quality of reporting of systematic reviews published in “evidence-based” Chinese journals
Li, J.-L., Ge, L., Ma, J.-C., Zeng, Q.-L., Yao, L., An, N., Ding, J.-X., Gan, Y.-H., & Tian, J.-H. (2014) · Systematic Reviews · doi:10.1186/2046-4053-3-58 · Cited by 33
Need for Quality Improvement in Renal Systematic Reviews
Mrkobrada, M., Thiessen‐Philbrook, H., Haynes, R. B., Iansavichus, A. V., Rehman, F., & Garg, A. X. (2008) · Clinical Journal of the American Society of Nephrology · doi:10.2215/cjn.04401007 · Cited by 31
Conway, A., Inglis, S. C., Chang, A. M., Horton-Breshears, M., Cleland, J. G., & Clark, R. A. (2013) · Journal of Telemedicine and Telecare · doi:10.1177/1357633x13503427 · Cited by 30
The Quality of Systematic Reviews in Hand Surgery: An Analysis Using AMSTAR
Momeni, A., Lee, G., & Talley, J. R. (2013) · Plastic & Reconstructive Surgery · doi:10.1097/prs.0b013e3182818d24 · Cited by 26
Quality Assessment of Systematic Reviews on Periodontal Regeneration in Humans
Elangovan, S., Ávila‐Ortiz, G., Johnson, G. K., Karimbux, N. Y., & Allareddy, V. (2012) · Journal of Periodontology · doi:10.1902/jop.2012.120021 · Cited by 26
Systematic reviews supporting practice guideline recommendations lack protection against bias
Brito, J. P., Τσάπας, Α., Griebeler, M. L., Wang, Z., Prutsky, G. J., Domecq, J. P., Murad, M. H., & Montori, V. M. (2013) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2013.01.008 · Cited by 26
Ma, B., Qi, G.-Q., Lin, X.-T., Wang, T., Chen, Z.-M., & Yang, K.-H. (2012) · The Journal of Alternative and Complementary Medicine · doi:10.1089/acm.2011.0274 · Cited by 24
Boluyt, N., van der Lee, J. H., Moyer, V. A., Brand, P. L. P., & Offringa, M. (2007) · PEDIATRICS · doi:10.1542/peds.2006-3381 · Cited by 23
Impact of choice of quality appraisal tool for systematic reviews in overviews
Pieper, D., Mathes, T., & Eikermann, M. (2014) · Journal of Evidence-Based Medicine · doi:10.1111/jebm.12097 · Cited by 21
Santaguida, P. L., Oremus, M., Walker, K. L., Wishart, L. R., Siegel, K. L., & Raina, P. S. (2012) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.10.012 · Cited by 20
Methodological quality and descriptive characteristics of prosthodontic‐related systematic reviews
Aziz, T., Compton, S. M., Nassar, U., Matthews, D., Ansari, K., & Flores‐Mir, C. (2013) · Journal of Oral Rehabilitation · doi:10.1111/joor.12028 · Cited by 17
Overview of systematic reviews on invasive treatment of stable coronary artery disease
Kuukasjärvi, P., Malmivaara, A. O. V., Halinen, M., Hartikainen, J. E. K., Keto, P. E., Talvensaari, T., Tierala, I., & Mākelä, M. (2006) · International Journal of Technology Assessment in Health Care · doi:10.1017/s026646230605104x · Cited by 17
Characteristics and Quality of Systematic Reviews of Acupuncture, Herbal Medicines, and Homeopathy
Linde, K., ter Riet, G., Hondras, M., Melchart, D., & Willich, S. (2003) · Complementary Medicine Research · doi:10.1159/000071668 · Cited by 16
Methodological and reporting quality of systematic reviews on tuberculosis
Nicolau, I. A., Ling, D. I., Tian, L., Lienhardt, C., & Pai, M. (2013) · The International Journal of Tuberculosis and Lung Disease · doi:10.5588/ijtld.13.0050 · Cited by 15
How to Develop a Reporting Guideline
Moher, D., Altman, D. G., Schulz, K. F., & Simera, I. (2014) · Wiley eBooks · doi:10.1002/9781118715598.ch2 · Cited by 13 · cited 1×
“A rigorous development of tools for MQ and RQ is important and should involve several steps and appropriate consideration of stakeholders and methodological experts’ participation [88].”
Self-citationSublingual immunotherapy in children with allergic rhinitis: quality of systematic reviews
de Bot, C. M. A., Moed, H. D., Berger, M. Y., Röder, E., van Wijk, R. G., & van der Wouden, J. C. (2011) · Pediatric Allergy and Immunology · doi:10.1111/j.1399-3038.2011.01165.x · Cited by 12
Systematic reviews of surgical procedures in children: Quantity, coverage and quality
McGee, R., Craig, J. C., Rogerson, T. E., & Webster, A. C. (2013) · Journal of Paediatrics and Child Health · doi:10.1111/jpc.12156 · Cited by 9
Barbosa, F. T., Castro, A. A., & de Miranda, C. T. (2012) · Brazilian Journal of Anesthesiology · doi:10.1016/s0034-7094(12)70121-6 · Cited by 9
Luo, J., Xu, H., Yang, G., Qiu, Y., Liu, J., & Chen, K. (2014) · Complementary Therapies in Medicine · doi:10.1016/j.ctim.2014.05.011 · Cited by 9
Xiao, Z., Zhang, Y., Wang, Y., & Xu, F. (2012) · International Journal of Pharmacology · doi:10.3923/ijp.2012.614.620 · Cited by 6
Parmelli, E., Banzi, R., Rio, M. D. P. F. D., Minozzi, S., Moja, L., Pecoraro, V. L., & Liberati, A. M. (2011) · Cited by 1
Research works printed but not linked: 2
- Green S, Higgins JPT, Alderson P, Clarke M, Mulrow CD, Oxman AD. Chapter 1: Introduction. In: Higgins JPT, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 (updated March 2011). The Cochrane Collaboration, 2011. [Internet]. [ http://training.cochrane.org/handbook]. Accessed 27 Jan 2016.
- Becker LA, Oxman AD. Chapter 22: Overviews of reviews. In: Higgins JPT, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 (updated March 2011). The Cochrane Collaboration, 2011 [Internet]. [ http://training.cochrane.org/handbook]. Accessed 14 Jan 2016.
Cited by
Standards for distribution models in biodiversity assessments
Araújo, M. B., Anderson, R. P., Márcia Barbosa, A., Beale, C. M., Dormann, C. F., Early, R., Garcia, R. A., Guisan, A., Maiorano, L., Naimi, B., et al. (2019) · Science Advances · doi:10.1126/sciadv.aat4858 · Cited by 1,134
Haddaway, N. R., Macura, B., Whaley, P., & Pullin, A. S. (2018) · Environmental Evidence · doi:10.1186/s13750-018-0121-7 · Cited by 946
Gates, M., Gates, A., Pieper, D., Fernandes, R. M., Tricco, A. C., Moher, D., Brennan, S. E., Li, T., Pollock, M., Lunny, C., et al. (2022) · BMJ · doi:10.1136/bmj-2022-070849 · Cited by 856
Self-citationO'Dea, R. E., Lagisz, M., Jennions, M. D., Koricheva, J., Noble, D. W. A., Parker, T. H., Gurevitch, J., Page, M. J., Stewart, G., Moher, D., et al. (2021) · Biological Reviews · doi:10.1111/brv.12721 · Cited by 697
Self-citationPage, M. J., & Moher, D. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0663-8 · Cited by 769
Self-citation
Metadata
- Authors
- Pussegoda, K., Turner, L., Garritty, C., Mayhew, A., Skidmore, B., Stevens, A., Boutron, I., Sarkis-Onofre, R., Bjerre, L. M., Hróbjartsson, A., Altman, D. G., & Moher, D.
- Affiliations
- Ottawa Hospital Research Institute (CA)
- Bruyère (CA)
- Inserm (FR)
- University of Ottawa (CA)
- University of Southern Denmark (DK)
- University of Oxford (GB)
- Corresponding author
- Moher, David
- ORCID
- Venue
- Systematic Reviews
- Published
- 2017-07-19
- Volume
- 6
- Issue
- 1
- Pages
- 131-131
- ISSN
- 2046-4053 (ISSN-L)
- PMC
- PMC5516390
- OpenAlex
- W2724414049
- Cited by
- 322
- References
- 85
- Citations by year
- 2026: 21 · 2025: 30 · 2024: 45 · 2023: 42 · 2022: 47 · 2021: 65 · 2020: 29 · 2019: 26 · 2018: 12 · 2017: 4
- FWCI
- 4.80
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Systematic Reviews (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- uO Research (University of Ottawa) (submittedVersion, public-domain)
- University of Southern Denmark Research Portal (University of Southern Denmark) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Canadian Institute for Health Research (CON-105529)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Keywords
- methodological quality, reporting quality, PRISMA, AMSTAR, quality assessment tools, risk of bias, reporting guidelines adherence, research waste, evidence-based care, study selection criteria, data extraction
- MeSH terms
- Systematic Reviews as Topic, Humans, Research Design, Evidence-Based Medicine, Guideline Adherence, Research Report
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
