Methodological Concerns and Quality Appraisal of Contemporary Systematic Reviews and Meta-Analyses in Pediatric Urology
Abstract
PURPOSE: The usefulness of systematic reviews and meta-analyses in influencing clinical practice depends on their quality. We sought to analyze the quality of published systematic reviews and meta-analyses in pediatric urology. MATERIALS AND METHODS: We searched PubMed (MEDLINE) and Embase for all systematic reviews and meta-analyses published in the top 5 pediatric urology journals between January 2000 and November 2009. Two reviewers independently selected articles for full text review. Scientific methodological quality was evaluated using the Assessment of Multiple Systematic Reviews 11-item tool. RESULTS: Of 267 initial results 220 articles were excluded because they were surveys, case reports or narrative reviews. Full text evaluation of the remaining 47 articles further excluded 32 series of exclusively adult patients, leaving 15 for final analysis. Seven articles (47%) were published in 2009 (p <0.01). Only 1 review (7%) described a full search strategy and 3 (20%) allowed inclusion of non-English studies. In 8 reviews (53%) selection of studies was performed by 2 reviewers. Five systematic reviews (33%) described some form of quality assessment. Only 5 reviews (33%) described assessment of publication bias, while 8 (53%) checked for heterogeneity among studies. According to AMSTAR criteria, 7 systematic reviews (47%) were considered of less than fair methodological quality, 5 (33%) fair to good quality and 3 (20%) good quality. CONCLUSIONS: Despite a recent increase in the number of systematic reviews and meta-analyses published in pediatric urology journals, almost half of these reviews lack good scientific quality, raising concerns about their role in influencing clinical practice. Efforts should be made to improve the methodological quality of systematic reviews and meta-analyses in the pediatric urology literature.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
An appraisal of SRs/MAs in pediatric urology journals reported that almost half lacked good scientific quality.
“In agreement, an appraisal of the SRs/MAs in pediatric urology journals reported that almost half lacked good scientific quality, raising concerns about their role in clinical practice [16].”
From Reporting quality of abstracts of systematic reviews/meta-analyses: An appraisal of Arab Journal of Urology across 12 years: the PRISMA-Abstracts checklist (2023) · cites this paper as [16]
Cites
The Hazards of Scoring the Quality of Clinical Trials for Meta-analysis
Jüni, P. (1999) · JAMA · doi:10.1001/jama.282.11.1054 · Cited by 1,981 · see citing works
Systematic Reviews in Health Care: Meta‐Analysis in Context
(2001) · doi:10.1002/9780470693926 · Cited by 1,693 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Systematic review adherence to methodological or reporting quality
Pussegoda, K., Turner, L., Garritty, C., Mayhew, A., Skidmore, B., Stevens, A., Boutron, I., Sarkis-Onofre, R., Bjerre, L. M., Hróbjartsson, A., et al. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0527-2 · Cited by 322 · see citing works
Johal, A., Fleming, P. S., Manek, S., & Marinho, V. C. C. (2015) · Sleep and Breathing · doi:10.1007/s11325-015-1148-4 · Cited by 15 · see citing works
El Ansari, W., AlRumaihi, K., El-Ansari, K., Arafa, M., Elbardisi, H., Majzoub, A., Shamsodini, A., & Al Ansari, A. (2023) · Arab Journal of Urology · doi:10.1080/2090598x.2022.2113127 · Cited by 6 · see citing works
“quality, raising concerns about their role in clinical practice [16].”
Background
- Authors
- Braga, L. H., Pemberton, J., DeMaria, J., & Lorenzo, A. J.
- Affiliations
- University of Toronto (CA)
- Venue
- Journal of Urology
- Published
- 2011-07
- Volume
- 186
- Issue
- 1
- Pages
- 266-272
- ISSN
- 0022-5347 (ISSN-L)
- 1527-3792
- OpenAlex
- W2001959354
- Cited by
- 50
- References
- 2
- Citations by year
- 2025: 1 · 2024: 1 · 2023: 2 · 2022: 1 · 2021: 6 · 2020: 2 · 2019: 8 · 2018: 4 · 2017: 10 · 2015: 5 · 2014: 3 · 2013: 5 · 2012: 1
- FWCI
- 0.78
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- The Journal of Urology (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › scientometrics and bibliometrics research
- Keywords
- meta-analysis, methodological quality, AMSTAR, pediatric urology, publication bias, heterogeneity, search strategies, quality assessment
- MeSH terms
- Pediatrics, Review Literature as Topic, Urology, Meta-Analysis as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
