The DOI matches its Crossref record; 81 of the 89 references it cites were found in a registry or an index.
Evaluating Meta-analyses in the General Surgical Literature: A Critical Appraisal
Open accessAbstract
OBJECTIVE: To assess the methodologic quality of meta-analyses of general surgery topics published in peer-reviewed journals. SUMMARY BACKGROUND DATA: Systematic reviews and meta-analysis are used to seek, summarize, and interpret primary studies on a given topic. Accordingly, systematic reviews and meta-analyses of high-quality primary studies may be the highest level of evidence for issues of prevention and treatment in evidence-based medicine. However, not all published meta-analyses are rigorously performed. METHODS: We searched MEDLINE (from January 1, 1997, to September 1, 2002) and reference lists and solicited general surgery specialists to identify relevant meta-analyses. Inclusion criteria were use of meta-analytic methods to pool the results of primary studies in general surgery on issues of diagnosis, causation, prognosis, or treatment. Our search strategies identified 487 potentially relevant articles. After excluding articles based on a priori criteria, 51 meta-analyses fulfilled eligibility criteria. In duplicate and independently, 2 reviewers assessed the quality of these meta-analyses using a 10-item index called the Overview Quality Assessment Questionnaire. RESULTS: Overall concordance between 2 independent reviewers was good (interobserver agreement 81%, and a kappa of 0.62 (95% CI 0.55-0.69). Of 51 relevant articles, 38 were published in surgical journals. Most studies had major methodologic flaws (median score of 3.3, scale of 1-7). Factors associated with low overall scientific quality included the absence of any prior meta-analyses publications by authors and meta-analyses produced by surgical department members without external collaboration. CONCLUSIONS: This critical appraisal of meta-analyses published in the general surgery literature demonstrates frequent methodologic flaws. The quality of these reports limits the validity of the findings and the inferences that can be made about the primary studies reviewed. To improve the quality of future meta-analyses, we recommend following guidelines for the optimal conduct and reporting of meta-analyses in general surgery.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“However, despite advances, the quality of the conduct and reporting of systematic reviews remains well short of ideal [3],[4],[5],[6].”
From The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of Studies That Evaluate Health Care Interventions: Explanation and Elaboration (2009) · cites this paper as [4]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Bias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722
Schulz (1995) · doi:10.1001/jama.1995.03520290060030 · Cited by 5,634
Moher, D., Cook, D. J., Eastwood, S., Olkin, I., Rennie, D., & Stroup, D. F. (1999) · The Lancet · doi:10.1016/s0140-6736(99)04149-5 · Cited by 4,826
Self-citationMoher, D., Pham, B., Jones, A., Cook, D. J., Jadad, A. R., Moher, M., Tugwell, P., & Klassen, T. P. (1998) · The Lancet · doi:10.1016/s0140-6736(98)01085-x · Cited by 3,560
Self-citationSystematic Reviews in Health Care : Meta-Analysis in Context
Egger, M., Smith, G. D., & Altman, D. G. (2000) · Cited by 2,312
Gluud, L. L., Villumsen, J., & Gluud, C. N. (2001) · Annals of Internal Medicine · doi:10.7326/0003-4819-135-11-200112040-00010 · Cited by 2,266
Klinge, U., Park, J.-K., & Klosterhalfen, B. (2013) · Pathobiology · doi:10.1159/000348446 · Cited by 1,791
Meta-Analyses of Randomized Controlled Trials
Sacks, H. S., Berrier, J., Reitman, D., Ancona-Berk, V. A., & Chalmers, T. C. (1987) · New England Journal of Medicine · doi:10.1056/nejm198702193160806 · Cited by 1,257
Discrepancies between Meta-Analyses and Subsequent Large Randomized, Controlled Trials
LeLorier, J., Grégoire, G., Benhaddad, A. A., Lapierre, J., & Derderian, F. (1997) · New England Journal of Medicine · doi:10.1056/nejm199708213370806 · Cited by 1,164
Lewis, S. J., Egger, M., Sylvester, P. A., & Thomas, S. (2001) · BMJ · doi:10.1136/bmj.323.7316.773 · Cited by 858
Research works printed but not linked: 1
- Wiseman, 1999, Meta-analysis of the safety and efficacy of an adhesion barrier (Interceed TC7) in laparotomy., J Reprod Med, 44, 325
Cited by
Sorted by citations
Items 101–129 of 129
Likhvantsev, V. V., Landoni, G., Subbotin, V. V., Kadantseva, K., Zhukova, L. A., Yadgarov, M. Y., Belletti, A., Novikov, A., Бондоевич, Б. Л., & Kuzovlev, A. N. (2022) · General Reanimatology · doi:10.15360/1813-9779-2022-4-20-28 · Cited by 3
Handsuche nach kontrollierten klinischen Studien in deutschsprachigen chirurgischen Zeitschriften
Blümle, A., Antes, G., & Diener, M. K. (2007) · Der Chirurg · doi:10.1007/s00104-007-1372-y · Cited by 3
Reihe Evidenzbasierte Chirurgie
Herrle, F., & Güller, U. (2009) · Zeitschrift für Herz- Thorax- und Gefäßchirurgie · doi:10.1007/s00398-009-0705-1 · Cited by 2
Recommended clinical practices and evidence-based medicine in surgery
Masquelet, A. (2013) · Journal of Visceral Surgery · doi:10.1016/j.jviscsurg.2013.02.002 · Cited by 2
Ng, T. T., McGory, M. L., Ko, C. Y., & Maggard, M. A. (2017) · Cited by 2
Walter, G., Hirschberger, W., & Hoffmann, R. F. G. (2009) · Trauma und Berufskrankheit · doi:10.1007/s10039-009-1472-7 · Cited by 1
Diseño de una escala para la evaluación de calidad metodológica de estudios de pronóstico
Héctor Losada, M., Manterola, C., Viviana Pineda, N., Manuel Vial, G., & R, A. S. (2009) · Revista de Cirugía · doi:10.4067/s0718-40262009000100011 · Cited by 1
Are Our Publications Failing the Inspection?
Mora-Pinzon, M. C., Hayden, D. M., Ariel, D., Bartosiak, K. A., Chiodo, M. V., Kosmidis, K., Evans, A., & Saclarides, T. J. (2014) · Diseases of the Colon & Rectum · doi:10.1097/dcr.0000000000000169 · Cited by 1
Looking Beyond the Surface: a Beginner’s Guide to the Surgical Literature
Ceelen, W. P. (2005) · Acta chirurgica Belgica · doi:10.1080/00015458.2005.11679756 · Cited by 1
Marler, J. L., Curry, K. Y., & Suda, K. J. (2014) · Therapeutic Innovation & Regulatory Science · doi:10.1177/2168479013516774 · Cited by 1
Application of systematic reviews and meta-analyses in surgical clinical practice
Ríos, A., & Iniesta‐Sepúlveda, M. (2024) · Cirugía Española (English Edition) · doi:10.1016/j.cireng.2024.03.009 · Cited by 1
Die Arbeit der „Cochrane Collaboration“ in der Urologie
Kunath, F., Pittler, M. H., Grobe, H. R., Wullich, B., & Antes, G. (2011) · Der Urologe · doi:10.1007/s00120-010-2490-6
Masquelet, A. (2013) · Journal de Chirurgie Viscérale · doi:10.1016/j.jchirv.2012.01.017
Can Journals Help the Improvement of Meta-Analyses Quality in Plastic Surgery?
Labanaris, A. P., Polykriotis, E., & Horch, R. E. (2006) · Annals of Surgery · doi:10.1097/01.sla.0000198346.55968.d6
Meta-analyses in Surgical Publications
Sotiropoulos, G. C., & Dervenis, C. (2013) · Annals of Surgery · doi:10.1097/sla.0000000000000440
Critically appraise before you believe: The quality of meta-analyses in critical care medicine*
Needham, D. M., Desai, S. V., & Robinson, K. A. (2007) · Critical Care Medicine · doi:10.1097/01.ccm.0000251635.70798.10
Perioperative corticosteroids for patients undergoing elective major abdominal surgery (CORPUS)
Zimmermann, J. B., Turzo, M., Roggenbach, J., Jensen, K., Diener, M. K., Seiler, C. M., Kieser, M., Weigand, M. A., Martin, E., & Hofer, S. (2010) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd008899
RetractedHanna, M. N. (2013) · Journal of Anesthesia & Clinical Research · doi:10.4172/2155-6148.1000275
The Usefulness of Meta-Analyses to Hip and Knee Surgeons
Park, K. J., Lambert, B. S., Brown, L. D., Le, D. T., Chaliki, K., Lee, J., Rossano, A., Benjamin, K., Moreno, M. R., & Incavo, S. J. (2019) · Journal of Bone and Joint Surgery · doi:10.2106/jbjs.19.00420
Systematic Review and Meta-Analysis: A Clinical Exercise
Merchant, N. N., & Maggard-Gibbons, M. A. (2020) · Success in academic surgery · doi:10.1007/978-3-030-28357-5_17
Reihe Evidenzbasierte Chirurgie
Herrle, F., & Güller, U. (2009) · Zeitschrift für Herz- Thorax- und Gefäßchirurgie · doi:10.1007/s00398-009-0721-1
Systematic Review and Meta-analysis: A Clinical Exercise
Gibbons, M. A. M. (2014) · Success in academic surgery · doi:10.1007/978-1-4471-4718-3_15
Zandonadi, F. B., de Moura, F. R., & Amude, A. M. (2021) · UNICIÊNCIAS · doi:10.17921/1415-5141.2021v25n2p80-85
McLeod, R. S. (2008) · Springer eBooks · doi:10.1007/978-0-387-68113-9_2
UZABACI, E., & Can, F. E. (2023) · Kafkas Universitesi Veteriner Fakultesi Dergisi · doi:10.9775/kvfd.2023.30171
Fernández‐Moreno, M., Pérez‐Santiago, L., & Sabater-Orti, L. (2024) · Cirugía Española (English Edition) · doi:10.1016/j.cireng.2024.04.005
Aplicación en la práctica clínica quirúrgica de las revisiones sistemáticas y los metaanálisis
Ríos, A., & Iniesta‐Sepúlveda, M. (2024) · Cirugía Española · doi:10.1016/j.ciresp.2024.03.011
Fernández‐Moreno, M., Pérez‐Santiago, L., & Sabater, L. (2024) · Cirugía Española · doi:10.1016/j.ciresp.2024.04.002
How to Evaluate the Quality of the Published Literature
Chow, A., Purkayastha, S., & Athanasiou, T. (2010) · Springer eBooks · doi:10.1007/978-3-540-71915-1_41
Metadata
- Authors
- Dixon, E., Hameed, M., Sutherland, F., Cook, D. J., & Doig, C.
- Affiliations
- University of Calgary (CA)
- McMaster University (CA)
- Corresponding author
- Dixon, Elijah
- Hameed, Morad
- Sutherland, Francis
- Venue
- Annals of Surgery
- Published
- 2005-03
- Volume
- 241
- Issue
- 3
- Pages
- 450-459
- ISSN
- 0003-4932 (ISSN-L)
- PMC
- PMC1356983
- OpenAlex
- W2081069584
- Cited by
- 129
- References
- 89
- Citations by year
- 2024: 6 · 2023: 4 · 2022: 1 · 2021: 7 · 2020: 5 · 2019: 5 · 2017: 4 · 2016: 4 · 2015: 4 · 2014: 8 · 2013: 11 · 2012: 5
- FWCI
- 10.78
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Annals of Surgery (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Keywords
- meta-analysis, methodological quality, general surgery, critical appraisal, reporting guidelines, surgical journals, interobserver agreement
- MeSH terms
- Publishing, Review Literature as Topic, General Surgery, Meta-Analysis as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
