CoChat

A systematic evaluation of the quality of meta-analyses in the critical care literature

Open access
Delaney, Anthony, Bagshaw, Sean M, Ferland, Andre, Manns, Braden, Laupland, Kevin B, Doig, Christopher J · 2005 · Critical Care
Cited by 142

The DOI matches its Crossref record; 7 of the 7 references it cites were found in a registry or an index.

Abstract

Introduction: Meta-analyses have been suggested to be the highest form of evidence available to clinicians to guide clinical practice in critical care. The purpose of this study was to systematically evaluate the quality of meta-analyses that address topics pertinent to critical care. Methods: To identify potentially eligible meta-analyses for inclusion, a systematic search of Medline, EMBASE and the Cochrane Database of Systematic Reviews was undertaken, using broad search terms relevant to intensive care, including: intensive care, critical care, shock, resuscitation, inotropes and mechanical ventilation. Predetermined inclusion criteria were applied to each identified meta-analysis independently by two authors. To assess report quality, the included meta-analyses were assessed using the component and overall scores from the Overview Quality Assessment Questionnaire (OQAQ). The quality of reports published before and after the publication of the QUOROM statement was compared. Results: A total of 139 reports of meta-analyses were included (kappa = 0.93). The overall quality of reports of meta-analyses was found to be poor, with an estimated mean overall OQAQ score of 3.3 (95% CI; 3.0–3.6). Only 43 (30.9%) were scored as having minimal or minor flaws (>5). We noted problems with the reporting of key characteristics of meta-analyses, such as performing a thorough literature search, avoidance of bias in the inclusion of studies and appropriately referring to the validity of the included studies. After the release of the QUOROM statement, however, an improvement in the overall quality of published meta-analyses was noted. Conclusion: The overall quality of the reports of meta-analyses available to critical care physicians is poor. Physicians should critically evaluate these studies prior to considering applying the results of these studies in their clinical practice.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1186/cc3803

Cites

7 of 7 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 19

  1. Cook DJ, Meade MO, Fink MP: How to keep up with the critical care literature and avoid being buried alive. Crit Care Med 1996, 24: 1757-1768. 10.1097/00003246-199610000-00026 · DOI
  2. Guyatt GH, Haynes RB, Jaeschke RZ, Cook DJ, Green L, Naylor CD, Wilson MC, Richardson WS: Users' Guides to the Medical Literature: XXV. Evidence-based medicine: principles for applying the Users' Guides to patient care. Evidence-Based Medicine Working Group. JAMA 2000, 284: 1290-1296. 10.1001/jama.284.10.1290 · DOI · Paper page
  3. McIntyre LA, Fergusson DA, Hebert PC, Moher D, Hutchison JS: Prolonged therapeutic hypothermia after traumatic brain injury in adults: a systematic review. JAMA 2003, 289: 2992-2999. 10.1001/jama.289.22.2992 · DOI · Paper page
  4. Jadad AR, Cook DJ, Browman GP: A guide to interpreting discordant systematic reviews. CMAJ 1997, 156: 1411-1416.
  5. Henderson WR, Dhingra VK, Chittock DR, Fenwick JC, Ronco JJ: Hypothermia in the management of traumatic brain injury. A systematic review and meta-analysis. Intensive Care Med 2003, 29: 1637-1644. 10.1007/s00134-003-1848-2 · DOI · Paper page
  6. Finfer S, Bellomo R, Boyce N, French J, Myburgh J, Norton R: A comparison of albumin and saline for fluid resuscitation in the intensive care unit. N Engl J Med 2004, 350: 2247-2256. 10.1056/NEJMoa040232 · DOI · Paper page
  7. Cook DJ, Reeve BK, Guyatt GH, Heyland DK, Griffith LE, Buckingham L, Tryba M: Stress ulcer prophylaxis in critically ill patients. Resolving discordant meta-analyses. JAMA 1996, 275: 308-314. 10.1001/jama.275.4.308 · DOI · Paper page
  8. Cook D, Guyatt G, Marshall J, Leasa D, Fuller H, Hall R, Peters S, Rutledge F, Griffith L, McLellan A, et al.: A comparison of sucralfate and ranitidine for the prevention of upper gastrointestinal bleeding in patients requiring mechanical ventilation. Canadian Critical Care Trials Group. N Engl J Med 1998, 338: 791-797. 10.1056/NEJM199803193381203 · DOI · Paper page
  9. Cochrane Injuries Group Albumin Reviewers: Human albumin administration in critically ill patients: systematic review of randomised controlled trials. BMJ 1998, 317: 235-240. · DOI · Paper page
  10. LeLorier J, Gregoire G, Benhaddad A, Lapierre J, Derderian F: Discrepancies between meta-analyses and subsequent large randomized, controlled trials. N Engl J Med 1997, 337: 536-542. 10.1056/NEJM199708213370806 · DOI · Paper page
  11. Flather MD, Farkouh ME, Pogue JM, Yusuf S: Strengths and limitations of meta-analysis: larger studies may be more reliable. Control Clin Trials 1997, 18: 568-579. 10.1016/S0197-2456(97)00024-X · DOI · Paper page
  12. Celermajer DS: Evidence-based medicine: how good is the evidence? Med J Aust 2001, 174: 293-295. · doi:10.5694/j.1326-5377.2001.tb143274.x
  13. Oxman AD, Guyatt GH, Singer J, Goldsmith CH, Hutchison BG, Milner RA, Streiner DL: Agreement among reviewers of review articles. J Clin Epidemiol 1991, 44: 91-98. 10.1016/0895-4356(91)90205-N · DOI · Paper page
  14. Oxman AD, Guyatt GH: Validation of an index of the quality of review articles. J Clin Epidemiol 1991, 44: 1271-1278. 10.1016/0895-4356(91)90160-B · DOI · Paper page
  15. Kelly KD, Travers A, Dorgan M, Slater L, Rowe BH: Evaluating the quality of systematic reviews in the emergency medicine literature. Ann Emerg Med 2001, 38: 518-526. 10.1067/mem.2001.115881 · DOI
  16. Choi PT, Halpern SH, Malik N, Jadad AR, Tramer MR, Walder B: Examining the evidence in anesthesia literature: a critical appraisal of systematic reviews. Anesth Analg 2001, 92: 700-709. 10.1097/00000539-200103000-00029 · DOI
  17. Shojania KG, Bero LA: Taking advantage of the explosion of systematic reviews: an efficient MEDLINE search strategy. Eff Clin Pract 2001, 4: 157-162.
  18. Hunt DL, McKibbon KA: Locating and appraising systematic reviews. Ann Intern Med 1997, 126: 532-538. · DOI · Paper page
  19. Villar J, Carroli G, Belizan JM: Predictive ability of meta-analyses of randomised controlled trials. Lancet 1995, 345: 772-776. 10.1016/S0140-6736(95)90646-0 · DOI · Paper page

Cited by

See all 142

Metadata

Authors
Delaney, A., Bagshaw, S. M., Ferland, A., Manns, B., Laupland, K. B., & Doig, C. J.
Venue
Critical Care
Published
2005-09-09
Volume
9
Issue
5
Pages
R575-82
ISSN
1364-8535 (ISSN-L)
OpenAlex
W1774591799
Cited by
142
References
7
FWCI
8.11
Type
journal article
Language
en
Open access
gold
Licence

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.