Bench-to-bedside review: Avoiding pitfalls in critical care meta-analysis – funnel plots, risk estimates, types of heterogeneity, baseline risk and the ecologic fallacy
Open accessThe DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
Abstract
Meta-analysis can be a powerful tool for demonstrating the applicability of a concept beyond the context of individual clinical trials and observational studies, including exploration of effects across different subgroups. Meta-analysis avoids Simpson's paradox, in which a consistent effect in constituent trials is reversed when results are simply pooled. Meta-analysis in critical care medicine is made more complicated, however, by the heterogeneous nature of critically ill patients and the contexts within which they are treated. Failure to properly adjust for this heterogeneity risks missing important subgroup effects in, for example, the interaction of treatment with varying levels of baseline risk. When subgroups are defined by characteristics that vary within constituent trials (such as age) rather than features constant within each trial (such as drug dose), there is the additional risk of incorrect conclusions due to the ecological fallacy. The present review explains these problems and the strategies by which they are overcome.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Publication bias in clinical research
Easterbrook, P. J., Gopalan, R., Berlin, J. A., & Matthews, D. R. (1991) · The Lancet · doi:10.1016/0140-6736(91)90201-y · Cited by 3,109 · cited 1×
“scrutiny. Positive studies have a greater chance of being published, however, which can falsely exaggerate the results [ 11 ].”
ResultMeta-analysis: Principles and procedures
Egger, M., Smith, G. D., & Phillips, A. N. (1997) · BMJ · doi:10.1136/bmj.315.7121.1533 · Cited by 2,415
The interpretation of interaction in contingency tables
Simpson, E. H. (1951) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1951.tb00088.x · Cited by 2,301 · cited 1×
“[19], where (for example) a beneficial effect in each study can”
BackgroundInvestigating and dealing with publication and other biases in meta-analysis
Sterne, J. A. C., Egger, M., & Smith, G. D. (2001) · BMJ · doi:10.1136/bmj.323.7304.101 · Cited by 2,225
Efficacy and Safety of Recombinant Activated Factor VII for Acute Intracerebral Hemorrhage
Mayer, S. A., Brun, N. C., Begtrup, K., Broderick, J., Davis, S., Diringer, M. N., Skolnick, B. E., & Steiner, T. (2008) · New England Journal of Medicine · doi:10.1056/nejmoa0707534 · Cited by 1,255 · cited 1×
“trial of activated factor VII that hypothesised a 33% relative improvement in outcome for patients with acute intracerebral haemorrhage [37].”
BackgroundMcAuley, L., Pham, B., Tugwell, P., & Moher, D. (2000) · The Lancet · doi:10.1016/s0140-6736(00)02786-0 · Cited by 812 · cited 1×
“include unpublished data [12], a process facilitated by the”
Deeks, J. J. (2002) · Statistics in Medicine · doi:10.1002/sim.1188 · Cited by 660 · cited 3×
“If attempting to combine estimates of treatment effect in different populations, however, it is better to use measures of relative risk – such as odds ratios, risk ratios or hazard ratios [17,18].”
MethodResultCHALMERS, I. (1993) · Annals of the New York Academy of Sciences · doi:10.1111/j.1749-6632.1993.tb26345.x · Cited by 561
The strengths and limitations of meta-analyses based on aggregate data
Lyman, G. H., & Kuderer, N. M. (2005) · BMC Medical Research Methodology · doi:10.1186/1471-2288-5-14 · Cited by 250
Cited by
Younossi, Z. M., Golabi, P., de Avila, L., Paik, J. M., Srishord, M., Fukui, N., Qiu, Y., Burns, L., Afendy, A., & Nader, F. (2019) · Journal of Hepatology · doi:10.1016/j.jhep.2019.06.021 · Cited by 2,235
Systematic Review: Kidney Transplantation Compared With Dialysis in Clinically Relevant Outcomes
Tonelli, M., Wiebe, N., Knoll, G., Bello, A., Browne, S., Jadhav, D., Klarenbach, S., & Gill, J. (2011) · American Journal of Transplantation · doi:10.1111/j.1600-6143.2011.03686.x · Cited by 1,729
Griesdale, D. E. G., de Souza, R. J., van Dam, R. M., Heyland, D. K., Cook, D. J., Malhotra, A., Dhaliwal, R., Henderson, W. R., Chittock, D. R., Finfer, S., et al. (2009) · Canadian Medical Association Journal · doi:10.1503/cmaj.090206 · Cited by 1,096
Modern attitudes toward older adults in the aging world: A cross-cultural meta-analysis.
North, M. S., & Fiske, S. T. (2015) · Psychological Bulletin · doi:10.1037/a0039469 · Cited by 560
Griesdale, D. E. G., Liu, D., McKinney, J., & Choi, P. T. (2012) · Canadian Journal of Anesthesia/Journal canadien d'anesthésie · doi:10.1007/s12630-011-9620-5 · Cited by 363
The efficacy of recommended treatments for veterans with PTSD: A metaregression analysis
Haagen, J. F. G., Smid, G. E., Knipscheer, J. W., & Kleber, R. J. (2015) · Clinical Psychology Review · doi:10.1016/j.cpr.2015.06.008 · Cited by 246
Gagnier, J. J., Moher, D., Boon, H., Beyene, J., & Bombardier, C. (2012) · BMC Medical Research Methodology · doi:10.1186/1471-2288-12-111 · Cited by 222
Metadata
- Authors
- Reade, M. C., Delaney, A., Bailey, M. J., & Angus, D. C.
- Venue
- Critical Care
- Published
- 2008
- Volume
- 12
- Issue
- 4
- Pages
- 220
- ISSN
- 1364-8535 (ISSN-L)
- OpenAlex
- W1969662158
- Cited by
- 72
- References
- 9
- FWCI
- 3.00
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.