Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Sign Up
Cited by 1,138

The DOI matches its Crossref record; 8 of the 8 references it cites were found in a registry or an index.

Sensitivity of between-study heterogeneity in meta-analysis: proposed metrics and empirical evaluation

Open access
Patsopoulos, Nikolaos A, Evangelou, Evangelos, Ioannidis, John PA·2008·International Journal of Epidemiology
DOI 10.1093/ije/dyn065
Open this paper in CoChat — free account

Abstract

BACKGROUND: Several approaches are available for evaluating heterogeneity in meta-analysis. Sensitivity analyses are often used, but these are often implemented in various non-standardized ways. METHODS: We developed and implemented sequential and combinatorial algorithms that evaluate the change in between-study heterogeneity as one or more studies are excluded from the calculations. The algorithms exclude studies aiming to achieve either the maximum or the minimum final I(2) below a desired pre-set threshold. We applied these algorithms in databases of meta-analyses of binary outcome and >/=4 studies from Cochrane Database of Systematic Reviews (Issue 4, 2005, n = 1011) and meta-analyses of genetic associations (n = 50). Two I(2) thresholds were used (50% and 25%). RESULTS: Both algorithms have succeeded in achieving the pre-specified final I(2) thresholds. Differences in the number of excluded studies varied from 0% to 6% depending on the database and the heterogeneity threshold, while it was common to exclude different specific studies. Among meta-analyses with initial I(2) > 50%, in the large majority [19 (90.5%) and 208 (85.9%) in genetic and Cochrane meta-analyses, respectively] exclusion of one or two studies sufficed to decrease I(2) < 50%. Similarly, among meta-analyses with initial I(2) > 25%, in most cases [16 (57.1%) and 382 (81.3%), respectively) exclusion of one or two studies sufficed to decrease heterogeneity even <25%. The number of excluded studies correlated modestly with initial estimated I(2) (correlation coefficients 0.52-0.68 depending on algorithm used). CONCLUSIONS: The proposed algorithms can be routinely applied in meta-analyses as standardized sensitivity analyses for heterogeneity. Caution is needed evaluating post hoc which specific studies are responsible for the heterogeneity.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1093/ije/dyn065

Cites

8 of 8 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 8 references

Research works printed but not linked: 20

  1. Lau, 1998, Summing up evidence: one answer is not always enough, Lancet, 351, 123, 10.1016/S0140-6736(97)08468-7 · DOI · Paper page
  2. Glasziou, 2002, Investigating causes of heterogeneity in systematic reviews, Stat Med, 21, 1503, 10.1002/sim.1183 · DOI
  3. Sutton, 2000, Methods for Meta-analysis in Medical Research.
  4. Galbraith, 1988, A note on graphical presentation of estimated odds ratios from several clinical trials, Stat Med, 7, 889, 10.1002/sim.4780070807 · DOI · Paper page
  5. Baujat, 2002, A graphical method for exploring heterogeneity in meta-analyses: application to a meta-analysis of 65 trials, Stat Med, 21, 2641, 10.1002/sim.1221 · DOI · Paper page
  6. Ioannidis, 2006, Implications of small effect sizes of individual genetic variants on the design and interpretation of genetic association studies of complex diseases, Am J Epidemiol, 164, 609, 10.1093/aje/kwj259 · DOI
  7. Ioannidis, 2001, Replication validity of genetic association studies, Nat Genet, 29, 306, 10.1038/ng749 · DOI · Paper page
  8. Schierhout, 2001, Anti-epileptic drugs for preventing seizures following acute traumatic brain injury, The Cochrane Database of Systematic Reviews, 10.1002/14651858.CD000173 · DOI
  9. Faggiano, 2003, Methadone maintenance at different dosages for opioid dependence, The Cochrane Database of Systematic Reviews, 10.1002/14651858.CD002208 · DOI · Paper page
  10. Ghersi, 2005, Taxane containing regimens for metastatic breast cancer, The Cochrane Database of Systematic Reviews, 10.1002/14651858.CD003366.pub2 · DOI
  11. Gøtzsche, 2005, Somatostatin analogues for acute bleeding oesophageal varices, The Cochrane Database of Systematic Reviews, 10.1002/14651858.CD000193.pub2 · DOI
  12. Song, 2001, Methods for exploring heterogeneity in meta-analysis, Eval Health Prof, 24, 126, 10.1177/016327870102400203 · DOI · Paper page
  13. Hardy, 1998, Detecting and describing heterogeneity in meta-analysis, Stat Med, 17, 841, 10.1002/(SICI)1097-0258(19980430)17:8<841::AID-SIM781>3.0.CO;2-D · DOI · Paper page
  14. Schmid, 2004, Meta-regression detected associations between heterogeneous treatment effects and study-level, but not patient-level, factors, J Clin Epidemiol, 57, 683, 10.1016/j.jclinepi.2003.12.001 · DOI
  15. Lambert, 2002, A comparison of summary patient-level covariates in meta-regression with individual patient data meta-analysis, J Clin Epidemiol, 55, 86, 10.1016/S0895-4356(01)00414-0 · DOI · Paper page
  16. Berlin, 2002, Anti-lymphocyte antibody induction therapy study group. Individual patient- versus group-level data meta-regressions for the investigation of treatment effect modifiers: ecological bias rears its ugly head, Stat Med, 21, 371, 10.1002/sim.1023 · DOI · Paper page
  17. Petiti, 2001, Approaches to heterogeneity in meta-analysis, Stat Med, 20, 3625, 10.1002/sim.1091 · DOI · Paper page
  18. Simmonds, 2005, Meta-analysis of individual patient data from randomized trials: a review of methods used in practice, Clin Trials, 2, 209, 10.1191/1740774505cn087oa · DOI · Paper page
  19. Engels, 2000, Heterogeneity and statistical significance in meta-analysis: an empirical study of 125 meta-analyses, Stat. Med, 19, 1707, 10.1002/1097-0258(20000715)19:13<1707::AID-SIM491>3.0.CO;2-P · DOI
  20. Ioannidis, 2007, Uncertainty in heterogeneity estimates in meta-analyses, Br Med J, 335, 914, 10.1136/bmj.39343.408449.80 · DOI · Paper page

Cited by

Metadata

Authors
Patsopoulos, N. A., Evangelou, E., & Ioannidis, J. P.
Venue
International Journal of Epidemiology
Publisher
Oxford University Press (OUP)
Published
2008-04-18
Volume
37
Issue
5
Pages
1148-1157
ISSN
  • 0300-5771 (ISSN-L)
  • 1464-3685
DOI
10.1093/ije/dyn065
OpenAlex
W2112970335
Cited by
1,138
References
8
FWCI
12.74
Type
journal article
Language
en
Open access
green

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.