The DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.
Statistical heterogeneity in systematic reviews of clinical trials: a critical appraisal of guidelines and practice
Abstract
Objective: Heterogeneity between study results can be a problem in any systematic review or meta-analysis of clinical trials. Identifying its presence, investigating its cause and correctly accounting for it in analyses all involve difficult decisions for the researcher. Our objectives were: to collate recommendations on the subject of dealing with heterogeneity in systematic reviews of clinical trials; to investigate current practice in addressing heterogeneity in Cochrane reviews; and to compare current practice with recommendations. Methods: We review guidelines for those undertaking systematic reviews and examine how heterogeneity is addressed in practice in a sample of systematic reviews, and their protocols, from the Cochrane Database of Systematic Reviews. Results: Advice to reviewers is on the whole consistent and sensible. However, examination of a sample of Cochrane protocols and reviews demonstrates that the advice is difficult to follow given the small numbers of studies identified in many systematic reviews, the difficulty of pre-specifying important effect modifiers for subgroup analysis or meta-regression and the unresolved debate concerning fixed versus random effects metaanalyses. There was disagreement between protocols and reviews, often either regarding choice of important potential effect modifiers or due to the review identifying too few studies to perform planned analyses. Conclusion: Guidelines that address practical issues are required to reduce the risk of spurious findings from investigations of heterogeneity. This may involve discouraging statistical investigations such as subgroup analyses and meta-regression, rather than simply adopting a cautious approach to their interpretation, unless a large number of studies is available. The notion of a priori specification of potential effect modifiers for a retrospective review of studies is ill-defined, and the appropriateness of using a statistical test for heterogeneity to decide between analysis strategies is suspect.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Schulz (1995) · doi:10.1001/jama.1995.03520290060030 · Cited by 5,634
Systematic Review: Why sources of heterogeneity in meta-analysis should be investigated
Thompson, S. G. (1994) · BMJ · doi:10.1136/bmj.309.6965.1351 · Cited by 1,017 · cited 1×
“Reasons for heterogeneity should be explored in order to increase scientific understanding and clinical relevance.(3) But despite the inevitability of heterogeneity, how it should be handled in systematic reviews is far from clear.”
BackgroundHeterogeneity in Meta-analysis of Data from Epidemiologic Studies: A Commentary
Colditz, G. A., Burdick, E., & Mosteller, F. (1995) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a117644 · Cited by 245
Understanding research synthesis (meta-analysis).
Mosteller, F., & Colditz, G. A. (1996) · Annual Review of Public Health · doi:10.1146/annurev.pu.17.050196.000245 · Cited by 228
Invited commentary: benefits of heterogeneity in meta-analysis of data from epidemiologic studies
Berlin, J. A. (1995) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a117645 · Cited by 191 · cited 1×
“This is despite the strong and, some would say, untenable assumption of equal underlying treatment effects on which this method is based.(6;26)”
ResultMeta-analysis. Beyond the grand mean?
Smith, G. D., Egger, M., & Phillips, A. N. (1997) · BMJ · doi:10.1136/bmj.315.7122.1610 · Cited by 174
Cited by
Measuring inconsistency in meta-analyses
Higgins, J. P. T., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003) · BMJ · doi:10.1136/bmj.327.7414.557 · Cited by 63,139
Self-citationHow should meta‐regression analyses be undertaken and interpreted?
Thompson, S. G., & Higgins, J. P. T. (2002) · Statistics in Medicine · doi:10.1002/sim.1187 · Cited by 3,026
Self-citationInterpretation of random effects meta-analyses
Riley, R. D., Higgins, J. P. T., & Deeks, J. J. (2011) · BMJ · doi:10.1136/bmj.d549 · Cited by 2,884
Self-citationA Re-Evaluation of Random-Effects Meta-Analysis
Higgins, J. P. T., Thompson, S. G., & Spiegelhalter, D. J. (2009) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/j.1467-985x.2008.00552.x · Cited by 2,569
“Although sources of heterogeneity should ideally be investigated (Thompson, 1994), selection of covariates to explain heterogeneity can be difficult, both practically (Higgins et al., 2002) and theoretically (Thompson and Higgins, 2002).”
BackgroundSelf-citationChan, A.-W., Hróbjartsson, A., Haahr, M. T., Gøtzsche, P. C., & Altman, D. G. (2004) · JAMA · doi:10.1001/jama.291.20.2457 · Cited by 1,845
Self-citation
Metadata
- Authors
- Higgins, J., Thompson, S., Deeks, J., & Altman, D.
- Venue
- Journal of Health Services Research & Policy
- Publisher
- SAGE Publications
- Published
- 2002-01-01
- Volume
- 7
- Issue
- 1
- Pages
- 51-61
- ISSN
- 1355-8196 (ISSN-L)
- 1758-1060
- OpenAlex
- W2081320515
- Cited by
- 755
- References
- 6
- FWCI
- 22.54
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
