Dealing with clinical heterogeneity in meta‐analysis. Assumptions, methods, interpretation
Open accessThe DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Abstract
Objective: There is an ongoing debate how to interpret findings of meta‐analyses when substantial clinical heterogeneity is present among included trials. The aim of the present study was to demonstrate various ways of dealing with clinical heterogeneity along with underlying assumptions and interpretation. A recent meta‐analysis on long‐term psychodynamic psychotherapy (LTPP) was used as an illustrative example. Method: Re‐analysis of published data including calculation of a prediction interval, heterogeneity tests, Bayesian meta‐analysis, meta‐regression, and subgroup analysis to explore and interpret summary estimates in clinically heterogeneous studies. Results: Meta‐analytic results and their implications varied considerably depending on whether and how clinical heterogeneity was addressed. Conclusions: Whether or not to trust summary estimates in meta‐analysis depends largely on the subjective relevance of clinical heterogeneity present. No single analysis and interpretation strategy can be valid in every context or paradigm, thus, reflection of own beliefs on the role of heterogeneity is needed. Copyright © 2013 John Wiley & Sons, Ltd.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The combination of estimates from different experiments
Cochran, W. G. (1954) · Biometrics · doi:10.2307/3001666 · Cited by 5,380 · cited 1×
“Whether heterogeneity is present is commonly tested with Cochran’s Q test statistic that examines the null hypothesis that all studies estimate the same effect (Cochran, 1954).”
BackgroundRealist review - a new method of systematic review designed for complex policy interventions
Pawson, R., Greenhalgh, T., Harvey, G., & Walshe, K. (2005) · Journal of Health Services Research & Policy · doi:10.1258/1355819054308530 · Cited by 3,190 · cited 2×
“…treatments are likely to have at least as much in common with complex health services interventions (Craig et al., 2008) as with pharmacological clinical trials, the so called “realist review” methodology may also present an alternative to mechanic meta-analysis (Pawson et al., 2005).”
BackgroundInterpretation of random effects meta-analyses
Riley, R. D., Higgins, J. P. T., & Deeks, J. J. (2011) · BMJ · doi:10.1136/bmj.d549 · Cited by 2,884
A Re-Evaluation of Random-Effects Meta-Analysis
Higgins, J. P. T., Thompson, S. G., & Spiegelhalter, D. J. (2009) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/j.1467-985x.2008.00552.x · Cited by 2,569 · cited 1×
“…true treatment effect, while random-effects models assume that no single true treatment effect exists, but each trial estimates a trial-specific true treatment effect, with these effects following a certain (commonly normal) distribution (Lau et al., 1998; Higgins et al., 2009; Riley et al., 2011).”
BackgroundRandom-effects model for meta-analysis of clinical trials: An update
DerSimonian, R., & Kacker, R. (2007) · Contemporary Clinical Trials · doi:10.1016/j.cct.2006.04.004 · Cited by 2,374
Lipsey, M. W., & Wilson, D. B. (1993) · American Psychologist · doi:10.1037/0003-066x.48.12.1181 · Cited by 2,033 · cited 1×
“Several authors were interested in the overall effectiveness of psychotherapeutic treatments and contributed to the development of the methodological-statistical foundations of meta-analysis (Smith et al., 1980; Lipsey and Wilson, 1993).”
BackgroundStatistics notes: Absence of evidence is not evidence of absence
Altman, D. G., & Bland, J. M. (1995) · BMJ · doi:10.1136/bmj.311.7003.485 · Cited by 1,946
Systematic Reviews in Health Care: Meta‐Analysis in Context
(2001) · doi:10.1002/9780470693926 · Cited by 1,693
Systematic Review: Why sources of heterogeneity in meta-analysis should be investigated
Thompson, S. G. (1994) · BMJ · doi:10.1136/bmj.309.6965.1351 · Cited by 1,017 · cited 5×
“…heterogeneity description should be given priority over inference, not only because an “absence of evidence is not evidence of absence” (Altman and Bland, 1995), but also because heterogeneity tests frequently lack power to detect “even a moderate degree of genuine heterogeneity” (Thompson, 1994).”
BackgroundMethodInfluentialYusuf, S. (1991) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.1991.03470010097038 · Cited by 1,014 · cited 1×
“On the other hand, one may prefer to look at certain subgroups in clinical trials (Yusuf et al., 1991).”
Background
Cited by
Doi, S. A. R., Barendregt, J. J., Khan, S., Thalib, L., & Williams, G. M. (2015) · Contemporary Clinical Trials · doi:10.1016/j.cct.2015.05.009 · Cited by 581
Antimicrobial resistance in wildlife
Vittecoq, M., Godreuil, S., Prugnolle, F., Durand, P., Brazier, L., Renaud, N., Arnal, A., Aberkane, S., Jean‐Pierre, H., Gauthier‐Clerc, M., et al. (2016) · Journal of Applied Ecology · doi:10.1111/1365-2664.12596 · Cited by 284
Long-term effects of parental divorce on mental health – A meta-analysis
Auersperg, F., Vlasak, T., Ponocny, I., & Barth, A. (2019) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2019.09.011 · Cited by 223
Linde, K., Sigterman, K., Kriston, L., Rucker, G., Jamil, S., Meissner, K., & Schneider, A. (2015) · The Annals of Family Medicine · doi:10.1370/afm.1719 · Cited by 196
Self-citationLim, S. M., Shiau, C. W. C., Cheng, L. J., & Lau, Y. (2022) · Behavior Therapy · doi:10.1016/j.beth.2021.09.007 · Cited by 163
High statistical heterogeneity is more frequent in meta-analysis of continuous than binary outcomes
Alba, A. C., Alexander, P. E., Chang, J., MacIsaac, J., DeFry, S., & Guyatt, G. H. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.09.005 · Cited by 129
“Investigators can test such hypotheses by comparing the effect estimates from different groups of studies (subgroup analyses or meta-regression) or by excluding studies based on particular characteristics (sensitivity analyses)[3,4].”
BackgroundKriston, L., von Wolff, A., Westphal, A., Hölzel, L. P., & Härter, M. (2014) · Depression and Anxiety · doi:10.1002/da.22236 · Cited by 119
Self-citationPhilipp, R., Kriston, L., Lanio, J., Kühne, F., Härter, M., Moritz, S., & Meister, R. (2019) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.2345 · Cited by 114
Self-citationSchmidt, C. S., Schulte, B., Seo, H., Kuhn, S., O'Donnell, A., Kriston, L., Verthein, U., & Reimer, J. (2016) · Addiction · doi:10.1111/add.13263 · Cited by 107
Self-citationLinde, K., Rücker, G., Sigterman, K., Jamil, S., Meissner, K., Schneider, A., & Kriston, L. (2015) · BMC Family Practice · doi:10.1186/s12875-015-0314-x · Cited by 56
“We classified treatments consistently with our previous work [11] (pages 58–59): “As psychological treatments are considered complex interventions [21], grouping them can be performed along several dimensions and remains controversial [22].”
MethodSelf-citationYang, Q., Lang, X., Li, W., & Liang, Y. (2022) · European Journal of Clinical Nutrition · doi:10.1038/s41430-021-00927-0 · Cited by 45
Li, Z., Li, Y., Guo, L., Li, M., & Yang, K. (2021) · International Journal of Clinical Practice · doi:10.1111/ijcp.13982 · Cited by 38
A meta-analysis of obstructive sleep apnea in patients with cerebrovascular disease
Wu, Z., Chen, F., Yu, F., Wang, Y., & Guo, Z. (2018) · Sleep and Breathing · doi:10.1007/s11325-017-1604-4 · Cited by 38
Köhnen, M., Kriston, L., Härter, M., Dirmaier, J., & Liebherz, S. (2019) · BMJ Open · doi:10.1136/bmjopen-2018-028042 · Cited by 10
Self-citation
Metadata
- Authors
- Kriston, L.
- Venue
- International Journal of Methods in Psychiatric Research
- Publisher
- Wiley
- Published
- 2013-03
- Volume
- 22
- Issue
- 1
- Pages
- 1-15
- ISSN
- 1049-8931 (ISSN-L)
- 1557-0657
- OpenAlex
- W1536115762
- Cited by
- 94
- References
- 21
- FWCI
- 9.32
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.