The DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Avoiding common mistakes in meta‐analysis: Understanding the distinct roles of Q, I‐squared, tau‐squared, and the prediction interval in reporting heterogeneity
Abstract
In any meta‐analysis, it is critically important to report the dispersion in effects as well as the mean effect. If an intervention has a moderate clinical impact on average we also need to know if the impact is moderate for all relevant populations, or if it varies from trivial in some to major in others. Or indeed, if the intervention is beneficial in some cases but harmful in others. Researchers typically report a series of statistics such as the Q‐value, the p‐value, and I2, which are intended to address this issue. Often, they use these statistics to classify the heterogeneity as being low, moderate, or high and then use these classifications when considering the potential utility of the intervention. While this practice is ubiquitous, it is nevertheless incorrect. The statistics mentioned above do not actually tell us how much the effect size varies. Classifications of heterogeneity based on these statistics are uninformative at best, and often misleading. My goal in this paper is to explain what these statistics do tell us, and that none of them tells us how much the effect size varies. Then I will introduce the prediction interval, the statistic that does tell us how much the effect size varies, and that addresses the question we have in mind when we ask about heterogeneity. This paper is adapted from a chapter in “Common Mistakes in Meta‐Analysis and How to Avoid Them.” A free PDF of the book is available at https://www.Meta-Analysis.com/rsm.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Re-Evaluation of Random-Effects Meta-Analysis
Higgins, J. P. T., Thompson, S. G., & Spiegelhalter, D. J. (2009) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/j.1467-985x.2008.00552.x · Cited by 2,569 · cited 1×
“4 While the idea of a prediction interval applies in primary studies, we do not typically call it by that name.”
BackgroundPlea for routinely presenting prediction intervals in meta-analysis.
IntHout, J., Ioannidis, J. P. A., Rovers, M. M., & Goeman, J. J. (2016) · BMJ Open · doi:10.1136/bmjopen-2015-010247 · Cited by 2,264
Basics of meta‐analysis: I 2 is not an absolute measure of heterogeneity
Borenstein, M., Higgins, J. P. T., Hedges, L. V., & Rothstein, H. R. (2017) · Research Synthesis Methods · doi:10.1002/jrsm.1230 · Cited by 1,848
Self-citationCastells, X., Ramos-Quiroga, J. A., Rigau, D., Bosch, R., Nogueira, M., Vidal, X., & Casas, M. (2011) · CNS Drugs · doi:10.2165/11539440-000000000-00000 · Cited by 197
Siemens, W., Meerpohl, J. J., Rohe, M. S., Buroh, S., Schwarzer, G., & Becker, G. (2022) · Research Synthesis Methods · doi:10.1002/jrsm.1543 · Cited by 24
Cited by
Borenstein, M. (2023) · Integrative Medicine Research · doi:10.1016/j.imr.2023.101014 · Cited by 262
Self-citation
Metadata
- Authors
- Borenstein, M.
- Venue
- Research Synthesis Methods
- Publisher
- Wiley
- Published
- 2023-11-08
- Volume
- 15
- Issue
- 2
- Pages
- 354-368
- ISSN
- 1759-2879 (ISSN-L)
- 1759-2887
- OpenAlex
- W4388510553
- Cited by
- 183
- References
- 5
- FWCI
- 33.78
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.