Plea for routinely presenting prediction intervals in meta-analysis.
Open accessThe DOI matches its Crossref record; 22 of the 22 references it cites were found in a registry or an index.
Abstract
Objectives: Evaluating the variation in the strength of the effect across studies is a key feature of meta-analyses. This variability is reflected by measures like τ 2 or I 2 , but their clinical interpretation is not straightforward. A prediction interval is less complicated: it presents the expected range of true effects in similar studies. We aimed to show the advantages of having the prediction interval routinely reported in meta-analyses. Design: We show how the prediction interval can help understand the uncertainty about whether an intervention works or not. To evaluate the implications of using this interval to interpret the results, we selected the first meta-analysis per intervention review of the Cochrane Database of Systematic Reviews Issues 2009–2013 with a dichotomous (n=2009) or continuous (n=1254) outcome, and generated 95% prediction intervals for them. Results: In 72.4% of 479 statistically significant (random-effects p<0.05) meta-analyses in the Cochrane Database 2009–2013 with heterogeneity (I 2 >0), the 95% prediction interval suggested that the intervention effect could be null or even be in the opposite direction. In 20.3% of those 479 meta-analyses, the prediction interval showed that the effect could be completely opposite to the point estimate of the meta-analysis. We demonstrate also how the prediction interval can be used to calculate the probability that a new trial will show a negative effect and to improve the calculations of the power of a new trial. Conclusions: The prediction interval reflects the variation in treatment effects over different settings, including what effect is to be expected in future patients, such as the patients that a clinician is interested to treat. Prediction intervals should be routinely reported to allow more informative inferences in meta-analyses.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures

Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Given that treatment effects over different settings may vary, we also reported 95% prediction intervals (PIs) for the true effect [41].”
From Generative AI Mental Health Chatbots as Therapeutic Tools: Systematic Review and Meta-Analysis of Their Role in Reducing Mental Health Issues (2025) · cites this paper as [41]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“This review followed updated guidance of the Cochrane Handbook for Systematic Reviews of Intervention30 and Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 guidelines.31”
From Acceptance and Commitment Therapy Among Informal Caregivers of People With Chronic Health Conditions: A Systematic Review and Meta-Analysis (2023) · cites this paper as [31]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Only one study focused on adolescents exclusively [41]”
From Generative AI Mental Health Chatbots as Therapeutic Tools: Systematic Review and Meta-Analysis of Their Role in Reducing Mental Health Issues (2025) · cites this paper as [41]
Experimental — accuracy not yet measuredIn the citing paper
“Prediction intervals include the expected range of true effects in similar studies [22].”
From How are systematic reviews of prevalence conducted? A methodological study (2020) · cites this paper as [22]
In the citing paper
“As the between-studies standard deviation and lower and upper limits of the 95% PI were in the same metric as the effect estimates, they offer a reasonable quantification of the heterogeneity level [63, 67–69].”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
1 retracted
Topical steroids for nasal polyps
Kalish, L., Snidvongs, K., Sivasubramaniam, R., Cope, D., & Harvey, R. J. (2012) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd006549.pub2 · Cited by 122
RetractedMeasuring inconsistency in meta-analyses
Higgins, J. P. T., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003) · BMJ · doi:10.1136/bmj.327.7414.557 · Cited by 63,139
Cochrane Handbook for Systematic Reviews of Interventions: Cochrane Book Series
(2008) · doi:10.1002/9780470712184 · Cited by 25,048 · cited 1×
“with suggested limits for low or high inconsistency.(13) ▸ A direct clinical interpretation of I(2) is difficult.”
BackgroundConducting Meta-Analyses in R with the metafor Package
Viechtbauer, W. (2010) · Journal of Statistical Software · doi:10.18637/jss.v036.i03 · Cited by 20,464
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009) · doi:10.1002/9780470743386 · Cited by 14,225
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2021) · doi:10.1002/9781119558378 · Cited by 9,799
Cochrane Handbook for Systematic Reviews of Interventions Version 5.0.1. The Cochrane Collaboration
Higgins, J. P. T., Green, S. R., & Higgins, J. P. T. (2008) · Cited by 7,171
How should meta‐regression analyses be undertaken and interpreted?
Thompson, S. G., & Higgins, J. P. T. (2002) · Statistics in Medicine · doi:10.1002/sim.1187 · Cited by 3,026
Interpretation of random effects meta-analyses
Riley, R. D., Higgins, J. P. T., & Deeks, J. J. (2011) · BMJ · doi:10.1136/bmj.d549 · Cited by 2,884
A Re-Evaluation of Random-Effects Meta-Analysis
Higgins, J. P. T., Thompson, S. G., & Spiegelhalter, D. J. (2009) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/j.1467-985x.2008.00552.x · Cited by 2,569 · cited 2×
“effects are to be expected for 95% of similar (exchangeable) studies that might be conducted in the future.(4) Therefore, it is well suited to evaluate the variability of the effect of an intervention over different settings.”
Background
Cited by
The Measurement of Observer Agreement for Categorical Data
Landis, J. R., & Koch, G. G. (1977) · Biometrics · doi:10.2307/2529310 · Cited by 81,148
Stringhini, S., Carmeli, C., Jokela, M., Avendaño, M., Muennig, P., Guida, F., Ricceri, F., d'Errico, A., Barros, H., Bochud, M., et al. (2017) · The Lancet · doi:10.1016/s0140-6736(16)32380-7 · Cited by 1,319
Long-term outcomes of childhood sexual abuse: an umbrella review
Hailes, H. P., Yu, R., Danese, A., & Fazel, S. (2019) · The Lancet Psychiatry · doi:10.1016/s2215-0366(19)30286-x · Cited by 827
“42 We created an overall quality assessment.”
Orellano, P., Reynoso, J., Quaranta, N., Bardach, A., & Ciapponi, A. (2020) · Environment International · doi:10.1016/j.envint.2020.105876 · Cited by 792
Page, M. J., & Moher, D. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0663-8 · Cited by 769
“summarise findings when meta-analysis is not appropriate [146, 147]; report and synthesise intervention characteristics of included studies [138, 148]; use and interpret prediction intervals for randomeffects meta-analyses [149, 150];”
Metadata
- Authors
- IntHout, J., Ioannidis, J. P. A., Rovers, M. M., & Goeman, J. J.
- Affiliations
- Radboud University Medical Center (NL)
- Stanford Medicine (US)
- Corresponding author
- IntHout, Joanna
- Venue
- BMJ Open
- Publisher
- BMJ
- Published
- 2016-07
- Volume
- 6
- Issue
- 7
- Pages
- e010247
- ISSN
- 2044-6055 (ISSN-L)
- PMC
- PMC4947751
- OpenAlex
- W2464924628
- Cited by
- 2,264
- References
- 22
- Citations by year
- 2026: 422 · 2025: 372 · 2024: 369 · 2023: 300 · 2022: 227 · 2021: 234 · 2020: 167 · 2019: 97 · 2018: 47 · 2017: 22 · 2016: 6
- FWCI
- 53.50
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by-nc
- Locations
- BMJ Open (publishedVersion, cc-by-nc)
- Europe PMC (PubMed Central) (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- HighWire Press Open Archive (submittedVersion)
- Radboud Repository (Radboud University) (submittedVersion)
- Radboud Repository (Radboud University) (submittedVersion, other-oa)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Keywords
- prediction intervals, meta-analysis reporting, heterogeneity, null effect, trial power
- MeSH terms
- Humans, Probability, Publishing, Meta-Analysis as Topic, Research Report
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Plea for routinely presenting prediction intervals in meta-analysis. https://app.cochat.ai/papers/10.1136/bmjopen-2015-010247
- The paper
- IntHout, J., Ioannidis, J. P. A., Rovers, M. M., & Goeman, J. J. (2016). Plea for routinely presenting prediction intervals in meta-analysis. BMJ Open. https://doi.org/10.1136/bmjopen-2015-010247
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.