The Hartung-Knapp-Sidik-Jonkman method for random effects meta-analysis is straightforward and considerably outperforms the standard DerSimonian-Laird method
Open accessThe DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index.
Abstract
BACKGROUND: The DerSimonian and Laird approach (DL) is widely used for random effects meta-analysis, but this often results in inappropriate type I error rates. The method described by Hartung, Knapp, Sidik and Jonkman (HKSJ) is known to perform better when trials of similar size are combined. However evidence in realistic situations, where one trial might be much larger than the other trials, is lacking. We aimed to evaluate the relative performance of the DL and HKSJ methods when studies of different sizes are combined and to develop a simple method to convert DL results to HKSJ results. METHODS: We evaluated the performance of the HKSJ versus DL approach in simulated meta-analyses of 2-20 trials with varying sample sizes and between-study heterogeneity, and allowing trials to have various sizes, e.g. 25% of the trials being 10-times larger than the smaller trials. We also compared the number of "positive" (statistically significant at p < 0.05) findings using empirical data of recent meta-analyses with > = 3 studies of interventions from the Cochrane Database of Systematic Reviews. RESULTS: The simulations showed that the HKSJ method consistently resulted in more adequate error rates than the DL method. When the significance level was 5%, the HKSJ error rates at most doubled, whereas for DL they could be over 30%. DL, and, far less so, HKSJ had more inflated error rates when the combined studies had unequal sizes and between-study heterogeneity. The empirical data from 689 meta-analyses showed that 25.1% of the significant findings for the DL method were non-significant with the HKSJ method. DL results can be easily converted into HKSJ results. CONCLUSIONS: Our simulations showed that the HKSJ method consistently results in more adequate error rates than the DL method, especially when the number of studies is small, and can easily be applied routinely in meta-analyses. Even with the HKSJ method, extra caution is needed when there are = <5 studies of very unequal sizes.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures

Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Random-effects meta-regression with Knapp–Hartung adjustment was applied to account for between-study heterogeneity and to calculate 95% prediction intervals, estimating the likely range of effect sizes in future studies (30).”
From The association of screen time and the risk of sleep outcomes: a systematic review and meta-analysis. (2025) · cites this paper as [30]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“To evaluate robustness of pooled estimates, prespecified sensitivity analyses included fixed-effect models, Hartung–Knapp adjustment for random-effects inference, alternative τ² estimators, leave-one-out analyses, and Bayesian random-effects meta-analysis (45–49).”
In the citing paper
“An inverse-variance method with a random-effects model and Hartung-Knapp-Sidik-Jonkman adjustment (33) was used to adjust for study variability in sample sizes.”
From Digital Media Use and Adolescents' Mental Health During the Covid-19 Pandemic: A Systematic Review and Meta-Analysis (2022) · cites this paper as [33]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
2 retracted
Singh, M., & Das, R. R. (2013) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd001364.pub4 · Cited by 237
RetractedSingh, M., & Das, R. R. (2011) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd001364.pub3 · Cited by 86
RetractedMeasuring inconsistency in meta-analyses
Higgins, J. P. T., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003) · BMJ · doi:10.1136/bmj.327.7414.557 · Cited by 63,139 · cited 8×
“The levels correspond to no, low, moderate, high and very high heterogeneity, respectively [16].”
BackgroundMethodResultInfluentialMeta-analysis in clinical trials
DerSimonian, R., & Laird, N. (1986) · Controlled Clinical Trials · doi:10.1016/0197-2456(86)90046-2 · Cited by 39,552 · cited 2×
“Background The commonly used method for a random effects metaanalysis is the DerSimonian and Laird approach (DL method) [1].”
MethodQuantifying heterogeneity in a meta‐analysis
Higgins, J. P. T., & Thompson, S. G. (2002) · Statistics in Medicine · doi:10.1002/sim.1186 · Cited by 36,763 · cited 1×
“the degree of inconsistency in the studies’ results, in comparison to the total amount of variation [16,22].”
ResultConducting Meta-Analyses in R with the metafor Package
Viechtbauer, W. (2010) · Journal of Statistical Software · doi:10.18637/jss.v036.i03 · Cited by 20,464 · cited 2×
“The R package metafor [20] can also be used to perform an HKSJ analysis.”
MethodBorenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009) · doi:10.1002/9780470743386 · Cited by 14,225
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2021) · doi:10.1002/9781119558378 · Cited by 9,799
Improved tests for a random effects meta‐regression with a single covariate
Knapp, G., & Hartung, J. (2003) · Statistics in Medicine · doi:10.1002/sim.1482 · Cited by 2,075 · cited 1×
“The implementation is based on the meta-regression paper by Knapp and Hartung [34]: when no covariates or moderator variables are used, the meta-regression reduces to a random effects metaanalysis as proposed by Hartung/Knapp and Sidik/ Jonkman.”
MethodUncertainty in heterogeneity estimates in meta-analyses
Ioannidis, J. P. A., Patsopoulos, N. A., & Evangelou, E. (2007) · BMJ · doi:10.1136/bmj.39343.408449.80 · Cited by 1,254 · cited 1×
“Some detectable heterogeneity is present in about half of meta-analyses of clinical studies [15-18].”
BackgroundSelf-citation
Research works printed but not linked: 3
- The Cochrane Collaboration: Review Manager (RevMan) 5.1.4. 2011, Copenhagen: The Nordic Cochrane Centre
- Borenstein M, Hedges L, Higgins J, Rothstein H: Comprehensive Meta-analysis Version 2. 2005, Englewood NJ: Biostat
- Hartung J: An alternative method for meta-analysis. Biom J. 1999, 901-916.
Cited by
Meta-analysis in clinical trials revisited
DerSimonian, R., & Laird, N. (2015) · Contemporary Clinical Trials · doi:10.1016/j.cct.2015.09.002 · Cited by 2,677
“study are small, or highly unequal [25].”
BackgroundMethodInfluentialThe WHO Rapid Evidence Appraisal for COVID-19 Therapies (REACT) Working Group, Sterne, J. A. C., Murthy, S., Diaz, J. V., Slutsky, A. S., Villar, J., Angus, D. C., Annane, D., Azevedo, L. C. P., Berwanger, O., et al. (2020) · JAMA · doi:10.1001/jama.2020.17023 · Cited by 2,257
Plea for routinely presenting prediction intervals in meta-analysis.
IntHout, J., Ioannidis, J. P. A., Rovers, M. M., & Goeman, J. J. (2016) · BMJ Open · doi:10.1136/bmjopen-2015-010247 · Cited by 2,264
“The difference may be caused by publication bias, the higher number of primary studies in their sample (median 9 vs 4 in our set(15)) and by their use of the DerSimonian-Laird approach which can result in too many statistically significant findings, whereas we used the HKSJ approach.(16) However, results with respect…”
ResultSelf-citationStringhini, S., Carmeli, C., Jokela, M., Avendaño, M., Muennig, P., Guida, F., Ricceri, F., d'Errico, A., Barros, H., Bochud, M., et al. (2017) · The Lancet · doi:10.1016/s0140-6736(16)32380-7 · Cited by 1,319
A comparison of heterogeneity variance estimators in simulated random‐effects meta‐analyses
Langan, D., Higgins, J. P. T., Jackson, D., Bowden, J., Veroniki, A. A., Kontopantelis, E., Viechtbauer, W., & Simmonds, M. (2019) · Research Synthesis Methods · doi:10.1002/jrsm.1316 · Cited by 1,074
“These findings agree with a previous 533 simulation study (IntHout et al., 2014), in which the HKSJ method was compared with other 534 confidence interval methods for both continuous and binary outcome measures.”
Result
Metadata
- Authors
- IntHout, J., Ioannidis, J. P., & Borm, G. F.
- Affiliations
- Radboud University Medical Center (NL)
- Stanford Medicine (US)
- Corresponding author
- IntHout, Joanna
- Venue
- BMC Medical Research Methodology
- Published
- 2014-02-18
- Volume
- 14
- Issue
- 1
- Pages
- 25-25
- ISSN
- 1471-2288 (ISSN-L)
- PMC
- PMC4015721
- OpenAlex
- W2032506622
- Cited by
- 2,419
- References
- 32
- Citations by year
- 2026: 599 · 2025: 299 · 2024: 319 · 2023: 279 · 2022: 263 · 2021: 243 · 2020: 144 · 2019: 81 · 2018: 62 · 2017: 50 · 2016: 37 · 2015: 37 · 2014: 3
- FWCI
- 41.47
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- BMC Medical Research Methodology (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Radboud Repository (Radboud University) (submittedVersion)
- Radboud Repository (Radboud University) (submittedVersion, other-oa)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Keywords
- random-effects meta-analysis, DerSimonian-Laird method, type I error rate, study heterogeneity, significance testing
- MeSH terms
- Clinical Trials as Topic, Common Cold, Data Interpretation, Statistical, Humans, Zinc, Proportional Hazards Models, Odds Ratio, Sample Size, Precursor Cell Lymphoblastic Leukemia-Lymphoma
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The Hartung-Knapp-Sidik-Jonkman method for random effects meta-analysis is straightforward and considerably outperforms the standard DerSimonian-Laird method. https://app.cochat.ai/papers/10.1186/1471-2288-14-25
- The paper
- IntHout, J., Ioannidis, J. P., & Borm, G. F. (2014). The Hartung-Knapp-Sidik-Jonkman method for random effects meta-analysis is straightforward and considerably outperforms the standard DerSimonian-Laird method. BMC Medical Research Methodology. https://doi.org/10.1186/1471-2288-14-25
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.