Methods to calculate uncertainty in the estimated overall effect size from a random‐effects meta‐analysis
Open accessAbstract
Meta‐analyses are an important tool within systematic reviews to estimate the overall effect size and its confidence interval for an outcome of interest. If heterogeneity between the results of the relevant studies is anticipated, then a random‐effects model is often preferred for analysis. In this model, a prediction interval for the true effect in a new study also provides additional useful information. However, the DerSimonian and Laird method—frequently used as the default method for meta‐analyses with random effects—has been long challenged due to its unfavorable statistical properties. Several alternative methods have been proposed that may have better statistical properties in specific scenarios. In this paper, we aim to provide a comprehensive overview of available methods for calculating point estimates, confidence intervals, and prediction intervals for the overall effect size under the random‐effects model. We indicate whether some methods are preferable than others by considering the results of comparative simulation and real‐life data studies.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
For random-effects meta-analyses, there are different ways to estimate the between-study variance (such as DerSimonian and Laird, restricted maximum likelihood) and calculate the confidence interval for the summary effect (such as Wald-type confidence interval, Hartung-Knapp-Sidik-Jonkman).
“For random-effects meta-analyses, there are also different ways to estimate the between-study variance (such as DerSimonian and Laird, restricted maximum likelihood (REML)) and calculate the confidence interval for the summary effect (such as Wald-type confidence interval, Hartung-Knapp-Sidik-Jonkman108).”
From PRISMA 2020 explanation and elaboration: updated guidance and exemplars for reporting systematic reviews (2021) · cites this paper as [108]
Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Meta-analysis in clinical trials
DerSimonian, R., & Laird, N. (1986) · Controlled Clinical Trials · doi:10.1016/0197-2456(86)90046-2 · Cited by 39,552 · see citing works
An Introduction to the Bootstrap
Efron, B., & Tibshirani, R. J. (1993) · doi:10.1007/978-1-4899-4541-9 · Cited by 21,230 · see citing works
Conducting Meta-Analyses in <i>R</i> with the <b>metafor</b> Package
Viechtbauer, W. (2010) · Journal of Statistical Software · doi:10.18637/jss.v036.i03 · Cited by 20,464 · see citing works
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009) · doi:10.1002/9780470743386 · Cited by 14,225 · see citing works
Self-citationLittell, R. C. (1996) · Medical Entomology and Zoology · Cited by 10,345 · see citing works
Grading quality of evidence and strength of recommendations
(2004) · BMJ · doi:10.1136/bmj.328.7454.1490 · Cited by 8,477 · see citing works
Bootstrap Methods and their Application
Davison, A. C., & Hinkley, D. V. (1997) · doi:10.1017/cbo9780511802843 · Cited by 7,350 · see citing works
A basic introduction to fixed-effect and random-effects models for meta-analysis
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2010) · Research Synthesis Methods · doi:10.1002/jrsm.12 · Cited by 6,793 · see citing works
Self-citationThe combination of estimates from different experiments
Cochran, W. G. (1954) · Biometrics · doi:10.2307/3001666 · Cited by 5,380 · see citing works
WinBUGS - A Bayesian modelling framework: Concepts, structure, and extensibility
Lunn, D. J., Thomas, A., Best, N., & Spiegelhalter, D. J. (2000) · Statistics and Computing · doi:10.1023/a:1008929526011 · Cited by 5,372 · see citing works
Cochrane Handbook for Systematic Reviews of Interventions
Tarsilla, M. (2010) · Journal of MultiDisciplinary Evaluation · doi:10.56645/jmde.v6i14.284 · Cited by 5,228 · see citing works
Small sample inference for fixed effects from restricted maximum likelihood
Kenward, M. G., & Roger, J. H. (1997) · Biometrics · doi:10.2307/2533558 · Cited by 4,552 · see citing works
Interpretation of random effects meta-analyses
Riley, R. D., Higgins, J. P. T., & Deeks, J. J. (2011) · BMJ · doi:10.1136/bmj.d549 · Cited by 2,884 · see citing works
Self-citationThe Jackknife: The Bootstrap and Other Resampling Plans.
Low, L. Y., & Efron, B. (1983) · Journal of the American Statistical Association · doi:10.2307/2288218 · Cited by 2,728 · see citing works
A Re-Evaluation of Random-Effects Meta-Analysis
Higgins, J. P. T., Thompson, S. G., & Spiegelhalter, D. J. (2009) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/j.1467-985x.2008.00552.x · Cited by 2,569 · cited 1× · see citing works
“A prediction interval provides a predicted range for the true effect size in a new study, and its calculation is recommended to be conducted under the random‐effects model.(24,153) Assuming the random effects are normally distributed, an ad hoc 95% prediction interval can be obtained by”
BackgroundSelf-citationProperties of sufficiency and statistical tests
Bartlett, M. S. (1937) · Proceedings of the Royal Society of London A Mathematical and Physical Sciences · doi:10.1098/rspa.1937.0109 · Cited by 2,440 · see citing works
IntHout, J., Ioannidis, J. P., & Borm, G. F. (2014) · BMC Medical Research Methodology · doi:10.1186/1471-2288-14-25 · Cited by 2,419 · see citing works
Plea for routinely presenting prediction intervals in meta-analysis.
IntHout, J., Ioannidis, J. P. A., Rovers, M. M., & Goeman, J. J. (2016) · BMJ Open · doi:10.1136/bmjopen-2015-010247 · Cited by 2,264 · see citing works
Improved tests for a random effects meta‐regression with a single covariate
Knapp, G., & Hartung, J. (2003) · Statistics in Medicine · doi:10.1002/sim.1482 · Cited by 2,075 · see citing works
Self-citationMethods to estimate the between‐study variance and its uncertainty in meta‐analysis
Veroniki, A. A., Jackson, D., Viechtbauer, W., Bender, R., Bowden, J., Knapp, G., Kuss, O., Higgins, J. P., Langan, D., & Salanti, G. (2016) · Research Synthesis Methods · doi:10.1002/jrsm.1164 · Cited by 1,685 · see citing works
Self-citationR2WinBUGS : A Package for Running WinBUGS from R
Sturtz, S., Ligges, U., & Gelman, A. (2005) · Journal of Statistical Software · doi:10.18637/jss.v012.i03 · Cited by 1,566 · see citing works
Controlling the risk of spurious findings from meta-regression
Higgins, J. P. T., & Thompson, S. G. (2004) · Statistics in Medicine · doi:10.1002/sim.1752 · Cited by 1,267 · cited 2× · see citing works
“The HKSJ approach (method 4) is often preferred, as in the case of small k, it is conservative and on average produces wider CIs with more adequate type I error compared with the WTz method.(16,119,141,152,159) The HKSJ method provides exact inference when all study sizes are equal and the random‐effects model is true…”
MethodSelf-citationClosing the Gap between Methodologists and End-Users: <i>R</i> as a Computational Back-End
Wallace, B. C., Dahabreh, I. J., Trikalinos, T. A., Lau, J., Trow, P., & Schmid, C. H. (2012) · Journal of Statistical Software · doi:10.18637/jss.v049.i05 · Cited by 1,125 · see citing works
A comparison of heterogeneity variance estimators in simulated random‐effects meta‐analyses
Langan, D., Higgins, J. P. T., Jackson, D., Bowden, J., Veroniki, A. A., Kontopantelis, E., Viechtbauer, W., & Simmonds, M. (2019) · Research Synthesis Methods · doi:10.1002/jrsm.1316 · Cited by 1,074 · see citing works
Self-citationMetan: fixed- and random-effects meta-analysis
Harris, R. J., Deeks, J. J., Altman, D. G., Bradburn, M. J., Harbord, R. M., & Sterne, J. A. C. (2008) · The Stata Journal: Promoting communications on statistics and Stata · doi:10.1177/1536867x0800800102 · Cited by 1,027 · see citing works
Harbord, R. M., & Higgins, J. P. T. (2008) · The Stata Journal: Promoting communications on statistics and Stata · doi:10.1177/1536867x0800800403 · Cited by 897 · see citing works
Self-citationTurner, R. M., Davey, J., Clarke, M. J., Thompson, S. G., & Higgins, J. P. (2012) · International Journal of Epidemiology · doi:10.1093/ije/dys041 · Cited by 837 · see citing works
Self-citationHunter, J. E., & Schmidt, F. L. (2000) · International Journal of Selection and Assessment · doi:10.1111/1468-2389.00156 · Cited by 833 · see citing works
A refined method for the meta-analysis of controlled clinical trials with binary outcome
Hartung, J., & Knapp, G. (2001) · Statistics in Medicine · doi:10.1002/sim.1009 · Cited by 775 · see citing works
Self-citationThomas, A., O'Hara, R., Ligges, U., & Sturtz, S. (2006) · Cited by 713 · see citing works
Schmidt, F. L., Oh, I., & Hayes, T. L. (2008) · British Journal of Mathematical and Statistical Psychology · doi:10.1348/000711007x255327 · Cited by 713 · see citing works
Doi, S. A. R., Barendregt, J. J., Khan, S., Thalib, L., & Williams, G. M. (2015) · Contemporary Clinical Trials · doi:10.1016/j.cct.2015.05.009 · Cited by 585 · see citing works
Consensus Values and Weighting Factors
Paule, R. C., & Mandel, J. (1982) · Journal of Research of the National Bureau of Standards · doi:10.6028/jres.087.022 · Cited by 580 · see citing works
A comparison of statistical methods for meta‐analysis
Brockwell, S. E., & Gordon, I. R. (2001) · Statistics in Medicine · doi:10.1002/sim.650 · Cited by 573 · see citing works
Bayesian approaches to random-effects meta-analysis: a comparative study
Smith, T. C., Spiegelhalter, D. J., & Thomas, A. (1995) · Statistics in Medicine · doi:10.1002/sim.4780142408 · Cited by 568 · see citing works
Some Statistical Methods for Combining Experimental Results
Laird, N. M., & Mosteller, F. C. (1990) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462300008916 · Cited by 568 · see citing works
A Nondegenerate Penalized Likelihood Estimator for Variance Parameters in Multilevel Models
Chung, Y., Rabe‐Hesketh, S., Dorie, V., Gelman, A., & Liu, J. (2013) · Psychometrika · doi:10.1007/s11336-013-9328-2 · Cited by 566 · see citing works
Bowden, J., Tierney, J. F., Copas, A. J., & Burdett, S. (2011) · BMC Medical Research Methodology · doi:10.1186/1471-2288-11-41 · Cited by 551 · see citing works
On tests of the overall treatment effect in meta-analysis with normally distributed responses
Hartung, J., & Knapp, G. (2001) · Statistics in Medicine · doi:10.1002/sim.791 · Cited by 543 · see citing works
Self-citationRöver, C., Knapp, G., & Friede, T. (2015) · BMC Medical Research Methodology · doi:10.1186/s12874-015-0091-1 · Cited by 527 · see citing works
Self-citationA LIKELIHOOD APPROACH TO META-ANALYSIS WITH RANDOM EFFECTS
Hardy, R., & Thompson, S. G. (1996) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19960330)15:6 · Cited by 517 · see citing works
Bax, L., Yu, L.-M., Ikeda, N., Tsuruta, H., & Moons, K. G. (2006) · BMC Medical Research Methodology · doi:10.1186/1471-2288-6-50 · Cited by 514 · see citing works
Lambert, P. C., Sutton, A. J., Burton, P. R., Abrams, K. R., & Jones, D. R. (2005) · Statistics in Medicine · doi:10.1002/sim.2112 · Cited by 511 · see citing works
A Re-Analysis of the Cochrane Library Data: The Dangers of Unobserved Heterogeneity in Meta-Analyses
Kontopantelis, E., Springate, D. A., & Reeves, D. (2013) · PLoS ONE · doi:10.1371/journal.pone.0069930 · Cited by 508 · see citing works
Random-Effects Meta-analysis of Inconsistent Effects: A Time for Change
Cornell, J. E., Mulrow, C. D., Localio, R., Stack, C. B., Meibohm, A. R., Guallar, E., & Goodman, S. N. (2014) · Annals of Internal Medicine · doi:10.7326/m13-2886 · Cited by 453 · see citing works
A simple confidence interval for meta-analysis
Sidik, K., & Jonkman, J. N. (2002) · Statistics in Medicine · doi:10.1002/sim.1262 · Cited by 431 · cited 10× · see citing works
“Also, the FP method provided coverage close to the nominal level, regardless the included number of studies, at the expense of highly lengthy CIs.”
BackgroundMethodResultInfluentialTurner, R. M., Jackson, D., Wei, Y., Thompson, S. G., & Higgins, J. P. T. (2015) · Statistics in Medicine · doi:10.1002/sim.6381 · Cited by 410 · see citing works
Self-citationDavey, J., Turner, R. M., Clarke, M. J., & Higgins, J. P. (2011) · BMC Medical Research Methodology · doi:10.1186/1471-2288-11-160 · Cited by 404 · see citing works
Self-citationConfidence intervals for the amount of heterogeneity in meta‐analysis
Viechtbauer, W. (2006) · Statistics in Medicine · doi:10.1002/sim.2514 · Cited by 399 · see citing works
Rhodes, K. M., Turner, R., & Higgins, J. P. T. (2014) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2014.08.012 · Cited by 398 · see citing works
Self-citationA comparison of heterogeneity variance estimators in combining results of studies
Sidik, K., & Jonkman, J. N. (2007) · Statistics in Medicine · doi:10.1002/sim.2688 · Cited by 380 · see citing works
Random-Effects Meta-Analyses Are Not Always Conservative
Poole, C., & Greenland, S. (1999) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a010035 · Cited by 338 · see citing works
Multivariate Random-effects Meta-analysis
White, I. R. (2009) · The Stata Journal Promoting communications on statistics and Stata · doi:10.1177/1536867x0900900103 · Cited by 289 · see citing works
Kontopantelis, E., & Reeves, D. (2010) · Statistical Methods in Medical Research · doi:10.1177/0962280210392008 · Cited by 264 · see citing works
Simple Heterogeneity Variance Estimation for Meta-Analysis
Sidik, K., & Jonkman, J. N. (2005) · Journal of the Royal Statistical Society Series C (Applied Statistics) · doi:10.1111/j.1467-9876.2005.00489.x · Cited by 254 · see citing works
Jackson, D., Law, M. W.-M., Rücker, G., & Schwarzer, G. (2017) · Statistics in Medicine · doi:10.1002/sim.7411 · Cited by 252 · see citing works
Self-citationMethods for evidence synthesis in the case of very few studies
Bender, R., Friede, T., Koch, A., Kuss, O., Schlattmann, P., Schwarzer, G., & Skipka, G. (2018) · Research Synthesis Methods · doi:10.1002/jrsm.1297 · Cited by 250 · see citing works
Self-citationNeither fixed nor random: weighted least squares meta‐analysis
Stanley, T. D., & Doucouliagos, H. (2015) · Statistics in Medicine · doi:10.1002/sim.6481 · Cited by 249 · see citing works
Systematic reviews and meta-analyses of randomized trials: principles and pitfalls
da Costa, B. R., & Jüni, P. (2014) · European Heart Journal · doi:10.1093/eurheartj/ehu424 · Cited by 247 · see citing works
Confidence intervals for the overall effect size in random-effects meta-analysis.
Sánchez‐Meca, J., & Marín‐Martínez, F. (2008) · Psychological Methods · doi:10.1037/1082-989x.13.1.31 · Cited by 242 · see citing works
A comparison of procedures to test for moderators in mixed-effects meta-regression models
Viechtbauer, W., López-López, J. A., Sánchez-Meca, J., & Marín-Martínez, F. (2015) · Psychological Methods · doi:10.1037/met0000023 · Cited by 241 · see citing works
Partlett, C., & Riley, R. D. (2017) · Statistics in Medicine · doi:10.1002/sim.7140 · Cited by 240 · see citing works
Jackson, D., Bowden, J., & Baker, R. (2010) · Journal of Statistical Planning and Inference · doi:10.1016/j.jspi.2009.09.017 · Cited by 226 · see citing works
Self-citationWeighting by Inverse Variance or by Sample Size in Random-Effects Meta-Analysis
Marín‐Martínez, F., & Sánchez‐Meca, J. (2009) · Educational and Psychological Measurement · doi:10.1177/0013164409344534 · Cited by 214 · see citing works
Random-effects meta-analysis: The number of studies matters
Guolo, A., & Varin, C. (2017) · Statistical Methods in Medical Research · doi:10.1177/0962280215583568 · Cited by 203 · see citing works
Meta‐analysis of few small studies in orphan diseases
Friede, T., Röver, C., Wandel, S., & Neuenschwander, B. E. (2016) · Research Synthesis Methods · doi:10.1002/jrsm.1217 · Cited by 183 · see citing works
Confidence intervals for random effects meta-analysis and robustness to publication bias
Henmi, M., & Copas, J. B. (2010) · Statistics in Medicine · doi:10.1002/sim.4029 · Cited by 163 · see citing works
Metaan: Random-effects Meta-analysis
Kontopantelis, E., & Reeves, D. (2010) · The Stata Journal Promoting communications on statistics and Stata · doi:10.1177/1536867x1001000307 · Cited by 149 · see citing works
Langan, D., Higgins, J. P. T., & Simmonds, M. (2017) · Research Synthesis Methods · doi:10.1002/jrsm.1198 · Cited by 149 · see citing works
Self-citationFlexible parametric models for random‐effects distributions
Lee, K. J., & Thompson, S. G. (2007) · Statistics in Medicine · doi:10.1002/sim.2897 · Cited by 148 · see citing works
Combining information from independent sources through confidence distributions
Singh, K., Xie, M., & Strawderman, W. E. (2005) · The Annals of Statistics · doi:10.1214/009053604000001084 · Cited by 142 · see citing works
Robust variance estimation for random effects meta-analysis
Sidik, K., & Jonkman, J. N. (2006) · Computational Statistics & Data Analysis · doi:10.1016/j.csda.2005.07.019 · Cited by 138 · see citing works
The exact distribution of Cochran's heterogeneity statistic in one‐way random effects meta‐analysis
Biggerstaff, B. J., & Jackson, D. (2008) · Statistics in Medicine · doi:10.1002/sim.3428 · Cited by 130 · see citing works
Self-citationValid Inference in Random Effects Meta‐Analysis
Follmann, D. A., & Proschan, M. A. (1999) · Biometrics · doi:10.1111/j.0006-341x.1999.00732.x · Cited by 127 · see citing works
Confidence Distributions and a Unifying Framework for Meta-Analysis
Xie, M., Singh, K., & Strawderman, W. E. (2011) · Journal of the American Statistical Association · doi:10.1198/jasa.2011.tm09803 · Cited by 123 · see citing works
Some general points in estimating heterogeneity variance with the DerSimonian-Laird estimator
Böhning, D. (2002) · Biostatistics · doi:10.1093/biostatistics/3.4.445 · Cited by 122 · see citing works
A general framework for the use of logistic regression models in meta-analysis
Simmonds, M. C., & Higgins, J. P. T. (2014) · Statistical Methods in Medical Research · doi:10.1177/0962280214534409 · Cited by 107 · see citing works
Self-citationAn Explicit Large-Deviation Approximation to One-Parameter Tests
Skovgaard, I. M. (1996) · Bernoulli · doi:10.3150/bj/1193839221 · Cited by 107 · see citing works
Graham, P. L., & Moran, J. L. (2012) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.09.012 · Cited by 105 · see citing works
Asymptotics and the theory of inference
Reid, N. M. (2003) · The Annals of Statistics · doi:10.1214/aos/1074290325 · Cited by 103 · see citing works
Testing overall and moderator effects in random effects meta‐regression
Huizenga, H. M., Visser, I., & Dolan, C. V. (2010) · British Journal of Mathematical and Statistical Psychology · doi:10.1348/000711010x522687 · Cited by 101 · see citing works
Introduction to systematic reviews and meta‐analysis
McKenzie, J. E., Beller, E. M., & Forbes, A. B. (2016) · Respirology · doi:10.1111/resp.12783 · Cited by 101 · see citing works
Πετροπούλου, Μ., & Mavridis, D. (2017) · Statistics in Medicine · doi:10.1002/sim.7431 · Cited by 94 · see citing works
A Review of Meta-Analysis Packages in R
Polanin, J. R., Hennessy, E. A., & Tanner-Smith, E. E. (2017) · Journal of Educational and Behavioral Statistics · doi:10.3102/1076998616674315 · Cited by 92 · cited 1× · see citing works
“In Table 1, we summarize the methods available in several software options.(112)”
MethodMalzahn, U. (2000) · Biometrika · doi:10.1093/biomet/87.3.619 · Cited by 84 · see citing works
No solution yet for combining two independent studies in the presence of heterogeneity
Gonnermann, A., Framke, T., Großhennig, A., & Koch, A. (2015) · Statistics in Medicine · doi:10.1002/sim.6473 · Cited by 83 · see citing works
Jackson, D. (2013) · Research Synthesis Methods · doi:10.1002/jrsm.1081 · Cited by 83 · see citing works
Self-citationThorlund, K., Wetterslev, J., Awad, T., Thabane, L., & Gluud, C. N. (2011) · Research Synthesis Methods · doi:10.1002/jrsm.53 · Cited by 83 · see citing works
Weighting by Inverse Variance or by Sample Size in Meta-Analysis: A Simulation Study
Sánchez‐Meca, J., & Marín‐Martínez, F. (1998) · Educational and Psychological Measurement · doi:10.1177/0013164498058002005 · Cited by 82 · see citing works
Confidence intervals for a random‐effects meta‐analysis based on Bartlett‐type corrections
Noma, H. (2011) · Statistics in Medicine · doi:10.1002/sim.4350 · Cited by 82 · see citing works
Hartung–Knapp method is not always conservative compared with fixed‐effect meta‐analysis
Wiksten, A. S., Rücker, G., & Schwarzer, G. (2016) · Statistics in Medicine · doi:10.1002/sim.6879 · Cited by 80 · see citing works
Mittmann, N., Evans, W. K., Rocchi, A., Longo, C. J., Au, H., Husereau, D., Isogai, P. K., Krahn, M. D., Coyle, D., & Working Group on Standardized Pharmacoeconomic Guidelines in Oncology (2009) · Journal of Clinical Oncology · doi:10.1200/jco.2009.27.15_suppl.e17572 · Cited by 80 · see citing works
Veroniki, A. A., Mavridis, D., Higgins, J. P. T., & Salanti, G. (2014) · BMC Medical Research Methodology · doi:10.1186/1471-2288-14-106 · Cited by 79 · see citing works
Self-citationLee, W. (1997) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19970415)16:7 · Cited by 74 · see citing works
Bayesian estimation in random effects meta‐analysis using a non‐informative prior
Bodnar, O., Link, A., Arendacká, B., Possolo, A., & Elster, C. (2017) · Statistics in Medicine · doi:10.1002/sim.7156 · Cited by 73 · see citing works
Kulinskaya, E., Döllinger, M., & Bjørkestøl, K. (2011) · Research Synthesis Methods · doi:10.1002/jrsm.54 · Cited by 69 · see citing works
Restricted maximum likelihood estimation of a common mean and the Mandel–Paule algorithm
Rukhin, A. L., Biggerstaff, B. J., & Vangel, M. G. (2000) · Journal of Statistical Planning and Inference · doi:10.1016/s0378-3758(99)00098-1 · Cited by 65 · see citing works
gemtc: Network Meta-Analysis Using Bayesian Methods
van Valkenhoef, G., & Kuiper, J. (2012) · doi:10.32614/cran.package.gemtc · Cited by 64 · see citing works
Testing for Homogeneity in Meta-Analysis I. The One-Parameter Case: Standardized Mean Difference
Kulinskaya, E., Döllinger, M., & Bjørkestøl, K. (2010) · Biometrics · doi:10.1111/j.1541-0420.2010.01442.x · Cited by 63 · see citing works
Parametric and nonparametric bootstrap methods for meta-analysis
Noortgate, W. V. D., & Onghena, P. (2005) · Behavior Research Methods · doi:10.3758/bf03206394 · Cited by 61 · see citing works
Thorlund, K., Thabane, L., & Mills, E. J. (2013) · BMC Medical Research Methodology · doi:10.1186/1471-2288-13-2 · Cited by 60 · see citing works
A simple method for inference on an overall effect in meta‐analysis
Brockwell, S. E., & Gordon, I. R. (2007) · Statistics in Medicine · doi:10.1002/sim.2883 · Cited by 59 · see citing works
Applicable or non-applicable: investigations of clinical heterogeneity in systematic reviews
Chess, L. E., & Gagnier, J. J. (2016) · BMC Medical Research Methodology · doi:10.1186/s12874-016-0121-7 · Cited by 58 · see citing works
MetaEasy: A Meta-Analysis Add-In for MicrosoftExcel
Kontopantelis, E., & Reeves, D. (2009) · Journal of Statistical Software · doi:10.18637/jss.v030.i07 · Cited by 57 · see citing works
On Constructing Confidence Intervals for a Standardized Mean Difference in Meta-analysis
Sidik, K., & Jonkman, J. N. (2003) · Communications in Statistics - Simulation and Computation · doi:10.1081/sac-120023885 · Cited by 57 · see citing works
On random-effects meta-analysis
Zeng, D., & Lin, D. Y. (2015) · Biometrika · doi:10.1093/biomet/asv011 · Cited by 51 · see citing works
A refined method for multivariate meta‐analysis and meta‐regression
Jackson, D., & Riley, R. D. (2013) · Statistics in Medicine · doi:10.1002/sim.5957 · Cited by 49 · see citing works
Self-citationRhodes, K. M., Turner, R., White, I. R., Jackson, D., Spiegelhalter, D. J., & Higgins, J. P. T. (2016) · Statistics in Medicine · doi:10.1002/sim.7090 · Cited by 49 · see citing works
Self-citationHigher‐order likelihood inference in meta‐analysis and meta‐regression
Guolo, A. (2011) · Statistics in Medicine · doi:10.1002/sim.4451 · Cited by 48 · see citing works
Maximum Likelihood Estimation of Generalized Linear Models with Covariate Measurement Error
Rabe‐Hesketh, S., Skrondal, A., & Pickles, A. R. (2003) · The Stata Journal Promoting communications on statistics and Stata · doi:10.1177/1536867x0400300408 · Cited by 47 · see citing works
The Effect of the Heterogeneity Variance Estimator on Some Tests of Treatment Efficacy
Makambi, K. H. (2004) · Journal of Biopharmaceutical Statistics · doi:10.1081/bip-120037191 · Cited by 46 · see citing works
Reducing the Number of Unjustified Significant Results in Meta-analysis
Hartung, J. G., & Makambi, K. H. (2003) · Communications in Statistics - Simulation and Computation · doi:10.1081/sac-120023884 · Cited by 39 · see citing works
Bootstrap estimates of standard errors in validity generalization.
Switzer, F. S., Paese, P. W., & Drasgow, F. (1992) · Journal of Applied Psychology · doi:10.1037/0021-9010.77.2.123 · Cited by 36 · see citing works
A re‐evaluation of the ‘quantile approximation method’ for random effects meta‐analysis
Jackson, D., & Bowden, J. (2008) · Statistics in Medicine · doi:10.1002/sim.3487 · Cited by 36 · see citing works
Self-citationJackson, D., & Bowden, J. (2016) · BMC Medical Research Methodology · doi:10.1186/s12874-016-0219-y · Cited by 35 · see citing works
Self-citationTheRPackagemetaLikfor Likelihood Inference in Meta-Analysis
Guolo, A., & Varin, C. (2012) · Journal of Statistical Software · doi:10.18637/jss.v050.i07 · Cited by 34 · see citing works
Alternatives for Mixed-Effects Meta-Regression Models in the Reliability Generalization Approach
López-López, J. A., Botella, J., Sánchez‐Meca, J., & Marín‐Martínez, F. (2012) · Journal of Educational and Behavioral Statistics · doi:10.3102/1076998612466142 · Cited by 32 · see citing works
Accounting for heterogeneity in meta‐analysis using a multiplicative model—an empirical study
Mawdsley, D., Higgins, J. P. T., Sutton, A. J., & Abrams, K. R. (2016) · Research Synthesis Methods · doi:10.1002/jrsm.1216 · Cited by 29 · see citing works
Self-citationZschorlich, B., Knelangen, M., & Bastian, H. (2010) · Das Gesundheitswesen · doi:10.1055/s-0030-1261879 · Cited by 26 · see citing works
The Concept of Attributable Risk in Epidemiological Practice
Uter, W. J. C., & Pfahlberg, A. B. (1999) · Biometrical Journal · doi:10.1002/(sici)1521-4036(199912)41:8 · Cited by 25 · see citing works
metaplus: An R Package for the Analysis of Robust Meta-Analysis and Meta-Regression
Beath, K. (2016) · The R Journal · doi:10.32614/rj-2016-001 · Cited by 25 · see citing works
We know less than we should about methods of meta‐analysis
Hoaglin, D. C. (2015) · Research Synthesis Methods · doi:10.1002/jrsm.1146 · Cited by 24 · see citing works
A SAS Interface for Bayesian Analysis With WinBUGS
Zhang, Z. J., McArdle, J. J., Wang, L., & Hamagami, F. (2008) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705510802339106 · Cited by 22 · see citing works
Maximum Likelihood Estimation of Generalized Linear Models with Covariate Measurement Error
Rabe‐Hesketh, S., Skrondal, A., & Pickles, A. R. (2003) · The Stata Journal Promoting communications on statistics and Stata · doi:10.1177/1536867x0300300408 · Cited by 17 · see citing works
Preuss, M. H., & Ziegler, A. (2013) · Methods of Information in Medicine · doi:10.3414/me13-01-0073 · Cited by 14 · see citing works
Positive estimation of the between-study variance in meta-analysis : theory and methods
Hartung, J., & Makambi, K. H. (2002) · South African Statistical Journal · Cited by 13 · see citing works
The Significance Level of the Standard Test for a Treatment Effect in Meta-analysis
Jackson, D. (2009) · Statistics in Biopharmaceutical Research · doi:10.1198/sbr.2009.0009 · Cited by 12 · see citing works
Self-citationIntegrated Likelihood Inference in Small Sample Meta‐analysis for Continuous Outcomes
Bellio, R., & Guolo, A. (2015) · Scandinavian Journal of Statistics · doi:10.1111/sjos.12172 · Cited by 12 · see citing works
Generalised interval estimation in the random effects meta regression model
Möbius, T. W. D., & Knapp, G. (2013) · Computational Statistics & Data Analysis · doi:10.1016/j.csda.2013.03.011 · Cited by 12 · see citing works
Self-citationHigher order inference for the consensus mean in inter-laboratory studies
Sharma, G., & Mathew, T. (2010) · Biometrical Journal · doi:10.1002/bimj.201000032 · Cited by 11 · see citing works
Sidik, K., & Jonkman, J. N. (2003) · Statistics in Medicine · doi:10.1002/sim.1729 · Cited by 7 · see citing works
Estimation of the effect size in meta‐analysis with few studies
Longford, N. T. (2009) · Statistics in Medicine · doi:10.1002/sim.3814 · Cited by 4 · see citing works
Confidence Regions and Intervals for Meta-Analysis Model Parameters
Rukhin, A. L. (2015) · Technometrics · doi:10.1080/00401706.2014.962707 · Cited by 4 · see citing works
‘Improving the accuracy of likelihood-based inference in meta-analysis and meta-regression’
Kosmidis, I., Guolo, A., & Varin, C. (2017) · Biometrika · doi:10.1093/biomet/asx040 · Cited by 2 · see citing works
mvtmeta: Multivariate Meta-Analysis
Chen, H. (2012) · doi:10.32614/cran.package.mvtmeta · Cited by 1 · see citing works
Explaining heterogeneity in meta-analysis: a comparison of methods
Thompson, S. G., & Sharp, S. J. (1999) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19991030)18:20
Hahn, S., Williamson, P. R., Hutton, J. L., Garner, P., & Flynn, E. V. (2000) · Statistics in Medicine · doi:10.1002/1097-0258(20001230)19:24
Flagged first, then by citations
Research works printed but not linked: 3
- [37]37. Rukhin AL. Estimating heterogeneity variance in meta-analysis. Journal of the Royal Statistical Society: Series B (Statistical Methodology). 2013;75(3):451-469.
- [51]51. Hartung J. An alternative method for meta-analysis. Biom J 1999;41(8):901-916.
- [89]89. Turner RM, Omar RZ, Yang M, Goldstein H, Thompson SG. A multilevel model framework for meta-analysis of clinical trials with binary outcomes. Stat Med. 2000;19(24):3417-3432.
Cited by
Sorted by citations
Page, M. J., Moher, D., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., et al. (2021) · BMJ · doi:10.1136/bmj.n160 · Cited by 11,883 · see citing works
Page, M. J., Moher, D., Bossuyt, P., Boutron, I., Hoffmann, T., mulrow, C., Shamseer, L., Tetzlaff, J., Akl, E., Brennan, S. E., et al. (2020) · doi:10.31222/osf.io/gwdhk · Cited by 916 · see citing works
Methods for evidence synthesis in the case of very few studies
Bender, R., Friede, T., Koch, A., Kuss, O., Schlattmann, P., Schwarzer, G., & Skipka, G. (2018) · Research Synthesis Methods · doi:10.1002/jrsm.1297 · Cited by 250 · see citing works
Self-citationHenderson, J. T., Vesco, K. K., Senger, C. A., Thomas, R. G., & Redmond, N. (2021) · JAMA · doi:10.1001/jama.2021.8551 · Cited by 227 · see citing works
Claus, B. B., Stahlschmidt, L., Dunford, E., Major, J., Harbeck-Weber, C., Bhandari, R. P., Baerveldt, A., Neß, V., Grochowska, K., Hübner-Möhler, B., et al. (2022) · Pain · doi:10.1097/j.pain.0000000000002636 · Cited by 59 · see citing works
Saueressig, T., Pedder, H., Bowe, S. J., Owen, P. J., & Belavy, D. L. (2021) · Journal of Orthopaedic & Sports Physical Therapy · doi:10.2519/jospt.2021.0107 · Cited by 41 · see citing works
Gaia, J. W. P., Schuch, F. B., Ferreira, R. W., Souza, E. D. L., Ferreira, V. M. S., & Pires, D. A. (2024) · Scandinavian Journal of Medicine & Science in Sports · doi:10.1111/sms.14618 · Cited by 26 · see citing works
McIsaac, D. I., Hutton, B., Veroniki, A., Gill, M., Boland, L., Branje, K., Grudzinski, A. L., Hladkowicz, E., Shaw, J., & Gillis, C. (2022) · BMJ Open · doi:10.1136/bmjopen-2022-068797 · Cited by 7 · see citing works
Self-citation
- Authors
- Veroniki, A. A., Jackson, D., Bender, R., Kuss, O., Langan, D., Higgins, J. P. T., Knapp, G., & Salanti, G.
- Affiliations
- St. Michael's Hospital (CA)
- AstraZeneca (United Kingdom) (GB)
- Institute for Quality and Efficiency in Health Care (DE)
- Deutsches Diabetes-Zentrum e.V. (DE)
- University College London (GB)
- University of Bristol (GB)
- TU Dortmund University (DE)
- University of Bern (CH)
- Corresponding author
- Veroniki, Areti Angeliki
- ORCID
- Venue
- Research Synthesis Methods
- Publisher
- Wiley
- Published
- 2018-10-09
- Volume
- 10
- Issue
- 1
- Pages
- 23-43
- ISSN
- 1759-2879 (ISSN-L)
- 1759-2887
- OpenAlex
- W2888732220
- Cited by
- 236
- References
- 137
- Citations by year
- 2026: 37 · 2025: 32 · 2024: 36 · 2023: 31 · 2022: 30 · 2021: 35 · 2020: 23 · 2019: 11 · 2018: 1
- FWCI
- 13.33
- Type
- journal article
- Language
- en
- Open access
- Licence
- Locations
- Research Synthesis Methods (publishedVersion)
- PubMed (publishedVersion)
- Explore Bristol Research (publishedVersion, other-oa)
- BORIS (University Library Bern) (submittedVersion, other-oa)
- UCL Discovery (University College London) (submittedVersion)
- Bristol Research (University of Bristol) (submittedVersion, other-oa)
- Open Access CRIS of the University of Bern
- Funders
- National Institute for Health and Care Research (NF-SI-0617-10145)
- Medical Research Council (U1052 60558)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Keywords
- random-effects meta-analysis, uncertainty estimation, prediction intervals, confidence intervals, heterogeneity, point estimates, alternative estimation methods
- MeSH terms
- Algorithms, Bayes Theorem, Computer Simulation, Data Interpretation, Statistical, Decision Making, Humans, Random Allocation, Research Design, Meta-Analysis as Topic, Reproducibility of Results, Models, Statistical, Likelihood Functions, Sample Size, Uncertainty
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
