The DOI matches its Crossref record; 52 of the 52 references it cites were found in a registry or an index.
A comparison of heterogeneity variance estimators in simulated random‐effects meta‐analyses
Open accessAbstract
Studies combined in a meta‐analysis often have differences in their design and conduct that can lead to heterogeneous results. A random‐effects model accounts for these differences in the underlying study effects, which includes a heterogeneity variance parameter. The DerSimonian‐Laird method is often used to estimate the heterogeneity variance, but simulation studies have found the method can be biased and other methods are available. This paper compares the properties of nine different heterogeneity variance estimators using simulated meta‐analysis data. Simulated scenarios include studies of equal size and of moderate and large differences in size. Results confirm that the DerSimonian‐Laird estimator is negatively biased in scenarios with small studies and in scenarios with a rare binary outcome. Results also show the Paule‐Mandel method has considerable positive bias in meta‐analyses with large differences in study size. We recommend the method of restricted maximum likelihood (REML) to estimate the heterogeneity variance over other methods. However, considering that meta‐analyses of health studies typically contain few studies, the heterogeneity variance estimate should not be used as a reliable gauge for the extent of heterogeneity in a meta‐analysis. The estimated summary effect of the meta‐analysis and its confidence interval derived from the Hartung‐Knapp‐Sidik‐Jonkman method are more robust to changes in the heterogeneity variance estimate and show minimal deviation from the nominal coverage of 95% under most of our simulated scenarios.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Meta-analysis was conducted using a random-effect model with restricted maximum likelihood, as the estimator of true heterogeneity, which seems to provide us with less biased estimates than other methods.33”
From Psychological interventions for depression and anxiety: a systematic review and meta-analysis of Iranian chronic pain trials. (2020) · cites this paper as [33]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“For random-effects models, many methods are available, and their performance has been shown to differ depending on the characteristics of the meta-analysis (such as the number and size of the included studies113 114).”
From PRISMA 2020 explanation and elaboration: updated guidance and exemplars for reporting systematic reviews (2021) · cites this paper as [114]
The cited source says what the claim attributes to it.
In the citing paper
“a random-effects model typically cannot adequately estimate heterogeneity with only two effects (Langan et al., 2019)”
From Investigating the replicability of preclinical cancer biology (2021) · cites this paper as Langan et al., 2019
The cited source says what the claim attributes to it.
In the citing paper
“This method assumes a t-distribution of the pooled effect rather than a normal distribution, which reduces the chance of obtaining false-positive results (Langan et al., 2019).”
From The state of hiring discrimination: A meta-analysis of (almost) all recent correspondence experiments (2023) · cites this paper as Langan et al., 2019
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Meta-analysis in clinical trials
DerSimonian, R., & Laird, N. (1986) · Controlled Clinical Trials · doi:10.1016/0197-2456(86)90046-2 · Cited by 39,552
Quantifying heterogeneity in a meta‐analysis
Higgins, J. P. T., & Thompson, S. G. (2002) · Statistics in Medicine · doi:10.1002/sim.1186 · Cited by 36,763
Self-citationConducting Meta-Analyses in R with the metafor Package
Viechtbauer, W. (2010) · Journal of Statistical Software · doi:10.18637/jss.v036.i03 · Cited by 20,464
Self-citationBorenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009) · doi:10.1002/9780470743386 · Cited by 14,225
Self-citationBorenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2021) · doi:10.1002/9781119558378 · Cited by 9,799
Self-citationHunter, J., & Schmidt, F. (2004) · doi:10.4135/9781412985031 · Cited by 3,023
Maximum Likelihood Approaches to Variance Component Estimation and to Related Problems
Harville, D. A. (1977) · Journal of the American Statistical Association · doi:10.1080/01621459.1977.10480998 · Cited by 2,667
IntHout, J., Ioannidis, J. P., & Borm, G. F. (2014) · BMC Medical Research Methodology · doi:10.1186/1471-2288-14-25 · Cited by 2,419
Random-effects model for meta-analysis of clinical trials: An update
DerSimonian, R., & Kacker, R. (2007) · Contemporary Clinical Trials · doi:10.1016/j.cct.2006.04.004 · Cited by 2,374
Improved tests for a random effects meta‐regression with a single covariate
Knapp, G., & Hartung, J. (2003) · Statistics in Medicine · doi:10.1002/sim.1482 · Cited by 2,075
Cited by
Sorted by citations
Items 201–300 of 1,000
Cariprazine in the treatment of Bipolar Disorder: A systematic review and meta‐analysis
Pinto, J. V., Saraf, G., Vigo, D., Keramatian, K., Chakrabarty, T., & Yatham, L. N. (2019) · Bipolar Disorders · doi:10.1111/bdi.12850 · Cited by 34
Terrisse, S. A., Karamouza, E., Parker, C. C., Sartor, O. A., James, N. D., Pirrie, S. J., Collette, L., TOMBAL, B. F., Chahoud, J., Smeland, S., et al. (2019) · JAMA Oncology · doi:10.1001/jamaoncol.2019.4097 · Cited by 31
Godoy-Cumillaf, A. E. R., Fuentes-Merino, P., Díaz-González, A., Díaz, J. J., Martínez‐Vizcaíno, V., Álvarez‐Bueno, C., & Cavero‐Redondo, I. (2020) · Nutrients · doi:10.3390/nu12051378 · Cited by 36
“Furthermore, a sensitivity analyses were performed using the restricted maximum likelihood (REML) method to estimate the heterogeneity variance [39].”
MethodA meta-analytical review of intervention experiments to reduce food waste
Tian, X., Xia, Z., Xie, J., Zhang, C., Liu, Y., & Xu, M. (2022) · Environmental Research Letters · doi:10.1088/1748-9326/ac72b6 · Cited by 31
Higgins, J. P. T., & Li, T. (2022) · Wiley eBooks · doi:10.1002/9781119099369.ch10 · Cited by 32
Self-citationJoshee, S. R., Lim, L., Wybrecht, A., Berriesford, R., & Riddle, M. S. (2022) · Journal of Investigative Medicine · doi:10.1136/jim-2021-002109 · Cited by 32
Perspectives on the Incidence of Acanthamoeba Keratitis
Aiello, F., Afflitto, G. G., Ceccarelli, F., Turco, M. V., Han, Y., Amescua, G., Dart, J. K. G., & Nucci, C. (2024) · Ophthalmology · doi:10.1016/j.ophtha.2024.08.003 · Cited by 30
Guo, L., Zhang, J., An, R., Wang, W., Fen, J., Wu, Y., & Wang, Y. (2025) · BMC Medicine · doi:10.1186/s12916-025-04064-4 · Cited by 30
Adeoye, J., Alade, A. A., Zhu, W., Wang, W., Choi, S. W., & Thomson, P. J. (2021) · Oral Diseases · doi:10.1111/odi.13773 · Cited by 32
Lin, L., & Chu, H. (2022) · PLoS ONE · doi:10.1371/journal.pone.0268754 · Cited by 31
“, the restricted maximum likelihood (REML) estimator, have been shown to perform better in general [67, 68].”
BackgroundKhreis, H., Bredell, C., Fung, K. W., Hong, L., Szybka, M., Phillips, V., Abbas, A., Lim, Y., Andersen, Z. J., Woodcock, J., et al. (2022) · Environment International · doi:10.1016/j.envint.2022.107596 · Cited by 34
Ahn, N., Nolde, M., Krause, E., Güntner, F., Günter, A., Tauscher, M., Gerlach, R., Meisinger, C., Linseisen, J., Baumeister, S., et al. (2022) · British Journal of Clinical Pharmacology · doi:10.1111/bcp.15583 · Cited by 32
“The Hartung–Knapp–Sidik–Jonkman random-effects metaanalysis approach with restricted maximum likelihood (REML) variance estimation was used to combine study-specific log RRs with the empirical Bayes estimator for the between-study variance.(27) The 95% CIs from the random-effects model include highly probable values f…”
MethodSharma, A., Tang, A. L., Takiar, V., Wise-Draper, T. M., & Langevin, S. M. (2021) · Cancers · doi:10.3390/cancers13153677 · Cited by 29
“Summary HR was generated via a fixed-effects model (inverse variance method) [47] and random-effects model using the restricted maximum likelihood (REML) heterogeneity variance estimator [48].”
MethodTesting the bottom-up and top-down models of self-esteem: A meta-analysis of longitudinal studies.
Dapp, L. C., Krauss, S., & Orth, U. (2022) · Journal of Personality and Social Psychology · doi:10.1037/pspp0000444 · Cited by 32
Oliva, V., Fanelli, G., Kasper, S., Zohar, J., Souery, D., Montgomery, S., Albani, D., Forloni, G., Ferentinos, P., Rujescu, D., et al. (2022) · International Clinical Psychopharmacology · doi:10.1097/yic.0000000000000384 · Cited by 29
Tian, G., Ye, Z., Zhao, Q., & Jiang, T. (2020) · Asian Journal of Surgery · doi:10.1016/j.asjsur.2019.12.011 · Cited by 28
for the IPPIC Collaborative Network, Snell, K. I. E., Allotey, J., Smuk, M., Hooper, R., Chan, C., Ahmed, A., Chappell, L. C., von Dadelszen, P., Green, M., et al. (2020) · BMC Medicine · doi:10.1186/s12916-020-01766-9 · Cited by 28
“Summary (average) performance statistics were reported with 95% confidence intervals (derived using the Hartung-KnappSidik-Jonkman approach as recommended) [27, 28].”
MethodExtracting data from graphs: A case‐study on animal research with implications for meta‐analyses
van der Mierden, S., Spineli, L. M., Talbot, S. R., Yiannakou, C., Zentrich, E., Weegh, N., Struve, B., Brügge, T. F. Z., Bleich, A., & Leenaars, C. H. C. (2021) · Research Synthesis Methods · doi:10.1002/jrsm.1481 · Cited by 28
“The between-trial variance was estimated using the restricted maximum likelihood (REML) estimator for being recommended from a recent simulation study.(35,36) As the original and extracted data should be similar the data is correlated.”
MethodLópez‐Pacheco, A., Soto‐Peñaloza, D., Gómez, M., Peñarrocha‐Oltra, D., & Alarcón, M. A. (2021) · International Journal of Implant Dentistry · doi:10.1186/s40729-021-00294-2 · Cited by 30
“This approach is more robust to changes in the heterogeneity variance estimates, especially in metaanalyses that contain few studies [30].”
BackgroundAiquel, L. L., Pitta, J., Antonoglou, G. N., Mischak, I., Sailer, I., & Payer, M. (2021) · Clinical Oral Implants Research · doi:10.1111/clr.13860 · Cited by 27
Dalgaard, N. T., Bondebjerg, A., Klokker, R. H., Viinholt, B. C. A., & Dietrichson, J. (2022) · Campbell Systematic Reviews · doi:10.1002/cl2.1239 · Cited by 58
Idoiaga, N., Legorburu, I., Ozamiz-Etxebarria, N., Lipnicki, D., Villagrasa, B., & Santabárbara, J. (2022) · Sustainability · doi:10.3390/su14137914 · Cited by 27
Systematic review of avian hatching failure and implications for conservation
Marshall, A. F., Balloux, F., Hemmings, N., & Brekke, P. (2023) · Biological reviews/Biological reviews of the Cambridge Philosophical Society · doi:10.1111/brv.12931 · Cited by 28
Prevalence of gastrointestinal symptoms in autism spectrum disorder: A meta-analysis
Lasheras, I., Real‐López, M., & Santabárbara, J. (2023) · Anales de Pediatría (English Edition) · doi:10.1016/j.anpede.2023.07.003 · Cited by 31
Dietrichson, J., Filges, T., Klokker, R. H., Viinholt, B. C. A., Bøg, M., & Jensen, U. H. (2020) · Campbell Systematic Reviews: Better Evidence for a Better World · doi:10.1002/cl2.1081 · Cited by 33
Hussain, U., Alam, S., Rehman, K., Antonoglou, G. N., & Papageorgiou, S. N. (2023) · European Journal of Orthodontics · doi:10.1093/ejo/cjac044 · Cited by 30
Social physical anxiety and eating disorders: A systematic review and meta-analysis
Alcaraz‐Ibáñez, M., Paterna, A., & Griffiths, M. D. (2023) · Body Image · doi:10.1016/j.bodyim.2023.02.008 · Cited by 31
Barker, T. G., O’Higgins, A., Fonagy, P., & Gardner, F. E. M. (2024) · Journal of Affective Disorders · doi:10.1016/j.jad.2023.12.080 · Cited by 28
Brennan, C. J., Roberts, C. A., & Cole, J. C. (2024) · BMJ Open · doi:10.1136/bmjopen-2023-071776 · Cited by 29
“33 Separate meta-analyses were conducted with the prevalence of PTED as the outcome, and then the prevalence of moral injury as the outcome.”
MethodOrtiz, J. A., Nolasco, J. M., Huang, Y. T., & Chow, J. C. (2024) · Journal of Speech Language and Hearing Research · doi:10.1044/2024_jslhr-24-00212 · Cited by 31
Samea, F., Mortazavi, N., Reimann, G. M., Ebneabbasi, A., Zarei, M., Khazaie, H., Goldstein‐Piekarski, A. N., Spiegelhalder, K., Baglioni, C., Sepehry, A. A., et al. (2025) · Sleep Medicine Reviews · doi:10.1016/j.smrv.2025.102111 · Cited by 34
Havakeshian, G., Koretsi, V., Eliades, T., & Papageorgiou, S. N. (2020) · Journal of Clinical Medicine · doi:10.3390/jcm9093015 · Cited by 29
“This systematic review has several strengths, comprising an a priori registered protocol [14], a comprehensive literature search, the inclusion of an untreated Class III control group, the use of modern analytic methods [21], the application of the GRADE approach to assess the strength of provided recommendations [17]…”
Wallace, E., Carberry, J. C., Toson, B., & Eckert, D. J. (2022) · Sleep Medicine Reviews · doi:10.1016/j.smrv.2022.101589 · Cited by 27
A Role for Visual Memory in Vocabulary Development: A Systematic Review and Meta-Analysis
Pickering, H. E., Peters, J., & Crewther, S. G. (2022) · Neuropsychology Review · doi:10.1007/s11065-022-09561-4 · Cited by 38
“A random-effects model (restricted maximum likelihood, REML) was chosen a priori , as significant methodological variability between included studies was anticipated (Langan et al., 2019).”
Altheer, C., Papageorgiou, S. N., Antonarakis, G. S., & Papadopoulou, A. K. (2024) · European Journal of Orthodontics · doi:10.1093/ejo/cjae010 · Cited by 28
“As far as identifiability is concerned, heterogeneity (tau 2 ) was assumed to be the same across all comparisons of each network [15] that was calculated with a REstricted Maximum Likelihood (REML) estimator and its 95% CI using the Q-profile approach, based on empirical evidence [21].”
MethodKerachi, A. J., Shahlaee, M. A., Habibi, P., Ebrahimi, N. D., Ala, M., & Sadeghi, A. (2025) · BMC Medicine · doi:10.1186/s12916-025-03935-0 · Cited by 26
Papageorgiou, S. N., Koletsi, D., & Eliades, T. (2019) · Journal of Orthodontics · doi:10.1177/1465312519880558 · Cited by 26
“, 2015a, 2015b), the use of modern analytic methods (Langan et al., 2019), application of the GRADE approach to assess the strength of provided recommendations (Guyatt et al.”
MethodWhen the Medium Is the Message: A Meta-Analysis of Creative Media Advertising Effects
van Berlo, Z. M. C., Meijers, M. H. C., Eelen, J., Voorveld, H. A. M., & Eisend, M. (2023) · Journal of Advertising · doi:10.1080/00913367.2023.2186986 · Cited by 26
Sand, F. L., Urbutė, A., Ring, L. L., Kjær, A. K., Belmonte, F., & Kjær, S. K. (2023) · Preventive Medicine · doi:10.1016/j.ypmed.2023.107519 · Cited by 29
Behers, B. J., Melchor, J., Behers, B. M., Meng, Z., Swanson, P. J., Paterson, H. I., Araque, S. J. M., Davis, J. L., Gerhold, C., Shah, R. S., et al. (2023) · Nutrients · doi:10.3390/nu15194223 · Cited by 27
“This between-study variance was estimated using the restricted maximum likelihood (REML) method [28].”
MethodBushi, G., Khatib, M. N., Rohilla, S., Singh, M. P., Uniyal, N., Ballal, S., Bansal, P., Bhopte, K., Gupta, M., Gaidhane, A. M., et al. (2025) · Diabetes/Metabolism Research and Reviews · doi:10.1002/dmrr.70037 · Cited by 27
Liu, Z., Amer, F. M. A., Xiao, M., Xu, C., Furuya‐Kanamori, L., Hong, H., Siegel, L. K., & Lin, L. (2023) · BMC Medicine · doi:10.1186/s12916-023-02823-9 · Cited by 27
“Third, although the REML method is generally recommended for estimating the between-study variance [36], it could have convergence problems that lead to a loss of about 2.”
MethodA meta‐analysis of creativity training in organizational settings
McKay, A. S., Reiter‐Palmon, R., Coombes, S. M. T., & Coombs, J. E. (2024) · Creativity and Innovation Management · doi:10.1111/caim.12605 · Cited by 32
Can Consumers’ Altruistic Inferences Solve the CSR Initiative Puzzle? A Meta-analytic Investigation
Carrillat, F. A., Plewa, C., Pupovac, L., Vanasse, C., Willmott, T., Legoux, R., & Napolova, E. (2024) · Journal of Business Ethics · doi:10.1007/s10551-024-05704-0 · Cited by 28
“We apply the Metafor package in R for our analyses (Viechtbauer, 2010), with restricted maximum likelihood as an estimator, because of its greater efficiency compared with alternatives (Langan et al., 2019).”
Eldabe, S. S., Nevitt, S. J., Copley, S., Maden, M., Goudman, L., Hayek, S. M., Mekhail, N. A., Moens, M., Rigoard, P., & Duarte, R. V. (2023) · Regional Anesthesia & Pain Medicine · doi:10.1136/rapm-2023-104674 · Cited by 25
Boyne, P., Miller, A. E., Kubalak, O., Mink, C., Reisman, D. S., & Fulk, G. D. (2023) · Neurology Report · doi:10.1097/npt.0000000000000456 · Cited by 24
“26,27 Since standard deviations such as SD response have poor statistical properties, 18 response variability meta-analyses were done using the variance of response (Var response ) 3,4,13,17,18 and then results were converted to standard deviation units.”
MethodFunctional appliance treatment for mandibular fractures: A systematic review with meta‐analyses
Stähli, C., Eliades, T., & Papageorgiou, S. N. (2021) · Journal of Oral Rehabilitation · doi:10.1111/joor.13178 · Cited by 23
Kim, Y., Lehrer, E. J., Wirth, P. J., Khesroh, E., Brewer, J. D., Billingsley, E. M., Zaorsky, N. G., & Lam, C. (2021) · Journal of the American Academy of Dermatology · doi:10.1016/j.jaad.2021.11.059 · Cited by 24
Blázquez-Rincón, D., Sánchez‐Meca, J., Botella, J., & Suero, M. (2023) · BMC Medical Research Methodology · doi:10.1186/s12874-022-01809-0 · Cited by 23
“[45] compared estimators similar to those previous simulation studies in terms of relative bias and MSE.”
BackgroundResultInfluentialDexter, F., & Hindman, B. J. (2023) · Journal of Clinical Anesthesia · doi:10.1016/j.jclinane.2023.111210 · Cited by 26
Pradeilles, R., Norris, T., Sellem, L., & Markey, O. (2023) · Advances in Nutrition · doi:10.1016/j.advnut.2023.09.003 · Cited by 24
“In addition, the Hartung – Knapp – Sidik – Jonkman (HKSJ) correction was applied to estimate the 95% CIs of the summary effects [39,40]; this is a more conservative approach for pooling a small number of studies when compared with the Wald-type method [38,41,42].”
MethodUmbrella reviews: a useful study design in need of standardisation
Shi, X., & Wallach, J. D. (2022) · BMJ · doi:10.1136/bmj.o1740 · Cited by 24
Kumpasoğlu, G. B., Campbell, C., Saunders, R., & Fonagy, P. (2024) · Psychotherapy Research · doi:10.1080/10503307.2023.2298000 · Cited by 23
“The restricted maximum-likelihood estimator (REML) was selected as the heterogeneity variance estimator for all meta-analyses, as recommended for continuous outcome meta-analyses of both small and large study sizes (Langan et al., 2019; Novianti et al., 2014).”
MethodSilverman, R. D., Keane, K., Darling-Hammond, E., & Khanna, S. (2024) · Review of Educational Research · doi:10.3102/00346543241261073 · Cited by 35
“One key advantage of REML over other estimation methods (e.g., maximum likelihood estimation) is that it tends to provide less biased estimates of the variance components, especially when there is significant heterogeneity among studies or there are few studies in the analysis (Langan et al., 2019).”
Maishman, T., Schaap, S., Silk, D. S., Nevitt, S. J., Woods, D. C., & Bowman, V. E. (2022) · Statistical Methods in Medical Research · doi:10.1177/09622802221109506 · Cited by 28
“The use of REML to calculate τ2 has been shown to be robust to deviations from normality and to perform well, particularly when utilising the KNHA method to calculate the CIs, when only a limited number of models are available for comparison [10,15,16].”
BackgroundMethodResultInfluentialLaubach, M., Weimer, L. P., Bläsius, F., Hildebrand, F., Kobbe, P., & Hutmacher, D. W. (2022) · Archives of Orthopaedic and Trauma Surgery · doi:10.1007/s00402-022-04621-z · Cited by 26
“The preferred heterogeneity variance estimator of restricted maximum likelihood (REML) [33] was applied.”
MethodPrevalencia de síntomas gastrointestinales en trastornos del espectro del autismo: un metaanálisis
Lasheras, I., Real‐López, M., & Santabárbara, J. (2023) · Anales de Pediatría · doi:10.1016/j.anpedi.2023.05.011 · Cited by 24
Orthodontic treatment of patients with severe (stage IV) periodontitis
Papageorgiou, S. N., Antonoglou, G. N., Eliades, T., Martin, C., & Sanz, M. (2024) · Seminars in Orthodontics · doi:10.1053/j.sodo.2024.01.004 · Cited by 23
Cosier, D., Lambert, K., Neale, E. P., Probst, Y., & Charlton, K. E. (2024) · Nutrition Reviews · doi:10.1093/nutrit/nuae002 · Cited by 27
“58 Restricted maximum likelihood to estimate heterogeneity variance was used, as recommended by Langan et al. 59 The WMD (with 95% CI) was computed from mean change or final values, where possible, and Hedge’s g weighted SMD (with 95% CI) where measurement methods hindered a weighted mean…”
Hamza, B., Papageorgiou, S. N., Patcas, R., & Schätzle, M. A. (2024) · European Journal of Orthodontics · doi:10.1093/ejo/cjae054 · Cited by 22
Methods for estimating the sampling variance of the standardized mean difference.
Suero, M., Botella, J., & Durán, J. I. (2021) · Psychological Methods · doi:10.1037/met0000446 · Cited by 22
Hüsing, P., Smakowski, A., Löwe, B., Kleinstäuber, M., Toussaint, A. C., & Mora, M. C. S. (2023) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2023.1142484 · Cited by 24
“Between-study variance (τ2) will be estimated the method of restricted maximum likelihood (56, 57).”
MethodSanchez‐Ramos, L., Lin, L., Vilchez-Lagos, G., Duncan, J. R., Condon, N. A., Wheatley, J., & Kaunitz, A. M. (2023) · American Journal of Obstetrics and Gynecology · doi:10.1016/j.ajog.2022.01.005 · Cited by 22
Greig, L., Aspe, R., Hall, A., Comfort, P., Cooper, K., & Swinton, P. A. (2023) · Sports Medicine · doi:10.1007/s40279-023-01854-9 · Cited by 30
“Of the six studies included for narrative synthesis only, each investigated a single exercise including the bench press [48, 49], back squat [50], deadlift [51] and power clean [39].”
BackgroundMethodNguyen, G. T., Bell, Z., Andreae, G., Scott, S., Sermin‐Reed, L., Lake, A. A., & Heslehurst, N. (2024) · Obesity Reviews · doi:10.1111/obr.13753 · Cited by 24
“36,37 The I 2 statistic was used to assess the heterogeneity among studies, 38 with a threshold of >75% representing significant heterogeneity.”
MethodKılıç, Ö. U. (2019) · Australian Journal of Career Development · doi:10.1177/1038416219857567 · Cited by 30
Can mindfulness-based interventions reduce PTSD symptoms? An umbrella review
Jovanović, B., & Garfin, D. R. (2024) · Journal of Anxiety Disorders · doi:10.1016/j.janxdis.2024.102859 · Cited by 22
Maternal adiposity measures and hypertensive disorders of pregnancy: a meta-analysis
Xiang, C., Sui, L., Ding, X., Cao, M., Li, G., & Du, Z. (2024) · BMC Pregnancy and Childbirth · doi:10.1186/s12884-024-06788-2 · Cited by 21
“In this study, RRs were directly approximated as ORs [22].”
MethodNeonatal Colonization With Antibiotic-Resistant Pathogens in Low- and Middle-Income Countries
Beaumont, A., Kermorvant‐Duchemin, E., Breurec, S., & Huynh, B.-T. (2024) · JAMA Network Open · doi:10.1001/jamanetworkopen.2024.41596 · Cited by 21
PARP Inhibitor Maintenance After First-Line Chemotherapy in Advanced-Stage Epithelial Ovarian Cancer
Petousis, S., Kahramanoğlu, İ., Appenzeller‐Herzog, C., Ángeles, M. A., Margioula‐Siarkou, C., Kacperczyk-Bartnik, J. P., Bilir, E., Chatzakis, C., Caruso, G., Bizzarri, N., et al. (2025) · JAMA Network Open · doi:10.1001/jamanetworkopen.2025.41648 · Cited by 22
Amer, F. M. A., Thompson, C. G., & Lin, L. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18073492 · Cited by 22
“Similar to the frequentist framework, where multiple estimators are available for τ2 [7], choosing the prior for the heterogeneity parameter is critical in Bayesian meta-analyses, especially when k is small or events are rare, as this prior may greatly impact the credible interval (CrI) length [48].”
BackgroundMethodInfluentialBelloni, S., Arrigoni, C., Sanctis, R. D., Arcidiacono, M. A., Dellafiore, F., & Caruso, R. (2021) · Critical Reviews in Oncology/Hematology · doi:10.1016/j.critrevonc.2021.103373 · Cited by 24
Heterogeneity estimates in a biased world
Hönekopp, J., & Linden, A. H. (2022) · PLoS ONE · doi:10.1371/journal.pone.0262809 · Cited by 25
“Confirming previous findings, we found that REML and PM did well when the sets of primary studies were unbiased [6, 32].”
BackgroundMethodResultInfluentialTsang, T. K., Huang, X., Guo, Y., Lau, E. H. Y., Cowling, B. J., & Ip, D. (2023) · JMIR Public Health and Surveillance · doi:10.2196/41329 · Cited by 21
“We conducted random effects meta-analyses using the inverse variance method and restricted maximum likelihood estimator for heterogeneity to obtain the pooled correlation between school absenteeism and community surveillance [26-29].”
Yang, X., Fridman, A., Nash Unsworth, A., & Casement, M. D. (2023) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2023.105125 · Cited by 22
Frankort, J., Mees, B., Doukas, P., Keszei, A. P., Kontopodis, N., Antoniou, G. A., Jacobs, M. J. H. M., & Gombert, A. (2023) · European Journal of Vascular and Endovascular Surgery · doi:10.1016/j.ejvs.2023.05.006 · Cited by 21
Fitzgerald, I., Sahm, L. J., Byrne, A., O’Connell, J., Ensor, J., Dhubhlaing, C. N., O’Dwyer, S., & Crowley, E. K. (2023) · European Psychiatry · doi:10.1192/j.eurpsy.2023.2417 · Cited by 22
“Restrictedmaximum likelihood estimation (REML) was used to fit all analyses, with 95% confidence intervals (CI) derived using the Hartung–Knapp Sidik– Jonkman (HKSJ) approach, to account for uncertainty in estimated variances [20].”
MethodKock, L. S., Erath, T. G., Coleman, S. R. M., Higgins, S. T., & Heil, S. H. (2023) · Preventive Medicine · doi:10.1016/j.ypmed.2023.107654 · Cited by 23
Evolution of statistical models for meta-analysis and implications for best practice
Kelley, G. A., & Kelley, K. S. (2023) · Current Opinion in Epidemiology and Public Health · doi:10.1097/pxh.0000000000000023 · Cited by 21
Colmenares, A. M., Prytherch, B., Thomas, M. L., & Burzynska, A. Z. (2023) · Imaging Neuroscience · doi:10.1162/imag_a_00045 · Cited by 24
“Heterogeneity variance was calculated using the restricted maximum likelihood (REML) method ( Langan et al., 2019).”
MethodWhite, C. M., Caroti, K. S., Bessada, Y., Hernández, A. V., Baker, W. L., Dobesh, P. P., van Haalen, H., Rhodes, K. M., & Coleman, C. I. (2024) · Pharmacotherapy The Journal of Human Pharmacology and Drug Therapy · doi:10.1002/phar.2925 · Cited by 21
“23 To reflect uncertainty in estimation of between-study variance, an adjustment to the 95% CI for the treatment effect proposed by Hartung and Knapp and by Sidik and Jonkman was used when five or more studies were meta-analyzed, given its enhanced performance in simulation evaluations and provides…”
MethodParr, N. J., Schweer‐Collins, M. L., Darlington, T. M., & Tanner‐Smith, E. E. (2019) · Journal of Adolescence · doi:10.1016/j.adolescence.2019.10.009 · Cited by 23
“Comparative strengths and drawbacks of different estimation approaches for τ2 are discussed in Langan, Higgins, and Simmonds (2017) and Langan et al. (2018).”
BackgroundMethodNejadghaderi, S. A., Mousavi, S. E., Fazlollahi, A., Asghari, K. M., & Garfin, D. R. (2024) · Psychiatry Research · doi:10.1016/j.psychres.2024.116098 · Cited by 27
“For analyses of retention in the trials, the odds ratio (OR) for each study was calculated; the between-study variance was estimated with the Paule-Mandel estimator (Langan et al., 2019).”
Tsompanaki, E., Koutoukidis, D. A., Wren, G., Tong, H., Theodoulou, A., Wang, D., Park, R. J., Jebb, S. A., & Aveyard, P. N. (2025) · EClinicalMedicine · doi:10.1016/j.eclinm.2024.103049 · Cited by 20
Mukherjee, A., Farsi, D. N., García-Gutiérrez, E., Akan, E., Salas-Millán, J. Á., Angelovski, L., Bintsis, Τ., Gérard, A., Güley, Z., Kabakcı, S. A., et al. (2025) · Frontiers in Nutrition · doi:10.3389/fnut.2025.1668889 · Cited by 26
“Heterogeneity was assessed using the Chi-squared test and quantified with the I 2 statistic and Tau 2 estimated using the restricted maximum likelihood (REML) method (46).”
Sun, J., Tan, J., Zhang, Q., Shen, Y., Shi, J., Zhang, H., & Shi, X. (2023) · River Research and Applications · doi:10.1002/rra.4194 · Cited by 22
“…analyzed without considering distinctions of four types of fishways initially and then was analyzed separately by fishway types to find out whether the relationship maintains no significant change (Hershey, 2021; Johansen et al., 2016; Langan et al., 2019; Nyaga et al., 2014; Wong et al., 2020).”
MethodLocke, B. W., Gómez-Lumbreras, A., Tan, C. J., Nonthasawadsri, T., Veettil, S. K., Patikorn, C., & Chaiyakunapruk, N. (2024) · Obesity Reviews · doi:10.1111/obr.13697 · Cited by 20
“33 Knapp – Hartung adjustment of standard errors was used given the small size of included studies.”
MethodShi, L., Chu, H., & Lin, L. (2020) · Research Synthesis Methods · doi:10.1002/jrsm.1415 · Cited by 22
Minh, L. H. N., Abozaid, A. A., Ha, N. X., Quang, L. L., Gad, A. G., Tiwari, R., Le, T., Quyen, D. T. K., AL-Manaseer, B. M., Nguyen, D. K., et al. (2021) · Reviews in Medical Virology · doi:10.1002/rmv.2288 · Cited by 18
“Each group was analysed independently using the single pooled proportion (PP) or raw mean inverse variance random effect size method of meta‐analysis with the recommended restricted maximum likelihood tau method.(25,26) We have chosen these estimate methods as there is no direct comparison between these groups in most”
MethodSassano, M., Seyyedsalehi, M. S., Kappil, E. M., Zhang, S., Zheng, T., & Boffetta, P. (2024) · Environmental Research · doi:10.1016/j.envres.2024.120606 · Cited by 22
Datson, N., Lolli, L., Drust, B., Atkinson, G., Weston, M., & Gregson, W. A. (2021) · Science and Medicine in Football · doi:10.1080/24733938.2021.1942538 · Cited by 19
“dence synthesis informed the meta-analytical framework of the present study (Veroniki et al. 2016; Langan et al. 2019).”
MethodInfluentialTewabe, T., Tesfaye, W. H., Abrha, S., & Kelly, M. J. (2021) · BMJ Open · doi:10.1136/bmjopen-2021-049565 · Cited by 19
“21 Then, the Begg’s funnel plot and Egger’s test for small-study effects 22 were used to objectively assess the risk of bias across studies.”
Zhang, E., & Liu, Y. (2022) · International Journal of Educational Research · doi:10.1016/j.ijer.2022.101949 · Cited by 27
Alcaraz‐Ibáñez, M., Paterna, A., Griffiths, M. D., Demetrovics, Z., & Sicilia, Á. (2022) · Psychology of sport and exercise · doi:10.1016/j.psychsport.2022.102280 · Cited by 24
Felsch, M., Beckmann, L., Bender, R., Kuß, O., Skipka, G., & Mathes, T. (2022) · BMC Medical Research Methodology · doi:10.1186/s12874-022-01779-3 · Cited by 19
Šmid, A. N., Golja, P., Hadžić, V., Abazović, E., Drole, K., & Paravlic, A. H. (2024) · Sports Medicine · doi:10.1007/s40279-024-01992-8 · Cited by 27
“The method of restricted maximum likelihood (REML) with Knap-Hartung standard error adjustment was used for all analyses [37].”
MethodMatsingos, A., Wilhelm, M., Noor, L., Yildiz, C., Rief, W., Hofmann, S. G., Falkenberg, I., & Kircher, T. T. J. (2024) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2024.1346697 · Cited by 20
“We opted for the REML model due to its favorable outcome in comparison with other heterogeneity variance estimators (29).”
Methoddel Val, B. M., Shukla, S. R., Oduoye, M. O., Nsengiyumva, M., Tesfaye, T., & Glinkowski, W. (2024) · Annals of Medicine and Surgery · doi:10.1097/ms9.0000000000002258 · Cited by 19
“The extent of heterogeneity, or variability in results across different studies, was quanti fi ed using I 2 and tau-squared statistics [30] .”
MethodTsang, T. K., Sullivan, S. G., Huang, X., Wang, C., Wang, Y., Nealon, J., Yang, B., Ainslie, K. E. C., & Cowling, B. J. (2024) · American Journal of Epidemiology · doi:10.1093/aje/kwae142 · Cited by 21
Meza‐Mauricio, J., Furquim, C. P., Torrez, W. B., Soto‐Peñaloza, D., Peñarrocha‐Oltra, D., Retamal‐Valdes, B., & Faveri, M. (2021) · Clinical Oral Investigations · doi:10.1007/s00784-021-03782-2 · Cited by 19
Metadata
- Authors
- Langan, D., Higgins, J. P. T., Jackson, D., Bowden, J., Veroniki, A. A., Kontopantelis, E., Viechtbauer, W., & Simmonds, M.
- Affiliations
- University of Manchester (GB)
- University of Bristol (GB)
- AstraZeneca (United Kingdom) (GB)
- St. Michael's Hospital (CA)
- Maastricht University (NL)
- University of York (GB)
- Corresponding author
- Langan, Dean
- ORCID
- Venue
- Research Synthesis Methods
- Publisher
- Wiley
- Published
- 2018-09-06
- Volume
- 10
- Issue
- 1
- Pages
- 83-98
- ISSN
- 1759-2879 (ISSN-L)
- 1759-2887
- OpenAlex
- W2885675882
- Semantic Scholar
- 264ababb39dc4b8737425a30c9fb75c5d8eb2bf4
- Cited by
- 1,074
- References
- 51
- Citations by year
- 2026: 214 · 2025: 173 · 2024: 198 · 2023: 144 · 2022: 140 · 2021: 113 · 2020: 60 · 2019: 28 · 2018: 2
- FWCI
- 38.65
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Research Synthesis Methods (publishedVersion)
- PubMed (publishedVersion)
- Research Publications (Maastricht University) (publishedVersion, other-oa)
- Research Explorer (The University of Manchester) (publishedVersion, other-oa)
- White Rose Research Online (University of Leeds, The University of Sheffield, University of York) (submittedVersion)
- Bristol Research (University of Bristol) (submittedVersion, other-oa)
- UCL Discovery (University College London) (submittedVersion)
- Research Publications (Maastricht University) (submittedVersion, other-oa)
- Funders
- National Institute for Health and Care Research (NF-SI-0617-10145)
- Medical Research Council (MC_UU_00011/2, MR/K006665/1)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Keywords
- random-effects meta-analysis, estimation bias, binary outcomes, confidence interval coverage
- MeSH terms
- Systematic Reviews as Topic, Algorithms, Analysis of Variance, Computer Simulation, Data Interpretation, Statistical, Humans, Probability, Research Design, Software, Meta-Analysis as Topic, Reproducibility of Results, Models, Statistical, Selection Bias, Likelihood Functions, Odds Ratio
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
