The DOI matches its Crossref record; 52 of the 52 references it cites were found in a registry or an index.
A comparison of heterogeneity variance estimators in simulated random‐effects meta‐analyses
Open accessAbstract
Studies combined in a meta‐analysis often have differences in their design and conduct that can lead to heterogeneous results. A random‐effects model accounts for these differences in the underlying study effects, which includes a heterogeneity variance parameter. The DerSimonian‐Laird method is often used to estimate the heterogeneity variance, but simulation studies have found the method can be biased and other methods are available. This paper compares the properties of nine different heterogeneity variance estimators using simulated meta‐analysis data. Simulated scenarios include studies of equal size and of moderate and large differences in size. Results confirm that the DerSimonian‐Laird estimator is negatively biased in scenarios with small studies and in scenarios with a rare binary outcome. Results also show the Paule‐Mandel method has considerable positive bias in meta‐analyses with large differences in study size. We recommend the method of restricted maximum likelihood (REML) to estimate the heterogeneity variance over other methods. However, considering that meta‐analyses of health studies typically contain few studies, the heterogeneity variance estimate should not be used as a reliable gauge for the extent of heterogeneity in a meta‐analysis. The estimated summary effect of the meta‐analysis and its confidence interval derived from the Hartung‐Knapp‐Sidik‐Jonkman method are more robust to changes in the heterogeneity variance estimate and show minimal deviation from the nominal coverage of 95% under most of our simulated scenarios.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Meta-analysis was conducted using a random-effect model with restricted maximum likelihood, as the estimator of true heterogeneity, which seems to provide us with less biased estimates than other methods.33”
From Psychological interventions for depression and anxiety: a systematic review and meta-analysis of Iranian chronic pain trials. (2020) · cites this paper as [33]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“For random-effects models, many methods are available, and their performance has been shown to differ depending on the characteristics of the meta-analysis (such as the number and size of the included studies113 114).”
From PRISMA 2020 explanation and elaboration: updated guidance and exemplars for reporting systematic reviews (2021) · cites this paper as [114]
The cited source says what the claim attributes to it.
In the citing paper
“a random-effects model typically cannot adequately estimate heterogeneity with only two effects (Langan et al., 2019)”
From Investigating the replicability of preclinical cancer biology (2021) · cites this paper as Langan et al., 2019
The cited source says what the claim attributes to it.
In the citing paper
“This method assumes a t-distribution of the pooled effect rather than a normal distribution, which reduces the chance of obtaining false-positive results (Langan et al., 2019).”
From The state of hiring discrimination: A meta-analysis of (almost) all recent correspondence experiments (2023) · cites this paper as Langan et al., 2019
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Meta-analysis in clinical trials
DerSimonian, R., & Laird, N. (1986) · Controlled Clinical Trials · doi:10.1016/0197-2456(86)90046-2 · Cited by 39,552
Quantifying heterogeneity in a meta‐analysis
Higgins, J. P. T., & Thompson, S. G. (2002) · Statistics in Medicine · doi:10.1002/sim.1186 · Cited by 36,763
Self-citationConducting Meta-Analyses in R with the metafor Package
Viechtbauer, W. (2010) · Journal of Statistical Software · doi:10.18637/jss.v036.i03 · Cited by 20,464
Self-citationBorenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009) · doi:10.1002/9780470743386 · Cited by 14,225
Self-citationBorenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2021) · doi:10.1002/9781119558378 · Cited by 9,799
Self-citationHunter, J., & Schmidt, F. (2004) · doi:10.4135/9781412985031 · Cited by 3,023
Maximum Likelihood Approaches to Variance Component Estimation and to Related Problems
Harville, D. A. (1977) · Journal of the American Statistical Association · doi:10.1080/01621459.1977.10480998 · Cited by 2,667
IntHout, J., Ioannidis, J. P., & Borm, G. F. (2014) · BMC Medical Research Methodology · doi:10.1186/1471-2288-14-25 · Cited by 2,419
Random-effects model for meta-analysis of clinical trials: An update
DerSimonian, R., & Kacker, R. (2007) · Contemporary Clinical Trials · doi:10.1016/j.cct.2006.04.004 · Cited by 2,374
Improved tests for a random effects meta‐regression with a single covariate
Knapp, G., & Hartung, J. (2003) · Statistics in Medicine · doi:10.1002/sim.1482 · Cited by 2,075
Flagged first, then by citations
Cited by
Sorted by citations
Items 401–500 of 1,000
Varney, B., Brett, J., Zoëga, H., Gillies, M. B., Powell, M., Bateman, B. T., Shand, A., Pearson, S., & Havard, A. (2024) · Anaesthesia · doi:10.1111/anae.16307 · Cited by 11
“We estimated the heterogeneity variance ( s 2 ) using the restricted maximum likelihood estimator [26].”
The versatility of the scapular free flap: A workhorse flap? A systematic review and meta‐analysis
Escobar‐Domingo, M. J., Bustos, V. P., Akintayo, R. M., Mahmoud, A., Fanning, J. E., Foppiani, J., Miller, A. S., Cauley, R. P., Lin, S. J., & Lee, B. T. (2024) · Microsurgery · doi:10.1002/micr.31203 · Cited by 11
“Significant heterogeneity was considered if I 2 >50% or p <.05 (Borenstein et al., 2010; Huedo-Medina et al., 2006; Langan et al., 2019; Pereira et al., 2010).”
Predictors of acute kidney injury in dengue patients: a systematic review and meta-analysis
Awad, A. A., Khatib, M. N., Gaidhane, A. M., Ballal, S., Bansal, P., Srivastava, M., Arora, I., Kumar, M., Sinha, A., Rawat, P., et al. (2024) · Virology Journal · doi:10.1186/s12985-024-02488-7 · Cited by 11
“[18, 19].”
Behers, B. J., Behers, B. M., Stephenson-Moe, C. A., Vargas, I., Meng, Z., Thompson, A. J., Melchor, J., Wojtas, C. N., Rosario, M. A., Baker, J. F., et al. (2024) · Nutrients · doi:10.3390/nu16213617 · Cited by 14
Papageorgiou, S. N., Seehra, J., Cobourne, M. T., & Kanavakis, G. (2024) · Orthodontics and Craniofacial Research · doi:10.1111/ocr.12878 · Cited by 11
Meta-analysis of water stress impact on rice quality in China
Han, S., Liu, X., Makowski, D., & Ciais, P. (2024) · Agricultural Water Management · doi:10.1016/j.agwat.2024.109230 · Cited by 12
Leishmaniasis in deployed military populations: A systematic review and meta-analysis
Rawlings, N. N., Bailey, M. S., & Courtenay, O. (2025) · PLoS neglected tropical diseases · doi:10.1371/journal.pntd.0012680 · Cited by 11
“For all meta-analyses, data were fitted to random effects models using restricted maximum likelihood [47,48].”
Minkin, M., Woodland, L., Williams, O. A., Hamilton, S., Hansell, A., Vienneau, D., Gong, X., & Fenech, B. (2025) · Environment International · doi:10.1016/j.envint.2025.109667 · Cited by 11
Neurological Manifestations in Oropouche Virus Infection: A Systematic Review and Meta‐Analysis
Sah, R., Satapathy, P., Gaidhane, A. M., Vadia, N., Menon, S. V., Chennakesavulu, K., Panigrahi, R., Bushi, G., Singh, M. P., Sah, S. K., et al. (2025) · Journal of Medical Virology · doi:10.1002/jmv.70532 · Cited by 11
Barboza, J. J., Albitres‐Flores, L., Rivera-Meza, M., Rodriguez-Huapaya, J., Caballero-Alvarado, J., Pasupuleti, V., & Hernández, A. V. (2020) · Pediatric Research · doi:10.1038/s41390-020-0902-x · Cited by 11
Exploring consequences of simulation design for apparent performance of methods of meta-analysis
Kulinskaya, E., Hoaglin, D. C., & Bakbergenuly, I. (2021) · Statistical Methods in Medical Research · doi:10.1177/09622802211013065 · Cited by 14
“9 Expanding [W(i)v 2 i + 1] −1 and taking into account the independence of W(i) from θi and v 2 i , we have Cov(θi, [W(i)v 2 i + 1] −1) = − E(W(i)) (E(W(i))E(v 2 i ) + 1) 2 Cov(θi, v 2 i ), (14)”
BackgroundJeon, J. P., Chen, C., Tsuang, F.-Y., Liu, J., Hill, M. D., Zhang, L., Yang, P., Wang, G., Cho, B., Kim, J., et al. (2021) · International Journal of Stroke · doi:10.1177/17474930211047337 · Cited by 11
“The summary of estimates was presented as odds ratio (OR) using the method of restricted maximum likelihood to estimate the heterogeneity variance within the randomeffects model.(24) A narrative synthesis was planned if pooling was inappropriate.”
MethodUsing secondary cases to characterize the severity of an emerging or re-emerging infection
Tsang, T. K., Wang, C., Yang, B., Cauchemez, S., & Cowling, B. J. (2021) · Nature Communications · doi:10.1038/s41467-021-26709-7 · Cited by 11
THE EFFECT DIRECTION SHOULD BE TAKEN INTO ACCOUNT WHEN ASSESSING SMALL-STUDY EFFECTS
Meng, Z., Wu, C., & Lin, L. (2022) · Journal of Evidence Based Dental Practice · doi:10.1016/j.jebdp.2022.101830 · Cited by 11
Peng, J., Lei, Y., Liu, Y., Zhang, B., & Chen, J. (2023) · Clinical Oral Investigations · doi:10.1007/s00784-023-04881-y · Cited by 11
“According to the recent guidance, we chose random-effects model to summarize the mean effect size, instead of just based on the I2 value and P value [15].”
MethodClinical benefit and safety associated with mRNA vaccines for advanced solid tumors: A meta‐analysis
Zhang, T., Xu, H., Zheng, X., Xiong, X., Zhang, S., Yi, X., Li, J., Wei, Q., & Ai, J. (2023) · MedComm · doi:10.1002/mco2.286 · Cited by 10
Mousavi, S. E., Anamag, F. T., & Sanaie, S. (2023) · Phytotherapy Research · doi:10.1002/ptr.7973 · Cited by 11
“The Hartung-Knapp adjustment improves coverage probabilities and Type I error control over the random-effects method (IntHout et al., 2014; Langan et al., 2019); however, The Hartung-Knapp variance estimate can be very small if the treatment estimates are homogeneous, resulting in a narrow…”
MethodYang, S., Yang, S., Jo, Y. K., Kim, S., Chang, M. J., Choi, J., Cheon, J. H., & Yu, Y. M. (2021) · Therapeutic Advances in Chronic Disease · doi:10.1177/20406223211041927 · Cited by 11
Duration of tooth alignment with fixed appliances: A systematic review and meta-analysis
Wazwaz, F., Seehra, J., Carpenter, G. H., Ireland, A. J., Papageorgiou, S. N., & Cobourne, M. T. (2021) · American Journal of Orthodontics and Dentofacial Orthopedics · doi:10.1016/j.ajodo.2021.06.016 · Cited by 12
Brunner, M., Stallasch, S. E., & Lüdtke, O. (2023) · Journal of Research on Educational Effectiveness · doi:10.1080/19345747.2023.2175753 · Cited by 16
“…to reliably gauge the heterogeneity of (true) effect sizes when less than k ¼ 10 effect sizes are available for the meta-analytic integration (Langan et al., 2019, p. 95), we applied (multi-variate) fixed-effects models (Rice et al., 2018) when 2 (cid:4) k 10, and (multivariate)…”
MethodGanzevoort, C. O. W., Wolters, L. H., Hornstra, R., Grieve, C. M., Højgaard, D. R. M. A., Skarphedinsson, G. A., Weidle, B., Waite, P., Bertie, L.-A., Tomlinson, M., et al. (2024) · Journal of Anxiety Disorders · doi:10.1016/j.janxdis.2024.102940 · Cited by 11
Global Burden of Violence Against Transgender and Gender-Diverse Adults
McLellan, C., Yeh, P. T., Kennedy, C. E., Barre-Quick, M., Rich, A. J., Poteat, T., & Cooney, E. E. (2026) · JAMA Network Open · doi:10.1001/jamanetworkopen.2025.52953 · Cited by 11
Rahman, A. A., He, N., Rej, S., Platt, R. W., & Renoux, C. (2022) · Thrombosis and Haemostasis · doi:10.1055/a-1932-8976 · Cited by 10
Do jihadist terrorist attacks cause changes in institutional trust? A multi‐site natural experiment
Nägel, C., Nivette, A. E., & Czymara, C. S. (2023) · European Journal of Political Research · doi:10.1111/1475-6765.12612 · Cited by 12
Satapathy, P., Khatib, M. N., Gaidhane, S., Zahiruddin, Q. S., Serhan, H. A. A., Sharma, R. K., Sharma, D., Arora, M., Rustagi, S., AlKaabi, N. A., et al. (2024) · BMC Infectious Diseases · doi:10.1186/s12879-024-09357-2 · Cited by 10
“We calculated the tau-squared value using maximum likelihood estimation to gain additional insights into heterogeneity [25–28].”
MethodMost meta-analyses in oral health do not have conclusive and robust results
Tatas, Z., Kyriakou, E., Koutsiouroumpa, O., Seehra, J., Mavridis, D., & Pandis, N. (2024) · Journal of Dentistry · doi:10.1016/j.jdent.2024.105309 · Cited by 10
Russo, M. A., Nevitt, S. J., Santarelli, D. M., Eldabe, S. S., & Duarte, R. V. (2025) · Pain Medicine · doi:10.1093/pm/pnaf004 · Cited by 12
Lingamaiah, D., Bushi, G., Gaidhane, S., Balaraman, A. K., PadmaPriya, G., Kaur, I., Lal, M., Iqbal, S., Prasad, G. V. S., Pramanik, A., et al. (2025) · BMC Geriatrics · doi:10.1186/s12877-025-05722-1 · Cited by 10
“An LFK index value between − 1 and + 1 indicated no asymmetry, values between ± 1 and ± 2 indicated minor asymmetry, and values beyond ± 2 suggested major asymmetry [12, 13].”
Gonçalves, J. P. F., Duran, M. L., Barreto, E. S. R., Júnior, C. R. A., D′Albuquerque, L. A. C., Lins-Kusterer, L., Azi, L. M. T. D. A., & Kraychete, D. C. (2025) · Journal of Clinical Anesthesia · doi:10.1016/j.jclinane.2025.111831 · Cited by 10
Antonoglou, G. N., Papageorgiou, S. N., de Albornoz, A. C., Payer, M., & Stavropoulos, A. (2025) · Journal Of Clinical Periodontology · doi:10.1111/jcpe.70021 · Cited by 10
Harms, N. E., & Walter, D. J. (2021) · Aquatic Botany · doi:10.1016/j.aquabot.2021.103391 · Cited by 10
Muñoz, J. A. (2021) · PLoS ONE · doi:10.1371/journal.pone.0252533 · Cited by 10
Duration of canine retraction with fixed appliances: A systematic review and meta-analysis
Wazwaz, F., Seehra, J., Carpenter, G. H., Papageorgiou, S. N., & Cobourne, M. T. (2022) · American Journal of Orthodontics and Dentofacial Orthopedics · doi:10.1016/j.ajodo.2022.08.009 · Cited by 13
Ozamiz-Etxebarria, N., Fernández, I. L., Mondragón, N. I., Olaya, B., Cornelius‐White, J. H. D., & Santabárbara, J. (2023) · Sustainability · doi:10.3390/su15065272 · Cited by 10
A Reliability Generalization Meta-Analysis of the Padua Inventory-Revised (PI-R)
Núñez, R. M. N., Rubio‐Aparicio, M., Marín‐Martínez, F., Sánchez‐Meca, J., Pina, J. A. L., & López-López, J. A. (2021) · International Journal of Clinical and Health Psychology · doi:10.1016/j.ijchp.2021.100277 · Cited by 10
Tressoldi, P. E., & Storm, L. (2021) · F1000Research · doi:10.12688/f1000research.51746.1 · Cited by 13
“Frequentist random-effect model Following the recommendations of Langan et al. (2019), we used the restricted maximum likelihood (REML) approach to estimate the heterogeneity variance with the Knapp and Hartung method for adjustment to the standard errors of the estimated coefficients…”
MethodZuriaga, A., Kaplan, M. S., Choi, N. G., Hodkinson, A., Storman, D., Brudasca, N. I., Hirani, S. P., & Brini, S. (2021) · Journal of Affective Disorders · doi:10.1016/j.jad.2021.05.005 · Cited by 12
Fournier, J. C. J., Forand, N. R., Wang, Z., Li, Z., Iyengar, S., DeRubeis, R. J., Shelton, R. C., Amsterdam, J. D., Jarrett, R. B., Vittengl, J. R., et al. (2022) · Cognitive Therapy and Research · doi:10.1007/s10608-021-10281-x · Cited by 11
“Formal estimation of heterogeneity across trials is often unreliable with fewer than 10 studies (Jong et al., 2020; Langan et al., 2018).”
BackgroundJandaghi, G., Firoozi, M., & Zia-Tohidi, A. (2020) · Health Promotion Perspectives · doi:10.34172/hpp.2020.31 · Cited by 9
Efficacy of oxycodone for postoperative dental pain: A systematic review and meta-analysis
Teoh, L., McCullough, M. J., & Taing, M. (2022) · Journal of Dentistry · doi:10.1016/j.jdent.2022.104254 · Cited by 9
The impact of violating the independence assumption in meta-analysis on biomarker discovery
Aghababazadeh, F. A., Xu, W. M., & Haibe‐Kains, B. (2023) · Frontiers in Genetics · doi:10.3389/fgene.2022.1027345 · Cited by 10
Ylijoki, M., Sentenac, M., Pape, B., Zeitlin, J. A., & Lehtonen, L. (2024) · Acta Paediatrica · doi:10.1111/apa.17118 · Cited by 11
“A restricted maximum-likelihood estimator with the Hartung–Knapp–Sidik–Jonkman adjustment 14–16 was used.”
MethodHip osteoarthritis and occupational mechanical exposures: a systematic review and meta-analysis
Jahn, A., Andersen, J. H., Seidler, A., Christiansen, D. H., & Dalbøge, A. (2024) · Scandinavian Journal of Work Environment & Health · doi:10.5271/sjweh.4152 · Cited by 9
“Heteroge - neity between studies was calculated using I 2 statistics, quantified by the restricted maximum likelihood method ( 38), and was interpreted using Cochrane’s thresholds for interpretation of the I 2 statistics ( 39 ).”
Meta-analysis: Confidence intervals and Prediction intervals
Botella, J., & Sánchez‐Meca, J. (2024) · Anales de Psicología · doi:10.6018/analesps.591831 · Cited by 10
Kim, A., Frecklington, M., Philps, A., & Stewart, S. (2024) · Journal of Foot and Ankle Research · doi:10.1002/jfa2.70003 · Cited by 14
“…comparing more than one AFO intervention with a comparator (and therefore contributing more than one effect size to the analysis), the means and standard deviations for all AFO interventions were pooled to avoid double ‐ counting and correlated effect sizes (as recommended by Cochrane [27]).”
MethodStallasch, S. E., Lüdtke, O., Artelt, C., Hedges, L. V., & Brunner, M. (2024) · Educational Psychology Review · doi:10.1007/s10648-024-09898-7 · Cited by 9
“…2010), we fitted two meta-analytic/meta-regression model classes, conditional on the number of R 2 effect sizes G per covariate set: either (multivariate) fixed-effect models if G 10 or (multivariate multilevel) random-effects models via REML if G ≥ 10 (see Langan et al., 2019, p. 95).”
Assessment of inverse publication bias in safety outcomes: an empirical analysis
Xing, X., Zhu, J., Shi, L., Xu, C., & Lin, L. (2024) · BMC Medicine · doi:10.1186/s12916-024-03707-2 · Cited by 9
“However, 10 meta-analyses could not be calculated due to the non-convergence of the Fisher scoring algorithm when using the restricted maximum-likelihood (REML) method, which was our primary and preferred method for estimating between-study variance [34].”
MethodHesseling, B., Prinsze, N., van Etten‐Jamaludin, F. S., Perry, S. I. B., Eygendaal, D. E., Mathijssen, N. M. C., & Snoeker, B. A. M. (2024) · Systematic Reviews · doi:10.1186/s13643-024-02694-y · Cited by 9
“We used random-effects models with Restricted Maximum Likelihood (REML) estimation [8, 9] to conduct the analyses, using R version 4.3.1 [10] in RStudio version 2023.6.1.524 [11] and the metafor package [12].”
InfluentialAnticoagulation Among Patients Hospitalized for COVID-19
The WHO Rapid Evidence Appraisal for COVID-19 Therapies (REACT) Working Group, Vale, C. L., Godolphin, P. J., Fisher, D. J., Higgins, J. P. T., McAleenan, A., Spiga, F., Tritschler, T., Silva, P. G. M. D. B. E., Berg, D. D., et al. (2024) · Annals of Internal Medicine · doi:10.7326/annals-24-00800 · Cited by 10
“In sensitivity analyses, we estimated subgroup effects that were adjusted so that they corresponded with the pooled ratios of ORs derived from the within-trial approach (11, 12) (for additional information on the sensitivity analyses, see the Supplement Methods in Supplement 1 [13 – 15]).”
Self-citationA meta-analytic review of cognitive processing therapy with and without the written account
Sager, J. C., DeJesus, C. R., Kearns, J. C., Thompson‐Hollands, J., Trendel, S. L., Marx, B. P., & Sloan, D. M. (2025) · Journal of Anxiety Disorders · doi:10.1016/j.janxdis.2025.102976 · Cited by 14
Michalopoulou, M., Jebb, S. A., Hobson, A., Khaw, S. C., Stevens, R., Melo, P., Haffner, S. J. P., Clay, K. S., Mounsey, S., Granne, I., et al. (2025) · Annals of Internal Medicine · doi:10.7326/annals-24-01025 · Cited by 9
“…inverse variance random-effects models, applying the Hartung-Knapp-Sidik-Jonkman option to the DerSimonian-Laird simple method of moments estimator (22), and also present the pooled outcomes using fixed-effects meta-analysis with the Mantel-Haenszel, given that some of the meta-analyses included a…”
GLP-1 Receptor Agonists and Noncardiometabolic Outcomes
Yang, K., Liu, C., Guo, Q., & Li, Y. (2026) · JAMA Network Open · doi:10.1001/jamanetworkopen.2026.4722 · Cited by 10
Combining estimators in interlaboratory studies and meta‐analyses
Huang, H. (2023) · Research Synthesis Methods · doi:10.1002/jrsm.1633 · Cited by 9
Chang, Y., Lin, H.-M., Chi, K., Lin, W.-Y., & Chou, T.-C. (2023) · PeerJ · doi:10.7717/peerj.15149 · Cited by 9
“Third, methodologically, we performed our meta-analysis through a more robust heterogeneity estimator based on the latest simulation study (Langan et al., 2019).”
MethodArmoon, B., Griffiths, M. D., Mohammadi, R., & Ahounbar, E. (2023) · The American Journal of Drug and Alcohol Abuse · doi:10.1080/00952990.2023.2201872 · Cited by 9
“(55) 2017 1149 Canada Cohort 14–26 790 (69) 359 (31) NR NR Islam et al.”
MethodInfluentialRemote Viewing: A 1974-2022 Systematic Review and Meta-Analysis
Tressoldi, P. E., & Katz, D. L. (2023) · Journal of Scientific Exploration · doi:10.31275/20232931 · Cited by 23
“…using the metafor package v. 3.8 (Viechtbauer, 2010), adopting the restricted maximum likelihood (REML) to estimate the heterogeneity variance (Langan et al., 2019) and the Hartung method to control effect size nonnormality (Ru-bio-Aparicio et al., 2018) and corresponding confidence intervals…”
MethodInfluentialSabatés, J., Belleville, S., Castellani, M., Dwolatzky, T., Hampstead, B. M., Lampit, A., Simon, S. S., Anstey, K. J., Goodenough, B., Mancuso, S. G., et al. (2021) · Systematic Reviews · doi:10.1186/s13643-021-01787-2 · Cited by 9
Igarashi, Y., Akazawa, N., & Maeda, S. (2021) · Endocrine · doi:10.1007/s12020-021-02817-8 · Cited by 10
Dimov, K. G. (2022) · Archiva zootechnica · doi:10.2478/azibna-2022-0009 · Cited by 9
Civitillo, S., Mayer, A., & Jugert, P. (2022) · doi:10.31234/osf.io/d4rva · Cited by 9
Lee, C. C., Soon, Y. Y., Vellayappan, B. A., Ho, F. C. H., & Tey, J. C. S. (2022) · Acta Oncologica · doi:10.1080/0284186x.2022.2062680 · Cited by 8
Tracking Truth Through Measurement and the Spyglass of Statistics
Possolo, A. M. (2023) · Statistical Science · doi:10.1214/23-sts899 · Cited by 8
Health-related quality of life in patients with conditions affecting the hand: meta-analysis
Geoghegan, L., Carolina, M., French, J., Harrison, C. J., & Rodrigues, J. N. (2024) · British journal of surgery · doi:10.1093/bjs/znae067 · Cited by 8
“The restricted maximum likelihood (REML) method was used to estimate the heterogeneity of variance due to the small sample sizes in eligible studies 17 .”
Loef, B., Boer, J. M. A., Beekman, M., Campman, S. L., Hoogendijk, E. O., Huider, F., Pagen, D. M. E., Splinter, M. J., van der Velde, J. H. P. M., Boomsma, D. I., et al. (2024) · International Journal of Obesity · doi:10.1038/s41366-024-01660-x · Cited by 8
“In three cohorts with available data on serology (i.e. Doetinchem Cohort Study, HELIUS, The Rotterdam Study), the SARS-CoV-2 infection rate varied between 9.8% and 22.2%.”
InfluentialPapageorgiou, S. N., Giannakopoulou, T. T., Eliades, T., & Vandevska‐Radunovic, V. (2024) · European Journal of Orthodontics · doi:10.1093/ejo/cjae060 · Cited by 8
Shabil, M., Gaidhane, S., Ballal, S., Kumar, S., Bhat, M., Sharma, S., Kumar, M., Rustagi, S., Khatib, M. N., Rai, N., et al. (2024) · Pneumonia · doi:10.1186/s41479-024-00149-5 · Cited by 8
MAZEROLLE, L. G., Petersen, K., Sydes, M. L., & Ransley, J. (2025) · Cambridge University Press eBooks · doi:10.1017/9781009472029 · Cited by 10
Use and reporting of historical control data for regulatory studies
(PPR), E. P. O. P. P. P. A. T. R., Čoja, T., Adriaanse, P., Choi, J. M., Finizio, A., Giraudo, M., Kühl, T., Metruccio, F., Paparella, M., Pieper, S., et al. (2025) · EFSA Journal · doi:10.2903/j.efsa.2025.9576 · Cited by 10
Prevalence of polypharmacy among older adults with diabetes: A systematic review and Meta-Analysis
Satapathy, P., Gaidhane, A. M., Vadia, N., Menon, S. V., Chennakesavulu, K., Panigrahi, R., Shabil, M., Singh, M., Sah, S. K., Lingamaiah, D., et al. (2025) · Aging Clinical and Experimental Research · doi:10.1007/s40520-025-03240-z · Cited by 8
“Publication bias was evaluated using Doi plots and quantified with the Luis Furuya-Kanamori index, with val - ues exceeding ± 2 indicating major asymmetry [18, 19].”
Meta-analysis on reporting practices as a source of heterogeneity in in vitro cancer research
Sander, T., Ghanawi, J., Wilson, E., Muhammad, S., Macleod, M. R., & Kahlert, U. D. (2022) · BMJ Open Science · doi:10.1136/bmjos-2021-100272 · Cited by 6
Ranibizumab for the treatment of retinopathy of prematurity: systematic review and meta-analysis
Alva, N., Ricardo-Martínez, A., Ortiz-Saavedra, B., Montes-Madariaga, E. S., Cotrina-Santome, A., Caballero-Alvarado, J., Sah, R., & Barboza, J. J. (2023) · Frontiers in Pediatrics · doi:10.3389/fped.2023.1202927 · Cited by 8
Storm, L., & Tressoldi, P. E. (2023) · Journal of Scientific Exploration · doi:10.31275/20232967 · Cited by 13
“…by using the metafor package v. 3.8 (Viechtbauer, 2010), adopting the restricted maximum likelihood (REML) to estimate the heterogeneity variance (Langan et al., 2019) and the Hartung method to control effect-size non-normality (Rubio-Aparicio et al., 2018) and corresponding confidence intervals…”
MethodInfluentialOsseous implications of proton pump inhibitor therapy: An umbrella review
Alanazi, A. S., Almutairi, H., Gupta, J. K., Mohanty, D., Rath, D., AlOdan, A. A., Mahal, A., Khatib, M. N., Gaidhane, S., Zahiruddin, Q. S., et al. (2024) · Bone Reports · doi:10.1016/j.bonr.2024.101741 · Cited by 8
“The degree of heterogeneity among study findings was measured using I 2 and tau-squared metrics (Langan et al., 2019).”
MethodKorevaar, E., Turner, S., Forbes, A. B., Karahalios, A., Taljaard, M., & McKenzie, J. E. (2024) · BMC Medical Research Methodology · doi:10.1186/s12874-024-02147-z · Cited by 8
“REML has been recommended as an alternative estimator because of its slightly improved performance compared with DL, and for this reason was selected for evaluation in this study [29, 40, 41].”
Smokeless tobacco consumption among women of reproductive age: a systematic review and meta-analysis
Itumalla, R., Khatib, M. N., Gaidhane, S., Zahiruddin, Q. S., Gaidhane, A. M., Neyazi, A., Hassam, A. F., Satapathy, P., Rustagi, S., Kukreti, N., et al. (2024) · BMC Public Health · doi:10.1186/s12889-024-18840-z · Cited by 8
“The analysis also included a 95% prediction interval and the tau-squared value for a deeper understanding of between-study variance [40, 41].”
MethodEfficacy of letrozole for the treatment of tubal ectopic pregnancy: A meta-analysis
Laganà, A. S., Etrusco, A., Haydamous, J., Semaan, S., Agrifoglio, V., Chiantera, V., Vitagliano, A., Riemma, G., D’Amato, A., & Montagna, E. (2024) · European Journal of Obstetrics & Gynecology and Reproductive Biology · doi:10.1016/j.ejogrb.2024.06.043 · Cited by 8
Hussain, U., Kunwar, S. S., Khan, U. W., Alnazeh, A. A., Kamran, M. A., Alam, S., Aziz, A., Zaheen, M., Pandis, N., & Campobasso, A. (2024) · European Journal of Orthodontics · doi:10.1093/ejo/cjae040 · Cited by 8
Beyond Surgery: Psychological Well-Being's Role in Breast Reconstruction Outcomes
Foppiani, J., Lee, T. C., Alvarez, A. H., Escobar‐Domingo, M. J., Taritsa, I. C., Lee, D., Schuster, K. A., Wood, S., Utz, B., Bai, C., et al. (2024) · Journal of Surgical Research · doi:10.1016/j.jss.2024.10.040 · Cited by 8
Early Invasive or Conservative Strategies for Older Patients With Acute Coronary Syndromes
Reddy, R. K., Koeckerling, D., Eichhorn, C., Jamil, Y., Ardissino, M., Braun, V., Sharar, H. A., Frey, N., Howard, J. P., & Ahmad, Y. (2025) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2025.2058 · Cited by 9
Huang, H. (2019) · Measurement Science and Technology · doi:10.1088/1361-6501/ab46fd · Cited by 8
Bayesian inference for heterogeneity in meta-analysis
Bodnar, O., Nalule, R. M., & Possolo, A. M. (2020) · Metrologia · doi:10.1088/1681-7575/abb064 · Cited by 8
Riley, R. D., Hattle, M., Collins, P. G. S., Whittle, R., & Ensor, J. (2022) · Statistics in Medicine · doi:10.1002/sim.9538 · Cited by 8
“2 89 (28) 85 (16) 53 (11) 55 (11) 73 67”
Saran, A., Singh, S., Gupta, N., Walke, S. C., Rao, R., Simiyu, C., Malhotra, S. K., Mishra, A., Puskur, R., Masset, E., et al. (2022) · Campbell Systematic Reviews · doi:10.1002/cl2.1274 · Cited by 8
“Standard errors will be calculated to ensure that appropriate clustering by group is done, making adjustments where necessary using standard approaches to estimate the design effect (Langan et al., 2019; Waddington et al., 2012).”
MethodVisual Processing During the Interictal Period Between Migraines: A Meta-Analysis
Sezai, T., Murphy, M., Riddell, N., Nguyen, V., & Crewther, S. G. (2022) · Neuropsychology Review · doi:10.1007/s11065-022-09562-3 · Cited by 8
“…meta-analysis was conducted using the restricted maximum likelihood method as a robust analysis to account for underlying differences between studies, minimise bias in the analysis, and control for heterogeneity (Kalaian & Raudenbush, 1996; Kontopantelis & Reeves, 2012; Langan et al., 2019).”
InfluentialTsang, N. N. Y., So, H. C., Cowling, B. J., Leung, G. M., & Ip, D. (2022) · European Respiratory Review · doi:10.1183/16000617.0014-2022 · Cited by 8
RIMeta : An R shiny tool for estimating the reference interval from a meta‐analysis
Jiang, Z., Cao, W., Chu, H., Bazerbachi, F., & Siegel, L. K. (2023) · Research Synthesis Methods · doi:10.1002/jrsm.1626 · Cited by 8
“Similarly, Langan et al.(19) also compared nine commonly used methods and recommended the REML method to estimate the heterogeneity variance over the JIANG ET AL.”
MethodTressoldi, P. E., & Storm, L. (2021) · F1000Research · doi:10.12688/f1000research.24868.3 · Cited by 7
Brunner, M., Stallasch, S. E., & Lüdtke, O. (2022) · doi:10.35542/osf.io/39gbq · Cited by 8
McIsaac, D. I., Hutton, B., Veroniki, A., Gill, M., Boland, L., Branje, K., Grudzinski, A. L., Hladkowicz, E., Shaw, J., & Gillis, C. (2022) · BMJ Open · doi:10.1136/bmjopen-2022-068797 · Cited by 7
Self-citationManosroi, W., Atthakomol, P., Phinyo, P., & Danpanichkul, P. (2023) · Journal of Orthopaedics and Traumatology · doi:10.1186/s10195-023-00707-5 · Cited by 7
Appenzeller‐Herzog, C., Rosat, A., Mathes, T., Baroja‐Mazo, A., Chruscinski, A. J., Feng, S., Herrero, J. I., Londoño, M., Mazariegos, G. V., Ohe, H., et al. (2023) · Liver International · doi:10.1111/liv.15764 · Cited by 7
“17,18 In the second set of meta-analyses, random-effects dose–response meta-analysis with the pre-withdrawal time intervals using the lower quartile of the pre-withdrawal time in each study as reference category were performed.”
MethodDu, J., Liu, D., Zhou, W., Ye, T., Zhang, C., Qian, T., Zhang, J., & Zhang, Z. (2024) · Journal of Proteome Research · doi:10.1021/acs.jproteome.3c00630 · Cited by 7
Muñoz‐Carrillo, J. L., Cortés, N. R., Levano, S. E. T., Morán‐Mariños, C., & Barboza, J. J. (2024) · Journal of Clinical Medicine · doi:10.3390/jcm13247791 · Cited by 7
“…served as the control drug, with induction doses of 1–2.5 mg/kg and maintenance doses ranging from 4 to Regarding the features of designs, there were nine randomized controlled trials and prospective, double-[2,4,7–12] and single-blind [6], phase IIb/III [6,7], and phase III [2,4,8–12] trials.”
InfluentialAutier, P., Doi, G., Mullie, P., Vankrunkelsven, P., D’Ecclesiis, O., & Gandini, S. (2025) · PLoS ONE · doi:10.1371/journal.pone.0303316 · Cited by 7
Wu, W., Wu, W.-H., Lee, M., Wu, P.-Y., Tu, Y., Lin, H. H., Chan, C., & Huang, C. (2025) · Sports Medicine · doi:10.1007/s40279-025-02179-5 · Cited by 7
“The restricted maximum likelihood (REML) method was used to estimate between-study variance [55, 56].”
Grashuis, J. (2025) · Journal of Rural Studies · doi:10.1016/j.jrurstud.2025.103639 · Cited by 10
Sillassen, C. D. B., Petersen, J. J., Kamp, C. B., Grand, J., Domínguez, H., Frølich, A., Gæde, P., Gluud, C. N., Mathiesen, O., & Jakobsen, J. C. (2025) · BMJ Open · doi:10.1136/bmjopen-2024-094947 · Cited by 8
Satapathy, P., Gaidhane, A. M., Vadia, N., Menon, S. V., Chennakesavulu, K., Panigrahi, R., Sah, S. K., Tiwari, S., Rao, S. G., Goh, K. W., et al. (2025) · JGH Open · doi:10.1002/jgh3.70190 · Cited by 7
Hutchison, J. E., Tabrizian, B. I., Bajnauth, N., Avdellas, A., Phillips, D., & Lyons, I. M. (2025) · Journal of Numerical Cognition · doi:10.5964/jnc.14335 · Cited by 8
Systematic Review and Network Meta-Analysis of Initial Treatments for Lupus Nephritis
Izcovich, A., Tortosa, F. G., Bengolea, A., Pissinis, M. M., Ragusa, M. A., Fielli, M., Agnoletti, C., Quintana, R., Malvar, A., Scolnik, M., et al. (2025) · Kidney International Reports · doi:10.1016/j.ekir.2025.06.047 · Cited by 7
Metadata
- Authors
- Langan, D., Higgins, J. P. T., Jackson, D., Bowden, J., Veroniki, A. A., Kontopantelis, E., Viechtbauer, W., & Simmonds, M.
- Affiliations
- University of Manchester (GB)
- University of Bristol (GB)
- AstraZeneca (United Kingdom) (GB)
- St. Michael's Hospital (CA)
- Maastricht University (NL)
- University of York (GB)
- Corresponding author
- Langan, Dean
- ORCID
- Venue
- Research Synthesis Methods
- Publisher
- Wiley
- Published
- 2018-09-06
- Volume
- 10
- Issue
- 1
- Pages
- 83-98
- ISSN
- 1759-2879 (ISSN-L)
- 1759-2887
- OpenAlex
- W2885675882
- Semantic Scholar
- 264ababb39dc4b8737425a30c9fb75c5d8eb2bf4
- Cited by
- 1,074
- References
- 51
- Citations by year
- 2026: 214 · 2025: 173 · 2024: 198 · 2023: 144 · 2022: 140 · 2021: 113 · 2020: 60 · 2019: 28 · 2018: 2
- FWCI
- 38.65
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Research Synthesis Methods (publishedVersion)
- PubMed (publishedVersion)
- Research Publications (Maastricht University) (publishedVersion, other-oa)
- Research Explorer (The University of Manchester) (publishedVersion, other-oa)
- White Rose Research Online (University of Leeds, The University of Sheffield, University of York) (submittedVersion)
- Bristol Research (University of Bristol) (submittedVersion, other-oa)
- UCL Discovery (University College London) (submittedVersion)
- Research Publications (Maastricht University) (submittedVersion, other-oa)
- Funders
- National Institute for Health and Care Research (NF-SI-0617-10145)
- Medical Research Council (MC_UU_00011/2, MR/K006665/1)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Keywords
- random-effects meta-analysis, estimation bias, binary outcomes, confidence interval coverage
- MeSH terms
- Systematic Reviews as Topic, Algorithms, Analysis of Variance, Computer Simulation, Data Interpretation, Statistical, Humans, Probability, Research Design, Software, Meta-Analysis as Topic, Reproducibility of Results, Models, Statistical, Selection Bias, Likelihood Functions, Odds Ratio
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
