Cited by 453Meta-analysis and systematic reviewsStatistical Methods in EpidemiologyDelphi Technique in Research

The DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.

Random-Effects Meta-analysis of Inconsistent Effects: A Time for Change

Cornell, John E., Mulrow, Cynthia D., Localio, Russell, Stack, Catharine B., Meibohm, Anne R., Guallar, Eliseo, Goodman, Steven N.·2014·Annals of Internal Medicine

DOI 10.7326/m13-2886

Open this paper in CoChat — free account

Abstract

A primary goal of meta-analysis is to improve the estimation of treatment effects by pooling results of similar studies. This article explains how the most widely used method for pooling heterogeneous studies--the Der Simonian-Laird (DL) estimator--can produce biased estimates with falsely high precision. A classic example is presented to show that use of the DL estimator can lead to erroneous conclusions. Particular problems with the DL estimator are discussed, and several alternative methods for summarizing heterogeneous evidence are presented. The authors support replacing universal use of the DL estimator with analyses based on a critical synthesis that recognizes the uncertainty in the evidence,focuses on describing and explaining the probable sources of variation in the evidence, and uses random-effects estimates that provide more accurate confidence limits than the DL estimator.

Integrity signals

Registry

Crossref · 10.7326/m13-2886

Cites

28 of 28 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Meta-analysis in clinical trials
DerSimonian, R., & Laird, N. (1986) · Controlled Clinical Trials · doi:10.1016/0197-2456(86)90046-2 · Cited by 39,552
Quantifying heterogeneity in a meta‐analysis
Higgins, J. P. T., & Thompson, S. G. (2002) · Statistics in Medicine · doi:10.1002/sim.1186 · Cited by 36,763
Conducting Meta-Analyses in R with the metafor Package
Viechtbauer, W. (2010) · Journal of Statistical Software · doi:10.18637/jss.v036.i03 · Cited by 20,464
A Re-Evaluation of Random-Effects Meta-Analysis
Higgins, J. P. T., Thompson, S. G., & Spiegelhalter, D. J. (2009) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/j.1467-985x.2008.00552.x · Cited by 2,569
Random-effects model for meta-analysis of clinical trials: An update
DerSimonian, R., & Kacker, R. (2007) · Contemporary Clinical Trials · doi:10.1016/j.cct.2006.04.004 · Cited by 2,374
Improved tests for a random effects meta‐regression with a single covariate
Knapp, G., & Hartung, J. (2003) · Statistics in Medicine · doi:10.1002/sim.1482 · Cited by 2,075
Meta‐Analysis Of Controlled Clinical Trials
Whitehead, A. (2002) · doi:10.1002/0470854200 · Cited by 827
A comparison of statistical methods for meta‐analysis
Brockwell, S. E., & Gordon, I. R. (2001) · Statistics in Medicine · doi:10.1002/sim.650 · Cited by 573
BORROWING STRENGTH FROM EXTERNAL TRIALS IN A META-ANALYSIS
Higgins, J. P. T., & WHITEHEAD, A. (1996) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19961230)15:24 · Cited by 559
A LIKELIHOOD APPROACH TO META-ANALYSIS WITH RANDOM EFFECTS
Hardy, R., & Thompson, S. G. (1996) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19960330)15:6 · Cited by 522

Flagged first, then by citations

Show all 28 references

Research works printed but not linked: 1

Weseley AC, 1962, Obstet Gynecol, 19, 355

Sorted by citations

Items 201–300 of 452

Protease activity as a prognostic factor for wound healing in venous leg ulcers
Westby, M. J., Dumville, J. C., Stubbs, N. C., Norman, G., Wong, J. K. F., Cullum, N. A., & Riley, R. D. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012841 · Cited by 31
Assessing and visualizing fragility of clinical results with binary outcomes in R using the fragility package
Lin, L., & Chu, H. (2022) · PLoS ONE · doi:10.1371/journal.pone.0268754 · Cited by 31
“, the restricted maximum likelihood (REML) estimator, have been shown to perform better in general [67, 68].”
Background
Comparison of Severity of Genitourinary Syndrome of Menopause Symptoms After Carbon Dioxide Laser vs Vaginal Estrogen Therapy
Jang, Y.-C., Leung, C. Y., & Huang, H.-L. (2022) · JAMA Network Open · doi:10.1001/jamanetworkopen.2022.32563 · Cited by 30
Pre-emptive correction for haemodialysis arteriovenous access stenosis
Ravani, P., Quinn, R. R., Oliver, M. J., Karsanji, D. J., James, M. T., MacRae, J. M., Palmer, S. C., & Strippoli, G. F. (2016) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd010709.pub2 · Cited by 29
Meta-Analysis of Normal Reference Values for Right and Left Ventricular Quantification by Cardiovascular Magnetic Resonance
Zhan, Y., Friedrich, M. G., Dendukuri, N., Lu, Y., Chetrit, M., Schiller, I., Joseph, L., Shaw, J. L., Chuang, M. L., Riffel, J. H., et al. (2024) · Circulation Cardiovascular Imaging · doi:10.1161/circimaging.123.016090 · Cited by 28
“We used a Bayesian approach as it is known to be advantageous for modeling heterogeneity in a meta-analysis 5 and for handling complex models 6 (arising in this case due to the use of 2 different meta-analysis models and the need to report the uncertainty around a combined result).”
Method
Hyaluronic acid injection therapy for osteoarthritis of the knee: concordant efficacy and conflicting serious adverse events in two systematic reviews
O’Hanlon, C. E., Newberry, S. J., Booth, M. S., Grant, S., Motala, A., Maglione, M., FitzGerald, J. D., & Shekelle, P. (2016) · Systematic Reviews · doi:10.1186/s13643-016-0363-9 · Cited by 28
“We conducted meta-analysis of the efficacy outcomes of interest in cases where there were three or more sufficiently homogeneous studies and estimated a pooled random-effects estimate of the overall effect size [12].”
Method
Which patients benefit specifically from short-term psychodynamic psychotherapy (STPP) for depression? Study protocol of a systematic review and meta-analysis of individual participant data
Driessen, E., Abbass, A. A., Barber, J. P., Gibbons, M. B. C., Dekker, J., Fokkema, M., Fonagy, P., Hollon, S. D., Jansma, E. P., de Maat, S. C. M., et al. (2018) · BMJ Open · doi:10.1136/bmjopen-2017-018900 · Cited by 28
“We will apply the Kenward-Roger denominator-degrees-of-freedom adjustment for CIs,(29) because it more adequately accounts for uncertainty than the standard Dersimonian-Laird estimator when the number of studies is small or when heterogeneity is present.(30) To account for clustering of patients within studies and to…”
Method
Risk scores for occult cancer in patients with unprovoked venous thromboembolism: Results from an individual patient data meta‐analysis
Mulder, F. I., Carrier, M., van Doormaal, F. F., Robin, P., Otten, H.-M. M. B., Salaün, P., Buller, H. R., Gal, G. L., & van Es, N. (2020) · Journal of Thrombosis and Haemostasis · doi:10.1111/jth.15001 · Cited by 27
Physical health-related quality of life at higher achieved hemoglobin levels among chronic kidney disease patients: a systematic review and meta-analysis
Guedes, M. H., Guetter, C. R., Erbano, L. H. O., Palone, A. G., Zee, J., Robinson, B. M., Pisoni, R. L., de Moraes, T. P., Pecoits‐Filho, R., & Baena, C. P. (2020) · BMC Nephrology · doi:10.1186/s12882-020-01912-8 · Cited by 33
The Effect of Telehealth Interventions on Function and Quality of Life for Older Adults with Pre-Frailty or Frailty: A Systematic Review and Meta-Analysis
Esfandiari, E., Miller, W. C., & Ashe, M. C. (2021) · Journal of Applied Gerontology · doi:10.1177/0733464820983630 · Cited by 28
“We used Review Manager (RevMan) Version 5.3 (Copenhagen, The Nordic Cochrane Center, The Cochrane Collaboration, 2014) for meta-analyses.”
MethodInfluential
Physical Activity and Bone Health in Men: A Systematic Review and Meta-Analysis
Ashe, M. C., dos Santos, Í. K., Edward, N. Y., Burnett, L. A., Barnes, R., Fleig, L., Puyat, J. H., Sale, J. E. M., McKay, H. A., & Giangregorio, L. (2021) · Journal of Bone Metabolism · doi:10.11005/jbm.2021.28.1.27 · Cited by 28
“[20] The effect estimates were computed using the DerSimonian-Laird method with the Knapp-Hartung small-sample adjustments.[21] We used R (version 4.”
Method
Deriving percentage study weights in multi-parameter meta-analysis models: with application to meta-regression, network meta-analysis and one-stage individual participant data models
Riley, R. D., Ensor, J., Jackson, D., & Burke, D. L. (2017) · Statistical Methods in Medical Research · doi:10.1177/0962280216688033 · Cited by 28
“…assume variance estimates are ‘known’ or, post-estimation, inflate confidence intervals for β̂ to account for uncertainty in variance estimates.49 For the latter situation, our approach should be valid as it focuses on decomposing the observed information toward β̂ itself, which does not…”
High Positive End-expiratory Pressure (PEEP) with Recruitment Maneuvers versus Low PEEP during General Anesthesia for Surgery: A Bayesian Individual Patient Data Meta-analysis of Three Randomized Clinical Trials
Mazzinari, G., Zampieri, F. G., Ball, L., Campos, N. S., Bluth, T., Hemmes, S. N. T., Ferrando, C., Librero, J., Soro, M., Pelosi, P. P., et al. (2024) · Anesthesiology · doi:10.1097/aln.0000000000005170 · Cited by 28
“28 Our results showed that (1) using high PEEP and recruitment maneuver led to a slight reduction in the posterior probability of complications occurrence; (2) this benefit was highly likely in general, although (3) it can be less likely to benefit depending on the variability of patients.”
Prostate Cancer Screening Patient Decision Aids: A Systematic Review and Meta-analysis
Ivlev, I., Jeřábková, S., Mishra, M., Cook, L. A., & Eden, K. (2018) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2018.06.016 · Cited by 27
Efficacy and safety of sodium-glucose cotransporter 2 inhibitors in patients with type 2 diabetes and moderate renal function impairment: A systematic review and meta-analysis
Zhang, L., Zhang, M., Lv, Q., & Tong, N. (2018) · Diabetes Research and Clinical Practice · doi:10.1016/j.diabres.2018.03.047 · Cited by 27
Antipsychotics for the Prevention and Treatment of Delirium
Neufeld, K. J., Needham, D. M., Oh, E. S., Wilson, L. M., Nikooie, R., Zhang, A. W., Koneru, M., Balagani, A., Singu, S., Aldabain, L., et al. (2019) · doi:10.23970/ahrqepccer219 · Cited by 27
Late Postoperative Complications in Laparoscopic Sleeve Gastrectomy (LVSG) Versus Laparoscopic Roux-en-y Gastric Bypass (LRYGB): Meta-analysis and Systematic Review
Osland, E., Yunus, R. M., Khan, S., Memon, B., & Memon, M. A. (2016) · Surgical Laparoscopy, Endoscopy & Percutaneous Techniques · doi:10.1097/sle.0000000000000279 · Cited by 26
Metformin and health outcomes: An umbrella review of systematic reviews with meta‐analyses
Li, X., Celotto, S., Pizzol, D., Gašević, D., Ji, M., Barnini, T., Solmi, M., Stubbs, B., Smith, L., Sánchez, G. F. L., et al. (2021) · European Journal of Clinical Investigation · doi:10.1111/eci.13536 · Cited by 27
Assessment of Heterogeneity in Meta-analyses
Hoaglin, D. C. (2014) · JAMA · doi:10.1001/jama.2014.14346 · Cited by 26
Effects of yoga on stress in stressed adults: a systematic review and meta-analysis
Schleinzer, A., Moosburner, A., Anheyer, D., Burgahn, L., & Cramer, H. (2024) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2024.1437902 · Cited by 41
“Additionally, the Hartung-Knapp small-sample correction was applied to account for uncertainty in pooling treatment effects from a limited number of heterogeneous studies (32 – 35).”
Risk prediction models for falls in hospitalized older patients: a systematic review and meta-analysis
Mao, A., Su, J., Ren, M., Chen, S., & Zhang, H. (2025) · BMC Geriatrics · doi:10.1186/s12877-025-05688-0 · Cited by 29
Effects of high‐intensity interval training on depressive and anxiety symptoms in healthy individuals: A systematic review and meta‐analysis of randomized clinical trials
Gaia, J. W. P., Schuch, F. B., Ferreira, R. W., Souza, E. D. L., Ferreira, V. M. S., & Pires, D. A. (2024) · Scandinavian Journal of Medicine & Science in Sports · doi:10.1111/sms.14618 · Cited by 26
“30,35,36 The SMD was calculated for each group based on the mean difference (mean change), standard deviation (SD), and sample size for each study condition to compare the interventions.”
The effects of anticholinergic medications on cognition in children: a systematic review and meta-analysis
Ghezzi, E. S., Chan, M., Ellett, L. M. K., Ross, T. J., Richardson, K., Ho, J. N., Copley, D., Steele, C., & Keage, H. A. D. (2021) · Scientific Reports · doi:10.1038/s41598-020-80211-6 · Cited by 24
Effectiveness of bile acids as a feed supplement to improve growth performance, feed utilization, lipid metabolism, digestive enzymes, and hepatic antioxidant status in aquaculture animals: A meta-analysis
Li, L., Liu, T., Li, J., Yang, Y., Liu, H., & Zhang, P. (2024) · Aquaculture Reports · doi:10.1016/j.aqrep.2024.102121 · Cited by 26
Should persons with acute myeloid leukemia have a transplant in first remission?
Gale, R. P., Wiernik, P. H., & Lazarus, H. M. (2014) · Leukemia · doi:10.1038/leu.2014.129 · Cited by 24
We know less than we should about methods of meta‐analysis
Hoaglin, D. C. (2015) · Research Synthesis Methods · doi:10.1002/jrsm.1146 · Cited by 24
“A survey of current software would probably find that the DerSimonian-Laird method is generally the default option for (frequentist) random-effects meta-analysis, despite accumulated evidence of its shortcomings and forceful calls for change (Cornell et al., 2014; IntHout et al., 2014).”
Method
Estimating the reference range from a meta‐analysis
Siegel, L. K., Murad, M. H., & Chu, H. (2020) · Research Synthesis Methods · doi:10.1002/jrsm.1442 · Cited by 24
Fixed-effect versus random-effect model in meta-analysis: How to decide?
Maitra, S. (2025) · Indian Journal of Anaesthesia · doi:10.4103/ija.ija_1203_24 · Cited by 24
“[16] Though the choice of the random-effect estimator depends upon several factors, a restricted maximum likelihood estimator is generally preferable.”
Non‐genetic biomarkers and colorectal cancer risk: Umbrella review and evidence triangulation
Zhang, X., Gill, D., He, Y., Yang, T., Li, X., Monori, G., Campbell, H. J., Dunlop, M. G., Tsilidis, K. K., Timofeeva, M. N., et al. (2020) · Cancer Medicine · doi:10.1002/cam4.3051 · Cited by 22
Effect of Anticoagulant Administration on the Mortality of Hospitalized Patients With COVID-19: An Updated Systematic Review and Meta-Analysis
Jiang, L., Li, Y., Du, H., Qin, Z., & Su, B. (2021) · Frontiers in Medicine · doi:10.3389/fmed.2021.698935 · Cited by 25
Malnutrition in Hospitalized Adults: A Systematic Review
Uhl, S., Siddique, S. M., McKeever, L., Bloschichak, A. M., D’Anci, K. E., Leas, B. F., Mull, N. K., & Tsou, A. Y. (2021) · doi:10.23970/ahrqepccer249 · Cited by 26
Hydrochlorothiazide and alternative diuretics versus renin–angiotensin system inhibitors for the regression of left ventricular hypertrophy
Roush, G. C., Abdelfattah, R. S., Song, S., Kostis, J. B., Ernst, M. E., & Sica, D. A. (2018) · Journal of Hypertension · doi:10.1097/hjh.0000000000001691 · Cited by 22
Associations between light at night and mental health: A systematic review and meta-analysis
Deprato, A., Haldar, P., Navarro, J. F., Harding, B. N., Lacy, P., Maidstone, R. J., Moitra, S., Cros, A. P., Durrington, H. J., Kogevinas, M., et al. (2025) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2025.179188 · Cited by 26
Feasibility and safety of image‐guided vacuum‐assisted breast biopsy: A PRISMA‐compliant systematic review and meta‐analysis of 20 000 population from 36 longitudinal studies
Fang, M., Liu, G., Luo, G., & Wu, T. (2019) · International Wound Journal · doi:10.1111/iwj.13224 · Cited by 21
Bayesian Methods for Meta-Analyses of Binary Outcomes: Implementations, Examples, and Impact of Priors
Amer, F. M. A., Thompson, C. G., & Lin, L. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18073492 · Cited by 22
“For example, the frequentist DL, ML, and REML methods estimated the overall OR as 4.59, 4.52, and 4.47, respectively, and ?”
MethodInfluential
The effectiveness and safety of treatments used for acute diarrhea and acute gastroenteritis in children: protocol for a systematic review and network meta-analysis
Flórez, I. D., Khalifah, R. A. A., Sierra, J. M., Granados, C., Yepes-Núñez, J. J., Cuello‐García, C. A., Pérez‐Gaxiola, G., Zea, A. M., Hernández, G., Veroniki, A. A., et al. (2016) · Systematic Reviews · doi:10.1186/s13643-016-0186-8 · Cited by 23
“The RE models assume that the observed treatment effect for a study is a combination of a treatment effect common to all studies plus a component specific to that study alone [32].”
A comparison of analytic approaches for individual patient data meta-analyses with binary outcomes
Thomas, D., Platt, R. W., & Benedetti, A. (2017) · BMC Medical Research Methodology · doi:10.1186/s12874-017-0307-7 · Cited by 21
“found that the DL method produced too-narrow confidence bounds and p values that were too small when the number of studies was small or there was high between-study heterogeneity [30].”
Method
Methodological Aspects of Meta-Analyses Assessing the Effect of Blood Pressure–Lowering Treatment on Clinical Outcomes
Brunström, M., Thomopoulos, C., Carlberg, B., Kreutz, R., & Mancia, G. (2021) · Hypertension · doi:10.1161/hypertensionaha.121.18413 · Cited by 21
Risk of Transmitting Coronavirus Disease 2019 During Nebulizer Treatment: A Systematic Review
Goldstein, K. M., Ghadimi, K., Mystakelis, H. A., Kong, Y., Meng, T., Cantrell, S. E., von Isenburg, M., Gordon, A. M., Ear, B., Gierisch, J. M., et al. (2021) · Journal of Aerosol Medicine and Pulmonary Drug Delivery · doi:10.1089/jamp.2020.1659 · Cited by 20
“(58) We identiﬁed several gaps in the existing literature (see Table 5 for a modiﬁed gap framework.”
Tests for publication bias are unreliable in case of heteroscedasticity
Almalik, O., Zhan, Z. Z., & van den Heuvel, E. R. (2021) · Contemporary Clinical Trials Communications · doi:10.1016/j.conctc.2021.100781 · Cited by 20
“However, the DerSimonian-Laird estimate has been found to underestimate the between-study variance estimate, and thereby producing narrow confidence intervals for the mean treatment effect [25].”
Background
A Bayesian approach to assessing small‐study effects in meta‐analysis of a binary outcome with controlled false positive rate
Shi, L., Chu, H., & Lin, L. (2020) · Research Synthesis Methods · doi:10.1002/jrsm.1415 · Cited by 22
A meta-analysis of teacher well-being: A job demands and resources perspective
Li, Y., Chen, J., Lu, J., & Wang, X. (2025) · Educational Research Review · doi:10.1016/j.edurev.2025.100719 · Cited by 19
Efficacy and safety of steroid-sparing treatments in giant cell arteritis according to the glucocorticoids tapering regimen: A systematic review and meta-analysis
Gerard, A.-L., Simon‐Tillaux, N., Yordanov, Y., Cacoub, P., Tubach, F., Saadoun, D., & Dechartres, A. (2021) · European Journal of Internal Medicine · doi:10.1016/j.ejim.2021.03.040 · Cited by 19
Methodological systematic review recommends improvements to conduct and reporting when meta-analyzing interrupted time series studies
Korevaar, E., Karahalios, A., Turner, S., Forbes, A. B., Taljaard, M., Cheng, A. C., Grimshaw, J. M., Bero, L., & McKenzie, J. E. (2022) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2022.01.010 · Cited by 19
Interventions for malnutrition in hospitalized adults: a systematic review and meta-analysis
Uhl, S., Siddique, S. M., Bloschichak, A., McKeever, W., D'Anci, K., Leas, B., Mull, N. K., Compher, C., Lewis, J. D., Wu, G. D., et al. (2022) · Journal of Hospital Medicine · doi:10.1002/jhm.12891 · Cited by 19
Management of chronic neuropathic pain: a protocol for a multiple treatment comparison meta-analysis of randomised controlled trials
Mulla, S. M., Buckley, D. N., Moulin, D. E., Couban, R., Izhar, Z., Agarwal, A., Panju, A. A., Wang, L., Makosso‐Kallyth, S., Turan, A., et al. (2014) · BMJ Open · doi:10.1136/bmjopen-2014-006112 · Cited by 19
“Recent methodological research has shown that while popular, the DerSimonian-Laird method(67) can produce narrow CIs when the number of studies is small or when they are substantively heterogeneous.(68) 69 Therefore, to pool outcome data for trials that make direct comparisons between interventions and alternatives, w…”
Potentially unnecessary and wasteful clinical trial research detected in cumulative meta-epidemiological and trial sequential analysis
Storz-Pfennig, P. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2016.11.003 · Cited by 19
The Effects of Walking or Nordic Walking in Adults 50 Years and Older at Elevated Risk of Fractures: A Systematic Review and Meta-Analysis
Rodrigues, I. B., Ponzano, M., Butt, D. A., Bartley, J., Bardai, Z., Ashe, M. C., Chilibeck, P. D., Thabane, L., Wark, J. D., Stapleton, J. E., et al. (2021) · Journal of Aging and Physical Activity · doi:10.1123/japa.2020-0262 · Cited by 19
Screening Coverage Needed to Reduce Mortality from Prostate Cancer: A Living Systematic Review
Rahal, A. K., Badgett, R. G., & Hoffman, R. M. (2016) · PLoS ONE · doi:10.1371/journal.pone.0153417 · Cited by 18
“We conducted meta-analysis with the Dersimonian-Laird inverse variance estimator using the Hartung-Knapp adjustment for calculating relative risk [18].”
The Impact of Placebo Response Rates on Clinical Trial Outcome: A Systematic Review and Meta-Analysis of Antidepressants in Children and Adolescents with Major Depressive Disorder
Li, Y., Huang, J., He, Y., Yang, J., Lv, Y., Liu, H., Liang, L., Li, H., Zheng, Q., & Li, L. (2019) · Journal of Child and Adolescent Psychopharmacology · doi:10.1089/cap.2019.0022 · Cited by 18
“Random effects models were used to estimate the effect size by the ‘‘random’’ command in Stata (Cornell et al. 2014).”
Influential
Machine learning models for risk prediction of cancer-associated thrombosis: a systematic review and meta-analysis
Chen, K., Zhang, Y., Zhang, L., Zhang, W., & Chen, Y. (2024) · Journal of Thrombosis and Haemostasis · doi:10.1016/j.jtha.2024.11.001 · Cited by 20
A Systematic Review of the Single-Item Burnout Question: Its Reliability Depends on Your Purpose
Hagan, G., Okut, H., & Badgett, R. G. (2024) · Journal of General Internal Medicine · doi:10.1007/s11606-024-08685-y · Cited by 18
“31 Random effects meta-analyses with the Hartung and Knapp approach 32 were done with the metacor function of the R package meta, 33 while the mixed effects analysis for creating the conversion equation was done with the lmer function of the package lme4.”
An approach to addressing subpopulation considerations in systematic reviews: the experience of reviewers supporting the U.S. Preventive Services Task Force
Whitlock, E. P., Eder, M. L., Thompson, J. H., Jonas, D. E., Evans, C. V., Guirguis‐Blake, J., & Lin, J. S. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0437-3 · Cited by 18
“decision to mathematically combine data depends on critical judgment [31].”
Background
Effects of antihypertensives, lipid-modifying drugs, glycaemic control drugs and sodium bicarbonate on the progression of stages 3 and 4 chronic kidney disease in adults: a systematic review and meta-analysis
Taylor, K., McLellan, J., Verbakel, J. Y. J., Aronson, J. K., Lasserson, D. S., Pidduck, N., Roberts, N. W., Fleming, S., O’Callaghan, C. A., Bankhead, C. R., et al. (2019) · BMJ Open · doi:10.1136/bmjopen-2019-030596 · Cited by 17
Effectiveness of Astaxanthin as a Feed Supplement to Improve Growth Performance and Feed Utilization in Aquaculture Animals: A Meta-Analysis
Li, B., Chen, C., Zhou, X., Liu, H., Zhou, Z., Wang, X., Liang, J., Guo, Y., & Liang, S. (2025) · Antioxidants · doi:10.3390/antiox14050609 · Cited by 18
“A random effects model was used to combine the effect size to deal with the heterogeneity of species differences and experimental cycles [30], which was realized by the rma function of the metafor package.”
Apolipoprotein E-ε4 allele predicts escalation of psychotic symptoms in late adulthood
Jonas, K. G., Clouston, S., Li, K., Fochtmann, L. J., Lencz, T., Malhotra, A. K., Cicero, D. C., Perlman, G., Bromet, E. J., & Kotov, R. (2018) · Schizophrenia Research · doi:10.1016/j.schres.2018.12.010 · Cited by 16
“Random-effects allow for differences among effects due random between-study differences (Cornell et al., 2014).”
Background
Postpartum NSAID Use and Adverse Outcomes among Women with Hypertensive Disorders of Pregnancy: A Systematic Review and Meta-analysis
Premkumar, A., Ayala, N. K., Miller, C. H., Grobman, W. A., & Miller, E. S. (2020) · American Journal of Perinatology · doi:10.1055/s-0040-1713180 · Cited by 16
Is meta-analysis of RCTs assessing the efficacy of interventions a reliable source of evidence for therapeutic decisions?
Maziarz, M. (2021) · Studies in History and Philosophy of Science Part A · doi:10.1016/j.shpsa.2021.11.007 · Cited by 17
Lies, damned lies and statistics: Clinical importance versus statistical significance in research
Mellis, C. (2017) · Paediatric Respiratory Reviews · doi:10.1016/j.prrv.2017.02.002 · Cited by 17
Ten interleukins and risk of prostate cancer
Li, B.-H., Yan, S., Luo, L.-S., Zeng, X., Wang, Y., & Wang, X. (2023) · Frontiers in Oncology · doi:10.3389/fonc.2023.1108633 · Cited by 16
The effectiveness of pharmacological and lifestyle interventions to reduce the risk of diabetes and hyperglycaemia following gestational diabetes: A systematic review and meta‐analysis
Dennison, R. A., Oliver‐Williams, C., Qi, H. L. J., Kotecha, D., Seed, L. M., Ward, R. J., & Griffin, S. J. (2024) · Diabetic Medicine · doi:10.1111/dme.15316 · Cited by 17
“As the DerSimonian–Laird method for random effects meta-analysis may have statistical limitations in the case of few studies, 29 we reran all meta-analyses with less than 10 studies with fixed-effects models to assess the consistency of the results and provide an overall estimation of the…”
Method
Mid- and Long-Term Outcome Comparisons of Everolimus-Eluting Bioresorbable Scaffolds Versus Everolimus-Eluting Metallic Stents
Zhang, X., Zhu, Q., Kang, L., Li, X., & Xu, B. (2017) · Annals of Internal Medicine · doi:10.7326/m17-1101 · Cited by 15
“The profile likelihood method was chosen because it captures the uncertainty associated with statistical heterogeneity better than the conventional DerSimonian–Laird method (30).”
Renal effectiveness and safety of the sodium-glucose cotransporter-2 inhibitors: a population-based cohort study
Alkabbani, W., Zongo, A., Minhas‐Sandhu, J. K., Eurich, D. T., Shah, B. R., Alsabbagh, M. W., & Gamble, J. (2021) · BMJ Open Diabetes Research & Care · doi:10.1136/bmjdrc-2021-002496 · Cited by 16
Proton Pump Inhibitors’ Use and Risk of Iron Deficiency Anaemia: A Systematic Review and Meta-analysis
Ali, M. D. (2022) · Current Clinical Pharmacology · doi:10.2174/2772432817666220307121220 · Cited by 14
Effects of the COVID-19 pandemic on people experiencing incarceration: a systematic review
Williams, D., Spinks, B., Williams, D., Lewis, R. A., Bull, F., & Edwards, A. (2024) · BMJ Open · doi:10.1136/bmjopen-2023-076451 · Cited by 14
Measurement reliability of automated oscillometric blood pressure monitor in the elderly with atrial fibrillation: a systematic review and meta-analysis
Park, S.-H., & Choi, Y.-K. (2020) · Blood Pressure Monitoring · doi:10.1097/mbp.0000000000000414 · Cited by 14
The effectiveness and safety of treatments used for polycystic ovarian syndrome management in adolescents: a systematic review and network meta-analysis protocol
Khalifah, R. A. A., Flórez, I. D., Dennis, B. B., Neupane, B., Thabane, L., & Bassilious, E. (2015) · Systematic Reviews · doi:10.1186/s13643-015-0105-4 · Cited by 14
“We will pool all direct evidence using random-effect metaanalysis with the maximal likelihood (ML) estimator [39].”
Method
Systematic review and meta-analysis of respiratory viral triggers for acute myocardial infarction and stroke
Nguyen, T. Q., Vlasenko, D., Shetty, A. N., Zhao, E., Reid, C. M., Clothier, H. J., & Buttery, J. P. (2025) · Cardiovascular Research · doi:10.1093/cvr/cvaf092 · Cited by 13
“20 For exposure–outcome pairs with 3 studies or due to statistical heterogeneity, effect estimate data were displayed in forest plots (i.e. without the summary diamond), including 95% confidence intervals, for visual comparison across studies.”
Autobiographical memory style and clinical outcomes following mindfulness-based cognitive therapy (MBCT): An individual patient data meta-analysis
Hitchcock, C., Rudokaite, J., Haag, C., Patel, S. D., Smith, A., Kuhn, I. L., Jermann, F., Ma, S. H., Kuyken, W., Williams, J., et al. (2022) · Behaviour Research and Therapy · doi:10.1016/j.brat.2022.104048 · Cited by 13
The Benefits and Risks of Receiving Investigational Solid Tumor Drugs in Randomized Trials
Iskander, R., Moyer, H., Fergusson, D. A., McGrath, S., Benedetti, A., & Kimmelman, J. (2024) · Annals of Internal Medicine · doi:10.7326/m23-2515 · Cited by 13
“We used the DerSimonian – Laird estimator of the between-study variance and the Knapp – Hartung adjustment for the con ﬁ dence intervals and P values around the pooled HR (26 – 28).”
Method
Yoga for Depressive Disorder: A Systematic Review and Meta‐Analysis
Moosburner, A., Cramer, H., Bîlc, M. I., Triana, J., & Anheyer, D. (2024) · Depression and Anxiety · doi:10.1155/da/6071055 · Cited by 22
“It offers a more appropriate handling of uncertainty when consolidating treatment effects from a limited number of heterogeneous studies [38 – 41].”
Five comparative cohorts to assess the risk of genital tract infections associated with sodium‐glucose cotransporter‐2 inhibitors initiation in type 2 diabetes mellitus
Alkabbani, W., Zongo, A., Minhas‐Sandhu, J. K., Eurich, D. T., Shah, B. R., Alsabbagh, M. W., & Gamble, J. (2022) · Diabetic Medicine · doi:10.1111/dme.14858 · Cited by 12
Impact of Ramadan fasting on kidney function and related outcomes in patients with chronic kidney disease: a systematic review protocol
Bello, A. K., Kurzawa, J., Osman, M. A., Olah, M. E., Lloyd, A. M., Wiebe, N., Habib, S., Qarni, U., Shojai, S., & Pauly, R. P. (2019) · BMJ Open · doi:10.1136/bmjopen-2018-022710 · Cited by 13
Adjustment for compliance behavior in trials of epidural analgesia in labor using instrumental variable meta-analysis
Bannister-Tyrrell, M., Miladinović, B., Roberts, C. L., & Ford, J. B. (2014) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2014.11.005 · Cited by 13
“Future metaanalyses contrasting ITT and IV approaches should also explore whether IV estimates reduce statistical heterogeneity, as although heterogeneity was not detected in this study, noncompliance and contamination may contribute to heterogeneity and there is increasing recognition of the need to adequately explor…”
Background
Fixed-effects model: the most convincing model for meta-analysis with few studiesPreprint
Lin, E., Tong, T., Chen, Y., & Wang, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.04211 · Cited by 24
“According to Davey et al. (2011) with a total of 22,453 meta-analyses from the January 2008 Issue of the Cochrane Database of Systematic Reviews, the median number of studies included in each meta-analysis is only three.”
Post-stroke epilepsy and risk of all-cause mortality: A systematic review and meta-analysis of cohort studies
Ren, Z., Wen, Q., Yan, X., Wang, Y., & Zhang, Y. (2022) · Clinical Neurology and Neurosurgery · doi:10.1016/j.clineuro.2022.107362 · Cited by 12
A penalization approach to random‐effects meta‐analysis
Wang, Y., Lin, L., Thompson, C. G., & Chu, H. (2021) · Statistics in Medicine · doi:10.1002/sim.9261 · Cited by 12
Early studies reported extreme findings with large variability: a meta-epidemiologic study in the field of endocrinology
Wang, Z., Alahdab, F., Almasri, J., Haydour, Q., Mohammed, K. S., Dabrh, A. M. A., Prokop, L. J., Alfarkh, W., Lakis, S., Montori, V. M., et al. (2015) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.10.015 · Cited by 12
Telehealth interventions for mobility after lower limb loss: A systematic review and meta-analysis of randomized controlled trials
Esfandiari, E., Miller, W. C., Berardi, A., King, S., & Ashe, M. C. (2021) · Prosthetics and Orthotics International · doi:10.1097/pxr.0000000000000075 · Cited by 12
A brief step-by-step guide on conducting a systematic review and meta-analysis of prognostic model studies
Rayner, D. G., Kim, B. Y., & Foroutan, F. (2024) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2024.111360 · Cited by 15
Preventing colorectal cancer or early diagnosis: Which is best? A re-analysis of the U.S. Preventive Services Task Force Evidence Report
Swartz, A. W., Eberth, J. M., & Strayer, S. M. (2018) · Preventive Medicine · doi:10.1016/j.ypmed.2018.10.014 · Cited by 12
Adverse outcomes associated with opioid prescription for acute low back pain: a systematic review protocol
Mouravska, N., Zielinski, L., Bhatt, M., Sanger, N., Bawor, M., Dennis, B. B., Banfield, L. E., MacKillop, J. M., Paul, J., Worster, A., et al. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0556-x · Cited by 11
“’s methods [32] for continuous outcomes and Akl et al.”
Method
Screening to prevent fragility fractures among adults 40 years and older in primary care: protocol for a systematic review
Gates, M., Pillay, J., Thériault, G., Limburg, H., Grad, R. M., Klarenbach, S. W., Korownyk, C. S., Reynolds, D. L., Riva, J. J., Thombs, B. D., et al. (2019) · Systematic Reviews · doi:10.1186/s13643-019-1094-5 · Cited by 12
“If appropriate, for KQ2 (accuracy of screening tests), we will pool model calibration data for each identified screening method separately using the restricted maximum likelihood estimation approach and the HartunKnapp-Sidnick-Jonkman correction to derive 95% CIs [139, 140].”
Influential
Dual anti-thrombotic treatment with direct anticoagulants improves clinical outcomes in patients with Atrial Fibrillation with ACS or undergoing PCI. A systematic review and meta-analysis
Rosa, S. D., Sabatino, J., Polimeni, A., Sorrentino, S., & Indolfi, C. (2020) · PLoS ONE · doi:10.1371/journal.pone.0235511 · Cited by 12
Diet and physical activity in pregnancy to prevent gestational diabetes: a protocol for an individual participant data (IPD) meta-analysis on the differential effects of interventions with economic evaluation
Coomar, D., Hazlehurst, J., Austin, F., Foster, C. E., Hitman, G. A., Heslehurst, N., Iliodromiti, S., Betrán, A. P., Moss, N., Poston, L., et al. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-048119 · Cited by 12
Searching ClinicalTrials.gov did not change the conclusions of a systematic review
Wilson, L. M., Sharma, R., Dy, S. M., Waldfogel, J. M., & Robinson, K. A. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.07.009 · Cited by 12
Aspirin vs clopidogrel in chronic coronary syndromes: a meta-analysis of ethnic differences
Laudani, C., Occhipinti, G., Costa, F., D’Amario, D., Palmerini, T., Sciarretta, S., Gibson, C. M., Bikdeli, B., Andreotti, F., Jeong, Y., et al. (2026) · European Heart Journal · doi:10.1093/eurheartj/ehag351 · Cited by 10
Traffic safety meta-analysis of reversible lanes
Manuel, A., de Barros, A., & Tay, R. (2020) · Accident Analysis & Prevention · doi:10.1016/j.aap.2020.105751 · Cited by 11
Corticosteroid Therapy for Patients Hospitalized With Community-Acquired Pneumonia
Siemieniuk, R. A. C., Alonso‐Coello, P., & Guyatt, G. H. (2016) · Annals of Internal Medicine · doi:10.7326/l15-0583 · Cited by 11
Apparently conclusive meta-analyses on interventions in critical care may be inconclusive—a meta-epidemiological study
Koster, T. M., Wetterslev, J., Gluud, C. N., Jakobsen, J. C., Kaufmann, T., Eck, R. J., Koster, G., Hiemstra, B., van der Horst, I. C. C., & Keus, E. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.05.011 · Cited by 10
“Fourth, we have used the DerSimonian-Laird random effects estimator, although newer methods have been found to have improved performance [36].”
Method
Predictors of the effects of treatment for shoulder pain: protocol of an individual participant data meta-analysis
van der Windt, D., Burke, D. L., Babatunde, O. O., Hattle, M., McRobert, C. J., Littlewood, C., Wynne‐Jones, G., Chesterton, L. S., van der Heijden, G. J. M. G., Winters, J. C., et al. (2019) · Diagnostic and Prognostic Research · doi:10.1186/s41512-019-0061-x · Cited by 11
Hospital Workforce Engagement and Inpatient Mortality Rate: Findings from the English National Health Service Staff Surveys
Badgett, R. G., Jonker, L., & Xirasagar, S. (2020) · Journal of General Internal Medicine · doi:10.1007/s11606-020-06045-0 · Cited by 10
Association between obstructive sleep apnea and multiple adverse clinical outcomes: evidence from an umbrella review
Wang, Q., Zeng, H., Dai, J., Zhang, M., & Shen, P. (2025) · Frontiers in Medicine · doi:10.3389/fmed.2025.1497703 · Cited by 9
Does vitamin D supplementation improve bone density in vitamin D-deficient children? Protocol for an individual patient data meta-analysis
Winzenberg, T. M., Lamberg-Allardt, C. J. E., Fuleihan, G. E., Mølgaard, C., Zhu, K., Wu, F., & Riley, R. D. (2018) · BMJ Open · doi:10.1136/bmjopen-2017-019584 · Cited by 9
Association between antibiotic exposure and adverse outcomes of children and pregnant women: evidence from an umbrella review
Li, Y., Liu, L.-H., Jian, Z., Li, P.-H., Jin, X., Li, H., & Wang, K. (2023) · World Journal of Pediatrics · doi:10.1007/s12519-023-00711-z · Cited by 9
“Since the DL model may overestimate the effect in meta-analyses including a small number of studies, we also calculated the effect size by the modified Hartung–Knapp–Sidik–Jonkman (HKSJ) method in meta-analyses with ≤ 5 original studies [19, 20].”
Method
Does the benefit from treating to lower blood pressure targets vary with age? A systematic review and meta-analysis
Roush, G. C., Zubair, A., Singh, K. N., Kostis, W. J., Sica, D. A., & Kostis, J. B. (2019) · Journal of Hypertension · doi:10.1097/hjh.0000000000002079 · Cited by 9
“Consistent with prior publications [12], Knapp–Hartung models were used for 5þ trials, and DerSimonian–Laird models for less than five trials.”
Method
Delivery room interventions to prevent bronchopulmonary dysplasia in preterm infants: a protocol for a systematic review and network meta-analysis
Mitra, S., Disher, T. C., Pichler, G., D’Souza, B., McCord, H., Chayapathi, V., Jones, K., & Schmölzer, G. M. (2019) · BMJ Open · doi:10.1136/bmjopen-2018-028066 · Cited by 9
“The RE model assumes that the observed treatment effect for a study is a combination of a treatment effect common to all studies plus a component specific to that study alone.(36) 37 Models will be based on standard code modified to include minimally informative priors on baselines, treatment effects and between trial…”
Method
Cognitive Behavioral Therapy for Chronic Insomnia
Trauer, J. M., & Cunnington, D. (2016) · Annals of Internal Medicine · doi:10.7326/l15-0484 · Cited by 9
Meta-analysis methods for risk difference: A comparison of different models
Guo, J., Xiao, M., Chu, H., & Lin, L. (2022) · Statistical Methods in Medical Research · doi:10.1177/09622802221125913 · Cited by 8
“One of the most popular methods to estimate between-study heterogeneity is the DerSimonian–Laird (DL) estimator,(23) however, recent research shows that the DL estimator can be biased with incorrectly high precision.(24) Many other heterogeneity estimators are available, including Hedges (HE), Paule– Mandel (PM), maxi…”
Method
The Myth of Making Inferences for an Overall Treatment Efficacy with Data from Multiple Comparative Studies Via Meta-Analysis
Hasegawa, T., Claggett, B. L., Tian, L., Solomon, S. D., Pfeffer, M. A., & Wei, L. (2016) · Statistics in Biosciences · doi:10.1007/s12561-016-9179-3 · Cited by 8
“[12] in the clinical literature along with three specific alternative methods which attempt to account for the increased uncertainty induced by between-study differences.”

Authors: Cornell, J. E., Mulrow, C. D., Localio, R., Stack, C. B., Meibohm, A. R., Guallar, E., & Goodman, S. N.
Affiliations: Johns Hopkins University (US)
From University of Texas Health Science Center at San Antonio, San Antonio, Texas; University of Pennsylvania and American College of Physicians, Philadelphia, Pennsylvania; Johns Hopkins School of Public Health, Baltimore, Maryland; and Stanford University School of Medicine, Stanford, California.
ORCID: Cornell, John E.
Mulrow, Cynthia D.
Goodman, Steven N.
Venue: Annals of Internal Medicine
Publisher: American College of Physicians
Published: 2014-02-18
Volume: 160
Issue: 4
Pages: 267-270
ISSN: 0003-4819 (ISSN-L)
1539-3704
DOI: 10.7326/m13-2886
OpenAlex: W2129670822
Cited by: 453
References: 28
Citations by year: 2026: 17 · 2025: 34 · 2024: 30 · 2023: 17 · 2022: 26 · 2021: 54 · 2020: 34 · 2019: 58 · 2018: 42 · 2017: 41 · 2016: 51 · 2015: 28 · 2014: 20
FWCI: 12.77
Type: journal article
Language: en
Open access: closed
Licence: www.acpjournals.org/journal/aim/text-and-data-mining (tdm)
Locations: Annals of Internal Medicine (publishedVersion)
PubMed (publishedVersion)
Topics: Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
Social Sciences › Sociology and Political Science › Delphi Technique in Research
Keywords: random-effects meta-analysis, heterogeneity, treatment effect estimation, confidence limits, critical synthesis
MeSH terms: Data Interpretation, Statistical, Software, Meta-Analysis as Topic, Confidence Intervals

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Random-Effects Meta-analysis of Inconsistent Effects: A Time for Change

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks