Cited by 453Meta-analysis and systematic reviewsStatistical Methods in EpidemiologyDelphi Technique in Research

The DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.

Random-Effects Meta-analysis of Inconsistent Effects: A Time for Change

Cornell, John E., Mulrow, Cynthia D., Localio, Russell, Stack, Catharine B., Meibohm, Anne R., Guallar, Eliseo, Goodman, Steven N.·2014·Annals of Internal Medicine

DOI 10.7326/m13-2886

Open this paper in CoChat — free account

Abstract

A primary goal of meta-analysis is to improve the estimation of treatment effects by pooling results of similar studies. This article explains how the most widely used method for pooling heterogeneous studies--the Der Simonian-Laird (DL) estimator--can produce biased estimates with falsely high precision. A classic example is presented to show that use of the DL estimator can lead to erroneous conclusions. Particular problems with the DL estimator are discussed, and several alternative methods for summarizing heterogeneous evidence are presented. The authors support replacing universal use of the DL estimator with analyses based on a critical synthesis that recognizes the uncertainty in the evidence,focuses on describing and explaining the probable sources of variation in the evidence, and uses random-effects estimates that provide more accurate confidence limits than the DL estimator.

Integrity signals

Registry

Crossref · 10.7326/m13-2886

Cites

28 of 28 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Meta-analysis in clinical trials
DerSimonian, R., & Laird, N. (1986) · Controlled Clinical Trials · doi:10.1016/0197-2456(86)90046-2 · Cited by 39,552
Quantifying heterogeneity in a meta‐analysis
Higgins, J. P. T., & Thompson, S. G. (2002) · Statistics in Medicine · doi:10.1002/sim.1186 · Cited by 36,763
Conducting Meta-Analyses in R with the metafor Package
Viechtbauer, W. (2010) · Journal of Statistical Software · doi:10.18637/jss.v036.i03 · Cited by 20,464
A Re-Evaluation of Random-Effects Meta-Analysis
Higgins, J. P. T., Thompson, S. G., & Spiegelhalter, D. J. (2009) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/j.1467-985x.2008.00552.x · Cited by 2,569
Random-effects model for meta-analysis of clinical trials: An update
DerSimonian, R., & Kacker, R. (2007) · Contemporary Clinical Trials · doi:10.1016/j.cct.2006.04.004 · Cited by 2,374
Improved tests for a random effects meta‐regression with a single covariate
Knapp, G., & Hartung, J. (2003) · Statistics in Medicine · doi:10.1002/sim.1482 · Cited by 2,075
Meta‐Analysis Of Controlled Clinical Trials
Whitehead, A. (2002) · doi:10.1002/0470854200 · Cited by 827
A comparison of statistical methods for meta‐analysis
Brockwell, S. E., & Gordon, I. R. (2001) · Statistics in Medicine · doi:10.1002/sim.650 · Cited by 573
BORROWING STRENGTH FROM EXTERNAL TRIALS IN A META-ANALYSIS
Higgins, J. P. T., & WHITEHEAD, A. (1996) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19961230)15:24 · Cited by 559
A LIKELIHOOD APPROACH TO META-ANALYSIS WITH RANDOM EFFECTS
Hardy, R., & Thompson, S. G. (1996) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19960330)15:6 · Cited by 522

Flagged first, then by citations

Show all 28 references

Research works printed but not linked: 1

Weseley AC, 1962, Obstet Gynecol, 19, 355

Sorted by citations

Assessment of global incidence and mortality of hospital-treated sepsis. Current estimates and limitations.
Fleischmann, C., Scherag, A., Adhikari, N. K. J., Hartog, C. S., Tsaganos, T., Schlattmann, P., Angus, D. C., & Reinhart, K. (2016) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/rccm.201504-0781oc · Cited by 3,548
“(15) Balancing between taking care of heterogeneity by stratification and the availability of a considerable number of studies to be meta-analysed to derive a relatively robust estimator, we finally stratified all studies into two subgroups (hospital-treated sepsis and hospitaltreated severe sepsis).”
Background
Methods to estimate the between‐study variance and its uncertainty in meta‐analysis
Veroniki, A. A., Jackson, D., Viechtbauer, W., Bender, R., Bowden, J., Knapp, G., Kuss, O., Higgins, J. P., Langan, D., & Salanti, G. (2016) · Research Synthesis Methods · doi:10.1002/jrsm.1164 · Cited by 1,685
“…its default use has often been challenged in the sense that DL may underestimate the true between-study variance, potentially producing overly narrow confidence intervals (CIs) for the mean effect (Cornell et al., 2014), especially when the between-study variance is large (Novianti et al., 2014).”
A comparison of heterogeneity variance estimators in simulated random‐effects meta‐analyses
Langan, D., Higgins, J. P. T., Jackson, D., Bowden, J., Veroniki, A. A., Kontopantelis, E., Viechtbauer, W., & Simmonds, M. (2019) · Research Synthesis Methods · doi:10.1002/jrsm.1316 · Cited by 1,074
“Other methods include the profile likelihood method,(21) which has also been shown as a better alternative to the Wald method in simulated meta‐analysis data(42) and recommended elsewhere.(43) Second, a continuity correction of 0.”
Method
Fixed or random effects meta-analysis? Common methodological issues in systematic reviews of effectiveness
Tufanaru, C., Munn, Z., Stephenson, M., & Aromataris, E. (2015) · International Journal of Evidence-Based Healthcare · doi:10.1097/xeb.0000000000000065 · Cited by 1,057
A guide to systematic review and meta-analysis of prognostic factor studies
Riley, R. D., Moons, K. G. M., Snell, K. I. E., Ensor, J., Hooft, L., Altman, D. G., Hayden, J. A., Collins, P. G. S., & Debray, T. P. A. (2019) · BMJ · doi:10.1136/bmj.k4597 · Cited by 771
Diabetes Medications as Monotherapy or Metformin-Based Combination Therapy for Type 2 Diabetes: A Systematic Review and Meta-analysis
Maruthur, N. M., Tseng, E., Hutfless, S., Wilson, L. M., Suarez-Cuervo, C., Berger, Z., Chu, Y., Iyoha, E., Segal, J. B., & Bolen, S. (2016) · Annals of Internal Medicine · doi:10.7326/m15-2650 · Cited by 682
“We generated summary treatment effects with the random-effects model estimated by using the DerSimonian and Laird method in settings of low heterogeneity (I(2) < 50%) (12) and the profile likelihood estimate in settings of high heterogeneity (I(2) ≥ 50%) (13).”
Method
A guide to systematic review and meta-analysis of prediction model performance
Debray, T. P. A., Damen, J. A. A. G., Snell, K. I. E., Ensor, J., Hooft, L., Reitsma, J. B., Riley, R. D., & Moons, K. G. M. (2017) · BMJ · doi:10.1136/bmj.i6460 · Cited by 657
Impact of patient and public involvement on enrolment and retention in clinical trials: systematic review and meta-analysis
Crocker, J. C., Ricci‐Cabello, I., Parker, A., Hirst, J. A., Chant, A., Petit-Zeman, S., Evans, D., & Rees, S. (2018) · BMJ · doi:10.1136/bmj.k4738 · Cited by 650
Advances in the meta-analysis of heterogeneous clinical trials I: The inverse variance heterogeneity model
Doi, S. A. R., Barendregt, J. J., Khan, S., Thalib, L., & Williams, G. M. (2015) · Contemporary Clinical Trials · doi:10.1016/j.cct.2015.05.009 · Cited by 596
Statins for Prevention of Cardiovascular Disease in Adults
Chou, R., Dana, T., Blazina, I., Daeges, M., & Jeanne, T. L. (2016) · JAMA · doi:10.1001/jama.2015.15629 · Cited by 570
Global association between ambient air pollution and blood pressure: A systematic review and meta-analysis
Yang, B., Qian, Z. M., Howard, S. W., Vaughn, M. G., Fan, S.-J., Liu, K.-K., & Dong, G. (2018) · Environmental Pollution · doi:10.1016/j.envpol.2018.01.001 · Cited by 609
Effect of diet and physical activity based interventions in pregnancy on gestational weight gain and pregnancy outcomes: meta-analysis of individual participant data from randomised trials
The International Weight Management in Pregnancy (i-WIP) Collaborative Group (2017) · BMJ · doi:10.1136/bmj.j3119 · Cited by 529
Low-Dose Aspirin for Prevention of Morbidity and Mortality From Preeclampsia: A Systematic Evidence Review for the U.S. Preventive Services Task Force
Henderson, J. T., Whitlock, E. P., O’Connor, E., Senger, C. A., Thompson, J. H., & Rowland, M. G. (2014) · Annals of Internal Medicine · doi:10.7326/m13-2844 · Cited by 517
Association of Blood Pressure Lowering With Mortality and Cardiovascular Disease Across Blood Pressure Levels
Brunström, M., & Carlberg, B. (2017) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2017.6015 · Cited by 483
Birth cohort trends in the global epidemiology of alcohol use and alcohol-related harms in men and women: systematic review and metaregression
Slade, T., Chapman, C., Swift, W., Keyes, K., Tonks, Z., & Teesson, M. (2016) · BMJ Open · doi:10.1136/bmjopen-2016-011827 · Cited by 463
Internet-Delivered Cognitive Behavioral Therapy to Treat Insomnia: A Systematic Review and Meta-Analysis.
Seyffert, M., Lagisetty, P., Landgraf, J., Chopra, V., Pfeiffer, P. N., Conte, M. L., & Rogers, M. A. M. (2016) · PLOS ONE · doi:10.1371/journal.pone.0149139 · Cited by 323
“Our review of 15 randomized trials with 2392 participants employed slightly different analytic methods in that the Hartung-Knapp-SidakJonkman random effects model is more conservative but recommended above the fixed models used in the Cheng review and the DerSimonian-Laird random effects model [21,22].”
Method
Association of Inhaled Corticosteroids and Long-Acting β-Agonists as Controller and Quick Relief Therapy With Exacerbations and Symptom Control in Persistent Asthma
Sobieraj, D. M., Weeda, E. R., Nguyen, E., Coleman, C. I., White, C. M., Lazarus, S. C., Blake, K., Lang, J. E., & Baker, W. L. (2018) · JAMA · doi:10.1001/jama.2018.2769 · Cited by 343
Physical inactivity, cardiometabolic disease, and risk of dementia: an individual-participant meta-analysis
Kivimäki, M., Singh-Manoux, A., Pentti, J., Sabia, S., Nyberg, S. T., Alfredsson, L., Goldberg, M., Knutsson, A., Koskenvuo, M., Koskinen, A., et al. (2019) · BMJ · doi:10.1136/bmj.l1495 · Cited by 313
Statin Use for the Primary Prevention of Cardiovascular Disease in Adults: Updated Evidence Report and Systematic Review for the US Preventive Services Task Force
Chou, R., Cantor, A., Dana, T., Wagner, J., Ahmed, A. Y., Fu, R., & Ferencik, M. (2022) · JAMA · doi:10.1001/jama.2022.12138 · Cited by 287
Evaluation of various estimators for standardized mean difference in meta‐analysis
Lin, L., & Aloe, A. M. (2021) · Statistics in Medicine · doi:10.1002/sim.8781 · Cited by 297
Return to work after critical illness: a systematic review and meta-analysis
Kamdar, B. B., Suri, R., Suchyta, M. R., DiGrande, K., Sherwood, K. D., Colantuoni, E., Dinglas, V. D., Needham, D. M., & Hopkins, R. O. (2019) · Thorax · doi:10.1136/thoraxjnl-2019-213803 · Cited by 286
Transfusion strategies in non-bleeding critically ill adults: a clinical practice guideline from the European Society of Intensive Care Medicine
Vlaar, A. P. J., Oczkowski, S. J. W., de Bruin, S., Wijnberge, M., Antonelli, M. M., Aubron, C., Ariès, P., Duranteau, J., Juffermans, N. P., Meier, J. M., et al. (2020) · Intensive Care Medicine · doi:10.1007/s00134-019-05884-8 · Cited by 294
“Fixed-effects models were used whenever the number of studies was three or less or when there were large discrepancies between large and small study results [30].”
Method
A framework for meta-analysis of prediction model studies with binary and time-to-event outcomes
Debray, T. P., Damen, J. A., Riley, R. D., Snell, K., Reitsma, J. B., Hooft, L., Collins, G. S., & Moons, K. G. (2019) · Statistical Methods in Medical Research · doi:10.1177/0962280218785504 · Cited by 278
“Furthermore, when the heterogeneity is small and the study sizes are mixed, the Hartung–Knapp method produces confidence intervals that are too wide.(46,47) Similarly, prediction intervals have poor coverage when the extent of heterogeneity is limited, when few (less than 5) studies are included, or when there are a m…”
Method
Inhaled Cannabis for Chronic Neuropathic Pain: A Meta-analysis of Individual Patient Data
Andreae, M. H., Carter, G. M., Shaparin, N., Suslov, K., Ellis, R. J., Ware, M. A., Abrams, D. I., Prasad, H., Wilsey, B. L., Indyk, D., et al. (2015) · Journal of Pain · doi:10.1016/j.jpain.2015.07.009 · Cited by 268
Preexposure Prophylaxis for the Prevention of HIV Infection
Chou, R., Evans, C., Hoverman, A. S., Sun, C. J., Dana, T., Bougatsos, C., Grusing, S., & Korthuis, P. T. (2019) · JAMA · doi:10.1001/jama.2019.2591 · Cited by 275
Methods for evidence synthesis in the case of very few studies
Bender, R., Friede, T., Koch, A., Kuss, O., Schlattmann, P., Schwarzer, G., & Skipka, G. (2018) · Research Synthesis Methods · doi:10.1002/jrsm.1297 · Cited by 250
“The main problem of the DerSimonian‐Laird method is that the uncertainty of the estimation of the between‐study and within‐study variance is ignored.(4,12,18) A number of alternative methods to perform meta‐analyses with random effects have been proposed (see below) including various methods to estimate the between‐st…”
Method
Methods to calculate uncertainty in the estimated overall effect size from a random‐effects meta‐analysis
Veroniki, A. A., Jackson, D., Bender, R., Kuss, O., Langan, D., Higgins, J. P. T., Knapp, G., & Salanti, G. (2019) · Research Synthesis Methods · doi:10.1002/jrsm.1319 · Cited by 236
“The estimation of the CI for overall effect size is often conducted with the Wald‐type (WT) method using a normal distribution, with variance equal to the inverse of the sum of the study weights, and the DerSimonian and Laird(11) estimator for the between‐study variance, and this has been used routinely in many meta‐a…”
Method
Systematic reviews and meta-analyses of randomized trials: principles and pitfalls
da Costa, B. R., & Jüni, P. (2014) · European Heart Journal · doi:10.1093/eurheartj/ehu424 · Cited by 250
Association of Type 2 Diabetes With Cancer: A Meta-analysis With Bias Analysis for Unmeasured Confounding in 151 Cohorts Comprising 32 Million People
Ling, S., Brown, K., Miksza, J. K., Howells, L. M., Morrison, A., Issa, E., Yates, T. E., Khunti, K., Davies, M. J., & Zaccardi, F. (2020) · Diabetes Care · doi:10.2337/dc20-0204 · Cited by 267
“We used a random-effects meta-analysis with the Knapp-Hartung method to better account for suchheterogeneity and to give a more robust inference about the strength of causal evidence in potentially heterogeneous studies (9,49).”
Method
Behavioral counseling and pharmacotherapy interventions for tobacco cessation in adults, including pregnant women: a review of reviews for the U.S. Preventive Services Task Force.
Patnode, C. D., Henderson, J. T., Thompson, J. H., Senger, C. A., Fortmann, S. P., & Whitlock, E. P. (2015) · Annals of Internal Medicine · doi:10.7326/m15-0171 · Cited by 238
“We evaluated the appropriateness of meta-analytic procedures and used our technical judgment to interpret pooled analyses accounting for limitations or concerns around heterogeneity, statistical approaches (18, 19), and other factors.”
Random effects meta‐analysis: Coverage performance of 95 % confidence and prediction intervals following REML estimation
Partlett, C., & Riley, R. D. (2017) · Statistics in Medicine · doi:10.1002/sim.7140 · Cited by 240
“[3] demonstrate aNational Perinatal Epidemiology Unit, Oxford, U.”
Background
Meta-analysis of Results of Testosterone Therapy on Sexual Function Based on International Index of Erectile Function Scores
Corona, G., Rastrelli, G., Morgentaler, A., Sforza, A., Mannucci, E., & Maggi, M. (2017) · European Urology · doi:10.1016/j.eururo.2017.03.032 · Cited by 226
Association of Optimism With Cardiovascular Events and All-Cause Mortality: A Systematic Review and Meta-analysis
Rozanski, A., Bavishi, C., Kubzansky, L. D., & Cohen, R. (2019) · JAMA Network Open · doi:10.1001/jamanetworkopen.2019.12200 · Cited by 216
Local anaesthetics and regional anaesthesia versus conventional analgesia for preventing persistent postoperative pain in adults and children
Weinstein, E. J., Levene, J. L., Cohen, M. S., Andreae, D. A., Chao, J. Y., Johnson, M., Hall, C. B., & Andreae, M. H. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd007105.pub3 · Cited by 209
Mindfulness-Based Stress Reduction for Treating Low Back Pain: A Systematic Review and Meta-analysis
Anheyer, D., Haller, H., Barth, J., Lauche, R., Dobos, G., & Cramer, H. (2017) · Annals of Internal Medicine · doi:10.7326/m16-1997 · Cited by 197
“The Knapp–Hartung small-sample correction was used because it provides a more adequate accounting of uncertainty when treatment effects from a small number of heterogeneous studies are pooled (28, 29).”
Method
Epidural Corticosteroid Injections for Radiculopathy and Spinal Stenosis
Chou, R., Hashimoto, R. E., Friedly, J. L., Fu, R. F., Bougatsos, C., Dana, T., Sullivan, S. D., & Jarvik, J. G. (2015) · Annals of Internal Medicine · doi:10.7326/m15-0934 · Cited by 209
Cardiovascular Events and Mortality in White Coat Hypertension
Cohen, J. B., Lotito, M. J., Trivedi, U. K., Denker, M. G., Cohen, D. L., & Townsend, R. R. (2019) · Annals of Internal Medicine · doi:10.7326/m19-0223 · Cited by 212
Inverse relationship between body mass index and mortality in older nursing home residents: a meta‐analysis of 19,538 elderly subjects
Veronese, N., Cereda, E., Solmi, M., Fowler, S. A., Manzato, E., Maggi, S., Manu, P., Abe, E., Hayashi, K., Allard, J. P., et al. (2015) · Obesity Reviews · doi:10.1111/obr.12309 · Cited by 187
“05) (30), we conducted stratified analyses (29) exploring effects of the following pre-specified moderators: study origin (European vs.”
BackgroundMethod
Comparison of four heterogeneity measures for meta‐analysis
Lin, L. (2019) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.13159 · Cited by 186
“However, although the method‐of‐moments estimate has been popular so far, it has been found to be inferior in some cases.(42) Alternative estimates, such as those based on the restricted maximum likelihood or Bayesian analysis, may be better options to be used in the heterogeneity measures.”
Background
Effectiveness of Prevention Strategies for Contrast-Induced Nephropathy
Subramaniam, R. M., Suarez‐Cuervo, C., Wilson, R. F., Turban, S., Zhang, A. W., Sherrod, C. E., Aboagye, J., Eng, J., Choi, M. J., Hutfless, S. M., et al. (2016) · Annals of Internal Medicine · doi:10.7326/m15-1456 · Cited by 185
“For large comparisons with 18 or more studies, we used the DerSimonian–Laird random-effects estimator, with the estimate of heterogeneity taken from the inverse-variance, fixed-effect model (15).”
MethodInfluential
Intravesical Therapy for the Treatment of Nonmuscle Invasive Bladder Cancer: A Systematic Review and Meta-Analysis
Chou, R., Selph, S., Buckley, D., Fu, R. F., Griffin, J. C., Grusing, S., & Gore, J. L. (2016) · The Journal of Urology · doi:10.1016/j.juro.2016.12.090 · Cited by 185
“Some meta-analyses were based on small numbers of trials and should be interpreted cautiously.(13) We excluded nonEnglish language articles and abstract-only publications.”
MethodInfluential
Adverse Pregnancy Outcomes and Long-term Maternal Kidney Disease
Barrett, P., McCarthy, F. P., Kublickiene, K. R., Cormican, S., Judge, C. S., Evans, M., Kublickas, M., Perry, I. J., Stenvinkel, P., & Khashan, A. S. (2020) · JAMA Network Open · doi:10.1001/jamanetworkopen.2019.20964 · Cited by 187
Direct oral anticoagulants for cancer-associated venous thromboembolism: a systematic review and meta-analysis
Mulder, F. I., Bosch, F. T. M., Young, A., Marshall, A., McBane, R. D., Zemla, T., Carrier, M., Kamphuisen, P. W., Bossuyt, P. M., Buller, H. R., et al. (2020) · Blood · doi:10.1182/blood.2020005819 · Cited by 181
Ultrasonography Screening for Abdominal Aortic Aneurysms: A Systematic Evidence Review for the U.S. Preventive Services Task Force
Guirguis‐Blake, J., Beil, T. L., Senger, C. A., & Whitlock, E. P. (2014) · Annals of Internal Medicine · doi:10.7326/m13-1844 · Cited by 176
“Other potential risk factors include a history of other vascular aneurysms (9), coronary artery disease (10), cerebrovascular disease (9), atherosclerosis (10), hypercholesterolemia (10), and hypertension (1, 10).”
Background
Counseling and Behavioral Interventions for Healthy Weight and Weight Gain in Pregnancy
Cantor, A. G., Jungbauer, R. M., McDonagh, M. S., Blazina, I., Marshall, N., Weeks, C., Fu, R. F., LeBlanc, E. S., & Chou, R. (2021) · JAMA · doi:10.1001/jama.2021.4230 · Cited by 175
Effect of Statins on Kidney Disease Outcomes: A Systematic Review and Meta-analysis
Su, X., Zhang, L., Lv, J., Wang, J., Hou, W., Xie, X., & Zhang, H. (2016) · American Journal of Kidney Diseases · doi:10.1053/j.ajkd.2016.01.016 · Cited by 164
Mid- and Long-Term Health Risks in Living Kidney Donors
O’Keeffe, L. M., Ramond, A., Oliver‐Williams, C., Willeit, P., Paige, E., Trotter, P., Evans, J., Wadström, J., Nicholson, M. L., Collett, D., et al. (2018) · Annals of Internal Medicine · doi:10.7326/m17-1235 · Cited by 165
“Summary RRs, incidence rates, and SMDs comparing donors with control participants were calculated for each outcome by pooling the study-specific estimates using the random-effects profile likelihood metaanalysis method (13).”
Method
Association Between Atrial Fibrillation and Silent Cerebral Infarctions
Kalantarian, S., Ay, H., Gollub, R. L., Lee, H., Retzepi, K., Mansour, M., & Ruskin, J. N. (2014) · Annals of Internal Medicine · doi:10.7326/m14-0538 · Cited by 169
“In the presence of heterogeneity, we used random-effects models and the profile likelihood method (27) to perform the meta-analysis of the association between AF and SCIs.”
Method
The characterization of seafood mislabeling: A global meta-analysis
Luque, G. M., & Donlan, C. J. (2019) · Biological Conservation · doi:10.1016/j.biocon.2019.04.006 · Cited by 163
“Like many clinical medical trials (Cornell et al., 2014), this is not the case with mislabeling studies: seafood is sampled in different ways, with different products, for different reasons, and in different locations.”
Background
Effects on Health Outcomes of a Mediterranean Diet With No Restriction on Fat Intake: A Systematic Review and Meta-analysis
Bloomfield, H. E., Koeller, E., Greer, N., MacDonald, R., Kane, R., & Wilt, T. J. (2016) · Annals of Internal Medicine · doi:10.7326/m16-0361 · Cited by 159
“Second, we used DerSimonian and Laird random-effects models, which may produce estimates with erroneously high precision if the number of studies meta-analyzed is small or if the estimates vary widely among the studies (123).”
Method
Lung Cancer Screening with Low-Dose CT: a Meta-Analysis
Hoffman, R. M., Atallah, R. P., Struble, R. D., & Badgett, R. G. (2020) · Journal of General Internal Medicine · doi:10.1007/s11606-020-05951-7 · Cited by 156
“We used a random-effects model with the Hartung-Knapp estimator.(17) We performed all statistical analyses online at openCPU(18) with the meta package of the R programming language.”
Method
Pharmacotherapy for diabetic peripheral neuropathy pain and quality of life
Waldfogel, J. M., Nesbit, S. A., Dy, S. M., Sharma, R., Zhang, A. W., Wilson, L. M., Bennett, W. L., Yeh, H., Chelladurai, Y., Feldman, D. R., et al. (2017) · Neurology · doi:10.1212/wnl.0000000000003882 · Cited by 158
“50%).(12) We graded strength of evidence as recommended by the Methods Guide for Conducting Comparative Effectiveness Reviews.”
Risk of relapse after antidepressant discontinuation in anxiety disorders, obsessive-compulsive disorder, and post-traumatic stress disorder: systematic review and meta-analysis of relapse prevention trials
Batelaan, N. M., Bosman, R. C., Muntingh, A., Scholten, W. D., Huijbregts, K. M., & van Balkom, A. J. L. M. (2017) · BMJ · doi:10.1136/bmj.j3927 · Cited by 153
Extended Daily Dialysis Versus Continuous Renal Replacement Therapy for Acute Kidney Injury: A Meta-analysis
Zhang, L., Yang, J., Eastwood, G. M., Zhu, G., Tanaka, A., & Bellomo, R. (2015) · American Journal of Kidney Diseases · doi:10.1053/j.ajkd.2015.02.328 · Cited by 151
Random-Effects Meta-analysis
Serghiou, S., & Goodman, S. N. (2018) · JAMA · doi:10.1001/jama.2018.19684 · Cited by 151
Self-citation
Road Traffic Noise Exposure and Depression/Anxiety: An Updated Systematic Review and Meta-Analysis
Dzhambov, A. M., & Lercher, P. (2019) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph16214134 · Cited by 151
Mindfulness- And acceptance-based interventions for patients with fibromyalgia - A systematic review and meta-analyses
Haugmark, T., Hagen, K. B., Smedslund, G., & Zangi, H. A. (2019) · PLoS ONE · doi:10.1371/journal.pone.0221897 · Cited by 161
Efficacy of Hospital at Home in Patients with Heart Failure: A Systematic Review and Meta-Analysis
Qaddoura, A., Yazdan-Ashoori, P., Kabali, C., Thabane, L., Haynes, R. B., Connolly, S. J., & Spall, H. G. C. V. (2015) · PLoS ONE · doi:10.1371/journal.pone.0129282 · Cited by 147
“We applied the DerSimonian-Laird random effects model with a Knapp-Hartrung modification for pooled estimates as recent methodological research has shown that the standard DerSimonian-Laird random effects approach can produce narrow confidence intervals when the number of studies is small or when substantive heterogen…”
Method
Alpha blockers for treatment of ureteric stones: systematic review and meta-analysis
Hollingsworth, J. M., Canales, B. K., Rogers, M. A. M., Sukumar, S., Yan, P. L., Kuntz, G. M., & Dahm, P. (2016) · BMJ · doi:10.1136/bmj.i6112 · Cited by 143
Conducting Meta-analyses of Proportions in R
Wang, N. (2023) · Journal of Behavioral Data Science · doi:10.35566/jbds/v3n2/wang · Cited by 144
Society of Critical Care Medicine Guidelines on Family-Centered Care for Adult ICUs: 2024
Hwang, D. Y., Oczkowski, S. J. W., Lewis, K. A., Birriel, B., Downar, J., Farrier, C. E., Fiest, K. M., Gerritsen, R. T., Hart, J. L., Hartog, C. S., et al. (2025) · Critical Care Medicine · doi:10.1097/ccm.0000000000006549 · Cited by 140
Prognostic Value of Cardiac Troponin in Patients With Chronic Kidney Disease Without Suspected Acute Coronary Syndrome
Michos, E. D., Wilson, L. M., Yeh, H., Berger, Z. D., Suarez‐Cuervo, C., Stacy, S. R., & Bass, E. B. (2014) · Annals of Internal Medicine · doi:10.7326/m14-0743 · Cited by 136
“This method provides better accounting of uncertainty in the estimation of between-study variance than the DerSimonian–Laird estimator (16).”
Method
Procalcitonin-Guided Antibiotic Discontinuation and Mortality in Critically Ill Adults
Pepper, D. J., Sun, J., Rhee, C., Welsh, J. A., Powers, J. H., Danner, R. L., & Kadri, S. S. (2019) · CHEST Journal · doi:10.1016/j.chest.2018.12.029 · Cited by 138
“Mortality outcomes between intervention and comparator groups were analyzed as risk ratios (RRs) using random-effects models and the Knapp-Hartung adjustment for small study numbers.(37-39) Because biologic plausibility would be strengthened if a survival benefit was observed in studies with high (> 80%) algorithm adh…”
Method
Screening for type 2 diabetes mellitus: a systematic review for the U.S. Preventive Services Task Force
Selph, S., Dana, T., Blazina, I., Bougatsos, C., Patel, H., & Chou, R. (2015) · Annals of Internal Medicine · doi:10.7326/m14-2221 · Cited by 134
“When statistical heterogeneity was present, we performed sensitivity analyses using the profile likelihood method because the DerSimonian–Laird model results in overly narrow 95% CIs (27).”
Method
Screening for Vitamin D Deficiency: A Systematic Review for the U.S. Preventive Services Task Force
LeBlanc, E. S., Zakher, B., Daeges, M., Pappas, M., & Chou, R. (2014) · Annals of Internal Medicine · doi:10.7326/m14-1659 · Cited by 133
“In these cases, the DerSimonian–Laird random-effects model may result in CIs that are too narrow (107).”
Background
Future Cardiovascular Disease Risk for Women With Gestational Hypertension: A Systematic Review and Meta‐Analysis
Lo, C. C. W., Lo, A., Leow, S. H., Fisher, G., Corker, B., Batho, O., Morris, B., Chowaniec, M., Vladutiu, C. J., Fraser, A., et al. (2020) · Journal of the American Heart Association · doi:10.1161/jaha.119.013991 · Cited by 134
Transfusion strategies in bleeding critically ill adults: a clinical practice guideline from the European Society of Intensive Care Medicine
Vlaar, A. P. J., Dionne, J. C., de Bruin, S., Wijnberge, M., Raasveld, J., van Baarle, F. L. F., Antonelli, M. M., Aubron, C., Duranteau, J., Juffermans, N. P., et al. (2021) · Intensive Care Medicine · doi:10.1007/s00134-021-06531-x · Cited by 129
“For all outcomes, we compared fixed- and random-effects estimates, generally using fixed-effects models when the number of studies was very small (5 or fewer) or there were concerns that small-study effects may be impacting the pooled estimate; otherwise we used random-effects models [13].”
Method
Transcatheter Aortic Valve Implantation Versus Surgical Aortic Valve Replacement
Gargiulo, G., Sannino, A., Capodanno, D., Barbanti, M., Buccheri, S., Perrino, C., Capranzano, P., Indolfi, C., Trimarco, B., Tamburino, C., et al. (2016) · Annals of Internal Medicine · doi:10.7326/m16-0060 · Cited by 126
“Data Synthesis and Analysis Effect sizes were calculated with the Knapp– Hartung random-effects estimator and expressed as odds ratios (ORs) and 95% CIs, as previously reported (22).”
Method
Incidence of infection with multidrug-resistant Gram-negative bacteria and vancomycin-resistant enterococci in carriers: a systematic review and meta-regression analysis
Willems, R. P. J., van Dijk, K., Vehreschild, M. J. G. T., Biehl, L. M., Ket, J. C. F., Remmelzwaal, S., & Vandenbroucke‐Grauls, C. M. J. E. (2023) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(22)00811-8 · Cited by 129
Weight History and All-Cause and Cause-Specific Mortality in Three Prospective Cohort Studies
Yu, E., Ley, S. H., Manson, J. E., Willett, W. C., Satija, A., Hu, F. B., & Stokes, A. (2017) · Annals of Internal Medicine · doi:10.7326/m16-1390 · Cited by 120
“The HR estimates for all cohorts were combined via fixed-effects meta-analysis (19).”
Method
Parenteral anticoagulation in patients with cancer who have no therapeutic or prophylactic indication for anticoagulation
Akl, E. A., Gunukula, S. K., Barba, M., Yosuico, V. E. D., van Doormaal, F. F., Kuipers, S., Middeldorp, S., Dickinson, H. O., Bryant, A., & Schünemann, H. J. (2011) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd006652.pub3 · Cited by 119
Benefits and Harms of Osteoporosis Medications in Patients With Chronic Kidney Disease
Wilson, L. M., Rebholz, C. M., Jirru, E., Liu, M. C., Zhang, A. W., Gayleard, J. L., Chu, Y., & Robinson, K. A. (2017) · Annals of Internal Medicine · doi:10.7326/m16-2752 · Cited by 115
Noninvasive Ventilation With vs Without Early Surfactant to Prevent Chronic Lung Disease in Preterm Infants
Isayama, T., Chai‐Adisaksopha, C., & McDonald, S. D. (2015) · Archives of Pediatrics and Adolescent Medicine · doi:10.1001/jamapediatrics.2015.0510 · Cited by 110
Iron intake, body iron status, and risk of breast cancer: a systematic review and meta-analysis
Chang, V. C., Cotterchio, M., & Khoo, E. (2019) · BMC Cancer · doi:10.1186/s12885-019-5642-0 · Cited by 111
Comparison of commonly used methods in random effects meta-analysis: application to preclinical data in drug discovery research
Tanriver-Ayder, E., Faes, C., de Casteele, T. V., McCann, S., & Macleod, M. R. (2021) · BMJ Open Science · doi:10.1136/bmjos-2020-100074 · Cited by 108
Comparative Effectiveness of Fluorescent Versus White Light Cystoscopy for Initial Diagnosis or Surveillance of Bladder Cancer on Clinical Outcomes: Systematic Review and Meta-Analysis
Chou, R., Selph, S., Buckley, D., Fu, R. F., Griffin, J. C., Grusing, S., & Gore, J. L. (2016) · The Journal of Urology · doi:10.1016/j.juro.2016.10.061 · Cited by 107
“than 20% we also pooled studies using the profile likelihood method.(17) All analyses were stratified by the pho-”
BackgroundMethodResultInfluential
Evaluation of Daratumumab for the Treatment of Multiple Myeloma in Patients With High-risk Cytogenetic Factors
Giri, S., Grimshaw, A. A., Bal, S., Godby, K. N., Kharel, P., Djulbegović, B., Dimopoulos, M. A., Facon, T. T. F., Usmani, S. Z., Mateos, M., et al. (2020) · JAMA Oncology · doi:10.1001/jamaoncol.2020.4338 · Cited by 107
Transcatheter versus surgical aortic valve replacement in lower-risk and higher-risk patients: a meta-analysis of randomized trials
Ahmad, Y., Howard, J. P., Arnold, A. D., Madhavan, M. V., Cook, C., Alu, M. C., Mack, M. J., Reardon, M. J., Thourani, V. H., Kapadia, S. R., et al. (2022) · European Heart Journal · doi:10.1093/eurheartj/ehac642 · Cited by 108
Calcineurin Inhibitor Minimization, Conversion, Withdrawal, and Avoidance Strategies in Renal Transplantation: A Systematic Review and Meta‐Analysis
Sawinski, D. L., Trofe‐Clark, J., Leas, B. F., Uhl, S., Tuteja, S., Kaczmarek, J. L., French, B. C., & Umscheid, C. A. (2016) · American Journal of Transplantation · doi:10.1111/ajt.13710 · Cited by 106
“Data synthesis We performed random-effects meta-analyses with Hartung–Knapp adjustment of risk ratios (RRs) and confidence intervals (CIs) (4, 5).”
Method
Buttonhole Versus Rope-Ladder Cannulation of Arteriovenous Fistulas for Hemodialysis: A Systematic Review
Wong, B. C., Muneer, M., Wiebe, N., Storie, D., Shurraw, S., Pannu, N., Klarenbach, S. W., Grudzinski, A. L., NESRALLAH, G. E., & Pauly, R. P. (2014) · American Journal of Kidney Diseases · doi:10.1053/j.ajkd.2014.06.018 · Cited by 105
“Due to other differences (eg, study design, patient population, and variations in buttonhole cannulation technique) expected between studies, we combined results using the KnappHartung random-effects model.(16) Statistical heterogeneity was quantified using the I(2) statistic.”
Method
The effect of vitamin D supplementation on survival in patients with colorectal cancer: systematic review and meta-analysis of randomised controlled trials
Vaughan-Shaw, P., Buijs, L., Blackmur, J., Theodoratou, E., Zgaga, L., Din, F., Farrington, S., & Dunlop, M. (2020) · British Journal of Cancer · doi:10.1038/s41416-020-01060-8 · Cited by 91
Local anaesthetics and regional anaesthesia versus conventional analgesia for preventing persistent postoperative pain in adults and children
Weinstein, E. J., Levene, J. L., Cohen, M. S., Andreae, D. A., Chao, J. Y., Johnson, M. C., Hall, C. B., & Andreae, M. H. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd007105.pub4 · Cited by 105
“Classical meta-analysis may underestimate the between-study variability for small numbers of studies, making estimates for the confidence intervals less reliable when they rely only on a small number of studies (Cornell 2014; Song 2012).”
Method
The Safety and Efficacy of Mineralocorticoid Receptor Antagonists in Patients Who Require Dialysis: A Systematic Review and Meta-analysis
Quach, K., Lvtvyn, L., Baigent, C., Bueti, J., Garg, A. X., Hawley, C. M., Haynes, R., Manns, B., Perkovic, V., Rabbat, C. G., et al. (2016) · American Journal of Kidney Diseases · doi:10.1053/j.ajkd.2016.04.011 · Cited by 101
“Finally, we also estimated effect estimates using the profile likelihood method, which considers the uncertainty of the between-studies variance and may be more reliable than DerSimonian and Laird models in meta-analyses of small trials.(17) A 2-sided P, 0.”
Method
Meta-analysis in evidence-based healthcare
Doi, S. A. R., Furuya‐Kanamori, L., Thalib, L., & Barendregt, J. J. (2017) · International Journal of Evidence-Based Healthcare · doi:10.1097/xeb.0000000000000125 · Cited by 98
Screening for Gestational Diabetes
Pillay, J., Donovan, L., Guitard, S., Zakher, B., Gates, M., Gates, A., Vandermeer, B., Bougatsos, C., Chou, R., & Hartling, L. (2021) · JAMA · doi:10.1001/jama.2021.10404 · Cited by 106
A systematic review and meta-analysis of beta-blockers and renin–angiotensin system inhibitors for preventing left ventricular dysfunction due to anthracyclines or trastuzumab in patients with breast cancer
Lewinter, C., Nielsen, T. H., Edfors, L. R., Linde, C. M., Bland, M., LeWinter, M. M., Cleland, J. G. F., Køber, L., Braunschweig, F., & Månsson‐Broberg, A. (2021) · European Heart Journal · doi:10.1093/eurheartj/ehab843 · Cited by 100
Risk of solid cancer in low dose-rate radiation epidemiological studies and the dose-rate effectiveness factor
Shore, R. E., Walsh, L., Azizova, T. V., & Rühm, W. (2017) · International Journal of Radiation Biology · doi:10.1080/09553002.2017.1319090 · Cited by 99
“Since methods for assessing risk heterogeneity are in flux (Cornell et al. 2014), heterogeneity also was evaluated using the several other methods described in DerSimonian & Kacker (2007).”
Method
Do You Get the Picture? A Meta-Analysis of the Effect of Graphics on Reading Comprehension
Guo, D., Zhang, S., Wright, K. L., & McTigue, E. M. (2020) · AERA Open · doi:10.1177/2332858420901696 · Cited by 137
“Cornell et al. (2014) suggest this method over the DerSimonian Laird method because the latter may produce biased results.”
Method
Colchicine for primary prevention of atrial fibrillation after open-heart surgery: Systematic review and meta-analysis
Lennerz, C., Barman, M., Tantawy, M. A., Sopher, M., & Whittaker, P. (2017) · International Journal of Cardiology · doi:10.1016/j.ijcard.2017.08.039 · Cited by 96
Effect of dietary sodium restriction on arterial stiffness
D’Elia, L., Galletti, F., Fata, E. L., Sabino, P., & Strazzullo, P. (2017) · Journal of Hypertension · doi:10.1097/hjh.0000000000001604 · Cited by 97
Statins and Multiple Noncardiovascular Outcomes
He, Y., Li, X., Gašević, D., Brunt, E., McLachlan, F., Millenson, M. E., Timofeeva, M. N., Ioannidis, J. P. A., Campbell, H. J., & Τheodoratou, E. (2018) · Annals of Internal Medicine · doi:10.7326/m18-0808 · Cited by 97
Radial Versus Femoral Access in Invasively Managed Patients With Acute Coronary Syndrome
Andò, G., & Capodanno, D. (2015) · Annals of Internal Medicine · doi:10.7326/m15-1277 · Cited by 95
Comparative Effectiveness of 12 Treatment Strategies for Preventing Contrast-Induced Acute Kidney Injury: A Systematic Review and Bayesian Network Meta-analysis
Su, X., Xie, X., Liu, L., Lv, J., Song, F., Perkovic, V., & Zhang, H. (2016) · American Journal of Kidney Diseases · doi:10.1053/j.ajkd.2016.07.033 · Cited by 95
Primary Care Screening and Treatment for Latent Tuberculosis Infection in Adults
Kahwati, L. C., Feltner, C., Halpern, M. T., Woodell, C., Boland, E., Amick, H. R., Weber, R. P., & Jonas, D. E. (2016) · JAMA · doi:10.1001/jama.2016.10357 · Cited by 107
Preexposure Prophylaxis for the Prevention of HIV
Chou, R., Spencer, H. C., Bougatsos, C., Blazina, I., Ahmed, A. Y., & Selph, S. (2023) · JAMA · doi:10.1001/jama.2023.9865 · Cited by 101
Postoperative Early Major and Minor Complications in Laparoscopic Vertical Sleeve Gastrectomy (LVSG) Versus Laparoscopic Roux-en-Y Gastric Bypass (LRYGB) Procedures: A Meta-Analysis and Systematic Review
Osland, E., Yunus, R. M., Khan, S., Alodat, T., Memon, B., & Memon, M. A. (2016) · Obesity Surgery · doi:10.1007/s11695-016-2101-8 · Cited by 93
“Similarly, re-analysis of existing meta-analyses in the literature utilizing the IVhet model or similar models that produce more conservative estimates than the REM demonstrates the potential impact the different models may have on the results obtained, of which the clinical implications are significant [47, 48].”
Result
Warfarin use and stroke, bleeding and mortality risk in patients with end stage renal disease and atrial fibrillation: a systematic review and meta-analysis
Tan, J., Liu, S., Segal, J. B., Alexander, G. C., & McAdams‐DeMarco, M. (2016) · BMC Nephrology · doi:10.1186/s12882-016-0368-6 · Cited by 95
“We conducted our quantitative assessment using the Knapp-Hartung method based on small-sample adjustments [12], which provided more accurate confidence limits than the DerSimonian-Laird estimator and has been advocated as an alternative method for metaanalysis with a limited number of studies [39, 40].”
Method
Causality between non-alcoholic fatty liver disease and risk of cardiovascular disease and type 2 diabetes: a meta-analysis with bias analysis
Morrison, A. E., Zaccardi, F., Khunti, K., & Davies, M. J. (2019) · Liver International · doi:10.1111/liv.13994 · Cited by 92
Effects of uric acid-lowering therapy in patients with chronic kidney disease: A meta-analysis
Su, X., Xu, B., Yan, B., Qiao, X., & Wang, L. (2017) · PLoS ONE · doi:10.1371/journal.pone.0187550 · Cited by 91
Device Closure Versus Medical Therapy Alone for Patent Foramen Ovale in Patients With Cryptogenic Stroke
Shah, R., Nayyar, M., Jovin, I. S., Abdul-Rashid, H. A., Bondy, B. R., Fan, T. M., Flaherty, M. P., & Rao, S. V. (2018) · Annals of Internal Medicine · doi:10.7326/m17-2679 · Cited by 89
“We chose the Knapp–Hartung method because it captures the uncertainty associated with statistical heterogeneity better than the traditional DerSimonian– Laird method (16).”

Authors: Cornell, J. E., Mulrow, C. D., Localio, R., Stack, C. B., Meibohm, A. R., Guallar, E., & Goodman, S. N.
Affiliations: Johns Hopkins University (US)
From University of Texas Health Science Center at San Antonio, San Antonio, Texas; University of Pennsylvania and American College of Physicians, Philadelphia, Pennsylvania; Johns Hopkins School of Public Health, Baltimore, Maryland; and Stanford University School of Medicine, Stanford, California.
ORCID: Cornell, John E.
Mulrow, Cynthia D.
Goodman, Steven N.
Venue: Annals of Internal Medicine
Publisher: American College of Physicians
Published: 2014-02-18
Volume: 160
Issue: 4
Pages: 267-270
ISSN: 0003-4819 (ISSN-L)
1539-3704
DOI: 10.7326/m13-2886
OpenAlex: W2129670822
Cited by: 453
References: 28
Citations by year: 2026: 17 · 2025: 34 · 2024: 30 · 2023: 17 · 2022: 26 · 2021: 54 · 2020: 34 · 2019: 58 · 2018: 42 · 2017: 41 · 2016: 51 · 2015: 28 · 2014: 20
FWCI: 12.77
Type: journal article
Language: en
Open access: closed
Licence: www.acpjournals.org/journal/aim/text-and-data-mining (tdm)
Locations: Annals of Internal Medicine (publishedVersion)
PubMed (publishedVersion)
Topics: Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
Social Sciences › Sociology and Political Science › Delphi Technique in Research
Keywords: random-effects meta-analysis, heterogeneity, treatment effect estimation, confidence limits, critical synthesis
MeSH terms: Data Interpretation, Statistical, Software, Meta-Analysis as Topic, Confidence Intervals

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Random-Effects Meta-analysis of Inconsistent Effects: A Time for Change

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks