The DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
Random-Effects Meta-analysis of Inconsistent Effects: A Time for Change
Abstract
A primary goal of meta-analysis is to improve the estimation of treatment effects by pooling results of similar studies. This article explains how the most widely used method for pooling heterogeneous studies--the Der Simonian-Laird (DL) estimator--can produce biased estimates with falsely high precision. A classic example is presented to show that use of the DL estimator can lead to erroneous conclusions. Particular problems with the DL estimator are discussed, and several alternative methods for summarizing heterogeneous evidence are presented. The authors support replacing universal use of the DL estimator with analyses based on a critical synthesis that recognizes the uncertainty in the evidence,focuses on describing and explaining the probable sources of variation in the evidence, and uses random-effects estimates that provide more accurate confidence limits than the DL estimator.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The Knapp-Hartung approach (SJ) is known to suit systematic reviews including few studies, particularly when dealing with 5 or fewer trials [49].”
From Mindfulness- And acceptance-based interventions for patients with fibromyalgia - A systematic review and meta-analyses (2019) · cites this paper as [49]
In the citing paper
“Furthermore, many frequentist estimation methods (including REML) sometimes fail to produce reliable confidence or prediction intervals.”
From A framework for meta-analysis of prediction model studies with binary and time-to-event outcomes (2019) · cites this paper as [47]
In the citing paper
“when the heterogeneity is small and the study sizes are mixed, the Hartung–Knapp method produces confidence intervals that are too wide.46,47”
From A framework for meta-analysis of prediction model studies with binary and time-to-event outcomes (2019) · cites this paper as [47]
Citations
Cites
Meta-analysis in clinical trials
DerSimonian, R., & Laird, N. (1986) · Controlled Clinical Trials · doi:10.1016/0197-2456(86)90046-2 · Cited by 39,552
Quantifying heterogeneity in a meta‐analysis
Higgins, J. P. T., & Thompson, S. G. (2002) · Statistics in Medicine · doi:10.1002/sim.1186 · Cited by 36,763
Conducting Meta-Analyses in R with the metafor Package
Viechtbauer, W. (2010) · Journal of Statistical Software · doi:10.18637/jss.v036.i03 · Cited by 20,464
A Re-Evaluation of Random-Effects Meta-Analysis
Higgins, J. P. T., Thompson, S. G., & Spiegelhalter, D. J. (2009) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/j.1467-985x.2008.00552.x · Cited by 2,569
Random-effects model for meta-analysis of clinical trials: An update
DerSimonian, R., & Kacker, R. (2007) · Contemporary Clinical Trials · doi:10.1016/j.cct.2006.04.004 · Cited by 2,374
Improved tests for a random effects meta‐regression with a single covariate
Knapp, G., & Hartung, J. (2003) · Statistics in Medicine · doi:10.1002/sim.1482 · Cited by 2,075
Meta‐Analysis Of Controlled Clinical Trials
Whitehead, A. (2002) · doi:10.1002/0470854200 · Cited by 827
A comparison of statistical methods for meta‐analysis
Brockwell, S. E., & Gordon, I. R. (2001) · Statistics in Medicine · doi:10.1002/sim.650 · Cited by 573
BORROWING STRENGTH FROM EXTERNAL TRIALS IN A META-ANALYSIS
Higgins, J. P. T., & WHITEHEAD, A. (1996) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19961230)15:24 · Cited by 559
Lambert, P. C., Sutton, A. J., Burton, P. R., Abrams, K. R., & Jones, D. R. (2005) · Statistics in Medicine · doi:10.1002/sim.2112 · Cited by 511
Flagged first, then by citations
Research works printed but not linked: 1
- Weseley AC, 1962, Obstet Gynecol, 19, 355
Cited by
Sorted by citations
Items 101–200 of 452
Yoga for treating low back pain: a systematic review and meta-analysis
Anheyer, D., Haller, H., Lauche, R., Dobos, G. J., & Cramer, H. (2021) · Pain · doi:10.1097/j.pain.0000000000002416 · Cited by 95
Dzhambov, A. M., & Dimitrova, D. (2018) · Environmental Pollution · doi:10.1016/j.envpol.2018.04.122 · Cited by 87
A meta-analysis of metal biosorption by suspended bacteria from three phyla
Fathollahi, A., Khasteganan, N., Coupe, S. J., & Newman, A. (2020) · Chemosphere · doi:10.1016/j.chemosphere.2020.129290 · Cited by 93
Subclinical Thyroid Dysfunction and the Risk for Fractures: A Systematic Review and Meta-analysis
Wirth, C. D., Blum, M. R., da Costa, B. R., Baumgartner, C., Collet, T.-H., Medici, M., Peeters, R. P., Aujesky, D., Bauer, D. C., & Rodondi, N. (2014) · Annals of Internal Medicine · doi:10.7326/m14-0125 · Cited by 85
Percutaneous Closure Versus Medical Treatment in Stroke Patients With Patent Foramen Ovale
Rosa, S. D., Sievert, H., Sabatino, J., Polimeni, A., Sorrentino, S., & Indolfi, C. (2018) · Annals of Internal Medicine · doi:10.7326/m17-3033 · Cited by 84
“The random-effects model with the Hartung–Knapp–Sidik–Jonkman method to estimate tau was used to compute estimates for the summary effect (21, 22).”
MethodYoga for Treating Headaches: a Systematic Review and Meta-analysis
Anheyer, D., Klose, P., Lauche, R., Saha, F. J., & Cramer, H. (2019) · Journal of General Internal Medicine · doi:10.1007/s11606-019-05413-9 · Cited by 89
Tonsillectomy for IgA Nephropathy: A Meta-analysis
Liu, L., Wang, L., Jiang, Y., Yao, L., Dong, L., Li, Z., & Li, X. (2014) · American Journal of Kidney Diseases · doi:10.1053/j.ajkd.2014.06.036 · Cited by 86
Goode, A. P., Hall, K. S., Batch, B. C., Huffman, K. M., Hastings, S. N., Allen, K. D., Shaw, R. J., Kanach, F. A., McDuffie, J. R., Kosinski, A. S., et al. (2017) · Annals of Behavioral Medicine · doi:10.1007/s12160-016-9829-1 · Cited by 80
Hartung–Knapp method is not always conservative compared with fixed‐effect meta‐analysis
Wiksten, A. S., Rücker, G., & Schwarzer, G. (2016) · Statistics in Medicine · doi:10.1002/sim.6879 · Cited by 78
“In recent publications, the use of the classic DerSimonian–Laird method has been condemned, and the Hartung–Knapp method was suggested as possible alternative both in simulations [9] and in empirical evaluations either based on 689 meta-analyses from Cochrane reviews [9] or a single classic example [11].”
MethodJutkowitz, E., Brasure, M., Fuchs, E. L., Shippee, T. P., Kane, R. A., Fink, H. A., Butler, M. E., Sylvanus, T., & Kane, R. L. (2016) · Journal of the American Geriatrics Society · doi:10.1111/jgs.13936 · Cited by 88
“When it was judged that comparisons were reasonable to pool (comparable interventions, outcome measures, and effect sizes), Knapp-Hartung random effects models in R was used to generate pooled estimates of weighted mean differences and standardized mean differences with corresponding 95% confidence intervals (CIs).(24…”
Methoddo Nascimento, I. J. B., von Groote, T., O’Mathúna, D. P., Abdulazeem, H. M., Henderson, C., Jayarajah, U., Weerasekara, I., Peričić, T. P., Klapproth, H., Puljak, L., et al. (2020) · PLoS ONE · doi:10.1371/journal.pone.0239235 · Cited by 82
“QE stabilizes the variance and mitigates the issue of a conventional random effects model underestimating standard errors in presence of high heterogeneity [17, 18].”
BackgroundAlharbi, T. A., Paudel, S., Gasevic, D., Ryan, J., Freak-Poli, R., & Owen, A. J. (2021) · Age and Ageing · doi:10.1093/ageing/afaa231 · Cited by 80
Steps in undertaking meta-analysis and addressing heterogeneity in meta-analysis
Gandhi, A. P., Shamim, M. A., & Padhi, B. K. (2023) · The Evidence · doi:10.61505/evidence.2023.1.1.7 · Cited by 76
Bosch, F. T. M., Mulder, F. I., Kamphuisen, P. W., Middeldorp, S., Bossuyt, P. M., Buller, H. R., & van Es, N. (2020) · Blood Advances · doi:10.1182/bloodadvances.2020003115 · Cited by 73
“Strengths of this study include the use of a profile-likelihood random-effects model to provide a more conservative yet more reliable CI than the more widely used Dersimonian-Laird randomeffects model.(24,26) Certainty of evidence was rated using the standardized GRADE approach to aid in the interpretation of the find…”
MethodMeta-analysis supports GWAS-implicated link between GRM3 and schizophrenia risk
Saini, S. M., Mancuso, S. G., Mostaid, M. S., Liu, C., Pantelis, C., Everall, I., & Bousman, C. (2017) · Translational Psychiatry · doi:10.1038/tp.2017.172 · Cited by 73
Masclee, G. M. C., Straatman, H., Arfè, A., Castellsagué, J., Garbe, E., Herings, R. M. C., Kollhorst, B., Lucchi, S. R., Perez‐Gutthann, S., Romio, S., et al. (2018) · PLoS ONE · doi:10.1371/journal.pone.0204746 · Cited by 81
Bayesian estimation in random effects meta‐analysis using a non‐informative prior
Bodnar, O., Link, A., Arendacká, B., Possolo, A., & Elster, C. (2017) · Statistics in Medicine · doi:10.1002/sim.7156 · Cited by 73
Boehmer, K. R., Barakat, S., Ahn, S., Prokop, L. J., Erwin, P. J., & Murad, M. H. (2016) · Systematic Reviews · doi:10.1186/s13643-016-0316-3 · Cited by 81
“If the number of studies in a particular analysis is large (15–20), we will use the DerSimonian Laird method [22]; if the number is smaller, we will use the Knapp–Hartung small-sample estimator approach method [23].”
MethodPump, pipes, and filter: do SGLT2 inhibitors cover it all?
Verma, S., Jüni, P., & Mazer, C. D. (2018) · The Lancet · doi:10.1016/s0140-6736(18)32824-1 · Cited by 74
Pain Management Injection Therapies for Low-back Pain
Chou, R. (2015) · Brown Digital Repository · doi:10.7301/z0ks6phj · Cited by 71
Wang, C., & Lee, W. (2019) · Research Synthesis Methods · doi:10.1002/jrsm.1345 · Cited by 72
“Although the DLmethod is the most popular method, it is possibly biased when there is substantial heterogeneity and/or the number of studies is small.(29,30) There are many other estimators and simulation studies suggested using alternatives such as the Paule‐Mandel method(31) or the restricted maximum likelihood meth…”
MethodKunutsor, S. K., Seidu, S., & Khunti, K. K. (2017) · European Heart Journal · doi:10.1093/eurheartj/ehx107 · Cited by 70
“There were only a small number of studies available for pooling and there is a possibility that the precision of the summary estimates for PE and DVT may have been overestimated; given that the DerSimonian and Laird random effect model tends to produce narrow confidence intervals when there are small number of studies…”
ResultFu, R., & Holmer, H. K. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2016.01.034 · Cited by 71
Self-Reported Trait Mindfulness and Couples’ Relationship Satisfaction: a Meta-Analysis
Quinn‐Nilas, C. (2020) · Mindfulness · doi:10.1007/s12671-020-01303-y · Cited by 78
D’Elia, L., Rossi, G., di Cola, M. S., Savino, I. G., Galletti, F., & Strazzullo, P. (2015) · Clinical Journal of the American Society of Nephrology · doi:10.2215/cjn.09110914 · Cited by 68
Empirical evidence about inconsistency among studies in a pair‐wise meta‐analysis
Rhodes, K. M., Turner, R., & Higgins, J. P. T. (2015) · Research Synthesis Methods · doi:10.1002/jrsm.1193 · Cited by 65
“When it is appropriate to combine studies whose estimates differ considerably, methods that better account for uncertainty than the DL procedure are recommended (Cornell et al , 2014).”
Selecting the best meta-analytic estimator for evidence-based practice: a simulation study
Doi, S. A. R., & Furuya‐Kanamori, L. (2019) · International Journal of Evidence-Based Healthcare · doi:10.1097/xeb.0000000000000207 · Cited by 66
“The second issue arises due to the assumption that the differences between the study results originate because the studies are measuring different effects.(17,28,29) However, if this is true, the appropriateness of meta-analysing the studies should be questioned.”
ResultTwelve recommendations for integrating existing systematic reviews into new reviews: EPC guidance
Robinson, K. A., Chou, R., Berkman, N. D., Newberry, S. J., Fu, R. F., Hartling, L., Dryden, D. M., Butler, M. E., Foisy, M., Anderson, J., et al. (2015) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.05.035 · Cited by 65
Zaragoza‐Macias, E., Zaidi, A. N., Dendukuri, N., & Marelli, A. J. (2018) · Circulation · doi:10.1161/cir.0000000000000604 · Cited by 65
Nonpharmacologic Treatments for Attention-Deficit/Hyperactivity Disorder: A Systematic Review
Goode, A. P., Coeytaux, R. R., Maslow, G., Davis, N. O., Hill, S., Namdari, B., LaPointe, N. M. A., Befus, D., Lallinger, K. R., Bowen, S., et al. (2018) · PEDIATRICS · doi:10.1542/peds.2018-0094 · Cited by 64
Endotrophin, a Collagen VI Formation–Derived Peptide, in Heart Failure
Chirinos, J. A., Zhao, L., Reese‐Petersen, A. L., Cohen, J. B., Genovese, F., Richards, M., Doughty, R. N., Dı́ez, J., González, A., Querejeta, R., et al. (2022) · NEJM Evidence · doi:10.1056/evidoa2200091 · Cited by 62
Dennis, B. B., Naji, L., Bawor, M., Bonner, A., Varenbut, M., Daiter, J., Plater, C., Paré, G., Marsh, D. C., Worster, A., et al. (2014) · Systematic Reviews · doi:10.1186/2046-4053-3-105 · Cited by 63
Increased risks for random errors are common in outcomes graded as high certainty of evidence
Gartlehner, G., Nußbaumer-Streit, B., Wagner, G., Patel, S. V., Evans, T. S., Dobrescu, A. I., & Gluud, C. N. (2018) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2018.10.009 · Cited by 63
Dorman, D. C., Chiu, W. A., Hales, B. F., Hauser, R. B., Johnson, K. J., Mantus, E., Martel, S., Robinson, K. A., Rooney, A. A., Rudel, R. A., et al. (2018) · Journal of Toxicology and Environmental Health Part B · doi:10.1080/10937404.2018.1514829 · Cited by 64
Alageel, S., Gulliford, M. C., McDermott, L. M., & Wright, A. J. (2017) · BMJ Open · doi:10.1136/bmjopen-2016-015375 · Cited by 58
Daliri, S., Boujarfi, S., el Mokaddam, A., Reimer, W. J. M. S. O., ter Riet, G., den Haan, C., Buurman, B. M., & Karapinar‐Çarkit, F. (2020) · BMJ Quality & Safety · doi:10.1136/bmjqs-2020-010927 · Cited by 61
“As stated by Cornell et al , 59 pooling studies in the face of heterogeneity can cause uncertainty of evidence.”
Kuo, C.-C., Wang, C.-C., Chang, W., Liao, T.-C., Chen, P., & Tung, T. (2021) · Evidence-based Complementary and Alternative Medicine · doi:10.1155/2021/6651238 · Cited by 61
“-e statistical assumption of the normal distribution for random effects does not fit the principle of randomization [35].”
BackgroundChen, J., Hsu, T., Liu, J.-H., Pan, H., Lai, C., Yang, S., & Wu, V. (2025) · American Journal of Kidney Diseases · doi:10.1053/j.ajkd.2024.11.013 · Cited by 71
Zaragoza‐Macias, E., Zaidi, A. N., Dendukuri, N., & Marelli, A. J. (2018) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2018.08.1030 · Cited by 61
Anheyer, D., Leach, M. J., Klose, P., Dobos, G., & Cramer, H. (2019) · Cephalalgia · doi:10.1177/0333102418781795 · Cited by 76
Twisk, J. W. R. (2019) · Cambridge University Press eBooks · doi:10.1017/9781108635660 · Cited by 59
Collet, C. A., Asano, T., Miyazaki, Y., Tenekecioğlu, E., Katagiri, Y., Sotomi, Y., Cavalcante, R., de Winter, R. J., Kimura, T., Gao, R., et al. (2017) · European Heart Journal · doi:10.1093/eurheartj/ehx155 · Cited by 58
“The incidence of VLDT was 1.4% (95% CI 0.8–2.5%), which represents a three-fold increase in the risk of VLDT compared with EES.”
MethodInfluentialWang, Y., Parpia, S., Couban, R., Wang, Q., Armijo-Olivo, S., Bassler, D., Briel, M., Brignardello-Petersen, R., Gluud, L. L., Keitz, S. A., et al. (2024) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2023.11.001 · Cited by 58
Liao, D., Ma, L., Liu, J., & Fu, P. (2019) · PLoS ONE · doi:10.1371/journal.pone.0210213 · Cited by 60
Early goal-directed therapy for severe sepsis and septic shock: A living systematic review
Simpson, S. Q., Gaines, M., Hussein, Y., & Badgett, R. G. (2016) · Journal of Critical Care · doi:10.1016/j.jcrc.2016.06.017 · Cited by 55
Su, X., Yan, B., Wang, L., Lv, J., Cheng, H., & Chen, Y. (2021) · American Journal of Kidney Diseases · doi:10.1053/j.ajkd.2021.02.328 · Cited by 54
Do worms protect against the metabolic syndrome? A systematic review and meta-analysis
Tracey, E. F., McDermott, R. A., & McDonald, M. (2016) · Diabetes Research and Clinical Practice · doi:10.1016/j.diabres.2016.08.014 · Cited by 53
Chou, R., Totten, A. M., Carney, N., Dandy, S., Fu, R. F., Grusing, S., Pappas, M., Wasson, N., & Newgard, C. D. (2017) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2016.11.032 · Cited by 56
“We also performed analyses with the profile likelihood method, an alternative random-effects model.(23) We defined a small difference in the AUROC a priori as a difference of less than 0.”
MethodKołodziejczak, M. M., Andreotti, F., Kowalewski, M., Buffon, A., Ciccone, M. M., Parati, G., Scicchitano, P., Umińska, J. M., Servi, S. D., Bliden, K. P., et al. (2017) · Annals of Internal Medicine · doi:10.7326/m17-0120 · Cited by 55
“Pooled HRs were calculated by using the Knapp–Hartung small-sample estimator method (10).”
MethodZhao, G., Bhatia, D., Jung, F., & Lipscombe, L. (2021) · Diabetologia · doi:10.1007/s00125-020-05343-w · Cited by 54
“The DerSimonian– Laird method may also underestimate confidence intervals and lead to erroneous conclusions about the statistical significance of findings when the meta-analysis sample size is small and statistical heterogeneity is high [60, 61].”
MethodWang, K.-L., Lópes, R. D., Patel, M. R., Buller, H. R., Tan, D. S., Chiang, C., & Giugliano, R. P. (2018) · European Heart Journal · doi:10.1093/eurheartj/ehy802 · Cited by 54
Complications After Percutaneous Ultrasound‐Guided Liver Biopsy
Tian, G., Kong, D.-X., Jiang, T., & Li, L. (2020) · Journal of Ultrasound in Medicine · doi:10.1002/jum.15229 · Cited by 53
“Thus we, used random-effect models to pool incidence rates for those subgroups, which could have caused narrow confidence intervals when heterogeneity existed, particularly when there were few studies.(61) In addition, subgroup analyses were not performed for the biopsy site and tumor pathologic type, which were not a…”
MethodTyler, B., Walford, H., Tamblyn, J. A., Keay, S. D., Mavrelos, D., Yasmin, E., & Wattar, B. H. A. (2022) · Human Reproduction Update · doi:10.1093/humupd/dmac009 · Cited by 52
“We were unable to explore certain effect modifiers due to reporting limitations, however, we attempted to address this by implementing a random-effect model to adjust our pooled effect estimate (Cornell et al., 2014).”
Transitional Care Interventions for Older Residents of Long-term Care Facilities
Birtwell, K., Planner, C., Hodkinson, A., Hall, A., Giles, S. J., Campbell, S., Tyler, N., Panagioti, M., & Daker‐White, G. (2022) · JAMA Network Open · doi:10.1001/jamanetworkopen.2022.10192 · Cited by 50
The effects of cognitive‐behavioral therapy for eating disorders on quality of life: A meta‐analysis
Linardon, J., & Brennan, L. (2017) · International Journal of Eating Disorders · doi:10.1002/eat.22719 · Cited by 57
“However, Bayesian procedures require knowledge of a prior distribution (the range of possible values) for s, which is often not known (Cornell et al., 2014; Liem et al., 2010).”
BackgroundYeung, T., Shannon, B., Perillo, S., Nehme, Z., Jennings, P., & Olaussen, A. (2019) · Emergency Medicine Australasia · doi:10.1111/1742-6723.13288 · Cited by 50
Screening and Supplementation for Iron Deficiency and Iron Deficiency Anemia During Pregnancy
Cantor, A. G., Holmes, R. S., Bougatsos, C., Atchison, C., DeLoughery, T. G., & Chou, R. (2024) · JAMA · doi:10.1001/jama.2024.13546 · Cited by 48
Road Traffic Noise Exposure and Birth Outcomes: An Updated Systematic Review and Meta-Analysis
Dzhambov, A. M., & Lercher, P. (2019) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph16142522 · Cited by 48
Saleem, K., Franz, J. K., Klem, M. L., Yabes, J. G., Boyiadzis, M. M., Jones, J. R., Shaikh, N., & Lontos, K. (2022) · The Lancet Haematology · doi:10.1016/s2352-3026(22)00289-7 · Cited by 50
Dzhambov, A. M., & Dimitrova, D. (2017) · Annals of Work Exposures and Health · doi:10.1093/annweh/wxx078 · Cited by 48
“Coverage of the RE model declines below the intended nominal level of 95% with heterogeneity because the computed variance is smaller than the true variance (Brockwell and Gordon, 2007; Cornell et al., 2014).”
BackgroundSiontis, G. C. M., Praz, F., Lanz, J., Vollenbroich, R., Roten, L., Stortecky, S., Räber, L., Windecker, S., & Pilgrim, T. (2017) · Heart · doi:10.1136/heartjnl-2017-312310 · Cited by 47
Shepherd‐Banigan, M. E., Goldstein, K. M., Coeytaux, R. R., McDuffie, J. R., Goode, A. P., Kosinski, A. S., Noord, M. V., Befus, D., Adam, S. S., Masilamani, V., et al. (2017) · Complementary Therapies in Medicine · doi:10.1016/j.ctim.2017.08.011 · Cited by 46
Dual Versus Triple Therapy for Atrial Fibrillation After Percutaneous Coronary Intervention
Khan, S. U., Osman, M., Khan, M. U., Khan, M. S., Zhao, D., Mamas, M. A., Savji, N., Al‐Abdouh, A., Hasan, R. K., & Michos, E. D. (2020) · Annals of Internal Medicine · doi:10.7326/m19-3763 · Cited by 49
“The DerSimonian–Laird random-effects model is known to substantially underestimate the 95% CI when only a few studies report effect estimates (31).”
BackgroundLow‐event‐rate meta‐analyses of clinical trials: implementing good practices
Shuster, J. J., & Walker, M. A. (2016) · Statistics in Medicine · doi:10.1002/sim.6844 · Cited by 45
“[37] suggests sunsetting the method in these low-event scenarios.”
MethodActive video games for rehabilitation in respiratory conditions: Systematic review and meta-analysis
Simmich, J., Deacon, A. J., & Russell, T. G. (2019) · JMIR Serious Games · doi:10.2196/10116 · Cited by 43
“This method is recommended over the more common DerSimonian and Laird (DSL) method [46,47] for meta-analyses where only few studies are available (the DSL method was also calculated for comparison).”
Methoddo Nascimento, I. J. B., Jerončić, A., Arantes, A., Brady, W. J., Guimarães, N. S., Antunes, N. S., Junior, G. C. C., & Marcolino, M. S. (2021) · The American Journal of Emergency Medicine · doi:10.1016/j.ajem.2021.04.010 · Cited by 48
Effect of a low-salt diet on chronic kidney disease outcomes: a systematic review and meta-analysis
Shi, H., Su, X., Chun-fang, L., Guo, W., & Wang, L. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-050843 · Cited by 43
Wong, B. C., Collister, D., Muneer, M., Storie, D., Courtney, M. J., Lloyd, A. M., Campbell, S., & Pauly, R. P. (2017) · American Journal of Kidney Diseases · doi:10.1053/j.ajkd.2017.01.047 · Cited by 44
Bavishi, C., Whelton, P. K., Mancia, G., Corrao, G., & Messerli, F. H. (2021) · Journal of Hypertension · doi:10.1097/hjh.0000000000002784 · Cited by 44
“We performed sensitivity analyses using a more conservative Hartung– Knapp–Sidik–Jonkman model for random-effects metaanalysis [7].”
MethodProtease activity as a prognostic factor for wound healing in venous leg ulcers
Westby, M. J., Dumville, J. C., Stubbs, N. C., Norman, G., Wong, J. K. F., Cullum, N. A., & Riley, R. D. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012841.pub2 · Cited by 40
Number of studies, heterogeneity, generalisability, and the choice of method for meta-analysis
Singh, A., Hussain, S., & Najmi, A. K. (2017) · Journal of the Neurological Sciences · doi:10.1016/j.jns.2017.09.026 · Cited by 43
Howse, P. M., Chibrikova, L., Twells, L., Barrett, B. J., & Gamble, J. (2016) · American Journal of Kidney Diseases · doi:10.1053/j.ajkd.2016.06.014 · Cited by 44
Nikolic, M., Pezzoli, P., Jaworska, N., & Seto, M. C. (2022) · Progress in Neuro-Psychopharmacology and Biological Psychiatry · doi:10.1016/j.pnpbp.2022.110596 · Cited by 41
Gates, M., Pillay, J., Nuspl, M., Wingert, A., Vandermeer, B., & Hartling, L. (2023) · Systematic Reviews · doi:10.1186/s13643-023-02181-w · Cited by 46
“In all cases, we used the restricted maximum likelihood estimation approach and the Hartun-Knapp-Sidnick-Jonkman correction to derive 95% CIs [73, 74].”
MethodRoush, G. C., Abdelfattah, R. S., Song, S., Ernst, M. E., Sica, D. A., & Kostis, J. B. (2018) · Journal of Clinical Hypertension · doi:10.1111/jch.13386 · Cited by 40
Mostaid, M. S., Mancuso, S. G., Liu, C., Sundram, S., Pantelis, C., Everall, I., & Bousman, C. (2017) · Translational Psychiatry · doi:10.1038/tp.2016.279 · Cited by 42
Huang, Y., Wei, S., Jiang, N., Zhang, L., Wang, S., Cao, X., Zhao, Y., & Wang, P. (2018) · BMC Cancer · doi:10.1186/s12885-018-5136-5 · Cited by 39
Janiaud, P., Agarwal, A., Tzoulaki, I., Τheodoratou, E., Tsilidis, K. K., Εvangelou, Ε., & Ioannidis, J. P. A. (2021) · BMC Medicine · doi:10.1186/s12916-021-02020-6 · Cited by 41
“P-value thresholds are set arbitrarily, the random effects meta-analysis may produce inconsistent results [84], the excess significance of bias has limited power if only a few studies are statistically significant [21, 22], and similarly both small-study effect and excess significance testing may be misleading when th…”
BackgroundWambua, S., Singh, M., Okoth, K., Snell, K. I. E., Riley, R. D., Yau, C., Thangaratinam, S., Nirantharakumar, K., Crowe, F. L., & on behalf of the MuM-PreDiCT Group (2024) · BMC Medicine · doi:10.1186/s12916-024-03284-4 · Cited by 46
“…with fewer than four studies should be interpreted with caution as the precision of the estimates might be affected where the number of studies included in a meta-analysis is small and heterogeneity is high [121, 122], and may be biased by confounding from other study-level characteristics.”
BackgroundJiesisibieke, Z. L., Tung, T., Xu, Q., Chen, P., Hsu, S.-Y., Liu, Y., & Chien, C.-W. (2019) · Renal Failure · doi:10.1080/0886022x.2019.1679644 · Cited by 38
“Secondly, surrounding random-effects model is controversial, that is, the assumption of normal distribution for random effects against the principle of randomization in statistical inference [29].”
BackgroundAndò, G., & Costa, F. (2019) · International Journal of Cardiology · doi:10.1016/j.ijcard.2019.12.054 · Cited by 40
Oral manifestation of the monkeypox virus: a systematic review and meta-analysis
Gandhi, A. P., Patro, S. K., Sandeep, M., Satapathy, P., Shamim, M. A., Kumar, V., Aggarwal, A. K., Padhi, B. K., & Sah, R. (2023) · EClinicalMedicine · doi:10.1016/j.eclinm.2022.101817 · Cited by 38
“42,43 Geography, endemicity, sexual practices and site-wise lesions could not explain the high heterogeneity among the included studies in the current meta-analysis.”
de Haan, A., Meiser‐Stedman, R., Landolt, M. A., Kuhn, I. L., Black, M., Klaus, K., Patel, S. D., Fisher, D. J., Haag, C., Ukoumunne, O. C., et al. (2023) · The Lancet Child & Adolescent Health · doi:10.1016/s2352-4642(23)00253-5 · Cited by 37
“22 A value of 0 indicated no heterogeneity.”
BackgroundAlkabbani, W., Zongo, A., Minhas‐Sandhu, J. K., Eurich, D. T., Shah, B. R., Alsabbagh, M. W., & Gamble, J. (2021) · Canadian Journal of Diabetes · doi:10.1016/j.jcjd.2021.12.005 · Cited by 38
Fixed Effect and Random Effects Meta-Analysis
Schwarzer, G., Carpenter, J. R., & Rücker, G. (2015) · Use R! · doi:10.1007/978-3-319-21416-0_2 · Cited by 34
Mitra, S., Flórez, I. D., Tamayo, M. E., Aune, D., Mbuagbaw, L. C. E., Veroniki, A. A., & Thabane, L. (2016) · BMJ Open · doi:10.1136/bmjopen-2016-011271 · Cited by 39
“The RE model assumes that the observed treatment effect for a study is a combination of a treatment effect common to all studies plus a component specific to that study alone.(29) 30 We will pool the outcome data using a Bayesian RE model.”
MethodHyperbaric oxygen therapy for people with autism spectrum disorder (ASD)
Xiong, T., Chen, H., Luo, R., & Mu, D. (2016) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd010922.pub2 · Cited by 37
Παπαδόπουλος, Α., Palaiopanos, K., Protogerou, A. D., Paraskevas, G. P., Tsivgoulis, G. K., & Georgakis, M. K. (2020) · Journal of Stroke · doi:10.5853/jos.2019.03335 · Cited by 36
Meta-analyses triggered by previous (false-)significant findings: problems and solutions
Schuit, E., Roes, K. C. B., Mol, B. W., Kwee, A., Moons, K. G. M., & Groenwold, R. H. H. (2015) · Systematic Reviews · doi:10.1186/s13643-015-0048-9 · Cited by 35
“Therefore, alternative, more reliable methods could be considered including small-sample adjustments, profile likelihood, or hierarchical Bayesian models [16].”
MethodResultSeroprevalence of Hepatitis B, C and D in Vietnam: A systematic review and meta-analysis
Flower, B., Duc, D. H., Kim, H. V. T., Minh, K. P., Geskus, R. B., Day, J., & Cooke, G. (2022) · The Lancet Regional Health - Western Pacific · doi:10.1016/j.lanwpc.2022.100468 · Cited by 35
Performing Meta-analyses with Very Few Studies
Schulz, A., Schürmann, C., Skipka, G., & Bender, R. (2021) · Methods in molecular biology · doi:10.1007/978-1-0716-1566-9_5 · Cited by 34
The P300 in alcohol use disorder: A meta-analysis and meta-regression
Hamidovic, A., & Wang, Y. (2019) · Progress in Neuro-Psychopharmacology and Biological Psychiatry · doi:10.1016/j.pnpbp.2019.109716 · Cited by 33
Petersen, J. M., Barrett, M., Ahrens, K. A., Murray, E. J., Bryant, A., HOGUE, C. J. R., Mumford, S. L., Gadupudi, S. S., Fox, M. P., & Trinquart, L. (2021) · Research Synthesis Methods · doi:10.1002/jrsm.1544 · Cited by 34
“We estimated between-study variance τ(2) by using the restricted maximum-likelihood estimator with KnappHartung adjustment.(63) We performed subgroup analyses stratified by overall confounding control and by control of each core construct.”
MethodChen, T., Tung, T., Chen, P., Wang, S., Chao, C.-M., Hsiung, N.-H., & Chi, C. (2016) · Evidence-based Complementary and Alternative Medicine · doi:10.1155/2016/9147974 · Cited by 32
“Another bias in this study is the controversy surrounding random-effects models; that is, the assumption of normally distributed random effects violates the basic principle of randomization in statistical inference [26].”
BackgroundLegha, A., Riley, R. D., Ensor, J., Snell, K. I. E., Morris, T. P., & Burke, D. L. (2018) · Statistics in Medicine · doi:10.1002/sim.7930 · Cited by 32
Su, X., Yan, B., Wang, L., Lv, J., Cheng, H., & Chen, Y. (2019) · BMC Nephrology · doi:10.1186/s12882-019-1499-3 · Cited by 32
Zhang, X.-L., Zhu, L., Wei, Z.-H., Zhu, Q.-Q., Qiao, J.-Z., Dai, Q., Huang, W., Li, X.-H., Xie, J., Kang, L.-N., et al. (2016) · Annals of Internal Medicine · doi:10.7326/m16-0006 · Cited by 31
López‐Pernas, S. (2023) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2023.3328913 · Cited by 39
“TheHartung-Knapp adjustment [67], [68] was also applied: a refined variance estimator that has been found to be preferable over standard methods of estimating between-study variance [69].”
MethodSinclair, A., Xie, X., Saab, L., & Dendukuri, N. (2016) · CMAJ Open · doi:10.9778/cmajo.20160087 · Cited by 31
Brahim, L. O., Lambert, S., Feeley, N., Coumoundouros, C., Schaffler, J. L., McCusker, J., Moodie, E. E. M., Kayser, J. W., Kolne, K., Belzile, É., et al. (2021) · BMC Psychiatry · doi:10.1186/s12888-021-03504-8 · Cited by 39
“This more conservative approach produces a wider con dence limit appropriate when the sample size is small (69).”
BackgroundMethod
Metadata
- Authors
- Cornell, J. E., Mulrow, C. D., Localio, R., Stack, C. B., Meibohm, A. R., Guallar, E., & Goodman, S. N.
- Affiliations
- Johns Hopkins University (US)
- From University of Texas Health Science Center at San Antonio, San Antonio, Texas; University of Pennsylvania and American College of Physicians, Philadelphia, Pennsylvania; Johns Hopkins School of Public Health, Baltimore, Maryland; and Stanford University School of Medicine, Stanford, California.
- Venue
- Annals of Internal Medicine
- Publisher
- American College of Physicians
- Published
- 2014-02-18
- Volume
- 160
- Issue
- 4
- Pages
- 267-270
- ISSN
- 0003-4819 (ISSN-L)
- 1539-3704
- OpenAlex
- W2129670822
- Cited by
- 453
- References
- 28
- Citations by year
- 2026: 17 · 2025: 34 · 2024: 30 · 2023: 17 · 2022: 26 · 2021: 54 · 2020: 34 · 2019: 58 · 2018: 42 · 2017: 41 · 2016: 51 · 2015: 28 · 2014: 20
- FWCI
- 12.77
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Annals of Internal Medicine (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Keywords
- random-effects meta-analysis, heterogeneity, treatment effect estimation, confidence limits, critical synthesis
- MeSH terms
- Data Interpretation, Statistical, Software, Meta-Analysis as Topic, Confidence Intervals
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
