The DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
Random-Effects Meta-analysis of Inconsistent Effects: A Time for Change
Abstract
A primary goal of meta-analysis is to improve the estimation of treatment effects by pooling results of similar studies. This article explains how the most widely used method for pooling heterogeneous studies--the Der Simonian-Laird (DL) estimator--can produce biased estimates with falsely high precision. A classic example is presented to show that use of the DL estimator can lead to erroneous conclusions. Particular problems with the DL estimator are discussed, and several alternative methods for summarizing heterogeneous evidence are presented. The authors support replacing universal use of the DL estimator with analyses based on a critical synthesis that recognizes the uncertainty in the evidence,focuses on describing and explaining the probable sources of variation in the evidence, and uses random-effects estimates that provide more accurate confidence limits than the DL estimator.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The Knapp-Hartung approach (SJ) is known to suit systematic reviews including few studies, particularly when dealing with 5 or fewer trials [49].”
From Mindfulness- And acceptance-based interventions for patients with fibromyalgia - A systematic review and meta-analyses (2019) · cites this paper as [49]
In the citing paper
“Furthermore, many frequentist estimation methods (including REML) sometimes fail to produce reliable confidence or prediction intervals.”
From A framework for meta-analysis of prediction model studies with binary and time-to-event outcomes (2019) · cites this paper as [47]
In the citing paper
“when the heterogeneity is small and the study sizes are mixed, the Hartung–Knapp method produces confidence intervals that are too wide.46,47”
From A framework for meta-analysis of prediction model studies with binary and time-to-event outcomes (2019) · cites this paper as [47]
Citations
Cites
Meta-analysis in clinical trials
DerSimonian, R., & Laird, N. (1986) · Controlled Clinical Trials · doi:10.1016/0197-2456(86)90046-2 · Cited by 39,552
Quantifying heterogeneity in a meta‐analysis
Higgins, J. P. T., & Thompson, S. G. (2002) · Statistics in Medicine · doi:10.1002/sim.1186 · Cited by 36,763
Conducting Meta-Analyses in R with the metafor Package
Viechtbauer, W. (2010) · Journal of Statistical Software · doi:10.18637/jss.v036.i03 · Cited by 20,464
A Re-Evaluation of Random-Effects Meta-Analysis
Higgins, J. P. T., Thompson, S. G., & Spiegelhalter, D. J. (2009) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/j.1467-985x.2008.00552.x · Cited by 2,569
Random-effects model for meta-analysis of clinical trials: An update
DerSimonian, R., & Kacker, R. (2007) · Contemporary Clinical Trials · doi:10.1016/j.cct.2006.04.004 · Cited by 2,374
Improved tests for a random effects meta‐regression with a single covariate
Knapp, G., & Hartung, J. (2003) · Statistics in Medicine · doi:10.1002/sim.1482 · Cited by 2,075
Meta‐Analysis Of Controlled Clinical Trials
Whitehead, A. (2002) · doi:10.1002/0470854200 · Cited by 827
A comparison of statistical methods for meta‐analysis
Brockwell, S. E., & Gordon, I. R. (2001) · Statistics in Medicine · doi:10.1002/sim.650 · Cited by 573
BORROWING STRENGTH FROM EXTERNAL TRIALS IN A META-ANALYSIS
Higgins, J. P. T., & WHITEHEAD, A. (1996) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19961230)15:24 · Cited by 559
A LIKELIHOOD APPROACH TO META-ANALYSIS WITH RANDOM EFFECTS
Hardy, R., & Thompson, S. G. (1996) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19960330)15:6 · Cited by 522
Flagged first, then by citations
Research works printed but not linked: 1
- Weseley AC, 1962, Obstet Gynecol, 19, 355
Cited by
Sorted by citations
Fleischmann, C., Scherag, A., Adhikari, N. K. J., Hartog, C. S., Tsaganos, T., Schlattmann, P., Angus, D. C., & Reinhart, K. (2016) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/rccm.201504-0781oc · Cited by 3,548
“(15) Balancing between taking care of heterogeneity by stratification and the availability of a considerable number of studies to be meta-analysed to derive a relatively robust estimator, we finally stratified all studies into two subgroups (hospital-treated sepsis and hospitaltreated severe sepsis).”
BackgroundMethods to estimate the between‐study variance and its uncertainty in meta‐analysis
Veroniki, A. A., Jackson, D., Viechtbauer, W., Bender, R., Bowden, J., Knapp, G., Kuss, O., Higgins, J. P., Langan, D., & Salanti, G. (2016) · Research Synthesis Methods · doi:10.1002/jrsm.1164 · Cited by 1,685
“…its default use has often been challenged in the sense that DL may underestimate the true between-study variance, potentially producing overly narrow confidence intervals (CIs) for the mean effect (Cornell et al., 2014), especially when the between-study variance is large (Novianti et al., 2014).”
A comparison of heterogeneity variance estimators in simulated random‐effects meta‐analyses
Langan, D., Higgins, J. P. T., Jackson, D., Bowden, J., Veroniki, A. A., Kontopantelis, E., Viechtbauer, W., & Simmonds, M. (2019) · Research Synthesis Methods · doi:10.1002/jrsm.1316 · Cited by 1,074
“Other methods include the profile likelihood method,(21) which has also been shown as a better alternative to the Wald method in simulated meta‐analysis data(42) and recommended elsewhere.(43) Second, a continuity correction of 0.”
MethodTufanaru, C., Munn, Z., Stephenson, M., & Aromataris, E. (2015) · International Journal of Evidence-Based Healthcare · doi:10.1097/xeb.0000000000000065 · Cited by 1,057
A guide to systematic review and meta-analysis of prognostic factor studies
Riley, R. D., Moons, K. G. M., Snell, K. I. E., Ensor, J., Hooft, L., Altman, D. G., Hayden, J. A., Collins, P. G. S., & Debray, T. P. A. (2019) · BMJ · doi:10.1136/bmj.k4597 · Cited by 771
Metadata
- Authors
- Cornell, J. E., Mulrow, C. D., Localio, R., Stack, C. B., Meibohm, A. R., Guallar, E., & Goodman, S. N.
- Affiliations
- Johns Hopkins University (US)
- From University of Texas Health Science Center at San Antonio, San Antonio, Texas; University of Pennsylvania and American College of Physicians, Philadelphia, Pennsylvania; Johns Hopkins School of Public Health, Baltimore, Maryland; and Stanford University School of Medicine, Stanford, California.
- Venue
- Annals of Internal Medicine
- Publisher
- American College of Physicians
- Published
- 2014-02-18
- Volume
- 160
- Issue
- 4
- Pages
- 267-270
- ISSN
- 0003-4819 (ISSN-L)
- 1539-3704
- OpenAlex
- W2129670822
- Cited by
- 453
- References
- 28
- Citations by year
- 2026: 17 · 2025: 34 · 2024: 30 · 2023: 17 · 2022: 26 · 2021: 54 · 2020: 34 · 2019: 58 · 2018: 42 · 2017: 41 · 2016: 51 · 2015: 28 · 2014: 20
- FWCI
- 12.77
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Annals of Internal Medicine (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Keywords
- random-effects meta-analysis, heterogeneity, treatment effect estimation, confidence limits, critical synthesis
- MeSH terms
- Data Interpretation, Statistical, Software, Meta-Analysis as Topic, Confidence Intervals
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
