Empirical vs natural weighting in random effects meta‐analysis
Open accessThe DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.
Abstract
This article brings into serious question the validity of empirically based weighting in random effects meta‐analysis. These methods treat sample sizes as non‐random, whereas they need to be part of the random effects analysis. It will be demonstrated that empirical weighting risks substantial bias. Two alternate methods are proposed. The first estimates the arithmetic mean of the population of study effect sizes per the classical model for random effects meta‐analysis. We show that anything other than an unweighted mean of study effect sizes will risk serious bias for this targeted parameter. The second method estimates a patient level effect size, something quite different from the first. To prevent inconsistent estimation for this population parameter, the study effect sizes must be weighted in proportion to their total sample sizes for the trial. The two approaches will be presented for a meta‐analysis of a nasal decongestant, while at the same time will produce counter‐intuitive results for the DerSimonian–Laird approach, the most popular empirically based weighted method. It is concluded that all past publications based on empirically weighted random effects meta‐analysis should be revisited to see if the qualitative conclusions hold up under the methods proposed herein. It is also recommended that empirically based weighted random effects meta‐analysis not be used in the future, unless strong cautions about the assumptions underlying these analyses are stated, and at a minimum, some form of secondary analysis based on the principles set forth in this article be provided to supplement the primary analysis. Copyright © 2009 John Wiley & Sons, Ltd.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Effect of Rosiglitazone on the Risk of Myocardial Infarction and Death from Cardiovascular Causes
Nissen, S. E., & Wolski, K. (2007) · New England Journal of Medicine · doi:10.1056/nejmoa072761 · Cited by 4,675 · cited 2×
“[2] and Nissen–Wolski [6], where a very few studies play a dominant role.”
Shuster, J. J., Jones, L. S., & Salmon, D. A. (2007) · Statistics in Medicine · doi:10.1002/sim.3060 · Cited by 105 · cited 1×
“See Shuster and colleagues [7], who provide a parallel development for an unweighted random effects meta-analysis to cover that situation.”
Self-citationKollar, C., Schneider, H., Waksman, J., & Krusinska, E. (2007) · Clinical Therapeutics · doi:10.1016/j.clinthera.2007.05.021 · Cited by 39
Cited by
A basic introduction to fixed-effect and random-effects models for meta-analysis
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2010) · Research Synthesis Methods · doi:10.1002/jrsm.12 · Cited by 6,793
IntHout, J., Ioannidis, J. P., & Borm, G. F. (2014) · BMC Medical Research Methodology · doi:10.1186/1471-2288-14-25 · Cited by 2,419
“Several other methods have been developed, like the Quantile Approximation (QA) method [28], the Profile Likelihood approach [29], natural weighting instead of empirically based weighting of studies [30], use of fixed effects estimates with a random effects approach to heterogeneity [31] and more recently, higher-orde…”
MethodHoaglin, D. C., Hawkins, N., Jansen, J. P., Scott, D. A., Itzler, R., Cappelleri, J. C., Boersma, C., Thompson, D., Larholt, K. M., Diaz, M., et al. (2011) · Value in Health · doi:10.1016/j.jval.2011.01.011 · Cited by 770
Meta-analysis methods for genome-wide association studies and beyond
Evangelou, E., & Ioannidis, J. P. A. (2013) · Nature Reviews Genetics · doi:10.1038/nrg3472 · Cited by 596
Doi, S. A. R., Barendregt, J. J., Khan, S., Thalib, L., & Williams, G. M. (2015) · Contemporary Clinical Trials · doi:10.1016/j.cct.2015.05.009 · Cited by 581
Metadata
- Authors
- Shuster, J. J.
- Venue
- Statistics in Medicine
- Publisher
- Wiley
- Published
- 2010-05-19
- Volume
- 29
- Issue
- 12
- Pages
- 1259-1265
- ISSN
- 0277-6715 (ISSN-L)
- 1097-0258
- OpenAlex
- W2144426506
- Cited by
- 122
- References
- 3
- FWCI
- 11.94
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Empirical vs natural weighting in random effects meta‐analysis. https://app.cochat.ai/papers/10.1002/sim.3607
- The paper
- Shuster, J. J. (2010). Empirical vs natural weighting in random effects meta‐analysis. Statistics in Medicine. https://doi.org/10.1002/sim.3607
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.