Assessing the Sensitivity of Meta-analysis to Selection Bias: A Multiple Imputation Approach
The DOI matches its Crossref record; 7 of the 7 references it cites were found in a registry or an index.
Abstract
Evidence synthesis, both qualitatively and quantitatively through meta-analysis, is central to the development of evidence-based medicine. Unfortunately, meta-analysis is often complicated by the suspicion that the available studies represent a biased subset of the evidence, possibly due to publication bias or other systematically different effects in small studies. A number of statistical methods have been proposed to address this, among which the trim-and-fill method and the Copas selection model are two of the most widely discussed. However, both methods have drawbacks: the trim-and-fill method is based on strong assumptions about the symmetry of the funnel plot; the Copas selection model is less accessible to systematic reviewers, and sometimes encounters estimation problems. In this article, we adopt a logistic selection model, and show how treatment effects can be rapidly estimated via multiple imputation. Specifically, we impute studies under a missing at random assumption, and then reweight to obtain estimates under nonrandom selection. Our proposal is computationally straightforward. It allows users to increase selection while monitoring the extent of remaining funnel plot asymmetry, and also visualize the results using the funnel plot. We illustrate our approach using a small meta-analysis of benign prostatic hyperplasia.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Ripley, B. D. (1987) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316726 · Cited by 2,241
Chan, A.-W., Hróbjartsson, A., Haahr, M. T., Gøtzsche, P. C., & Altman, D. G. (2004) · JAMA · doi:10.1001/jama.291.20.2457 · Cited by 1,845 · cited 2×
“, Chan et al., 2004; Rothstein, Sutton, and Borenstein, 2005; Nieminen et al., 2007). Such biases distort the evidence base for meta-analysis, potentially resulting in unduly optimistic inference for intervention effects. It is therefore important to detect and where appropriate correct for them. Commonly used figures…”
BackgroundMethodSterne, J. A., Egger, M., & Moher, D. (2008) · Cochrane Handbook for Systematic Reviews of Interventions · doi:10.1002/9780470712184.ch10 · Cited by 651
Meta-analysis, funnel plots and sensitivity analysis
Copas, J. (2000) · Biostatistics · doi:10.1093/biostatistics/1.3.247 · Cited by 503 · cited 4×
“3.2 Copas Selection Model An alternative that avoids making the relatively strong assumptions underpinning the trim-and-fill method is the probit-selection model approach proposed by Copas and coauthors, e.g., Copas and Shi (2000) and references therein.”
BackgroundMethodInfluentialPublication Bias in Research Synthesis: Sensitivity Analysis Using A Priori Weight Functions.
Vevea, J. L., & Woods, C. M. (2005) · Psychological Methods · doi:10.1037/1082-989x.10.4.428 · Cited by 461 · cited 2×
“Our approach is in the spirit of Vevea and Woods (2005), who propose prespecifying, rather than estimating, the weights in Hedges’ selection model, and look at the sensitivity of the conclusions as they vary these weights.”
MethodMoreno, S. G., Sutton, A. J., Ades, A., Stanley, T. D., Abrams, K. R., Peters, J. L., & Cooper, N. J. (2009) · BMC Medical Research Methodology · doi:10.1186/1471-2288-9-2 · Cited by 373
Modeling Publication Selection Effects in Meta-Analysis
Hedges, L. V. (1992) · Statistical Science · doi:10.1214/ss/1177011364 · Cited by 359
Cited by
Statistical methods for dealing with publication bias in meta‐analysis
Jin, Z., Zhou, X., & He, J. (2015) · Statistics in Medicine · doi:10.1002/sim.6342 · Cited by 252
Detecting and adjusting for small‐study effects in meta‐analysis
Rücker, G., Carpenter, J. R., & Schwarzer, G. (2011) · Biometrical Journal · doi:10.1002/bimj.201000151 · Cited by 132
“…known that for small or heterogeneous metaanalyses all funnel plot methods work poorly with respect to both size and power (Rücker et al., 2010; Carpenter et al., 2010). r 2010 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim www.biometrical-journal.com Another limitation is that we did not…”
BackgroundSelf-citationAdjustment for reporting bias in network meta-analysis of antidepressant trials
Trinquart, L., Chatellier, G., & Ravaud, P. (2012) · BMC Medical Research Methodology · doi:10.1186/1471-2288-12-150 · Cited by 122
“We adopt an approach based on a logistic selection model, as previously used in conventional MAs [18, 31].”
MethodMueller, K. F., Meerpohl, J. J., Briel, M., Antes, G., von Elm, E., Lang, B., Motschall, E., Schwarzer, G., & Bassler, D. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2016.04.015 · Cited by 48
Self-citation
Metadata
- Authors
- Carpenter, J., Rücker, G., & Schwarzer, G.
- Venue
- Biometrics
- Publisher
- Oxford University Press (OUP)
- Published
- 2010-10-29
- Volume
- 67
- Issue
- 3
- Pages
- 1066-1072
- ISSN
- 0006-341X (ISSN-L)
- OpenAlex
- W2028853315
- Cited by
- 39
- References
- 7
- FWCI
- 3.97
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Assessing the Sensitivity of Meta-analysis to Selection Bias: A Multiple Imputation Approach. https://app.cochat.ai/papers/10.1111/j.1541-0420.2010.01498.x
- The paper
- Carpenter, J., Rücker, G., & Schwarzer, G. (2011). Assessing the Sensitivity of Meta-analysis to Selection Bias: A Multiple Imputation Approach. Biometrics. https://doi.org/10.1111/j.1541-0420.2010.01498.x
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.