The DOI matches its Crossref record; 40 of the 40 references it cites were found in a registry or an index.
A Nonparametric “Trim and Fill” Method of Accounting for Publication Bias in Meta-Analysis
Abstract
Meta-analysis collects and synthesizes results from individual studies to estimate an overall effect size. If published studies are chosen, say through a literature review, then an inherent selection bias may arise, because, for example, studies may tend to be published more readily if they are statistically significant, or deemed to be more “interesting” in terms of the impact of their outcomes. We develop a simple rank-based data augmentation technique, formalizing the use of funnel plots, to estimate and adjust for the numbers and outcomes of missing studies. Several nonparametric estimators are proposed for the number of missing studies, and their properties are developed analytically and through simulations. We apply the method to simulated and epidemiological datasets and show that it is both effective and consistent with other criteria in the literature.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Publication bias was investigated by applying the trim and fill method, where effect sizes are recalculated to adjust for supposed publication biases (Duval & Tweedie, 2000).”
From Taking stock: A meta-analysis of the effects of foreign language enjoyment (2022) · cites this paper as Duval & Tweedie, 2000
In the citing paper
“The trim‐and‐fill procedure determined that nine studies should be added to create symmetry.”
From Problem-oriented policing for reducing crime and disorder: An updated systematic review and meta-analysis (2020) · cites this paper as Duval and Tweedie (2000)
In the citing paper
“Like many systematic reviews, our meta‐analyses used the trim‐and‐fill procedure to explore whether publication bias might be affecting the results and to estimate how the reported effects would change if the bias were to be removed (Duval & Tweedie, 2000; Duval, 2005).”
From Hot spots policing of small geographic areas effects on crime (2019) · cites this paper as Duval & Tweedie, 2000
In the citing paper
“When we encountered publication bias, we conducted a trim and fill adjusted analysis 25 to remove the most extreme small studies from the positive side of the funnel plot, and recalculated the effect size iteratively, until the funnel plot was symmetrical around the (new) effect size.”
From Diabetes mellitus in people with schizophrenia, bipolar disorder and major depressive disorder: a systematic review and large scale meta‐analysis (2016) · cites this paper as [25]
In the citing paper
“Variations in the risk for glucose abnormalities are evident in the literature, with the highest risk being associated with clozapine, olanzapine and quetiapine in carefully designed studies 25,34,35 .”
From Diabetes mellitus in people with schizophrenia, bipolar disorder and major depressive disorder: a systematic review and large scale meta‐analysis (2016) · cites this paper as [25]
Citations
Cites
Bias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722
Meta-analysis in clinical trials
DerSimonian, R., & Laird, N. (1986) · Controlled Clinical Trials · doi:10.1016/0197-2456(86)90046-2 · Cited by 39,552
Operating characteristics of a rank correlation test for publication bias
Begg, C. B., & Mazumdar, M. (1994) · Biometrics · doi:10.2307/2533446 · Cited by 16,955
Duval, S., & Tweedie, R. (2000) · Biometrics · doi:10.1111/j.0006-341x.2000.00455.x · Cited by 14,099
Self-citationWasserman, S. S., Hedges, L. V., & Olkin, I. (1988) · Journal of Educational Statistics · doi:10.2307/1164953 · Cited by 10,377
Casella, G., & Berger, R. L. (1993) · Chapman & Hall/CRC eBooks · doi:10.1201/9781003456285 · Cited by 5,835
The Handbook of Research Synthesis.
Howel, D., Cooper, H., & Hedges, L. V. (1995) · Biometrics · doi:10.2307/2533295 · Cited by 3,657
Publication bias in clinical research
Easterbrook, P. J., Gopalan, R., Berlin, J. A., & Matthews, D. R. (1991) · The Lancet · doi:10.1016/0140-6736(91)90201-y · Cited by 3,109 · cited 1×
“The negative binomial forms here are reminiscent of those used by Eberly and Casella (1999) and Givens et al.”
MethodSumming up: The Science of Reviewing Research
Wilkinson, I., Light, R. J., & Pillemer, D. B. (1987) · Journal of Educational Statistics · doi:10.2307/1164691 · Cited by 1,486
Light, R. J., & Pillemer, D. B. (1984) · Harvard University Press eBooks · doi:10.4159/9780674040243 · Cited by 1,422
Research works printed but not linked: 10
- 1983, British Medical Journal, 280, 508
- Cooper H., 1998, Synthesizing Research
- Dear K., 1997, Statistical Science, 12, 245
- DuMouchel W., 1997, Statistical Science, 12, 244
- Duval S. J., 1999, “Effects of Publication Bias in Meta-Analysis,”
- Duval S. J., “Robustness of the ‘Trim and Fill’ publication bias approach to different mechanisms for study suppression,”
- Eberly L. E., 1999, Journal of Official Statistics, 15
- Hedges L., 1985, Statistical Methods for Meta-Analysis
- Sugita M., 1994, Tokai Journal of Experimental and Clinical Medicine, 19, 29
- Sutton A. J., 1999, “The Impact of Publication Bias on Meta-Analyses Within the Cochrane Database of Systematic Reviews,”
Cited by
Sorted by citations
Conducting Meta-Analyses in R with the metafor Package
Viechtbauer, W. (2010) · Journal of Statistical Software · doi:10.18637/jss.v036.i03 · Cited by 20,464
Hao (2015) · doi:10.1002/14651858 · Cited by 11,321
A meta-analytic test of intergroup contact theory.
Pettigrew, T. F., & Tropp, L. R. (2006) · Journal of personality and social psychology · doi:10.1037/0022-3514.90.5.751 · Cited by 9,708
Social Relationships and Mortality Risk: A Meta-analytic Review
Holt-Lunstad, J., Smith, T. B., & Layton, J. B. (2010) · PLoS Medicine · doi:10.1371/journal.pmed.1000316 · Cited by 7,272
“Second, we employed the ‘‘trim and fill’’ methodology described by Duval and Tweedie [178,179] to estimate the number of studies missing due to publication bias, but this analysis failed to reveal any studies that would need to be created on the opposite side of the distribution, meaning that adjustment to the omnibus…”
MethodHow to perform a meta-analysis with R: a practical tutorial
Balduzzi, S., Rücker, G., & Schwarzer, G. (2019) · Evidence Based Mental Health · doi:10.1136/ebmental-2019-300117 · Cited by 5,827
Metadata
- Authors
- Duval, S., & Tweedie, R.
- Affiliations
- Colorado State University (US)
- University of Minnesota (US)
- ORCID
- Duval, Sue
- Venue
- Journal of the American Statistical Association
- Publisher
- Informa UK Limited
- Published
- 2000-03
- Volume
- 95
- Issue
- 449
- Pages
- 89-98
- ISSN
- 0162-1459 (ISSN-L)
- 1537-274X
- OpenAlex
- W2043565183
- Cited by
- 2,955
- References
- 40
- Citations by year
- 2026: 106 · 2025: 167 · 2024: 189 · 2023: 171 · 2022: 233 · 2021: 275 · 2020: 259 · 2019: 179 · 2018: 184 · 2017: 160 · 2016: 142 · 2015: 149 · 2014: 128 · 2013: 91 · 2012: 101
- FWCI
- 21.09
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of the American Statistical Association (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Keywords
- publication bias, trim and fill, missing studies, funnel plot, nonparametric estimators, selection bias, meta-analysis, effect size estimation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
