The DOI matches its Crossref record; 96 of the 96 references it cites were found in a registry or an index.
Publication Bias in Research Synthesis: Sensitivity Analysis Using A Priori Weight Functions.
Abstract
Publication bias, sometimes known as the "file-drawer problem" or "funnel-plot asymmetry," is common in empirical research. The authors review the implications of publication bias for quantitative research synthesis (meta-analysis) and describe existing techniques for detecting and correcting it. A new approach is proposed that is suitable for application to meta-analytic data sets that are too small for the application of existing methods. The model estimates parameters relevant to fixed-effects, mixed-effects or random-effects meta-analysis contingent on a hypothetical pattern of bias that is fixed independently of the data. The authors illustrate this approach for sensitivity analysis using 3 data sets adapted from a commonly cited reference work on research synthesis (H. M. Cooper & L. V. Hedges, 1994).
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“To evaluate publication bias, selection modeling was used. This approach used a weight function model created by Vevea and Woods [44], implemented through the weightr package.”
From Generative AI Mental Health Chatbots as Therapeutic Tools: Systematic Review and Meta-Analysis of Their Role in Reducing Mental Health Issues (2025) · cites this paper as [44]
In the citing paper
“confirming evidence of selection bias.”
From Generative AI Mental Health Chatbots as Therapeutic Tools: Systematic Review and Meta-Analysis of Their Role in Reducing Mental Health Issues (2025) · cites this paper as [44]
In the citing paper
“After adjustment for potential selection bias, the two-step model (P value cutpoints=.025, .50, 1) yielded a smaller and nonsignificant change in the pooled effect (g=0.54, SE=0.26, z=2.10; P=.04, 95% CI 0.04, 1.04).”
From Generative AI Mental Health Chatbots as Therapeutic Tools: Systematic Review and Meta-Analysis of Their Role in Reducing Mental Health Issues (2025) · cites this paper as [44]
In the citing paper
“The unadjusted model (k=44) estimated a pooled effect size of g=0.41 (SE=0.10, z=4.24; P<.001, 95% CI 0.22, 0.60).”
From Generative AI Mental Health Chatbots as Therapeutic Tools: Systematic Review and Meta-Analysis of Their Role in Reducing Mental Health Issues (2025) · cites this paper as [44]
In the citing paper
“studies with P<.025* were about 3.54 times more likely to be included than those with larger P values”
From Generative AI Mental Health Chatbots as Therapeutic Tools: Systematic Review and Meta-Analysis of Their Role in Reducing Mental Health Issues (2025) · cites this paper as [44]
In the citing paper
“suggesting moderate publication bias favoring statistically significant findings”
From Generative AI Mental Health Chatbots as Therapeutic Tools: Systematic Review and Meta-Analysis of Their Role in Reducing Mental Health Issues (2025) · cites this paper as [44]
In the citing paper
“The likelihood ratio test comparing adjusted and unadjusted models was significant (χ²2=11.25, P=.004)”
From Generative AI Mental Health Chatbots as Therapeutic Tools: Systematic Review and Meta-Analysis of Their Role in Reducing Mental Health Issues (2025) · cites this paper as [44]
In the citing paper
“Vevea and Woods’ (2005) selection model calculated a similar estimate (g¯ = 0.119).”
From Cognitive and academic benefits of music training with children: A multilevel meta-analysis (2020) · cites this paper as Vevea and Woods’ (2005)
In the citing paper
“The selection model estimate was g¯ = 0.039.”
From Cognitive and academic benefits of music training with children: A multilevel meta-analysis (2020) · cites this paper as Vevea and Woods’ (2005)
In the citing paper
“Vevea and Woods’ (2005) selection model calculated a more conservative estimate (g¯ = 0.126).”
From Cognitive and academic benefits of music training with children: A multilevel meta-analysis (2020) · cites this paper as Vevea and Woods’ (2005)
In the citing paper
“Vevea and Woods’ (2005) selection model yielded a similar estimate (g¯ = -0.002).”
From Cognitive and academic benefits of music training with children: A multilevel meta-analysis (2020) · cites this paper as Vevea and Woods’ (2005)
In the citing paper
“Sensitivity analysis (Vevea and Woods 2005) was also conducted to evaluate and account for the level of publication bias.”
From A meta-analytic review of the efficacy of physical exercise interventions on cognition in individuals with autism spectrum disorder and ADHD (2016) · cites this paper as Vevea and Woods 2005
In the citing paper
“Although the result of Kendall's tau is non-significant, in view of the small number of studies included in the metaanalysis, publication bias cannot be rejected and therefore a sensitivity analysis (Vevea and Woods, 2005) was conducted to estimate and account for the likelihood of publication bias.”
From A meta-analytic review of the efficacy of physical exercise interventions on cognition in individuals with autism spectrum disorder and ADHD (2016) · cites this paper as Vevea and Woods, 2005
Citations
Cites
Bias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722
Operating characteristics of a rank correlation test for publication bias
Begg, C. B., & Mazumdar, M. (1994) · Biometrics · doi:10.2307/2533446 · Cited by 16,955
Duval, S., & Tweedie, R. (2000) · Biometrics · doi:10.1111/j.0006-341x.2000.00455.x · Cited by 14,099
Wasserman, S. S., Hedges, L. V., & Olkin, I. (1988) · Journal of Educational Statistics · doi:10.2307/1164953 · Cited by 10,377
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068
Statistical Methods for Meta-Analysis.
Freeman, P. R., Hedges, L. V., & Olkin, I. (1986) · Biometrics · doi:10.2307/2531069 · Cited by 4,840
The Handbook of Research Synthesis.
Howel, D., Cooper, H., & Hedges, L. V. (1995) · Biometrics · doi:10.2307/2533295 · Cited by 3,657
Publication bias in clinical research
Easterbrook, P. J., Gopalan, R., Berlin, J. A., & Matthews, D. R. (1991) · The Lancet · doi:10.1016/0140-6736(91)90201-y · Cited by 3,109 · cited 2×
“…are surveyed about unpublished work (e.g., Chan, Sacks, & Chalmers, 1982; Coursol & Wagner, 1986; Dickersin, Chan, Chalmers, Sacks, & Smith, 1987; Easterbrook et al., 1991; Misakian & Bero, 1998; Rotton, Foos, Vanmeek, & Levitt, 1995; Scherer, Dickersin, & Langenberg, 1994; Shadish, Doherty, &…”
BackgroundFixed-and random-effects models in meta-analysis.
Hedges, L. V., & Vevea, J. L. (1998) · Psychological Methods · doi:10.1037/1082-989x.3.4.486 · Cited by 2,977
Self-citationA Nonparametric “Trim and Fill” Method of Accounting for Publication Bias in Meta-Analysis
Duval, S., & Tweedie, R. (2000) · Journal of the American Statistical Association · doi:10.1080/01621459.2000.10473905 · Cited by 2,955
Research works printed but not linked: 8
- Becker, 1986, Influence again., 178
- Becker, (in press). Failsafe N of file-drawer number., 111
- Chan, 1982, The epidemiology of unpublished randomized control trials., Clinical Research, 30, 234A
- 10.2307/1165375 · DOI
- Givens, 1997b, Statistical Science, 12, 247, 10.1214/ss/1030037958 · DOI
- 10.2307/1164832 · DOI · Paper page
- 10.2307/1164836 · DOI · Paper page
- 10.1001/jama.272.2.158 · DOI · Paper page
Cited by
Sorted by citations
The Handbook of Research Synthesis and Meta-Analysis
(2019) · doi:10.7758/9781610448864 · Cited by 4,553
Salk, R. H., Hyde, J. S., & Abramson, L. Y. (2017) · Psychological Bulletin · doi:10.1037/bul0000102 · Cited by 2,622
The effects of physical activity on sleep: A meta-analytic review
Kredlow, M. A., Capozzoli, M. C., Hearon, B. A., Calkins, A. W., & Otto, M. W. (2015) · Journal of Behavioral Medicine · doi:10.1007/s10865-015-9617-6 · Cited by 1,333
“In considering these analyses, it is important to note that funnel plot asymmetry could be due to factors other than publication bias and because of this the Trim and Fill method may lead to overcorrection (Vevea & Woods, 2005).”
MethodTwo Decades of Terror Management Theory: A Meta-Analysis of Mortality Salience Research
Burke, B. L., Martens, A., & Faucher, E. H. (2010) · Personality and Social Psychology Review · doi:10.1177/1088868309352321 · Cited by 1,188
“Yet another way to interpret the MS ES is to investigate the possibility of publication bias, the tendency for the availability of research to depend on the results (Vevea & Woods, 2005).”
Field, A. P., & Gillett, R. (2010) · British Journal of Mathematical and Statistical Psychology · doi:10.1348/000711010x502733 · Cited by 1,162
“Opening this file in a text editor reveals the code that controls the weight functions: # Enter fixed weight function w1 ,- matrix(c(1.0,.99,.95,.90,.80,.75,.65,.60,.55,.50,.50,.50,.50,.50), ncol ¼ 1) w2 ,- matrix(c(1.0,.99,.90,.75,.60,.50,.40,.35,.30,.25,.10,.10,.10,.10), ncol ¼ 1) w3 ,- matrix(c(1.0,.99,.95,.90,.80,…”
BackgroundMethodInfluential
Metadata
- Authors
- Vevea, J. L., & Woods, C. M.
- Affiliations
- University of California, Santa Cruz (US)
- Washington University in St. Louis (US)
- ORCID
- Woods, Carol M.
- Venue
- Psychological Methods
- Published
- 2005-12
- Volume
- 10
- Issue
- 4
- Pages
- 428-443
- ISSN
- 1082-989X (ISSN-L)
- 1939-1463
- OpenAlex
- W1974479479
- Cited by
- 461
- References
- 94
- Citations by year
- 2026: 24 · 2025: 33 · 2024: 35 · 2023: 35 · 2022: 36 · 2021: 41 · 2020: 35 · 2019: 49 · 2018: 33 · 2017: 23 · 2016: 12 · 2015: 22 · 2014: 13 · 2013: 10 · 2012: 11
- FWCI
- 9.19
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychological Methods (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Institutes of Health (4r44ag024771-02)
- National Institute on Aging (AG024771)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Keywords
- publication bias, file drawer problem, funnel plot asymmetry, meta-analysis, sensitivity analysis, bias detection, fixed effects model, random effects models, mixed models
- MeSH terms
- Data Interpretation, Statistical, Humans, Models, Psychological, Publishing, Research, Research Design, Sensitivity and Specificity, Meta-Analysis as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
