The DOI matches its Crossref record; 96 of the 96 references it cites were found in a registry or an index.
Publication Bias in Research Synthesis: Sensitivity Analysis Using A Priori Weight Functions.
Abstract
Publication bias, sometimes known as the "file-drawer problem" or "funnel-plot asymmetry," is common in empirical research. The authors review the implications of publication bias for quantitative research synthesis (meta-analysis) and describe existing techniques for detecting and correcting it. A new approach is proposed that is suitable for application to meta-analytic data sets that are too small for the application of existing methods. The model estimates parameters relevant to fixed-effects, mixed-effects or random-effects meta-analysis contingent on a hypothetical pattern of bias that is fixed independently of the data. The authors illustrate this approach for sensitivity analysis using 3 data sets adapted from a commonly cited reference work on research synthesis (H. M. Cooper & L. V. Hedges, 1994).
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“To evaluate publication bias, selection modeling was used. This approach used a weight function model created by Vevea and Woods [44], implemented through the weightr package.”
From Generative AI Mental Health Chatbots as Therapeutic Tools: Systematic Review and Meta-Analysis of Their Role in Reducing Mental Health Issues (2025) · cites this paper as [44]
In the citing paper
“confirming evidence of selection bias.”
From Generative AI Mental Health Chatbots as Therapeutic Tools: Systematic Review and Meta-Analysis of Their Role in Reducing Mental Health Issues (2025) · cites this paper as [44]
In the citing paper
“After adjustment for potential selection bias, the two-step model (P value cutpoints=.025, .50, 1) yielded a smaller and nonsignificant change in the pooled effect (g=0.54, SE=0.26, z=2.10; P=.04, 95% CI 0.04, 1.04).”
From Generative AI Mental Health Chatbots as Therapeutic Tools: Systematic Review and Meta-Analysis of Their Role in Reducing Mental Health Issues (2025) · cites this paper as [44]
In the citing paper
“The unadjusted model (k=44) estimated a pooled effect size of g=0.41 (SE=0.10, z=4.24; P<.001, 95% CI 0.22, 0.60).”
From Generative AI Mental Health Chatbots as Therapeutic Tools: Systematic Review and Meta-Analysis of Their Role in Reducing Mental Health Issues (2025) · cites this paper as [44]
In the citing paper
“studies with P<.025* were about 3.54 times more likely to be included than those with larger P values”
From Generative AI Mental Health Chatbots as Therapeutic Tools: Systematic Review and Meta-Analysis of Their Role in Reducing Mental Health Issues (2025) · cites this paper as [44]
In the citing paper
“suggesting moderate publication bias favoring statistically significant findings”
From Generative AI Mental Health Chatbots as Therapeutic Tools: Systematic Review and Meta-Analysis of Their Role in Reducing Mental Health Issues (2025) · cites this paper as [44]
In the citing paper
“The likelihood ratio test comparing adjusted and unadjusted models was significant (χ²2=11.25, P=.004)”
From Generative AI Mental Health Chatbots as Therapeutic Tools: Systematic Review and Meta-Analysis of Their Role in Reducing Mental Health Issues (2025) · cites this paper as [44]
In the citing paper
“Vevea and Woods’ (2005) selection model calculated a similar estimate (g¯ = 0.119).”
From Cognitive and academic benefits of music training with children: A multilevel meta-analysis (2020) · cites this paper as Vevea and Woods’ (2005)
In the citing paper
“The selection model estimate was g¯ = 0.039.”
From Cognitive and academic benefits of music training with children: A multilevel meta-analysis (2020) · cites this paper as Vevea and Woods’ (2005)
In the citing paper
“Vevea and Woods’ (2005) selection model calculated a more conservative estimate (g¯ = 0.126).”
From Cognitive and academic benefits of music training with children: A multilevel meta-analysis (2020) · cites this paper as Vevea and Woods’ (2005)
In the citing paper
“Vevea and Woods’ (2005) selection model yielded a similar estimate (g¯ = -0.002).”
From Cognitive and academic benefits of music training with children: A multilevel meta-analysis (2020) · cites this paper as Vevea and Woods’ (2005)
In the citing paper
“Sensitivity analysis (Vevea and Woods 2005) was also conducted to evaluate and account for the level of publication bias.”
From A meta-analytic review of the efficacy of physical exercise interventions on cognition in individuals with autism spectrum disorder and ADHD (2016) · cites this paper as Vevea and Woods 2005
In the citing paper
“Although the result of Kendall's tau is non-significant, in view of the small number of studies included in the metaanalysis, publication bias cannot be rejected and therefore a sensitivity analysis (Vevea and Woods, 2005) was conducted to estimate and account for the likelihood of publication bias.”
From A meta-analytic review of the efficacy of physical exercise interventions on cognition in individuals with autism spectrum disorder and ADHD (2016) · cites this paper as Vevea and Woods, 2005
Citations
Cites
Bias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722
Operating characteristics of a rank correlation test for publication bias
Begg, C. B., & Mazumdar, M. (1994) · Biometrics · doi:10.2307/2533446 · Cited by 16,955
Duval, S., & Tweedie, R. (2000) · Biometrics · doi:10.1111/j.0006-341x.2000.00455.x · Cited by 14,099
Wasserman, S. S., Hedges, L. V., & Olkin, I. (1988) · Journal of Educational Statistics · doi:10.2307/1164953 · Cited by 10,377
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068
Statistical Methods for Meta-Analysis.
Freeman, P. R., Hedges, L. V., & Olkin, I. (1986) · Biometrics · doi:10.2307/2531069 · Cited by 4,840
The Handbook of Research Synthesis.
Howel, D., Cooper, H., & Hedges, L. V. (1995) · Biometrics · doi:10.2307/2533295 · Cited by 3,657
Publication bias in clinical research
Easterbrook, P. J., Gopalan, R., Berlin, J. A., & Matthews, D. R. (1991) · The Lancet · doi:10.1016/0140-6736(91)90201-y · Cited by 3,109 · cited 2×
“…are surveyed about unpublished work (e.g., Chan, Sacks, & Chalmers, 1982; Coursol & Wagner, 1986; Dickersin, Chan, Chalmers, Sacks, & Smith, 1987; Easterbrook et al., 1991; Misakian & Bero, 1998; Rotton, Foos, Vanmeek, & Levitt, 1995; Scherer, Dickersin, & Langenberg, 1994; Shadish, Doherty, &…”
BackgroundFixed-and random-effects models in meta-analysis.
Hedges, L. V., & Vevea, J. L. (1998) · Psychological Methods · doi:10.1037/1082-989x.3.4.486 · Cited by 2,977
Self-citationA Nonparametric “Trim and Fill” Method of Accounting for Publication Bias in Meta-Analysis
Duval, S., & Tweedie, R. (2000) · Journal of the American Statistical Association · doi:10.1080/01621459.2000.10473905 · Cited by 2,955
Research works printed but not linked: 8
- Becker, 1986, Influence again., 178
- Becker, (in press). Failsafe N of file-drawer number., 111
- Chan, 1982, The epidemiology of unpublished randomized control trials., Clinical Research, 30, 234A
- 10.2307/1165375 · DOI
- Givens, 1997b, Statistical Science, 12, 247, 10.1214/ss/1030037958 · DOI
- 10.2307/1164832 · DOI · Paper page
- 10.2307/1164836 · DOI · Paper page
- 10.1001/jama.272.2.158 · DOI · Paper page
Cited by
Sorted by citations
Items 401–460 of 460
Tan, J. J. X. (2016) · Illinois Digital Environment for Access to Learning and Scholarship (University of Illinois at Urbana-Champaign) · Cited by 1
Infinite diameter confidence sets in Hedges’ publication bias model
Moss, J. (2022) · Journal of the Korean Statistical Society · doi:10.1007/s42952-022-00169-1 · Cited by 1
“Variants of this model, with and without covariates, has been studied by e.g. Citkowicz and Vevea (2017); Dear and Begg (1992); Vevea and Hedges (1995); Vevea and Woods (2005).”
MethodDöring, N. M. (2023) · Springer eBooks · doi:10.1007/978-3-662-64762-2_16 · Cited by 1
Bruns, A. K., Henriksson, I., Nordström, T., & Carlsson, R. (2024) · Campbell Systematic Reviews · doi:10.1002/cl2.1389
Song, R., & Sun, N. (2025) · Journal of Applied Developmental Psychology · doi:10.1016/j.appdev.2025.101798 · Cited by 1
PERSONALITY PROCESSES AND INDIVIDUAL DIFFERENCES
Dittmar, H., & Hurst, M. (2014) · Cited by 1
Chaka, C. (2021) · Per Linguam · doi:10.5785/37-2-905 · Cited by 2
Undertake a Critical Review of the Evidence: Evaluate Its Rigor, Impact, and Applicability
Macgowan, M. J. (2008) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780195183450.003.0004 · Cited by 1
Research on Research: A Meta-Scientific Study of Problems and Solutions in Psychological SciencePreprint
Nuijten, M. B. (2018) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/qtk7e · Cited by 1
Stress Regulation via Being in Nature and Social Support in Adults, a Meta-analysisPreprint
Sparacio, A., Ropovik, I., Jiga‐Boy, G. M., Cem, A., & IJzerman, H. (2021) · doi:10.31234/osf.io/a4zmj · Cited by 1
PEER-ADMINISTERED INTERVENTIONS FOR DEPRESSION: A META-ANALYTIC REVIEW
Bryan, A. E. B. (2013) · UA Campus Repository (The University of Arizona)
Formulate an Answerable Practice Question
Macgowan, M. J. (2008) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780195183450.003.0002
Advancing Evidence-Based Group Work in Research, Clinical Practice, and Education
Macgowan, M. J. (2008) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780195183450.003.0006
Apply the Evidence and Evaluate the Achievement of Desired Outcomes
Macgowan, M. J. (2008) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780195183450.003.0005
Petkova, B. N., & van Wezel, W. (2008)
김성훈 (2012) · Korean Journal of Military Art and Science · doi:10.31066/kjmas.2012.68.3.006
Pan and Rickard (2018) Transfer of test-enhanced learning meta-analysis (preprint)Preprint
Pan, S. C. (2018) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/4qpyn · Cited by 1
Döring, N. M., & Bortz, J. (2015) · Springer-Lehrbuch · doi:10.1007/978-3-642-41089-5_16 · Cited by 3
Tekrarlama Krizi ve Psikoterapi Etkililik Çalışmaları
Türkarslan, K. K. (2022) · Psikoloji Çalışmaları / Studies in Psychology · doi:10.26650/sp2021-959170
vanAert_Niemeyer21Preprint
van Aert, R. C. M., & Niemeyer, H. (2020) · MetArXiv (OSF Preprints) · doi:10.31222/osf.io/3rdys
Duggan, M., & Tressoldi, P. E. (2018) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/5u48e
Mellenbergh, G. J. (2019) · Springer eBooks · doi:10.1007/978-3-030-12272-0_19
(2008) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780195183450.002.0005
(2008) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780195183450.002.0003
(2008) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780195183450.002.0004
(2008) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780195183450.005.0001
Contextualisation of Relational AI Governance in Existing Research
Wiesmüller, S. (2023) · Relational economics and organization governance · doi:10.1007/978-3-031-25023-1_4
Asefa, Y. A., Persson, L. Å., Seale, A. C., & Assefa, N. (2024) · Gates Open Research · doi:10.12688/gatesopenres.13915.2
“Sensitivity analysis will also be done using a risk of bias assessment result to ensure the robustness of the conclusion 39,40 .”
Relationship between parental involvement and school readiness: a meta-analysisPreprint
Cui, S., Ropovik, I., & Greger, D. (2024) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/t4we2
A systematic review and meta-analysis of the relationship between genes and reflexive attention
Myres, S., Christensen, K. E., & Lundwall, R. A. (2025) · Frontiers in Neuroscience · doi:10.3389/fnins.2025.1449354
Zhang, Q., Wang, X., Tian, X., & Gehlbach, H. (2025) · EdArXiv (OSF Preprints) · doi:10.35542/osf.io/nfwm4_v1 · Cited by 1
Hedges, L. V. (2025) · Elsevier eBooks · doi:10.1016/b978-0-443-26629-4.00033-2
Espinoza, J. A., Allen, N. J., Su, S., & Thompson, B. (2025) · Team Performance Management · doi:10.1108/tpm-02-2024-0014
Pękala, K. A., Seweryn, M. T., & Żak, A. M. (2025) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2025.1602060
“Publication bias will also be assessed using funnel plot test (27) or other suitable method.”
VanderWeele, T. J. (2025) · American Journal of Epidemiology · doi:10.1093/aje/kwaf276
Sha, S., Loveys, K., Qualter, P., Shi, H., Krpan, D., & Galizzi, M. M. (2024) · Figshare · doi:10.6084/m9.figshare.26744636
Liang, W., Xiang, W., Cheng, S., Jiao, J., Zhu, X., & Duan, Y. (2024) · Figshare · doi:10.6084/m9.figshare.27012315
Liang, W., Xiang, W., Cheng, S., Jiao, J., Zhu, X., & Duan, Y. (2024) · Figshare · doi:10.6084/m9.figshare.27012312
Liang, W., Xiang, W., Cheng, S., Jiao, J., Zhu, X., & Duan, Y. (2024) · Figshare · doi:10.6084/m9.figshare.27012309
Sha, S., Loveys, K., Qualter, P., Shi, H., Krpan, D., & Galizzi, M. M. (2024) · Figshare · doi:10.6084/m9.figshare.26744630
Liang, W., Xiang, W., Cheng, S., Jiao, J., Zhu, X., & Duan, Y. (2024) · Figshare · doi:10.6084/m9.figshare.27012306
Liang, W., Xiang, W., Cheng, S., Jiao, J., Zhu, X., & Duan, Y. (2024) · Figshare · doi:10.6084/m9.figshare.27012300
Effect Size Thresholds in Early Literacy: Defining Benchmarks for Phonemic Awareness Research
Erbeli, F., Rice, M., Xu, Y., & Savage, R. S. (2025) · Early Education and Development · doi:10.1080/10409289.2025.2581998
“Specifically, the weights were estimated as part of the selection model implemented using the weightr R package (Coburn & Vevea, 2019), which follows the approach outlined by Vevea and Hedges (1995) and Vevea and Woods (2005).”
Type I, III, and IV Robust Design Through Unsupervised Learning
Guo, L., Allen, J. K., & Mistree, F. (2026) · Springer eBooks · doi:10.1007/978-3-032-04577-5_6
Wang, X., Geng, Z., & Pan, Z. (2026) · Science & Sports · doi:10.1016/j.scispo.2025.12.002
Moving in the presence of others – a systematic review and meta-analysis on social facilitationPreprint
Strauß, B., Greve, J., & van Meurs, E. (2021) · OSF Preprints (OSF Preprints)
Zhou, L., Wang, J., Lou, K., Zheng, X., Li, X., & Zeng, X. (2026) · Cyberpsychology Behavior and Social Networking · doi:10.1177/21522715261449054
“To further verify this, we applied both trim-and-fi ll procedures and step-function selection models 105,106 to correct for potential bias.”
Llorens, L. S., Peñalver, J., Fernández-Castilla, B., & Salanova, M. (2026) · Journal of Work and Organizational Psychology · doi:10.5093/jwop2026a4
The Focal v Control Test: Dealing With P‐hacking Concerns in Meta‐analytical Research
Kunst, V. E., & Becker, P. B. (2026) · British Journal of Management · doi:10.1111/1467-8551.70092
Metaanálisis de generalización de la fiabilidad de la Bergen Shopping Addiction Scale (BSAS)
Carrasco-Reig, M., Vilar, M. M., González‐Sala, F., & Esparza-Reig, J. (2026) · Quaderns de Psicologia · doi:10.5565/rev/qpsicologia.2317
Azooz, R. E. (2026) · Journal of Energy Storage · doi:10.1016/j.est.2026.124214
Vera, A. B. (2026) · Universitas Psychologica · doi:10.11144/javeriana.upsy25.resf
“Publication bias was assessed using funnel plot inspection, Egger’s regression test, trim-and-fill analysis, p -curve analysis (Simonsohn et al., 2014), and three-parameter selection models (Vevea & Woods, 2005).”
Zhang, Z., Chen, Q., & Chan, K. L. (2026) · Trauma Violence & Abuse · doi:10.1177/15248380261480155
Estimation and inference for step-function selection models in meta-analyses with dependent effects
Pustejovsky, J. E., Citkowicz, M., & Joshi, M. (2026) · Research Synthesis Methods · doi:10.1017/rsm.2026.10117
Sex Differences in Piaget’s Water Level Tasks: A Systematic Review and Multiverse Meta-Analysis
Dejardin, F., Oberleiter, S., Voracek, M., & Pietschnig, J. (2026) · Journal of Intelligence · doi:10.3390/jintelligence14100243
“…Sterne and Egger’s regression model (Sterne & Egger, 2005), the rank-correlation test (Begg & Mazumdar, 1994), PET-PEESE (Stanley & Doucouliagos, 2014), trim-and-fill (Duval & Tweedie, 2000), the test of excess significance (Ioannidis & Trikalinos, 2007), and selection models (Vevea & Woods, 2005).”
Spender, K., Parsons, L., Wilkes‐Gillan, S., Chen, Y.-W. R., So, J., & Cordier, R. (2026) · Review of Education · doi:10.1002/rev3.70235
Chen, M. (2026) · Behavior Research Methods · doi:10.3758/s13428-026-03206-7
“…et al., 1997), non-parametric detection and adjustment methods such as trim-and-fill (Duval & Tweedie, 2000b,a), regression-based methods such as PET/PEESE (Stanley & Doucouliagos, 2014), and selection modeling approach based on statistical significance (Vevea & Hedges, 1995; Vevea & Woods, 2005).”
Sala, G., & Gobet, F. (2019) · Collabra: Psychology · doi:10.2478/collabra.2019.5.1.18
Metadata
- Authors
- Vevea, J. L., & Woods, C. M.
- Affiliations
- University of California, Santa Cruz (US)
- Washington University in St. Louis (US)
- ORCID
- Woods, Carol M.
- Venue
- Psychological Methods
- Published
- 2005-12
- Volume
- 10
- Issue
- 4
- Pages
- 428-443
- ISSN
- 1082-989X (ISSN-L)
- 1939-1463
- OpenAlex
- W1974479479
- Cited by
- 461
- References
- 94
- Citations by year
- 2026: 24 · 2025: 33 · 2024: 35 · 2023: 35 · 2022: 36 · 2021: 41 · 2020: 35 · 2019: 49 · 2018: 33 · 2017: 23 · 2016: 12 · 2015: 22 · 2014: 13 · 2013: 10 · 2012: 11
- FWCI
- 9.19
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychological Methods (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Institutes of Health (4r44ag024771-02)
- National Institute on Aging (AG024771)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Keywords
- publication bias, file drawer problem, funnel plot asymmetry, meta-analysis, sensitivity analysis, bias detection, fixed effects model, random effects models, mixed models
- MeSH terms
- Data Interpretation, Statistical, Humans, Models, Psychological, Publishing, Research, Research Design, Sensitivity and Specificity, Meta-Analysis as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
