Publication Bias in Research Synthesis: Sensitivity Analysis Using A Priori Weight Functions.
Abstract
Publication bias, sometimes known as the "file-drawer problem" or "funnel-plot asymmetry," is common in empirical research. The authors review the implications of publication bias for quantitative research synthesis (meta-analysis) and describe existing techniques for detecting and correcting it. A new approach is proposed that is suitable for application to meta-analytic data sets that are too small for the application of existing methods. The model estimates parameters relevant to fixed-effects, mixed-effects or random-effects meta-analysis contingent on a hypothetical pattern of bias that is fixed independently of the data. The authors illustrate this approach for sensitivity analysis using 3 data sets adapted from a commonly cited reference work on research synthesis (H. M. Cooper & L. V. Hedges, 1994).
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
To evaluate publication bias, selection modeling was used.
“To evaluate publication bias, selection modeling was used. This approach used a weight function model created by Vevea and Woods [44], implemented through the weightr package.”
From Generative AI Mental Health Chatbots as Therapeutic Tools: Systematic Review and Meta-Analysis of Their Role in Reducing Mental Health Issues (2025) · cites this paper as [44]
There is evidence of selection bias.
“confirming evidence of selection bias.”
From Generative AI Mental Health Chatbots as Therapeutic Tools: Systematic Review and Meta-Analysis of Their Role in Reducing Mental Health Issues (2025) · cites this paper as [44]
After adjustment for potential selection bias, the two-step model yielded a pooled effect of g=0.54 (SE=0.26, z=2.10; P=.04, 95% CI 0.04, 1.04).
“After adjustment for potential selection bias, the two-step model (P value cutpoints=.025, .50, 1) yielded a smaller and nonsignificant change in the pooled effect (g=0.54, SE=0.26, z=2.10; P=.04, 95% CI 0.04, 1.04).”
From Generative AI Mental Health Chatbots as Therapeutic Tools: Systematic Review and Meta-Analysis of Their Role in Reducing Mental Health Issues (2025) · cites this paper as [44]
The unadjusted model estimated a pooled effect size of g=0.41 (SE=0.10, z=4.24; P<.001, 95% CI 0.22, 0.60).
“The unadjusted model (k=44) estimated a pooled effect size of g=0.41 (SE=0.10, z=4.24; P<.001, 95% CI 0.22, 0.60).”
From Generative AI Mental Health Chatbots as Therapeutic Tools: Systematic Review and Meta-Analysis of Their Role in Reducing Mental Health Issues (2025) · cites this paper as [44]
Studies with P<.025 were about 3.54 times more likely to be included than those with larger P values.
“studies with P<.025* were about 3.54 times more likely to be included than those with larger P values”
From Generative AI Mental Health Chatbots as Therapeutic Tools: Systematic Review and Meta-Analysis of Their Role in Reducing Mental Health Issues (2025) · cites this paper as [44]
The estimated weights suggest moderate publication bias favoring statistically significant findings.
“suggesting moderate publication bias favoring statistically significant findings”
From Generative AI Mental Health Chatbots as Therapeutic Tools: Systematic Review and Meta-Analysis of Their Role in Reducing Mental Health Issues (2025) · cites this paper as [44]
The likelihood ratio test comparing adjusted and unadjusted models was significant (χ22=11.25, P=.004).
“The likelihood ratio test comparing adjusted and unadjusted models was significant (χ²2=11.25, P=.004)”
From Generative AI Mental Health Chatbots as Therapeutic Tools: Systematic Review and Meta-Analysis of Their Role in Reducing Mental Health Issues (2025) · cites this paper as [44]
For non-active controls, Vevea and Woods' selection model calculated a similar estimate (g = 0.119).
“Vevea and Woods’ (2005) selection model calculated a similar estimate (g¯ = 0.119).”
From Cognitive and academic benefits of music training with children: A multilevel meta-analysis (2020) · cites this paper as Vevea and Woods’ (2005)
For active controls, the selection model estimate was g = 0.039.
“The selection model estimate was g¯ = 0.039.”
From Cognitive and academic benefits of music training with children: A multilevel meta-analysis (2020) · cites this paper as Vevea and Woods’ (2005)
For non-randomized studies, Vevea and Woods' selection model calculated a more conservative estimate (g = 0.126).
“Vevea and Woods’ (2005) selection model calculated a more conservative estimate (g¯ = 0.126).”
From Cognitive and academic benefits of music training with children: A multilevel meta-analysis (2020) · cites this paper as Vevea and Woods’ (2005)
For randomized samples, Vevea and Woods' selection model yielded an estimate of g = -0.002.
“Vevea and Woods’ (2005) selection model yielded a similar estimate (g¯ = -0.002).”
From Cognitive and academic benefits of music training with children: A multilevel meta-analysis (2020) · cites this paper as Vevea and Woods’ (2005)
Sensitivity analysis was conducted to evaluate and account for the level of publication bias.
“Sensitivity analysis (Vevea and Woods 2005) was also conducted to evaluate and account for the level of publication bias.”
From A meta-analytic review of the efficacy of physical exercise interventions on cognition in individuals with autism spectrum disorder and ADHD (2016) · cites this paper as Vevea and Woods 2005
Publication bias cannot be rejected due to the small number of studies included in the meta-analysis.
“Although the result of Kendall's tau is non-significant, in view of the small number of studies included in the metaanalysis, publication bias cannot be rejected and therefore a sensitivity analysis (Vevea and Woods, 2005) was conducted to estimate and account for the likelihood of publication bias.”
From A meta-analytic review of the efficacy of physical exercise interventions on cognition in individuals with autism spectrum disorder and ADHD (2016) · cites this paper as Vevea and Woods, 2005
Cites
Bias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722 · see citing works
Operating characteristics of a rank correlation test for publication bias
Begg, C. B., & Mazumdar, M. (1994) · Biometrics · doi:10.2307/2533446 · Cited by 16,955 · see citing works
Duval, S., & Tweedie, R. (2000) · Biometrics · doi:10.1111/j.0006-341x.2000.00455.x · Cited by 14,099 · see citing works
Wasserman, S. S., Hedges, L. V., & Olkin, I. (1988) · Journal of Educational Statistics · doi:10.2307/1164953 · Cited by 10,347 · see citing works
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068 · see citing works
Statistical Methods for Meta-Analysis.
Freeman, P. R., Hedges, L. V., & Olkin, I. (1986) · Biometrics · doi:10.2307/2531069 · Cited by 4,832 · see citing works
The Handbook of Research Synthesis.
Howel, D., Cooper, H., & Hedges, L. V. (1995) · Biometrics · doi:10.2307/2533295 · Cited by 3,657 · see citing works
Publication bias in clinical research
Easterbrook, P. J., Gopalan, R., Berlin, J. A., & Matthews, D. R. (1991) · The Lancet · doi:10.1016/0140-6736(91)90201-y · Cited by 3,109 · cited 2× · see citing works
“…are surveyed about unpublished work (e.g., Chan, Sacks, & Chalmers, 1982; Coursol & Wagner, 1986; Dickersin, Chan, Chalmers, Sacks, & Smith, 1987; Easterbrook et al., 1991; Misakian & Bero, 1998; Rotton, Foos, Vanmeek, & Levitt, 1995; Scherer, Dickersin, & Langenberg, 1994; Shadish, Doherty, &…”
BackgroundFixed-and random-effects models in meta-analysis.
Hedges, L. V., & Vevea, J. L. (1998) · Psychological Methods · doi:10.1037/1082-989x.3.4.486 · Cited by 2,977 · see citing works
Self-citationA Nonparametric “Trim and Fill” Method of Accounting for Publication Bias in Meta-Analysis
Duval, S., & Tweedie, R. (2000) · Journal of the American Statistical Association · doi:10.1080/01621459.2000.10473905 · Cited by 2,955 · see citing works
Publication Bias in Meta‐Analysis: Prevention, Assessment and Adjustments
(2005) · doi:10.1002/0470870168 · Cited by 2,368 · see citing works
A Fail-Safe N for Effect Size in Meta-Analysis
Orwin, R. G. (1983) · Journal of Educational Statistics · doi:10.2307/1164923 · Cited by 2,064 · cited 1× · see citing works
“Accordingly, the fail-safe n has been modified for use with effect sizes, which can be nonzero (Ashworth, Osburn, Callender, & Boyle, 1992; Orwin, 1983).”
MethodLipsey, M. W., & Wilson, D. B. (1993) · American Psychologist · doi:10.1037/0003-066x.48.12.1181 · Cited by 2,033 · see citing works
Publication bias in meta-analysis: Prevention, assessment and adjustments
Viechtbauer, W. (2007) · Psychometrika · doi:10.1007/s11336-006-1450-y · Cited by 1,507 · see citing works
Summing up: The Science of Reviewing Research
Wilkinson, I., Light, R. J., & Pillemer, D. B. (1987) · Journal of Educational Statistics · doi:10.2307/1164691 · Cited by 1,486 · see citing works
Light, R. J., & Pillemer, D. B. (1984) · Harvard University Press eBooks · doi:10.4159/9780674040243 · Cited by 1,422 · see citing works
Empirical assessment of effect of publication bias on meta-analyses
Sutton, A. J. (2000) · BMJ · doi:10.1136/bmj.320.7249.1574 · Cited by 1,398 · see citing works
Publication bias in meta-analysis its causes and consequences
Thornton, A. (2000) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(99)00161-4 · Cited by 1,291 · see citing works
Language bias in randomised controlled trials published in English and German
Egger, M., Zellweger-Zähner, T., Schneider, M., Junker, C., Lengeler, C., & Antes, G. (1997) · The Lancet · doi:10.1016/s0140-6736(97)02419-7 · Cited by 1,028 · see citing works
Consequences of prejudice against the null hypothesis.
Greenwald, A. G. (1975) · Psychological Bulletin · doi:10.1037/h0076157 · Cited by 1,022 · cited 5× · see citing works
“…and failure to publish studies with inadequate sample sizes (Newcombe, 1987), development of a more positive attitude toward nonsignificant results (Greenwald, 1975; Rennie & Flanagin, 1992), and improvement of the peer-review and publication processes (Higginson, 1987; Iyengar & Greenhouse, 1988;…”
BackgroundInfluentialPublication prejudices: An experimental study of confirmatory bias in the peer review system
Mahoney, M. J. (1977) · Cognitive Therapy and Research · doi:10.1007/bf01173636 · Cited by 1,013 · see citing works
Sterling, T. D. (1959) · Journal of the American Statistical Association · doi:10.1080/01621459.1959.10501497 · Cited by 964 · cited 2× · see citing works
“Empirical research using various methodologies, and conducted in diverse substantive areas, suggests that statistically significant results are more likely than nonsignificant results to be published and presented (Bozarth & Roberts, 1972; Greenwald, 1975; Smart, 1964; Sterling, 1959; Sterling, Rosenbaum, & Weinkam, 1…”
Combining results of independent studies.
Rosenthal, R. (1978) · Psychological Bulletin · doi:10.1037/0033-2909.85.1.185 · Cited by 910 · see citing works
Dickersin, K., Yuan, M.-Q., & Meinert, C. (1992) · Journal of the American Medical Association (JAMA) · doi:10.1001/jama.1992.03480030052036 · Cited by 900 · see citing works
Hunter, J. E., & Schmidt, F. L. (2000) · International Journal of Selection and Assessment · doi:10.1111/1468-2389.00156 · Cited by 833 · see citing works
Cohen, P. A. (1981) · Review of Educational Research · doi:10.3102/00346543051003281 · Cited by 819 · see citing works
Publication bias: evidence of delayed publication in a cohort study of clinical research projects
Stern, J. M., & Simes, R. J. (1997) · BMJ · doi:10.1136/bmj.315.7109.640 · Cited by 818 · see citing works
Publication bias—a problem in interpreting medical data
Begg, C. B., & Berlin, J. A. (1988) · Journal of the Royal Statistical Society. Series A (Statistics in Society) · doi:10.2307/2982993 · Cited by 814 · see citing works
Adjusting for publication bias in the presence of heterogeneity
Terrin, N., Schmid, C. H., Lau, J., & Olkin, I. (2003) · Statistics in Medicine · doi:10.1002/sim.1461 · Cited by 809 · see citing works
Publication bias: the case for an international registry of clinical trials.
Simes, R. J. (1986) · Journal of Clinical Oncology · doi:10.1200/jco.1986.4.10.1529 · Cited by 747 · see citing works
Publication bias and clinical trials
Dickersin, K., Chan, S., Chalmersx, T. C., Sacks, H. S., & Smith, H. (1987) · Controlled Clinical Trials · doi:10.1016/0197-2456(87)90155-3 · Cited by 715 · see citing works
A Nonparametric "Trim and Fill" Method of Accounting for Publication Bias in Meta-Analysis
Duval, S., & Tweedie, R. L. (2000) · Journal of the American Statistical Association · doi:10.2307/2669529 · Cited by 694 · see citing works
Underreporting Research Is Scientific Misconduct
Chalmers, I. (1990) · JAMA · doi:10.1001/jama.263.10.1405 · Cited by 634 · see citing works
A Fail-Safe N for Effect Size in Meta-Analysis
Orwin, R. G. (1983) · Journal of Educational Statistics · doi:10.3102/10769986008002157 · Cited by 562 · see citing works
Grégoire, G., Derderian, F., & LeLorier, J. (1995) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(94)00098-b · Cited by 527 · see citing works
Eagly, A. H., & Carli, L. L. (1981) · Psychological Bulletin · doi:10.1037/0033-2909.90.1.1 · Cited by 525 · see citing works
Sterling, T. D. (1959) · Journal of the American Statistical Association · doi:10.2307/2282137 · Cited by 524 · see citing works
Meta-analysis, funnel plots and sensitivity analysis
Copas, J. (2000) · Biostatistics · doi:10.1093/biostatistics/1.3.247 · Cited by 503 · see citing works
(1994) · Cited by 463 · see citing works
Sterling, T. D., Rosenbaum, W. L., & Weinkam, J. J. (1995) · The American Statistician · doi:10.1080/00031305.1995.10476125 · Cited by 453 · see citing works
Publication Bias and Dissemination of Clinical Research
Begg, C. B., & Berlin, J. A. (1989) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/81.2.107 · Cited by 442 · see citing works
Full publication of results initially presented in abstracts. A meta-analysis
Scherer, R. W. (1994) · JAMA · doi:10.1001/jama.272.2.158 · Cited by 403 · see citing works
Inference for Non-random Samples
Copas, J. B., & Li, H. G. (1997) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/1467-9868.00055 · Cited by 398 · see citing works
Selection Models and the File Drawer Problem
Iyengar, S., & Greenhouse, J. B. (1988) · Statistical Science · doi:10.1214/ss/1177013012 · Cited by 388 · see citing works
A general linear model for estimating effect size in the presence of publication bias
Vevea, J. L., & Hedges, L. (1995) · Psychometrika · doi:10.1007/bf02294384 · Cited by 367 · see citing works
Self-citationModeling Publication Selection Effects in Meta-Analysis
Hedges, L. V. (1992) · Statistical Science · doi:10.1214/ss/1177011364 · Cited by 359 · see citing works
Empirical bayes meta-analysis.
Raudenbush, S. W., & Bryk, A. S. (1985) · Journal of Educational Statistics · doi:10.3102/10769986010002075 · Cited by 299 · see citing works
Unpublished Research From a Medical Specialty Meeting
Weber, E. J., Callaham, M. L., Wears, R. L., Barton, C., & Young, G. P. (1998) · JAMA · doi:10.1001/jama.280.3.257 · Cited by 262 · see citing works
Hedges, L. V. (1984) · Journal of Educational Statistics · doi:10.3102/10769986009001061 · Cited by 241 · see citing works
Publication bias and meta-analysis
Smith, M. L. (1980) · Evaluation in Education · doi:10.1016/0191-765x(80)90004-x · Cited by 216 · see citing works
Using the normal quantile plot to explore meta-analytic data sets.
Wang, M. C., & Bushman, B. J. (1998) · Psychological Methods · doi:10.1037/1082-989x.3.1.46 · Cited by 216 · see citing works
Confronting publication bias: A cohort design for meta‐analysis
Simes, R. J. (1987) · Statistics in Medicine · doi:10.1002/sim.4780060104 · Cited by 216 · see citing works
An Approach for Assessing Publication Bias Prior to Performing a Meta-Analysis
Dear, K., & Begg, C. (1992) · Statistical Science · doi:10.1214/ss/1177011363 · Cited by 202 · see citing works
A cohort study of summary reports of controlled trials.
Chalmers, I., Adams, M., Dickersin, K., Hetherington, J., Tarnow-Mordi, W., Meinert, C., Tonascia, S., & Chalmers, T. (1990) · Journal of the American Medical Association (JAMA) · doi:10.1001/jama.1990.03440100117017 · Cited by 200 · see citing works
The Place of Statistics in Psychology
Nunnally, J. C. (1960) · Educational and Psychological Measurement · doi:10.1177/001316446002000401 · Cited by 195 · see citing works
Dickersin, K. (1992) · JAMA · doi:10.1001/jama.267.3.374 · Cited by 193 · see citing works
BIAS AGAINST THE NULL HYPOTHESIS: THE REPRODUCTIVE HAZARDS OF COCAINE
Koren, G., Shear, H. H., Graham, K., & Einarson, T. R. (1989) · The Lancet · doi:10.1016/s0140-6736(89)92044-8 · Cited by 173 · see citing works
Cooper, H. M., & Hedges, L. V. (1994) · Journal of International Crisis and Risk Communication Research · Cited by 169 · see citing works
What Works?: Selectivity Models and Meta-Analysis
Copas, J. B. (1999) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/1467-985x.00123 · Cited by 168 · see citing works
Sterling, T. D., Rosenbaum, W. L., & Jj, W. (1995) · The American Statistician · doi:10.2307/2684823 · Cited by 168 · see citing works
Effect of positive findings on submission and acceptance rates: A note on meta-analysis bias.
Coursol, A., & Wagner, E. E. (1986) · Professional Psychology Research and Practice · doi:10.1037/0735-7028.17.2.136 · Cited by 165 · see citing works
Signifying significant significance.
Bozarth, J. D., & Roberts, R. R. (1972) · American Psychologist · doi:10.1037/h0038034 · Cited by 164 · see citing works
The importance of negative results in psychological research.
Smart, R. (1964) · The Canadian Psychologist · doi:10.1037/h0083036 · Cited by 156 · see citing works
Bias Against Negative Studies in Newspaper Reports of Medical Research
Koren, G. (1991) · JAMA · doi:10.1001/jama.1991.03470130104037 · Cited by 144 · see citing works
Finding the missing science: The fate of studies submitted for review by a human subjects committee.
Cooper, H., DeNeve, K., & Charlton, K. (1997) · Psychological Methods · doi:10.1037/1082-989x.2.4.447 · Cited by 144 · see citing works
Kraemer, H. C., Gardner, C., Brooks, J. O., & Yesavage, J. A. (1998) · Psychological Methods · doi:10.1037/1082-989x.3.1.23 · Cited by 141 · see citing works
An Assessment of Publication Bias Using a Sample of Published Clinical Trials
Berlin, J. A., Begg, C. B., & Louis, T. A. (1989) · Journal of the American Statistical Association · doi:10.1080/01621459.1989.10478782 · Cited by 134 · see citing works
Publication problems in psychology and an explicit evaluation schema for manuscripts.
Wolff, W. M. (1973) · American Psychologist · doi:10.1037/h0034678 · Cited by 133 · see citing works
Givens, G. H., Smith, D. D., & Tweedie, R. L. (1997) · Statistical Science · doi:10.1214/ss/1030037958 · Cited by 127 · see citing works
Publication Bias and Research on Passive Smoking
Misakian, A. L., & Bero, L. A. (1998) · JAMA · doi:10.1001/jama.280.3.250 · Cited by 126 · see citing works
Models for estimating the number of unpublished studies.
Gleser, L., & Olkin, I. (1996) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19961215)15:23 · Cited by 116 · see citing works
Towards a reduction in publication bias.
Newcombe, R. G. (1987) · BMJ · doi:10.1136/bmj.295.6599.656 · Cited by 113 · see citing works
Cohen, P. A. (1981) · Review of Educational Research · doi:10.2307/1170209 · Cited by 107 · see citing works
A measure to aid in the interpretation of published clinical trials
Begg, C. B. (1985) · Statistics in Medicine · doi:10.1002/sim.4780040103 · Cited by 100 · see citing works
Statistical Data Analysis and Inference
Ziegel, E. R., & Dodge, Y. (1992) · Technometrics · doi:10.2307/1269272 · Cited by 71 · see citing works
Likelihood and Bayesian Inference from Selectively Reported Data
Dawid, A., & Dickey, J. (1977) · Journal of the American Statistical Association · doi:10.1080/01621459.1977.10479968 · Cited by 66 · see citing works
Publication Bias against Null Results
Hubbard, R., & Armstrong, J. S. (1997) · Psychological Reports · doi:10.2466/pr0.1997.80.1.337 · Cited by 59 · see citing works
Rennie, D. (1992) · JAMA · doi:10.1001/jama.1992.03480030089045 · Cited by 55 · see citing works
Shadish, W. R., Doherty, M., & Montgomery, L. M. (1989) · Clinical Psychology Review · doi:10.1016/0272-7358(89)90013-5 · Cited by 45 · see citing works
Publication practices and the file drawer problem: A survey of published authors.
Rotton, J., Foos, P. W., Meek, L. V., & Levitt, M. J. (1995) · Journal of social behavior and personality · Cited by 41 · see citing works
Bias against negative studies in newspaper reports of medical research
Koren, G. (1991) · JAMA · doi:10.1001/jama.266.13.1824 · Cited by 40 · see citing works
PROBING ENCOUNTERED DATA, META ANALYSIS AND WEIGHTED DISTRIBUTION METHODS
Patil, G. P., & Taillie, C. (1989) · Elsevier eBooks · doi:10.1016/b978-0-444-88029-1.50035-6 · Cited by 39 · see citing works
Modeling publication bias using weighted distributions in a Bayesian framework
Larose, D. T., & Dey, D. (1998) · Computational Statistics & Data Analysis · doi:10.1016/s0167-9473(97)00039-x · Cited by 30 · see citing works
An Application of Gibbs Sampling to Estimation in Meta-Analysis: Accounting for Publication Bias
Cleary, R. J., & Casella, G. (1997) · Journal of Educational and Behavioral Statistics · doi:10.3102/10769986022002141 · Cited by 29 · see citing works
An Assessment of Publication Bias Using a Sample of Published Clinical Trials
Berlin, J. A., Begg, C. B., & Louis, T. A. (1989) · Journal of the American Statistical Association · doi:10.2307/2289921 · Cited by 29 · see citing works
Publication bias and the evaluation of psychotherapy efficacy in reviews of the research literature
Sohn, D. A. (1996) · Clinical Psychology Review · doi:10.1016/0272-7358(96)00005-0 · Cited by 28 · see citing works
A cohort study of summary reports of controlled trials
Chalmers, I. (1990) · JAMA · doi:10.1001/jama.263.10.1401 · Cited by 27 · see citing works
The File Drawer Effect and Publication Rates in Menstrual Cycle Research
Sommer, B. A. (1987) · Psychology of Women Quarterly · doi:10.1111/j.1471-6402.1987.tb00786.x · Cited by 24 · see citing works
Assessing the effects of selection bias on validity data for the General Aptitude Test Battery.
Vevea, J. L., Clements, N. C., & Hedges, L. V. (1993) · Journal of Applied Psychology · doi:10.1037/0021-9010.78.6.981 · Cited by 23 · see citing works
Self-citationTHE EFFECTS OF UNREPRESENTED STUDIES ON THE ROBUSTNESS OF VALIDITY GENERALIZATION RESULTS
Ashworth, S. D., Osburn, H. G., Callender, J. C., & Boyle, K. A. (1992) · Personnel Psychology · doi:10.1111/j.1744-6570.1992.tb00853.x · Cited by 23 · see citing works
Publication bias. The triumph of hope over experience
Rennie, D. (1992) · JAMA · doi:10.1001/jama.267.3.411 · Cited by 22 · see citing works
Nonparametric classes of weight functions to model publication bias
Silliman, N. (1997) · Biometrika · doi:10.1093/biomet/84.4.909 · Cited by 20 · see citing works
Likelihood and Bayesian Inference from Selectively Reported Data
Dawid, A. P., & Dickey, J. (1977) · Journal of the American Statistical Association · doi:10.2307/2286472 · Cited by 19 · see citing works
Assessing rationales for inclusiveness in meta-analytic samples
Greenwald, S., & Russell, R. L. (1991) · Psychotherapy Research · doi:10.1080/10503309112331334021 · Cited by 13 · see citing works
Publication of “negative” epidemiological studies
Higginson, J. F. (1987) · Journal of Chronic Diseases · doi:10.1016/0021-9681(87)90170-6 · Cited by 9 · see citing works
An Application of Gibbs Sampling to Estimation in Meta-Analysis: Accounting for Publication Bias
Cleary, R. J., & Casella, G. (1997) · Journal of Educational and Behavioral Statistics · doi:10.2307/1165375 · Cited by 6 · see citing works
The Effects of Publication Selection on Test Probabilities and Estimator Distributions
Denton, F. (1990) · Risk Analysis · doi:10.1111/j.1539-6924.1990.tb01027.x · Cited by 4 · see citing works
The power function of a published hypothesis test
Denton, F. T. (1987) · Economics Letters · doi:10.1016/0165-1765(87)90043-7 · Cited by 2 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
The Handbook of Research Synthesis and Meta-Analysis
(2019) · doi:10.7758/9781610448864 · Cited by 4,553 · see citing works
Salk, R. H., Hyde, J. S., & Abramson, L. Y. (2017) · Psychological Bulletin · doi:10.1037/bul0000102 · Cited by 2,622 · see citing works
The effects of physical activity on sleep: A meta-analytic review
Kredlow, M. A., Capozzoli, M. C., Hearon, B. A., Calkins, A. W., & Otto, M. W. (2015) · Journal of Behavioral Medicine · doi:10.1007/s10865-015-9617-6 · Cited by 1,333 · see citing works
“In considering these analyses, it is important to note that funnel plot asymmetry could be due to factors other than publication bias and because of this the Trim and Fill method may lead to overcorrection (Vevea & Woods, 2005).”
MethodTwo Decades of Terror Management Theory: A Meta-Analysis of Mortality Salience Research
Burke, B. L., Martens, A., & Faucher, E. H. (2010) · Personality and Social Psychology Review · doi:10.1177/1088868309352321 · Cited by 1,188 · see citing works
“Yet another way to interpret the MS ES is to investigate the possibility of publication bias, the tendency for the availability of research to depend on the results (Vevea & Woods, 2005).”
Field, A. P., & Gillett, R. (2010) · British Journal of Mathematical and Statistical Psychology · doi:10.1348/000711010x502733 · Cited by 1,162 · see citing works
“Opening this file in a text editor reveals the code that controls the weight functions: # Enter fixed weight function w1 ,- matrix(c(1.0,.99,.95,.90,.80,.75,.65,.60,.55,.50,.50,.50,.50,.50), ncol ¼ 1) w2 ,- matrix(c(1.0,.99,.90,.75,.60,.50,.40,.35,.30,.25,.10,.10,.10,.10), ncol ¼ 1) w3 ,- matrix(c(1.0,.99,.95,.90,.80,…”
BackgroundMethodInfluentialPsychotherapy for depression in adults: A meta-analysis of comparative outcome studies.
Cuijpers, P., van Straten, A., Andersson, G., & van Oppen, P. (2008) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0013075 · Cited by 1,050 · see citing works
Chapter 10: Addressing reporting biases
Egger, M., & Moher, D. (2011) · Cited by 977 · see citing works
Debunking: A Meta-Analysis of the Psychological Efficacy of Messages Countering Misinformation
Chan, M.-P. S., Jones, C. R., Hall Jamieson, K., & Albarracín, D. (2017) · Psychological Science · doi:10.1177/0956797617714579 · Cited by 898 · see citing works
“Next, we performed the bias tests using selection models (Vevea & Woods, 2005), which are weightfunction models accounting for the fact that not all effect sizes have the same probability of being published.”
MethodThe relationship between materialism and personal well-being: A meta-analysis
Dittmar, H., Bond, R., Hurst, M., & Kasser, T. (2014) · Journal of Personality and Social Psychology · doi:10.1037/a0037409 · Cited by 890 · see citing works
“We applied the same four different weight functions in our analyses as were used by Vevea and Woods (2005).”
A social comparison theory meta-analysis 60+ years on.
Gerber, J. P., Wheeler, L., & Suls, J. (2018) · Psychological Bulletin · doi:10.1037/bul0000127 · Cited by 808 · see citing works
“Of note, Vevea and Woods’ (2005) weight function involves choice of direction and size for the weight function.”
BackgroundMethodInfluentialA meta-analysis of risk factors for post-traumatic stress disorder in children and adolescents
Trickey, D., Siddaway, A. P., Meiser-Stedman, R., Serpell, L., & Field, A. P. (2012) · Clinical Psychology Review · doi:10.1016/j.cpr.2011.12.001 · Cited by 828 · see citing works
“…Qouta, Sarraj, & Montgomery, 2006 PTSD Reaction Index 3 0.11 0.11 0.00 0.22 Smith et al., 2001 Impact of Events Scale — Revised 2 0.35 0.03 0.33 0.37 Wickrama & Kaspar, 2007 DSM-IV diagnostic 1 0.40 interview items Parental depression Meiser-Stedman, Yule, Dalgleish, Smith, & Glucksman,…”
BackgroundInfluentialMertens, S., Herberz, M., Hahnel, U. J. J., & Brosch, T. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2107346118 · Cited by 778 · see citing works
“3) revealed an asymmetric distribution that suggested a one-tailed overrepresentation of positive effect sizes in studies with comparatively low statistical power (43).”
BackgroundMethodCoping flexibility and psychological adjustment to stressful life changes: A meta-analytic review.
Cheng, C., Lau, H.-P. B., & Chan, M.-P. S. (2014) · Psychological Bulletin · doi:10.1037/a0037913 · Cited by 664 · see citing works
“The weights of all four selection methods are obtainable from Table 1 of Vevea and Woods’ (2005) article.”
MethodIs actual similarity necessary for attraction? A meta-analysis of actual and perceived similarity
Montoya, R. M., Horton, R. S., & Kirchner, J. (2008) · Journal of Social and Personal Relationships · doi:10.1177/0265407508096700 · Cited by 855 · see citing works
“To investigate the role of publication bias in determining the actual presence of this effect, we employed a computer program designed by Vevea and Woods (2005).”
MethodA meta-analytic review of authentic and transformational leadership: A test for redundancy
Banks, G. C., McCauley, K. D., Gardner, W. L., & Guler, C. E. (2016) · The Leadership Quarterly · doi:10.1016/j.leaqua.2016.02.006 · Cited by 681 · see citing works
“In order to triangulate the possibility of publication bias, we report the results of the trim and fill (Duval, 2005), moderate selection models (Vevea & Woods, 2005), and cumulative meta-analysis (Kepes, Banks, McDaniel, & Whetzel, 2012).”
Recent developments in meta‐analysis
Sutton, A. J., & Higgins, J. P. T. (2007) · Statistics in Medicine · doi:10.1002/sim.2934 · Cited by 583 · see citing works
“While it has been suggested that correcting for publication bias is not possible without making untestable assumptions [130], a bound for the bias as a function of the fraction of missing studies has been proposed [131], as has the use of a priori specified selection models [132].”
BackgroundCorrecting for Bias in Psychology: A Comparison of Meta-Analytic Methods
Carter, E. C., Schönbrodt, F. D., Gervais, W. M., & Hilgard, J. (2019) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245919847196 · Cited by 588 · see citing works
Does marriage and relationship education work? A meta-analytic study.
Hawkins, A. J., Blanchard, V. L., Baldwin, S. A., & Fawcett, E. B. (2008) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0012584 · Cited by 556 · see citing works
“Indeed, meta-analyses that ignore unpublished studies likely overstate the true effect size (e.g., Vevea & Woods, 2005).”
BackgroundTan, J. J. X., Kraus, M. W., Carpenter, N. C., & Adler, N. E. (2020) · Psychological Bulletin · doi:10.1037/bul0000258 · Cited by 582 · see citing works
Dawel, A., O’Kearney, R., McKone, E., & Palermo, R. (2012) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2012.08.006 · Cited by 525 · see citing works
Siegel, E. H., Sands, M. K., Van den Noortgate, W., Condon, P., Chang, Y., Dy, J., Quigley, K. S., & Barrett, L. F. (2018) · Psychological Bulletin · doi:10.1037/bul0000128 · Cited by 515 · see citing works
“We used the method described by Hedges and Vevea (1996) and Vevea and Woods (2005) to compute a sensitivity analysis because it performs well in random-effects designs, can accommodate the inclusion of moderators, and allows for the flexibility to explore specific conditions of publication bias…”
BackgroundMethodThe Effectiveness of Customer Participation in New Product Development: A Meta-Analysis
Chang, W., & Taylor, S. A. (2015) · Journal of Marketing · doi:10.1509/jm.14.0057 · Cited by 515 · see citing works
“…model that calculates the adjusted metaanalytic effect size estimate by considering the statistical significance of the effect size in certain samples (Vevea and Woods 2005) shows that the availability bias in our meta–data set is not likely to be a serious issue in moderate asymmetry situations.”
BackgroundFerguson, C. J., & Brannick, M. Τ. (2011) · Psychological Methods · doi:10.1037/a0024445 · Cited by 493 · see citing works
“There are sophisticated methods available for such analyses (e.g., Vevea & Woods, 2005), but the authors of the meta-analyses in our sample rarely used them (the rare exceptions were Pettigrew & Tropp, 2006; Shadish & Baldwin, 2005).”
MethodStrategic CSR: A Concept Building Meta‐Analysis
Vishwanathan, P., van Oosterhout, H., Heugens, P. P. M. A. R., Durán, P., & van Essen, M. (2019) · Journal of Management Studies · doi:10.1111/joms.12514 · Cited by 505 · see citing works
“We identified the specific weights and probability value cutoff points for our sample based on Vevea and Woods (2005).”
McShane, B. B., Böckenholt, U., & Hansen, K. T. (2016) · Perspectives on Psychological Science · doi:10.1177/1745691616662243 · Cited by 398 · see citing works
“…less for obtaining a single estimate that purports to adjust for publication bias ex post and more for sensitivity analysis—that is, exploring the range of estimates that result from assuming different forms of and severity of publication bias (see Vevea & Woods, 2005, and Hedges & Vevea, 2005).”
BackgroundMethodZeegers, M. A. J., Colonnesi, C., Stams, G. J. J. M., & Meins, E. (2017) · Psychological Bulletin · doi:10.1037/bul0000114 · Cited by 431 · see citing works
“That is, as the number of studies we found on mentalization was small, single studies may be very influential to the calculation of the pooled correlation on mentalization and attachment/ sensitivity (Vevea & Woods, 2005).”
BackgroundThe antecedents and outcomes of workplace ostracism: A meta-analysis.
Howard, M. C., Cogswell, J. E., & Smith, M. B. (2019) · Journal of Applied Psychology · doi:10.1037/apl0000453 · Cited by 431 · see citing works
“These analyses were fail-safe k, Egger’s test, randomeffects trim-and-fill method, and weight-function model analysis (Coburn & Vevea, 2015; Egger, Davey Smith, Schneider, & Minder, 1997; Vevea & Hedges, 1995; Vevea & Woods, 2005).”
Zell, E., Strickhouser, J. E., Sedikides, C., & Alicke, M. D. (2019) · Psychological Bulletin · doi:10.1037/bul0000218 · Cited by 410 · see citing works
Publication Bias in the Organizational Sciences
Kepes, S., Banks, G. C., McDaniel, M. A., & Whetzel, D. L. (2012) · Organizational Research Methods · doi:10.1177/1094428112452760 · Cited by 400 · see citing works
“Unfortunately, the estimation of publication bias using selection models has been limited by data set requirements because a large number of samples is necessary to estimate the model with adequate accuracy (e.g., at least 100 samples; Vevea & Woods, 2005).”
BackgroundMethodResultInfluentialPancultural self-enhancement reloaded: A meta-analytic reply to Heine (2005).
Sedikides, C., Gaertner, L., & Vevea, J. L. (2005) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.89.4.539 · Cited by 383 · see citing works
“We estimated the parameters for the random-or mixed-effects models with unconditional maximum likelihood using a computer program developed for that purpose (Vevea & Hedges, 1995; Vevea & Woods, 2004).”
MethodSelf-citationStockinger, K., Pekrun, R. H., & Lester, J. C. (2018) · Learning and Instruction · doi:10.1016/j.learninstruc.2018.08.002 · Cited by 403 · see citing works
“The R-packages Weightr (Coburn & Vevea, 2016; weighting schemes provided by; Vevea & Woods, 2005) and Metafor (Viechtbauer, 2010) were used for WFM and contour-en-hanced funnel plot production, respectively.”
MethodThe Effectiveness of Second Language Strategy Instruction: A Meta‐analysis
Plonsky, L. (2011) · Language Learning · doi:10.1111/j.1467-9922.2011.00663.x · Cited by 394 · see citing works
Antecedents and consequences of employee organizational cynicism: A meta-analysis
Chiaburu, D. S., Peng, A. C., Oh, I., Banks, G. C., & Lomeli, L. C. (2013) · Journal of Vocational Behavior · doi:10.1016/j.jvb.2013.03.007 · Cited by 391 · see citing works
Rewards of kindness? A meta-analysis of the link between prosociality and well-being.
Hui, B. P. H., Ng, J. C. K., Berzaghi, E., Cunningham-Amos, L. A., & Kogan, A. (2020) · Psychological Bulletin · doi:10.1037/bul0000298 · Cited by 398 · see citing works
Moreno, S. G., Sutton, A. J., Ades, A., Stanley, T. D., Abrams, K. R., Peters, J. L., & Cooper, N. J. (2009) · BMC Medical Research Methodology · doi:10.1186/1471-2288-9-2 · Cited by 373 · see citing works
Sensitivity Analysis for Publication Bias in Meta-Analyses
Mathur, M. B., & VanderWeele, T. J. (2020) · Journal of the Royal Statistical Society Series C: Applied Statistics · doi:10.1111/rssc.12440 · Cited by 354 · see citing works
“…meta-analysis are only mildly attenuated even under severe publication bias, then the results might be considered fairly robust to publication bias (Vevea and Woods, 2005), whereas if the results are severely attenuated under mild publication bias the results might be considered sensitive to…”
BackgroundMethodResultInfluentialLoh, R., Stamatakis, E., Folkerts, D., Allgrove, J. E., & Moir, H. J. (2020) · Sports Medicine · doi:10.1007/s40279-019-01183-w · Cited by 326 · see citing works
“The Vevea and Woods [58] method estimated that the SMD for TAG measures comparing PA breaks with sitting would be reduced from − 0.”
MethodPietschnig, J., Penke, L., Wicherts, J. M., Zeiler, M., & Voracek, M. (2015) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2015.09.017 · Cited by 316 · see citing works
“We used four different study weight functions as specified by Vevea and Woods (2005) assuming either moderate or severe and one- or two-tailed selection of effect sizes.”
Murphy, N. A., & Isaacowitz, D. M. (2008) · Psychology and Aging · doi:10.1037/0882-7974.23.2.263 · Cited by 338 · see citing works
“In general, funnel plots can be useful to help detect publication bias, but interpretation can be dubious (Vevea & Woods, 2005).”
BackgroundPublication bias examined in meta-analyses from psychology and medicine: A meta-meta-analysis
van Aert, R. C. M., Wicherts, J. M., & van Assen, M. A. L. M. (2019) · PLoS ONE · doi:10.1371/journal.pone.0215052 · Cited by 345 · see citing works
“Some approaches estimate the selection model [74, 77] whereas others assume a known selection model [79].”
BackgroundCognitive Training Does Not Enhance General Cognition
Sala, G., & Gobet, F. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2018.10.004 · Cited by 303 · see citing works
“05) effect sizes [108] or test the robustness of the results by postulating a probability for the nonsignificant effects to be published [109,110].”
The impact of shared book reading on children's language skills: A meta-analysis
Noble, C. H., Sala, G., Peter, M., Lingwood, J., Rowland, C. F., Gobet, F. R., & Pine, J. M. (2019) · Educational Research Review · doi:10.1016/j.edurev.2019.100290 · Cited by 326 · see citing works
Agostini, M., & van Zomeren, M. (2021) · Psychological Bulletin · doi:10.1037/bul0000256 · Cited by 301 · see citing works
Sanchez, A. L., Cornacchio, D., Poznanski, B., Golik, A. M., Chou, T., & Comer, J. S. (2017) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1016/j.jaac.2017.11.022 · Cited by 303 · see citing works
No evidence for nudging after adjusting for publication bias
Maier, M., Bartos̆, F., Stanley, T. D., Shanks, D. R., Harris, A. J. L., & Wagenmakers, E. (2022) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2200300119 · Cited by 289 · see citing works
“Their sensitivity analysis (4) indicates that the true effect size could be as low as d = 0.”
BackgroundPinquart, M., & Ebeling, M. (2019) · Educational Psychology Review · doi:10.1007/s10648-019-09506-z · Cited by 305 · see citing works
Banks, G. C., Batchelor, J. H., Seers, A., O’Boyle, E. H., Pollack, J. M., & Gower, K. (2013) · Journal of Organizational Behavior · doi:10.1002/job.1885 · Cited by 272 · see citing works
“Third, a priori selection models (Hedges & Vevea, 2005; Vevea & Woods, 2005) can be used to weight samples in the analysis on the basis of the probability that they are available to be included in the meta-analysis.”
MethodShadish, W. R., Hedges, L. V., & Pustejovsky, J. E. (2014) · Journal of School Psychology · doi:10.1016/j.jsp.2013.11.005 · Cited by 270 · see citing works
“Although that was the case in the PRT dataset, it will not always hold—though in such cases the methods of Vevea and colleagues (Citkowicz & Vevea, 2011; Hedges & Vevea, 1996; Vevea & Hedges, 1995; Vevea & Woods, 2005) should be considered.”
BackgroundMethodVideo game training does not enhance cognitive ability: A comprehensive meta-analytic investigation
Sala, G., Tatlidil, K. S., & Gobet, F. (2018) · Psychological Bulletin · doi:10.1037/bul0000139 · Cited by 257 · see citing works
Meta-analysis using effect size distributions of only statistically significant studies.
van Assen, M. A. L. M., van Aert, R. C. M., & Wicherts, J. M. (2015) · Psychological Methods · doi:10.1037/met0000025 · Cited by 265 · see citing works
Conn, V. S., Hafdahl, A. R., Brown, S. A., & Brown, L. M. (2007) · Patient Education and Counseling · doi:10.1016/j.pec.2007.10.004 · Cited by 263 · see citing works
“Although this is consistent with a publication bias against non-significant or negative results, it may also arise from phenomena such as certain patterns of heterogeneity [221].”
BackgroundLauriola, M., Panno, A., Levin, I. P., & Lejuez, C. W. (2013) · Journal of Behavioral Decision Making · doi:10.1002/bdm.1784 · Cited by 257 · see citing works
“As a final test of publication bias, we carried out a sensitivity analysis hypothesizing both moderate and severe one-tailed selection of studies (Vevea & Woods, 2005).”
MethodMeta-analytic structural equation modeling with moderating effects on SEM parameters.
Jak, S., & Cheung, M. W. L. (2020) · Psychological Methods · doi:10.1037/met0000245 · Cited by 262 · see citing works
Statistical methods for dealing with publication bias in meta‐analysis
Jin, Z., Zhou, X., & He, J. (2014) · Statistics in Medicine · doi:10.1002/sim.6342 · Cited by 252 · see citing works
“The main change in the modification model was imposing a set of fixed weights determined a priori, but not estimating the weights from the data [43]; yet, a priori chosen weights are usually arbitrary, so it should be used and interpreted cautiously.”
BackgroundMethodPromoting memory consolidation during sleep: A meta-analysis of targeted memory reactivation
Hu, X., Cheng, L. Y., Chiu, M. H., & Paller, K. A. (2020) · Psychological Bulletin · doi:10.1037/bul0000223 · Cited by 232 · see citing works
“Fifth, we employed a selection model with a priori weight functions that could model four different scenarios of publication biases: moderate one-tailed selection, severe one-tailed selection, moderate two-tailed selection, and severe two-tailed selection (Vevea & Woods, 2005).”
InfluentialGrowth mindsets and psychological distress: A meta-analysis
Burnette, J. L., Knouse, L. E., Vavra, D. T., O'Boyle, E., & Brooks, M. A. (2020) · Clinical Psychology Review · doi:10.1016/j.cpr.2020.101816 · Cited by 271 · see citing works
Burnout and depression in nurses: a systematic review and meta-analysis
Chen, C., & Meier, S. T. (2021) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2021.104099 · Cited by 246 · see citing works
Effects of stress on decisions under uncertainty: A meta-analysis.
Starcke, K., & Brand, M. (2016) · Psychological Bulletin · doi:10.1037/bul0000060 · Cited by 247 · see citing works
Signaling text-picture relations in multimedia learning: A comprehensive meta-analysis
Richter, J., Scheiter, K., & Eitel, A. (2016) · Educational Research Review · doi:10.1016/j.edurev.2015.12.003 · Cited by 242 · see citing works
The development of gender differences in spatial reasoning: A meta-analytic review.
Lauer, J. E., Yhang, E., & Lourenco, S. F. (2019) · Psychological Bulletin · doi:10.1037/bul0000191 · Cited by 239 · see citing works
Transfer of test- enhanced learning: meta-analytic review and synthesis
Pan, S. C., & Rickard, T. C. (2018) · Psychological Bulletin · doi:10.1037/bul0000151 · Cited by 286 · see citing works
Meta-analysis in Second Language Research: Choices and Challenges
Oswald, F. L., & Plonsky, L. (2010) · Annual Review of Applied Linguistics · doi:10.1017/s0267190510000115 · Cited by 237 · see citing works
“…bias, namely, the tendency of editors, reviewers, and individual researchers to favor only statistically significant or theoretically appealing findings (Moncrieff, 1998; Torgerson, 2006; Vevea & Woods, 2005; for a comprehensive review of this issue, see Rothstein, Sutton, & Borenstein, 2005).”
BackgroundExistential therapies: A meta-analysis of their effects on psychological outcomes.
Vos, J., Craig, M., & Cooper, M. (2014) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0037167 · Cited by 237 · see citing works
A meta-analysis of virtual reality training programs for social skill development
Howard, M. C., & Gutworth, M. B. (2020) · Computers & Education · doi:10.1016/j.compedu.2019.103707 · Cited by 234 · see citing works
Cognitive and academic benefits of music training with children: A multilevel meta-analysis
Sala, G., & Gobet, F. (2020) · Memory & Cognition · doi:10.3758/s13421-020-01060-2 · Cited by 235 · see citing works
Posttraumatic growth and adjustment among individuals with cancer or HIV/AIDS: A meta-analysis
Sawyer, A., Ayers, S., & Field, A. P. (2010) · Clinical Psychology Review · doi:10.1016/j.cpr.2010.02.004 · Cited by 238 · see citing works
“2 ) the cloud is a little sparse around zero for small studies, which indicates two-tailed publication bias (Vevea & Woods, 2005) .”
BackgroundMethodInfluentialThe effects of shared storybook reading on word learning: A meta-analysis.
Flack, Z. M., Field, A. P., & Horst, J. S. (2018) · Developmental Psychology · doi:10.1037/dev0000512 · Cited by 244 · see citing works
“To quantify the likely effect of publication bias, a sensitivity analysis based on Vevea and Woods (2005) was conducted.”
Mozart effect–Shmozart effect: A meta-analysis
Pietschnig, J., Voracek, M., & Formann, A. K. (2010) · Intelligence · doi:10.1016/j.intell.2010.03.001 · Cited by 230 · see citing works
“…2005), trimand-fill analysis (Duval & Tweedie, 2000) following the approach as suggested by Peters, Sutton, Jones, Abrams and Rushton (2007), a sensitivity analysis for publications bias as suggested by Vevea and Woods (2005), and a method based on truncated normal distributions (Formann, 2008).”
MethodPalmer, C. A., Bower, J. L., Cho, K. W., Clementi, M. A., Lau, S., Oosterhoff, B. J., & Alfano, C. A. (2023) · Psychological Bulletin · doi:10.1037/bul0000410 · Cited by 237 · see citing works
How to conduct a meta-analysis in eight steps: A practical guide
Hansen, C., Steinmetz, H., & Block, J. (2022) · Management Review Quarterly · doi:10.1007/s11301-021-00247-4 · Cited by 218 · see citing works
“2009), the trim and fill method (Duval and Tweedie 2000a, b), Egger et al.’s (1997) regression test, failsafe N (Rosenthal 1979), or selection models (Hedges and Vevea 2005; Vevea and Woods 2005).”
BackgroundMethodLawrence, P. J., Murayama, K., & Creswell, C. (2019) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1016/j.jaac.2018.07.898 · Cited by 214 · see citing works
“We used funnel plots and Egger tests to assess the risk of publication bias and used a priori weight functions for sensitivity analyses.(60)”
MethodThe relationship between adult attachment style and post-traumatic stress symptoms: A meta-analysis
Woodhouse, S., Ayers, S., & Field, A. P. (2015) · Journal of Anxiety Disorders · doi:10.1016/j.janxdis.2015.07.002 · Cited by 210 · see citing works
“To quantify the likely effect of publication bias, a sensitivity analysis based on Vevea and Woods (2005) was conducted which adjusts the population effect size estimate for moderate and severe one-and two-tailed selection bias.”
MethodPublication bias in meta‐analyses from the Cochrane Database of Systematic Reviews
Kiciński, M., Springate, D. A., & Kontopantelis, E. (2015) · Statistics in Medicine · doi:10.1002/sim.6525 · Cited by 214 · see citing works
“, negative-exponential) [34,57], step functions with more than one cut point [34,36,58], and nonparametric functions [38, 39].”
BackgroundLang, C., Kalak, N., Brand, S., Holsboer‐Trachsler, E., Pühse, U., & Gerber, M. (2015) · Sleep Medicine Reviews · doi:10.1016/j.smrv.2015.07.004 · Cited by 204 · see citing works
The Effects of Family Therapies for Adolescent Delinquency and Substance Abuse: A Meta‐analysis
Baldwin, S. A., Christian, S., Berkeljon, A., & Shadish, W. R. (2012) · Journal of Marital and Family Therapy · doi:10.1111/j.1752-0606.2011.00248.x · Cited by 244 · see citing works
“…plots (Peters, Sutton, Jones, Abrams, & Ruston, 2008; Sterne & Harbord, 2004), Begg’s rank correlation test (Begg & Mazumdar, 1994), and Egger’s regression test (Egger, Smith, Schnieder, & Minder, 1997), trim-and-fill analysis (Duval & Tweedie, 2000a,b), and selection models (Vevea & Woods, 2005).”
BackgroundMethodInfluentialAchievement Motivation and Academic Dishonesty: A Meta-Analytic Investigation
Krou, M. R., Fong, C. J., & Hoff, M. A. (2020) · Educational Psychology Review · doi:10.1007/s10648-020-09557-7 · Cited by 219 · see citing works
“Weights reflect the process by which studies tend to be published based on higher levels of statistical significance (Vevea and Woods 2005).”
BackgroundAfonso, J. R., Ramírez‐Campillo, R., Clemente, F. M., Büttner, F., & Andrade, R. (2023) · Sports Medicine · doi:10.1007/s40279-023-01927-9 · Cited by 213 · see citing works
“Fixed Weights Selection Model [61–63].”
BackgroundSiddaway, A. P., Taylor, P. J., Wood, A. M., & Schulz, J. (2015) · Journal of Affective Disorders · doi:10.1016/j.jad.2015.05.046 · Cited by 186 · see citing works
“tion bias effects (Vevea and Woods, 2005).”
BackgroundInfluentialDonald, J. N., Bradshaw, E. L., Ryan, R. M., Basarkod, G., Ciarrochi, J., Duineveld, J. J., Guo, J., & Sahdra, B. K. (2020) · Personality and Social Psychology Bulletin · doi:10.1177/0146167219896136 · Cited by 182 · see citing works
“However, it should be noted that the 3PSM sensitivity test is less robust with less than 10 data points (Vevea & Woods, 2005), which was the case for correlational amotivation studies and intervention studies of identified regulation.”
BackgroundMethodCue-reactivity in behavioral addictions: a meta-analysis and methodological considerations
Starcke, K., Antons, S., Trotzke, P., & Brand, M. (2018) · Journal of Behavioral Addictions · doi:10.1556/2006.7.2018.39 · Cited by 180 · see citing works
“Funnel plots as proposed, for example, by Vevea and Woods (2005) are principally suited for small meta-analyses, but at least 10 data sets should be included in each calculation (Higgins & Green, 2011), which was not the case for all calculations.”
BackgroundMethodChandler, M. J., Parks, A. C., Marsiske, M., Rotblatt, L. J., & Smith, G. E. (2016) · Neuropsychology Review · doi:10.1007/s11065-016-9330-4 · Cited by 169 · see citing works
“Third, we employed a sensitivity analysis approach described by Vevea and Woods (2005), which produces adjusted effect size estimates.”
BackgroundMethodInfluentialCheng, C., Wang, H., Sigerson, L., & Chau, C.-L. (2019) · Psychological Bulletin · doi:10.1037/bul0000198 · Cited by 182 · see citing works
Oliveira, S., Roberto, M. S., Pereira, N., Pinto, A. M., & Simão, A. M. V. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.677217 · Cited by 210 · see citing works
Otten, J., Åhrén, B., & Olsson, T. (2014) · Diabetologia · doi:10.1007/s00125-014-3285-x · Cited by 170 · see citing works
“1, R Foundation for Statistical Computing, Vienna, Austria) using a code written byVevea andWoods [19] and applying a script from Field and Gillett to feed SPSS data into R [18].”
MethodGaluschka, K., Görgen, R., Kalmar, J., Haberstroh, S., Schmalz, X., & Schulte‐Körne, G. (2020) · Educational Psychologist · doi:10.1080/00461520.2019.1659794 · Cited by 176 · see citing works
“We used the packages compute.es (Del Re, 2013), robumeta (Fisher & Tipton, 2015), metafor (Viechtbauer, 2010), clubSandwich (Pustejovsky, 2016) and the Shiny application weightr that implements the weight function model published by Vevea and Woods (2005).”
MethodAssessing Publication Bias in Meta-Analyses in the Presence of Between-Study Heterogeneity
Peters, J., Sutton, A. J., Jones, D. R., Abrams, K. R., Rushton, L., & Moreno, S. G. (2010) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/j.1467-985x.2009.00629.x · Cited by 168 · see citing works
“Following Vevea and Woods (2005), we specify the weights at each cut point (rather than estimate them as in Hedges and Vevea (1996)).”
MethodA historical review of publication bias
Marks‐Anglin, A., & Chen, Y. (2020) · Research Synthesis Methods · doi:10.1002/jrsm.1452 · Cited by 165 · see citing works
Sanada, K., Montero-Marin, J., Alda Díez, M., Salas-Valero, M., Pérez-Yus, M. C., Morillo, H., Demarzo, M. M. P., García-Toro, M., & García-Campayo, J. (2016) · Frontiers in Physiology · doi:10.3389/fphys.2016.00471 · Cited by 161 · see citing works
“A t p o st in te rv e n tio n , M B S R g ro u p sh o w e d a te n d e n c y to w a rd a lo w e r A U C G th a n d id th e in a c tiv e c o n tr o ls .”
BackgroundMethodLevine, T. R., Asada, K. J. K., & Carpenter, C. (2009) · Communication Monographs · doi:10.1080/03637750903074685 · Cited by 159 · see citing works
“Vevea and Woods (2005) demonstrated that there are situations where visual examination of the funnel plot may make it difficult to detect evidence of subtle but important publication bias.”
BackgroundTan, B. W. Z., Pooley, J. A., & Speelman, C. P. (2016) · Journal of Autism and Developmental Disorders · doi:10.1007/s10803-016-2854-x · Cited by 157 · see citing works
“Although the result of Kendall’s tau is non-significant, in view of the small number of studies included in the metaanalysis, publication bias cannot be rejected and therefore a sensitivity analysis (Vevea and Woods, 2005) was conducted to estimate and account for the likelihood of publication bias.”
BackgroundMethodZhang, Q., Wang, J., & Neitzel, A. J. (2022) · Journal of Youth and Adolescence · doi:10.1007/s10964-022-01684-4 · Cited by 156 · see citing works
“This study used weightr package (Coburn & Vevea, 2019) to apply the weight-function model (Vevea & Woods, 2005).”
MethodThe efficacy of mindfulness-based interventions in primary care: a meta-analytic review.
Demarzo, M. M. P., Montero-Marin, J., Cuijpers, P., Zabaleta-del-Olmo, E., Mahtani, K. R., Vellinga, A., Vicens, C., Lopez-del-Hoyo, Y., & Garcia-Campayo, J. (2015) · The Annals of Family Medicine · doi:10.1370/afm.1863 · Cited by 152 · see citing works
“de Vibe and Moum39 2006 Norway Patients with stress and chronic illness I: 47 (20-69) PC: 48 (17-63) 88.”
Acar, F., Seurinck, R., Eickhoff, S. B., & Moerkerke, B. (2018) · PLOS ONE · doi:10.1371/journal.pone.0208177 · Cited by 154 · see citing works
“The method of Vevea and Woods [42] corrects for publication bias by adjusting weights assigned to different studies based on study characteristics that could influence likeliness of publication, such as statistical significance.”
MethodStuijfzand, S., Creswell, C., Field, A. P., Pearcey, S., & Dodd, H. F. (2017) · Journal of Child Psychology and Psychiatry · doi:10.1111/jcpp.12822 · Cited by 154 · see citing works
“Following Vevea and Woods (2005), a prespecified sensitivity analysis was conducted using a priori weight functions.”
Bartos̆, F., Maier, M., Wagenmakers, E., Doucouliagos, H., & Stanley, T. D. (2022) · Research Synthesis Methods · doi:10.1002/jrsm.1594 · Cited by 138 · see citing works
“28 – 30 The new method proposed here is another useful tool to accommodate publication bias, and researchers are free to supplement it with other methods.”
Effects of Mindfulness-Based Therapies for Female Sexual Dysfunction: A Meta-Analytic Review
Stephenson, K. R., & Kerth, J. (2017) · The Journal of Sex Research · doi:10.1080/00224499.2017.1331199 · Cited by 145 · see citing works
“Thus, we also used a more recent approach suggested by McShane et al. (2016; see also Vevea & Woods, 2005): sensitivity analysis.”
MethodConducting Meta-analyses of Proportions in R
Wang, N. (2023) · Journal of Behavioral Data Science · doi:10.35566/jbds/v3n2/wang · Cited by 143 · see citing works
Panadero, E., Jönsson, A., Pinedo, L., & Fernández‐Castilla, B. (2023) · Educational Psychology Review · doi:10.1007/s10648-023-09823-4 · Cited by 212 · see citing works
“Given the mixed evidence on the potential presence of publication bias, we decided to apply Vevea and Woods (2005) selection model to obtain a corrected, overall effect size.”
Publication Bias in Strategic Management Research
Harrison, J. S., Banks, G. C., Pollack, J. M., O’Boyle, E. H., & Short, J. C. (2014) · Journal of Management · doi:10.1177/0149206314535438 · Cited by 141 · see citing works
“The selection model approach takes into consideration the probability that certain samples are included in a meta-analysis due to specific characteristics (Hedges & Vevea, 2005; Vevea & Woods, 2005).”
BackgroundMethodInfluentialKredlow, M. A., Unger, L. D., & Otto, M. W. (2016) · Psychological Bulletin · doi:10.1037/bul0000034 · Cited by 130 · see citing works
“It is important to consider that funnel plot asymmetry may be due to factors other than publication bias and because of this, the Trim and Fill method may lead to overcorrection (Vevea & Woods, 2005).”
- Authors
- Vevea, J. L., & Woods, C. M.
- Affiliations
- University of California, Santa Cruz (US)
- Washington University in St. Louis (US)
- ORCID
- Woods, Carol M.
- Venue
- Psychological Methods
- Published
- 2005-12
- Volume
- 10
- Issue
- 4
- Pages
- 428-443
- ISSN
- 1082-989X (ISSN-L)
- 1939-1463
- OpenAlex
- W1974479479
- Cited by
- 461
- References
- 94
- Citations by year
- 2026: 24 · 2025: 33 · 2024: 35 · 2023: 35 · 2022: 36 · 2021: 41 · 2020: 35 · 2019: 49 · 2018: 33 · 2017: 23 · 2016: 12 · 2015: 22 · 2014: 13 · 2013: 10 · 2012: 11
- FWCI
- 9.19
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychological Methods (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Institutes of Health (4r44ag024771-02)
- National Institute on Aging (AG024771)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › scientometrics and bibliometrics research
- Keywords
- publication bias, file drawer problem, funnel plot asymmetry, meta-analysis, sensitivity analysis, bias detection, fixed effects model, random effects models, mixed models
- MeSH terms
- Data Interpretation, Statistical, Humans, Models, Psychological, Publishing, Research, Research Design, Sensitivity and Specificity, Meta-Analysis as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
