The DOI matches its Crossref record; 61 of the 63 references it cites were found in a registry or an index.
Modelling publication bias in meta-analysis: a review
Abstract
Meta-analysis is now a widely used technique for summarizing evidence from multiple studies. Publication bias, the bias induced by the fact that research with statistically significant results is potentially more likely to be submitted and published than work with null or non-significant results, poses a thereat to the validity of such analyses. The implication of this is that combining only the identified published studies uncritically may lead to an incorrect, usually over optimistic, conclusion. How publication bias should be addressed when carrying out a meta-analysis is currently a hotly debated subject. While statistical methods to test for its presence are starting be used, they do not address the problem of how to proceed if publication bias is suspected. This paper provides a review of methods, which can be employed as a sensitivity analysis to assess the likely impact of publication bias on a meta-analysis. It is hoped that this will raise awareness of such methods, and promote their use and development, as well as provide an agenda for future research.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“There are two main threats to the validity of meta-analytic results: the systematic omission of studies (e.g., publication bias suppressing studies with null results36,37), and the flexible selection of analysis approaches within each study inflating the rate of false positives (e.g., p-hacking38).”
From Accuracy prompts are a replicable and generalizable approach for reducing the spread of misinformation (2022) · cites this paper as [36]
Citations
Cites
Bias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722
Maximum Likelihood from Incomplete Data Via the EM Algorithm
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1977.tb01600.x · Cited by 50,444
Meta-analysis in clinical trials
DerSimonian, R., & Laird, N. (1986) · Controlled Clinical Trials · doi:10.1016/0197-2456(86)90046-2 · Cited by 39,552
Operating characteristics of a rank correlation test for publication bias
Begg, C. B., & Mazumdar, M. (1994) · Biometrics · doi:10.2307/2533446 · Cited by 16,955
Monte Carlo sampling methods using Markov chains and their applications.
Hastings, W. K. (1970) · Biometrika · doi:10.1093/biomet/57.1.97 · Cited by 15,195
Duval, S., & Tweedie, R. (2000) · Biometrics · doi:10.1111/j.0006-341x.2000.00455.x · Cited by 14,099
Wasserman, S. S., Hedges, L. V., & Olkin, I. (1988) · Journal of Educational Statistics · doi:10.2307/1164953 · Cited by 10,377
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068
Journal of The American Statistical Association
Fienberg, S. E. (2004) · Encyclopedia of Statistical Sciences · doi:10.1002/0471667196.ess1317 · Cited by 6,289
Publication bias in clinical research
Easterbrook, P. J., Gopalan, R., Berlin, J. A., & Matthews, D. R. (1991) · The Lancet · doi:10.1016/0140-6736(91)90201-y · Cited by 3,109
A Nonparametric “Trim and Fill” Method of Accounting for Publication Bias in Meta-Analysis
Duval, S., & Tweedie, R. (2000) · Journal of the American Statistical Association · doi:10.1080/01621459.2000.10473905 · Cited by 2,955
A Fail-Safe N for Effect Size in Meta-Analysis
Orwin, R. G. (1983) · Journal of Educational Statistics · doi:10.2307/1164923 · Cited by 2,064
The Handbook of Research Synthesis.
Light, R. J., Cooper, H., & Hedges, L. V. (1994) · Journal of the American Statistical Association · doi:10.2307/2291021 · Cited by 1,880
(2000) · Queen Mary Research Online (Queen Mary University of London) · Cited by 1,795
Self-citationSumming up: The Science of Reviewing Research
Wilkinson, I., Light, R. J., & Pillemer, D. B. (1987) · Journal of Educational Statistics · doi:10.2307/1164691 · Cited by 1,486
Light, R. J., & Pillemer, D. B. (1984) · Harvard University Press eBooks · doi:10.4159/9780674040243 · Cited by 1,422
Empirical assessment of effect of publication bias on meta-analyses
Sutton, A. J. (2000) · BMJ · doi:10.1136/bmj.320.7249.1574 · Cited by 1,398
Self-citationLanguage bias in randomised controlled trials published in English and German
Egger, M., Zellweger-Zähner, T., Schneider, M., Junker, C., Lengeler, C., & Antes, G. (1997) · The Lancet · doi:10.1016/s0140-6736(97)02419-7 · Cited by 1,028
Linde, K., Clausius, N., Ramirez, G., Melchart, D., Eitel, F., Hedges, L. V., & Jonas, W. B. (1997) · The Lancet · doi:10.1016/s0140-6736(97)02293-9 · Cited by 979
Publication bias: evidence of delayed publication in a cohort study of clinical research projects
Stern, J. M., & Simes, R. J. (1997) · BMJ · doi:10.1136/bmj.315.7109.640 · Cited by 818
Publication bias—a problem in interpreting medical data
Begg, C. B., & Berlin, J. A. (1988) · Journal of the Royal Statistical Society. Series A (Statistics in Society) · doi:10.2307/2982993 · Cited by 814
Bayesian approaches to random-effects meta-analysis: a comparative study
Smith, T. C., Spiegelhalter, D. J., & Thomas, A. (1995) · Statistics in Medicine · doi:10.1002/sim.4780142408 · Cited by 568
Grégoire, G., Derderian, F., & LeLorier, J. (1995) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(94)00098-b · Cited by 521
The effectiveness and cost-effectiveness of prophylactic removal of wisdom teeth.
Song, O'Meara, Wilson, P. M., Golder, S., & Kleijnen, J. (2000) · Health Technology Assessment · doi:10.3310/hta4150 · Cited by 478
Publication Bias and Dissemination of Clinical Research
Begg, C. B., & Berlin, J. A. (1989) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/81.2.107 · Cited by 447
Publication and related biases.
Song, Eastwood, Gilbody, Duley, & Sutton (2000) · Health Technology Assessment · doi:10.3310/hta4100 · Cited by 427
Inference for Non-random Samples
Copas, J. B., & Li, H. G. (1997) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/1467-9868.00055 · Cited by 398
Selection Models and the File Drawer Problem
Iyengar, S., & Greenhouse, J. B. (1988) · Statistical Science · doi:10.1214/ss/1177013012 · Cited by 388
Robust Bayesian analysis: sensitivity to the prior
Berger, J. O. (1990) · Journal of Statistical Planning and Inference · doi:10.1016/0378-3758(90)90079-a · Cited by 373
A general linear model for estimating effect size in the presence of publication bias
Vevea, J. L., & Hedges, L. (1995) · Psychometrika · doi:10.1007/bf02294384 · Cited by 367
Modeling Publication Selection Effects in Meta-Analysis
Hedges, L. V. (1992) · Statistical Science · doi:10.1214/ss/1177011364 · Cited by 359
Bayesian methods in health technology assessment: a review.
Spiegelhalter, Myles, Jones, & Abrams, K. R. (2000) · Health Technology Assessment · doi:10.3310/hta4380 · Cited by 359
Self-citationSystematic reviews of trials and other studies.
Sutton, A. J., Abrams, K. R., Jones, D. R., Sheldon, T., & Сонг (1998) · Health Technology Assessment · doi:10.3310/hta2190 · Cited by 270
Self-citationHedges, L. V. (1984) · Journal of Educational Statistics · doi:10.3102/10769986009001061 · Cited by 241
Hedges, L., & Vevea, J. L. (1996) · Journal of Educational and Behavioral Statistics · doi:10.3102/10769986021004299 · Cited by 223
An Approach for Assessing Publication Bias Prior to Performing a Meta-Analysis
Dear, K., & Begg, C. (1992) · Statistical Science · doi:10.1214/ss/1177011363 · Cited by 202
What Works?: Selectivity Models and Meta-Analysis
Copas, J. B. (1999) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/1467-985x.00123 · Cited by 168
Estimating effect size: Bias resulting from the significance criterion in editorial decisions
Lane, D. M., & Dunlap, W. P. (1978) · British Journal of Mathematical and Statistical Psychology · doi:10.1111/j.2044-8317.1978.tb00578.x · Cited by 161
Givens, G. H., Smith, D. D., & Tweedie, R. L. (1997) · Statistical Science · doi:10.1214/ss/1030037958 · Cited by 127
Piano, F. D., & Giorlamini-Smith, M. (1993) · Psychotherapy in Private Practice · doi:10.1300/j294v12n03_07 · Cited by 103
Time to register randomised trials
Horton, R., & Smith, R. W. (1999) · BMJ · doi:10.1136/bmj.319.7214.865 · Cited by 91
Estimating Publication Bias in Meta-Analysis
Rust, R. T., Lehmann, D. R., & Farley, J. U. (1990) · Springer eBooks · doi:10.1007/978-3-031-88060-5_10 · Cited by 79
Reanalysis of epidemiological evidence on lung cancer and passive smoking
Copas, J. B. (2000) · BMJ · doi:10.1136/bmj.320.7232.417 · Cited by 60
Hierarchical Selection Models With Applications in Meta-Analysis
Silliman, N. (1997) · Journal of the American Statistical Association · doi:10.1080/01621459.1997.10474047 · Cited by 47
Horton, R. C. (1997) · The Lancet · doi:10.1016/s0140-6736(05)62564-0 · Cited by 45
Linearization of Bayesian robustness problems
Wasserman, L., Lavine, M., & Wolpert, R. L. (1993) · Journal of Statistical Planning and Inference · doi:10.1016/0378-3758(93)90109-j · Cited by 44
DIFFERENTIAL RECALL BIAS AND SPURIOUS ASSOCIATIONS IN CASE/CONTROL STUDIES
Barry, D. (1996) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19961215)15:23 · Cited by 39
PROBING ENCOUNTERED DATA, META ANALYSIS AND WEIGHTED DISTRIBUTION METHODS
Patil, G. P., & Taillie, C. (1989) · Elsevier eBooks · doi:10.1016/b978-0-444-88029-1.50035-6 · Cited by 37
Modeling publication bias using weighted distributions in a Bayesian framework
Larose, D. T., & Dey, D. K. (1998) · Computational Statistics & Data Analysis · doi:10.1016/s0167-9473(97)00039-x · Cited by 35
CHAPTER 9 – Random Effects Models for Effect Sizes
Hedges, L. (1985) · Elsevier eBooks · doi:10.1016/b978-0-08-057065-5.50014-2 · Cited by 35
Estimating Publication Bias in Meta-Analysis
Rust, R. T., Lehmann, D. R., & Farley, J. U. (1990) · Journal of Marketing Research · doi:10.2307/3172848 · Cited by 32
An Application of Gibbs Sampling to Estimation in Meta-Analysis: Accounting for Publication Bias
Cleary, R. J., & Casella, G. (1997) · Journal of Educational and Behavioral Statistics · doi:10.3102/10769986022002141 · Cited by 30
Relation between hostility and coronary heart disease
Petticrew, M. P., Gilbody, S. M., Sheldon, T., Hemingway, H., & Marmot, M. G. (1999) · BMJ · doi:10.1136/bmj.319.7214.917 · Cited by 24
Self-citationNonparametric classes of weight functions to model publication bias
Silliman, N. (1997) · Biometrika · doi:10.1093/biomet/84.4.909 · Cited by 20
Sutton, A. J., Lambert, P. F., Hellmich, M., Jones, D. S., & Abrams, K. R. (2000) · Chapman & Hall/CRC biostatistics series · doi:10.1201/9780203909935.ch15 · Cited by 18
Self-citationEstimating a Summarized Odds Ratio whilst Eliminating Publication Bias in Meta-analysis *
Sugita, M., Kanamor, M., Izuno, T., & Miyakawa, M. (1992) · Japanese Journal of Clinical Oncology · doi:10.1093/oxfordjournals.jjco.a039577 · Cited by 17
Informed consent, clinical research and the practice of medicine.
Tc, C. (1983) · PubMed · Cited by 12
Estimating the Normal Mean and Variance Under A Publication Selection Model
Hedges, L. V. (1989) · Springer eBooks · doi:10.1007/978-1-4612-3678-8_31 · Cited by 8
Sugita, M., Yamaguchi, N., Izuno, T., Kanamori, M., & Kasuga, H. (1994) · PubMed · Cited by 8
Meta-analysis: The fashion of summing-up evidence
Gelber, R. D., Coates, A. S., & Goldhirsch, A. (1992) · Annals of Oncology · doi:10.1093/oxfordjournals.annonc.a058318 · Cited by 4
Explaining heterogeneity in meta-analysis: a comparison of methods
Thompson, S. G., & Sharp, S. J. (1999) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19991030)18:20
Research works printed but not linked: 8
- Chalmers TC., 1982, Transactions of the American Clinical Climatology Association, 94, 204
- 10.2307/1164832 · DOI · Paper page
- Champney TF. Adjustments for selection: publication bias in quantitative research synthesis. Doctoral thesis, University of Chicago, 1983.
- Hedges LV., 1998, Statistical analysis of medical data
- Wasserman L., 1998, Bayesian statistics 4, 483
- Eberly LE, 1996, Estimating the number of unseen studies
- Sugita M, 1992, Japnese Journal of Clinical Oncology, 22, 354
- Sugita M, 1994, Tokai Journal of Experimental Clinical Medicine, 19, 29
Cited by
Sorted by citations
A meta-analytic test of intergroup contact theory.
Pettigrew, T. F., & Tropp, L. R. (2006) · Journal of personality and social psychology · doi:10.1037/0022-3514.90.5.751 · Cited by 9,708
The Handbook of Research Synthesis and Meta-Analysis
(2019) · doi:10.7758/9781610448864 · Cited by 4,553
Younossi, Z. M., Golabi, P., Paik, J. M., Henry, A., Van Dongen, C., & Henry, L. (2023) · Hepatology · doi:10.1097/hep.0000000000000004 · Cited by 3,383
Eagly, A. H., Johannesen‐Schmidt, M. C., & Engen, M. L. V. (2003) · Psychological Bulletin · doi:10.1037/0033-2909.129.4.569 · Cited by 2,783
Younossi, Z. M., Golabi, P., de Avila, L., Paik, J. M., Srishord, M., Fukui, N., Qiu, Y., Burns, L., Afendy, A., & Nader, F. (2019) · Journal of Hepatology · doi:10.1016/j.jhep.2019.06.021 · Cited by 2,235
Metadata
- Authors
- Sutton, A. J., Song, F., Gilbody, S. M., & Abrams, K. R.
- Affiliations
- Department of Epidemiology and Public Health, University of Leicester, Leicester, UK,
- University of York (GB)
- University of Leicester (GB)
- Venue
- Statistical Methods in Medical Research
- Publisher
- SAGE Publications
- Published
- 2000-10
- Volume
- 9
- Issue
- 5
- Pages
- 421-445
- ISSN
- 0962-2802 (ISSN-L)
- 1477-0334
- OpenAlex
- W2009813561
- Cited by
- 288
- References
- 63
- Citations by year
- 2026: 6 · 2025: 8 · 2024: 10 · 2023: 27 · 2022: 15 · 2021: 10 · 2020: 11 · 2019: 12 · 2018: 14 · 2017: 25 · 2016: 10 · 2015: 15 · 2014: 14 · 2013: 12 · 2012: 11
- FWCI
- 11.45
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Statistical Methods in Medical Research (publishedVersion)
- PubMed (publishedVersion)
- Leicester Research Archive (University of Leicester) (submittedVersion)
- UEA Digital Repository (University of East Anglia) (submittedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Keywords
- publication bias, meta-analysis, sensitivity analysis, bias correction, statistical significance, negative results, research validity
- MeSH terms
- Bayes Theorem, Data Interpretation, Statistical, United Kingdom, Meta-Analysis as Topic, Models, Statistical, Publication Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review