The DOI matches its Crossref record; 44 of the 45 references it cites were found in a registry or an index, and 1 are not checked yet.
Assessment of regression-based methods to adjust for publication bias through a comprehensive simulation study
Open accessAbstract
BACKGROUND: In meta-analysis, the presence of funnel plot asymmetry is attributed to publication or other small-study effects, which causes larger effects to be observed in the smaller studies. This issue potentially mean inappropriate conclusions are drawn from a meta-analysis. If meta-analysis is to be used to inform decision-making, a reliable way to adjust pooled estimates for potential funnel plot asymmetry is required. METHODS: A comprehensive simulation study is presented to assess the performance of different adjustment methods including the novel application of several regression-based methods (which are commonly applied to detect publication bias rather than adjust for it) and the popular Trim & Fill algorithm. Meta-analyses with binary outcomes, analysed on the log odds ratio scale, were simulated by considering scenarios with and without i) publication bias and; ii) heterogeneity. Publication bias was induced through two underlying mechanisms assuming the probability of publication depends on i) the study effect size; or ii) the p-value. RESULTS: The performance of all methods tended to worsen as unexplained heterogeneity increased and the number of studies in the meta-analysis decreased. Applying the methods conditional on an initial test for the presence of funnel plot asymmetry generally provided poorer performance than the unconditional use of the adjustment method. Several of the regression based methods consistently outperformed the Trim & Fill estimators. CONCLUSION: Regression-based adjustments for publication bias and other small study effects are easy to conduct and outperformed more established methods over a wide range of simulation scenarios.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, more novel and promising methods, such as p-curve (Simonsohn, Nelson, & Simmons, 2014) and variations of Egger's regression (Egger, Smith, Schneider, & Minder, 1997;Moreno et al., 2009), have been found to perform better than the trim and fill method.”
From Working memory training revisited: A multi-level meta-analysis of n-back training studies (2017) · cites this paper as Moreno et al., 2009
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Bias adjustments can improve performance relative to a meta-analysis that does not adjust for publication bias (Carter et al., 2019; McShane et al., 2016; Moreno et al., 2009; Simonsohn et al., 2014).”
From Do Salient Social Norms Moderate Mortality Salience Effects? A (Challenging) Meta-Analysis of Terror Management Studies (2023) · cites this paper as Moreno et al., 2009
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Bias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722
Quantifying heterogeneity in a meta‐analysis
Higgins, J. P. T., & Thompson, S. G. (2002) · Statistics in Medicine · doi:10.1002/sim.1186 · Cited by 36,763
(2005) · Springer eBooks · doi:10.1007/0-387-27255-0_9 · Cited by 20,333
McCullagh, P., & Nelder, J. A. (1989) · doi:10.1007/978-1-4899-3242-6 · Cited by 16,836
Duval, S., & Tweedie, R. (2000) · Biometrics · doi:10.1111/j.0006-341x.2000.00455.x · Cited by 14,099
Funnel plots for detecting bias in meta-analysis
Sterne, J. A. C., & Egger, M. (2001) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(01)00377-8 · Cited by 3,779
A Nonparametric “Trim and Fill” Method of Accounting for Publication Bias in Meta-Analysis
Duval, S., & Tweedie, R. (2000) · Journal of the American Statistical Association · doi:10.1080/01621459.2000.10473905 · Cited by 2,955
Selective publication of antidepressant trials and its influence on apparent efficacy
Turner, E. H., Matthews, A. M., Linardatos, E., Tell, R. A., & Rosenthal, R. (2008) · New England Journal of Medicine · doi:10.1056/nejmsa065779 · Cited by 2,435
Publication Bias in Meta‐Analysis: Prevention, Assessment and Adjustments
(2005) · doi:10.1002/0470870168 · Cited by 2,368
Publication and related bias in meta-analysis
Sterne, J. A. C., Gavaghan, D., & Egger, M. (2000) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(00)00242-0 · Cited by 2,282
Comparison of two methods to detect publication bias in meta-analysis.
Peters, J. L. (2006) · JAMA · doi:10.1001/jama.295.6.676 · Cited by 2,278
Self-citationA modified test for small‐study effects in meta‐analyses of controlled trials with binary endpoints
Harbord, R. M., Egger, M., & Sterne, J. A. C. (2006) · Statistics in Medicine · doi:10.1002/sim.2380 · Cited by 2,267
Combination of direct and indirect evidence in mixed treatment comparisons
Lu, G., & Ades, A. E. (2004) · Statistics in Medicine · doi:10.1002/sim.1875 · Cited by 2,172
Self-citationPeters, J. L., Sutton, A. J., Jones, D. R., Abrams, K. R., & Rushton, L. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.11.010 · Cited by 2,053
Self-citation(2000) · Queen Mary Research Online (Queen Mary University of London) · Cited by 1,795
Self-citationThe case of the misleading funnel plot
Lau, J., Ioannidis, J. P. A., Terrin, N., Schmid, C. H., & Olkin, I. (2006) · BMJ · doi:10.1136/bmj.333.7568.597 · Cited by 1,766
Summing up: The Science of Reviewing Research
Wilkinson, I., Light, R. J., & Pillemer, D. B. (1987) · Journal of Educational Statistics · doi:10.2307/1164691 · Cited by 1,486
Light, R. J., & Pillemer, D. B. (1984) · Harvard University Press eBooks · doi:10.4159/9780674040243 · Cited by 1,422
Publication bias in meta-analysis its causes and consequences
Thornton, A. (2000) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(99)00161-4 · Cited by 1,291
A comparison of methods to detect publication bias in meta‐analysis
Macaskill, P., Walter, S. D., & Irwig, L. (2001) · Statistics in Medicine · doi:10.1002/sim.698 · Cited by 1,286
The appropriateness of asymmetry tests for publication bias in meta-analyses: A large survey
Ioannidis, J. P. A., & Trikalinos, T. A. (2007) · Canadian Medical Association Journal · doi:10.1503/cmaj.060410 · Cited by 1,052
Publication bias—a problem in interpreting medical data
Begg, C. B., & Berlin, J. A. (1988) · Journal of the Royal Statistical Society. Series A (Statistics in Society) · doi:10.2307/2982993 · Cited by 814
Adjusting for publication bias in the presence of heterogeneity
Terrin, N., Schmid, C. H., Lau, J., & Olkin, I. (2003) · Statistics in Medicine · doi:10.1002/sim.1461 · Cited by 809
The design of simulation studies in medical statistics
Burton, A., Altman, D. G., Royston, P. J., & HOLDER, R. L. (2006) · Statistics in Medicine · doi:10.1002/sim.2673 · Cited by 755
Peters, J. L., Sutton, A. J., Jones, D. R., Abrams, K. R., & Rushton, L. (2007) · Statistics in Medicine · doi:10.1002/sim.2889 · Cited by 716
Self-citationA Nonparametric "Trim and Fill" Method of Accounting for Publication Bias in Meta-Analysis
Duval, S., & Tweedie, R. L. (2000) · Journal of the American Statistical Association · doi:10.2307/2669529 · Cited by 694
Stanley, T. D. (2008) · Oxford Bulletin of Economics and Statistics · doi:10.1111/j.1468-0084.2007.00487.x · Cited by 605
Self-citationA LIKELIHOOD APPROACH TO META-ANALYSIS WITH RANDOM EFFECTS
Hardy, R., & Thompson, S. G. (1996) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19960330)15:6 · Cited by 522
Egger, M., & Smith, G. D. (1995) · BMJ · doi:10.1136/bmj.310.6982.752 · Cited by 508
Publication Bias in Research Synthesis: Sensitivity Analysis Using A Priori Weight Functions.
Vevea, J. L., & Woods, C. M. (2005) · Psychological Methods · doi:10.1037/1082-989x.10.4.428 · Cited by 461
Arcsine test for publication bias in meta‐analyses with binary outcomes
Rücker, G., Schwarzer, G., & Carpenter, J. R. (2007) · Statistics in Medicine · doi:10.1002/sim.2971 · Cited by 442
Hedges, L., & Vevea, J. L. (1996) · Journal of Educational and Behavioral Statistics · doi:10.3102/10769986021004299 · Cited by 223
Meta-Analysis in Medicine and Health Policy
Stangl, D. K. (2000) · CRC Press eBooks · doi:10.1201/9780203909935 · Cited by 213
Publication bias and meta-analysis
Smith, M. L. (1980) · Evaluation in Education · doi:10.1016/0191-765x(80)90004-x · Cited by 209
What Works?: Selectivity Models and Meta-Analysis
Copas, J. B. (1999) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/1467-985x.00123 · Cited by 168
Tests for publication bias in meta-analysis
Steichen, T. J. (1998) · Stata technical bulletin · Cited by 103
Schwarzer, G., Antes, G., & Schumacher, M. C. (2002) · Statistics in Medicine · doi:10.1002/sim.1224 · Cited by 81
The implications of publication bias for meta‐analysis' other parameter
Jackson, D. (2006) · Statistics in Medicine · doi:10.1002/sim.2293 · Cited by 65
Adjusting for publication bias: modelling the selection process.
Preston, C., Ashby, D., & Smyth, R. (2004) · Journal of Evaluation In Clinical Practice · doi:10.1111/j.1365-2753.2003.00457.x · Cited by 40
A robust P‐value for treatment effect in meta‐analysis with publication bias
Copas, J., & Malley, P. F. (2008) · Statistics in Medicine · doi:10.1002/sim.3284 · Cited by 39
Dealing with Non-Ignorable Non-Response by Using an ‘Enthusiasm-to-Respond’ Variable
Copas, A. J., & Farewell, V. T. (1998) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/1467-985x.00115 · Cited by 28
Using Journal Impact Factors to Correct for the Publication Bias of Medical Studies
Baker, R., & Jackson, D. (2006) · Biometrics · doi:10.1111/j.1541-0420.2005.00513.x · Cited by 25
Trastuzumab: possible publication bias
Metcalfe, S., Burgess, C. D., Laking, G. R., Evans, J., Wells, S. J., & Crausaz, S. (2008) · The Lancet · doi:10.1016/s0140-6736(08)60706-0 · Cited by 9
Generalised synthesis methods in human health risk assessment
Peters, J. (2006) · Figshare · Cited by 1
Self-citationExplaining heterogeneity in meta-analysis: a comparison of methods
Thompson, S. G., & Sharp, S. J. (1999) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19991030)18:20
Flagged first, then by citations
Research works printed but not linked: 2
- Peters JL, Sutton AJ, Jones DR, Abrams KR: Performance of Tests and Adjustments for Publication Bias in the Presence of Heterogeneity. Technical report 05-01. 2005, Department of Health Sciences, University of Leicester
- Stata: Stata Statistical Software: Release 9.2. College Station, Texas. 2008
Cited by
Sorted by citations
Conducting Meta-Analyses in R with the metafor Package
Viechtbauer, W. (2010) · Journal of Statistical Software · doi:10.18637/jss.v036.i03 · Cited by 20,464
Sterne, J. A. C., Sutton, A. J., Ioannidis, J. P. A., Terrin, N., Jones, D. R., Lau, J., Carpenter, J., Rucker, G., Harbord, R. M., Schmid, C. H., et al. (2011) · BMJ · doi:10.1136/bmj.d4002 · Cited by 7,709
Self-citationLeucht, S., Cipriani, A., Spineli, L., Mavridis, D., Örey, D., Richter, F., Samara, M., Barbui, C., Engel, R. R., Geddes, J. R., et al. (2013) · The Lancet · doi:10.1016/s0140-6736(13)60733-3 · Cited by 2,561
An opportunity cost model of subjective effort and task performance
Kurzban, R., Duckworth, A., Kable, J. W., & Myers, J. (2013) · Behavioral and Brain Sciences · doi:10.1017/s0140525x12003196 · Cited by 1,381
Dissemination and publication of research findings: an updated review of related biases
Song, F., Parekh, Hooper, L., Loke, Y., Ryder, J., Sutton, A., Hing, C., Kwok, C., Pang, C., & Harvey, I. (2010) · Health Technology Assessment · doi:10.3310/hta14080 · Cited by 1,039
Self-citation
Metadata
- Authors
- Moreno, S. G., Sutton, A. J., Ades, A., Stanley, T. D., Abrams, K. R., Peters, J. L., & Cooper, N. J.
- Affiliations
- University of Leicester (GB)
- Hendrix College (US)
- Corresponding author
- Moreno, Santiago G
- Venue
- BMC Medical Research Methodology
- Published
- 2009-01-12
- Volume
- 9
- Issue
- 1
- Pages
- 2-2
- ISSN
- 1471-2288 (ISSN-L)
- PMC
- PMC2649158
- OpenAlex
- W1970128129
- Cited by
- 373
- References
- 44
- Citations by year
- 2026: 9 · 2025: 11 · 2024: 11 · 2023: 11 · 2022: 31 · 2021: 28 · 2020: 26 · 2019: 21 · 2018: 39 · 2017: 31 · 2016: 18 · 2015: 26 · 2014: 24 · 2013: 26 · 2012: 19
- FWCI
- 24.04
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- BMC Medical Research Methodology (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Own your potential (DEAKIN) (submittedVersion)
- Figshare (submittedVersion, cc-by)
- Leicester Research Archive (University of Leicester) (submittedVersion, cc-by)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Keywords
- publication bias, small-study effects, trim and fill, funnel plot asymmetry, meta-analysis, heterogeneity, binary outcomes, odds ratio, simulation study
- MeSH terms
- Computer Simulation, Humans, Regression Analysis, Meta-Analysis as Topic, Odds Ratio, Randomized Controlled Trials as Topic, Publication Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
