Publication and related bias in meta-analysis
The DOI matches its Crossref record, with a small difference in title or year; 67 of the 67 references it cites were found in a registry or an index.
Abstract
Publication and selection biases in meta-analysis are more likely to affect small studies, which also tend to be of lower methodological quality. This may lead to "small-study effects," where the smaller studies in a meta-analysis show larger treatment effects. Small-study effects may also arise because of between-trial heterogeneity. Statistical tests for small-study effects have been proposed, but their validity has been questioned. A set of typical meta-analyses containing 5, 10, 20, and 30 trials was defined based on the characteristics of 78 published meta-analyses identified in a hand search of eight journals from 1993 to 1997. Simulations were performed to assess the power of a weighted regression method and a rank correlation test in the presence of no bias, moderate bias or severe bias. We based evidence of small-study effects on P < 0.1. The power to detect bias increased with increasing numbers of trials. The rank correlation test was less powerful than the regression method. For example, assuming a control group event rate of 20% and no treatment effect, moderate bias was detected with the regression test in 13.7%, 23.5%, 40.1% and 51.6% of meta-analyses with 5, 10, 20 and 30 trials. The corresponding figures for the correlation test were 8.5%, 14.7%, 20.4% and 26.0%, respectively. Severe bias was detected with the regression method in 23.5%, 56.1%, 88.3% and 95.9% of meta-analyses with 5, 10, 20 and 30 trials, as compared to 11.9%, 31.1%, 45.3% and 65.4% with the…
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Second, this meta-analysis focused on inactive control conditions and the quality of many studies included in the meta-analysis was variable, which may produce an inflated estimate for the efficacy of these interventions (Cuijpers et al., 2010b; Sterne et al., 2000; Thornton and Lee, 2000).”
From Meta-analytic review of online guided self-help interventions for depressive symptoms among college students (2021) · cites this paper as Sterne et al., 2000
In the citing paper
“If there is a minimum of 10 studies in any meta-analysis, funnel plot asymmetry will be examined for sources of possible bias (eg, publication bias, language bias, inclusion of small studies with poor methodological quality and heterogeneity).99 100”
In the citing paper
“Moreover, many VR trials are conducted on a small number of participants, which exposes meta-analyses to “small study effects”10, the notion that smaller studies show different, often larger, treatment effects than large ones.”
From The effectiveness of virtual reality based interventions for symptoms of anxiety and depression: A meta-analysis (2018) · cites this paper as [10]
In the citing paper
“Asymmetry of the funnel plot means that there is an association between study size and effect estimates or a ‘small study effect’ [76], with smaller studies typically showing larger effects.”
From COSMOS-E: Guidance on conducting systematic reviews and meta-analyses of observational studies of etiology (2019) · cites this paper as [76]
In the citing paper
“However, the rank correlation test may only have moderate power for small meta-analyses (Begg & Mazumdar, 1994; Sterne, Gavaghan, & Egger, 2000).”
From A comprehensive meta-analysis on Problematic Facebook Use (2018) · cites this paper as Sterne, Gavaghan, & Egger, 2000
2 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Bias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722
Self-citationOperating characteristics of a rank correlation test for publication bias
Begg, C. B., & Mazumdar, M. (1994) · Biometrics · doi:10.2307/2533446 · Cited by 16,955
Schulz (1995) · doi:10.1001/jama.1995.03520290060030 · Cited by 5,634
Moher, D., Pham, B., Jones, A., Cook, D. J., Jadad, A. R., Moher, M., Tugwell, P., & Klassen, T. P. (1998) · The Lancet · doi:10.1016/s0140-6736(98)01085-x · Cited by 3,560
Altman, R., Carreras, L., Díaz, R. S., Figueroa, E. B., Paolasso, E., Parodi, J. C., Cade, J. F., Donnan, G. A., Eadie, M. J., Gavaghan, T. P., et al. (1994) · Cited by 3,322
Publication bias in clinical research
Easterbrook, P. J., Gopalan, R., Berlin, J. A., & Matthews, D. R. (1991) · The Lancet · doi:10.1016/0140-6736(91)90201-y · Cited by 3,109
Yusuf, S., Zucker, D., Passamani, E., Peduzzi, P., Takaro, T., Fisher, L. D., Kennedy, J. W., Davis, K., Killip, T., Norris, R., et al. (1994) · The Lancet · doi:10.1016/s0140-6736(94)91963-1 · Cited by 2,134
The Hazards of Scoring the Quality of Clinical Trials for Meta-analysis
Jüni, P. (1999) · JAMA · doi:10.1001/jama.282.11.1054 · Cited by 1,981
Comprehensive geriatric assessment: a meta-analysis of controlled trials
Stuck, A. E., Siu, A. L., Wieland, G. D., Rubenstein, L. Z., & Adams, J. (1993) · The Lancet · doi:10.1016/0140-6736(93)92884-v · Cited by 1,974
Garg, R. (1995) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.1995.03520420066040 · Cited by 1,771
Research works printed but not linked: 4
- Irwig, 1998, Graphical test is itself biased, Br Med J, 316, 470
- Egger, 1998, Bias in meta-analysis detected by a simple, graphical test—authors reply, Br Med J, 316, 470
- Thompson, 1999, Explaining heterogeneity in meta-analysis, Stat Med, 18, 2693, 10.1002/(SICI)1097-0258(19991030)18:20<2693::AID-SIM235>3.0.CO;2-V · doi:10.1002/(sici)1097-0258(19991030)18:20<2693::aid-sim235>3.0.co;2-v
- Steichen, 1998, sbe19.1, Stata Technical Bulletin, 44, 3
Cited by
Sterne, J. A. C., Sutton, A. J., Ioannidis, J. P. A., Terrin, N., Jones, D. R., Lau, J., Carpenter, J., Rucker, G., Harbord, R. M., Schmid, C. H., et al. (2011) · BMJ · doi:10.1136/bmj.d4002 · Cited by 7,709
Self-citationHow to perform a meta-analysis with R: a practical tutorial
Balduzzi, S., Rücker, G., & Schwarzer, G. (2019) · Evidence Based Mental Health · doi:10.1136/ebmental-2019-300117 · Cited by 5,827
The Handbook of Research Synthesis and Meta-Analysis
(2019) · doi:10.7758/9781610448864 · Cited by 4,553
Development of AMSTAR: A measurement tool to assess the methodological quality of systematic reviews
Shea, B. J., Grimshaw, J. M., Wells, G. A., Boers, M., Andersson, N., Hamel, C., Porter, A. C., Tugwell, P., Moher, D., & Bouter, L. M. (2007) · BMC Medical Research Methodology · doi:10.1186/1471-2288-7-10 · Cited by 4,374
Funnel plots for detecting bias in meta-analysis
Sterne, J. A. C., & Egger, M. (2001) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(01)00377-8 · Cited by 3,779
“We have shown elsewhere [8] that this regression test for funnel plot asymmetry is more powerful than the rank correlation method proposed by Begg and Mazumdar [17].”
BackgroundMethodResultInfluentialSelf-citation
Metadata
- Authors
- Sterne, J. A. C., Gavaghan, D., & Egger, M.
- Affiliations
- University of Bristol (GB)
- Corresponding author
- Egger, Matthias
- Venue
- Journal of Clinical Epidemiology
- Publisher
- Elsevier BV
- Published
- 2000-11
- Volume
- 53
- Issue
- 11
- Pages
- 1119-1129
- ISSN
- 0895-4356 (ISSN-L)
- OpenAlex
- W204046373
- Cited by
- 2,282
- References
- 67
- Citations by year
- 2026: 114 · 2025: 157 · 2024: 178 · 2023: 145 · 2022: 164 · 2021: 179 · 2020: 124 · 2019: 131 · 2018: 115 · 2017: 124 · 2016: 101 · 2015: 102 · 2014: 102 · 2013: 88 · 2012: 82
- FWCI
- 26.20
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
- Locations
- Journal of Clinical Epidemiology (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Keywords
- publication bias, selection bias, small-study effects, statistical power, false positive rate, between-trial heterogeneity
- MeSH terms
- Clinical Trials as Topic, Regression Analysis, Statistics as Topic, Meta-Analysis as Topic, Reproducibility of Results, Bias, Publication Bias, Statistics, Nonparametric
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Publication and related bias in meta-analysis. https://app.cochat.ai/papers/10.1016/s0895-4356(00)00242-0
- The paper
- Sterne, J. A. C., Gavaghan, D., & Egger, M. (2000). Publication and related bias in meta-analysis. Journal of Clinical Epidemiology. https://doi.org/10.1016/s0895-4356(00)00242-0
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.