Detecting and adjusting for small‐study effects in meta‐analysis
The DOI matches its Crossref record; 77 of the 77 references it cites were found in a registry or an index.
Abstract
Publication bias and related types of small‐study effects threaten the validity of systematic reviews. The existence of small‐study effects has been demonstrated in empirical studies. Small‐study effects are graphically diagnosed by inspection of the funnel plot. Though observed funnel plot asymmetry cannot be easily linked to a specific reason, tests based on funnel plot asymmetry have been proposed. Beyond a vast range of funnel plot tests, there exist several methods for adjusting treatment effect estimates for these biases. In this article, we consider the trim‐and‐fill method, the Copas selection model, and more recent regression‐based approaches. The methods are exemplified using a meta‐analysis from the literature and compared in a simulation study, based on binary response data. They are also applied to a large set of meta‐analyses. Some fundamental differences between the approaches are discussed. An assumption common to the trim‐and‐fill method and the Copas selection model is that the small‐study effect is caused by selection. The trim‐and‐fill method corresponds to an unknown implicit model generated by the symmetry assumption, whereas the Copas selection model is a parametric statistical model. However, it requires a sensitivity analysis. Regression‐based approaches are easier to implement and not based on a specific selection model. Both simulations and applications suggest that in the presence of strong selection both the trim‐and‐fill method and the Copas selection model may not fully eliminate bias, while regression‐based approaches seem to be a promising alternative.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“We checked the model for possible small-study effects by visually examining a funnel plot (20), supplemented by Egger's regression-based test for funnel plot asymmetry (21).”
From Referral of patients to psychiatric emergency departments by police: A systematic review and meta-analysis (2025) · cites this paper as [20]
In the citing paper
“Previous summary work on liver enzymes and inter-hepatic fat, in people with non-alcoholic fatty liver disease, has shown that an aggregate exercise program energy expenditure of around 10,000 kcal is required to elicit improvement [37], and this may take several weeks to achieve.”
From The Effect of Exercise Training on Blood Lipids: A Systematic Review and Meta-analysis (2025) · cites this paper as [37]
Citations
Cites
Bias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722
Meta-analysis in clinical trials
DerSimonian, R., & Laird, N. (1986) · Controlled Clinical Trials · doi:10.1016/0197-2456(86)90046-2 · Cited by 39,552 · cited 3×
“Therefore, our aim was (i) to extend the established random effects model (DerSimonian and Laird, 1986) by a parameter that represents small-study effects in a natural way and (ii) to use a shrinkage procedure to improve the properties of parameter estimation (Rücker, 2010; Rücker et al.”
MethodCochrane Handbook for Systematic Reviews of Interventions: Cochrane Book Series
(2008) · doi:10.1002/9780470712184 · Cited by 25,048
Operating characteristics of a rank correlation test for publication bias
Begg, C. B., & Mazumdar, M. (1994) · Biometrics · doi:10.2307/2533446 · Cited by 16,955 · cited 1×
“Broadly, tests fall into two classes: (i) non-parametric tests using rank-correlation methods, going back to Begg and Mazumdar (1994) and (ii) regression tests, represented by a test proposed by Egger et al. (1997a).”
MethodMcCullagh, P., & Nelder, J. A. (1989) · doi:10.1007/978-1-4899-3242-6 · Cited by 16,836
Duval, S., & Tweedie, R. (2000) · Biometrics · doi:10.1111/j.0006-341x.2000.00455.x · Cited by 14,099
Measurement in Medicine: The Analysis of Method Comparison Studies
Altman, D. G., & Bland, J. M. (1983) · The Statistician · doi:10.2307/2987937 · Cited by 4,300 · cited 2×
“Figure 5 (top left panel) shows a scatterplot of b̂0 against b̂limðr2 ¼ 0:9006Þ: Bland–Altman plots (not shown; Altman and Bland, 1983) give no hints of systematic trends in the difference.”
BackgroundFunnel plots for detecting bias in meta-analysis
Sterne, J. A. C., & Egger, M. (2001) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(01)00377-8 · Cited by 3,779 · cited 1×
“This is a scatter plot showing the treatment effect (usually on the x-axis) depending on the precision of the study, preferably represented by the standard error on a reverse scale (Sterne and Egger, 2001).”
Publication bias in clinical research
Easterbrook, P. J., Gopalan, R., Berlin, J. A., & Matthews, D. R. (1991) · The Lancet · doi:10.1016/0140-6736(91)90201-y · Cited by 3,109 · cited 1×
“The first form of this effect that was acknowledged was publication bias (Sterling, 1959; Simes, 1986; Begg and Berlin, 1988; Berlin et al., 1989; Rust et al., 1990; Sterling et al., 1995; Dickersin, 1990; Easterbrook et al., 1991; Sutton et al., 2000a).”
BackgroundHow should meta‐regression analyses be undertaken and interpreted?
Thompson, S. G., & Higgins, J. P. T. (2002) · Statistics in Medicine · doi:10.1002/sim.1187 · Cited by 3,026 · cited 1×
“If study-level covariates are available that describe the baseline risk of patients and may explain heterogeneity, they can be included in a meta-regression model (Sharp and Thompson, 2000; Thompson and Higgins, 2002).”
Background
Research works printed but not linked: 1
- Rücker G.(2010). Small‐study effects and heterogeneity in meta‐analysis. Ph.D. Thesis.http://www.freidok.uni‐freiburg.de/volltexte/7486/
Cited by
How to perform a meta-analysis with R: a practical tutorial
Balduzzi, S., Rücker, G., & Schwarzer, G. (2019) · Evidence Based Mental Health · doi:10.1136/ebmental-2019-300117 · Cited by 5,827
“Different methods exist 22 ; in this paper, we report the more established trim-and-fill method 24 and the method of adjusting by regression.”
Self-citationA new improved graphical and quantitative method for detecting bias in meta-analysis
Furuya-Kanamori, L., Barendregt, J. J., & Doi, S. A. R. (2018) · International Journal of Evidence-Based Healthcare · doi:10.1097/xeb.0000000000000141 · Cited by 1,160
A meta-analysis of procedures to change implicit measures
Forscher, P. S., Lai, C. K., Axt, J. R., Ebersole, C. R., Herman, M., Devine, P. G., & Nosek, B. A. (2019) · Journal of Personality and Social Psychology · doi:10.1037/pspa0000160 · Cited by 700
The impact of study size on meta-analyses: examination of underpowered studies in Cochrane reviews
Turner, R. M., Bird, S. M., & Higgins, J. P. T. (2013) · PLoS ONE · doi:10.1371/journal.pone.0059202 · Cited by 672
Correcting for Bias in Psychology: A Comparison of Meta-Analytic Methods
Carter, E. C., Schönbrodt, F. D., Gervais, W. M., & Hilgard, J. (2019) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245919847196 · Cited by 588
Influence of trial sample size on treatment effect estimates: meta-epidemiological study
Dechartres, A., Trinquart, L., Boutron, I., & Ravaud, P. (2013) · BMJ · doi:10.1136/bmj.f2304 · Cited by 544
Chaimani, A., & Salanti, G. (2012) · Research Synthesis Methods · doi:10.1002/jrsm.57 · Cited by 521
“Study size is often viewed as a single proxy for the quality of randomised trials, and differences between small and large studies can be due to clinical and methodological heterogeneity (Peters et al., 2010; Rucker et al., 2011).”
BackgroundPublication and other reporting biases in cognitive sciences: detection, prevalence, and prevention
Ioannidis, J. P. A., Munafò, M. R., Fusar-Poli, P., Nosek, B. A., & David, S. P. (2014) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2014.02.010 · Cited by 471
Assessing risk of bias due to missing results in a synthesis
Page, M. J., Higgins, J. P., & Sterne, J. A. (2019) · Cochrane Handbook for Systematic Reviews of Interventions · doi:10.1002/9781119536604.ch13 · Cited by 402
Carter, E. C., & McCullough, M. E. (2014) · Frontiers in Psychology · doi:10.3389/fpsyg.2014.00823 · Cited by 381
Page, M. J., Sterne, J. A. C., Higgins, J. P. T., & Egger, M. (2021) · Research Synthesis Methods · doi:10.1002/jrsm.1468 · Cited by 294
“However, the available evidence suggests that no single method outperforms others in all scenarios, and thus there is a danger that uncritical application can lead to inappropriate conclusions being drawn.(69,74,75) Vevea et al(71) and Marks-Anglin and Chen(45) summarize the advantages and disadvantages of different m…”
BackgroundTreatment-effect estimates adjusted for small-study effects via a limit meta-analysis
Rucker, G., Schwarzer, G., Carpenter, J. R., Binder, H., & Schumacher, M. (2011) · Biostatistics · doi:10.1093/biostatistics/kxq046 · Cited by 286
Self-citationAssociation Between Analytic Strategy and Estimates of Treatment Outcomes in Meta-analyses
Dechartres, A., Altman, D. G., Trinquart, L., Boutron, I., & Ravaud, P. (2014) · JAMA · doi:10.1001/jama.2014.8166 · Cited by 233
Health effects associated with smoking: a Burden of Proof study
Dai, X., Gil, G. F., Reitsma, M. B., Ahmad, N. S., Anderson, J. A., Bisignano, C., Carr, S., Feldman, R., Hay, S. I., He, J., et al. (2022) · Nature Medicine · doi:10.1038/s41591-022-01978-x · Cited by 224
Debray, T. P. A., Moons, K. G. M., & Riley, R. D. (2018) · Research Synthesis Methods · doi:10.1002/jrsm.1266 · Cited by 212
When our hands help us understand: A meta-analysis into the effects of gesture on comprehension.
Dargue, N., Sweller, N., & Jones, M. P. (2019) · Psychological Bulletin · doi:10.1037/bul0000202 · Cited by 202
“Given publication bias cannot adequately be measured through a single method, multiple approaches were used (Rücker, Carpenter, & Schwarzer, 2011).”
White, C. A., Uttl, B., & Holder, M. D. (2019) · PLOS ONE · doi:10.1371/journal.pone.0216588 · Cited by 200
Gouveia e Melo, R., Silva Duarte, G., Lopes, A., Alves, M., Caldeira, D., Fernandes e Fernandes, R., & Mendes Pedro, L. (2022) · Seminars in Thoracic and Cardiovascular Surgery · doi:10.1053/j.semtcvs.2021.02.029 · Cited by 171
A historical review of publication bias
Marks‐Anglin, A., & Chen, Y. (2020) · Research Synthesis Methods · doi:10.1002/jrsm.1452 · Cited by 165
“Evaluating the source of asymmetry is important for determining the most appropriate means of addressing it.(67,83) For example, in cases where clinical heterogeneity is the cause, a subgroup analysis may be more appropriate than correction, as a combined estimate is less clinically useful.”
BackgroundPandor, A., Gomersall, T., Stevens, J. W., Wang, J., Al-Mohammad, A., Bakhai, A., Cleland, J. G. F., Cowie, M. R., & Wong, R. (2013) · Heart · doi:10.1136/heartjnl-2013-303811 · Cited by 147
Adjustment for reporting bias in network meta-analysis of antidepressant trials
Trinquart, L., Chatellier, G., & Ravaud, P. (2012) · BMC Medical Research Methodology · doi:10.1186/1471-2288-12-150 · Cited by 122
“Numerous methods have been used as sensitivity analyses to assess the robustness of conventional MAs to publication bias and related small-study effects [9-20].”
MethodWu, Y., Wang, F., Fan, X., Bao, R., Bo, L., Li, J., & Deng, X. (2012) · Critical Care · doi:10.1186/cc11884 · Cited by 120
Sarri, G., Patorno, E., Yuan, H., Guo, J., Bennett, D., Wen, X., Zullo, A. R., Largent, J. A., Panaccio, M. P., Gokhale, M., et al. (2020) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2020-111493 · Cited by 136
Curcumin Supplementation and Human Disease: A Scoping Review of Clinical Trials
Panknin, T., Howe, C. L., Hauer, M., Bucchireddigari, B., Rossi, A., & Funk, J. L. (2023) · International Journal of Molecular Sciences · doi:10.3390/ijms24054476 · Cited by 133
Small-Study Effects in Meta-Analysis
Schwarzer, G., Carpenter, J. R., & Rücker, G. (2015) · Use R! · doi:10.1007/978-3-319-21416-0_5 · Cited by 145
Self-citationPublication Bias in Neuroimaging Research: Implications for Meta-Analyses
Jennings, R. G., & Van Horn, J. D. (2012) · Neuroinformatics · doi:10.1007/s12021-011-9125-y · Cited by 102
“Often these pressures may force researchers to publish statistically significant results as soon as they have them, despite their study having a low sample size (Rucker et al. 2011).”
BackgroundThe Effect of Exercise Training on Blood Lipids: A Systematic Review and Meta-analysis
Smart, N. A., Downes, D., van der Touw, T., Hada, S., Dieberg, G., Pearson, M. J., Wolden, M., King, N., & Goodman, S. P. J. (2025) · Sports Medicine · doi:10.1007/s40279-024-02115-z · Cited by 126
“Previous summary work on liver enzymes and inter-hepatic fat, in people with non-alcoholic fatty liver disease, has shown that an aggregate exercise program energy expenditure of around 10,000 kcal is required to elicit improvement [37], and this may take several weeks to achieve.”
Hague, B., Hall, J., & Kellett, S. (2016) · Journal of Behavioral Addictions · doi:10.1556/2006.5.2016.064 · Cited by 103
Moreno, S. G., Sutton, A. J., Thompson, J. R., Ades, A. E., Abrams, K. R., & Cooper, N. J. (2012) · Statistics in Medicine · doi:10.1002/sim.4488 · Cited by 86
Mavridis, D., Sutton, A., Cipriani, A., & Salanti, G. (2013) · Statistics in Medicine · doi:10.1002/sim.5494 · Cited by 87
Chaimani, A., Vasiliadis, H. S., Pandis, N., Schmid, C. H., Welton, N. J., & Salanti, G. (2013) · International Journal of Epidemiology · doi:10.1093/ije/dyt074 · Cited by 83
Gjerdevik, M., & Heuch, I. (2014) · BMC Medical Research Methodology · doi:10.1186/1471-2288-14-109 · Cited by 82
“[34] state that the trim-and-fill method, as well as the Copas selection model, may not fully eliminate bias.”
MethodHartung–Knapp method is not always conservative compared with fixed‐effect meta‐analysis
Wiksten, A., Rücker, G., & Schwarzer, G. (2016) · Statistics in Medicine · doi:10.1002/sim.6879 · Cited by 79
Self-citationFarraia, M., Paciência, I., Mendes, F. C., Rufo, J. C., Shamji, M. H., Agache, I. O., & Moreira, A. M. (2022) · Allergy · doi:10.1111/all.15295 · Cited by 116
Presenting simulation results in a nested loop plot
Rücker, G., & Schwarzer, G. (2014) · BMC Medical Research Methodology · doi:10.1186/1471-2288-14-129 · Cited by 70
“In both earlier articles [9,10], additional focus was on MSE and coverage of 95% confidence intervals.”
BackgroundMethodInfluentialSelf-citationCalorie delivery and clinical outcomes in the critically ill: a systematic review and meta-analysis
Parikh, H. G., Miller, A., Chapman, M., Moran, J. L., & Peake, S. L. (2016) · Critical Care and Resuscitation · doi:10.1016/s1441-2772(23)00927-4 · Cited by 69
Finell, J., Sammallahti, E., Korhonen, J., Eklöf, H., & Jonsson, B. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.798090 · Cited by 121
Meta-analysis of biomarkers for severe dengue infections
Soo, K.-M., Khalid, B., Ching, S. M., Tham, C. L., Basir, R., & Chee, H. (2017) · PeerJ · doi:10.7717/peerj.3589 · Cited by 71
Real-time fMRI neurofeedback as a new treatment for psychiatric disorders: A meta-analysisPreprint
Pindi, P., Houenou, J., Piguet, C., & Favre, P. (2021) · PsyArXiv (OSF Preprints) · doi:10.1016/j.pnpbp.2022.110605 · Cited by 67
Renkewitz, F., & Keiner, M. (2019) · Zeitschrift für Psychologie · doi:10.1027/2151-2604/a000386 · Cited by 55
Hypertonic saline (HS) for acute bronchiolitis: Systematic review and meta-analysis
Maguire, C., Cantrill, H., Hind, D., Bradburn, M. J., & Everard, M. L. (2015) · BMC Pulmonary Medicine · doi:10.1186/s12890-015-0140-x · Cited by 61
“…the single most precise trial (the trial with the narrowest confidence interval [65]); ii) the pooled average of largest 25 % of trials; iii) the beta0 limit-analysis proposed by Rücker et al. (2011) to control for small-study effects [76]; and iv) restricted to trials at low overall risk of bias.”
BackgroundProtein delivery and clinical outcomes in the critically ill: a systematic review and meta-analysis
Davies, M. L., Chapple, L. S., Chapman, M. J., Moran, J. L., & Peake, S. (2017) · Critical Care and Resuscitation · doi:10.1016/s1441-2772(23)00783-4 · Cited by 60
Meta-analysis and The Cochrane Collaboration: 20 years of the Cochrane Statistical Methods Group
McKenzie, J. E., Salanti, G., Lewis, S. C., & Altman, D. G. (2013) · Systematic Reviews · doi:10.1186/2046-4053-2-80 · Cited by 52
“Consequently, SMG members have published several methodological developments evaluating the possibility of small study effects [46-52], visualizing them [53] and accounting for them [54-58].”
Assessing Publication Bias: a 7-Step User’s Guide with Best-Practice Recommendations
Kepes, S., Wang, W., & Cortina, J. M. (2022) · Journal of Business and Psychology · doi:10.1007/s10869-022-09840-0 · Cited by 49
Brini, S., Brudasca, N. I., Hodkinson, A., Kaluzinska, K., Wach, A., Storman, D., Prokop-Dorner, A., Jemioło, P., & Bala, M. M. (2023) · Clinical Psychology Review · doi:10.1016/j.cpr.2022.102236 · Cited by 44
Schwab, S., Giuachin, K., & Held, L. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-045942 · Cited by 49
Khasapane, N. G., Byaruhanga, C., Thekisoe, O. M. M., Nkhebenyane, S. J., & Khumalo, Z. T. H. (2023) · BMC Veterinary Research · doi:10.1186/s12917-023-03673-6 · Cited by 46
Stileman, H. M., & Jones, C. A. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1248924 · Cited by 49
LFK index does not reliably detect small‐study effects in meta‐analysis: A simulation study
Schwarzer, G., Rücker, G., & Semaca, C. (2024) · Research Synthesis Methods · doi:10.1002/jrsm.1714 · Cited by 47
“32 A reliable assessment of the extent of asymmetry based on the value of the LFK index is fruitless in our view as long as the approximate distribution of the LFK index under the assumption of symmetry is unknown.”
Self-citationDuan, H., Xia, L., He, W., Lin, Y., Zhi-hui, L., & Lan, Q. (2019) · International Journal of Pediatric Otorhinolaryngology · doi:10.1016/j.ijporl.2019.01.011 · Cited by 48
Zhao, W., Ou, X., Liu, J., & Zhang, W. (2017) · Journal of Pain Research · doi:10.2147/jpr.s138212 · Cited by 39
“24,25 The basic information of the ten studies is presented in Table 1.”
A meta-analysis of entrepreneurial ecosystem elements and entrepreneurial activity
Queissner, M., Stolz, L., & Weiß, M. (2024) · Small Business Economics · doi:10.1007/s11187-024-00953-9 · Cited by 64
Raugh, I. M., Berglund, A. M., & Strauss, G. P. (2024) · Affective Science · doi:10.1007/s42761-024-00281-x · Cited by 49
“…rank correlation test (Begg & Mazumdar, 1994), trim-and-fill analysis (Duval & Tweedie, 2000), limit meta-analysis (Rücker, Carpenter et al., 2011; Rücker, Schwarzer et al., 2011), Rosenthal’s fail-safe N to reduce a significant effect to be nonsignificant, Orwin’s fail-safe N to reduce the…”
Chang, W., Peng, F., Meng, S., Xu, J., & Yang, Y. (2020) · BMC Immunology · doi:10.1186/s12865-020-0332-x · Cited by 36
“However, the sample size of the trials we enrolled in this study was relatively small (in most of studies the participants not exceeding 100), which could lead to type II error with elevated false positive [33].”
BackgroundCryoanalgesia for postsurgical pain relief in adults: A systematic review and meta-analysis
Park, R., Coomber, M., Gilron, I., & Shanthanna, H. (2021) · Annals of Medicine and Surgery · doi:10.1016/j.amsu.2021.102689 · Cited by 33
“Most studies had fewer than 50 participants per treatment arm, increasing the risk of small study bias [50].”
BackgroundMeta-analysis of aspirin use and risk of lung cancer shows notable results
Hochmuth, F., Jochem, M., & Schlattmann, P. (2015) · European Journal of Cancer Prevention · doi:10.1097/cej.0000000000000176 · Cited by 33
“Further investigation of publication bias was carried out using the limit meta-analysis method by Rücker et al. (2011a, 2011b) implemented in the R package metasens (Schwarzer et al., 2014).”
MethodAcuña, S. A., Zhang, W., Yoon, P. D., Ivanics, T., Zhu, M. P., Claasen, M. P. A. W., Zuckerman, J., Ghanekar, A., Cattral, M. S., & Sapisochín, G. (2022) · Transplantation · doi:10.1097/tp.0000000000004213 · Cited by 30
“The inclusion of small studies increases the risk of small-study effects, which may threaten the validity of meta-analyses because these small studies are more prone to show different, often larger, treatment effects than large studies and are at higher risk for outcome reporting bias.(79) Despite the high risk of any…”
Sun, C., Zhao, M., & Li, X. (2012) · Current Eye Research · doi:10.3109/02713683.2011.635401 · Cited by 32
Adam, G. P., Springs, S., Trikalinos, T. A., Williams, J. W., Eaton, J. L., von Isenburg, M., Gierisch, J. M., Wilson, L. M., Robinson, K. A., Viswanathan, M., et al. (2018) · Systematic Reviews · doi:10.1186/s13643-018-0726-5 · Cited by 27
Meta-analysis using Python: a hands-on tutorial
Masoumi, S., & Shahraz, S. (2022) · BMC Medical Research Methodology · doi:10.1186/s12874-022-01673-y · Cited by 25
“Clinical heterogeneity is another unchecked source of variability that might explain the treatment effect diversification [22].”
BackgroundBae, J. (2014) · Epidemiology and Health · doi:10.4178/epih/e2014019 · Cited by 32
Adams, D., Ambrose, K., Simpson, K., Malone, S. A., & Dargue, N. (2023) · Clinical Child and Family Psychology Review · doi:10.1007/s10567-023-00450-7 · Cited by 30
Increase value and reduce waste in research on psychological therapies
Cristea, I. A., & Naudet, F. (2019) · Behaviour Research and Therapy · doi:10.1016/j.brat.2019.103479 · Cited by 22
Risum, Ø., Sandven, I., Sundhagen, J. O., & Abdelnoor, M. (2020) · European Journal of Vascular and Endovascular Surgery · doi:10.1016/j.ejvs.2020.08.007 · Cited by 20
Linde, K., Schumann, I., Meißner, K., Jamil, S., Kriston, L., Rücker, G., Antes, G., & Schneider, A. (2011) · BMC Family Practice · doi:10.1186/1471-2296-12-127 · Cited by 21
Self-citationThe sphenoidal emissary foramina prevalence: a meta-analysis of 6,369 subjects
Piagkou, M., Kostares, M., Duparc, F., Papanagiotou, P., Politis, C., Tsakotos, G. A., Pantazis, N., & Natsis, K. (2022) · Surgical and Radiologic Anatomy · doi:10.1007/s00276-022-03051-1 · Cited by 22
A Systematic Literature Review of Complementary and Alternative Veterinary Medicine: Laser Therapy
Millis, D. L., & Bergh, A. (2023) · Animals · doi:10.3390/ani13040667 · Cited by 18
Clark, R. C., Reese, M., Attalla, P., Camacho, J. M., Hirpara, M., DeLong, M. R., & Reid, C. M. (2024) · Aesthetic Surgery Journal Open Forum · doi:10.1093/asjof/ojae066 · Cited by 18
SALVAGING PSYCHOTHERAPY RESEARCH: A MANIFESTO
Coyne, J. C., & Kok, R. N. (2014) · Journal of Evidence-Based Psychotherapies · Cited by 19
Shah, U. J., Nguyen, D. L., Karuppiah, N., Martin, J. E., & Sehmbi, H. (2021) · Regional Anesthesia & Pain Medicine · doi:10.1136/rapm-2020-102024 · Cited by 15
A test for reporting bias in trial networks: simulation and case studies
Trinquart, L., Ioannidis, J. P. A., Châtellier, G., & Ravaud, P. (2014) · BMC Medical Research Methodology · doi:10.1186/1471-2288-14-112 · Cited by 15
“Full list of author information is available at the end of the article Numerous statistical tests have been introduced to detect the presence of reporting bias in conventional meta-analyses [5-11].”
BackgroundRenkewitz, F., & Keiner, M. (2018) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/w94ep · Cited by 19
Meta-analysis II: Interpretation and Use of Outputs
Onitilo, A. A., Doi, S. A. R., & Barendregt, J. J. (2013) · Springer series on epidemiology and public health · doi:10.1007/978-3-642-37131-8_15 · Cited by 16
Lee, W., Pulbrook, M., Sheehan, C., Kochovska, S., Chang, S., Hosie, A., Lobb, E. A., Parker, D., Draper, B., Agar, M. R., et al. (2020) · Journal of Pain and Symptom Management · doi:10.1016/j.jpainsymman.2020.07.011 · Cited by 14
Lin, L., Gu, H.-Y., Luo, J., Wang, L., Zhang, C., Niu, Y., & Zuo, H. (2019) · Frontiers in Pharmacology · doi:10.3389/fphar.2019.01101 · Cited by 14
Sleep dysregulation in ADHD children: a systematic review and meta-analysis
Xian, P., Sheng, X., Liu, S., Liu, Z., & Guo, X. (2025) · Psychological Medicine · doi:10.1017/s0033291725102158 · Cited by 13
“To confirm the stability of the aforementioned analytical outcomes, consistent with Rücker, Carpenter, and Schwarzer (2011), numerous methods were employed to assess the data for publication bias and related small-study effects.”
Boström, A. F., Asplund, K., Bergh, A., & Hyytiäinen, H. K. (2022) · Animals · doi:10.3390/ani12223144 · Cited by 11
“detect and adjust for publication bias require that there are at least some larger studies to be used as a reference [49].”
Alcouffe, S., Maurice, J., Galy, N., & Gaté, L. (2019) · Comptabilité - Contrôle - Audit · doi:10.3917/cca.251.0133 · Cited by 14
Bae, J., & Shim, S. R. (2020) · Asian Pacific Journal of Cancer Prevention · doi:10.31557/apjcp.2020.21.9.2793 · Cited by 12
“Egger test for small-study effects was conducted to evaluate a publication bias (Rücker et al., 2011).”
MethodDiabetes History and Gastric Cancer Risk: Different Results by Types of Follow-Up Studies
Bae, J. (2022) · Asian Pacific Journal of Cancer Prevention · doi:10.31557/apjcp.2022.23.5.1523 · Cited by 10
“Asian Pacific Journal of Cancer Prevention, Vol 231526 al., 2012; Oberaigner et al., 2014; Sekikawa et al., 2014; Sakitani et al., 2015; Zheng et al., 2019; Li et al., 2020; Yang et al., 2020) and 9 prospective studies (Inoue et al., 2006; Khan et al., 2006; Lin et al., 2011; Nakamura et al., 2013;…”
BackgroundNeoh, M. J. Y., Carollo, A., Lee, A., & Esposito, G. (2023) · Royal Society Open Science · doi:10.1098/rsos.230677 · Cited by 17
“10 : 230677 articles in the cluster revolved around statistics [42,73,74], p -values [43,75] and positive results in science [7,69,76].”
BackgroundBowles, D., & Quinton, A. E. (2016) · Journal of medical imaging and radiation sciences · doi:10.1016/j.jmir.2016.06.005 · Cited by 10
Investigating and Dealing with Publication Bias and Other Reporting Biases
Page, M. J., Sterne, J. A. C., Higgins, J. P. T., & Egger, M. (2022) · Wiley eBooks · doi:10.1002/9781119099369.ch5 · Cited by 13
Bae, J. (2021) · World Journal of Diabetes · doi:10.4239/wjd.v12.i6.908 · Cited by 10
Bae, J. (2021) · Journal of Menopausal Medicine · doi:10.6118/jmm.21023 · Cited by 7
Holper, L. (2020) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2020.00633 · Cited by 7
“The present analysis therefore aimed to account for possible SSE by allowing placebo outcomes to depend on the standard error (24, 25).”
Yang, L., Zhang, Q., Wen, M., Li, S., Lu, A., Li, K., Peng, C., & Chen, J. (2025) · General Hospital Psychiatry · doi:10.1016/j.genhosppsych.2025.09.002 · Cited by 11
Preventing postpartum hemorrhage: A network meta-analysis on routes of administration of uterotonics
Papadopoulou, A., Tournas, G., Georgiopoulos, G. A., Antsaklis, P., Daskalakis, G., Coomarasamy, A., & Devall, A. (2024) · European Journal of Obstetrics & Gynecology and Reproductive Biology · doi:10.1016/j.ejogrb.2024.02.021 · Cited by 6
Villacura-Herrera, C., Pérez, J. M., Jones, P. B., & Núñez, D. (2024) · Psychiatry Research · doi:10.1016/j.psychres.2024.115988 · Cited by 7
Mediastinitis in open heart surgery: a systematic review and meta-analysis of risk factors
Abdelnoor, M., Sandven, I., Vengen, Ø., & Risnes, I. (2019) · Scandinavian Cardiovascular Journal · doi:10.1080/14017431.2019.1642508 · Cited by 5
“This increased infection risk in diabetic patients can be imputed in part to physiologic modifications precipitated by deficient glucose control as underlined by Talbot TR [89].”
BackgroundMethodAlmutairi, M. F., Gulden, S., Hundscheid, T. M., Bartos̆, F., Cavallaro, G., & Villamor, E. (2023) · Children · doi:10.3390/children10121903 · Cited by 4
Akwongo, C. J., & Byaruhanga, C. (2024) · Preventive Veterinary Medicine · doi:10.1016/j.prevetmed.2024.106214 · Cited by 4
Yu, T., Metelli, S., Papakonstantinou, T., & Chaimani, A. (2026) · Research Synthesis Methods · doi:10.1017/rsm.2026.10074 · Cited by 8
“38 Small-study effects can be visually assessed using funnel plots, where asymmetry may indicate the presence of small-study effects and requires further investigation.”
Brini, S., & Leung, T. I. (2026) · Journal of Medical Internet Research · doi:10.2196/92132 · Cited by 8
“Visually, this appears as an asymmetrical funnel plot, with a gap in one of the bottom corners [37,39].”
Hussey, I. (2025) · Journal of Behavior Therapy and Experimental Psychiatry · doi:10.1016/j.jbtep.2024.102015 · Cited by 3
Shahzil, M., Kashif, T., Jamil, Z., Khaqan, M. A., Munir, L., Amjad, Z., Faisal, M. S., Chaudhary, A. J., Ali, H., Khan, S., et al. (2025) · DEN Open · doi:10.1002/deo2.70128 · Cited by 3
“All statistical analyses adhered to the guidelines provided in the Cochrane Handbook for Systematic Reviews of Interventions and the methodological recommendations by Rücker et al. 15 Continuous and dichotomous data were combined following the appropriate formulas and guidance to ensure accurate synthesis of the evide…”
Shahzil, M., Hasan, F., Qureshi, A. A., Jamil, Z., Kashif, T., Faisal, M. S., Arif, T. B., Chaudhary, A. J., Farooq, U., Ali, H., et al. (2025) · Journal of Digestive Diseases · doi:10.1111/1751-2980.70008 · Cited by 3
Rix, A., Girbig, R. M., Porte, C., Lederle, W., Leenaars, C. H. C., & Kiessling, F. M. (2021) · Molecular Imaging and Biology · doi:10.1007/s11307-021-01686-x · Cited by 3
“Finally, the distribution of all effect sizes in a funnel plot can be used to identify the occurrence and degree of small-study effects [12].”
MethodK-M., S. (2022) · Tropical biomedicine · doi:10.47665/tb.39.1.015 · Cited by 3
Appraising Between-Study Homogeneity, Small-Study Effects, Moderators, and Confounders
Veroniki, A., Huedo-Medina, T. B., & Fountoulakis, K. (2016) · Springer eBooks · doi:10.1007/978-3-319-25655-9_12 · Cited by 3
Metadata
- Authors
- Rücker, G., Carpenter, J. R., & Schwarzer, G.
- Affiliations
- University of Freiburg (DE)
- Corresponding author
- Rücker, Gerta
- Venue
- Biometrical Journal
- Publisher
- Wiley
- Published
- 2011-01-14
- Volume
- 53
- Issue
- 2
- Pages
- 351-368
- ISSN
- 0323-3847 (ISSN-L)
- 1521-4036
- OpenAlex
- W2069224123
- Cited by
- 132
- References
- 76
- Citations by year
- 2026: 11 · 2025: 14 · 2024: 9 · 2023: 8 · 2022: 16 · 2021: 9 · 2020: 10 · 2019: 13 · 2018: 5 · 2017: 5 · 2016: 5 · 2015: 3 · 2014: 8 · 2013: 7 · 2012: 6
- FWCI
- 7.83
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Biometrical Journal (publishedVersion)
- PubMed (publishedVersion)
- LSHTM Research Online (London School of Hygiene and Tropical Medicine) (submittedVersion)
- Funders
- Deutsche Forschungsgemeinschaft
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Keywords
- small-study effects, publication bias, funnel plot asymmetry, selection bias, binary response data, meta-analysis bias
- MeSH terms
- Algorithms, Clinical Trials as Topic, Computer Simulation, Humans, Regression Analysis, Research Design, Time Factors, Meta-Analysis as Topic, Reproducibility of Results, Models, Statistical, Bias, Randomized Controlled Trials as Topic, Publication Bias, Evidence-Based Medicine
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.