Detecting and adjusting for small‐study effects in meta‐analysis
The DOI matches its Crossref record; 77 of the 77 references it cites were found in a registry or an index.
Abstract
Publication bias and related types of small‐study effects threaten the validity of systematic reviews. The existence of small‐study effects has been demonstrated in empirical studies. Small‐study effects are graphically diagnosed by inspection of the funnel plot. Though observed funnel plot asymmetry cannot be easily linked to a specific reason, tests based on funnel plot asymmetry have been proposed. Beyond a vast range of funnel plot tests, there exist several methods for adjusting treatment effect estimates for these biases. In this article, we consider the trim‐and‐fill method, the Copas selection model, and more recent regression‐based approaches. The methods are exemplified using a meta‐analysis from the literature and compared in a simulation study, based on binary response data. They are also applied to a large set of meta‐analyses. Some fundamental differences between the approaches are discussed. An assumption common to the trim‐and‐fill method and the Copas selection model is that the small‐study effect is caused by selection. The trim‐and‐fill method corresponds to an unknown implicit model generated by the symmetry assumption, whereas the Copas selection model is a parametric statistical model. However, it requires a sensitivity analysis. Regression‐based approaches are easier to implement and not based on a specific selection model. Both simulations and applications suggest that in the presence of strong selection both the trim‐and‐fill method and the Copas selection model may not fully eliminate bias, while regression‐based approaches seem to be a promising alternative.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“We checked the model for possible small-study effects by visually examining a funnel plot (20), supplemented by Egger's regression-based test for funnel plot asymmetry (21).”
From Referral of patients to psychiatric emergency departments by police: A systematic review and meta-analysis (2025) · cites this paper as [20]
In the citing paper
“Previous summary work on liver enzymes and inter-hepatic fat, in people with non-alcoholic fatty liver disease, has shown that an aggregate exercise program energy expenditure of around 10,000 kcal is required to elicit improvement [37], and this may take several weeks to achieve.”
From The Effect of Exercise Training on Blood Lipids: A Systematic Review and Meta-analysis (2025) · cites this paper as [37]
Citations
Cites
Bias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722
Meta-analysis in clinical trials
DerSimonian, R., & Laird, N. (1986) · Controlled Clinical Trials · doi:10.1016/0197-2456(86)90046-2 · Cited by 39,552 · cited 3×
“Therefore, our aim was (i) to extend the established random effects model (DerSimonian and Laird, 1986) by a parameter that represents small-study effects in a natural way and (ii) to use a shrinkage procedure to improve the properties of parameter estimation (Rücker, 2010; Rücker et al.”
MethodCochrane Handbook for Systematic Reviews of Interventions: Cochrane Book Series
(2008) · doi:10.1002/9780470712184 · Cited by 25,048
Operating characteristics of a rank correlation test for publication bias
Begg, C. B., & Mazumdar, M. (1994) · Biometrics · doi:10.2307/2533446 · Cited by 16,955 · cited 1×
“Broadly, tests fall into two classes: (i) non-parametric tests using rank-correlation methods, going back to Begg and Mazumdar (1994) and (ii) regression tests, represented by a test proposed by Egger et al. (1997a).”
MethodMcCullagh, P., & Nelder, J. A. (1989) · doi:10.1007/978-1-4899-3242-6 · Cited by 16,836
Duval, S., & Tweedie, R. (2000) · Biometrics · doi:10.1111/j.0006-341x.2000.00455.x · Cited by 14,099
Measurement in Medicine: The Analysis of Method Comparison Studies
Altman, D. G., & Bland, J. M. (1983) · The Statistician · doi:10.2307/2987937 · Cited by 4,300 · cited 2×
“Figure 5 (top left panel) shows a scatterplot of b̂0 against b̂limðr2 ¼ 0:9006Þ: Bland–Altman plots (not shown; Altman and Bland, 1983) give no hints of systematic trends in the difference.”
BackgroundFunnel plots for detecting bias in meta-analysis
Sterne, J. A. C., & Egger, M. (2001) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(01)00377-8 · Cited by 3,779 · cited 1×
“This is a scatter plot showing the treatment effect (usually on the x-axis) depending on the precision of the study, preferably represented by the standard error on a reverse scale (Sterne and Egger, 2001).”
Publication bias in clinical research
Easterbrook, P. J., Gopalan, R., Berlin, J. A., & Matthews, D. R. (1991) · The Lancet · doi:10.1016/0140-6736(91)90201-y · Cited by 3,109 · cited 1×
“The first form of this effect that was acknowledged was publication bias (Sterling, 1959; Simes, 1986; Begg and Berlin, 1988; Berlin et al., 1989; Rust et al., 1990; Sterling et al., 1995; Dickersin, 1990; Easterbrook et al., 1991; Sutton et al., 2000a).”
BackgroundHow should meta‐regression analyses be undertaken and interpreted?
Thompson, S. G., & Higgins, J. P. T. (2002) · Statistics in Medicine · doi:10.1002/sim.1187 · Cited by 3,026 · cited 1×
“If study-level covariates are available that describe the baseline risk of patients and may explain heterogeneity, they can be included in a meta-regression model (Sharp and Thompson, 2000; Thompson and Higgins, 2002).”
Background
Research works printed but not linked: 1
- Rücker G.(2010). Small‐study effects and heterogeneity in meta‐analysis. Ph.D. Thesis.http://www.freidok.uni‐freiburg.de/volltexte/7486/
Cited by
Items 101–132 of 132
Goldschmidt, T., Bermpohl, F., Schreiter, S., van Bergen, A. H., Kupka, R. W., Schouler-Ocak, M., Gutwinski, S., & Deutscher, K. (2025) · Psychiatric Services · doi:10.1176/appi.ps.20240323 · Cited by 2
Quintero‐Cardona, P., Tangarife, M. A., Blandon, J. A., Arango‐Lasprilla, J. C., Ibáñez, A. M., & Báez, S. (2026) · Alzheimer s & Dementia · doi:10.1002/alz.71506 · Cited by 3
Conducting and Reporting Meta.analysis
Ghosal, S. (2022) · International Journal of Diabetes and Technology · doi:10.4103/ijdt.ijdt_16_23 · Cited by 1
Oke, J. L., Clements, A., McLellan, J., Bankhead, C. R., Taylor, C. J., Spence, G. T., Banerjee, A., & Perera, R. (2018) · Open Heart · doi:10.1136/openhrt-2018-000826 · Cited by 2
“This was to account for biases in which smaller studies report larger effect sizes.(11) We first identified the trial with the largest RR reduction (single-trial choice) following re-estimation and assessed this version of the intervention.”
MethodAssessing the robustness of direct meta-analysis in the presence of heterogeneity
Ainsworth, C. (2018) · Journal of Comparative Effectiveness Research · doi:10.2217/cer-2018-0024 · Cited by 2
Artsitas, S., Artsitas, D., Koronaki, I. P., Toutouzas, K. G., & Zografos, G. (2025) · Bulletin of the National Research Centre/Bulletin of the National Research Center · doi:10.1186/s42269-025-01350-2 · Cited by 1
“To assess the relevance and presence of publication bias (PB) and small-study effects (SSEs), both funnel plots and radial (Galbraith-style) plots were generated (Rücker et al. 2011).”
Sampaio, B. O. A., Santos, A. R. D. A., Ferreira, M. V. N., & de Moraes, A. C. F. (2025) · Preventive Medicine Reports · doi:10.1016/j.pmedr.2025.103230 · Cited by 7
“…part of this heterogeneity is due to publication bias or study size, as publication bias tends to distort the funnel (producing an asymmetric shape) and may affect the pooled estimate and, consequently, the heterogeneity test (Higgins and Thompson, 2002; Egger et al., 1997; Rücker et al., 2011).”
Lin, D., Gunaga, S., Mowbray, F., Isaacs, E. D., Markwalter, D., George, N. R., Hay, A., Manfredi, R. A., Westlake, E., Akhter, M., et al. (2025) · BMJ Open · doi:10.1136/bmjopen-2024-095894 · Cited by 2
“26 27 As I 2 depends on the number of participants rather than the number of studies, we will only employ this statistic if we have more than 200 individuals per polling.”
Shahzil, M., Habiba, U., Irfan, M. Z., Qureshi, M. A., Faisal, M. S., Kashif, T., Qureshi, A. A., Ali, H., Jahagirdar, V., & Vinayek, R. (2025) · World Journal of Transplantation · doi:10.5500/wjt.v15.i4.106444 · Cited by 1
Jiang, J., Guo, W., & Zhixiang, D. I. N. G. (2026) · Frontiers in Public Health · doi:10.3389/fpubh.2026.1832691 · Cited by 1
A series of meta-analytic tests of the depletion effect
Carter, E. C. (2013) · Cited by 1
Cognitive Impairment in Paediatric PCNS: A Systematic Review and Meta-analysisPreprint
Falcone, L., Riddell, N., & Murphy, M. (2025) · medRxiv · doi:10.1101/2025.05.29.25328543 · Cited by 2
Carpenter, J. R., Rücker, G., & Schwarzer, G. (2020) · Biometrics · doi:10.1111/biom.13344
“For this reason, we and others (Rücker et al. (2011)) have argued this should be adjusted for before presenting the results.”
Self-citationXu, C., Furuya‐Kanamori, L., Lin, L., & Doi, S. A. R. (2020) · medRxiv · doi:10.1101/2020.10.13.20212290
“For example, in the case where pooled small studies suggest a positive result, we may need to assess all these other potential sources of uncertainty (additional analyses) before reporting results [35-38].”
BackgroundSchwartz, S. J. (2021) · Oxford University Press eBooks · doi:10.1093/oso/9780190095918.003.0003
(2021) · Wiley eBooks · doi:10.1002/9781119558378.ref
Chao, Y. (2022) · medRxiv · doi:10.1101/2022.04.05.22273469
“It is unclear whether the included smaller studies tended to obtain more high-risk 208 individuals or whether small-study effects have been well addressed.[37] The reasons for the 209 discrepancies in the estimated infection rates between studies were not clear.”
ResultLiu, Y. (2019) · American Educational Research Association Annual Meeting (AERA) · doi:10.3102/1441769
Barba, M., Bernasconi, D. P., Manodoro, S., & Frigerio, M. (2023) · International Journal of Gynecology & Obstetrics · doi:10.1002/ijgo.14940
Bharadwaj, A., Sweller, N., Dargue, N., & Jones, M. (2025) · Review Journal of Autism and Developmental Disorders · doi:10.1007/s40489-025-00521-1
Li, X., Luo, A., Deng, Y., Yang, Y., Yang, Q., Chen, Y., He, L., & Zhou, M. (2025) · Archives of Gerontology and Geriatrics · doi:10.1016/j.archger.2025.106089
Additional file 2 of Meta-analysis using Python: a hands-on tutorial
Masoumi, S., & Shahraz, S. (2022) · Figshare · doi:10.6084/m9.figshare.20295714
Shahzil, M., Kazmi, S. K., Shehzadi, M., Hasan, F. N., Danish, M. W., Faisal, M. S., Canakis, J. P., & Clarke, K. (2025) · Research Square · doi:10.21203/rs.3.rs-7035116/v1
Putra, B., Karti, P. D. M. H., Abdullah, L., Prihantoro, I., Wirnas, D., Gopar, R. A., Surachman, M., Darmawan, I. W. A., Fanindi, A., Akhadiarto, S., et al. (2026) · Journal of soil science and plant nutrition · doi:10.1007/s42729-026-03059-1
Shahzil, M., Kazmi, S. K., Shehzadi, M., Hasan, F. N., Danish, M. W., Faisal, M. S., Canakis, J. P., & Clarke, K. (2026) · Gastroenterology Research and Practice · doi:10.1155/grp/2994340
Shahzil, M., Faisal, M. H., Khan, M. H., Abid, S. F., Shehzadi, M., Faisal, M. S., Chaudhary, A. J., Warraich, M. S., & Maranki, J. L. (2026) · European Journal of Gastroenterology & Hepatology · doi:10.1097/meg.0000000000003190
Virtual Reality in Education: A Stratified Meta-Analysis of Experimental Rigor and Bias
Layadi, S., Huguet, P., Pavic, K., & Chevalère, J. (2026) · Educational Psychology Review · doi:10.1007/s10648-026-10130-x · Cited by 3
Laurent, J. S., Limond, J., & Karl, A. (2026) · European Journal of Trauma & Dissociation · doi:10.1016/j.ejtd.2026.100715 · Cited by 2
Jo, H. E., Woo, S. H., Lee, Y. K., Lee, H., Kim, J. S., & Lee, J. H. (2026) · Journal of Acupuncture Research · doi:10.13045/jar.26.0021
Tudoraşcu, D. R., Vere, C. C., Petrescu, M., Ciurea, A.-M., Pana, R. C., Dan, A. O., Târtea, E. A., Greșiță, A., Hădăreanu, D.-R., Obleagă, C. V., et al. (2026) · International Journal of Molecular Sciences · doi:10.3390/ijms27209090
Metadata
- Authors
- Rücker, G., Carpenter, J. R., & Schwarzer, G.
- Affiliations
- University of Freiburg (DE)
- Corresponding author
- Rücker, Gerta
- Venue
- Biometrical Journal
- Publisher
- Wiley
- Published
- 2011-01-14
- Volume
- 53
- Issue
- 2
- Pages
- 351-368
- ISSN
- 0323-3847 (ISSN-L)
- 1521-4036
- OpenAlex
- W2069224123
- Cited by
- 132
- References
- 76
- Citations by year
- 2026: 11 · 2025: 14 · 2024: 9 · 2023: 8 · 2022: 16 · 2021: 9 · 2020: 10 · 2019: 13 · 2018: 5 · 2017: 5 · 2016: 5 · 2015: 3 · 2014: 8 · 2013: 7 · 2012: 6
- FWCI
- 7.83
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Biometrical Journal (publishedVersion)
- PubMed (publishedVersion)
- LSHTM Research Online (London School of Hygiene and Tropical Medicine) (submittedVersion)
- Funders
- Deutsche Forschungsgemeinschaft
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Keywords
- small-study effects, publication bias, funnel plot asymmetry, selection bias, binary response data, meta-analysis bias
- MeSH terms
- Algorithms, Clinical Trials as Topic, Computer Simulation, Humans, Regression Analysis, Research Design, Time Factors, Meta-Analysis as Topic, Reproducibility of Results, Models, Statistical, Bias, Randomized Controlled Trials as Topic, Publication Bias, Evidence-Based Medicine
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.