Detecting and adjusting for small‐study effects in meta‐analysis
The DOI matches its Crossref record; 77 of the 77 references it cites were found in a registry or an index.
Abstract
Publication bias and related types of small‐study effects threaten the validity of systematic reviews. The existence of small‐study effects has been demonstrated in empirical studies. Small‐study effects are graphically diagnosed by inspection of the funnel plot. Though observed funnel plot asymmetry cannot be easily linked to a specific reason, tests based on funnel plot asymmetry have been proposed. Beyond a vast range of funnel plot tests, there exist several methods for adjusting treatment effect estimates for these biases. In this article, we consider the trim‐and‐fill method, the Copas selection model, and more recent regression‐based approaches. The methods are exemplified using a meta‐analysis from the literature and compared in a simulation study, based on binary response data. They are also applied to a large set of meta‐analyses. Some fundamental differences between the approaches are discussed. An assumption common to the trim‐and‐fill method and the Copas selection model is that the small‐study effect is caused by selection. The trim‐and‐fill method corresponds to an unknown implicit model generated by the symmetry assumption, whereas the Copas selection model is a parametric statistical model. However, it requires a sensitivity analysis. Regression‐based approaches are easier to implement and not based on a specific selection model. Both simulations and applications suggest that in the presence of strong selection both the trim‐and‐fill method and the Copas selection model may not fully eliminate bias, while regression‐based approaches seem to be a promising alternative.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“We checked the model for possible small-study effects by visually examining a funnel plot (20), supplemented by Egger's regression-based test for funnel plot asymmetry (21).”
From Referral of patients to psychiatric emergency departments by police: A systematic review and meta-analysis (2025) · cites this paper as [20]
In the citing paper
“Previous summary work on liver enzymes and inter-hepatic fat, in people with non-alcoholic fatty liver disease, has shown that an aggregate exercise program energy expenditure of around 10,000 kcal is required to elicit improvement [37], and this may take several weeks to achieve.”
From The Effect of Exercise Training on Blood Lipids: A Systematic Review and Meta-analysis (2025) · cites this paper as [37]
Citations
Cites
Bias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722
Meta-analysis in clinical trials
DerSimonian, R., & Laird, N. (1986) · Controlled Clinical Trials · doi:10.1016/0197-2456(86)90046-2 · Cited by 39,552 · cited 3×
“Therefore, our aim was (i) to extend the established random effects model (DerSimonian and Laird, 1986) by a parameter that represents small-study effects in a natural way and (ii) to use a shrinkage procedure to improve the properties of parameter estimation (Rücker, 2010; Rücker et al.”
MethodCochrane Handbook for Systematic Reviews of Interventions: Cochrane Book Series
(2008) · doi:10.1002/9780470712184 · Cited by 25,048
Operating characteristics of a rank correlation test for publication bias
Begg, C. B., & Mazumdar, M. (1994) · Biometrics · doi:10.2307/2533446 · Cited by 16,955 · cited 1×
“Broadly, tests fall into two classes: (i) non-parametric tests using rank-correlation methods, going back to Begg and Mazumdar (1994) and (ii) regression tests, represented by a test proposed by Egger et al. (1997a).”
MethodMcCullagh, P., & Nelder, J. A. (1989) · doi:10.1007/978-1-4899-3242-6 · Cited by 16,836
Duval, S., & Tweedie, R. (2000) · Biometrics · doi:10.1111/j.0006-341x.2000.00455.x · Cited by 14,099
Measurement in Medicine: The Analysis of Method Comparison Studies
Altman, D. G., & Bland, J. M. (1983) · The Statistician · doi:10.2307/2987937 · Cited by 4,300 · cited 2×
“Figure 5 (top left panel) shows a scatterplot of b̂0 against b̂limðr2 ¼ 0:9006Þ: Bland–Altman plots (not shown; Altman and Bland, 1983) give no hints of systematic trends in the difference.”
BackgroundFunnel plots for detecting bias in meta-analysis
Sterne, J. A. C., & Egger, M. (2001) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(01)00377-8 · Cited by 3,779 · cited 1×
“This is a scatter plot showing the treatment effect (usually on the x-axis) depending on the precision of the study, preferably represented by the standard error on a reverse scale (Sterne and Egger, 2001).”
Publication bias in clinical research
Easterbrook, P. J., Gopalan, R., Berlin, J. A., & Matthews, D. R. (1991) · The Lancet · doi:10.1016/0140-6736(91)90201-y · Cited by 3,109 · cited 1×
“The first form of this effect that was acknowledged was publication bias (Sterling, 1959; Simes, 1986; Begg and Berlin, 1988; Berlin et al., 1989; Rust et al., 1990; Sterling et al., 1995; Dickersin, 1990; Easterbrook et al., 1991; Sutton et al., 2000a).”
BackgroundHow should meta‐regression analyses be undertaken and interpreted?
Thompson, S. G., & Higgins, J. P. T. (2002) · Statistics in Medicine · doi:10.1002/sim.1187 · Cited by 3,026 · cited 1×
“If study-level covariates are available that describe the baseline risk of patients and may explain heterogeneity, they can be included in a meta-regression model (Sharp and Thompson, 2000; Thompson and Higgins, 2002).”
BackgroundA Nonparametric “Trim and Fill” Method of Accounting for Publication Bias in Meta-Analysis
Duval, S., & Tweedie, R. (2000) · Journal of the American Statistical Association · doi:10.1080/01621459.2000.10473905 · Cited by 2,955
A Re-Evaluation of Random-Effects Meta-Analysis
Higgins, J. P. T., Thompson, S. G., & Spiegelhalter, D. J. (2009) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/j.1467-985x.2008.00552.x · Cited by 2,569 · cited 1×
“…estimates for all parameters and the observed study residual for ei obtaining x1;i ¼ b̂1 ffiffiffiffiffiffiffiffiffiffiffiffi t̂2 s2i 1t̂2 s ðxi b̂Þ: The limit meta-analysis yields shrunken estimates of the study-specific effects, comparable to empirical Bayes estimates (Higgins et al., 2009).”
ResultPublication Bias in Meta‐Analysis: Prevention, Assessment and Adjustments
(2005) · doi:10.1002/0470870168 · Cited by 2,368
Publication and related bias in meta-analysis
Sterne, J. A. C., Gavaghan, D., & Egger, M. (2000) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(00)00242-0 · Cited by 2,282 · cited 2×
“2 Publication bias and small-study effects Against the background of ongoing discussions on the meaning of asymmetry in funnel plots, Sterne et al. (2000) coined the term ‘small-study effects’ for the phenomenon that sometimes small trials show different, often larger, treatment effects than the…”
BackgroundComparison of two methods to detect publication bias in meta-analysis.
Peters, J. L. (2006) · JAMA · doi:10.1001/jama.295.6.676 · Cited by 2,278 · cited 3×
“They were followed by modifications for the special case of binary response data (Macaskill et al., 2001; Harbord et al., 2006; Peters et al., 2006; Schwarzer et al., 2007; Rücker et al., 2008a).”
MethodA modified test for small‐study effects in meta‐analyses of controlled trials with binary endpoints
Harbord, R. M., Egger, M., & Sterne, J. A. C. (2006) · Statistics in Medicine · doi:10.1002/sim.2380 · Cited by 2,267 · cited 2×
“For the odds ratio as an effect measure, Harbord et al. (2006) proposed taking for each trial the score approximation of the treatment effect estimate, and evaluating this under the null hypothesis of no treatment effect.”
MethodChan, A.-W., Hróbjartsson, A., Haahr, M. T., Gøtzsche, P. C., & Altman, D. G. (2004) · JAMA · doi:10.1001/jama.291.20.2457 · Cited by 1,845
The case of the misleading funnel plot
Lau, J., Ioannidis, J. P. A., Terrin, N., Schmid, C. H., & Olkin, I. (2006) · BMJ · doi:10.1136/bmj.333.7568.597 · Cited by 1,766 · cited 1×
“It is questioned whether funnel plots are a suited means for judging publication bias (Tang and Liu, 2000; Terrin et al., 2003, 2005; Lau et al., 2006; Ioannidis and Trikalinos, 2007; Ioannidis, 2008).”
BackgroundSumming Up: The Science of Reviewing Research
Light, R. J., & Pillemer, D. B. (1984) · Harvard University Press eBooks · doi:10.4159/9780674040243 · Cited by 1,427
The Existence of Publication Bias and Risk Factors for Its Occurrence
Dickersin, K. (1990) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.1990.03440100097014 · Cited by 1,417 · cited 1×
“The first form of this effect that was acknowledged was publication bias (Sterling, 1959; Simes, 1986; Begg and Berlin, 1988; Berlin et al., 1989; Rust et al., 1990; Sterling et al., 1995; Dickersin, 1990; Easterbrook et al., 1991; Sutton et al., 2000a).”
BackgroundEmpirical assessment of effect of publication bias on meta-analyses
Sutton, A. J. (2000) · BMJ · doi:10.1136/bmj.320.7249.1574 · Cited by 1,398 · cited 2×
“The first form of this effect that was acknowledged was publication bias (Sterling, 1959; Simes, 1986; Begg and Berlin, 1988; Berlin et al., 1989; Rust et al., 1990; Sterling et al., 1995; Dickersin, 1990; Easterbrook et al., 1991; Sutton et al., 2000a).”
BackgroundMethodUndue reliance on I 2 in assessing heterogeneity may mislead
Rücker, G., Schwarzer, G., Carpenter, J. R., & Schumacher, M. (2008) · BMC Medical Research Methodology · doi:10.1186/1471-2288-8-79 · Cited by 1,287
Self-citationA comparison of methods to detect publication bias in meta‐analysis
Macaskill, P., Walter, S. D., & Irwig, L. (2001) · Statistics in Medicine · doi:10.1002/sim.698 · Cited by 1,286 · cited 3×
“They were followed by modifications for the special case of binary response data (Macaskill et al., 2001; Harbord et al., 2006; Peters et al., 2006; Schwarzer et al., 2007; Rücker et al., 2008a).”
MethodThe appropriateness of asymmetry tests for publication bias in meta-analyses: A large survey
Ioannidis, J. P. A., & Trikalinos, T. A. (2007) · Canadian Medical Association Journal · doi:10.1503/cmaj.060410 · Cited by 1,052 · cited 1×
“It is questioned whether funnel plots are a suited means for judging publication bias (Tang and Liu, 2000; Terrin et al., 2003, 2005; Lau et al., 2006; Ioannidis and Trikalinos, 2007; Ioannidis, 2008).”
BackgroundLanguage bias in randomised controlled trials published in English and German
Egger, M., Zellweger-Zähner, T., Schneider, M., Junker, C., Lengeler, C., & Antes, G. (1997) · The Lancet · doi:10.1016/s0140-6736(97)02419-7 · Cited by 1,028
Sterling, T. D. (1959) · Journal of the American Statistical Association · doi:10.1080/01621459.1959.10501497 · Cited by 964 · cited 1×
“The first form of this effect that was acknowledged was publication bias (Sterling, 1959; Simes, 1986; Begg and Berlin, 1988; Berlin et al., 1989; Rust et al., 1990; Sterling et al., 1995; Dickersin, 1990; Easterbrook et al., 1991; Sutton et al., 2000a).”
BackgroundPublication bias: evidence of delayed publication in a cohort study of clinical research projects
Stern, J. M., & Simes, R. J. (1997) · BMJ · doi:10.1136/bmj.315.7109.640 · Cited by 818 · cited 1×
“…et al., 1995; Moher et al., 1996; Egger et al., 1997b; Moher et al., 2000); (ii) time lag bias (‘significant’ studies are published faster than others, Stern and Simes, 1997); (iii) multiple publication bias (‘significant’ studies are published more than once, Tramèr et al., 1997);…”
BackgroundPublication bias—a problem in interpreting medical data
Begg, C. B., & Berlin, J. A. (1988) · Journal of the Royal Statistical Society. Series A (Statistics in Society) · doi:10.2307/2982993 · Cited by 814 · cited 1×
“The first form of this effect that was acknowledged was publication bias (Sterling, 1959; Simes, 1986; Begg and Berlin, 1988; Berlin et al., 1989; Rust et al., 1990; Sterling et al., 1995; Dickersin, 1990; Easterbrook et al., 1991; Sutton et al., 2000a).”
BackgroundAdjusting for publication bias in the presence of heterogeneity
Terrin, N., Schmid, C. H., Lau, J., & Olkin, I. (2003) · Statistics in Medicine · doi:10.1002/sim.1461 · Cited by 809 · cited 1×
“It is questioned whether funnel plots are a suited means for judging publication bias (Tang and Liu, 2000; Terrin et al., 2003, 2005; Lau et al., 2006; Ioannidis and Trikalinos, 2007; Ioannidis, 2008).”
BackgroundPublication bias: the case for an international registry of clinical trials.
Simes, R. J. (1986) · Journal of Clinical Oncology · doi:10.1200/jco.1986.4.10.1529 · Cited by 747 · cited 2×
“The first form of this effect that was acknowledged was publication bias (Sterling, 1959; Simes, 1986; Begg and Berlin, 1988; Berlin et al., 1989; Rust et al., 1990; Sterling et al., 1995; Dickersin, 1990; Easterbrook et al., 1991; Sutton et al., 2000a).”
BackgroundImpact of covert duplicate publication on meta-analysis: a case study
Tramèr, M. R., Reynolds, D. J. M., Moore, R. A., & McQuay, H. J. (1997) · BMJ · doi:10.1136/bmj.315.7109.635 · Cited by 702 · cited 1×
“…studies are published faster than others, Stern and Simes, 1997); (iii) multiple publication bias (‘significant’ studies are published more than once, Tramèr et al., 1997); selective r 2010 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim www.biometrical-journal.com outcome reporting bias (Chan…”
BackgroundInterpretation of tests of heterogeneity and bias in meta-analysis
Ioannidis, J. P. A. (2008) · Journal of Evaluation in Clinical Practice · doi:10.1111/j.1365-2753.2008.00986.x · Cited by 696 · cited 1×
“It is questioned whether funnel plots are a suited means for judging publication bias (Tang and Liu, 2000; Terrin et al., 2003, 2005; Lau et al., 2006; Ioannidis and Trikalinos, 2007; Ioannidis, 2008).”
BackgroundAnalysis of cost data in randomized trials: an application of the non-parametric bootstrap
Barber, J. A., & Thompson, S. G. (2000) · Statistics in Medicine · doi:10.1002/1097-0258(20001215)19:23<3219::aid-sim623>3.0.co;2-p · Cited by 690
Stanley, T. D. (2008) · Oxford Bulletin of Economics and Statistics · doi:10.1111/j.1468-0084.2007.00487.x · Cited by 605 · cited 2×
“3.3 Regression-based approach Stanley (2008) seems to be the first author who proposed a regression-based treatment effect estimate adjusting for small-study effects.”
BackgroundMethodInfluentialMoher, D., Fortin, P., Jadad, A. R., Jüni, P., Klassen, T., Le Lorier, J., Liberati, A., Penna, A., & Linde, K. (1996) · The Lancet · doi:10.1016/s0140-6736(96)90538-3 · Cited by 537 · cited 2×
“Examples are (i) language bias (‘significant’ results tend to be published in high-ranking English language journals, while studies without a ‘significant’ result are published in local language journals, Grégoire et al., 1995; Moher et al., 1996; Egger et al., 1997b; Moher et al., 2000); (ii) time lag bias (‘signific…”
BackgroundGrégoire, G., Derderian, F., & Le Lorier, J. (1995) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(94)00098-b · Cited by 526 · cited 1×
“…tend to be published in high-ranking English language journals, while studies without a ‘significant’ result are published in local language journals, Grégoire et al., 1995; Moher et al., 1996; Egger et al., 1997b; Moher et al., 2000); (ii) time lag bias (‘significant’ studies are published…”
BackgroundSterling, T. D. (1959) · Journal of the American Statistical Association · doi:10.2307/2282137 · Cited by 524
Meta-analysis, funnel plots and sensitivity analysis
Copas, J. (2000) · Biostatistics · doi:10.1093/biostatistics/1.3.247 · Cited by 503 · cited 3×
“In this article, we consider the trim-and-fill method (Taylor and Tweedie, 1998; Duval and Tweedie, 2000a,b), selection models (Copas, 1999; Copas and Shi, 2000, 2001), and regression-based approaches (also called extrapolation approaches, Stanley, 2008; Moreno et al., 2009a,b; Peters et al., 2010;…”
BackgroundMethodInfluentialTerrin, N., Schmid, C. H., & Lau, J. (2005) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2005.01.006 · Cited by 492 · cited 1×
“It is questioned whether funnel plots are a suited means for judging publication bias (Tang and Liu, 2000; Terrin et al., 2003, 2005; Lau et al., 2006; Ioannidis and Trikalinos, 2007; Ioannidis, 2008).”
BackgroundSterling, T. D., Rosenbaum, W. L., & Weinkam, J. J. (1995) · The American Statistician · doi:10.1080/00031305.1995.10476125 · Cited by 453 · cited 1×
“The first form of this effect that was acknowledged was publication bias (Sterling, 1959; Simes, 1986; Begg and Berlin, 1988; Berlin et al., 1989; Rust et al., 1990; Sterling et al., 1995; Dickersin, 1990; Easterbrook et al., 1991; Sutton et al., 2000a).”
BackgroundWhat contributions do languages other than English make on the results of meta-analyses?
Moher, D., Pham, Klassen, T. P., Schulz, K. F., Berlin, J. A., Jadad, A. R., & Liberati, A. (2000) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(00)00188-8 · Cited by 453 · cited 2×
“Examples are (i) language bias (‘significant’ results tend to be published in high-ranking English language journals, while studies without a ‘significant’ result are published in local language journals, Grégoire et al., 1995; Moher et al., 1996; Egger et al., 1997b; Moher et al., 2000); (ii) time lag bias (‘signific…”
BackgroundQuantitive systematic review of topically applied non-steroidal anti-inflammatory drugs
Moore, R. A., Tramer, M. R., Carroll, D., Wiffen, P. J., & McQuay, H. J. (1998) · BMJ · doi:10.1136/bmj.316.7128.333 · Cited by 446 · cited 1×
“The main outcome was treatment success, defined as at least 50% reduction in pain (Moore et al., 1998).”
BackgroundArcsine test for publication bias in meta‐analyses with binary outcomes
Rücker, G., Schwarzer, G., & Carpenter, J. (2008) · Statistics in Medicine · doi:10.1002/sim.2971 · Cited by 444 · cited 9×
“At first, we have developed a new type of test of small-study effects, the arcsine test, that is available in three versions (Rücker et al., 2008a).”
MethodInfluentialSelf-citationOutcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research
Chan, A.-W. (2004) · Canadian Medical Association Journal · doi:10.1503/cmaj.1041086 · Cited by 444
Misleading funnel plot for detection of bias in meta-analysis
Tang, J.-L., & Liu, J. L. (2000) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(99)00204-8 · Cited by 420
Confidence intervals for the amount of heterogeneity in meta‐analysis
Viechtbauer, W. (2007) · Statistics in Medicine · doi:10.1002/sim.2514 · Cited by 394 · cited 2×
“…estimating t2 by the method-of-moments-based estimator proposed by DerSimonian and Laird (1986) or, alternatively, the REML (restricted maximum likelihood) or another estimator (Sidik and Jonkman, 2005; Knapp et al., 2006; Viechtbauer, 2007), a and b can be estimated by ML (maximum likelihood).”
MethodMoreno, S. G., Sutton, A. J., Ades, A., Stanley, T. D., Abrams, K. R., Peters, J. L., & Cooper, N. J. (2009) · BMC Medical Research Methodology · doi:10.1186/1471-2288-9-2 · Cited by 373
Modelling publication bias in meta-analysis: a review
Sutton, A. J., Song, F., Gilbody, S. M., & Abrams, K. R. (2000) · Statistical Methods in Medical Research · doi:10.1177/096228020000900503 · Cited by 288 · cited 2×
“The first form of this effect that was acknowledged was publication bias (Sterling, 1959; Simes, 1986; Begg and Berlin, 1988; Berlin et al., 1989; Rust et al., 1990; Sterling et al., 1995; Dickersin, 1990; Easterbrook et al., 1991; Sutton et al., 2000a).”
BackgroundMethodA sensitivity analysis for publication bias in systematic reviews
Copas, J. B., & Shi, J. Q. (2001) · Statistical Methods in Medical Research · doi:10.1177/096228020101000402 · Cited by 259 · cited 5×
“Following our work on the Copas selection model (Copas, 1999; Copas and Shi, 2000, 2001; Carpenter et al., 2009a,b; Schwarzer et al., 2010), we focused on methods for adjusting estimates for these biases.”
BackgroundMethodInfluentialSimple Heterogeneity Variance Estimation for Meta-Analysis
Sidik, K., & Jonkman, J. N. (2005) · Journal of the Royal Statistical Society Series C: Applied Statistics · doi:10.1111/j.1467-9876.2005.00489.x · Cited by 253 · cited 2×
“…estimating t2 by the method-of-moments-based estimator proposed by DerSimonian and Laird (1986) or, alternatively, the REML (restricted maximum likelihood) or another estimator (Sidik and Jonkman, 2005; Knapp et al., 2006; Viechtbauer, 2007), a and b can be estimated by ML (maximum likelihood).”
MethodThe relationship between quality of research and citation frequency
Nieminen, P., Carpenter, J., Rucker, G., & Schumacher, M. (2006) · BMC Medical Research Methodology · doi:10.1186/1471-2288-6-42 · Cited by 184
Self-citationMoreno, S. G., Sutton, A. J., Turner, E. H., Abrams, K. R., Cooper, N. J., Palmer, T. M., & Ades, A. E. (2009) · BMJ · doi:10.1136/bmj.b2981 · Cited by 184 · cited 7×
“…the trim-and-fill method (Taylor and Tweedie, 1998; Duval and Tweedie, 2000a,b), selection models (Copas, 1999; Copas and Shi, 2000, 2001), and regression-based approaches (also called extrapolation approaches, Stanley, 2008; Moreno et al., 2009a,b; Peters et al., 2010; Rücker et al., 2010).”
BackgroundMethodResultInfluentialCould It Be Better to Discard 90% of the Data? A Statistical Paradox
Stanley, T. D., Jarrell, S. B., & Doucouliagos, H. (2010) · The American Statistician · doi:10.1198/tast.2009.08205 · Cited by 170 · cited 1×
“Stanley et al. (2010) went a step further and asked whether ‘it could be better to discard 90% of the data’.”
BackgroundWhat Works?: Selectivity Models and Meta-Analysis
Copas, J. (1999) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/1467-985x.00123 · Cited by 168 · cited 5×
“Following our work on the Copas selection model (Copas, 1999; Copas and Shi, 2000, 2001; Carpenter et al., 2009a,b; Schwarzer et al., 2010), we focused on methods for adjusting estimates for these biases.”
BackgroundMethodInfluentialAssessing Publication Bias in Meta-Analyses in the Presence of Between-Study Heterogeneity
Peters, J. L., Sutton, A. J., Jones, D. R., Abrams, K. R., Rushton, L., & Moreno, S. G. (2010) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/j.1467-985x.2009.00629.x · Cited by 166 · cited 2×
“In this article, we consider the trim-and-fill method (Taylor and Tweedie, 1998; Duval and Tweedie, 2000a,b), selection models (Copas, 1999; Copas and Shi, 2000, 2001), and regression-based approaches (also called extrapolation approaches, Stanley, 2008; Moreno et al., 2009a,b; Peters et al., 2010; Rücker et al., 2010…”
MethodSterling, T. D., Rosenbaum, W. L., & Jj, W. (1995) · The American Statistician · doi:10.2307/2684823 · Cited by 158
An Assessment of Publication Bias Using a Sample of Published Clinical Trials
Berlin, J. A., Begg, C. B., & Louis, T. A. (1989) · Journal of the American Statistical Association · doi:10.1080/01621459.1989.10478782 · Cited by 134 · cited 1×
“The first form of this effect that was acknowledged was publication bias (Sterling, 1959; Simes, 1986; Begg and Berlin, 1988; Berlin et al., 1989; Rust et al., 1990; Sterling et al., 1995; Dickersin, 1990; Easterbrook et al., 1991; Sutton et al., 2000a).”
BackgroundSchwarzer, G., Carpenter, J. R., & Rücker, G. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.05.008 · Cited by 132 · cited 7×
“In addition, the model was compared to the more popular trim-and-fill method (Schwarzer et al., 2010).”
MethodResultInfluentialSelf-citationIdentification and impact of outcome selection bias in meta‐analysis
Williamson, P. R., & Gamble, C. (2005) · Statistics in Medicine · doi:10.1002/sim.2025 · Cited by 132 · cited 2×
“…Tramèr et al., 1997); selective r 2010 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim www.biometrical-journal.com outcome reporting bias (Chan et al., 2004a,b; Williamson and Gamble, 2005), and citation bias (‘significant’ results are cited more often than studies without a ‘significant’ result,…”
BackgroundSharp, S. J., & Thompson, S. G. (2000) · Statistics in Medicine · doi:10.1002/1097-0258(20001215)19:23<3251::aid-sim625>3.0.co;2-2 · Cited by 102
Statistically significant papers in psychiatry were cited more often than others
Nieminen, P. A., Rücker, G., Miettunen, J., Carpenter, J. R., & Schumacher, M. C. (2007) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2006.11.014 · Cited by 89 · cited 3×
“, 2004a,b; Williamson and Gamble, 2005), and citation bias (‘significant’ results are cited more often than studies without a ‘significant’ result, and therefore are better detectable, Kjaergard and Gluud, 2002; Nieminen et al., 2007).”
BackgroundSelf-citationPractical estimates of the effect of publication bias in meta- analysis
Duval, S., & Tweedie, R. L. (1998) · Australasian epidemiologist · Cited by 86
Citation bias of hepato-biliary randomized clinical trials
Gluud, L. L., & Gluud, C. N. (2002) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(01)00513-3 · Cited by 84 · cited 1×
“…www.biometrical-journal.com outcome reporting bias (Chan et al., 2004a,b; Williamson and Gamble, 2005), and citation bias (‘significant’ results are cited more often than studies without a ‘significant’ result, and therefore are better detectable, Kjaergard and Gluud, 2002; Nieminen et al., 2007).”
BackgroundAssessing the Amount of Heterogeneity in Random‐Effects Meta‐Analysis
Knapp, G., Biggerstaff, B. J., & Hartung, J. (2006) · Biometrical Journal · doi:10.1002/bimj.200510175 · Cited by 83 · cited 2×
“…estimating t2 by the method-of-moments-based estimator proposed by DerSimonian and Laird (1986) or, alternatively, the REML (restricted maximum likelihood) or another estimator (Sidik and Jonkman, 2005; Knapp et al., 2006; Viechtbauer, 2007), a and b can be estimated by ML (maximum likelihood).”
MethodSchwarzer, G., Antes, G., & Schumacher, M. (2002) · Statistics in Medicine · doi:10.1002/sim.1224 · Cited by 80 · cited 3×
“Beyond publication bias and related biases, we point out that small-study effects may occur for a number of other potential reasons, for example clinical heterogeneity between patients in large and small trials, a mathematical artifact (Schwarzer et al., 2002) or mere coincidence (Rothstein et al.”
BackgroundMethodSelf-citationEstimating Publication Bias in Meta-Analysis
Rust, R. T., Lehmann, D. R., & Farley, J. U. (1990) · Springer eBooks · doi:10.1007/978-3-031-88060-5_10 · Cited by 78
Easier parallel computing in R with snowfall and sfCluster
Knaus, J., Porzelius, C., Binder, H., & Schwarzer, G. (2009) · The R Journal · doi:10.32614/rj-2009-004 · Cited by 61 · cited 2×
“, 2010) and to use parallel computing (Knaus et al., 2009).”
MethodSelf-citationGalandi, D., Schwarzer, G., & Antes, G. (2006) · BMC Medical Research Methodology · doi:10.1186/1471-2288-6-30 · Cited by 59
Self-citationA test for publication bias in meta‐analysis with sparse binary data
Schwarzer, G., Antes, G., & Schumacher, M. (2007) · Statistics in Medicine · doi:10.1002/sim.2588 · Cited by 56 · cited 4×
“They were followed by modifications for the special case of binary response data (Macaskill et al., 2001; Harbord et al., 2006; Peters et al., 2006; Schwarzer et al., 2007; Rücker et al., 2008a).”
BackgroundMethodInfluentialSelf-citationThe evidence provided by a single trial is less reliable than its statistical analysis suggests
Borm, G. F., Lemmers, O., Fransen, J., & Donders, R. A. R. T. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2008.09.013 · Cited by 54 · cited 1×
“By contrast, others have advocated conducting many small studies, although their arguments have been questioned (Borm et al., 2009; Rücker et al., 2009).”
ResultCarpenter, J. R., Schwarzer, G., Rücker, G., & Künstler, R. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2008.12.002 · Cited by 51
Self-citationBlümle, A., Antes, G., Schumacher, M., Just, H., & von Elm, E. (2008) · Journal of Medical Ethics · doi:10.1136/jme.2008.024521 · Cited by 47
Assessing the Sensitivity of Meta-analysis to Selection Bias: A Multiple Imputation Approach
Carpenter, J., Rücker, G., & Schwarzer, G. (2011) · Biometrics · doi:10.1111/j.1541-0420.2010.01498.x · Cited by 39 · cited 1×
“…known that for small or heterogeneous metaanalyses all funnel plot methods work poorly with respect to both size and power (Rücker et al., 2010; Carpenter et al., 2010). r 2010 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim www.biometrical-journal.com Another limitation is that we did not…”
BackgroundSelf-citationA robust P‐value for treatment effect in meta‐analysis with publication bias
Copas, J. B., & Malley, P. F. (2008) · Statistics in Medicine · doi:10.1002/sim.3284 · Cited by 37 · cited 2×
“At roughly the same time, in a related approach which was derived from radial plots, Copas and Malley (2008) developed robust P-values adjusting for smallstudy effects.”
MethodEstimating Publication Bias in Meta-Analysis
Rust, R. T., Lehmann, D. R., & Farley, J. U. (1990) · Journal of Marketing Research · doi:10.2307/3172848 · Cited by 32
copas: An R package for Fitting the Copas Selection Model
Carpenter, J. R., Rücker, G., & Schwarzer, G. (2009) · The R Journal · doi:10.32614/rj-2009-012 · Cited by 16
Self-citationAre large trials less reliable than small trials?
Rücker, G., Schwarzer, G., Schumacher, M. C., & Carpenter, J. R. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.03.007 · Cited by 6 · cited 2×
“By contrast, others have advocated conducting many small studies, although their arguments have been questioned (Borm et al., 2009; Rücker et al., 2009).”
ResultSelf-citation
Research works printed but not linked: 1
- Rücker G.(2010). Small‐study effects and heterogeneity in meta‐analysis. Ph.D. Thesis.http://www.freidok.uni‐freiburg.de/volltexte/7486/
Cited by
How to perform a meta-analysis with R: a practical tutorial
Balduzzi, S., Rücker, G., & Schwarzer, G. (2019) · Evidence Based Mental Health · doi:10.1136/ebmental-2019-300117 · Cited by 5,827
“Different methods exist 22 ; in this paper, we report the more established trim-and-fill method 24 and the method of adjusting by regression.”
Self-citationA new improved graphical and quantitative method for detecting bias in meta-analysis
Furuya-Kanamori, L., Barendregt, J. J., & Doi, S. A. R. (2018) · International Journal of Evidence-Based Healthcare · doi:10.1097/xeb.0000000000000141 · Cited by 1,160
A meta-analysis of procedures to change implicit measures
Forscher, P. S., Lai, C. K., Axt, J. R., Ebersole, C. R., Herman, M., Devine, P. G., & Nosek, B. A. (2019) · Journal of Personality and Social Psychology · doi:10.1037/pspa0000160 · Cited by 700
The impact of study size on meta-analyses: examination of underpowered studies in Cochrane reviews
Turner, R. M., Bird, S. M., & Higgins, J. P. T. (2013) · PLoS ONE · doi:10.1371/journal.pone.0059202 · Cited by 672
Correcting for Bias in Psychology: A Comparison of Meta-Analytic Methods
Carter, E. C., Schönbrodt, F. D., Gervais, W. M., & Hilgard, J. (2019) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245919847196 · Cited by 588
Metadata
- Authors
- Rücker, G., Carpenter, J. R., & Schwarzer, G.
- Affiliations
- University of Freiburg (DE)
- Corresponding author
- Rücker, Gerta
- Venue
- Biometrical Journal
- Publisher
- Wiley
- Published
- 2011-01-14
- Volume
- 53
- Issue
- 2
- Pages
- 351-368
- ISSN
- 0323-3847 (ISSN-L)
- 1521-4036
- OpenAlex
- W2069224123
- Cited by
- 132
- References
- 76
- Citations by year
- 2026: 11 · 2025: 14 · 2024: 9 · 2023: 8 · 2022: 16 · 2021: 9 · 2020: 10 · 2019: 13 · 2018: 5 · 2017: 5 · 2016: 5 · 2015: 3 · 2014: 8 · 2013: 7 · 2012: 6
- FWCI
- 7.83
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Biometrical Journal (publishedVersion)
- PubMed (publishedVersion)
- LSHTM Research Online (London School of Hygiene and Tropical Medicine) (submittedVersion)
- Funders
- Deutsche Forschungsgemeinschaft
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Keywords
- small-study effects, publication bias, funnel plot asymmetry, selection bias, binary response data, meta-analysis bias
- MeSH terms
- Algorithms, Clinical Trials as Topic, Computer Simulation, Humans, Regression Analysis, Research Design, Time Factors, Meta-Analysis as Topic, Reproducibility of Results, Models, Statistical, Bias, Randomized Controlled Trials as Topic, Publication Bias, Evidence-Based Medicine
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.