The trim-and-fill method for publication bias: practical guidelines and recommendations based on a large database of meta-analyses

Abstract

Publication bias is a type of systematic error when synthesizing evidence that cannot represent the underlying truth. Clinical studies with favorable results are more likely published and thus exaggerate the synthesized evidence in meta-analyses. The trim-and-fill method is a popular tool to detect and adjust for publication bias. Simulation studies have been performed to assess this method, but they may not fully represent realistic settings about publication bias. Based on real-world meta-analyses, this article provides practical guidelines and recommendations for using the trim-and-fill method. We used a worked illustrative example to demonstrate the idea of the trim-and-fill method, and we reviewed three estimators (R0, L0, and Q0) for imputing missing studies. A resampling method was proposed to calculate P values for all 3 estimators. We also summarized available meta-analysis software programs for implementing the trim-and-fill method. Moreover, we applied the method to 29,932 meta-analyses from the Cochrane Database of Systematic Reviews, and empirically evaluated its overall performance. We carefully explored potential issues occurred in our analysis. The estimators L0 and Q0 detected at least one missing study in more meta-analyses than R0, while Q0 often imputed more missing studies than L0. After adding imputed missing studies, the significance of heterogeneity and overall effect sizes changed in many meta-analyses. All estimators generally converged fast. However, L0 and Q0 failed to converge in a few meta-analyses that contained studies with identical effect sizes. Also, P values produced by different estimators could yield different conclusions of publication bias significance. Outliers and the pre-specified direction of missing studies could have influential impact on the trim-and-fill results. Meta-analysts are recommended to perform the trim-and-fill method with great caution when using meta-analysis software programs. Some default settings (e.g., the choice of estimators and the direction of missing studies) in the programs may not be optimal for a certain meta-analysis; they should be determined on a case-by-case basis. Sensitivity analyses are encouraged to examine effects of different estimators and outlying studies. Also, the trim-and-fill estimator should be routinely reported in meta-analyses, because the results depend highly on it.