Generalized synthesis of evidence and the threat of dissemination bias

Abstract

Assessment of the potential impact of dissemination bias is necessary for meta-analysis. When evidence is available from studies of different designs, the different study types may be affected by dissemination bias to differing degrees. The evidence relating to electronic fetal heart rate monitoring (EFM) for preventing perinatal mortality is used to explore the feasibility of carrying out a dissemination bias assessment in a generalized synthesis of evidence (gse) framework. Visual inspection of funnel plots, statistical tests, and methods to "adjust" the results of a meta-analysis are all used in an extensive sensitivity analysis. The potential impact of dissemination bias on gse models synthesizing all the evidence together is also reported. Detailed consideration is given to the influence of meta-analysis model choice, and outcome scale used. Using the risk difference scale, funnel plots of the observational studies appeared highly asymmetric. However, further explorations show these conclusions are not robust over use of different outcome measures or different meta-analysis models. Researchers should be aware that dissemination bias may affect different sources of evidence differently. Although assessments such as those described here are recommended, awareness of their lack of robustness to outcome scale and model choice is important. Further research into methods to assess dissemination bias that are invariant to these factors is needed.