Assessing the potential for bias in meta-analysis due to selective reporting of subgroup analyses within studies

Abstract

Evidence regarding the potential benefits of a particular health care intervention is often available from a variety of disparate sources. However, formal synthesis of such evidence has traditionally concentrated almost exclusively on that derived from randomized studies, although for a range of conditions the randomized evidence will be less than adequate due to economic, organizational or ethical considerations. In such situations a formal synthesis of the evidence that is available from observational studies can be valuable whilst awaiting higher quality evidence from randomized trials. Consideration of randomized studies alone may be appropriate when assessing the efficacy of an intervention, but assessment of the effectiveness of such an intervention within a more general target population may be improved by consideration of evidence from non-randomized studies as well. Standard meta-analysis methods may allow for both within- and between-study heterogeneity; however when multiple sources of evidence are considered an extra level of complexity is introduced, namely study type. One possible solution to the problem of making inferences, particularly regarding an overall population effect, in such situations is to model the heterogeneity, both quantitative and qualitative, using a Bayesian hierarchical model. The hierarchical nature of such models specifically allows for the quantitative within and between sources of heterogeneity, whilst the Bayesian approach can accommodate a priori beliefs regarding qualitative differences between the various sources of evidence. The use of such methods in practice is illustrated in the context of screening for breast cancer; in this example evidence is available from both randomized clinical trials and observational studies. A particular appeal of a Bayesian approach for this type of problem lies in the prediction of future benefits likely to be observed in a target population. This approach to health service monitoring in general is discussed.