Do pooled estimates from meta-analyses of observational epidemiology studies contribute to causal inference?

Abstract

The purpose of meta-analyses that generate pooled risk estimates is generally to inform causal inference, compiling evidence to address an aetiological question. Logically, drawing on information from all relevant studies is likely to be superior to relying on any single study. Meta-analysis applies an objective, quantitative method to integrate evidence across studies. Objectivity in identifying studies, evaluating their relevance, assessing the quality of the methods and extracting results has clear benefits over informal, subjective approaches. It guards against arbitrary selection of studies and allows for replication of at least parts of the review protocol (eg, literature identification). Clearly, meta-analysis has great appeal in the research community. A cursory examination of PubMed searching on ‘epidemiology’ and ‘meta-analysis’ yielded the expected pattern of proliferation—fewer than 100 publications per year prior to 1990, around 400 per year in 2000, 800 per year in 2005, 2000 per year in 2010, 5600 in 2015 and over 6000 per year starting in 2019. In the 1990s, there was intense debate over the merits and demerits of meta-analysis in observational epidemiology, with some arguing for abandoning this approach entirely1 2 and others expressing reservations based largely on the heterogeneity of study methods.3–6 The role of meta-analysis in causal inference specifically also has been considered.7 8 Interestingly, the debate appeared to end over 20 years ago without a clear resolution, yet meta-analysis became the default approach to summarising and evaluating evidence. We suggest that the debate should be reopened and make the case that the negative features of this approach often outweigh its benefits. The primary competitor to generating pooled estimates through meta-analysis is some variant of expert review. Obviously, pooled estimates through meta-analysis and expert reviews may be divergent, deviating in either direction—seemingly clear evidence for an effect based on meta-analysis that is not …