Commentary: How to Report Instrumental Variable Analyses (Suggestions Welcome)

Abstract

Instrumental variable (IV) methods are becoming mainstream in comparative effectiveness research,1 but IV methods are radically different from traditional epidemiologic methods. The goal of IV methods is to eliminate confounding without ever measuring the confounders. This apparent miracle can be achieved only when four conditions are met (see below).2,3 Here we suggest a checklist for investigators who use IV methods. Like others before,4 we hope this step-by-step guide will improve the reporting of IV estimates and increase the transparency of the underlying assumptions. Our discussion focuses on reports of causal effects of medical interventions and is informed by two papers by Davies et al5,6 that appear in this issue of EPIDEMIOLOGY: an application that estimates the causal effects of cyclooxygenase-2 (COX-2) selective versus nonselective nonsteroidal anti-inflammatory drugs (NSAIDs), and a literature review of IV papers that describes their use, and perhaps misuse, in epidemiology. We supplement this review with additional information from our own review of IV analyses of observational studies with a relatively well-defined medical intervention. Details of our review can be found in the online supplement (https://links.lww.com/EDE/A664). IV methods require a variable—the “instrument”—that meets the three so-called instrumental conditions: (1) the instrument is associated with the treatment, (2) the instrument does not affect the outcome except through treatment (also known as the exclusion restriction assumption), and (3) the instrument does not share any causes with the outcome. An example of such an instrument is the randomization indicator in double-blind randomized experiments.2 Davies et al6 summarize instruments that have been proposed in epidemiologic studies, including the physician preference type they use themselves.5 Unfortunately, no variable can be proved to be an instrument in observational studies because only condition (1) can be empirically verified. We outline some steps for reporting IV analyses based on the variables proposed as instruments (Figure) and discuss how these steps have been reported in previous studies (Table). A detailed specification of how IV methods should be implemented is beyond the scope of this commentary.FIGURE: Flowchart for reporting of IV analyses with a single, non–time-varying, dichotomous instrument.TABLE: Proportions Reporting According to Suggested Guidelines in a Review of IV Publications Assessing Effects of Medical Interventions (n = 81)The first step in our reporting flowchart is to empirically verify condition (1). When the association between the proposed instrument and the treatment is weak, the proposed “weak” instrument may amplify biases due to small violations of conditions (2) or (3), producing very biased effect estimates.7,8 Alternatively, if the proposed instrument is very strong, it may be more likely to violate conditions (2) or (3); in the extreme, a perfect correlation between the proposed instrument and the observational treatment implies the proposed instrument is associated with the same set of (possibly unmeasured) confounders as the treatment.9 Most prior studies have evaluated the strength of their proposed instrument via risk differences, odds ratios, or partial R2 or F statistics like Davies et al5 did and several authors have recommended.6,8 (In contrast, the use of R2 or F statistics as effect measures is subject to many study-specific artifacts and should generally be avoided.10,11) Unlike condition (1), conditions (2) and (3) are not empirically verifiable: they are assumptions. Therefore, the next step is to use subject-matter knowledge to build a case for why the proposed instrument may be reasonably assumed to meet each of these two conditions; this is similar to how one would use subject-matter knowledge to justify the no unmeasured confounding assumption of traditional methods. When dichotomizing a polytomous or continuous treatment, condition (2) may hold for the original treatment, but not for its dichotomous version. Investigators can provide some support for condition (3) by showing the proposed instrument is not associated with measured confounders.6 Although any associations found with measured covariates could be adjusted for by modeling (see below), associations with measured confounders may raise concerns over associations with unmeasured confounders. Davies et al5 provide arguments to support conditions (2) and (3) when physician’s preference is proposed as an instrument; other authors have provided counterarguments.2 In our review, we notice a tendency for conditions (2) and (3) to be treated as a single condition. Although these conditions are statistically similar, it is important to consider them separately in order to incorporate subject-matter knowledge in discussions about their validity and in decisions on adjustment. For example, Davies et al5 found some evidence that the proposed instrument is ass