CoChat
Sign Up
Cited by 198Advanced Causal Inference TechniquesMeta-analysis and systematic reviewsStatistical Methods in Epidemiology

Commentary: How to Report Instrumental Variable Analyses (Suggestions Welcome)

Swanson, Sonja A., Hernán, Miguel A.·2013·Epidemiology
DOI 10.1097/ede.0b013e31828d0590
Open this paper in CoChat — free account

Abstract

Instrumental variable (IV) methods are becoming mainstream in comparative effectiveness research,1 but IV methods are radically different from traditional epidemiologic methods. The goal of IV methods is to eliminate confounding without ever measuring the confounders. This apparent miracle can be achieved only when four conditions are met (see below).2,3 Here we suggest a checklist for investigators who use IV methods. Like others before,4 we hope this step-by-step guide will improve the reporting of IV estimates and increase the transparency of the underlying assumptions. Our discussion focuses on reports of causal effects of medical interventions and is informed by two papers by Davies et al5,6 that appear in this issue of EPIDEMIOLOGY: an application that estimates the causal effects of cyclooxygenase-2 (COX-2) selective versus nonselective nonsteroidal anti-inflammatory drugs (NSAIDs), and a literature review of IV papers that describes their use, and perhaps misuse, in epidemiology. We supplement this review with additional information from our own review of IV analyses of observational studies with a relatively well-defined medical intervention. Details of our review can be found in the online supplement (https://links.lww.com/EDE/A664). IV methods require a variable—the “instrument”—that meets the three so-called instrumental conditions: (1) the instrument is associated with the treatment, (2) the instrument does not affect the outcome except through treatment (also known as the exclusion restriction assumption), and (3) the instrument does not share any causes with the outcome. An example of such an instrument is the randomization indicator in double-blind randomized experiments.2 Davies et al6 summarize instruments that have been proposed in epidemiologic studies, including the physician preference type they use themselves.5 Unfortunately, no variable can be proved to be an instrument in observational studies because only condition (1) can be empirically verified. We outline some steps for reporting IV analyses based on the variables proposed as instruments (Figure) and discuss how these steps have been reported in previous studies (Table). A detailed specification of how IV methods should be implemented is beyond the scope of this commentary.FIGURE: Flowchart for reporting of IV analyses with a single, non–time-varying, dichotomous instrument.TABLE: Proportions Reporting According to Suggested Guidelines in a Review of IV Publications Assessing Effects of Medical Interventions (n = 81)The first step in our reporting flowchart is to empirically verify condition (1). When the association between the proposed instrument and the treatment is weak, the proposed “weak” instrument may amplify biases due to small violations of conditions (2) or (3), producing very biased effect estimates.7,8 Alternatively, if the proposed instrument is very strong, it may be more likely to violate conditions (2) or (3); in the extreme, a perfect correlation between the proposed instrument and the observational treatment implies the proposed instrument is associated with the same set of (possibly unmeasured) confounders as the treatment.9 Most prior studies have evaluated the strength of their proposed instrument via risk differences, odds ratios, or partial R2 or F statistics like Davies et al5 did and several authors have recommended.6,8 (In contrast, the use of R2 or F statistics as effect measures is subject to many study-specific artifacts and should generally be avoided.10,11) Unlike condition (1), conditions (2) and (3) are not empirically verifiable: they are assumptions. Therefore, the next step is to use subject-matter knowledge to build a case for why the proposed instrument may be reasonably assumed to meet each of these two conditions; this is similar to how one would use subject-matter knowledge to justify the no unmeasured confounding assumption of traditional methods. When dichotomizing a polytomous or continuous treatment, condition (2) may hold for the original treatment, but not for its dichotomous version. Investigators can provide some support for condition (3) by showing the proposed instrument is not associated with measured confounders.6 Although any associations found with measured covariates could be adjusted for by modeling (see below), associations with measured confounders may raise concerns over associations with unmeasured confounders. Davies et al5 provide arguments to support conditions (2) and (3) when physician’s preference is proposed as an instrument; other authors have provided counterarguments.2 In our review, we notice a tendency for conditions (2) and (3) to be treated as a single condition. Although these conditions are statistically similar, it is important to consider them separately in order to incorporate subject-matter knowledge in discussions about their validity and in decisions on adjustment. For example, Davies et al5 found some evidence that the proposed instrument is ass

Integrity signals

Registry

Crossref · 10.1097/ede.0b013e31828d0590

Cites

29 of 29 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Instrumental variable methods have not yet been widely used to evaluate public health interventions because it can be difficult to find suitable instruments and to demonstrate convincingly, using theory or data, that they meet the second and third conditions.

    “IV methods have not yet been widely used to evaluate public health interventions because it can be difficult to find suitable instruments and to demonstrate convincingly, using theory or data, that they meet the second and third conditions above (35, 71).”

    From Natural Experiments: An Overview of Methods, Approaches, and Contributions to Public Health Intervention Research (2017) · cites this paper as [71]

Cites

Flagged first, then by citations

1 of the 30 references couldn’t be matched to a known paper.

Cited by

Sorted by citations

Related

Authors
Swanson, S. A., & Hernán, M. A.
Affiliations
Harvard University (US)
Corresponding author
Swanson, Sonja A.
ORCID
Swanson, Sonja A.
Venue
Epidemiology
Publisher
Ovid Technologies (Wolters Kluwer Health)
Published
2013-05
Volume
24
Issue
3
Pages
370-374
ISSN
1044-3983 (ISSN-L)
DOI
10.1097/ede.0b013e31828d0590
OpenAlex
W2330290021
Cited by
197
References
30
Citations by year
2026: 8 · 2025: 15 · 2024: 15 · 2023: 12 · 2022: 23 · 2021: 14 · 2020: 9 · 2019: 18 · 2018: 16 · 2017: 15 · 2016: 19 · 2015: 21 · 2014: 10 · 2013: 1
FWCI
14.82
Type
journal article
Language
en
Open access
closed
Locations
  • Epidemiology (publishedVersion)
  • PubMed (publishedVersion)
Topics
  • Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
Keywords
instrumental variables, reporting guidelines, confounding, exclusion restriction, weak instruments, reporting checklist, observational studies, medical interventions, COX-2 inhibitors, nonsteroidal anti-inflammatory drugs, causal effect estimation
MeSH terms
Data Interpretation, Statistical, Female, Gastrointestinal Diseases, Humans, Male, Myocardial Infarction, Models, Statistical, Epidemiologic Research Design, Causality, Confounding Factors, Epidemiologic, Cyclooxygenase 2 Inhibitors, Research Report

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review