How to compare instrumental variable and conventional regression analyses using negative controls and bias plots
Open accessAbstract
There is increasing interest in the use of instrumental variable analysis to overcome unmeasured confounding in observational pharmacoepidemiological studies. This is partly because instrumental variable analyses are potentially less biased than conventional regression analyses. However, instrumental variable analyses are less precise, and regulators and clinicians find it difficult to interpret conflicting evidence from instrumental variable compared with conventional regression analyses. In this paper, we describe three techniques to assess which approach (instrumental variable versus conventional regression analyses) is least biased. These techniques are negative control outcomes, negative control populations and tests of covariate balance. We illustrate these methods using an analysis of the effects of smoking cessation therapies (varenicline) prescribed in primary care.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The use of negative control outcomes or negative control populations might also allow evaluation of the exclusion restriction assumption.
“The use of negative control outcomes or negative control populations might also allow evaluation of this assumption.70”
From Strengthening the Reporting of Observational Studies in Epidemiology using Mendelian Randomisation (STROBE-MR): explanation and elaboration. (2021) · cites this paper as [70]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Identification of Causal Effects Using Instrumental Variables
Angrist, J. D., Imbens, G. W., & Rubin, D. B. (1996) · Journal of the American Statistical Association · doi:10.1080/01621459.1996.10476902 · Cited by 4,230 · see citing works
Identification and estimation of local average treatment effects
Imbens, G. W., & Angrist, J. D. (1994) · Econometrica · doi:10.2307/2951620 · Cited by 4,101 · see citing works
Identification and Estimation of Local Average Treatment Effects
Angrist, J. D., & Imbens, G. W. (1995) · National Bureau of Economic Research · doi:10.3386/t0118 · Cited by 2,898 · see citing works
Angrist, J. D., & Krueger, A. B. (2001) · Journal of Economic Perspectives · doi:10.1257/jep.15.4.69 · Cited by 2,355 · see citing works
An introduction to instrumental variables for epidemiologists
Greenland, S. (2000) · International Journal of Epidemiology · doi:10.1093/ije/29.4.722 · Cited by 1,156 · see citing works
Instruments for Causal Inference: An Epidemiologist's Dream?
Hernán, M. A., & Robins, J. M. (2006) · Epidemiology · doi:10.1097/01.ede.0000222409.00878.37 · Cited by 961 · see citing works
Risk of Death in Elderly Users of Conventional vs. Atypical Antipsychotic Medications
Wang, P. S., Schneeweiss, S., Avorn, J., Fischer, M. A., Mogun, H., Solomon, D. H., & Brookhart, M. A. (2005) · New England Journal of Medicine · doi:10.1056/nejmoa052827 · Cited by 959 · see citing works
A weak instrument F -test in linear IV models with multiple endogenous variables
Sanderson, E. C. M., & Windmeijer, F. A. G. (2015) · Journal of Econometrics · doi:10.1016/j.jeconom.2015.06.004 · Cited by 752 · see citing works
Self-citationInstrumental variable methods for causal inference
Baiocchi, M., Cheng, J., & Small, D. S. (2014) · Statistics in Medicine · doi:10.1002/sim.6128 · Cited by 687 · see citing works
Aprotinin during Coronary-Artery Bypass Grafting and Risk of Death
Schneeweiß, S., Seeger, J. D., Landon, J. E., & Walker, A. M. (2008) · New England Journal of Medicine · doi:10.1056/nejmoa0707571 · Cited by 356 · see citing works
Brookhart, M. A., Wang, P. S., Solomon, D. H., & Schneeweiß, S. (2006) · Epidemiology · doi:10.1097/01.ede.0000193606.58671.c5 · Cited by 341 · see citing works
Brookhart, M. A., & Schneeweiß, S. (2007) · The International Journal of Biostatistics · doi:10.2202/1557-4679.1072 · Cited by 221 · see citing works
Rassen, J. A., Brookhart, M. A., Glynn, R. J., Mittleman, M. A., & Schneeweiss, S. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2008.12.005 · Cited by 164 · see citing works
Instrumental Variable Analysis for Estimation of Treatment Effects With Dichotomous Outcomes
Rassen, J. A., Schneeweiß, S., Glynn, R., Mittleman, M. A., & Brookhart, M. A. (2008) · American Journal of Epidemiology · doi:10.1093/aje/kwn299 · Cited by 164 · see citing works
Instrumental Variable Estimators for Binary Outcomes
Clarke, P. S., & Windmeijer, F. A. G. (2012) · Journal of the American Statistical Association · doi:10.1080/01621459.2012.734171 · Cited by 149 · see citing works
Self-citationIssues in the Reporting and Conduct of Instrumental Variable Studies
Davies, N. M., Smith, G. D., Windmeijer, F. A. G., & Martin, R. M. (2013) · Epidemiology · doi:10.1097/ede.0b013e31828abafb · Cited by 146 · see citing works
Self-citationRassen, J. A., Brookhart, M. A., Glynn, R. J., Mittleman, M. A., & Schneeweiß, S. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2008.12.006 · Cited by 126 · see citing works
Thomas, K. H., Martin, R. M., Davies, N. M., Metcalfe, C., Windmeijer, F., & Gunnell, D. (2013) · BMJ · doi:10.1136/bmj.f5704 · Cited by 119 · see citing works
Self-citationAdjusting for Unmeasured Confounders in Pharmacoepidemiologic Claims Data Using External Information
Schneeweiß, S., Glynn, R. J., Tsai, E. H., Avorn, J., & Solomon, D. H. (2004) · Epidemiology · doi:10.1097/01.ede.0000147164.11879.b5 · Cited by 111 · see citing works
Lawlor, D. A. (2004) · International Journal of Epidemiology · doi:10.1093/ije/dyh124 · Cited by 109 · see citing works
Setoguchi, S., Wang, P. S., Brookhart, M. A., Canning, C. F., Kaci, L., & Schneeweiß, S. (2008) · Journal of the American Geriatrics Society · doi:10.1111/j.1532-5415.2008.01839.x · Cited by 100 · see citing works
Negative Control Exposures in Epidemiologic Studies
Smith, G. D. (2012) · Epidemiology · doi:10.1097/ede.0b013e318245912c · Cited by 88 · cited 1× · see citing works
“the validity of instrumental variables for dealing with confounding by indication.(22)”
BackgroundToward a Clearer Portrayal of Confounding Bias in Instrumental Variable Applications
Jackson, J. W., & Swanson, S. A. (2015) · Epidemiology · doi:10.1097/ede.0000000000000287 · Cited by 74 · see citing works
A weak instrument F-test in linear IV models with multiple endogenous variables
Sanderson, E., & Windmeijer, F. (2026) · doi:10.1920/wp.cem.2015.3115 · Cited by 72 · see citing works
Self-citationSelecting on Treatment: A Pervasive Form of Bias in Instrumental Variable Analyses
Swanson, S. A., Robins, J. M., Miller, M., & Hernán, M. Á. (2015) · American Journal of Epidemiology · doi:10.1093/aje/kwu284 · Cited by 69 · see citing works
The Role of Known Effects in Observational Studies
Rosenbaum, P. R. (1989) · Biometrics · doi:10.2307/2531497 · Cited by 68 · see citing works
Davies, N. M., Gunnell, D., Thomas, K. H., Metcalfe, C. D., Windmeijer, F. A. G., & Martin, R. M. (2013) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2013.06.008 · Cited by 54 · see citing works
Self-citationIdentification of causal effects on binary outcomes using structural mean models
Clarke, P. S., & Windmeijer, F. A. G. (2010) · Biostatistics · doi:10.1093/biostatistics/kxq024 · Cited by 49 · see citing works
Self-citationDavies, N. M., Smith, G. D., Windmeijer, F. A. G., & Martin, R. M. (2013) · Epidemiology · doi:10.1097/ede.0b013e318289e024 · Cited by 40 · see citing works
Self-citationA weak instrument F-test in linear IV models with multiple endogenous variablesPreprint
Sanderson, E., & Windmeijer, F. (n.d.) · RePEc: Research Papers in Economics · Cited by 40 · see citing works
Self-citationHuybrechts, K. F., Brookhart, M. A., Rothman, K. J., Silliman, R. A., Gerhard, T., Crystal, S., & Schneeweiß, S. (2011) · American Journal of Epidemiology · doi:10.1093/aje/kwr213 · Cited by 25 · see citing works
Davies, N. M. (2015) · Epidemiology · doi:10.1097/ede.0000000000000302 · Cited by 20 · see citing works
Self-citationKollhorst, B., Abrahamowicz, M., & Pigeot, I. (2015) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.08.008 · Cited by 6 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Skrivankova, V. W., Richmond, R. C., Woolf, B. A. R., Davies, N. M., Swanson, S. A., VanderWeele, T. J., Timpson, N. J., Higgins, J. P. T., Dimou, N., Langenberg, C., et al. (2021) · BMJ · doi:10.1136/bmj.n2233 · Cited by 1,648 · see citing works
Self-citationLabrecque, J., & Swanson, S. A. (2018) · Current Epidemiology Reports · doi:10.1007/s40471-018-0152-1 · Cited by 190 · see citing works
Davies, N. M., Hill, W. D., Anderson, E. L., Sanderson, E., Deary, I. J., & Davey Smith, G. (2019) · eLife · doi:10.7554/elife.43990 · Cited by 135 · see citing works
Self-citationYang, Q., Sanderson, E., Tilling, K., Borges, M. C., & Lawlor, D. A. (2019) · doi:10.1101/19009605 · Cited by 85 · see citing works
- Authors
- Davies, N. M., Thomas, K. H., Taylor, A. E., Taylor, G. M., Martin, R. M., Munafò, M. R., & Windmeijer, F.
- Affiliations
- MRC Integrative Epidemiology Unit (GB)
- UK Centre for Tobacco & Alcohol Studies (GB)
- Medical Research Council Integrative Epidemiology Unit,
- University of Bristol (GB)
- Corresponding author
- Davies, Neil M
- Venue
- International Journal of Epidemiology
- Publisher
- Oxford University Press (OUP)
- Published
- 2017-04-07
- Volume
- 46
- Issue
- 6
- Pages
- 2067-2077
- ISSN
- 0300-5771 (ISSN-L)
- 1464-3685
- PMC
- PMC5837536
- OpenAlex
- W2587176846
- Cited by
- 55
- References
- 33
- Citations by year
- 2026: 4 · 2025: 9 · 2024: 10 · 2023: 7 · 2022: 6 · 2021: 6 · 2020: 3 · 2019: 4 · 2018: 5 · 2017: 1
- FWCI
- 2.47
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- International Journal of Epidemiology (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Pure (University of Bath) (publishedVersion, cc-by)
- Explore Bristol Research (publishedVersion, cc-by)
- Bristol Research (University of Bristol) (submittedVersion, cc-by)
- Funders
- National Institutes of Health
- Medical Research Council (MR/N01006X/1)
- Cancer Research UK (C18281/A19169)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Keywords
- instrumental variables, negative control outcomes, covariate balance, unmeasured confounding, smoking cessation therapy, varenicline, pharmacoepidemiology, primary care prescribing
- MeSH terms
- Data Interpretation, Statistical, Humans, Regression Analysis, Bias, Causality, Confounding Factors, Epidemiologic, Case-Control Studies, Outcome Assessment, Health Care, Pharmacoepidemiology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
