Sensitivity Analysis in Observational Research: Introducing the E-Value
Open accessThe DOI matches its Crossref record; 58 of the 59 references it cites were found in a registry or an index.
Abstract
Sensitivity analysis is useful in assessing how robust an association is to potential unmeasured or uncontrolled confounding. This article introduces a new measure called the "E-value," which is related to the evidence for causality in observational studies that are potentially subject to confounding. The E-value is defined as the minimum strength of association, on the risk ratio scale, that an unmeasured confounder would need to have with both the treatment and the outcome to fully explain away a specific treatment-outcome association, conditional on the measured covariates. A large E-value implies that considerable unmeasured confounding would be needed to explain away an effect estimate. A small E-value implies little unmeasured confounding would be needed to explain away an effect estimate. The authors propose that in all observational studies intended to produce evidence for causality, the E-value be reported or some other sensitivity analysis be used. They suggest calculating the E-value for both the observed association estimate (after adjustments for measured confounders) and the limit of the confidence interval closest to the null. If this were to become standard practice, the ability of the scientific community to assess evidence from observational studies would improve considerably, and ultimately, science would be strengthened.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Quantitative bias analyses, such as the calculation of E-values to estimate the strength of unmeasured confounding required to explain away an observed association, provide useful context for the robustness of findings.64”
From Artificial Intelligence in Oncology: Practical Applications Across Clinical Care, Scholarship, and Translation. (2026) · cites this paper as [64]
The cited source says what the claim attributes to it.
In the citing paper
“We performed the sensitivity analysis using the E value [29] to evaluate the robustness of our findings to potential unmeasured confounding, with the calculation based on the observed odds ratio (OR) point estimate for rare outcome (<15%) or the square root of OR otherwise.”
The cited source says what the claim attributes to it.
In the citing paper
“To further assess the robustness of our findings to potential unmeasured confounding, we conducted an E-value analysis, a quantitative method for sensitivity analysis in observational research [31].”
From Acute Respiratory Tract Infection and Sudden Sensorineural Hearing Loss: A Multinational Cohort Study (2025) · cites this paper as [31]
In the citing paper
“We also computed E-values: indicators of how robust findings are to potential unmeasured confounding”
From Leading a meaningful life at older ages and its relationship with social engagement, prosperity, health, biology, and time use (2019) · cites this paper as [12]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009) · doi:10.1002/9780470743386 · Cited by 14,225
The environment and disease: association or causation?
Hill, A. B. (2015) · Journal of the Royal Society of Medicine · doi:10.1177/0141076814562718 · Cited by 7,285
The Environment and Disease: Association or Causation?
Hill, A. B. (1965) · Proceedings of the Royal Society of Medicine · doi:10.1177/003591576505800503 · Cited by 6,974 · cited 1×
“Sensitivity analysis makes clear quantitatively why Bradford Hill's criterion of “strength of association” (12) is important in establishing that a given association is, in fact, causal.”
BackgroundThe ASA Statement on p -Values: Context, Process, and Purpose
Wasserstein, R. L., & Lazar, N. A. (2016) · The American Statistician · doi:10.1080/00031305.2016.1154108 · Cited by 6,306 · cited 2×
“While potentially subject to mis-use and misinterpretation (1,5-12), the p-value can be informative as a continuous measure of evidence as to whether an association is present.”
BackgroundCausal Inference for Statistics, Social, and Biomedical Sciences: An Introduction
Imbens, G. W., & Rubin, D. B. (2015) · doi:10.1017/cbo9781139025751 · Cited by 4,210
Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations
Greenland, S., Senn, S. J., Rothman, K. J., Carlin, J. B., Poole, C., Goodman, S. N., & Altman, D. G. (2016) · European Journal of Epidemiology · doi:10.1007/s10654-016-0149-3 · Cited by 3,200
Explanation in Causal Inference: Methods for Mediation and Interaction
VanderWeele, T. J. (2015) · Cited by 2,217
Self-citationSmithson, M. (2000) · SAGE Publications Ltd eBooks · doi:10.4135/9781446218525 · Cited by 1,871
Fundamentals of Biostatistics.
Cornell, R. G., & Rosner, B. (1987) · Journal of the American Statistical Association · doi:10.2307/2289416 · Cited by 1,603
Breastfeeding and Maternal and Infant Health Outcomes In Developed Countries
(2007) · AAP Grand Rounds · doi:10.1542/gr.18-2-15 · Cited by 1,540 · cited 2×
“A study by the Agency for Healthcare Research and Quality (42) reported the association between breastfeeding and childhood leukemia as RR=0.”
MethodInfluentialSterne, J. A. C. (2001) · BMJ · doi:10.1136/bmj.322.7280.226 · Cited by 1,466
Toward Evidence-Based Medical Statistics. 1: The P Value Fallacy
Goodman, S. N. (1999) · Annals of Internal Medicine · doi:10.7326/0003-4819-130-12-199906150-00008 · Cited by 1,185 · cited 2×
“While potentially subject to mis-use and misinterpretation (1,5-12), the p-value can be informative as a continuous measure of evidence as to whether an association is present.”
BackgroundDesign of Observational Studies
Rosenbaum, P. R. (2010) · Springer Series in Statistics · doi:10.1007/978-1-4419-1213-8 · Cited by 1,150
Rosenbaum, P. R., & Rubin, D. B. (1983) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1983.tb01242.x · Cited by 1,124 · cited 1×
“Such assumptions often stipulate that the unmeasured confounder is binary (22, 24, 26) or that only 1 unmeasured confounder exists (24–28), or that there is no interaction between the effects of the unmeasured confounder and of the treatment on the outcome (25–28).”
BackgroundSmoking and Lung Cancer: Recent Evidence and a Discussion of Some Questions
(1959) · JNCI: Journal of the National Cancer Institute · doi:10.1093/jnci/22.1.173 · Cited by 841 · cited 1×
“Such assumptions often stipulate that the unmeasured confounder is binary (22, 24, 26) or that only 1 unmeasured confounder exists (24–28), or that there is no interaction between the effects of the unmeasured confounder and of the treatment on the outcome (25–28).”
BackgroundTaubes, G. (1995) · Science · doi:10.1126/science.7618077 · Cited by 841 · cited 1×
“Observational research is sometimes criticized on the grounds that its results constantly are being overturned (44).”
BackgroundAssessing the sensitivity of regression results to unmeasured confounders in observational studies
Lin, D. Y., Psaty, B. M., & Kronmal, R. A. (1998) · Biometrics · doi:10.2307/2533848 · Cited by 742 · cited 2×
“Such assumptions often stipulate that the unmeasured confounder is binary (22,24,26), or that there is only one unmeasured confounder (24-28) or that there is no interaction between the effects of the unmeasured confounder and the treatment on the outcome (25-28).”
BackgroundMethodEVIDENCE FOR PROTECTION BY BREAST-FEEDING AGAINST INFANT DEATHS FROM INFECTIOUS DISEASES IN BRAZIL
Victora, C., Vaughan, J. P., Lombardi, C., Fuchs, S. C., Gigante, L., Smith, P., Nobre, L., Teixeira, A., Moreira, L., & Barros, F. (1987) · The Lancet · doi:10.1016/s0140-6736(87)90902-0 · Cited by 709 · cited 4×
“9 from the study by Victora and colleagues (23).”
BackgroundInfluentialRothman, K. J. (1988) · Cited by 690
A Dirty Dozen: Twelve P-Value Misconceptions
Goodman, S. (2008) · Seminars in Hematology · doi:10.1053/j.seminhematol.2008.04.003 · Cited by 680 · cited 2×
“While potentially subject to mis-use and misinterpretation (1,5-12), the p-value can be informative as a continuous measure of evidence as to whether an association is present.”
BackgroundMeta-analysis of screening and diagnostic tests.
Hasselblad, V., & Hedges, L. V. (1995) · Psychological Bulletin · doi:10.1037/0033-2909.117.1.167 · Cited by 579
Sensitivity Analysis Without Assumptions
Ding, P., & VanderWeele, T. J. (2016) · Epidemiology · doi:10.1097/ede.0000000000000457 · Cited by 570 · cited 8×
“A potential remedy would be to provide a large table with different values of RRUD and RREU, including some which are large, to give readers and researchers a sense as to how sensitive the conclusions are to potential unmeasured confounders (37).”
BackgroundMethodInfluentialSelf-citation(2009) · doi:10.1007/978-0-387-87959-8 · Cited by 554
Sensitivity to Exogeneity Assumptions in Program Evaluation
Imbens, G. W. (2003) · American Economic Review · doi:10.1257/000282803321946921 · Cited by 529 · cited 2×
“Such assumptions often stipulate that the unmeasured confounder is binary (22,24,26), or that there is only one unmeasured confounder (24-28) or that there is no interaction between the effects of the unmeasured confounder and the treatment on the outcome (25-28).”
BackgroundMethodMultiple-bias modelling for analysis of observational data
Greenland, S. (2005) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/j.1467-985x.2004.00349.x · Cited by 497
Rosenbaum, P. R. (2002) · Springer Series in Statistics · doi:10.1007/978-1-4757-3692-2_1 · Cited by 484
VanderWeele, T. J., & Arah, O. A. (2011) · Epidemiology · doi:10.1097/ede.0b013e3181f74493 · Cited by 452 · cited 1×
“However, information about the distribution of the unmeasured confounder, when available, may be helpful in sensitivity analysis, and techniques are available to use such information (21, 25, 29).”
MethodSelf-citationAntidepressant Use in Pregnancy and the Risk of Cardiac Defects
Huybrechts, K. F., Palmsten, K., Avorn, J., Cohen, L. S., Holmes, L. B., Franklin, J. M., Mogun, H., Levin, R., Kowal, M., Setoguchi, S., et al. (2014) · New England Journal of Medicine · doi:10.1056/nejmoa1312828 · Cited by 402
Randomization, Statistics, and Causal Inference
Greenland, S. (1990) · Epidemiology · doi:10.1097/00001648-199011000-00003 · Cited by 385 · cited 2×
“, that some third factor related to both the treatment and the outcome might explain their association, with no true causal effect(12-22).”
BackgroundRobins, J. M., Rotnitzky, A., & Scharfstein, D. O. (2000) · The IMA Volumes in Mathematics and its Applications · doi:10.1007/978-1-4612-1284-3_1 · Cited by 363 · cited 1×
“Numerous sensitivity analysis techniques have been developed for different statistical models (14-22,24-40).”
MethodApplying Quantitative Bias Analysis to Epidemiologic Data
Fink, A. K., Fox, M. P., & Lash, T. L. (2009) · Statistics for Biology and Health · doi:10.1007/b97920 · Cited by 343
Wasserstein, R., & Lazar, N. A. (2016) · Figshare · doi:10.6084/m9.figshare.3085162 · Cited by 336
SMOKING AND DEATH RATES—REPORT ON FORTY-FOUR MONTHS OF FOLLOW-UP OF 187,783 MEN: I. TOTAL MORTALITY
Hammond, E. C. (1958) · Journal of the American Medical Association · doi:10.1001/jama.1958.02990100047009 · Cited by 320
Sifting the evidence—what's wrong with significance tests?
Sterne, J. A. C., & Smith, G. D. (2001) · Physical Therapy · doi:10.1093/ptj/81.8.1464 · Cited by 301
The ASA's statement on p-values: context, process, and purpose
Wasserstein, R., & Lazar, N. A. (2016) · Figshare · doi:10.6084/m9.figshare.3085162.v3 · Cited by 288
Dirty Dozen: Twelve P-Value Misconceptions
Goodman, T. (2008) · Cited by 238
In utero exposure to phenobarbital and intelligence deficits in adult men
Reinisch, J. M. (1995) · JAMA · doi:10.1001/jama.274.19.1518 · Cited by 238
In Utero Exposure to Phenobarbital and Intelligence Deficits in Adult Men
Reinisch, J. M., Sanders, S. A., Mortensen, E. L., & Rubin, D. B. (1995) · JAMA · doi:10.1001/jama.1995.03530190032031 · Cited by 235
The ongoing tyranny of statistical significance testing in biomedical research
Stang, A., Poole, C., & Kuss, O. (2010) · European Journal of Epidemiology · doi:10.1007/s10654-010-9440-x · Cited by 220 · cited 2×
“While potentially subject to mis-use and misinterpretation (1,5-12), the p-value can be informative as a continuous measure of evidence as to whether an association is present.”
BackgroundNick, T. G., Pagano, M., & Gauvreau, K. (1994) · The American Statistician · doi:10.2307/2684733 · Cited by 214
Discussing Hidden Bias in Observational Studies
Rosenbaum, P. R. (1991) · Annals of Internal Medicine · doi:10.7326/0003-4819-115-11-901 · Cited by 213
Brumback, B. A., Hernán, M. A., Haneuse, S. J. P. A., & Robins, J. M. (2004) · Statistics in Medicine · doi:10.1002/sim.1657 · Cited by 212 · cited 1×
“Numerous sensitivity analysis techniques have been developed for different statistical models (14-22,24-40).”
MethodSpurious effects from an extraneous variable
Bross, I. D. J. (1966) · Journal of Chronic Diseases · doi:10.1016/0021-9681(66)90062-2 · Cited by 182 · cited 2×
“Such assumptions often stipulate that the unmeasured confounder is binary (22,24,26), or that there is only one unmeasured confounder (24-28) or that there is no interaction between the effects of the unmeasured confounder and the treatment on the outcome (25-28).”
BackgroundMethodBayesian sensitivity analysis for unmeasured confounding in observational studies
McCandless, L. C., Gustafson, P., & Levy, A. (2007) · Statistics in Medicine · doi:10.1002/sim.2711 · Cited by 181 · cited 1×
“Numerous sensitivity analysis techniques have been developed for different statistical models (14-22,24-40).”
MethodGilbert, P. B., Bosch, R. J., & Hudgens, M. G. (2003) · Biometrics · doi:10.1111/1541-0420.00063 · Cited by 161 · cited 1×
“Numerous sensitivity analysis techniques have been developed for different statistical models (14-22,24-40).”
MethodASSESSING EFFECTS OF CONFOUNDING VARIABLES
(1978) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a112581 · Cited by 160 · cited 1×
“However, information about the distribution of the unmeasured confounder, when available, may be helpful in sensitivity analysis, and techniques are available to use such information (21, 25, 29).”
MethodSensitivity Analysis for Contagion Effects in Social Networks
VanderWeele, T. J. (2011) · Sociological Methods & Research · doi:10.1177/0049124111404821 · Cited by 148 · cited 1×
“Numerous sensitivity analysis techniques have been developed for different statistical models (14-22,24-40).”
MethodSelf-citationA Simple Method for Principal Strata Effects When the Outcome Has Been Truncated Due to Death
Chiba, Y., & VanderWeele, T. J. (2011) · American Journal of Epidemiology · doi:10.1093/aje/kwq418 · Cited by 146
Self-citationHormonal Risk Factors for Ovarian Cancer in Premenopausal and Postmenopausal Women
Moorman, P. G., Calingaert, B., Weber, R. P., Iversen, E. S., Bentley, R. C., Halabi, S., Berchuck, A., & Schildkraut, J. M. (2008) · American Journal of Epidemiology · doi:10.1093/aje/kwn006 · Cited by 139 · cited 1×
“As another example, a study by Moorman and colleagues (43) indicated that in premenopausal women who breastfed for 6 to 12 months, the odds of developing ovarian cancer were 0.”
On a Square-Root Transformation of the Odds Ratio for a Common Outcome
VanderWeele, T. J. (2017) · Epidemiology · doi:10.1097/ede.0000000000000733 · Cited by 134 · cited 2×
“HR for common outcomes When the outcome is common (>15% at the end of follow-up), an approximate E-value may be obtained (45) by applying the approximation RR ≈ (1 − 0.”
MethodSelf-citationPertinency of an extraneous variable
Bross, I. D. J. (1967) · Journal of Chronic Diseases · doi:10.1016/0021-9681(67)90080-x · Cited by 114 · cited 1×
“Numerous sensitivity analysis techniques have been developed for different statistical models (14-22,24-40).”
MethodDesign of Observational Studies
Bedogni, G. (2011) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/j.1467-985x.2010.00676_8.x · Cited by 108
Null misinterpretation in statistical testing and its impact on health risk assessment
Greenland, S. (2011) · Preventive Medicine · doi:10.1016/j.ypmed.2011.08.010 · Cited by 103 · cited 2×
“While potentially subject to mis-use and misinterpretation (1,5-12), the p-value can be informative as a continuous measure of evidence as to whether an association is present.”
BackgroundDesign Sensitivity and Efficiency in Observational Studies
Rosenbaum, P. R. (2010) · Journal of the American Statistical Association · doi:10.1198/jasa.2010.tm09570 · Cited by 97 · cited 4×
“, that some third factor related to both the treatment and the outcome might explain their association, with no true causal effect(12-22).”
BackgroundMethodInfluentialAssessing the sensitivity of regression results to unmeasured confounders in observational studies
Lin, D. Y., Kronmal, R. A., & Psaty, B. M. (2019) · Carolina Digital Repository (University of North Carolina at Chapel Hill) · doi:10.17615/0qzs-af86 · Cited by 52
Bounding the bias of unmeasured factors with confounding and effect‐modifying potentials
Lee, W. (2011) · Statistics in Medicine · doi:10.1002/sim.4151 · Cited by 27 · cited 1×
“Numerous sensitivity analysis techniques have been developed for different statistical models (14-22,24-40).”
MethodBounding Formulas for Selection Bias
Huang, T.-H., & Lee, W. (2015) · American Journal of Epidemiology · doi:10.1093/aje/kwv130 · Cited by 19 · cited 1×
“Numerous sensitivity analysis techniques have been developed for different statistical models (14-22,24-40).”
MethodOddy, W. H., Smith, G. J., & Jacoby, P. (2014) · Annals of Nutrition and Metabolism · doi:10.1159/000360548 · Cited by 10
Research works printed but not linked: 3
- Greenland S, 2011, Jurimetrics, 51, 11329
- Cornfield J, 1959, J Natl Cancer Inst, 22, 173
- Schlesselman JJ, 1978, Am J Epidemiol, 108, 3
Cited by
Bjornevik, K., Cortese, M., Healy, B. C., Kuhle, J., Mina, M. J., Leng, Y., Elledge, S. J., Niebuhr, D. W., Scher, A. I., Munger, K. L., et al. (2022) · Science · doi:10.1126/science.abj8222 · Cited by 2,049
Wu, X., Nethery, R. C., Sabath, M. B., Braun, D., & Dominici, F. (2020) · Science Advances · doi:10.1126/sciadv.abd4049 · Cited by 1,974
Skrivankova, V. W., Richmond, R. C., Woolf, B. A. R., Davies, N. M., Swanson, S. A., VanderWeele, T. J., Timpson, N. J., Higgins, J. P. T., Dimou, N., Langenberg, C., et al. (2021) · BMJ · doi:10.1136/bmj.n2233 · Cited by 1,648
Self-citationZhu, L., She, Z.-G., Cheng, X., Qin, J.-J., Zhang, X.-J., Cai, J., Lei, F., Wang, H., Xie, J., Wang, W., et al. (2020) · Cell Metabolism · doi:10.1016/j.cmet.2020.04.021 · Cited by 1,602
Principles of confounder selection
VanderWeele, T. J. (2019) · European Journal of Epidemiology · doi:10.1007/s10654-019-00494-6 · Cited by 1,535
Self-citation
Metadata
- Authors
- VanderWeele, T. J., & Ding, P.
- Affiliations
- Harvard University (US)
- Venue
- Annals of Internal Medicine
- Publisher
- American College of Physicians
- Published
- 2017-08-15
- Volume
- 167
- Issue
- 4
- Pages
- 268-274
- ISSN
- 0003-4819 (ISSN-L)
- 1539-3704
- OpenAlex
- W2735301415
- Cited by
- 6,350
- References
- 59
- Citations by year
- 2026: 1516 · 2025: 1020 · 2024: 860 · 2023: 736 · 2022: 716 · 2021: 583 · 2020: 498 · 2019: 290 · 2018: 113 · 2017: 15
- FWCI
- 174.58
- Type
- journal article
- Language
- en
- Open access
- green
- nrs.harvard.edu
- Licence
- cc-by
- www.acpjournals.org/journal/aim/text-and-data-mining (tdm)
- Locations
- Annals of Internal Medicine (publishedVersion)
- PubMed (publishedVersion)
- Digital Access to Scholarship at Harvard (DASH) (Harvard University) (submittedVersion, cc-by)
- Funders
- National Institute of Environmental Health Sciences (ES017876)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- E-value, unmeasured confounding, sensitivity analysis, causal inference, observational studies, relative risk
- MeSH terms
- Humans, Sensitivity and Specificity, Epidemiologic Research Design, Confounding Factors, Epidemiologic, Observational Studies as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.

