Sensitivity Analysis in Observational Research: Introducing the E-Value
Open accessThe DOI matches its Crossref record; 58 of the 59 references it cites were found in a registry or an index.
Abstract
Sensitivity analysis is useful in assessing how robust an association is to potential unmeasured or uncontrolled confounding. This article introduces a new measure called the "E-value," which is related to the evidence for causality in observational studies that are potentially subject to confounding. The E-value is defined as the minimum strength of association, on the risk ratio scale, that an unmeasured confounder would need to have with both the treatment and the outcome to fully explain away a specific treatment-outcome association, conditional on the measured covariates. A large E-value implies that considerable unmeasured confounding would be needed to explain away an effect estimate. A small E-value implies little unmeasured confounding would be needed to explain away an effect estimate. The authors propose that in all observational studies intended to produce evidence for causality, the E-value be reported or some other sensitivity analysis be used. They suggest calculating the E-value for both the observed association estimate (after adjustments for measured confounders) and the limit of the confidence interval closest to the null. If this were to become standard practice, the ability of the scientific community to assess evidence from observational studies would improve considerably, and ultimately, science would be strengthened.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Quantitative bias analyses, such as the calculation of E-values to estimate the strength of unmeasured confounding required to explain away an observed association, provide useful context for the robustness of findings.64”
From Artificial Intelligence in Oncology: Practical Applications Across Clinical Care, Scholarship, and Translation. (2026) · cites this paper as [64]
The cited source says what the claim attributes to it.
In the citing paper
“We performed the sensitivity analysis using the E value [29] to evaluate the robustness of our findings to potential unmeasured confounding, with the calculation based on the observed odds ratio (OR) point estimate for rare outcome (<15%) or the square root of OR otherwise.”
The cited source says what the claim attributes to it.
In the citing paper
“To further assess the robustness of our findings to potential unmeasured confounding, we conducted an E-value analysis, a quantitative method for sensitivity analysis in observational research [31].”
From Acute Respiratory Tract Infection and Sudden Sensorineural Hearing Loss: A Multinational Cohort Study (2025) · cites this paper as [31]
In the citing paper
“We also computed E-values: indicators of how robust findings are to potential unmeasured confounding”
From Leading a meaningful life at older ages and its relationship with social engagement, prosperity, health, biology, and time use (2019) · cites this paper as [12]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009) · doi:10.1002/9780470743386 · Cited by 14,225
The environment and disease: association or causation?
Hill, A. B. (2015) · Journal of the Royal Society of Medicine · doi:10.1177/0141076814562718 · Cited by 7,285
The Environment and Disease: Association or Causation?
Hill, A. B. (1965) · Proceedings of the Royal Society of Medicine · doi:10.1177/003591576505800503 · Cited by 6,974 · cited 1×
“Sensitivity analysis makes clear quantitatively why Bradford Hill's criterion of “strength of association” (12) is important in establishing that a given association is, in fact, causal.”
BackgroundThe ASA Statement on p -Values: Context, Process, and Purpose
Wasserstein, R. L., & Lazar, N. A. (2016) · The American Statistician · doi:10.1080/00031305.2016.1154108 · Cited by 6,306 · cited 2×
“While potentially subject to mis-use and misinterpretation (1,5-12), the p-value can be informative as a continuous measure of evidence as to whether an association is present.”
BackgroundCausal Inference for Statistics, Social, and Biomedical Sciences: An Introduction
Imbens, G. W., & Rubin, D. B. (2015) · doi:10.1017/cbo9781139025751 · Cited by 4,210
Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations
Greenland, S., Senn, S. J., Rothman, K. J., Carlin, J. B., Poole, C., Goodman, S. N., & Altman, D. G. (2016) · European Journal of Epidemiology · doi:10.1007/s10654-016-0149-3 · Cited by 3,200
Explanation in Causal Inference: Methods for Mediation and Interaction
VanderWeele, T. J. (2015) · Cited by 2,217
Self-citationSmithson, M. (2000) · SAGE Publications Ltd eBooks · doi:10.4135/9781446218525 · Cited by 1,871
Fundamentals of Biostatistics.
Cornell, R. G., & Rosner, B. (1987) · Journal of the American Statistical Association · doi:10.2307/2289416 · Cited by 1,603
Breastfeeding and Maternal and Infant Health Outcomes In Developed Countries
(2007) · AAP Grand Rounds · doi:10.1542/gr.18-2-15 · Cited by 1,540 · cited 2×
“A study by the Agency for Healthcare Research and Quality (42) reported the association between breastfeeding and childhood leukemia as RR=0.”
MethodInfluential
Research works printed but not linked: 3
- Greenland S, 2011, Jurimetrics, 51, 11329
- Cornfield J, 1959, J Natl Cancer Inst, 22, 173
- Schlesselman JJ, 1978, Am J Epidemiol, 108, 3
Cited by
Items 101–200 of 1,031
Kim, E. S., Whillans, A. V., Lee, M. T., Chen, Y., & VanderWeele, T. J. (2020) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2020.03.004 · Cited by 161
“First, to assess the robustness of the volunteering−health/well-being associations to unmeasured confounding, E-values were calculated to assess the minimum strength that unmeasured confounding must have on the RR scale (with both volunteering and the outcome) to entirely explain the association away.(51) Second, all…”
MethodSelf-citationBuprenorphine versus Methadone for Opioid Use Disorder in Pregnancy
Suarez, E. A., Huybrechts, K. F., Straub, L., Hernández-Díaz, S., Jones, H. E., Connery, H. S., Davis, J. M., Gray, K. J., Lester, B., Terplan, M., et al. (2022) · New England Journal of Medicine · doi:10.1056/nejmoa2203318 · Cited by 162
O'Brien, R., Johnson, E., Haneuse, S., Coleman, K. J., O'Connor, P. J., Fisher, D. P., Sidney, S., Bogart, A., Theis, M. K., Anau, J., et al. (2018) · Annals of Internal Medicine · doi:10.7326/m17-2383 · Cited by 161
“We did 3 sensitivity analyses to examine the effect of not adjusting for potential confounders excluded from the matching process (vs. adjusting for those confounders in our main analysis), the effect of using 1:1 and 10:1 matching, and the effect of assessing robustness of the 5-year results to unmeasured confounding…”
InfluentialLinden, A., Mathur, M. B., & VanderWeele, T. J. (2020) · The Stata Journal: Promoting communications on statistics and Stata · doi:10.1177/1536867x20909696 · Cited by 155
“…confidence limit closest to the null) is 3.0, which can be interpreted as “[a]n unmeasured confounder associated with respiratory death and breastfeeding by a risk ratio of 3.0-fold each could explain away the lower confidence limit, but weaker confounding could not” (VanderWeele and Ding 2017).”
BackgroundMethodSelf-citationRisk for newly diagnosed diabetes after COVID-19: a systematic review and meta-analysis
Zhang, T., Mei, Q., Zhang, Z., Walline, J. H., Liu, Y., Zhu, H., & Zhang, S. (2022) · BMC Medicine · doi:10.1186/s12916-022-02656-y · Cited by 158
“The E-value, a sensitivity analysis of unmeasured confounders, is a relatively new method for measuring the association between exposure and outcome robustness and to assess evidence of causality [50].”
MethodLi, S., Guo, B., Jiang, Y., Wang, X., Chen, L., Wang, X., Chen, T., Yang, L., Silang, Y., Hong, F., et al. (2022) · Diabetes Care · doi:10.2337/dc22-1585 · Cited by 164
“The E-value is defined as the minimum unmeasured confounding effect required to completely subvert the OR in the study, controlling for the measured confounding factor (25,26).”
BackgroundDrouin‐Chartier, J., Chen, S., Li, Y., Schwab, A. L., Stampfer, M. J., Sacks, F. M., ROSNER, B. A., Willett, W. C., Hu, F. B., & Bhupathiraju, S. N. (2020) · BMJ · doi:10.1136/bmj.m513 · Cited by 184
Cheng, X., Liu, Y.-M., Li, H., Zhang, X., Lei, F., Qin, J.-J., Chen, Z., Deng, K.-Q., Lin, L., Chen, M.-M., et al. (2020) · Cell Metabolism · doi:10.1016/j.cmet.2020.08.013 · Cited by 157
Use of real‐world evidence from healthcare utilization data to evaluate drug safety during pregnancy
Huybrechts, K. F., Bateman, B. T., & Hernández–Díaz, S. (2019) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.4789 · Cited by 173
“This can be done using a graphical or tabular approach that illustrates the various combinations of the strength of the association between the unmeasured confounder and the exposure and outcome, respectively,(78-80) or by using the E value approach.(81) In addition to showing the minimum strength of association that…”
BackgroundMethodInfluentialSensitivity Analysis for Unmeasured Confounding in Meta-Analyses
Mathur, M. B., & VanderWeele, T. J. (2020) · Journal of the American Statistical Association · doi:10.1080/01621459.2018.1529598 · Cited by 152
“This approach to sensitivity analysis is essentially a meta-analytic extension of a recently proposed metric (the E-value) that quantifies, for a single study, the minimum confounding bias capable of reducing the true effect to a chosen threshold (VanderWeele and Ding 2017).”
InfluentialSelf-citationOral Nirmatrelvir and Ritonavir in Nonhospitalized Vaccinated Patients With Coronavirus Disease 2019
Ganatra, S., Dani, S. S., Ahmad, J., Kumar, A., Shah, J., Abraham, G. M., McQuillen, D. P., Wachter, R. M., & Sax, P. E. (2022) · Clinical Infectious Diseases · doi:10.1093/cid/ciac673 · Cited by 168
“As a sensitivity analysis, we measured the E-value, a measure to check for robustness against 4 bias from unmeasured confounding or omitted covariates in the observational studies, for both 5 primary and secondary outcomes.[13] A higher E value implies a stronger unmeasured 6 confounder would be needed to negate the e…”
MethodDybvik, J. S., Svendsen, M., & Aune, D. (2023) · European Journal of Nutrition · doi:10.1007/s00394-022-02942-8 · Cited by 150
“Furthermore, we calculated E-values for the association between vegetarian diets and CVD, IHD and stroke to assess the potential impact of unmeasured or uncontrolled confounding [44].”
MethodThe Global Flourishing Study: Study Profile and Initial Results on Flourishing
VanderWeele, T. J., Johnson, B. R., Białowolski, P., Bonhag, R., Bradshaw, M., Breedlove, T., Case, B. W., Chen, Y., Chen, Z. J., Counted, V., et al. (2025) · Nature Mental Health · doi:10.1038/s44220-025-00423-5 · Cited by 253
“An E-value is the minimum strength of the association an unmeasured confounder must have with both the outcome and the predictor, above and beyond all measured covariates, for an unmeasured confounder to explain away an association 35 .”
Self-citationLong, K. N. G., Kim, E. S., Chen, Y., Wilson, M. F., Worthington, E. L., & VanderWeele, T. J. (2020) · Global Epidemiology · doi:10.1016/j.gloepi.2020.100018 · Cited by 181
Self-citationFernando, S. M., Qureshi, D., Talarico, R., Tanuseputro, P., Dowlatshahi, D. D., Sood, M. M., Smith, E. E., Hill, M. D., McCREDIE, V. A., Scales, D. C., et al. (2021) · Stroke · doi:10.1161/strokeaha.120.032550 · Cited by 169
Richmond-Rakerd, L. S., D’Souza, S., Milne, B. J., Caspi, A., & Moffitt, T. E. (2022) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2021.4377 · Cited by 146
The use of the E-value for sensitivity analysis
Chung, W. T., & Chung, K. C. (2023) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2023.09.014 · Cited by 144
Ehret, D. E. Y., Edwards, E. M., Greenberg, L. T., Bernstein, I. M., Buzas, J. S., Soll, R. F., & Horbar, J. D. (2018) · JAMA Network Open · doi:10.1001/jamanetworkopen.2018.3235 · Cited by 152
Well-Being in Life and Well-Being at Work: Which Comes First? Evidence From a Longitudinal Study
Węziak‐Białowolska, D., Białowolski, P., Sacco, P. L., VanderWeele, T. J., & McNeely, E. (2020) · Frontiers in Public Health · doi:10.3389/fpubh.2020.00103 · Cited by 215
Self-citationPositive parenting improves multiple aspects of health and well-being in young adulthood
Chen, Y., Haines, J., Charlton, B. M., & VanderWeele, T. J. (2019) · Nature Human Behaviour · doi:10.1038/s41562-019-0602-x · Cited by 157
Self-citationAssociation of Preeclampsia in Term Births With Neurodevelopmental Disorders in Offspring
Sun, B. Z., Moster, D., Harmon, Q. E., & Wilcox, A. J. (2020) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2020.0306 · Cited by 159
dos Santos, M., Ferrari, G., Lee, D. H., Rey-López, J. P., Aune, D., Liao, B., Huang, W., Nie, J., Wang, Y., Giovannucci, E. L., et al. (2022) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2022.2488 · Cited by 157
Chen, Y., Koh, H. K., Kawachi, I. I., Botticelli, M. P., & VanderWeele, T. J. (2020) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2020.0175 · Cited by 172
Self-citationMartins, S. S., Segura, L. E., Levy, N. S., Mauro, P. M., Mauro, C., Philbin, M. M., & Hasin, D. S. (2021) · JAMA Network Open · doi:10.1001/jamanetworkopen.2021.27002 · Cited by 150
Andersen, K. M., Bates, B. A., Rashidi, E., Olex, A. L., Mannon, R. B., Patel, R. C., Singh, J., Sun, J., Auwaerter, P. G., Ng, D. K., et al. (2021) · The Lancet Rheumatology · doi:10.1016/s2665-9913(21)00325-8 · Cited by 149
Bajema, K. L., Berry, K., Streja, E., Rajeevan, N., Li, Y., Mutalik, P., Yan, L., Cunningham, F. E., Hynes, D. M., Rowneki, M., et al. (2023) · Annals of Internal Medicine · doi:10.7326/m22-3565 · Cited by 139
“The calculated E-value for illness severity using estimates of RR for death within 30 days in trials 1 and 2 was 8.16 (Supplement Figure 6, available at Annals.org).”
MethodInfluentialPenner, L. S., Gavan, S. P., Ashcroft, D. M., Peek, N. B., & Elliott, R. A. (2022) · British Journal of Clinical Pharmacology · doi:10.1111/bcp.15371 · Cited by 142
“For risk ratios (RR) >1, the E-value is calculated as follows.(53)”
BackgroundMethodInfluentialKim, E. S., Chen, Y., Nakamura, J. S., Ryff, C. D., & VanderWeele, T. J. (2021) · American Journal of Health Promotion · doi:10.1177/08901171211038545 · Cited by 162
“First, we performed sensitivity analysis using E-values to assess the robustness of an exposure-outcome association to unmeasured confounding by assessing the minimum strength that an unmeasured confounder must have on the risk ratio scale (with both the exposure and the outcome) to explain away the association.(49) S…”
MethodSelf-citationHemoglobin and anemia in relation to dementia risk and accompanying changes on brain MRI
Wolters, F. J., Zonneveld, H. I., Licher, S., Cremers, L. G. M., on behalf of the Heart Brain Connection Collaborative Research Group, Ikram, M. K., Koudstaal, P. J., Vernooij, M. W., & Ikram, M. A. (2019) · Neurology · doi:10.1212/wnl.0000000000008003 · Cited by 132
“edu/app/).(25) This fairly straightforward computation using point estimate and confidence intervals (CIs) provides an indication of the magnitude of confounding required to change the conclusions on the basis of the presented results.”
BackgroundYazdi, M. D., Wang, Y., Di, Q., Wei, Y., Réquia, W. J., Shi, L., Sabath, M. B., Dominici, F., Coull, B. A., Evans, J. S., et al. (2021) · Circulation · doi:10.1161/circulationaha.120.050252 · Cited by 140
Lee, Y.-B., Han, K., Kim, B., Lee, S.-E., Jun, J. E., Ahn, J., Kim, G., Jin, S.-M., & Kim, J. H. (2019) · Cardiovascular Diabetology · doi:10.1186/s12933-019-0953-7 · Cited by 131
Clinical Outcomes of Tirzepatide or GLP-1 Receptor Agonists in Individuals With Type 2 Diabetes
Chuang, M.-H., Chen, J.-Y., Wang, H.-Y., Jiang, Z.-H., & Wu, V.-C. (2024) · JAMA Network Open · doi:10.1001/jamanetworkopen.2024.27258 · Cited by 133
Brown, J. C., Caan, B. J., Meyerhardt, J. A., Weltzien, E. K., Xiao, J., Feliciano, E. M. C., Kroenke, C. H., Castillo, A., Kwan, M. L., & Prado, C. M. M. (2018) · Journal of Cachexia Sarcopenia and Muscle · doi:10.1002/jcsm.12305 · Cited by 150
“Sensitivity analyses were conducted to quantify the strength that an unmeasured confounder must have to explain the observed associations.(27) P < 0.”
BackgroundPrenatal anemia and postpartum hemorrhage risk: A systematic review and meta‐analysis
Omotayo, M. O., Abioye, A. I., Kuyebi, M. A., & Eke, A. C. (2021) · Journal of obstetrics and gynaecology research · doi:10.1111/jog.14834 · Cited by 186
“69, but weaker confounding could not do so.(45) An E-value greater than 3.”
BackgroundAir pollutants, genetic susceptibility and risk of incident idiopathic pulmonary fibrosis
Cui, F., Sun, Y., Xie, J., Li, D., Wu, M., Song, L., Hu, Y., & Tian, Y. (2022) · European Respiratory Journal · doi:10.1183/13993003.00777-2022 · Cited by 150
Causes and consequences of child growth faltering in low-resource settings
Mertens, A., Benjamin-Chung, J., Colford, J. M., Coyle, J., van der Laan, M. J., Hubbard, A. E., Rosete, S., Malenica, I., Hejazi, N., Sofrygin, O., et al. (2023) · Nature · doi:10.1038/s41586-023-06501-x · Cited by 130
Applying Causal Inference Methods in Psychiatric Epidemiology: A Review
Ohlsson, H., & Kendler, K. S. (2020) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2019.3758 · Cited by 128
Cai, J., Li, H., Zhang, C., Chen, Z., Liu, H., Lei, F., Qin, J.-J., Liu, Y.-M., Zhou, F., Song, X., et al. (2021) · Cell Metabolism · doi:10.1016/j.cmet.2021.01.002 · Cited by 129
Gao, X., Jiang, M., Huang, N., Guo, X., & Huang, T. (2023) · Environmental Health Perspectives · doi:10.1289/ehp10391 · Cited by 127
“35 This E-value could test the robustness of an association between exposure and outcome and to evaluate evidence for causation.”
Li, S., Fonarow, G. C., Mukamal, K., Xu, H., Matsouaka, R. A., Devore, A. D., & Bhatt, D. L. (2018) · Journal of the American Heart Association · doi:10.1161/jaha.117.008088 · Cited by 127
“We also performed sensitivity analysis to estimate the influence of unmeasured confounding factors on our effect estimates.(35)”
Technical Considerations in the Use of the E-Value
VanderWeele, T. J., Ding, P., & Mathur, M. B. (2019) · Journal of Causal Inference · doi:10.1515/jci-2018-0007 · Cited by 127
Self-citationNichols, G. A., Philip, S., Reynolds, K., Granowitz, C. B., & Fazio, S. (2019) · Diabetes, Obesity and Metabolism · doi:10.1111/dom.13537 · Cited by 125
“We assessed the robustness of the reported rate ratios using the recently proposed E-value, a measure of the minimum association that an unmeasured confounder must have with respect to both treatment and outcome to explain the significant associations.(15) A large E-value implies that a large amount of residual confou…”
MethodGuglielminotti, J. R., Landau, R., & Li, G. (2019) · Anesthesia & Analgesia · doi:10.1213/ane.0000000000004336 · Cited by 136
Bao, W., Liu, B., Rong, S., Dai, S. Y., Trasande, L., & Lehmler, H. (2020) · JAMA Network Open · doi:10.1001/jamanetworkopen.2020.11620 · Cited by 131
Fancourt, D., & Steptoe, A. (2019) · BMJ · doi:10.1136/bmj.l6377 · Cited by 125
Johansson, F., Rozental, A., Edlund, K., Côté, P., Sundberg, T., Onell, C., Rudman, A., & Skillgate, E. (2023) · JAMA Network Open · doi:10.1001/jamanetworkopen.2022.49346 · Cited by 124
Henriksson, H., Henriksson, P., Tynelius, P., Ekstedt, M., Berglind, D., Labayen, I., Ruiz, J. R., Lavie, C. J., & Ortega, F. B. (2020) · European Heart Journal · doi:10.1093/eurheartj/ehz774 · Cited by 123
“Finally, we assessed the robustness of the associations of cardiorespiratory fitness with disability pension by calculating Evalues.16 As shown in Supplementary material online, Table S3, calculated E-values were generally high indicating robustness to residual confounding.”
BackgroundMethodInfluentialGervès‐Pinquié, C., Bailly, S., Goupil, F., Pigeanne, T., Launois, S. H., Leclair‐Visonneau, L., Masson, P., Bizieux‐Thaminy, A., Blanchard, M., Sabil, A., et al. (2022) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/rccm.202202-0366oc · Cited by 138
Agricultural land-uses consistently exacerbate infectious disease risks in Southeast Asia
Shah, H. A., Huxley, P. J., Elmes, J., & Murray, K. A. (2019) · Nature Communications · doi:10.1038/s41467-019-12333-z · Cited by 141
Yazdi, M. D., Wang, Y., Di, Q., Réquia, W. J., Wei, Y., Shi, L., Sabath, M. B., Dominici, F., Coull, B. A., Evans, J. S., et al. (2021) · The Lancet Planetary Health · doi:10.1016/s2542-5196(21)00204-7 · Cited by 131
Cannabis Use Disorder and Perioperative Outcomes in Major Elective Surgeries
Goel, A., McGuinness, B. R., Jivraj, N. K., Wijeysundera, D. N., Mittleman, M. A., Bateman, B. T., Clarke, H. A., Kotra, L. P., & Ladha, K. S. (2019) · Anesthesiology · doi:10.1097/aln.0000000000003067 · Cited by 121
Sedlmeier, A. M., Baumeister, S. E., Weber, A., Fischer, B., Thorand, B., Ittermann, T., Dörr, M., Felix, S. B., Völzke, H., Peters, A., et al. (2021) · The American Journal of Clinical Nutrition · doi:10.1093/ajcn/nqaa339 · Cited by 121
Castelnuovo, A. F. D., Costanzo, S., Antinori, A., Berselli, N., Blandi, L., Bonaccio, M., Cauda, R., Guaraldi, G., Menicanti, L., Mennuni, M. G., et al. (2021) · Thrombosis and Haemostasis · doi:10.1055/a-1347-6070 · Cited by 121
Kim, S. Y., Min, C., Oh, D. J., & Choi, H. G. (2019) · The Journal of Allergy and Clinical Immunology In Practice · doi:10.1016/j.jaip.2019.10.043 · Cited by 120
Liu, D., Li, Z., Shen, D., Zhang, P., Song, W.-Q., Zhang, W., Huang, Q., Chen, P., Zhang, X., & Mao, C. (2022) · Annals of Internal Medicine · doi:10.7326/m21-2977 · Cited by 131
“Last, the E-value (21) was estimated to examine the magnitude of an unmeasured confounding factor that could affect the association between coffee consumption andmortality by random chance.”
BackgroundInfluentialHolmer, J., Eriksdotter, M., Schultzberg, M., Pussinen, P. J., & Buhlin, K. (2018) · Journal of Clinical Periodontology · doi:10.1111/jcpe.13016 · Cited by 119
Kim, D. H., Pawar, A. M., Gagne, J. J., Bessette, L. G., Lee, H., Glynn, R. J., & Schneeweiß, S. (2021) · Annals of Internal Medicine · doi:10.7326/m20-7141 · Cited by 136
“” We also examined how our results would change under different scenarios of unmeasured confounding and calculated the minimum strength of the confounder–exposure and confounder– outcome associations on the risk ratio scale to fully explain nonnull associations (E-values) (34).”
ResultGong, X., Fenech, B., Blackmore, C., Chen, Y., Rodgers, G., Gulliver, J., & Hansell, A. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph19052696 · Cited by 146
“We converted the relative risk and 95% CI into odds ratio by using the formula OR ≈ RR2, assuming that mental health is a common health problem among participants (reported by >15% participants) [30].”
MethodChan, Y., Lee, H., See, L., Tu, H., Chao, T., Yeh, Y., Wu, L., Kuo, C.-T., Chang, S., & Lip, G. Y. H. (2019) · CHEST Journal · doi:10.1016/j.chest.2019.04.108 · Cited by 119
Risks of 23 specific malformations associated with prenatal exposure to 10 antiepileptic drugs
Blotière, P., RAGUIDEAU, F., Weill, A., Éléfant, É., Perthus, I., Goulet, V., Rouget, F., Zureik, M., Coste, J., & Dray‐Spira, R. (2019) · Neurology · doi:10.1212/wnl.0000000000007696 · Cited by 141
“However, we calculated e-values to assess the robustness of our results to potential unmeasured or uncontrolled confounding.(41)Given the strength of the associations between the risk of MCMs and parental history of MCMs(6) and alcohol intake(42,43) found in the literature, it is unlikely that these factors could indi…”
MethodTobacco Smoking and Risk for Pulmonary Fibrosis
Bellou, V., Belbasis, L., & Εvangelou, Ε. (2021) · CHEST Journal · doi:10.1016/j.chest.2021.04.035 · Cited by 136
Białowolski, P., Węziak‐Białowolska, D., Lee, M. T., Chen, Y., VanderWeele, T. J., & McNeely, E. (2021) · Social Science & Medicine · doi:10.1016/j.socscimed.2021.114041 · Cited by 160
Self-citationJenkins, D. J. A., Willett, W. C., Yusuf, S., Hu, F. B., Glenn, A. J., Liu, S., Mente, A., Miller, V., Bangdiwala, S. I., Gerstein, H. C., et al. (2024) · The Lancet Diabetes & Endocrinology · doi:10.1016/s2213-8587(23)00344-3 · Cited by 117
Mediational E-values: Approximate Sensitivity Analysis for Unmeasured Mediator–Outcome Confounding
Smith, L. H., & VanderWeele, T. J. (2019) · Epidemiology · doi:10.1097/ede.0000000000001064 · Cited by 119
Self-citationPack, A. I., Magalang, U. J., Singh, B., Kuna, S. T., Keenan, B. T., & Maislin, G. (2021) · Sleep · doi:10.1093/sleep/zsaa229 · Cited by 114
““The E-value is defined as the minimum strength of association, on the risk ratio scale, that an unmeasured confounder would need to have with both the treatment and the outcome to fully explain away a specific treatment–outcome association, conditional on the measured covariates” [116].”
BackgroundKushima, M., Kojima, R., Shinohara, R., Horiuchi, S., Otawa, S., Ooka, T., Akiyama, Y., Miyake, K., Yokomichi, H., Yamagata, Z., et al. (2022) · JAMA Pediatrics · doi:10.1001/jamapediatrics.2021.5778 · Cited by 114
Zhang, S., Gan, S., Zhang, Q., Liu, L., Meng, G., Yao, Z., Wu, H., Gu, Y., Wang, Y., Zhang, T., et al. (2022) · International Journal of Epidemiology · doi:10.1093/ije/dyab174 · Cited by 113
Effect Estimates in Randomized Trials and Observational Studies: Comparing Apples With Apples
Lodi, S., Phillips, A., Lundgren, J., Logan, R., Sharma, S., Cole, S. R., Babiker, A., Law, M., Chu, H., Byrne, D., et al. (2019) · American Journal of Epidemiology · doi:10.1093/aje/kwz100 · Cited by 112
“Future work can extend our general framework to incorporate quantitative assessments of the impact of imperfect harmonization on the randomized-observational discrepancies (46, 47).”
BackgroundKaura, A., Sterne, J. A. C., Trickey, A., Abbott, S., Mulla, A., Glampson, B., Panoulas, V. F., Davies, J., Woods, K., Omigie, J., et al. (2020) · The Lancet · doi:10.1016/s0140-6736(20)30930-2 · Cited by 124
Welk, B. K., & McArthur, E. (2020) · British Journal of Urology · doi:10.1111/bju.15040 · Cited by 119
Association Between Time to Source Control in Sepsis and 90-Day Mortality
Reitz, K. M., Kennedy, J., Li, S. R., Handzel, R., Tonetti, D. A., Neal, M. D., Zuckerbraun, B. S., Hall, D. E., Sperry, J. L., Angus, D. C., et al. (2022) · JAMA Surgery · doi:10.1001/jamasurg.2022.2761 · Cited by 112
Watts, E. L., Matthews, C. E., Freeman, J. R., Gorzelitz, J., Hong, H. G., Liao, L. M., McClain, K. M., Saint‐Maurice, P. F., Shiroma, E. J., & Moore, S. C. (2022) · JAMA Network Open · doi:10.1001/jamanetworkopen.2022.28510 · Cited by 140
Shao, Y., Hu, H., Li, Q., Cao, C., Liu, D., & Han, Y. Y. (2024) · Cardiovascular Diabetology · doi:10.1186/s12933-024-02165-7 · Cited by 120
“The E-value was calculated to assess the possibility of unmeasured confounders between TyG-BMI and stroke risk [36].”
MethodCommentary: Developing best-practice guidelines for the reporting of E-values
VanderWeele, T. J., & Mathur, M. B. (2020) · International Journal of Epidemiology · doi:10.1093/ije/dyaa094 · Cited by 111
“for their interesting analysis of the current reporting practices around the use of the E-value to evaluate sensitivity to unmeasured confounding.(2,3) As with nearly any quantitative tool, the Evalue is potentially subject to misuse,(4,5) examples of which are indeed documented in their paper.”
BackgroundSelf-citationLiu, B., Luo, Z., Pinto, J. M., Shiroma, E. J., Tranah, G. J., Wirdefeldt, K., Fang, F., Harris, T. B., & Chen, H. (2019) · Annals of Internal Medicine · doi:10.7326/m18-0775 · Cited by 116
“The E-values (Appendix Table 6, available at Annals.org) for effects of poor olfaction ranged from 1.54 to 2.28 for the total effect, 1.16 to 1.99 for the natural direct effect, 1.31 to 1.57 for the natural indirect effect through pathways involving dementia or Parkinson disease, and 1.00 to 1.21 for the partial indir…”
InfluentialHaberman, R. H., Castillo, R., Chen, A., Yan, D., Ramirez, D., Sekar, V., Lesser, R. L., Solomon, G. E., Neimann, A. L., Blank, R. B., et al. (2020) · Arthritis & Rheumatology · doi:10.1002/art.41456 · Cited by 117
Mortality risk from United States coal electricity generation
Henneman, L., Choirat, C., Dedoussi, I. C., Dominici, F., Roberts, J., & Zigler, C. M. (2023) · Science · doi:10.1126/science.adf4915 · Cited by 118
Nava, G. R., Trelles-Garcia, D. P., Yanez‐Bello, M. A., Chung, C. W., Trelles-Garcia, V. P., & Friedman, H. (2020) · Critical Care · doi:10.1186/s13054-020-03154-4 · Cited by 108
Lyu, H., Yoshida, K., Zhao, S. S., Wei, J., Zeng, C., Tedeschi, S. K., Leder, B. Z., Lei, G., Tang, P., & Solomon, D. H. (2020) · Annals of Internal Medicine · doi:10.7326/m20-0882 · Cited by 120
“To examine the effect of unmeasured confounding, we found an E-value of 7.28 with a lower confidence limit of 2.62 for vertebral fracture, meaning that substantial unmeasured confounding would be needed to explain away the association between long delay and vertebral fracture risk to the null.”
InfluentialHong, J. H., Lachman, M. E., Charles, S. T., Chen, Y., Wilson, C. L., Nakamura, J. S., VanderWeele, T. J., & Kim, E. S. (2021) · Preventive Medicine · doi:10.1016/j.ypmed.2021.106612 · Cited by 132
Self-citationBlum, M. R., Tan, Y. J., & Ioannidis, J. P. A. (2019) · International Journal of Epidemiology · doi:10.1093/ije/dyz261 · Cited by 136
Castelnuovo, A. F. D., Costanzo, S., Antinori, A., Berselli, N., Blandi, L., Bruno, R., Cauda, R., Guaraldi, G., Menicanti, L., My, I., et al. (2020) · European Journal of Internal Medicine · doi:10.1016/j.ejim.2020.08.019 · Cited by 110
Qiu, C. X., Lin, J. C., Shi, J. M., Chow, T., Desai, V. N., Nguyen, V. T., Riewerts, R. J., Feldman, R. K., Segal, S., & Xiang, A. H. (2020) · Archives of Pediatrics and Adolescent Medicine · doi:10.1001/jamapediatrics.2020.3231 · Cited by 114
Marsden, J., White, M., Annand, F., Burkinshaw, P., Carville, S., Eastwood, B., Kelleher, M., Knight, J., O'Connor, R., Tran, A., et al. (2019) · The Lancet Psychiatry · doi:10.1016/s2215-0366(19)30331-1 · Cited by 106
Limitations and Misinterpretations of E-Values for Sensitivity Analyses of Observational Studies
Ioannidis, J. P. A., Tan, Y. J., & Blum, M. R. (2018) · Annals of Internal Medicine · doi:10.7326/m18-2159 · Cited by 127
“that an unmeasured confounder would need to have with both the treatment and the outcome to fully explain away a specific treatment– outcome association, conditional on the measured covariates” (12).”
BackgroundInfluentialFralick, M., Chen, S. K., Patorno, E., & Kim, S. C. (2020) · Annals of Internal Medicine · doi:10.7326/m19-2610 · Cited by 122
“An E-value was calculated to estimate the robustness of the HR of our primary outcome to unmeasured confounders (22).”
MethodTrasande, L., Liu, B., & Bao, W. (2021) · Environmental Pollution · doi:10.1016/j.envpol.2021.118021 · Cited by 114
Associations between mental health, blood pressure and the development of hypertension
Schaare, H. L., Blöchl, M., Kumral, D., Uhlig, M., Lemcke, L., Valk, S. L., & Villringer, A. (2023) · Nature Communications · doi:10.1038/s41467-023-37579-6 · Cited by 135
Xu, X., Huang, R., Lin, Y., Guo, Y., Xiong, Z., Zhong, X., Ye, X., Li, M., Zhuang, X., & Liao, X. (2022) · Cardiovascular Diabetology · doi:10.1186/s12933-022-01593-7 · Cited by 117
“In addition, we calculated the E-value to evaluate the robustness of the results to potential uncontrolled confounders [20].”
MethodEarly exposure to antibiotic drugs and risk for psychiatric disorders: a population-based study
Lavebratt, C., Yang, L. L., Giacobini, M., Forsell, Y., Schalling, M., Partonen, T., & Gissler, M. (2019) · Translational Psychiatry · doi:10.1038/s41398-019-0653-9 · Cited by 103
Hong, I., Goodwin, J. S., Reistetter, T., Kuo, Y., Mallinson, T., Karmarkar, A. M., Lin, Y., & Ottenbacher, K. J. (2019) · JAMA Network Open · doi:10.1001/jamanetworkopen.2019.16646 · Cited by 104
Beesley, L. J., Salvatore, M., Fritsche, L. G., Pandit, A., Rao, A., Brummett, C., Willer, C. J., Lisabeth, L. D., & Mukherjee, B. (2020) · Statistics in Medicine · doi:10.1002/sim.8445 · Cited by 104
Fralick, M., Colacci, M., Schneeweiß, S., Huybrechts, K. F., Lin, K. J., & Gagne, J. J. (2020) · Annals of Internal Medicine · doi:10.7326/m19-2522 · Cited by 116
“Finally, we calculated an E-value to assess the impact of unmeasured confounding (31).”
MethodGLP-1 Receptor Agonist Use and Risk of Suicide Death
Ueda, P., Söderling, J., Wintzell, V., Svanström, H., Pazzagli, L., Eliasson, B., Melbye, M., Hviid, A., & Pasternak, B. (2024) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2024.4369 · Cited by 103
Internet use and subsequent health and well-being in older adults: An outcome-wide analysis
Nakagomi, A., Shiba, K., Kawachi, I. I., Ide, K., Nagamine, Y., Kondo, N., Hanazato, M., & Kondo, K. (2022) · Computers in Human Behavior · doi:10.1016/j.chb.2021.107156 · Cited by 111
Venous Thromboembolism in Total Hip and Total Knee Arthroplasty
Simon, S. J., Patell, R. D., Zwicker, J. I., Kazi, D. S., & Hollenbeck, B. L. (2023) · JAMA Network Open · doi:10.1001/jamanetworkopen.2023.45883 · Cited by 124
Association Between Fertility Treatment and Cancer Risk in Children
Hargreave, M., Jensen, A., Hansen, M. K., Dehlendorff, C., Winther, J. F., Schmiegelow, K., & Kjær, S. K. (2019) · JAMA · doi:10.1001/jama.2019.18037 · Cited by 109
Meyer, A., Rudant, J., Drouin, J., Weill, A., Carbonnel, F., & Coste, J. (2018) · Annals of Internal Medicine · doi:10.7326/m18-1512 · Cited by 110
“We also calculated the E-value, which is the minimum strength of association on the risk ratio scale that an unmeasured confounder would need to have with both the treatment and the outcome, conditional on the measured covariates, to explain away a treatment– outcome association (35).”
MethodMadsen, H., Sen, A., & Aune, D. (2023) · European Journal of Nutrition · doi:10.1007/s00394-023-03145-5 · Cited by 144
Zhang, Y., Mustieles, V., Wang, Y., Sun, Y., Agudelo, J., Bibi, Z., Torres, N., Oulhote, Y., Slitt, A. L., & Messerlian, C. (2023) · The Lancet Planetary Health · doi:10.1016/s2542-5196(23)00088-8 · Cited by 108
“79 Finally, we were unable to examine short-chain PFAS since the NHANES had few data on this, necessitating further research.”
Metadata
- Authors
- VanderWeele, T. J., & Ding, P.
- Affiliations
- Harvard University (US)
- Venue
- Annals of Internal Medicine
- Publisher
- American College of Physicians
- Published
- 2017-08-15
- Volume
- 167
- Issue
- 4
- Pages
- 268-274
- ISSN
- 0003-4819 (ISSN-L)
- 1539-3704
- OpenAlex
- W2735301415
- Cited by
- 6,350
- References
- 59
- Citations by year
- 2026: 1516 · 2025: 1020 · 2024: 860 · 2023: 736 · 2022: 716 · 2021: 583 · 2020: 498 · 2019: 290 · 2018: 113 · 2017: 15
- FWCI
- 174.58
- Type
- journal article
- Language
- en
- Open access
- green
- nrs.harvard.edu
- Licence
- cc-by
- www.acpjournals.org/journal/aim/text-and-data-mining (tdm)
- Locations
- Annals of Internal Medicine (publishedVersion)
- PubMed (publishedVersion)
- Digital Access to Scholarship at Harvard (DASH) (Harvard University) (submittedVersion, cc-by)
- Funders
- National Institute of Environmental Health Sciences (ES017876)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- E-value, unmeasured confounding, sensitivity analysis, causal inference, observational studies, relative risk
- MeSH terms
- Humans, Sensitivity and Specificity, Epidemiologic Research Design, Confounding Factors, Epidemiologic, Observational Studies as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.

