Sensitivity Analysis in Observational Research: Introducing the E-Value
Open accessThe DOI matches its Crossref record; 58 of the 59 references it cites were found in a registry or an index.
Abstract
Sensitivity analysis is useful in assessing how robust an association is to potential unmeasured or uncontrolled confounding. This article introduces a new measure called the "E-value," which is related to the evidence for causality in observational studies that are potentially subject to confounding. The E-value is defined as the minimum strength of association, on the risk ratio scale, that an unmeasured confounder would need to have with both the treatment and the outcome to fully explain away a specific treatment-outcome association, conditional on the measured covariates. A large E-value implies that considerable unmeasured confounding would be needed to explain away an effect estimate. A small E-value implies little unmeasured confounding would be needed to explain away an effect estimate. The authors propose that in all observational studies intended to produce evidence for causality, the E-value be reported or some other sensitivity analysis be used. They suggest calculating the E-value for both the observed association estimate (after adjustments for measured confounders) and the limit of the confidence interval closest to the null. If this were to become standard practice, the ability of the scientific community to assess evidence from observational studies would improve considerably, and ultimately, science would be strengthened.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Quantitative bias analyses, such as the calculation of E-values to estimate the strength of unmeasured confounding required to explain away an observed association, provide useful context for the robustness of findings.64”
From Artificial Intelligence in Oncology: Practical Applications Across Clinical Care, Scholarship, and Translation. (2026) · cites this paper as [64]
The cited source says what the claim attributes to it.
In the citing paper
“We performed the sensitivity analysis using the E value [29] to evaluate the robustness of our findings to potential unmeasured confounding, with the calculation based on the observed odds ratio (OR) point estimate for rare outcome (<15%) or the square root of OR otherwise.”
The cited source says what the claim attributes to it.
In the citing paper
“To further assess the robustness of our findings to potential unmeasured confounding, we conducted an E-value analysis, a quantitative method for sensitivity analysis in observational research [31].”
From Acute Respiratory Tract Infection and Sudden Sensorineural Hearing Loss: A Multinational Cohort Study (2025) · cites this paper as [31]
In the citing paper
“We also computed E-values: indicators of how robust findings are to potential unmeasured confounding”
From Leading a meaningful life at older ages and its relationship with social engagement, prosperity, health, biology, and time use (2019) · cites this paper as [12]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009) · doi:10.1002/9780470743386 · Cited by 14,225
The environment and disease: association or causation?
Hill, A. B. (2015) · Journal of the Royal Society of Medicine · doi:10.1177/0141076814562718 · Cited by 7,285
The Environment and Disease: Association or Causation?
Hill, A. B. (1965) · Proceedings of the Royal Society of Medicine · doi:10.1177/003591576505800503 · Cited by 6,974 · cited 1×
“Sensitivity analysis makes clear quantitatively why Bradford Hill's criterion of “strength of association” (12) is important in establishing that a given association is, in fact, causal.”
BackgroundThe ASA Statement on p -Values: Context, Process, and Purpose
Wasserstein, R. L., & Lazar, N. A. (2016) · The American Statistician · doi:10.1080/00031305.2016.1154108 · Cited by 6,306 · cited 2×
“While potentially subject to mis-use and misinterpretation (1,5-12), the p-value can be informative as a continuous measure of evidence as to whether an association is present.”
BackgroundCausal Inference for Statistics, Social, and Biomedical Sciences: An Introduction
Imbens, G. W., & Rubin, D. B. (2015) · doi:10.1017/cbo9781139025751 · Cited by 4,210
Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations
Greenland, S., Senn, S. J., Rothman, K. J., Carlin, J. B., Poole, C., Goodman, S. N., & Altman, D. G. (2016) · European Journal of Epidemiology · doi:10.1007/s10654-016-0149-3 · Cited by 3,200
Explanation in Causal Inference: Methods for Mediation and Interaction
VanderWeele, T. J. (2015) · Cited by 2,217
Self-citationSmithson, M. (2000) · SAGE Publications Ltd eBooks · doi:10.4135/9781446218525 · Cited by 1,871
Fundamentals of Biostatistics.
Cornell, R. G., & Rosner, B. (1987) · Journal of the American Statistical Association · doi:10.2307/2289416 · Cited by 1,603
Breastfeeding and Maternal and Infant Health Outcomes In Developed Countries
(2007) · AAP Grand Rounds · doi:10.1542/gr.18-2-15 · Cited by 1,540 · cited 2×
“A study by the Agency for Healthcare Research and Quality (42) reported the association between breastfeeding and childhood leukemia as RR=0.”
MethodInfluential
Research works printed but not linked: 3
- Greenland S, 2011, Jurimetrics, 51, 11329
- Cornfield J, 1959, J Natl Cancer Inst, 22, 173
- Schlesselman JJ, 1978, Am J Epidemiol, 108, 3
Cited by
Items 701–800 of 1,031
Association between Dietary Salt Intake and Progression in the Gastric Precancerous Process
Thapa, S., Fischbach, L. A., Delongchamp, R. R., Faramawi, M. F., & Orloff, M. S. (2019) · Cancers · doi:10.3390/cancers11040467 · Cited by 38
“An e-value can assist us in evaluating whether our results can likely be attributed to confounding and is defined as “the minimum strength of association, on a risk ratio scale, that an unmeasured confounder would need to have with both [exposure] and outcome to fully explain away a specific [exposure]–outcome associa…”
BackgroundDario, A. B., Kamper, S. J., O’Keeffe, M., Zadro, J. R., Lee, H., Wolfenden, L., & Williams, C. M. (2019) · Pain · doi:10.1097/j.pain.0000000000001639 · Cited by 43
“This uncertainty could be reduced in future studies by implementing sensitivity analyses that estimate the strength of residual confounding that would invalidate the observed associations.(70) Furthermore, we do not know whether associations of different strength would be found in samples collected from clinical setti…”
BackgroundDepression, antidepressants and the risk of cardiovascular events and death in older men
Almeida, O. P., Ford, A. H., Hankey, G. J., Golledge, J., Yeap, B. B., & Flicker, L. A. (2019) · Maturitas · doi:10.1016/j.maturitas.2019.06.009 · Cited by 42
Sensitivity Analysis via the Proportion of Unmeasured Confounding
Bonvini, M., & Kennedy, E. H. (2020) · Journal of the American Statistical Association · doi:10.1080/01621459.2020.1864382 · Cited by 73
“In addition, Ding and VanderWeele (2016) and VanderWeele and Ding (2017) have derived a bounding factor for certain treatment effects in terms of two sensitivity parameters governing the U-A and U-Y relationships.”
BackgroundTakeuchi, M., Ogura, M., Minoura, T., Inagaki, N., & Kawakami, K. (2020) · Mayo Clinic Proceedings · doi:10.1016/j.mayocp.2019.12.004 · Cited by 38
“Quantitative Bias Analysis We calculated E-value (as relative risk) to evaluate the minimal strength of unmeasured confounding to negate the observed HR in our survival analysis.(18) The higher the E-value is, the stronger the unmeasured confounding should be to explain away the observed association.”
MethodHormonal contraceptives and onset of asthma in reproductive-age women: Population-based cohort study
Nwaru, B. I., Pillinger, R., Tibble, H., Shah, S. A., Ryan, D. P., Critchley, H., Price, D. B., Hawrylowicz, C. M., Simpson, C. R., Soyiri, I. N., et al. (2020) · Journal of Allergy and Clinical Immunology · doi:10.1016/j.jaci.2020.02.027 · Cited by 37
Pang, Y., Lv, J., Kartsonaki, C., Yu, C., Guo, Y., Du, H., Bennett, D., Bian, Z., Chen, Y., Yang, L., et al. (2020) · British Journal of Sports Medicine · doi:10.1136/bjsports-2020-102174 · Cited by 40
“46 In addition, the associations remained for hepatobiliary diseases when excluding the first 5 years of follow-up or participants with chronic diseases or poor self-rated health at baseline, suggesting that the inverse associations were not affected by reverse causality.”
A pilot investigation of couple-level phthalates exposure and in vitro fertilization (IVF) outcomes
Begum, T. F., Fujimoto, V. Y., Gerona, R. R. L., McGough, A., Lenhart, N. J., Wong, R., Mok-Lin, E. Y., Melamed, J., Butts, C. D., & Bloom, M. S. (2020) · Reproductive Toxicology · doi:10.1016/j.reprotox.2020.11.014 · Cited by 38
Rossides, M., Kullberg, S., Giuseppe, D. D., Eklúnd, A., Grünewald, J., Askling, J., & Arkema, E. V. (2021) · Respirology · doi:10.1111/resp.14001 · Cited by 44
Cabanas‐Sánchez, V., Duarte, M. A. D. S., Lavie, C. J., Celis-Morales, C. A., Rodríguez‐Artalejo, F., & Martínez‐Gómez, D. (2023) · Mayo Clinic Proceedings · doi:10.1016/j.mayocp.2023.05.028 · Cited by 40
Magar, R. R., Knight, S. R., Maggiore, U., Lafranca, J. A., Dor, F. J. M. F., & Pengel, L. H. M. (2023) · Transplantation Reviews · doi:10.1016/j.trre.2023.100798 · Cited by 37
Choi, M. H., Wan, E. Y. F., Wong, I. C. K., Chan, E. W., Chu, W. M., Tam, A. R., Yuen, K. Y., & Hung, I. F. (2024) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(24)00353-0 · Cited by 45
Continuous opioid substitution treatment over five years: Heroin use trajectories and outcomes
Eastwood, B., Strang, J., & Marsden, J. (2018) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2018.03.052 · Cited by 39
Tétu, P., Allayous, C., Oriano, B., Dalle, S., Mortier, L., Leccia, M. ‐., Guillot, B., Dalac, S., Dutriaux, C., Lacour, J. ‐., et al. (2019) · European Journal of Cancer · doi:10.1016/j.ejca.2019.02.009 · Cited by 44
Bounds on the conditional and average treatment effect with unobserved confounding factors
Yadlowsky, S., Namkoong, H., Basu, S., Duchi, J. C., & Tian, L. (2022) · The Annals of Statistics · doi:10.1214/22-aos2195 · Cited by 112
“While our result is valid for each fixed , providing uniform inference results over a set of would allow estimation of the smallest value of consistent with zero treatment effect in the data (a sensitivity value analogous to the E value for risk ratios from VanderWeele and Ding [59]).”
ResultIn-hospital mortality after hip arthroplasty in China
Zeng, C., Lane, N. E., Englund, M., Xie, D., Hu, C., Zhang, Y., Wang, H., & Lei, G. (2019) · The Bone & Joint Journal · doi:10.1302/0301-620x.101b10.bjj-2018-1608.r1 · Cited by 38
Yang, C., Kwak, L., Ballew, S. H., Jaar, B. G., Deal, J. A., Folsom, A. R., HEISS, G. M., Sharrett, A. R., Selvin, E., Sabanayagam, C., et al. (2019) · Atherosclerosis · doi:10.1016/j.atherosclerosis.2019.10.012 · Cited by 41
Patterns of prenatal antidepressant exposure and risk of preeclampsia and postpartum haemorrhage
Palmsten, K. K., Chambers, C., Wells, A., & Bandoli, G. (2020) · Paediatric and Perinatal Epidemiology · doi:10.1111/ppe.12660 · Cited by 37
Raffetti, E., Donat‐Vargas, C., Mentasti, S., Chinotti, A., & Donato, F. (2020) · Chemosphere · doi:10.1016/j.chemosphere.2020.126984 · Cited by 40
A cohort study on long-term exposure to air pollution and incidence of liver cirrhosis
Orioli, R., Solimini, A. G., Michelozzi, P., Forastiere, F., Davoli, M., & Cesaroni, G. (2020) · Environmental Epidemiology · doi:10.1097/ee9.0000000000000109 · Cited by 37
Baum, A., Wisnivesky, J. P., Basu, S., Siu, A. L., & Schwartz, M. D. (2020) · JAMA · doi:10.1001/jama.2020.14381 · Cited by 43
Gondek, D., Bann, D., Brown, M., Hamer, M., Sullivan, A., & Ploubidis, G. B. (2021) · BMC Public Health · doi:10.1186/s12889-021-11291-w · Cited by 40
Iavarone, M. A., Invernizzi, F., Ivanics, T., Mazza, S., Zavaglia, C., Sanduzzi‐Zamparelli, M., López, M. F., Czauderna, C., Costanzo, G. D., Bhoori, S., et al. (2021) · Liver Transplantation · doi:10.1002/lt.26264 · Cited by 41
Wang, T., Lv, Z., Wen, Y., Zou, X., Zhou, G., Cheng, J., Zhong, D., Zhang, Y., Yu, S., Liu, N., et al. (2021) · Chemosphere · doi:10.1016/j.chemosphere.2021.132305 · Cited by 40
Hyperglycaemic disorders associated with PCSK9 inhibitors: a real-world, pharmacovigilance study
Goldman, A., Raschi, E., Cukierman‐Yaffe, T., Dankner, R. S., Shouval, R., Shechter, M., Ben‐Zvi, I., Gerstein, H. C., & Maor, E. (2021) · European Journal of Preventive Cardiology · doi:10.1093/eurjpc/zwab209 · Cited by 38
Overcoming Stenotrophomonas maltophilia Resistance for a More Rational Therapeutic Approach
Kullar, R., Wenzler, E., Alexander, J. L. R., & Goldstein, E. J. C. (2022) · Open Forum Infectious Diseases · doi:10.1093/ofid/ofac095 · Cited by 41
“11 could move the CI to include an RR of 1 and therefore no longer be statistically significant [80].”
BackgroundMiller, M., Zhang, Y., Prince, L., Swanson, S. A., Wintemute, G. J., Holsinger, E. E., & Studdert, D. M. (2022) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2022.0793 · Cited by 40
Sawadogo, W., Adera, T., Alattar, M., Perera, R. A., & Burch, J. B. (2023) · Neurology · doi:10.1212/wnl.0000000000207449 · Cited by 42
“The E-value is the minimum strength of association that an unmeasured confounder would need to have with both the treatment and the outcome, conditional on the measured covariates, to fully explain a specific exposureoutcome association.(43) The analyses were conducted using SAS version 9.”
MethodHuang, C., Iio, E., Jun, D. W., Ogawa, E., Toyoda, H., Hsu, Y., Haga, H., Iwane, S., Enomoto, M., Lee, D. H., et al. (2019) · Hepatology International · doi:10.1007/s12072-019-09974-z · Cited by 36
Arrais, M. L. T., Maricoto, T., Nwaru, B. I., Cooper, P. J., Gama, J. M. R., Brito, M., & Taborda‐Barata, L. (2021) · Journal of Allergy and Clinical Immunology · doi:10.1016/j.jaci.2021.12.777 · Cited by 40
“In the meta-analysis, estimates from studies not presented as RRs were converted to RRs using the recently proposed formulas provided by VanderWeele and Ding.31 Data for continuous outcomes were not available.”
MethodWęziak‐Białowolska, D., Białowolski, P., Lee, M. T., Chen, Y., VanderWeele, T. J., & McNeely, E. (2022) · International Journal of Public Health · doi:10.3389/ijph.2022.1604710 · Cited by 47
Self-citationRacial and ethnic disparities in sentencing: What do we know, and where should we go?
Veiga, A., Pina-Sánchez, J., & Lewis, S. (2022) · The Howard Journal of Crime and Justice · doi:10.1111/hojo.12496 · Cited by 43
Methamphetamine exposure and depression—A systematic review and meta‐analysis
Leung, J., Mekonen, T., Wang, X., Arunogiri, S., Degenhardt, L. J., & McKetin, R. (2023) · Drug and Alcohol Review · doi:10.1111/dar.13670 · Cited by 43
Positive Airway Pressure, Mortality, and Cardiovascular Risk in Older Adults With Sleep Apnea
Mazzotti, D. R., Waitman, L. R., Miller, J. N., Sundar, K. M., Stewart, N. H., Gozal, D., Song, X., Collaborative, G. P., Chandaka, S., Anuforo, K., et al. (2024) · JAMA Network Open · doi:10.1001/jamanetworkopen.2024.32468 · Cited by 47
Maternal exposures and risk of oral clefts in South Vietnam
Dien, V. H. A., McKinney, C. M., Pisek, A., & Pitiphat, W. (2018) · Birth Defects Research · doi:10.1002/bdr2.1192 · Cited by 38
“To address the impact of uncontrolled confounding, we calculated an E-value, a novel measure that assesses how far off the true risk estimate may be given unmeasured confounding in a study in a sensitivity analysis (VanderWeele & Ding, 2017).”
BackgroundMethodInfluentialMínguez‐Alarcón, L., Chiu, Y., Nassan, F. L., Williams, P. L., Petrozza, J. C., Ford, J. B., Calafat, A. M., Hauser, R. B., & Chavarro, J. E. (2019) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2019.04.452 · Cited by 40
“In addition, a sensitivity analysis was performed and the bias adjusted risk ratio (aRR) was calculated to quantify the impact of unmeasured confounding on the relationship between urinary benzophenone-3 concentrations and pregnancy outcomes (VanderWeele and Ding 2017).”
MethodInfluentialLeonard, S. A., Carmichael, S. L., Main, E. K., Lyell, D. J., & Abrams, B. F. (2019) · Paediatric and Perinatal Epidemiology · doi:10.1111/ppe.12555 · Cited by 38
Leopold, S. J., Watson, J. A., Jeeyapant, A., Simpson, J. A., Phu, N. H., Hien, T. T., Day, N., Dondorp, A. M., & White, N. J. (2019) · PLoS Medicine · doi:10.1371/journal.pmed.1002858 · Cited by 37
Autism Spectrum Disorders in Children Exposed in Utero to Hyperemesis Gravidarum
Getahun, D., Fassett, M. J., Jacobsen, S. J., Xiang, A. H., Takhar, H. S., Wing, D. A., & Peltier, M. R. (2019) · American Journal of Perinatology · doi:10.1055/s-0039-1696670 · Cited by 41
Ogawa, E., Toyoda, H., Iio, E., Jun, D. W., Huang, C., Enomoto, M., Hsu, Y., Haga, H., Iwane, S., Wong, G. L., et al. (2019) · Clinical Infectious Diseases · doi:10.1093/cid/ciz1160 · Cited by 37
Effect of maternal asthma exacerbations on perinatal outcomes: a population-based study
Robijn, A. L., Brew, B. K., Jensen, M. E., Rejnö, G., Lundholm, C., Murphy, V. E., & Almqvist, C. (2020) · ERJ Open Research · doi:10.1183/23120541.00295-2020 · Cited by 36
Reece, A. S., & Hulse, G. K. (2020) · BMC Pediatrics · doi:10.1186/s12887-020-02431-z · Cited by 37
Nwaru, B. I., Tibble, H., Shah, S. A., Pillinger, R., McLean, S., Ryan, D. P., Critchley, H., Price, D. B., Hawrylowicz, C. M., Simpson, C. R., et al. (2020) · Thorax · doi:10.1136/thoraxjnl-2020-215540 · Cited by 38
Stress Related Disorders and the Risk of Kidney Disease
Su, G., Song, H., Lanka, V., Liu, X., Fang, F., Valdimarsdóttir, U. A., & Carrero, J. J. (2021) · Kidney International Reports · doi:10.1016/j.ekir.2020.12.032 · Cited by 45
“Second, we estimated the E-value to evaluate the risk of unmeasured confounding.(24) Conditional on the”
Mumford, S. L., Flannagan, K. S. J., Radoc, J. G., Sjaarda, L. A., Zolton, J. R., Metz, T. D., Plowden, T. C., Perkins, N. J., DeVilbiss, E. A., Andriessen, V. C., et al. (2020) · Human Reproduction · doi:10.1093/humrep/deaa355 · Cited by 44
“We utilized two sensitivity analysis approaches for unmeasured confounding, including the e-value method (VanderWeele and Ding, 2017) and a simulation study to address this concern.”
MethodChan, G., Becker, D., Butterworth, P., Hines, L. A., Coffey, C., Hall, W., & Patton, G. C. (2021) · Drug and Alcohol Review · doi:10.1111/dar.13239 · Cited by 38
“We used sensitivity analyses based on recent advances in biostatistics to test the plausibility of causal interpretations [27].”
BackgroundMethodInfluentialMarkoulidakis, A., Taiyari, K., Holmans, P., Pallmann, P., Busse, M. E., Godley, M. D., & Griffin, B. A. (2022) · Health Services and Outcomes Research Methodology · doi:10.1007/s10742-022-00280-0 · Cited by 53
Meyer, A., Drouin, J., Weill, A., Carbonnel, F., & Dray‐Spira, R. (2021) · Alimentary Pharmacology & Therapeutics · doi:10.1111/apt.16448 · Cited by 36
Ochoa, L. B., Bertens, L. C. M., García‐Gómez, P., Ourti, T. V., Steegers, E. A. P., & Been, J. V. (2021) · The Lancet Regional Health - Europe · doi:10.1016/j.lanepe.2021.100205 · Cited by 38
Long-term survival after coronary bypass surgery with multiple versus single arterial grafts
Thuijs, D. J. F. M., Davierwala, P. M., Milojevic, M., Deo, S. V., Noack, T., Kappetein, A. P., Serruys, P. W. J. C., Mohr, F., Morice, M., Mack, M. J., et al. (2021) · European Journal of Cardio-Thoracic Surgery · doi:10.1093/ejcts/ezab392 · Cited by 36
“The E-value, representing the potential impact of unobserved variables, was 1.7 [11].”
BackgroundInfluentialAutoimmune diseases in microscopic colitis: A Danish nationwide case–control study
Wildt, S., Munck, L. K., Winther‐Jensen, M., Jess, T., & Andersen, N. N. (2021) · Alimentary Pharmacology & Therapeutics · doi:10.1111/apt.16614 · Cited by 36
Kamitaki, B. K., Janmohamed, M., Kandula, P., Elder, C. J., Mani, R., Wong, S., Perucca, P., O'Brien, T. J., Lin, H., Heiman, G. A., et al. (2021) · Epilepsia · doi:10.1111/epi.17104 · Cited by 41
Baek, Y., Kang, E. J., Hong, S., Park, S., Kim, J. H., & Shin, J. (2021) · International Journal of Cancer · doi:10.1002/ijc.33892 · Cited by 41
“Finally, we conducted a post hoc sensitivity analysis by calculating the E-value to quantify the effect of an unmeasured confounder that would have affected our findings (Supplementary Method S2).(17,18)”
MethodVanderWeele, T. J., Balboni, T. A., & Koh, H. K. (2021) · American Journal of Epidemiology · doi:10.1093/aje/kwab134 · Cited by 51
“However sensitivity analyses(22) indicate that to explain away the metaanalytic estimate for mortality, an unmeasured confounder associated with both increased attendance and reduced mortality by risk-ratios of 2.”
Self-citationMaher, G. M., Dalman, C., O’Keeffe, G. W., Kearney, P. M., McCarthy, F. P., Kenny, L. C., & Khashan, A. S. (2020) · Acta Psychiatrica Scandinavica · doi:10.1111/acps.13162 · Cited by 35
“In summary, an E-value is a continuous measure that quantifies the minimum strength of association an unmeasured confounder would need to have with both preeclampsia and ADHD in order to explain away an effect estimate (22).”
BackgroundMethodCorticosteroid treatment has no effect on hospital mortality in COVID-19 patients
Albani, F., Fusina, F., Granato, E., Capotosto, C., Ceracchi, C., Gargaruti, R., Santangelo, G., Schiavone, L., Taranto, M., Tosati, C., et al. (2021) · Scientific Reports · doi:10.1038/s41598-020-80654-x · Cited by 35
Chow, J. H., Yin, Y., Yamane, D. P., Davison, D. L., Keneally, R. J., Hawkins, K., Parr, K. G., Al‐Mashat, M., Berger, J. S., Bushardt, R. L., et al. (2021) · Journal of Thrombosis and Haemostasis · doi:10.1111/jth.15517 · Cited by 37
Association of Historical Redlining and Present-Day Social Vulnerability with Cancer Screening
Moazzam, Z., Woldesenbet, S., Endo, Y., Alaimo, L., Lima, H. A., Cloyd, J. M., Dillhoff, M., Ejaz, A. M., & Pawlik, T. M. (2023) · Journal of the American College of Surgeons · doi:10.1097/xcs.0000000000000779 · Cited by 39
McCormick, N., Yokose, C., Wei, J., Lu, N., Wexler, D. J., Aviña-Zubieta, J. A., Vera, M. A. D., Zhang, Y., & Choi, H. K. (2023) · Annals of Internal Medicine · doi:10.7326/m23-0724 · Cited by 39
Herlemann, A., Cowan, J. E., Washington, S. L., Wong, A., Broering, J. M., Carroll, P. R., & Cooperberg, M. R. (2023) · European Urology · doi:10.1016/j.eururo.2023.09.024 · Cited by 39
Patil, S. P., Billings, M. E., Bourjeily, G. R., Collop, N. A., Gottlieb, D. J., Johnson, K. G., Kimoff, R. J., & Pack, A. I. (2023) · Journal of Clinical Sleep Medicine · doi:10.5664/jcsm.10832 · Cited by 46
“132 If this is a large number, it adds confidence to the robustness of results.”
Coburn, B. W., Baker, J. F., Hsu, J. Y., Wu, Q., Xie, F., Curtis, J. R., & George, M. (2024) · Arthritis & Rheumatology · doi:10.1002/art.42928 · Cited by 40
“31,32 Other sensitivity analyses included the following: treating glucocorticoid dose as a time-varying covariate, adding cumulative glucocorticoid dose in the fi rst 9 months of the baseline period as a covariate, excluding those with COPD or asthma who have alternative reasons for glucocorticoid…”
Long-Term Kidney Outcomes after Pediatric Acute Kidney Injury
Robinson, C., Jeyakumar, N., Luo, B., Askenazi, D. J., Deep, A., Garg, A. X., Goldstein, S. L., Greenberg, J. H., Mammen, C., Nash, D. M., et al. (2024) · Journal of the American Society of Nephrology · doi:10.1681/asn.0000000000000445 · Cited by 38
“53 To address proportionality assumption violations, Cox models were time-strati fi ed using data-derived time periods, so that the proportionality assumption was met within each period.”
Kim, G., Oh, D. K., Lee, S. Y., Park, M. H., Lim, C., & the Korean Sepsis Alliance (KSA) investigators (2024) · Critical Care · doi:10.1186/s13054-024-05064-1 · Cited by 39
“The effect size of unmeasured confounders was estimated using the E-value, calculated with the website for computing E-values [23, 24].”
MethodCannabis Use Disorder Emergency Department Visits and Hospitalizations and 5-Year Mortality
Myran, D. T., Pugliese, M., McDonald, A. J., Xiao, J., Fischer, B., Finkelstein, Y., Tanuseputro, P., Firth, J., Pakpour, A. H., Hsu, C., et al. (2025) · JAMA Network Open · doi:10.1001/jamanetworkopen.2024.57852 · Cited by 40
Jüni, P., Antoniou, S., Arbelo, E., Buccheri, S., Čikeš, M., da Costa, B. R., Fauchier, L., Gale, C. P., Halvorsen, S., James, S. K., et al. (2025) · European Heart Journal · doi:10.1093/eurheartj/ehaf009 · Cited by 44
Strand, E., Rebnord, E. W., Flygel, M. R., Lysne, V., Svingen, G. F. T., Tell, G. S., Løland, K. H., Berge, R. K., Svardal, A. M., Nygard, O. K., et al. (2018) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/jc.2017-02139 · Cited by 44
“(20) Because the event rate was relatively low in this cohort, the E-value was calculated on the basis of the OR (95% CI).”
Wen, H.-J., Wang, S.-L., Chuang, Y.-C., Chen, P.-C., & Guo, Y. L. (2019) · Chemosphere · doi:10.1016/j.chemosphere.2019.05.100 · Cited by 35
Meyer, A., Rudant, J., Drouin, J., Coste, J., Carbonnel, F., & Weill, A. (2019) · Alimentary Pharmacology & Therapeutics · doi:10.1111/apt.15323 · Cited by 38
Smith, M. L., Kakuhikire, B., Baguma, C., Rasmussen, J. D., Perkins, J. M., Cooper‐Vince, C. E., Venkataramani, A. S., Ashaba, S., Bangsberg, D. R., & Tsai, A. C. (2019) · SSM - Population Health · doi:10.1016/j.ssmph.2019.100448 · Cited by 39
Preoperative Sedative-hypnotic Medication Use and Adverse Postoperative Outcomes
Gaulton, T. G., Wunsch, H., Gaskins, L. J., Leonard, C. E., Hennessy, S., Ashburn, M. A., Brensinger, C. M., Newcomb, C., Wijeysundera, D. N., Bateman, B. T., et al. (2019) · Annals of Surgery · doi:10.1097/sla.0000000000003556 · Cited by 38
“To determine the robustness of our primary association to unmeasured or uncontrolled confounding, we calculated an E value for the association between benzodiazepine or Z-drug use and adverse events after surgery.(29) The E value is the minimum strength of association, on the relative scale, that a confounder would ne…”
MethodSiu, J., McCarty, J. C., Gadkaree, S. K., Caterson, E. J., Randolph, G. W., Witterick, I., Eskander, A. E., & Bergmark, R. W. (2019) · JAMA Surgery · doi:10.1001/jamasurg.2019.3146 · Cited by 35
Childhood Pancreatitis and Risk for Incident Diabetes in Adulthood
Bendor, C. D., Bardugo, A., Zucker, I., Cukierman‐Yaffe, T., Lutski, M., Derazne, E., Shohat, T., Mosenzon, O., Tzur, D., Sapir, A., et al. (2019) · Diabetes Care · doi:10.2337/dc19-1562 · Cited by 36
“for adolescence obesity, analysis was limited to individuals with normal BMI at enrollment; and 4) to assess the robustness of the adjusted OR of the entire cohort, E-value was calculated (21,22).”
MethodEl‐Galaly, A., Nielsen, P. T., Kappel, A., & Jensen, S. L. (2020) · Acta Orthopaedica · doi:10.1080/17453674.2019.1709711 · Cited by 36
“The robustness of the estimated hazard ratios (HR) was evaluated by calculating their E-values, which estimates the magnitude of association unmeasured confounders must have with both the exposure and outcome to negate the observed HRs (Van Der Weele and Ding 2017).”
MethodSong, L., Wang, Y., Chen, B., Tan, Y., Zhang, W., & Wang, Y. (2020) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph17051525 · Cited by 36
“For the residual confounder to explain the observed association, the unmeasured risk should have a risk ratio associated with the exposure-outcome greater than the E value [18].”
BackgroundMultiple-bias Sensitivity Analysis Using Bounds
Smith, L. H., Mathur, M. B., & VanderWeele, T. J. (2021) · Epidemiology · doi:10.1097/ede.0000000000001380 · Cited by 36
Self-citationImpact of Azithromycin and/or Hydroxychloroquine on Hospital Mortality in COVID-19
Albani, F., Fusina, F., Giovannini, A., Ferretti, P., Granato, A., Prezioso, C., Divizia, D., Sabaini, A., Marri, M., Malpetti, E., et al. (2020) · Journal of Clinical Medicine · doi:10.3390/jcm9092800 · Cited by 39
“In addition, the association of patients’ characteristics with mortality was similar to that previously reported: male sex, age, and lymphopenia were independently associated with a negative outcome at hospital discharge [7,8,16].”
MethodResultPiske, M., Thomson, T. J., Krebs, E., Hongdilokkul, N., Bruneau, J., Greenland, S., Gustafson, P., Karim, M. E., McCandless, L. C., Maclure, M., et al. (2020) · BMJ Open · doi:10.1136/bmjopen-2019-036102 · Cited by 38
Scailteux, L., Campillo‐Gimenez, B., Kerbrat, S., Despas, F., Mathiéu, R., Vincendeau, S., Balusson, F., Happe, A., Nowak, E., & Oger, E. (2020) · American Journal of Epidemiology · doi:10.1093/aje/kwaa190 · Cited by 37
“We reported an E-value, calculated through a straightforward formula using the HR(25), to estimate how strongly an unmeasured confounder would have to be associated with the treatment and”
BackgroundMethodYeo, Y. H., Tseng, T., Hosaka, T., Cunningham, C. W., Fung, J. Y. Y., Ho, H. J., Kwak, M., Trinh, H. N., Ungtrakul, T., Yu, M., et al. (2020) · Clinical and Translational Gastroenterology · doi:10.14309/ctg.0000000000000196 · Cited by 39
“We also assessed goodness of fit of the model by estimating the Harrell C statistics and determined the E values of variables to assess themagnitude that an unmeasured confounder would have to explain the association between the corresponding variable and spontaneous HBsAg seroclearance (16).”
MethodIyer, H. S., James, P., Valeri, L., Bajunirwe, F., Nankya-Mutyoba, J., Njelekela, M. A., Chiwanga, F. S., Sewram, V., Ajayi, I. O., Adebamowo, C., et al. (2020) · Environmental Research · doi:10.1016/j.envres.2020.110397 · Cited by 39
Bennett, S., Coburn, N. G., Law, C., Mahar, A. L., Zhao, H., Singh, S., Zuk, V., Myrehaug, S. D., Gupta, V., Levy, J., et al. (2020) · Annals of Surgery · doi:10.1097/sla.0000000000004647 · Cited by 40
“Finally, we assessed the potential effect of unmeasured confounding on the results using the E-value methodology, which is an alternative approach to sensitivity analyses which avoid making assumptions.(33,34) This method estimates the minimum strength of the association an unmeasured confounder would need to have wit…”
MethodResultRoberts, D. J., Nagpal, S., Kubelik, D., Brandys, T., Stelfox, H. T., Lalu, M. M., Forster, A. J., McCartney, C. J. L., & McIsaac, D. I. (2020) · BMJ · doi:10.1136/bmj.m4104 · Cited by 37
Ibrahim, B., Mendonca, M. I. D. F., Gombar, S., Callahan, A., Jung, K., & Capasso, R. (2021) · JAMA Otolaryngology–Head & Neck Surgery · doi:10.1001/jamaoto.2020.5179 · Cited by 39
Dhana, K., Aggarwal, N. T., Rajan, K. B., Barnes, L. L., Evans, D. A., & Morris, M. C. (2021) · American Journal of Epidemiology · doi:10.1093/aje/kwab033 · Cited by 35
“We also assessed the robustness of our associations by computing the E value, which provides a measure of the minimum strength of the association that an unmeasured confounder would need to have with both exposure and outcome to completely explain away observed associations (35, 36).”
MethodSun, Y., Jackson, K. E., Dalmon, C. A., Shapiro, B. L., Nie, S., Wong, C., Arnold, B. F., Porco, T. C., & Acharya, N. R. (2021) · Vaccine · doi:10.1016/j.vaccine.2021.05.056 · Cited by 40
Li, S., Meng, Q., Laba, C., Guan, H., Wang, Z., Pan, Y., Wei, J., Xu, H. Y., Zeng, C., Wang, X., et al. (2022) · Ecotoxicology and Environmental Safety · doi:10.1016/j.ecoenv.2022.113851 · Cited by 37
Becerril, A. J., Pfoh, E. R., Hashmi, A. Z., Mourany, L., Gunzler, D. D., Berg, K. A., Krieger, N. I., Krishnan, K., Moore, S. E., Kahana, E. F., et al. (2023) · Journal of the American Geriatrics Society · doi:10.1111/jgs.18322 · Cited by 38
Kondracki, A. J., Valente, M. J., Ibrahimou, B., & Bursac, Z. (2022) · Paediatric and Perinatal Epidemiology · doi:10.1111/ppe.12809 · Cited by 34
Madaniyazi, L., Jung, C., Ng, C. F. S., Seposo, X., Hashizume, M., & Nakayama, S. F. (2021) · Environment International · doi:10.1016/j.envint.2021.107004 · Cited by 43
Bruzelius, E., Cerdá, M., Davis, C. S., Jent, V. A., Wheeler-Martin, K., Mauro, C., Crystal, S., Keyes, K. M., Samples, H., Hasin, D. S., et al. (2023) · International Journal of Drug Policy · doi:10.1016/j.drugpo.2023.103980 · Cited by 36
Suicide and Transportation Noise: A Prospective Cohort Study from Switzerland
Wicki, B., Schäffer, B., Wunderli, J. M., Müller, T., Pervilhac, C., Röösli, M., & Vienneau, D. (2023) · Environmental Health Perspectives · doi:10.1289/ehp11587 · Cited by 43
“61 The absolute excess risk for the main fi ndings was calculated by multiplying the suicides/100,000 person-years (PY) by the excess risk (HR-1).”
Ide, K., Nakagomi, A., Tsuji, T., Yamamoto, T., Watanabe, R., Yokoyama, M., Shirai, K., Kondo, K., & Shiba, K. (2023) · Innovation in Aging · doi:10.1093/geroni/igad084 · Cited by 39
“To evaluate the robustness of the estimated associations to unmeasured confounding, we calculated E -values for each exposure-outcome association (45).”
MethodShi, J., Huang, J., Guo, M., Tian, L., Wang, J. P., Wong, T. W., Webster, C. J., Leung, G. M., & Ni, M. Y. (2023) · Environmental Pollution · doi:10.1016/j.envpol.2023.122641 · Cited by 45
Smit, J. M., Krijthe, J. H., Kant, W. M. R., Labrecque, J. A., Komorowski, M., Gommers, D. A. M. P. J., van Bommel, J., Reinders, M. J. T., & van Genderen, M. E. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00961-1 · Cited by 45
“For the conditional exchangeability assumption, two types of attempts were differentiated: ‘ indirect approaches ’ 11 and ‘ bias analyses ’ 71 .”
Cipolletta, E., Nakafero, G., McCormick, N., Yokose, C., Avery, A. J., Mamas, M. A., Choi, H. K., Tata, L. J., & Abhishek, A. (2024) · The Lancet Rheumatology · doi:10.1016/s2665-9913(24)00248-0 · Cited by 45
Lee, Y., Han, K., Kim, B., Jun, J. E., Lee, S., Ahn, J., Kim, G., Jin, S., & Kim, J. H. (2019) · Diabetes/Metabolism Research and Reviews · doi:10.1002/dmrr.3197 · Cited by 37
Peripheral Nerve Blocks for Ambulatory Shoulder Surgery
Hamilton, G. M., Ramlogan, R. R., Lui, A. C. P., McCartney, C. J. L., Abdallah, F. W., McVicar, J. A., & McIsaac, D. I. (2019) · Anesthesiology · doi:10.1097/aln.0000000000002865 · Cited by 35
Mao, R., Li, K., Cai, J., Luo, S., Turner, M. C., Blazer, D. G., & Zhao, H. (2019) · Annals of Surgery · doi:10.1097/sla.0000000000003562 · Cited by 36
Whitney, D. G., Bell, S., Etter, J. P., & Prisby, R. D. (2020) · Bone · doi:10.1016/j.bone.2020.115376 · Cited by 36
Metadata
- Authors
- VanderWeele, T. J., & Ding, P.
- Affiliations
- Harvard University (US)
- Venue
- Annals of Internal Medicine
- Publisher
- American College of Physicians
- Published
- 2017-08-15
- Volume
- 167
- Issue
- 4
- Pages
- 268-274
- ISSN
- 0003-4819 (ISSN-L)
- 1539-3704
- OpenAlex
- W2735301415
- Cited by
- 6,350
- References
- 59
- Citations by year
- 2026: 1516 · 2025: 1020 · 2024: 860 · 2023: 736 · 2022: 716 · 2021: 583 · 2020: 498 · 2019: 290 · 2018: 113 · 2017: 15
- FWCI
- 174.58
- Type
- journal article
- Language
- en
- Open access
- green
- nrs.harvard.edu
- Licence
- cc-by
- www.acpjournals.org/journal/aim/text-and-data-mining (tdm)
- Locations
- Annals of Internal Medicine (publishedVersion)
- PubMed (publishedVersion)
- Digital Access to Scholarship at Harvard (DASH) (Harvard University) (submittedVersion, cc-by)
- Funders
- National Institute of Environmental Health Sciences (ES017876)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- E-value, unmeasured confounding, sensitivity analysis, causal inference, observational studies, relative risk
- MeSH terms
- Humans, Sensitivity and Specificity, Epidemiologic Research Design, Confounding Factors, Epidemiologic, Observational Studies as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.

