Sensitivity Analysis in Observational Research: Introducing the E-Value
Open accessThe DOI matches its Crossref record; 58 of the 59 references it cites were found in a registry or an index.
Abstract
Sensitivity analysis is useful in assessing how robust an association is to potential unmeasured or uncontrolled confounding. This article introduces a new measure called the "E-value," which is related to the evidence for causality in observational studies that are potentially subject to confounding. The E-value is defined as the minimum strength of association, on the risk ratio scale, that an unmeasured confounder would need to have with both the treatment and the outcome to fully explain away a specific treatment-outcome association, conditional on the measured covariates. A large E-value implies that considerable unmeasured confounding would be needed to explain away an effect estimate. A small E-value implies little unmeasured confounding would be needed to explain away an effect estimate. The authors propose that in all observational studies intended to produce evidence for causality, the E-value be reported or some other sensitivity analysis be used. They suggest calculating the E-value for both the observed association estimate (after adjustments for measured confounders) and the limit of the confidence interval closest to the null. If this were to become standard practice, the ability of the scientific community to assess evidence from observational studies would improve considerably, and ultimately, science would be strengthened.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Quantitative bias analyses, such as the calculation of E-values to estimate the strength of unmeasured confounding required to explain away an observed association, provide useful context for the robustness of findings.64”
From Artificial Intelligence in Oncology: Practical Applications Across Clinical Care, Scholarship, and Translation. (2026) · cites this paper as [64]
The cited source says what the claim attributes to it.
In the citing paper
“We performed the sensitivity analysis using the E value [29] to evaluate the robustness of our findings to potential unmeasured confounding, with the calculation based on the observed odds ratio (OR) point estimate for rare outcome (<15%) or the square root of OR otherwise.”
The cited source says what the claim attributes to it.
In the citing paper
“To further assess the robustness of our findings to potential unmeasured confounding, we conducted an E-value analysis, a quantitative method for sensitivity analysis in observational research [31].”
From Acute Respiratory Tract Infection and Sudden Sensorineural Hearing Loss: A Multinational Cohort Study (2025) · cites this paper as [31]
In the citing paper
“We also computed E-values: indicators of how robust findings are to potential unmeasured confounding”
From Leading a meaningful life at older ages and its relationship with social engagement, prosperity, health, biology, and time use (2019) · cites this paper as [12]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009) · doi:10.1002/9780470743386 · Cited by 14,225
The environment and disease: association or causation?
Hill, A. B. (2015) · Journal of the Royal Society of Medicine · doi:10.1177/0141076814562718 · Cited by 7,285
The Environment and Disease: Association or Causation?
Hill, A. B. (1965) · Proceedings of the Royal Society of Medicine · doi:10.1177/003591576505800503 · Cited by 6,974 · cited 1×
“Sensitivity analysis makes clear quantitatively why Bradford Hill's criterion of “strength of association” (12) is important in establishing that a given association is, in fact, causal.”
BackgroundThe ASA Statement on p -Values: Context, Process, and Purpose
Wasserstein, R. L., & Lazar, N. A. (2016) · The American Statistician · doi:10.1080/00031305.2016.1154108 · Cited by 6,306 · cited 2×
“While potentially subject to mis-use and misinterpretation (1,5-12), the p-value can be informative as a continuous measure of evidence as to whether an association is present.”
BackgroundCausal Inference for Statistics, Social, and Biomedical Sciences: An Introduction
Imbens, G. W., & Rubin, D. B. (2015) · doi:10.1017/cbo9781139025751 · Cited by 4,210
Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations
Greenland, S., Senn, S. J., Rothman, K. J., Carlin, J. B., Poole, C., Goodman, S. N., & Altman, D. G. (2016) · European Journal of Epidemiology · doi:10.1007/s10654-016-0149-3 · Cited by 3,200
Explanation in Causal Inference: Methods for Mediation and Interaction
VanderWeele, T. J. (2015) · Cited by 2,217
Self-citationSmithson, M. (2000) · SAGE Publications Ltd eBooks · doi:10.4135/9781446218525 · Cited by 1,871
Fundamentals of Biostatistics.
Cornell, R. G., & Rosner, B. (1987) · Journal of the American Statistical Association · doi:10.2307/2289416 · Cited by 1,603
Breastfeeding and Maternal and Infant Health Outcomes In Developed Countries
(2007) · AAP Grand Rounds · doi:10.1542/gr.18-2-15 · Cited by 1,540 · cited 2×
“A study by the Agency for Healthcare Research and Quality (42) reported the association between breastfeeding and childhood leukemia as RR=0.”
MethodInfluential
Research works printed but not linked: 3
- Greenland S, 2011, Jurimetrics, 51, 11329
- Cornfield J, 1959, J Natl Cancer Inst, 22, 173
- Schlesselman JJ, 1978, Am J Epidemiol, 108, 3
Cited by
Items 201–300 of 1,031
GLP-1RA and SGLT2i Medications for Type 2 Diabetes and Alzheimer Disease and Related Dementias
Tang, H., Donahoo, W. T., DeKosky, S. T., Lee, Y. A., Kotecha, P., Svensson, M., Bian, J., & Guo, J. S. (2025) · JAMA Neurology · doi:10.1001/jamaneurol.2025.0353 · Cited by 116
Disparities in Care and Mortality Among Homeless Adults Hospitalized for Cardiovascular Conditions
Wadhera, R. K., Khatana, S. A. M., Choi, E. H., Jiang, G. Y., Shen, C., Yeh, R. W., & Maddox, K. E. J. (2019) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2019.6010 · Cited by 103
Impact Of Long-Term Buprenorphine Treatment On Adverse Health Care Outcomes In Medicaid
Samples, H., Williams, A. R., Crystal, S., & Olfson, M. (2020) · Health Affairs · doi:10.1377/hlthaff.2019.01085 · Cited by 106
Kjellbom, A., Lindgren, O., Puvaneswaralingam, S., Löndahl, M., & Olsen, H. (2021) · Annals of Internal Medicine · doi:10.7326/m20-7946 · Cited by 98
“Sensitivity analysis was used to determine the minimum association an unmeasured confounder would need to have with the covariate and outcome to have the possibility to explain the result (21).”
BackgroundMethodInfluentialStatin Discontinuation and Cardiovascular Events Among Older People in Denmark
Thompson, W., Morin, L., Jarbøl, D. E., Andersen, J. H., Ernst, M. T., Nielsen, J. B., Haastrup, P., Schmidt, M., & Pottegård, A. (2021) · JAMA Network Open · doi:10.1001/jamanetworkopen.2021.36802 · Cited by 98
Hong, J. H., Nakamura, J. S., Berkman, L. F., Chen, F. S., Shiba, K., Chen, Y., Kim, E. S., & VanderWeele, T. J. (2023) · SSM - Population Health · doi:10.1016/j.ssmph.2023.101459 · Cited by 128
Self-citationSingla, D. R., Shinde, S., Patton, G. C., & Patel, V. (2020) · Journal of Adolescent Health · doi:10.1016/j.jadohealth.2020.09.030 · Cited by 117
Jorgensen, S. C. J., Zasowski, E. J., Trinh, T. D., Lagnf, A. M., Bhatia, S., Sabagha, N., Abdul‐Mutakabbir, J. C., Alosaimy, S., Mynatt, R. P., Davis, S. L., et al. (2019) · Clinical Infectious Diseases · doi:10.1093/cid/ciz746 · Cited by 105
Deschasaux, M., Huybrechts, I., Julia, C., Herçberg, S., Egnell, M., Srour, B., Kesse‐Guyot, E., Latino‐Martel, P., Biessy, C., Casagrande, C., et al. (2020) · BMJ · doi:10.1136/bmj.m3173 · Cited by 126
Sun, Y., Kim, E. H., Kong, C., Arnold, B. F., Porco, T. C., & Acharya, N. R. (2021) · Clinical Infectious Diseases · doi:10.1093/cid/ciab121 · Cited by 112
Life’s Essential 8 and Risk of Stroke: A Prospective Community-Based Study
Wu, S., Wu, Z., Yu, D., Chen, S., Wang, A., Wang, A., & Gao, X. (2023) · Stroke · doi:10.1161/strokeaha.123.042525 · Cited by 103
Real-world studies no substitute for RCTs in establishing efficacy
Gerstein, H. C., McMurray, J., & Holman, R. R. (2019) · The Lancet · doi:10.1016/s0140-6736(18)32840-x · Cited by 97
Gestational diabetes and the risk of late stillbirth: a case–control study from England, UK
Stacey, T., Tennant, P., McCowan, L., Mitchell, E., Budd, J., Li, M., Thompson, J., Martin, B., Roberts, D., & Heazell, A. (2019) · BJOG: An International Journal of Obstetrics & Gynaecology · doi:10.1111/1471-0528.15659 · Cited by 95
Hagström, H., Thiele, M., Roelstraete, B., Söderling, J., & Ludvigsson, J. F. (2021) · Gut · doi:10.1136/gutjnl-2019-320446 · Cited by 97
“19 In brief, this is a method to estimate the effect an unmeasured confounder, such as smoking, would need to have to completely obviate the findings of an association.”
Permanent Pacemaker Reduction Using Cusp-Overlapping Projection in TAVR
Pascual, I., Hernández‐Vaquero, D., Alperi, A., Almendárez, M., y, P. A., Kalavrouziotis, D., Lorca, R., Mesnier, J., Arboine, L. A., Mohammadi, S., et al. (2022) · JACC: Cardiovascular Interventions · doi:10.1016/j.jcin.2021.10.002 · Cited by 97
Nedelec, T., Couvy-Duchesne, B., Monnet, F., Daly, T., Ansart, M., Gantzer, L., Lekens, B., Epelbaum, S., Dufouil, C., & Durrleman, S. (2022) · The Lancet Digital Health · doi:10.1016/s2589-7500(21)00275-2 · Cited by 96
Baumeister, S. E., Schlesinger, S., Aleksandrova, K., Jochem, C., Jenab, M., Gunter, M. J., Overvad, K., Tjønneland, A., Boutron-Ruault, M.-C., Carbonnel, F., et al. (2019) · Journal of Hepatology · doi:10.1016/j.jhep.2018.12.014 · Cited by 95
Religious service attendance, health behaviors and well-being—an outcome-wide longitudinal analysis
Pawlikowski, J., Białowolski, P., Węziak‐Białowolska, D., & VanderWeele, T. J. (2019) · European Journal of Public Health · doi:10.1093/eurpub/ckz075 · Cited by 107
Self-citationSmith, N. A., Voisin, D. R., Yang, J. P., & Tung, E. L. (2019) · Health Affairs · doi:10.1377/hlthaff.2019.00560 · Cited by 119
Zhong, G.-C., Gu, H.-T., Peng, Y., Wang, K., Wu, Y.-Q.-L., Hu, T.-Y., Jing, F.-C., & Hao, F.-B. (2021) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/s12966-021-01081-3 · Cited by 96
Antithyroid Drugs and Congenital Malformations: A Nationwide Korean Cohort Study
Seo, G. H., Kim, T. H., & Chung, J. H. (2018) · Annals of Internal Medicine · doi:10.7326/m17-1398 · Cited by 108
“We calculated E-values as a sensitivity analysis to assess the robustness of results to potential residual or unmeasured confounders (20, 21).”
MethodAmoah, J., Stuart, E. A., Cosgrove, S. E., Harris, A. D., Han, J., Lautenbach, E., & Tamma, P. D. (2020) · Clinical Infectious Diseases · doi:10.1093/cid/ciaa169 · Cited by 99
“Sensitivity analyses (eg, calculating an E-value) can be considered to better assess how much an effect estimate is subject to unmeasured confounding [36, 37].”
BackgroundA prospective study of type 2 diabetes, metformin use, and risk of breast cancer
Park, Y., Bookwalter, D. B., O’Brien, K. M., Jackson, C. L., Weinberg, C. R., & Sandler, D. P. (2021) · Annals of Oncology · doi:10.1016/j.annonc.2020.12.008 · Cited by 105
“These values are not especially large because our observed HR are not large.[27] We adjusted for a wide range of known or potential confounders.”
BackgroundMethodCullen, P., Möller, H., Woodward, M., Senserrick, T. M., Boufous, S., Rogers, K. D., Brown, J. K., & Ivers, R. (2021) · SSM - Population Health · doi:10.1016/j.ssmph.2021.100816 · Cited by 122
Tran, V.-T., Perrodeau, É., Saldanha, J. S., Pane, I., & Ravaud, P. (2023) · BMJ Medicine · doi:10.1136/bmjmed-2022-000229 · Cited by 146
“26 We performed several sensitivity analyses.”
Demographic, lifestyle, and reproductive risk factors for ectopic pregnancy
Gaskins, A. J., Missmer, S. A., Rich-Edwards, J. W., Williams, P. L., Souter, I. C., & Chavarro, J. E. (2018) · Fertility and Sterility · doi:10.1016/j.fertnstert.2018.08.022 · Cited by 101
“We also plotted the value of the joint minimum strength of association that a history of STIs/PID must have with the exposure (RREU) and the outcome (RRUD) to fully explain away the observed associations.”
MethodInfluentialBerkowitz, S. A., Kalkhoran, S. M., Edwards, S. T., Essien, U. R., & Baggett, T. P. (2018) · Diabetes Care · doi:10.2337/dc17-1812 · Cited by 102
“In sensitivity analyses designed to detect the amount of unmeasured confounding that would be needed to explain away this association (29), we found”
BackgroundHeffler, K. F., Sienko, D. M., Subedi, K., McCann, K. A., & Bennett, D. S. (2020) · JAMA Pediatrics · doi:10.1001/jamapediatrics.2020.0230 · Cited by 92
Iqbal, R., Dehghan, M., Mente, A., Rangarajan, S., Wielgosz, A. T., Avezum, Á., Serón, P., AlHabib, K. F., López-Jaramillo, P. A., Swaminathan, S., et al. (2021) · American Journal of Clinical Nutrition · doi:10.1093/ajcn/nqaa448 · Cited by 104
“Further, we examined the association between processed meat and death due to injury as a negative control and computed E-values using the VanderWeele method (13).”
MethodKandola, A. A., del Pozo Cruz, B., Osborn, D. P. J., Stubbs, B., Choi, K. W., & Hayes, J. F. (2021) · BMC Medicine · doi:10.1186/s12916-021-02007-3 · Cited by 91
“tions between our exposure and outcomes while accounting for all measured covariates [55].”
BackgroundHjorthøj, C. R., Posselt, C. M., & Nordentoft, M. (2021) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2021.1471 · Cited by 100
Li, R., Ware, J. L., Chen, A., Nelson, J. M., Kmet, J. M., Parks, S. E., Morrow, A. L., Chen, J., & Perrine, C. G. (2021) · The Lancet Regional Health - Americas · doi:10.1016/j.lana.2021.100094 · Cited by 109
“To address how robust our findings are to potential uncontrolled confounding, we have conducted a sensitivity analyses using E-value.32 To explain away the observed associations between breastfeeding and overall infant death, late-neonatal death, and post-neonatal death, the minimum strength of the…”
InfluentialMorrison, A. E., Zaccardi, F., Khunti, K., & Davies, M. J. (2019) · Liver International · doi:10.1111/liv.13994 · Cited by 92
Newbury, J. B., Stewart, R., Fisher, H. L., Beevers, S., Dajnak, D., Broadbent, M., Pritchard, M., Shiode, N., Heslin, M., Hammoud, R., et al. (2021) · The British Journal of Psychiatry · doi:10.1192/bjp.2021.119 · Cited by 92
Lifestyle factors for the prevention of inflammatory bowel disease
Lopes, E. W., Chan, S. S. M., Song, M., Ludvigsson, J. F., Håkansson, N., Lochhead, P. J., Clark, A., Burke, K. E., Ananthakrishnan, A. N., Cross, A. J., et al. (2022) · Gut · doi:10.1136/gutjnl-2022-328174 · Cited by 104
Fernainy, P., Cohen, A. A., Murray, E. J., Losina, E., Lamontagne, F., & Sourial, N. (2024) · BMC Proceedings · doi:10.1186/s12919-023-00285-8 · Cited by 154
“…o n a l o n t h e m e a s u r e d c o v a r i a t e s ” [ 24 ] T h e E - v a l u e i s a n i n t u i t i v e m e t r i c t o h e l p d e t e r m i n e h o w r o b u s t t h e r e s u l t s o f a s t u d y a r e t o u n m e a s u r e d c o n f o u n d i n g [ 24 ] B e n d e r I g n a c i o e t a l .”
BackgroundInfluentialWater insecurity and gendered risk for depression in rural Uganda: a hotspot analysis
Cooper-Vince, C. E., Arachy, H., Kakuhikire, B., Vořechovská, D., Mushavi, R. C., Baguma, C., McDonough, A. Q., Bangsberg, D. R., & Tsai, A. C. (2018) · BMC Public Health · doi:10.1186/s12889-018-6043-z · Cited by 90
Laureati, P., Xu, Y., Trevisan, M., Schalin, L., Mariani, I., Bellocco, R., Sood, M. M., Barany, P. F., Sjölander, A., Evans, M., et al. (2019) · Nephrology Dialysis Transplantation · doi:10.1093/ndt/gfz150 · Cited by 98
Obesity and Risk for First Ischemic Stroke Depends on Metabolic Syndrome: The HUNT Study
Horn, J. W., Feng, T., Mørkedal, B., Strand, L. B., Horn, J., Mukamal, K. J., & Janszky, I. (2021) · Stroke · doi:10.1161/strokeaha.120.033016 · Cited by 97
Ingason, A. B., Hreinsson, J. P., Agustsson, A. S., Lund, S. H., Rumba, E., Pálsson, D. A., Reynisson, I. E., Guðmundsdóttir, B. R., Onundarson, P. T., & Bjornsson, E. S. (2021) · Annals of Internal Medicine · doi:10.7326/m21-1474 · Cited by 105
“To assess the effect of potential unmeasured confounders, we calculated the Evalue for comparisons of GIB rates (32).”
MethodRenal Transplantation and Survival Among Patients With Lupus Nephritis: A Cohort Study
Jorge, A. M., Wallace, Z. S., Lu, N., Zhang, Y., & Choi, H. K. (2019) · Annals of Internal Medicine · doi:10.7326/m18-1570 · Cited by 98
“We calculated E-values in a sensitivity analysis to assess the potential effect of unmeasured confounders (23).”
Selection Mechanisms and Their Consequences: Understanding and Addressing Selection Bias
Smith, L. H. (2020) · Current Epidemiology Reports · doi:10.1007/s40471-020-00241-6 · Cited by 115
Inoue‐Choi, M., Ramirez, Y., Cornelis, M. C., de González, A. B., Freedman, N. D., & Loftfield, E. (2022) · Annals of Internal Medicine · doi:10.7326/m22-0041 · Cited by 107
“We have performed a formal quantitative sensitivity analysis and computed the E-value, which is the minimum strength of association with both the exposure and outcome needed for an unmeasured confounder to explain an observed risk (34, 35).”
MethodProton Pump Inhibitor Use and Risk of Serious Infections in Young Children
Lassalle, M., Zureik, M., & Dray‐Spira, R. (2023) · Archives of Pediatrics and Adolescent Medicine · doi:10.1001/jamapediatrics.2023.2900 · Cited by 104
Sotos‐Prieto, M., Ortolá, R., Ruiz‐Canela, M., García‐Esquinas, E., Martínez‐Gómez, D., López‐García, E., Martínez‐González, M. Á., & Rodríguez‐Artalejo, F. (2021) · Cardiovascular Diabetology · doi:10.1186/s12933-020-01195-1 · Cited by 98
“In addition, despite extensive adjustment for known risk factors, some residual confounding may persist; to address this issue we calculated the E-value [42], and it showed that after accounting for confounders measured in this study, the HR of 0.58 for total mortality could only be explained away by an unmeasured con…”
BackgroundMethodInfluentialAssociation of Medical Financial Hardship and Mortality Among Cancer Survivors in the United States
Yabroff, K. R., Han, X., Song, W., Zhao, J., Nogueira, L., Pollack, C. E., Jemal, A., & Zheng, Z. (2022) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/djac044 · Cited by 102
Qiu, X., Yazdi, M. D., Wei, Y., Di, Q., Just, A. C., Zanobetti, A., Weisskopf, M. G., Dominici, F., & Schwartz, J. D. (2022) · The Lancet Planetary Health · doi:10.1016/s2542-5196(22)00017-1 · Cited by 100
“The robustness of the estimates was assessed with respect to moving averages of lag day of choice; residual confounding (e-value); 36 alternative adjustment of specific humidity; alternative adjustment of moving average of lag0 to 3-day temperature exposure; and temperature-only modelling.”
Immune correlates analysis of the ENSEMBLE single Ad26.COV2.S dose vaccine efficacy clinical trial
Fong, Y., McDermott, A. B., Benkeser, D. C., Roels, S., Stieh, D. J., Vandebosch, A., Gars, M. L., Roey, G. A. V., Houchens, C. R., Martins, K. A. O., et al. (2022) · Nature Microbiology · doi:10.1038/s41564-022-01262-1 · Cited by 106
“The first sensitivity analysis, based on E -values 28 , assessed the robustness of the inference that vaccine efficacy is greater at High vs Low ID 50 tertile.”
Wicklow, B. A., Sellers, E. A. C., Sharma, A. K., Kroeker, K., Nickel, N. C., Philips-Beck, W., & Shen, G. X. (2018) · JAMA Pediatrics · doi:10.1001/jamapediatrics.2018.1201 · Cited by 88
“A sensitivity analysis was performed to investigate the association of unmeasured confounders.(25) Dataanalysiswascompletedusingstatistical software (SAS, version9.”
MethodOhbe, H., Jo, T., Yamana, H., Matsui, H., Fushimi, K., & Yasunaga, H. (2018) · Intensive Care Medicine · doi:10.1007/s00134-018-5319-1 · Cited by 88
Anderson, T. S., Jing, B., Auerbach, A. D., Wray, C. M., Lee, S., Boscardin, W. J., Fung, K. Z., Ngo, S., Silvestrini, M. C., & Steinman, M. A. (2019) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2019.3007 · Cited by 88
Patel, D. K., Duncan, M. S., Shah, A. S., Lindman, B. R., Greevy, R. A., Savage, P. D., Whooley, M. A., Matheny, M. E., Freiberg, M. S., & Bachmann, J. M. (2019) · JAMA Cardiology · doi:10.1001/jamacardio.2019.4032 · Cited by 90
Hallet, J. I., Jerath, A., Turgeon, A. F., McIsaac, D. I., Eskander, A. E., Zuckerman, J., Zuk, V., Sohail, S., Darling, G. E., Dharma, C., et al. (2021) · JAMA Surgery · doi:10.1001/jamasurg.2021.0135 · Cited by 93
Satinsky, E. N., Kakuhikire, B., Baguma, C., Rasmussen, J. D., Ashaba, S., Cooper‐Vince, C. E., Perkins, J. M., Kiconco, A., Namara, E. B., Bangsberg, D. R., et al. (2021) · PLoS Medicine · doi:10.1371/journal.pmed.1003642 · Cited by 103
“To probe the robustness of our findings to confounding by unobserved variables, we used methods proposed by Vanderweele and Ding [71].”
MethodKhan, T. A., Field, D., Chen, V., Ahmad, S., Mejía, S. B., Kahleová, H., Rahelić, D., Salas‐Salvadó, J., Leiter, L. A., Uusitupa, M. I. J., et al. (2023) · Diabetes Care · doi:10.2337/dc22-1024 · Cited by 104
“We also computed the E-value to ascertain the effect of an unmeasured or uncontrolled confounder on the exposureoutcome relationship (26).”
BackgroundMethodIob, E., Lacey, R. E., & Steptoe, A. (2019) · Brain Behavior and Immunity · doi:10.1016/j.bbi.2019.12.019 · Cited by 99
“This allowed us to determine the average effect required for any possible unmeasured confounder to fully explain the associations between the ACEs score and the biomarkers (VanderWeele and Ding, 2017).”
BackgroundOptimal Vasopressin Initiation in Septic Shock: The OVISS Reinforcement Learning Study
Kalimouttou, A., Kennedy, J. N., Feng, J., Singh, H., Saria, S., Angus, D. C., Seymour, C. W., & Pirracchio, R. (2025) · JAMA · doi:10.1001/jama.2025.3046 · Cited by 81
Methods to Address Confounding and Other Biases in Meta-Analyses: Review and Recommendations
Mathur, M. B., & VanderWeele, T. J. (2022) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-051920-114020 · Cited by 85
Self-citationAssociation of Oral Corticosteroid Bursts With Severe Adverse Events in Children
Yao, T., Wang, J., Chang, S., Chang, Y., Tsai, Y.-F., Wu, A. C., Huang, J., & Tsai, H. (2021) · Archives of Pediatrics and Adolescent Medicine · doi:10.1001/jamapediatrics.2021.0433 · Cited by 93
Digital health intervention in acute myocardial infarction
Marvel, F. A., Spaulding, E. M., Lee, M. A., Yang, W. E., Demo, R., Ding, J., Wang, J., Xun, H., Shah, L. M., Weng, D., et al. (2021) · Circulation Cardiovascular Quality and Outcomes · doi:10.1161/circoutcomes.121.007741 · Cited by 84
Okuzono, S. S., Shiba, K., Kim, E. S., Shirai, K., Kondo, N., Fujiwara, T., Kondo, K., Lomas, T., Trudel‐Fitzgerald, C., Kawachi, I. I., et al. (2022) · The Lancet Regional Health - Western Pacific · doi:10.1016/j.lanwpc.2022.100391 · Cited by 113
Self-citationRe, V. L., Dutcher, S. K., Connolly, J. G., Pérez-Vilar, S., Carbonari, D. M., DeFor, T. A., Djibo, D. A., Harrington, L. B., Hou, L., Hennessy, S., et al. (2022) · JAMA · doi:10.1001/jama.2022.13072 · Cited by 90
Association of Routine Preoperative Frailty Assessment With 1-Year Postoperative Mortality
Varley, P. R., Buchanan, D., Bilderback, A. L., Wisniewski, M. K., Johanning, J. M., Nelson, J. B., Johnson, J. T., Minnier, T. E., & Hall, D. E. (2023) · JAMA Surgery · doi:10.1001/jamasurg.2022.8341 · Cited by 94
Axial symptoms predict mortality in patients with Parkinson disease and subthalamic stimulation
Lau, B., Meier, N., Serra, G., Czernecki, V., Schüpbach, M. W. M., Navarro, S., Cornu, P., Grabli, D., Agid, Y., Vidailhet, M., et al. (2019) · Neurology · doi:10.1212/wnl.0000000000007562 · Cited by 83
“We performed a sensitivity analysis and calculated E-values to determine the degree to which potential unmeasured confounding could explain away associations between symptom severity and death.(15) The confounder associations are represented on risk ratio scale, and the HRs for current outcome severity from the joint…”
MethodTovoli, F., Ielasi, L., Gardini, A. C., Granito, A., Foschi, F. G., Rovesti, G., Negrini, G., Orsi, G., Renzulli, M., & Piscaglia, F. (2019) · Journal of Hepatology · doi:10.1016/j.jhep.2019.08.015 · Cited by 83
Association of Anticoagulant Therapy With Risk of Fracture Among Patients With Atrial Fibrillation
Lutsey, P. L., Norby, F. L., Ensrud, K. E., MacLehose, R. F., Diem, S. J., Chen, L. Y., & Alonso, Á. (2019) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2019.5679 · Cited by 89
Long, K. N. G., Worthington, E. L., VanderWeele, T. J., & Chen, Y. (2020) · BMC Psychology · doi:10.1186/s40359-020-00470-w · Cited by 101
“First, we calculated E-values to evaluate robustness of the associations to potential unmeasured confounding [28, 29].”
BackgroundMethodSelf-citationSheehy, O., Zhao, J., & Bérard, A. (2019) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2019.0963 · Cited by 85
Lau, W. C. Y., Cheung, C., Man, K. K. C., Chan, E. W., Sing, C., Lip, G. Y. H., Siu, C., Lam, J. K. Y., Lee, A. C. H., & Wong, I. C. K. (2020) · Annals of Internal Medicine · doi:10.7326/m19-3671 · Cited by 87
Siedner, M. J., Moorhouse, M. A., Simmons, B., de Oliveira, T., Lessells, R. J., Giandhari, J., Kemp, S. A., Chimukangara, B., Akpomiemie, G., Serenata, C. M., et al. (2020) · Nature Communications · doi:10.1038/s41467-020-19801-x · Cited by 84
Application of an analytical framework for multivariate mediation analysis of environmental data
Aung, M. T., Song, Y., Ferguson, K. K., Cantonwine, D. E., Zeng, L., McElrath, T. F., Pennathur, S., Meeker, J. D., & Mukherjee, B. (2020) · Nature Communications · doi:10.1038/s41467-020-19335-2 · Cited by 90
Schwahn, C., Frenzel, S., Holtfreter, B., der Auwera, S. V., Pink, C., Bülow, R., Friedrich, N., Völzke, H., Biffar, R., Kocher, T., et al. (2021) · Alzheimer s & Dementia · doi:10.1002/alz.12378 · Cited by 94
Textbook Outcome and Survival in Patients With Gastric Cancer
Levy, J., Gupta, V., Amirazodi, E., Allen-Ayodabo, C., Jivraj, N. K., Jeong, Y., Davis, L., Mahar, A. L., de Mestral, C., Saarela, O., et al. (2020) · Annals of Surgery · doi:10.1097/sla.0000000000003849 · Cited by 88
“We computed an E-value for the primary analysis using an online platform.(23,24)”
MethodThe effect of screening on melanoma incidence and biopsy rates
Whiteman, D. C., Olsen, C. M., MacGregor, S., Law, M. H., Thompson, B. S., Dusingize, J. C., Green, A. C., Neale, R. Ε., Pandeya, N., & Study, F. T. Q. (2022) · British Journal of Dermatology · doi:10.1111/bjd.21649 · Cited by 81
Associations of “Weekend Warrior” Physical Activity With Incident Disease and Cardiometabolic Health
Kany, S., Al‐Alusi, M. A., Rämö, J. T., Pirruccello, J. P., Churchill, T. W., Lubitz, S. A., Maddah, M., Guseh, J. S., Ellinor, P. T., & Khurshid, S. S. (2024) · Circulation · doi:10.1161/circulationaha.124.068669 · Cited by 91
“17 Fourth, we compared weekend warrior and regular activity without adjustment for MVPA volume.”
Lee, K. K., Spath, N., Miller, M. R., Mills, N. L., & Shah, A. S. V. (2020) · Environment International · doi:10.1016/j.envint.2020.105901 · Cited by 80
Bosch, N. A., Teja, B. J., Law, A. C., Pang, B., Jafarzadeh, S. R., & Walkey, A. J. (2023) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2023.0258 · Cited by 91
Kioumourtzoglou, M.-A., Coull, B. A., O’Reilly, É. J., Ascherio, A., & Weisskopf, M. G. (2018) · JAMA Pediatrics · doi:10.1001/jamapediatrics.2018.0727 · Cited by 79
Gaudino, M. F. L., Samadashvili, Z., Hameed, I., Chikwe, J., Girardi, L. N., & Hannan, E. L. (2020) · JAMA Cardiology · doi:10.1001/jamacardio.2020.6585 · Cited by 79
Bayesian causal inference: a critical review
Li, F., Ding, P., & Mealli, F. (2023) · Philosophical Transactions of the Royal Society A Mathematical Physical and Engineering Sciences · doi:10.1098/rsta.2022.0153 · Cited by 178
“Based on that theory, VanderWeele and Ding [110] further simplified by assuming the two associations to be the same and called the resulting threshold the E-value , as a measure of robustness of the causal conclusions with respect to unmeasured confounding.”
BackgroundInfluentialSelf-citationHow to use directed acyclic graphs: guide for clinical researchers
Feeney, T., Hartwig, F. P., & Davies, N. M. (2025) · BMJ · doi:10.1136/bmj-2023-078226 · Cited by 103
“…on a key source of confounding, researchers can consider whether the question can be reliably answered with available data or whether sufficient sensitivity analyses and quantitative bias analysis, 40-42 aided by the graph, can be designed to better answer the question with the variables at hand.”
Raaschou, P., Söderling, J. K., Turesson, C., & Askling, J. (2018) · Annals of Internal Medicine · doi:10.7326/m17-2812 · Cited by 80
Association Between Proton Pump Inhibitor Use and Risk of Asthma in Children
Wang, Y., Wintzell, V., Ludvigsson, J. F., Svanström, H., & Pasternak, B. (2021) · Archives of Pediatrics and Adolescent Medicine · doi:10.1001/jamapediatrics.2020.5710 · Cited by 78
Simple Sensitivity Analysis for Differential Measurement Error
VanderWeele, T. J., & Li, Y. (2019) · American Journal of Epidemiology · doi:10.1093/aje/kwz133 · Cited by 81
“Some similar simple sensitivity analysis results for unmeasured confounding have been provided previously.(12,13) We will say that the measurement error for the outcome Y is non-differential with respect to exposure A if the distribution of Y* conditional on Y and C is independent of A which we can denote in terms of…”
BackgroundSelf-citationSeib, C. D., Meng, T., Suh, I., Harris, A. H. S., Covinsky, K. E., Shoback, D., Trickey, A. W., Kebebew, E., & Tamura, M. K. (2021) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2021.6437 · Cited by 86
Nagendrababu, V., Duncan, H. F., FOUAD, A. F., Kirkevang, L., Parashos, P., Pigg, M., Væth, M., Jayaraman, J., Suresh, N., Arias, A., et al. (2022) · International Endodontic Journal · doi:10.1111/iej.13873 · Cited by 90
Elsaid, M. I., Li, N., Firkins, S. A., Rustgi, V. K., Paskett, E. D., Acharya, C., Reddy, K. R., Chiang, C. W., & Mumtaz, K. (2024) · Alimentary Pharmacology & Therapeutics · doi:10.1111/apt.17925 · Cited by 77
“38 In our study, the observed adjusted associations of GLP-1RAs initiation and the risk of ALO was aHR: 0.64 (95% CI: 0.54–0.76).”
BackgroundInfluentialQin, M., & Chen, B. (2024) · Cardiovascular Diabetology · doi:10.1186/s12933-024-02359-z · Cited by 77
“…we repeated the analysis after excluding base-line cancer and severe cardiovascular disease patients; (2) We calculated the E-value to quantify the minimum strength of association that an unmeasured confounder would require with both AIP and mortality to explain away the observed associations [19].”
MethodDonohue, J. M., Kennedy, J. N., Seymour, C. W., Girard, T. D., Lo‐Ciganic, W., Kim, C. H., Marroquin, O. C., Moyo, P., Chang, C. H., & Angus, D. C. (2019) · Annals of Internal Medicine · doi:10.7326/m18-2864 · Cited by 79
“These E-values are substantially higher than the largest observed confounder for either the 90- or 365-day outcome (risk ratio, 2.20 [CI, 2.00 to 2.44]).”
InfluentialLee, M.-J., Lim, M., & Kim, J. (2019) · British Journal Of Nutrition · doi:10.1017/s000711451900165x · Cited by 84
“Although most of the included studies adjusted for most of the major confounding factors, there is a possibility of residual or unmeasured confounding factors on the association between fruit intake and the MetS, based on a rather small E-value(40).”
BackgroundIskander, C., Cherney, D. Z., Clemens, K. K., Dixon, S. N., Harel, Z., Jeyakumar, N., McArthur, E., Muanda, F. T., Parikh, C. R., Paterson, J. M., et al. (2020) · Canadian Medical Association Journal · doi:10.1503/cmaj.191283 · Cited by 76
Kandola, A. A., Osborn, D., Stubbs, B., Choi, K. W., & Hayes, J. (2020) · BMC Medicine · doi:10.1186/s12916-020-01782-9 · Cited by 90
A US Population Health Survey on the Impact of COVID-19 Using the EQ-5D-5L
Hay, J. W., Gong, C. L., Jiao, X., Zawadzki, N. K., Zawadzki, R. S., Pickard, A. S., Xie, F., Crawford, S. A., & Gu, N. Y. (2021) · Journal of General Internal Medicine · doi:10.1007/s11606-021-06674-z · Cited by 87
Lim, S., Cherian, T., Katyal, M., Goldfeld, K. S., McDonald, R. D., Wiewel, E. W., Khan, M. R., Krawczyk, N., Braunstein, S., Murphy, S. M., et al. (2022) · Addiction · doi:10.1111/add.16071 · Cited by 84
Nobile, F., Forastiere, A., Michelozzi, P., Forastiere, F., & Stafoggia, M. (2023) · Environment International · doi:10.1016/j.envint.2023.108302 · Cited by 85
Epidural analgesia during labour and severe maternal morbidity: population based study
Kearns, R. J., Kyzayeva, A., Halliday, L., Lawlor, D. A., Shaw, M., & Nelson, S. M. (2024) · BMJ · doi:10.1136/bmj-2023-077190 · Cited by 101
“43 The E-value was defined as the minimum strength of association that one confounder or several unmeasured confounders would need to have with both epidural analgesia and SMM, conditional on the confounders we adjusted for, to fully explain a specific exposure-outcome association.”
BackgroundChen, Y., Harris, S. K., Worthington, E. L., & VanderWeele, T. J. (2018) · The Journal of Positive Psychology · doi:10.1080/17439760.2018.1519591 · Cited by 89
“Fourth, both forgiveness and health a See Vanderweele & Ding, 2017 for the formula for calculating E-values. b The E-values for effect estimates are the minimum strength of association on the risk ratio scale that an unmeasured confounder would need to have with both the exposure and the outcome, above and beyond the…”
BackgroundMethodInfluentialSelf-citationKennedy, M. C., Hayashi, K., Milloy, M.-J., Wood, E., & Kerr, T. (2019) · PLOS Medicine · doi:10.1371/journal.pmed.1002964 · Cited by 75
“Although we sought to reduce the potential for this bias by restricting our study sample based on SIF utilization patterns and by adjusting multivariable analyses for key confounding factors, an e-value analysis [82] indicated that an unmeasured confounder associated with frequent SIF use and mortality by a HR equival…”
MethodMongkhon, P., Fanning, L., Lau, W. C. Y., Tse, G., Lau, K. K., Wei, L., Kongkaew, C., & Wong, I. C. K. (2020) · Heart Rhythm · doi:10.1016/j.hrthm.2020.01.007 · Cited by 81
Molina‐Montes, E., Coscia, C., Gómez-Rubio, P., Fernández, A. M., Boenink, R., Rava, M., Márquez, M., Molero, X., Löhr, M. J., Sharp, L., et al. (2020) · Gut · doi:10.1136/gutjnl-2019-319990 · Cited by 100
Metadata
- Authors
- VanderWeele, T. J., & Ding, P.
- Affiliations
- Harvard University (US)
- Venue
- Annals of Internal Medicine
- Publisher
- American College of Physicians
- Published
- 2017-08-15
- Volume
- 167
- Issue
- 4
- Pages
- 268-274
- ISSN
- 0003-4819 (ISSN-L)
- 1539-3704
- OpenAlex
- W2735301415
- Cited by
- 6,350
- References
- 59
- Citations by year
- 2026: 1516 · 2025: 1020 · 2024: 860 · 2023: 736 · 2022: 716 · 2021: 583 · 2020: 498 · 2019: 290 · 2018: 113 · 2017: 15
- FWCI
- 174.58
- Type
- journal article
- Language
- en
- Open access
- green
- nrs.harvard.edu
- Licence
- cc-by
- www.acpjournals.org/journal/aim/text-and-data-mining (tdm)
- Locations
- Annals of Internal Medicine (publishedVersion)
- PubMed (publishedVersion)
- Digital Access to Scholarship at Harvard (DASH) (Harvard University) (submittedVersion, cc-by)
- Funders
- National Institute of Environmental Health Sciences (ES017876)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- E-value, unmeasured confounding, sensitivity analysis, causal inference, observational studies, relative risk
- MeSH terms
- Humans, Sensitivity and Specificity, Epidemiologic Research Design, Confounding Factors, Epidemiologic, Observational Studies as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.

