Sensitivity Analysis in Observational Research: Introducing the E-Value
Open accessThe DOI matches its Crossref record; 58 of the 59 references it cites were found in a registry or an index.
Abstract
Sensitivity analysis is useful in assessing how robust an association is to potential unmeasured or uncontrolled confounding. This article introduces a new measure called the "E-value," which is related to the evidence for causality in observational studies that are potentially subject to confounding. The E-value is defined as the minimum strength of association, on the risk ratio scale, that an unmeasured confounder would need to have with both the treatment and the outcome to fully explain away a specific treatment-outcome association, conditional on the measured covariates. A large E-value implies that considerable unmeasured confounding would be needed to explain away an effect estimate. A small E-value implies little unmeasured confounding would be needed to explain away an effect estimate. The authors propose that in all observational studies intended to produce evidence for causality, the E-value be reported or some other sensitivity analysis be used. They suggest calculating the E-value for both the observed association estimate (after adjustments for measured confounders) and the limit of the confidence interval closest to the null. If this were to become standard practice, the ability of the scientific community to assess evidence from observational studies would improve considerably, and ultimately, science would be strengthened.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Quantitative bias analyses, such as the calculation of E-values to estimate the strength of unmeasured confounding required to explain away an observed association, provide useful context for the robustness of findings.64”
From Artificial Intelligence in Oncology: Practical Applications Across Clinical Care, Scholarship, and Translation. (2026) · cites this paper as [64]
The cited source says what the claim attributes to it.
In the citing paper
“We performed the sensitivity analysis using the E value [29] to evaluate the robustness of our findings to potential unmeasured confounding, with the calculation based on the observed odds ratio (OR) point estimate for rare outcome (<15%) or the square root of OR otherwise.”
The cited source says what the claim attributes to it.
In the citing paper
“To further assess the robustness of our findings to potential unmeasured confounding, we conducted an E-value analysis, a quantitative method for sensitivity analysis in observational research [31].”
From Acute Respiratory Tract Infection and Sudden Sensorineural Hearing Loss: A Multinational Cohort Study (2025) · cites this paper as [31]
In the citing paper
“We also computed E-values: indicators of how robust findings are to potential unmeasured confounding”
From Leading a meaningful life at older ages and its relationship with social engagement, prosperity, health, biology, and time use (2019) · cites this paper as [12]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009) · doi:10.1002/9780470743386 · Cited by 14,225
The environment and disease: association or causation?
Hill, A. B. (2015) · Journal of the Royal Society of Medicine · doi:10.1177/0141076814562718 · Cited by 7,285
The Environment and Disease: Association or Causation?
Hill, A. B. (1965) · Proceedings of the Royal Society of Medicine · doi:10.1177/003591576505800503 · Cited by 6,974 · cited 1×
“Sensitivity analysis makes clear quantitatively why Bradford Hill's criterion of “strength of association” (12) is important in establishing that a given association is, in fact, causal.”
BackgroundThe ASA Statement on p -Values: Context, Process, and Purpose
Wasserstein, R. L., & Lazar, N. A. (2016) · The American Statistician · doi:10.1080/00031305.2016.1154108 · Cited by 6,306 · cited 2×
“While potentially subject to mis-use and misinterpretation (1,5-12), the p-value can be informative as a continuous measure of evidence as to whether an association is present.”
BackgroundCausal Inference for Statistics, Social, and Biomedical Sciences: An Introduction
Imbens, G. W., & Rubin, D. B. (2015) · doi:10.1017/cbo9781139025751 · Cited by 4,210
Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations
Greenland, S., Senn, S. J., Rothman, K. J., Carlin, J. B., Poole, C., Goodman, S. N., & Altman, D. G. (2016) · European Journal of Epidemiology · doi:10.1007/s10654-016-0149-3 · Cited by 3,200
Explanation in Causal Inference: Methods for Mediation and Interaction
VanderWeele, T. J. (2015) · Cited by 2,217
Self-citationSmithson, M. (2000) · SAGE Publications Ltd eBooks · doi:10.4135/9781446218525 · Cited by 1,871
Fundamentals of Biostatistics.
Cornell, R. G., & Rosner, B. (1987) · Journal of the American Statistical Association · doi:10.2307/2289416 · Cited by 1,603
Breastfeeding and Maternal and Infant Health Outcomes In Developed Countries
(2007) · AAP Grand Rounds · doi:10.1542/gr.18-2-15 · Cited by 1,540 · cited 2×
“A study by the Agency for Healthcare Research and Quality (42) reported the association between breastfeeding and childhood leukemia as RR=0.”
MethodInfluential
Research works printed but not linked: 3
- Greenland S, 2011, Jurimetrics, 51, 11329
- Cornfield J, 1959, J Natl Cancer Inst, 22, 173
- Schlesselman JJ, 1978, Am J Epidemiol, 108, 3
Cited by
Items 801–900 of 1,031
Juraschek, S. P., Kovell, L. C., Appel, L. J., Miller, E. R., Sacks, F. M., Christenson, R. H., Rebuck, H., Chang, A. R., & Mukamal, K. J. (2020) · Annals of Internal Medicine · doi:10.7326/m20-0336 · Cited by 40
“Standardized mean difference E-values (using unclustered SDs) were calculated for each of the between-diet contrasts to estimate the minimum strength of association an unmeasured confounder would need to explain away the dietary associations with each cardiac biomarker (19).”
MethodSchnell, P., & Papadogeorgou, G. (2020) · The Annals of Applied Statistics · doi:10.1214/20-aoas1377 · Cited by 56
Raffetti, E., Donato, F., Palma, G. D., Leonardi, L., Sileo, C. V., & Magoni, M. (2020) · Chemosphere · doi:10.1016/j.chemosphere.2020.127522 · Cited by 38
Development and validation of a predictive model of drug-resistant genetic generalized epilepsy
Choi, H., Detyniecki, K., Bazil, C. W., Thornton, S., Crosta, P., Tolba, H., Muneeb, M., Hirsch, L. J., Heinzen, E. L., Sen, A., et al. (2020) · Neurology · doi:10.1212/wnl.0000000000010597 · Cited by 40
Ashaba, S., Cooper‐Vince, C. E., Maling, S., Satinsky, E. N., Baguma, C., Akena, D. H., Nansera, D., Bajunirwe, F., & Tsai, A. C. (2021) · Journal of Affective Disorders Reports · doi:10.1016/j.jadr.2021.100094 · Cited by 46
Evaluating the Long-term Efficacy of Coronavirus Disease 2019 (COVID-19) Vaccines
Lin, D.-Y., Zeng, D., & Gilbert, P. B. (2021) · Clinical Infectious Diseases · doi:10.1093/cid/ciab226 · Cited by 34
Koenecke, A., Powell, M. A., Xiong, R., Shen, Z., Fischer, N. M., Huq, S., Khalafallah, A. M., Trevisan, M., Sparén, P., Carrero, J. J., et al. (2021) · eLife · doi:10.7554/elife.61700 · Cited by 47
“…represents ‘the minimum strength of association on the risk ratio scale that an unmeasured confounder would need to have with both the treatment and the outcome to fully explain away a specific treatment- outcome association, conditional on the measured covariates’ (VanderWeele and Ding, 2017).”
BackgroundInfluentialSousa, B. L. A., Brentani, A., Ribeiro, C. C. C., Dolhnikoff, M., Grisi, S. J. F. E., Ferrer, A. P. S., & Ferraro, A. A. (2021) · BMJ Open · doi:10.1136/bmjopen-2021-050724 · Cited by 34
Isaksen, K. T., Mastroianni, M. A., Rinde, M., Rusten, L. S., Barzenje, D. A., Ramslien, L. F., Slaaen, M., Jerm, M. B., Smeland, E. B., Rostoft, S., et al. (2021) · Blood Advances · doi:10.1182/bloodadvances.2021004777 · Cited by 39
“Sensitivity analyses in the form of E-values were calculated to assess how robust the observed association between treatment intensity and survival was to potential unmeasured confounding.(42) Baseline characteristics were compared using the x(2) or Fisher’s exact test, when indicated.”
Recommendations for the use of propensity score methods in multiple sclerosis research
Simoneau, G., Pellegrini, F., Debray, T. P. A., Rouette, J., Muñoz, J., Platt, R. W., Petkau, J., Bohn, J. M., Shen, C., de Moor, C., et al. (2022) · Multiple Sclerosis Journal · doi:10.1177/13524585221085733 · Cited by 35
Serper, M., Kaplan, D. E., Taddei, T. H., Tapper, E. B., Cohen, J. B., & Mahmud, N. (2022) · Hepatology · doi:10.1002/hep.32737 · Cited by 39
“The minimum E‐value is 1; high E‐values indicate that substantial unmeasured confounding would be needed to render the observed associations null.[30,31]”
BackgroundRisk of Inflammatory Bowel Disease in Patients With Atopic Dermatitis
Fuxench, Z. C. C., Wan, J., Wang, S., Syed, M. N., Shin, D. B., Abuabara, K., & Gelfand, J. M. (2023) · JAMA Dermatology · doi:10.1001/jamadermatol.2023.2875 · Cited by 39
“Age, median (IQR), y 4 (2-9) 4 (1-8) 9 (4-14) 5 (1-10) 47 (32-64) 45 (30-63) 50 (34-68) 47 (32-63)”
Causes and consequences of child growth faltering in low- and middle-income countriesPreprint
Mertens, A., Benjamin‐Chung, J., Colford, J. M., Coyle, J., van der Laan, M. J., Hubbard, A., Rosete, S., Malenica, I., Hejazi, N. S., Sofrygin, O., et al. (2020) · medRxiv · doi:10.1101/2020.06.09.20127100 · Cited by 34
Mosholder, A. D., Ma, Y., Akhtar, S., Podskalny, G. D., Feng, Y., Lyu, H., Liao, J., Wei, Y., Wernecke, M., Leishear, K., et al. (2022) · American Journal of Psychiatry · doi:10.1176/appi.ajp.21090876 · Cited by 33
Long, K. N. G., Chen, Y., Potts, M., Hanson, J., & VanderWeele, T. J. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.01337 · Cited by 33
“To further investigate understudied forms of forgiveness, this study used an outcome-wide analytic approach (VanderWeele, 2017; VanderWeele et al., 2020) to prospectively examine the association between religiously or spiritually motivated selfforgiveness and divine forgiveness and a wide array of…”
BackgroundMethodSelf-citationWeller, J. H., Peter, S. D. S., Fallat, M. E., Saito, J. M., Burns, R. C., Deans, K. J., Fraser, J. D., Gadepalli, S. K., Helmrath, M. A., Hirschl, R. B., et al. (2021) · Journal of Pediatric Surgery · doi:10.1016/j.jpedsurg.2021.04.013 · Cited by 33
Federated causal inference in heterogeneous observational data
Xiong, R., Koenecke, A., Powell, M. A., Shen, Z., Vogelstein, J. T., & Athey, S. C. (2023) · Statistics in Medicine · doi:10.1002/sim.9868 · Cited by 62
“22To quantify the impact of unobserved confounders, E-value analysis was conducted in Koenecke et al. (2021) which showed that a relatively strong unobserved confounder(s) would be necessary to nullify the results; see VanderWeele and Ding (2017) for the definition of E-values.”
BackgroundAre E-values too optimistic or too pessimistic? Both and neither!
Sjölander, A., & Greenland, S. (2022) · International Journal of Epidemiology · doi:10.1093/ije/dyac018 · Cited by 41
Skogen, J. C., Andersen, A. I. O., Finserås, T. R., Ranganath, P., Brunborg, G. S., & Hjetland, G. J. (2023) · Frontiers in Public Health · doi:10.3389/fpubh.2023.1192788 · Cited by 52
“In fact, a meta-analysis showed that cyberbullying is more strongly related to suicidal ideation than traditional bullying (16).”
BackgroundPiaserico, S., Riedl, E., Pavlovsky, L., Vender, R. B., Mert, C., Tangsirisap, N., Haustrup, N., Gallo, G., Schuster, C., & Brunner, P. M. (2023) · Frontiers in Medicine · doi:10.3389/fmed.2023.1185523 · Cited by 33
Risk of Alzheimer's Disease and Related Dementias in Persons with Glaucoma
Crump, C., Sundquist, J., Sieh, W., & Sundquist, K. (2023) · Ophthalmology · doi:10.1016/j.ophtha.2023.10.014 · Cited by 41
Associations between types and sources of dietary carbohydrates and liver fat: a UK Biobank study
Orliacq, J., Perez‐Cornago, A., Parry, S. A., Kelly, R. K., Koutoukidis, D. A., & Carter, J. L. (2023) · BMC Medicine · doi:10.1186/s12916-023-03135-8 · Cited by 39
“…the analysis, including energy intake, although the calculation of an E-value indicated that unmeasured confounders with associations of 2.31 with both the exposure and outcome could explain away the inverse relationship of fibre with the odds of liver fat in the cross-sectional analyses [64, 65].”
BackgroundInfluentialChen, N., Ma, L., Zhang, Y., & Yan, Y. (2023) · Environmental Pollution · doi:10.1016/j.envpol.2023.123091 · Cited by 39
Rahman, A. A., Dell’Aniello, S., Moodie, E. E. M., Durand, M., Coulombe, J., Boivin, J., Suissa, S., Ernst, P., & Renoux, C. (2024) · Annals of Internal Medicine · doi:10.7326/m23-0849 · Cited by 38
“Sixth, we computed an E-value to assess the robustness of fi ndings to potential residual confounding (42).”
MethodKaracabeyli, D., Lacaille, D. V., Lu, N., McCormick, N., Xie, H. W., Choi, H. K., & Aviña-Zubieta, J. A. (2024) · PLoS ONE · doi:10.1371/journal.pone.0308533 · Cited by 39
Deprescribing of Antihypertensive Medications and Cognitive Function in Nursing Home Residents
Jing, B., Liu, X., Graham, L. A., Dave, C. V., Li, Y., Fung, K. Z., Liu, C., Magid, H. S. A., Growdon, M. E., Deardorff, W. J., et al. (2024) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2024.4851 · Cited by 35
Association of IgE-Mediated Allergy With Risk of Complicated Appendicitis in a Pediatric Population
Salö, M., Gudjonsdottir, J., Omling, E., Hagander, L. E., & Stenström, P. (2018) · Archives of Pediatrics and Adolescent Medicine · doi:10.1001/jamapediatrics.2018.1634 · Cited by 33
Shokoohi, M., Bauer, G. R., Kaida, A., Logie, C. H., Lacombe‐Duncan, A., Milloy, M., Lloyd‐Smith, E., Carter, A., & Loutfy, M. R. (2019) · Addiction · doi:10.1111/add.14566 · Cited by 38
“Sensitivity analysis We reported E-value to evaluate the extent to which residual (unmeasured) confounding might explain away the observed associations, and computed as: E = RR + sqrt{RR × RR – 1}, where RR = 1/RR for RR < 1 [51].”
MethodMouri, H., Jo, T., Matsui, H., Fushimi, K., & Yasunaga, H. (2019) · Anesthesia & Analgesia · doi:10.1213/ane.0000000000004239 · Cited by 34
Don’t abandon RCTs in IVF. We don’t even understand them
Wilkinson, J., Brison, D. R., Duffy, J. M., Farquhar, C. M., Lensen, S. F., Mastenbroek, S., van Wely, M., & Vail, A. (2019) · Human Reproduction · doi:10.1093/humrep/dez199 · Cited by 35
“large, stable treatment effects, since the bias would have to be large to fully explain away the result (VanderWeele and Ding, 2017).”
BackgroundKim, S. Y., Min, C., Oh, D. J., & Choi, H. G. (2020) · Scientific Reports · doi:10.1038/s41598-020-58783-0 · Cited by 38
Iyer, H. S., Valeri, L., James, P., Chen, J. T., Hart, J. E., Laden, F., Holmes, M. D., & Rebbeck, T. R. (2020) · Environmental Epidemiology · doi:10.1097/ee9.0000000000000087 · Cited by 38
“63) Age at diagnosis, years 66 (59, 73) 67 (60, 74) 67 (60, 74) 66 (60, 73) 66 (60, 73) 66 (60, 73) Black reported race [n (%)] 8,599 (33) 2,512 (10) 1,371 (5) 1,028 (4) 654 (3) 14,164 (11) Married [n (%)] Yes 9,868 (38) 11,887 (46) 13,142 (51) 13,956 (54) 14,515 (56) 63,368 (49) No 5,910 (23) 3,752 (15) 3,342 (13) 3,…”
Nakamura, A., Pryor, L., Ballon, M., Lioret, S., Heude, B., Charles, M.-A., Melchior, M., & El-Khoury Lesueur, F. (2020) · European Journal of Public Health · doi:10.1093/eurpub/ckaa076 · Cited by 29
Schubauer‐Berigan, M. K., de González, A. B., Cardis, E., Laurier, D., Lubin, J. H., Hauptmann, M., & Richardson, D. B. (2020) · JNCI Monographs · doi:10.1093/jncimonographs/lgaa008 · Cited by 38
“Several approaches have been proposed for calculating externally corrected relative risks (RRs) or the maximum potential confounding bias (1,10,11).”
BackgroundSilverberg, M. J., Leyden, W. A., Hernández‐Ramírez, R. U., Li, Q., Lin, H., Justice, A. C., Hessol, N. A., Achenbach, C. J., DʼSouza, G., Engels, E. A., et al. (2020) · Clinical Infectious Diseases · doi:10.1093/cid/ciaa1046 · Cited by 37
Zhao, S. S., Jones, G. T., Hughes, D. M., Moots, R. J., & Goodson, N. J. (2021) · Rheumatology · doi:10.1093/rheumatology/keab242 · Cited by 41
“These point estimates could be explained away by an unmeasured confounder that was associated with both the exposure and outcome by over 2-fold each, above and beyond measured confounders; weaker confounding could not do so [22].”
BackgroundMemel, Z. N., Lee, J. J., Foulkes, A. S., Chung, R. T., Thaweethai, T., & Bloom, P. P. (2021) · The Journal of Infectious Diseases · doi:10.1093/infdis/jiab539 · Cited by 36
Niu, H., & Zhou, Y. (2021) · International Journal of General Medicine · doi:10.2147/ijgm.s341790 · Cited by 42
“Thus, E-values were calculated, which were defined as the minimum strength of association that an unmeasured factor would need to have with both the exposure and the outcome to completely explain the observed associations between any unmeasured confounder: AST/ALT ratio (exposure) and T2DM (outcome).(13) E-values rang…”
BackgroundAssociation of Incarceration With Mortality by Race From a National Longitudinal Cohort Study
Bovell-Ammon, B. J., Xuan, Z., Paasche‐Orlow, M. K., & Larochelle, M. R. (2021) · JAMA Network Open · doi:10.1001/jamanetworkopen.2021.33083 · Cited by 37
Kerola, A. M., Palomäki, A., Rautava, P. T., & Kytö, V. (2022) · European Journal of Preventive Cardiology · doi:10.1093/eurjpc/zwac049 · Cited by 34
Suffering, psychological distress, and well‐being in Indonesia: A prospective cohort study
Ho, S., Cook, K. V., Chen, Z. J., Kurniati, N. M. T., Suwartono, C., Widyarini, N., Wong, P. T. P., & Cowden, R. G. (2022) · Stress and Health · doi:10.1002/smi.3139 · Cited by 44
“…EE, E‐value for the effect estimate; LCI, E‐value for the limit of the confidence interval. aThe formula for calculating E‐values can be found in VanderWeele and Ding (2017). bE‐values for effect estimates are the minimum strength of association that an unmeasured confounder would need to…”
BackgroundMethodInfluentialTian, F., Shen, Q., Hu, Y., Ye, W., Valdimarsdóttir, U. A., Song, H., & Fang, F. (2022) · The Lancet Regional Health - Europe · doi:10.1016/j.lanepe.2022.100402 · Cited by 42
Risk of cardiovascular disease in women and men with subfertility: the Trøndelag Health Study
Skåra, K. H., Åsvold, B. O., Hernáez, Á., Fraser, A., Rich-Edwards, J. W., Farland, L. V., Næss, Ø., Lawlor, D. A., Brumpton, B. M., & Magnus, M. C. (2022) · Fertility and Sterility · doi:10.1016/j.fertnstert.2022.05.038 · Cited by 30
Ngamvichchukorn, T., Ruengorn, C., Noppakun, K., Thavorn, K., Hutton, B., Sood, M. M., Knoll, G. A., & Nochaiwong, S. (2022) · JAMA Network Open · doi:10.1001/jamanetworkopen.2022.37580 · Cited by 40
Sensitivity analysis of individual treatment effects: A robust conformal inference approach
Jin, Y., Ren, Z., & Candès, E. J. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2214889120 · Cited by 123
“Imbens (2003), Ding and VanderWeele (2016) and VanderWeele and Ding (2017) also considered unmeasured confounders with ‘limited’ effect, but under different sensitivity models.”
BackgroundAssociation of sulfonylureas with the risk of dementia: A population‐based cohort study
Wu, C., Iskander, C., Wang, C., Xiong, L. Y., Shah, B. R., Edwards, J. D., Kapral, M. K., Herrmann, N., Lanctôt, K. L., Masellis, M., et al. (2023) · Journal of the American Geriatrics Society · doi:10.1111/jgs.18397 · Cited by 38
“E-value as a quantitative bias analysis was used to quantify the robustness of our estimate to an unmeasured confounder.(22) As an exploratory analysis to validate the within-class differences, new glyburide and gliclazide users were selected from the primary cohort, and glyburide users were compared to gliclazide use…”
MethodEbrahimi, F., Simon, T. G., Hagström, H., Söderling, J. K., Wester, A., Roelstraete, B., & Ludvigsson, J. F. (2023) · Clinical Gastroenterology and Hepatology · doi:10.1016/j.cgh.2023.05.013 · Cited by 40
Glotzbach, J. P., Hanson, H. A., Tonna, J. E., Horns, J. J., Allen, C. M., Presson, A. P., Griffin, C. L., Zak, M., Sharma, V., Tristani‐Firouzi, M., et al. (2023) · Circulation · doi:10.1161/circulationaha.122.060439 · Cited by 31
Kerr, D., Levy, N. S., Bae, H. T., Boustead, A. E., & Martins, S. S. (2023) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2023.06.009 · Cited by 36
McCracken, C., Raisi‐Estabragh, Z., Szabó, L. E., Robson, J. P., Raman, B., Topiwala, A., Roca‐Fernández, A., Husain, M., Petersen, S. E., Neubauer, S., et al. (2024) · BMC Medicine · doi:10.1186/s12916-023-03187-w · Cited by 45
“Finally, we applied the e -value methodology described by VanderWeele and colleagues [31] to evaluate the potential nullifying impact of unobserved confounding variables.”
MethodLei, L., Li, J., Wang, W., Yu, Y., Pu, B., Peng, Y., Zhang, L., & Zhao, Z. (2024) · Obesity · doi:10.1002/oby.23986 · Cited by 38
“…statistically significant results in the main analysis, we calculated the E -value to estimate the minimum strength of association, on the risk ratio scale, that an unmeasured confounder would need to explain away an exposure – outcome association when accounting for the measured covariates [23].”
BackgroundBone shape mediates the relationship between sex and incident knee osteoarthritis
Wise, B. L., Niu, J., Zhang, Y., Liu, F., Pang, J. H., Lynch, J. A., & Lane, N. E. (2018) · BMC Musculoskeletal Disorders · doi:10.1186/s12891-018-2251-z · Cited by 31
“E-value is a recently developed measure of the minimum strength of association, on the risk ratio scale, that an unmeasured confounder would need to have with both the treatment and the outcome to fully explain away a specific treatment–outcome association, conditional on the measured covariates [40].”
BackgroundBaravelli, C. M., Sandberg, S., Aarsand, A. K., & Tollånes, M. C. (2019) · Orphanet Journal of Rare Diseases · doi:10.1186/s13023-019-1051-3 · Cited by 35
Reasons why osteoarthritis predicts mortality: path analysis within a Cox proportional hazards model
Wilkie, R., Parmar, S., Bucknall, M., Smith, D., Thomas, M. J., Seale, B., Mansell, G., & Peat, G. (2019) · RMD Open · doi:10.1136/rmdopen-2019-001048 · Cited by 31
“Sensitivity analysis was performed to assess the robustness of findings to potential unmeasured confounding, using the E value approach introduced by VanderWeele and Ding 36 ; E value represents the minimal needed association between a confounder and both the outcome and exposure to fully explain observed associations.”
MethodInfluentialGuha, A., Dey, A. K., Arora, S., Cavender, M. A., Vavalle, J. P., Sabik, J. F., Jiménez, E., Jneid, H. M., & Addison, D. (2020) · Journal of the American Heart Association · doi:10.1161/jaha.119.014248 · Cited by 32
“The wage index helps correct for geographic variations in costs among hospitals.(21) Charges and costs were inflation adjusted to 2015.”
BackgroundPark, J., Kim, J., Lee, S. H., Lee, J. H., Min, J. J., Kwon, J., Oh, A. R., Seo, W., Hyeon, C. W., Yang, K., et al. (2020) · Scientific Reports · doi:10.1038/s41598-020-68511-3 · Cited by 37
Sheffield, K. M., Dreyer, N. A., Murray, J. F., Faries, D. E., & Klopchin, M. N. (2020) · Journal of Comparative Effectiveness Research · doi:10.2217/cer-2020-0161 · Cited by 34
Fernando, S. M., Fan, E., Rochwerg, B. N., Burns, K. E. A., Brochard, L. J., Cook, D. J., Walkey, A. J., Ferguson, N. D., Hough, C. L., Brodie, D., et al. (2020) · CHEST Journal · doi:10.1016/j.chest.2020.09.100 · Cited by 32
Maldonado, L. Y., Bone, J. N., Scanlon, M., Anusu, G., Chelagat, S., Jumah, A., Ikemeri, J. E., Songok, J. J., Christoffersen‐Deb, A., & Ruhl, L. J. (2020) · BMJ Global Health · doi:10.1136/bmjgh-2020-003370 · Cited by 47
“We plan to additionally distribute printed summaries of key findings following the trial’s publication.”
Nwaru, B. I., Shah, S. A., Tibble, H., Pillinger, R., McLean, S., Ryan, D. P., Critchley, H., Hawrylowicz, C. M., Simpson, C. R., Soyiri, I. N., et al. (2021) · The Journal of Allergy and Clinical Immunology In Practice · doi:10.1016/j.jaip.2021.02.052 · Cited by 37
Prevention of delayed bleeding with vonoprazan in upper gastrointestinal endoscopic treatment
Abe, H., Hatta, W., Ogata, Y., Koike, T., Saito, M., Jin, X., Nakagawa, K., Kanno, T., Uno, K., Asano, N., et al. (2021) · Journal of Gastroenterology · doi:10.1007/s00535-021-01781-4 · Cited by 33
“Second, we computed an E value to evaluate the extent of unmeasured confounders that would be needed to negate the observed significant association [37, 38].”
MethodDietary Factors and Prevention: Risk of End-Stage Kidney Disease by Fruit and Vegetable Consumption
Banerjee, T., Carrero, J. J., McCulloch, C. E., Burrows, N. R., Siegel, K. R., Morgenstern, H., Saran, R., & Powe, N. R. (2021) · American Journal of Nephrology · doi:10.1159/000514754 · Cited by 35
“We attribute this loss of statistical significance to potentially other unmeasured aspects of health status in these participants with advanced CKD (E-value [50] for the risk estimate = 1.”
BackgroundReece, A. S., & Hulse, G. K. (2021) · BMC Pharmacology and Toxicology · doi:10.1186/s40360-021-00505-x · Cited by 35
“e-Values were calculated using the e-Value algorithms of the EValue package [39].”
MethodRied‐Larsen, M., Rasmussen, M. G. B., Blond, K., Overvad, T. F., Overvad, K., Steindorf, K., Katzke, V. A., Andersen, J. L. M., Petersen, K. E. N., Aune, D., et al. (2021) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2021.3836 · Cited by 36
Shiba, K., Hikichi, H., Okuzono, S. S., VanderWeele, T. J., Arcaya, M. C., Daoud, A., Cowden, R. G., Yazawa, A., Zhu, D. T., Aida, J., et al. (2022) · Environmental Health Perspectives · doi:10.1289/ehp10903 · Cited by 36
“02 Missing 291 (13) 253 (12) 27 (29) Senior clubs 1:38± 0:84 1:36± 0:82 1:81± 1:16 <0:001 Missing 315 (15) 283 (14) 20 (22) Frequency of meeting friends 3:79± 1:45 3:78± 1:44 4:13± 1:59 0.”
Self-citationVezzali, L., Lolliot, S., Trifiletti, E., Cocco, V. M., Rae, J. R., Capozza, D., & Hewstone, M. (2022) · British Journal of Social Psychology · doi:10.1111/bjso.12558 · Cited by 42
“VanderWeele et al. (2020) propose the ‘outcome- wide longitudinal design’ which offers new metrics (e.g., the E- value that measures how robust the parameter estimates are; VanderWeele & Ding, 2017) and approaches to estimating and controlling for unmeasured confounding variables.”
BackgroundSeib, C. D., Meng, T., Cisco, R. M., Suh, I., Lin, D. T., Harris, A. H. S., Trickey, A. W., Tamura, M. K., & Kebebew, E. (2022) · Annals of Surgery · doi:10.1097/sla.0000000000005691 · Cited by 34
Li, A., Qiu, B., Goettsch, M., Chen, Y., Ge, S., Xu, S., & Tjakkes, G. E. (2023) · Journal Of Clinical Periodontology · doi:10.1111/jcpe.13785 · Cited by 32
“In addition, we computed the E -value to assess whether an unmeasured confounder could completely explain the associations between plant-based diets and periodontitis and elevated antibodies (VanderWeele & Ding, 2017).”
MethodLack of Statin Therapy and Outcomes After Ischemic Stroke: A Population-Based Study
Åivo, J., Ruuskanen, J. O., Tornio, A., Rautava, P. T., & Kytö, V. (2023) · Stroke · doi:10.1161/strokeaha.122.040536 · Cited by 37
“22 Our study was primarily designed to be as treated analysis.”
MethodJeong, H. E., Park, S., Noh, Y., Bea, S., Filion, K. B., Yu, O. H. Y., Jang, S. H., Cho, Y. M., Yon, D. K., & Shin, J. (2023) · BMC Medicine · doi:10.1186/s12916-023-02765-2 · Cited by 35
Elevated body temperature is associated with depressive symptoms: results from the TemPredict Study
Mason, A. E., Kasl, P., Soltani, S., Green, A., Hartogensis, W., Dilchert, S., Chowdhary, A., Pandya, L. S., Siwik, C. J., Foster, S. L., et al. (2024) · Scientific Reports · doi:10.1038/s41598-024-51567-w · Cited by 31
“Although we were unable to assess many potential confounders, sensitivity analysis using E-values showed that unmeasured confounders would need to be larger than observed effects of several known relevant variables to explain away all observed effects in these models.”
InfluentialAntiemetic Administration and Its Association with Race: A Retrospective Cohort Study
White, R. S., Andreae, M. H., Lui, B., Ma, X., Tangel, V. E., Turnbull, Z. A., Jiang, S. Y., Nachamie, A. S., Pryor, K. O., & Multicenter Perioperative Outcomes Group Collaborators (2023) · Anesthesiology · doi:10.1097/aln.0000000000004549 · Cited by 31
“Post hoc, for our primary outcome model, we calculated an E value and a confidence limit to show the minimum strength of an unmeasured confounder’s association with our variable of interest (Black race) and outcome (perioperative antiemetic administration) that would suffice to undercut our inferences based on the obs…”
BackgroundMethodAdherence to 5 Diet Quality Indices and Pancreatic Cancer Risk in a Large US Prospective Cohort
Julián‐Serrano, S., Reedy, J., Robien, K., & Stolzenberg‐Solomon, R. Z. (2022) · American Journal of Epidemiology · doi:10.1093/aje/kwac082 · Cited by 32
Węziak‐Białowolska, D., & Białowolski, P. (2022) · Psychology and Health · doi:10.1080/08870446.2022.2105842 · Cited by 41
“This sensitivity measure assesses the extent to which a potential uncontrolled confounder would need to be associated with both the exposure and the outcome to cancel out the observed association (VanderWeele & Ding, 2017).”
BackgroundMind-stimulating leisure activities: Prospective associations with health, wellbeing, and longevity
Węziak‐Białowolska, D., Białowolski, P., & Sacco, P. L. (2023) · Frontiers in Public Health · doi:10.3389/fpubh.2023.1117822 · Cited by 40
“This sensitivity measure assesses the extent to which a potential uncontrolled confounder would need to be associated with both the exposure and outcome to explain the observed association (45).”
BackgroundStatin use and risk of colorectal cancer in patients with inflammatory bowel disease
Sun, J., Halfvarson, J., Bergman, D., Ebrahimi, F., Roelstraete, B., Lochhead, P. J., Song, M., Olén, O., & Ludvigsson, J. F. (2023) · EClinicalMedicine · doi:10.1016/j.eclinm.2023.102182 · Cited by 38
Assayag, J., Kim, C. H., Chu, H., & Webster, J. (2023) · Frontiers in Oncology · doi:10.3389/fonc.2023.1194718 · Cited by 40
Ng, H. S., Zhu, F., Zhao, Y., Yao, S., Lu, X., Ekuma, O., Evans, C., Fisk, J. D., Marrie, R. A., & Tremlett, H. (2024) · Neurology · doi:10.1212/wnl.0000000000208006 · Cited by 31
“Sensitivity analyses included (1) computing E-values to assess how likely the fi ndings were due to unobserved confounding, 31 (2) computing summary to assess the susceptibility in italics), which was performed in British Columbia only where the total source cohort was 19,360. of unadjusted and…”
MethodSustainable diets and risk of overweight and obesity: A systematic review and meta‐analysis
Reger, C., Leitzmann, M. F., Rohrmann, S., Kuhn, T. E., Sedlmeier, A. M., & Jochem, C. (2024) · Obesity Reviews · doi:10.1111/obr.13707 · Cited by 37
“31 One previous meta-analysis examined the association between organic food consumption and obesity and showed a modest 11% reduction in the risk of obesity.”
BackgroundIndividualized prevention of proton pump inhibitor related adverse events by risk stratification
Xia, B., He, Q., Smith, F. G., Gkoutos, G. V., Nirantharakumar, K., Kuo, Z. C., Wang, D., Feng, Q., Cheung, E. C., Dai, L., et al. (2024) · Nature Communications · doi:10.1038/s41467-024-48007-8 · Cited by 36
Wu, C., Liu, J., Li, Y., Qin, L., Gu, R., Feng, J., Xu, L., Meng, X., Chen, J., Chen, R., et al. (2024) · EBioMedicine · doi:10.1016/j.ebiom.2024.105376 · Cited by 32
“45 Seventh, we reran the models excluding smoking, alcohol use, and family history to assess the possible in-fl uences of their data missingness.”
MethodAnalytic methodology for childhood predictor analyses for wave 1 of the Global Flourishing Study
Padgett, R. N., Bradshaw, M., Chen, Y., Cowden, R. G., Jang, S. J., Kim, E. S., Shiba, K., Johnson, B. R., & VanderWeele, T. J. (2025) · BMC Global and Public Health · doi:10.1186/s44263-025-00142-0 · Cited by 62
“…outcome (sense of mastery) and the predictor, above and beyond all measured covariates, for an unmeasured confounder to explain away an association [36]; and a Group is very small (< 0.1% of the observed sample) within several countries potentially leading to complete separation and large…”
InfluentialSelf-citationBay, B., Boie, S., & Kesmodel, U. (2018) · BJOG An International Journal of Obstetrics & Gynaecology · doi:10.1111/1471-0528.15509 · Cited by 32
Developmental origins of endometriosis: a Swedish cohort study
Gao, M., Allebeck, P., Mishra, G. D., & Koupil, I. (2019) · Journal of Epidemiology & Community Health · doi:10.1136/jech-2018-211811 · Cited by 36
Gockley, A. A., Melamed, A., Cronin, A. M., Bookman, M. A., Burger, R. A., Cristae, M. C., Griggs, J. J., Mantia-Smaldone, G. M., Matulonis, U. A., Meyer, L. A., et al. (2019) · American Journal of Obstetrics and Gynecology · doi:10.1016/j.ajog.2019.06.009 · Cited by 33
Joseph, G. B., McCulloch, C. E., Nevitt, M. C., Neumann, J., Lynch, J. A., Lane, N. E., & Link, T. M. (2019) · Arthritis Care & Research · doi:10.1002/acr.24021 · Cited by 31
“confounding on the associations reported in this study using the E value statistic (47).”
MethodHeard, K. J., Ledbetter, C. M., & Hoppe, J. A. (2020) · Academic Emergency Medicine · doi:10.1111/acem.14071 · Cited by 34
Anticoagulation Type and Early Recurrence in Cardioembolic Stroke
Yaghi, S. R., Mistry, E. A., Liberman, A. L., Giles, J. A., Asad, S. D., Liu, A., Nagy, M., Kaushal, A., Azher, I., Grory, B. M., et al. (2020) · Stroke · doi:10.1161/strokeaha.120.028867 · Cited by 31
“Time to initiating anticoagulation (median, IQR) 2 (6) 5 (8) <0.”
Use of ACE (Angiotensin-Converting Enzyme) Inhibitors and Risk of Lung Cancer
Kristensen, K. B., Hicks, B. M., Azoulay, L., & Pottegård, A. (2021) · Circulation Cardiovascular Quality and Outcomes · doi:10.1161/circoutcomes.120.006687 · Cited by 31
Ruggiero, E., Castelnuovo, A. F. D., Costanzo, S., Persichillo, M., Curtis, A. D., Cerletti, C., Donati, M. B., de Gaetano, G., Iacoviello, L., & Bonaccio, M. (2020) · Journal of Nutrition · doi:10.1093/jn/nxaa365 · Cited by 38
Scott, F. I., Vajravelu, R. K., Mamtani, R., Bianchina, N., Mahmoud, N. N., Hou, J. K., Wu, Q., Wang, X., Haynes, K., & Lewis, J. D. (2021) · JAMA Network Open · doi:10.1001/jamanetworkopen.2020.36315 · Cited by 35
Reece, A. S., & Hulse, G. K. (2021) · BMC Cancer · doi:10.1186/s12885-021-07924-3 · Cited by 31
Association Between Buprenorphine for Opioid Use Disorder and Mortality Risk
Vakkalanka, J. P., Lund, B. C., Arndt, S. V., Field, W., Charlton, M. E., Ward, M. M., & Carnahan, R. M. (2021) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2021.02.026 · Cited by 34
Herzig, S. J., Anderson, T. S., Jung, Y., Ngo, L. H., Kim, D. H., & McCarthy, E. P. (2021) · PLoS Medicine · doi:10.1371/journal.pmed.1003804 · Cited by 34
“Finally, using the E-value [18,19], we estimated the strength of association between an unmeasured confounder and both opioid use and death that would be needed to explain away the observed risk of death from opioid use.”
MethodInfluentialHou, J., Kim, N. H., Cai, T. A., Dahal, K. P., Weiner, H. L., Chitnis, T., Cai, T., & Xia, Z. (2021) · JAMA Network Open · doi:10.1001/jamanetworkopen.2021.34627 · Cited by 33
Beer, R. J., Cnattingius, S., Susser, E., & Villamor, E. (2022) · Acta Paediatrica · doi:10.1111/apa.16375 · Cited by 32
NSAID use and clinical outcomes in COVID-19 patients: a 38-center retrospective cohort study
Reese, J., Coleman, B., Chan, L., Blau, H., Callahan, T. J., Cappelletti, L., Fontana, T., Bradwell, K. R., Harris, N. L., Casiraghi, E., et al. (2022) · Virology Journal · doi:10.1186/s12985-022-01813-2 · Cited by 41
“We treated the outcome of increased COVID-19 severity as a non-rare outcome (as it occurred more frequently than 15%), and death, invasive ventilation, AKI, and ECMO as rare outcomes (since the frequency of these outcomes was less than 15%) [36].”
Pereyra‐Elías, R., QUIGLEY, M. A., & Carson, C. (2022) · PLoS ONE · doi:10.1371/journal.pone.0267326 · Cited by 42
“This was assessed through the calculation of the E-values [56].”
MethodInfluentialTang, E. H. M., Mak, I. L., Tse, E. T. Y., Wan, E. Y. F., Yu, E. Y. T., Chen, J. Y., Chin, W. Y., Chao, D. V. K., Tsui, W. W. S., Ha, T. K. H., et al. (2022) · Diabetes Care · doi:10.2337/dc22-0387 · Cited by 34
Assisted Reproductive Technology and Risk of Childhood Cancers
Weng, S., Huang, Y., Huang, Y.-T., Li, Y.-P., & Chien, L. (2022) · JAMA Network Open · doi:10.1001/jamanetworkopen.2022.30157 · Cited by 37
Metadata
- Authors
- VanderWeele, T. J., & Ding, P.
- Affiliations
- Harvard University (US)
- Venue
- Annals of Internal Medicine
- Publisher
- American College of Physicians
- Published
- 2017-08-15
- Volume
- 167
- Issue
- 4
- Pages
- 268-274
- ISSN
- 0003-4819 (ISSN-L)
- 1539-3704
- OpenAlex
- W2735301415
- Cited by
- 6,350
- References
- 59
- Citations by year
- 2026: 1516 · 2025: 1020 · 2024: 860 · 2023: 736 · 2022: 716 · 2021: 583 · 2020: 498 · 2019: 290 · 2018: 113 · 2017: 15
- FWCI
- 174.58
- Type
- journal article
- Language
- en
- Open access
- green
- nrs.harvard.edu
- Licence
- cc-by
- www.acpjournals.org/journal/aim/text-and-data-mining (tdm)
- Locations
- Annals of Internal Medicine (publishedVersion)
- PubMed (publishedVersion)
- Digital Access to Scholarship at Harvard (DASH) (Harvard University) (submittedVersion, cc-by)
- Funders
- National Institute of Environmental Health Sciences (ES017876)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- E-value, unmeasured confounding, sensitivity analysis, causal inference, observational studies, relative risk
- MeSH terms
- Humans, Sensitivity and Specificity, Epidemiologic Research Design, Confounding Factors, Epidemiologic, Observational Studies as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.

