Sensitivity Analysis in Observational Research: Introducing the E-Value
Open accessThe DOI matches its Crossref record; 58 of the 59 references it cites were found in a registry or an index.
Abstract
Sensitivity analysis is useful in assessing how robust an association is to potential unmeasured or uncontrolled confounding. This article introduces a new measure called the "E-value," which is related to the evidence for causality in observational studies that are potentially subject to confounding. The E-value is defined as the minimum strength of association, on the risk ratio scale, that an unmeasured confounder would need to have with both the treatment and the outcome to fully explain away a specific treatment-outcome association, conditional on the measured covariates. A large E-value implies that considerable unmeasured confounding would be needed to explain away an effect estimate. A small E-value implies little unmeasured confounding would be needed to explain away an effect estimate. The authors propose that in all observational studies intended to produce evidence for causality, the E-value be reported or some other sensitivity analysis be used. They suggest calculating the E-value for both the observed association estimate (after adjustments for measured confounders) and the limit of the confidence interval closest to the null. If this were to become standard practice, the ability of the scientific community to assess evidence from observational studies would improve considerably, and ultimately, science would be strengthened.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Quantitative bias analyses, such as the calculation of E-values to estimate the strength of unmeasured confounding required to explain away an observed association, provide useful context for the robustness of findings.64”
From Artificial Intelligence in Oncology: Practical Applications Across Clinical Care, Scholarship, and Translation. (2026) · cites this paper as [64]
The cited source says what the claim attributes to it.
In the citing paper
“We performed the sensitivity analysis using the E value [29] to evaluate the robustness of our findings to potential unmeasured confounding, with the calculation based on the observed odds ratio (OR) point estimate for rare outcome (<15%) or the square root of OR otherwise.”
The cited source says what the claim attributes to it.
In the citing paper
“To further assess the robustness of our findings to potential unmeasured confounding, we conducted an E-value analysis, a quantitative method for sensitivity analysis in observational research [31].”
From Acute Respiratory Tract Infection and Sudden Sensorineural Hearing Loss: A Multinational Cohort Study (2025) · cites this paper as [31]
In the citing paper
“We also computed E-values: indicators of how robust findings are to potential unmeasured confounding”
From Leading a meaningful life at older ages and its relationship with social engagement, prosperity, health, biology, and time use (2019) · cites this paper as [12]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009) · doi:10.1002/9780470743386 · Cited by 14,225
The environment and disease: association or causation?
Hill, A. B. (2015) · Journal of the Royal Society of Medicine · doi:10.1177/0141076814562718 · Cited by 7,285
The Environment and Disease: Association or Causation?
Hill, A. B. (1965) · Proceedings of the Royal Society of Medicine · doi:10.1177/003591576505800503 · Cited by 6,974 · cited 1×
“Sensitivity analysis makes clear quantitatively why Bradford Hill's criterion of “strength of association” (12) is important in establishing that a given association is, in fact, causal.”
BackgroundThe ASA Statement on p -Values: Context, Process, and Purpose
Wasserstein, R. L., & Lazar, N. A. (2016) · The American Statistician · doi:10.1080/00031305.2016.1154108 · Cited by 6,306 · cited 2×
“While potentially subject to mis-use and misinterpretation (1,5-12), the p-value can be informative as a continuous measure of evidence as to whether an association is present.”
BackgroundCausal Inference for Statistics, Social, and Biomedical Sciences: An Introduction
Imbens, G. W., & Rubin, D. B. (2015) · doi:10.1017/cbo9781139025751 · Cited by 4,210
Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations
Greenland, S., Senn, S. J., Rothman, K. J., Carlin, J. B., Poole, C., Goodman, S. N., & Altman, D. G. (2016) · European Journal of Epidemiology · doi:10.1007/s10654-016-0149-3 · Cited by 3,200
Explanation in Causal Inference: Methods for Mediation and Interaction
VanderWeele, T. J. (2015) · Cited by 2,217
Self-citationSmithson, M. (2000) · SAGE Publications Ltd eBooks · doi:10.4135/9781446218525 · Cited by 1,871
Fundamentals of Biostatistics.
Cornell, R. G., & Rosner, B. (1987) · Journal of the American Statistical Association · doi:10.2307/2289416 · Cited by 1,603
Breastfeeding and Maternal and Infant Health Outcomes In Developed Countries
(2007) · AAP Grand Rounds · doi:10.1542/gr.18-2-15 · Cited by 1,540 · cited 2×
“A study by the Agency for Healthcare Research and Quality (42) reported the association between breastfeeding and childhood leukemia as RR=0.”
MethodInfluential
Research works printed but not linked: 3
- Greenland S, 2011, Jurimetrics, 51, 11329
- Cornfield J, 1959, J Natl Cancer Inst, 22, 173
- Schlesselman JJ, 1978, Am J Epidemiol, 108, 3
Cited by
Bjornevik, K., Cortese, M., Healy, B. C., Kuhle, J., Mina, M. J., Leng, Y., Elledge, S. J., Niebuhr, D. W., Scher, A. I., Munger, K. L., et al. (2022) · Science · doi:10.1126/science.abj8222 · Cited by 2,049
Wu, X., Nethery, R. C., Sabath, M. B., Braun, D., & Dominici, F. (2020) · Science Advances · doi:10.1126/sciadv.abd4049 · Cited by 1,974
Skrivankova, V. W., Richmond, R. C., Woolf, B. A. R., Davies, N. M., Swanson, S. A., VanderWeele, T. J., Timpson, N. J., Higgins, J. P. T., Dimou, N., Langenberg, C., et al. (2021) · BMJ · doi:10.1136/bmj.n2233 · Cited by 1,648
Self-citationZhu, L., She, Z.-G., Cheng, X., Qin, J.-J., Zhang, X.-J., Cai, J., Lei, F., Wang, H., Xie, J., Wang, W., et al. (2020) · Cell Metabolism · doi:10.1016/j.cmet.2020.04.021 · Cited by 1,602
Principles of confounder selection
VanderWeele, T. J. (2019) · European Journal of Epidemiology · doi:10.1007/s10654-019-00494-6 · Cited by 1,535
Self-citationJohnston, R., Jones, K., & Manley, D. (2018) · Quality & Quantity · doi:10.1007/s11135-017-0584-6 · Cited by 1,181
Rosenberg, E. S., Dufort, E. M., Udo, T., Wilberschied, L. A., Kumar, J., Tesoriero, J., Weinberg, P., Kirkwood, J., Muse, A., DeHovitz, J., et al. (2020) · JAMA · doi:10.1001/jama.2020.8630 · Cited by 1,156
Handbook of Religion and Health
Koenig, H. G., VanderWeele, T., & Peteet, J. R. (2024) · doi:10.1093/oso/9780190088859.001.0001 · Cited by 1,139
Self-citationImmune correlates analysis of the mRNA-1273 COVID-19 vaccine efficacy clinical trial
Gilbert, P. B., Montefiori, D. C., McDermott, A. B., Fong, Y., Benkeser, D., Deng, W., Zhou, H., Houchens, C. R., Martins, K., Jayashankar, L., et al. (2022) · Science · doi:10.1126/science.abm3425 · Cited by 1,137
Zhang, P., Zhu, L., Cai, J., Lei, F., Qin, J.-J., Xie, J., Liu, Y.-M., Zhao, Y.-C., Huang, X., Lin, L., et al. (2020) · Circulation Research · doi:10.1161/circresaha.120.317134 · Cited by 1,133
Making sense of sensitivity: extending omitted variable bias
Cinelli, C., & Hazlett, C. (2020) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/rssb.12348 · Cited by 1,088
“Regarding the robustness value in particular, we now discuss its relation to two other proposals advocated in the literature: the impact thresholds of Frank (2000) and the E-value of VanderWeele and Ding (2017).”
Larochelle, M. R., Bernson, D., Land, T., Stopka, T. J., Wang, N., Xuan, Z., Bagley, S. M., Liebschutz, J. M., & Walley, A. Y. (2018) · Annals of Internal Medicine · doi:10.7326/m17-3107 · Cited by 1,076
“We calculated the E-value to identify the minimum strength of association that an unmeasured confounder would need to have with both treatment and outcome, conditional on the measured covariates, to explain away the observed associations between MOUD and mortality (23).”
MethodUsing the E-Value to Assess the Potential Effect of Unmeasured Confounding in Observational Studies
Haneuse, S., VanderWeele, T. J., & Arterburn, D. (2019) · JAMA · doi:10.1001/jama.2018.21554 · Cited by 988
Self-citationThe Mediterranean Diet and Cardiovascular Health: A Critical Review
Martínez-González, M. A., Gea, A., & Ruiz-Canela, M. (2019) · Circulation Research · doi:10.1161/circresaha.118.313348 · Cited by 878
Orellano, P., Reynoso, J., Quaranta, N., Bardach, A., & Ciapponi, A. (2020) · Environment International · doi:10.1016/j.envint.2020.105876 · Cited by 792
Web Site and R Package for Computing E-values
Mathur, M. B., Ding, P., Riddell, C. A., & VanderWeele, T. J. (2018) · Epidemiology · doi:10.1097/ede.0000000000000864 · Cited by 761
Self-citationMental health outcomes in transgender and nonbinary youths receiving gender-affirming care
Tordoff, D. M., Wanta, J. W., Collin, A., Stepney, C., Inwards-Breland, D. J., & Ahrens, K. (2022) · JAMA Network Open · doi:10.1001/jamanetworkopen.2022.0978 · Cited by 692
The Future of Sensitivity Analysis: An essential discipline for systems modeling and policy support
Razavi, S., Jakeman, A., Saltelli, A., Prieur, C., Iooss, B., Borgonovo, E., Plischke, E., Lo Piano, S., Iwanaga, T., Becker, W., et al. (2021) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2020.104954 · Cited by 628
“…modeling (Koo et al., 2020a and 2020b), hydrology (Shin et al., 2013; Haghne-gahdar and Razavi, 2017), water security (Puy et al., 2020c), nuclear safety (Saltelli and Tarantola, 2002; Iooss and Marrel, 2019) and epidemiology (Burgess et al., 2017; VanderWeele and Ding, 2017), to name a few.”
Cerdá, M., Mauro, C., Hamilton, A., Levy, N. S., Santaella-Tenorio, J., Hasin, D., Wall, M. M., Keyes, K. M., & Martins, S. S. (2020) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2019.3254 · Cited by 598
Western Cape Department of Health in collaboration with the National Institute for Communicable Diseases, S. A., Boulle, A., Davies, M.-A., Hussey, H., Ismail, M., Morden, E., Vundle, Z., Zweigenthal, V., Mahomed, H., Paleker, M., et al. (2021) · Clinical Infectious Diseases · doi:10.1093/cid/ciaa1198 · Cited by 565
“association between HIV/tuberculosis and COVID-19 death [22].”
Basu, S., Berkowitz, S. A., Phillips, R. L., Bitton, A., Landon, B. E., & Phillips, R. S. (2019) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2018.7624 · Cited by 555
Rico-Campà, A., Martínez-González, M. A., Alvarez-Alvarez, I., Mendonça, R. D. D., de la Fuente-Arrillaga, C., Gómez-Donoso, C., & Bes-Rastrollo, M. (2019) · BMJ · doi:10.1136/bmj.l1949 · Cited by 534
Huangfu, P., & Atkinson, R. (2020) · Environment International · doi:10.1016/j.envint.2020.105998 · Cited by 522
“To this end, the application of the E-value approach is helpful (VanderWeele and Ding, 2017; Haneuse et al., 2019; Ioannidis et al., 2019; VanderWeele et al., 2019).”
BackgroundMethodInfluentialAminian, A., Zajichek, A., Arterburn, D. E., Wolski, K. E., Brethauer, S. A., Schauer, P. R., Kattan, M. W., & Nissen, S. E. (2019) · JAMA · doi:10.1001/jama.2019.14231 · Cited by 505
“2 UACR, median (IQR), mg/g 14 (6-40) 14 (5-40) −0.”
Zhang, X.-J., Qin, J.-J., Cheng, X., Shen, L., Zhao, Y.-C., Yuan, Y., Lei, F., Chen, M.-M., Yang, H., Bai, L., et al. (2020) · Cell Metabolism · doi:10.1016/j.cmet.2020.06.015 · Cited by 474
Affective polarization, local contexts and public opinion in America
Druckman, J. N., Klar, S., Krupnikov, Y., Levendusky, M., & Ryan, J. B. (2020) · Nature Human Behaviour · doi:10.1038/s41562-020-01012-5 · Cited by 463
Cross-sex Hormones and Acute Cardiovascular Events in Transgender Persons: A Cohort Study
Getahun, D., Nash, R., Flanders, W. D., Baird, T. C., Becerra-Culqui, T. A., Cromwell, L., Hunkeler, E., Lash, T. L., Millman, A., Quinn, V. P., et al. (2018) · Annals of Internal Medicine · doi:10.7326/m17-2785 · Cited by 431
“8) Not done (missing, age <40 y) 374 (13) 8696 (31) 8114 (29) 369 (17) 9888 (48) 9280 (45) Unknown (missing, age ≥40 y) 181 (6.”
MethodInfluentialSOME DESIRABLE PROPERTIES OF THE BONFERRONI CORRECTION: IS THE BONFERRONI CORRECTION REALLY SO BAD?
VanderWeele, T. J., & Mathur, M. B. (2019) · American Journal of Epidemiology · doi:10.1093/aje/kwy250 · Cited by 424
Self-citationIntraoperative Hypotension Is Associated With Adverse Clinical Outcomes After Noncardiac Surgery
Gregory, A., Stapelfeldt, W. H., Khanna, A. K., Smischney, N. J., Boero, I. J., Chen, Q., Stevens, M., & Shaw, A. D. (2021) · Anesthesia & Analgesia · doi:10.1213/ane.0000000000005250 · Cited by 405
Bachy, E., Le Gouill, S., Di Blasi, R., Sesques, P., Manson, G., Cartron, G., Beauvais, D., Roulin, L., Gros, F. X., Rubio, M. T., et al. (2022) · Nature Medicine · doi:10.1038/s41591-022-01969-y · Cited by 405
Pottegård, A., Lund, L. C., Karlstad, Ø., Dahl, J., Andersen, M., Hallas, J., Lidegaard, Ø., Tapia, G., Gulseth, H. L., Ruiz, P. L.-D., et al. (2021) · BMJ · doi:10.1136/bmj.n1114 · Cited by 401
Aminian, A., Al-Kurd, A., Wilson, R., Bena, J., Fayazzadeh, H., Singh, T., Albaugh, V. L., Shariff, F. U., Rodriguez, N. A., Jin, J., et al. (2021) · JAMA · doi:10.1001/jama.2021.19569 · Cited by 375
Multisystem Inflammatory Syndrome in Children — Initial Therapy and Outcomes
Son, M. B. F., Murray, N., Friedman, K., Young, C. C., Newhams, M. M., Feldstein, L. R., Loftis, L. L., Tarquinio, K. M., Singh, A. R., Heidemann, S. M., et al. (2021) · New England Journal of Medicine · doi:10.1056/nejmoa2102605 · Cited by 366
Association of Bariatric Surgery With Cancer Risk and Mortality in Adults With Obesity
Aminian, A., Wilson, R., Al-Kurd, A., Tu, C., Milinovich, A., Kroh, M., Rosenthal, R. J., Brethauer, S. A., Schauer, P. R., Kattan, M. W., et al. (2022) · JAMA · doi:10.1001/jama.2022.9009 · Cited by 357
Attention-deficit/hyperactivity disorder
Faraone, S. V., Bellgrove, M. A., Brikell, I., Cortese, S., Hartman, C. A., Hollis, C., Newcorn, J. H., Philipsen, A., Polanczyk, G. V., Rubia, K., et al. (2024) · Nature Reviews Disease Primers · doi:10.1038/s41572-024-00495-0 · Cited by 355
“Low risk ratios are difficult to interpret because low levels of risk could be due to confounding 95,96 .”
Sensitivity Analysis for Publication Bias in Meta-Analyses
Mathur, M. B., & VanderWeele, T. J. (2020) · Journal of the Royal Statistical Society Series C: Applied Statistics · doi:10.1111/rssc.12440 · Cited by 354
“That is, analogously to recent work on sensitivity analysis for unmeasured confounding (VanderWeele and Ding, 2017), the present methods allow statements such as ‘For publication bias to “explain away” the meta-analytic pooled point estimate completely (i.e. to attenuate the population effect to…”
MethodResultInfluentialSelf-citationAddressing context dependence in ecology
Catford, J. A., Wilson, J. R. U., Pyšek, P., Hulme, P. E., & Duncan, R. P. (2022) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2021.09.007 · Cited by 344
“Different methods are available for identifying different types of confounders [42,96,97].”
MethodThe Taboo Against Explicit Causal Inference in Nonexperimental Psychology
Grosz, M. P., Rohrer, J. M., & Thoemmes, F. (2020) · Perspectives on Psychological Science · doi:10.1177/1745691620921521 · Cited by 322
“…a researcher might want to compute what would happen if unobservable variables were to confound the effect (for more information on sensitivity analysis, see, e.g., Frank, Maroulis, Duong, & Kelcey, 2013; Greenland, 1996; Rosenbaum, 2005; Rosenberg, Xu, & Frank, 2019; VanderWeele & Ding, 2017).”
BackgroundOutcome-Wide Longitudinal Designs for Causal Inference: A New Template for Empirical Studies
VanderWeele, T. J., Mathur, M. B., & Chen, Y. (2020) · Statistical Science · doi:10.1214/19-sts728 · Cited by 310
“…at least approximate, scale for assessing robustness to unmeasured confounding across different types of outcomes, though the E-value itself must always be interpreted within the context of the particular exposure, outcome and set of covariates under consideration (VanderWeele and Ding, 2017).”
BackgroundMethodInfluentialSelf-citationChow, J. H., Khanna, A. K., Kethireddy, S., Yamane, D., Levine, A., Jackson, A. M., McCurdy, M. T., Tabatabai, A., Kumar, G., Park, P., et al. (2021) · Anesthesia & Analgesia · doi:10.1213/ane.0000000000005292 · Cited by 309
“would be needed to mitigate the association between aspirin use and the need for mechanical ventilation.(29) AC CE PT ED”
BackgroundVaccine Effectiveness of JYNNEOS against Mpox Disease in the United States
Deputy, N. P., Deckert, J., Chard, A. N., Sandberg, N., Moulia, D. L., Barkley, E., Dalton, A. F., Sweet, C., Cohn, A. C., Little, D. R., et al. (2023) · New England Journal of Medicine · doi:10.1056/nejmoa2215201 · Cited by 305
The impact of automation and artificial intelligence on worker well-being
Nazareno, L., & Schiff, D. S. (2021) · Technology in Society · doi:10.1016/j.techsoc.2021.101679 · Cited by 302
Fisher, D. P., Johnson, E., Haneuse, S., Arterburn, D., Coleman, K. J., O’Connor, P. J., O’Brien, R., Bogart, A., Theis, M. K., Anau, J., et al. (2018) · JAMA · doi:10.1001/jama.2018.14619 · Cited by 305
Williams, A., Kamper, S. J., Wiggers, J. H., O’Brien, K. M., Lee, H., Wolfenden, L., Yoong, S. L., Robson, E., McAuley, J. H., Hartvigsen, J., et al. (2018) · BMC Medicine · doi:10.1186/s12916-018-1151-2 · Cited by 305
Vai, B., Mazza, M. G., Delli Colli, C., Foiselle, M., Allen, B., Benedetti, F., Borsini, A., Casanova Dias, M., Tamouza, R., Leboyer, M., et al. (2021) · The Lancet Psychiatry · doi:10.1016/s2215-0366(21)00232-7 · Cited by 305
“16 Larger E-values indicate a greater amount of unmeasured confounding is needed to explain away an effect estimate.”
Bramante, C. T., Ingraham, N. E., Murray, T. A., Marmor, S., Hovertsen, S., Gronski, J., McNeil, C., Feng, R., Guzman, G., Abdelwahab, N., et al. (2021) · The Lancet Healthy Longevity · doi:10.1016/s2666-7568(20)30033-7 · Cited by 291
Propensity score matching with R: conventional methods and new features
Zhao, Q.-Y., Luo, J.-C., Su, Y., Zhang, Y.-J., Tu, G.-W., & Luo, Z. (2021) · Annals of Translational Medicine · doi:10.21037/atm-20-3998 · Cited by 291
Gazit, S., Shlezinger, R., Perez, G., Lotan, R., Peretz, A., Ben-Tov, A., Herzel, E., Alapi, H., Cohen, D., Muhsen, K., et al. (2022) · Clinical Infectious Diseases · doi:10.1093/cid/ciac262 · Cited by 283
“The E-value is defined as the minimum strength of association that an unmeasured confounder would need to have with both the exposure and the outcome to fully explain away a specific exposure-outcome association, conditional on the measured covariates [18].”
Stuart, J. J., Tanz, L. J., Missmer, S. A., Rimm, E. B., Spiegelman, D., James-Todd, T. M., & Rich-Edwards, J. W. (2018) · Annals of Internal Medicine · doi:10.7326/m17-2740 · Cited by 279
Schultze, A., Walker, A. J., MacKenna, B., Morton, C. E., Bhaskaran, K., Brown, J. P., Rentsch, C. T., Williamson, E., Drysdale, H., Croker, R., et al. (2020) · The Lancet Respiratory Medicine · doi:10.1016/s2213-2600(20)30415-x · Cited by 275
“37 We managed the data using Python (version 3.8) and SQL, with analysis carried out using Stata (version 16.1).”
InfluentialLindenauer, P. K., Stefan, M. S., Pekow, P. S., Mazor, K. M., Priya, A., Spitzer, K. A., Lagu, T. C., Pack, Q. R., Pinto-Plata, V. M., & ZuWallack, R. (2020) · JAMA · doi:10.1001/jama.2020.4437 · Cited by 272
Religious Communities and Human Flourishing
VanderWeele, T. J. (2017) · Current Directions in Psychological Science · doi:10.1177/0963721417721526 · Cited by 268
“It is, however, possible to use sensitivity analysis (VanderWeele and Ding, 2017) to examine how strong such unmeasured confounding would have to be to explain away the associations.”
Self-citationHultcrantz, M., Björkholm, M., Dickman, P. W., Landgren, O., Derolf, Å. R., Kristinsson, S. Y., & Andersson, T. M. L. (2018) · Annals of Internal Medicine · doi:10.7326/m17-0028 · Cited by 268
Salter, A., Fox, R. J., Newsome, S. D., Halper, J., Li, D. K. B., Kanellis, P., Costello, K., Bebo, B., Rammohan, K., Cutter, G. R., et al. (2021) · JAMA Neurology · doi:10.1001/jamaneurol.2021.0688 · Cited by 267
Orellano, P., Reynoso, J., & Quaranta, N. (2021) · Environment International · doi:10.1016/j.envint.2021.106434 · Cited by 256
Steptoe, A., & Fancourt, D. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1814723116 · Cited by 254
“We also computed E-values: indicators of how robust findings are to potential unmeasured confounding (12).”
BackgroundMethodLife Satisfaction and Subsequent Physical, Behavioral, and Psychosocial Health in Older Adults
KIM, E. S., DELANEY, S. W., TAY, L., CHEN, Y., DIENER, E., & VANDERWEELE, T. J. (2021) · The Milbank Quarterly · doi:10.1111/1468-0009.12497 · Cited by 245
Self-citationEpstein–Barr virus as a leading cause of multiple sclerosis: mechanisms and implications
Bjornevik, K., Münz, C., Cohen, J. I., & Ascherio, A. (2023) · Nature Reviews Neurology · doi:10.1038/s41582-023-00775-5 · Cited by 246
Ling, S., Brown, K., Miksza, J. K., Howells, L., Morrison, A., Issa, E., Yates, T., Khunti, K., Davies, M. J., & Zaccardi, F. (2020) · Diabetes Care · doi:10.2337/dc20-0204 · Cited by 241
“1)using the “Evalue” and “metafor” packages (10,22); Stata/IC 16.”
BackgroundResultMeaningful associations in the adolescent brain cognitive development study
Dick, A. S., Lopez, D. A., Watts, A. L., Heeringa, S., Reuter, C., Bartsch, H., Fan, C. C., Kennedy, D. N., Palmer, C., Marshall, A., et al. (2021) · NeuroImage · doi:10.1016/j.neuroimage.2021.118262 · Cited by 241
Causal Inference About the Effects of Interventions From Observational Studies in Medical Journals
Dahabreh, I. J., & Bibbins-Domingo, K. (2024) · JAMA · doi:10.1001/jama.2024.7741 · Cited by 242
Job Insecurity and Symptoms of Anxiety and Depression Among U.S. Young Adults During COVID-19
Ganson, K. T., Tsai, A. C., Weiser, S. D., Benabou, S. E., & Nagata, J. M. (2021) · Journal of Adolescent Health · doi:10.1016/j.jadohealth.2020.10.008 · Cited by 237
Walmsley, R., Chan, S., Smith-Byrne, K., Ramakrishnan, R., Woodward, M., Rahimi, K., Dwyer, T., Bennett, D., & Doherty, A. (2022) · British Journal of Sports Medicine · doi:10.1136/bjsports-2021-104050 · Cited by 238
Rentsch, C. T., Beckman, J. A., Tomlinson, L., Gellad, W. F., Alcorn, C., Kidwai-Khan, F., Skanderson, M., Brittain, E., King, J. T., Ho, Y.-L., et al. (2021) · BMJ · doi:10.1136/bmj.n311 · Cited by 237
“27 To assess for undue effects from outliers with very high or very low estimated propensity of treatment, we capped propensity score distributions at the 1st and 99th centiles and again at the 5th and 95th centiles.”
Wadström, B. N., Wulff, A. B., Pedersen, K. M., Jensen, G. B., & Nordestgaard, B. G. (2022) · European Heart Journal · doi:10.1093/eurheartj/ehab705 · Cited by 234
Zheng, X.-Y., Orellano, P., Lin, H.-L., Jiang, M., & Guan, W.-J. (2021) · Environment International · doi:10.1016/j.envint.2021.106435 · Cited by 229
Wang, Y., Nie, J., Ferrari, G., Rey-Lopez, J. P., & Rezende, L. F. M. (2021) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2020.6331 · Cited by 229
Zeng, C., Lane, N. E., Hunter, D. J., Wei, J., Choi, H. K., McAlindon, T. E., Li, H., Lu, N., Lei, G., & Zhang, Y. (2019) · Osteoarthritis and Cartilage · doi:10.1016/j.joca.2019.01.007 · Cited by 228
Wu, X., Braun, D., Schwartz, J., Kioumourtzoglou, M. A., & Dominici, F. (2020) · Science Advances · doi:10.1126/sciadv.aba5692 · Cited by 226
“We conducted further sensitivity analyses to unmeasured confounding by calculating the E-value (10, 11).”
InfluentialHandgun Ownership and Suicide in California
Studdert, D. M., Zhang, Y., Swanson, S. A., Prince, L., Rodden, J. A., Holsinger, E. E., Spittal, M. J., Wintemute, G. J., & Miller, M. (2020) · New England Journal of Medicine · doi:10.1056/nejmsa1916744 · Cited by 224
Gerber, J. S., Ross, R. K., Bryan, M., Localio, A. R., Szymczak, J. E., Wasserman, R., Barkman, D., Odeniyi, F., Conaboy, K., Bell, L., et al. (2017) · JAMA · doi:10.1001/jama.2017.18715 · Cited by 224
Virtanen, M., Jokela, M., Madsen, I. E., Magnusson Hanson, L. L., Lallukka, T., Nyberg, S. T., Alfredsson, L., Batty, G. D., Bjorner, J. B., Borritz, M., et al. (2018) · Scandinavian Journal of Work, Environment & Health · doi:10.5271/sjweh.3712 · Cited by 268
“In combination, unmeasured confounders should be associated both with long working hours and depressive symptoms with an odds ratio of 1.54 (95% CI 1.21–1.81) to entirely explain the present findings (95).”
BackgroundInfluentialChen, Y., & VanderWeele, T. J. (2018) · American Journal of Epidemiology · doi:10.1093/aje/kwy142 · Cited by 218
“To assess the robustness of the observed associations to unmeasured confounding, we calculated E-values (33) for the associations of religious service attendance (at least weekly vs.”
BackgroundMethodInfluentialSelf-citationAnderson, M. R., Geleris, J., Anderson, D. R., Zucker, J., Nobel, Y. R., Freedberg, D., Small-Saunders, J., Rajagopalan, K. N., Greendyk, R., Chae, S.-R., et al. (2020) · Annals of Internal Medicine · doi:10.7326/m20-3214 · Cited by 216
Ravulizumab in Aquaporin‐4–Positive Neuromyelitis Optica Spectrum Disorder
Pittock, S. J., Barnett, M., Bennett, J. L., Berthele, A., de Sèze, J., Levy, M., Nakashima, I., Oreja‐Guevara, C., Palace, J., Paul, F., et al. (2023) · Annals of Neurology · doi:10.1002/ana.26626 · Cited by 212
“Because using an external comparator could introduce confounding, a sensitivity analysis was conducted using propensity scores to balance treatment groups,(18) and a tipping-point analysis (E-value) was used to estimate, in terms of a relative risk, the amount of unmeasured confounding that would have been required to…”
MethodMenopausal hormone therapy and women’s health: An umbrella review
Zhang, G.-Q., Chen, J.-L., Luo, Y., Mathur, M. B., Anagnostis, P., Nurmatov, U., Talibov, M., Zhang, J., Hawrylowicz, C. M., Lumsden, M. A., et al. (2021) · PLOS Medicine · doi:10.1371/journal.pmed.1003731 · Cited by 208
“All estimates were converted to the RR scale [48,53– 55], and results are presented on that scale, except where otherwise noted.”
MethodAssociation of Short Interpregnancy Interval With Pregnancy Outcomes According to Maternal Age
Schummers, L., Hutcheon, J. A., Hernandez-Diaz, S., Williams, P. L., Hacker, M. R., VanderWeele, T. J., & Norman, W. V. (2018) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2018.4696 · Cited by 205
Self-citationRonit, A., Kirkegaard-Klitbo, D. M., Dohlmann, T. L., Lundgren, J., Sabin, C. A., Phillips, A. N., Nordestgaard, B. G., & Afzal, S. (2020) · Arteriosclerosis, Thrombosis, and Vascular Biology · doi:10.1161/atvbaha.119.313681 · Cited by 204
Durand, C. M., Bowring, M. G., Thomas, A. G., Kucirka, L. M., Massie, A. B., Cameron, A., Desai, N. M., Sulkowski, M., & Segev, D. L. (2018) · Annals of Internal Medicine · doi:10.7326/m17-2451 · Cited by 200
“E-values were calculated using the E-value package in R statistical software (24, 25).”
Boehm, J. K., Chen, Y., Koga, H., Mathur, M. B., Vie, L. L., & Kubzansky, L. D. (2018) · Circulation Research · doi:10.1161/circresaha.117.310828 · Cited by 194
Abnormalities of Potassium in Heart Failure
Ferreira, J. P., Butler, J., Rossignol, P., Pitt, B., Anker, S. D., Kosiborod, M., Lund, L. H., Bakris, G. L., Weir, M. R., & Zannad, F. (2020) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2020.04.021 · Cited by 190
Davies, M.-A. (2020) · medRxiv · doi:10.1101/2020.07.02.20145185 · Cited by 192
Chen, Y., Kim, E. S., & VanderWeele, T. J. (2021) · International Journal of Epidemiology · doi:10.1093/ije/dyaa120 · Cited by 185
“See VanderWeele and Ding (ref no.(39)) for the formula for calculating E-values.”
BackgroundSelf-citationBonaccio, M., Di Castelnuovo, A., Costanzo, S., De Curtis, A., Persichillo, M., Sofi, F., Cerletti, C., Donati, M. B., de Gaetano, G., & Iacoviello, L. (2021) · The American Journal of Clinical Nutrition · doi:10.1093/ajcn/nqaa299 · Cited by 181
“Finally, by the use of E-values [a parameter that calculates the minimum strength of association that an unmeasured confounder would need to have with both the exposure and outcome, to cancel a specific exposure–outcome association (24)], we found that to nullify the reported association, unobserved confounders should…”
MethodClift, A. K., von Ende, A., Tan, P. S., Sallis, H. M., Lindson, N., Coupland, C. A. C., Munafò, M. R., Aveyard, P., Hippisley-Cox, J., & Hopewell, J. C. (2022) · Thorax · doi:10.1136/thoraxjnl-2021-217080 · Cited by 186
Chan, G. C. K., Stjepanović, D., Lim, C., Sun, T., Shanmuga Anandan, A., Connor, J. P., Gartner, C., Hall, W. D., & Leung, J. (2021) · Addiction · doi:10.1111/add.15246 · Cited by 183
Schubert, J., Lindahl, B., Melhus, H., Renlund, H., Leosdottir, M., Yari, A., Ueda, P., James, S., Reading, S. R., Dluzniewski, P. J., et al. (2021) · European Heart Journal · doi:10.1093/eurheartj/ehaa1011 · Cited by 185
Kusamura, S., Barretta, F., Yonemura, Y., Sugarbaker, P. H., Moran, B. J., Levine, E. A., Goere, D., Baratti, D., Nizri, E., Morris, D. L., et al. (2021) · JAMA Surgery · doi:10.1001/jamasurg.2020.6363 · Cited by 179
VanderWeele, T. J., Chen, Y., Long, K., Kim, E. S., Trudel-Fitzgerald, C., & Kubzansky, L. D. (2020) · Epidemiology · doi:10.1097/ede.0000000000001147 · Cited by 181
Self-citationAssociation Between Receipt of a Medically Tailored Meal Program and Health Care Use
Berkowitz, S. A., Terranova, J., Randall, L., Cranston, K., Waters, D. B., & Hsu, J. (2019) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2019.0198 · Cited by 176
Huo, G., Tang, Y., Liu, Z., Cao, J., Yao, Z., & Zhou, D. (2025) · Cardiovascular Diabetology · doi:10.1186/s12933-025-02686-9 · Cited by 167
“The calculation method of E-value was as follows: E = RR + sqrt{RR × (RR−1)} [36].”
Personality Trait Stability and Change
Bleidorn, W., Hopwood, C. J., Back, M. D., Denissen, J. J. A., Hennecke, M., Hill, P. L., Jokela, M., Kandler, C., Lucas, R. E., Luhmann, M., et al. (2021) · Personality Science · doi:10.5964/ps.6009 · Cited by 171
“However, as any single study design, correlational studies – even if longitudinal – are limited in their ability to inform questions about the causes of personality trait change (Schwaba et al., 2021; VanderWeele & Ding, 2017).”
BackgroundChen, H., Dhana, K., Huang, Y., Huang, L., Tao, Y., Liu, X., Melo van Lent, D., Zheng, Y., Ascherio, A., Willett, W., et al. (2023) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2023.0800 · Cited by 170
Lim, C. C., Hayes, R. B., Ahn, J., Shao, Y., Silverman, D. T., Jones, R. R., & Thurston, G. D. (2019) · Circulation · doi:10.1161/circulationaha.118.035742 · Cited by 169
Bounding Bias Due to Selection
Smith, L. H., & VanderWeele, T. J. (2019) · Epidemiology · doi:10.1097/ede.0000000000001032 · Cited by 164
Self-citationFest, J., Ruiter, T. R., Groot Koerkamp, B., Rizopoulos, D., Ikram, M. A., van Eijck, C. H. J., & Stricker, B. H. (2019) · European Journal of Epidemiology · doi:10.1007/s10654-018-0472-y · Cited by 166
Mushavi, R. C., Burns, B. F. O., Kakuhikire, B., Owembabazi, M., Vořechovská, D., McDonough, A. Q., Cooper-Vince, C. E., Baguma, C., Rasmussen, J. D., Bangsberg, D. R., et al. (2020) · Social Science & Medicine · doi:10.1016/j.socscimed.2019.112561 · Cited by 164
Li, H., Zuo, Y., Qian, F., Chen, S., Tian, X., Wang, P., Li, X., Guo, X., Wu, S., & Wang, A. (2022) · Cardiovascular Diabetology · doi:10.1186/s12933-022-01541-5 · Cited by 164
“2-y lag analysis was conducted, which excluded CVD events that occurred within 2 years of follow-up; (6) other variability measures including SD, coefficient of variation (CV), and independent of the mean (VIM) were calculated [19]; and (7) to assess the influence of unmeasured confounding, E-value, which is defined a…”
MethodWright, A. K., Carr, M. J., Kontopantelis, E., Leelarathna, L., Thabit, H., Emsley, R., Buchan, I., Mamas, M. A., van Staa, T. P., Sattar, N., et al. (2022) · Diabetes Care · doi:10.2337/dc21-1113 · Cited by 166
The Association of Preoperative Frailty and Postoperative Delirium: A Meta-analysis
Gracie, T. J., Caufield-Noll, C., Wang, N.-Y., & Sieber, F. E. (2021) · Anesthesia & Analgesia · doi:10.1213/ane.0000000000005609 · Cited by 164
Metadata
- Authors
- VanderWeele, T. J., & Ding, P.
- Affiliations
- Harvard University (US)
- Venue
- Annals of Internal Medicine
- Publisher
- American College of Physicians
- Published
- 2017-08-15
- Volume
- 167
- Issue
- 4
- Pages
- 268-274
- ISSN
- 0003-4819 (ISSN-L)
- 1539-3704
- OpenAlex
- W2735301415
- Cited by
- 6,350
- References
- 59
- Citations by year
- 2026: 1516 · 2025: 1020 · 2024: 860 · 2023: 736 · 2022: 716 · 2021: 583 · 2020: 498 · 2019: 290 · 2018: 113 · 2017: 15
- FWCI
- 174.58
- Type
- journal article
- Language
- en
- Open access
- green
- nrs.harvard.edu
- Licence
- cc-by
- www.acpjournals.org/journal/aim/text-and-data-mining (tdm)
- Locations
- Annals of Internal Medicine (publishedVersion)
- PubMed (publishedVersion)
- Digital Access to Scholarship at Harvard (DASH) (Harvard University) (submittedVersion, cc-by)
- Funders
- National Institute of Environmental Health Sciences (ES017876)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- E-value, unmeasured confounding, sensitivity analysis, causal inference, observational studies, relative risk
- MeSH terms
- Humans, Sensitivity and Specificity, Epidemiologic Research Design, Confounding Factors, Epidemiologic, Observational Studies as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.

