The DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index.
The Hazards of Hazard Ratios
Open accessAbstract
The hazard ratio (HR) is the main, and often the only, effect measure reported in many epidemiologic studies. For dichotomous, non–time-varying exposures, the HR is defined as the hazard in the exposed groups divided by the hazard in the unexposed groups. For all practical purposes, hazards can be thought of as incidence rates and thus the HR can be roughly interpreted as the incidence rate ratio. The HR is commonly and conveniently estimated via a Cox proportional hazards model, which can include potential confounders as covariates. Unfortunately, the use of the HR for causal inference is not straightforward even in the absence of unmeasured confounding, measurement error, and model misspecification. Endowing a HR with a causal interpretation is risky for 2 key reasons: the HR may change over time, and the HR has a built-in selection bias. Here I review these 2 problems and some proposed solutions. As an example, I will use the findings from a Women's Health Initiative randomized experiment that compared the risk of coronary heart disease of women assigned to combined (estrogen plus progestin) hormone therapy with that of women assigned to placebo.1 By using a randomized experiment as an example, the discussion can focus on the shortcomings of the HR, setting aside issues of confounding and other serious problems that arise in observational studies. The Women's Health Initiative followed over 16,000 women for an average of 5.2 years before the study was halted due to safety concerns. The primary result from the trial was a HR. As stated in the abstract1 and shown in Table 1 of the article, “Combined hormone therapy was associated with a hazard ratio of 1.24.”1 In addition, Table 2 provided the HRs during each year of follow-up: 1.81, 1.34, 1.27, 1.25, 1.45, and 0.70 for years 1, 2, 3, 4, 5, and 6+, respectively. Thus, the HR reported in the abstract and Table 1 can be viewed as some sort of weighted average of the period-specific HRs reported in Table 2. This bring us to Problem 1: although the HR may change over time, some studies report only a single HR averaged over the duration of the study's follow-up. As a result, the conclusions from the study may critically depend on the duration of the follow-up. For example, the average HR in the WHI would have been 1.8 if the study had been halted after 1 year of follow-up, 1.7 after 2 years,2 1.2 after 5 years, and—who knows—perhaps 1.0 after 10 years. The 24% increase in the rate of coronary heart disease that many researchers and journalists consider as the effect of combined hormone therapy is the result of the arbitrary choice of an average follow-up period of 5.2 years. A trial with a shorter follow-up could have reported an 80% increase, whereas a longer trial might have found little or no increase at all. The magnitude of the average HR depends on the length of follow-up because the average HR ignores the distribution of events during the follow-up. The average HR can take the value 1.0 if the hazard in the exposed is identical to the hazard in the unexposed during the entire follow-up, or if the hazard in the exposed is higher during, say, the first 5 years and lower afterward. Incidentally, the same problem arises whether the average HR is directly estimated in a cohort study, as discussed here, or estimated via the odds ratio of a properly designed case-control study with incidence density sampling. One might then conclude that we should forget about the average HR and restrict our attention to the period-specific HRs, which seem to capture the potentially time-varying magnitude of the effect. This brings us to Problem 2: the period-specific HRs have a built-in selection bias. To describe the bias, consider that the (discrete-time) hazard during period t is defined as the risk of the outcome during period t among those who reached period t free of the outcome. In the Women's Health Initiative, the calculation of the HR during year t was restricted to women who did not develop coronary heart disease—the “survivors”—between baseline and the beginning of year t. The HR after year 5 was 0.7, which means that the disease rate after year 5 was lower in the treatment arm (the hazard in the numerator of the HR) than in the placebo arm (the hazard in the denominator). However, this apparently protective effect of hormone therapy after year 5 is hardly surprising if one bears in mind that women vary in their susceptibility to heart disease. A certain proportion of all women enrolled in the trial were particularly prone to develop heart disease if they were exposed to hormone therapy or other factors (for simplicity, let's refer to them as the “susceptible women”). The proportion of susceptible women in the trial was of course unknown but, because of randomization, it was expected to be the same in both the treatment and placebo arms at baseline. However, these susceptible women were preferentially excluded from the treatment arm as they developed heart disease
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A structural approach to selection bias
Hernán, M. A., Hernández-Díaz, S., & Robins, J. M. (2004) · Epidemiology · doi:10.1097/01.ede.0000135174.63482.43 · Cited by 2,844 · cited 1×
“This built-in selection bias of the HR has also been described using causal diagrams.(3,4) In short, the average HR may be uninformative because of potentially time-varying period-specific HRs, and because the period-specific HRs may be time-varying because of built-in selection bias.”
MethodSelf-citationEstrogen plus Progestin and the Risk of Coronary Heart Disease
Manson, J. E., Hsia, J., Johnson, K. C., Rossouw, J. E., Assaf, A. R., Lasser, N. L., Trevisan, M., Black, H. R., Heckbert, S. R., Detrano, R., et al. (2003) · New England Journal of Medicine · doi:10.1056/nejmoa030808 · Cited by 2,065 · cited 1×
“As an example, I will use the findings from a Women’s Health Initiative randomized experiment that compared the risk of coronary heart disease of women assigned to combined (estrogen plus progestin) hormone therapy with that of women assigned to placebo.(1) By using a randomized experiment as an example, the discussio…”
MethodHernán, M. A., Alonso, A., Logan, R., Grodstein, F., Michels, K. B., Willett, W. C., Manson, J. E., & Robins, J. M. (2008) · Epidemiology · doi:10.1097/ede.0b013e3181875e61 · Cited by 818
Self-citationAdjusted survival curves with inverse probability weights
Cole, S. R., & Hernán, M. A. (2004) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2003.10.004 · Cited by 814
Self-citationImputation for Exposure Histories with Gaps, under an Excess Relative Risk Model
Weinberg, C. R., Moledor, E. S., Umbach, D. M., & Sandler, D. P. (1996) · Epidemiology · doi:10.1097/00001648-199609000-00007 · Cited by 178
Hernán, M. A., Cole, S. R., Margolick, J., Cohen, M., & Robins, J. M. (2005) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.1064 · Cited by 169
Self-citationStatistical Issues Arising in the Women's Health Initiative
Prentice, R. L., Pettinger, M. B., & Anderson, G. L. (2005) · Biometrics · doi:10.1111/j.0006-341x.2005.454_1.x · Cited by 109
Absence of Confounding Does Not Correspond to Collapsibility of the Rate Ratio or Rate Difference
Greenland, S. (1996) · Epidemiology · doi:10.1097/00001648-199609000-00008 · Cited by 82
Properties of 2 Counterfactual Effect Definitions of a Point Exposure
Flanders, W. D., & Klein, M. (2007) · Epidemiology · doi:10.1097/01.ede.0000261472.07150.4f · Cited by 50
Zhang, Y., Thamer, M., Cotter, D. J., Kaufman, J. S., & Hernán, M. Á. (2009) · Clinical Journal of the American Society of Nephrology · doi:10.2215/cjn.05071008 · Cited by 50
Self-citation
Cited by
BNT162b2 mRNA Covid-19 Vaccine in a Nationwide Mass Vaccination Setting
Dagan, N., Barda, N., Kepten, E., Miron, O., Perchik, S., Katz, M. A., Hernán, M. A., Lipsitch, M., Reis, B., & Balicer, R. D. (2021) · New England Journal of Medicine · doi:10.1056/nejmoa2101765 · Cited by 2,527
Self-citationInitial Invasive or Conservative Strategy for Stable Coronary Disease
Maron, D. J., Hochman, J. S., Reynolds, H. R., Bangalore, S., O’Brien, S. M., Boden, W. E., Chaitman, B. R., Senior, R., López-Sendón, J., Alexander, K. P., et al. (2020) · New England Journal of Medicine · doi:10.1056/nejmoa1915922 · Cited by 2,449
Lepeule, J., Laden, F., Dockery, D., & Schwartz, J. (2012) · Environmental Health Perspectives · doi:10.1289/ehp.1104660 · Cited by 983
Target Trial Emulation: A Framework for Causal Inference From Observational Data
Hernán, M. A., Wang, W., & Leaf, D. E. (2022) · JAMA · doi:10.1001/jama.2022.21383 · Cited by 994
Self-citationMoving Beyond the Hazard Ratio in Quantifying the Between-Group Difference in Survival Analysis
Uno, H., Claggett, B., Tian, L., Inoue, E., Gallo, P., Miyata, T., Schrag, D., Takeuchi, M., Uyama, Y., Zhao, L., et al. (2014) · Journal of Clinical Oncology · doi:10.1200/jco.2014.55.2208 · Cited by 810
Hurvitz, S. A., Hegg, R., Chung, W.-P., Im, S.-A., Jacot, W., Ganju, V., Chiu, J. W. Y., Xu, B., Hamilton, E., Madhusudan, S., et al. (2023) · The Lancet · doi:10.1016/s0140-6736(22)02420-5 · Cited by 687
Good practices for quantitative bias analysis
Lash, T. L., Fox, M. P., MacLehose, R. F., Maldonado, G., McCandless, L. C., & Greenland, S. (2014) · International Journal of Epidemiology · doi:10.1093/ije/dyu149 · Cited by 646
Why Test for Proportional Hazards?
Stensrud, M. J., & Hernán, M. A. (2020) · JAMA · doi:10.1001/jama.2020.1267 · Cited by 532
Self-citationManagement of coronary disease in patients with advanced kidney disease
Bangalore, S., Maron, D. J., O’Brien, S. M., Fleg, J. L., Kretov, E. I., Briguori, C., Kaul, U., Reynolds, H. R., Mazurek, T., Sidhu, M. S., et al. (2020) · New England Journal of Medicine · doi:10.1056/nejmoa1915925 · Cited by 499
Generalizing Evidence From Randomized Clinical Trials to Target Populations: The ACTG 320 Trial
Cole, S. R., & Stuart, E. A. (2010) · American Journal of Epidemiology · doi:10.1093/aje/kwq084 · Cited by 486
Selection Bias Due to Loss to Follow Up in Cohort Studies
Howe, C. J., Cole, S. R., Lau, B., Napravnik, S., & Eron, J. J. (2016) · Epidemiology · doi:10.1097/ede.0000000000000409 · Cited by 467
Cardiovascular Disease Mortality Among Breast Cancer Survivors
Bradshaw, P. T., Stevens, J., Khankari, N., Teitelbaum, S. L., Neugut, A. I., & Gammon, M. D. (2016) · Epidemiology · doi:10.1097/ede.0000000000000394 · Cited by 443
Intrinsic Breast Tumor Subtypes, Race, and Long-Term Survival in the Carolina Breast Cancer Study
O'Brien, K. M., Cole, S. R., Tse, C.-K., Perou, C. M., Carey, L. A., Foulkes, W. D., Dressler, L. G., Geradts, J., & Millikan, R. C. (2010) · Clinical Cancer Research · doi:10.1158/1078-0432.ccr-10-1533 · Cited by 429
Chlebowski, R. T., Anderson, G. L., Aragaki, A. K., Manson, J. E., Stefanick, M. L., Pan, K., Barrington, W., Kuller, L. H., Simon, M. S., Lane, D., et al. (2020) · JAMA · doi:10.1001/jama.2020.9482 · Cited by 427
PFAS and cancer, a scoping review of the epidemiologic evidence
Steenland, K., & Winquist, A. (2021) · Environmental Research · doi:10.1016/j.envres.2020.110690 · Cited by 426
Leening, M. J. G., Ferket, B. S., Steyerberg, E. W., Kavousi, M., Deckers, J. W., Nieboer, D., Heeringa, J., Portegies, M. L. P., Hofman, A., Ikram, M. A., et al. (2014) · BMJ · doi:10.1136/bmj.g5992 · Cited by 352
Young, J. G., Stensrud, M. J., Tchetgen Tchetgen, E. J., & Hernán, M. A. (2020) · Statistics in Medicine · doi:10.1002/sim.8471 · Cited by 338
Self-citationAdam, M. A., Pura, J., Goffredo, P., Dinan, M. A., Reed, S. D., Scheri, R. P., Hyslop, T., Roman, S. A., & Sosa, J. A. (2015) · Journal of Clinical Oncology · doi:10.1200/jco.2014.59.8391 · Cited by 335
Crowley, M. J., Diamantidis, C. J., McDuffie, J. R., Cameron, C. B., Stanifer, J. W., Mock, C. K., Wang, X., Tang, S., Nagi, A., Kosinski, A. S., et al. (2017) · Annals of Internal Medicine · doi:10.7326/m16-1901 · Cited by 287
Danaei, G., Rodríguez, L. A. G., Cantero, O. F., Logan, R., & Hernán, M. A. (2013) · Statistical Methods in Medical Research · doi:10.1177/0962280211403603 · Cited by 292
Self-citationIntention to treat and per protocol analysis in clinical trials
Tripepi, G., Chesnaye, N. C., Dekker, F. W., Zoccali, C., & Jager, K. J. (2020) · Nephrology · doi:10.1111/nep.13709 · Cited by 284
Balan, T. A., & Putter, H. (2020) · Statistical Methods in Medical Research · doi:10.1177/0962280220921889 · Cited by 279
Polis, C. B., Curtis, K. M., Hannaford, P. C., Phillips, S. J., Chipato, T., Kiarie, J. N., Westreich, D. J., & Steyn, P. S. (2016) · AIDS · doi:10.1097/qad.0000000000001228 · Cited by 196
Ambient air pollution and clinical dementia: systematic review and meta-analysis
Wilker, E. H., Osman, M., & Weisskopf, M. G. (2023) · BMJ · doi:10.1136/bmj-2022-071620 · Cited by 186
Cepoiu-Martin, M., Tam-Tham, H., Patten, S., Maxwell, C. J., & Hogan, D. B. (2016) · International Journal of Geriatric Psychiatry · doi:10.1002/gps.4449 · Cited by 175
Israelsen, M., Torp, N., Johansen, S., Hansen, C. D., Hansen, E. D., Thorhauge, K., Hansen, J. K., Villesen, I., Bech, K., Wernberg, C., et al. (2024) · The Lancet Gastroenterology & Hepatology · doi:10.1016/s2468-1253(23)00443-0 · Cited by 174
Daniel, R., Zhang, J., & Farewell, D. (2021) · Biometrical Journal · doi:10.1002/bimj.201900297 · Cited by 167
Ospina-Romero, M., Glymour, M. M., Hayes-Larson, E., Mayeda, E. R., Graff, R. E., Brenowitz, W. D., Ackley, S. F., Witte, J. S., & Kobayashi, L. C. (2020) · JAMA Network Open · doi:10.1001/jamanetworkopen.2020.25515 · Cited by 124
Mendoza, K., Smith-Warner, S. A., Rossato, S. L., Khandpur, N., Manson, J. E., Qi, L., Rimm, E. B., Mukamal, K. J., Willett, W. C., Wang, M., et al. (2024) · The Lancet Regional Health - Americas · doi:10.1016/j.lana.2024.100859 · Cited by 101
A Structural Description of Biases That Generate Immortal Time
Hernán, M. A., Sterne, J. A. C., Higgins, J. P. T., Shrier, I., & Hernández-Díaz, S. (2025) · Epidemiology · doi:10.1097/ede.0000000000001808 · Cited by 99
Self-citationYau, T., Kaseb, A., Cheng, A.-L., Qin, S., Zhu, A. X., Chan, S. L., Melkadze, T., Sukeepaisarnjaroen, W., Breder, V., Verset, G., et al. (2024) · The Lancet Gastroenterology & Hepatology · doi:10.1016/s2468-1253(23)00454-5 · Cited by 97
Muzambi, R., Bhaskaran, K., Smeeth, L., Brayne, C., Chaturvedi, N., & Warren-Gash, C. (2021) · The Lancet Healthy Longevity · doi:10.1016/s2666-7568(21)00118-5 · Cited by 84
Renoux, C., Dell'Aniello, S., Brenner, B., & Suissa, S. (2017) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.4197 · Cited by 80
Winquist, A., Hodge, J. M., Diver, W. R., Rodriguez, J. L., Troeschel, A. N., Daniel, J., & Teras, L. R. (2023) · Environmental Health Perspectives · doi:10.1289/ehp13174 · Cited by 75
“103,104 – 107 Ongoing exposure prior to enrollment also complicates control for some variables, such as BMI, which might confound some PFAS associations but might also be caused by PFAS exposure and therefore potentially on the causal pathway between PFAS exposure and the outcome.”
BackgroundThyroid Function, Cardiovascular Events, and Mortality in Diabetic Hemodialysis Patients
Drechsler, C., Schneider, A., Gutjahr-Lengsfeld, L., Kroiss, M., Carrero, J. J., Krane, V., Allolio, B., Wanner, C., & Fassnacht, M. (2014) · American Journal of Kidney Diseases · doi:10.1053/j.ajkd.2013.10.009 · Cited by 70
Vandenbroucke, J., & Pearce, N. (2015) · American Journal of Epidemiology · doi:10.1093/aje/kwv225 · Cited by 67
Ambati, J., Magagnoli, J., Leung, H., Wang, S.-B., Andrews, C. A., Fu, D., Pandey, A., Sahu, S., Narendran, S., Hirahara, S., et al. (2020) · Nature Communications · doi:10.1038/s41467-020-18528-z · Cited by 66
Xu, Y., Derakhshan, A., Hysaj, O., Wildisen, L., Ittermann, T., Pingitore, A., Abolhassani, N., Medici, M., Kiemeney, L. A. L. M., Riksen, N. P., et al. (2023) · The Lancet Diabetes & Endocrinology · doi:10.1016/s2213-8587(23)00227-9 · Cited by 61
on behalf of the CHANCES consortium, Arnold, M., Freisling, H., Stolzenberg-Solomon, R., Kee, F., O’Doherty, M. G., Ordóñez-Mena, J. M., Wilsgaard, T., May, A. M., Bueno-de-Mesquita, H. B., et al. (2016) · European Journal of Epidemiology · doi:10.1007/s10654-016-0169-z · Cited by 58
Depletion-of-susceptibles bias in influenza vaccine waning studies: how to ensure robust results
Lipsitch, M., Goldstein, E., Ray, G. T., & Fireman, B. (2019) · Epidemiology and Infection · doi:10.1017/s0950268819001961 · Cited by 56
Richardson, D. B., Laurier, D., Schubauer-Berigan, M. K., Tchetgen, E. T., & Cole, S. R. (2014) · American Journal of Epidemiology · doi:10.1093/aje/kwu211 · Cited by 51
Depletion-of-susceptibles Bias in Analyses of Intra-season Waning of Influenza Vaccine Effectiveness
Ray, G. T., Lewis, N., Klein, N. P., Daley, M. F., Lipsitch, M., & Fireman, B. (2020) · Clinical Infectious Diseases · doi:10.1093/cid/ciz706 · Cited by 48
Association of SARS-CoV-2 BA.4/BA.5 Omicron lineages with immune escape and clinical outcome
Lewnard, J. A., Hong, V., Kim, J. S., Shaw, S. F., Lewin, B., Takhar, H., & Tartof, S. Y. (2022) · doi:10.1101/2022.07.31.22278258 · Cited by 35
Davies, M. J., van der Meer, P., Verma, S., Patel, S., Chinnakondepalli, K. M., Borlaug, B. A., Butler, J., Kitzman, D. W., Shah, S. J., Harring, S., et al. (2025) · The Lancet Diabetes & Endocrinology · doi:10.1016/s2213-8587(24)00304-8 · Cited by 32
Letellier, N., Gutierrez, L., Duchesne, J., Chen, C., Ilango, S., Helmer, C., Berr, C., Mortamais, M., & Benmarhnia, T. (2022) · Alzheimer's & Dementia · doi:10.1002/alz.12606 · Cited by 23
Moholdt, T., Skarpsno, E. S., Moe, B., & Nilsen, T. I. L. (2021) · British Journal of Sports Medicine · doi:10.1136/bjsports-2020-102350 · Cited by 18
Grewal, S., Zaman, N., Borgatta, L., Nudy, M., Foy, A. J., & Peterson, B. (2021) · The American Journal of Cardiology · doi:10.1016/j.amjcard.2021.05.043 · Cited by 14
Xie, Y., Choi, T., & Al-Aly, Z. (2026) · BMJ Medicine · doi:10.1136/bmjmed-2025-002150 · Cited by 8
“67 68 We used multiple approaches to better understand the association between various patterns of use, including scenarios with different durations and timings of discontinuations and interruptions.”
Rewley, J., Fawzi, M. C. S., McAdam, K., Kaaya, S., Liu, Y., Todd, J., Andrew, I., & Onnela, J. P. (2020) · BMJ Open · doi:10.1136/bmjopen-2019-033759 · Cited by 7
Lemmetyinen, R., Karjalainen, J., But, A., Renkonen, R., Pekkanen, J., Haukka, J., & Toppila-Salmi, S. (2021) · BMJ Open · doi:10.1136/bmjopen-2021-053133 · Cited by 7
Metadata
- Authors
- Hernán, M. A.
- Venue
- Epidemiology
- Published
- 2010-01
- Volume
- 21
- Issue
- 1
- Pages
- 13-15
- ISSN
- 1044-3983 (ISSN-L)
- OpenAlex
- W1973885323
- Cited by
- 1,370
- References
- 10
- FWCI
- 9.57
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.