Controversy and Debate: Questionable utility of the relative risk in clinical research: Paper 1: A call for change to practice
Open accessAbstract
BACKGROUND AND OBJECTIVES: In clinical trials, the relative risk or risk ratio (RR) is a mainstay of reporting of the effect magnitude for an intervention. The RR is the ratio of the probability of an outcome in an intervention group to its probability in a control group. Thus, the RR provides a measure of change in the likelihood of an event linked to a given intervention. This measure has been widely used because it is today considered a measure with "portability" across varying outcome prevalence, especially when the outcome is rare. It turns out, however, that there is a much more important problem with this ratio, and this paper aims to demonstrate this problem. METHODS: We used mathematical derivation to determine if the RR is a measure of effect magnitude alone (i.e., a larger absolute value always indicating a stronger effect) or not. We also used the same derivation to determine its relationship to the prevalence of an outcome. We confirm the derivation results with a follow-up analysis of 140,620 trials scraped from the Cochrane. RESULTS: We demonstrate that the RR varies for reasons other than the magnitude of the effect because it is a ratio of two posterior probabilities, both of which are dependent on baseline prevalence of an outcome. In addition, we demonstrate that the RR shifts toward its null value with increasing outcome prevalence. The shift toward the null happens regardless of the strength of the association between intervention and outcome. The odds ratio (OR), the other commonly used ratio, measures solely the effect magnitude and has no relationship to the prevalence of an outcome in a study nor does it overestimate the RR as is commonly thought. CONCLUSIONS: The results demonstrate the need to (1) end the primary use of the RR in clinical trials and meta-analyses as its direct interpretation is not meaningful, (2) replace the RR by the OR, and (3) only use the postintervention risk recalculated from the OR for any expected level of baseline risk in absolute terms for purposes of interpretation such as the number needed to treat. These results will have far-reaching implications such as reducing misleading results from clinical trials and meta-analyses and ushering in a new era in the reporting of such trials or meta-analyses in practice.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Consistency of an effect measure in meta-analysis does not mean that it is the effect measure of choice, because if the relative risk does not measure effect magnitude and simply reflects the prevalence of the outcome in the study, consistency is no longer meaningful.
““It is a common perception that consistency of an effect measure in meta-analysis means that it is the effect measure of choice. This again is a misconception because if the RR does not measure effect magnitude and simply reflects the prevalence of the outcome in the study, consistency is no longer meaningful” [15].”
From Relative risk reduction: Misinformative measure in clinical trials and COVID-19 vaccine efficacy (2022) · cites this paper as [15]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
The citation lists haven’t been fetched yet
References 35 · Cited by 150
Research works printed but not linked: 18
- Walter SD. Choice of effect measure for epidemiological data. J Clin Epidemiol 2000; 53:931-9.
- Cornfield J. A method of estimating comparative rates from clinical data; applications to cancer of the lung, breast, and cervix. J Natl Cancer Inst 1951; 11:1269-75.
- Greenland S. Cornfield, risk relativism, and research synthesis. Stat Med 2012; 31:2773-7.
- Persoskie A, Ferrer RA. A Most Odd Ratio:: Interpreting and Describing Odds Ratios. Am J Prev Med 2017; 52:224-228.
- Balasubramanian H, Ananthan A, Rao S, Patole S. Odds ratio vs risk ratio in randomized controlled trials. Postgrad Med 2015; 127:359-67.
- Tajeu GS, Sen B, Allison DB, Menachemi N. Misuse of odds ratios in obesity literature: an empirical analysis of published studies. Obesity (Silver Spring) 2012; 20:1726-31.
- Lee J. Odds ratio or relative risk for cross-sectional data? Int J Epidemiol 1994; 23:201-3.
- Bracken MB, Sinclair JC. When can odds ratios mislead? Avoidable systematic error in estimating treatment effects must not be tolerated. BMJ 1998; 317:1156; author reply 1156-7
- Sinclair JC, Bracken MB. Clinically useful measures of effect in binary analyses of randomized trials. J Clin Epidemiol 1994; 47:881-9.
- Eckermann S, Coory M, Willan AR. Indirect comparison: relative risk fallacies and odds solution. J Clin Epidemiol 2009; 62:1031-6.
- Sonis J. Odds Ratios vs Risk Ratios. JAMA 2018; 320:2041
- Furuya-Kanamori L, Doi SA. The outcome with higher baseline risk should be selected for relative risk in clinical studies: a proposal for change to practice. J Clin Epidemiol 2014; 67:364-7.
- Grimes DA, Schulz KF. Refining clinical diagnosis with likelihood ratios. Lancet 2005; 365:1500-5.
- Zhang J, Yu KF. What's the relative risk? A method of correcting the odds ratio in cohort studies of common outcomes. JAMA 1998; 280:1690-1.
- McNutt LA, Wu C, Xue X, Hafner JP. Estimating the relative risk in cohort studies and clinical trials of common outcomes. Am J Epidemiol 2003; 157:940-3.
- Murphy JR. The relationship of relative risk and positive predictive value in 2 X 2 tables. Am J Epidemiol 1983; 117:86-9.
- Hoppe FM, Hoppe DJ, Walter SD. Odds ratios deconstructed: A new way to understand and explain odds ratios as conditional risk ratios. J Clin Epidemiol 2017; 82:87-93.
- Furuya-Kanamori L, Xu C, Lin L, Doan T, Chu H, Thalib L, et al. P value-driven methods were underpowered to detect publication bias: analysis of Cochrane review meta-analyses. J Clin Epidemiol 2019; 118:86-92.
- Authors
- Doi, S. A., Furuya-Kanamori, L., Xu, C., Lin, L., Chivese, T., & Thalib, L.
- Affiliations
- Qatar University (QA)
- Sichuan University (CN)
- Florida State University (US)
- Corresponding author
- Doi, Suhail A.
- Venue
- Journal of Clinical Epidemiology
- Publisher
- Elsevier BV
- Published
- 2022-02
- Volume
- 142
- Pages
- 271-279
- ISSN
- 0895-4356 (ISSN-L)
- OpenAlex
- W3096389326
- Cited by
- 150
- References
- 35
- Citations by year
- 2026: 8 · 2025: 19 · 2024: 43 · 2023: 20 · 2022: 32 · 2021: 27 · 2020: 1
- FWCI
- 12.20
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Journal of Clinical Epidemiology (publishedVersion, cc-by)
- PubMed (publishedVersion)
- ANU Open Research (Australian National University) (submittedVersion, cc-by)
- Figshare (submittedVersion, cc-by)
- Qatar University QSpace (Qatar University) (submittedVersion, cc-by)
- Griffith Research Online (submittedVersion, cc-by)
- Funders
- Qatar National Research Fund (NPRP10-0129-170274)
- National Health and Medical Research Council (APP1158469)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Keywords
- relative risk, odds ratio, baseline prevalence, number needed to treat, clinical trial reporting, meta-analysis reporting, posterior probability, rare outcomes
- MeSH terms
- Humans, Risk
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
