Pitfalls of using the risk ratio in meta‐analysis
Open accessThe DOI matches its Crossref record; 7 of the 7 references it cites were found in a registry or an index.
Abstract
For meta‐analysis of studies that report outcomes as binomial proportions, the most popular measure of effect is the odds ratio (OR), usually analyzed as log(OR). Many meta‐analyses use the risk ratio (RR) and its logarithm because of its simpler interpretation. Although log(OR) and log(RR) are both unbounded, use of log(RR) must ensure that estimates are compatible with study‐level event rates in the interval (0, 1). These complications pose a particular challenge for random‐effects models, both in applications and in generating data for simulations. As background, we review the conventional random‐effects model and then binomial generalized linear mixed models (GLMMs) with the logit link function, which do not have these complications. We then focus on log‐binomial models and explore implications of using them; theoretical calculations and simulation show evidence of biases. The main competitors to the binomial GLMMs use the beta‐binomial (BB) distribution, either in BB regression or by maximizing a BB likelihood; a simulation produces mixed results. Two examples and an examination of Cochrane meta‐analyses that used RR suggest bias in the results from the conventional inverse‐variance–weighted approach. Finally, we comment on other measures of effect that have range restrictions, including risk difference, and outline further research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
IntHout, J., Ioannidis, J. P., & Borm, G. F. (2014) · BMC Medical Research Methodology · doi:10.1186/1471-2288-14-25 · Cited by 2,419
Bias and Efficiency of Meta-Analytic Variance Estimators in the Random-Effects Model
Viechtbauer, W. (2005) · Journal of Educational and Behavioral Statistics · doi:10.3102/10769986030003261 · Cited by 1,699
Meta‐Analysis Of Controlled Clinical Trials
Whitehead, A. (2002) · doi:10.1002/0470854200 · Cited by 827
Stijnen, T., Hamza, T. H., & Özdemir, P. (2010) · Statistics in Medicine · doi:10.1002/sim.4040 · Cited by 757 · cited 1×
“i=1 ( nji Yji ) Beta(πi(ρ − 1) + Yji, (1 − πi)(ρ − 1) + nji − Yji) Beta(πi(ρ−1 − 1), (1 − πi)(ρ−1 − 1)) , (16)”
A re‐evaluation of fixed effect(s) meta‐analysis
Rice, K., Higgins, J. P. T., & Lumley, T. (2018) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/rssa.12275 · Cited by 287
Warn, D. E., Thompson, S. G., & Spiegelhalter, D. J. (2002) · Statistics in Medicine · doi:10.1002/sim.1189 · Cited by 277
Kuss, O. (2015) · Statistics in Medicine · doi:10.1002/sim.6383 · Cited by 206
Cited by
Doi, S. A., Furuya-Kanamori, L., Xu, C., Lin, L., Chivese, T., & Thalib, L. (2022) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.08.019 · Cited by 150
Furukawa, T. A., Shinohara, K., Sahker, E., Karyotaki, E., Miguel, C., Ciharova, M., Bockting, C. L. H., Breedvelt, J. J. F., Tajika, A., Imai, H., et al. (2021) · World Psychiatry · doi:10.1002/wps.20906 · Cited by 126
Michopoulos, I., Furukawa, T. A., Noma, H., Kishimoto, S., Onishi, A., Ostinelli, E. G., Ciharova, M., Miguel, C., Karyotaki, E., & Cuijpers, P. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.12.012 · Cited by 69
Furukawa, Y., Oguro, S., Obata, S., Hamza, T., Ostinelli, E. G., & Kasai, K. (2022) · Psychiatry and Clinical Neurosciences · doi:10.1111/pcn.13438 · Cited by 14
Metadata
- Authors
- Bakbergenuly, I., Hoaglin, D. C., & Kulinskaya, E.
- Venue
- Research Synthesis Methods
- Publisher
- Wiley
- Published
- 2019-04-11
- Volume
- 10
- Issue
- 3
- Pages
- 398-419
- ISSN
- 1759-2879 (ISSN-L)
- 1759-2887
- OpenAlex
- W2920850085
- Cited by
- 84
- References
- 7
- FWCI
- 6.51
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Pitfalls of using the risk ratio in meta‐analysis. https://app.cochat.ai/papers/10.1002/jrsm.1347
- The paper
- Bakbergenuly, I., Hoaglin, D. C., & Kulinskaya, E. (2019). Pitfalls of using the risk ratio in meta‐analysis. Research Synthesis Methods. https://doi.org/10.1002/jrsm.1347
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.