Comparison of logistic regression versus propensity score when the number of events is low and there are multiple confounders
Abstract
The aim of this study was to use Monte Carlo simulations to compare logistic regression with propensity scores in terms of bias, precision, empirical coverage probability, empirical power, and robustness when the number of events is low relative to the number of confounders. The authors simulated a cohort study and performed 252,480 trials. In the logistic regression, the bias decreased as the number of events per confounder increased. In the propensity score, the bias decreased as the strength of the association of the exposure with the outcome increased. Propensity scores produced estimates that were less biased, more robust, and more precise than the logistic regression estimates when there were seven or fewer events per confounder. The logistic regression empirical coverage probability increased as the number of events per confounder increased. The propensity score empirical coverage probability decreased after eight or more events per confounder. Overall, the propensity score exhibited more empirical power than logistic regression. Propensity scores are a good alternative to control for imbalances when there are seven or fewer events per confounder; however, empirical power could range from 35% to 60%. Logistic regression is the technique of choice when there are at least eight events per confounder.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Risk stratification can control observed confounders more efficiently than regression and may outperform propensity scores when covariates are weakly correlated with exposure.
“These methods constitute a broader strategy of risk stratification, allowing for more efficient control of observed confounders than regression-based approaches [31,32], and are a better alternative to propensity score estimation when covariates are not highly correlated with the exposure [33].”
From Social Media Use and Depressive Symptoms Among United States Adolescents (2021) · cites this paper as [31]
Cites
The central role of the propensity score in observational studies for causal effects
ROSENBAUM, P. R., & RUBIN, D. B. (1983) · Biometrika · doi:10.1093/biomet/70.1.41 · Cited by 31,241 · see citing works
A simulation study of the number of events per variable in logistic regression analysis
Peduzzi, P., Concato, J., Kemper, E., Holford, T. R., & Feinstein, A. R. (1996) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(96)00236-3 · Cited by 9,655 · see citing works
Applied Regression Analysis and Other Multivariable Methods
Walker, E., Kleinbaum, D. G., Kupper, L. L., & Muller, K. E. (1989) · Technometrics · doi:10.2307/1270375 · Cited by 8,538 · see citing works
An Introduction to Categorical Data Analysis
Agresti, A. (2006) · Wiley series in probability and statistics · doi:10.1002/0470114754 · Cited by 6,697 · see citing works
Casella, G., & Berger, R. L. (1993) · Chapman & Hall/CRC eBooks · doi:10.1201/9781003456285 · Cited by 5,835 · see citing works
Estimating Causal Effects from Large Data Sets Using Propensity Scores
Rubin, D. V. (1997) · Annals of Internal Medicine · doi:10.7326/0003-4819-127-8_part_2-199710151-00064 · Cited by 3,104 · see citing works
The Effectiveness of Right Heart Catheterization in the Initial Care of Critically III Patients
Connors, A. F. (1996) · JAMA · doi:10.1001/jama.1996.03540110043030 · Cited by 2,239 · see citing works
Regression modelling strategies for improved prognostic prediction
Harrell, F. E., Lee, K. L., Califf, R. M., Pryor, D. B., & Rosati, R. A. (1984) · Statistics in Medicine · doi:10.1002/sim.4780030207 · Cited by 1,804 · see citing works
Multiple Regression in Practice
Berry, W., & Feldman, S. (1985) · doi:10.4135/9781412985208 · Cited by 1,227 · see citing works
The Risk of Determining Risk with Multivariable Models
Concato, J., Feinstein, A. R., & Holford, T. R. (1993) · Annals of Internal Medicine · doi:10.7326/0003-4819-118-3-199302010-00009 · Cited by 1,119 · see citing works
Invited commentary: propensity scores.
Joffe, M., & Rosenbaum, P. (1999) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a010011 · Cited by 1,117 · see citing works
The Effectiveness of Adjustment by Subclassification in Removing Bias in Observational Studies
Cochran, W. G. (1968) · Biometrics · doi:10.2307/2528036 · Cited by 961 · see citing works
Concato, J., Peduzzi, P. N., Holford, T. R., & Feinstein, A. R. (1995) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(95)00510-2 · Cited by 797 · see citing works
Gail, M. H., Wieand, S., & Piantadosi, S. (1984) · Biometrika · doi:10.1093/biomet/71.3.431 · Cited by 689 · see citing works
Identifiability, Exchangeability, and Epidemiological Confounding
GREENLAND, S., & ROBINS, J. M. (1986) · International Journal of Epidemiology · doi:10.1093/ije/15.3.413 · Cited by 686 · see citing works
INTERPRETATION AND CHOICE OF EFFECT MEASURES IN EPIDEMIOLOGIC ANALYSES1
Greenland, S. (1987) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a114593 · Cited by 546 · see citing works
Estimating Exposure Effects by Modelling the Expectation of Exposure Conditional on Confounders
Robins, J. M., Mark, S. D., & Newey, W. K. (1992) · Biometrics · doi:10.2307/2532304 · Cited by 384 · see citing works
CONFOUNDING: ESSENCE AND DETECTION1
Miettinen, O. S., & Cook, E. F. (1981) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a113225 · Cited by 350 · see citing works
Effects of Misspecification of the Propensity Score on Estimators of Treatment Effect
Drake, C. (1993) · Biometrics · doi:10.2307/2532266 · Cited by 347 · see citing works
Rubin, D. B., & Thomas, N. (1992) · Biometrika · doi:10.1093/biomet/79.4.797 · Cited by 188 · see citing works
Software for optimal matching in observational studies.
Bergstralh, E. J., Kosanke, J. L., & Jacobsen, S. J. (1996) · PubMed · Cited by 145 · see citing works
International monitoring of adverse health effects associated with herbal medicines
Farah, M. H., Edwards, R., Lindquist, M., Leon, C. J., & Shaw, D. (2000) · Pharmacoepidemiology and Drug Safety · doi:10.1002/(sici)1099-1557(200003/04)9:2 · Cited by 114 · see citing works
Cook, E. F., & Goldman, L. (1989) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(89)90036-x · Cited by 110 · see citing works
Prognostic Models and the Propensity Score
Drake, C., & Fisher, L. D. (1995) · International Journal of Epidemiology · doi:10.1093/ije/24.1.183 · Cited by 110 · see citing works
Sample Size Formulas for Normal TheoryTTests
Guenther, W. C. (1981) · The American Statistician · doi:10.1080/00031305.1981.10479363 · Cited by 49 · see citing works
Cook, E. F., & Goldman, L. (1988) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a114838 · Cited by 49 · see citing works
The Method of Maximum Likelihood
Brandt, S. (2014) · Springer eBooks · doi:10.1007/978-3-319-03762-2_7 · Cited by 11 · see citing works
(2012) · Princeton University Press eBooks · doi:10.1515/9781400843046.175 · Cited by 5 · see citing works
Using Monte Carlo Simulation in the Social Sciences
Mooney, C. Z. (1997) · Sage eBooks · doi:10.4135/9781412985116.n4 · Cited by 2 · see citing works
The Multiple Regression Model: A Review
Berry, W. D., & Feldman, S. (1985) · Sage eBooks · doi:10.4135/9781412985208.n1 · Cited by 1 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Data-driven prediction of drug effects and interactions
Tatonetti, N. P., Ye, P. P., Daneshjou, R., & Altman, R. B. (2012) · Science Translational Medicine · doi:10.1126/scitranslmed.3003377 · Cited by 936 · see citing works
Indications for propensity scores and review of their use in pharmacoepidemiology
Glynn, R. J., Schneeweiss, S., & Stürmer, T. (2006) · Basic & Clinical Pharmacology & Toxicology · doi:10.1111/j.1742-7843.2006.pto_293.x · Cited by 541 · see citing works
Shah, B. R., Laupacis, A., Hux, J. E., & Austin, P. C. (2005) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2004.10.016 · Cited by 493 · see citing works
An application of propensity score matching using claims data
Seeger, J. D., Williams, P. L., & Walker, A. M. (2005) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.1062 · Cited by 218 · see citing works
Dahabreh, I. J., Sheldrick, R. C., Paulus, J. K., Chung, M., Varvarigou, V., Jafri, H., Rassen, J. A., Trikalinos, T. A., & Kitsios, G. D. (2012) · European Heart Journal · doi:10.1093/eurheartj/ehs114 · Cited by 204 · see citing works
Glucagon-Like Peptide-1 Receptor Agonist Use and Residual Gastric Content Before Anesthesia
Sen, S., Potnuru, P. P., Hernandez, N., Goehl, C., Praestholm, C., Sridhar, S., & Nwokolo, O. O. (2024) · JAMA Surgery · doi:10.1001/jamasurg.2024.0111 · Cited by 202 · see citing works
Ali, M. S., Prieto-Alhambra, D., Lopes, L. C., Ramos, D., Bispo, N., Ichihara, M. Y., Pescarini, J. M., Williamson, E., Fiaccone, R. L., Barreto, M. L., et al. (2019) · Frontiers in Pharmacology · doi:10.3389/fphar.2019.00973 · Cited by 200 · see citing works
Fruit and vegetable intake and risk of breast cancer by hormone receptor status
Jung, S., Spiegelman, D., Baglietto, L., Bernstein, L., Boggs, D. A., van den Brandt, P. A., Buring, J. E., Cerhan, J. R., Gaudet, M. M., Giles, G. G., et al. (2013) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/djs635 · Cited by 183 · see citing works
Arbogast, P. G., & Ray, W. A. (2011) · American Journal of Epidemiology · doi:10.1093/aje/kwr143 · Cited by 138 · see citing works
Food insufficiency and health services utilization in a national sample of homeless adults
Baggett, T. P., Singer, D. E., Rao, S. R., O’Connell, J. J., Bharel, M., & Rigotti, N. A. (2011) · Journal of General Internal Medicine · doi:10.1007/s11606-011-1638-4 · Cited by 105 · see citing works
Social Media Use and Depressive Symptoms Among United States Adolescents
Kreski, N., Platt, J., Rutherford, C., Olfson, M., Odgers, C., Schulenberg, J., & Keyes, K. M. (2021) · Journal of Adolescent Health · doi:10.1016/j.jadohealth.2020.07.006 · Cited by 97 · see citing works
Rubin, D. B. (2010) · American Journal of Ophthalmology · doi:10.1016/j.ajo.2009.08.024 · Cited by 88 · see citing works
Cepeda, M. S., Berlin, J. A., Gao, C. Y., Wiegand, F., & Wada, D. R. (2012) · Pain Medicine · doi:10.1111/j.1526-4637.2012.01340.x · Cited by 71 · see citing works
Self-citationWebb, S. J., Garrison, M. M., Bernier, R., McClintic, A. M., King, B. H., & Mourad, P. D. (2017) · Autism Research · doi:10.1002/aur.1690 · Cited by 29 · see citing works
Melgarejo, J. D., Maestre, G. E., Mena, L. J., Lee, J. H., Petitto, M., Chávez, C. A., Calmon, G., Silva, E., Thijs, L., Al-Aswad, L. A., et al. (2021) · Hypertension Research · doi:10.1038/s41440-021-00687-1 · Cited by 11 · see citing works
- Authors
- Cepeda, M. S.
- Affiliations
- Pontificia Universidad Javeriana (CO)
- Corresponding author
- Cepeda, M. S.
- Venue
- American Journal of Epidemiology
- Publisher
- Oxford University Press (OUP)
- Published
- 2003-08-01
- Volume
- 158
- Issue
- 3
- Pages
- 280-287
- ISSN
- 0002-9262 (ISSN-L)
- OpenAlex
- W2140074098
- Cited by
- 859
- References
- 30
- Citations by year
- 2026: 15 · 2025: 27 · 2024: 58 · 2023: 42 · 2022: 58 · 2021: 68 · 2020: 68 · 2019: 51 · 2018: 41 · 2017: 50 · 2016: 50 · 2015: 43 · 2014: 37 · 2013: 37 · 2012: 34
- FWCI
- 12.73
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- American Journal of Epidemiology (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Keywords
- propensity score matching, logistic regression, small sample size, confounding, bias, statistical power, robustness, cohort studies
- MeSH terms
- Humans, Research Design, Reproducibility of Results, Models, Statistical, Bias, Confounding Factors, Epidemiologic, Logistic Models, Epidemiologic Studies
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review