Some Methods of Propensity‐Score Matching had Superior Performance to Others: Results of an Empirical Investigation and Monte Carlo simulations
The DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.
Abstract
Propensity‐score matching is increasingly being used to reduce the impact of treatment‐selection bias when estimating causal treatment effects using observational data. Several propensity‐score matching methods are currently employed in the medical literature: matching on the logit of the propensity score using calipers of width either 0.2 or 0.6 of the standard deviation of the logit of the propensity score; matching on the propensity score using calipers of 0.005, 0.01, 0.02, 0.03, and 0.1; and 5 → 1 digit matching on the propensity score. We conducted empirical investigations and Monte Carlo simulations to investigate the relative performance of these competing methods. Using a large sample of patients hospitalized with a heart attack and with exposure being receipt of a statin prescription at hospital discharge, we found that the 8 different methods produced propensity‐score matched samples in which qualitatively equivalent balance in measured baseline variables was achieved between treated and untreated subjects. Seven of the 8 propensity‐score matched samples resulted in qualitatively similar estimates of the reduction in mortality due to statin exposure. 5 → 1 digit matching resulted in a qualitatively different estimate of relative risk reduction compared to the other 7 methods. Using Monte Carlo simulations, we found that matching using calipers of width of 0.2 of the standard deviation of the logit of the propensity score and the use of calipers of width 0.02 and 0.03 tended to have superior performance for estimating treatment effects (© 2009 WILEY‐VCH Verlag GmbH & Co. KGaA, Weinheim)
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Also, calliper matchings with widths of 0.2, 0.02 and 0.03 have been reported to have superior results [49], which prompted us to use this method.”
Citations
Cites
R: A Language and Environment for Statistical Computing
R Core Team (2025) · doi:10.32614/r.manuals · Cited by 352,181
The central role of the propensity score in observational studies for causal effects
ROSENBAUM, P. R., & RUBIN, D. B. (1983) · Biometrika · doi:10.1093/biomet/70.1.41 · Cited by 31,241
Rosenbaum, P. R., & Rubin, D. B. (1985) · The American Statistician · doi:10.2307/2683903 · Cited by 4,774
Reducing Bias in Observational Studies Using Subclassification on the Propensity Score
Rosenbaum, P. R., & Rubin, D. B. (1984) · Journal of the American Statistical Association · doi:10.1080/01621459.1984.10478078 · Cited by 3,082
A critical appraisal of propensity‐score matching in the medical literature between 1996 and 2003
Austin, P. C. (2008) · Statistics in Medicine · doi:10.1002/sim.3150 · Cited by 1,249
Self-citationAustin, P. C., Grootendorst, P., & Anderson, G. M. (2007) · Statistics in Medicine · doi:10.1002/sim.2580 · Cited by 1,137
Self-citationThe performance of different propensity score methods for estimating marginal hazard ratios
Austin, P. C. (2013) · Statistics in Medicine · doi:10.1002/sim.5705 · Cited by 1,006
Self-citationControlling Bias in Observational Studies: A Review
Cochran, W. G. (2006) · Matched Sampling for Causal Effects · doi:10.1017/cbo9780511810725.005 · Cited by 958
Reducing bias in a propensity score matched-pair sample using greedy matching techniques
Parsons, L. (2001) · Medical Entomology and Zoology · Cited by 733
Austin, P. C. (2007) · The Journal of Thoracic and Cardiovascular Surgery · doi:10.1016/j.jtcvs.2007.07.021 · Cited by 589
Self-citationAustin, P. C., & Mamdani, M. M. (2006) · Statistics in Medicine · doi:10.1002/sim.2328 · Cited by 560
Self-citationRosenbaum, P. R. (1995) · Springer Series in Statistics · doi:10.1007/978-1-4757-2443-1 · Cited by 351
Austin, P. C., & Tu, J. V. (2004) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2004.04.003 · Cited by 341
Self-citationVikram, H. R., Buenconsejo, J., Hasbun, R., & Quagliarello, V. J. (2003) · JAMA · doi:10.1001/jama.290.24.3207 · Cited by 335
Christakis, N. A., & Iwashyna, T. J. (2003) · Social Science & Medicine · doi:10.1016/s0277-9536(02)00370-2 · Cited by 314
Standard Distance in Univariate and Multivariate Analysis
Flury, B. K., & Riedwyl, H. (1986) · The American Statistician · doi:10.1080/00031305.1986.10475403 · Cited by 307
The performance of different propensity score methods for estimating marginal odds ratios
Austin, P. C. (2007) · Statistics in Medicine · doi:10.1002/sim.2781 · Cited by 249
Self-citationAustin, P. C., Grootendorst, P., Normand, S. T., & Anderson, G. M. (2007) · Statistics in Medicine · doi:10.1002/sim.2618 · Cited by 230
Self-citationAustin, P. C., Mamdani, M. M., Stukel, T. A., Anderson, G. M., & Tu, J. V. (2005) · Statistics in Medicine · doi:10.1002/sim.2053 · Cited by 222
Self-citationDoes off-pump coronary surgery reduce morbidity and mortality?
Sabik, J. F., Gillinov, A. M., Blackstone, E. H., Vacha, C., Houghtaling, P. L., Navia, J., Smedira, N. G., McCarthy, P. M., Cosgrove, D. M., & Lytle, B. W. (2002) · The Journal of Thoracic and Cardiovascular Surgery · doi:10.1067/mtc.2002.121975 · Cited by 222
The performance of different propensity-score methods for estimating relative risks
Austin, P. C. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.07.011 · Cited by 213
Self-citationAyanian, J. Z., Landrum, M. B., Guadagnoli, E., & Gaccione, P. (2002) · New England Journal of Medicine · doi:10.1056/nejmsa020080 · Cited by 205
Iwashyna, T. J., & Lamont, E. B. (2002) · Journal of Clinical Oncology · doi:10.1200/jco.2002.03.083 · Cited by 157
Austin, P. C. (2007) · Statistics in Medicine · doi:10.1002/sim.2770 · Cited by 156
Self-citationOutcome of Mitral Valve Repair or Replacement: A Comparison by Propensity Score Analysis
Moss, R., Humphries, K. H., Gao, M., Thompson, C., Abel, J. G., Fradet, G. J., & Munt, B. I. (2003) · Circulation · doi:10.1161/01.cir.0000089182.44963.bb · Cited by 109
Patient Selection and Current Practice Strategy for Off-pump Coronary Artery Bypass Surgery
Magee, M. J., Coombs, L. P., Peterson, E. D., & Mack, M. J. (2003) · Circulation · doi:10.1161/01.cir.0000089187.51855.77 · Cited by 97
Yu, D., Platt, R. G., Lanken, P. N., Black, E. R., Sands, K., Schwartz, J. S., Hibberd, P. L., GRAMAN, P. S., Kahn, K. L., Snydman, D. R., et al. (2003) · Critical Care Medicine · doi:10.1097/01.ccm.0000098028.68323.64 · Cited by 93
Agresti, A., & Min, Y. (2004) · Statistics in Medicine · doi:10.1002/sim.1589 · Cited by 89
Aronow, H. D., Novaro, G. M., Lauer, M. S., Brennan, D. M., Lincoff, A. M., Topol, E. J., Kereiakes, D. J., & Nissen, S. E. (2003) · Archives of Internal Medicine · doi:10.1001/archinte.163.21.2576 · Cited by 84
Standard Distance in Univariate and Multivariate Analysis
Flury, B. K., & Riedwyl, H. (1986) · The American Statistician · doi:10.2307/2684560 · Cited by 77
Seeger, J. D., Walker, A. M., Williams, P. L., Saperia, G. M., & Sacks, F. M. (2003) · The American Journal of Cardiology · doi:10.1016/j.amjcard.2003.08.057 · Cited by 70
Weiss, J. P., Saynina, O., McDonald, K. M., McClellan, M., & Hlatky, M. A. (2002) · The American Journal of Medicine · doi:10.1016/s0002-9343(02)01078-1 · Cited by 58
Ferguson, T. B., Coombs, L. P., & Peterson, E. D. (2002) · Journal of Thoracic and Cardiovascular Surgery · doi:10.1067/mtc.2002.121679 · Cited by 55
Austin, P. C., Mamdani, M., Juurlink, D. N., Alter, D. A., & Tu, J. V. (2006) · American Heart Journal · doi:10.1016/j.ahj.2005.06.034 · Cited by 55
Self-citationMurray, P. K., Singer, M. E., Dawson, N. V., Thomas, C. L., & Cebul, R. D. (2003) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/s0003-9993(03)00149-7 · Cited by 53
Effect of preoperative aspirin use in off-pump coronary artery bypass operations
Srinivasan, A. K., Grayson, A. D., Pullan, D. M., Fabri, B. M., & Dihmis, W. C. (2003) · The Annals of Thoracic Surgery · doi:10.1016/s0003-4975(03)00182-6 · Cited by 51
Austin, P. C., & Tu, J. V. (2005) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/j.1467-985x.2005.00380.x · Cited by 29
Self-citationElad, Y., French, W. J., Shavelle, D. M., Parsons, L. S., Sada, M. J., & Every, N. R. (2002) · Journal of the American College of Cardiology · doi:10.1016/s0735-1097(01)01817-4 · Cited by 29
The effect of zanamivir treatment on influenza complications: A retrospective cohort study
Cole, J. A., Loughlin, J. E., Ajene, A. N., Rosenberg, D., Cook, S. F., & Walker, A. M. (2002) · Clinical Therapeutics · doi:10.1016/s0149-2918(02)80082-0 · Cited by 25
Hall, J., Summers, K. A., & Obenchain, R. L. (2003) · Journal of Managed Care Pharmacy · doi:10.18553/jmcp.2003.9.3.263 · Cited by 22
Austin, P. C. (2007) · Statistics in Medicine · Cited by 14
Self-citationComparing Clinical Data with Administrative Data for Producing AMI Report Cards
Austin, P. C. (2006) · Journal of the Royal Statistical Society Series A (Statistics in Society) · Cited by 8
Self-citation
Cited by
Austin, P. C. (2009) · Statistics in Medicine · doi:10.1002/sim.3697 · Cited by 6,758
“2 of the standard deviation of the logit of the propensity score [3, 17, 18].”
BackgroundSelf-citationAustin, P. C. (2011) · Pharmaceutical Statistics · doi:10.1002/pst.433 · Cited by 3,949
“10) [27].”
Self-citationWhy Propensity Scores Should Not Be Used for Matching
King, G., & Nielsen, R. (2019) · Political Analysis · doi:10.1017/pan.2019.11 · Cited by 1,706
“Then, following the most common implementation of either PSM or MDM (see Austin 2009, p.173 and the content analysis below) we (greedily) match the first treated unit to the closest control; then, we match the second treated unit with the closest control unit that has not already been matched, and…”
MethodA comparison of 12 algorithms for matching on the propensity score
Austin, P. C. (2014) · Statistics in Medicine · doi:10.1002/sim.6004 · Cited by 1,085
“Although prior studies have compared the relative performance of different caliper widths when using nearest neighbor matching within specified caliper widths [10, 11], there is a paucity of research comparing different matching algorithms.”
ResultSelf-citationThe performance of different propensity score methods for estimating marginal hazard ratios
Austin, P. C. (2013) · Statistics in Medicine · doi:10.1002/sim.5705 · Cited by 1,006
“This approach was included as it has been found to perform well compared with other commonly used alternatives [34].”
MethodSelf-citation
Metadata
- Authors
- Austin, P. C.
- Affiliations
- University of Toronto (CA)
- Corresponding author
- Austin, Peter C.
- ORCID
- Austin, Peter C.
- Venue
- Biometrical Journal
- Publisher
- Wiley
- Published
- 2009-02
- Volume
- 51
- Issue
- 1
- Pages
- 171-184
- ISSN
- 0323-3847 (ISSN-L)
- 1521-4036
- OpenAlex
- W2016403661
- Cited by
- 769
- References
- 42
- Citations by year
- 2026: 29 · 2025: 50 · 2024: 54 · 2023: 61 · 2022: 78 · 2021: 72 · 2020: 62 · 2019: 48 · 2018: 53 · 2017: 42 · 2016: 48 · 2015: 34 · 2014: 40 · 2013: 33 · 2012: 35
- FWCI
- 18.03
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Biometrical Journal (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Keywords
- propensity score matching, treatment selection bias, observational data, statin exposure, myocardial infarction, mortality reduction, method comparison
- MeSH terms
- Algorithms, Biometry, Canada, Data Interpretation, Statistical, Humans, Monte Carlo Method, Myocardial Infarction, Bias, Confounding Factors, Epidemiologic, Incidence, Survival Rate, Proportional Hazards Models, Survival Analysis, Hydroxymethylglutaryl-CoA Reductase Inhibitors, Prescriptions
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.