Selecting an Appropriate Caliper Can Be Essential for Achieving Good Balance With Propensity Score Matching
Open accessThe DOI matches its Crossref record; 23 of the 25 references it cites were found in a registry or an index.
Abstract
Matching on the propensity score is widely used to estimate the effect of an exposure in observational studies. However, the quality of the matches can be affected by decisions made during the matching process, particularly the order in which subjects are selected for matching and the maximum permitted difference between matched subjects (the "caliper"). This study used simulations to explore the effects of these decisions on both the imbalance of covariates and the closeness of matching, while allowing the numbers of potential matches and strengths of association between the confounding variable and the exposure to vary. It was found that, without a caliper, substantial bias was possible, particularly with a relatively small reservoir of potential matches and strong confounder-exposure association. Use of the recommended caliper reduced the bias considerably, but bias remained if subjects were selected by increasing or decreasing propensity score. A tighter caliper led to greatly reduced bias and closer matches, although some subjects could not be matched. This study suggests that a narrow caliper can improve the performance of propensity score matching. In situations where it is impossible to find appropriate matches for all exposed subjects, it is better to select subjects in order of the best available matches, rather than increasing or decreasing the propensity score.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures




Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Most of the literature suggests that the calliper method outperforms other methods such as nearest neighbour with and without replacement and optimal matching [55] and gives better results if it is difficult to find the matches from the control group [56].”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The central role of the propensity score in observational studies for causal effects
ROSENBAUM, P. R., & RUBIN, D. B. (1983) · Biometrika · doi:10.1093/biomet/70.1.41 · Cited by 31,241 · cited 4×
“It can be difficult to find appropriate matches when trying to match on several variables, but Rosenbaum and Rubin (7) showed that matching on the propensity score (the conditional probability of exposure given a set of covariates) could produce samples with the same distribution of covariates in exposed and unexposed…”
BackgroundInfluentialSOME PRACTICAL GUIDANCE FOR THE IMPLEMENTATION OF PROPENSITY SCORE MATCHING
Caliendo, M., & Kopeinig, S. (2008) · Journal of Economic Surveys · doi:10.1111/j.1467-6419.2007.00527.x · Cited by 6,238 · cited 1×
“2 standard deviations), more recent papers on the practicalities of matching have not given recommendations for setting a caliper (12, 13).”
BackgroundMatching Methods for Causal Inference: A Review and a Look Forward
Stuart, E. A. (2010) · Statistical Science · doi:10.1214/09-sts313 · Cited by 5,478 · cited 2×
“2 standard deviations), more recent papers on the practicalities of matching have not given recommendations for setting a caliper (12, 13).”
BackgroundPropensity Score-Matching Methods for Nonexperimental Causal Studies
Dehejia, R. H., & Wahba, S. (2002) · Review of Economics and Statistics · doi:10.1162/003465302317331982 · Cited by 4,846
Rosenbaum, P. R., & Rubin, D. B. (1985) · The American Statistician · doi:10.2307/2683903 · Cited by 4,774 · cited 6×
“It can be difficult to find appropriate matches when trying to match on several variables, but Rosenbaum and Rubin (7) showed that matching on the propensity score (the conditional probability of exposure given a set of covariates) could produce samples with the same distribution of covariates in exposed and unexposed…”
BackgroundMethodInfluentialAustin, P. C. (2011) · Pharmaceutical Statistics · doi:10.1002/pst.433 · Cited by 3,949 · cited 5×
“In practice, a wide variety of calipers is used (1) and, with the exception of Austin (11) (who recommended reducing the caliper from 0.25 standard deviations to 0.2 standard deviations), more recent papers on the practicalities of matching have not given recommendations for setting a caliper (12,…”
BackgroundMethodInfluentialA critical appraisal of propensity‐score matching in the medical literature between 1996 and 2003
Austin, P. C. (2008) · Statistics in Medicine · doi:10.1002/sim.3150 · Cited by 1,249
Matching Using Estimated Propensity Scores: Relating Theory to Practice
Rubin, D. B., & Thomas, N. (1996) · Biometrics · doi:10.2307/2533160 · Cited by 1,074
Matching Using Estimated Propensity Scores: Relating Theory to Practice
Thomas, N. (2006) · Matched Sampling for Causal Effects · doi:10.1017/cbo9780511810725.024 · Cited by 1,018 · cited 1×
“In order to be able to find suitable matches for all exposed subjects, the number of controls available needs to be greater than the number of exposed subjects; the ratio typically lies in the range of 2–20, although it may be higher (8).”
BackgroundControlling Bias in Observational Studies: A Review
Cochran, W. G. (2006) · Matched Sampling for Causal Effects · doi:10.1017/cbo9780511810725.005 · Cited by 958 · cited 5×
“It can be difficult to find appropriate matches when trying to match on several variables, but Rosenbaum and Rubin (7) showed that matching on the propensity score (the conditional probability of exposure given a set of covariates) could produce samples with the same distribution of covariates in exposed and unexposed…”
MethodInfluential
Cited by
Brown, J. W. L., Coles, A., Horakova, D., Havrdova, E., Izquierdo, G., Prat, A., Girard, M., Duquette, P., Trojano, M., Lugaresi, A., et al. (2019) · JAMA · doi:10.1001/jama.2018.20588 · Cited by 546
Yao, X. I., Wang, X., Speicher, P. J., Hwang, E. S., Cheng, P., Harpole, D. H., Berry, M. F., Schrag, D., & Pang, H. H. (2017) · JNCI: Journal of the National Cancer Institute · doi:10.1093/jnci/djw323 · Cited by 386
Bramante, C. T., Ingraham, N. E., Murray, T. A., Marmor, S., Hovertsen, S., Gronski, J., McNeil, C., Feng, R., Guzman, G., Abdelwahab, N., et al. (2021) · The Lancet Healthy Longevity · doi:10.1016/s2666-7568(20)30033-7 · Cited by 291
Ali, M. S., Groenwold, R. H. H., Belitser, S. V., Pestman, W. R., Hoes, A. W., Roes, K. C. B., Boer, A. D., & Klungel, O. H. (2015) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2014.08.011 · Cited by 222
“It is important to note that exclusion of unmatched patients from the analysis not only affect the precision of the effect estimate but also have consequences on the generalizability of results [5,6].”
BackgroundInfluentialStatistical matching for conservation science
Schleicher, J., Eklund, J., D. Barnes, M., Geldmann, J., Oldekop, J. A., & Jones, J. P. G. (2020) · Conservation Biology · doi:10.1111/cobi.13448 · Cited by 229
Metadata
- Authors
- Lunt, M.
- Affiliations
- University of Manchester (GB)
- Corresponding author
- Lunt, Mark
- ORCID
- Lunt, Mark
- Venue
- American Journal of Epidemiology
- Publisher
- Oxford University Press (OUP)
- Published
- 2013-10-10
- Volume
- 179
- Issue
- 2
- Pages
- 226-235
- ISSN
- 0002-9262 (ISSN-L)
- 1476-6256
- PMC
- PMC3873103
- OpenAlex
- W2154872949
- Cited by
- 358
- References
- 25
- Citations by year
- 2026: 30 · 2025: 50 · 2024: 21 · 2023: 35 · 2022: 31 · 2021: 39 · 2020: 36 · 2019: 39 · 2018: 24 · 2017: 16 · 2016: 15 · 2015: 16 · 2014: 6
- FWCI
- 10.88
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by-nc
- Locations
- American Journal of Epidemiology (publishedVersion, cc-by-nc)
- Europe PMC (PubMed Central) (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- Research Explorer (The University of Manchester) (publishedVersion)
- Funders
- Versus Arthritis (/17552)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- propensity score matching, covariate imbalance, matching bias, confounding, observational studies
- MeSH terms
- Epidemiologic Methods, Humans, Bias, Propensity Score
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Selecting an Appropriate Caliper Can Be Essential for Achieving Good Balance With Propensity Score Matching. https://app.cochat.ai/papers/10.1093/aje/kwt212
- The paper
- Lunt, M. (2014). Selecting an Appropriate Caliper Can Be Essential for Achieving Good Balance With Propensity Score Matching. American Journal of Epidemiology. https://doi.org/10.1093/aje/kwt212
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.