Squeezing the Balloon: Propensity Scores and Unmeasured Covariate Balance
Open accessThe DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index.
Abstract
Objective: To assess the covariate balancing properties of propensity score‐based algorithms in which covariates affecting treatment choice are both measured and unmeasured. Data Sources/Study Setting: A simulation model of treatment choice and outcome. Study Design: Simulation. Data Collection/Extraction Methods: Eight simulation scenarios varied with the values placed on measured and unmeasured covariates and the strength of the relationships between the measured and unmeasured covariates. The balance of both measured and unmeasured covariates was compared across patients either grouped or reweighted by propensity scores methods. Principal Findings: Propensity score algorithms require unmeasured covariate variation that is unrelated to measured covariates, and they exacerbate the imbalance in this variation between treated and untreated patients relative to the full unweighted sample. Conclusions: The balance of measured covariates between treated and untreated patients has opposite implications for unmeasured covariates in randomized and observational studies. Measured covariate balance between treated and untreated patients in randomized studies reinforces the notion that all covariates are balanced. In contrast, forced balance of measured covariates using propensity score methods in observational studies exacerbates the imbalance in the independent portion of the variation in the unmeasured covariates, which can be likened to squeezing a balloon. If the unmeasured covariates affecting treatment choice are confounders, propensity score methods can exacerbate the bias in treatment effect estimates.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Mostly Harmless Econometrics: An Empiricist's Companion
Angrist, J. D., & Pischke, J.-S. (2009) · doi:10.1515/9781400829828 · Cited by 9,467
Marginal Structural Models and Causal Inference in Epidemiology
Robins, J. M., Hernán, M. Á., & Brumback, B. (2000) · Epidemiology · doi:10.1097/00001648-200009000-00011 · Cited by 5,581
Matching Methods for Causal Inference: A Review and a Look Forward
Stuart, E. A. (2010) · Statistical Science · doi:10.1214/09-sts313 · Cited by 5,478 · cited 3×
“These algorithms range from stratification (D’Agostino 1998) and matching based on propensity scores (Hall, Summers, and Oberchain 2003; Frolich 2007; Stuart 2010) to using patient-specific propensity scores to weight observations (Rosenbaum 1987; Robins, Hernan, and Brunback 2000).”
BackgroundMethodInfluentialMostly Harmless Econometrics: An Empiricist's Companion
Angrist, J. D., & Pischke, J.-S. (2008) · doi:10.2307/j.ctvcm4j72 · Cited by 5,051
Using Propensity Scores to Help Design Observational Studies: Application to the Tobacco Litigation
Rubin, D. B. (2001) · Health Services and Outcomes Research Methodology · doi:10.1023/a:1020363010465 · Cited by 2,228 · cited 5×
“…the assumption that randomized treatment assignment yields a balanced distribution of covariates thought to be related to outcome between the treatment and control groups ©Health Research and Educational Trust DOI: 10.1111/1475-6773.12020 METHODSARTICLE 1487 Health Services Research (Rubin 2001).”
BackgroundMethodInfluentialSchneeweiss, S., Rassen, J. A., Glynn, R. J., Avorn, J., Mogun, H., & Brookhart, M. A. (2009) · Epidemiology · doi:10.1097/ede.0b013e3181a663cc · Cited by 1,128 · cited 2×
“These results have implications on the use of higher dimensional propensity scores for balancing treated and untreated patients (Rassen, Brookhart et al. 2009; Schneeweiss et al. 2009).”
BackgroundRubin, D. B. (2007) · Statistics in Medicine · doi:10.1002/sim.2739 · Cited by 1,047
Assessing the sensitivity of regression results to unmeasured confounders in observational studies
Lin, D. Y., Psaty, B. M., & Kronmal, R. A. (1998) · Biometrics · doi:10.2307/2533848 · Cited by 742
Shah, B. R., Laupacis, A., Hux, J. E., & Austin, P. C. (2005) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2004.10.016 · Cited by 493
Do Instrumental Variables Belong in Propensity Scores?
Bhattacharya, J., & Vogt, W. (2007) · doi:10.3386/t0343 · Cited by 81
Cited by
Methods for Constructing and Assessing Propensity Scores
Garrido, M. M., Kelley, A. S., Paris, J., Roza, K., Meier, D. E., Morrison, R. S., & Aldridge, M. D. (2014) · Health Services Research · doi:10.1111/1475-6773.12182 · Cited by 882
A flexible, interpretable framework for assessing sensitivity to unmeasured confounding
Dorie, V., Harada, M., Carnegie, N. B., & Hill, J. (2016) · Statistics in Medicine · doi:10.1002/sim.6973 · Cited by 104
Webb, S. J., Garrison, M. M., Bernier, R., McClintic, A. M., King, B. H., & Mourad, P. D. (2017) · Autism Research · doi:10.1002/aur.1690 · Cited by 29
“…covariate adjustment using propensity scores, as described by Austin [2011a,b] and as employed by other researchers [Arabi et al., 2013; Goldin, Sawin, Garrison & Christakis, 2007; Gagliardi, Bellu, Zanini, Dammann, 2009; Lund et al., 2013 but see Brooks & Ohsfeldt, 2013; Weitzen et al., 2004].”
Propensity score weighting: an application to anEarlyHeadStart dental study
Burgette, J. M., Preisser, J. S., & Rozier, R. G. (2016) · Journal of Public Health Dentistry · doi:10.1111/jphd.12106 · Cited by 25
“While propensity score analyses can control for selection into the EHS group using observable characteristics, they cannot be used to control for selection based on characteristics that are unobserved (20,21).”
Method
Metadata
- Authors
- Brooks, J. M., & Ohsfeldt, R. L.
- Venue
- Health Services Research
- Publisher
- Wiley
- Published
- 2012-12-06
- Volume
- 48
- Issue
- 4
- Pages
- 1487-1507
- ISSN
- 0017-9124 (ISSN-L)
- 1475-6773
- OpenAlex
- W2062417462
- Cited by
- 113
- References
- 10
- FWCI
- 8.39
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Squeezing the Balloon: Propensity Scores and Unmeasured Covariate Balance. https://app.cochat.ai/papers/10.1111/1475-6773.12020
- The paper
- Brooks, J. M., & Ohsfeldt, R. L. (2013). Squeezing the Balloon: Propensity Scores and Unmeasured Covariate Balance. Health Services Research. https://doi.org/10.1111/1475-6773.12020
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.