New Evidence on the Finite Sample Properties of Propensity Score Reweighting and Matching Estimators
The DOI matches its Crossref record; 51 of the 51 references it cites were found in a registry or an index.
Abstract
Frölich (2004) compares the finite sample properties of reweighting and matching estimators of average treatment effects and concludes that reweighting performs far worse than even the simplest matching estimator. We argue that this conclusion is unjustified. Neither approach dominates the other uniformly across data-generating processes (DGPs). Expanding on Frölich's analysis, this paper analyzes empirical as well as hypothetical DGPs and also examines the effect of misspecification. We conclude that reweighting is competitive with the most effective matching estimators when overlap is good, but that matching may be more effective when overlap is sufficiently poor.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“As illustrated by Busso et al. (2014), this usually leads to improved finite sample properties.”
From Difference-in-Differences with multiple time periods (2021) · cites this paper as Busso et al. (2014)
In the citing paper
“This type of normalized weights is said to be of the Hájek-type (Hájek (1971)) and typically results in estimators with improved finite sample properties relative to its unnormalized counterpart (Busso et al. (2014)).”
From Policy evaluation during a pandemic (2023) · cites this paper as Busso et al. (2014)
Citations
Cites
The central role of the propensity score in observational studies for causal effects
ROSENBAUM, P. R., & RUBIN, D. B. (1983) · Biometrika · doi:10.1093/biomet/70.1.41 · Cited by 31,241
(2001) · RePEc: Research Papers in Economics · Cited by 27,577
Estimating causal effects of treatments in randomized and nonrandomized studies
Rubin, D. B. (1974) · Journal of Educational Psychology · doi:10.1037/h0037350 · Cited by 9,436
Wage discrimination: Reduced form and structural estimates
Blinder, A. S. (1973) · The Journal of Human Resources · doi:10.2307/144855 · Cited by 6,922
A generalization of sampling without replacement from a finite universe
Horvitz, D. G., & Thompson, D. J. (1952) · Journal of the American Statistical Association · doi:10.1080/01621459.1952.10483446 · Cited by 4,293
Matching As An Econometric Evaluation Estimator
Heckman, J. J., Ichimura, H., & Todd, P. (1998) · Review of Economic Studies · doi:10.1111/1467-937x.00044 · Cited by 3,807
Does matching overcome lalonde’s critique of nonexperimental estimators?
A. Smith, J., & E. Todd, P. (2005) · Journal of Econometrics · doi:10.1016/j.jeconom.2004.04.011 · Cited by 3,176
A Generalization of Sampling Without Replacement From a Finite Universe
Horvitz, D. G., & Thompson, D. J. (1952) · Journal of the American Statistical Association · doi:10.2307/2280784 · Cited by 3,011
Nonparametric Estimation of Average Treatment Effects Under Exogeneity: A Review
Imbens, G. W. (2004) · Review of Economics and Statistics · doi:10.1162/003465304323023651 · Cited by 2,994
Large sample properties of matching estimators for average treatment effects
Abadie, A., & Imbens, G. W. (2006) · Econometrica · doi:10.1111/j.1468-0262.2006.00655.x · Cited by 2,908
Estimation of Regression Coefficients When Some Regressors are not Always Observed
Robins, J. M., Rotnitzky, A., & Zhao, L. P. (1994) · Journal of the American Statistical Association · doi:10.1080/01621459.1994.10476818 · Cited by 2,823
Causal effects in nonexperimental studies: Reevaluating the evaluation of training programs
Dehejia, R. H., & Wahba, S. (1999) · Journal of the American Statistical Association · doi:10.1080/01621459.1999.10473858 · Cited by 2,395
Efficient Estimation of Average Treatment Effects Using the Estimated Propensity Score
Hirano, K., Imbens, G. W., & Ridder, G. (2003) · Econometrica · doi:10.1111/1468-0262.00442 · Cited by 2,355
Lunceford, J. K., & Davidian, M. (2004) · Statistics in Medicine · doi:10.1002/sim.1903 · Cited by 1,674
Bias-Corrected Matching Estimators for Average Treatment Effects
Abadie, A., & Imbens, G. W. (2011) · Journal of Business & Economic Statistics · doi:10.1198/jbes.2009.07333 · Cited by 1,416
Implementing Matching Estimators for Average Treatment Effects in Stata
Abadie, A., Drukker, D., Herr, J. L., & Imbens, G. W. (2004) · The Stata Journal: Promoting communications on statistics and Stata · doi:10.1177/1536867x0400400307 · Cited by 1,386
Inverse probability weighted estimation for general missing data problems
Wooldridge, J. M. (2007) · Journal of Econometrics · doi:10.1016/j.jeconom.2007.02.002 · Cited by 1,029
Rosenbaum, P. R. (1987) · Journal of the American Statistical Association · doi:10.1080/01621459.1987.10478441 · Cited by 997
Causal Effects in Non-Experimental Studies: Re-Evaluating the Evaluation of Training Programs
Dehejia, R., & Wahba, S. (1998) · National Bureau of Economic Research · doi:10.3386/w6586 · Cited by 805
Kang, J. D. Y., & Schafer, J. L. (2007) · Statistical Science · doi:10.1214/07-sts227 · Cited by 797
How robust is the evidence on the effects of college quality? Evidence from matching
Black, D. A., & Smith, J. A. (2004) · Journal of Econometrics · doi:10.1016/j.jeconom.2003.10.006 · Cited by 673
Finite-Sample Properties of Propensity-Score Matching and Weighting Estimators
Frölich, M. (2004) · Review of Economics and Statistics · doi:10.1162/003465304323023697 · Cited by 399
Kang, J., & Schafer, J. L. (2007) · Statistical Science · Cited by 389
Effects of Misspecification of the Propensity Score on Estimators of Treatment Effect
Drake, C. (1993) · Biometrics · doi:10.2307/2532266 · Cited by 347
Robins, J. M., Sued, M., Lei-Gomez, Q., Rotnitzky, A. G., Ciencias, F., & Naturales, E. (2010) · Cited by 346
Robins, J., Sued, M., Lei-Gomez, Q., & Rotnitzky, A. (2007) · Statistical Science · doi:10.1214/07-sts227d · Cited by 313
The performance of estimators based on the propensity score
Huber, M., Lechner, M., & Wunsch, C. (2013) · Journal of Econometrics · doi:10.1016/j.jeconom.2012.11.006 · Cited by 311
Bounded, efficient and doubly robust estimation with inverse weighting
Tan, Z. (2010) · Biometrika · doi:10.1093/biomet/asq035 · Cited by 305
Irregular Identification, Support Conditions, and Inverse Weight Estimation
(2010) · Econometrica · doi:10.3982/ecta7372 · Cited by 251
Does Collaboration Make Any Difference? Linking Collaborative Governance to Environmental Outcomes
Scott, T. (2015) · Journal of Policy Analysis and Management · doi:10.1002/pam.21836 · Cited by 224
Executive overconfidence and compensation structure
Humphery-Jenner, M., Lisic, L. L., Nanda, V., & Silveri, S. D. (2016) · Journal of Financial Economics · doi:10.1016/j.jfineco.2016.01.022 · Cited by 180
Inverse probability tilting for moment condition models with missing data
Graham, B. S., De Xavier Pinto, C. C., & Egel, D. (2012) · The Review of Economic Studies · doi:10.1093/restud/rdr047 · Cited by 174 · cited 4×
“…being married, an indicator for 1974 unemployment, an indicator for 1975 unemployment, 1974 earnings in thousands of dollars and its square, 1975 earnings in thousands of dollars and its square, and interactions between the 1974 and 1975 unemployment indicators and between 1974 and 1975 earnings.”
BackgroundMethodInfluentialCross-border M&As and innovative activity of acquiring and target firms
Stiebale, J. (2015) · Journal of International Economics · doi:10.1016/j.jinteco.2015.12.005 · Cited by 158
Rosenbaum, P. R. (1987) · Journal of the American Statistical Association · doi:10.2307/2289440 · Cited by 138
Annals of Economics and Statistics
Collard, F. (2015) · Cited by 113
Executive Overconfidence and Compensation StructurePreprint
Humphery‐Jenner, M., Lisic, L. L., Nanda, V. K., et al. (2014) · SSRN Electronic Journal · doi:10.2139/ssrn.2416431 · Cited by 106
Reynolds, C. L., & Rohlin, S. M. (2015) · Journal of Urban Economics · doi:10.1016/j.jue.2015.04.003 · Cited by 80
Bandwidth Selection and the Estimation of Treatment Effects with Unbalanced Data
Galdo, J. C., SMITH, & BLACK (2008) · Annales d Économie et de Statistique · doi:10.2307/27917245 · Cited by 69
Data Adaptive Ridging in Local Polynomial Regression
Seifert, B., & Gasser, T. (2000) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.2000.10474884 · Cited by 67
Model averaging in semiparametric estimation of treatment effects
Kitagawa, T., & Muris, C. (2016) · Journal of Econometrics · doi:10.1016/j.jeconom.2016.03.002 · Cited by 48
Bandwidth Selection and the Estimation of Treatment Effects with Unbalanced DataPreprint
Galdo, J. C., Smith, J. A., & Black, D. A. (2007) · SSRN Electronic Journal · doi:10.2139/ssrn.1028208 · Cited by 47
How to Control for Many Covariates? Reliable Estimators Based on the Propensity ScorePreprint
Huber, M., Lechner, M., & Wunsch, C. (2010) · SSRN Electronic Journal · doi:10.2139/ssrn.1696892 · Cited by 43
Health Outcomes for Children Born to Teen Mothers in Cape Town, South Africa
Branson, N., Ardington, C., & Leibbrandt, M. (2015) · Economic Development and Cultural Change · doi:10.1086/679737 · Cited by 40
Propensity score and doubly robust methods for estimating the effect of treatment on censored cost
Li, J., Handorf, E. A., Bekelman, J. E., & Mitra, N. (2015) · Statistics in Medicine · doi:10.1002/sim.6842 · Cited by 31
Data Adaptive Ridging in Local Polynomial Regression
Seifert, B., & Gasser, T. (2000) · Journal of Computational and Graphical Statistics · doi:10.2307/1390658 · Cited by 31
Semiparametric decomposition of the gender achievement gap: An application for Turkey
Gevrek, Z. E., & Seiberlich, R. R. (2014) · Labour Economics · doi:10.1016/j.labeco.2014.08.002 · Cited by 26
Kolodziejczyk, C., & Heinesen, E. (2015) · Economics & Human Biology · doi:10.1016/j.ehb.2015.12.002 · Cited by 17
Mullally, C., & Maffioli, A. (2015) · American Journal of Agricultural Economics · doi:10.1093/ajae/aav050 · Cited by 14
Mamza, J., Mehta, R. L., Donnelly, R., & IDRIS, I. R. (2015) · Diabetes Therapy · doi:10.1007/s13300-015-0110-6 · Cited by 8
Empirical Bias Bandwidth Choice for Local Polynomial Matching EstimatorsPreprint
Flossmann, A. L. (2007) · SSRN Electronic Journal · doi:10.2139/ssrn.920787 · Cited by 8
The impact of import-related displacement on local business activity
Park, J., Reynolds, C. L., & Rohlin, S. M. (2014) · Regional Science and Urban Economics · doi:10.1016/j.regsciurbeco.2014.05.003 · Cited by 5
Research works printed but not linked: 1
- Huber Martin, 2010, IZA discussion paper, 5268
Cited by
Difference-in-Differences with multiple time periods
Callaway, B., & Sant’Anna, P. H. C. (2021) · Journal of Econometrics · doi:10.1016/j.jeconom.2020.12.001 · Cited by 9,126
Bias-Corrected Matching Estimators for Average Treatment Effects
Abadie, A., & Imbens, G. W. (2011) · Journal of Business & Economic Statistics · doi:10.1198/jbes.2009.07333 · Cited by 1,416
Decomposition Methods in Economics
Fortin, N., Lemieux, T., & Firpo, S. (2011) · Handbook of Labor Economics · doi:10.1016/s0169-7218(11)00407-2 · Cited by 1,029
Balancing Covariates via Propensity Score Weighting
Li, F., Morgan, K. L., & Zaslavsky, A. M. (2018) · Journal of the American Statistical Association · doi:10.1080/01621459.2016.1260466 · Cited by 962
Methods for Constructing and Assessing Propensity Scores
Garrido, M. M., Kelley, A. S., Paris, J., Roza, K., Meier, D. E., Morrison, R. S., & Aldridge, M. D. (2014) · Health Services Research · doi:10.1111/1475-6773.12182 · Cited by 882
Metadata
- Authors
- Busso, M., DiNardo, J., & McCrary, J.
- Affiliations
- University of Michigan (US)
- National Bureau of Economic Research (US)
- Venue
- Review of Economics and Statistics
- Publisher
- MIT Press - Journals
- Published
- 2014-12
- Volume
- 96
- Issue
- 5
- Pages
- 885-897
- ISSN
- 0034-6535 (ISSN-L)
- 1530-9142
- OpenAlex
- W1987537206
- Cited by
- 428
- References
- 46
- Citations by year
- 2026: 22 · 2025: 25 · 2024: 26 · 2023: 23 · 2022: 44 · 2021: 34 · 2020: 41 · 2019: 37 · 2018: 32 · 2017: 29 · 2016: 28 · 2015: 26 · 2014: 22
- FWCI
- 37.00
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- The Review of Economics and Statistics (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Berkley Law Scholarship Repository (University of California, Berkeley) (submittedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Social Sciences › Education › School Choice and Performance
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Spatial and Panel Data Analysis
- Keywords
- propensity score matching, finite sample properties, average treatment effect, overlap, model misspecification, data generating process, causal inference
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.