On the Failure of the Bootstrap for Matching Estimators
Open accessThe DOI matches its Crossref record; 8 of the 8 references it cites were found in a registry or an index.
Abstract
Matching estimators are widely used for the evaluation of programs or treatments.Often researchers use bootstrapping methods for inference.However, no formal justification for the use of the bootstrap has been provided.Here we show that the bootstrap is in general not valid, even in the simple case with a single continuous covariate when the estimator is root-N consistent and asymptotically normally distributed with zero asymptotic bias.Due to the extreme non-smoothness of nearest neighbor matching, the standard conditions for the bootstrap are not satisfied, leading the bootstrap variance to diverge from the actual variance.Simulations confirm the difference between actual and nominal coverage rates for bootstrap confidence intervals predicted by the theoretical calculations.To our knowledge, this is the first example of a root-N consistent and asymptotically normal estimator for which the bootstrap fails to work.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Matching as an econometric evaluation estimator: Evidence from evaluating a job training programme
Heckman, J. J., Ichimura, H., & Todd, P. E. (1997) · The Review of Economic Studies · doi:10.2307/2971733 · Cited by 5,486
Vaart, A. W. V. D. (1998) · doi:10.1017/cbo9780511802256 · Cited by 4,159
Does matching overcome lalonde’s critique of nonexperimental estimators?
A. Smith, J., & E. Todd, P. (2005) · Journal of Econometrics · doi:10.1016/j.jeconom.2004.04.011 · Cited by 3,176
Nonparametric Estimation of Average Treatment Effects Under Exogeneity: A Review
Imbens, G. W. (2004) · Review of Economics and Statistics · doi:10.1162/003465304323023651 · Cited by 2,994
Self-citationLarge sample properties of matching estimators for average treatment effects
Abadie, A., & Imbens, G. W. (2006) · Econometrica · doi:10.1111/j.1468-0262.2006.00655.x · Cited by 2,908
Self-citationCausal effects in nonexperimental studies: Reevaluating the evaluation of training programs
Dehejia, R. H., & Wahba, S. (1999) · Journal of the American Statistical Association · doi:10.1080/01621459.1999.10473858 · Cited by 2,395
Negative Association of Random Variables with Applications
Joag-Dev, K., & Proschan, F. (1983) · The Annals of Statistics · doi:10.1214/aos/1176346079 · Cited by 1,486
Politis, D. N., Romano, J. P., & Wolf, M. (1999) · Springer Series in Statistics · doi:10.1007/978-1-4612-1554-7 · Cited by 760
Cited by
SOME PRACTICAL GUIDANCE FOR THE IMPLEMENTATION OF PROPENSITY SCORE MATCHING
Caliendo, M., & Kopeinig, S. (2008) · Journal of Economic Surveys · doi:10.1111/j.1467-6419.2007.00527.x · Cited by 6,238
Large sample properties of matching estimators for average treatment effects
Abadie, A., & Imbens, G. W. (2006) · Econometrica · doi:10.1111/j.1468-0262.2006.00655.x · Cited by 2,908
Self-citationMeasuring the effectiveness of protected area networks in reducing deforestation
Andam, K. S., Ferraro, P. J., Pfaff, A., Sanchez-Azofeifa, G. A., & Robalino, J. A. (2008) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0800437105 · Cited by 1,028
Don't Believe the Hype: Local Media Slant, Local Advertising, and Firm Value
GURUN, U. G., & BUTLER, A. W. (2012) · The Journal of Finance · doi:10.1111/j.1540-6261.2012.01725.x · Cited by 612
Protected areas reduced poverty in Costa Rica and Thailand
Andam, K. S., Ferraro, P. J., Sims, K. R. E., Healy, A., & Holland, M. B. (2010) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0914177107 · Cited by 486
Metadata
- Authors
- Abadie, A., & Imbens, G.
- Venue
- National Bureau of Economic Research
- Publisher
- National Bureau of Economic Research
- Published
- 2006-06
- OpenAlex
- W3123208646
- Cited by
- 152
- References
- 8
- Type
- report
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.