The DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
Causal Effects in Non-Experimental Studies: Re-Evaluating the Evaluation of Training Programs
Open accessAbstract
This paper uses propensity score methods to address the question: how well can an observational study estimate the treatment impact of a program? Using data from Lalonde's (1986) influential evaluation of non-experimental methods, we demonstrate that propensity score methods succeed in estimating the treatment impact of the National Supported Work Demonstration. Propensity score methods reduce the task of controlling for differences in pre-intervention variables between the treatment and the non-experimental comparison groups to controlling for differences in the estimated propensity score (the probability of assignment to treatment, conditional on covariates). It is difficult to control for differences in pre-intervention variables when they are numerous and when the treatment and comparison groups are dissimilar, whereas controlling for the estimated propensity score, a single variable on the unit interval, is a straightforward task. We apply several methods, such as stratification on the propensity score and matching on the propensity score, and show that they result in accurate estimates of the treatment impact.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The central role of the propensity score in observational studies for causal effects
ROSENBAUM, P. R., & RUBIN, D. B. (1983) · Biometrika · doi:10.1093/biomet/70.1.41 · Cited by 31,241
Sample Selection Bias as a Specification Error
Heckman, J. J. (1979) · Econometrica · doi:10.2307/1912352 · Cited by 28,779
Estimating causal effects of treatments in randomized and nonrandomized studies
Rubin, D. B. (1974) · Journal of Educational Psychology · doi:10.1037/h0037350 · Cited by 9,436
Matching as an econometric evaluation estimator: Evidence from evaluating a job training programme
Heckman, J. J., Ichimura, H., & Todd, P. E. (1997) · The Review of Economic Studies · doi:10.2307/2971733 · Cited by 5,486
Statistics and Causal Inference
Holland, P. W. (1986) · Journal of the American Statistical Association · doi:10.1080/01621459.1986.10478354 · Cited by 5,293
Fisher, R. A. (1936) · BMJ · doi:10.1136/bmj.1.3923.554-a · Cited by 4,980
Identification of Causal Effects Using Instrumental Variables
Angrist, J. D., Imbens, G. W., & Rubin, D. B. (1996) · Journal of the American Statistical Association · doi:10.1080/01621459.1996.10476902 · Cited by 4,230
Matching As An Econometric Evaluation Estimator
Heckman, J. J., Ichimura, H., & Todd, P. (1998) · Review of Economic Studies · doi:10.1111/1467-937x.00044 · Cited by 3,807
The Central Role of the Propensity Score in Observational Studies for Causal Effects
Rosenbaum, P. R., & Rubin, D. B. (1983) · Biometrika · doi:10.2307/2335942 · Cited by 3,261
Reducing Bias in Observational Studies Using Subclassification on the Propensity Score
Rosenbaum, P. R., & Rubin, D. B. (1984) · Journal of the American Statistical Association · doi:10.1080/01621459.1984.10478078 · Cited by 3,082
Cited by
Mostly Harmless Econometrics: An Empiricist's Companion
Angrist, J. D., & Pischke, J.-S. (2009) · doi:10.1515/9781400829828 · Cited by 9,467
SOME PRACTICAL GUIDANCE FOR THE IMPLEMENTATION OF PROPENSITY SCORE MATCHING
Caliendo, M., & Kopeinig, S. (2008) · Journal of Economic Surveys · doi:10.1111/j.1467-6419.2007.00527.x · Cited by 6,238
Matching Methods for Causal Inference: A Review and a Look Forward
Stuart, E. A. (2010) · Statistical Science · doi:10.1214/09-sts313 · Cited by 5,478
Unobservable Selection and Coefficient Stability: Theory and Evidence
Oster, E. (2019) · Journal of Business & Economic Statistics · doi:10.1080/07350015.2016.1227711 · Cited by 5,367
Recent Developments in the Econometrics of Program Evaluation
Imbens, G. W., & Wooldridge, J. M. (2009) · Journal of Economic Literature · doi:10.1257/jel.47.1.5 · Cited by 4,926
MatchIt : Nonparametric Preprocessing for Parametric Causal Inference
Ho, D. E., Imai, K., King, G., & Stuart, E. A. (2011) · Journal of Statistical Software · doi:10.18637/jss.v042.i08 · Cited by 4,385
Matching as Nonparametric Preprocessing for Reducing Model Dependence in Parametric Causal Inference
Ho, D. E., Imai, K., King, G., & Stuart, E. A. (2007) · Political Analysis · doi:10.1093/pan/mpl013 · Cited by 4,205
Large sample properties of matching estimators for average treatment effects
Abadie, A., & Imbens, G. W. (2006) · Econometrica · doi:10.1111/j.1468-0262.2006.00655.x · Cited by 2,908
Estimation of Average Treatment Effects Based on Propensity Scores
Becker, S. O., & Ichino, A. (2002) · The Stata Journal: Promoting communications on statistics and Stata · doi:10.1177/1536867x0200200403 · Cited by 2,754
Efficient Estimation of Average Treatment Effects Using the Estimated Propensity Score
Hirano, K., Imbens, G. W., & Ridder, G. (2003) · Econometrica · doi:10.1111/1468-0262.00442 · Cited by 2,355
Using Propensity Scores to Help Design Observational Studies: Application to the Tobacco Litigation
Rubin, D. B. (2001) · Health Services and Outcomes Research Methodology · doi:10.1023/a:1020363010465 · Cited by 2,228
Implementing Matching Estimators for Average Treatment Effects in Stata
Abadie, A., Drukker, D., Herr, J. L., & Imbens, G. W. (2004) · The Stata Journal: Promoting communications on statistics and Stata · doi:10.1177/1536867x0400400307 · Cited by 1,386
McCaffrey, D. F., Ridgeway, G., & Morral, A. R. (2004) · Psychological Methods · doi:10.1037/1082-989x.9.4.403 · Cited by 1,317
Covariate Balancing Propensity Score
Imai, K., & Ratkovic, M. (2014) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/rssb.12027 · Cited by 1,315
Using Randomization in Development Economics Research: A Toolkit
Duflo, E., Glennerster, R., & Kremer, M. (2006) · National Bureau of Economic Research · doi:10.3386/t0333 · Cited by 1,191
Environmental Policy and Directed Technological Change: Evidence from the European Carbon Market
Calel, R., & Dechezleprêtre, A. (2016) · Review of Economics and Statistics · doi:10.1162/rest_a_00470 · Cited by 1,147
ebalance : A Stata Package for Entropy Balancing
Hainmueller, J., & Xu, Y. (2013) · Journal of Statistical Software · doi:10.18637/jss.v054.i07 · Cited by 1,113
Diamond, A., & Sekhon, J. S. (2013) · Review of Economics and Statistics · doi:10.1162/rest_a_00318 · Cited by 1,102
Why high-order polynomials should not be used in regression discontinuity designs
Gelman, A., & Imbens, G. (2019) · Journal of Business & Economic Statistics · doi:10.1080/07350015.2017.1366909 · Cited by 1,101
Identification and Inference in Nonlinear Difference-in-Differences Models
Athey, S., & Imbens, G. W. (2006) · Econometrica · doi:10.1111/j.1468-0262.2006.00668.x · Cited by 1,084
An Evaluation of the Swedish System of Active Labor Market Programs in the 1990s
Sianesi, B. (2004) · Review of Economics and Statistics · doi:10.1162/003465304323023723 · Cited by 1,053
Causal Inference With General Treatment Regimes: Generalizing the Propensity Score
Imai, K., & van Dyk, D. A. (2004) · Journal of the American Statistical Association · doi:10.1198/016214504000001187 · Cited by 868
Country-level institutions, firm value, and the role of corporate social responsibility initiatives
Ghoul, S. E., Guedhami, O., & Kim, Y. (2017) · Journal of International Business Studies · doi:10.1057/jibs.2016.4 · Cited by 822
On the Failure of the Bootstrap for Matching Estimators
(2008) · Econometrica · doi:10.3982/ecta6474 · Cited by 798
Electoral Rules and Corruption
Persson, T., Tabellini, G., & Trebbi, F. (2003) · Journal of the European Economic Association · doi:10.1162/154247603322493203 · Cited by 782
Using matching, instrumental variables and control functions to estimate economic choice models
Heckman, J., & Navarro-Lozano, S. (2004) · Review of Economics and Statistics · doi:10.1162/003465304323023660 · Cited by 768
Does Diversification Cause the 'Diversification Discount'?
Villalonga, B. (2000) · SSRN Electronic Journal · doi:10.2139/ssrn.227828 · Cited by 737
Lechner, M. (2001) · ZEW Economic Studies · doi:10.1007/978-3-642-57615-7_3 · Cited by 710
Earnings Losses of Displaced Workers Revisited
Couch, K. A., & Placzek, D. W. (2010) · American Economic Review · doi:10.1257/aer.100.1.572 · Cited by 676
How robust is the evidence on the effects of college quality? Evidence from matching
Black, D. A., & Smith, J. A. (2004) · Journal of Econometrics · doi:10.1016/j.jeconom.2003.10.006 · Cited by 673
Efficient Semiparametric Estimation of Quantile Treatment Effects
Firpo, S. (2007) · Econometrica · doi:10.1111/j.1468-0262.2007.00738.x · Cited by 631
Sensitivity analysis for average treatment effects
Becker, S. O., & Caliendo, M. (2007) · The Stata Journal: Promoting communications on statistics and Stata · doi:10.1177/1536867x0700700104 · Cited by 620
Agency Problems at Dual‐Class Companies
MASULIS, R. W., WANG, C., & XIE, F. (2009) · The Journal of Finance · doi:10.1111/j.1540-6261.2009.01477.x · Cited by 605
Propensity Score Matching: A Statistical Method
Kane, L. T., Fang, T., Galetta, M. S., Goyal, D. K. C., Nicholson, K. J., Kepler, C. K., Vaccaro, A. R., & Schroeder, G. D. (2020) · Clinical Spine Surgery: A Spine Publication · doi:10.1097/bsd.0000000000000932 · Cited by 603
Untangling the Causal Effects of Sex on Judging
Boyd, C. L., Epstein, L., & Martin, A. D. (2010) · American Journal of Political Science · doi:10.1111/j.1540-5907.2010.00437.x · Cited by 591
Slavin, R. E. (2008) · Educational Researcher · doi:10.3102/0013189x08314117 · Cited by 560
Oberlander, T. F., Warburton, W., Misri, S., Aghajanian, J., & Hertzman, C. (2006) · Archives of General Psychiatry · doi:10.1001/archpsyc.63.8.898 · Cited by 551
Diagnosing and responding to violations in the positivity assumption
Petersen, M. L., Porter, K. E., Gruber, S., Wang, Y., & van der Laan, M. J. (2012) · Statistical Methods in Medical Research · doi:10.1177/0962280210386207 · Cited by 548
Rishika, R., Kumar, A., Janakiraman, R., & Bezawada, R. (2013) · Information Systems Research · doi:10.1287/isre.1120.0460 · Cited by 547
Recent Developments in the Econometrics of Program Evaluation
Imbens, G., & Wooldridge, J. (2008) · National Bureau of Economic Research · doi:10.3386/w14251 · Cited by 533
Evaluating the impact of R&D tax credits on innovation: A microeconometric study on Canadian firms
Czarnitzki, D., Hanel, P., & Rosa, J. M. (2011) · Research Policy · doi:10.1016/j.respol.2010.09.017 · Cited by 522
Evaluating Anti-Poverty Programs
Ravallion, M. (2005) · World Bank, Washington, DC eBooks · doi:10.1596/1813-9450-3625 · Cited by 507
Evaluating the Employment Impact of a Mandatory Job Search Program
Blundell, R., Dias, M. C., Meghir, C., & van Reenen, J. (2004) · Journal of the European Economic Association · doi:10.1162/1542476041423368 · Cited by 500
Lechner, M. (2002) · Review of Economics and Statistics · doi:10.1162/003465302317411488 · Cited by 499
Estimating treatment effect heterogeneity in randomized program evaluation
Imai, K., & Ratkovic, M. (2013) · The Annals of Applied Statistics · doi:10.1214/12-aoas593 · Cited by 480
Efficient Estimation of Average Treatment Effects Using the Estimated Propensity Score
Hirano, K., Imbens, G. W., & Ridder, G. (2002) · SSRN Electronic Journal · doi:10.2139/ssrn.324940 · Cited by 480
Covariance Adjustment in Randomized Experiments and Observational Studies
Rosenbaum, P. R. (2002) · Statistical Science · doi:10.1214/ss/1042727942 · Cited by 477
Evaluating the Effect of Teachers’ Group Performance Incentives on Pupil Achievement
Lavy, V. (2002) · Journal of Political Economy · doi:10.1086/342810 · Cited by 448
Nonparametric Estimation of Average Treatment Effects under Exogeneity: A Review
Imbens, G. (2003) · National Bureau of Economic Research · doi:10.3386/t0294 · Cited by 446
Matching Methods in Practice: Three Examples
Imbens, G. W. (2015) · Journal of Human Resources · doi:10.3368/jhr.50.2.373 · Cited by 445
Endogenous product versus process innovation and a firm’s propensity to export
Becker, S. O., & Egger, P. H. (2013) · Empirical Economics · doi:10.1007/s00181-009-0322-6 · Cited by 431
New evidence on the finite sample properties of propensity score reweighting and matching estimators
Busso, M., DiNardo, J., & McCrary, J. (2014) · Review of Economics and Statistics · doi:10.1162/rest_a_00431 · Cited by 428
Constitutional Rules and Fiscal Policy Outcomes
Persson, T., & Tabellini, G. (2004) · American Economic Review · doi:10.1257/000282804322970689 · Cited by 413
Practical propensity score matching: a reply to Smith and Todd
Dehejia, R. (2005) · Journal of Econometrics · doi:10.1016/j.jeconom.2004.04.012 · Cited by 390
Self-citationLife satisfaction and self-employment: a matching approach
Binder, M., & Coad, A. (2013) · Small Business Economics · doi:10.1007/s11187-011-9413-9 · Cited by 375
Too Much Ado about Propensity Score Models? Comparing Methods of Propensity Score Matching
Baser, O. (2006) · Value in Health · doi:10.1111/j.1524-4733.2006.00130.x · Cited by 372
The prognostic analogue of the propensity score
Hansen, B. B. (2008) · Biometrika · doi:10.1093/biomet/asn004 · Cited by 371
Lechner, M. (2002) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/1467-985x.0asp2 · Cited by 369
From the Editors: Endogeneity in international business research
Reeb, D., Sakakibara, M., & Mahmood, I. P. (2012) · Journal of International Business Studies · doi:10.1057/jibs.2011.60 · Cited by 358
Propensity Score Matching Methods for Non-experimental Causal Studies
Rajeev, D., & Wahba, S. (1999) · SSRN Electronic Journal · doi:10.2139/ssrn.138259 · Cited by 355
Self-citationKai, L., & Prabhala, N. R. (2007) · Handbook of Empirical Corporate Finance · doi:10.1016/b978-0-444-53265-7.50016-0 · Cited by 352
Blundell, R., Dearden, L., & Sianesi, B. (2005) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/j.1467-985x.2004.00360.x · Cited by 343
Simple and Bias-Corrected Matching Estimators for Average Treatment Effects
Abadie, A., & Imbens, G. (2002) · National Bureau of Economic Research · doi:10.3386/t0283 · Cited by 342
Best Practices in Quasi–Experimental Designs: Matching Methods for Causal Inference
Stuart, E. A., & Rubin, D. B. (2008) · Best Practices in Quantitative Methods · doi:10.4135/9781412995627.d14 · Cited by 335
The effect of charter schools on charter students and public schools
Bettinger, E. P. (2005) · Economics of Education Review · doi:10.1016/j.econedurev.2004.04.009 · Cited by 325
Does Digital Divide or Provide? The Impact of Cell Phones on Grain Markets in Niger
Aker, J. C. (2008) · SSRN Electronic Journal · doi:10.2139/ssrn.1093374 · Cited by 324
What Can We Learn about Neighborhood Effects from the Moving to Opportunity Experiment?
Ludwig, J., Liebman, J. B., Kling, J. R., Duncan, G. J., Katz, L. F., Kessler, R. C., & Sanbonmatsu, L. (2008) · American Journal of Sociology · doi:10.1086/588741 · Cited by 320
Opiates for the Matches: Matching Methods for Causal Inference
Sekhon, J. S. (2009) · Annual Review of Political Science · doi:10.1146/annurev.polisci.11.060606.135444 · Cited by 319
The performance of estimators based on the propensity score
Huber, M., Lechner, M., & Wunsch, C. (2013) · Journal of Econometrics · doi:10.1016/j.jeconom.2012.11.006 · Cited by 311
Why High-order Polynomials Should not be Used in Regression Discontinuity Designs
Gelman, A., & Imbens, G. (2014) · National Bureau of Economic Research · doi:10.3386/w20405 · Cited by 301
Estimating and Using Propensity Scores with Partially Missing Data
D'Agostino, R. B., & Rubin, D. B. (2000) · Journal of the American Statistical Association · doi:10.1080/01621459.2000.10474263 · Cited by 300
Levine, D. I., & Toffel, M. W. (2010) · Management Science · doi:10.1287/mnsc.1100.1159 · Cited by 293
The impact of entrepreneurship education in high school on long-term entrepreneurial performance
Elert, N., Andersson, F. W., & Wennberg, K. (2015) · Journal of Economic Behavior & Organization · doi:10.1016/j.jebo.2014.12.020 · Cited by 290
Does knowledge diffusion between university and industry increase innovativeness?
Lööf, H., & Broström, A. (2008) · The Journal of Technology Transfer · doi:10.1007/s10961-006-9001-3 · Cited by 264
The Effect of Classmate Characteristics on Post-Secondary Outcomes: Evidence from the Add Health
Bifulco, R., Fletcher, J. M., & Ross, S. L. (2011) · American Economic Journal: Economic Policy · doi:10.1257/pol.3.1.25 · Cited by 260
Cook, T. D. (2002) · Educational Evaluation and Policy Analysis · doi:10.3102/01623737024003175 · Cited by 257
What Works to Prevent Child Marriage: A Review of the Evidence
Lee‐Rife, S., Malhotra, A., Warner, A., & Glinski, A. M. (2012) · Studies in Family Planning · doi:10.1111/j.1728-4465.2012.00327.x · Cited by 254
Diaz, J. J., & Handa, S. (2006) · Journal of Human Resources · doi:10.3368/jhr.xli.2.319 · Cited by 196
Predicting the efficacy of future training programs using past experiences at other locations
Joseph Hotz, V., Imbens, G. W., & Mortimer, J. H. (2005) · Journal of Econometrics · doi:10.1016/j.jeconom.2004.04.009 · Cited by 190
A decision support framework to implement optimal personalized marketing interventions
Guelman, L., Guillén, M., & Pérez-Marín, A. M. (2015) · Decision Support Systems · doi:10.1016/j.dss.2015.01.010 · Cited by 48
Barbosa, C., Cowell, A., Dowd, W., Landwehr, J., Aldridge, A., & Bray, J. (2017) · Addiction · doi:10.1111/add.13658 · Cited by 46
What was lost? A causal estimate of fourth down behavior in the National Football League
Yam, D. R., & Lopez, M. J. (2019) · Journal of Sports Analytics · doi:10.3233/jsa-190294 · Cited by 33
Metadata
- Authors
- Dehejia, R., & Wahba, S.
- Venue
- National Bureau of Economic Research
- Publisher
- National Bureau of Economic Research
- Published
- 1998-06
- OpenAlex
- W2171442673
- Cited by
- 805
- References
- 38
- Type
- report
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.