The DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.
Choosing Among Alternative Nonexperimental Methods for Estimating the Impact of Social Programs: The Case of Manpower Training
Open accessAbstract
The recent literature on evaluating manpower training programs demonstrates that alternative nonexperimental estimators of the same program produce an array of estimates of program impact. These findings have led to the call for experiments to be used to perform credible program evaluations. Missing in all of the recent pessimistic analyses of nonexperimental methods is any systematic discussion of how to choose among competing estimators. This article explores the value of simple specification tests in selecting an appropriate nonexperimental estimator. A reanalysis of the National Supported Work Demonstration data previously analyzed by proponents of social experiments reveals that a simple testing procedure eliminates the range of nonexperimental estimators at variance with the experimental estimates of program impact.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The central role of the propensity score in observational studies for causal effects
ROSENBAUM, P. R., & RUBIN, D. B. (1983) · Biometrika · doi:10.1093/biomet/70.1.41 · Cited by 31,241
Evaluation: A Systematic Approach
Rossi, P. H., Lipsey, M. W., & Freeman, H. E. (1979) · Cited by 2,943
Dummy Endogenous Variables in a Simultaneous Equation System
Heckman, J. (1977) · National Bureau of Economic Research · doi:10.3386/w0177 · Cited by 1,837
Self-citationSpecification Searches: Ad Hoc Inference with Non-Experimental Data.
Richard, J.-F., & Leamer, E. E. (1981) · Economica · doi:10.2307/2553491 · Cited by 1,114
Alternative methods for evaluating the impact of interventions
Heckman, J. J., & Robb, R. (1985) · Longitudinal Analysis of Labor Market Data · doi:10.1017/ccol0521304539.004 · Cited by 1,101
Self-citationUsing the Longitudinal Structure of Earnings to Estimate the Effect of Training Programs
Ashenfelter, O., & Card, D. (1984) · National Bureau of Economic Research · doi:10.3386/w1489 · Cited by 876
Systematic Thinking for Social Action
Tullock, G., & Rivlin, A. M. (1972) · Journal of Money, Credit and Banking · doi:10.2307/1991056 · Cited by 363
Alternative Estimators and Sample Designs for Discrete Choice Analysis
Manski, C. F. (1981) · Cited by 337
The Adequacy of Comparison Group Designs for Evaluations of Employment-Related Programs
Fraker, T., & Maynard, R. (1987) · The Journal of Human Resources · doi:10.2307/145902 · Cited by 323
Evaluation: A Systematic Approach.
Essi, N. O., Rossi, P. H., & Freeman, H. E. (1987) · The Statistician · doi:10.2307/2348859 · Cited by 274
Cited by
(2001) · RePEc: Research Papers in Economics · Cited by 27,577
Matching as an econometric evaluation estimator: Evidence from evaluating a job training programme
Heckman, J. J., Ichimura, H., & Todd, P. E. (1997) · The Review of Economic Studies · doi:10.2307/2971733 · Cited by 5,486
Self-citationUnobservable Selection and Coefficient Stability: Theory and Evidence
Oster, E. (2019) · Journal of Business & Economic Statistics · doi:10.1080/07350015.2016.1227711 · Cited by 5,367
Regression discontinuity designs: A guide to practice
Imbens, G. W., & Lemieux, T. (2008) · Journal of Econometrics · doi:10.1016/j.jeconom.2007.05.001 · Cited by 3,694
Does matching overcome lalonde’s critique of nonexperimental estimators?
A. Smith, J., & E. Todd, P. (2005) · Journal of Econometrics · doi:10.1016/j.jeconom.2004.04.011 · Cited by 3,176
The Economics and Econometrics of Active Labor Market Programs
Heckman, J. J., Lalonde, R. J., & Smith, J. A. (1999) · Handbook of Labor Economics · doi:10.1016/s1573-4463(99)03012-6 · Cited by 3,059
Self-citationEarnings Losses of Displaced Workers
Jacobson, L. S., LaLonde, R. J., & Sullivan, D. G. (1992) · doi:10.17848/wp92-11 · Cited by 2,895
Comparative Politics and the Synthetic Control Method
Abadie, A., Diamond, A., & Hainmueller, J. (2015) · American Journal of Political Science · doi:10.1111/ajps.12116 · Cited by 2,406
Causal effects in nonexperimental studies: Reevaluating the evaluation of training programs
Dehejia, R. H., & Wahba, S. (1999) · Journal of the American Statistical Association · doi:10.1080/01621459.1999.10473858 · Cited by 2,395
Using Propensity Scores to Help Design Observational Studies: Application to the Tobacco Litigation
Rubin, D. B. (2001) · Health Services and Outcomes Research Methodology · doi:10.1023/a:1020363010465 · Cited by 2,228
Does Entrepreneurship Pay? An Empirical Analysis of the Returns to Self‐Employment
Hamilton, B. H. (2000) · Journal of Political Economy · doi:10.1086/262131 · Cited by 2,228
Empirical Strategies in Labor Economics
Angrist, J. D., & Krueger, A. B. (1999) · Handbook of Labor Economics · doi:10.1016/s1573-4463(99)03004-7 · Cited by 1,628
Using Synthetic Controls: Feasibility, Data Requirements, and Methodological Aspects
Abadie, A. (2021) · Journal of Economic Literature · doi:10.1257/jel.20191450 · Cited by 1,486
Bias-Corrected Matching Estimators for Average Treatment Effects
Abadie, A., & Imbens, G. W. (2011) · Journal of Business & Economic Statistics · doi:10.1198/jbes.2009.07333 · Cited by 1,416
Diamond, A., & Sekhon, J. S. (2013) · Review of Economics and Statistics · doi:10.1162/rest_a_00318 · Cited by 1,102
Uggen, C. (2000) · American Sociological Review · doi:10.1177/000312240006500403 · Cited by 1,079
Evaluation Methods for Non‐Experimental Data
Blundell, R., & Costa Dias, M. (2000) · Fiscal Studies · doi:10.1111/j.1475-5890.2000.tb00031.x · Cited by 1,003
What Works? A Meta Analysis of Recent Active Labor Market Program Evaluations
Card, D., Kluve, J., & Weber, A. (2018) · Journal of the European Economic Association · doi:10.1093/jeea/jvx028 · Cited by 990
Confounding and Collapsibility in Causal Inference
Greenland, S., Pearl, J., & Robins, J. M. (1999) · Statistical Science · doi:10.1214/ss/1009211805 · Cited by 933
Assessing the Case for Social Experiments
Heckman, J. J., & Smith, J. A. (1995) · Journal of Economic Perspectives · doi:10.1257/jep.9.2.85 · Cited by 936
Self-citationTHE ESTIMATION OF CAUSAL EFFECTS FROM OBSERVATIONAL DATA
Winship, C., & Morgan, S. L. (1999) · Annual Review of Sociology · doi:10.1146/annurev.soc.25.1.659 · Cited by 890
Alternative approaches to evaluation in empirical microeconomics
Blundell, R., & Costa Dias, M. (2008) · doi:10.1920/wp.cem.2008.2608 · Cited by 881
Water For Life: The impact of the privatization of water services on child mortality
Galiani, S., Gertler, P., & Schargrodsky, E. (2005) · Journal of Political Economy · doi:10.1086/426041 · Cited by 877
Chief Executive Officer Equity Incentives and Accounting Irregularities
ARMSTRONG, C. S., JAGOLINZER, A. D., & LARCKER, D. F. (2010) · Journal of Accounting Research · doi:10.1111/j.1475-679x.2009.00361.x · Cited by 875
Behavioral science at the crossroads in public health: Extending horizons, envisioning the future
Glass, T. A., & McAtee, M. J. (2006) · Social Science & Medicine · doi:10.1016/j.socscimed.2005.08.044 · Cited by 821
Abadie, A., Diamond, A., & Hainmueller, J. (2007) · National Bureau of Economic Research · doi:10.3386/t0335 · Cited by 815
Models for Sample Selection Bias
Winship, C., & Mare, R. D. (1992) · Annual Review of Sociology · doi:10.1146/annurev.so.18.080192.001551 · Cited by 812
An Analysis of Sample Attrition in Panel Data: The Michigan Panel Study of Income Dynamics
Fitzgerald, J., Gottschalk, P., & Moffitt, R. (1998) · The Journal of Human Resources · doi:10.2307/146433 · Cited by 805
Causal Effects in Non-Experimental Studies: Re-Evaluating the Evaluation of Training Programs
Dehejia, R., & Wahba, S. (1998) · National Bureau of Economic Research · doi:10.3386/w6586 · Cited by 805
Using matching, instrumental variables and control functions to estimate economic choice models
Heckman, J., & Navarro-Lozano, S. (2004) · Review of Economics and Statistics · doi:10.1162/003465304323023660 · Cited by 768
Self-citationThe (mis)estimation of neighborhood effects: causal inference for a practicable social epidemiology
Oakes, J. M. (2004) · Social Science & Medicine · doi:10.1016/j.socscimed.2003.08.004 · Cited by 760
Information from Markets Near and Far: Mobile Phones and Agricultural Markets in Niger
Aker, J. C. (2010) · American Economic Journal: Applied Economics · doi:10.1257/app.2.3.46 · Cited by 746
New Evidence on College Remediation
Attewell, P., Lavin, D., Domina, T., & Levey, T. (2006) · The Journal of Higher Education · doi:10.1080/00221546.2006.11778948 · Cited by 730
Van Der Klaauw, W. (2002) · International Economic Review · doi:10.1111/1468-2354.t01-1-00055 · Cited by 722
The Conceptualisation and Measurement of Mega Sport Event Legacies
Preuss, H. (2007) · Journal of Sport & Tourism · doi:10.1080/14775080701736957 · Cited by 715
Frydman, R., Gray, C., Hessel, M., & Rapaczynski, A. (1999) · The Quarterly Journal of Economics · doi:10.1162/003355399556241 · Cited by 697
Matched Sampling for Causal Effects
Rubin, D. B. (2006) · Cambridge University Press eBooks · doi:10.1017/cbo9780511810725 · Cited by 676
Earnings Losses of Displaced Workers Revisited
Couch, K. A., & Placzek, D. W. (2010) · American Economic Review · doi:10.1257/aer.100.1.572 · Cited by 676
Rubin, D. B. (1990) · Statistical Science · doi:10.1214/ss/1177012032 · Cited by 665
Employment Effects of Minimum and Subminimum Wages: Panel Data on State Minimum Wage Laws
Neumark, D., & Wascher, W. (1992) · ILR Review · doi:10.1177/001979399204600105 · Cited by 655
Efficient Semiparametric Estimation of Quantile Treatment Effects
Firpo, S. (2007) · Econometrica · doi:10.1111/j.1468-0262.2007.00738.x · Cited by 631
Propensity Score Matching: A Statistical Method
Kane, L. T., Fang, T., Galetta, M. S., Goyal, D. K. C., Nicholson, K. J., Kepler, C. K., Vaccaro, A. R., & Schroeder, G. D. (2020) · Clinical Spine Surgery: A Spine Publication · doi:10.1097/bsd.0000000000000932 · Cited by 603
Sensitivity to Exogeneity Assumptions in Program Evaluation
Imbens, G. W. (2003) · American Economic Review · doi:10.1257/000282803321946921 · Cited by 529
Farrelly, M. C., Davis, K. C., Haviland, M. L., Messeri, P., & Healton, C. G. (2005) · American Journal of Public Health · doi:10.2105/ajph.2004.049692 · Cited by 500
Evaluating the Employment Impact of a Mandatory Job Search Program
Blundell, R., Dias, M. C., Meghir, C., & van Reenen, J. (2004) · Journal of the European Economic Association · doi:10.1162/1542476041423368 · Cited by 500
Breierova, L., & Duflo, E. (2004) · National Bureau of Economic Research · doi:10.3386/w10513 · Cited by 459
The Economic Impact of the Coal Boom and Bust
Black, D., McKinnish, T., & Sanders, S. (2005) · The Economic Journal · doi:10.1111/j.1468-0297.2005.00996.x · Cited by 446
Covenants with weak swords: ISO 14001 and facilities' environmental performance
Potoski, M., & Prakash, A. (2005) · Journal of Policy Analysis and Management · doi:10.1002/pam.20136 · Cited by 436
Robustness checks and robustness tests in applied economics
Lu, X., & White, H. (2014) · Journal of Econometrics · doi:10.1016/j.jeconom.2013.08.016 · Cited by 412
Building Bridges between Structural and Program Evaluation Approaches to Evaluating Policy
Heckman, J. J. (2010) · Journal of Economic Literature · doi:10.1257/jel.48.2.356 · Cited by 410
Self-citationFrank, K. A., Maroulis, S. J., Duong, M. Q., & Kelcey, B. M. (2013) · Educational Evaluation and Policy Analysis · doi:10.3102/0162373713493129 · Cited by 406
The Small Firm Effect and the Entrepreneurial Spawning of Scientists and Engineers
Elfenbein, D. W., Hamilton, B. H., & Zenger, T. R. (2010) · Management Science · doi:10.1287/mnsc.1090.1130 · Cited by 405
Lebovic, J. H., & Voeten, E. (2009) · Journal of Peace Research · doi:10.1177/0022343308098405 · Cited by 402
1. The Scientific Model of Causality
Heckman, J. J. (2005) · Sociological Methodology · doi:10.1111/j.0081-1750.2006.00164.x · Cited by 401
Self-citationCausal Inference in Sociological Research
Gangl, M. (2010) · Annual Review of Sociology · doi:10.1146/annurev.soc.012809.102702 · Cited by 401
Inference for Non-random Samples
Copas, J. B., & Li, H. G. (1997) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/1467-9868.00055 · Cited by 392
Practical propensity score matching: a reply to Smith and Todd
Dehejia, R. (2005) · Journal of Econometrics · doi:10.1016/j.jeconom.2004.04.012 · Cited by 390
Confounding in Health Research
Greenland, S., & Morgenstern, H. (2001) · Annual Review of Public Health · doi:10.1146/annurev.publhealth.22.1.189 · Cited by 388
Heckman, J. J., & Vytlacil, E. J. (2007) · Handbook of Econometrics · doi:10.1016/s1573-4412(07)06071-0 · Cited by 369
Self-citationEarnings and Employment Effects of Continuous Gff-the-Job Training in East Germany After Unification
Lechner, M. (1999) · Journal of Business & Economic Statistics · doi:10.1080/07350015.1999.10524798 · Cited by 368
Using the Propensity Score Method to Estimate Causal Effects: A Review and Practical Guide
Li, M. (2013) · Organizational Research Methods · doi:10.1177/1094428112447816 · Cited by 366
Wooldridge, J. M. (2005) · Review of Economics and Statistics · doi:10.1162/0034653053970320 · Cited by 363
Hsiao, C., Steve Ching, H., & Ki Wan, S. (2012) · Journal of Applied Econometrics · doi:10.1002/jae.1230 · Cited by 347
Ham, J. C., & Lalonde, R. J. (1996) · Econometrica · doi:10.2307/2171928 · Cited by 344
The Case for Randomized Field Trials in Economic and Policy Research
Burtless, G. (1995) · Journal of Economic Perspectives · doi:10.1257/jep.9.2.63 · Cited by 341
The Impact of Incarceration on Employment during the Transition to Adulthood
Apel, R., & Sweeten, G. (2010) · Social Problems · doi:10.1525/sp.2010.57.3.448 · Cited by 337
Reconciling Conflicting Evidence on the Performance of Propensity-Score Matching Methods
Smith, J. A., & Todd, P. E. (2001) · American Economic Review · doi:10.1257/aer.91.2.112 · Cited by 331
Brown, J. D., Earle, J. S., & Telegdy, Á. (2006) · Journal of Political Economy · doi:10.1086/499547 · Cited by 326
Debt Relief and Debtor Outcomes: Measuring the Effects of Consumer Bankruptcy Protection
Dobbie, W., & Song, J. (2015) · American Economic Review · doi:10.1257/aer.20130612 · Cited by 317
Partial Identification in Econometrics
Tamer, E. (2010) · Annual Review of Economics · doi:10.1146/annurev.economics.050708.143401 · Cited by 314
Alon, S., & Tienda, M. (2005) · Sociology of Education · doi:10.1177/003804070507800402 · Cited by 308
School decentralization: Helping the good get better, but leaving the poor behind
Galiani, S., Gertler, P., & Schargrodsky, E. (2008) · Journal of Public Economics · doi:10.1016/j.jpubeco.2008.05.004 · Cited by 304
Business start-ups by the unemployed — an econometric analysis based on firm data
Pfeiffer, F., & Reize, F. (2000) · Labour Economics · doi:10.1016/s0927-5371(00)00016-6 · Cited by 297
Bounding a Matching Estimator: The Case of a Norwegian Training Program
Aakvik, A. (2001) · Oxford Bulletin of Economics and Statistics · doi:10.1111/1468-0084.00211 · Cited by 296
Estimating the returns to community college schooling for displaced workers
Jacobson, L., LaLonde, R., & G. Sullivan, D. (2005) · Journal of Econometrics · doi:10.1016/j.jeconom.2004.04.010 · Cited by 290
Identification of Causal Effects Using Instrumental Variables: Comment
Robins, J. M., & Greenland, S. (1996) · Journal of the American Statistical Association · doi:10.2307/2291630 · Cited by 290
Recent developments in the econometrics of program evaluation
Wooldridge, J. M., & Imbens, G. (2008) · doi:10.1920/wp.cem.2008.2408 · Cited by 283
Aakvik, A., Heckman, J. J., & Vytlacil, E. J. (2005) · Journal of Econometrics · doi:10.1016/j.jeconom.2004.04.002 · Cited by 274
Self-citationMarriage, Motherhood, and Wages
Korenman, S., & Neumark, D. (1990) · National Bureau of Economic Research · doi:10.3386/w3473 · Cited by 268
The Highs and Lows of the Minimum Wage Effect: A Time‐Series Cross‐Section Study of the Canadian Law
Baker, M., Benjamin, D., & Stanger, S. (1999) · Journal of Labor Economics · doi:10.1086/209923 · Cited by 263
What Works to Prevent Child Marriage: A Review of the Evidence
Lee‐Rife, S., Malhotra, A., Warner, A., & Glinski, A. M. (2012) · Studies in Family Planning · doi:10.1111/j.1728-4465.2012.00327.x · Cited by 254
Juster, H. R., Loomis, B. R., Hinman, T. M., Farrelly, M. C., Hyland, A., Bauer, U. E., & Birkhead, G. S. (2007) · American Journal of Public Health · doi:10.2105/ajph.2006.099994 · Cited by 240
Darolia, R. (2014) · Economics of Education Review · doi:10.1016/j.econedurev.2013.10.004 · Cited by 238
Finance and Growth at the Firm Level: Evidence from SBA Loans
BROWN, J. D., & EARLE, J. S. (2017) · The Journal of Finance · doi:10.1111/jofi.12492 · Cited by 226
Quantile treatment effects in difference in differences models with panel data
Callaway, B., & Li, T. (2019) · Quantitative Economics · doi:10.3982/qe935 · Cited by 110
Formal Processing and Future Delinquency: Deviance Amplification as Selection Artifact
Smith, D. A., & Paternoster, R. (1990) · Law & Society Review · doi:10.2307/3053663 · Cited by 106
Is educational achievement a turning point for incarcerated delinquents across race and sex?
Blomberg, T. G., Bales, W. D., & Piquero, A. R. (2012) · Journal of Youth and Adolescence · doi:10.1007/s10964-011-9680-4 · Cited by 84
Butry, D. T. (2009) · Environmental and Ecological Statistics · doi:10.1007/s10651-007-0083-3 · Cited by 33
MEDICAL REGULATION AND HEALTH OUTCOMES: THE EFFECT OF THE PHYSICIAN EXAMINATION REQUIREMENT
Cotet, A. M., & Benjamin, D. K. (2013) · Health Economics · doi:10.1002/hec.2807 · Cited by 13
Metadata
- Authors
- Heckman, J.
- Venue
- National Bureau of Economic Research
- Publisher
- National Bureau of Economic Research
- Published
- 1989-02
- OpenAlex
- W3125727931
- Cited by
- 882
- References
- 16
- Type
- report
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.