Program Evaluation and Research Designs
Open accessThe DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.
Abstract
This chapter provides a selective review of some contemporary approaches to program evaluation. One motivation for our review is the recent emergence and increasing use of a particular kind of "program" in applied microeconomic research, the so-called Regression Discontinuity (RD) Design of Thistlethwaite and Campbell (1960). We organize our discussion of these various research designs by how they secure internal validity: in this view, the RD design can been seen as a close "cousin" of the randomized experiment. An important distinction which emerges from our discussion of "heterogeneous treatment effects" is between ex post (descriptive) and ex ante (predictive) evaluations; these two types of evaluations have distinct, but complementary goals. A second important distinction we make is between statistical statements that are descriptions of our knowledge of the program assignment process and statistical statements that are structural assumptions about individual behavior. Using these distinctions, we examine some commonly employed evaluation strategies, and assess them with a common set of criteria for "internal validity", the foremost goal of an ex post evaluation. In some cases, we also provide some concrete illustrations of how internally valid causal estimates can be supplemented with specific structural assumptions to address "external validity": the estimate from an internally valid "experimental" estimate can be viewed as a "leading term" in an extrapolation for a parameter of interest in an ex ante evaluation.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Estimating causal effects of treatments in randomized and nonrandomized studies
Rubin, D. B. (1974) · Journal of Educational Psychology · doi:10.1037/h0037350 · Cited by 9,436 · cited 2×
“The distinction between these situations has sometimes been blurred: the Neyman–Holland–Rubin Model (Splawa-Neyman et al., 1990, 1935; Rubin, 1990, 1974, 1986; Holland, 1986), which we discuss later, has been used in situations both where the investigator does have detailed institutional knowledge…”
Econometric policy evaluation: A critique
Lucas, R. E. (1976) · Carnegie-Rochester Conference Series on Public Policy · doi:10.1016/s0167-2231(76)80003-6 · Cited by 6,110
Statistics and Causal Inference
Holland, P. W. (1986) · Journal of the American Statistical Association · doi:10.1080/01621459.1986.10478354 · Cited by 5,293 · cited 2×
“…between these situations has sometimes been blurred: the Neyman–Holland–Rubin Model (Splawa-Neyman et al., 1990, 1935; Rubin, 1990, 1974, 1986; Holland, 1986), which we discuss later, has been used in situations both where the investigator does have detailed institutional knowledge of the…”
Regression discontinuity designs in economics
Lee, D. S., & Lemieux, T. (2010) · Journal of Economic Literature · doi:10.1257/jel.48.2.281 · Cited by 4,534
Self-citationIdentification of Causal Effects Using Instrumental Variables
Angrist, J. D., Imbens, G. W., & Rubin, D. B. (1996) · Journal of the American Statistical Association · doi:10.1080/01621459.1996.10476902 · Cited by 4,230
Identification and estimation of local average treatment effects
Imbens, G. W., & Angrist, J. D. (1994) · Econometrica · doi:10.2307/2951620 · Cited by 4,101
Matching As An Econometric Evaluation Estimator
Heckman, J. J., Ichimura, H., & Todd, P. (1998) · Review of Economic Studies · doi:10.1111/1467-937x.00044 · Cited by 3,807
Shadow Prices, Market Wages, and Labor Supply
Heckman, J. (1974) · Econometrica · doi:10.2307/1913937 · Cited by 1,852 · cited 1×
“6 The framework also accommodates standard latent variable threshold-crossing models (Heckman, 1974, 1976, 1978) such as: Y = α + Dβ + Xγ + ε D = 1 [Xδ +V > 0] where X, ε, (with an arbitrary joint distribution) are elements of W , and P * = Pr [V > −Xδ |X, ε].”
Empirical Strategies in Labor Economics
Angrist, J. D., & Krueger, A. B. (1999) · Handbook of Labor Economics · doi:10.1016/s1573-4463(99)03004-7 · Cited by 1,628
Estimating the Effect of Training Programs on Earnings
Ashenfelter, O. (1978) · The Review of Economics and Statistics · doi:10.2307/1924332 · Cited by 1,515
Cited by
Robust nonparametric confidence intervals for regression-discontinuity designs
Calonico, S., Cattaneo, M. D., & Titiunik, R. (2014) · Econometrica · doi:10.3982/ecta11757 · Cited by 3,036
Do Investors Value Sustainability? A Natural Experiment Examining Ranking and Fund Flows
HARTZMARK, S. M., & SUSSMAN, A. B. (2019) · The Journal of Finance · doi:10.1111/jofi.12841 · Cited by 1,711
Why high-order polynomials should not be used in regression discontinuity designs
Gelman, A., & Imbens, G. (2019) · Journal of Business & Economic Statistics · doi:10.1080/07350015.2017.1366909 · Cited by 1,101
Robust Data-Driven Inference in the Regression-Discontinuity Design
Calonico, S., Cattaneo, M. D., & Titiunik, R. (2014) · The Stata Journal: Promoting communications on statistics and Stata · doi:10.1177/1536867x1401400413 · Cited by 706
The Effect of Education on Adult Mortality and Health: Evidence from Britain
Clark, D., & Royer, H. (2013) · American Economic Review · doi:10.1257/aer.103.6.2087 · Cited by 505
Metadata
- Authors
- DiNardo, J., & Lee, D. S.
- Venue
- Handbook of Labor Economics
- Publisher
- Elsevier
- Published
- 2011
- Pages
- 463-536
- ISSN
- 1573-4463 (ISSN-L)
- OpenAlex
- W2096530780
- Cited by
- 140
- References
- 19
- FWCI
- 34.54
- Type
- book chapter
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Program Evaluation and Research Designs. https://app.cochat.ai/papers/10.1016/s0169-7218(11)00411-4
- The paper
- DiNardo, J., & Lee, D. S. (2011). Program Evaluation and Research Designs. Handbook of Labor Economics. https://doi.org/10.1016/s0169-7218(11)00411-4
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.