CoChat

Characterizing Selection Bias Using Experimental Data

Open access
Heckman, James, Ichimura, Hidehiko, Smith, Jeffrey, Todd, Petra · 1998 · National Bureau of Economic Research

The DOI matches its Crossref record.

Abstract

This paper develops and applies semiparametric econometric methods to estimate the form of selection bias that arises from using nonexperimental comparison groups to evaluate social programs and to test the identifying assumptions that justify three widely-used classes of estimators and our extensions of them: (a) the method of matching; (b) the classical econometric selection model which represents the bias solely as a function of the probability of participation; and (c) the method of difference-in-differences. Using data from an experiment on a prototypical social program combined with unusually rich data from a nonexperimental comparison group, we reject the assumptions justifying matching and our extensions of that method but find evidence in support of the index-sufficient selection bias model and the assumptions that justify application of a conditional semiparametric version of the method of difference-in-difference. Fa comparable people and to appropriately weight participants and nonparticipants a sources of selection bias as conveniently measured. We present a rigorous defin bias and find that in our data it is a small component of conventially meausred it is still substantial when compared with experimentally-estimated program impa matching participants to comparison group members in the same labor market, givi same questionnaire, and making sure they have comparable characteristics substan the performance of any econometric program evaluation estimator. We show how t analysis to estimate the impact of treatment on the treated using ordinary obser

Overview

Integrity signals

Registry

Crossref · 10.3386/w6699

Cites

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Still being checked.

Cited by

Metadata

Authors
Heckman, J., Ichimura, H., Smith, J., & Todd, P.
Affiliations
  • University of Chicago (US)
  • The University of Tokyo (JP)
  • National Bureau of Economic Research (US)
Venue
National Bureau of Economic Research
Published
1998-08
OpenAlex
W2167047802
Cited by
696
References
0
Citations by year
2026: 9 · 2025: 25 · 2024: 30 · 2023: 23 · 2022: 30 · 2021: 47 · 2020: 48 · 2019: 23 · 2018: 27 · 2017: 29 · 2016: 28 · 2015: 37 · 2014: 30 · 2013: 27 · 2012: 39
Type
report
Language
en
Open access
Locations
National Bureau of Economic Research (publishedVersion)
Topics
  • Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
  • Social Sciences › Public Administration › Evaluation and Performance Assessment
  • Social Sciences › Education › School Choice and Performance
Keywords
selection bias, program evaluation, matching estimators, difference-in-differences, selection model, social programs, treatment effect estimation, semiparametric estimation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.