Estimating Heterogeneous Treatment Effects with Observational Data
Open accessThe DOI matches its Crossref record; 101 of the 101 references it cites were found in a registry or an index.
Abstract
Individuals differ not only in their background characteristics but also in how they respond to a particular treatment, intervention, or stimulation. In particular, treatment effects may vary systematically by the propensity for treatment. In this paper, we discuss a practical approach to studying heterogeneous treatment effects as a function of the treatment propensity, under the same assumption commonly underlying regression analysis: ignorability. We describe one parametric method and two nonparametric methods for estimating interactions between treatment and the propensity for treatment. For the first method, we begin by estimating propensity scores for the probability of treatment given a set of observed covariates for each unit and construct balanced propensity score strata; we then estimate propensity score stratum-specific average treatment effects and evaluate a trend across them. For the second method, we match control units to treated units based on the propensity score and transform the data into treatment-control comparisons at the most elementary level at which such comparisons can be constructed; we then estimate treatment effects as a function of the propensity score by fitting a nonparametric model as a smoothing device. For the third method, we first estimate nonparametric regressions of the outcome variable as a function of the propensity score separately for treated units and for control units and then take the difference between the two nonparametric regressions. We illustrate the application of these methods with an empirical example of the effects of college attendance on women’s fertility.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The central role of the propensity score in observational studies for causal effects
ROSENBAUM, P. R., & RUBIN, D. B. (1983) · Biometrika · doi:10.1093/biomet/70.1.41 · Cited by 31,241
Gelman, A., Carlin, J. B., Stern, H. S., & Rubin, D. B. (2003) · doi:10.1201/9780429258480 · Cited by 27,276
Hierarchical Linear Models: Applications and Data Analysis Methods.
Longford, N. T., Bryk, A. S., & Raudenbush, S. W. (1993) · Contemporary Sociology · doi:10.2307/2075823 · Cited by 18,724
Hierarchical Linear Models: Applications and Data Analysis Methods
Davidian, M. (2003) · Journal of the American Statistical Association · doi:10.1198/jasa.2003.s288 · Cited by 13,723
Causality: Models, Reasoning, and Inference
Pearl, J. (2009) · doi:10.1017/cbo9780511803161 · Cited by 11,653
Robust Locally Weighted Regression and Smoothing Scatterplots
Cleveland, W. S. (1979) · Journal of the American Statistical Association · doi:10.1080/01621459.1979.10481038 · Cited by 11,594
Mostly Harmless Econometrics: An Empiricist's Companion
Angrist, J. D., & Pischke, J.-S. (2009) · doi:10.1515/9781400829828 · Cited by 9,467
Estimating causal effects of treatments in randomized and nonrandomized studies
Rubin, D. B. (1974) · Journal of Educational Psychology · doi:10.1037/h0037350 · Cited by 9,436 · cited 1×
“Causal effects, treatment effects, heterogeneity, propensity scores, matching”
BackgroundJournal of The American Statistical Association
Fienberg, S. E. (2005) · Encyclopedia of Statistical Sciences · doi:10.1002/0471667196.ess1317 · Cited by 6,125
Matching as an econometric evaluation estimator: Evidence from evaluating a job training programme
Heckman, J. J., Ichimura, H., & Todd, P. E. (1997) · The Review of Economic Studies · doi:10.2307/2971733 · Cited by 5,486
Journal of the American Statistical Association
Koehler, A. B. (1995) · International Journal of Forecasting · doi:10.1016/0169-2070(95)90067-5 · Cited by 5,337
Statistics and Causal Inference
Holland, P. W. (1986) · Journal of the American Statistical Association · doi:10.1080/01621459.1986.10478354 · Cited by 5,293 · cited 1×
“The large methodological literature on causal inference recognizes the importance of and consequently allows for population heterogeneity (Heckman 2005; Holland 1986; Manski 1995; Rubin 1974; Winship and Morgan 1999).”
BackgroundMostly Harmless Econometrics: An Empiricist's Companion
Angrist, J. D., & Pischke, J.-S. (2008) · doi:10.2307/j.ctvcm4j72 · Cited by 5,051
Identification of Causal Effects Using Instrumental Variables
Angrist, J. D., Imbens, G. W., & Rubin, D. B. (1996) · Journal of the American Statistical Association · doi:10.1080/01621459.1996.10476902 · Cited by 4,230
Matching As An Econometric Evaluation Estimator
Heckman, J. J., Ichimura, H., & Todd, P. (1998) · Review of Economic Studies · doi:10.1111/1467-937x.00044 · Cited by 3,807
Bound, J., Jaeger, D. A., & Baker, R. M. (1995) · Journal of the American Statistical Association · doi:10.1080/01621459.1995.10476536 · Cited by 3,618
Muthén, B., & Muthén, L. K. (2000) · Alcoholism: Clinical and Experimental Research · doi:10.1111/j.1530-0277.2000.tb02070.x · Cited by 3,149
Modern Epidemiology 3rd edition
Rothman, K. J., Lash, T. L., & Greenland, S. (1986) · Cited by 3,096
Reducing Bias in Observational Studies Using Subclassification on the Propensity Score
Rosenbaum, P. R., & Rubin, D. B. (1984) · Journal of the American Statistical Association · doi:10.1080/01621459.1984.10478078 · Cited by 3,082
The Economics and Econometrics of Active Labor Market Programs
Heckman, J. J., Lalonde, R. J., & Smith, J. A. (1999) · Handbook of Labor Economics · doi:10.1016/s1573-4463(99)03012-6 · Cited by 3,059
Nonparametric Estimation of Average Treatment Effects Under Exogeneity: A Review
Imbens, G. W. (2004) · Review of Economics and Statistics · doi:10.1162/003465304323023651 · Cited by 2,994
Estimating Causal Effects from Large Data Sets Using Propensity Scores
Rubin, D. B. (1997) · Annals of Internal Medicine · doi:10.7326/0003-4819-127-8_part_2-199710151-00064 · Cited by 2,877
Local Polynomial Modelling and its Applications
Fan, J., & Gijbels, I. (1994) · Springer eBooks · doi:10.1007/978-1-4899-3150-4 · Cited by 2,849
Identification and Estimation of Local Average Treatment Effects
Angrist, J., & Imbens, G. (1995) · National Bureau of Economic Research · doi:10.3386/t0118 · Cited by 2,808
Heckman, J., & Singer, B. (1984) · Econometrica · doi:10.2307/1911491 · Cited by 2,788
Estimation of Average Treatment Effects Based on Propensity Scores
Becker, S. O., & Ichino, A. (2002) · The Stata Journal: Promoting communications on statistics and Stata · doi:10.1177/1536867x0200200403 · Cited by 2,754
Causal effects in nonexperimental studies: Reevaluating the evaluation of training programs
Dehejia, R. H., & Wahba, S. (1999) · Journal of the American Statistical Association · doi:10.1080/01621459.1999.10473858 · Cited by 2,395
Rosenbaum, P. R. (2002) · Springer Series in Statistics · doi:10.1007/978-1-4757-3692-2 · Cited by 2,394
The Growth of Biological Thought: Diversity, Evolution, and Inheritance
Mayr, E. (1982) · CERN Document Server (European Organization for Nuclear Research) · Cited by 2,295
(2012) · American Economic Review · doi:10.1257/aer.102.3.i · Cited by 2,235
Leuven, E., & Sianesi, B. (2003) · RePEc: Research Papers in Economics · Cited by 2,168
Dummy Endogenous Variables in a Simultaneous Equation System
Heckman, J. J. (1978) · Econometrica · doi:10.2307/1909757 · Cited by 1,838
Dummy Endogenous Variables in a Simultaneous Equation System
Heckman, J. (1977) · National Bureau of Economic Research · doi:10.3386/w0177 · Cited by 1,837
Bayesian Data Analysis, 2nd edn
Brix, A. (2005) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/j.1467-985x.2004.00347_4.x · Cited by 1,767
Empirical Strategies in Labor Economics
Angrist, J. D., & Krueger, A. B. (1999) · Handbook of Labor Economics · doi:10.1016/s1573-4463(99)03004-7 · Cited by 1,628
Principal Stratification in Causal Inference
Frangakis, C. E., & Rubin, D. B. (2002) · Biometrics · doi:10.1111/j.0006-341x.2002.00021.x · Cited by 1,511
(2007) · Cited by 1,428
Structural Equations, Treatment Effects, and Econometric Policy Evaluation1
Heckman, J. J., & Vytlacil, E. (2005) · Econometrica · doi:10.1111/j.1468-0262.2005.00594.x · Cited by 1,220
Local Polynomial Modelling and its Applications
Copeland, K. A. F. (1997) · Journal of Quality Technology · doi:10.1080/00224065.1997.11979762 · Cited by 1,195
Counterfactuals and Causal Inference: Methods and Principles for Social Research
Morgan, S. L., & Winship, C. (2007) · doi:10.1017/cbo9780511804564 · Cited by 1,146
Investigating population heterogeneity with factor mixture models.
Lubke, G. H., & Muthén, B. (2005) · Psychological Methods · doi:10.1037/1082-989x.10.1.21 · Cited by 1,143
Identification Problems in the Social Sciences
Manski, C. F. (1993) · Sociological Methodology · doi:10.2307/271005 · Cited by 1,130
Willis, R., & Rosen, S. (1978) · National Bureau of Economic Research · doi:10.3386/w0249 · Cited by 1,110
Diamond, A., & Sekhon, J. S. (2013) · Review of Economics and Statistics · doi:10.1162/rest_a_00318 · Cited by 1,102
Alternative methods for evaluating the impact of interventions
Heckman, J. J., & Robb, R. (1985) · Longitudinal Analysis of Labor Market Data · doi:10.1017/ccol0521304539.004 · Cited by 1,101
Bauer, D. J., & Curran, P. J. (2003) · Psychological Methods · doi:10.1037/1082-989x.8.3.338 · Cited by 1,059
Misunderstandings Between Experimentalists and Observationalists about Causal Inference
Imai, K., King, G., & Stuart, E. A. (2008) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/j.1467-985x.2007.00527.x · Cited by 906
THE ESTIMATION OF CAUSAL EFFECTS FROM OBSERVATIONAL DATA
Winship, C., & Morgan, S. L. (1999) · Annual Review of Sociology · doi:10.1146/annurev.soc.25.1.659 · Cited by 890 · cited 3×
“The large methodological literature on causal inference recognizes the importance of and consequently allows for population heterogeneity (Heckman 2005; Holland 1986; Manski 1995; Rubin 1974; Winship and Morgan 1999).”
BackgroundStatistical Methods for Categorical Data Analysis
Powers, D. A. (1999) · Cited by 880
Understanding Instrumental Variables in Models with Essential Heterogeneity
Heckman, J. J., Urzua, S., & Vytlacil, E. (2006) · Review of Economics and Statistics · doi:10.1162/rest.88.3.389 · Cited by 874
Smoking and Lung Cancer: Recent Evidence and a Discussion of Some Questions
(1959) · JNCI: Journal of the National Cancer Institute · doi:10.1093/jnci/22.1.173 · Cited by 841
A Hierarchical Model for Studying School Effects
Raudenbush, S., & Bryk, A. S. (1986) · Sociology of Education · doi:10.2307/2112482 · Cited by 818
Willis, R. J., & Rosen, S. (1979) · Journal of Political Economy · doi:10.1086/260821 · Cited by 786
Journal of the National Cancer Institute
(1958) · JNCI: Journal of the National Cancer Institute · doi:10.1093/jnci/20.1.227 · Cited by 770
DiPrete, T. A., & Gangl, M. (2004) · Sociological Methodology · doi:10.1111/j.0081-1750.2004.00154.x · Cited by 732
Micro Data, Heterogeneity, and the Evaluation of Public Policy: Nobel Lecture
Heckman, J. J. (2001) · Journal of Political Economy · doi:10.1086/322086 · Cited by 675
Harding, D. J. (2003) · American Journal of Sociology · doi:10.1086/379217 · Cited by 644
Heckman, J. J., & Vytlacil, E. J. (1999) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.96.8.4730 · Cited by 627
Estimating Marginal Returns to Education
Carneiro, P., Heckman, J. J., & Vytlacil, E. J. (2011) · American Economic Review · doi:10.1257/aer.101.6.2754 · Cited by 612
Identification problems in the social sciences
(1994) · Mathematical Social Sciences · doi:10.1016/0165-4896(94)90027-2 · Cited by 608
Matching to Remove Bias in Observational Studies
(2006) · Matched Sampling for Causal Effects · doi:10.1017/cbo9780511810725.007 · Cited by 600
Brand, J. E., & Xie, Y. (2010) · American Sociological Review · doi:10.1177/0003122410363567 · Cited by 587
Self-citationAngrist, J. D. (1990) · American Economic Review · Cited by 547
Identification for Prediction and Decision
Manski, C. F. (2009) · Harvard University Press eBooks · doi:10.2307/j.ctv219kxm0 · Cited by 502
7. Multilevel Latent Class Models
Vermunt, J. K. (2003) · Sociological Methodology · doi:10.1111/j.0081-1750.2003.t01-1-00131.x · Cited by 460
The Estimation of Wage Gains and Welfare Gains in Self-Selection Models
Bjorklund, A., & Moffitt, R. (1987) · The Review of Economics and Statistics · doi:10.2307/1937899 · Cited by 452
The Use of Matched Sampling and Regression Adjustment to Remove Bias in Observational Studies
(2006) · Matched Sampling for Causal Effects · doi:10.1017/cbo9780511810725.008 · Cited by 442
D'Unger, A., Land, K. C., McCall, P. L., & Nagin, D. S. (1998) · American Journal of Sociology · doi:10.1086/231402 · Cited by 420
1. The Scientific Model of Causality
Heckman, J. J. (2005) · Sociological Methodology · doi:10.1111/j.0081-1750.2006.00164.x · Cited by 401
Causal Inference in Sociological Research
Gangl, M. (2010) · Annual Review of Sociology · doi:10.1146/annurev.soc.012809.102702 · Cited by 401
Matching Estimators of Causal Effects: Prospects and Pitfalls in Theory and Practice
Morgan, S. L., & Harding, D. J. (2006) · Sociological Methods & Research · doi:10.1177/0049124106289164 · Cited by 392
Blundell, R., Dearden, L., & Sianesi, B. (2005) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/j.1467-985x.2004.00360.x · Cited by 343
After Admission: From College Access to College Success
Rosenbaum, J. E., Deil‐Amen, R., & Person, A. E. (2007) · Cited by 303
Fitting Categorical Models to Effect Sizes from a Series of Experiments
Hedges, L. V. (1982) · Journal of Educational Statistics · doi:10.2307/1164961 · Cited by 296
Identification of Causal Effects Using Instrumental Variables: Comment
Robins, J. M., & Greenland, S. (1996) · Journal of the American Statistical Association · doi:10.2307/2291630 · Cited by 290
Policy-Relevant Treatment Effects
Heckman, J. J., & Vytlacil, E. (2001) · American Economic Review · doi:10.1257/aer.91.2.107 · Cited by 285
Counterfactuals, Causal Effect Heterogeneity, and the Catholic School Effect on Learning
Morgan, S. L. (2001) · Sociology of Education · doi:10.2307/2673139 · Cited by 269
6. A Diagnostic Routine for the Detection of Consequential Heterogeneity of Causal Effects
Morgan, S. L., & Todd, J. J. (2008) · Sociological Methodology · doi:10.1111/j.1467-9531.2008.00204.x · Cited by 207
Brand, J. E., & Halaby, C. N. (2006) · Social Science Research · doi:10.1016/j.ssresearch.2005.06.006 · Cited by 193
Self-citationThe Impact of College Education on Fertility: Evidence for Heterogeneous Effects
Brand, J. E., & Davis, D. (2011) · Demography · doi:10.1007/s13524-011-0034-3 · Cited by 163
Self-citationCivic Returns to Higher Education: A Note on Heterogeneous Effects
Brand, J. E. (2010) · Social Forces · doi:10.1353/sof.2010.0095 · Cited by 156
Self-citationInvariants and noninvariants in the concept of interdependent effects.
Greenland, S., & Poole, C. (1988) · Scandinavian Journal of Work, Environment & Health · doi:10.5271/sjweh.1945 · Cited by 150
Readings in Mathematical Social Science
Künzi, H. P., Lazarsfeld, P. F., & Henry, N. W. (1968) · Econometrica · doi:10.2307/1909535 · Cited by 144
Notes on Social Measurement: Historical and Critical
Bogue, A. G., & Duncan, O. D. (1986) · The Journal of Interdisciplinary History · doi:10.2307/204809 · Cited by 138
Population heterogeneity and causal inference
Xie, Y. (2013) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1303102110 · Cited by 137
Self-citationHeuristics, Probability and Causality. A Tribute to Judea Pearl
Dechter, R., Geffner, H., & Halpern, J. Y. (2010) · Cited by 136
Cognitive Ability, Wages, and Meritocracy
Cawley, J., Conneely, K., Heckman, J. J., & Vytlacil, E. (1996) · Springer eBooks · doi:10.1007/978-1-4612-0669-9_8 · Cited by 135
Causal Inference in the Social Sciences
Sobel, M. E. (2000) · Journal of the American Statistical Association · doi:10.1080/01621459.2000.10474243 · Cited by 127
Matching on the Estimated Propensity Score
Abadie, A., & Imbens, G. W. (2009) · National Bureau of Economic Research · doi:10.3386/w15301 · Cited by 127
Bayesian Factor Analysis for Multilevel Binary Observations
Ansari, A., & Jedidi, K. (2000) · Psychometrika · doi:10.1007/bf02296339 · Cited by 116
Matching Estimators of Causal Effects
Harding, D. J. (2007) · Cambridge University Press eBooks · doi:10.1017/cbo9780511804564.004 · Cited by 84
Into Adulthood: A Study of the Effects of Head Start
Oden, S. L., Schweinhart, L. J., & Weikart, D. P. (2000) · Cited by 66
Otis Dudley Duncan's legacy: The demographic approach to quantitative reasoning in social science
Xie, Y. (2007) · Research in Social Stratification and Mobility · doi:10.1016/j.rssm.2007.05.006 · Cited by 58
Self-citationVoluntary and involuntary job mobility and earnings inequality in urban China, 1993–2000
Wu, X. (2009) · Social Science Research · doi:10.1016/j.ssresearch.2009.11.003 · Cited by 45
Reply to Jann: Market Premium, Social Process, and Statisticism
Xie, Y., & Wu, X. (2005) · American Sociological Review · doi:10.1177/000312240507000508 · Cited by 40
Self-citationLatent class analysis of complex sample survey data : Application to dietary data
Vermunt, J. K. (2002) · Quarterly Publications of the American Statistical Association · Cited by 40
Brand, J. E., & Xie, Y. (2007) · Sociological Methodology · doi:10.1111/j.1467-9531.2007.00185.x · Cited by 38
Self-citationEffect Heterogeneity and Bias in Main-Effects-Only Regression Models
Winship, C., Elwert, F., Dechter, R., Geffner, H., & Halpern, J. Y. (2010) · Cited by 30
Changes in Earnings Returns to Higher Education in Taiwan since the 1990s
Tsai, S., & Xie, Y. (2008) · Population review · doi:10.1353/prv.0.0003 · Cited by 29
Self-citationHeterogeneity in returns to college education: Selection bias in contemporary Taiwan
Tsai, S., & Xie, Y. (2010) · Social Science Research · doi:10.1016/j.ssresearch.2010.12.008 · Cited by 26
Self-citationA New Non-Parametric Matching Method for Bias Adjustment with Applications to Economic EvaluationsPreprint
Sekhon, J. S., & Grieve, R. (2007) · SSRN Electronic Journal · doi:10.2139/ssrn.1138926 · Cited by 24
Cited by
Bapna, R., & Umyarov, A. (2015) · Management Science · doi:10.1287/mnsc.2014.2081 · Cited by 350
Verhofstadt, E., & Maertens, M. (2015) · Applied Economic Perspectives and Policy · doi:10.1093/aepp/ppu021 · Cited by 271
Conformal Inference of Counterfactuals and Individual Treatment Effects
Lei, L., & Candès, E. J. (2021) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/rssb.12445 · Cited by 166
Averitt, A. J., Weng, C., Ryan, P., & Perotte, A. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0277-8 · Cited by 137
Mayer, A., Zimmermann, J., Hoyer, J., Salzer, S., Wiltink, J., Leibing, E., & Leichsenring, F. (2020) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705511.2019.1671196 · Cited by 21
Metadata
- Authors
- Xie, Y., Brand, J. E., & Jann, B.
- Venue
- Sociological Methodology
- Publisher
- SAGE Publications
- Published
- 2012-08
- Volume
- 42
- Issue
- 1
- Pages
- 314-347
- ISSN
- 0081-1750 (ISSN-L)
- 1467-9531
- OpenAlex
- W2101626831
- Cited by
- 333
- References
- 101
- FWCI
- 16.48
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.