The DOI matches its Crossref record; 42 of the 43 references it cites were found in a registry or an index.
Recursive partitioning for heterogeneous causal effects
Open accessAbstract
In this paper we propose methods for estimating heterogeneity in causal effects in experimental and observational studies and for conducting hypothesis tests about the magnitude of differences in treatment effects across subsets of the population. We provide a data-driven approach to partition the data into subpopulations that differ in the magnitude of their treatment effects. The approach enables the construction of valid confidence intervals for treatment effects, even with many covariates relative to the sample size, and without “sparsity” assumptions. We propose an “honest” approach to estimation, whereby one sample is used to construct the partition and another to estimate treatment effects for each subpopulation. Our approach builds on regression tree methods, modified to optimize for goodness of fit in treatment effects and to account for honest estimation. Our model selection criterion anticipates that bias will be eliminated by honest estimation and also accounts for the effect of making additional splits on the variance of treatment effect estimates within each subpopulation. We address the challenge that the “ground truth” for a causal effect is not observed for any individual unit, so that standard approaches to cross-validation must be modified. Through a simulation study, we show that for our preferred method honest estimation results in nominal coverage for 90% confidence intervals, whereas coverage ranges between 74% and 84% for nonhonest approaches. Honest estimation requires estimating the model with a smaller sample size; the cost in terms of mean squared error of treatment effects for our preferred method ranges between 7–22%.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In particular, Athey and Imbens (2016), Wager and Athey (2018), and Athey et al. (2019) developed random forests methods designed to capture honest heterogeneity in treatment effects, circumventing overfitting issues inherent to traditional methods and thus avoiding mistaking statistical noise for heterogeneity.”
From Prison Rehabilitation Programs: Efficiency and Targeting (2021) · cites this paper as Athey and Imbens (2016)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
Regression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482
The central role of the propensity score in observational studies for causal effects
ROSENBAUM, P. R., & RUBIN, D. B. (1983) · Biometrika · doi:10.1093/biomet/70.1.41 · Cited by 31,241
Wu, Y., & Vapnik, V. N. (1999) · Technometrics · doi:10.2307/1271368 · Cited by 27,236
The Elements of Statistical Learning
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7 · Cited by 25,970
Classification and Regression Trees.
Gordon, A. D., Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1984) · Biometrics · doi:10.2307/2530946 · Cited by 23,867
Classification and Regression Trees.
Ryzin, J. V., Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1986) · Journal of the American Statistical Association · doi:10.2307/2288003 · Cited by 21,030
Regularization Paths for Generalized Linear Models via Coordinate Descent
Friedman, J., Hastie, T., & Tibshirani, R. (2010) · Journal of Statistical Software · doi:10.18637/jss.v033.i01 · Cited by 19,927
The Elements of Statistical Learning: Data Mining, Inference, and Prediction
Lu, Z. Q. J. (2010) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/j.1467-985x.2010.00646_6.x · Cited by 12,916
Causality: Models, Reasoning, and Inference
Pearl, J. (2009) · doi:10.1017/cbo9780511803161 · Cited by 11,653
Estimating causal effects of treatments in randomized and nonrandomized studies
Rubin, D. B. (1974) · Journal of Educational Psychology · doi:10.1037/h0037350 · Cited by 9,436
Statistics and Causal Inference
Holland, P. W. (1986) · Journal of the American Statistical Association · doi:10.1080/01621459.1986.10478354 · Cited by 5,293
A generalization of sampling without replacement from a finite universe
Horvitz, D. G., & Thompson, D. J. (1952) · Journal of the American Statistical Association · doi:10.1080/01621459.1952.10483446 · Cited by 4,293
Causal Inference for Statistics, Social, and Biomedical Sciences: An Introduction
Imbens, G. W., & Rubin, D. B. (2015) · doi:10.1017/cbo9781139025751 · Cited by 4,210
Self-citationThe Central Role of the Propensity Score in Observational Studies for Causal Effects
Rosenbaum, P. R., & Rubin, D. B. (1983) · Biometrika · doi:10.2307/2335942 · Cited by 3,495
Causality: models, reasoning, and inference
(2000) · Choice Reviews Online · doi:10.5860/choice.38-1494a · Cited by 2,968
Large sample properties of matching estimators for average treatment effects
Abadie, A., & Imbens, G. W. (2006) · Econometrica · doi:10.1111/j.1468-0262.2006.00655.x · Cited by 2,908
Self-citationEfficient Estimation of Average Treatment Effects Using the Estimated Propensity Score
Hirano, K., Imbens, G. W., & Ridder, G. (2003) · Econometrica · doi:10.1111/1468-0262.00442 · Cited by 2,602
Self-citationRosenbaum, P. R. (2002) · Springer series in statistics · doi:10.1007/978-1-4757-3692-2 · Cited by 2,543
Bayesian inference for causal effects: The role of randomization
Rubin, D. B. (1978) · The Annals of Statistics · doi:10.1214/aos/1176344064 · Cited by 2,503
van der Laan, M. J., & Rose, S. (2011) · Springer series in statistics · doi:10.1007/978-1-4419-9782-1 · Cited by 1,198
Bishop, C. (2004) · Institute of Physics eBooks · doi:10.1887/0750303123/b365c43 · Cited by 859
Causal Inference for Statistics, Social, and Biomedical Sciences: An Introduction
Imbens, G. W., & Rubin, D. B. (2015) · Thammasat University Digital Collections · Cited by 838
Self-citationModel-Based Recursive Partitioning
Zeileis, A., Hothorn, T., & Hornik, K. (2008) · Journal of Computational and Graphical Statistics · doi:10.1198/106186008x319331 · Cited by 771
Doubly Robust Policy Evaluation and LearningPreprint
Dudı́k, M., Langford, J. C., & Li, L. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1103.4601 · Cited by 571
Subgroup identification from randomized clinical trial data
Foster, J. C., Taylor, J. M. G., & Ruberg, S. J. (2011) · Statistics in Medicine · doi:10.1002/sim.4322 · Cited by 559
Rosenbaum, P. R. (2002) · Springer Series in Statistics · doi:10.1007/978-1-4757-3692-2_1 · Cited by 484
Efficient Estimation of Average Treatment Effects Using the Estimated Propensity ScorePreprint
Hirano, K., Imbens, G. W., & Ridder, G. (2002) · SSRN Electronic Journal · doi:10.2139/ssrn.324940 · Cited by 480
Self-citationEstimating treatment effect heterogeneity in randomized program evaluation
Imai, K., & Ratkovic, M. (2013) · The Annals of Applied Statistics · doi:10.1214/12-aoas593 · Cited by 479
A Simple Method for Estimating Interactions Between a Treatment and a Large Number of Covariates
Tian, L., Alizadeh, A. A., Gentles, A. J., & Tibshirani, R. J. (2014) · Journal of the American Statistical Association · doi:10.1080/01621459.2014.951443 · Cited by 379
Subgroup Analysis via Recursive Partitioning
Su, X., Tsai, C., Wang, H., Nickerson, D. M., Li, B., & Rosset, S. (2009) · Cited by 310
GREEN, D. P., & Kern, H. (2012) · Public Opinion Quarterly · doi:10.1093/poq/nfs036 · Cited by 251
Nonparametric Tests for Treatment Effect Heterogeneity
Crump, R. K., Hotz, V. J., Imbens, G. W., & Mitnik, O. A. (2008) · The Review of Economics and Statistics · doi:10.1162/rest.90.3.389 · Cited by 236
Self-citationMultiple Hypothesis Testing in Experimental Economics
List, J. A., Shaikh, A. M., & Xu, Y. (2016) · National Bureau of Economic Research · doi:10.3386/w21875 · Cited by 188
Subgroup Analysis via Recursive PartitioningPreprint
Su, X., Tsai, C., Wang, H., Nickerson, D. M., & Li, B. (2009) · SSRN Electronic Journal · doi:10.2139/ssrn.1341380 · Cited by 150
The offset tree for learning with partial labels
Beygelzimer, A., & Langford, J. C. (2009) · ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/1557019.1557040 · Cited by 130
Causal Inference for Statistics, Social, and Biomedical Sciences: An Introduction
Blumberg, C. J. (2016) · International Statistical Review · doi:10.1111/insr.12170 · Cited by 110
Optimizing randomized trial designs to distinguish which subpopulations benefit from treatment
Rosenblum, M., & van der Laan, M. J. (2011) · Biometrika · doi:10.1093/biomet/asr055 · Cited by 62
Post hoc subgroups in clinical trials: Anathema or analytics?
Weisberg, H. I., & Pontes, V. P. (2015) · Clinical Trials · doi:10.1177/1740774515588096 · Cited by 38
Uniform Convergence of Random Forests via Adaptive ConcentrationPreprint
Wager, S., & Walther, G. (2015) · arXiv (Cornell University) · Cited by 8
Heterogeneous Treatment Effects in Digital ExperimentationPreprint
Taddy, M. A., Gardner, M., Chen, L., & Draper, D. (2014) · arXiv (Cornell University) · Cited by 7
Estimation and Inference of Heterogeneous Treatment Effects using Random ForestsPreprint
Wager, S., & Athey, S. C. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1510.04342 · Cited by 6
Self-citation
Research works printed but not linked: 1
- J Sigovitch Identifying informative biological markers in high-dimensional genomic data and clinical trials. PhD thesis (Harvard Univ Cambridge MA). (2007).
Cited by
Sorted by citations
Estimation and Inference of Heterogeneous Treatment Effects using Random Forests
Wager, S., & Athey, S. (2018) · Journal of the American Statistical Association · doi:10.1080/01621459.2017.1319839 · Cited by 2,917
Self-citationMachine Learning: An Applied Econometric Approach
Mullainathan, S., & Spiess, J. (2017) · Journal of Economic Perspectives · doi:10.1257/jep.31.2.87 · Cited by 1,989
Athey, S. C., Tibshirani, J., & Wager, S. (2019) · The Annals of Statistics · doi:10.1214/18-aos1709 · Cited by 2,108
Self-citationThe State of Applied Econometrics: Causality and Policy Evaluation
Athey, S. C., & Imbens, G. W. (2017) · The Journal of Economic Perspectives · doi:10.1257/jep.31.2.3 · Cited by 1,406
Self-citationGentzkow, M. A., Kelly, B., & Taddy, M. A. (2019) · Journal of Economic Literature · doi:10.1257/jel.20181020 · Cited by 1,353
Metadata
- Authors
- Athey, S., & Imbens, G.
- Affiliations
- Stanford University (US)
- Venue
- Proceedings of the National Academy of Sciences
- Publisher
- National Academy of Sciences
- Published
- 2016-07-05
- Volume
- 113
- Issue
- 27
- Pages
- 7353-7360
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- PMC
- PMC4941430
- OpenAlex
- W2305754340
- Cited by
- 1,586
- References
- 43
- Citations by year
- 2026: 213 · 2025: 235 · 2024: 198 · 2023: 175 · 2022: 170 · 2021: 178 · 2020: 161 · 2019: 110 · 2018: 66 · 2017: 61 · 2016: 15
- FWCI
- 87.13
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Proceedings of the National Academy of Sciences (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Deep Blue (University of Michigan) (submittedVersion)
- Funders
- Microsoft Research (0)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Inference
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Keywords
- treatment effect heterogeneity, recursive partitioning, regression trees, cross-validation, model selection criteria, high-dimensional covariates, causal inference
- MeSH terms
- Machine Learning, Computer Simulation, Models, Theoretical, Research Design, Statistics as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
