The DOI matches its Crossref record; 43 of the 43 references it cites were found in a registry or an index.
Least Squares After Model Selection in High-Dimensional Sparse Models
Open accessAbstract
In this article we study post-model selection estimators that apply ordinary least squares (OLS) to the model selected by first-step penalized estimators, typically Lasso. It is well known that Lasso can estimate the nonparametric regression function at nearly the oracle rate, and is thus hard to improve upon. We show that the OLS post-Lasso estimator performs at least as well as Lasso in terms of the rate of convergence, and has the advantage of a smaller bias. Remarkably, this performance occurs even if the Lasso-based model selection “fails” in the sense of missing some components of the “true” regression model. By the “true” model, we mean the best $s$-dimensional approximation to the nonparametric regression function chosen by the oracle. Furthermore, OLS post-Lasso estimator can perform strictly better than Lasso, in the sense of a strictly faster rate of convergence, if the Lasso-based model selection correctly includes all components of the “true” model as a subset and also achieves sufficient sparsity. In the extreme case, when Lasso perfectly selects the “true” model, the OLS post-Lasso estimator becomes the oracle estimator. An important ingredient in our analysis is a new sparsity bound on the dimension of the model selected by Lasso, which guarantees that this dimension is at most of the same order as the dimension of the “true” model. Our rate results are nonasymptotic and hold in both parametric and nonparametric models. Moreover, our analysis is not limited to the Lasso estimator acting as a selector in the first step, but also applies to any other estimator, for example, various forms of thresholded Lasso, with good rates and good sparsity properties. Our analysis covers both traditional thresholding and a new practical, data-driven thresholding scheme that induces additional sparsity subject to maintaining a certain goodness of fit. The latter scheme has theoretical guarantees similar to those of Lasso or OLS post-Lasso, but it dominates those procedures as well as traditional thresholding in a wide variety of experiments.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Regression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482
Estimating the Dimension of a Model
Schwarz, G. (1978) · The Annals of Statistics · doi:10.1214/aos/1176344136 · Cited by 47,033
Ideal spatial adaptation by wavelet shrinkage
Donoho, D. L., & Johnstone, I. M. (1994) · Biometrika · doi:10.1093/biomet/81.3.425 · Cited by 8,760
Weak Convergence and Empirical Processes
van der Vaart, A. W., & Wellner, J. A. (1996) · Springer Series in Statistics · doi:10.1007/978-1-4757-2545-2 · Cited by 5,014
Sure Independence Screening for Ultrahigh Dimensional Feature Space
Fan, J., & Lv, J. (2008) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2008.00674.x · Cited by 2,711
Donoho, D. L. (2006) · Communications on Pure and Applied Mathematics · doi:10.1002/cpa.20132 · Cited by 2,490
Introduction to Nonparametric Estimation
Tsybakov, A. B. (2009) · Springer Series in Statistics · doi:10.1007/b13794 · Cited by 2,486
Simultaneous analysis of Lasso and Dantzig selector
Bickel, P. J., Ritov, Y., & Tsybakov, A. B. (2009) · The Annals of Statistics · doi:10.1214/08-aos620 · Cited by 2,396
Robust Regression: Asymptotics, Conjectures and Monte Carlo
Huber, P. J. (1973) · The Annals of Statistics · doi:10.1214/aos/1176342503 · Cited by 2,328
Stable recovery of sparse overcomplete representations in the presence of noise
Donoho, D. L., Elad, M., & Temlyakov, V. N. (2006) · IEEE Transactions on Information Theory · doi:10.1109/tit.2005.860430 · Cited by 2,183
Cited by
Double/debiased machine learning for treatment and structural parameters
Chernozhukov, V., Chetverikov, D., Demirer, M., Duflo, E., Hansen, C., Newey, W., & Robins, J. (2018) · The Econometrics Journal · doi:10.1111/ectj.12097 · Cited by 2,946
Self-citationInference on Treatment Effects after Selection among High-Dimensional Controls
Belloni, A., Chernozhukov, V., & Hansen, C. (2014) · The Review of Economic Studies · doi:10.1093/restud/rdt044 · Cited by 1,508
Self-citationCovariate Balancing Propensity Score
Imai, K., & Ratkovic, M. (2014) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/rssb.12027 · Cited by 1,315
Sparse Models and Methods for Optimal Instruments With an Application to Eminent Domain
(2012) · Econometrica · doi:10.3982/ecta9626 · Cited by 889
On asymptotically optimal confidence regions and tests for high-dimensional models
van de Geer, S., Bühlmann, P., Ritov, Y., & Dezeure, R. (2014) · The Annals of Statistics · doi:10.1214/14-aos1221 · Cited by 822
High-Dimensional Methods and Inference on Structural and Treatment Effects
Belloni, A., Chernozhukov, V., & Hansen, C. (2014) · Journal of Economic Perspectives · doi:10.1257/jep.28.2.29 · Cited by 722
Self-citationTaming the Factor Zoo: A Test of New Factors
FENG, G., GIGLIO, S., & XIU, D. (2020) · The Journal of Finance · doi:10.1111/jofi.12883 · Cited by 707
Confidence Intervals and Hypothesis Testing for High-Dimensional Regression
Javanmard, A., & Andrea Montanari, A. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1306.3171 · Cited by 635
Double/Debiased Machine Learning for Treatment and Structural Parameters
Chernozhukov, V., Chetverikov, D., Demirer, M., Duflo, E., Hansen, C., Newey, W., & Robins, J. (2017) · National Bureau of Economic Research · doi:10.3386/w23564 · Cited by 535
Self-citationSources of Geographic Variation in Health Care: Evidence From Patient Migration*
Finkelstein, A., Gentzkow, M., & Williams, H. (2016) · The Quarterly Journal of Economics · doi:10.1093/qje/qjw023 · Cited by 500
Double/debiased machine learning for treatment and structural parameters
Newey, W. K., Hansen, C., Duflo, E., Robins, J., Chernozhukov, V., Demirer, M., & Chetverikov, D. (2017) · doi:10.1920/wp.cem.2017.2817 · Cited by 486
Self-citationMcNeish, D. M. (2015) · Multivariate Behavioral Research · doi:10.1080/00273171.2015.1036965 · Cited by 432
The Welfare Effects of Nudges: A Case Study of Energy Use Social Comparisons
Allcott, H., & Kessler, J. B. (2019) · American Economic Journal: Applied Economics · doi:10.1257/app.20170328 · Cited by 386
Robust inference on average treatment effects with possibly more covariates than observations
Farrell, M. H. (2015) · Journal of Econometrics · doi:10.1016/j.jeconom.2015.06.017 · Cited by 320
Chernozhukov, V., Chetverikov, D., & Kato, K. (2013) · The Annals of Statistics · doi:10.1214/13-aos1161 · Cited by 320
Self-citationProgram evaluation and causal inference with high-dimensional data
Belloni, A., Chernozhukov, V., Fern�ndez-Val, I., & Hansen, C. (2017) · Econometrica · doi:10.3982/ecta12723 · Cited by 317
Self-citationChernozhukov, V., Chetverikov, D., & Kato, K. (2013) · doi:10.1920/wp.cem.2013.7613 · Cited by 312
Self-citationMachine Learning and Portfolio Optimization
Ban, G.-Y., El Karoui, N., & Lim, A. E. B. (2018) · Management Science · doi:10.1287/mnsc.2016.2644 · Cited by 282
Using Gossips to Spread Information: Theory and Evidence from Two Randomized Controlled Trials
Banerjee, A., Chandrasekhar, A. G., Duflo, E., & Jackson, M. O. (2019) · The Review of Economic Studies · doi:10.1093/restud/rdz008 · Cited by 272
Identifying Latent Structures in Panel Data
Su, L., Shi, Z., & Phillips, P. C. B. (2016) · Econometrica · doi:10.3982/ecta12560 · Cited by 265
Lassopack: model selection and prediction with regularized regression in Stata
Ahrens, A., Hansen, C. B., & Schaffer, M. E. (2020) · The Stata Journal Promoting communications on statistics and Stata · doi:10.1177/1536867x20909697 · Cited by 212
Lasso adjustments of treatment effect estimates in randomized experiments
Bloniarz, A., Liu, H., Zhang, C.-H., Sekhon, J. S., & Yu, B. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1510506113 · Cited by 169
On the Use of the Lasso for Instrumental Variables Estimation with Some Invalid Instruments
Windmeijer, F., Farbmacher, H., Davies, N., & Davey Smith, G. (2019) · Journal of the American Statistical Association · doi:10.1080/01621459.2018.1498346 · Cited by 144
“(Bowden et al. (2015) and Kolesar et al. (2015) allowed for all instruments to be invalid and showed that the causal effect can be consistently estimated if the number of instruments increases with the sample size under the assumption of uncorrelatedness of the instrument strength and their direct effects on the outco…”
Efficient Difference-in-Differences Estimation with High-Dimensional Common Trend Confounding
Zimmert, M. (2018) · arXiv · doi:10.48550/arxiv.1809.01643 · Cited by 9
Metadata
- Authors
- Belloni, A., & Chernozhukov, V.
- Venue
- SSRN Electronic Journal
- Publisher
- Elsevier BV
- Published
- 2009
- Volume
- 19
- Issue
- 2
- Pages
- 521-547
- ISSN
- 1350-7265 (ISSN-L)
- 1556-5068
- OpenAlex
- W3103221895
- Cited by
- 683
- References
- 43
- FWCI
- 1.91
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.