Bolasso: model consistent Lasso estimation through the bootstrap
The DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.
Abstract
We consider the least-square linear regression problem with regularization by the l1-norm, a problem usually referred to as the Lasso. In this paper, we present a detailed asymptotic analysis of model consistency of the Lasso. For various decays of the regularization parameter, we compute asymptotic equivalents of the probability of correct model selection (i.e., variable selection). For a specific rate decay, we show that the Lasso selects all the variables that should enter the model with probability tending to one exponentially fast, while it selects all other variables with strictly positive probability. We show that this property implies that if we run the Lasso for several bootstrapped replications of a given sample, then intersecting the supports of the Lasso bootstrap estimates leads to consistent model selection. This novel variable selection algorithm, referred to as the Bolasso, is compared favorably to other linear regression methods on synthetic data and datasets from the UCI machine learning repository.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Regression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482
An Introduction to the Bootstrap
Efron, B., & Tibshirani, R. J. (1994) · doi:10.1201/9780429246593 · Cited by 40,500
UCI Machine Learning Repository
Asuncion, A. (2007) · Medical Entomology and Zoology · Cited by 25,410
Breiman, L. (1996) · Machine Learning · doi:10.1007/bf00058655 · Cited by 16,325 · cited 5×
“…linear regression problems and compare it to alternative methods for linear regression, namely, ridge regression, Lasso, bagging of Lasso estimates (Breiman, 1996a), and a soft version of the Bolasso (referred to as Bolasso-S), where instead of intersecting the supports for each bootstrap…”
MethodInfluentialEfron, B., Hastie, T., Johnstone, I., & Tibshirani, R. (2004) · The Annals of Statistics · doi:10.1214/009053604000000067 · Cited by 9,357
The adaptive lasso and its oracle properties
Zou, H. (2006) · Journal of the American Statistical Association · doi:10.1198/016214506000000735 · Cited by 7,470 · cited 9×
“…exclusive possible situations which explain various portions of the regularization path (we assume ( A1-3 )); many of these results appear elsewhere (Yuan & Lin, 2007; Zhao & Yu, 2006; Fu & Knight, 2000; Zou, 2006; Bach, 2007) but some of the finer results presented below are new (see Section 2.4).”
BackgroundMethodResultInfluentialAsymptotics for lasso-type estimators
Fu, W., & Knight, K. (2000) · The Annals of Statistics · doi:10.1214/aos/1015957397 · Cited by 1,276
Wainwright, M. J. (2009) · IEEE Transactions on Information Theory · doi:10.1109/tit.2009.2016018 · Cited by 1,198
Heuristics of instability and stabilization in model selection
Breiman, L. (1996) · The Annals of Statistics · doi:10.1214/aos/1032181158 · Cited by 1,146
On Model Selection Consistency of Lasso
ZhaoPeng, & Yubin, Y. (2006) · Journal of Machine Learning Research · doi:10.5555/1248547.1248637 · Cited by 996
Lasso-type recovery of sparse representations for high-dimensional data
Meinshausen, N., & Yu, B. (2006) · Cited by 744
Consistency of the group Lasso and multiple kernel learning
Bach, F. (2007) · arXiv (Cornell University) · doi:10.48550/arxiv.0707.3390 · Cited by 669
Self-citationBoosting for high-dimensional linear models
Bühlmann, P. (2006) · The Annals of Statistics · doi:10.1214/009053606000000092 · Cited by 410
On the Non-Negative Garrotte Estimator
Yuan, M., & Lin, Y. (2007) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2007.00581.x · Cited by 264
On the dependence of the Berry–Esseen bound on dimension
Bentkus, V. (2003) · Journal of Statistical Planning and Inference · doi:10.1016/s0378-3758(02)00094-0 · Cited by 249
Sup-norm convergence rate and sign concentration property of Lasso and Dantzig estimatorsPreprint
Lounici, K. (2008) · HAL · Cited by 158
Cited by
Nearly unbiased variable selection under minimax concave penalty
Zhang, C.-H. (2010) · The Annals of Statistics · doi:10.1214/09-aos729 · Cited by 3,835
Meinshausen, N., & Bühlmann, P. (2010) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2010.00740.x · Cited by 2,215
Consistency of the group Lasso and multiple kernel learning
Bach, F. (2007) · arXiv (Cornell University) · doi:10.48550/arxiv.0707.3390 · Cited by 669
Self-citationA systematic literature review of methods and datasets for anomaly-based network intrusion detection
Yang, Z., Liu, X., Li, T., Wu, D., Wang, J., Zhao, Y., & Han, H. (2022) · Computers & Security · doi:10.1016/j.cose.2022.102675 · Cited by 453
p-Values for High-Dimensional Regression
Meinshausen, N., Meier, L., & Bühlmann, P. (2009) · Journal of the American Statistical Association · doi:10.1198/jasa.2009.tm08647 · Cited by 442
Metadata
- Authors
- Bach, F. R.
- Venue
- Proceedings of the 25th international conference on Machine learning - ICML '08
- Publisher
- ACM Press
- Published
- 2008
- Pages
- 33-40
- OpenAlex
- W2160222246
- Cited by
- 387
- References
- 16
- FWCI
- 22.22
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.