The DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index.
Strong Rules for Discarding Predictors in Lasso-Type Problems
Open accessAbstract
Summary: We consider rules for discarding predictors in lasso regression and related problems, for computational efficiency. El Ghaoui and his colleagues have proposed ‘SAFE’ rules, based on univariate inner products between each predictor and the outcome, which guarantee that a coefficient will be 0 in the solution vector. This provides a reduction in the number of variables that need to be entered into the optimization. We propose strong rules that are very simple and yet screen out far more predictors than the SAFE rules. This great practical improvement comes at a price: the strong rules are not foolproof and can mistakenly discard active predictors, i.e. predictors that have non-zero coefficients in the solution. We therefore combine them with simple checks of the Karush–Kuhn–Tucker conditions to ensure that the exact solution to the convex problem is delivered. Of course, any (approximate) screening method can be combined with the Karush–Kuhn–Tucker conditions to ensure the exact solution; the strength of the strong rules lies in the fact that, in practice, they discard a very large number of the inactive predictors and almost never commit mistakes. We also derive conditions under which they are foolproof. Strong rules provide substantial savings in computational time for a variety of statistical optimization problems.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Regression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482
Self-citationRegularization and Variable Selection Via the Elastic Net
Zou, H., & Hastie, T. (2005) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2005.00503.x · Cited by 20,957
Self-citationEfron, B., Hastie, T., Johnstone, I., & Tibshirani, R. (2004) · The Annals of Statistics · doi:10.1214/009053604000000067 · Cited by 9,357
Self-citationModel Selection and Estimation in Regression with Grouped Variables
Yuan, M., & Lin, Y. (2006) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2005.00532.x · Cited by 7,261
Atomic Decomposition by Basis Pursuit
Chen, S. S., Donoho, D. L., & Saunders, M. A. (1998) · SIAM Journal on Scientific Computing · doi:10.1137/s1064827596304010 · Cited by 6,845
Sure Independence Screening for Ultrahigh Dimensional Feature Space
Fan, J., & Lv, J. (2008) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2008.00674.x · Cited by 2,711
High dimensional graphs and variable selection with the Lasso
Meinshausen, N., & Bühlmann, P. (2006) · Cited by 2,539
High-dimensional graphs and variable selection with the Lasso
Meinshausen, N., & Bühlmann, P. (2006) · The Annals of Statistics · doi:10.1214/009053606000000281 · Cited by 2,521
NewsWeeder: Learning to Filter Netnews
Lang, K. (1995) · Machine Learning Proceedings 1995 · doi:10.1016/b978-1-55860-377-6.50048-7 · Cited by 2,051
Pathwise coordinate optimization
Friedman, J., Hastie, T., Höfling, H., & Tibshirani, R. (2007) · The Annals of Applied Statistics · doi:10.1214/07-aoas131 · Cited by 1,895
Self-citationJust relax: convex programming methods for identifying sparse signals in noise
Tropp, J. A. (2006) · IEEE Transactions on Information Theory · doi:10.1109/tit.2005.864420 · Cited by 1,407
Wainwright, M. J. (2009) · IEEE Transactions on Information Theory · doi:10.1109/tit.2009.2016018 · Cited by 1,198
On Model Selection Consistency of Lasso
ZhaoPeng, & Yubin, Y. (2006) · Journal of Machine Learning Research · doi:10.5555/1248547.1248637 · Cited by 996
Genome-wide association analysis by lasso penalized logistic regression
Wu, T. T., Chen, Y. F., Hastie, T., Sobel, E., & Lange, K. (2009) · Bioinformatics · doi:10.1093/bioinformatics/btp041 · Cited by 785 · cited 5×
“Wu et al. (2009) study !1 penalized logistic regression and build a set D to discard based on the inner products between the outcome and each feature. As with the strong rules, their construction does not guarantee that the variables in D actually have zero coefficients in the solution, and so after fitting on XDc , t…”
BackgroundMethodInfluentialSelf-citationAn Interior-Point Method for Large-Scale '1-Regularized Logistic Regression
Koh, K., Kim, S.-J., & Boyd, S. (2007) · Cited by 579
The solution path of the generalized lasso
Tibshirani, R. J., & Taylor, J. (2011) · The Annals of Statistics · doi:10.1214/11-aos878 · Cited by 553
Self-citationNear-ideal model selection by ℓ1 minimization
Candès, E. J., & Plan, Y. (2008) · Cited by 469
New Insights and Faster Computations for the Graphical Lasso
Witten, D. M., Friedman, J. H., & Simon, N. (2011) · Journal of Computational and Graphical Statistics · doi:10.1198/jcgs.2011.11051a · Cited by 325
Self-citationRecovery of Exact Sparse Representations in the Presence of Bounded Noise
Fuchs, J. J. (2005) · IEEE Transactions on Information Theory · doi:10.1109/tit.2005.855614 · Cited by 314
Near-ideal model selection by ℓ1 minimization
Candès, E. J., & Plan, Y. (2009) · The Annals of Statistics · doi:10.1214/08-aos653 · Cited by 302
An Interior-Point Method for Large-Scale l1-Regularized Logistic Regression
KohKwangmoo, KimSeung-Jean, & BoydStephen (2007) · Journal of Machine Learning Research · doi:10.5555/1314498.1314550 · Cited by 231
Safe Feature Elimination for the LASSO and Sparse Supervised Learning ProblemsPreprint
Ghaoui, L. E., Viallon, V., & Rabbani, T. (2010) · arXiv (Cornell University) · doi:10.48550/arxiv.1009.4219 · Cited by 165
Exact covariance thresholding into connected components for large-scale Graphical LassoPreprint
Mazumder, R., & Hastie, T. J. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1108.3829 · Cited by 132
Self-citation
Cited by
Elastic Net Regularization Paths for All Generalized Linear Models
Tay, J. K., Narasimhan, B., & Hastie, T. (2023) · Journal of Statistical Software · doi:10.18637/jss.v106.i01 · Cited by 653
Self-citationExtension of SBL Algorithms for the Recovery of Block Sparse Signals With Intra-Block Correlation
Zhang, Z., & Rao, B. D. (2013) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2013.2241055 · Cited by 558
Personalized evidence based medicine: predictive approaches to heterogeneous treatment effects
Kent, D. M., Steyerberg, E., & van Klaveren, D. (2018) · BMJ · doi:10.1136/bmj.k4245 · Cited by 479
Efficient analysis of large-scale genome-wide data with two R packages: bigstatsr and bigsnpr
Privé, F., Aschard, H., Ziyatdinov, A., & Blum, M. G. B. (2018) · Bioinformatics · doi:10.1093/bioinformatics/bty185 · Cited by 368
Learning Interactions via Hierarchical Group-Lasso Regularization
Lim, M., & Hastie, T. (2015) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.2014.938812 · Cited by 273
Self-citation
Metadata
- Authors
- Tibshirani, R., Bien, J., Friedman, J., Hastie, T., Simon, N., Taylor, J., & Tibshirani, R. J.
- Venue
- Journal of the Royal Statistical Society Series B: Statistical Methodology
- Publisher
- Oxford University Press (OUP)
- Published
- 2011-11-03
- Volume
- 74
- Issue
- 2
- Pages
- 245-266
- ISSN
- 1369-7412 (ISSN-L)
- 1467-9868
- OpenAlex
- W2131060185
- Cited by
- 653
- References
- 23
- FWCI
- 18.92
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.