Coordinate descent algorithms for nonconvex penalized regression, with applications to biological feature selection
Open accessThe DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.
Abstract
A number of variable selection methods have been proposed involving nonconvex penalty functions. These methods, which include the smoothly clipped absolute deviation (SCAD) penalty and the minimax concave penalty (MCP), have been demonstrated to have attractive theoretical properties, but model fitting is not a straightforward task, and the resulting solutions may be unstable. Here, we demonstrate the potential of coordinate descent algorithms for fitting these models, establishing theoretical convergence properties and demonstrating that they are significantly faster than competing approaches. In addition, we demonstrate the utility of convexity diagnostics to determine regions of the parameter space in which the objective function is locally convex, even though the penalty is not. Our simulation study and data examples indicate that nonconvex penalties like MCP and SCAD are worthwhile alternatives to the lasso in many applications. In particular, our numerical results suggest that MCP is the preferred approach among the three methods.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Regression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482
Regularization Paths for Generalized Linear Models via Coordinate Descent
Friedman, J., Hastie, T., & Tibshirani, R. (2010) · Journal of Statistical Software · doi:10.18637/jss.v033.i01 · Cited by 19,927
McCullagh, P., & Nelder, J. A. (1989) · doi:10.1007/978-1-4899-3242-6 · Cited by 16,836
Golub, T. R., Slonim, D. K., Tamayo, P., Huard, C., Gaasenbeek, M., Mesirov, J. P., Coller, H., Loh, M. L., Downing, J. R., Caligiuri, M. A., et al. (1999) · Science · doi:10.1126/science.286.5439.531 · Cited by 11,516
Efron, B., Hastie, T., Johnstone, I., & Tibshirani, R. (2004) · The Annals of Statistics · doi:10.1214/009053604000000067 · Cited by 9,357
Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
Fan, J., & Li, R. (2001) · Journal of the American Statistical Association · doi:10.1198/016214501753382273 · Cited by 8,900
Ideal spatial adaptation by wavelet shrinkage
Donoho, D. L., & Johnstone, I. M. (1994) · Biometrika · doi:10.1093/biomet/81.3.425 · Cited by 8,760
Nearly unbiased variable selection under minimax concave penalty
Zhang, C.-H. (2010) · The Annals of Statistics · doi:10.1214/09-aos729 · Cited by 3,835
Convergence of a Block Coordinate Descent Method for Nondifferentiable Minimization
Tseng, P. (2001) · Journal of Optimization Theory and Applications · doi:10.1023/a:1017501703105 · Cited by 1,989
Pathwise coordinate optimization
Friedman, J., Hastie, T., Höfling, H., & Tibshirani, R. (2007) · The Annals of Applied Statistics · doi:10.1214/07-aoas131 · Cited by 1,895
Heuristics of instability and stabilization in model selection
Breiman, L. (1996) · The Annals of Statistics · doi:10.1214/aos/1032181158 · Cited by 1,146
L 1-Regularization Path Algorithm for Generalized Linear Models
Park, M. Y., & Hastie, T. (2007) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2007.00607.x · Cited by 838
One-step sparse estimates in nonconcave penalized likelihood models
Zou, H., & Li, R. (2008) · The Annals of Statistics · doi:10.1214/009053607000000802 · Cited by 683
Coordinate descent algorithms for lasso penalized regression
Wu, T. T., & Lange, K. (2008) · The Annals of Applied Statistics · doi:10.1214/07-aoas147 · Cited by 563
Understanding WaveShrink: variance and bias estimation
Bruce, A. (1996) · Biometrika · doi:10.1093/biomet/83.4.727 · Cited by 146
Feature Selection and Molecular Classification of Cancer Using Genetic Programming
Yu, J., Yu, J., Almal, A. A., Dhanasekaran, S. M., Ghosh, D., Worzel, W. P., & Chinnaiyan, A. M. (2007) · Neoplasia · doi:10.1593/neo.07121 · Cited by 79
Cited by
Challenges of Big Data analysis
Fan, J., Han, F., & Liu, H. (2014) · National Science Review · doi:10.1093/nsr/nwt032 · Cited by 1,482
Wright, S. J. (2015) · Mathematical Programming · doi:10.1007/s10107-015-0892-3 · Cited by 1,438
Elastic Net Regularization Paths for All Generalized Linear Models
Tay, J. K., Narasimhan, B., & Hastie, T. (2023) · Journal of Statistical Software · doi:10.18637/jss.v106.i01 · Cited by 653
Pavlou, M., Ambler, G., Seaman, S., De Iorio, M., & Omar, R. Z. (2016) · Statistics in Medicine · doi:10.1002/sim.6782 · Cited by 418
Cox-nnet: An artificial neural network method for prognosis prediction of high-throughput omics data
Ching, T., Zhu, X., & Garmire, L. X. (2018) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1006076 · Cited by 400
Metadata
- Authors
- Breheny, P., & Huang, J.
- Venue
- The Annals of Applied Statistics
- Publisher
- Institute of Mathematical Statistics
- Published
- 2011-03-01
- Volume
- 5
- Issue
- 1
- Pages
- 232-253
- ISSN
- 1932-6157 (ISSN-L)
- OpenAlex
- W2149144946
- Cited by
- 671
- References
- 16
- FWCI
- 28.38
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.