A generalized Dantzig selector with shrinkage tuning
Abstract
The Dantzig selector performs variable selection and model fitting in linear regression. It uses an L1 penalty to shrink the regression coefficients towards zero, in a similar fashion to the lasso. While both the lasso and Dantzig selector potentially do a good job of selecting the correct variables, they tend to overshrink the final coefficients. This results in an unfortunate trade-off. One can either select a high shrinkage tuning parameter that produces an accurate model but poor coefficient estimates or a low shrinkage parameter that produces more accurate coefficients but includes many irrelevant variables. We extend the Dantzig selector to fit generalized linear models while eliminating overshrinkage of the coefficient estimates, and develop a computationally efficient algorithm, similar in nature to least angle regression, to compute the entire path of coefficient estimates. A simulation study illustrates the advantages of our approach relative to others. We apply the methodology to two datasets.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Cites
Regression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482 · see citing works
An Introduction to the Bootstrap
Efron, B., & Tibshirani, R. J. (1994) · doi:10.1201/9780429246593 · Cited by 40,500 · see citing works
The Elements of Statistical Learning
Hastie, T., Friedman, J., & Tibshirani, R. (2001) · Springer Series in Statistics · doi:10.1007/978-0-387-21606-5 · Cited by 24,162 · see citing works
Regularization and Variable Selection Via the Elastic Net
Zou, H., & Hastie, T. (2005) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2005.00503.x · Cited by 20,957 · see citing works
(2005) · Springer eBooks · doi:10.1007/0-387-27255-0_9 · Cited by 20,333 · see citing works
Efron, B., Hastie, T., Johnstone, I., & Tibshirani, R. (2004) · The Annals of Statistics · doi:10.1214/009053604000000067 · Cited by 9,357 · see citing works
Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
B, J. F. A., & Li, R. (2001) · Journal of the American Statistical Association · doi:10.1198/016214501753382273 · Cited by 8,919 · see citing works
The adaptive lasso and its oracle properties
Zou, H. (2006) · Journal of the American Statistical Association · doi:10.1198/016214506000000735 · Cited by 7,470 · see citing works
Atomic Decomposition by Basis Pursuit
Chen, S. S., Donoho, D. L., & Saunders, M. A. (1998) · SIAM Journal on Scientific Computing · doi:10.1137/s1064827596304010 · Cited by 6,734 · see citing works
Generalized linear models. 2nd ed.
McCullagh, P., & Nelder, J. (1993) · Journal of the American Statistical Association · doi:10.2307/2290358 · Cited by 1,581 · see citing works
L 1-Regularization Path Algorithm for Generalized Linear Models
Park, M. Y., & Hastie, T. J. (2007) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.1467-9868.2007.00607.x · Cited by 859 · see citing works
Large-Scale Bayesian Logistic Regression for Text Categorization
Genkin, A., Lewis, D. D., & Madigan, D. (2007) · Technometrics · doi:10.1198/004017007000000245 · Cited by 812 · see citing works
Meinshausen, N. (2006) · Computational Statistics & Data Analysis · doi:10.1016/j.csda.2006.12.019 · Cited by 500 · see citing works
A simple and efficient algorithm for gene selection using sparse logistic regression
Shevade, S., & Keerthi, S. S. (2003) · Bioinformatics · doi:10.1093/bioinformatics/btg308 · Cited by 400 · see citing works
Grouped and Hierarchical Model Selection through Composite Absolute Penalties
Zhao, P., Rocha, G. V., & Yu, B. (2007) · Cited by 196 · see citing works
DASSO: Connections Between the Dantzig Selector and Lasso
James, G. M., Radchenko, P., & Lv, J. (2008) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.1467-9868.2008.00668.x · Cited by 172 · see citing works
Self-citationDiscussion of "Least Angle Regression" by Efron, Hastie, Johnstone and Tibshirani
Turlach, B. A. (2004) · UWA Profiles and Research Repository (University of Western Australia) · Cited by 147 · see citing works
Zhao, P., & Yu, B. (2007) · Cited by 94 · see citing works
The Dantzig Selector in Cox's Proportional Hazards Model
Antoniadis, A., Fryźlewicz, P., & Letué, F. (2010) · Scandinavian Journal of Statistics · doi:10.1111/j.1467-9469.2009.00685.x · Cited by 79 · see citing works
Discussion: The Dantzig selector: Statistical estimation when p is much larger than n
Efron, B., Hastie, T., & Tibshirani, R. J. (2007) · The Annals of Statistics · doi:10.1214/009053607000000433 · Cited by 76 · see citing works
DISCUSSION: A TALE OF THREE COUSINS: LASSO, L2BOOSTING AND DANTZIGPreprint
Meinshausen, N., Rocha, G., & Yu, B. (2007) · Cited by 69 · see citing works
Variable Inclusion and Shrinkage Algorithms
Radchenko, P., & James, G. M. (2008) · Journal of the American Statistical Association · doi:10.1198/016214508000000481 · Cited by 61 · see citing works
Self-citationRidge estimation for multinomial logit models with symmetric side constraints
Zahid, F. M., & Tutz, G. (2012) · Computational Statistics · doi:10.1007/s00180-012-0341-1 · Cited by 19 · see citing works
Discussion of "the Dantzig selector"
Hastie, B. E. T., & Tibshirani, R. J. (2007) · Cited by 9 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Theory-Guided Data Science: A New Paradigm for Scientific Discovery from Data
Karpatne, A., Atluri, G., Faghmous, J. H., Steinbach, M., Banerjee, A., Ganguly, A., Shekhar, S., Samatova, N., & Kumar, V. (2017) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2017.2720168 · Cited by 1,752 · see citing works
- Authors
- James, G. M., & Radchenko, P.
- Affiliations
- University of Southern California (US)
- Venue
- Biometrika
- Publisher
- Oxford University Press (OUP)
- Published
- 2009-05-20
- Volume
- 96
- Issue
- 2
- Pages
- 323-337
- ISSN
- 0006-3444 (ISSN-L)
- 1464-3510
- OpenAlex
- W2019440647
- Cited by
- 67
- References
- 24
- Citations by year
- 2025: 2 · 2024: 3 · 2023: 3 · 2022: 1 · 2021: 1 · 2019: 3 · 2018: 3 · 2017: 5 · 2016: 3 · 2015: 1 · 2014: 3 · 2013: 7 · 2012: 7
- FWCI
- 8.32
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Biometrika (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Funders
- National Science Foundation
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Inference
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Statistical Methods and Models
- Physical Sciences › Engineering › Computational Mechanics › Sparse and Compressive Sensing Techniques
- Keywords
- Dantzig selector, feature selection, l1 penalty, coefficient estimation, generalized linear models, least angle regression, linear regression
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
