The DOI matches its Crossref record; 24 of the 24 references it cites were found in a registry or an index.
The LASSO Risk for Gaussian Matrices
Abstract
We consider the problem of learning a coefficient vector xο∈ RNfrom noisy linear observation y = Axo+ ∈ Rn. In many contexts (ranging from model selection to image processing), it is desirable to construct a sparse estimator x̂. In this case, a popular approach consists in solving an ℓ1-penalized least-squares problem known as the LASSO or basis pursuit denoising. For sequences of matrices A of increasing dimensions, with independent Gaussian entries, we prove that the normalized risk of the LASSO converges to a limit, and we obtain an explicit expression for this limit. Our result is the first rigorous derivation of an explicit formula for the asymptotic mean square error of the LASSO for random instances. The proof technique is based on the analysis of AMP, a recently developed efficient algorithm, that is inspired from graphical model ideas. Simulations on real data matrices suggest that our results can be relevant in a broad array of practical applications.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Elements of Statistical Learning
Hastie, T., Friedman, J., & Tibshirani, R. (2001) · Springer Series in Statistics · doi:10.1007/978-0-387-21606-5 · Cited by 24,162
A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems
Beck, A., & Teboulle, M. (2009) · SIAM Journal on Imaging Sciences · doi:10.1137/080716542 · Cited by 11,952
Capacity of Multi‐antenna Gaussian Channels
Telatar, E. (1999) · European Transactions on Telecommunications · doi:10.1002/ett.4460100604 · Cited by 11,234
Stable signal recovery from incomplete and inaccurate measurements
Candès, E. J., Romberg, J. K., & Tao, T. (2006) · Communications on Pure and Applied Mathematics · doi:10.1002/cpa.20124 · Cited by 7,036
Regression Shrinkage and Selection via The Lasso: A Retrospective
Tibshirani, R. (2011) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2011.00771.x · Cited by 3,589
Message-passing algorithms for compressed sensing
Donoho, D. L., Maleki, A., & Montanari, A. (2009) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0909892106 · Cited by 2,416
Self-citationInformation, Physics, and Computation
Mézard, M., & Montanari, A. (2009) · doi:10.1093/acprof:oso/9780198570837.001.0001 · Cited by 1,913
Self-citationThe Dantzig selector: Statistical estimation when p is much larger than n
Candes, E., & Tao, T. (2007) · The Annals of Statistics · doi:10.1214/009053606000001523 · Cited by 1,747
The Concentration of Measure Phenomenon
Ledoux, M. (2005) · Mathematical Surveys and Monographs · doi:10.1090/surv/089 · Cited by 1,700
Spectral Analysis of Large Dimensional Random Matrices
Bai, Z., & Silverstein, J. W. (2010) · Springer Series in Statistics · doi:10.1007/978-1-4419-0661-8 · Cited by 1,544
Minimax estimation via wavelet shrinkage
Donoho, D. L., & Johnstone, I. M. (1998) · The Annals of Statistics · doi:10.1214/aos/1024691081 · Cited by 1,005
On Model Selection Consistency of Lasso
ZhaoPeng, & Yubin, Y. (2006) · Journal of Machine Learning Research · doi:10.5555/1248547.1248637 · Cited by 996
The Dynamics of Message Passing on Dense Graphs, with Applications to Compressed Sensing
Bayati, M., & Montanari, A. (2011) · IEEE Transactions on Information Theory · doi:10.1109/tit.2010.2094817 · Cited by 780
Self-citationLimit of the Smallest Eigenvalue of a Large Dimensional Sample Covariance Matrix
Bai, Z. D., & Yin, Y. Q. (1993) · The Annals of Probability · doi:10.1214/aop/1176989118 · Cited by 519
Scalable training of L1-regularized log-linear models
Andrew, G., & Gao, J. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273501 · Cited by 506
The Noise-Sensitivity Phase Transition in Compressed Sensing
Donoho, D. L., Maleki, A., & Montanari, A. (2011) · IEEE Transactions on Information Theory · doi:10.1109/tit.2011.2165823 · Cited by 339
Self-citationDIAMETERS OF SOME FINITE-DIMENSIONAL SETS AND CLASSES OF SMOOTH FUNCTIONS
Kašin, B. S. (1977) · Mathematics of the USSR-Izvestiya · doi:10.1070/im1977v011n02abeh001719 · Cited by 251
Minimax risk overl p -balls forl p -error
Donoho, D. L., & Johnstone, I. M. (1994) · Probability Theory and Related Fields · doi:10.1007/bf01199026 · Cited by 218
High dimensional statistical inference and random matrices
Johnstone, I. M. (2007) · Proceedings of the International Congress of Mathematicians Madrid, August 22–30, 2006 · doi:10.4171/022-1/13 · Cited by 209
Asymptotic Analysis of MAP Estimation via the Replica Method and Applications to Compressed Sensing
Rangan, S., Fletcher, A. K., & Goyal, V. K. (2012) · IEEE Transactions on Information Theory · doi:10.1109/tit.2011.2177575 · Cited by 189
A typical reconstruction limit for compressed sensing based on Lp-norm minimization
Kabashima, Y., Wadayama, T., & Tanaka, T. (2009) · Journal of Statistical Mechanics Theory and Experiment · doi:10.1088/1742-5468/2009/09/l09003 · Cited by 177
High Dimensional Statistical Inference and Random MatricesPreprint
Johnstone, I. M. (2006) · arXiv (Cornell University) · doi:10.48550/arxiv.math/0611589 · Cited by 154
A single-letter characterization of optimal noisy compressed sensing
Guo, D., Baron, D., & Shamai, S. S. (2009) · Allerton Conference on Communication, Control, and Computing · doi:10.1109/allerton.2009.5394838 · Cited by 113
Chen, S., & Donoho, D. L. (1995) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.217610 · Cited by 49
Cited by
Phase Retrieval via Wirtinger Flow: Theory and Algorithms
Candès, E. J., Li, X., & Soltanolkotabi, M. (2015) · IEEE Transactions on Information Theory · doi:10.1109/tit.2015.2399924 · Cited by 1,049
A Probabilistic and RIPless Theory of Compressed Sensing
Candes, E. J., & Plan, Y. (2011) · IEEE Transactions on Information Theory · doi:10.1109/tit.2011.2161794 · Cited by 598
Living on the edge: phase transitions in convex programs with random data
Amelunxen, D., Lotz, M., McCoy, M. B., & Tropp, J. A. (2014) · Information and Inference · doi:10.1093/imaiai/iau005 · Cited by 413
The Noise-Sensitivity Phase Transition in Compressed Sensing
Donoho, D. L., Maleki, A., & Montanari, A. (2011) · IEEE Transactions on Information Theory · doi:10.1109/tit.2011.2165823 · Cited by 339
Self-citationAsymptotic Analysis of Complex LASSO via Complex Approximate Message Passing (CAMP)
Maleki, A., Anitori, L., Yang, Z., & Baraniuk, R. G. (2013) · IEEE Transactions on Information Theory · doi:10.1109/tit.2013.2252232 · Cited by 264
Metadata
- Authors
- Bayati, M., & Montanari, A.
- Venue
- IEEE Transactions on Information Theory
- Published
- 2012-04
- Volume
- 58
- Issue
- 4
- Pages
- 1997-2017
- ISSN
- 0018-9448 (ISSN-L)
- 1557-9654
- OpenAlex
- W2123202508
- Cited by
- 277
- References
- 24
- FWCI
- 14.84
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.