The DOI matches its Crossref record; 81 of the 81 references it cites were found in a registry or an index.
Surprises in high-dimensional ridgeless least squares interpolation
Open accessAbstract
Interpolators—estimators that achieve zero training error—have attracted growing attention in machine learning, mainly because state-of-the art neural networks appear to be models of this type. In this paper, we study minimum ℓ2 norm (“ridgeless”) interpolation least squares regression, focusing on the high-dimensional regime in which the number of unknown parameters p is of the same order as the number of samples n. We consider two different models for the feature distribution: a linear model, where the feature vectors xi∈Rp are obtained by applying a linear transform to a vector of i.i.d. entries, xi=Σ1/2zi (with zi∈Rp); and a nonlinear model, where the feature vectors are obtained by passing the input through a random one-layer neural network, xi=φ(Wzi) (with zi∈Rd, W∈Rp×d a matrix of i.i.d. entries, and φ an activation function acting componentwise on Wzi). We recover—in a precise quantitative way—several phenomena that have been observed in large-scale neural networks and kernel machines, including the “double descent” behavior of the prediction risk, and the potential benefits of overparametrization.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(2016) · Cited by 11,679
Atomic Decomposition by Basis Pursuit
Chen, S. S., Donoho, D. L., & Saunders, M. A. (1998) · SIAM Journal on Scientific Computing · doi:10.1137/s1064827596304010 · Cited by 6,845
Generalized Cross-Validation as a Method for Choosing a Good Ridge Parameter
Golub, G. H., Heath, M., & Wahba, G. (1979) · Technometrics · doi:10.1080/00401706.1979.10489751 · Cited by 3,934
A survey of cross-validation procedures for model selectionPreprint
Arlot, S., & Célisse, A. (2010) · Cited by 3,625
A survey of cross-validation procedures for model selection
Arlot, S., & Celisse, A. (2010) · Statistics Surveys · doi:10.1214/09-ss054 · Cited by 3,522
Craven, P., & Wahba, G. (1978) · Numerische Mathematik · doi:10.1007/bf01404567 · Cited by 3,388
Understanding deep learning requires rethinking generalization
Zhang, C., Bengio, S., Hardt, M., Recht, B., & Vinyals, O. (2016) · arXiv · doi:10.48550/arxiv.1611.03530 · Cited by 2,838
Random Features for Large-Scale Kernel Machines
Rahimi, A., & Recht, B. (2007) · Cited by 2,783
Random Matrix Theory and Wireless Communications
Tulino, A. M., & Verdú, S. (2004) · Foundations and Trends™ in Communications and Information Theory · doi:10.1561/0100000001 · Cited by 2,034
Reconciling modern machine-learning practice and the classical bias–variance trade-off
(2019) · Proceedings of the National Academy of Sciences · Cited by 1,956
Cited by
Deep double descent: where bigger models and more data hurt*
Nakkiran, P., Kaplun, G., Bansal, Y., Yang, T., Barak, B., & Sutskever, I. (2021) · Journal of Statistical Mechanics: Theory and Experiment · doi:10.1088/1742-5468/ac3a74 · Cited by 565
Does learning require memorization? a short tale about a long tail
Feldman, V. (2020) · Proceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing · doi:10.1145/3357713.3384290 · Cited by 287
Explaining Neural Scaling Laws
Bahri, Y., Dyer, E., Kaplan, J., Lee, J., & Sharma, U. (2021) · arXiv · doi:10.48550/arxiv.2102.06701 · Cited by 182
Explaining neural scaling laws
Bahri, Y., Dyer, E., Kaplan, J., Lee, J., & Sharma, U. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2311878121 · Cited by 181
Harmless interpolation of noisy data in regression
Muthukumar, V., Vodrahalli, K., & Sahai, A. (2019) · 2019 IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2019.8849614 · Cited by 162
Metadata
- Authors
- Hastie, T., Montanari, A., Rosset, S., & Tibshirani, R. J.
- Venue
- The Annals of Statistics
- Publisher
- Institute of Mathematical Statistics
- Published
- 2022-04-01
- Volume
- 50
- Issue
- 2
- Pages
- 949-986
- ISSN
- 0090-5364 (ISSN-L)
- OpenAlex
- W2923764619
- Cited by
- 503
- References
- 81
- FWCI
- 20.28
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Surprises in high-dimensional ridgeless least squares interpolation. https://app.cochat.ai/papers/10.1214/21-aos2133
- The paper
- Hastie, T., Montanari, A., Rosset, S., & Tibshirani, R. J. (2022). Surprises in high-dimensional ridgeless least squares interpolation. The Annals of Statistics. https://doi.org/10.1214/21-aos2133
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
