Understanding deep learning (still) requires rethinking generalization
Open accessThe DOI matches its Crossref record; 45 of the 47 references it cites were found in a registry or an index; none of the 12 sources its claims cite is checked for support yet.
Abstract
Despite their massive size, successful deep artificial neural networks can exhibit a remarkably small gap between training and test performance. Conventional wisdom attributes small generalization error either to properties of the model family or to the regularization techniques used during training. Through extensive systematic experiments, we show how these traditional approaches fail to explain why large neural networks generalize well in practice. Specifically, our experiments establish that state-of-the-art convolutional networks for image classification trained with stochastic gradient methods easily fit a random labeling of the training data. This phenomenon is qualitatively unaffected by explicit regularization and occurs even if we replace the true images by completely unstructured random noise. We corroborate these experimental findings with a theoretical construction showing that simple depth two neural networks already have perfect finite sample expressivity as soon as the number of parameters exceeds the number of data points as it usually does in practice. We interpret our experimental findings by comparison with traditional models. We supplement this republication with a new section at the end summarizing recent progresses in the field since the original version of this paper.
Overview
CoChat Summary
TL;DR
Deep neural networks can fit random labels and random pixels with zero training error, so traditional complexity and regularization explanations do not account for their generalization. The paper argues that explicit regularization is neither necessary nor sufficient, and offers a finite-sample expressivity construction plus linear-model implicit-regularization analysis.
Key findings
- On CIFAR10, Inception, AlexNet, and MLPs reach 100% (or 99.82%/99.34%) training accuracy on random labels while test accuracy is near chance: Inception 9.78%, Inception w/o BatchNorm 10.12%, AlexNet 9.86%, MLP 3×512 10.48%, MLP 1×512 10.61%.
- On ImageNet, Inception reaches 95.20% top-1 accuracy on 1.2 million random labels from 1000 categories without hyperparameter tuning; with explicit regularizers on it still reaches ∼90% top-1.
- Explicit regularization may improve generalization but is neither necessary nor sufficient: on CIFAR10 all models still generalize well with regularizers off; on ImageNet top-1 accuracy without regularization is 59.80% vs 0.1% random guessing, and with data augmentation only Inception reaches 72.95% top-1.
- Theoretically, a two-layer ReLU network with p = 2n + d parameters can represent any labeling of any sample of size n in d dimensions; depth-k networks can use O(n/k) parameters per layer. In linear models, SGD converges to a minimum-norm solution, but minimum norm is not predictive: on MNIST the minimum norm with no preprocessing is ∼220 and with wavelet preprocessing ∼390, yet test error drops by a factor of 2.
Methods
- Randomization tests on CIFAR10 and ImageNet ILSVRC 2012: train Inception V3 on ImageNet and smaller Inception, AlexNet, and MLPs on CIFAR10 under true labels, partially corrupted labels, random labels, shuffled pixels, random pixels, and Gaussian pixels, toggling data augmentation, weight decay, and dropout.
- Compare training/test accuracy and convergence time; supplement with a finite-sample expressivity theorem for two-layer ReLU networks and with linear/kernel experiments on MNIST and CIFAR10 (Gaussian kernels, random convolutional filters).
Limitations
- Stated: the paper offers no new theory of generalization; it says a full mathematical characterization remains challenging and that it cannot fully survey recent progress. It also states that minimum-norm intuition is only a small piece of the generalization story and that it does not explain why certain architectures generalize better.
- Evident: the empirical scope is limited to image classification benchmarks (CIFAR10, ImageNet) and a small set of architectures/regularizers, so the findings may not extend to other domains or model families. The theoretical construction uses a depth-2 network with large width, and the paper notes depth-k alternatives with O(n/k) parameters per layer.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 Valid · 7 Not valid · 12 Not checked · 85 Own results and interpretations
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In other words, by randomizing labels alone we can force the generalization error of a model to jump up considerably without changing the model, its size, hyperparameters, or the optimizer.”
§ Understanding Deep Learning (Still) Requires Rethinking Generalization > 1.1. The randomization test
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“although it does not reach the perfect 100% top-1 accuracy, 95.20% accuracy is still very surprising for 1.2 million random labels from 1000 categories.”
§ Understanding Deep Learning (Still) Requires Rethinking Generalization > 2. EFFECTIVE CAPACITY OF NEURAL NETWORKS
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The networks fit the corrupted training set perfectly for all the cases.”
§ Understanding Deep Learning (Still) Requires Rethinking Generalization > Partially corrupted labels.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“On the CIFAR10 dataset, Alexnet and MLPs all converge to zero loss on the training set.”
§ Understanding Deep Learning (Still) Requires Rethinking Generalization > 2.1. Fitting random labels and pixels
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We corroborate these experimental findings with a theoretical construction showing that simple depth two neural networks already have perfect finite sample expressivity as soon as the number of parameters exceeds the number of data points as it usually does in practice.”
- Also stated in: § 1.3. Finite sample expressivity · plain
“More formally, we exhibit a very simple two-layer ReLU network with p = 2n + d parameters that can express any labeling of any sample of size n in d dimensions.”
§ Understanding Deep Learning (Still) Requires Rethinking Generalization > Abstract
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 1.3. Finite sample expressivity · plain
70 more claims
Create a free account to see all 104 claims and every source verdict.
- It may state two separate claims as one. 17
- Probably not a substantial claim. 15
- The quote only partly states this claim. 1
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Based on existing research(Saha et al., 2021; Zhang et al., 2021), the following conclusion can be drawn: the gradient updates of a linear layer always lie within the space spanned by its input.”
From Knowledge Decoupling via Orthogonal Projection for Lifelong Editing of Large Language Models (2025) · cites this paper as Zhang et al., 2021
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“If a model is successfully trained on a dataset with random labels, which has been shown to be possible with deep neural networks [294], then it has learned the data generation process by memorizing the random labels.”
From From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI (2023) · cites this paper as [294]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Existing works in privacy-preserving federated learning are mostly developed based on the wellknown privacy-preserving techniques, including: (1) homomorphic encryption (HE), such as Paillier [90], Elgamal [91] and Brakerski-Gentry-Vaikuntanathan cryptosystems [92]; (2) Secure Multiparty Computation (SMC), such as garbled circuits [93] and secret sharing [94]; and (3) differential privacy (DP) [95”
From Privacy and Robustness in Federated Learning: Attacks and Defenses (2024) · cites this paper as [95]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Understanding Deep Learning (Still) Requires Rethinking Generalization > 2.1. Fitting random labels and pixels
“On the CIFAR10 dataset, Alexnet and MLPs all converge to zero loss on the training set.”
§ Understanding Deep Learning (Still) Requires Rethinking Generalization > Rademacher complexity and VC-dimension.
“(fitting random labels) No No 99.82 9.86 MLP 3 × 512 1,735,178 No Yes 100.0 53.35No No 100.0 52.39 (fitting random labels) No No 100.0 10.48 MLP 1 × 512 1,209,866 No Yes 99.80 50.39No No 100.0 50.51 (fitting random labels) No No 99.34 10.61”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Understanding Deep Learning (Still) Requires Rethinking Generalization > 1.1. The randomization test
“More precisely, when trained on a completely random labeling of the true data, neural networks achieve 0 training error.”
§ Understanding Deep Learning (Still) Requires Rethinking Generalization > 2. EFFECTIVE CAPACITY OF NEURAL NETWORKS
“although it does not reach the perfect 100% top-1 accuracy, 95.20% accuracy is still very surprising for 1.2 million random labels from 1000 categories.”
Citations
Cites
(2014) · Cited by 33,419
Wu, Y. (1999) · Technometrics · doi:10.1080/00401706.1999.10485951 · Cited by 27,221
Advances in neural information processing systems 7
Cios, K., & Shields, M. (1997) · Neurocomputing · doi:10.1016/s0925-2312(97)90018-7 · Cited by 22,059
Approximation by superpositions of a sigmoidal function
Cybenko, G. (1989) · Mathematics of Control, Signals, and Systems · doi:10.1007/bf02551274 · Cited by 13,883
Advances in Neural Information Processing Systems 25
Zhang, Y., Sutton, C. A., Storkey, A., & Zoubin Ghahramani, A. (2012) · Cited by 12,653
Understanding deep learning requires rethinking generalization
Zhang, C., Bengio, S., Hardt, M., Recht, B., & Vinyals, O. (2016) · arXiv · doi:10.48550/arxiv.1611.03530 · Cited by 2,838
Self-citationReconciling modern machine-learning practice and the classical bias–variance trade-off
Belkin, M., Hsu, D., Ma, S., & Mandal, S. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1903070116 · Cited by 1,595 · cited 1×
“23, 7, 6, 16 Despite significant progress on theoretical understanding of deep learning in the past few years, a full mathematical characterization of the whole story remains challenging.”
BackgroundA Generalized Representer Theorem
Schölkopf, B., Herbrich, R., & Smola, A. J. (2001) · Lecture Notes in Computer Science · doi:10.1007/3-540-44581-1_27 · Cited by 1,589 · cited 1×
“We have thus derived the “kernel trick” 34 —albeit in a round-about fashion.”
MethodA Closer Look at Memorization in Deep Networks
Arpit, D., Jastrzębski, S., Ballas, N., Krueger, D., Bengio, E., Kanwal, M. S., Maharaj, T., Fischer, A., Courville, A., Bengio, Y., et al. (2017) · arXiv · doi:10.48550/arxiv.1706.05394 · Cited by 1,307
Bartlett, P. L. (1998) · IEEE Transactions on Information Theory · doi:10.1109/18.661502 · Cited by 1,166
Research works printed but not linked: 8
- Cohen, N., Shashua, A. Convolutional rectifier networks as generalized tensor decompositions. In International Conference on Machine Learning. M.F. Balcan and K.Q. Weinberger, eds. 2016, 955--963.
- Cybenko, G. Approximation by superposition of sigmoidal functions. Math. Control Signal. Syst. 4, 2 (1989), 303--314.
- Golowich, N., Rakhlin, A., Shamir, O. Size-independent sample complexity of neural networks. In Conference On Learning Theory. P.R. Sébastien Bubeck, V. Perchet, eds., 2018, 297--299.
- Ma, S., Bassily, R., Belkin, M. The power of interpolation: Understanding the effectiveness of sgd in modern over-parametrized learning. In International Conference on Machine Learning. J. Dy and A. Krause, eds. 2018, 3325--3334.
- Morcos, A., Raghu, M., Bengio, S. Insights on representational similarity in neural networks with canonical correlation. Adv. Neural Inform. Process. Syst.. 2018, 5727--5736.
- Neyshabur, B., Tomioka, R., Srebro, N. Norm-based capacity control in neural networks. In Conference on Learning Theory. S.K. Peter Grünwald and E. Hazan, eds. 2015, 1376--1401.
- Smith, S.L., Le, Q.V. A bayesian perspective on generalization and stochastic gradient descent. In International Conference on Learning Representations, 2018.
- Zhou, W., Veitch, V., Austern, M., Adams, R.P., Orbanz, P. Non-vacuous generalization bounds at the ImageNet scale: A PAC-Bayesian compression approach. In International Conference on Learning Representations, 2019.
Cited by
Items 2,401–2,500 of 2,667
From Garbage to Gold: A Data-Architectural Theory of Predictive RobustnessPreprint
John, T. J. L.-S., Lawson, J. L., & Piechowski‐Jóźwiak, B. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2603.12288
Improving Robustness to Noisy Labels by Encouraging Confident Predictions in DNNsPreprint
Zeng, L., Liu, Z., Huang, Y., & Shi, X. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6432143
Ding, Z., Ding, D., Cao, J., Zhang, Y., & Xue, G. (2026) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2026.3675166
“Additionally,thesemethodsoftenstrugglewith featureextractionlimitationsoroverfitting,reducingprediction accuracyandimpactingresourceallocation[20],[21],[22].”
From empirical rules to data-driven design of Al-based amorphous alloys
liu, X., Luo, J., Zhang, E., Jing, Z., Bu, W., Hao, J., Zhang, Z., Gao, Q., & Liang, X. (2026) · Journal of Alloys and Compounds · doi:10.1016/j.jallcom.2026.187543
Feature Space-Preserving Machine Unlearning for Robust Image Classification With Noisy Labels
Yamamoto, S., Yoshida, S., & Muneyasu, M. (2026) · IEEE Access · doi:10.1109/access.2026.3676403
“Zhang et al. [3] demonstrated that deep networks can easily fit random labels and highlighted their propensity to memorize rather than generalize when they encounter noisy training data.”
Ma, J., Ma, K., Chen, L., & Jia, B. (2026) · Environment Development and Sustainability · doi:10.1007/s10668-026-07537-5
Gaona, D. S. B., Guerrero, B. S. V., & Clavijo, S. J. (2026) · Minerva Digital Library (Universidad EAN)
Hu, C., Li, J., Zhang, G., Li, Y., Zhao, T., Wang, X., Liu, Q., & Xu, Y. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6487011
Qamruzzaman, M., Mindia, P. M., & Swarna, S. R. (2026) · Quality & Quantity · doi:10.1007/s11135-026-02708-y
“…with robust digital infrastructure and urban development are more likely to invest in renewable energy technologies, aligning with the literature, which indicates that digital transformation facilitates clean energy transitions through smart grids and improved energy efficiency (Zhang et al. 2021).”
Knolle, M. A., Menten, M. J., Rueckert, D., Kaissis, G. A., & Glocker, B. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-16128-4_3 · Cited by 1
GNN Defender: Overcoming Mixed Noise Through Cross-Correction Over Nodes and Edges
Zhang, X., Liu, Y., Wang, H., Xie, W., & Wang, W. (2026) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2026.3678725
“Subsequently, experiments demonstrated that deep neural networks would recall clean labels and, over time, recall noisy labels [13], [14].”
Understanding and the Black Box Problem in Artificial Intelligence
Crook, B. (2025) · EPub Bayreuth (University of Bayreuth) · doi:10.15495/epub_ubt_00008992
Zhang, C. (2025) · Asian Conference on Frontiers of Power and Energy (ACFPE) · doi:10.1109/acfpe67232.2025.11451158
Self-citationNooraiepour, M., Masoudi, M., Song, Z., & Hellevang, H. (2026) · ACS Omega · doi:10.1021/acsomega.6c08678 · Cited by 1
Distributional properties of Bayesian neural networks
Vladimirova, M. (2022) · theses.fr (ABES) · doi:10.70675/cb210357zc25bz49e0zb0b4z19c9fd3bcaa4
Bontonou, M. (2021) · theses.fr (ABES) · doi:10.70675/4e46bed6zde51z4571z94caz198d67555e82
Maillard, A. (2021) · theses.fr (ABES) · doi:10.70675/6e19f979z2cd9z41bbzb187z4f91da50814e · Cited by 1
Precipitation Forecasting with Deep Transfer Active Learning for Agricultural Adaptation
Gikunda, P. (2021) · theses.fr (ABES) · doi:10.70675/cedfbb84z35daz4d9czb9a4z17dbdc7e3963
Garnier, R. (2021) · theses.fr (ABES) · doi:10.70675/608dfb1ezceb6z4dbcza4aezd6b056e4aab0
Mei, H., Guo, S., Wang, P., Ding, J., Chen, W., Yan, J., & Liu, H. (2026) · IEEE Transactions on Aerospace and Electronic Systems · doi:10.1109/taes.2026.3680886
“In real-world noncooperative radar scenarios, label noise can mislead the model and significantly degrade its generalization performance in target recognition [19].”
Deep learning based physical-statistics modeling of ocean dynamics
Déchelle-Marquet, M. (2023) · theses.fr (ABES) · doi:10.70675/03a2f675zc671z4ff4zad6bza1821d730e84
Bridging Deep Learning and Classical Profiled Side-Channel Attacks
Zaid, G. (2021) · theses.fr (ABES) · doi:10.70675/5a2c0f7ez9972z42c8za23az41f50140f51a
Bonnaire, T. (2021) · theses.fr (ABES) · doi:10.70675/553b007dz2008z47fez9d3az684a68a11cb4
Encoding and decoding of information through efficient neural representations
Malerba, S. B. (2022) · theses.fr (ABES) · doi:10.70675/fb444c77z28a6z476bza8d4z4d6b60064e35
Hypergraph based learning method : Embedding, Clustering, and Classification
Tran, L. (2023) · theses.fr (ABES) · doi:10.70675/ff3c4d6bz5497z4571z9527zcab59661158e
Constrained deep learning for MEMS sensors-based applications
Lê, M. T. (2023) · theses.fr (ABES) · doi:10.70675/cd0d8c3fze723z48f0zba6czeaaab993828c
Topics in high-dimensional and non-parametric inference
Chhor, J. (2022) · theses.fr (ABES) · doi:10.70675/24f761cbz67f0z4760zbeb2z340aaddb0a57
Towards lighter and faster deep neural networks with parameter pruning
Hubens, N. (2022) · theses.fr (ABES) · doi:10.70675/c018d7baz3aa3z43b2z9f61z0d9bb05ff94e
Beyond the training task in classification : looking at extension of the notion of generalization
Baena, R. (2023) · theses.fr (ABES) · doi:10.70675/8cf1a9e7z9e5ez4d5bza3a4zc92795ef92b5
Deep learning for subgrid-scale modeling in large eddy simulations of turbulent premixed combustion
Xing, V. (2022) · theses.fr (ABES) · doi:10.70675/67eddd96z6757z4c3ez8db6z50affe708228 · Cited by 2
Biquality learning : from weakly supervised learning to distribution shifts
Nodet, P. (2023) · theses.fr (ABES) · doi:10.70675/41d22fb9z3d38z4a64z9a3cz4a385058629c
Growth dynamics of large networks using hidden Markov chains
Duchemin, Q. (2022) · theses.fr (ABES) · doi:10.70675/37febdf8z99fbz4e94z8208zd41331764707
Réseaux de neurones impulsionnels appliqués à la vision par ordinateur
Oudjail, V. (2022) · theses.fr (ABES) · doi:10.70675/c491ede8z9a59z434fz982az2397d09c5ea6 · Cited by 1
Study of the abstraction capabilities of neural language models
Li, B. (2023) · theses.fr (ABES) · doi:10.70675/0236e084zee1ez4013z8f5bz6eebee3f0e4e
On the tradeoffs of statistical learning with privacy
Lalanne, C. (2023) · theses.fr (ABES) · doi:10.70675/7dc698fbze06ez4055zbed7z705880dd8daf · Cited by 4
Contributions to structured high-dimensional inference
Sigalla, S. (2022) · theses.fr (ABES) · doi:10.70675/ffef887ez4944z4cb3z82b9zcaa0294ed85a
From weakly supervised learning to active labeling
Cabannes, V. (2022) · theses.fr (ABES) · doi:10.70675/7153dc52z3c48z4a31z8b7bzd0b3fa4a7b66 · Cited by 1
Mahiout, T. (2022) · theses.fr (ABES) · doi:10.70675/de8dc2a9zc7fez489ezaa6ezb911b79eb56f
Achour, E. M. (2022) · theses.fr (ABES) · doi:10.70675/844f7ddaz4abbz4c26z83d3z9a3ce9519a42
Plassier, V. (2023) · theses.fr (ABES) · doi:10.70675/9b6101b0zc3dez4c4az9bb1zc3b4392f012a
Refining machine learning evaluation : statistical insights into model performance and fairness
Soumm, M. (2024) · theses.fr (ABES) · doi:10.70675/a653ed6fz3b6ez4629za555z89b7bcadd995
Optimizing Memory Usage when Training Deep Neural Networks
Zhao, X. (2024) · theses.fr (ABES) · doi:10.70675/14b989cezef3bz4263z8c05z34a16bd91adc · Cited by 2
A data modeling perspective on few-shot image classification
Bendou, Y. (2024) · theses.fr (ABES) · doi:10.70675/53e8985az0dc3z4a9azaaf6z925272599bbe
Experimental and Theoretical Analysis of Reinforcement Learning Algorithms
Brellmann, D. (2024) · theses.fr (ABES) · doi:10.70675/179f7b5azad23z4e41zab4fz104a7ade2bf2
Electromagnetic Field Exposure Reconstruction by Artificial Intelligence
Mallik, M. (2023) · theses.fr (ABES) · doi:10.70675/9fed277bzedaaz4398zb592z621b51499d81
Algorithmic and theoretical aspects of sparse deep neural networks
Le, Q.-T. (2023) · theses.fr (ABES) · doi:10.70675/01b91cd8zefbaz442az9887zbfcec3e1132f · Cited by 1
Contrastive techniques for enhanced Representation Learning
Gauffre, A. (2024) · theses.fr (ABES) · doi:10.70675/6bba74bbzce53z45aaza0a6z947f937b77ba
Quantifying and understanding uncertainty in deep-learning-based medical image segmentation
Lambert, B. (2024) · theses.fr (ABES) · doi:10.70675/a043da9aze69dz4ea2z978dzcd9a570ce0a7 · Cited by 1
Vortex gust mitigation using deep reinforcement learning
Martin, B. (2024) · theses.fr (ABES) · doi:10.70675/fcdcc63ez1e6fz42daz9006z9fbda52d6413
Label ambiguity in crowdsourcing for classification and expert feedback
Lefort, T. (2024) · theses.fr (ABES) · doi:10.70675/6de072beza1acz41ffzbcbdzb4103d751759
Mathematics of deep learning : generalization, optimization, continuous-time models
Marion, P. (2023) · theses.fr (ABES) · doi:10.70675/3253291bz2d55z4d65zab77z36bac6882cf0
Neural networks security under realistic scenario
Maho, T. (2023) · theses.fr (ABES) · doi:10.70675/690e25b2z1fb9z4d7fza6cbzcb09e2b773cf · Cited by 1
Nastasiu, D. (2024) · theses.fr (ABES) · doi:10.70675/f7530ab4z2185z4564zaf84zc239a380eaa7
Harnessing symmetries for modern deep learning challenges : a path-lifting perspective
Gonon, A. (2024) · theses.fr (ABES) · doi:10.70675/c0dd6b18z746ez4cfezae65zaba0b12d7641 · Cited by 5
Chalvidal, M. (2023) · theses.fr (ABES) · doi:10.70675/d35a8a98zd3cfz4df4za6e9zf26f90d6f965
Distributed and Parallel Computing for very Large Neural Networks
Petit, Q. R. (2025) · theses.fr (ABES) · doi:10.70675/b6bee80czf209z49b2z8063z0e05ef0d1c8c
Enhanced Deepfake Speech Classification With Ensemble Learning
Wójcicki, P. (2026) · IEEE Access · doi:10.1109/access.2026.3682478 · Cited by 2
“Ensemble techniques are particularly relevant in deep learning, where single high-capacity models are prone to overfitting [32].”
Distributed Rehearsal Buffers for Continual Learning at Scale
Bouvier, T. (2024) · theses.fr (ABES) · doi:10.70675/8cddaf62z4662z4848zb9e1z45041507de78 · Cited by 1
Membership Inference Security : An Alternative Path to Confidentiality
Aubinais, E. (2025) · theses.fr (ABES) · doi:10.70675/a3f76dabzff1dz4360za608z20aa9ce00cc5
Data-centric artificial intelligence for ecoacoustics
Michaud, F. (2025) · theses.fr (ABES) · doi:10.70675/d4db6e9cz957az4525z8e75z0d804ccb15c3
On the Oracle Complexity of Interpolation-Based Gradient Descent
Lee, D. D., Lu, W., & Makur, A. (2026) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2026.3682210
“For example, in machine learning contexts where overparameterization ( p > n ) can often be beneficial [79], it is natural to consider a regime of p = O ( n b ) (where b ≥ 1 ) and ε = Θ(1 / √ n ) , with the error bound matching the scaling of the standard error.”
TFN-FAA-EIRM: a frequency-aware adaptive framework for flat-minima optimization against label noise
Xie, F., & Zhong, J. (2026) · Measurement Science and Technology · doi:10.1088/1361-6501/ae5df2
“This performance degradation is hypothesized to stem from the ‘Memorization Effect’ inherent in DNNs [23].”
Shahzad, A., Akram, S., & Mehmood, T. (2026) · BMC Bioinformatics · doi:10.1186/s12859-026-06440-0
“This trend indicates that optimization properties of GGAR are better when moderated training parameters are used [48].”
InfluentialA Transport Geometry of Benign OverfittingPreprint
Lundström-Imanov, G. O. Y. L.-F., Turhan, R. I., & Jafarli, I. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2604.08625
“…Shalev-Shwartz and Ben-David 2014; Mohri et al. 2018; Hastie et al. 2009; Belkin et al. 2019; Belkin 2021; Belkin et al. 2020; Bartlett et al. 2020; Hastie et al. 2022; Nakkiran et al. 2021; Zhang et al. 2021; Chhor et al. 2024; Cherkassky and Lee 2024; Lee and Cherkassky 2024; Suh and Cheng 2025).”
Wu, Y., Gao, X., Wang, J., & Gong, T. (2026) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2026.3684037
“Neural networks exhibit a memorization effect, whereby they prioritize learning from cleanly labeled data in the early stages of training before eventually fitting noisy data [25].”
Dey, V., Thamizhselvi, M., Libeesh, P. C., & Joshi, P. (2026) · Advances in Social Science, Education and Humanities Research/Advances in social science, education and humanities research · doi:10.2991/978-2-38476-559-1_9
Federated Learning and Optimization for Lung Cancer Detection Using FedRes-Net Model
Wei, X. (2026) · International Journal of Pattern Recognition and Artificial Intelligence · doi:10.1142/s021800142657017x
Topology-Enhanced Deep Learning
Bautista, R. B. (2026) · Dipòsit Digital de la Universitat de Barcelona (Universitat de Barcelona)
Oliveira, A. S., Yaserifar, M., & Pîrșcoveanu, C.-I. (2026) · Sensors · doi:10.3390/s26082502 · Cited by 2
“One advantage of linear models is that they have stronger interpretability and reduced risks of overfitting [57,58], which is particularly relevant given the variability in cohort size and reporting quality across the accessed literature.”
Adegbosin, O. T., & Patel, H. (2026) · medRxiv · doi:10.64898/2026.04.18.26350809
XMIX: Combating Extremely Noisy Labels via Local Smoothness in Self-Supervised Feature Space
Li, C., Lu, Y., Shi, Z., He, W., Talhi, C., & Kara, N. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11463541
Positive-and-Multi-Negative Learning with Adaptive Reweighting for Noisy Labels
Zong, C.-C., Chi, Y.-Q., Jin, T., & Huang, S.-J. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11464415
Mehmood, H., Xu, J., Saravanan, K., van Dalen, R., & Özay, M. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11460789
Rong, Q., Zhang, L., Yuan, L., & Li, G. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11463739
Learning from Noisy Labels: A Conformal Prediction Perspective
Chi, Y.-Q., Zong, C.-C., Jin, T., & Huang, S.-J. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11464144
StylusNet: ML-Based Coordinate Extractor for EMR Stylus Touch Devices
Park, E., Seo, D., Shin, H., & Lee, Y. K. (2026) · IEEE Sensors Journal · doi:10.1109/jsen.2026.3684492
Classifier-Agnostic Ambiguity Detection Framework for Targeted Oversampling in Imbalanced Learning
Prexawanprasut, T., & Paireekreng, W. (2026) · IEEE Access · doi:10.1109/access.2026.3686480
Robust graph contrastive learning under label noise
Zhang, J., Liu, G., & Liang, Y. (2026) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-026-03113-5
“Some studies have shown that GNN-based models tend to memorize noisy samples and overfit label noise, leading to a reduction in the generalization ability of models [13, 14].”
Mean teacher with pseudo-label correction for cross-domain image classification
Pei, Y., & Yang, F. (2026) · Neurocomputing · doi:10.1016/j.neucom.2026.133670
From Convergence to Generalization: Stability of Stationary-Point Learning Algorithms
Lei, Y., Wang, Z., & Yuan, X. (2026) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2026.3688491
“Contrarytothisexpectation,empiricalevidenceshowsthat suchoverparameterizedmodelscanachievenotonlynear-zero training error but also strong generalization performance [10].”
Functional Analysis and Convergence Dynamics of the Decoder-Only Transformer
Guan, R., Wang, J., Sun, L., & Zhan, C. (2026) · IEEE Transactions on Consumer Electronics · doi:10.1109/tce.2026.3688624
Wei, C., Cheng, M., Dong, X., Lu, S., Guo, Z., & Wei, Z. (2026) · Geophysics · doi:10.1190/geo-2025-0020
Neuromanifold-Regularized KANs for Shape-fair Feature Representations
Arslan, M. F., Guo, W., & Li, S. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.01188
SAFIR: Spatially-aware Activation Function for Implicit Neural Representations
Zeraatkar, E., & Tešić, J. (2026) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2026.104754
Hybrid CNN Model for Brain Hemorrhage Detection with Grad- Cam-Powered Visualization
Vanavil, P., Pravina, P., Saranya, S., Thirisha, B., & Thabasiq, J. S. (2026) · World Conference on Computational Science and Technology (WcCST) · doi:10.1109/wccst67302.2026.11495856
“The hybrid framework proposed in this study is intended to deliver precise and interpretable brain hemorrhage diagnosis while addressing limitations in the existing literature regarding explainability and region-level identification [20].”
Hu, H., Ma, R., & Yuan, W. (2026) · IEEE Signal Processing Letters · doi:10.1109/lsp.2026.3690099
“However, conventional ES methods and model selection strategies based on minimum validation loss or maximum validation accuracy are sensitive to training fluctuations and outliers, which may degrade the generalization ability of the model [21], [22], [23].”
Ghojogh, B., & Ghodsi, A. (2026) · Springer eBooks · doi:10.1007/978-3-032-10738-1_6
Editorial: Machine Learning and Statistical Learning with Applications 2025
Zhang, Y. (2026) · Computers · doi:10.3390/computers15050295
Liu, Q., Zhang, L., Chen, B., Yuwen, G., Song, X., Si, J., Zhao, Y., Fu, P., Zhang, M., & Chen, Y. (2026) · Fuel · doi:10.1016/j.fuel.2026.139691
Deonarain, G. (2026) · Digital Commons - Michigan Tech (Michigan Technological University) · doi:10.37099/mtu.dc.etdr/2085 · Cited by 1
Chung, S., Yoo, I., Lee, J., Kim, D., Kim, K., Kim, K., & Lee, S.-B. (2026) · IEEE Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbbio.2026.3686921
A Synthetic Data Augmented Deep Learning Framework for Low-Data Image Classification
Sabu, J., & Bais, A. S. (2026) · IEEE International Conference on Communication Systems and Network Technologies (CSNT) · doi:10.1109/csnt69054.2026.11502370
Large Action Models: A Unified Smart Framework for Autonomous Intelligent Systems
Patel, P., Patil, A., Parshuramkar, H., & Raut, P. U. (2026) · International Conference on Smart Futuristic Technology (SMARTTECH) · doi:10.1109/icsft66733.2026.11506682
Towards ROI-Free Modality Translation for MRI with Test-Time Augmentation
Park, J. W., Kim, S., Park, G., & Hwang, S. J. (2026) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi61048.2026.11515360
“This difference makes the performance of the MRI translation models with smaller datasets incompatible [14].”
Lehmler, S. J., Saif-ur-Rehman, M., Glasmachers, T., & Iossifidis, I. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-21477-5_27
Soft LearningPreprint
Aledhari, M., Aledhari, A., Aledhari, F., & Rahouti, M. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.18889
“On the disagreement set, Corollary 12 (Diversity Lower Bound) .”
InfluentialAji, R. P., Mukti, B. P., Arsi, P., Safitri, A., & Mahardhika, G. S. (2026) · International Conference on Current Research in Artificial Intelligence and Data Science (ICCRAIDS) · doi:10.1109/iccraids67816.2026.11519633
Metadata
- Authors
- Zhang, C., Bengio, S., Hardt, M., Recht, B., & Vinyals, O.
- Affiliations
- Google (United States) (US)
- Google Brain, Mountain View, CA
- University of California, Berkeley (US)
- Google DeepMind (United Kingdom) (GB)
- Venue
- Communications of the ACM
- Published
- 2021-02-22
- Volume
- 64
- Issue
- 3
- Pages
- 107-115
- ISSN
- 0001-0782 (ISSN-L)
- 1557-7317
- OpenAlex
- W2566079294
- Cited by
- 2,400
- References
- 47
- Citations by year
- 2026: 295 · 2025: 558 · 2024: 579 · 2023: 500 · 2022: 316 · 2021: 108
- FWCI
- 132.37
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Communications of the ACM (publishedVersion, cc-by-nc-sa)
- Funders
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- generalization, deep neural network, overfitting, overparameterization, expressivity, stochastic gradient descent, regularization, image classification, convolutional neural networks
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-11-08, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-11-08 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.