Understanding deep learning (still) requires rethinking generalization
Open accessThe DOI matches its Crossref record; 45 of the 47 references it cites were found in a registry or an index; none of the 12 sources its claims cite is checked for support yet.
Abstract
Despite their massive size, successful deep artificial neural networks can exhibit a remarkably small gap between training and test performance. Conventional wisdom attributes small generalization error either to properties of the model family or to the regularization techniques used during training. Through extensive systematic experiments, we show how these traditional approaches fail to explain why large neural networks generalize well in practice. Specifically, our experiments establish that state-of-the-art convolutional networks for image classification trained with stochastic gradient methods easily fit a random labeling of the training data. This phenomenon is qualitatively unaffected by explicit regularization and occurs even if we replace the true images by completely unstructured random noise. We corroborate these experimental findings with a theoretical construction showing that simple depth two neural networks already have perfect finite sample expressivity as soon as the number of parameters exceeds the number of data points as it usually does in practice. We interpret our experimental findings by comparison with traditional models. We supplement this republication with a new section at the end summarizing recent progresses in the field since the original version of this paper.
Overview
CoChat Summary
TL;DR
Deep neural networks can fit random labels and random pixels with zero training error, so traditional complexity and regularization explanations do not account for their generalization. The paper argues that explicit regularization is neither necessary nor sufficient, and offers a finite-sample expressivity construction plus linear-model implicit-regularization analysis.
Key findings
- On CIFAR10, Inception, AlexNet, and MLPs reach 100% (or 99.82%/99.34%) training accuracy on random labels while test accuracy is near chance: Inception 9.78%, Inception w/o BatchNorm 10.12%, AlexNet 9.86%, MLP 3×512 10.48%, MLP 1×512 10.61%.
- On ImageNet, Inception reaches 95.20% top-1 accuracy on 1.2 million random labels from 1000 categories without hyperparameter tuning; with explicit regularizers on it still reaches ∼90% top-1.
- Explicit regularization may improve generalization but is neither necessary nor sufficient: on CIFAR10 all models still generalize well with regularizers off; on ImageNet top-1 accuracy without regularization is 59.80% vs 0.1% random guessing, and with data augmentation only Inception reaches 72.95% top-1.
- Theoretically, a two-layer ReLU network with p = 2n + d parameters can represent any labeling of any sample of size n in d dimensions; depth-k networks can use O(n/k) parameters per layer. In linear models, SGD converges to a minimum-norm solution, but minimum norm is not predictive: on MNIST the minimum norm with no preprocessing is ∼220 and with wavelet preprocessing ∼390, yet test error drops by a factor of 2.
Methods
- Randomization tests on CIFAR10 and ImageNet ILSVRC 2012: train Inception V3 on ImageNet and smaller Inception, AlexNet, and MLPs on CIFAR10 under true labels, partially corrupted labels, random labels, shuffled pixels, random pixels, and Gaussian pixels, toggling data augmentation, weight decay, and dropout.
- Compare training/test accuracy and convergence time; supplement with a finite-sample expressivity theorem for two-layer ReLU networks and with linear/kernel experiments on MNIST and CIFAR10 (Gaussian kernels, random convolutional filters).
Limitations
- Stated: the paper offers no new theory of generalization; it says a full mathematical characterization remains challenging and that it cannot fully survey recent progress. It also states that minimum-norm intuition is only a small piece of the generalization story and that it does not explain why certain architectures generalize better.
- Evident: the empirical scope is limited to image classification benchmarks (CIFAR10, ImageNet) and a small set of architectures/regularizers, so the findings may not extend to other domains or model families. The theoretical construction uses a depth-2 network with large width, and the paper notes depth-k alternatives with O(n/k) parameters per layer.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 Valid · 7 Not valid · 12 Not checked · 85 Own results and interpretations
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In other words, by randomizing labels alone we can force the generalization error of a model to jump up considerably without changing the model, its size, hyperparameters, or the optimizer.”
§ Understanding Deep Learning (Still) Requires Rethinking Generalization > 1.1. The randomization test
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“although it does not reach the perfect 100% top-1 accuracy, 95.20% accuracy is still very surprising for 1.2 million random labels from 1000 categories.”
§ Understanding Deep Learning (Still) Requires Rethinking Generalization > 2. EFFECTIVE CAPACITY OF NEURAL NETWORKS
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The networks fit the corrupted training set perfectly for all the cases.”
§ Understanding Deep Learning (Still) Requires Rethinking Generalization > Partially corrupted labels.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“On the CIFAR10 dataset, Alexnet and MLPs all converge to zero loss on the training set.”
§ Understanding Deep Learning (Still) Requires Rethinking Generalization > 2.1. Fitting random labels and pixels
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We corroborate these experimental findings with a theoretical construction showing that simple depth two neural networks already have perfect finite sample expressivity as soon as the number of parameters exceeds the number of data points as it usually does in practice.”
- Also stated in: § 1.3. Finite sample expressivity · plain
“More formally, we exhibit a very simple two-layer ReLU network with p = 2n + d parameters that can express any labeling of any sample of size n in d dimensions.”
§ Understanding Deep Learning (Still) Requires Rethinking Generalization > Abstract
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 1.3. Finite sample expressivity · plain
70 more claims
Create a free account to see all 104 claims and every source verdict.
- It may state two separate claims as one. 17
- Probably not a substantial claim. 15
- The quote only partly states this claim. 1
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Based on existing research(Saha et al., 2021; Zhang et al., 2021), the following conclusion can be drawn: the gradient updates of a linear layer always lie within the space spanned by its input.”
From Knowledge Decoupling via Orthogonal Projection for Lifelong Editing of Large Language Models (2025) · cites this paper as Zhang et al., 2021
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“If a model is successfully trained on a dataset with random labels, which has been shown to be possible with deep neural networks [294], then it has learned the data generation process by memorizing the random labels.”
From From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI (2023) · cites this paper as [294]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Existing works in privacy-preserving federated learning are mostly developed based on the wellknown privacy-preserving techniques, including: (1) homomorphic encryption (HE), such as Paillier [90], Elgamal [91] and Brakerski-Gentry-Vaikuntanathan cryptosystems [92]; (2) Secure Multiparty Computation (SMC), such as garbled circuits [93] and secret sharing [94]; and (3) differential privacy (DP) [95”
From Privacy and Robustness in Federated Learning: Attacks and Defenses (2024) · cites this paper as [95]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Understanding Deep Learning (Still) Requires Rethinking Generalization > 2.1. Fitting random labels and pixels
“On the CIFAR10 dataset, Alexnet and MLPs all converge to zero loss on the training set.”
§ Understanding Deep Learning (Still) Requires Rethinking Generalization > Rademacher complexity and VC-dimension.
“(fitting random labels) No No 99.82 9.86 MLP 3 × 512 1,735,178 No Yes 100.0 53.35No No 100.0 52.39 (fitting random labels) No No 100.0 10.48 MLP 1 × 512 1,209,866 No Yes 99.80 50.39No No 100.0 50.51 (fitting random labels) No No 99.34 10.61”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Understanding Deep Learning (Still) Requires Rethinking Generalization > 1.1. The randomization test
“More precisely, when trained on a completely random labeling of the true data, neural networks achieve 0 training error.”
§ Understanding Deep Learning (Still) Requires Rethinking Generalization > 2. EFFECTIVE CAPACITY OF NEURAL NETWORKS
“although it does not reach the perfect 100% top-1 accuracy, 95.20% accuracy is still very surprising for 1.2 million random labels from 1000 categories.”
Citations
Cites
(2014) · Cited by 33,419
Wu, Y. (1999) · Technometrics · doi:10.1080/00401706.1999.10485951 · Cited by 27,221
Advances in neural information processing systems 7
Cios, K., & Shields, M. (1997) · Neurocomputing · doi:10.1016/s0925-2312(97)90018-7 · Cited by 22,059
Approximation by superpositions of a sigmoidal function
Cybenko, G. (1989) · Mathematics of Control, Signals, and Systems · doi:10.1007/bf02551274 · Cited by 13,883
Advances in Neural Information Processing Systems 25
Zhang, Y., Sutton, C. A., Storkey, A., & Zoubin Ghahramani, A. (2012) · Cited by 12,653
Understanding deep learning requires rethinking generalization
Zhang, C., Bengio, S., Hardt, M., Recht, B., & Vinyals, O. (2016) · arXiv · doi:10.48550/arxiv.1611.03530 · Cited by 2,838
Self-citationReconciling modern machine-learning practice and the classical bias–variance trade-off
Belkin, M., Hsu, D., Ma, S., & Mandal, S. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1903070116 · Cited by 1,595 · cited 1×
“23, 7, 6, 16 Despite significant progress on theoretical understanding of deep learning in the past few years, a full mathematical characterization of the whole story remains challenging.”
BackgroundA Generalized Representer Theorem
Schölkopf, B., Herbrich, R., & Smola, A. J. (2001) · Lecture Notes in Computer Science · doi:10.1007/3-540-44581-1_27 · Cited by 1,589 · cited 1×
“We have thus derived the “kernel trick” 34 —albeit in a round-about fashion.”
MethodA Closer Look at Memorization in Deep Networks
Arpit, D., Jastrzębski, S., Ballas, N., Krueger, D., Bengio, E., Kanwal, M. S., Maharaj, T., Fischer, A., Courville, A., Bengio, Y., et al. (2017) · arXiv · doi:10.48550/arxiv.1706.05394 · Cited by 1,307
Bartlett, P. L. (1998) · IEEE Transactions on Information Theory · doi:10.1109/18.661502 · Cited by 1,166
Research works printed but not linked: 8
- Cohen, N., Shashua, A. Convolutional rectifier networks as generalized tensor decompositions. In International Conference on Machine Learning. M.F. Balcan and K.Q. Weinberger, eds. 2016, 955--963.
- Cybenko, G. Approximation by superposition of sigmoidal functions. Math. Control Signal. Syst. 4, 2 (1989), 303--314.
- Golowich, N., Rakhlin, A., Shamir, O. Size-independent sample complexity of neural networks. In Conference On Learning Theory. P.R. Sébastien Bubeck, V. Perchet, eds., 2018, 297--299.
- Ma, S., Bassily, R., Belkin, M. The power of interpolation: Understanding the effectiveness of sgd in modern over-parametrized learning. In International Conference on Machine Learning. J. Dy and A. Krause, eds. 2018, 3325--3334.
- Morcos, A., Raghu, M., Bengio, S. Insights on representational similarity in neural networks with canonical correlation. Adv. Neural Inform. Process. Syst.. 2018, 5727--5736.
- Neyshabur, B., Tomioka, R., Srebro, N. Norm-based capacity control in neural networks. In Conference on Learning Theory. S.K. Peter Grünwald and E. Hazan, eds. 2015, 1376--1401.
- Smith, S.L., Le, Q.V. A bayesian perspective on generalization and stochastic gradient descent. In International Conference on Learning Representations, 2018.
- Zhou, W., Veitch, V., Austern, M., Adams, R.P., Orbanz, P. Non-vacuous generalization bounds at the ImageNet scale: A PAC-Bayesian compression approach. In International Conference on Learning Representations, 2019.
Cited by
Items 2,301–2,400 of 2,667
Wang, Y., Wang, W., Liu, D., Hou, W., Zhou, T., & Ji, Z. (2024) · Figshare · doi:10.6084/m9.figshare.26628964
Metric magnitude and topological methods for machine learning and biomedical data analysis
Andreeva, R. (2025) · ERA · doi:10.7488/era/6133
Explore the Use of Time Series Foundation Model for Car-Following Behavior AnalysisPreprint
Zeng, L., & Yan, R. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2501.07034
Methods for Constructing and Exploiting Information Measures for Neural Networks
Flood, P. D. L. (2025) · Apollo (University of Cambridge) · doi:10.17863/cam.119648
Machine Learning in Intensive Care Medicine
Kapral, L. (2025) · reposiTUm (TU Wien) · doi:10.34726/hss.2025.113846
Wang, Y., Wang, W., Liu, D., Hou, W., Zhou, T., & Ji, Z. (2024) · Figshare · doi:10.6084/m9.figshare.24406895
Developing TomoNet: A Deep Learning Feature Extractor for Medical Image Classification
Guerra - Librero Camacho, J. (2023) · doi:10.11575/prism/41484
Wang, Y., Wang, W., Liu, D., Hou, W., Zhou, T., & Ji, Z. (2023) · Figshare · doi:10.6084/m9.figshare.24406642
Yang, J., Triendl, H., Soltan, A. A. S., Prakash, M., & Clifton, D. A. (2024) · Figshare · doi:10.6084/m9.figshare.26122686
Ensemble learning with discrete classifiers on small devices
Buschjäger, S. (2022) · Technische Universität Dortmund Eldorado (Technische Universität Dortmund) · doi:10.17877/de290r-22979 · Cited by 1
Strategies for Generalisable Machine Learning with Small Data for Exercise Fatigue Detection
Jeworutzki, A. (2025) · REPOSIT (Hamburg University of Applied Sciences (HAW Hamburg)) · doi:10.48441/4427.2383 · Cited by 1
Integrace technologií jazykových modelů a LangChain do moderních aplikací
Kratochvíl, P. (2024) · DSpace VŠB-TUO (VŠB-TUO)
Exploring Generalisation Performance through PAC-Bayes
Biggs, F. (2024) · UCL Discovery (University College London) · Cited by 1
A Novel SciML Framework using Mechanistic Constraints
Hughes, G. (2024) · eScholarship (California Digital Library)
Wang, H. (2024) · Chalmers Research (Chalmers University of Technology) · Cited by 2
Automatic segmentation of satellite images using noisy labels
Šuler, K. (2024) · Repository of the University of Ljubljana (University of Ljubljana)
Formal Verification of ML-based Systems in Avionics
Stein, V. (2023) · elib (German Aerospace Center)
Belcher, D. (2025) · ePrints Soton (University of Southampton)
Towards Better Representation Learning in the Absence of Sufficient Supervision
Rahbar, A. (2022) · Chalmers Research (Chalmers University of Technology)
Rakennusten lämpöenergian erittely syväoppimismenetelmin
Forsman, K. (2023) · Aaltodoc (Aalto University)
Simplifying the usage and construction of deep image classification models
Inés, A. (2022) · RIUR (Universidad de La Rioja)
Rejeki, S. (2024) · ePrints - UNY (Yogyakarta State University)
Three Strategies for Salvaging Epistemic Value in Deep Neural Network Modeling
Verreault-Julien, P. (n.d.) · PhilPapers (PhilPapers Foundation) · Cited by 1
Uncanny Performance, Divergent Competence
Johnson, G. M., & Dupré, G. (2026) · Oxford University Press eBooks · doi:10.1093/9780191998317.003.0015
Dui, L. G., Bardi, E., & Antonietti, A. (2025) · Springer eBooks · doi:10.1007/978-3-031-88024-7_3
Enhancing crime prediction with machine vision: Convolutional neural network approach
Akinmuyisitan, T. M. (2024) · Brunel University Research Archive (BURA) (Brunel University London) · Cited by 1
ANALYSIS OF NEURAL NETWORK SOLUTIONS FOR UNMANNED VEHICLE CONTROL
Tereikovskiy, I. A., & Shevchenko, O. V. (2025) · Scientific notes of Taurida National V I Vernadsky University Series Technical Sciences · doi:10.32782/2663-5941/2025.3.2/52
A bearing fault diagnosis method based on MBCNN and Transformer encoder with DANN
Li, X., Lin, P., & Liu, K. (2025) · Youth Academic Annual Conference of Chinese Association of Automation (YAC) · doi:10.1109/yac66630.2025.11150318
“Nevertheless, methods based on This work was supported in part by the National Natural Science Foundation of China under Grant 62203085. deep learning can memorize entire datasets, resulting in poor generalization to new working conditions [16].”
Beyond Control: Will to Power in AI
Hipólito, I., & Podosky, P. C. (2025) · Studies in applied philosophy, epistemology and rational ethics · doi:10.1007/978-3-031-99654-2_4
Subtract the Corruption: Training-Data-Free Corrective Machine Unlearning Using Task Arithmetic
Mozafari, M., Wani, F. A., Silvestri, F., & Bucarelli, M. S. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37657-2_19
“Deep neural networks are known to memorize arbitrary data [1, 72].”
Data-driven real-time prediction of cutting's transport index in directional wellbores
Al-Shargabi, M., Davoodi, S., Wood, D. A., Mehrad, M., & Al-Rubaii, M. (2025) · Petroleum Research · doi:10.1016/j.ptlrs.2025.11.010
From Explanations to Interpretability and Back
Räz, T. (2025) · Synthese Library/Synthese library · doi:10.1007/978-3-032-03083-2_9 · Cited by 2
Ensuring the Trustworthy Development of AI-Based Applications Compatible to Future EASA Regulations
Werner, F. (2025) · elib (German Aerospace Center) · Cited by 2
Winning the Lottery by Preserving Network Training Dynamics with Concrete Ticket SearchPreprint
Arora, T., & Teuscher, C. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2512.07142
“Although traditionally implemented after training due to conventional knowledge that overparameterization was necessary for effective gradient descent [1], [2], Frankle and Carbin [3] provide empirical evidence that this computationally extensive process can be avoided.”
Campolo, A. (2025) · Journal for the History of Knowledge · doi:10.55283/jhk.18717
A precision teaching model for college mathematics based on information technology
Duan, L., Li, B., & Han, X. (2025) · Current Psychology · doi:10.1007/s12144-025-08822-8
“Besides, Zhang et al. (2021) and Xu et al. (2023) have explored precision teaching models leveraging advanced IT to customize educational interventions.”
InfluentialLiu, J. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5971317
Fine-Tuning and Optimizing Lightweight LLMs for Domain-Specific Use Cases
Das, T. K., & Biswas, A. (2025) · Innovations in Power and Advanced Computing Technologies (i-PACT) · doi:10.1109/i-pact65952.2025.11307857
“Another study challenged traditional notions of generalization in deep learning, arguing that deep networks can fit random labels and still generalize well [3].”
Ballester, R., Casacuberta, C., & Escalera, S. (2026) · SpringerBriefs in computer science · doi:10.1007/978-3-032-08283-1_4
Zhang, H., & Shao, Y. (2025) · International Conference on Mathematics and Machine Learning (ICMML) · doi:10.1145/3783779.3783792
Zhang, X., & Zhang, Q. (2026) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2026.113804
A two-stage active cleaning strategy for long-tail label noise
Lin, X., Rong, Z., Li, Y., Yang, Q., Li, P., & Huang, W. (2026) · Neural Networks · doi:10.1016/j.neunet.2026.108585
Hou, S., Jiang, G., Guo, Y., & Wang, W. (2026) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2026.3650911
“Research found that Deep neural networks (DNNs) tend to memorize noisy samples more than clean ones because of the memorization effect [42], [43], [44], [45].”
Wang, L. N., Guo, Z., Li, H., & Chen, C. (2025) · IEEE International Conference on Big Data and Artificial Intelligence (BDAI) · doi:10.1109/bdai66031.2025.11325344
“However, their advanced learning capabilities can sometimes lead to overfitting, especially in the presence of random noise [1].”
Principles of Lipschitz continuity in neural networksPreprint
Luo, R. (2026) · arXiv (Cornell University) · doi:10.13025/30167 · Cited by 1
Adaptive Correction in NARS Inspired by Neural Error Monitoring
Qi, Y., & Li, X. (2025) · IEEE International Conference on Frontiers Technology of Information and Computer (ICFTIC) · doi:10.1109/icftic68075.2025.11324813
“Supervised learning follows an offline paradigm where model parameters are fixed after training, preventing error identification in novel scenarios [15].”
Mahomed, M., Koppel, S., Kaviani, F., & Truelove, V. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6100370
Liu, J. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.5964774
Architectural Trade-offs in Transformer Models: A Controlled Grid Search and Complexity Analysis
Hossain, M. N., Khan, M. R., Rahman, M., & Prianka, I. A. H. (2025) · International Conference on Computing, Communication and Automation · doi:10.1109/iccca66364.2025.11325452
“Nevertheless, scaling introduces considerable challenges: computational cost grows super-linearly with model size [3], while overparameterization increases danger of over-fitting [4], complicating deployment in resource-limited settings.”
Unified Analysis of SGD-Type Methods
Gorbunov, E. (2025) · Springer eBooks · doi:10.1007/978-3-030-54621-2_863-1
A Brief Survey of Emerging Threats to AI Security
Tannous, C., Faraoun, H., & Kallas, K. (2026) · Springer eBooks · doi:10.1007/978-3-031-99447-0_1
Neural Collapse Violated: Why Dirty-Label Backdoor Samples Stand Out in Feature Space
Wang, Y. (2026) · Lecture notes in computer science · doi:10.1007/978-981-95-5764-6_18
Hallo, F., Jażdżewski, T., Bala, P., Korpała, G., & Regulski, K. (2026) · Materials · doi:10.3390/ma19020429 · Cited by 1
“…between the training and test datasets, the inherent sensitivity of the CNN model to edge-related features, the limited dataset size [23], the requirement to resize segments to a fixed 224 × 224 resolution—which may introduce artifacts or reduce spatial fidelity—and the intrinsic…”
DetectiSign: Signature Validation and Forgery Control with ForgeGuard
Rao, V. S., Reddy, T. T., Sneha, Y., Arunlal, C. K., Prasad, K. S., & Vaishnavi, L. (2025) · IEEE International Conference on Information Technology, Electronics and Intelligent Communication Systems (ICITEICS) · doi:10.1109/iciteics64870.2025.11340846
Ensembled Neural Networks Enhanced by Multi-Optimization Strategies for Robust Intelligent Systems
Konduru, R., & Vanitha, M. (2025) · International Conference on Smart Structures and Systems (ICSSS) · doi:10.1109/icsss66939.2025.11346160
“On top of that, the generalization capability of deep networks is still only poorly understood and the robustness of models remains an open problem [6].”
Abnormal Network Traffic Detection based on Data Enhancement and Temporal Convolutional Network
Zhang, B., Deng, Y., & Yan, X. (2025) · International Symposium on AI and Cybersecurity (ISAICS) · doi:10.1109/isaics66888.2025.11350048
Dias, G. H. M., do Rego, T. G., & Malheiros, Y. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-15984-7_34
Automatic Identification of Metastatic Tissue in Lymphatic Node Images Using Machine Learning
Jegatheesan, A., Muthukumar, B., Sugathi, G., Bindu, G., Nafiza, & Jose Anand, A. (2025) · International Conference for Advancement in Technology (ICONAT) · doi:10.1109/iconat66879.2025.11362427
The Role of Computational Parallelism in Artificial Intelligence Learning Models
Sousa, C., Couto, P., Rocha, G. A., Quintas, J. M., & Lopes, S. (2026) · Springer eBooks · doi:10.1007/978-3-031-99147-9_20
Xu, Y., Fang, X., Gong, J., Zhao, Y., Wei, S., Fu, P., & Lv, L. (2026) · Multimedia Systems · doi:10.1007/s00530-025-02142-4
“Traditional methods usually assume that user preferences remain static, which prevents making training and inference more challenging [9], which complicates their application in resource-constrained SR-related scenarios.”
Elderly Fall Prevention Using Advanced Machine Learning Algorithms: A Review
Verma, L., & Pillai, A. (2026) · Algorithms for intelligent systems · doi:10.1007/978-3-032-14156-9_36
The Cross‐Kernel Margin: A Robustness Measure for Quantum Kernel Methods
Govender, S., & Sinayskiy, I. (2026) · Advanced Quantum Technologies · doi:10.1002/qute.70459
“Systematic randomization experiments have highlighted the ability of modern deep learning models to memorize randomly labeled datasets [ 45 ].”
VC-Theoretical Explanation of Double Descent
Cherkassky, V., & Lee, E. H. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-15120-9_9
Xu, D., Wang, Y., Zhang, H., Jia, N., Guo, L., Zhao, D., & Zhang, K. (2026) · Environment Development and Sustainability · doi:10.1007/s10668-025-07306-w · Cited by 1
Correcting noisy labels via comparative distillation: a domain adaptation approach
Feng, Y., Liu, J., & Zhong, H. (2026) · Scientific Reports · doi:10.1038/s41598-026-37935-8
A novel causal meta-learning method based on causality for the prediction of CBM well productivityPreprint
Min, C., Ge, J., Wen, G., Li, X., & Yang, Z. (2026) · Research Square · doi:10.21203/rs.3.rs-7845158/v1
Federated Learning in Healthcare Finance: A Systematic Review of Privacy-Preserving Models
Garg, M., Tank, S., & Tank, B. K. (2026) · Studies in computational intelligence · doi:10.1007/978-3-032-03985-9_34 · Cited by 1
Chen, Z., Haucke, T., Beery, S., Bennett, K., Powell, A., Zuehsow, L., Vincent, R. E., & Deegan, L. A. (2026) · Remote Sensing in Ecology and Conservation · doi:10.1002/rse2.70055
“CV algorithms currently evolve rapidly in image and video processing tasks such as object detection, species classification (Jocher & Qiu, 2024) and multiple object tracking (Aharon et al., 2022; Zhang et al., 2021b).”
Okine, A. N. A., & Garr, D. K. (2026) · Ilomata International Journal of Tax and Accounting · doi:10.61194/ijtc.v6i4.1930
Unlocking the Power of ChatGPT: A Generative AI Technology
KUMAR, A., Makkar, A., Taneja, S., & Sharma, V. (2026) · International Journal Of Informative and Futuristic Research · doi:10.64672/ijifr/26.02.13.06.004
Why Machine Learning Needs Theoretical Guidance To Support Future Theory Building
Xu, H., Zhang, N., & Wang, M. (2025) · MIS Quarterly · doi:10.25300/misq/2025/18900
Dynamic federated learning: coping with the challenge of label noise in real-time network data
Ouyang, C., Wan, J., Zhu, D., Hu, G., Zhou, C., Liu, Z., & Li, T. (2026) · Journal of King Saud University - Computer and Information Sciences · doi:10.1007/s44443-026-00488-x
“When training on noisy data, DNN first learns simpler patterns and then overfits the noisy data (Zhang et al. 2021; Arpit et al. 2017; Patel and Sastry 2023).”
Fundamentals of convolutional neural networks
Bhattacharya, S., Dasgupta, A., Sengupta, A., & Dutta, K. (2026) · Elsevier eBooks · doi:10.1016/b978-0-443-33082-7.00019-x
Clinical AI is Not (Yet) Trustworthy-But It Could Be
Saada, A., Dias, S. B., Alhussein, G., Lyreskog, D. M., Gerasimou, I., Alves, B., de Vos, Μ., Drivas, I., Zaras, J., Stergioulas, A., et al. (2026) · Journal of Medical Internet Research · doi:10.2196/85433
“Closely related is generalization, the model’s ability to extrapolate to unseen data, which remains a fundamental challenge given the risk of underfitting or overfitting, especially in small or biased datasets [19,22,23].”
Fortified Concept Forgetting for text-to-image generative models by machine unlearning on CLIP
Fan, J., Ma, X., Dong, C., Chu, H., & Yang, B. (2026) · Computer Standards & Interfaces · doi:10.1016/j.csi.2026.104142
A Regularization-Sharpness Tradeoff for Linear InterpolatorsPreprint
Hu, Q., & Hodgkinson, L. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2602.12680
“…often operates far beyond this regime: overparameterized models can achieve (near) zero training error and still generalize well, and test error can exhibit double descent around the interpolation threshold (Zhang et al., 2021; Rolnick et al., 2018; Belkin et al., 2019; Nakkiran et al., 2021).”
Liu, J., Yang, K., He, J., Yang, C., Yang, S., & Zhou, Y. (2026) · Neurocomputing · doi:10.1016/j.neucom.2026.133059
Naser, M. Z. (2026) · AI & Society · doi:10.1007/s00146-026-02927-9
“This adherence to standardized protocols aligns with contemporary best practices in machine learning research and ensures reproducibility while mitigating overfitting concerns (Géron et al. 2019; Zhang et al. 2021). in different forms.”
Towards a Representation Framework for EO Deep Learning Models in SDI
Yue, P., Teng, B., Wu, H., & Wang, K. (2026) · Transactions in GIS · doi:10.1111/tgis.70216
“Owing to its strong learning and generalization capabilities (Zhang et al. 2021), it is being widely used in computer vision (Chai et al. 2021), natural language processing (Khan et al. 2023), and recommendation systems (Zhang et al. 2019).”
Harnessing machine learning for decoding Caenorhabditis elegans behavior
Babu, K. R. (2026) · Machine Learning with Applications · doi:10.1016/j.mlwa.2026.100871
Evaluation of the Impact of Image Mutations on the Origin Classification of Digital Images
Koka, V., Ramanand, Mondal, S., & Meena, Y. K. (2026) · Information and Software Technology · doi:10.1016/j.infsof.2026.108071
Xia, Y., Liu, W., Sun, X., Yang, J., Tong, Y., & Zhang, Y. (2026) · Artificial Intelligence Review · doi:10.1007/s10462-026-11523-8
Yang, B., Kim, S.-H., Kim, G.-H., Min, G., Jang, I., Kye, D. E., Kim, K., JUNG, J. Y., Na, J. O., Park, S., et al. (2026) · Scientific Reports · doi:10.1038/s41598-026-38968-9
“This imbalance could predispose the ML models to overfitting 12,13 , although the use of the AUROC curve as a quantitative performance indicator likely mitigated this issue.”
Device Type Classification Using WiFi Probe Requests: From Signals to Insights
Bogahawatta, N., Karunanayaka, Y. S., Seneviratne, S., Thilakarathna, K., Masood, R., Kanhere, S. S., Seneviratne, A. P., & Zomaya, A. Y. (2026) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2026.3668688
“Their effectiveness tends to be limited to specific datasets and exhibits diminished accuracy when scaled across diverse datasets [40].”
Kullanıcı geribeslemeli makine öğrenmesi ile beyin bilgisayar arayüzü tasarımı ve uygulaması
Arı, E. (2023) · Marmara University Open Access System
Palacios, K. D. M. (2025) · SIE (Muisca Goddess of Water) (University of Cundinamarca)
Zhang, W., Fan, W., & Chen, Y. (2026) · Neural Networks · doi:10.1016/j.neunet.2026.108801 · Cited by 2
A Unified preprocessing framework for high-throughput diffraction pattern analysis
Liu, M., Mao, Z., Liu, Z., Guo, J., Zhang, H., Huang, X., Cheng, C., Ding, J. C., Hui, J., Chu, S., et al. (2026) · npj Computational Materials · doi:10.1038/s41524-026-01993-3
Automatic Detection of Beggiatoa Microbial Mats in Salmon Farms Videos
Martínez, N., Moreno, S., Plaza, F., Castillo, J.-E., & Carrasco, M. (2026) · IEEE Access · doi:10.1109/access.2026.3669848
Latif, S., & Ouarda, T. B. M. J. (2026) · Journal of Hydrology Regional Studies · doi:10.1016/j.ejrh.2026.103265
A Mobile Application Approach to Deep Learning for Symptom-Based Disease Detection From User Textual
Garidzira, T. C., & Matobobo, C. (2025) · International Multidisciplinary Information Technology and Engineering Conference (IMITEC) · doi:10.1109/imitec67386.2025.11410504
“This is often an indicator that the model has not yet plateaued, and further training epochs could potentially yield even better results [44].”
Non-asymptotic analysis of online noisy stochastic gradient descent
Bhattacharya, R., & Jiang, T. (2026) · Advances in Applied Probability · doi:10.1017/apr.2026.10056
Neural co-optimization of knots and parameterization for B-spline curve approximation
Tang, W., Zhenqian, E., & Yang, Z. (2026) · Computer-Aided Design · doi:10.1016/j.cad.2026.104059
Emp: enhance memory in data pruning
Xiao, J., Li, P., Nie, J., Ji, B., Li, S., Liu, X., Ma, J., Wu, Q., & Yu, J. (2026) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-025-01183-7
de Freitas, L. A., Ferreira, R. E. P., Alves, A. A. C., Dorea, J. R. R., Paz, C. C. P., & Rosa, G. J. M. (2026) · Preventive Veterinary Medicine · doi:10.1016/j.prevetmed.2026.106843
Risks and Challenges of Secrecy in the Age of Artificial Intelligence
Hu, J., Hu, Y., Tang, X., Feng, Y., Lin, X., & Liu, L. (2025) · International Conference on Intelligent Computing and Next Generation Networks (ICNGN) · doi:10.1109/icngn67480.2025.11413644
Fortifying Robustness in Graph Neural Networks: A Loss Correction Approach to Mitigate Label Noise
Hsieh, I.-C., & Li, C. (2026) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2026.3661886 · Cited by 1
“The inherent noise in neural networks poses various challenges for GNNs, including misdirected learning [39], performance decline [12], which cause the inaccurate aggregation, and harmful edge incorporation.”
Udugama, A., Westendorff, N., Schmitt, O., Dede, J., Manteuffel, C., Wewetzer, D., & Förster, A. (2025) · IEEE Annual Congress on Artificial Intelligence of Things (AIoT) · doi:10.1109/aiot66900.2025.00057
“Zhang et al. (2021) [11] showed that neural networks benefit from label smoothing effects introduced by annotation inconsistency.”
Robust Federated Learning With Double Denoising
Wei, X., Chen, S., Li, X., & Chen, S. (2026) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2026.3673539
“Since the pre-training process is noise-tolerant, when executing (15), the global models do not exhibit significant generalization error [39].”
Singh, J., Khanna, N. N., Singh, R., Mantella, L. E., Johri, A. M., Faa, G., Tiwari, E., Laird, J. R., Al-Maini, M., Isenović, E. R., et al. (2026) · Journal of Applied Biomedicine · doi:10.1016/j.bbe.2026.02.005
Metadata
- Authors
- Zhang, C., Bengio, S., Hardt, M., Recht, B., & Vinyals, O.
- Affiliations
- Google (United States) (US)
- Google Brain, Mountain View, CA
- University of California, Berkeley (US)
- Google DeepMind (United Kingdom) (GB)
- Venue
- Communications of the ACM
- Published
- 2021-02-22
- Volume
- 64
- Issue
- 3
- Pages
- 107-115
- ISSN
- 0001-0782 (ISSN-L)
- 1557-7317
- OpenAlex
- W2566079294
- Cited by
- 2,400
- References
- 47
- Citations by year
- 2026: 295 · 2025: 558 · 2024: 579 · 2023: 500 · 2022: 316 · 2021: 108
- FWCI
- 132.37
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Communications of the ACM (publishedVersion, cc-by-nc-sa)
- Funders
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- generalization, deep neural network, overfitting, overparameterization, expressivity, stochastic gradient descent, regularization, image classification, convolutional neural networks
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-11-08, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-11-08 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.