Understanding deep learning (still) requires rethinking generalization
Open accessThe DOI matches its Crossref record; 45 of the 47 references it cites were found in a registry or an index; none of the 12 sources its claims cite is checked for support yet.
Abstract
Despite their massive size, successful deep artificial neural networks can exhibit a remarkably small gap between training and test performance. Conventional wisdom attributes small generalization error either to properties of the model family or to the regularization techniques used during training. Through extensive systematic experiments, we show how these traditional approaches fail to explain why large neural networks generalize well in practice. Specifically, our experiments establish that state-of-the-art convolutional networks for image classification trained with stochastic gradient methods easily fit a random labeling of the training data. This phenomenon is qualitatively unaffected by explicit regularization and occurs even if we replace the true images by completely unstructured random noise. We corroborate these experimental findings with a theoretical construction showing that simple depth two neural networks already have perfect finite sample expressivity as soon as the number of parameters exceeds the number of data points as it usually does in practice. We interpret our experimental findings by comparison with traditional models. We supplement this republication with a new section at the end summarizing recent progresses in the field since the original version of this paper.
Overview
CoChat Summary
TL;DR
Deep neural networks can fit random labels and random pixels with zero training error, so traditional complexity and regularization explanations do not account for their generalization. The paper argues that explicit regularization is neither necessary nor sufficient, and offers a finite-sample expressivity construction plus linear-model implicit-regularization analysis.
Key findings
- On CIFAR10, Inception, AlexNet, and MLPs reach 100% (or 99.82%/99.34%) training accuracy on random labels while test accuracy is near chance: Inception 9.78%, Inception w/o BatchNorm 10.12%, AlexNet 9.86%, MLP 3×512 10.48%, MLP 1×512 10.61%.
- On ImageNet, Inception reaches 95.20% top-1 accuracy on 1.2 million random labels from 1000 categories without hyperparameter tuning; with explicit regularizers on it still reaches ∼90% top-1.
- Explicit regularization may improve generalization but is neither necessary nor sufficient: on CIFAR10 all models still generalize well with regularizers off; on ImageNet top-1 accuracy without regularization is 59.80% vs 0.1% random guessing, and with data augmentation only Inception reaches 72.95% top-1.
- Theoretically, a two-layer ReLU network with p = 2n + d parameters can represent any labeling of any sample of size n in d dimensions; depth-k networks can use O(n/k) parameters per layer. In linear models, SGD converges to a minimum-norm solution, but minimum norm is not predictive: on MNIST the minimum norm with no preprocessing is ∼220 and with wavelet preprocessing ∼390, yet test error drops by a factor of 2.
Methods
- Randomization tests on CIFAR10 and ImageNet ILSVRC 2012: train Inception V3 on ImageNet and smaller Inception, AlexNet, and MLPs on CIFAR10 under true labels, partially corrupted labels, random labels, shuffled pixels, random pixels, and Gaussian pixels, toggling data augmentation, weight decay, and dropout.
- Compare training/test accuracy and convergence time; supplement with a finite-sample expressivity theorem for two-layer ReLU networks and with linear/kernel experiments on MNIST and CIFAR10 (Gaussian kernels, random convolutional filters).
Limitations
- Stated: the paper offers no new theory of generalization; it says a full mathematical characterization remains challenging and that it cannot fully survey recent progress. It also states that minimum-norm intuition is only a small piece of the generalization story and that it does not explain why certain architectures generalize better.
- Evident: the empirical scope is limited to image classification benchmarks (CIFAR10, ImageNet) and a small set of architectures/regularizers, so the findings may not extend to other domains or model families. The theoretical construction uses a depth-2 network with large width, and the paper notes depth-k alternatives with O(n/k) parameters per layer.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 Valid · 7 Not valid · 12 Not checked · 85 Own results and interpretations
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In other words, by randomizing labels alone we can force the generalization error of a model to jump up considerably without changing the model, its size, hyperparameters, or the optimizer.”
§ Understanding Deep Learning (Still) Requires Rethinking Generalization > 1.1. The randomization test
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“although it does not reach the perfect 100% top-1 accuracy, 95.20% accuracy is still very surprising for 1.2 million random labels from 1000 categories.”
§ Understanding Deep Learning (Still) Requires Rethinking Generalization > 2. EFFECTIVE CAPACITY OF NEURAL NETWORKS
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The networks fit the corrupted training set perfectly for all the cases.”
§ Understanding Deep Learning (Still) Requires Rethinking Generalization > Partially corrupted labels.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“On the CIFAR10 dataset, Alexnet and MLPs all converge to zero loss on the training set.”
§ Understanding Deep Learning (Still) Requires Rethinking Generalization > 2.1. Fitting random labels and pixels
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We corroborate these experimental findings with a theoretical construction showing that simple depth two neural networks already have perfect finite sample expressivity as soon as the number of parameters exceeds the number of data points as it usually does in practice.”
- Also stated in: § 1.3. Finite sample expressivity · plain
“More formally, we exhibit a very simple two-layer ReLU network with p = 2n + d parameters that can express any labeling of any sample of size n in d dimensions.”
§ Understanding Deep Learning (Still) Requires Rethinking Generalization > Abstract
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 1.3. Finite sample expressivity · plain
70 more claims
Create a free account to see all 104 claims and every source verdict.
- It may state two separate claims as one. 17
- Probably not a substantial claim. 15
- The quote only partly states this claim. 1
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Based on existing research(Saha et al., 2021; Zhang et al., 2021), the following conclusion can be drawn: the gradient updates of a linear layer always lie within the space spanned by its input.”
From Knowledge Decoupling via Orthogonal Projection for Lifelong Editing of Large Language Models (2025) · cites this paper as Zhang et al., 2021
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“If a model is successfully trained on a dataset with random labels, which has been shown to be possible with deep neural networks [294], then it has learned the data generation process by memorizing the random labels.”
From From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI (2023) · cites this paper as [294]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Existing works in privacy-preserving federated learning are mostly developed based on the wellknown privacy-preserving techniques, including: (1) homomorphic encryption (HE), such as Paillier [90], Elgamal [91] and Brakerski-Gentry-Vaikuntanathan cryptosystems [92]; (2) Secure Multiparty Computation (SMC), such as garbled circuits [93] and secret sharing [94]; and (3) differential privacy (DP) [95”
From Privacy and Robustness in Federated Learning: Attacks and Defenses (2024) · cites this paper as [95]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Understanding Deep Learning (Still) Requires Rethinking Generalization > 2.1. Fitting random labels and pixels
“On the CIFAR10 dataset, Alexnet and MLPs all converge to zero loss on the training set.”
§ Understanding Deep Learning (Still) Requires Rethinking Generalization > Rademacher complexity and VC-dimension.
“(fitting random labels) No No 99.82 9.86 MLP 3 × 512 1,735,178 No Yes 100.0 53.35No No 100.0 52.39 (fitting random labels) No No 100.0 10.48 MLP 1 × 512 1,209,866 No Yes 99.80 50.39No No 100.0 50.51 (fitting random labels) No No 99.34 10.61”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Understanding Deep Learning (Still) Requires Rethinking Generalization > 1.1. The randomization test
“More precisely, when trained on a completely random labeling of the true data, neural networks achieve 0 training error.”
§ Understanding Deep Learning (Still) Requires Rethinking Generalization > 2. EFFECTIVE CAPACITY OF NEURAL NETWORKS
“although it does not reach the perfect 100% top-1 accuracy, 95.20% accuracy is still very surprising for 1.2 million random labels from 1000 categories.”
Citations
Cites
(2014) · Cited by 33,419
Wu, Y. (1999) · Technometrics · doi:10.1080/00401706.1999.10485951 · Cited by 27,221
Advances in neural information processing systems 7
Cios, K., & Shields, M. (1997) · Neurocomputing · doi:10.1016/s0925-2312(97)90018-7 · Cited by 22,059
Approximation by superpositions of a sigmoidal function
Cybenko, G. (1989) · Mathematics of Control, Signals, and Systems · doi:10.1007/bf02551274 · Cited by 13,883
Advances in Neural Information Processing Systems 25
Zhang, Y., Sutton, C. A., Storkey, A., & Zoubin Ghahramani, A. (2012) · Cited by 12,653
Understanding deep learning requires rethinking generalization
Zhang, C., Bengio, S., Hardt, M., Recht, B., & Vinyals, O. (2016) · arXiv · doi:10.48550/arxiv.1611.03530 · Cited by 2,838
Self-citationReconciling modern machine-learning practice and the classical bias–variance trade-off
Belkin, M., Hsu, D., Ma, S., & Mandal, S. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1903070116 · Cited by 1,595 · cited 1×
“23, 7, 6, 16 Despite significant progress on theoretical understanding of deep learning in the past few years, a full mathematical characterization of the whole story remains challenging.”
BackgroundA Generalized Representer Theorem
Schölkopf, B., Herbrich, R., & Smola, A. J. (2001) · Lecture Notes in Computer Science · doi:10.1007/3-540-44581-1_27 · Cited by 1,589 · cited 1×
“We have thus derived the “kernel trick” 34 —albeit in a round-about fashion.”
MethodA Closer Look at Memorization in Deep Networks
Arpit, D., Jastrzębski, S., Ballas, N., Krueger, D., Bengio, E., Kanwal, M. S., Maharaj, T., Fischer, A., Courville, A., Bengio, Y., et al. (2017) · arXiv · doi:10.48550/arxiv.1706.05394 · Cited by 1,307
Bartlett, P. L. (1998) · IEEE Transactions on Information Theory · doi:10.1109/18.661502 · Cited by 1,166
Research works printed but not linked: 8
- Cohen, N., Shashua, A. Convolutional rectifier networks as generalized tensor decompositions. In International Conference on Machine Learning. M.F. Balcan and K.Q. Weinberger, eds. 2016, 955--963.
- Cybenko, G. Approximation by superposition of sigmoidal functions. Math. Control Signal. Syst. 4, 2 (1989), 303--314.
- Golowich, N., Rakhlin, A., Shamir, O. Size-independent sample complexity of neural networks. In Conference On Learning Theory. P.R. Sébastien Bubeck, V. Perchet, eds., 2018, 297--299.
- Ma, S., Bassily, R., Belkin, M. The power of interpolation: Understanding the effectiveness of sgd in modern over-parametrized learning. In International Conference on Machine Learning. J. Dy and A. Krause, eds. 2018, 3325--3334.
- Morcos, A., Raghu, M., Bengio, S. Insights on representational similarity in neural networks with canonical correlation. Adv. Neural Inform. Process. Syst.. 2018, 5727--5736.
- Neyshabur, B., Tomioka, R., Srebro, N. Norm-based capacity control in neural networks. In Conference on Learning Theory. S.K. Peter Grünwald and E. Hazan, eds. 2015, 1376--1401.
- Smith, S.L., Le, Q.V. A bayesian perspective on generalization and stochastic gradient descent. In International Conference on Learning Representations, 2018.
- Zhou, W., Veitch, V., Austern, M., Adams, R.P., Orbanz, P. Non-vacuous generalization bounds at the ImageNet scale: A PAC-Bayesian compression approach. In International Conference on Learning Representations, 2019.
Cited by
Items 2,501–2,600 of 2,667
Perera, D., Moura, V., dos Santos, L. I. A., Haddad, M. F. C., & Figueiredo, F. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.21692
“Neural networks combine strong memorization capabilities with architectural and optimization biases that shape their behavior outside of the training dataset (Hornik, 1991; Kaplan et al., 2020b; Kubo et al., 2019; Zhang et al., 2021).”
Proximal regularization of deep residual neural networks applied to high-dimensional genomic data
Fan, Y., Launonen, I. K., Sillanpää, M. J., & Waldmann, P. (2026) · Briefings in Bioinformatics · doi:10.1093/bib/bbag246
“Other widely used regularization methods include stochastic gradient descent [ 15] and weight decay [ 16].”
Forecasting with Deep Pooled Panel Neural Networks
Chronopoulos, I., Chrysikou, K., Kapetanios, G., Mitchell, J., & Raftapostolos, A. (2026) · Econometric Reviews · doi:10.1080/07474938.2026.2660660
Harnessing variational auto-encoder for binary classification in extremely low data regime
Nedbal, R., & Mahdian, B. (2026) · Progress in Artificial Intelligence · doi:10.1007/s13748-026-00442-y
Rethinking Weak Supervision in Anomaly Detection: A Comprehensive Benchmark
Yao, X., Zhou, S., Wu, Z., Hou, C., Liang, S., Wang, S., Huang, H., Han, S., & Jiang, M. (2026) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3770855.3817536
“Although robust learning has been extensively studied in classification [45, 57], its application to AD remains less systematic, with emerging work on noise-tolerant losses and confidence-based filtering [43].”
Liu, R., Li, T. D., Migdal, J., Meszaros, K., & Dardik, T. (2026) · IEEE International Conference on Mechatronics and Automation (ICMA) · doi:10.1109/icma69663.2026.11647374
Color Difference Histogram Capsule Network (CDH-CapsNet) for Plant Disease Recognition
Okyere-Gyamfi, S., Asante, M., Peasah, K. O., Missah, Y. M., & Akoto-Adjepong, V. (2026) · International Journal of Intelligent Systems and Applications · doi:10.5815/ijisa.2026.03.03
“• Area under the curve (AUC) : To evaluate the model’s effectiveness on datasets with uneven class distributions, Receiver Operating Characteristic (ROC) and precision-recall (PR) curves are generated [30,31].”
Феррара, М. (2026) · Neurocomputing · doi:10.1016/j.neucom.2026.134176
Tirupati, K. K., Methuku, N. K., Chilukury, P. K., & Reddy, S. R. B. (2026) · International Conference on Artificial Intelligence, Systems, and Emerging Technologies (ICAISET) · doi:10.1109/icaiset66439.2026.11541930
Hart, S. N., & Oberg, A. L. (2026) · BMC Medical Research Methodology · doi:10.1186/s12874-026-02894-1
“Networks with billions of parameters generalize well despite p ≫ n [26].”
Ma, J., Li, M., Liao, M., (20030), W. L., Yang, Y., Wang, Y., Zhu, Z., & Wu, H. (2026) · Measurement · doi:10.1016/j.measurement.2026.122155
Formal Concept Lattices are Good Semantic Scaffolds for Concept-Based LearningPreprint
Vemuri, D. S., Adhikari, S., Saha, A., Kher, K. V., & Balasubramanian, V. N. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.05471
“In general, multiple parameter configurations can attain the same empirical risk (Zhang et al., 2021).”
Προηγμένες Τεχνικές Ανάλυσης και Επεξεργασίας Καρδιολογικών Δεδομένων
Mavridis, C. (2026) · DSpace - NTUA (National Technical University of Athens) · doi:10.26240/heal.ntua.32538
It Takes One to Bias Them All: Breaking Bad with One-Shot GRPOPreprint
Deng, N., Zhu, Y., Shi, N., Scott, C., & Mihalcea, R. F. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.10931
“Learning with label noise has been extensively studied from both theoretical and algorithmic perspectives (Angluin & Laird, 1988; Liu & Tao, 2015; Zhang et al., 2021a; Zhu et al., 2024).”
LIMITAÇÕES NA APLICABILIDADE DE INTELIGÊNCIA ARTIFICIAL
de Araújo, D. B. M., Amate, M. C., & dos Santos, R. C. (2026) · Revista Acadêmica Online · doi:10.36238/2359-5787.2026.v12n63.2215
A stochastic gradient algorithm for non-separable optimization with convergence guaranteePreprint
Li, Y., & Wu, R. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.10383
Active forgetting with selective labeling for multi-task learning
Lee, J., & Kim, E. (2026) · Neurocomputing · doi:10.1016/j.neucom.2026.134231
FDDPG: a federated deep deterministic policy gradient-based method for attack detection in IoV
Wang, S., Cong, L., Liu, X., Hong, Y., & Wang, R. (2026) · The Journal of Supercomputing · doi:10.1007/s11227-026-08607-1
“Insuf-ficient generalization capability and the risk of overfitting [19] remain fundamental theoretical challenges.”
Deep Neural Networks, Architectural Constraints, and Scientific Representation
Kieval, P. H. (2026) · PhilSci-Archive (University of Pittsburgh)
Sarutipaisan, N., Maksap, P., Keawphinit, C., & Watanapa, B. (2026) · doi:10.1145/3816713.3816783
Kucherov, D., Dolgikh, S., & Podelskyi, S. (2026) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2026.1636604
Sludnikov, V., Ovseenko, G. A., & Mukhamadiev, A. R. (2026) · International Conference on Industrial Engineering, Applications and Manufacturing (ICIEAM) · doi:10.1109/icieam69213.2026.11549892
“%) is consistent with the findings of Arpit et al. [7] and Zhang et al. [8], who showed that overparameterized networks can memorize arbitrary labels at the expense of generalization ability.”
InfluentialExplainability and Interpretability in Machine Learning and Artificial Intelligence
Rowat, C. (2026) · Oxford University Press eBooks · doi:10.1093/9780197851418.003.0994
Random matrix perspective in deep learning: spectral analysis, training dynamics, and generalization
Li, H., & Bai, Z. (2026) · Handbook of statistics · doi:10.1016/bs.host.2026.05.003
Alkhatib, A., Alkhatib, S., Alhobeira, H. A., & Bijle, M. N. (2026) · Research Square · doi:10.21203/rs.3.rs-9878062/v1
SG-IRM: Sparsity-Gated Invariant Risk Minimization
Zhang, X., & Su, J. (2026) · Chinese Control and Decision Conference (CCDC) · doi:10.1109/ccdc69976.2026.11560490
CPLN: Consistent pseudo-label noise learning for semi-supervised medical image classification
Liu, J., Wang, Y., He, D., Zhou, J., & Pan, Z. (2026) · Pattern Recognition · doi:10.1016/j.patcog.2026.114270 · Cited by 1
Yang, Y.-C., Lee, M.-C., Pan, C., & Shiue, Y. (2026) · Scientific Reports · doi:10.1038/s41598-026-59470-2
“Excessive epochs increase the risk of overfitting; prior research suggests that model performance typically peaks near 250 epochs, beyond which further gains are marginal 25 .”
FedLID: Leveraging Dimensionality to Overcome Noisy Labels in Federated Learning
Kang, Y., Hu, C., & Li, B. (2026) · IEEE Transactions on Cloud Computing · doi:10.1109/tcc.2026.3708715
“Evenasmallfractionofsuchmislabeled datacanseverelydegrademodelperformance,asdeepneural networkshaveastrongtendencytomemorizeincorrectlabelsat differentlevelsofrepresentation[3].”
Memorization of Training Data of Large Language Models: A Survey
Ishihara, S., & Takahashi, H. (2026) · Transactions of the Japanese Society for Artificial Intelligence · doi:10.1527/tjsai.41-4_an40-f
A Dual-Branch Fusion Framework for Height Estimation From a Single Satellite Image
Ma, W., Yan, Y., Zhang, Z., Gao, F., He, Y., & Zhang, Q. (2026) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2026.3709390
“These issues make it difficult for traditional methods and early deep learning models to achieve both high accuracy and strong detail recovery capabilities [10], [11], [12].”
Artificial Intelligence and Deep Learning in Plant Disease Detection and Diagnosis
Mondal, A., Sadhu, S., Dey, S., Biswas, A., & Chowdhury, A. (2026) · doi:10.52756/efahciai.2026.e01.002
Zhang, Y., Zhao, J., Chen, L., Li, M., Han, J. J., Lin, Q., Wu, B., & Hu, Z. (2026) · IEEE Access · doi:10.1109/access.2026.3710417
Unsupervised Attenuation of Aliased Ground Roll in Undersampled 3-D Land Seismic Data
Li, J., Liu, D., & Trad, D. O. (2026) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2026.3710605
“This notion of frequency is closely related to the low-frequency bias observed in conventional CNN-based seismic networks, in that both favor the early learning of smooth structures [48].”
Xu, Z., Zhong, H., & Lu, Y. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7076334
Comparing Optimization Algorithms for Training Deep Neural Networks: A Large-Scale Empirical Study
Sridharan, V. K., Chaubey, P. K., Imran, S. M., Anandhasilambarasan, D., Singh, A. K., & Sharma, M. K. (2026) · International Conference on AI, Data Science, Cyber Security and Smart Manufacturing for Sustainable Development (ICADCS) · doi:10.1109/icadcs70036.2026.11583709
“Deep Neural Networks (DNNs) are commonly considered as the most efficient learning device when it comes to working with large datasets, and it has been already effectively employed in thousands of applications in many different spheres [1,2].”
AGAM: A randomly initialized policy network for action mask fusion in reinforcement learning
Xue, B., Chen, Q., Wang, Z., Du, L., Zhao, J., & Hu, T. (2026) · Knowledge-Based Systems · doi:10.1016/j.knosys.2026.116563
Early Stopping Without Validation Data in Weakly Supervised Learning
Yuan, S., Li, M., Feng, L., Han, B., Niu, G., Sugiyama, M., & Liu, T. (2026) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2026.3712551
From gene to enzyme: Microbial engineering of bovine chymosin for efficient cheese coagulation
Amjad, H., Nisar, U., Yaseen, A. R., Saleem, F., Ahmad, M., & Yaqoob, T. (2026) · World Journal of Microbiology and Biotechnology · doi:10.1007/s11274-026-05128-0
Poornima, D., Harshana, C., Sharma, K., Yadav, K., Suma, V., & Shobha, N. (2025) · International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC) · doi:10.1109/i-smac65734.2025.11597669
Hersko, I., Grolman, E., Cahana, Y., Shabtai, A., & Elovici, Y. (2026) · IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW) · doi:10.1109/icstw72326.2026.00053
Bacanto, J. R. D., Meraña, J. M., Simbajon, C. M. G., & Pacaon, M. R. S. (2026) · International Conference on Frontiers of Engineering and Emerging Technologies (FET) · doi:10.1109/fet68771.2026.11601212
“A study applied an improved CBAM-ResNet-18 model for apple leaf disease identification and reported that the attention module enh anced the network’s ability to focus on diseased regions and extract more discriminative spatial-channel features, resulting in 98.44 % accuracy [15].”
InfluentialDecentralized Machine Learning in Healthcare: Federated Learning for IoMT and Beyond
MUGIGAYI, F., Karacan, H., Alli, A. A., Ali, A. B., & Mwondi, L. A. (2026) · International Symposium on AI-Driven Engineering Systems (ISADES) · doi:10.1109/isades69945.2026.11608204
“Validation performance remained stable and closely tracked training performance, signaling effective generalization and appropriate regularization through the federated learning process[37].”
Deonarain, G., & Blough, J. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7154124
A Hybrid LSTM-Enhanced Reinforcement Learning Framework for Intelligent Smart Agriculture
Ghodbane, I., Gaba, H. L., Ghalmi, I., Azizi, N., Guerch, L. D., Dendani, N., & Bounour, M. O. (2026) · International Conference on Pattern Analysis and Intelligent Systems (PAIS) · doi:10.1109/pais69842.2026.11605305
Heterogeneous transfer learning for robust infectious disease forecasting: A data-centric approach
Lee, D.-H. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.133728
Benchmarking Hybrid Deep Learning Architectures for Predictive Maintenance in Industry 4.0
Gu, Z. C., Hernandez, J. E., Cook, T., Burtenshaw, J., Scott, S., & Couch, C. (2026) · IEEE International Conference on AI and Data Analytics (ICAD) · doi:10.1109/icad69378.2026.11608648
Zhang, G., Yin, G., Wang, Y., Zhang, Y., Hu, J., Sun, G., Ni, Y., Chen, R., Zhao, W., Wang, M., et al. (2026) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2026.3715899
“The success of this strategy is consistent with the latest theory of neural network optimization, that is, low-level features have stronger generality, while high-level features are more suitable for specific tasks [69].”
Lv, Y., Hong, Z., Zhang, Z., Li, L., & Zhu, Z. (2026) · IEEE Data Driven Control and Learning Systems Conference (DDCLS) · doi:10.1109/ddcls71227.2026.11610251
The nuclear route: sharp asymptotics of ERM in overparameterized quadratic networks *
Erba, V., Troiani, E., Zdeborová, L., & Krząkała, F. (2026) · Journal of Statistical Mechanics Theory and Experiment · doi:10.1088/1742-5468/ae8248 · Cited by 2
The curve-fitting problem revisited
Buchholz, O. (2026) · European Journal for Philosophy of Science · doi:10.1007/s13194-026-00772-7
“42 For an example, see Shamir (2023, Lemma 7) to ensure that they make accurate predictions (Bartlett et al., 2021, p. 116; Zhang et al., 2021, p. 112).”
Bozhbanbayeva, N., Abilbayeva, A., Tarabayeva, A., Isgandarov, I., Tolykbayev, T., Адилбекова, И. М., & Yelyubayeva, D. (2026) · Frontiers in Pediatrics · doi:10.3389/fped.2026.1801840
Customer Sentiment Analysis of E-Commerce Products Using the Naïve Bayes Method and Word Embedding
Harpad, B., Azahari, A., & Salmon, S. (2025) · Building of Informatics Technology and Science (BITS) · doi:10.47065/bits.v7i3.8879 · Cited by 1
“The trained model is tested using the test dataset to assess its generalization capability on unseen data[41][42].”
Qodri, K. N., Desmana, S., Putri, B. A., Ratih, R., & Muin, M. A. (2026) · Blend Sains Jurnal Teknik · doi:10.56211/blendsains.v5i1.1841
“…terhadap kurva pelatihan memberikan wawasan yang bermanfaat mengenai dinamika optimasi dan membantu memverifikasi bahwa perbedaan kinerja yang teramati berasal dari peningkatan pembelajaran yang sesungguhnya dan bukan sekadar artefak dari inisialisasi acak atau ketidakstabilan optimasi [32].”
Multi-Round Meta-Learning for Label Purification Without Trusted Labels
Otsu, M., Zhou, J., Han, B., & Pan, S. (2026) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2026.3718926
Modeling diesel output particulate matter as the Ornstein-Uhlenbeck process
Bolt, M., Alberts, A., Desai, A. S., Meckl, P. H., & Bilionis, I. (2026) · International Journal of Engine Research · doi:10.1177/14680874261471315
A Unified Framework of Strong and Weak Constraints for Robust Multimodal Learning
Zhang, H., Huang, S., Wang, Y., & Kuruoğlu, E. E. (2026) · Multidimensional Signal Processing Workshop · doi:10.1109/ivmsp68335.2026.11618812
Bhandari, H. (2026) · Digital Commons-DePaul (DePaul University)
EMASAM: a Computationally Efficient Sharpness-Aware Minimization via EMA-Guided Perturbations
Ratchatorn, T., & Tanaka, M. (2026) · International Conference on Pattern Recognition · doi:10.1007/978-3-032-31666-0_12
“This phenomenon, referred to as overfitting, results in a gap in performance between training and testing datasets [38].”
Singh, D., Mohanta, A., Tymofyeyeva, E., & Singh, S. (2026) · International Journal of Remote Sensing · doi:10.1080/01431161.2026.2709712
“…have been applied in energy resource exploration and management through surface deformation analysis and structural mapping, while its capabilities in land-use and land-cover change detection have supported sustainable land management, mining regulation, and urban planning (C. Zhang et al. 2021).”
Yagual, D. Q., Fernandez-Iglesias, D., Nóvoa, F., & Garabato, D. (2026) · International Conference on Data Technologies and Applications · doi:10.3390/data11080199
“Taken together, these results indicate that the dataset preserves realistic traffic characteristics while maintaining sufficient discriminative structure for anomaly detection research in encrypted network environments [28].”
Метод та модель автономного управління БПЛА
Шевченко, О. В. (2025) · Електронний архів наукових та освітніх матеріалів КПІ ім. Ігоря Сікорського (КПІ ім. Ігоря Сікорського)
NEADS - an Improved Neuro-Evolution Method for Self-perfecting Modeling
Lyubchev, D., & Marchev, A. (2026) · doi:10.1109/icarai70085.2026.11635704
CoRas: Cooperative Rationale Selection for Graph Rationale Learning in Out-of-Distribution Graphs
Yao, F., Yue, L., Gao, W., Liu, Q., & Chen, E. (2026) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2026.3722666
“Small-loss criterion is widely used in learning with noisy labels, where instances with small training loss are clean [45], [46].”
Scaling Laws of Deep-Learning Neural Networks: Taxonomy and Survey
Wang, C., & Lin, Y. (2026) · Springer eBooks · doi:10.1007/978-981-95-8110-8_4
Scaling Laws of Deep Learning Neural Networks: Information Loss
Wang, C., & Lin, Y. (2026) · Springer eBooks · doi:10.1007/978-981-95-8110-8_6
TemplateQO: Template-Aware and Scalable Query Optimization with Data-Efficient Learning
Zheng, P., Li, G., Xu, L., Zhu, R., Li, Y., & Zheng, B. (2026) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde65706.2026.00061
Zhang, Z., & Ma, W. (2026) · Proceedings - International Conference on Image Processing · doi:10.1109/icip61757.2026.11630126
Learning Generalizable Reconstruction of High-Dimensional Neural DynamicsPreprint
Kujur, A., & Monfared, Z. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2608.16569
“High-dimensional time series arise throughout science, including neuroscience, biology, climate science, and engineering, where a central objective is to discover compact representations that preserve the essential Preprint to the underlying physiological dynamics Zhang et al. (2021).”
Xu, J., & Jia, X. (2026) · Wireless networks · doi:10.1007/978-3-032-30978-5_1
Sambyal, S., Shastri, S., Taloor, A. K., Kumar, A., & Mansotra, V. (2026) · Proceedings of the Geologists Association · doi:10.1016/j.pgeola.2026.101224
Amin, R., Rahman, Q. M., Rana, M. M. R., & Aktar, S. (2026) · Machine learning for computational science and engineering · doi:10.1007/s44379-026-00088-6
Qu, H., Guo, Q., & Zou, J. (2026) · Remote Sensing · doi:10.3390/rs18162833
Wu, Y., & Liu, H. (2026) · International Conference on IoT, Data Science and Advanced Computing (IDSAC) · doi:10.1109/idsac70973.2026.11650646
An inductive bias for generalization in mouse olfactory learningPreprint
Xia, N., & Murthy, V. N. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.08.17.745288
“However, it remains uncertain how much the state-of-art models (including generative AIs) are still merely memorizing the training data 4–6 .”
Volatility Smiles and Structural Non-Identification under Black-Scholes InversionPreprint
Pereira, R. R. F. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7319498
Sun, P., Jia, X., Huang, L., Tan, W., Gao, W., & Li, Y. (2026) · Signal Processing · doi:10.1016/j.sigpro.2026.110883
Uncertainty-aware human mask segmentation and pose estimation using Radar heatmapsPreprint
Kartik, K., Karmakar, C. K., Nasirzadeh, F., Brusey, J., & Herring, D. H. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7331692
Certainty-Aware Partition and Sufficient Utilization of Noisy Samples
Zhang, J., Jiang, G., Hou, S., Zhang, H., & Wang, W. (2026) · Machine Learning · doi:10.1007/s10994-026-07138-3
“…Negative-LS (Wei et al., 2021 ), VolMinNet (Li et al., 2021 ), CAL (Zhu et al., 2021 ), MOIT (Ortego et al., 2021 ), RoLT (Wei et al., 2021 ), PLC (Zhang et al., 2021 ), RRL (Li and Hoi, 2021 ), PES (Bai et al., 2021 ), CORES* (Cheng et al., 2021 ), DISC (Li et al., 2023 ), LongReMix (Cordeiro et…”
Charles, M., Banda, K. E., Chundu, M. L., Muchanga, M., Samboko, H., Zimba, H. M., & Simangolwa, F. K. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7352272
Antor, M. S. U. M., Imran, T. A., Hossain, M. R., Podder, N. K., & Khan, I. (2026) · 2026 International Conference on Power, Electronics, Communications, Computing, and Intelligent Infrastructure (PECCII) · doi:10.1109/peccii70991.2026.11661945
“Classical Convolutional Neural Networks (CNNs) have become the benchmark of processing high-dimensional images and thus far, the rapid progress in deep learning has transformed the sphere of computer vision, marking its revolution [1], [2].”
Shang, F., Li, Z., Qin, X., Gao, T., Liu, Z., Meng, X., & Yang, M. (2026) · Measurement Science and Technology · doi:10.1088/1361-6501/ae9f85
“Successful generalization testing confirmed that the network did not merely perform a memory fit of the training samples [41], but rather learned the intrinsic mapping relationship between the weld parameter space and the wavefield response.”
Robust discharge coefficient modeling in triangular weirs via an enhanced Adam-optimized BP network
Zeng, G., Xu, M., & Zheng, Y. (2026) · Frontiers in Water · doi:10.3389/frwa.2026.1890874
Neuromimetic Signal Processing for Bearing Fault Diagnosis Based on Second-Order CyclostationarityPreprint
Gao, S., Antoni, J., Peng, Z., & Borghesani, P. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7382644
The Accommodation Head: A Projected Field for Monitoring and Steering Latent Space OrganisationPreprint
Galvez, J. J. H., Pérez, R. C., Cabrera, J. J. H., Gómez, J. É., Caballero, M., & Andrés, O. R. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7390616
High-Frequency Guided Feature Aggregation for Strip Steel Surface Defect Detection
Sun, Y., Xu, C., Guo, H., Zhang, L., Feng, Y., Zhang, W., & Tang, Y. (2026) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2026.3728911
GMM-BiLSTM: A Novel Framework for Robust Stock Price Forecasting Using GMM-based Data Augmentation
Zhu, J. (2026) · International Conference on Innovation Management and Information Systems (ICIMIS) · doi:10.1145/3821966.3821972
Yan, Y., Wang, J., Miao, J., & Xu, Z. (2026) · Smart Agricultural Technology · doi:10.1016/j.atech.2026.102618
Synthetic training data for neural networks in optical scanners used in the foundry industry
Zender, J., Pinzer, B., Herkommer, A., & Layh, M. (2026) · Machine Vision and Applications · doi:10.1007/s00138-026-01900-2
Wang, Z., Guo, B., Liu, S., Yu, Z., & Zhang, D. (2026) · IEEE Internet of Things Journal · doi:10.1109/jiot.2026.3729805
Large deviations for linear regressionsPreprint
Bartolucci, S., Caccioli, F., Caravelli, F., & Vivo, P. (2026) · arXiv (Cornell University) · doi:10.13140/rg.2.2.16426.99522
Nature-Inspired Algorithms in Data Analytics: A Unified Cross Domain Benchmarking Framework
Fawei, B. (2026) · INTERNATIONAL JOURNAL OF COMPUTER SCIENCE AND MATHEMATICAL THEORY E-ISSN · doi:10.56201/ijcsmt.vol.12.no4.2026.pg95.107
DNN vs CNN for YouTube Shorts Abusive Word Classification
Pratama, E. F., Setyawan, R. A., & Bororing, J. E. (2026) · Journal of System and Computer Engineering (JSCE) · doi:10.61628/jsce.v7i3.2759
EEG Based Emotion Recognition with Hybrid CNN-LSTM Deep Learning Model
Selvi, R., & Vijayakumaran, C. (2026) · International Journal of Automation and Smart Technology · doi:10.5875/2ae4y165
Structural knowledge-guided complex-noise handling for open intent classification
Li, Y., Ouyang, X., Zhang, J., Pan, C. L., Ren, L., & Yang, X. (2026) · Neural Networks · doi:10.1016/j.neunet.2026.109588
Li, M., Wang, L., Huang, S., & Zhang, Q. (2026) · Pattern Recognition · doi:10.1016/j.patcog.2026.114848
Kogitans, A., Nurcholis, M., Khalifah, N., Sayyidina, N., & Riza, D. F. A. (2026) · Journal of Information Technology and Computer Science · doi:10.25126/jitecs.2026112666
Default Assumptions as Hidden Parameters: A Theory of Implicit Priors in Materials AI
Mansour, A., & Saeed, O. (2022) · Journal of Artificial Intelligence for Materials Science · doi:10.68159/j862290346
More Data Is Not Enough: Neural Network Robustness Under Data Degradation
York, C., & Bihl, T. (2026) · IEEE National Aerospace and Electronics Conference (NAECON) · doi:10.1109/naecon70028.2026.11674937
A Novel Differentially Private Low Rank Weight Fusion Approach
Yao, Y., Wang, Y., Huang, C., Ji, W., & He, D. (2026) · Chinese Journal of Electronics · doi:10.23919/cje.2024.00.123
Metadata
- Authors
- Zhang, C., Bengio, S., Hardt, M., Recht, B., & Vinyals, O.
- Affiliations
- Google (United States) (US)
- Google Brain, Mountain View, CA
- University of California, Berkeley (US)
- Google DeepMind (United Kingdom) (GB)
- Venue
- Communications of the ACM
- Published
- 2021-02-22
- Volume
- 64
- Issue
- 3
- Pages
- 107-115
- ISSN
- 0001-0782 (ISSN-L)
- 1557-7317
- OpenAlex
- W2566079294
- Cited by
- 2,400
- References
- 47
- Citations by year
- 2026: 295 · 2025: 558 · 2024: 579 · 2023: 500 · 2022: 316 · 2021: 108
- FWCI
- 132.37
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Communications of the ACM (publishedVersion, cc-by-nc-sa)
- Funders
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- generalization, deep neural network, overfitting, overparameterization, expressivity, stochastic gradient descent, regularization, image classification, convolutional neural networks
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-11-08, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-11-08 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.