Understanding deep learning (still) requires rethinking generalization
Open accessThe DOI matches its Crossref record; 45 of the 47 references it cites were found in a registry or an index; none of the 12 sources its claims cite is checked for support yet.
Abstract
Despite their massive size, successful deep artificial neural networks can exhibit a remarkably small gap between training and test performance. Conventional wisdom attributes small generalization error either to properties of the model family or to the regularization techniques used during training. Through extensive systematic experiments, we show how these traditional approaches fail to explain why large neural networks generalize well in practice. Specifically, our experiments establish that state-of-the-art convolutional networks for image classification trained with stochastic gradient methods easily fit a random labeling of the training data. This phenomenon is qualitatively unaffected by explicit regularization and occurs even if we replace the true images by completely unstructured random noise. We corroborate these experimental findings with a theoretical construction showing that simple depth two neural networks already have perfect finite sample expressivity as soon as the number of parameters exceeds the number of data points as it usually does in practice. We interpret our experimental findings by comparison with traditional models. We supplement this republication with a new section at the end summarizing recent progresses in the field since the original version of this paper.
Overview
CoChat Summary
TL;DR
Deep neural networks can fit random labels and random pixels with zero training error, so traditional complexity and regularization explanations do not account for their generalization. The paper argues that explicit regularization is neither necessary nor sufficient, and offers a finite-sample expressivity construction plus linear-model implicit-regularization analysis.
Key findings
- On CIFAR10, Inception, AlexNet, and MLPs reach 100% (or 99.82%/99.34%) training accuracy on random labels while test accuracy is near chance: Inception 9.78%, Inception w/o BatchNorm 10.12%, AlexNet 9.86%, MLP 3×512 10.48%, MLP 1×512 10.61%.
- On ImageNet, Inception reaches 95.20% top-1 accuracy on 1.2 million random labels from 1000 categories without hyperparameter tuning; with explicit regularizers on it still reaches ∼90% top-1.
- Explicit regularization may improve generalization but is neither necessary nor sufficient: on CIFAR10 all models still generalize well with regularizers off; on ImageNet top-1 accuracy without regularization is 59.80% vs 0.1% random guessing, and with data augmentation only Inception reaches 72.95% top-1.
- Theoretically, a two-layer ReLU network with p = 2n + d parameters can represent any labeling of any sample of size n in d dimensions; depth-k networks can use O(n/k) parameters per layer. In linear models, SGD converges to a minimum-norm solution, but minimum norm is not predictive: on MNIST the minimum norm with no preprocessing is ∼220 and with wavelet preprocessing ∼390, yet test error drops by a factor of 2.
Methods
- Randomization tests on CIFAR10 and ImageNet ILSVRC 2012: train Inception V3 on ImageNet and smaller Inception, AlexNet, and MLPs on CIFAR10 under true labels, partially corrupted labels, random labels, shuffled pixels, random pixels, and Gaussian pixels, toggling data augmentation, weight decay, and dropout.
- Compare training/test accuracy and convergence time; supplement with a finite-sample expressivity theorem for two-layer ReLU networks and with linear/kernel experiments on MNIST and CIFAR10 (Gaussian kernels, random convolutional filters).
Limitations
- Stated: the paper offers no new theory of generalization; it says a full mathematical characterization remains challenging and that it cannot fully survey recent progress. It also states that minimum-norm intuition is only a small piece of the generalization story and that it does not explain why certain architectures generalize better.
- Evident: the empirical scope is limited to image classification benchmarks (CIFAR10, ImageNet) and a small set of architectures/regularizers, so the findings may not extend to other domains or model families. The theoretical construction uses a depth-2 network with large width, and the paper notes depth-k alternatives with O(n/k) parameters per layer.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 Valid · 7 Not valid · 12 Not checked · 85 Own results and interpretations
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In other words, by randomizing labels alone we can force the generalization error of a model to jump up considerably without changing the model, its size, hyperparameters, or the optimizer.”
§ Understanding Deep Learning (Still) Requires Rethinking Generalization > 1.1. The randomization test
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“although it does not reach the perfect 100% top-1 accuracy, 95.20% accuracy is still very surprising for 1.2 million random labels from 1000 categories.”
§ Understanding Deep Learning (Still) Requires Rethinking Generalization > 2. EFFECTIVE CAPACITY OF NEURAL NETWORKS
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The networks fit the corrupted training set perfectly for all the cases.”
§ Understanding Deep Learning (Still) Requires Rethinking Generalization > Partially corrupted labels.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“On the CIFAR10 dataset, Alexnet and MLPs all converge to zero loss on the training set.”
§ Understanding Deep Learning (Still) Requires Rethinking Generalization > 2.1. Fitting random labels and pixels
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We corroborate these experimental findings with a theoretical construction showing that simple depth two neural networks already have perfect finite sample expressivity as soon as the number of parameters exceeds the number of data points as it usually does in practice.”
- Also stated in: § 1.3. Finite sample expressivity · plain
“More formally, we exhibit a very simple two-layer ReLU network with p = 2n + d parameters that can express any labeling of any sample of size n in d dimensions.”
§ Understanding Deep Learning (Still) Requires Rethinking Generalization > Abstract
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 1.3. Finite sample expressivity · plain
70 more claims
Create a free account to see all 104 claims and every source verdict.
- It may state two separate claims as one. 17
- Probably not a substantial claim. 15
- The quote only partly states this claim. 1
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Based on existing research(Saha et al., 2021; Zhang et al., 2021), the following conclusion can be drawn: the gradient updates of a linear layer always lie within the space spanned by its input.”
From Knowledge Decoupling via Orthogonal Projection for Lifelong Editing of Large Language Models (2025) · cites this paper as Zhang et al., 2021
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“If a model is successfully trained on a dataset with random labels, which has been shown to be possible with deep neural networks [294], then it has learned the data generation process by memorizing the random labels.”
From From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI (2023) · cites this paper as [294]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Existing works in privacy-preserving federated learning are mostly developed based on the wellknown privacy-preserving techniques, including: (1) homomorphic encryption (HE), such as Paillier [90], Elgamal [91] and Brakerski-Gentry-Vaikuntanathan cryptosystems [92]; (2) Secure Multiparty Computation (SMC), such as garbled circuits [93] and secret sharing [94]; and (3) differential privacy (DP) [95”
From Privacy and Robustness in Federated Learning: Attacks and Defenses (2024) · cites this paper as [95]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Understanding Deep Learning (Still) Requires Rethinking Generalization > 2.1. Fitting random labels and pixels
“On the CIFAR10 dataset, Alexnet and MLPs all converge to zero loss on the training set.”
§ Understanding Deep Learning (Still) Requires Rethinking Generalization > Rademacher complexity and VC-dimension.
“(fitting random labels) No No 99.82 9.86 MLP 3 × 512 1,735,178 No Yes 100.0 53.35No No 100.0 52.39 (fitting random labels) No No 100.0 10.48 MLP 1 × 512 1,209,866 No Yes 99.80 50.39No No 100.0 50.51 (fitting random labels) No No 99.34 10.61”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Understanding Deep Learning (Still) Requires Rethinking Generalization > 1.1. The randomization test
“More precisely, when trained on a completely random labeling of the true data, neural networks achieve 0 training error.”
§ Understanding Deep Learning (Still) Requires Rethinking Generalization > 2. EFFECTIVE CAPACITY OF NEURAL NETWORKS
“although it does not reach the perfect 100% top-1 accuracy, 95.20% accuracy is still very surprising for 1.2 million random labels from 1000 categories.”
Citations
Cites
(2014) · Cited by 33,419
Wu, Y. (1999) · Technometrics · doi:10.1080/00401706.1999.10485951 · Cited by 27,221
Advances in neural information processing systems 7
Cios, K., & Shields, M. (1997) · Neurocomputing · doi:10.1016/s0925-2312(97)90018-7 · Cited by 22,059
Approximation by superpositions of a sigmoidal function
Cybenko, G. (1989) · Mathematics of Control, Signals, and Systems · doi:10.1007/bf02551274 · Cited by 13,883
Advances in Neural Information Processing Systems 25
Zhang, Y., Sutton, C. A., Storkey, A., & Zoubin Ghahramani, A. (2012) · Cited by 12,653
Understanding deep learning requires rethinking generalization
Zhang, C., Bengio, S., Hardt, M., Recht, B., & Vinyals, O. (2016) · arXiv · doi:10.48550/arxiv.1611.03530 · Cited by 2,838
Self-citationReconciling modern machine-learning practice and the classical bias–variance trade-off
Belkin, M., Hsu, D., Ma, S., & Mandal, S. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1903070116 · Cited by 1,595 · cited 1×
“23, 7, 6, 16 Despite significant progress on theoretical understanding of deep learning in the past few years, a full mathematical characterization of the whole story remains challenging.”
BackgroundA Generalized Representer Theorem
Schölkopf, B., Herbrich, R., & Smola, A. J. (2001) · Lecture Notes in Computer Science · doi:10.1007/3-540-44581-1_27 · Cited by 1,589 · cited 1×
“We have thus derived the “kernel trick” 34 —albeit in a round-about fashion.”
MethodA Closer Look at Memorization in Deep Networks
Arpit, D., Jastrzębski, S., Ballas, N., Krueger, D., Bengio, E., Kanwal, M. S., Maharaj, T., Fischer, A., Courville, A., Bengio, Y., et al. (2017) · arXiv · doi:10.48550/arxiv.1706.05394 · Cited by 1,307
Bartlett, P. L. (1998) · IEEE Transactions on Information Theory · doi:10.1109/18.661502 · Cited by 1,166
Research works printed but not linked: 8
- Cohen, N., Shashua, A. Convolutional rectifier networks as generalized tensor decompositions. In International Conference on Machine Learning. M.F. Balcan and K.Q. Weinberger, eds. 2016, 955--963.
- Cybenko, G. Approximation by superposition of sigmoidal functions. Math. Control Signal. Syst. 4, 2 (1989), 303--314.
- Golowich, N., Rakhlin, A., Shamir, O. Size-independent sample complexity of neural networks. In Conference On Learning Theory. P.R. Sébastien Bubeck, V. Perchet, eds., 2018, 297--299.
- Ma, S., Bassily, R., Belkin, M. The power of interpolation: Understanding the effectiveness of sgd in modern over-parametrized learning. In International Conference on Machine Learning. J. Dy and A. Krause, eds. 2018, 3325--3334.
- Morcos, A., Raghu, M., Bengio, S. Insights on representational similarity in neural networks with canonical correlation. Adv. Neural Inform. Process. Syst.. 2018, 5727--5736.
- Neyshabur, B., Tomioka, R., Srebro, N. Norm-based capacity control in neural networks. In Conference on Learning Theory. S.K. Peter Grünwald and E. Hazan, eds. 2015, 1376--1401.
- Smith, S.L., Le, Q.V. A bayesian perspective on generalization and stochastic gradient descent. In International Conference on Learning Representations, 2018.
- Zhou, W., Veitch, V., Austern, M., Adams, R.P., Orbanz, P. Non-vacuous generalization bounds at the ImageNet scale: A PAC-Bayesian compression approach. In International Conference on Learning Representations, 2019.
Cited by
Items 2,601–2,667 of 2,667
Geometry as a shield: High-utility privacy preservation via manifold-aware feature compactness
Zhang, S., & Sun, D. (2026) · Information Sciences · doi:10.1016/j.ins.2026.124133
Pereira, R. R. F. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7462598
Jeong, J. C., Kim, J. S., Buman, M. P., Turaga, P. K., & Jeon, E. S. (2026) · IEEE Internet of Things Journal · doi:10.1109/jiot.2026.3733757
Pedagogical Transformation Through Local Ecosystems Advances Civic Character Building
Bantahari, D., Kamuli, S., & Yunus, R. (2026) · Academia Open · doi:10.21070/acopen.11.2026.14647
Zhang, S., Pan, Y., & Ge, X. (2026) · IEEE/CIC International Conference on Communications in China (ICCC) · doi:10.1109/iccc70295.2026.11680353
Cluster-Based Noise Inversion for Structure-Aware Noise-Gradient Estimation in NCSAM
Xu, J., Wang, Z., & Pang, J. (2026) · International Conference on Image, Vision and Computing (ICIVC) · doi:10.1109/icivc70586.2026.11686566
Gamage, D. A., & Nizzad, A. R. M. (2026) · Moratuwa Engineering Research Conference (MERCon) · doi:10.1109/mercon71835.2026.11691492
Anil, S., Srinivas, B., Liu, B., Li, A., Interian, Y., Chen, W., Prionas, N. D., Hong, J. C., Braunstein, S. E., Morin, O., et al. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-03280-3
Nikitha, V., & Sree Kala, T. (2026) · Procedia Computer Science · doi:10.1016/j.procs.2026.08.357
Provable Guarantees for Over-the-Air Federated Learning With Polynomial Neural Networks
Yang, H., Xia, S., Zhu, J., Wu, Y., Zhou, Y., & Shi, Y. (2026) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2026.3737404
PlanktonLake-CEREEP-A Freshwater Plankton Image Dataset with Semi-Automated Label CleaningPreprint
Dechaumet, L., Puppo, C., Carmignac, D., Blusseau, S., Marcotegui, B., Decencière, É., Lacroix, G., & Galliard, J. L. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.09.24.753096
Time — Series Artificial Neural Networks for Fault and Drift Detection in GSR Signals
Dewi, A. L., Prasetyo, A., & Engel, V. J. L. (2026) · International Conference on Information Management and Technology (ICIMTech) · doi:10.1109/icimtech202671078.2026.11699037
“This training approach and formulation is common in the deep learning and anomaly detection literature [34][35].”
An information identity reveals the geometry of deviation eventsPreprint
Balsubramani, A. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.38159
“A learner that fits every observed point [38] sits at the other end when its class is symmetric.”
Deep generative modeling of groundwater level dynamics using a conditional LSTM-WGAN-GP architecture
Tahroudi, M. N. (2026) · Journal of Hydrology Regional Studies · doi:10.1016/j.ejrh.2026.104013
How 'Foundational' Are Current Molecular Foundation Models?Preprint
Grisoni, F. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.37550 · Cited by 1
“87 These observations echo a well-documented behavior of deep networks, which can memorize individual training instances instead of learning the abstract, reusable regularities that support generalization 88,89 and compositional recombination.”
Rodrigues, J. M. B., Borges, H., Lopes, P., Lopes, L., Acher, M., & Pereira, J. A. (2026) · Journal of Software Engineering Research and Development · doi:10.5753/jserd.2026.6399
“Subsequent layers were gradually reduced in size to foster abstraction and mitigate overfitting, consistent with empirical analyses on depth–width trade-offs and double-descent behaviour in modern networks (Zhang et al., 2021; He et al., 2016).”
Significance of a Classification Scheme for Assessing Evidence Quality in AI Systems
Andrae, S. (2026) · IGI Global eBooks · doi:10.4018/979-8-2600-0090-8.ch014
Label-Corrected Weighted Multi-Similarity Loss for noisy cross-modal retrieval
Jiang, S., Yang, S., & Li, Z. (2026) · Information Processing & Management · doi:10.1016/j.ipm.2026.105204
Evaluation Of COCOMO Model Accuracy In Software Effort Estimation
Jeklin, U., Sa’ad, M. I., & Ekawati, H. (2025) · Bulletin of Information Technology (BIT) · doi:10.47065/bit.v6i2.2027
Syauri, S., Defiyanti, S., & Yusup, D. (2026) · IIKRA-ITH Informatika Jurnal Komputer dan Informatika
Fatmawati, E., Shuhidan, S. M., Samsuddin, S. F., Husna, J., Irawati, Y. E., & Rafa, M. F. F. (2026) · ENDLESS International Journal of Future Studies
Statistical Physics Methods for Non-Convex Neural Network Models
Annesi, B. L. (n.d.) · Archivio istituzionale della Ricerca - Bocconi
Exploring the Generalization Capacity of Over-Parameterized Networks
Pandey, E., & Kumar, S. (2023) · International Journal of Intelligent Systems and Applications in Engineering
Fitriyati, N., Liebenlito, M., & Tsabitah, S. H. (2026) · Jurnal Fourier
Navigating Rough Landscapes: Statistical Physics Approaches to Machine Learning and Inference
Straziota, D. (n.d.) · Archivio istituzionale della Ricerca - Bocconi
Integrated Web-Based Motorcycle Rental System with RealTime Fleet Availability Tracking
Rahmatulloh, A. H., & Maududy, R. (2026) · Jurnal Kecerdasan Buatan dan Teknologi Informasi. · doi:10.69916/jkbti.v5i3.526
Nouraniboosjin, S., & Ganji, F. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-25149-7_13
Novel Siamese CNN-Transformer Architecture for Efficient and Robust Cross-Domain Image Recognition
Abdian, A. Z., Javidi, M. M., & Mansouri, N. (2026) · Optical Memory and Neural Networks · doi:10.3103/s1060992x25600429
Conflict-aware uncertainty alignment for tool-augmented document-centric multimodal intelligence
Muthusami, R., & Saritha, K. (2026) · Discover Artificial Intelligence · doi:10.1007/s44163-026-02409-3
GBmooer: Granular-ball text label smoothing with hypersphere contrastive learning
Zhai, J., Wang, Z., Xia, S., & Lin, L. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.134643
Zhan, M., Zhao, R., Deng, X., Xue, Z., Li, Q. L., Liu, Z., Cheng, G., & Xu, K. (2026) · IEEE Transactions on Networking · doi:10.1109/ton.2026.3739909
“However, they are particularly prone to label noise, as their high capacity allows them to overfit incorrect labels [20], [39].”
Dual‐Branch Network With Enhanced Feature Fusion for Improved Rice Classification Accuracy
Ma, R., Shi, W., Wang, M., Chen, X., Zhang, C., Pan, B., & Xiao, X. (2026) · Journal of Sustainable Agriculture and Environment · doi:10.1002/sae2.70218
Self-citationMART: Mechanism-Disentanglement Anchor-Routed Training for Learning with Open-World Noisy Data
Hu, C., Nagarajan, B., Marques, R., & Radeva, P. I. (2026) · doi:10.1109/cvprf71909.2026.00674
Learning transferable noise recognizers from training dynamics for robust classification
Li, M.-Y., Zhou, S.-Q., Lv, J., & Zhang, Z. (2026) · Pattern Recognition · doi:10.1016/j.patcog.2026.115112
MAP4CS: A Multi-dimensional Data Pruning Framework for Efficient Code Retriever Fine-tuningPreprint
Chen, Y., Liu, M., Ou, G., Zhang, Z., Li, Z., Wang, Y., & Zheng, P. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.11727
“It often leads to the Loss of Long-Tail Knowledge, discarding crucial but infrequent domain-specific APIs [20], while simultaneously suffering from the Retention of High-Entropy Noise, preserving misaligned pairs that confuse the model’s decision boundaries [59].”
Intrinsic dimension for evaluating and analyzing neural networks for medical image segmentation
Li, W., Li, Z., Xu, J., Zhou, S., Guo, Y., Tao, Q., & Wang, Y. (2026) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2026.109680
A Survey on Loss-based Regularization for Model Generalization
Huang, Y., Zhang, C., Xu, R., & Zhang, Q. (2026) · Machine Learning · doi:10.1007/s10994-026-07139-2
“In addition, some studies suggest that increasing the complexity may lead to better model generalization (Zhang et al., 2021; Kawaguchi et al., 2017; Li and Sompolinsky, 2021).”
Self-citationWhy simple neural networks outperform traditional classifiers in cyberbullying detection
Anggrainingsih, R., Noor, F. A., Prakoso, G. B., Wiharto, W., Wardani, D. W., & Cahyono, H. D. (2026) · IAES International Journal of Artificial Intelligence · doi:10.11591/ijai.v15.i5.pp4852-4861
“This composition of neurons enables the model to learn a mapping function y = f ( x ) between input x and output y through a series of linear transformations, each followed by a nonlinear activation function [12], [13].”
Zhang, Q., Chen, Y., Yuan, Y., Jin, G., & Cheng, Y. (2026) · Symmetry · doi:10.3390/sym18101691
Phase Space Compression Optimization on Photonic Neural Networks
Xiao, Z., Li, J., Su, Y., Chen, Z., Li, X., Hao, W., & Xie, Y. (2026) · IEEE Journal of Selected Topics in Quantum Electronics · doi:10.1109/jstqe.2026.3742258
Learning from Ambiguity: Uncertainty-Aware Intervention for Multimodal Learning
Wen, B., Pan, Z., Ji, J., Wang, Y., & Ying, Z. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-6834-4_13
MIDAS: Manifold-Inspired Diverse Feature Acquisition for Simplicity Bias Mitigation
Kumar, B., Vivek, B. S., Gubbi, J., & Pal, A. (2026) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn69854.2026.11729742
“To achieve zero training error on noise, deep networks must utilize their maximum available capacity, leading to representations with high Intrinsic Dimension [22].”
Labiba, N. N., Islam, M. M., Hall, L., & Simmons, D. (2026) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.6c00178
Analysis of Power System Signal Processing Method Based on Deep Convolutional Neural Network
Xue, H., Yao, N., Fan, Q., Liu, Z., & Liu, Z. (2026) · Automatic Control and Computer Sciences · doi:10.3103/s0146411624603824
“As these systems grow increasingly complex and interconnected, the challenges associated with their management, operation, and maintenance have become more pronounced [1].”
Consistency-Aware Cross-Modal Noisy Correspondence Rectification via Perturbation-Invariant Modeling
Wang, Z., Peng, S., Liu, J., & Wenxiu, A. O. (2026) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn69854.2026.11728718
“Recent methods for noisy correspondence learning, including NCR [8] and CTPR [9], exploit the observation that DNNs learn simple patterns first [10].”
SARE-GT: Structure-Aware Robust Enhancement Graph Transformer for Corporate Credit Rating
Kong, L., Yang, Z., Wang, Y., & Zhou, L. (2026) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn69854.2026.11728665
Jennaoui, I., Bachaoui, E. M., Ghmari, A. E., Biniz, M., Harti, A. E., Bousbaa, M., El-Jamaoui, I., Ayoub, Y., & Mouaddine, A. (2026) · DYSONA – Applied Science · doi:10.30493/das.2026.012110
Syed, M. M., & Mohammed, H. A. B. (2026) · Journal of Artificial Intelligence Machine Learning and Neural Network · doi:10.55529/jaimlnn.62.84.97
MRI-Based Brain Tumor Classification Using A Customized Xception Transfer Learning Framework
Kumar, S., Dubey, A. K., & Shukla, A. K. (2026) · doi:10.1109/incect72370.2026.11742822
“The scarcity of large, high-quality labeled datasets in medical imaging, such as MRI scans, limits the training of deep neural networks and often leads to poor generalization [6].”
Samarin, A., Nazarenko, A., Nazarenko, D., Afonina, J., Kotenko, E., Dozortseva, A., Mamaeva, A., Savelev, A., Narova, G., Mikhailova, E., et al. (2026) · Big Data and Cognitive Computing · doi:10.3390/bdcc10110363
Lin, P., Gu, F., Chen, R., Deng, Y., & Heitzman, M. (2026) · Journal of Transportation Engineering Part B Pavements · doi:10.1061/jpeodx.pveng-2142
Pan, C.-T., Chen, N.-Y., Lee, M.-C., & Lin, M.-C. (2026) · Sensors and Materials · doi:10.18494/sam6566
Representation Complexity Along Learning Trajectories: Trading Width for Precision and TimePreprint
Singh, R., Llanza, M. D., & Buckley, C. L. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.39027
Formalizing the Intrinsic Dimension of Objective Landscapes by a Generalized Notion of Co-dimensionPreprint
Araya, E., Gitta Kutyniok, B., Li, W., Maly, J., & Singh, M. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.39674
Ерохин, В. В., & В., А. А. (n.d.) · CyberLeninK (CyberLeninka)
Software Defect Prediction Using Machine Learning Models
Kumar, M. A., Ravi, V., Hussain, S. A., Sravanthi, D., Duduka, S., & Susmija, P. (2026) · International Conference on Distributed Systems, Computer Networks and Cybersecurity (ICDSCNC) · doi:10.1109/icdscnc70670.2026.11746318
Information-Theoretic Neighbourhood Analysis for Detecting Confusion-Directed Label Noise
Ghosh, S. (2026) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3799682.3841063
Robust out-of-distribution detection under noisy labels
Lee, Y., & Kang, S. (2026) · Neural Networks · doi:10.1016/j.neunet.2026.109826
Review of computer vision algorithms in histopathology to decipher complex features in cancer
Illarionova, S., Studenikina, A., Carreras, J., Zapevalina, M., Bernstein, A., Hamoudi, R., & Sharaev, M. (2026) · Discover Artificial Intelligence · doi:10.1007/s44163-026-02592-3
Deng, C., Yan, J., Liu, Y., Shen, Y., & Zhong, F. (2026) · doi:10.1109/iotaiee70863.2026.11749413
Yang, W., Yang, W., & Zhang, G. (2026) · ACS Omega · doi:10.1021/acsomega.6c08912
Scale-Invariant Sharpness Measures and Their Impact on Generalization in Deep Neural Networks
N, N. G. K., Belamagi, J. U., & Muthukumaran, L. (2026) · International Conference on Circuits, Control, Communication and Computing (I4C) · doi:10.1109/i4c69082.2026.11760152
Gorelov, D. S., Мищенко, А. А., Malikiev, I. E., Михайлович, О. В., Горгоцкий, И. А., Семенякин, И. В., Ali, S., Okishev, A. V., Петров, С. Б., & Gadzhiev, N. K. (2026) · Urology reports (St - Petersburg) · doi:10.17816/uroved686690
Learning to (Learn at Test Time)
Sun, Y., Li, X., Dalal, K., Hsu, C., Koyejo, S., Guestrin, C., Wang, X., Hashimoto, T., & Chen, X. (2023) · arXiv · doi:10.48550/arxiv.2310.13807
“Since over-parameterized neural networks are known to be able to regularize themselves (Zhang et al., 2021), MTTT-MLP should not do poorly, but might not justify the extra compute.”
Background
Metadata
- Authors
- Zhang, C., Bengio, S., Hardt, M., Recht, B., & Vinyals, O.
- Affiliations
- Google (United States) (US)
- Google Brain, Mountain View, CA
- University of California, Berkeley (US)
- Google DeepMind (United Kingdom) (GB)
- Venue
- Communications of the ACM
- Published
- 2021-02-22
- Volume
- 64
- Issue
- 3
- Pages
- 107-115
- ISSN
- 0001-0782 (ISSN-L)
- 1557-7317
- OpenAlex
- W2566079294
- Cited by
- 2,400
- References
- 47
- Citations by year
- 2026: 295 · 2025: 558 · 2024: 579 · 2023: 500 · 2022: 316 · 2021: 108
- FWCI
- 132.37
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Communications of the ACM (publishedVersion, cc-by-nc-sa)
- Funders
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- generalization, deep neural network, overfitting, overparameterization, expressivity, stochastic gradient descent, regularization, image classification, convolutional neural networks
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-11-08, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-11-08 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.