Exploring Generalization in Deep Learning
Open accessThe DOI matches its OpenAlex record; 23 of the 23 references it cites were found in a registry or an index.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
Understanding Machine Learning: From Theory to Algorithms
Shalev-Shwartz, S., & Ben-David, S. (2014) · doi:10.1017/cbo9781107298019 · Cited by 3,115
(2015) · Cited by 2,857
Understanding deep learning requires rethinking generalization
Zhang, C., Bengio, S., Hardt, M., Recht, B., & Vinyals, O. (2016) · arXiv · doi:10.48550/arxiv.1611.03530 · Cited by 2,838 · cited 3×
“• It is possible to obtain zero training error on random labels using the same architecture for which training with real labels leads to good generalization [30].”
BackgroundRademacher and Gaussian Complexities: Risk Bounds and Structural Results
Bartlett, P. L., & Mendelson, S. (2001) · Lecture Notes in Computer Science · doi:10.1007/3-540-44581-1_15 · Cited by 1,949
Bartlett, P. L. (1998) · IEEE Transactions on Information Theory · doi:10.1109/18.661502 · Cited by 1,166
Maximum-Margin Matrix Factorization
Srebro, N., Rennie, J. D. M., & Jaakkola, T. (2004) · Cited by 962
Self-citationThe connection between regularization operators and support vector kernels
Smola, A. J., Schölkopf, B., & Müller, K.-R. (1998) · Neural Networks · doi:10.1016/s0893-6080(98)00032-x · Cited by 625
Train faster, generalize better: stability of stochastic gradient descent
Hardt, M., Recht, B., & Singer, Y. (2016) · International Conference on Machine Learning · Cited by 541
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima
Keskar, N. S., Mudigere, D., Nocedal, J., Smelyanskiy, M., & Tang, P. T. P. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1609.04836 · Cited by 525
McAllester, D. A. (1998) · Proceedings of the eleventh annual conference on Computational learning theory · doi:10.1145/279943.279989 · Cited by 464
Self-citationXu, H., & Mannor, S. (2012) · Machine Learning · doi:10.1007/s10994-011-5268-1 · Cited by 377
McAllester, D. A. (1999) · Proceedings of the twelfth annual conference on Computational learning theory · doi:10.1145/307400.307435 · Cited by 354
Self-citationSrebro, N., & Shraibman, A. (2005) · Lecture Notes in Computer Science · doi:10.1007/11503415_37 · Cited by 316
Self-citationNorm-Based Capacity Control in Neural Networks
Neyshabur, B., Tomioka, R., & Srebro, N. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1503.00036 · Cited by 279
Self-citationDziugaite, G. K., & Roy, D. M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.11008 · Cited by 217
Path-SGD: Path-Normalized Optimization in Deep Neural Networks
Neyshabur, B., Salakhutdinov, R., & Srebro, N. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.02617 · Cited by 149
Self-citationIn Search of the Real Inductive Bias: On the Role of Implicit Regularization in Deep Learning
Neyshabur, B., Tomioka, R., & Srebro, N. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.6614 · Cited by 125
Self-citationAlmost Linear VC-Dimension Bounds for Piecewise Polynomial Networks
Bartlett, P. L., Maiorov, V., & Meir, R. (1998) · Neural Computation · doi:10.1162/089976698300017016 · Cited by 119
Entropy-SGD: Biasing Gradient Descent Into Wide Valleys
Chaudhari, P., Choromanska, A., Soatto, S., LeCun, Y., Baldassi, C., Borgs, C., Chayes, J., Sagun, L., & Zecchina, R. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.01838 · Cited by 92
Generalization Error of Invariant ClassifiersPreprint
Sokolić, J., Giryes, R., Sapiro, G., & Rodrigues, M. R. D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.04574 · Cited by 81
Langford, J. C., & Caruana, R. (2001) · Cited by 74
Path-Normalized Optimization of Recurrent Neural Networks with ReLU ActivationsPreprint
Neyshabur, B., Wu, Y., Salakhutdinov, R., & Srebro, N. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.07154 · Cited by 20
Self-citation
Cited by
Understanding deep learning (still) requires rethinking generalization
Zhang, C., Bengio, S., Hardt, M., Recht, B., & Vinyals, O. (2021) · Communications of the ACM · doi:10.1145/3446776 · Cited by 2,400
“…bounds based on uniform convergence or uniform stability are inadequate for overparameterized deep neural networks, extensive efforts were made toward tighter generalization bounds (e.g., Kawaguchi et al., 19 Bartlett et al., 5 Neyshabur et al., 28 Golowich et al., 17 Liang et al. 20 ).”
BackgroundStudy on artificial intelligence: The state of the art and future prospects
Zhang, C., & Lu, Y. (2021) · Journal of Industrial Information Integration · doi:10.1016/j.jii.2021.100224 · Cited by 1,066
Frontiers in Artificial Intelligence and Applications
Yadav, J., Shukla, S., Sharma, K., Soni, N., Agarwal, S., & Pathak, P. C. (2022) · 2022 3rd International Conference on Computation, Automation and Knowledge Management (ICCAKM) · doi:10.1109/iccakm54721.2022.9990098 · Cited by 660
High-Frequency Component Helps Explain the Generalization of Convolutional Neural Networks
Wang, H., Wu, X., Huang, Z., & Xing, E. P. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00871 · Cited by 638
Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networks
John Xu, Z.-Q., Zhang, Y., Luo, T., Xiao, Y., & Ma, Z. (2020) · Communications in Computational Physics · doi:10.4208/cicp.oa-2020-0085 · Cited by 522
Metadata
- Authors
- Neyshabur, B., Bhojanapalli, S., McAllester, D., & Srebro, N.
- Venue
- arXiv (Cornell University)
- Publisher
- Cornell University
- Published
- 2017-06-27
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1706.08947
- OpenAlex
- W2732724430
- Cited by
- 284
- References
- 23
- Type
- posted content
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.