The DOI matches its OpenAlex record; 30 of the 30 references it cites were found in a registry or an index.
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima
Open accessOverview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
(2014) · Cited by 33,419
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,395
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S. (2024) · arXiv (Cornell University) · doi:10.57702/o9raffed · Cited by 14,214
Playing Atari with Deep Reinforcement Learning
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013) · arXiv · doi:10.48550/arxiv.1312.5602 · Cited by 9,471
Speech recognition with deep recurrent neural networks
Graves, A., Mohamed, A.-R., & Hinton, G. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6638947 · Cited by 8,893 · cited 1×
“These configura-tions have been successfully used in practice for a large number of applications; see e.g. (Simonyan & Zisserman, 2014; Graves et al., 2013; Mnih et al., 2013).”
Method(2010) · Cited by 8,261
A Limited Memory Algorithm for Bound Constrained Optimization
Byrd, R. H., Lu, P., Nocedal, J., & Zhu, C. (1995) · SIAM Journal on Scientific Computing · doi:10.1137/0916069 · Cited by 6,221
Self-citationLarge Scale Distributed Deep Networks
Dean, J. B., Corrado, G. S., Monga, R., Chen, K., Devin, M., Mao, M., Ranzato, M., Senior, A., Tucker, P. A., Yang, K., et al. (2012) · neural information processing systems · Cited by 4,489
(2013) · Cited by 3,740
Optimization Methods for Large-Scale Machine Learning
Bottou, L., Curtis, F. E., & Nocedal, J. (2018) · SIAM Review · doi:10.1137/16m1080173 · Cited by 3,128
Self-citationA practical Bayesian framework for backpropagation networks
MacKay, D. J. C. (1992) · Neural Computation · doi:10.1162/neco.1992.4.3.448 · Cited by 2,917
TIMIT Acoustic-Phonetic Continuous Speech Corpus
Garofolo, J. S., Lamel, L. F., Fisher, W. M., Pallett, D. S., Dahlgren, N. L., Zue, V. W., & Fiscus, J. G. (2020) · Americanae (AECID Library) · doi:10.35111/17gk-bn40 · Cited by 2,508
A Universal Prior for Integers and Estimation by Minimum Description Length
Rissanen, J. (1983) · The Annals of Statistics · doi:10.1214/aos/1176346150 · Cited by 1,672
LeCun, Y., Bottou, L., Orr, G., & Müller, K. (1998) · neural information processing systems · Cited by 1,333
Efficient mini-batch training for stochastic optimization
Li, M., Zhang, T., Chen, Y., & Smola, A. J. (2014) · Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/2623330.2623612 · Cited by 742
The Loss Surfaces of Multilayer Networks
Choromanska, A., Henaff, M., Mathieu, M., Arous, G. B., & LeCun, Y. (2014) · arXiv · doi:10.48550/arxiv.1412.0233 · Cited by 733
On-line Learning and Stochastic Approximations
Bottou, L. (1999) · On-Line Learning in Neural Networks · doi:10.1017/cbo9780511569920.003 · Cited by 564
Train faster, generalize better: stability of stochastic gradient descent
Hardt, M., Recht, B., & Singer, Y. (2016) · International Conference on Machine Learning · Cited by 541
Sample size selection in optimization methods for machine learning
Byrd, R. H., Chin, G. M., Nocedal, J., & Wu, Y. (2012) · Mathematical Programming · doi:10.1007/s10107-012-0572-5 · Cited by 369
Self-citationShaham, U., Yamada, Y., & Negahban, S. (2018) · Neurocomputing · doi:10.1016/j.neucom.2018.04.027 · Cited by 355
Qualitatively characterizing neural network optimization problems
Goodfellow, I. J., Vinyals, O., & Saxe, A. M. (2014) · arXiv · doi:10.48550/arxiv.1412.6544 · Cited by 341
Hybrid deterministic-stochastic methods for data fitting
Friedlander, M. P., & Schmidt, M. (2011) · Cited by 314
No bad local minima: Data independent training error guarantees for multilayer neural networksPreprint
Soudry, D., & Carmon, Y. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.08361 · Cited by 246
Distributed Deep Learning Using Synchronous Stochastic Gradient DescentPreprint
Das, D., Avancha, S., Mudigere, D., Vaidynathan, K., Sridharan, S., Kalamkar, D. D., Kaul, B., & Dubey, P. K. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.06709 · Cited by 207
Self-citationGradient Descent Converges to MinimizersPreprint
Lee, J. D., Simchowitz, M., Jordan, M. I., & Recht, B. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.04915 · Cited by 198
Robust Optimization for Unconstrained Simulation-Based Problems
Bertsimas, D., Nohadani, O., & Teo, K. M. (2009) · Operations Research · doi:10.1287/opre.1090.0715 · Cited by 142
Training Recurrent Neural Networks by DiffusionPreprint
Mobahi, H. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1601.04114 · Cited by 46
Cited by
Zhang, W., Li, C., Peng, G., Chen, Y., & Zhang, Z. (2018) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2017.06.022 · Cited by 1,421
Super-convergence: very fast training of neural networks using large learning rates
Smith, L. N., & Topin, N. (2019) · Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications · doi:10.1117/12.2520589 · Cited by 1,335
A Closer Look at Memorization in Deep Networks
Arpit, D., Jastrzębski, S., Ballas, N., Krueger, D., Bengio, E., Kanwal, M. S., Maharaj, T., Fischer, A., Courville, A., Bengio, Y., et al. (2017) · arXiv · doi:10.48550/arxiv.1706.05394 · Cited by 1,307
“…proposed before, including looking at generalization error, trajectory length evolution (Raghu et al., 2016), analyzing Jacobians associated to different layers (Wang; Saxe et al., 2013), or the shape of the loss minima found by SGD (Im et al., 2016; Chaudhari et al., 2016; Keskar et al., 2016).”
BackgroundA Comprehensive Survey of Continual Learning: Theory, Method and Application
Wang, L., Zhang, X., Su, H., & Zhu, J. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3367329 · Cited by 1,113
“Therefore, the robustness of an empirical optimal solution θ∗ i to parameter changes is closely related to λ i , which has been commonly used to describe the flatness of loss landscape [162], [211], [309].”
MethodFederated Learning via Over-the-Air Computation
Yang, K., Jiang, T., Shi, Y., & Ding, Z. (2020) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2019.2961673 · Cited by 1,084
“Furthermore, we notice that the aggregation error also causes a notable drop of the prediction accuracy [32].”
Background
Metadata
- Authors
- Keskar, N. S., Mudigere, D., Nocedal, J., Smelyanskiy, M., & Tang, P. T. P.
- Venue
- arXiv (Cornell University)
- Publisher
- Cornell University
- Published
- 2016-09-15
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1609.04836
- OpenAlex
- W2523060838
- Cited by
- 525
- References
- 30
- Type
- posted content
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.