The DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index.
Asymptotic statistical theory of overtraining and cross-validation
Abstract
A statistical theory for overtraining is proposed. The analysis treats general realizable stochastic neural networks, trained with Kullback-Leibler divergence in the asymptotic case of a large number of training examples. It is shown that the asymptotic gain in the generalization error is small if we perform early stopping, even if we have access to the optimal stopping time. Based on the cross-validation stopping we consider the ratio the examples should be divided into training and cross-validation sets in order to obtain the optimum performance. Although cross-validated early stopping is useless in the asymptotic region, it surely decreases the generalization error in the nonasymptotic region. Our large scale simulations done on a CM5 are in good agreement with our analytical findings.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A new look at the statistical model identification
Akaike, H. (1974) · IEEE Transactions on Automatic Control · doi:10.1109/tac.1974.1100705 · Cited by 51,879
An Introduction to the Bootstrap
Efron, B., & Tibshirani, R. J. (1994) · doi:10.1201/9780429246593 · Cited by 40,500
The Nature of Statistical Learning Theory
Vapnik, V. N. (1995) · doi:10.1007/978-1-4757-2440-0 · Cited by 39,138
An Introduction to the Bootstrap
Efron, B., & Tibshirani, R. J. (1993) · doi:10.1007/978-1-4899-4541-9 · Cited by 21,230
Learning Internal Representations by Error Propagation
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1985) · doi:10.21236/ada164453 · Cited by 16,172
Fundamentals of Artificial Neural Networks
Hassoun, M. H. (1996) · Proceedings of the IEEE · doi:10.1109/jproc.1996.503146 · Cited by 1,887
Regularization Algorithms for Learning That Are Equivalent to Multilayer Networks
Poggio, T., & Girosi, F. (1990) · Science · doi:10.1126/science.247.4945.978 · Cited by 1,026
Stochastic Complexity and Modeling
Rissanen, J. (1986) · The Annals of Statistics · doi:10.1214/aos/1176350051 · Cited by 922
Murata, N., Yoshizawa, S., & Amari, S. (1994) · IEEE Transactions on Neural Networks · doi:10.1109/72.329683 · Cited by 645
Self-citationA Theory of Adaptive Pattern Classifiers
Amari, S. (1967) · IEEE Transactions on Electronic Computers · doi:10.1109/pgec.1967.264666 · Cited by 534
Self-citation
Cited by
Maier, H. R., & Dandy, G. C. (2000) · Environmental Modelling & Software · doi:10.1016/s1364-8152(99)00007-9 · Cited by 2,377
ICLabel: An automated electroencephalographic independent component classifier, dataset, and website
Pion-Tonachini, L., Kreutz-Delgado, K., & Makeig, S. (2019) · NeuroImage · doi:10.1016/j.neuroimage.2019.05.026 · Cited by 2,267
Prechelt, L. (1998) · Lecture Notes in Computer Science · doi:10.1007/3-540-49430-8_3 · Cited by 1,649
Introduction to multi-layer feed-forward neural networks
Svozil, D., Kvasnicka, V., & Pospichal, J. (1997) · Chemometrics and Intelligent Laboratory Systems · doi:10.1016/s0169-7439(97)00061-0 · Cited by 1,600
Prechelt, L. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-35289-8_5 · Cited by 951
Introduction to machine learning for brain imaging
Lemm, S., Blankertz, B., Dickhaus, T., & Müller, K.-R. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2010.11.004 · Cited by 630
The connection between regularization operators and support vector kernels
Smola, A. J., Schölkopf, B., & Müller, K.-R. (1998) · Neural Networks · doi:10.1016/s0893-6080(98)00032-x · Cited by 625
Assessment and Validation of Machine Learning Methods for Predicting Molecular Atomization Energies
Hansen, K., Montavon, G., Biegler, F., Fazli, S., Rupp, M., Scheffler, M., von Lilienfeld, O. A., Tkatchenko, A., & Müller, K.-R. (2013) · Journal of Chemical Theory and Computation · doi:10.1021/ct400195d · Cited by 613
Metaheuristic design of feedforward neural networks: A review of two decades of research
Ojha, V. K., Abraham, A., & Snášel, V. (2017) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2017.01.013 · Cited by 606
Neural networks based predictive control for thermal comfort and energy savings in public buildings
Ferreira, P. M., Ruano, A. E., Silva, S., & Conceição, E. Z. E. (2012) · Energy and Buildings · doi:10.1016/j.enbuild.2012.08.002 · Cited by 445
Chan, K. Y., Dillon, T. S., Singh, J., & Chang, E. (2012) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2011.2174051 · Cited by 430
No Free Lunch Theorem: A Review
Adam, S. P., Alexandropoulos, S.-A. N., Pardalos, P. M., & Vrahatis, M. N. (2019) · Springer Optimization and Its Applications · doi:10.1007/978-3-030-12767-1_5 · Cited by 401
Taormina, R., Chau, K.-W., & Sethi, R. (2012) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2012.02.009 · Cited by 396
River flow forecasting: use of phase-space reconstruction and artificial neural networks approaches
Sivakumar, B., Jayawardena, A. W., & Fernando, T. M. K. G. (2002) · Journal of Hydrology · doi:10.1016/s0022-1694(02)00112-9 · Cited by 315
Landschützer, P., Gruber, N., Bakker, D. C. E., Schuster, U., Nakaoka, S., Payne, M. R., Sasse, T. P., & Zeng, J. (2013) · Biogeosciences · doi:10.5194/bg-10-7793-2013 · Cited by 296
Brusic, V, Rudy, G., Honeyman, G., Hammer, J., & Harrison, L. (1998) · Bioinformatics · doi:10.1093/bioinformatics/14.2.121 · Cited by 292
High-dimensional dynamics of generalization error in neural networks
Advani, M. S., Saxe, A. M., & Sompolinsky, H. (2020) · Neural Networks · doi:10.1016/j.neunet.2020.08.022 · Cited by 289
Forecasting daily streamflow using hybrid ANN models
Wang, W., Gelder, P. H. A. J. M. V., Vrijling, J. K., & Ma, J. (2006) · Journal of Hydrology · doi:10.1016/j.jhydrol.2005.09.032 · Cited by 279
Agirre-Basurko, E., Ibarra-Berastegi, G., & Madariaga, I. (2006) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2004.07.008 · Cited by 273
Unsupervised pre-trained filter learning approach for efficient convolution neural network
Rehman, S. U., Tu, S., Waqas, M., Huang, Y., Rehman, O. U., Ahmad, B., & Ahmad, S. (2019) · Neurocomputing · doi:10.1016/j.neucom.2019.06.084 · Cited by 270
Piotrowski, A. P., & Napiorkowski, J. J. (2013) · Journal of Hydrology · doi:10.1016/j.jhydrol.2012.10.019 · Cited by 257
A dynamic artificial neural network model for forecasting time series events
Ghiassi, M., Saidane, H., & Zimbra, D. K. (2005) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2004.10.008 · Cited by 257
Regularization for Deep Learning: A Taxonomy
Kukačka, J., Golkov, V., & Cremers, D. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.10686 · Cited by 255
Zhou, H. F., Ni, Y. Q., & Ko, J. M. (2010) · Engineering Structures · doi:10.1016/j.engstruct.2010.02.026 · Cited by 102
Eliminating Temperature Effect in Vibration-Based Structural Damage Detection
Zhou, H. F., Ni, Y. Q., & Ko, J. M. (2011) · Journal of Engineering Mechanics · doi:10.1061/(asce)em.1943-7889.0000273 · Cited by 91
Lai, D. K.-H., Cheng, E. S.-W., So, B. P.-H., Mao, Y.-J., Cheung, S. M.-Y., Cheung, D. S. K., Wong, D. W.-C., & Cheung, J. C.-W. (2023) · Mathematics · doi:10.3390/math11143081 · Cited by 18
Metadata
- Authors
- Amari, S., Murata, N., Muller, K.-R., Finke, M., & Yang, H. H.
- Venue
- IEEE Transactions on Neural Networks
- Published
- 1997-09
- Volume
- 8
- Issue
- 5
- Pages
- 985-996
- ISSN
- 1045-9227 (ISSN-L)
- 1941-0093
- OpenAlex
- W2141212940
- Cited by
- 351
- References
- 23
- FWCI
- 13.79
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.