The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
What Size Net Gives Valid Generalization?
Abstract
We address the question of when a network can be expected to generalize from m random training examples chosen from some arbitrary probability distribution, assuming that future test examples are drawn from the same distribution. Among our results are the following bounds on appropriate sample vs. network size. Assume 0 < ∊ ≤ 1/8. We show that if m ≥ O(W/∊ log N/∊) random examples can be loaded on a feedforward network of linear threshold functions with N nodes and W weights, so that at least a fraction 1 − ∊/2 of the examples are correctly classified, then one has confidence approaching certainty that the network will correctly classify a fraction 1 − ∊ of future test examples drawn from the same distribution. Conversely, for fully-connected feedforward nets with one hidden layer, any learning algorithm using fewer than Ω(W/∊) random training examples will, for some distributions of examples consistent with an appropriate weight choice, fail at least some fixed fraction of the time to find a weight choice that will correctly classify more than a 1 − ∊ fraction of the future test examples.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(1973) · CERN Document Server (European Organization for Nuclear Research) · Cited by 12,895
Backpropagation Applied to Handwritten Zip Code Recognition
LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., & Jackel, L. D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.4.541 · Cited by 11,891
On the Uniform Convergence of Relative Frequencies of Events to Their Probabilities
Vapnik, V. N., & Chervonenkis, A. Y. (2015) · Measures of Complexity · doi:10.1007/978-3-319-21852-6_3 · Cited by 3,105
Estimation of Dependences Based on Empirical Data
Vapnik, V. (2006) · Information Science and Statistics · doi:10.1007/0-387-34239-7 · Cited by 2,238
Convergence of Stochastic Processes.
Robinson, P. M., & Pollard, D. (1985) · Economica · doi:10.2307/2553898 · Cited by 2,184
Cover, T. M. (1965) · IEEE Transactions on Electronic Computers · doi:10.1109/pgec.1965.264137 · Cited by 2,044
Learnability and the Vapnik-Chervonenkis dimension
Blumer, A., Ehrenfeucht, A., Haussler, D., & Warmuth, M. K. (1989) · Journal of the ACM · doi:10.1145/76359.76371 · Cited by 1,839
Self-citationParallel Networks that Learn to Pronounce English Text
B, T. J. S. A. (1987) · Cited by 1,514
Connectionist learning procedures
Hinton, G. E. (1989) · Artificial Intelligence · doi:10.1016/0004-3702(89)90049-0 · Cited by 1,430
Predicting the secondary structure of globular proteins using neural network models
Qian, N., & Sejnowski, T. J. (1988) · Journal of Molecular Biology · doi:10.1016/0022-2836(88)90564-5 · Cited by 1,227
Blumer, A., Ehrenfeucht, A., Haussler, D., & Warmuth, M. K. (1987) · Information Processing Letters · doi:10.1016/0020-0190(87)90114-1 · Cited by 1,054
Self-citationLearning in Artificial Neural Networks: A Statistical Perspective
White, H. (1989) · Neural Computation · doi:10.1162/neco.1989.1.4.425 · Cited by 922
On the density of families of sets
Sauer, N. (1972) · Journal of Combinatorial Theory, Series A · doi:10.1016/0097-3165(72)90019-2 · Cited by 844
Learning quickly when irrelevant attributes abound: A new linear-threshold algorithm
Littlestone, N. (1988) · Machine Learning · doi:10.1007/bf00116827 · Cited by 497
Quantifying inductive bias: AI learning algorithms and Valiant's learning framework
Haussler, D. (1988) · Artificial Intelligence · doi:10.1016/0004-3702(88)90002-1 · Cited by 479
Self-citationA general lower bound on the number of examples needed for learning
Ehrenfeucht, A., Haussler, D., Kearns, M., & Valiant, L. (1989) · Information and Computation · doi:10.1016/0890-5401(89)90002-3 · Cited by 403
Self-citationOn the capabilities of multilayer perceptrons
Baum, E. B. (1988) · Journal of Complexity · doi:10.1016/0885-064x(88)90020-9 · Cited by 379
Self-citationUsing Relevance to Reduce Network Size Automatically
MOZER, M. C., & SMOLENSKY, P. (1989) · Connection Science · doi:10.1080/09540098908915626 · Cited by 339
Large Automatic Learning, Rule Extraction, and Generalization.
Denker, J. S., Schwartz, D., Wittner, B. S., Solla, S. A., Howard, R. E., Jackel, L. D., & Hopfield, J. J. (1987) · Complex Systems · Cited by 248
A statistical approach to learning and generalization in layered neural networks
Levin, E., Tishby, N., & Solla, S. A. (1990) · Proceedings of the IEEE · doi:10.1109/5.58339 · Cited by 248
Learning conjunctive concepts in structural domains
Haussler, D. (1989) · Machine Learning · doi:10.1007/bf00114802 · Cited by 171
Self-citationThe Vapnik-Chervonenkis Dimension: Information versus Complexity in Learning
Abu‐Mostafa, Y. S. (1989) · Neural Computation · doi:10.1162/neco.1989.1.3.312 · Cited by 151
Automatic pattern recognition: a study of the probability of error
Devroye, L. (1988) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.3915 · Cited by 144
Some special vapnik-chervonenkis classes
Wenocur, R. S., & Dudley, R. M. (1981) · Discrete Mathematics · doi:10.1016/0012-365x(81)90274-0 · Cited by 143
A general lower bound on the number of examples needed for learning
Ehrenfeucht, A., Haussler, D. H., Kearns, M. J., & Valiant, L. G. (1988) · Conference on Learning Theory · doi:10.5555/93025.93068 · Cited by 138
Self-citationAn Approach to Combining Explanation-based and Neural Learning Algorithms
SHAVLIK, J. W., & Towell, G. G. (1989) · Connection Science · doi:10.1080/09540098908915640 · Cited by 97
Bounds on the number of samples needed for neural learning
Mehrotra, K., Mohan, C. K., & Ranka, S. (1991) · IEEE Transactions on Neural Networks · doi:10.1109/72.97932 · Cited by 84
A Proposal for More Powerful Learning Algorithms
Baum, E. B. (1989) · Neural Computation · doi:10.1162/neco.1989.1.2.201 · Cited by 73
Self-citationOn subspaces spanned by random selections of ±1 vectors
Odlyzko, A. M. (1988) · Journal of Combinatorial Theory Series A · doi:10.1016/0097-3165(88)90046-5 · Cited by 59
Information theory, complexity and neural networks
Abu‐Mostafa, Y. S. (1989) · IEEE Communications Magazine · doi:10.1109/35.41397 · Cited by 33
A 'Neural' Network that Learns to Play Backgammon
Tesauro, G. J., & B, T. J. S. A. (1987) · Cited by 25
Valiant, L. G. (2026) · Foundations of Computation and Machine Learning · doi:10.1145/3749740.3749746
Cited by
A decision-theoretic generalization of on-line learning and an application to boosting
Freund, Y., & Schapire, R. E. (1997) · Journal of Computer and System Sciences · doi:10.1006/jcss.1997.1504 · Cited by 20,466
Deep learning in neural networks: an overview
Schmidhuber, J. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.09.003 · Cited by 18,570
Approximation by superpositions of a sigmoidal function
Cybenko, G. (1989) · Mathematics of Control, Signals, and Systems · doi:10.1007/bf02551274 · Cited by 13,883
A training algorithm for optimal margin classifiers
Boser, B. E., Guyon, I. M., & Vapnik, V. N. (1992) · Proceedings of the fifth annual workshop on Computational learning theory · doi:10.1145/130385.130401 · Cited by 11,992
Backpropagation Applied to Handwritten Zip Code Recognition
LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., & Jackel, L. D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.4.541 · Cited by 11,891
Metadata
- Authors
- Baum, E. B., & Haussler, D.
- Venue
- Neural Computation
- Publisher
- MIT Press - Journals
- Published
- 1989-03
- Volume
- 1
- Issue
- 1
- Pages
- 151-160
- ISSN
- 0899-7667 (ISSN-L)
- 1530-888X
- OpenAlex
- W2165758113
- Cited by
- 1,541
- References
- 32
- FWCI
- 62.41
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.