The DOI matches its Crossref record; 103 of the 103 references it cites were found in a registry or an index.
Neural Networks and the Bias/Variance Dilemma
Abstract
Feedforward neural networks trained by error backpropagation are examples of nonparametric regression estimators. We present a tutorial on nonparametric inference and its relation to neural networks, and we use the statistical viewpoint to highlight strengths and weaknesses of neural models. We illustrate the main points with some recognition experiments involving artificial data as well as handwritten numerals. In way of conclusion, we suggest that current-generation feedforward neural networks are largely inadequate for difficult problems in machine perception and machine learning, regardless of parallel-versus-serial hardware or other implementation issues. Furthermore, we suggest that the fundamental challenges in neural modeling are about representation rather than learning per se. This last point is supported by additional experiments with handwritten numerals.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The bias-variance trade-off implies that a model should balance under-fitting and over-fitting: rich enough to express underlying structure in data, simple enough to avoid fitting spurious patterns.”
From Reconciling modern machine-learning practice and the classical bias–variance trade-off (2019) · cites this paper as [17]
In the citing paper
“The concept of bias in machine learning (ML) research and more generally in the field of predictive modeling is intrinsically tied to the concept of variance.14”
From Bias in artificial intelligence for medical imaging: fundamentals, detection, avoidance, mitigation, challenges, ethics, and prospects (2024) · cites this paper as [14]
Citations
Cites
Maximum Likelihood from Incomplete Data Via the EM Algorithm
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1977.tb01600.x · Cited by 50,444
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892
Classification and Regression Trees.
Gordon, A. D., Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1984) · Biometrics · doi:10.2307/2530946 · Cited by 23,867
Multilayer feedforward networks are universal approximators
Hornik, K., Stinchcombe, M., & White, H. (1989) · Neural Networks · doi:10.1016/0893-6080(89)90020-8 · Cited by 21,728
Classification and Regression Trees.
Ryzin, J. V., Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1986) · Journal of the American Statistical Association · doi:10.2307/2288003 · Cited by 21,030
Information Theory and an Extension of the Maximum Likelihood Principle
Akaike, H. (1998) · Springer series in statistics · doi:10.1007/978-1-4612-1694-0_15 · Cited by 18,636
Learning Internal Representations by Error Propagation
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1985) · doi:10.21236/ada164453 · Cited by 16,501
Approximation by superpositions of a sigmoidal function
Cybenko, G. (1989) · Mathematics of Control, Signals, and Systems · doi:10.1007/bf02551274 · Cited by 13,883
(1973) · CERN Document Server (European Organization for Nuclear Research) · Cited by 12,895
Backpropagation Applied to Handwritten Zip Code Recognition
LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., & Jackel, L. D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.4.541 · Cited by 11,891
Research works printed but not linked: 7
- 10.1016/S0364-0213(85)80012-4 · DOI · Paper page
- Barron A. R., 1989, Florida, 280
- Baum L. E., 1972, Inequalities, 3, 1
- Burr D. J., 1980, IEEE Transact. Pattern Anal. Machine Intell. PAMI-3, 6, 708
- Chauvin Y., 1990, Neural Information Processing Systems 11, D. S. Touretzky, ed., 642
- Denker J., 1987, Complex Syst., 1, 877
- Goldman L., 1982, J. Med., 307, 588
Cited by
Sorted by citations
Items 1,001–1,006 of 1,006
Schmidhuber, J. (2003) · Natural Computing Series · doi:10.1007/978-3-642-18965-4_23 · Cited by 65
AutoMoDe-Mate: Automatic off-line design of spatially-organizing behaviors for robot swarms
Mendiburu, F. J., Garzón Ramos, D., Morais, M. R. A., Lima, A. M. N., & Birattari, M. (2022) · Swarm and Evolutionary Computation · doi:10.1016/j.swevo.2022.101118 · Cited by 15
(1996) · Cited by 4,738
“Inductive biases often trade flexibility for improved sample complexity and can be understood in terms of the bias-variance tradeoff (Geman et al., 1992).”
Background(2010) · doi:10.5555/1756006.1859921 · Cited by 1,924
“The model parameters ( w , b ) are given by the minimum of a regularised (Tikhonov and Arsenin, 1977) least-squares loss function, where l is a regularisation parameter controlling the bias-variance trade-off (Geman et al., 1992).”
BackgroundMethod
Metadata
- Authors
- Geman, S., Bienenstock, E., & Doursat, R.
- Affiliations
- Brown University (US)
- ESPCI Paris (FR)
- ORCID
- Doursat, René
- Venue
- Neural Computation
- Publisher
- MIT Press
- Published
- 1992-01
- Volume
- 4
- Issue
- 1
- Pages
- 1-58
- ISSN
- 0899-7667 (ISSN-L)
- 1530-888X
- OpenAlex
- W2076118331
- Cited by
- 3,578
- References
- 96
- Citations by year
- 2026: 125 · 2025: 131 · 2024: 115 · 2023: 128 · 2022: 148 · 2021: 201 · 2020: 181 · 2019: 142 · 2018: 126 · 2017: 97 · 2016: 115 · 2015: 99 · 2014: 103 · 2013: 82 · 2012: 110
- FWCI
- 60.04
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Neural Computation (publishedVersion)
- e-space (Manchester Metropolitan University) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and ELM
- Keywords
- bias-variance trade-off, feedforward neural networks, backpropagation, nonparametric regression, nonparametric inference, handwritten digit recognition, machine perception, neural network representations, statistical learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
