The DOI matches its Crossref record; 65 of the 65 references it cites were found in a registry or an index.
Learning in Artificial Neural Networks: A Statistical Perspective
Abstract
The premise of this article is that learning procedures used to train artificial neural networks are inherently statistical techniques. It follows that statistical theory can provide considerable insight into the properties, advantages, and disadvantages of different network learning methods. We review concepts and analytical results from the literatures of mathematical statistics, econometrics, systems identification, and optimization theory relevant to the analysis of learning in artificial neural networks. Because of the considerable variety of available learning procedures and necessary limitations of space, we cannot provide a comprehensive treatment. Our focus is primarily on learning procedures for feedforward networks. However, many of the concepts and issues arising in this framework are also quite broadly relevant to other network learning paradigms. In addition to providing useful insights, the material reviewed here suggests some potentially useful new training methods for artificial neural networks.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Mathematical Theory of Communication
Shannon, C. E. (1948) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1948.tb01338.x · Cited by 88,486
Genetic algorithms in search, optimization, and machine learning
(1989) · Choice Reviews Online · doi:10.5860/choice.27-0936 · Cited by 49,010
Holland, J. H. (1992) · The MIT Press eBooks · doi:10.7551/mitpress/1090.001.0001 · Cited by 47,002
Optimization by Simulated Annealing
Kirkpatrick, S., Gelatt, C. D., & Vecchi, M. P. (1983) · Science · doi:10.1126/science.220.4598.671 · Cited by 45,134
Multilayer feedforward networks are universal approximators
Hornik, K., Stinchcombe, M., & White, H. (1989) · Neural Networks · doi:10.1016/0893-6080(89)90020-8 · Cited by 21,728
Self-citationOn Information and Sufficiency
Kullback, S., & Leibler, R. A. (1951) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177729694 · Cited by 20,193
Learning Internal Representations by Error Propagation
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1985) · doi:10.21236/ada164453 · Cited by 16,172
Approximation by superpositions of a sigmoidal function
Cybenko, G. (1989) · Mathematics of Control, Signals, and Systems · doi:10.1007/bf02551274 · Cited by 13,883
Cross validatory choice and assessment of statistical predictions
Stone, M. (1974) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1974.tb00994.x · Cited by 10,881
A Stochastic Approximation Method
Robbins, H., & Monro, S. (1951) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177729586 · Cited by 9,643
Multilayer feedforward networks are universal approximators
HornikK., StinchcombeM., & WhiteH. (1989) · Neural Networks · doi:10.5555/70405.70408 · Cited by 9,081
Iterative Solution of Nonlinear Equations in Several Variables
Ortega, J. M., & Rheinboldt, W. C. (2000) · Society for Industrial and Applied Mathematics eBooks · doi:10.1137/1.9780898719468 · Cited by 6,860
PE, & Billingsley, P. (1996) · Journal of the American Statistical Association · doi:10.2307/2291440 · Cited by 6,481
Approximation Theorems of Mathematical Statistics
Serfling, R. J. (1980) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316481 · Cited by 5,851
On the approximate realization of continuous mappings by neural networks
Funahashi, K.-I. (1989) · Neural Networks · doi:10.1016/0893-6080(89)90003-8 · Cited by 4,206 · cited 1×
“…as a single hidden layer are capable of arbitrarily accurate approximation to an arbitrary mapping provided that sufficiently many hidden units are available [see Carroll and Dick-inson 1989; Cybenko 1989; Funahashi 1989; Hecht-Nielsen 1989; Hornik et al. 1989a,b (HSW); Stinchcombe and White 19891.”
BackgroundMaximum Likelihood Estimation of Misspecified Models
White, H. (1982) · Econometrica · doi:10.2307/1912526 · Cited by 4,098
Self-citationOn Measures of Entropy and Information
Rényi, A. (1961) · Project Euclid (Cornell University) · Cited by 4,031
Maximum Likelihood Estimation of Misspecified Models
White, H. (1983) · Econometrica · doi:10.2307/1912004 · Cited by 3,674
Self-citationThermodynamical approach to the traveling salesman problem: An efficient simulation algorithm
Černý, V. (1985) · Journal of Optimization Theory and Applications · doi:10.1007/bf00940812 · Cited by 3,436
Beyond Regression : "New Tools for Prediction and Analysis in the Behavioral Sciences
Werbos, P. J. (1974) · Medical Entomology and Zoology · Cited by 2,789
Beyond regression : new fools for prediction and analysis in the behavioral sciences
Werbos, P. J. (1974) · Medical Entomology and Zoology · Cited by 2,739
Stochastic Estimation of the Maximum of a Regression Function
Kiefer, J., & Wolfowitz, J. (1952) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177729392 · Cited by 2,133
Hornik, K., Stinchcombe, M., & White, H. (1990) · Neural Networks · doi:10.1016/0893-6080(90)90005-6 · Cited by 2,111
Self-citationHypothesis testing when a nuisance parameter is present only under the alternative
DAVIES, R. B. (1987) · Biometrika · doi:10.1093/biomet/74.1.33 · Cited by 2,056
Genetic Algorithms and Simulated Annealing
Davis, L. D. (1987) · Cited by 1,759
What Size Net Gives Valid Generalization?
Baum, E. B., & Haussler, D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.1.151 · Cited by 1,541
Analysis of recursive stochastic algorithms
Ljung, L. (1977) · IEEE Transactions on Automatic Control · doi:10.1109/tac.1977.1101561 · Cited by 1,467
Stochastic Approximation Methods for Constrained and Unconstrained Systems
Kushner, H. J., & Clark, D. S. (1978) · Applied Mathematical Sciences · doi:10.1007/978-1-4684-9352-8 · Cited by 1,459
Asymptotic Properties of Non-Linear Least Squares Estimators
Jennrich, R. I. (1969) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177697731 · Cited by 1,327
Asymptotic Theory for Econometricians.
Robinson, P. M., & White, H. (1988) · Economica · doi:10.2307/2554258 · Cited by 1,305
Self-citationCooling Schedules for Optimal Annealing
Hajek, B. (1988) · Mathematics of Operations Research · doi:10.1287/moor.13.2.311 · Cited by 1,246
Semi-Nonparametric Maximum Likelihood Estimation
Gallant, A. R., & Nychka, D. W. (1987) · Econometrica · doi:10.2307/1913241 · Cited by 913
Decision theoretic generalizations of the PAC model for neural net and other learning applications
Haussler, D. (1992) · Information and Computation · doi:10.1016/0890-5401(92)90010-d · Cited by 843
The multilayer perceptron as an approximation to a Bayes optimal discriminant function
Ruck, D. W., Rogers, S. K., Kabrisky, M., Oxley, M. E., & Suter, B. W. (1990) · IEEE Transactions on Neural Networks · doi:10.1109/72.80266 · Cited by 834
Connectionist nonparametric regression: Multilayer feedforward networks can learn arbitrary mappings
White, H. (1990) · Neural Networks · doi:10.1016/0893-6080(90)90004-5 · Cited by 679
Self-citation(2005) · Interdisciplinary Mathematical Sciences · doi:10.1142/9789812701039_0003 · Cited by 611
Multidimensional Stochastic Approximation Methods
Blum, J. R. (1954) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177728659 · Cited by 523
Hypothesis testing when a nuisance parameter is present only under the alternative
Davies, R. B. (1977) · Biometrika · doi:10.1093/biomet/64.2.247 · Cited by 471
A Unified Theory of Estimation and Inference for Nonlinear Dynamic Models.
Davidson, J., Gallant, A. R., & White, H. (1988) · The Economic Journal · doi:10.2307/2233735 · Cited by 468
Self-citationSome Asymptotic Results for Learning in Single Hidden-Layer Feedforward Network Models
White, H. (1989) · Journal of the American Statistical Association · doi:10.1080/01621459.1989.10478865 · Cited by 450
Self-citationDynamic node creation in backpropagation networks
ASH, T. (1989) · Connection Science · doi:10.1080/09540098908915647 · Cited by 415
Efficient distribution-free learning of probabilistic concepts
Kearns, M. J., & Schapire, R. E. (1994) · Journal of Computer and System Sciences · doi:10.1016/s0022-0000(05)80062-5 · Cited by 373
Nonparametric Maximum Likelihood Estimation by the Method of Sieves
Geman, S., & Hwang, C.-R. (1982) · The Annals of Statistics · doi:10.1214/aos/1176345782 · Cited by 367
Methods for Solving Systems of Nonlinear Equations
Rheinboldt, W. C. (1998) · Society for Industrial and Applied Mathematics eBooks · doi:10.1137/1.9781611970012 · Cited by 364
Partially Identified Econometric Models
Phillips, P. C. B. (1989) · Econometric Theory · doi:10.1017/s0266466600012408 · Cited by 337
Theory of the Back Propagation Neural Network
Hecht-Nielsen, R. (1989) · International Joint Conference on Neural Networks (IJCNN) · Cited by 324
Consequences and Detection of Misspecified Nonlinear Regression Models
White, H. (1981) · Journal of the American Statistical Association · doi:10.1080/01621459.1981.10477663 · Cited by 295
Self-citationNeural network classification: a Bayesian interpretation
Wan, E. A. (1990) · IEEE Transactions on Neural Networks · doi:10.1109/72.80269 · Cited by 274
Stinchcombe, & White (1989) · International Joint Conference on Neural Networks · doi:10.1109/ijcnn.1989.118640 · Cited by 251
Misspecified models with dependent observations
Domowitz, I., & White, H. (1982) · Journal of Econometrics · doi:10.1016/0304-4076(82)90102-6 · Cited by 250
Self-citationA statistical approach to learning and generalization in layered neural networks
Levin, E., Tishby, N., & Solla, S. A. (1990) · Proceedings of the IEEE · doi:10.1109/5.58339 · Cited by 248
Oxtoby, J. C., & Bartle, R. G. (1968) · The American Mathematical Monthly · doi:10.2307/2315250 · Cited by 164
Construction of neural nets using the radon transform
Carroll, & Dickinson (1989) · International Joint Conference on Neural Networks · doi:10.1109/ijcnn.1989.118639 · Cited by 162
Consistent inference of probabilities in layered networks: predictions and generalizations
Tishby, Levin, & Solla (1989) · International Joint Conference on Neural Networks · doi:10.1109/ijcnn.1989.118274 · Cited by 161
A tutorial survey of theory and applications of simulated annealing
Hajek, B. (1985) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc.1985.268599 · Cited by 148
Kushner, H. J. (1987) · SIAM Journal on Applied Mathematics · doi:10.1137/0147010 · Cited by 144
Neural network models of sensory integration for improved vowel recognition
Yuhas, B. P., Goldstein, M. H., B, T. J. S. A., & Jenkins, R. E. (1990) · Proceedings of the IEEE · doi:10.1109/5.58349 · Cited by 79
Statistical properties of artificial neural networks
Barron, A. R. (2003) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc.1989.70117 · Cited by 77
Consequences and Detection of Misspecified Nonlinear Regression Models
White, H. (1981) · Journal of the American Statistical Association · doi:10.2307/2287845 · Cited by 74
Self-citationA unified framework for connectionist systems
Golden, R. M. (1988) · Biological Cybernetics · doi:10.1007/bf00317773 · Cited by 73
A Stochastic Approach to Global Optimization
Kan, A. H. G. R., Boender, C. G. E., & Timmer, G. T. (1985) · Springer eBooks · doi:10.1007/978-3-642-82450-0_10 · Cited by 65
Global optimization : a stochastic approach
Boender, C. G. E., Kan, A. H. G. R., Stougie, L., Timmer, G. T., Archetti, F., & Cugiani, M. (1980) · TU/e Research Portal · Cited by 53
Consistent inference of probabilities in layered networks: Predictions and generalization
Tishby, N. (1989) · International Joint Conference on Neural Network · Cited by 40
A note on least-squares learning procedures and classification by neural network models
Shoemaker, P. A. (1991) · IEEE Transactions on Neural Networks · doi:10.1109/72.80304 · Cited by 30
A protein structure predictor based on an energy model with learned parameters
Bryngelson, J. D., Hopfield, J. J., & Southard, S. (1990) · Tetrahedron Computer Methodology · doi:10.1016/0898-5529(90)90048-d · Cited by 16
Cited by
Deep learning in neural networks: an overview
Schmidhuber, J. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.09.003 · Cited by 18,570
McClelland, J. L., McNaughton, B. L., & O'Reilly, R. C. (1995) · Psychological Review · doi:10.1037/0033-295x.102.3.419 · Cited by 5,256
Forecasting with artificial neural networks:
Zhang, G., Eddy Patuwo, B., & Y. Hu, M. (1998) · International Journal of Forecasting · doi:10.1016/s0169-2070(97)00044-7 · Cited by 4,229
Neural Networks and the Bias/Variance Dilemma
Geman, S., Bienenstock, E., & Doursat, R. (1992) · Neural Computation · doi:10.1162/neco.1992.4.1.1 · Cited by 3,578
Artificial neural networks: fundamentals, computing, design, and application
Basheer, I. A., & Hajmeer, M. (2000) · Journal of Microbiological Methods · doi:10.1016/s0167-7012(00)00201-3 · Cited by 3,231
Metadata
- Authors
- White, H.
- Venue
- Neural Computation
- Publisher
- MIT Press
- Published
- 1989-12
- Volume
- 1
- Issue
- 4
- Pages
- 425-464
- ISSN
- 0899-7667 (ISSN-L)
- 1530-888X
- OpenAlex
- W2020246210
- Cited by
- 922
- References
- 65
- FWCI
- 23.17
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.