The DOI matches its Crossref record; 43 of the 43 references it cites were found in a registry or an index.
Learning in linear neural networks: a survey
Abstract
Networks of linear units are the simplest kind of networks, where the basic questions related to learning, generalization, and self-organization can sometimes be answered analytically. We survey most of the known results on linear networks, including: 1) backpropagation learning and the structure of the error function landscape, 2) the temporal evolution of generalization, and 3) unsupervised learning algorithms and their properties. The connections to classical statistical ideas, such as principal component analysis (PCA), are emphasized as well as several simple but challenging open questions. A few new results are also spread across the paper, including an analysis of the effect of noise on backpropagation networks and a unified view of all unsupervised algorithms.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Horn, R. A., & Johnson, C. R. (1985) · Cited by 22,203
A logical calculus of the ideas immanent in nervous activity
McCulloch, W. S., & Pitts, W. (1943) · The Bulletin of Mathematical Biophysics · doi:10.1007/bf02478259 · Cited by 18,566
Horn, R. A., & Johnson, C. R. (1985) · Cambridge University Press eBooks · doi:10.1017/cbo9780511810817 · Cited by 10,280
WIDROW, B., & HOFF, M. E. (1960) · doi:10.21236/ad0241531 · Cited by 4,009
Simplified neuron model as a principal component analyzer
Oja, E. (1982) · Journal of Mathematical Biology · doi:10.1007/bf00275687 · Cited by 2,309
Advances in Neural Information Processing Systems 5
Zemel, R. S., Williams, C. K. I., & Mozer, M. C. (1993) · Cited by 1,896
Matrix Differential Calculus With Applications in Statistics and Econometrics
Ziegel, E. (1989) · Technometrics · doi:10.1080/00401706.1989.10488622 · Cited by 1,607
What Size Net Gives Valid Generalization?
Baum, E. B., & Haussler, D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.1.151 · Cited by 1,541
Optimal unsupervised learning in a single-layer linear feedforward neural network
Sanger, T. D. (1989) · Neural Networks · doi:10.1016/0893-6080(89)90044-0 · Cited by 1,509
Self-organization in a perceptual network
Linsker, R. (1988) · Computer · doi:10.1109/2.36 · Cited by 1,469
Neural networks and principal component analysis: Learning from examples without local minima
Baldi, P., & Hornik, K. (1989) · Neural Networks · doi:10.1016/0893-6080(89)90014-2 · Cited by 1,383
Self-citationAuto-association by multilayer perceptrons and singular value decomposition
Bourlard, H., & Kamp, Y. (1988) · Biological Cybernetics · doi:10.1007/bf00332918 · Cited by 1,345
Principles and practice of information theory
Blahut, R. E. (1987) · Medical Entomology and Zoology · Cited by 1,008
Advances in Neural Information Processing Systems 7
Tesauro, G., Touretzky, D. S., & Leen, T. K. (1995) · Cited by 869
NEURAL NETWORKS, PRINCIPAL COMPONENTS, AND SUBSPACES
Oja, E. (1989) · International Journal of Neural Systems · doi:10.1142/s0129065789000475 · Cited by 821
Principal components, minor components, and linear neural networks
Oja, E. (1992) · Neural Networks · doi:10.1016/s0893-6080(05)80089-9 · Cited by 814
Ocular Dominance Column Development: Analysis and Simulation
Miller, K. D., Keller, J. B., & Stryker, M. P. (1989) · Science · doi:10.1126/science.2762813 · Cited by 645
The Use of Control Systems Analysis in the Neurophysiology of Eye Movements
Robinson, D. A. (1981) · Annual Review of Neuroscience · doi:10.1146/annurev.ne.04.030181.002335 · Cited by 641
The eigenvalues of random symmetric matrices
Füredi, Z., & Komlós, J. (1981) · Combinatorica · doi:10.1007/bf02579329 · Cited by 633
From basic network principles to neural architecture: emergence of spatial-opponent cells.
Linsker, R. (1986) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.83.19.7508 · Cited by 386
Least mean square error reconstruction principle for self-organizing neural-nets
Xu, L. (1993) · Neural Networks · doi:10.1016/s0893-6080(05)80107-8 · Cited by 341
Adaptation and decorrelation in the cortex
Barlow, H. B., & Földiák, P. (1989) · Addison-Wesley Longman Publishing Co., Inc. eBooks · Cited by 340
Adaptive network for optimal linear feature extraction
Foldiak (1989) · International Joint Conference on Neural Networks · doi:10.1109/ijcnn.1989.118615 · Cited by 269
Gibbs, R. W., & Sharkey, N. E. (1988) · The American Journal of Psychology · doi:10.2307/1423234 · Cited by 257
A statistical approach to learning and generalization in layered neural networks
Levin, E., Tishby, N., & Solla, S. A. (1990) · Proceedings of the IEEE · doi:10.1109/5.58339 · Cited by 248
From basic network principles to neural architecture: emergence of orientation-selective cells.
Linsker, R. (1986) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.83.21.8390 · Cited by 244
A Self-Organizing Network for Principal-Component Analysis
Rubner, J., & Tavan, P. (1989) · Europhysics Letters (EPL) · doi:10.1209/0295-5075/10/7/015 · Cited by 237
Artificial neural networks for speech and vision
Mammone, R. J. (1994) · Cited by 231
Neale, M. C., & Cardon, L. R. (1992) · Methodology for Genetic Studies of Twins and Families · doi:10.1007/978-94-015-8018-2_12 · Cited by 204
A neural network learning algorithm for adaptive principal component extraction (APEX)
Kung, S. Y., & Diamantaras, K. I. (2002) · International Conference on Acoustics, Speech, and Signal Processing · doi:10.1109/icassp.1990.115975 · Cited by 163
Learning from examples in large neural networks
Sompolinsky, H., Tishby, N., & Seung, H. S. (1990) · Physical Review Letters · doi:10.1103/physrevlett.65.1683 · Cited by 152
Convergence of learning algorithms with constant learning rates
Kuan, C.-M., & Hornik, K. (1991) · IEEE Transactions on Neural Networks · doi:10.1109/72.134285 · Cited by 140
Self-citationGeneralization in a linear perceptron in the presence of noise
Krogh, A., & Hertz, J. (1992) · Journal of Physics A Mathematical and General · doi:10.1088/0305-4470/25/5/020 · Cited by 138
Convergence analysis of local feature extraction algorithms
Hornik, K., & Kuan, C.-M. (1992) · Neural Networks · doi:10.1016/s0893-6080(05)80022-x · Cited by 105
Self-citationAnalysis of Linsker's Simulations of Hebbian Rules
Mackay, D., & Miller, K. D. (1990) · Neural Computation · doi:10.1162/neco.1990.2.2.173 · Cited by 67
Neural networks for computing eigenvalues and eigenvectors
Cichocki, A. S., & Unbehauen, R. (1992) · Biological Cybernetics · doi:10.1007/bf00201437 · Cited by 64
Multilayer perceptrons and data analysis
Gallinari, Thiria, & Soulie (1988) · IEEE International Conference on Neural Networks/IEEE ... International Conference on Neural Networks · doi:10.1109/icnn.1988.23871 · Cited by 59
Efficient information transfer and anti-Hebbian neural networks
Plumbley, M. D. (1993) · Neural Networks · doi:10.1016/s0893-6080(05)80127-3 · Cited by 53
Anti-Hebbian learning in topologically constrained linear networks: a tutorial
Palmieri, F., Zhu, J., & Chang, C. (1993) · IEEE Transactions on Neural Networks · doi:10.1109/72.248453 · Cited by 50
Principal component analysis by gradient descent on a constrained linear Hebbian cell
Chauvin (1989) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.1989.118611 · Cited by 42
Dynamics of learning in linear feature-discovery networks
Leen, T. K. (1991) · Network Computation in Neural Systems · doi:10.1088/0954-898x_2_1_005 · Cited by 42
Temporal Evolution of Generalization during Learning in Linear Networks
Baldi, P., & Chauvin, Y. (1991) · Neural Computation · doi:10.1162/neco.1991.3.4.589 · Cited by 41
Self-citationLinear Learning: Landscapes and Algorithms
Baldi, P. (1988) · neural information processing systems · Cited by 31
Self-citation
Cited by
Deep learning in neural networks: an overview
Schmidhuber, J. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.09.003 · Cited by 18,570
Neural networks for classification: a survey
Zhang, G. P. (2000) · IEEE Transactions on Systems, Man and Cybernetics, Part C (Applications and Reviews) · doi:10.1109/5326.897072 · Cited by 1,667
Computational Intelligence in Wireless Sensor Networks: A Survey
Kulkarni, R. V., Forster, A., & Venayagamoorthy, G. K. (2011) · IEEE Communications Surveys & Tutorials · doi:10.1109/surv.2011.040310.00002 · Cited by 695
A class of neural networks for independent component analysis
Karhunen, J., Oja, E., Wang, L., Vigario, R., & Joutsensalo, J. (1997) · IEEE Transactions on Neural Networks · doi:10.1109/72.572090 · Cited by 402
Nonconvex Optimization Meets Low-Rank Matrix Factorization: An Overview
Chi, Y., Lu, Y. M., & Chen, Y. (2019) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2019.2937282 · Cited by 360
Metadata
- Authors
- Baldi, P. F., & Hornik, K.
- Venue
- IEEE Transactions on Neural Networks
- Published
- 1995-07
- Volume
- 6
- Issue
- 4
- Pages
- 837-858
- ISSN
- 1045-9227 (ISSN-L)
- OpenAlex
- W2137902180
- Cited by
- 252
- References
- 43
- FWCI
- 12.45
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.