The DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index.
On the problem of local minima in backpropagation
Abstract
The authors propose a theoretical framework for backpropagation (BP) in order to identify some of its limitations as a general learning procedure and the reasons for its success in several experiments on pattern recognition. The first important conclusion is that examples can be found in which BP gets stuck in local minima. A simple example in which BP can get stuck during gradient descent without having learned the entire training set is presented. This example guarantees the existence of a solution with null cost. Some conditions on the network architecture and the learning environment that ensure the convergence of the BP algorithm are proposed. It is proven in particular that the convergence holds if the classes are linearly separable. In this case, the experience gained in several experiments shows that multilayered neural networks (MLNs) exceed perceptrons in generalization to new examples.>
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Multilayer feedforward networks are universal approximators
Hornik, K., Stinchcombe, M., & White, H. (1989) · Neural Networks · doi:10.1016/0893-6080(89)90020-8 · Cited by 21,728
Learning Internal Representations by Error Propagation
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1985) · doi:10.21236/ada164453 · Cited by 16,172
Parallel Distributed Processing: Explorations in the Microstructure of Cognition: Foundations
Rumelhart, D. E., McClelland, J. L., & AU (1986) · doi:10.7551/mitpress/5236.001.0001 · Cited by 15,686
Approximation by superpositions of a sigmoidal function
Cybenko, G. (1989) · Mathematics of Control, Signals, and Systems · doi:10.1007/bf02551274 · Cited by 13,883
Multilayer feedforward networks are universal approximators
HornikK., StinchcombeM., & WhiteH. (1989) · Neural Networks · doi:10.5555/70405.70408 · Cited by 9,081
An introduction to computing with neural nets
Lippmann, R. (1987) · IEEE ASSP Magazine · doi:10.1109/massp.1987.1165576 · Cited by 7,825
Applied Optimal Control: Optimization, Estimation, and Control
Bryson, A. E., & Ho, Y.-C. (2018) · Routledge eBooks · doi:10.1201/9781315137667 · Cited by 6,015
J., J. E. (1979) · Technometrics · doi:10.1080/00401706.1979.10489803 · Cited by 3,073
Introduction to Matrix Analysis
Bellman, R. (1960) · Cited by 3,020
Phoneme recognition using time-delay neural networks
Waibel, A., Hanazawa, T., Hinton, G., Shikano, K., & Lang, K. J. (1989) · IEEE Transactions on Acoustics, Speech, and Signal Processing · doi:10.1109/29.21701 · Cited by 2,643
Introduction to matrix analysis
Amar, H. (1960) · Journal of the Franklin Institute · doi:10.1016/0016-0032(60)90904-2 · Cited by 2,316
30 years of adaptive neural networks: perceptron, Madaline, and backpropagation
Widrow, B., & Lehr, M. A. (1990) · Proceedings of the IEEE · doi:10.1109/5.58323 · Cited by 2,242
Theory of the backpropagation neural network
Hecht-Nielsen (1989) · International Joint Conference on Neural Networks · doi:10.1109/ijcnn.1989.118638 · Cited by 2,222
Increased rates of convergence through learning rate adaptation
Jacobs, R. A. (1988) · Neural Networks · doi:10.1016/0893-6080(88)90003-2 · Cited by 1,806
Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms
White, B. W., & Rosenblatt, F. (1963) · The American Journal of Psychology · doi:10.2307/1419730 · Cited by 1,792
Parallel Networks that Learn to Pronounce English Text
B, T. J. S. A. (1987) · Cited by 1,514
Neural networks and principal component analysis: Learning from examples without local minima
Baldi, P., & Hornik, K. (1989) · Neural Networks · doi:10.1016/0893-6080(89)90014-2 · Cited by 1,383
Generalization of back-propagation to recurrent neural networks
Pineda, F. J. (1987) · Physical Review Letters · doi:10.1103/physrevlett.59.2229 · Cited by 931
Generalization and network design strategies
LeCun, Y. (1989) · Cited by 885
A theoretical framework for back-propagation
LeCun, Y. (1988) · Cited by 583
An introduction to computing with neural nets
Lippmann, R. P. (1988) · ACM SIGARCH Computer Architecture News · doi:10.1145/44571.44572 · Cited by 539
Minsky, M., & Papert, S. A. (1988) · Cited by 281
Learning the hidden structure of speech
Elman, J. L., & Zipser, D. (1988) · The Journal of the Acoustical Society of America · doi:10.1121/1.395916 · Cited by 268
An Introduction to Neural and Electronic Networks
Zornetzer, S. F., Davis, J. L., & Lau, C. G. Y. (1995) · Cited by 258
Learning Algorithms for Connectionist Networks: Applied Gradient Methods of Nonlinear Optimization
Watrous, R. L. (1988) · ScholarlyCommons (University of Pennsylvania) · Cited by 242
Backpropagation: past and future
Werbos (1988) · IEEE International Conference on Neural Networks · doi:10.1109/icnn.1988.23866 · Cited by 228
VLSI implementation of a neural network model
Graf, H. P., Jackel, L. D., & Hubbard, W. E. (1988) · Computer · doi:10.1109/2.30 · Cited by 175
Learning Process in an Asymmetric Threshold Network
LeCun, Y. (1986) · Springer eBooks · doi:10.1007/978-3-642-82657-3_24 · Cited by 171
Neural Net and Traditional Classifiers
Huang, W. Y. C., & Lippmann, R. P. (1987) · neural information processing systems · Cited by 144
Back propagation fails to separate where perceptrons succeed
Brady, M., Raghavan, R., & Slawny, J. (1989) · IEEE Transactions on Circuits and Systems · doi:10.1109/31.31314 · Cited by 90
Speech pattern discrimination and multilayer perceptrons
Bourlard, H., & Wellekens, C. J. (1989) · Computer Speech & Language · doi:10.1016/0885-2308(89)90011-9 · Cited by 89
BPS: a learning algorithm for capturing the dynamic nature of speech
Gori, M., Bengio, & Mori, R. D. (1989) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.1989.118276 · Cited by 60
Self-citationPhonetically-based multi-layered neural networks for vowel classification
Cosi, P., Bengio, Y., & Mori, R. D. (1990) · Speech Communication · doi:10.1016/0167-6393(90)90041-7 · Cited by 20
Cited by
Raza, M. Q., & Khosravi, A. (2015) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2015.04.065 · Cited by 1,021
Comprehensive Review of Artificial Neural Network Applications to Pattern Recognition
Abiodun, O. I., Kiru, M. U., Jantan, A., Omolara, A. E., Dada, K. V., Umar, A. M., Linus, O. U., Arshad, H., Kazaure, A. A., & Gana, U. (2019) · IEEE Access · doi:10.1109/access.2019.2945545 · Cited by 750
Zhang, J.-R., Zhang, J., Lok, T.-M., & Lyu, M. R. (2007) · Applied Mathematics and Computation · doi:10.1016/j.amc.2006.07.025 · Cited by 706
Metaheuristic design of feedforward neural networks: A review of two decades of research
Ojha, V. K., Abraham, A., & Snášel, V. (2017) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2017.01.013 · Cited by 606
Mirjalili, S., Mohd Hashim, S. Z., & Moradian Sardroudi, H. (2012) · Applied Mathematics and Computation · doi:10.1016/j.amc.2012.04.069 · Cited by 505
Metadata
- Authors
- Gori, M., & Tesi, A.
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 1992
- Volume
- 14
- Issue
- 1
- Pages
- 76-86
- ISSN
- 0162-8828 (ISSN-L)
- OpenAlex
- W2022740958
- Cited by
- 650
- References
- 33
- FWCI
- 13.86
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.