The DOI matches its Crossref record; 92 of the 92 references it cites were found in a registry or an index.
30 years of adaptive neural networks: perceptron, Madaline, and backpropagation
Abstract
Fundamental developments in feedforward artificial neural networks from the past thirty years are reviewed. The history, origination, operating characteristics, and basic theory of several supervised neural-network training algorithms (including the perceptron rule, the least-mean-square algorithm, three Madaline rules, and the backpropagation technique) are described. The concept underlying these iterative adaptation algorithms is the minimal disturbance principle, which suggests that during training it is advisable to inject new information into a network in a manner that disturbs stored information to the smallest extent possible. The two principal kinds of online rules that have developed for altering the weights of a network are examined for both single-threshold elements and multielement networks. They are error-correction rules, which alter the weights of a network to correct error in the output response to the present input pattern, and gradient rules, which alter the weights of a network during each pattern presentation by gradient descent with the objective of reducing mean-square error (averaged over all training patterns).>
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Neural networks and physical systems with emergent collective computational abilities.
Hopfield, J. J. (1982) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.79.8.2554 · Cited by 19,270
Learning Internal Representations by Error Propagation
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1985) · doi:10.21236/ada164453 · Cited by 16,172
Parallel Distributed Processing: Explorations in the Microstructure of Cognition: Foundations
Rumelhart, D. E., McClelland, J. L., & AU (1986) · doi:10.7551/mitpress/5236.001.0001 · Cited by 15,686
Approximation by superpositions of a sigmoidal function
Cybenko, G. (1989) · Mathematics of Control, Signals, and Systems · doi:10.1007/bf02551274 · Cited by 13,883
Self-organized formation of topologically correct feature maps
Kohonen, T. (1982) · Biological Cybernetics · doi:10.1007/bf00337288 · Cited by 9,529
Self-Organization and Associative Memory
Kohonen, T. (1989) · Springer Series in Information Sciences · doi:10.1007/978-3-642-88163-3 · Cited by 8,740
Identification and control of dynamical systems using neural networks
Narendra, K. S., & Parthasarathy, K. (1990) · IEEE Transactions on Neural Networks · doi:10.1109/72.80202 · Cited by 8,014
An introduction to computing with neural nets
Lippmann, R. (1987) · IEEE ASSP Magazine · doi:10.1109/massp.1987.1165576 · Cited by 7,825
indicated, N. A. (1950) · Journal of Consulting Psychology · doi:10.1037/h0049612 · Cited by 6,923
Hopfield, J. J. (1984) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.81.10.3088 · Cited by 6,918
“Neural” computation of decisions in optimization problems
Hopfield, J. J., & Tank, D. W. (1985) · Biological Cybernetics · doi:10.1007/bf00339943 · Cited by 6,031
Fukushima, K. (1980) · Biological Cybernetics · doi:10.1007/bf00344251 · Cited by 5,759
(1991) · Springer eBooks · doi:10.1007/978-3-7091-2840-4 · Cited by 5,095
Extrapolation, Interpolation, and Smoothing of Stationary Time Series: With Engineering Applications
Wiener, N. (1949) · The MIT Press eBooks · doi:10.7551/mitpress/2946.001.0001 · Cited by 4,826
Linear and nonlinear programming.
(1986) · Mathematics and Computers in Simulation · doi:10.1016/0378-4754(86)90092-3 · Cited by 4,805
A Learning Algorithm for Continually Running Fully Recurrent Neural Networks
Williams, R. J., & Zipser, D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.2.270 · Cited by 4,369
WIDROW, B., & HOFF, M. E. (1960) · doi:10.21236/ad0241531 · Cited by 4,009
Self-citationAdaptive noise cancelling: Principles and applications
Widrow, B., Glover, J. R., McCool, J. M., Kaunitz, J., Williams, C. S., Hearn, R. H., Zeidler, J. R., Eugene Dong, Jr., & Goodlin, R. C. (1975) · Proceedings of the IEEE · doi:10.1109/proc.1975.10036 · Cited by 3,939
Self-citationSelf-Organization and Associative Memory
Kohonen, T. (1988) · Springer Series in Information Sciences · doi:10.1007/978-3-662-00784-6 · Cited by 3,812
Perceptrons: An Introduction to Computational Geometry
Minsky, M., & Papert, S. A. (1969) · Cited by 3,718
McClelland, J. L., Rumelhart, D. E., & AU (1987) · doi:10.7551/mitpress/5237.001.0001 · Cited by 3,315
Neuronlike adaptive elements that can solve difficult learning control problems
Barto, A. G., Sutton, R. S., & Anderson, C. W. (1983) · IEEE Transactions on Systems, Man, and Cybernetics · doi:10.1109/tsmc.1983.6313077 · Cited by 3,233
Beyond Regression : "New Tools for Prediction and Analysis in the Behavioral Sciences
Werbos, P. J. (1974) · Medical Entomology and Zoology · Cited by 2,789
Beyond regression : new fools for prediction and analysis in the behavioral sciences
Werbos, P. J. (1974) · Medical Entomology and Zoology · Cited by 2,739
Alexander, S. T. (1986) · Springer eBooks · doi:10.1007/978-1-4612-4978-8 · Cited by 2,646
Phoneme recognition using time-delay neural networks
Waibel, A., Hanazawa, T., Hinton, G., Shikano, K., & Lang, K. J. (1989) · IEEE Transactions on Acoustics, Speech, and Signal Processing · doi:10.1109/29.21701 · Cited by 2,643
A massively parallel architecture for a self-organizing neural pattern recognition machine
Carpenter, G. A., & Grossberg, S. (1987) · Computer Vision, Graphics, and Image Processing · doi:10.1016/s0734-189x(87)80014-2 · Cited by 2,525
A New Approach to Manipulator Control: The Cerebellar Model Articulation Controller (CMAC)
Albus, J. S. (1975) · Journal of Dynamic Systems, Measurement, and Control · doi:10.1115/1.3426922 · Cited by 2,177
Grossberg, S. (1976) · Biological Cybernetics · doi:10.1007/bf00344744 · Cited by 1,909
Increased rates of convergence through learning rate adaptation
Jacobs, R. A. (1988) · Neural Networks · doi:10.1016/0893-6080(88)90003-2 · Cited by 1,806
Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms
White, B. W., & Rosenblatt, F. (1963) · The American Journal of Psychology · doi:10.2307/1419730 · Cited by 1,792
Polynomial Theory of Complex Systems
Ivakhnenko, A. G. (1971) · IEEE Transactions on Systems, Man, and Cybernetics · doi:10.1109/tsmc.1971.4308320 · Cited by 1,684
Analog VLSI Implementation of Neural Systems
(1989) · The Kluwer International Series in Engineering and Computer Science · doi:10.1007/978-1-4613-1639-8 · Cited by 1,680
What Size Net Gives Valid Generalization?
Baum, E. B., & Haussler, D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.1.151 · Cited by 1,541
Parallel Networks that Learn to Pronounce English Text
B, T. J. S. A. (1987) · Cited by 1,514
Self-organization of orientation sensitive cells in the striate cortex
von der Malsburg, C. (1973) · Kybernetik · doi:10.1007/bf00288907 · Cited by 1,461
ART 2: self-organization of stable category recognition codes for analog input patterns
Carpenter, G. A., & Grossberg, S. (1987) · Applied Optics · doi:10.1364/ao.26.004919 · Cited by 1,442
Nguyen, D., & Widrow, B. (1990) · 1990 IJCNN International Joint Conference on Neural Networks · doi:10.1109/ijcnn.1990.137819 · Cited by 1,375
Self-citationAdaptive bidirectional associative memories
Kosko, B. (1987) · Applied Optics · doi:10.1364/ao.26.004947 · Cited by 1,103
Widrow, B., Mantey, P. E., Griffiths, L. J., & Goode, B. B. (1967) · Proceedings of the IEEE · doi:10.1109/proc.1967.6092 · Cited by 1,051
Self-citationLearning and relearning in Boltzmann machines
Hinton, G. E., & Sejnowski, T. J. (2001) · Graphical Models · doi:10.7551/mitpress/3349.003.0005 · Cited by 1,037
Grossberg, S. (1976) · Biological Cybernetics · doi:10.1007/bf00340335 · Cited by 961
Generalization of back-propagation to recurrent neural networks
Pineda, F. J. (1987) · Physical Review Letters · doi:10.1103/physrevlett.59.2229 · Cited by 931
Neural Information Processing Systems
Anderson, D. Z. (1988) · neural information processing systems · Cited by 784
Tukey, J. W., & Wiener, N. (1952) · Journal of the American Statistical Association · doi:10.2307/2280758 · Cited by 760
Cognitron: A self-organizing multilayered neural network
Fukushima, K. (1975) · Biological Cybernetics · doi:10.1007/bf00342633 · Cited by 757
Principles of data communication
Lucky, R. W., Weldon, E. J., & Salz, J. (1968) · CERN Document Server (European Organization for Nuclear Research) · Cited by 754
A view of three decades of linear filtering theory
Kailath, T. (1974) · IEEE Transactions on Information Theory · doi:10.1109/tit.1974.1055174 · Cited by 702
Buehler, R. J., & Wilde, D. J. (1965) · Econometrica · doi:10.2307/1909808 · Cited by 695
Learning, invariance, and generalization in high-order neural networks
Giles, C. L., & Maxwell, T. (1987) · Applied Optics · doi:10.1364/ao.26.004972 · Cited by 676
NETtalk: a parallel network that learns to read aloud
B, T. J. S. A., & Rosenberg, C. (1988) · The MIT Press eBooks · Cited by 670
Learning State Space Trajectories in Recurrent Neural Networks
Pearlmutter, B. A. (1989) · Neural Computation · doi:10.1162/neco.1989.1.2.263 · Cited by 656
Widrow, B., & Bilello, M. (2002) · Proceedings of 8th IEEE International Symposium on Intelligent Control · doi:10.1109/isic.1993.397732 · Cited by 594
Self-citationA theoretical framework for back-propagation
LeCun, Y. (1988) · Cited by 583
The least mean fourth (LMF) adaptive algorithm and its family
Walach, E., & Widrow, B. (1984) · IEEE Transactions on Information Theory · doi:10.1109/tit.1984.1056886 · Cited by 568
Self-citationPosition, rotation, and scale invariant optical correlation
Casasent, D., & Psaltis, D. (1976) · Applied Optics · doi:10.1364/ao.15.001795 · Cited by 520
Carpenter, G. A., & Grossberg, S. (1990) · Neural Networks · doi:10.1016/0893-6080(90)90085-y · Cited by 481
Neural nets for adaptive filtering and adaptive pattern recognition
Widrow, B., & Winter, R. (1988) · Computer · doi:10.1109/2.29 · Cited by 469
Self-citationThe truck backer-upper: an example of self-learning in neural networks
Nguyen, & Widrow (1989) · International Joint Conference on Neural Networks · doi:10.1109/ijcnn.1989.118723 · Cited by 449
Widrow, B., Mantey, P., Griffiths, L., & Goode, B. (1967) · The Journal of the Acoustical Society of America · doi:10.1121/1.2144028 · Cited by 415
Self-citationCapabilities of three-layered perceptrons
Irie, & Miyake (1988) · IEEE International Conference on Neural Networks · doi:10.1109/icnn.1988.23901 · Cited by 398
The Perceptron: A Model for Brain Functioning. I
Block, H. D. (1962) · Reviews of Modern Physics · doi:10.1103/revmodphys.34.123 · Cited by 396
On the capabilities of multilayer perceptrons
Baum, E. B. (1988) · Journal of Complexity · doi:10.1016/0885-064x(88)90020-9 · Cited by 379
Automatic equalization for digital communication
Becker, F. K., Holzman, L. N., Lucky, R. W., & Port, E. (1965) · Proceedings of the IEEE · doi:10.1109/proc.1965.3546 · Cited by 317
Sensor-based control of robotic manipulators using a general learning algorithm
Miller, W. (1987) · IEEE Journal on Robotics and Automation · doi:10.1109/jra.1987.1087081 · Cited by 310
A learning rule for asynchronous perceptrons with feedback in a combinatorial environment
Almeida, L. B. (1990) · IEEE Press eBooks · Cited by 280
Punish/Reward: Learning with a Critic in Adaptive Threshold Systems
Widrow, B., Gupta, N. K., & Maitra, S. (1973) · IEEE Transactions on Systems, Man, and Cybernetics · doi:10.1109/tsmc.1973.4309272 · Cited by 280
Self-citationA Simplified Derivation of Linear Least Square Smoothing and Prediction Theory
Bode, H. W., & Shannon, C. E. (1950) · Proceedings of the IRE · doi:10.1109/jrproc.1950.231821 · Cited by 276
Survey of neural network technology for automatic target recognition
Roth, M. W. (1990) · IEEE Transactions on Neural Networks · doi:10.1109/72.80203 · Cited by 268
Information capacity of the Hopfield model
Abu-Mostafa, Y., & St. Jacques, J. (1985) · IEEE Transactions on Information Theory · doi:10.1109/tit.1985.1057069 · Cited by 267
Sondhi, M. M. (1967) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1967.tb04231.x · Cited by 259
Abu‐Mostafa, Y. S. (1987) · Cited by 257
Stinchcombe, & White (1989) · International Joint Conference on Neural Networks · doi:10.1109/ijcnn.1989.118640 · Cited by 251
Generation of Polynomial Discriminant Functions for Pattern Recognition
Specht, D. F. (1967) · IEEE Transactions on Electronic Computers · doi:10.1109/pgec.1967.264667 · Cited by 249
Accelerated learning in layered neural networks
Solla, S. A., Levin, E., & Fleisher, M. (1988) · Complex Systems · Cited by 213
Denker, J. S. (1998) · Cited by 197
Pattern recognition by labeled graph matching
von der Malsburg, C. (1988) · Neural Networks · doi:10.1016/0893-6080(88)90016-0 · Cited by 162
Wilder, J. (1962) · American Journal of Psychotherapy · doi:10.1176/appi.psychotherapy.1962.16.2.326 · Cited by 162
Linear and nonlinear programming
Berck, P., & Sydsæter, K. (1991) · Springer eBooks · doi:10.1007/978-3-662-02678-6_14 · Cited by 155
A multilayer perceptron network for the diagnosis of low back pain
Bounds, Lloyd, Mathew, & Waddell (1988) · IEEE International Conference on Neural Networks/IEEE ... International Conference on Neural Networks · doi:10.1109/icnn.1988.23963 · Cited by 137
Efficient Parallel Learning Algorithms for Neural Networks
Kramer, A., & Sangiovanni-Vincentelli, A. L. (1988) · neural information processing systems · Cited by 134
Memory capacity in neural network models: Rigorous lower bounds
Newman, C. M. (1988) · Neural Networks · doi:10.1016/0893-6080(88)90028-7 · Cited by 120
Learning matrices and their applications
Steinbuch, K., & Piske, U. A. W. (1963) · IEEE Transactions on Electronic Computers · doi:10.1109/pgec.1963.263588 · Cited by 120
Widrow, B. C. (1987) · IFAC Proceedings Volumes · doi:10.1016/s1474-6670(17)55929-7 · Cited by 111
Self-citationLewis, A. C. L. C. (1967) · Cited by 97
Performance advantage of complex LMS for controlling narrow-band adaptive arrays
Horowitz, L. L., & Senne, K. D. (1981) · IEEE Transactions on Circuits and Systems · doi:10.1109/tcs.1981.1085024 · Cited by 77
Whitteridge, D. (1959) · The Lancet · doi:10.1016/s0140-6736(59)90152-7 · Cited by 69
Owens, & Filkin (1989) · doi:10.1109/ijcnn.1989.118726 · Cited by 61
Epsilon capacity of neural networks
Venkatesh, S. S. (1986) · AIP conference proceedings · doi:10.1063/1.36217 · Cited by 46
Functional link nets: removing hidden layers
Pao, Y. (1989) · AI Expert archive · Cited by 24
Generalization and information storage in network of adaline 'neurons'
Widrow, B. C. (1962) · Medical Entomology and Zoology · Cited by 24
Self-citationNeutral networks for computing?
Abu‐Mostafa, Y. S. (1986) · AIP conference proceedings · doi:10.1063/1.36256 · Cited by 18
Cited by
Artificial neural networks: fundamentals, computing, design, and application
Basheer, I. A., & Hajmeer, M. (2000) · Journal of Microbiological Methods · doi:10.1016/s0167-7012(00)00201-3 · Cited by 3,231
Rokach, L. (2010) · Artificial Intelligence Review · doi:10.1007/s10462-009-9124-7 · Cited by 2,657
Neuro-fuzzy modeling and control
Jang, J.-S. R., & Sun, C.-T. (1995) · Proceedings of the IEEE · doi:10.1109/5.364486 · Cited by 2,328
Godara, L. C. (1997) · Proceedings of the IEEE · doi:10.1109/5.622504 · Cited by 2,073
Causability and explainability of artificial intelligence in medicine
Holzinger, A., Langs, G., Denk, H., Zatloukal, K., & Müller, H. (2019) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1312 · Cited by 1,885
Metadata
- Authors
- Widrow, B., & Lehr, M. A.
- Venue
- Proceedings of the IEEE
- Published
- 1990
- Volume
- 78
- Issue
- 9
- Pages
- 1415-1442
- ISSN
- 0018-9219 (ISSN-L)
- OpenAlex
- W2116424792
- Cited by
- 2,242
- References
- 92
- FWCI
- 43.92
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.