The DOI matches its Crossref record; 41 of the 41 references it cites were found in a registry or an index.
An iterative pruning algorithm for feedforward neural networks
Abstract
The problem of determining the proper size of an artificial neural network is recognized to be crucial, especially for its practical implications in such important issues as learning and generalization. One popular approach for tackling this problem is commonly known as pruning and it consists of training a larger than necessary network and then removing unnecessary weights/nodes. In this paper, a new pruning method is developed, based on the idea of iteratively eliminating units and adjusting the remaining weights in such a way that the network performance does not worsen over the entire training set. The pruning problem is formulated in terms of solving a system of linear equations, and a very efficient conjugate gradient algorithm is used for solving it, in the least-squares sense. The algorithm also provides a simple criterion for choosing the units to be removed, which has proved to work well in practice. The results obtained over various test problems demonstrate the effectiveness of the proposed approach.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Golub, G. (2012) · doi:10.56021/9781421407944 · Cited by 34,505
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892
Classification and Regression Trees.
Gordon, A. D., Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1984) · Biometrics · doi:10.2307/2530946 · Cited by 23,522
Classification and Regression Trees.
Ryzin, J. V., Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1986) · Journal of the American Statistical Association · doi:10.2307/2288003 · Cited by 20,825
Learning Internal Representations by Error Propagation
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1985) · doi:10.21236/ada164453 · Cited by 16,172
Classification and regression trees
Praagman, J. (1985) · European Journal of Operational Research · doi:10.1016/0377-2217(85)90321-2 · Cited by 10,053
An introduction to computing with neural nets
Lippmann, R. (1987) · IEEE ASSP Magazine · doi:10.1109/massp.1987.1165576 · Cited by 7,825
Introduction to the theory of neural computation
(1994) · Neural Networks · doi:10.1016/0893-6080(94)90014-0 · Cited by 6,401
(1989) · Cited by 3,214
The Cascade-Correlation learning architecture
Fahlman, S. E., & Lebiere, C. J. (2018) · doi:10.1184/r1/6610403 · Cited by 2,372
Reed, R. (1993) · IEEE Transactions on Neural Networks · doi:10.1109/72.248452 · Cited by 1,708
What Size Net Gives Valid Generalization?
Baum, E. B., & Haussler, D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.1.151 · Cited by 1,541
(1993) · CaltechAUTHORS (California Institute of Technology) · Cited by 1,498
Connectionist learning procedures
Hinton, G. E. (1989) · Artificial Intelligence · doi:10.1016/0004-3702(89)90049-0 · Cited by 1,430
Progress in supervised neural networks
Hush, D. R., & Horne, B. G. (1993) · IEEE Signal Processing Magazine · doi:10.1109/79.180705 · Cited by 1,247
Generalization and network design strategies
LeCun, Y. (1989) · Cited by 885
PREDICTING THE FUTURE: A CONNECTIONIST APPROACH
Weigend, A. S., Huberman, B. A., & Rumelhart, D. E. (1990) · International Journal of Neural Systems · doi:10.1142/s0129065790000102 · Cited by 721
A simple procedure for pruning back-propagation trained neural networks
Karnin, E. D. (1990) · IEEE Transactions on Neural Networks · doi:10.1109/72.80236 · Cited by 670
Creating artificial neural networks that generalize
Sietsma, J., & Dow, R. J. F. (1991) · Neural Networks · doi:10.1016/0893-6080(91)90033-2 · Cited by 597
Hinton, G. E. (1990) · Machine Learning · doi:10.1016/b978-0-08-051055-2.50029-8 · Cited by 516
Learning in feedforward layered networks: the tiling algorithm
Mezard, M., & Nadal, J.-P. (1989) · Journal of Physics A: Mathematical and General · doi:10.1088/0305-4470/22/12/019 · Cited by 426
Dynamic node creation in backpropagation networks
ASH, T. (1989) · Connection Science · doi:10.1080/09540098908915647 · Cited by 415
Using Relevance to Reduce Network Size Automatically
MOZER, M. C., & SMOLENSKY, P. (1989) · Connection Science · doi:10.1080/09540098908915626 · Cited by 339
Bounds on the number of hidden neurons in multilayer perceptrons
Huang, S.-C., & Huang, Y.-F. (1991) · IEEE Transactions on Neural Networks · doi:10.1109/72.80290 · Cited by 328
Experiments on neural net recognition of spoken and written text
Burr, D. J. (1988) · IEEE Transactions on Acoustics, Speech, and Signal Processing · doi:10.1109/29.1643 · Cited by 263
A survey of sparse matrix research
Duff, I. S. (1977) · Proceedings of the IEEE · doi:10.1109/proc.1977.10514 · Cited by 262
Large Automatic Learning, Rule Extraction, and Generalization.
Denker, J. S., Schwartz, D., Wittner, B. S., Solla, S. A., Howard, R. E., Jackel, L. D., & Hopfield, J. J. (1987) · Complex Systems · Cited by 248
Neural net pruning-why and how
Sietsma, & Dow (1988) · IEEE International Conference on Neural Networks · doi:10.1109/icnn.1988.23864 · Cited by 232
Generalization and Parameter Estimation in Feedforward Nets: Some Experiments
Morgan, N., & Bourlard, H. (1989) · neural information processing systems · Cited by 225
Improving model selection by nonconvergent methods
Finnoff, W., Hergert, F., & Zimmermann, H. G. (1993) · Neural Networks · doi:10.1016/s0893-6080(05)80122-4 · Cited by 219
Neural net pruning-why and how.
Sietsma, J., & Dow, R. J. F. (1988) · Cited by 193
A simple method to derive bounds on the size and to train multilayer neural networks
Sartori, M. A., & Antsaklis, P. J. (1991) · IEEE Transactions on Neural Networks · doi:10.1109/72.88168 · Cited by 182
A Back-Propagation Algorithm with Optimal Use of Hidden Units
Chauvin, Y. (1988) · neural information processing systems · Cited by 168
Learning sets of filters using back-propagation
Plaut, D. C., & Hinton, G. E. (1987) · Computer Speech & Language · doi:10.1016/0885-2308(87)90026-x · Cited by 116
Speed up learning and network optimization with extended back propagation
Sperduti, A., & Starita, A. (1993) · Neural Networks · doi:10.1016/0893-6080(93)90004-g · Cited by 116
Can backpropagation error surface not have local minima
Yu, X.-H. (1992) · IEEE Transactions on Neural Networks · doi:10.1109/72.165604 · Cited by 105
Emmerson, M. D., & Damper, R. I. (1993) · IEEE Transactions on Neural Networks · doi:10.1109/72.248456 · Cited by 101
Structure Level Adaptation for Artificial Neural Networks
Lee, T.-C. (1991) · Springer eBooks · doi:10.1007/978-1-4615-3954-4 · Cited by 63
Generalization performance of overtrained back-propagation networks
Chauvin, Y. (1990) · Lecture notes in computer science · doi:10.1007/3-540-52255-7_26 · Cited by 54
Niles, L. T., Silverman, H. F., Tajchman, G. N., & Bush, M. A. (2003) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.1989.266352 · Cited by 37
A Frobenius approximation reduction method (FARM) for determining optimal number of hidden units
Kung, S., & Hu, Y. (2002) · 2022 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.1991.155331 · Cited by 29
Cited by
Statistical pattern recognition: a review
Jain, A. K., Duin, P. W., & Mao, J. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.824819 · Cited by 6,767
Maier, H. R., & Dandy, G. C. (2000) · Environmental Modelling & Software · doi:10.1016/s1364-8152(99)00007-9 · Cited by 2,377
Machine learning: a review of classification and combining techniques
Kotsiantis, S. B., Zaharakis, I. D., & Pintelas, P. E. (2006) · Artificial Intelligence Review · doi:10.1007/s10462-007-9052-3 · Cited by 1,582
Structured Pruning of Deep Convolutional Neural Networks
Anwar, S., Hwang, K., & Sung, W. (2017) · ACM Journal on Emerging Technologies in Computing Systems · doi:10.1145/3005348 · Cited by 698
Hoefler, T., Alistarh, D., Ben-Nun, T., Dryden, N., & Peste, A. (2021) · ETH Zurich · doi:10.3929/ethz-b-000519202 · Cited by 423
Metadata
- Authors
- Castellano, G., Fanelli, A. M., & Pelillo, M.
- Venue
- IEEE Transactions on Neural Networks
- Published
- 1997-05
- Volume
- 8
- Issue
- 3
- Pages
- 519-531
- ISSN
- 1045-9227 (ISSN-L)
- 1941-0093
- OpenAlex
- W2126333738
- Cited by
- 295
- References
- 41
- FWCI
- 8.56
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.