The DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.
Pruning algorithms-a survey
Abstract
A rule of thumb for obtaining good generalization in systems trained by examples is that one should use the smallest system that will fit the data. Unfortunately, it usually is not obvious what size is best; a system that is too small will not be able to learn the data while one that is just big enough may learn very slowly and be very sensitive to initial conditions and learning parameters. This paper is a survey of neural network pruning algorithms. The approach taken by the methods described here is to train a network that is larger than necessary and then remove the parts that are not needed.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Valiant, L. G. (1984) · Communications of the ACM · doi:10.1145/1968.1972 · Cited by 3,279
Learnability and the Vapnik-Chervonenkis dimension
Blumer, A., Ehrenfeucht, A., Haussler, D., & Warmuth, M. K. (1989) · Journal of the ACM · doi:10.1145/76359.76371 · Cited by 1,839
What Size Net Gives Valid Generalization?
Baum, E. B., & Haussler, D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.1.151 · Cited by 1,541
A simple procedure for pruning back-propagation trained neural networks
Karnin, E. D. (1990) · IEEE Transactions on Neural Networks · doi:10.1109/72.80236 · Cited by 670
Creating artificial neural networks that generalize
Sietsma, J., & Dow, R. J. F. (1991) · Neural Networks · doi:10.1016/0893-6080(91)90033-2 · Cited by 597
Simplifying Neural Networks by Soft Weight-Sharing
Nowlan, S. J., & Hinton, G. E. (1992) · Neural Computation · doi:10.1162/neco.1992.4.4.473 · Cited by 595
Skeletonization: A Technique for Trimming the Fat from a Network via Relevance Assessment
Mozer, M. C., & Smolensky, P. (1988) · neural information processing systems · Cited by 540
Comparing Biases for Minimal Network Construction with Back-Propagation
Hanson, S. J., & Pratt, L. (1988) · neural information processing systems · Cited by 480
A general lower bound on the number of examples needed for learning
Ehrenfeucht, A., Haussler, D., Kearns, M., & Valiant, L. (1989) · Information and Computation · doi:10.1016/0890-5401(89)90002-3 · Cited by 403
Experiments on Learning by Back Propagation.
Plaut, D. C., Nowlan, S. J., & Hinton, G. E. (1986) · Cited by 323
Cited by
The graph neural network model
Scarselli, F., Gori, M., Tsoi, A. C., Hagenbuchner, M., & Monfardini, G. (2009) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2008.2005605 · Cited by 9,582
Forecasting with artificial neural networks:
Zhang, G., Eddy Patuwo, B., & Y. Hu, M. (1998) · International Journal of Forecasting · doi:10.1016/s0169-2070(97)00044-7 · Cited by 4,229
Gardner, M. W., & Dorling, S. R. (1998) · Atmospheric Environment · doi:10.1016/s1352-2310(97)00447-0 · Cited by 3,324
Maier, H. R., & Dandy, G. C. (2000) · Environmental Modelling & Software · doi:10.1016/s1364-8152(99)00007-9 · Cited by 2,377
Neural networks for short-term load forecasting: a review and evaluation
Hippert, H. S., Pedreira, C. E., & Souza, R. C. (2001) · IEEE Transactions on Power Systems · doi:10.1109/59.910780 · Cited by 2,221
Nonlinear black-box modeling in system identification: a unified overview
Sjöberg, J., Zhang, Q., Ljung, L., Benveniste, A., Delyon, B., Glorennec, P.-Y., Hjalmarsson, H., & Juditsky, A. (1995) · Automatica · doi:10.1016/0005-1098(95)00120-8 · Cited by 1,942
Neural networks for classification: a survey
Zhang, G. P. (2000) · IEEE Transactions on Systems, Man and Cybernetics, Part C (Applications and Reviews) · doi:10.1109/5326.897072 · Cited by 1,667
Prechelt, L. (1998) · Lecture Notes in Computer Science · doi:10.1007/3-540-49430-8_3 · Cited by 1,649
Artificial Neural Networks in Hydrology. I: Preliminary Concepts
ASCE Task Committee on Application of Artificial Neural Networks in Hydrology (2000) · Journal of Hydrologic Engineering · doi:10.1061/(asce)1084-0699(2000)5:2(115 · Cited by 1,476
Kasabov, N. K., & Song, Q. (2002) · IEEE Transactions on Fuzzy Systems · doi:10.1109/91.995117 · Cited by 1,264
Pruning Convolutional Neural Networks for Resource Efficient Inference
Molchanov, P. A., Tyree, S., Karras, T., Aila, T., & Kautz, J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.06440 · Cited by 1,239
Deep multimodal learning: a survey on recent advances and trends
Ramachandram, D., & Taylor, G. W. (2017) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2738401 · Cited by 1,139
Automatic early stopping using cross validation: quantifying the criteria
Prechelt, L. (1998) · Neural Networks · doi:10.1016/s0893-6080(98)00010-0 · Cited by 990
Review of surrogate modeling in water resources
Razavi, S., Tolson, B. A., & Burn, D. H. (2012) · Water Resources Research · doi:10.1029/2011wr011527 · Cited by 983
Prechelt, L. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-35289-8_5 · Cited by 951
Pruning and quantization for deep neural network acceleration: A survey
Liang, T., Glossner, J., Wang, L., Shi, S., & Zhang, X. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.07.045 · Cited by 943
Neural Networks and Deep Learning
Theodoridis, S. (2015) · Machine Learning · doi:10.1016/b978-0-12-801522-3.00018-5 · Cited by 900
A new evolutionary system for evolving artificial neural networks
Yao, X., & Liu, Y. (1997) · IEEE Transactions on Neural Networks · doi:10.1109/72.572107 · Cited by 874
BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models
Ben Zaken, E., Goldberg, Y., & Ravfogel, S. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/2022.acl-short.1 · Cited by 765
Bebis, G., & Georgiopoulos, M. (1994) · IEEE Potentials · doi:10.1109/45.329294 · Cited by 746
Designing Energy-Efficient Convolutional Neural Networks Using Energy-Aware Pruning
Yang, T.-J., Chen, Y.-H., & Sze, V. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.643 · Cited by 732
“ods evaluate whether to prune each weight independently and do not account for correlation between weights [11].”
BackgroundStructured Pruning of Deep Convolutional Neural Networks
Anwar, S., Hwang, K., & Sung, W. (2017) · ACM Journal on Emerging Technologies in Computing Systems · doi:10.1145/3005348 · Cited by 698
SNIP: Single-shot Network Pruning based on Connection Sensitivity
Lee, N., Ajanthan, T., & Torr, P. H. S. (2018) · arXiv · doi:10.48550/arxiv.1810.02340 · Cited by 685
Artificial neural networks in bankruptcy prediction: General framework and cross-validation analysis
Zhang, G., Y. Hu, M., Eddy Patuwo, B., & C. Indro, D. (1999) · European Journal of Operational Research · doi:10.1016/s0377-2217(98)00051-4 · Cited by 677
Simplifying Neural Networks by Soft Weight-Sharing
Nowlan, S. J., & Hinton, G. E. (1992) · Neural Computation · doi:10.1162/neco.1992.4.4.473 · Cited by 595
A systematic review on overfitting control in shallow and deep neural networks
Bejani, M. M., & Ghatee, M. (2021) · Artificial Intelligence Review · doi:10.1007/s10462-021-09975-1 · Cited by 573
What is the State of Neural Network Pruning?
Blalock, D., Ortiz, J. J. G., Frankle, J., & Guttag, J. (2020) · arXiv · doi:10.48550/arxiv.2003.03033 · Cited by 560
A dynamic ensemble learning algorithm for neural networks
Alam, K. M. R., Siddique, N., & Adeli, H. (2020) · Neural Computing and Applications · doi:10.1007/s00521-019-04359-7 · Cited by 536
Artificial immune systems as a novel soft computing paradigm
Castro, L. N. D., & Timmis, J. I. (2003) · Soft Computing - A Fusion of Foundations, Methodologies and Applications · doi:10.1007/s00500-002-0237-z · Cited by 497
A Survey on Deep Neural Network Pruning: Taxonomy, Comparison, Analysis, and Recommendations
Cheng, H., Zhang, M., & Shi, J. Q. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3447085 · Cited by 497
Kwok, T.-Y., & Yeung, D.-Y. (1997) · IEEE Transactions on Neural Networks · doi:10.1109/72.572102 · Cited by 493
Evolving fuzzy neural networks for supervised/unsupervised online knowledge-based learning
Kasabov, N. (2001) · IEEE Transactions on Systems, Man and Cybernetics, Part B (Cybernetics) · doi:10.1109/3477.969494 · Cited by 482
A neural network classifier based on Dempster-Shafer theory
Denoeux, T. (2000) · IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans · doi:10.1109/3468.833094 · Cited by 432
Hoefler, T., Alistarh, D., Ben-Nun, T., Dryden, N., & Peste, A. (2021) · ETH Zurich · doi:10.3929/ethz-b-000519202 · Cited by 423
Neural networks: An overview of early research, current frameworks and new challenges
Prieto, A., Prieto, B., Ortigosa, E. M., Ros, E., Pelayo, F., Ortega, J., & Rojas, I. (2016) · Neurocomputing · doi:10.1016/j.neucom.2016.06.014 · Cited by 391
Pacifici, F., Chini, M., & Emery, W. J. (2009) · Remote Sensing of Environment · doi:10.1016/j.rse.2009.02.014 · Cited by 376
Model selection in neural networks
Anders, U., & Korn, O. (1999) · Neural Networks · doi:10.1016/s0893-6080(98)00117-8 · Cited by 367
Iteratively Pruned Deep Learning Ensembles for COVID-19 Detection in Chest X-Rays
Rajaraman, S., Siegelman, J., Alderson, P. O., Folio, L. S., Folio, L. R., & Antani, S. K. (2020) · IEEE Access · doi:10.1109/access.2020.3003810 · Cited by 365
Neural networks in geophysical applications
van der Baan, M., & Jutten, C. (2000) · Geophysics · doi:10.1190/1.1444797 · Cited by 342
Learning to Prune Deep Neural Networks via Layer-wise Optimal Brain Surgeon
Dong, X., Chen, S., & Pan, S. J. (2017) · arXiv · doi:10.48550/arxiv.1705.07565 · Cited by 339
A simulation study of artificial neural networks for nonlinear time-series forecasting
Zhang, G. P., Patuwo, B. E., & Hu, M. Y. (2001) · Computers & Operations Research · doi:10.1016/s0305-0548(99)00123-9 · Cited by 336
Structural learning with forgetting
Ishikawa, M. (1996) · Neural Networks · doi:10.1016/0893-6080(96)83696-3 · Cited by 300
An iterative pruning algorithm for feedforward neural networks
Castellano, G., Fanelli, A. M., & Pelillo, M. (1997) · IEEE Transactions on Neural Networks · doi:10.1109/72.572092 · Cited by 295
Ahmadi, M. A., Ebadi, M., Shokrollahi, A., & Majidi, S. M. J. (2013) · Applied Soft Computing · doi:10.1016/j.asoc.2012.10.009 · Cited by 290
Short-term inter-urban traffic forecasts using neural networks
Dougherty, M. S., & Cobbett, M. R. (1997) · International Journal of Forecasting · doi:10.1016/s0169-2070(96)00697-8 · Cited by 279
Wavelet neural networks: A practical guide
Alexandridis, A. K., & Zapranis, A. D. (2013) · Neural Networks · doi:10.1016/j.neunet.2013.01.008 · Cited by 277
Applications of artificial neural networks in chemical engineering
Himmelblau, D. M. (2000) · Korean Journal of Chemical Engineering · doi:10.1007/bf02706848 · Cited by 273
A survey on the application of recurrent neural networks to statistical language modeling
De Mulder, W., Bethard, S., & Moens, M.-F. (2015) · Computer Speech & Language · doi:10.1016/j.csl.2014.09.005 · Cited by 272
Feng, F., Zhang, C., Ma, J., & Zhang, Q.-J. (2016) · IEEE Transactions on Microwave Theory and Techniques · doi:10.1109/tmtt.2015.2504099 · Cited by 262
Support Vector Echo-State Machine for Chaotic Time-Series Prediction
Shi, Z., & Han, M. (2007) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2006.885113 · Cited by 258
A dynamic artificial neural network model for forecasting time series events
Ghiassi, M., Saidane, H., & Zimbra, D. K. (2005) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2004.10.008 · Cited by 257
Evolutionary artificial neural networks by multi-dimensional particle swarm optimization
Kiranyaz, S., Ince, T., Yildirim, A., & Gabbouj, M. (2009) · Neural Networks · doi:10.1016/j.neunet.2009.05.013 · Cited by 257
Monthly Electric Energy Demand Forecasting Based on Trend Extraction
Gonzalez-Romera, E., Jaramillo-Moran, M. A., & Carmona-Fernandez, D. (2006) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2006.883666 · Cited by 254
A genetic algorithms based multi-objective neural net applied to noisy blast furnace data
Pettersson, F., Chakraborti, N., & Saxén, H. (2007) · Applied Soft Computing · doi:10.1016/j.asoc.2005.09.001 · Cited by 252
Feature selection with neural networks
Verikas, A., & Bacauskiene, M. (2002) · Pattern Recognition Letters · doi:10.1016/s0167-8655(02)00081-8 · Cited by 252
A new hybrid ant colony optimization algorithm for feature selection
Kabir, M. M., Shahjahan, M., & Murase, K. (2012) · Expert Systems with Applications · doi:10.1016/j.eswa.2011.09.073 · Cited by 252
Deep Learning for Mobile Multimedia: A Survey
Ota, K., Dao, M. S., Mezaris, V., & Natale, F. G. B. D. (2017) · ACM Transactions on Multimedia Computing, Communications, and Applications · doi:10.1145/3092831 · Cited by 135
Neuroinspired unsupervised learning and pruning with subquantum CBRAM arrays
Shi, Y., Nguyen, L., Oh, S., Liu, X., Koushan, F., Jameson, J. R., & Kuzum, D. (2018) · Nature Communications · doi:10.1038/s41467-018-07682-0 · Cited by 106
Training-free Transformer Architecture Search
Zhou, Q., Sheng, K., Zheng, X., Li, K., Sun, X., Tian, Y., Chen, J., & Ji, R. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01062 · Cited by 64
A Survey of Security Strategies in Federated Learning: Defending Models, Data, and Privacy
Manzoor, H. U., Shabbir, A., Chen, A., Flynn, D., & Zoha, A. (2024) · Future Internet · doi:10.3390/fi16100374 · Cited by 58
Parameter Efficient Multimodal Transformers for Video Representation Learning
Lee, S., Yu, Y., Kim, G., Breuel, T., Kautz, J., & Song, Y. (2020) · arXiv · doi:10.48550/arxiv.2012.04124 · Cited by 49
“Network pruning (Reed, 1993; Caron et al., 2020) trains a large model and then reduces its size while maintaining performance.”
BackgroundWeak and strong convergence analysis of Elman neural networks via weight decay regularization
Zhou, L., Fan, Q., Huang, X., & Liu, Y. (2023) · Optimization · doi:10.1080/02331934.2022.2057852 · Cited by 42
ToxCast EPA in Vitro to in Vivo Challenge: Insight into the Rank-I Model
Novotarskyi, S., Abdelaziz, A., Sushko, Y., Körner, R., Vogt, J., & Tetko, I. V. (2016) · Chemical Research in Toxicology · doi:10.1021/acs.chemrestox.5b00481 · Cited by 34
From Algorithm to Hardware: A Survey on Efficient and Safe Deployment of Deep Neural Networks
Geng, X., Wang, Z., Chen, C., Xu, Q., Xu, K., Jin, C., Gupta, M., Yang, X., Chen, Z., Sabry Aly, M. M., et al. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3394494 · Cited by 26
Michel, E., Zernikow, B., & Wichert, S. A. (2000) · Medical Informatics and the Internet in Medicine · doi:10.1080/14639230050058329 · Cited by 4
Metadata
- Authors
- Reed, R.
- Venue
- IEEE Transactions on Neural Networks
- Published
- 1993
- Volume
- 4
- Issue
- 5
- Pages
- 740-747
- ISSN
- 1045-9227 (ISSN-L)
- OpenAlex
- W2145085734
- Cited by
- 1,708
- References
- 26
- FWCI
- 26.17
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.