The DOI matches its Crossref record; 193 of the 193 references it cites were found in a registry or an index.
Structured Pruning for Deep Convolutional Neural Networks: A Survey
Open accessAbstract
The remarkable performance of deep Convolutional neural networks (CNNs) is generally attributed to their deeper and wider architectures, which can come with significant computational costs. Pruning neural networks has thus gained interest since it effectively lowers storage and computational costs. In contrast to weight pruning, which results in unstructured models, structured pruning provides the benefit of realistic acceleration by producing models that are friendly to hardware implementation. The special requirements of structured pruning have led to the discovery of numerous new challenges and the development of innovative solutions. This article surveys the recent progress towards structured pruning of deep CNNs. We summarize and compare the state-of-the-art structured pruning techniques with respect to filter ranking methods, regularization methods, dynamic execution, neural architecture search, the lottery ticket hypothesis, and the applications of pruning. While discussing structured pruning algorithms, we briefly introduce the unstructured pruning counterpart to emphasize their differences. Furthermore, we provide insights into potential research opportunities in the field of structured pruning. A curated list of neural network pruning papers can be found at: https://github.com/he-y/Awesome-Pruning. A dedicated website offering a more interactive comparison of structured pruning methods can be found at: https://huggingface.co/spaces/he-yang/Structured-Pruning-Survey.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Exploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
Boyd, S., & Vandenberghe, L. (2004) · doi:10.1017/cbo9780511804441 · Cited by 56,601
You Only Look Once: Unified, Real-Time Object Detection
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.91 · Cited by 53,803
Densely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Boyd, S., Parikh, N., Chu, E., Peleato, B., & Eckstein, J. (2011) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000016 · Cited by 17,075
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782
Proximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625
De-noising by soft-thresholding
Donoho, D. L. (1995) · IEEE Transactions on Information Theory · doi:10.1109/18.382009 · Cited by 9,619
Liu, Z., Mao, H., Wu, C.-Y., Feichtenhofer, C., Darrell, T., & Xie, S. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01167 · Cited by 8,494
Model Selection and Estimation in Regression with Grouped Variables
Yuan, M., & Lin, Y. (2006) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2005.00532.x · Cited by 7,261
Representation Learning with Contrastive Predictive Coding
Oord, A. V. D., Li, Y., & Vinyals, O. (2018) · arXiv · doi:10.48550/arxiv.1807.03748 · Cited by 6,242 · cited 1×
“Therefore, a similarity-based contrastive loss [256] is used for gradient-based learning without the labeled data.”
MethodAN IDEA BASED ON HONEY BEE SWARM FOR NUMERICAL OPTIMIZATION
Karaboğa, D. (2005) · Cited by 5,466
Dimensionality Reduction by Learning an Invariant Mapping
Hadsell, R., Chopra, S., & LeCun, Y. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.100 · Cited by 5,336
Horn, R. A., & Johnson, C. R. (2012) · doi:10.1017/cbo9781139020411 · Cited by 4,642
XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks
Rastegari, M., Ordonez, V., Redmon, J., & Farhadi, A. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_32 · Cited by 4,408
Efficient Processing of Deep Neural Networks: A Tutorial and Survey
Sze, V., Chen, Y.-H., Yang, T.-J., & Emer, J. S. (2017) · Proceedings of the IEEE · doi:10.1109/jproc.2017.2761740 · Cited by 4,215
Parikh, N., & Boyd, S. (2014) · Foundations and Trends® in Optimization · doi:10.1561/2400000003 · Cited by 4,102
Knowledge Distillation: A Survey
Gou, J., Yu, B., Maybank, S. J., & Tao, D. (2021) · International Journal of Computer Vision · doi:10.1007/s11263-021-01453-z · Cited by 3,931
Image segmentation using deep learning: A survey
Minaee, S., Boykov, Y. Y., Porikli, F., Plaza, A. J., Kehtarnavaz, N., & Terzopoulos, D. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3059968 · Cited by 3,796
Categorical Reparameterization with Gumbel-Softmax
Jang, E., Gu, S., & Poole, B. (2016) · arXiv · doi:10.48550/arxiv.1611.01144 · Cited by 3,616
Learning Efficient Convolutional Networks through Network Slimming
Liu, Z., Li, J., Shen, Z., Huang, G., Yan, S., & Zhang, C. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.298 · Cited by 2,761
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
Channel Pruning for Accelerating Very Deep Neural Networks
He, Y., Zhang, X., & Sun, J. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.155 · Cited by 2,647 · cited 2×
“1) Current Layer: Channel Pruning (CP) [59] uses layer l ’s (current) activation maps to guide the pruning of layer l ’s filters.”
MethodSelf-citationMeasurement Reliability and Validity
(2000) · Research Methods in Applied Settings · doi:10.4324/9781410605337-29 · Cited by 2,541
Buciluǎ, C., Caruana, R., & Niculescu-Mizil, A. (2006) · Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1150402.1150464 · Cited by 2,068
Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
Bengio, Y., Léonard, N., & Courville, A. (2013) · arXiv · doi:10.48550/arxiv.1308.3432 · Cited by 1,910
ThiNet: A Filter Level Pruning Method for Deep Neural Network Compression
Luo, J.-H., Wu, J., & Lin, W. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.541 · Cited by 1,832
Neural Architecture Search: A Survey
Elsken, T., Metzen, J. H., & Hutter, F. (2018) · arXiv · doi:10.48550/arxiv.1808.05377 · Cited by 1,763
Minimization Methods for Non-Differentiable Functions
Shor, N. Z. (1985) · Springer Series in Computational Mathematics · doi:10.1007/978-3-642-82118-9 · Cited by 1,551
The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks
Frankle, J., & Carbin, M. (2018) · arXiv · doi:10.48550/arxiv.1803.03635 · Cited by 1,529
Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation
Denton, R., Zaremba, W., Bruna, J., LeCun, Y., & Fergus, R. (2014) · arXiv · doi:10.48550/arxiv.1404.0736 · Cited by 1,414
AMC: AutoML for Model Compression and Acceleration on Mobile Devices
He, Y., Lin, J., Liu, Z., Wang, H., Li, L.-J., & Han, S. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_48 · Cited by 1,400
Self-citationPackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning
Mallya, A., & Lazebnik, S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00810 · Cited by 1,258
A Survey of Quantization Methods for Efficient Neural Network Inference
Gholami, A., Kim, S., Dong, Z., Yao, Z., Mahoney, M. W., & Keutzer, K. (2022) · Low-Power Computer Vision · doi:10.1201/9781003162810-13 · Cited by 1,219
Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Acceleration
He, Y., Liu, P., Wang, Z., Hu, Z., & Yang, Y. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00447 · Cited by 1,217
Self-citationCooperative Coevolution: An Architecture for Evolving Coadapted Subcomponents
Potter, M. A., & Jong, K. A. D. (2000) · Evolutionary Computation · doi:10.1162/106365600568086 · Cited by 1,196
Soft Filter Pruning for Accelerating Deep Convolutional Neural Networks
He, Y., Kang, G., Dong, X., Fu, Y., & Yang, Y. (2018) · Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/309 · Cited by 1,020
Self-citationModel Compression and Hardware Acceleration for Neural Networks: A Comprehensive Survey
Deng, B. L., Li, G., Han, S., Shi, L., & Xie, Y. (2020) · Proceedings of the IEEE · doi:10.1109/jproc.2020.2976475 · Cited by 996 · cited 1×
“[26], [46], [47], [48], [49], [50] X X X X X”
Importance Estimation for Neural Network Pruning
Molchanov, P., Mallya, A., Tyree, S., Frosio, I., & Kautz, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01152 · Cited by 983
HAQ: Hardware-Aware Automated Quantization With Mixed Precision
Wang, K., Liu, Z., Lin, Y., Lin, J., & Han, S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00881 · Cited by 978
Pruning and quantization for deep neural network acceleration: A survey
Liang, T., Glossner, J., Wang, L., Shi, S., & Zhang, X. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.07.045 · Cited by 943
HRank: Filter Pruning Using High-Rank Feature Map
Lin, M., Ji, R., Wang, Y., Zhang, Y., Zhang, B., Tian, Y., & Shao, L. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00160 · Cited by 866
Once-for-All: Train One Network and Specialize it for Efficient Deployment
Cai, H., Gan, C., Wang, T., Zhang, Z., & Han, S. (2019) · arXiv · doi:10.48550/arxiv.1908.09791 · Cited by 845 · cited 1×
“A once-for-all network [273] supporting a large search space for channel number is trained.”
BackgroundCheng, Y., Wang, D., Zhou, P., & Zhang, T. (2018) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2765695 · Cited by 817
Single Path One-Shot Neural Architecture Search with Uniform Sampling
Guo, Z., Zhang, X., Mu, H., Heng, W., Liu, Z., Wei, Y., & Sun, J. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58517-4_32 · Cited by 813
NISP: Pruning Networks Using Neuron Importance Score Propagation
Yu, R., Li, A., Chen, C.-F., Lai, J.-H., Morariu, V. I., Han, X., Gao, M., Lin, C.-Y., & Davis, L. S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00958 · Cited by 794
Elsken, T., Metzen, J. H., & Hutter, F. (2019) · The Springer Series on Challenges in Machine Learning · doi:10.1007/978-3-030-05318-5_3 · Cited by 785
ConViT: improving vision transformers with soft convolutional inductive biases
d’Ascoli, S., Touvron, H., Leavitt, M. L., Morcos, A. S., Biroli, G., & Sagun, L. (2022) · Journal of Statistical Mechanics: Theory and Experiment · doi:10.1088/1742-5468/ac9830 · Cited by 782
Wang, L., & Yoon, K.-J. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3055564 · Cited by 752
Federated Learning: A Survey on Enabling Technologies, Protocols, and Applications
Aledhari, M., Razzak, R., Parizi, R. M., & Saeed, F. (2020) · IEEE Access · doi:10.1109/access.2020.3013541 · Cited by 733 · cited 1×
“Pruning for Federated Learning: Federated learning [277] aims to solve the problem of training a model without transferring data to a central location.”
BackgroundDesigning Energy-Efficient Convolutional Neural Networks Using Energy-Aware Pruning
Yang, T.-J., Chen, Y.-H., & Sze, V. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.643 · Cited by 732 · cited 4×
“4 Pruning Targets Hardware [220], Energy [221], Robustness [222], [223], [224]”
BackgroundInfluentialEfficient Algorithms for Computing a Strong Rank-Revealing QR Factorization
Gu, M., & Eisenstat, S. C. (1996) · SIAM Journal on Scientific Computing · doi:10.1137/0917055 · Cited by 677
Network Trimming: A Data-Driven Neuron Pruning Approach towards Efficient Deep Architectures
Hu, H., Peng, R., Tai, Y., & Tang, C. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1607.03250 · Cited by 668
A Survey on Evolutionary Neural Architecture Search
Liu, Y., Sun, Y., Xue, B., Zhang, M., Yen, G. G., & Tan, K. C. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3100554 · Cited by 644
Panning for Gold: ‘Model-X’ Knockoffs for High Dimensional Controlled Variable Selection
Candès, E., Fan, Y., Janson, L., & Lv, J. (2018) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/rssb.12265 · Cited by 632
Efficient Deep Learning: A Survey on Making Deep Learning Models Smaller, Faster, and Better
Menghani, G. (2023) · ACM Computing Surveys · doi:10.1145/3578938 · Cited by 605
A Comprehensive Survey of Neural Architecture Search: Challenges and Solutions
Ren, P., Xiao, Y., Chang, X., Huang, P.-Y., Li, Z., Chen, X., & Wang, X. (2022) · ACM Computing Surveys · doi:10.1145/3447582 · Cited by 590
Model Pruning Enables Efficient Federated Learning on Edge Devices
Jiang, Y., Wang, S., Valls, V., Ko, B. J., Lee, W.-H., Leung, K. K., & Tassiulas, L. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3166101 · Cited by 589
Understanding intermediate layers using linear classifier probes
Alain, G., & Bengio, Y. (2016) · arXiv · doi:10.48550/arxiv.1610.01644 · Cited by 581
MetaPruning: Meta Learning for Automatic Neural Network Channel Pruning
Liu, Z., Mu, H., Zhang, X., Guo, Z., Yang, X., Cheng, K.-T., & Sun, J. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00339 · Cited by 563
What is the State of Neural Network Pruning?
Blalock, D., Ortiz, J. J. G., Frankle, J., & Guttag, J. (2020) · arXiv · doi:10.48550/arxiv.2003.03033 · Cited by 560
Towards Optimal Structured CNN Pruning via Generative Adversarial Learning
Lin, S., Ji, R., Yan, C., Zhang, B., Cao, L., Ye, Q., Huang, F., & Doermann, D. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00290 · Cited by 555
Data-Driven Sparse Structure Selection for Deep Neural Networks
Huang, Z., & Wang, N. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01270-0_19 · Cited by 549
Binary Neural Networks: A Survey
Qin, H., Gong, R., Liu, X., Bai, X., Song, J., & Sebe, N. (2020) · Pattern Recognition · doi:10.1016/j.patcog.2020.107281 · Cited by 546
Scaling for edge inference of deep neural networks
Xu, X., Ding, Y., Hu, S. X., Niemier, M., Cong, J., Hu, Y., & Shi, Y. (2018) · Nature Electronics · doi:10.1038/s41928-018-0059-3 · Cited by 462
When did Bayesian inference become "Bayesian"?
Fienberg, S. E. (2006) · Bayesian Analysis · doi:10.1214/06-ba101 · Cited by 399
Fast ConvNets Using Group-Wise Brain Damage
Lebedev, V., & Lempitsky, V. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.280 · Cited by 373
Variational Convolutional Neural Network Pruning
Zhao, C., Ni, B., Zhang, J., Zhao, Q., Zhang, W., & Tian, Q. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00289 · Cited by 361
Universally Slimmable Networks and Improved Training Techniques
Yu, J., & Huang, T. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00189 · Cited by 361
A tutorial on variational Bayesian inference
Fox, C. W., & Roberts, S. J. (2012) · Artificial Intelligence Review · doi:10.1007/s10462-011-9236-8 · Cited by 326
Yang, Y., Zhuang, Y., & Pan, Y. (2021) · Frontiers of Information Technology & Electronic Engineering · doi:10.1631/fitee.2100463 · Cited by 309
Inductive biases for deep learning of higher-level cognition
Goyal, A., & Bengio, Y. (2022) · Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rspa.2021.0068 · Cited by 300
Roffo, G., Melzi, S., & Cristani, M. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.478 · Cited by 290
ViTAEv2: Vision Transformer Advanced by Exploring Inductive Bias for Image Recognition and Beyond
Zhang, Q., Xu, Y., Zhang, J., & Tao, D. (2023) · International Journal of Computer Vision · doi:10.1007/s11263-022-01739-w · Cited by 277
Learning Filter Pruning Criteria for Deep Convolutional Neural Networks Acceleration
He, Y., Ding, Y., Liu, P., Zhu, L., Zhang, H., & Yang, Y. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00208 · Cited by 262
Self-citationChannel Pruning via Automatic Structure Search
Lin, M., Ji, R., Zhang, Y., Zhang, B., Wu, Y., & Tian, Y. (2020) · Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/94 · Cited by 256
An Entropy-based Pruning Method for CNN CompressionPreprint
Luo, J.-H., & Wu, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.05791 · Cited by 249
A fast Iterative Shrinkage-Thresholding Algorithm with application to wavelet-based image deblurring
Beck, A., & Teboulle, M. (2009) · 2009 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2009.4959678 · Cited by 236
Recent advances in efficient computation of deep convolutional neural networks
Cheng, J., Wang, P.-S., Li, G., Hu, Q.-H., & Lu, H.-Q. (2018) · Frontiers of Information Technology & Electronic Engineering · doi:10.1631/fitee.1700789 · Cited by 230
Methods for Pruning Deep Neural Networks
Vadera, S., & Ameen, S. (2022) · IEEE Access · doi:10.1109/access.2022.3182659 · Cited by 223
Capra, M., Bussolino, B., Marchisio, A., Masera, G., Martina, M., & Shafique, M. (2020) · IEEE Access · doi:10.1109/access.2020.3039858 · Cited by 222
CLIP-Q: Deep Network Compression Learning by In-parallel Pruning-Quantization
Tung, F., & Mori, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00821 · Cited by 220
Group Sparsity: The Hinge Between Filter Pruning and Decomposition for Network Compression
Li, Y., Gu, S., Mayer, C., Van Gool, L., & Timofte, R. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00804 · Cited by 208
Convolutional Neural Network Pruning with Structural Redundancy Reduction
Wang, Z., Li, C., & Wang, X. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01467 · Cited by 205
AutoCompress: An Automatic DNN Structured Pruning Framework for Ultra-High Compression Rates
Liu, N., Ma, X., Xu, Z., Wang, Y., Tang, J., & Ye, J. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.5924 · Cited by 202
Accelerating Convolutional Networks via Global & Dynamic Filter Pruning
Lin, S., Ji, R., Li, Y., Wu, Y., Huang, F., & Zhang, B. (2018) · Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/336 · Cited by 201
DMCP: Differentiable Markov Channel Pruning for Neural Networks
Guo, S., Wang, Y., Li, Q., & Yan, J. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00161 · Cited by 198
A Survey on Efficient Convolutional Neural Networks and Hardware Acceleration
Ghimire, D., Kil, D., & Kim, S.-H. (2022) · Electronics · doi:10.3390/electronics11060945 · Cited by 198
Centripetal SGD for Pruning Very Deep Convolutional Networks With Complicated Structure
Ding, X., Ding, G., Guo, Y., & Han, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00508 · Cited by 193
Ma, X., Guo, F.-M., Niu, W., Lin, X., Tang, J., Ma, K., Ren, B., & Wang, Y. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.5954 · Cited by 193
A Survey on Methods and Theories of Quantized Neural Networks
Guo, Y. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.04752 · Cited by 188
APQ: Joint Search for Network Architecture, Pruning and Quantization Policy
Wang, T., Wang, K., Cai, H., Lin, J., Liu, Z., Wang, H., Lin, Y., & Han, S. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00215 · Cited by 187
Shallowing Deep Networks: Layer-Wise Pruning Based on Feature Representations
Chen, S., & Zhao, Q. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2874634 · Cited by 186
Toward Compact ConvNets via Structure-Sparsity Regularized Filter Pruning
Lin, S., Ji, R., Li, Y., Deng, C., & Li, X. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2906563 · Cited by 180
ResRep: Lossless CNN Pruning via Decoupling Remembering and Forgetting
Ding, X., Hao, T., Tan, J., Liu, J., Han, J., Guo, Y., & Ding, G. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00447 · Cited by 177
Bayesian Bits: Unifying Quantization and PruningPreprint
van Baalen, M., Louizos, C., Nagel, M., Amjad, R. A., Wang, Y., Blankevoort, T., & Welling, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.07093 · Cited by 175
Non-Structured DNN Weight Pruning—Is It Beneficial in Any Platform?
Ma, X., Lin, S., Ye, S., He, Z., Zhang, L., Yuan, G., Tan, S. H., Li, Z., Fan, D., Qian, X., et al. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3063265 · Cited by 167
Revisiting Random Channel Pruning for Neural Network Compression
Li, Y., Adamczewski, K., Li, W., Gu, S., Timofte, R., & Gool, L. V. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00029 · Cited by 166
Towards Efficient Model Compression via Learned Global Ranking
Chin, T.-W., Ding, R., Zhang, C., & Marculescu, D. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00159 · Cited by 165
DHP: Differentiable Meta Pruning via HyperNetworks
Li, Y., Gu, S., Zhang, K., Gool, L. V., & Timofte, R. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58598-3_36 · Cited by 165
EagleEye: Fast Sub-net Evaluation for Efficient Neural Network Pruning
Li, B., Wu, B., Su, J., & Wang, G. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58536-5_38 · Cited by 163
Evaluation and Selection of Biases in Machine Learning
Gordon, D. F., & desJardins, M. (1995) · Machine Learning · doi:10.1007/bf00993472 · Cited by 154
Few Sample Knowledge Distillation for Efficient Network Compression
Li, T., Li, J., Liu, Z., & Zhang, C. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01465 · Cited by 149
A Survey of Methods for Low-Power Deep Learning and Computer Vision
Goel, A., Tung, C., Lu, Y.-H., & Thiruvathukal, G. K. (2020) · IEEE World Forum on Internet of Things (WF-IoT) · doi:10.1109/wf-iot48130.2020.9221198 · Cited by 146
DearKD: Data-Efficient Early Knowledge Distillation for Vision Transformers
Chen, X., Cao, Q., Zhong, Y., Zhang, J., Gao, S., & Tao, D. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01174 · Cited by 142
DSA: More Efficient Budgeted Pruning via Differentiable Sparsity Allocation
Ning, X., Zhao, T., Li, W., Lei, P., Wang, Y., & Yang, H. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58580-8_35 · Cited by 133
Towards Compact CNNs via Collaborative Compression
Li, Y., Lin, S., Liu, J., Ye, Q., Wang, M., Chao, F., Yang, F., Ma, J., Tian, Q. C., & Ji, R. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00637 · Cited by 130
Zhao, T., Xie, Y., Wang, Y., Cheng, J., Guo, X., Hu, B., & Chen, Y. J. (2022) · Proceedings of the IEEE · doi:10.1109/jproc.2022.3153408 · Cited by 128
A Survey on Evolutionary Construction of Deep Neural Networks
Zhou, X., Qin, A. K., Gong, M., & Tan, K. C. (2021) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2021.3079985 · Cited by 127
Dynamic Model Pruning with Feedback
Lin, T., Stich, S. U., Barba, L., Dmitriev, D., & Jaggi, M. (2020) · arXiv · doi:10.48550/arxiv.2006.07253 · Cited by 126
Manifold Regularized Dynamic Network Pruning
Tang, Y., Wang, Y., Xu, Y., Deng, Y., Xu, C., Tao, D., & Xu, C. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00498 · Cited by 122
Differentiable Joint Pruning and Quantization for Hardware Efficiency
Wang, Y., Lu, Y., & Blankevoort, T. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58526-6_16 · Cited by 116
Discrete Model Compression With Resource Constraint for Deep Neural Networks
Gao, S., Huang, F., Pei, J., & Huang, H. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00197 · Cited by 115
Dynamical Channel Pruning by Conditional Accuracy Change for Deep Neural Networks
Chen, Z., Xu, T.-B., Du, C., Liu, C., & He, H. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2979517 · Cited by 108
Pruning Networks With Cross-Layer Ranking & k-Reciprocal Nearest Filters
Lin, M., Cao, L., Zhang, Y., Shao, L., Lin, C., & Ji, R. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3156047 · Cited by 106
Carrying Out CNN Channel Pruning in a White Box
Zhang, Y., Lin, M., Lin, C., Chen, J., Wu, Y., Tian, Y., & Ji, R. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3147269 · Cited by 99
Approximated Oracle Filter Pruning for Destructive CNN Width Optimization
Ding, X., Ding, G., Guo, Y., Han, J., & Yan, C. (2019) · arXiv · doi:10.48550/arxiv.1905.04748 · Cited by 96
Achieving on-Mobile Real-Time Super-Resolution with Neural Architecture and Pruning Search
Zhan, Z., Gong, Y., Zhao, P., Yuan, G., Niu, W., Wu, Y., Zhang, T., Jayaweera, M., Kaeli, D. R., Ren, B., et al. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00478 · Cited by 95
Co-Evolutionary Compression for Unpaired Image Translation
Shu, H., Wang, Y., Jia, X., Han, K., Chen, H., Xu, C., Tian, Q., & Xu, C. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00333 · Cited by 93
CHEX: CHannel EXploration for CNN Model Compression
Hou, Z., Qin, M., Sun, F., Ma, X., Yuan, K., Xu, Y. T., Chen, Y.-K., Jin, R., Xie, Y., & Kung, S. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01197 · Cited by 93
Structured Pruning of Neural Networks With Budget-Aware Regularization
Lemaire, C., Achkar, A., & Jodoin, P.-M. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00932 · Cited by 90
When to Prune? A Policy towards Early Structural Pruning
Shen, M.-Y., Molchanov, P., Yin, H., & Álvarez, J. (2021) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr52688.2022.01193 · Cited by 88
Progressive Local Filter Pruning for Image Retrieval Acceleration
Wang, X., Zheng, Z., He, Y., Yan, F., Zeng, Z., & Yang, Y. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3256092 · Cited by 86
Self-citationHessian-Aware Pruning and Optimal Neural Implant
Yu, S., Yao, Z., Gholami, A., Dong, Z., Kim, S., Mahoney, M. W., & Keutzer, K. (2022) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv51458.2022.00372 · Cited by 86
COP: Customized Deep Model Compression via Regularized Correlation-Based Filter-Level Pruning
Wang, W., Fu, C., Guo, J., Cai, D., & He, X. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/525 · Cited by 85
Filter Pruning by Switching to Neighboring CNNs With Good Attributes
He, Y., Liu, P., Zhu, L., & Yang, Y. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3149332 · Cited by 84
Self-citationPatel, N. P., Sarraf, É., & Tsai, M. H. (2018) · Anesthesiology · doi:10.1097/aln.0000000000002350 · Cited by 83
Network Pruning Using Adaptive Exemplar Filters
Lin, M., Ji, R., Li, S., Wang, Y., Wu, Y., Huang, F., & Ye, Q. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3084856 · Cited by 81
1xN Pattern for Pruning Convolutional Neural Networks
Lin, M., Zhang, Y., Li, Y., Chen, B., Chao, F., Wang, M., Li, S., Tian, Y., & Ji, R. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3195774 · Cited by 81
Model Compression Based on Differentiable Network Channel Pruning
Zheng, Y.-J., Chen, S.-B., Ding, C. H. Q., & Luo, B. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3165123 · Cited by 80
DECORE: Deep Compression with Reinforcement Learning
Alwani, M., Wang, Y., & Madhavan, V. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01203 · Cited by 80
Learning Optimized Structure of Neural Networks by Hidden Node Pruning With $L_{1}$ Regularization
Xie, X., Zhang, H., Wang, J., Chang, Q., Wang, J., & Pal, N. R. (2019) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2019.2950105 · Cited by 77
OICSR: Out-In-Channel Sparsity Regularization for Compact Deep Neural Networks
Li, J., Qi, Q., Wang, J., Ge, C., Li, Y., Yue, Z., & Sun, H. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00721 · Cited by 76
Accelerate CNN via Recursive Bayesian Pruning
Zhou, Y., Zhang, Y., Wang, Y., & Tian, Q. C. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00340 · Cited by 75
Convolutional Neural Network Pruning: A Survey
Xu, S. H., Huang, A., Chen, L., & Zhang, B. (2020) · Chinese Control Conference (CCC) · doi:10.23919/ccc50068.2020.9189610 · Cited by 74
Fire Together Wire Together: A Dynamic Pruning Approach with Self-Supervised Mask Prediction
Elkerdawy, S., Elhoushi, M., Zhang, H., & Ray, N. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01213 · Cited by 74
Coreset-Based Neural Network Compression
Dubey, A., Chatterjee, M., & Ahuja, N. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01234-2_28 · Cited by 73
Automatic Searching and Pruning of Deep Neural Networks for Medical Imaging Diagnostic
Fernandes, F. E., & Yen, G. G. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3027308 · Cited by 73
Liu, Y., Shu, Z., Li, Y., Lin, Z., Perazzi, F., & Kung, S. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01198 · Cited by 73
Li, Y., Zhao, P., Yuan, G., Lin, X., Wang, Y., & Chen, X. (2022) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/449 · Cited by 73
Vision Transformer Slimming: Multi-Dimension Searching in Continuous Optimization Space
Chavan, A., Shen, Z., Liu, Z., Liu, Z., Cheng, K.-T., & Xing, E. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00488 · Cited by 72
EDP: An Efficient Decomposition and Pruning Scheme for Convolutional Neural Network Compression
Ruan, X., Liu, Y., Yuan, C., Li, B., Hu, W., Li, Y., & Maybank, S. J. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3018177 · Cited by 72
Dynamic Dual Gating Neural Networks
Li, F., Li, G., He, X., & Cheng, J. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00528 · Cited by 72
Where to Prune: Using LSTM to Guide Data-Dependent Soft Pruning
Ding, G., Zhang, S., Jia, Z., Zhong, J., & Han, J. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.3035028 · Cited by 71
EDropout: Energy-Based Dropout and Pruning of Deep Neural Networks
Salehinejad, H., & Valaee, S. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3069970 · Cited by 67
FedDUAP: Federated Learning with Dynamic Update and Adaptive Pruning Using Shared Data on the Server
Zhang, H., Liu, J., Jia, J., Yang, Z., Dai, H., & Dou, D. (2022) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/385 · Cited by 67
Auto Graph Encoder-Decoder for Neural Network Pruning
Yu, S., Mazaheri, A., & Jannesari, A. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00630 · Cited by 65
GDP: Stabilized Neural Network Pruning via Gates with Differentiable Polarization
Guo, Y., Yuan, H., Tan, J., Wang, Z., Yang, S., & Liu, J. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00519 · Cited by 61
StructADMM: Achieving Ultrahigh Efficiency in Structured Pruning for DNNs
Zhang, T., Ye, S., Feng, X., Ma, X., Zhang, K., Li, Z., Tang, J., Liu, S., Lin, X., Liu, Y., et al. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3045153 · Cited by 59
Co-advise: Cross Inductive Bias Distillation
Ren, S., Gao, Z., Hua, T., Xue, Z., Tian, Y., He, S., & Zhao, H. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01627 · Cited by 59
Peng, J., Tang, B., Jiang, H., Li, Z., Lei, Y., Lin, T., & Li, H. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3056201 · Cited by 54
DAIS: Automatic Channel Pruning via Differentiable Annealing Indicator Search
Guan, Y., Liu, N., Zhao, P., Che, Z., Bian, K., Wang, Y., & Tang, J. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3161284 · Cited by 53
Li, Z., Yuan, G., Niu, W., Zhao, P., Li, Y., Cai, Y., Shen, X., Zhan, Z., Kong, Z., Jin, Q., et al. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01403 · Cited by 53
Wang, J., Chang, Q., Chang, Q., Liu, Y., & Pal, N. R. (2018) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2018.2864142 · Cited by 51
Soft Person Reidentification Network Pruning via Blockwise Adjacent Filter Decaying
Wang, X., Zheng, Z., He, Y., Yan, F., Zeng, Z., & Yang, Y. (2021) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2021.3130047 · Cited by 47
Self-citationCheng, P., Li, Y., Shang, R., & Jiao, L. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3192169 · Cited by 47
Learning Bayesian Sparse Networks with Full Experience Replay for Continual Learning
Yan, Q., Gong, D., Liu, Y., van den Hengel, A., & Shi, Q. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00021 · Cited by 47
Exploring Lottery Ticket Hypothesis in Spiking Neural Networks
Kim, Y., Li, Y., Park, H., Venkatesha, Y., Yin, R., & Panda, P. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19775-8_7 · Cited by 45
FairGRAPE: Fairness-Aware GRAdient Pruning mEthod for Face Attribute Classification
Lin, X., Kim, S., & Joo, J. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19778-9_24 · Cited by 43
Dimensionality reduced training by pruning and freezing parts of a deep neural network: a survey
Wimmer, P., Mehnert, J., & Condurache, A. P. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-023-10489-1 · Cited by 43
On the Channel Pruning using Graph Convolution Network for Convolutional Neural Network Acceleration
Jiang, D., Cao, Y., & Yang, Q. (2022) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/431 · Cited by 41
Speeding-up convolutional neural networks: A survey
Lebedev, V., & Lempitsky, V. (2018) · Bulletin of the Polish Academy of Sciences Technical Sciences · doi:10.24425/bpas.2018.125927 · Cited by 38
RED++ : Data-Free Pruning of Deep Neural Networks via Input Splitting and Output Merging
Yvinec, E., Dapogny, A., Cord, M., & Bailly, K. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3179616 · Cited by 38
Channel Pruning via Lookahead Search Guided Reinforcement Learning
Wang, Z., & Li, C. (2022) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv51458.2022.00357 · Cited by 38
Exploration and Estimation for Model Compression
Zhang, Y., Gao, S., & Huang, H. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00054 · Cited by 35
Filter Pruning via Learned Representation Median in the Frequency Domain
Zhang, X., Xie, W., Li, Y., Lei, J., & Du, Q. (2021) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2021.3124284 · Cited by 35
SOKS: Automatic Searching of the Optimal Kernel Shapes for Stripe-Wise Network Pruning
Liu, G., Zhang, K., & Lv, M. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3162067 · Cited by 35
Prune and Tune Ensembles: Low-Cost Ensemble Learning with Sparse Independent Subnetworks
Whitaker, T., & Whitley, D. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i8.20842 · Cited by 35
SP-GAN: Self-Growing and Pruning Generative Adversarial Networks
Song, X., Chen, Y., Feng, Z., Hu, G., Yu, D., & Wu, X. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3005574 · Cited by 34
Joint Multi-Dimension Pruning via Numerical Gradient Update
Liu, Z., Zhang, X., Shen, Z., Wei, Y., Cheng, K.-T. T., & Sun, J. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3112041 · Cited by 33
Neural Network Pruning by Cooperative Coevolution
Shang, H., Wu, J., Hong, W., & Qian, C. (2022) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/667 · Cited by 33
Subarchitecture Ensemble Pruning in Neural Architecture Search
Bian, Y., Song, Q., Du, M., Yao, J., Chen, H., & Hu, X. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3085299 · Cited by 32
Bootstrapping ViTs: Towards Liberating Vision Transformers from Pre-training
Zhang, H., Duan, J., Xue, M., Song, J., Li, S., & Song, M. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00874 · Cited by 30
On Implicit Filter Level Sparsity in Convolutional Neural Networks
Mehta, D., Kim, K. I., & Theobalt, C. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00061 · Cited by 28
Reborn Filters: Pruning Convolutional Neural Networks with Limited Data
Tang, Y., You, S., Xu, C., Han, J., Qian, C., Shi, B., Xu, C., & Zhang, C. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.6058 · Cited by 27
Soft Masking for Cost-Constrained Channel Pruning
Humble, R., Shen, M., Latorre, J. A., Darve, E., & Álvarez, J. M. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20083-0_38 · Cited by 26
Filter Pruning via Feature Discrimination in Deep Neural Networks
He, Z., Qian, Y., Wang, Y., Wang, B., Guan, X., Gu, Z., Ling, X., Zeng, S., Wang, H., & Zhou, W. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19803-8_15 · Cited by 26
Interpretations Steered Network Pruning via Amortized Inferred Saliency Maps
Ganjdanesh, A., Gao, S., & Huang, H. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19803-8_17 · Cited by 26
Gonzalez-Carabarin, L., Huijben, I. A. M., Veeling, B., Schmid, A., & van Sloun, R. J. G. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3176809 · Cited by 25
Tian, G., Sun, Y., Liu, Y., Zeng, X., Wang, M., Liu, Y., Zhang, J., & Chen, J. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3106917 · Cited by 16
A Survey for Sparse Regularization Based Compression Methods
Tang, A., Quan, P., Niu, L., & Shi, Y. (2022) · Annals of Data Science · doi:10.1007/s40745-022-00389-6 · Cited by 16
Disentangled Differentiable Network Pruning
Gao, S., Huang, F., Zhang, Y., & Huang, H. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20083-0_20 · Cited by 16
You, H., Li, B., Sun, Z., Ouyang, X., & Lin, Y. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20083-0_40 · Cited by 16
A Survey on Filter Pruning Techniques for Optimization of Deep Neural Networks
Kulkarni, U., Hallad, S. S., Patil, A., Bhujannavar, T., Kulkarni, S., & Meena, S. M. (2022) · International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC) · doi:10.1109/i-smac55078.2022.9987264 · Cited by 12
CPrune: Compiler-Informed Model Pruning for Efficient Target-Aware DNN Execution
Kim, T., Kwon, Y., Lee, J., Kim, T., & Ha, S. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20044-1_37 · Cited by 11
Ensemble Knowledge Guided Sub-network Search and Fine-Tuning for Filter Pruning
Lee, S., & Song, B. C. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20083-0_34 · Cited by 10
Contrastive Dual Gating: Learning Sparse Features With Contrastive Learning
Meng, J., Yang, L., Shin, J., Fan, D., & Seo, J.-S. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01194 · Cited by 6
Multi-granularity Pruning for Model Acceleration on Mobile Devices
Zhao, T., Zhang, X. S., Zhu, W., Wang, J., Yang, S., Liu, J., & Cheng, J. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20083-0_29 · Cited by 5
Bayesian Optimization with Clustering and Rollback for CNN Auto Pruning
Fan, H., Mu, J., & Zhang, W. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20050-2_29 · Cited by 5
Research works printed but not linked: 89
- Han, Deep compression: Compressing deep neural networks with pruning, trained quantization and Huffman coding
- Simonyan, Very deep convolutional networks for large-scale image recognition
- Han, Learning both weights and connections for efficient neural network, 1135
- Li, Pruning filters for efficient convnets
- 10.48550/ARXIV.1706.03762 · DOI · Paper page
- Hoefler, 2021, Sparsity in deep learning: Pruning and growth for efficient inference and training in neural networks, J. Mach. Learn. Res., 22, 1
- Yvinec, RED: Looking for redundancies for data-freestructured compression of deep neural networks, 20863
- Sui, Chip: Channel independence-based pruning for compact neural networks, 24604
- Tan, DropNet: Reducing neural network complexity via iterative pruning, 9356
- Ye, Good subnetworks provably exist: Pruning via greedy forward selection, 10820
- Zhuang, Discrimination-aware channel pruning for deep neural networks, 883
- Liebenwein, Provable filter pruning for efficient neural networks
- You, Gate decorator: Global filter pruning method for accelerating deep convolutional neural networks
- Zhuang, Neuron-level structured pruning using polarization regularizer, 9865
- Ye, Rethinking the smaller-norm-less-informative assumption in channel pruning of convolution layers
- Kang, Operation-aware soft channel pruning using differentiable masks, 5122
- Tang, SCOP: Scientific control for reliable neural network pruning, 10936
- 10.5555/3157096.3157329 · doi:10.5555/3157096.3157329
- Chen, Only train once: A one-shot neural network training and pruning framework, 19637
- Wang, Neural pruning via growing regularization
- Molchanov, Pruning convolutional neural networks for resource efficient inference
- Peng, Collaborative channel pruning for deep networks, 5113
- Wang, Eigendamage: Structured pruning in the kronecker-factored eigenbasis, 6566
- Liu, Group Fisher pruning for practical network compression, 7021
- Nonnenmacher, SOSP: Efficiently capturing global correlations by second-order structured pruning
- Dai, Compressing neural networks using the variational information bottleneck, 1135
- Louizos, Bayesian compression for deep learning, 3290
- Neklyudov, Structured Bayesian pruning via log-normal multiplicative noise, 6778
- Huang, Training structured neural networks through manifold identification and variance reduction
- Kusupati, Soft threshold weight reparameterization for learnable sparsity, 5544
- Liu, Dynamic sparse graph for efficient deep learning
- Lin, Runtime neural pruning
- Gao, Dynamic channel pruning: Feature boosting and suppression
- Chen, Storage efficient and dynamic flexible runtime channel pruning via deep reinforcement learning, 14747
- Yu, Topology-aware network pruning using multi-stage graph embedding and reinforcement learning, 25656
- Dong, Network pruning via transformable architecture search, 760
- Liu, Rethinking the value of network pruning
- You, Drawing early-bird tickets: Towards more efficient training of deep networks
- Alizadeh, Prospect pruning: Finding trainable weights at initialization using meta-gradients
- Rachwan, Winning the lottery ahead of time: Efficient early network pruning
- Fischer, Plant’n’seek: Can you find the winning ticket?
- Cunha, Proving the lottery ticket hypothesis for convolutional neural networks
- Wang, Fast lossless neural compression with integer-only discrete flows, 22562
- Kim, Neuron merging: Compensating for pruned neurons, 585
- Meng, Pruning filter in filter, 17629
- Zhong, Revisit kernel pruning with lottery regulated grouped convolutions
- Tanaka, Pruning neural networks without any data by iteratively conserving synaptic flow, 6377
- Lee, A signal propagation perspective for pruning neural networks at initialization
- Bartoldson, The generalization-stability tradeoff in neural network pruning, 20852
- Orseau, Logarithmic pruning is all you need, 2925
- Yang, PROXSGD: Training structured neural networks under regularization and constraints
- Ding, Global sparse momentum SGD for pruning very deep neural networks, 6382
- Barsbey, Heavy tails in SGD and compressibility of overparametrized neural networks, 29364
- Miao, Learning pruning-friendly networks via Frank-Wolfe: One-shot, any-sparsity, and no retraining
- Zhang, One-shot pruning of recurrent neural networks by Jacobian spectrum evaluation
- Peste, AC/DC: Alternating compressed/decompressed training of deep neural networks, 8557
- Lee, Layer-adaptive sparsity for the magnitude-based pruning
- Liebenwein, Compressing neural networks: Towards determining the optimal layer-wise decomposition, 8557
- Liu, Frequency-domain dynamic pruning for convolutional neural networks, 1051
- Golkar, Continual learning via neural pruning
- Li, Revisiting discriminator in GAN compression: A generator-discriminator cooperative compression scheme, 28560
- Reed, 2022, A generalist agent, Trans. Mach. Learn. Res.
- Serra, Scaling up exact neural network compression by RELU stability, 27081
- 10.1007/978-3-031-20083-0_42 · doi:10.1007/978-3-031-20083-0_42
- Chen, Linearity grafting: Relaxed neuron pruning helps certifiable robustness, 3760
- Madaan, Adversarial neural pruning with latent vulnerability suppression, 6575
- Baykal, Data-dependent coresets for compressing neural networks with applications to generalization bounds
- 10.1155/2019/1978154 · doi:10.1155/2019/1978154
- Nitanda, Stochastic proximal gradient descent with acceleration techniques, 1574
- Kingma, Variational dropout and the local reparameterization trick, 2575
- 10.1137/1015034 · doi:10.1137/1015034
- LeCun, Optimal brain damage, 598
- Hassibi, Second order derivatives for network pruning: Optimal brain surgeon, 164
- Martens, Optimizing neural networks with kronecker-factored approximate curvature, 2408
- George, Fast approximate natural gradient descent in a kronecker factored eigenbasis, 9573
- Xiao, Dual averaging method for regularized stochastic learning and online optimization, 2116
- Srinivas, Gaussian process optimization in the bandit setting: No regret and experimental design, 1015
- Guo, Dynamic network surgery for efficient DNNs, 1387
- Hua, Channel gating neural networks, 1886
- Kipf, Semi-supervised classification with graph convolutional networks
- Chen, Stochastic gradient Hamiltonian Monte Carlo, 1683
- Frankle, Linear mode connectivity and the lottery ticket hypothesis, 3259
- Lee, Snip: Single-shot network pruning based on connection sensitivity
- Wang, Picking winning tickets before training by preserving gradient flow
- Jacot, Neural tangent kernel: Convergence and generalization in neural networks, 8580
- Lubana, A gradient flow framework for analyzing network pruning
- Renda, Comparing rewinding and fine-tuning in neural network pruning
- Yang, Dataset pruning: Reducing training data by examining generalization influence
- Guo, Sparse DNNs with improved adversarial robustness, 240
Cited by
A Survey on Deep Neural Network Pruning: Taxonomy, Comparison, Analysis, and Recommendations
Cheng, H., Zhang, M., & Shi, J. Q. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3447085 · Cited by 497
Raza, S. M., Abidi, S. M. H., Masuduzzaman, M., & Shin, S. Y. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.131357 · Cited by 13
Nguyen-Le, H.-H., Tran, V.-T., Nguyen, D.-T., & Le-Khac, N.-A. (2026) · Artificial Intelligence Review · doi:10.1007/s10462-026-11608-4 · Cited by 28
Metadata
- Authors
- He, Y., & Xiao, L.
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2024-05
- Volume
- 46
- Issue
- 5
- Pages
- 2900-2919
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- 1939-3539
- OpenAlex
- W4389104738
- Cited by
- 363
- References
- 193
- FWCI
- 33.00
- Type
- journal article
- Language
- en
- Open access
- green
- Licence
- Funders
- A*STAR Centre for Frontier AI Research
- CFAR Internship Award and Research Excellence
- A*STAR Science and Engineering Research Council
- Maritime AI Research Programme (SMI-2022-MTP-06)
- Singapore Maritime Institute
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.