The DOI matches its Crossref record; 63 of the 64 references it cites were found in a registry or an index.
Learning Filter Pruning Criteria for Deep Convolutional Neural Networks Acceleration
Open accessAbstract
Filter pruning has been widely applied to neural network compression and acceleration. Existing methods usually utilize pre-defined pruning criteria, such as Lp-norm, to prune unimportant filters. There are two major limitations to these methods. First, existing methods fail to consider the variety of filter distribution across layers. To extract features of the coarse level to the fine level, the filters of different layers have various distributions. Therefore, it is not suitable to utilize the same pruning criteria to different functional layers. Second, prevailing layer-by-layer pruning methods process each layer independently and sequentially, failing to consider that all the layers in the network collaboratively make the final prediction. In this paper, we propose Learning Filter Pruning Criteria (LFPC) to solve the above problems. Specifically, we develop a differentiable pruning criteria sampler. This sampler is learnable and optimized by the validation loss of the pruned network obtained from the sampled criteria. In this way, we could adaptively select the appropriate pruning criteria for different functional layers. Besides, when evaluating the sampled criteria, LFPC comprehensively consider the contribution of all the layers at the same time. Experiments validate our approach on three image classification benchmarks. Notably, on ILSVRC-2012, our LFPC reduces more than 60% FLOPs on ResNet-50 with only 0.83% top-5 accuracy loss.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Pruning is the well-studied topic in model compression and many methods have been proposed to advance the network pruning [103].”
From Bringing ai to edge: from deep learning’s perspective (2022) · cites this paper as [103]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,889
(2017) · Cited by 11,625
Identity mappings in deep residual networks
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_38 · Cited by 10,741
Intriguing properties of neural networks
Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., & Fergus, R. (2013) · arXiv · doi:10.48550/arxiv.1312.6199 · Cited by 8,143
Zagoruyko, S., & Komodakis, N. (2016) · Procedings of the British Machine Vision Conference 2016 · doi:10.5244/c.30.87 · Cited by 7,244
Neural Architecture Search with Reinforcement LearningPreprint
Zoph, B., & Le, Q. V. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.01578 · Cited by 5,705
SGDR: Stochastic Gradient Descent with Warm Restarts
Loshchilov, I., & Hutter, F. (2016) · arXiv · doi:10.48550/arxiv.1608.03983 · Cited by 5,309
Han, S., Mao, H., & Dally, W. J. (2015) · arXiv · doi:10.48550/arxiv.1510.00149 · Cited by 4,662
AutoAugment: Learning Augmentation Strategies From Data
Cubuk, E. D., Zoph, B., Mané, D., Vasudevan, V., & Le, Q. V. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00020 · Cited by 4,445
Categorical Reparameterization with Gumbel-Softmax
Jang, E., Gu, S., & Poole, B. (2016) · arXiv · doi:10.48550/arxiv.1611.01144 · Cited by 3,616
DARTS: Differentiable architecture search
Liu, H., Simonyan, K., & Yang, Y. (2018) · arXiv · doi:10.48550/arxiv.1806.09055 · Cited by 3,248
Self-citationPruning Filters for Efficient ConvNets
Li, H., Kadav, A., Durdanovic, I., Samet, H., & Graf, H. P. (2016) · arXiv · doi:10.48550/arxiv.1608.08710 · Cited by 2,928
Learning Efficient Convolutional Networks through Network Slimming
Liu, Z., Li, J., Shen, Z., Huang, G., Yan, S., & Zhang, C. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.298 · Cited by 2,761
Channel Pruning for Accelerating Very Deep Neural Networks
He, Y., Zhang, X., & Sun, J. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.155 · Cited by 2,647
Self-citationThe Concrete Distribution: A Continuous Relaxation of Discrete Random VariablesPreprint
Maddison, C. J., Mnih, A., & Teh, Y. W. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.00712 · Cited by 2,170
ThiNet: A Filter Level Pruning Method for Deep Neural Network Compression
Luo, J.-H., Wu, J., & Lin, W. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.541 · Cited by 1,934
Understanding Neural Networks Through Deep VisualizationPreprint
Yosinski, J., Clune, J., Nguyen, A. M., Fuchs, T. J., & Lipson, H. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.06579 · Cited by 1,918
The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks
Frankle, J., & Carbin, M. (2018) · arXiv · doi:10.48550/arxiv.1803.03635 · Cited by 1,529
Rethinking the Value of Network PruningPreprint
Liu, Z., Sun, M., Zhou, T., Huang, G. M., & Darrell, T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.05270 · Cited by 1,448
AMC: AutoML for Model Compression and Acceleration on Mobile Devices
He, Y., Lin, J., Liu, Z., Wang, H., Li, L.-J., & Han, S. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_48 · Cited by 1,400
Self-citationFilter Pruning via Geometric Median for Deep Convolutional Neural Networks Acceleration
He, Y., Liu, P., Wang, Z., Hu, Z., & Yang, Y. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00447 · Cited by 1,280
Self-citationSoft Filter Pruning for Accelerating Deep Convolutional Neural Networks
He, Y., Kang, G., Dong, X., Fu, Y., & Yang, Y. (2018) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/309 · Cited by 1,179
Self-citationImportance Estimation for Neural Network Pruning
Molchanov, P., Mallya, A., Tyree, S., Frosio, I., & Kautz, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01152 · Cited by 983
Dynamic Network Surgery for Efficient DNNsPreprint
Guo, Y., Yao, A., & Chen, Y. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1608.04493 · Cited by 856
NISP: Pruning Networks Using Neuron Importance Score Propagation
Yu, R., Li, A., Chen, C.-F., Lai, J.-H., Morariu, V. I., Han, X., Gao, M., Lin, C., & Davis, L. S. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00958 · Cited by 854
Unbiased Scene Graph Generation From Biased Training
Tang, K., Niu, Y., Huang, J., Shi, J., & Zhang, H. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00377 · Cited by 810
Self-citationThe Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks.Preprint
Frankle, J., & Carbin, M. (2019) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 701
Searching for a Robust Neural Architecture in Four GPU Hours
Dong, X., & Yang, Y. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00186 · Cited by 675
Self-citationMetaPruning: Meta Learning for Automatic Neural Network Channel Pruning
Liu, Z., Mu, H., Zhang, X., Guo, Z., Yang, X. Y., Cheng, K.-T. T., & Sun, J. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00339 · Cited by 599
Towards Optimal Structured CNN Pruning via Generative Adversarial Learning
Lin, S., Ji, R., Yan, C., Zhang, B., Cao, L., Ye, Q., Huang, F., & Doermann, D. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00290 · Cited by 592
Data-Driven Sparse Structure Selection for Deep Neural Networks
Huang, Z., & Wang, N. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01270-0_19 · Cited by 584
Discrimination-aware Channel Pruning for Deep Neural NetworksPreprint
Zhuang, Z., Tan, M., Zhuang, B., Liu, J., Guo, Y. F., Wu, Q., Huang, J., & Zhu, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.11809 · Cited by 541
A Systematic DNN Weight Pruning Framework Using Alternating Direction Method of Multipliers
Zhang, T., Ye, S., Zhang, K., Tang, J., Wen, W., Fardad, M., & Wang, Y. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01237-3_12 · Cited by 492
Learning to Prune Deep Neural Networks via Layer-wise Optimal Brain Surgeon
Dong, X., Chen, S., & Pan, S. J. (2017) · arXiv · doi:10.48550/arxiv.1705.07565 · Cited by 339
Rethinking the Smaller-Norm-Less-Informative Assumption in Channel Pruning of Convolution Layers
Ye, J., Lu, X., Lin, Z., & Wang, J. Z. (2018) · arXiv · doi:10.48550/arxiv.1802.00124 · Cited by 328
More is Less: A More Complicated Network with Less Inference Complexity
Dong, X., Huang, J., Yang, Y., & Yan, S. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.205 · Cited by 305
Self-citationPruning Convolutional Neural Networks for Resource Efficient Transfer Learning.Preprint
Molchanov, P. A., Tyree, S., Karras, T., Aila, T., & Kautz, J. (2016) · arXiv (Cornell University) · Cited by 303
Benjamin, A. T., & Quinn, J. J. (2003) · American Mathematical Society eBooks · doi:10.5948/9781614442080 · Cited by 238
Proofs that really count: the art of combinatorial proof
(2004) · Choice Reviews Online · doi:10.5860/choice.41-4086 · Cited by 226
CLIP-Q: Deep Network Compression Learning by In-parallel Pruning-Quantization
Tung, F., & Mori, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00821 · Cited by 220
Gate Decorator: Global Filter Pruning Method for Accelerating Deep Convolutional Neural NetworksPreprint
You, Z., Yan, K., Ye, J., Ma, M., & Wang, P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.08174 · Cited by 218
Network Pruning via Transformable Architecture SearchPreprint
Dong, X., & Yang, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.09717 · Cited by 213
Self-citationCentripetal SGD for Pruning Very Deep Convolutional Networks With Complicated Structure
Ding, X., Ding, G., Guo, Y., & Han, J. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00508 · Cited by 202
Asymptotic Soft Filter Pruning for Deep Convolutional Neural Networks
He, Y., Dong, X., Kang, G., Fu, Y., Yan, C. C., & Yang, Y. (2019) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2019.2933477 · Cited by 201
Self-citationAccelerating Convolutional Networks via Global & Dynamic Filter Pruning
Lin, S., Ji, R., Li, Y., Wu, Y., Huang, F., & Zhang, B. (2018) · Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/336 · Cited by 201
Learning to Prune Filters in Convolutional Neural Networks
Huang, Q., Zhou, K., You, S. O., & Neumann, U. (2018) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2018.00083 · Cited by 191
Holistic CNN Compression via Low-Rank Decomposition with Knowledge Transfer
Lin, S., Ji, R., Chen, C., Tao, D., & Luo, J. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2873305 · Cited by 164
Collaborative Channel Pruning for Deep Networks
Peng, H., Wu, J., Chen, S., & Huang, J. (2019) · International Conference on Machine Learning · Cited by 117
Approximated Oracle Filter Pruning for Destructive CNN Width Optimization
Ding, X., Ding, G., Guo, Y., Han, J., & Yan, C. (2019) · arXiv · doi:10.48550/arxiv.1905.04748 · Cited by 96
Progressive Local Filter Pruning for Image Retrieval Acceleration
Wang, X., Zheng, Z., He, Y., Yan, F., Zeng, Z., & Yang, Y. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3256092 · Cited by 81
Self-citationCoreset-Based Neural Network Compression
Dubey, A., Chatterjee, M., & Ahuja, N. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01234-2_28 · Cited by 72
Exploring Linear Relationship in Feature Map Subspace for ConvNets Compression
Wang, D., Zhou, L., Zhang, X., Bai, X., & Zhou, J. (2018) · arXiv · doi:10.48550/arxiv.1803.05729 · Cited by 42
Leveraging Filter Correlations for Deep Model Compression
Singh, P. P., Verma, V. K., Rai, P., & Namboodiri, V. P. (2020) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv45572.2020.9093331 · Cited by 32
Principal Filter Analysis for Guided Network Compression.Preprint
Suau, X., Zappella, L., Palakkode, V., & Apostoloff, N. (2018) · arXiv (Cornell University) · Cited by 23
Pruning Filter via Geometric Median for Deep Convolutional Neural Networks Acceleration.Preprint
He, Y., Liu, P., Wang, Z., & Yang, Y. (2018) · arXiv (Cornell University) · Cited by 22
Self-citationMeta Filter Pruning to Accelerate Deep Convolutional Neural Networks.Preprint
He, Y., Liu, P., Zhu, L., & Yang, Y. (2019) · arXiv (Cornell University) · Cited by 8
Self-citationCoreset-Based Neural Network CompressionPreprint
Dubey, A., Chatterjee, M., & Ahuja, N. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.09810
Filter Pruning by Switching to Neighboring CNNs with Good Attributes
He, Y., Liu, P., Zhu, L., & Yang, Y. (2019) · arXiv · doi:10.48550/arxiv.1904.03961
Self-citationLeveraging Filter Correlations for Deep Model Compression
Singh, P., Verma, V. K., Rai, P., & Namboodiri, V. P. (2018) · arXiv · doi:10.48550/arxiv.1811.10559
Filter Distillation for Network Compression
Suau, X., Zappella, L., & Apostoloff, N. (2018) · arXiv · doi:10.48550/arxiv.1807.10585
Research works printed but not linked: 2
- carreira-perpinán, 0, learning-compression” algorithms for neural net pruning, CVPR
- han, 0, Learning both weights and connections for efficient neural network, NeurIPS
Cited by
Sorted by citations
Unbiased Scene Graph Generation From Biased Training
Tang, K., Niu, Y., Huang, J., Shi, J., & Zhang, H. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00377 · Cited by 810
Self-citationStructured Pruning for Deep Convolutional Neural Networks: A Survey
He, Y., & Xiao, L. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3334614 · Cited by 468
Self-citationLightweight Deep Learning for Resource-Constrained Environments: A Survey
Liu, H.-I., Galindo, M., Xie, H., Wong, L.-K., Shuai, H.-H., Li, Y., & Cheng, W.-H. (2024) · ACM Computing Surveys · doi:10.1145/3657282 · Cited by 332
Quantization and Deployment of Deep Neural Networks on Microcontrollers
Novac, P.-E., Hacene, G. B., Pégatoquet, A., Miramond, B., & Gripon, V. (2021) · Sensors · doi:10.3390/s21092984 · Cited by 250
ResRep: Lossless CNN Pruning via Decoupling Remembering and Forgetting
Ding, X., Hao, T., Tan, J., Liu, J., Han, J., Guo, Y., & Ding, G. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00447 · Cited by 203
Metadata
- Authors
- He, Y., Ding, Y., Liu, P., Zhu, L., Zhang, H., & Yang, Y.
- Affiliations
- University of Technology Sydney (AU)
- Baidu (China) (CN)
- Nanyang Technological University (SG)
- Venue
- 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2020-06
- Pages
- 2006-2015
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W3035467254
- Cited by
- 262
- References
- 64
- Citations by year
- 2026: 9 · 2025: 35 · 2024: 43 · 2023: 62 · 2022: 55 · 2021: 49 · 2020: 9
- FWCI
- 13.50
- Type
- conference paper
- Language
- en
- Open access
- green
- hdl.handle.net
- Licence
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- OPUS - Open Publications of UTS Scholars (University of Technology Sydney) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- filter pruning, neural network compression, deep convolutional neural networks, layer-wise pruning, model acceleration, FLOPs reduction, image classification, ResNet50
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
