The DOI matches its Crossref record; 55 of the 55 references it cites were found in a registry or an index.
MetaPruning: Meta Learning for Automatic Neural Network Channel Pruning
Open accessAbstract
In this paper, we propose a novel meta learning approach for automatic channel pruning of very deep neural networks. We first train a PruningNet, a kind of meta network, which is able to generate weight parameters for any pruned structure given the target network. We use a simple stochastic structure sampling method for training the PruningNet. Then, we apply an evolutionary procedure to search for good-performing pruned networks. The search is highly efficient because the weights are directly generated by the trained PruningNet and we do not need any finetuning at search time. With a single PruningNet trained for the target network, we can search for various Pruned Networks under different constraints with little human participation. Compared to the state-of-the-art pruning methods, we have demonstrated superior performances on MobileNet V1/V2 and ResNet. Codes are available on https://github.com/liuzechun/MetaPruning.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Self-citationImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
MobileNetV2: Inverted Residuals and Linear Bottlenecks
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L.-C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00474 · Cited by 28,755
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., & Adam, H. (2017) · arXiv · doi:10.48550/arxiv.1704.04861 · Cited by 19,015
ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
Zhang, X., Zhou, X., Lin, M., & Sun, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00716 · Cited by 9,982
Self-citationShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
Zhang, X., Zhou, X., Lin, M., & Sun, J. (2017) · arXiv · doi:10.48550/arxiv.1707.01083 · Cited by 9,982
Self-citationShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design
Ma, N., Zhang, X., Zheng, H.-T., & Sun, J. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01264-9_8 · Cited by 7,356 · cited 2×
“We adopt the same training scheme as [39] for experiments on MobileNets and the training scheme in [19] for ResNet.”
BackgroundMethodSelf-citationSqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
Iandola, F. N., Han, S., Moskewicz, M. W., Ashraf, K., Dally, W. J., & Keutzer, K. (2016) · arXiv · doi:10.48550/arxiv.1602.07360 · Cited by 6,201
Han, S., Mao, H., & Dally, W. J. (2015) · arXiv · doi:10.48550/arxiv.1510.00149 · Cited by 4,662
XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks
Rastegari, M., Ordonez, V., Redmon, J., & Farhadi, A. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_32 · Cited by 4,408
Efficient Processing of Deep Neural Networks: A Tutorial and Survey
Sze, V., Chen, Y.-H., Yang, T.-J., & Emer, J. S. (2017) · Proceedings of the IEEE · doi:10.1109/jproc.2017.2761740 · Cited by 4,215 · cited 1×
“A comprehensive survey is provided in [47].”
Neural Architecture Search with Reinforcement Learning
Zoph, B., & Le, Q. V. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.01578 · Cited by 3,974
DARTS: Differentiable architecture search
Liu, H., Simonyan, K., & Yang, Y. (2018) · arXiv · doi:10.48550/arxiv.1806.09055 · Cited by 3,248
(1989) · Cited by 3,214
Optimization as a Model for Few-Shot Learning
Ravi, S., & Larochelle, H. (2017) · International Conference on Learning Representations · Cited by 2,766
Learning Efficient Convolutional Networks through Network Slimming
Liu, Z., Li, J., Shen, Z., Huang, G., Yan, S., & Zhang, C. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.298 · Cited by 2,761
Self-citationChannel Pruning for Accelerating Very Deep Neural Networks
He, Y., Zhang, X., & Sun, J. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.155 · Cited by 2,647 · cited 7×
“Traditional channel pruning methods trim channels based on the importance of each channel either in an iterative mode [22, 38] or by adding a data-driven sparsity [28, 35].”
BackgroundMethodInfluentialSelf-citationThiNet: A Filter Level Pruning Method for Deep Neural Network Compression
Luo, J.-H., Wu, J., & Lin, W. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.541 · Cited by 1,832
(2018) · International Conference on Machine Learning · Cited by 1,465
AMC: AutoML for Model Compression and Acceleration on Mobile Devices
He, Y., Lin, J., Liu, Z., Wang, H., Li, L.-J., & Han, S. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_48 · Cited by 1,400
Self-citationFBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search
Wu, B., Keutzer, K., Dai, X., Zhang, P., Wang, Y., Sun, F., Wu, Y., Tian, Y., Vajda, P., & Jia, Y. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01099 · Cited by 1,261
Soft Filter Pruning for Accelerating Deep Convolutional Neural Networks
He, Y., Kang, G., Dong, X., Fu, Y., & Yang, Y. (2018) · Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/309 · Cited by 1,020
Rethinking the Value of Network Pruning
Liu, Z., Sun, M., Zhou, T., Huang, G. M., & Darrell, T. J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.05270 · Cited by 872
Self-citationPredicting Parameters in Deep Learning
Denil, M., Shakibi, B., Dinh, L., Ranzato, M., & de Freitas, N. (2013) · arXiv · doi:10.48550/arxiv.1306.0543 · Cited by 853
Understanding and Simplifying One-Shot Architecture Search
Bender, G., Kindermans, P.-J., Zoph, B., Vasudevan, V., & Le, Q. V. (2018) · International Conference on Machine Learning · Cited by 848
Xie, L., & Yuille, A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.154 · Cited by 740
Structured Pruning of Deep Convolutional Neural Networks
Anwar, S., Hwang, K., & Sung, W. (2017) · ACM Journal on Emerging Technologies in Computing Systems · doi:10.1145/3005348 · Cited by 698
LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks
Zhang, D., Yang, J., Ye, D., & Hua, G. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01237-3_23 · Cited by 693
Sparse Convolutional Neural Networks
Liu, B., Wang, M., Foroosh, H., Tappen, M., & Penksy, M. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298681 · Cited by 680
Network Trimming: A Data-Driven Neuron Pruning Approach towards Efficient Deep Architectures
Hu, H., Peng, R., Tai, Y., & Tang, C. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1607.03250 · Cited by 668
Optimal Brain Surgeon and general network pruning
Hassibi, B., Stork, D. G., & Wolff, G. J. (2002) · IEEE International Conference on Neural Networks · doi:10.1109/icnn.1993.298572 · Cited by 597
NetAdapt: Platform-Aware Neural Network Adaptation for Mobile Applications
Yang, T.-J., Howard, A., Chen, B., Zhang, X., Go, A., Sandler, M., Sze, V., & Adam, H. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01249-6_18 · Cited by 572
Self-citationLiu, Z., Wu, B., Luo, W., Yang, X., Liu, W., & Cheng, K.-T. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01267-0_44 · Cited by 565
Self-citationData-Driven Sparse Structure Selection for Deep Neural Networks
Huang, Z., & Wang, N. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01270-0_19 · Cited by 549
Meta-SR: A Magnification-Arbitrary Network for Super-Resolution
Hu, X., Mu, H., Zhang, X., Wang, Z., Tan, T., & Sun, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00167 · Cited by 526
Self-citationLarge-Scale Evolution of Image Classifiers
Real, E., Moore, S., Selle, A. P., Saxena, S., Suematsu, Y. I. L., Tan, J., Le, Q. V., & Kurakin, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.01041 · Cited by 466
Predicting Parameters in Deep Learning
Shakibi, B. (2014) · University of British Columbia · doi:10.14288/1.0165555 · Cited by 462
Xie, L., & Yuille, A. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · Cited by 377
Metalearning: A survey of trends and technologies
Lemke, C., Budka, M., & Gabrys, B. (2015) · Artificial Intelligence Review · doi:10.1007/s10462-013-9406-y · Cited by 365
SMASH: One-Shot Model Architecture Search through HyperNetworks
Brock, A., Lim, T., Ritchie, J. M., & Weston, N. J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.05344 · Cited by 351
Pruning Convolutional Neural Networks for Resource Efficient Transfer Learning.Preprint
Molchanov, P. A., Tyree, S., Karras, T., Aila, T., & Kautz, J. (2016) · arXiv (Cornell University) · Cited by 303
Learning to Learn: Model Regression Networks for Easy Small Sample Learning
Wang, Y.-X., & Hebert, M. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46466-4_37 · Cited by 282
Learning to Segment Every Thing
Hu, R., Dollar, P., He, K., Darrell, T., & Girshick, R. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00445 · Cited by 281
ChamNet: Towards Efficient Network Design Through Platform-Aware Model Adaptation
Dai, X., Zhang, P., Wu, B., Yin, H., Sun, F., Wang, Y., Dukhan, M., Hu, Y., Wu, Y., Jia, Y., et al. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01166 · Cited by 279
Write a Classifier: Zero-Shot Learning Using Purely Textual Descriptions
Elhoseiny, M., Saleh, B., & Elgammal, A. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.321 · Cited by 257
SMASH: One-Shot Model Architecture Search through HyperNetworks
Brock, A., Lim, T., Ritchie, J. M., & Weston, N. J. (2018) · International Conference on Learning Representations · Cited by 217
Yu, J., Yang, L., Xu, N., Yang, J., & Huang, T. S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.08928 · Cited by 216
DPP-Net: Device-Aware Progressive Search for Pareto-Optimal Neural Architectures
Dong, J.-D., Cheng, A.-C., Juan, D.-C., Wei, W., & Sun, M. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01252-6_32 · Cited by 189
MetaAnchor: Learning to Detect Objects with Customized AnchorsPreprint
Yang, T., Zhang, X., Li, Z., Zhang, W., & Sun, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.00980 · Cited by 180
Self-citationLearning the Number of Neurons in Deep Networks
Álvarez, J. M., & Salzmann, M. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.06321 · Cited by 164
Structured Binary Neural Networks for Accurate Image Classification and Semantic Segmentation
Zhuang, B., Shen, C., Tan, M., Liu, L., & Reid, I. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00050 · Cited by 158
Explicit Loss-Error-Aware Quantization for Low-Bit Deep Neural Networks
Zhou, A., Yao, A., Wang, K., & Chen, Y. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00982 · Cited by 101
Loss-aware Weight Quantization of Deep Networks
Hou, L., & Kwok, J. T. (2018) · arXiv · doi:10.48550/arxiv.1802.08635 · Cited by 99
Constraint-Aware Deep Neural Network Compression
Chen, C., Tung, F., Vedula, N., & Mori, G. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01237-3_25 · Cited by 71
Compact Deep Convolutional Neural Networks With Coarse PruningPreprint
Anwar, S., & Sung, W. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.09639 · Cited by 56
Cited by
GhostNet: More Features From Cheap Operations
Han, K., Wang, Y., Tian, Q., Guo, J., Xu, C., & Xu, C. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00165 · Cited by 5,222
Pruning and quantization for deep neural network acceleration: A survey
Liang, T., Glossner, J., Wang, L., Shi, S., & Zhang, X. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.07.045 · Cited by 943
Self-citationMeta-Learning in Neural Networks: A Survey
Hospedales, T., Antoniou, A., Micaelli, P., & Storkey, A. (2020) · arXiv · doi:10.48550/arxiv.2004.05439 · Cited by 886
“Meta-learning is beginning to be applied to this objective as well, such as training gradient generator meta-networks that allow quantized networks to be trained [199], and weight generator meta-networks that allow quantized networks to be trained with gradient [281].”
Meta-Learning in Neural Networks: A Survey
Hospedales, T. M., Antoniou, A., Micaelli, P., & Storkey, A. J. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3079209 · Cited by 886
Searching Efficient 3D Architectures with Sparse Point-Voxel Convolution
Tang, H., Liu, Z., Zhao, S., Lin, Y., Lin, J., Wang, H., & Han, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58604-1_41 · Cited by 733
Self-citation
Metadata
- Authors
- Liu, Z., Mu, H., Zhang, X., Guo, Z., Yang, X., Cheng, K.-T., & Sun, J.
- Venue
- 2019 IEEE/CVF International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2019-10
- Pages
- 3295-3304
- OpenAlex
- W2984618279
- Cited by
- 563
- References
- 55
- FWCI
- 24.54
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.