Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Sign Up
Cited by 363

The DOI matches its Crossref record; 189 of the 189 references it cites were found in a registry or an index.

Structured Pruning for Deep Convolutional Neural Networks: A Survey

Open access
He, Yang, Xiao, Lingao·2024·IEEE Transactions on Pattern Analysis and Machine Intelligence
DOI 10.1109/tpami.2023.3334614
Open this paper in CoChat — free account

Abstract

The remarkable performance of deep Convolutional neural networks (CNNs) is generally attributed to their deeper and wider architectures, which can come with significant computational costs. Pruning neural networks has thus gained interest since it effectively lowers storage and computational costs. In contrast to weight pruning, which results in unstructured models, structured pruning provides the benefit of realistic acceleration by producing models that are friendly to hardware implementation. The special requirements of structured pruning have led to the discovery of numerous new challenges and the development of innovative solutions. This article surveys the recent progress towards structured pruning of deep CNNs. We summarize and compare the state-of-the-art structured pruning techniques with respect to filter ranking methods, regularization methods, dynamic execution, neural architecture search, the lottery ticket hypothesis, and the applications of pruning. While discussing structured pruning algorithms, we briefly introduce the unstructured pruning counterpart to emphasize their differences. Furthermore, we provide insights into potential research opportunities in the field of structured pruning. A curated list of neural network pruning papers can be found at: https://github.com/he-y/Awesome-Pruning. A dedicated website offering a more interactive comparison of structured pruning methods can be found at: https://huggingface.co/spaces/he-yang/Structured-Pruning-Survey.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/tpami.2023.3334614

Cites

189 of 189 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 189 references

Research works printed but not linked: 92

  1. Han, Deep compression: Compressing deep neural networks with pruning, trained quantization and Huffman coding
  2. Simonyan, Very deep convolutional networks for large-scale image recognition
  3. Han, Learning both weights and connections for efficient neural network, 1135
  4. Denton, Exploiting linear structure within convolutional networks for efficient evaluation, 1269
  5. Li, Pruning filters for efficient convnets
  6. 10.48550/ARXIV.1706.03762 · DOI · Paper page
  7. Hoefler, 2021, Sparsity in deep learning: Pruning and growth for efficient inference and training in neural networks, J. Mach. Learn. Res., 22, 1
  8. Yvinec, RED: Looking for redundancies for data-freestructured compression of deep neural networks, 20863
  9. Sui, Chip: Channel independence-based pruning for compact neural networks, 24604
  10. Tan, DropNet: Reducing neural network complexity via iterative pruning, 9356
  11. Ye, Good subnetworks provably exist: Pruning via greedy forward selection, 10820
  12. Zhuang, Discrimination-aware channel pruning for deep neural networks, 883
  13. Liebenwein, Provable filter pruning for efficient neural networks
  14. You, Gate decorator: Global filter pruning method for accelerating deep convolutional neural networks
  15. Zhuang, Neuron-level structured pruning using polarization regularizer, 9865
  16. Ye, Rethinking the smaller-norm-less-informative assumption in channel pruning of convolution layers
  17. Kang, Operation-aware soft channel pruning using differentiable masks, 5122
  18. Tang, SCOP: Scientific control for reliable neural network pruning, 10936
  19. 10.5555/3157096.3157329 · doi:10.5555/3157096.3157329
  20. Chen, Only train once: A one-shot neural network training and pruning framework, 19637
  21. Wang, Neural pruning via growing regularization
  22. Molchanov, Pruning convolutional neural networks for resource efficient inference
  23. Peng, Collaborative channel pruning for deep networks, 5113
  24. Wang, Eigendamage: Structured pruning in the kronecker-factored eigenbasis, 6566
  25. Liu, Group Fisher pruning for practical network compression, 7021
  26. Nonnenmacher, SOSP: Efficiently capturing global correlations by second-order structured pruning
  27. Dai, Compressing neural networks using the variational information bottleneck, 1135
  28. Louizos, Bayesian compression for deep learning, 3290
  29. Neklyudov, Structured Bayesian pruning via log-normal multiplicative noise, 6778
  30. Huang, Training structured neural networks through manifold identification and variance reduction
  31. Kusupati, Soft threshold weight reparameterization for learnable sparsity, 5544
  32. Lin, Dynamic model pruning with feedback
  33. Liu, Dynamic sparse graph for efficient deep learning
  34. Lin, Runtime neural pruning
  35. Gao, Dynamic channel pruning: Feature boosting and suppression
  36. Chen, Storage efficient and dynamic flexible runtime channel pruning via deep reinforcement learning, 14747
  37. Yu, Topology-aware network pruning using multi-stage graph embedding and reinforcement learning, 25656
  38. Dong, Network pruning via transformable architecture search, 760
  39. Liu, Rethinking the value of network pruning
  40. You, Drawing early-bird tickets: Towards more efficient training of deep networks
  41. Alizadeh, Prospect pruning: Finding trainable weights at initialization using meta-gradients
  42. Rachwan, Winning the lottery ahead of time: Efficient early network pruning
  43. Fischer, Plant’n’seek: Can you find the winning ticket?
  44. Cunha, Proving the lottery ticket hypothesis for convolutional neural networks
  45. Wang, Fast lossless neural compression with integer-only discrete flows, 22562
  46. Kim, Neuron merging: Compensating for pruned neurons, 585
  47. Meng, Pruning filter in filter, 17629
  48. Zhong, Revisit kernel pruning with lottery regulated grouped convolutions
  49. Tanaka, Pruning neural networks without any data by iteratively conserving synaptic flow, 6377
  50. Lee, A signal propagation perspective for pruning neural networks at initialization
  51. Bartoldson, The generalization-stability tradeoff in neural network pruning, 20852
  52. Orseau, Logarithmic pruning is all you need, 2925
  53. Yang, PROXSGD: Training structured neural networks under regularization and constraints
  54. Ding, Global sparse momentum SGD for pruning very deep neural networks, 6382
  55. Barsbey, Heavy tails in SGD and compressibility of overparametrized neural networks, 29364
  56. Miao, Learning pruning-friendly networks via Frank-Wolfe: One-shot, any-sparsity, and no retraining
  57. Zhang, One-shot pruning of recurrent neural networks by Jacobian spectrum evaluation
  58. Peste, AC/DC: Alternating compressed/decompressed training of deep neural networks, 8557
  59. Lee, Layer-adaptive sparsity for the magnitude-based pruning
  60. Liebenwein, Compressing neural networks: Towards determining the optimal layer-wise decomposition, 8557
  61. Liu, Frequency-domain dynamic pruning for convolutional neural networks, 1051
  62. Golkar, Continual learning via neural pruning
  63. Li, Revisiting discriminator in GAN compression: A generator-discriminator cooperative compression scheme, 28560
  64. Reed, 2022, A generalist agent, Trans. Mach. Learn. Res.
  65. Serra, Scaling up exact neural network compression by RELU stability, 27081
  66. 10.1007/978-3-031-20083-0_42 · doi:10.1007/978-3-031-20083-0_42
  67. Gui, Model compression with adversarial robustness: A unified optimization framework, 2116
  68. Chen, Linearity grafting: Relaxed neuron pruning helps certifiable robustness, 3760
  69. Madaan, Adversarial neural pruning with latent vulnerability suppression, 6575
  70. Baykal, Data-dependent coresets for compressing neural networks with applications to generalization bounds
  71. 10.1155/2019/1978154 · doi:10.1155/2019/1978154
  72. Nitanda, Stochastic proximal gradient descent with acceleration techniques, 1574
  73. Kingma, Variational dropout and the local reparameterization trick, 2575
  74. 10.1137/1015034 · doi:10.1137/1015034
  75. LeCun, Optimal brain damage, 598
  76. Hassibi, Second order derivatives for network pruning: Optimal brain surgeon, 164
  77. Martens, Optimizing neural networks with kronecker-factored approximate curvature, 2408
  78. George, Fast approximate natural gradient descent in a kronecker factored eigenbasis, 9573
  79. Xiao, Dual averaging method for regularized stochastic learning and online optimization, 2116
  80. Srinivas, Gaussian process optimization in the bandit setting: No regret and experimental design, 1015
  81. Guo, Dynamic network surgery for efficient DNNs, 1387
  82. Hua, Channel gating neural networks, 1886
  83. Kipf, Semi-supervised classification with graph convolutional networks
  84. Chen, Stochastic gradient Hamiltonian Monte Carlo, 1683
  85. Frankle, Linear mode connectivity and the lottery ticket hypothesis, 3259
  86. Lee, Snip: Single-shot network pruning based on connection sensitivity
  87. Wang, Picking winning tickets before training by preserving gradient flow
  88. Jacot, Neural tangent kernel: Convergence and generalization in neural networks, 8580
  89. Lubana, A gradient flow framework for analyzing network pruning
  90. Renda, Comparing rewinding and fine-tuning in neural network pruning
  91. Yang, Dataset pruning: Reducing training data by examining generalization influence
  92. Guo, Sparse DNNs with improved adversarial robustness, 240

Cited by

See all 363

Metadata

Authors
He, Y., & Xiao, L.
Venue
IEEE Transactions on Pattern Analysis and Machine Intelligence
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Published
2024-05
Volume
46
Issue
5
Pages
2900-2919
ISSN
  • 0162-8828 (ISSN-L)
  • 2160-9292
  • 1939-3539
DOI
10.1109/tpami.2023.3334614
OpenAlex
W4389104738
Cited by
363
References
189
FWCI
33.00
Type
journal article
Language
en
Open access
green
Licence
Funders
  • A*STAR Centre for Frontier AI Research
  • CFAR Internship Award and Research Excellence
  • A*STAR Science and Engineering Research Council
  • Maritime AI Research Programme (SMI-2022-MTP-06)
  • Singapore Maritime Institute

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Structured Pruning for Deep Convolutional Neural Networks: A Survey. https://app.cochat.ai/papers/10.1109/tpami.2023.3334614
The paper
He, Y., & Xiao, L. (2024). Structured Pruning for Deep Convolutional Neural Networks: A Survey. IEEE Transactions on Pattern Analysis and Machine Intelligence. https://doi.org/10.1109/tpami.2023.3334614

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.