The DOI matches its DataCite record; 25 of the 29 references it cites were found in a registry or an index.
Learning Sparse Neural Networks through $L_0$ Regularization
Open accessAbstract
We propose a practical method for $L_0$ norm regularization for neural networks: pruning the network during training by encouraging weights to become exactly zero. Such regularization is interesting since (1) it can greatly speed up training and inference, and (2) it can improve generalization. AIC and BIC, well-known model selection criteria, are special cases of $L_0$ regularization. However, since the $L_0$ norm of weights is non-differentiable, we cannot incorporate it directly as a regularization term in the objective function. We propose a solution through the inclusion of a collection of non-negative stochastic gates, which collectively determine which weights to set to zero. We show that, somewhat surprisingly, for certain distributions over the gates, the expected $L_0$ norm of the resulting gated weights is differentiable with respect to the distribution parameters. We further propose the \emph{hard concrete} distribution for the gates, which is obtained by "stretching" a binary concrete distribution and then transforming its samples with a hard-sigmoid. The parameters of the distribution over the gates can then be jointly optimized with the original network parameters. As a result our method allows for straightforward and efficient learning of model structures with stochastic gradient descent and allows for conditional computation in a principled way. We perform various experiments to demonstrate the effectiveness of the resulting approach and regularizer.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Many methods based on the sparse learning (SL) framework have also been proven to be effective [43-47]”
From A survey of model compression strategies for object detection (2023) · cites this paper as [44]
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Self-citationGradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Regression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482
Estimating the Dimension of a Model
Schwarz, G. (1978) · The Annals of Statistics · doi:10.1214/aos/1176344136 · Cited by 47,033
Auto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067
Self-citationInformation Theory and an Extension of the Maximum Likelihood Principle
Akaike, H. (1998) · Springer series in statistics · doi:10.1007/978-1-4612-1694-0_15 · Cited by 18,636
Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
Williams, R. J. (1992) · Machine Learning · doi:10.1007/bf00992696 · Cited by 8,608
Zagoruyko, S., & Komodakis, N. (2016) · Procedings of the British Machine Vision Conference 2016 · doi:10.5244/c.30.87 · Cited by 7,244
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1512.03385 · Cited by 4,196
Stochastic Backpropagation and Approximate Inference in Deep Generative ModelsPreprint
Rezende, D. J., Mohamed, S., & Wierstra, D. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1401.4082 · Cited by 3,921
Cited by
Sorted by citations
Items 201–300 of 628
Ding, L., Kang, G., Liu, J., Xiao, Y., & Cao, B. (2021) · IEEE International Conference on Web Services (ICWS) · doi:10.1109/icws53863.2021.00076 · Cited by 17
Modelling of physical systems with a Hopf bifurcation using mechanistic models and machine learning
Lee, K. H., Barton, D. A. W., & Renson, L. (2023) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2023.110173 · Cited by 17
Structure-Aware Low-Rank Adaptation for Parameter-Efficient Fine-Tuning
Hu, Y., Xie, Y., Wang, T., Chen, M., & Pan, Z. (2023) · Mathematics · doi:10.3390/math11204317 · Cited by 21
L0 regularization-based compressed sensing with quantum–classical hybrid approach
Aonishi, T., Mimura, K., Okada, M., & Yamamoto, Y. (2022) · Quantum Science and Technology · doi:10.1088/2058-9565/ac6f19 · Cited by 21
Alignment Rationale for Natural Language Inference
Jiang, Z., Zhang, Y., Zhao, Y., Zhao, J., & Liu, K. (2021) · doi:10.18653/v1/2021.acl-long.417 · Cited by 17
Adaptively Sparse TransformersPreprint
Correia, G. M., Niculae, V., & Martins, A. F. T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.00015 · Cited by 95
MEMO: A Deep Network for Flexible Combination of Episodic MemoriesPreprint
Banino, A., Badia, A. P., Koster, R., Chadwick, M. J., Zambaldi, V., Hassabis, D., Barry, C., Botvinick, M., Kumaran, D., & Blundell, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.10913 · Cited by 31
Keep the Gradients Flowing: Using Gradient Flow to Study Sparse Network OptimizationPreprint
Tessera, K.-A., Hooker, S., & Rosman, B. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.01670 · Cited by 17
Effective Model Sparsification by Scheduled Grow-and-Prune MethodsPreprint
Ma, X., Qin, M., Sun, F., Hou, Z., Yuan, K., Xu, Y. T., Wang, Y., Chen, Y.-K., Jin, R., & Xie, Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.09857 · Cited by 25
Automatic Mixed-Precision Quantization Search of BERT
Zhao, C., Hua, T., Shen, Y., Lou, Q., & Jin, H. (2021) · Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/472 · Cited by 16
Pruning-aware Sparse Regularization for Network Pruning
Jiang, N.-F., Zhao, X. F., Zhao, C., An, Y.-Q., Tang, M., & Wang, J.-Q. (2023) · Machine Intelligence Research · doi:10.1007/s11633-022-1353-0 · Cited by 16
Learning broad learning system with controllable sparsity through L0 regularization
Chu, F., Wang, G., Wang, J., Chen, C. L. P., & Wang, X. (2023) · Applied Soft Computing · doi:10.1016/j.asoc.2023.110068 · Cited by 16
On the compression of neural networks using ℓ 0 -norm regularization and weight pruning
Oliveira, F. D. D. R., Batista, E. L. O., & Seara, R. (2023) · Neural Networks · doi:10.1016/j.neunet.2023.12.019 · Cited by 17
Learning Sparse & Ternary Neural Networks with Entropy-Constrained Trained Ternarization (EC2T)
Marbán, A., Becking, D., Wiedemann, S., & Samek, W. (2020) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00369 · Cited by 16
Dynamic hard pruning of Neural Networks at the edge of the internet
Valerio, L., Nardini, F. M., Passarella, A., & Perego, R. (2022) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2021.103330 · Cited by 17
Understanding Multi-Head Attention in Abstractive SummarizationPreprint
Baan, J., ter Hoeve, M., van der Wees, M., Schuth, A., & de Rijke, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.03898 · Cited by 26
A Winning Hand: Compressing Deep Networks Can Improve Out-Of-Distribution RobustnessPreprint
Diffenderfer, J., Bartoldson, B. R., Chaganti, S., Zhang, J., & Kailkhura, B. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.09129 · Cited by 20
Federated Learning using Smart Contracts on Blockchains, based on Reward Driven ApproachPreprint
Behera, M. R., Upadhyay, S., & Shetty, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.10243 · Cited by 16
$$L_0$$-ARM: Network Sparsification via Stochastic Binary Optimization
Li, Y., & Ji, S. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-46147-8_26 · Cited by 15
ADFQ-ViT: Activation-Distribution-Friendly post-training Quantization for Vision Transformers
Jiang, Y., Sun, N., Xie, X., Yang, F., & Li, T. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.107289 · Cited by 20
Real-Time Vehicle Detection from Short-range Aerial Image with Compressed MobileNet
He, Y., Pan, Z., Li, L., Shan, Y., Cao, D., & Chen, L. F. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8793673 · Cited by 16
ReSprop: Reuse Sparsified Backpropagation
Goli, N., & Aamodt, T. M. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00162 · Cited by 18
A generalization of regularized dual averaging and its dynamicsPreprint
Chao, S.-K., & Cheng, G. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.10072 · Cited by 15
Full deep neural network training on a pruned weight budgetPreprint
Golub, M., Lemieux, G. G. F., & Lis, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.06949 · Cited by 23
Taxonomy and Evaluation of Structured Compression of Convolutional Neural NetworksPreprint
Kuzmin, A., Nagel, M., Pitre, S., Pendyam, S., Blankevoort, T., & Welling, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.09802 · Cited by 15
Self-citationStructure Optimization of Neural Networks with L1 Regularization on Gates
Chang, Q., Wang, J., Zhang, H., Shi, L., Wang, J., & Pal, N. R. (2018) · IEEE Symposium Series on Computational Intelligence (SSCI) · doi:10.1109/ssci.2018.8628632 · Cited by 14
SymbolNet: neural symbolic regression with adaptive dynamic pruning for compression
Tsoi, H. F., Lončar, V., Dasu, S., & Harris, P. C. (2025) · Machine Learning Science and Technology · doi:10.1088/2632-2153/adaad8 · Cited by 22
Polyconvex physics-augmented neural network constitutive models in principal stretches
Tepole, A. B., Jadoon, A. A., Rausch, M. K., & Fuhg, J. N. (2025) · International Journal of Solids and Structures · doi:10.1016/j.ijsolstr.2025.113469 · Cited by 14
Bayesian neural networks with variable selection for prediction of genotypic values
van Bergen, G. H. H., Duenk, P., Albers, C. A., Bijma, P., Calus, M. P. L., Wientjes, Y. C. J., & Kappen, H. J. (2020) · Genetics Selection Evolution · doi:10.1186/s12711-020-00544-8 · Cited by 14
Scalable multiple changepoint detection for functional data sequences
Harris, T. A. L. V., Li, B., & Tucker, J. D. (2021) · Environmetrics · doi:10.1002/env.2710 · Cited by 14
Huang, J., Jiao, Y., Kang, L., Liu, J., Liu, Y., & Lu, X. (2021) · Computational Statistics · doi:10.1007/s00180-021-01098-z · Cited by 15
Layer-Wise Data-Free CNN Compression
Horton, M., Jin, Y., Farhadi, A., & Rastegari, M. (2022) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr56361.2022.9956237 · Cited by 17
Differentiable Unsupervised Feature Selection based on a Gated LaplacianPreprint
Lindenbaum, O., Shaham, U., Svirsky, J., Peterfreund, E., & Kluger, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.04728 · Cited by 16
Good Students Play Big Lottery Better.Preprint
Ma, H., Chen, T., Hu, T.-K., You, C., Xie, X. S., & Wang, Z. (2021) · arXiv (Cornell University) · Cited by 14
Optimal regularizations for data generation with probabilistic graphical models
Fanthomme, A., Rizzato, F., Cocco, S., & Monasson, R. (2022) · Journal of Statistical Mechanics Theory and Experiment · doi:10.1088/1742-5468/ac650c · Cited by 13
Neural Image Compression and Explanation
Li, X., & Ji, S. (2020) · IEEE Access · doi:10.1109/access.2020.3041416 · Cited by 17
One-class anomaly detection via novelty normalization
Wu, J.-C., Lu, S., Fuh, C., & Liu, T.-L. (2021) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2021.103226 · Cited by 15
A High-Performance Accelerator for Super-Resolution Processing on Embedded GPU
Zhao, W., Bai, Y., Sun, Q. L., Li, W., Zheng, H., Jiang, N., Lu, J., Yu, B., & Wong, M. D. F. (2023) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2023.3241110 · Cited by 13
TreeConnect: A Sparse Alternative to Fully Connected Layers
Richter, O., & Wattenhofer, R. P. (2018) · IEEE International Conference on Tools with Artificial Intelligence (ICTAI) · doi:10.1109/ictai.2018.00143 · Cited by 12
Structural Analysis of Sparse Neural Networks
Stier, J., & Granitzer, M. (2019) · Procedia Computer Science · doi:10.1016/j.procs.2019.09.165 · Cited by 14
Pruning Artificial Neural Networks: A Way to Find Well-Generalizing, High-Entropy Sharp Minima
Tartaglione, E., Bragagnolo, A., & Grangetto, M. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-61616-8_6 · Cited by 13
Sparse Representations for Object- and Ego-Motion Estimations in Dynamic Scenes
Kashyap, H. J., Fowlkes, C. C., & Krichmar, J. L. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3006467 · Cited by 13
Functionality-Oriented Convolutional Filter PruningPreprint
Qin, Z., Yu, F., Liu, C., & Chen, X. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.07322 · Cited by 16
Lipschitz Bounds and Provably Robust Training by Laplacian SmoothingPreprint
Krishnan, V., Makdah, A. A. A., & Pasqualetti, F. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.03712 · Cited by 13
Fast Graph Attention Networks Using Effective Resistance Based Graph SparsificationPreprint
Srinivasa, R. S., Xiao, C., Glass, L. M., Romberg, J., & Sun, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.08796 · Cited by 13
Finding trainable sparse networks through Neural Tangent TransferPreprint
Liu, T., & Zenke, F. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.08228 · Cited by 15
Layer-wise Pruning of Transformer Attention Heads for Efficient Language Modeling
Shim, K., Choi, I., Sung, W., & Choi, J. (2021) · International SoC Design Conference · doi:10.1109/isocc53507.2021.9613933 · Cited by 19
FLOPs-efficient filter pruning via transfer scale for neural network acceleration
Guo, Z., Xiao, Y., Liao, W., Veelaert, P., & Philips, W. R. (2021) · Journal of Computational Science · doi:10.1016/j.jocs.2021.101459 · Cited by 12
Towards explaining graph neural networks via preserving prediction ranking and structural dependency
Zhang, Y., Cheung, W. K., Liu, Q., Wang, G., Yang, L., & Liu, L. (2023) · Information Processing & Management · doi:10.1016/j.ipm.2023.103571 · Cited by 12
Torque based Structured Pruning for Deep Neural Network
Gupta, A., Bau, T. C., Kim, J., Zhu, Z., Jha, S., & Garud, H. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00269 · Cited by 12
Progressive Kernel Pruning Based on the Information Mapping Sparse Index for CNN Compression
Zhu, J., Zhao, Y., & Pei, J. (2021) · IEEE Access · doi:10.1109/access.2021.3051504 · Cited by 13
Layer-wise Model Pruning based on Mutual Information
Fan, C. C., Li, J., Zhang, T., Ao, X., Wu, F. Q., Meng, Y., & Sun, X. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.246 · Cited by 20
On Network Science and Mutual Information for Explaining Deep Neural Networks
Davis, B., Bhatt, U., Bhardwaj, K., Mărculescu, R., & Moura, J. M. F. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9053078 · Cited by 14
Efficient Inference For Neural Machine Translation
Hsu, Y.-T., Garg, S., Liao, Y.-H., & Chatsviorkin, I. (2020) · Workshop on Simple and Efficient Natural Language Processing (SustaiNLP) · doi:10.18653/v1/2020.sustainlp-1.7 · Cited by 12
Filter Pruning using Hierarchical Group Sparse Regularization for Deep Convolutional Neural Networks
Mitsuno, K., & Kurita, T. (2021) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr48806.2021.9413113 · Cited by 12
Compressing Convolutional Neural Networks by Pruning Density Peak Filters
Jang, Y., Lee, S., & Kim, J. (2021) · IEEE Access · doi:10.1109/access.2021.3049470 · Cited by 12
Deeply Tensor Compressed Transformers for End-to-End Object Detection
Zhen, P., Gao, Z., Hou, T., Cheng, Y. G., & Chen, H. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i4.20397 · Cited by 12
TimeGate: Conditional Gating of Segments in Long-range ActivitiesPreprint
Hussein, N., Jain, M., & Bejnordi, B. E. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.01808 · Cited by 12
Connectivity Matters: Neural Network Pruning Through the Lens of Effective SparsityPreprint
Vysogorets, A., & Kempe, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.02306 · Cited by 15
Bertsimas, D., Carballo, K. V., Boussioux, L., Li, M. L., Paskov, A., & Paskov, I. (2023) · Machine Learning · doi:10.1007/s10994-023-06482-y · Cited by 11
A Lightweight Model for 3D Point Cloud Object Detection
Li, Z., Li, Y., Wang, Y., Xie, G., Qu, H., & Lyu, Z. (2023) · Applied Sciences · doi:10.3390/app13116754 · Cited by 11
Huang, Z., Li, M., Shen, L., Yu, J., Gong, C., Han, B., & Liu, T. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-024-02075-x · Cited by 18
Multiaxial damage parameter evaluation by neural network-based symbolic regression
Cao, W., Sun, X., Li, Y., & Chen, X. (2025) · Engineering Fracture Mechanics · doi:10.1016/j.engfracmech.2025.110809 · Cited by 11
INVITED: Computation on Sparse Neural Networks and its Implications for Future Hardware
Sun, F., Qin, M., Zhang, T., Liu, L., Chen, Y.-K., & Xie, Y. (2020) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1109/dac18072.2020.9218645 · Cited by 11
Deep Learning for Visual Analytics of the Spread of COVID-19 Infection in Crowded Urban Environments
Pi, Y., Nath, N. D., Sampathkumar, S., & Behzadan, A. H. (2021) · Natural Hazards Review · doi:10.1061/(asce)nh.1527-6996.0000492 · Cited by 11
Generation of Istanbul road data set using Google Map API for deep learning-based segmentation
Öztürk, O., Ișık, M. S., Sarıtürk, B., & Şeker, D. Z. (2022) · International Journal of Remote Sensing · doi:10.1080/01431161.2022.2068989 · Cited by 11
Feature flow regularization: Improving structured sparsity in deep neural networks
Wu, Y., Lan, Y., Zhang, L., & Xiang, Y. (2023) · Neural Networks · doi:10.1016/j.neunet.2023.02.013 · Cited by 11
SpArX: Sparse Argumentative Explanations for Neural Networks
Ayoobi, H., Potyka, N., & Toni, F. (2023) · Frontiers in artificial intelligence and applications · doi:10.3233/faia230265 · Cited by 11
Convergence Rates of Variational Inference in Sparse Deep LearningPreprint
Chérief-Abdellatif, B.-E. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.04847 · Cited by 15
A Deep Neural Network Pruning Method Based on Gradient L1-norm
Liu, X., Xia, W., & Fan, Z. (2020) · IEEE International Conference on Computer and Communications (ICCC) · doi:10.1109/iccc51575.2020.9345039 · Cited by 10
A Survey of Sparse-Learning Methods for Deep Neural Networks
Ma, R., & Niu, L. (2018) · IEEE/WIC/ACM International Conference on Web Intelligence · doi:10.1109/wi.2018.00-20 · Cited by 12
Online Weight Pruning Via Adaptive Sparsity Loss
Retsinas, G., Elafrou, A., Goumas, G., & Maragos, P. A. (2021) · Proceedings - International Conference on Image Processing · doi:10.1109/icip42928.2021.9506301 · Cited by 10
Graph Structure Learning on User Mobility Data for Social Relationship Inference
Qin, G., Song, L., Yu, Y., Huang, C., Jia, W., Cao, Y. F., & Dong, J. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i4.25580 · Cited by 10
Modular and On-demand Bias Mitigation with Attribute-Removal Subnetworks
Hauzenberger, L., Masoudian, S., Kumar, D., Schedl, M., & Rekabsaz, N. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.386 · Cited by 16
Prox-SGD: Training Structured Neural Networks under Regularization and Constraints
Yang, Y., Yuan, Y., Chatzimichailidis, A., van Sloun, R. J. G., Lei, L., & Chatzinotas, S. (2020) · Open Repository and Bibliography (University of Luxembourg) · Cited by 10
Rank and run-time aware compression of NLP Applications
Thakker, U., Beu, J. G., Gope, D., Dasika, G. S., & Mattina, M. (2020) · Workshop on Simple and Efficient Natural Language Processing (SustaiNLP) · doi:10.18653/v1/2020.sustainlp-1.2 · Cited by 10
Sparse Training Theory for Scalable and Efficient Agents
Mocanu, D. C., Mocanu, E., Pinto, T. D. O., Curci, S., Nguyen, P. H., Gibescu, M., Ernst, D., & Vale, Z. (2021) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/jkww1249 · Cited by 16
Cluster-Promoting Quantization with Bit-Drop for Minimizing Network Quantization Loss
Lee, J. H., Yun, J., Hwang, S. J., & Yang, E. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00532 · Cited by 12
CSQ: Growing Mixed-Precision Quantization Scheme with Bi-level Continuous Sparsification
Xiao, L., Yang, H., Dong, Z., Keutzer, K., Du, L., & Zhang, S. (2023) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1109/dac56929.2023.10247982 · Cited by 10
A Latent Morphology Model for Open-Vocabulary Neural Machine TranslationPreprint
Ataman, D., Aziz, W. F., & Birch, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.13890 · Cited by 10
Rethinking Class-Discrimination Based CNN Channel PruningPreprint
Liu, Y., Wentzlaff, D., & Kung, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.14492 · Cited by 11
Fast Neural Models for Symbolic Regression at ScalePreprint
Costa, A. D. S., Dangovski, R., Dugan, O., Kim, S., Goyal, P., Marin Soljacic, A., & Jacobson, J. M. (2020) · arXiv (Cornell University) · Cited by 11
Evaluation of Chinese Natural Language Processing System Based on Metamorphic Testing
Jin, L., Ding, Z., & Zhou, H. (2022) · Mathematics · doi:10.3390/math10081276 · Cited by 9
AstroNet: When Astrocyte Meets Artificial Neural Network
Han, M., Pan, L., & Liu, X. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01940 · Cited by 9
Padmanabha, G. A., Fuhg, J. N., Safta, C., Jones, R. E., & Bouklas, N. (2024) · Computer Methods in Applied Mechanics and Engineering · doi:10.1016/j.cma.2024.117359 · Cited by 9
Adaptive Multiscale Slimming Network Learning for Remote Sensing Image Feature Extraction
Ye, D., Peng, J., Guo, W., & Li, H. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3490666 · Cited by 9
Dinh, T., & Xin, J. X. (2020) · Advances in intelligent systems and computing · doi:10.1007/978-3-030-55180-3_27 · Cited by 13
Ps and Qs: Quantization-Aware Pruning for Efficient Low Latency Neural Network Inference.
Hawks, B., Duarte, J. M., Fraser, N. J., Pappalardo, A., Tran, N. V., & Umuroglu, Y. (2021) · eScholarship (California Digital Library) · Cited by 9
RAPQ: Rescuing Accuracy for Power-of-Two Low-bit Post-training Quantization
Yao, H., Li, P., Cao, J., Liu, X., Xie, C., & Wang, B. (2022) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/219 · Cited by 9
Sparseout: Controlling Sparsity in Deep Networks
Khan, N. U., & Stavness, I. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-18305-9_24 · Cited by 9
Joint Regularization on Activations and Weights for Efficient Neural Network Pruning
Yang, Q. X., Wen, W., Wang, Z., & Li, H. H. (2019) · International Conference on Machine Learning and Applications · doi:10.1109/icmla.2019.00139 · Cited by 9
Neural network pruning and hardware acceleration
Jeong, T., Ghasemi, E., Tuyls, J., Delaye, E., & Sirasao, A. (2020) · IEEE/ACM International Conference on Utility and Cloud Computing (UCC) · doi:10.1109/ucc48980.2020.00069 · Cited by 9
Training Adversarially Robust Sparse Networks via Bayesian Connectivity Sampling
Özdenizci, O., & Legenstein, R. A. (2021) · International Conference on Machine Learning · Cited by 9
HEMP: High-order entropy minimization for neural network compression
Tartaglione, E., Lathuilière, S., Fiandrotti, A., Cagnazzo, M., & Grangetto, M. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.07.022 · Cited by 12
On the Role of Structured Pruning for Neural Network Compression
Bragagnolo, A., Tartaglione, E., Fiandrotti, A., & Grangetto, M. (2021) · Proceedings - International Conference on Image Processing · doi:10.1109/icip42928.2021.9506708 · Cited by 10
Pruning untrained neural networks: Principles and AnalysisPreprint
Hayou, S., Ton, J.-F., Arnaud Doucet, B., & Teh, Y. W. (2020) · arXiv (Cornell University) · Cited by 9
Khona, M., Chandra, S., Ma, J. J., & Fiete, I. (2023) · Neural Computation · doi:10.1162/neco_a_01613 · Cited by 16
Gao, F., Liu, Y., Fu, W., Zhang, M., Ballard, A., & Zhao, L. L. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.123058 · Cited by 8
Deep Sparse Learning for Automatic Modulation Classification Using Recurrent Neural Networks
Zang, K., Wu, W., & Luo, W. (2021) · Sensors · doi:10.3390/s21196410 · Cited by 9
Parameterized $L_{0}$ Image Smoothing With Unsupervised Learning
Yang, Y., Tang, L. Y., Yan, T., Zeng, L., Shen, X., & Zhan, Y. (2024) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2024.3359060 · Cited by 8
Metadata
- Authors
- Louizos, C., Welling, M., & Kingma, D. P.
- ORCID
- Welling, Max
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2017
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1712.01312
- OpenAlex
- W2771655537
- Cited by
- 628
- References
- 29
- Citations by year
- 2026: 3 · 2025: 16 · 2024: 31 · 2023: 83 · 2022: 54 · 2021: 189 · 2020: 143 · 2019: 82 · 2018: 27
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- TNO Repository (submittedVersion)
- Repository hosted by TU Delft Library (TU Delft) (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Keywords
- sparse neural networks, L0 regularization, weight pruning, concrete distribution, model selection, AIC and BIC, conditional computation, generalization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
