The DOI matches its DataCite record; 25 of the 29 references it cites were found in a registry or an index.
Learning Sparse Neural Networks through $L_0$ Regularization
Open accessAbstract
We propose a practical method for $L_0$ norm regularization for neural networks: pruning the network during training by encouraging weights to become exactly zero. Such regularization is interesting since (1) it can greatly speed up training and inference, and (2) it can improve generalization. AIC and BIC, well-known model selection criteria, are special cases of $L_0$ regularization. However, since the $L_0$ norm of weights is non-differentiable, we cannot incorporate it directly as a regularization term in the objective function. We propose a solution through the inclusion of a collection of non-negative stochastic gates, which collectively determine which weights to set to zero. We show that, somewhat surprisingly, for certain distributions over the gates, the expected $L_0$ norm of the resulting gated weights is differentiable with respect to the distribution parameters. We further propose the \emph{hard concrete} distribution for the gates, which is obtained by "stretching" a binary concrete distribution and then transforming its samples with a hard-sigmoid. The parameters of the distribution over the gates can then be jointly optimized with the original network parameters. As a result our method allows for straightforward and efficient learning of model structures with stochastic gradient descent and allows for conditional computation in a principled way. We perform various experiments to demonstrate the effectiveness of the resulting approach and regularizer.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Many methods based on the sparse learning (SL) framework have also been proven to be effective [43-47]”
From A survey of model compression strategies for object detection (2023) · cites this paper as [44]
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Self-citationGradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Regression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482
Estimating the Dimension of a Model
Schwarz, G. (1978) · The Annals of Statistics · doi:10.1214/aos/1176344136 · Cited by 47,033
Auto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067
Self-citationInformation Theory and an Extension of the Maximum Likelihood Principle
Akaike, H. (1998) · Springer series in statistics · doi:10.1007/978-1-4612-1694-0_15 · Cited by 18,636
Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
Williams, R. J. (1992) · Machine Learning · doi:10.1007/bf00992696 · Cited by 8,608
Zagoruyko, S., & Komodakis, N. (2016) · Procedings of the British Machine Vision Conference 2016 · doi:10.5244/c.30.87 · Cited by 7,244
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1512.03385 · Cited by 4,196
Stochastic Backpropagation and Approximate Inference in Deep Generative ModelsPreprint
Rezende, D. J., Mohamed, S., & Wierstra, D. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1401.4082 · Cited by 3,921
Cited by
Sorted by citations
Items 501–600 of 628
On the landscape of one-hidden-layer sparse networks and beyond
Lin, D., Sun, R., & Zhang, Z. (2022) · Artificial Intelligence · doi:10.1016/j.artint.2022.103739 · Cited by 2
Boosting the Convergence of Reinforcement Learning-Based Auto-Pruning Using Historical Data
Mu, J., Wang, M., Zhu, F., Yang, J. J., Lin, W. G., & Zhang, W. (2023) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2023.3319594 · Cited by 2
Sparse Communication via Mixed Distributions
Farinhas, A., Aziz, W. F., Niculae, V., & Martins, A. F. T. (2021) · UvA-DARE (University of Amsterdam) · doi:10.48550/arxiv.2108.02658 · Cited by 3
Dep-$$L_0$$: Improving $$L_0$$-Based Network Sparsification via Dependency Modeling
Li, Y., & Ji, S. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-86523-8_11 · Cited by 1
Simulation of Fault Diagnosis Model for Managing Aeronautical Multivariate Heterogeneous Inputs
Zhang, Y., Peng, D., Meng, G., Zhao, Q. L., & Li, T. (2023) · doi:10.1109/prmvia58252.2023.00043 · Cited by 1
Nguyen, V.-T., Tartaglione, E., & Dinh, T. (2023) · Symposium on Software Performance · doi:10.1109/ssp53291.2023.10208067 · Cited by 1
Chintam, A., Beloch, R., Zuidema, W. H., Hanna, M., & van der Wal, O. (2023) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/2023.blackboxnlp-1.29 · Cited by 10
Multi-Resolution Model Compression for Deep Neural Networks: A Variational Bayesian Approach
Xia, C., Guo, H., Ma, H., Tsang, D. H. K., & Lau, V. K. N. (2024) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2024.3382405 · Cited by 1
Jiang, J., & Huang, H. (2024) · Neural Networks · doi:10.1016/j.neunet.2024.106664 · Cited by 1
Evaluating Model Robustness Using Adaptive Sparse L0 Regularization
Liu, W., Li, Z., & Chen, W. (2024) · Lecture notes in computer science · doi:10.1007/978-981-96-0814-0_1 · Cited by 1
ЗГОРТКОВА НЕЙРОМЕРЕЖА ДЛЯ ВИЯВЛЕННЯ ШАХРАЙСЬКИХ ОПЕРАЦІЙ З КРЕДИТНИМИ КАРТКАМИ
Корнієнко, О. В., & Субботин, С. А. (2019) · Automation technological and business processes · doi:10.15673/atbp.v11i3.1503 · Cited by 1
Semi-Supervised Learning with Variational Bayesian Inference and Maximum Uncertainty Regularization
Do, K., Tran, T., & Venkatesh, S. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i8.16889 · Cited by 1
ES-ENAS: Controller-Based Architecture Search for Evolutionary Reinforcement Learning
Song, X., Choromański, K., Parker-Holder, J., Tang, Y., Peng, D., Jain, D., Gao, W., Pacchiano, A., Sarlós, T., & Yang, Y. (2021) · Cited by 2
Feng, D., & Ren, F. (2021) · Electronics · doi:10.3390/electronics10111329 · Cited by 1
Zhang, H., & Chan, W. K. (2021) · IEEE Transactions on Reliability · doi:10.1109/tr.2021.3086189 · Cited by 1
CE-PeopleSeg: Real-time people segmentation with 10% CPU usage for video conference
Jiang, Z., He, Z., Huang, X., Yang, Z., & Tan, P. (2021) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00102 · Cited by 1
Learning a Neural Diff for Speech Models
Macoskey, J. J., Strimel, G. P., & Rastrow, A. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-1575 · Cited by 1
On Sparsifying Encoder Outputs in Sequence-to-Sequence Models
Zhang, B., Titov, I., & Sennrich, R. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.255 · Cited by 1
Learning Deeper Non-Monotonic Networks by Softly Transferring Solution Space
Wu, Z.-F., Xue, H., & Bai, W. (2021) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/440 · Cited by 1
Galchonkov, O. N., Nevrev, A., Shevchuk, B., & Baranov, N. (2022) · Eastern-European Journal of Enterprise Technologies · doi:10.15587/1729-4061.2022.253103 · Cited by 1
A Sparse Gating Convolutional Recurrent Network for Traffic Flow Prediction
Huang, X., Tang, J., Peng, Z., Chen, Z., & Zeng, H. (2022) · Mathematical Problems in Engineering · doi:10.1155/2022/6446941 · Cited by 1
Disjunctive Threshold Networks for Tabular Data Classification
Wang, W., Qiao, L., & Lin, B. (2023) · IEEE Open Journal of the Computer Society · doi:10.1109/ojcs.2023.3282948 · Cited by 1
M-estimation with the Trimmed l1 PenaltyPreprint
Yun, J., Zheng, P., Yang, E., Lozano, A. C., & Aravkin, A. Y. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.07495 · Cited by 1
Deep supervised feature selection using Stochastic Gates.Preprint
Yamada, Y., Lindenbaum, O., Negahban, S. N., & Kluger, Y. (2018) · arXiv (Cornell University) · Cited by 1
A Relational Tucker Decomposition for Multi-Relational Link PredictionPreprint
Wang, Y., Broscheit, S., & Gemulla, R. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.00898 · Cited by 1
Not All Features Are Equal: Feature Leveling Deep Neural Networks for Better InterpretationPreprint
Lu, Y., & Yang, R. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.10009 · Cited by 1
Reinforcement Learning with Chromatic NetworksPreprint
Song, X., Choromański, K., Parker-Holder, J., Tang, Y., Gao, W., Pacchiano, A., Sarlós, T., Jain, D., & Yang, Y. (2019) · arXiv (Cornell University) · Cited by 1
𝓁 0 Regularized Structured Sparsity Convolutional Neural Networks.Preprint
Bui, K., Park, F., Zhang, S., Qi, Y., & Xin, J. X. (2019) · arXiv (Cornell University) · Cited by 1
How Does BN Increase Collapsed Neural Network Filters?Preprint
Zhou, S., Wang, X., Luo, P., Feng, L., Li, W., & Zhang, W. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.11216 · Cited by 1
Rodent: Relevance determination in differential equationsPreprint
Heim, N., Šmídl, V., & Pevný, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.00656 · Cited by 1
Dynamic Narrowing of VAE Bottlenecks Using GECO and L0 RegularizationPreprint
Boom, C. D., Wauthier, S. T., Verbelen, T., & Dhoedt, B. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.10901 · Cited by 1
Learning to Ask Medical Questions using Reinforcement LearningPreprint
Shaham, U., Zahavy, T., Caraballo, C., Mahajan, S., Massey, D. S., & Krumholz, H. M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.00994 · Cited by 1
Out-of-the-box channel pruned networksPreprint
Venkatesan, R., Gurumurthy, S., Zhou, X. H., & Luo, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.14584 · Cited by 1
The Role of Regularization in Shaping Weight and Node Pruning Dependency and DynamicsPreprint
Ben-Guigui, Y., Goldberger, J., & Raviv, T. R. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.03827 · Cited by 1
Inducing Meaningful Units from Character Sequences with Slot Attention.Preprint
Behjati, M., & Henderson, J. (2021) · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · Cited by 1
Thakker, U., Whatmough, P. N., Liu, Z., Mattina, M., & Beu, J. G. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.07071 · Cited by 2
Reconciling the Discrete-Continuous Divide: Towards a Mathematical Theory of Sparse CommunicationPreprint
Martins, A. F. T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.00755 · Cited by 1
The Flip Side of the Reweighted Coin: Duality of Adaptive Dropout and RegularizationPreprint
LeJeune, D., Javadi, H., & Baraniuk, R. G. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.07769 · Cited by 1
Boucaud, L., Aloise, D., & Saunier, N. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.15321 · Cited by 1
Exact Backpropagation in Binary Weighted Networks with Group Weight TransformationsPreprint
Shulman, Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.01400 · Cited by 2
Differentiable Feature Selection, a Reparameterization ApproachPreprint
Donà, J., & Gallinari, P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.10030 · Cited by 1
One-Cycle Pruning: Pruning ConvNets Under a Tight Training BudgetPreprint
Hubens, N., Mancaş, M., Gosselin, B. H., Preda, M., & Zaharia, T. B. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.02086 · Cited by 1
A fast asynchronous MCMC sampler for sparse Bayesian inferencePreprint
Atchadé, Y. F., & Wang, L. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.06446 · Cited by 1
High-dimensional Bayesian Optimization for CNN Auto Pruning with Clustering and Rollback.Preprint
Mu, J., Fan, H., & Zhang, W. (2021) · arXiv (Cornell University) · Cited by 1
On the Compression of Neural Networks Using 𝓁 0 -Norm Regularization and Weight Pruning.Preprint
Oliveira, F. D. D. R., Batista, E. L. O., & Seara, R. (2021) · arXiv (Cornell University) · Cited by 1
Evolving Transferable Pruning FunctionsPreprint
Liu, Y., Kung, S., & Wentzlaff, D. (2021) · arXiv (Cornell University) · Cited by 1
Mining the Weights Knowledge for Optimizing Neural Network StructuresPreprint
Han, M., Liu, X., Hai, Z., & Duan, X. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.05954 · Cited by 1
Sparse Regularization with the $\ell_0$ NormPreprint
Xu, Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.08244 · Cited by 1
Zhang, M., Huang, W., & Wang, L. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.11542 · Cited by 1
Latency-Constrained Dnn Architecture Learning for Edge Systems Using Zerorized Batch NormalizationPreprint
Huai, S., Liu, D., Kong, H., Liu, W., Subramaniam, R., Makaya, C., & Lin, Q. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4186455 · Cited by 1
Adfq-Vit: Activation-Distribution-Friendly Post-Training Quantization for Vision TransformersPreprint
Jiang, Y., Sun, N., Xie, X., Yang, F., & Li, T. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4893253 · Cited by 1
Structured Sparsification of Gated Recurrent Neural Networks
Lobacheva, E., Chirkova, N., Markovich, A., & Vetrov, D. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.5938
Dinh, T., & Xin, J. X. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.09731
L0 Regularization Based Neural Network Design and CompressionPreprint
Ahmed, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.13652
Batch Gradient Training Method with Smoothing $$l_0$$ Regularization for Echo State Networks
Ahmad, Z., Nie, K., Qiao, J., & Yang, C. (2019) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-030-32388-2_42
Lee, J. H., Yun, J., Hwang, S. J., & Yang, E. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.12990
Нейромережеве моделювання критичних температур піттінгоутворення сталі
Корнієнко, О. В., Subbotin, S. O., & Naryvskyi, O. E. (2019) · Реєстрація зберігання і обробка даних · doi:10.35681/1560-9189.2019.1.1.179699
Huang, J., Jiao, Y., Kang, L., Liu, J., Liu, Y., & Lu, X. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.05819
Variational Dropout Sparsification for Particle Identification speed-up
Ryzhikov, A., Деркач, Д., & Hushchyn, M. (2020) · Journal of Physics Conference Series · doi:10.1088/1742-6596/1525/1/012099
A Note on Latency Variability of Deep Neural Networks for Mobile InferencePreprint
Yang, L., Lu, B., & Ren, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.00138 · Cited by 2
Bayesian Sparsification Methods for Deep Complex-valued NetworksPreprint
Nazarov, I., & Burnaev, E. V. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.11413
Deep Convolutional Networks without Learning the Classifier Layer
Qian, Z. (2020) · RIT Scholar Works (Rochester Institute of Technology)
Nonconvex Sparse Representation With Slowly Vanishing Gradient Regularizers
Kim, E., Lee, M., & Oh, S. (2020) · IEEE Access · doi:10.1109/access.2020.3009971
Detecting Relevant Feature Interactions for Recommender Systems via Graph Neural Networks.Preprint
Su, Y., Zhang, R., Erfani, S., & Xu, Z. (2020) · arXiv (Cornell University)
FlipOut: Uncovering Redundant Weights via Sign Flipping
Apostol, A. C., Stol, M., & Forré, P. D. (2021) · Communications in computer and information science · doi:10.1007/978-3-030-76640-5_2
Byrne, D. W., Smith, S. J., Duran, J., & Santerre, J. (2020) · SMU Scholar (Southern Methodist University)
Cascaded Mixed-Precision Networks
Geng, X., Lin, J., & Li, S. (2020) · Proceedings - International Conference on Image Processing · doi:10.1109/icip40778.2020.9190760
Finding Dense Supermasks in Randomly Initialized Neural Networks.
Sándor, C. (2020)
Architecture Agnostic Neural NetworksPreprint
Talukder, S., Raghavan, G., & Yue, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.02712
FIREPruning: Learning-based Filter Pruning for Convolutional Neural Network Compression.
Fang, Y., Li, W., Zeng, Y., & Lu, S. (2020) · Asian Conference on Machine Learning
Channel selection using Gumbel SoftmaxPreprint
Herrmann, C., Bowen, R. S., & Zabih, R. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.04180 · Cited by 1
Sparsifying networks by traversing GeodesicsPreprint
Raghavan, G., & Thomson, M. W. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.09605 · Cited by 1
Interpreting Neural Networks as Gradual Argumentation Frameworks (Including Proof Appendix)Preprint
Potyka, N. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.05738 · Cited by 2
Adaptive Extreme Edge Computing for Wearable DevicesPreprint
Covi, E., Donati, E., Heidari, H., Kappel, D., Liang, X., Payvand, M., & Wang, W. J. (2020) · Zurich Open Repository and Archive (University of Zurich) · doi:10.5167/uzh-200278
Huang, D., & Chen, X. (2020) · International Conference on Robotics, Intelligent Control and Artificial Intelligence (RICAI) · doi:10.1145/3438872.3439080
CNN with large memory layersPreprint
Karimov, R., Malkov, Y., Iskakov, K., & Lempitsky, V. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.11685
Cascade Weight Shedding in Deep Neural Networks: Benefits and Pitfalls for Network PruningPreprint
Azarian, K., & Porikli, F. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.10629
Efficient Sparse Artificial Neural NetworksPreprint
Naji, S. M., Abtahi, A., & Marvasti, F. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.07674 · Cited by 3
Explore the Knowledge contained in Network Weights to Obtain Sparse Neural NetworksPreprint
Han, M., Liu, X., Hai, Z., & Li, Z. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.15590
Sequential Encryption of Sparse Neural Networks Toward Optimum Representation of Irregular Sparsity.Preprint
Park, B., Kwon, S. J., Lee, D., Oh, D., Kim, B., Jeon, Y., & Ro, Y. (2021) · arXiv (Cornell University)
Regularization and Compression of Deep Neural Networks
Khan, N. U. (2021) · HARVEST (University of Saskatchewan)
Channel Planting for Deep Neural Networks using Knowledge Distillation
Mitsuno, K., Nomura, Y., & Kurita, T. (2021) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr48806.2021.9412760
Grouping Words with Semantic Diversity
Chubarian, K., Khan, A. R., Sidiropoulos, A., & Xu, J. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.257
Locally Sparse Networks for Interpretable Predictions.Preprint
Yang, J., Lindenbaum, O., & Kluger, Y. (2021) · arXiv (Cornell University)
Learnt Sparsification for Interpretable Graph Neural NetworksPreprint
Rathee, M., Zhang, Z., Funke, T., Khosla, M., & Anand, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.12920
Adapting the Function Approximation Architecture in Online Reinforcement LearningPreprint
Martin, J. D., & Modayil, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.09776 · Cited by 2
HANT: Hardware-Aware Network TransformationPreprint
Molchanov, P. A., Hall, J., Yin, H., Kautz, J., Fusi, N., & Vahdat, A. (2021) · arXiv (Cornell University)
Differentiable Subset Pruning of Transformer Heads
Li, J., Cotterell, R., & Sachan, M. (2021) · Repository for Publications and Research Data (ETH Zurich) · doi:10.3929/ethz-b-000528141
Luong, H.-T., & Yamagishi, J. (2021) · ISCA Speech Synthesis Workshop (SSW) · doi:10.21437/ssw.2021-24
GCNNIRec: Graph Convolutional Networks with Neighbor Complex Interactions for Recommendation
Mei, T., Sun, T., Chen, R., Zhou, M., & Leong Hou, U. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-85899-5_25
Comparing Regularization Techniques Applied to a Perceptron
Boreland, B., Kunze, H. E., & Levere, K. M. (2021) · Springer proceedings in mathematics & statistics · doi:10.1007/978-3-030-63591-6_12
Learning Sparsity of Representations with Discrete Latent Variables
Zhao, X., Rubio, D. O., Serra, G., & Niepert, M. (2021) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9533762
ES-ENAS: Efficient Evolutionary Optimization for Large Hybrid Search SpacesPreprint
Song, X., Choromański, K., Parker-Holder, J., Tang, Y., Zhang, Q., Peng, D., Jain, D., Gao, W., Pacchiano, A., Sarlós, T., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.07415 · Cited by 1
Probabilistic fine-tuning of pruning masks and PAC-Bayes self-bounded learningPreprint
Hayou, S., He, B., & Dziugaite, G. K. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.11804 · Cited by 3
Back to Basics: Efficient Network Compression via IMP.Preprint
Zimmer, M., Spiegel, C., & Pokutta, S. (2021) · arXiv (Cornell University)
Zhang, H., Zhou, W. G., Wen, J., Liu, L., Zeng, J., & Gao, M. (2021) · IEEE International Conference on Services Computing · doi:10.1109/scc53864.2021.00019
Neural Weight Step Video CompressionPreprint
Czerkawski, M., Cardona, J., Atkinson, R. C., Michie, C., Andonović, I., Clemente, C., & Tachtatzis, C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2112.01504
Lan, W., Cheung, Y., & Lan, L. (2021) · doi:10.36227/techrxiv.17031917
Lan, W., Cheung, Y., & Lan, L. (2021) · doi:10.36227/techrxiv.17031917.v1
Visual Question and Answering System for NLP and Computer Vision Problems
Koziy, D., & Shakhovska, N. (2021) · IEEE International Conference on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications (IDAACS) · doi:10.1109/idaacs53288.2021.9660989
Metadata
- Authors
- Louizos, C., Welling, M., & Kingma, D. P.
- ORCID
- Welling, Max
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2017
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1712.01312
- OpenAlex
- W2771655537
- Cited by
- 628
- References
- 29
- Citations by year
- 2026: 3 · 2025: 16 · 2024: 31 · 2023: 83 · 2022: 54 · 2021: 189 · 2020: 143 · 2019: 82 · 2018: 27
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- TNO Repository (submittedVersion)
- Repository hosted by TU Delft Library (TU Delft) (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Keywords
- sparse neural networks, L0 regularization, weight pruning, concrete distribution, model selection, AIC and BIC, conditional computation, generalization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review