Network Sketching: Exploiting Binary Structure in Deep CNNs
Open accessAbstract
Convolutional neural networks (CNNs) with deep architectures have substantially advanced the state-of-the-art in computer vision tasks. However, deep networks are typically resource-intensive and thus difficult to be deployed on mobile devices. Recently, CNNs with binary weights have shown compelling efficiency to the community, whereas the accuracy of such models is usually unsatisfactory in practice. In this paper, we introduce network sketching as a novel technique of pursuing binary-weight CNNs, targeting at more faithful inference and better trade-off for practical applications. Our basic idea is to exploit binary structure directly in pre-trained filter banks and to produce binary-weight models via tensor expansion. The whole process can be treated as a coarse-to-fine model approximation, akin to the pencil drawing steps of outlining and shading. To further speedup the generated models, namely the sketches, we also propose an associative implementation of binary tensor convolutions. Experimental results demonstrate that a proper sketch of AlexNet (or ResNet) outperforms the existing binary-weight models by large margins on the ImageNet large scale classification task, while the committed memory for network parameters only exceeds a little.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Non-uniform quantization can set quantization points according to the distribution, ensuring the overall quantization error is small.
“Non-uniform quantization [10] can be used to solve the problem. It can set the quantization points according to the distribution, ensuring the overall quantization error is small.”
From PTQ4ViT: Post-Training Quantization for Vision Transformers with Twin Uniform Quantization (2022) · cites this paper as [10]
Section 3.1.1 (page unavailable) says the network-sketching method selects binary bases and scale factors to minimize reconstruction error. It does not say that non-uniform quantization sets quantization points according to a distribution; Section 2 instead describes sketching as distinct from quantization-based methods. (LLM, conf=0.82)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 74,248 · see citing works
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · see citing works
Caffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654889 · Cited by 11,077 · see citing works
Shortest Connection Networks And Some Generalizations
Prim, R. C. (1957) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1957.tb01515.x · Cited by 4,624 · see citing works
XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks
Rastegari, M., Ordonez, V., Redmon, J., & Farhadi, A. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_32 · Cited by 4,408 · see citing works
(1989) · Cited by 3,214 · see citing works
BinaryConnect: Training Deep Neural Networks with binary weights during propagations
Courbariaux, M., Bengio, Y., & David, J.-P. (2015) · arXiv · doi:10.48550/arxiv.1511.00363 · Cited by 2,180 · see citing works
Courbariaux, M., Hubara, I., Soudry, D., El-Yaniv, R., & Bengio, Y. (2016) · arXiv · doi:10.48550/arxiv.1602.02830 · Cited by 2,106 · see citing works
(1993) · CaltechAUTHORS (California Institute of Technology) · Cited by 1,498 · see citing works
Learning both Weights and Connections for Efficient Neural Networks
Han, S., Pool, J., Tran, J., & Dally, W. J. (2015) · arXiv · doi:10.48550/arxiv.1506.02626 · Cited by 1,481 · see citing works
Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation
Denton, R., Zaremba, W., Bruna, J., LeCun, Y., & Fergus, R. (2014) · arXiv · doi:10.48550/arxiv.1404.0736 · Cited by 1,414 · see citing works
Quantized Convolutional Neural Networks for Mobile Devices
Wu, J., Leng, C., Wang, Y., Hu, Q., & Cheng, J. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.521 · Cited by 1,244 · see citing works
Speeding up Convolutional Neural Networks with Low Rank Expansions
Jaderberg, M., Vedaldi, A., & Zisserman, A. (2014) · Proceedings of the British Machine Vision Conference 2014 · doi:10.5244/c.28.88 · Cited by 1,164 · see citing works
Adaptive Greedy Approximations
Davis, G. M., Mallat, S., & Avellaneda, M. (1997) · Constructive Approximation · doi:10.1007/bf02678430 · Cited by 1,154 · see citing works
Compressing Deep Convolutional Networks using Vector QuantizationPreprint
Gong, Y., Liu, L., Yang, M., & Bourdev, L. D. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.6115 · Cited by 1,047 · see citing works
Predicting Parameters in Deep Learning
Denil, M., Shakibi, B., Dinh, L., Ranzato, M., & de Freitas, N. (2013) · arXiv · doi:10.48550/arxiv.1306.0543 · Cited by 853 · see citing works
Dynamic Network Surgery for Efficient DNNsPreprint
Guo, Y., Yao, A., & Chen, Y. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1608.04493 · Cited by 804 · see citing works
Self-citationSparse Convolutional Neural Networks
Liu, B., Wang, M., Foroosh, H., Tappen, M. F., & Penksy, M. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298681 · Cited by 750 · see citing works
Fixed Point Quantization of Deep Convolutional Networks
Lin, D. D., Talathi, S. S., & Annapureddy, V. S. (2015) · arXiv · doi:10.48550/arxiv.1511.06393 · Cited by 684 · see citing works
Data-free Parameter Pruning for Deep Neural Networks
Srinivas, S., & Babu, R. V. (2015) · British Machine Vision Conference (BMVC) · doi:10.5244/c.29.31 · Cited by 518 · see citing works
Speeding-up Convolutional Neural Networks Using Fine-tuned CP-DecompositionPreprint
Lebedev, V., Ganin, Y., Rakhuba, M., Oseledets, I. V., & Lempitsky, V. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.6553 · Cited by 511 · see citing works
Predicting Parameters in Deep Learning
Shakibi, B. (2014) · University of British Columbia · doi:10.14288/1.0165555 · Cited by 462 · see citing works
Convolutional neural networks with low-rank regularizationPreprint
Tai, C., Xiao, T., Zhang, Y., Wang, X., & Weinan, E. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.06067 · Cited by 373 · see citing works
Fast Training of Convolutional Networks through FFTs
Mathieu, M., Henaff, M., & LeCun, Y. (2014) · International Conference on Learning Representations · Cited by 343 · see citing works
Local Binary Convolutional Neural Networks
Juefei-Xu, F., Boddeti, V. N., & Savvides, M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.456 · Cited by 297 · see citing works
Fast Training of Convolutional Networks through FFTs
Mathieu, M., Henaff, M., & LeCun, Y. (2013) · arXiv · doi:10.48550/arxiv.1312.5851 · Cited by 273 · see citing works
Integrated Imaging and Vision Techniques for Industrial Inspection
Liu, Z., Ukida, H., Ramuhalli, P., & Niel, K. S. (2015) · Advances in computer vision and pattern recognition · doi:10.1007/978-1-4471-6741-9 · Cited by 270 · see citing works
Efficient and accurate approximations of nonlinear convolutional networks
Zhang, X., Zou, J., Xiang, M., He, K., & Sun, J. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298809 · Cited by 267 · see citing works
Fast Convolutional Nets With fbfft: A GPU Performance Evaluation
Vasilache, N., Johnson, J., Mathieu, M., Chintala, S., Piantino, S., & LeCun, Y. (2014) · arXiv · doi:10.48550/arxiv.1412.7580 · Cited by 246 · see citing works
Rigamonti, R., Sironi, A., Lepetit, V., & Fua, P. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.355 · Cited by 213 · see citing works
Training CNNs with Low-Rank Filters for Efficient Image ClassificationPreprint
Ioannou, Y., Robertson, D., Shotton, J., Cipolla, R., & Criminisi, A. (2015) · arXiv (Cornell University) · doi:10.13140/rg.2.1.3727.8163 · Cited by 99 · see citing works
Residual Networks are Exponential Ensembles of Relatively Shallow Networks.Preprint
Veit, A., Wilber, M. J., & Belongie, S. (2016) · arXiv (Cornell University) · Cited by 76 · see citing works
(2011)
Flagged first, then by citations
4 of the 39 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
A Survey of Quantization Methods for Efficient Neural Network Inference
Gholami, A., Kim, S., Dong, Z., Yao, Z., Mahoney, M. W., & Keutzer, K. (2022) · Low-Power Computer Vision · doi:10.1201/9781003162810-13 · Cited by 1,219 · see citing works
“As such, there is a large body of work that has proposed different solutions to address this [18, 25, 47, 52, 77, 82, 91, 92, 118, 120, 122, 127, 129, 133, 139, 147, 152, 157, 190, 192, 210, 241, 243, 252, 254, 279, 281].”
Model Compression and Hardware Acceleration for Neural Networks: A Comprehensive Survey
Deng, B. L., Li, G., Han, S., Shi, L., & Xie, Y. (2020) · Proceedings of the IEEE · doi:10.1109/jproc.2020.2976475 · Cited by 996 · see citing works
A Survey on Methods and Theories of Quantized Neural NetworksPreprint
Guo, Y. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.04752 · Cited by 249 · see citing works
Self-citationCLIP-Q: Deep Network Compression Learning by In-parallel Pruning-Quantization
Tung, F., & Mori, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00821 · Cited by 220 · see citing works
Review of Lightweight Deep Convolutional Neural Networks
Chen, F., Li, S., Han, J., Ren, F., & Yang, Z. (2023) · Archives of Computational Methods in Engineering · doi:10.1007/s11831-023-10032-z · Cited by 172 · see citing works
Structured Binary Neural Networks for Accurate Image Classification and Semantic Segmentation
Zhuang, B., Shen, C., Tan, M., Liu, L., & Reid, I. D. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00050 · Cited by 160 · see citing works
PTQ4ViT: Post-Training Quantization for Vision Transformers with Twin Uniform Quantization
Yuan, Z., Xue, C., Chen, Y., Wu, Q., & Sun, G. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19775-8_12 · Cited by 163 · see citing works
“Non-uniform quantization [10] can be used to solve the problem.”
BackgroundSelf-citationBinary Ensemble Neural Network: More Bits per Network or More Networks per Bit?
Zhu, S., Dong, X., & Su, H. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00506 · Cited by 153 · see citing works
Deep Neural Network Compression by In-Parallel Pruning-Quantization
Tung, F., & Mori, G. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2886192 · Cited by 140 · see citing works
Recurrent Neural Networks: An Embedded Computing Perspective
Rezk, N. M., Purnaprajna, M., Nordström, T., & Ul-Abdin, Z. (2020) · IEEE Access · doi:10.1109/access.2020.2982416 · Cited by 141 · see citing works
Compression of Deep Learning Models for Text: A Survey
Gupta, M., & Agrawal, P. (2022) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3487045 · Cited by 112 · see citing works
Training Quantized Neural Networks With a Full-Precision Auxiliary Module
Zhuang, B., Liu, L., Tan, M., Shen, C., & Reid, I. D. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00156 · Cited by 74 · see citing works
Neural network quantization in federated learning at the edge
Tonellotto, N., Gotta, A., Nardini, F. M., Gadler, D., & Silvestri, F. (2021) · Information Sciences · doi:10.1016/j.ins.2021.06.039 · Cited by 69 · see citing works
Ji, Y., & Chen, L. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2022.3213650 · Cited by 56 · see citing works
AutoQ: Automated Kernel-Wise Neural Network QuantizationPreprint
Lou, Q., Guo, F., Liu, L., Kim, M., & Jiang, L. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.05690 · Cited by 56 · see citing works
Adaptive Loss-Aware Quantization for Multi-Bit Networks
Qu, Z., Zhou, Z., Cheng, Y., & Thiele, L. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00801 · Cited by 54 · see citing works
Constraint-Aware Deep Neural Network Compression
Chen, C., Tung, F., Vedula, N., & Mori, G. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01237-3_25 · Cited by 53 · see citing works
DFTerNet: Towards 2-bit Dynamic Fusion Networks for Accurate Human Activity Recognition
Yang, Z., Raymond, O. I., Zhang, C., Wan, Y., & Long, J. (2018) · IEEE Access · doi:10.1109/access.2018.2873315 · Cited by 45 · see citing works
Memristive Quantized Neural Networks: A Novel Approach to Accelerate Deep Learning On-Chip
Zhang, Y., Cui, M., Shen, L., & Zeng, Z. (2019) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2019.2912205 · Cited by 46 · see citing works
Effective Training of Convolutional Neural Networks With Low-Bitwidth Weights and Activations
Zhuang, B., Tan, M., Liu, J., Liu, L., Reid, I. D., & Shen, C. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3088904 · Cited by 39 · see citing works
Post-training Quantization with Multiple Points: Mixed Precision without Mixed Precision
Liu, X., Ye, M., Zhou, D., & Liu, Q. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i10.17054 · Cited by 40 · see citing works
TRQ: Ternary Neural Networks With Residual Quantization
Li, Y., Ding, W., Liu, C., Zhang, B., & Guo, G. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i10.17036 · Cited by 36 · see citing works
BiQGEMM: Matrix Multiplication with Lookup Table for Binary-Coding-Based Quantized DNNs
Jeon, Y., Park, B., Kwon, S. J., Kim, B., Yun, J., & Lee, D. (2020) · International Conference for High Performance Computing, Networking, Storage and Analysis (SC) · doi:10.1109/sc41405.2020.00099 · Cited by 33 · see citing works
Contemporary Advances in Neural Network Quantization: A Survey
Li, M., Huang, Z., Chen, L., Ren, J., Jiang, M., Li, F., Fu, J., & Gao, C. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10650109 · Cited by 33 · see citing works
Mr.BiQ: Post-Training Non-Uniform Quantization based on Minimizing the Reconstruction Error
Jeon, Y., Lee, C., Cho, E., & Ro, Y. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01201 · Cited by 31 · see citing works
Compressing Deep Neural Networks With Sparse Matrix Factorization
Wu, K., Guo, Y., & Zhang, C. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2946636 · Cited by 27 · see citing works
Self-citationFATNN: Fast and Accurate Ternary Neural Networks
Chen, P., Zhuang, B., & Shen, C. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00517 · Cited by 25 · see citing works
QSFM: Model Pruning Based on Quantified Similarity Between Feature Maps for AI on Edge
Wang, Z., Liu, X., Huang, L., Chen, Y., Zhang, Y., Lin, Z., & Wang, R. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2022.3190873 · Cited by 34 · see citing works
Self-citationEfficient Quantization for Neural Networks with Binary Weights and Low Bitwidth Activations
Huang, K., Ni, B., & Yang, X. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33013854 · Cited by 25 · see citing works
Hou, L., Zhang, R., & Kwok, J. T.-Y. (2019) · Rare & Special e-Zone (The Hong Kong University of Science and Technology) · Cited by 24 · see citing works
Content-Aware Adaptive Device–Cloud Collaborative Inference for Object Detection
Hu, Y., Li, Z., Chen, Y., Cheng, Y., Cao, Z., & Liu, J. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3279579 · Cited by 24 · see citing works
Self-citationExtremely Low Bit Transformer Quantization for On-Device Neural Machine Translation
Chung, I., Kim, B., Choi, Y., Kwon, S. J., Jeon, Y., Park, B., Kim, S., & Lee, D. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.433 · Cited by 24 · see citing works
Structured Binary Neural Networks for Image Recognition
Zhuang, B., Shen, C., Tan, M., Chen, P., Liu, L., & Reid, I. D. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-022-01638-0 · Cited by 23 · see citing works
Automatic Pruning for Quantized Neural Networks
Guerra, L., & Drummond, T. W. (2021) · International Conference on Digital Image Computing: Techniques and Applications (DICTA) · doi:10.1109/dicta52665.2021.9647074 · Cited by 22 · see citing works
Residual Quantization for Low Bit-Width Neural Networks
Li, Z., Ni, B., Yang, X., Zhang, W., & Gao, W. (2021) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2021.3124095 · Cited by 22 · see citing works
Joint Coding of Local and Global Deep Features in Videos for Visual Search
Ding, L., Tian, Y., Fan, H., Chen, C., & Huang, T. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.2965306 · Cited by 22 · see citing works
Efficient Visual Recognition: A Survey on Recent Advances and Brain-inspired Methodologies
Wu, Y., Wang, D., Lu, X., Yang, F., Yao, M., Dong, W., Shi, J., & Li, G. (2022) · Machine Intelligence Research · doi:10.1007/s11633-022-1340-5 · Cited by 21 · see citing works
Learning Recurrent Binary/Ternary WeightsPreprint
Ardakani, A., Ji, Z., Smithson, S. C., Meyer, B. H., & Gross, W. J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.11086 · Cited by 19 · see citing works
Ternary MobileNets via Per-Layer Hybrid Filter Banks
Gope, D., Beu, J. G., Thakker, U., & Mattina, M. (2020) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00362 · Cited by 16 · see citing works
Sun, H., Wang, A., Pu, N., Li, Z., Huang, J., Liu, H., & Qi, Z. (2021) · International Conference on Artificial Intelligence and Computer Engineering (ICAICE) · doi:10.1109/icaice54393.2021.00095 · Cited by 13 · see citing works
Kwon, S. J., Kim, J., Bae, J., Yoo, K. M., Kim, J.-H., Park, B., Kim, B., Ha, J.-W., Sung, N., & Lee, D. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.240 · Cited by 14 · see citing works
Neural Network Language Model Compression With Product Quantization and Soft Binarization
Yu, K., Ma, R., Shi, K., & Liu, Q. (2020) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2020.3015659 · Cited by 10 · see citing works
Multi-Precision Quantized Neural Networks via Encoding Decomposition of {-1,+1}
Sun, Q., Shang, F., Yang, K., Li, X., Ren, Y., & Jiao, L. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33015024 · Cited by 11 · see citing works
Retraining-Based Iterative Weight Quantization for Deep Neural NetworksPreprint
Lee, D., & Kim, B. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.11233 · Cited by 11 · see citing works
Multi-Objective Convex Quantization for Efficient Model Compression
Fan, C., Guo, D., Wang, Z., & Wang, M. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3521589 · Cited by 10 · see citing works
Weight Normalization based Quantization for Deep Neural Network CompressionPreprint
Cai, W.-P., & Li, W.-J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.00593 · Cited by 10 · see citing works
Deployment of Machine Learning Algorithms on Resource-Constrained Hardware Platforms for Prosthetics
Just, F., Ghinami, C., Zbinden, J., & Ortiz-Catalan, M. (2024) · IEEE Access · doi:10.1109/access.2024.3371251 · Cited by 8 · see citing works
SlimmeRF: Slimmable Radiance Fields
Yuan, S., & Zhao, H. (2024) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv62453.2024.00143 · Cited by 8 · see citing works
Self-citationEfficient Deep Learning in Network Compression and Acceleration
Ge, S. (2018) · IntechOpen eBooks · doi:10.5772/intechopen.79562 · Cited by 7 · see citing works
QSFL: Two-Level Communication-Efficient Federated Learning on Mobile Edge Devices
Yi, L., Wang, G., Wang, X., & Liu, X. (2024) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2024.3455098 · Cited by 6 · see citing works
AccEPT: An Acceleration Scheme for Speeding up Edge Pipeline-Parallel Training
Chen, Y.-H., Yan, Y., Yang, Q.-Q., Shu, Y.-C., He, S.-B., Shi, Z.-G., & Chen, J.-M. (2023) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2024.3389779 · Cited by 8 · see citing works
Self-citationSequential Characteristics Based Operators Disassembly Quantization Method for LSTM Layers
Wang, Y., Ma, Z., & Yang, Z. (2022) · Applied Sciences · doi:10.3390/app122412744 · Cited by 6 · see citing works
HadaNets: Flexible Quantization Strategies for Neural Networks
Akhauri, Y. (2019) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2019.00078 · Cited by 6 · see citing works
Optimization Strategies in Quantized Neural Networks: A Review
Lin, Y., Lei, M., & Niu, L. (2019) · IEEE ... International Conference on Data Mining workshops · doi:10.1109/icdmw.2019.00063 · Cited by 6 · see citing works
Highly Efficient Neural Network Language Model Compression Using Soft Binarization Training
Ma, R., Liu, Q., & Yu, K. (2019) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru46091.2019.9003744 · Cited by 6 · see citing works
HLHLp: Quantized Neural Networks Training for Reaching Flat Minima in Loss Surface
Shin, S., Park, J.-H., Boo, Y., & Sung, W. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.6035 · Cited by 6 · see citing works
Fuzzy Pruning for Compression of Convolutional Neural Networks
Zhao, W. B., Li, Y., & Shang, L. (2019) · IEEE International Conference on Fuzzy Systems (FUZZ-IEEE) · doi:10.1109/fuzz-ieee.2019.8858894 · Cited by 5 · see citing works
Application of Approximate Matrix Multiplication to Neural Networks and Distributed SLAM
Plancher, B., Brumar, C. D., Brumar, I., Pentecost, L., Rama, S., & Brooks, D. M. (2019) · IEEE High Performance Extreme Computing Conference · doi:10.1109/hpec.2019.8916468 · Cited by 5 · see citing works
Coal and Gangue Detection Networks with Compact and High-Performance Design
Cao, X., Liu, H., Liu, Y., Li, J., & Xu, K. (2024) · Sensors · doi:10.3390/s24227318 · Cited by 4 · see citing works
Circulant Binary Convolutional Networks for Object Recognition
Liu, C., Ding, W., Hu, Y. C., Xia, X., Zhang, B., Liu, J., & Doermann, D. (2020) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2020.2969516 · Cited by 4 · see citing works
Adversarial Deep Mutual Learning
Lai, X., & Qu, Y. (2019) · IEEE International Conference on Unmanned Systems and Artificial Intelligence (ICUSAI) · doi:10.1109/icusai47366.2019.9124828 · Cited by 4 · see citing works
GBCNs: Genetic Binary Convolutional Networks for Enhancing the Performance of 1-bit DCNNsPreprint
Liu, C., Ding, W., Hu, Y. C., Zhang, B., Liu, J., & Guo, G. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.11634 · Cited by 4 · see citing works
Translational Bit-by-Bit Multi-bit Quantization for CRNN on Keyword Spotting
Liu, B., Sun, Y., & Liu, B. (2019) · International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery (CyberC) · doi:10.1109/cyberc.2019.00082 · Cited by 3 · see citing works
High Throughput Matrix-Matrix Multiplication between Asymmetric Bit-Width OperandsPreprint
Gope, D., Beu, J. G., & Mattina, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.00638 · Cited by 3 · see citing works
BinArray: A Scalable Hardware Accelerator for Binary Approximated CNNs
Fischer, M., & Wassner, J. (2021) · IEEE Annual Computing and Communication Workshop and Conference (CCWC) · doi:10.1109/ccwc51732.2021.9375963 · Cited by 2 · see citing works
Adaptive Loss-Aware Quantization for Multi-Bit NetworksPreprint
Qu, Z., Zhou, Z., Cheng, Y., & Thiele, L. (2019) · arXiv (Cornell University) · doi:10.3929/ethz-b-000448782 · Cited by 2 · see citing works
Marginal samples for knowledge distillation
Chen, Z., Zheng, X., Shen, H., Zhang, J., Dai, P., & Zhao, R. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.08.004 · Cited by 1 · see citing works
Regularizing Binary Neural Networks via Ensembling for Efficient Person Re-Identification
Serbetci, A., & Akgül, Y. S. (2023) · IEEE Access · doi:10.1109/access.2023.3281737 · Cited by 1 · see citing works
DOPNet: Dynamic Optimized Pruning Net for Model Compression
Li, X., Dai, W., Chen, Y., & Wang, J. (2022) · IEEE Smart World Congress (SWC) · doi:10.1109/smartworld-uic-atc-scalcom-digitaltwin-pricomp-metaverse56740.2022.00072 · Cited by 1 · see citing works
Self-citationQuantization of Neural Networks
Zhang, B., Wang, T., Xu, S. H., & Doermann, D. (2024) · Computational intelligence methods and applications · doi:10.1007/978-981-99-5068-3_4 · Cited by 1 · see citing works
Towards Lossless Binary Convolutional Neural Networks Using Piecewise Approximation
Zhu, B., Al-Ars, Z., & Pan, W. (2020) · Frontiers in artificial intelligence and applications · doi:10.3233/faia200286 · Cited by 1 · see citing works
Training Quantized Network with Auxiliary Gradient Module.Preprint
Zhuang, B., Liu, L., Tan, M., Shen, C., & Reid, I. D. (2019) · arXiv (Cornell University) · Cited by 1 · see citing works
Shin, S., Boo, Y., & Sung, W. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.00343 · Cited by 1 · see citing works
GradFreeBits: Gradient Free Bit Allocation for Dynamic Low Precision Neural NetworksPreprint
Bodner, B. J., Shalom, G. B., & Treister, E. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.09298 · Cited by 1 · see citing works
Q-Rater: Non-Convex Optimization for Post-Training Uniform QuantizationPreprint
Kim, B., Lee, D., Ro, Y., Jeon, Y., Kwon, S. J., Park, B., & Oh, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.01868 · Cited by 1 · see citing works
ARB-LLM: Alternating Refined Binarizations for Large Language Models
Li, Z., Yan, X., Zhang, T., Qin, H., Xie, D., Tian, J., shi, Z., Kong, L., Zhang, Y., & Yang, X. (2024) · arXiv · doi:10.48550/arxiv.2410.03129 · Cited by 1 · see citing works
Multi-Precision Quantized Neural Networks via Encoding Decomposition of -1 and +1Preprint
Sun, Q., Shang, F., Yang, K., Li, X., Ren, Y., & Jiao, L. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.13389
Modulating Regularization Frequency for Efficient Compression-Aware Model TrainingPreprint
Lee, D., Kwon, S. J., Kim, B., Yun, J., Park, B., & Jeon, Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.01875
Quantized Neural Networks via {-1, +1} Encoding Decomposition and AccelerationPreprint
Sun, Q., Li, X., Shang, F., Liu, H., Yang, K., Jiao, L., & Lin, Z. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.09886
Tree-Structured Binary Neural Networks
Serbetci, A., & Akgül, Y. S. (2021) · Signal Processing and Communications Applications Conference (SIU) · doi:10.1109/siu53274.2021.9477811
A Review on Lightweighting Methods for Deep Learning Models of PHM in Airborne Mechatronic Systems
Liu, Y., Xue, L., Huang, T., Chen, C., & Tao, Y. (2025) · CAA Symposium on Fault Detection, Supervision and Safety for Technical Processes (SAFEPROCESS) · doi:10.1109/safeprocess67117.2025.11268211
Park, C., Hwang, I., Kang, B. J., & Kim, H. (2026) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1145/3770743.3804211
- Authors
- Guo, Y., Yao, A., Zhao, H., & Chen, Y.
- Affiliations
- Intel (United States) (US)
- Venue
- 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2017-07
- Pages
- 4040-4048
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2963576971
- Cited by
- 81
- References
- 39
- Citations by year
- 2026: 1 · 2025: 1 · 2024: 8 · 2023: 3 · 2022: 12 · 2021: 14 · 2020: 14 · 2019: 20 · 2018: 8
- FWCI
- 3.36
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Advanced Memory and Neural Computing
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- filter banks, model compression, mobile deployment, AlexNet, ResNet, ImageNet classification, memory efficiency
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
