BinaryConnect: Training Deep Neural Networks with binary weights during propagations
Open accessAbstract
Deep Neural Networks (DNN) have achieved state-of-the-art results in a wide range of tasks, with the best results obtained with large training sets and large models. In the past, GPUs enabled these breakthroughs because of their greater computational speed. In the future, faster computation at both training and test time is likely to be crucial for further progress and for consumer applications on low-power devices. As a result, there is much interest in research and development of dedicated hardware for Deep Learning (DL). Binary weights, i.e., weights which are constrained to only two possible values (e.g. -1 or 1), would bring great benefits to specialized DL hardware by replacing many multiply-accumulate operations by simple accumulations, as multipliers are the most space and power-hungry components of the digital implementation of neural networks. We introduce BinaryConnect, a method which consists in training a DNN with binary weights during the forward and backward propagations, while retaining precision of the stored weights in which gradients are accumulated. Like other dropout schemes, we show that BinaryConnect acts as regularizer and we obtain near state-of-the-art results with BinaryConnect on the permutation-invariant MNIST, CIFAR-10 and SVHN.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Since 2015, quantization methods have been applied in compression.
“Since 2015, quantization methods have been applied in compression.”
From A survey of model compression strategies for object detection (2023) · cites this paper as [194]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
Flagged first, then by citations
Research works printed but not linked: 40
- Geoffrey Hinton, Li Deng, George E. Dahl, Abdel-rahman Mohamed, Navdeep Jaitly, Andrew Senior, Vincent Vanhoucke, Patrick Nguyen, Tara Sainath, and Brian Kingsbury. Deep neural networks for acous- tic modeling in speech recognition. IEEE Signal Processing Magazine, 29(6):82-97, Nov. 2012.
- Tara Sainath, Abdel rahman Mohamed, Brian Kingsbury, and Bhuvana Ramabhadran. Deep convolutional neural networks for LVCSR. In ICASSP 2013, 2013.
- Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions. Technical report, arXiv:1409.4842, 2014. · arXiv
- Jacob Devlin, Rabih Zbib, Zhongqiang Huang, Thomas Lamar, Richard Schwartz, and John Makhoul. Fast and robust neural network joint models for statistical machine translation. In Proc. ACL'2014, 2014.
- Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. Sequence to sequence learning with neural networks. In NIPS'2014, 2014.
- Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. In ICLR'2015, arXiv:1409.0473, 2015. · arXiv
- Rajat Raina, Anand Madhavan, and Andrew Y. Ng. Large-scale deep unsupervised learning using graphics processors. In ICML'2009, 2009.
- Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Jauvin. A neural probabilistic language model. Journal of Machine Learning Research, 3:1137-1155, 2003.
- J. Dean, G.S Corrado, R. Monga, K. Chen, M. Devin, Q.V. Le, M.Z. Mao, M.A. Ranzato, A. Senior, P. Tucker, K. Yang, and A. Y. Ng. Large scale distributed deep networks. In NIPS'2012, 2012.
- Sang Kyun Kim, Lawrence C McAfee, Peter Leonard McMahon, and Kunle Olukotun. A highly scalable restricted Boltzmann machine FPGA implementation. In Field Programmable Logic and Applications, 2009. FPL 2009. International Conference on, pages 367-372. IEEE, 2009.
- Tianshi Chen, Zidong Du, Ninghui Sun, Jia Wang, Chengyong Wu, Yunji Chen, and Olivier Temam. Diannao: A small-footprint high-throughput accelerator for ubiquitous machine-learning. In Proceedings of the 19th international conference on Architectural support for programming languages and operating systems, pages 269-284. ACM, 2014.
- Yunji Chen, Tao Luo, Shaoli Liu, Shijin Zhang, Liqiang He, Jia Wang, Ling Li, Tianshi Chen, Zhiwei Xu, Ninghui Sun, et al. Dadiannao: A machine-learning supercomputer. In Microarchitecture (MICRO), 2014 47th Annual IEEE/ACM International Symposium on, pages 609-622. IEEE, 2014.
- Lorenz K Muller and Giacomo Indiveri. Rounding methods for neural networks with low resolution synaptic weights. arXiv preprint arXiv:1504.05767, 2015. · arXiv
- Suyog Gupta, Ankur Agrawal, Kailash Gopalakrishnan, and Pritish Narayanan. Deep learning with lim- ited numerical precision. In ICML'2015, 2015.
- Matthieu Courbariaux, Yoshua Bengio, and Jean-Pierre David. Low precision arithmetic for deep learn- ing. In Arxiv:1412.7024, ICLR'2015 Workshop, 2015. · arXiv
- Thomas M Bartol, Cailey Bromer, Justin P Kinney, Michael A Chirillo, Jennifer N Bourne, Kristen M Harris, and Terrence J Sejnowski. Hippocampal spine head sizes are highly precise. bioRxiv, 2015.
- Alex Graves. Practical variational inference for neural networks. In J. Shawe-Taylor, R.S. Zemel, P.L. Bartlett, F. Pereira, and K.Q. Weinberger, editors, Advances in Neural Information Processing Systems 24, pages 2348-2356.
- Nitish Srivastava. Improving neural networks with dropout. Master's thesis, U. Toronto, 2013.
- Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15:1929-1958, 2014.
- Li Wan, Matthew Zeiler, Sixin Zhang, Yann LeCun, and Rob Fergus. Regularization of neural networks using dropconnect. In ICML'2013, 2013.
- J.P. David, K. Kalach, and N. Tittley. Hardware complexity of modular multiplication and exponentiation. Computers, IEEE Transactions on, 56(10):1308-1319, Oct 2007.
- R. Collobert. Large Scale Machine Learning. PhD thesis, Université de Paris VI, LIP6, 2004.
- X. Glorot, A. Bordes, and Y. Bengio. Deep sparse rectifier neural networks. In AISTATS'2011, 2011.
- Xavier Glorot and Yoshua Bengio. Understanding the difficulty of training deep feedforward neural networks. In AISTATS'2010, 2010.
- Ian J. Goodfellow, David Warde-Farley, Mehdi Mirza, Aaron Courville, and Yoshua Bengio. Maxout networks. Technical Report Arxiv report 1302.4389, Université de Montréal, February 2013.
- Yichuan Tang. Deep learning using linear support vector machines. Workshop on Challenges in Repre- sentation Learning, ICML, 2013.
- Min Lin, Qiang Chen, and Shuicheng Yan. Network in network. arXiv preprint arXiv:1312.4400, 2013. · arXiv
- Chen-Yu Lee, Saining Xie, Patrick Gallagher, Zhengyou Zhang, and Zhuowen Tu. Deeply-supervised nets. arXiv preprint arXiv:1409.5185, 2014. · arXiv
- Yann LeCun, Leon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278-2324, November 1998.
- V. Nair and G.E. Hinton. Rectified linear units improve restricted Boltzmann machines. In ICML'2010, 2010.
- Benjamin Graham. Spatially-sparse convolutional neural networks. arXiv preprint arXiv:1409.6070, 2014. · arXiv
- Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recog- nition. In ICLR, 2015.
- Daniel Soudry, Itay Hubara, and Ron Meir. Expectation backpropagation: Parameter-free training of multilayer neural networks with continuous or discrete weights. In NIPS'2014, 2014.
- Zhiyong Cheng, Daniel Soudry, Zexi Mao, and Zhenzhong Lan. Training binary multilayer neural net- works for image classification using expectation backpropgation. arXiv preprint arXiv:1503.03562, 2015. · arXiv
- Kyuyeon Hwang and Wonyong Sung. Fixed-point feedforward deep neural network design using weights+ 1, 0, and-1. In Signal Processing Systems (SiPS), 2014 IEEE Workshop on, pages 1-6. IEEE, 2014.
- Jonghong Kim, Kyuyeon Hwang, and Wonyong Sung. X1000 real-time phoneme recognition vlsi using feed-forward deep neural networks. In Acoustics, Speech and Signal Processing (ICASSP), 2014 IEEE International Conference on, pages 7510-7514. IEEE, 2014.
- Thomas P Minka. Expectation propagation for approximate bayesian inference. In UAI'2001, 2001.
- James Bergstra, Olivier Breuleux, Frédéric Bastien, Pascal Lamblin, Razvan Pascanu, Guillaume Des- jardins, Joseph Turian, David Warde-Farley, and Yoshua Bengio. Theano: a CPU and GPU math expres- sion compiler. In Proceedings of the Python for Scientific Computing Conference (SciPy), June 2010. Oral Presentation.
- Frédéric Bastien, Pascal Lamblin, Razvan Pascanu, James Bergstra, Ian J. Goodfellow, Arnaud Bergeron, Nicolas Bouchard, and Yoshua Bengio. Theano: new features and speed improvements. Deep Learning and Unsupervised Feature Learning NIPS 2012 Workshop, 2012.
- Ian J. Goodfellow, David Warde-Farley, Pascal Lamblin, Vincent Dumoulin, Mehdi Mirza, Razvan Pas- canu, James Bergstra, Frédéric Bastien, and Yoshua Bengio. Pylearn2: a machine learning research library. arXiv preprint arXiv:1308.4214, 2013. · arXiv
Cited by
Sorted by citations
Howard, A., Sandler, M., Chen, B., Wang, W., Chen, L.-C., Tan, M., Chu, G., Vasudevan, V., Zhu, Y., Pang, R., et al. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00140 · Cited by 9,708 · see citing works
“We replace sigmoid function with its piece-wise linear hard analog: ReLU6(x+3) 6 similar to [11, 44].”
BackgroundMethodAdvances and Open Problems in Federated Learning
Kairouz, P., & McMahan, H. B. (2021) · Foundations and Trends in Machine Learning · doi:10.1561/2200000083 · Cited by 7,772 · see citing works
Efficient Processing of Deep Neural Networks: A Tutorial and Survey
Sze, V., Chen, Y.-H., Yang, T.-J., & Emer, J. S. (2017) · Proceedings of the IEEE · doi:10.1109/jproc.2017.2761740 · Cited by 4,215 · see citing works
“Reduce Weight BinaryConnect [127] 1 32 (float) 19.”
XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks
Rastegari, M., Ordonez, V., Redmon, J., & Farhadi, A. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_32 · Cited by 4,408 · see citing works
Convergence of Edge Computing and Deep Learning: A Comprehensive Survey
Wang, X., Han, Y., Leung, V. C. M., Niyato, D., Yan, X., & Chen, X. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.2970550 · Cited by 1,498 · see citing works
“general neural network structures, existing optimization methods can be divided into four categories [128]: 1) parameter pruning and sharing [129], [130], including also weights quantization [131]–[133]; 2) low-rank factorization [124]; 3) transferred/compact convolution filters [107], [134], [135]; 4) knowledge disti…”
MethodNeuro-Inspired Computing With Emerging Nonvolatile Memorys
Yu, S. (2018) · Proceedings of the IEEE · doi:10.1109/jproc.2018.2790840 · Cited by 1,170 · see citing works
A Survey of Quantization Methods for Efficient Neural Network Inference
Gholami, A., Kim, S., Dong, Z., Yao, Z., Mahoney, M. W., & Keutzer, K. (2022) · Low-Power Computer Vision · doi:10.1201/9781003162810-13 · Cited by 1,219 · see citing works
“Hence, [42] extends this to where Binary is a function to binarize the real value x , and σ ( ) is the sigmoid function.”
A Survey of Model Compression and Acceleration for Deep Neural Networks
Cheng, Y., Wang, D., Zhou, P., & Zhang, T. (2017) · arXiv · doi:10.48550/arxiv.1710.09282 · Cited by 857 · see citing works
Distributed Deep Neural Networks Over the Cloud, the Edge and End Devices
Teerapittayanon, S., McDanel, B., & Kung, H. T. (2017) · 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS) · doi:10.1109/icdcs.2017.226 · Cited by 843 · see citing works
Designing Energy-Efficient Convolutional Neural Networks Using Energy-Aware Pruning
Yang, T.-J., Chen, Y.-H., & Sze, V. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.643 · Cited by 732 · see citing works
Li, F., Liu, B., Wang, X., Zhang, B., & Yan, J. (2016) · arXiv · doi:10.48550/arxiv.1605.04711 · Cited by 732 · see citing works
HAWQ: Hessian AWare Quantization of Neural Networks With Mixed-Precision
Dong, Z., Yao, Z., Gholami, A., Mahoney, M., & Keutzer, K. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00038 · Cited by 492 · see citing works
Communication-efficient edge ai: algorithms and systems
Shi, Y., Yang, K., Jiang, T., Zhang, J., & Letaief, K. B. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.3007787 · Cited by 488 · see citing works
Scaling for edge inference of deep neural networks
Xu, X., Ding, Y., Hu, S. X., Niemier, M., Cong, J., Hu, Y., & Shi, Y. (2018) · Nature Electronics · doi:10.1038/s41928-018-0059-3 · Cited by 462 · see citing works
Efficient Acceleration of Deep Learning Inference on Resource-Constrained Edge Devices: A Review
Shuvo, M. M. H., Islam, S. K., Cheng, J., & Morshed, B. I. (2023) · Proceedings of the IEEE · doi:10.1109/jproc.2022.3226481 · Cited by 436 · see citing works
Q-BERT: Hessian Based Ultra Low Precision Quantization of BERT
Shen, S., Dong, Z., Ye, J., Ma, L., Yao, Z., Gholami, A., Mahoney, M. W., & Keutzer, K. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6409 · Cited by 437 · see citing works
Biswas, A., & Chandrakasan, A. P. (2019) · IEEE Journal of Solid-State Circuits · doi:10.1109/jssc.2018.2880918 · Cited by 414 · see citing works
Low-bit Quantization of Neural Networks for Efficient Inference
Choukroun, Y., Kravchik, E., Yang, F., & Kisilev, P. (2019) · 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW) · doi:10.1109/iccvw.2019.00363 · Cited by 404 · see citing works
ZeroQ: A Novel Zero Shot Quantization Framework
Cai, Y., Yao, Z., Dong, Z., Gholami, A., Mahoney, M. W., & Keutzer, K. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01318 · Cited by 381 · see citing works
Hierarchical Multiscale Recurrent Neural Networks
Chung, J., Ahn, S., & Bengio, Y. (2016) · arXiv · doi:10.48550/arxiv.1609.01704 · Cited by 386 · see citing works
Self-citationLearning to Quantize Deep Networks by Optimizing Quantization Intervals With Task Loss
Jung, S., Son, C., Lee, S., Son, J., Han, J.-J., Kwak, Y., Hwang, S. J., & Choi, C. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00448 · Cited by 385 · see citing works
Bai, X., Wang, X., Liu, X., Liu, Q., Song, J., Sebe, N., & Kim, B. (2021) · Pattern Recognition · doi:10.1016/j.patcog.2021.108102 · Cited by 313 · see citing works
Binary neural network with 16 Mb RRAM macro chip for classification and online training
Yu, S., Li, Z., Chen, P.-Y., Wu, H., Gao, B., Wang, D., Wu, W., & Qian, H. (2016) · 2016 IEEE International Electron Devices Meeting (IEDM) · doi:10.1109/iedm.2016.7838429 · Cited by 219 · see citing works
AdderNet: Do We Really Need Multiplications in Deep Learning?
Chen, H., Wang, Y., Xu, C., Shi, B., Xu, C., Tian, Q., & Xu, C. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00154 · Cited by 199 · see citing works
A Study of BFLOAT16 for Deep Learning Training
Kalamkar, D., Mudigere, D., Mellempudi, N., Das, D., Banerjee, K., Avancha, S., Vooturi, D. T., Jammalamadaka, N., Huang, J., Yuen, H., et al. (2019) · arXiv · doi:10.48550/arxiv.1905.12322 · Cited by 169 · see citing works
“This idea is further advanced in [7] where the authors showcase training with only binary weights while keeping all other tensors and operations in full precision.”
BackgroundHigh-Performance Mixed-Signal Neurocomputing With Nanoscale Floating-Gate Memory Cell Arrays
Merrikh-Bayat, F., Guo, X., Klachko, M., Prezioso, M., Likharev, K. K., & Strukov, D. B. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2017.2778940 · Cited by 167 · see citing works
Ando, K., Ueyoshi, K., Orimo, K., Yonekawa, H., Sato, S., Nakahara, H., Takamaeda-Yamazaki, S., Ikebe, M., Asai, T., Kuroda, T., et al. (2018) · IEEE Journal of Solid-State Circuits · doi:10.1109/jssc.2017.2778702 · Cited by 160 · see citing works
Binary Ensemble Neural Network: More Bits per Network or More Networks per Bit?
Zhu, S., Dong, X., & Su, H. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00506 · Cited by 153 · see citing works
OpenEI: An Open Framework for Edge Intelligence
Zhang, X., Wang, Y., Lu, S., Liu, L., xu, L., & Shi, W. (2019) · 2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS) · doi:10.1109/icdcs.2019.00182 · Cited by 139 · see citing works
Resiliency of Deep Neural Networks under Quantization
Sung, W., Shin, S., & Hwang, K. (2015) · arXiv · doi:10.48550/arxiv.1511.06488 · Cited by 132 · see citing works
Mixed-Precision Deep Learning Based on Computational Memory
Nandakumar, S. R., Le Gallo, M., Piveteau, C., Joshi, V., Mariani, G., Boybat, I., Karunaratne, G., Khaddam-Aljameh, R., Egger, U., Petropoulos, A., et al. (2020) · Frontiers in Neuroscience · doi:10.3389/fnins.2020.00406 · Cited by 120 · see citing works
Xu, W., Fang, W., Ding, Y., Zou, M., & Xiong, N. (2021) · IEEE Access · doi:10.1109/access.2021.3063291 · Cited by 112 · see citing works
Li, Z., Sun, M., Lu, A., Ma, H., Yuan, G., Xie, Y., Tang, H., Li, Y., Leeser, M., Wang, Z., et al. (2022) · 2022 32nd International Conference on Field-Programmable Logic and Applications (FPL) · doi:10.1109/fpl57034.2022.00027 · Cited by 85 · see citing works
Network Sketching: Exploiting Binary Structure in Deep CNNs
Guo, Y., Yao, A., Zhao, H., & Chen, Y. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.430 · Cited by 81 · see citing works
A Survey of Deep Learning Applications to Autonomous Vehicle Control
Kuutti, S., Bowden, R., Jin, Y., Barber, P., & Fallah, S. (2021) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2019.2962338 · Cited by 96 · see citing works
An Energy-Efficient Digital ReRAM-Crossbar-Based CNN With Bitwise Parallelism
Ni, L., Liu, Z., Yu, H., & Joshi, R. V. (2017) · IEEE Journal on Exploratory Solid-State Computational Devices and Circuits · doi:10.1109/jxcdc.2017.2697910 · Cited by 74 · see citing works
Design of Ternary Neural Network With 3-D Vertical RRAM Array
Li, Z., Chen, P.-Y., Xu, H., & Yu, S. (2017) · IEEE Transactions on Electron Devices · doi:10.1109/ted.2017.2697361 · Cited by 70 · see citing works
AddNet: Deep Neural Networks Using FPGA-Optimized Multipliers
Faraone, J., Kumm, M., Hardieck, M., Zipf, P., Liu, X., Boland, D., & Leong, P. H. W. (2020) · IEEE Transactions on Very Large Scale Integration (VLSI) Systems · doi:10.1109/tvlsi.2019.2939429 · Cited by 66 · see citing works
John, R. A., Acharya, J., Zhu, C., Surendran, A., Bose, S. K., Chaturvedi, A., Tiwari, N., Gao, Y., He, Y., Zhang, K. K., et al. (2020) · Nature Communications · doi:10.1038/s41467-020-16985-0 · Cited by 68 · see citing works
A survey of model compression strategies for object detection
Lyu, Z., Yu, T., Pan, F., Zhang, Y., Luo, J., Zhang, D., Chen, Y., Zhang, B., & Li, G. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-17192-x · Cited by 65 · see citing works
Fog-Embedded Deep Learning for the Internet of Things
Lyu, L., Bezdek, J. C., He, X., & Jin, J. (2019) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2019.2912465 · Cited by 61 · see citing works
Bit Efficient Quantization for Deep Neural Networks
Nayak, P., Zhang, D., & Chai, S. (2019) · 2019 Fifth Workshop on Energy Efficient Machine Learning and Cognitive Computing - NeurIPS Edition (EMC2-NIPS) · doi:10.1109/emc2-nips53020.2019.00020 · Cited by 51 · see citing works
FORESEEN: Towards Differentially Private Deep Inference for Intelligent Internet of Things
Lyu, L., Bezdek, J. C., Jin, J., & Yang, Y. (2020) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2020.3000374 · Cited by 34 · see citing works
IF-CNN: Image-Aware Inference Framework for CNN With the Collaboration of Mobile Devices and Cloud
Shu, G., Liu, W., Zheng, X., & Li, J. (2018) · IEEE Access · doi:10.1109/access.2018.2880196 · Cited by 30 · see citing works
NAS4RRAM: neural network architecture search for inference on RRAM-based accelerators
Yuan, Z., Liu, J., Li, X., Yan, L., Chen, H., Wu, B., Yang, Y., & Sun, G. (2021) · Science China Information Sciences · doi:10.1007/s11432-020-3245-7 · Cited by 30 · see citing works
Diagonalwise Refactorization: An Efficient Training Method for Depthwise Convolutions
Qin, Z., Zhang, Z., Li, D., Zhang, Y., & Peng, Y. (2018) · 2018 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2018.8489312 · Cited by 27 · see citing works
FLightNNs: Lightweight Quantized Deep Neural Networks for Fast and Accurate Inference
Ding, R., Liu, Z., Chin, T.-W., Marculescu, D., & Blanton, R. D. (. (2019) · Proceedings of the 56th Annual Design Automation Conference 2019 · doi:10.1145/3316781.3317828 · Cited by 19 · see citing works
A Novel Mixed-Precision Quantization Approach for CNNs
Wu, D., Wang, Y., Fei, Y., & Gao, G. (2025) · IEEE Access · doi:10.1109/access.2025.3551802 · Cited by 8 · see citing works
Oflamaz, C. U., & Yalçın, M. E. (2025) · Electronics · doi:10.3390/electronics14183686 · Cited by 4 · see citing works
ASVD: Activation-aware Singular Value Decomposition for Compressing Large Language Models
Yuan, Z., Shang, Y., Song, Y., Yang, D., Wu, Q., Yan, Y., & Sun, G. (2023) · arXiv · doi:10.48550/arxiv.2312.05821 · Cited by 4 · see citing works
“These methods primarily fall into three categories: weight quantization [Courbariaux et al., 2015, Dettmers et al., 2022], network pruning [LeCun et al., 1989, Fran-tar & Alistarh, 2023], and knowledge distillation [Hinton et al., 2015, Agarwal et al., 2023].”
MethodLittleBit: Ultra Low-Bit Quantization via Latent Factorization
Lee, B., Kim, D., You, Y., & Kim, Y. (2025) · arXiv · doi:10.48550/arxiv.2506.13771
“Although early works like Bina-ryConnect [28] and BNNs [26] showed feasibility, they faced significant accuracy degradation [29].”
- Authors
- Courbariaux, M., Bengio, Y., & David, J.-P.
- Affiliations
- Polytechnique Montréal (CA)
- Canadian Institute for Advanced Research (CA)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2015
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1511.00363
- OpenAlex
- W1902934009
- Cited by
- 2,180
- References
- 2
- Citations by year
- 2026: 12 · 2025: 89 · 2024: 201 · 2023: 215 · 2022: 234 · 2021: 311 · 2020: 330 · 2019: 318 · 2018: 259 · 2017: 157 · 2016: 47 · 2015: 7
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- PolyPublie (École Polytechnique de Montréal) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Canadian Institute for Advanced Research
- Compute Canada
- Nuance Foundation
- Canada Research Chairs
- Natural Sciences and Engineering Research Council of Canada
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Advanced Memory and Neural Computing
- Keywords
- binary weights, binary neural networks, weight quantization, gradient accumulation, regularization, low-power electronics, deep neural network training, image classification
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
