The DOI matches its DataCite record; 18 of the 18 references it cites were found in a registry or an index.
Training deep neural networks with low precision multiplications
Open accessAbstract
Multipliers are the most space and power-hungry arithmetic operators of the digital implementation of deep neural networks. We train a set of state-of-the-art neural networks (Maxout networks) on three benchmark datasets: MNIST, CIFAR-10 and SVHN. They are trained with three distinct formats: floating point, fixed point and dynamic fixed point. For each of those datasets and for each of those formats, we assess the impact of the precision of the multiplications on the final error after training. We find that very low precision is sufficient not just for running trained networks but also for training them. For example, it is possible to train Maxout networks with 10 bits multiplications.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Performing the backward pass with floating point is important, as accumulating the gradients in quantized precision can result in zerogradient or gradients that have high error, especially in low-precision [42,80,81,107,159,186,204,231].”
From A Survey of Quantization Methods for Efficient Neural Network Inference (2022) · cites this paper as [42]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“An important work here is BinaryConnect [42] which constrains the weights to either +1 or -1.”
From A Survey of Quantization Methods for Efficient Neural Network Inference (2022) · cites this paper as [42]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Self-citationLearning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,889
Improving neural networks by preventing co-adaptation of feature detectors
Hinton, G. E., Srivastava, N., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. R. (2012) · arXiv · doi:10.48550/arxiv.1207.0580 · Cited by 6,382
Reading digits in natural images with unsupervised feature learning
Netzer, Y. (2024) · doi:10.57702/e7wwo0zk · Cited by 4,261
Large Scale Distributed Deep Networks
Dean, J. B., Corrado, G. S., Monga, R., Chen, K., Devin, M., Mao, M., Ranzato, M., Senior, A., Tucker, P. A., Yang, K., et al. (2012) · Cited by 3,202
What is the best multi-stage architecture for object recognition?
Jarrett, K., Kavukcuoglu, K., Ranzato, M. A., & LeCun, Y. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459469 · Cited by 2,153
DaDianNao: A Machine-Learning Supercomputer
Chen, Y., Luo, T., Liu, S., Zhang, S., He, L., Wang, J., Li, L., Chen, T., Xu, Z., Sun, N., et al. (2014) · IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1109/micro.2014.58 · Cited by 1,547
Stochastic Pooling for Regularization of Deep Convolutional Neural NetworksPreprint
Zeiler, M. D., & Fergus, R. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.3557 · Cited by 1,010
Deep Learning with Limited Numerical Precision
Gupta, S., Agrawal, A., Gopalakrishnan, K., & Narayanan, P. (2015) · Cited by 757
(2011) · Cited by 713
Research works printed but not linked: 13
- Bastien, F., Lamblin, P., Pascanu, R., Bergstra, J., Goodfellow, I. J., Bergeron, A., Bouchard, N., and Bengio, Y. (2012). Theano: new features and speed improvements. Deep Learning and Unsupervised Feature Learning NIPS 2012 Workshop.
- Bergstra, J., Breuleux, O., Bastien, F., Lamblin, P., Pascanu, R., Desjardins, G., Turian, J., WardeFarley, D., and Bengio, Y. (2010). Theano: a CPU and GPU math expression compiler. In Proceedings of the Python for Scientific Computing Conference (SciPy) . Oral Presentation.
- Chen, T., Du, Z., Sun, N., Wang, J., Wu, C., Chen, Y., and Temam, O. (2014a). Diannao: A smallfootprint high-throughput accelerator for ubiquitous machine-learning. In Proceedings of the 19th international conference on Architectural support for programming languages and operating systems , pages 269-284. ACM.
- Farabet, C., Martini, B., Corda, B., Akselrod, P., Culurciello, E., and LeCun, Y. (2011). Neuflow: A runtime reconfigurable dataflow processor for vision. In Computer Vision and Pattern Recognition Workshops (CVPRW), 2011 IEEE Computer Society Conference on , pages 109-116. IEEE.
- Glorot, X., Bordes, A., and Bengio, Y. (2011). Deep sparse rectifier neural networks. In AISTATS'2011 .
- Goodfellow, I. J., Warde-Farley, D., Mirza, M., Courville, A., and Bengio, Y. (2013a). Maxout networks. Technical Report Arxiv report 1302.4389, Universit´ e de Montr´ eal.
- Goodfellow, I. J., Warde-Farley, D., Lamblin, P., Dumoulin, V., Mirza, M., Pascanu, R., Bergstra, J., Bastien, F., and Bengio, Y. (2013b). Pylearn2: a machine learning research library. arXiv preprint arXiv:1308.4214 . · arXiv
- Holt, J. L. and Baker, T. E. (1991). Back propagation simulations using limited precision calculations. In Neural Networks, 1991., IJCNN-91-Seattle International Joint Conference on , volume 2, pages 121-126. IEEE.
- Krizhevsky, A., Sutskever, I., and Hinton, G. (2012a). ImageNet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems 25 (NIPS'2012) .
- Krizhevsky, A., Sutskever, I., and Hinton, G. (2012b). ImageNet classification with deep convolutional neural networks. In NIPS'2012 .
- Nair, V. and Hinton, G. (2010). Rectified linear units improve restricted Boltzmann machines. In ICML'2010 .
- Pham, P.-H., Jelaca, D., Farabet, C., Martini, B., LeCun, Y., and Culurciello, E. (2012). Neuflow: dataflow vision processing system-on-a-chip. In Circuits and Systems (MWSCAS), 2012 IEEE 55th International Midwest Symposium on , pages 1044-1047. IEEE.
- Williamson, D. (1991//). Dynamically scaled fixed point arithmetic. pages 315 - 18, New York, NY, USA. dynamic scaling;iteration stages;digital filters;overflow probability;fixed point arithmetic;fixed-point filter;.
Cited by
Sorted by citations
Opportunities and obstacles for deep learning in biology and medicine
Ching, T., Himmelstein, D. S., Beaulieu-Jones, B. K., Kalinin, A. A., Do, B. T., Way, G. P., Ferrero, E., Agapow, P.-M., Zietz, M., Hoffman, M. M., et al. (2018) · Journal of The Royal Society Interface · doi:10.1098/rsif.2017.0387 · Cited by 2,310
Quantized Neural Networks: Training Neural Networks with Low Precision Weights and ActivationsPreprint
Hubara, I., Courbariaux, M., Soudry, D., El‐Yaniv, R., & Bengio, Y. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1609.07061 · Cited by 1,663
Self-citationQuantized Convolutional Neural Networks for Mobile Devices
Wu, J., Leng, C., Wang, Y., Hu, Q., & Cheng, J. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.521 · Cited by 1,273
A Survey of Quantization Methods for Efficient Neural Network Inference
Gholami, A., Kim, S., Dong, Z., Yao, Z., Mahoney, M. W., & Keutzer, K. (2022) · Low-Power Computer Vision · doi:10.1201/9781003162810-13 · Cited by 1,219
Pruning and quantization for deep neural network acceleration: A survey
Liang, T., Glossner, J., Wang, L., Shi, S., & Zhang, X. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.07.045 · Cited by 1,054
Efficient and self-adaptive in-situ learning in multilayer memristor neural networks
Li, C., Belkin, D., Li, Y., Yan, P., Hu, M., Ge, N., Jiang, H., Montgomery, E., Lin, P., Wang, Z., et al. (2018) · Nature Communications · doi:10.1038/s41467-018-04484-2 · Cited by 931
NISP: Pruning Networks Using Neuron Importance Score Propagation
Yu, R., Li, A., Chen, C.-F., Lai, J.-H., Morariu, V. I., Han, X., Gao, M., Lin, C., & Davis, L. S. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00958 · Cited by 854
Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communications
Gündüz, D., Qin, Z., Aguerri, I. E., Dhillon, H. S., Yang, Z., Yener, A., Wong, K., & Chae, C. (2022) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2022.3223408 · Cited by 893
Deep Learning With Spiking Neurons: Opportunities and Challenges
Pfeiffer, M., & Pfeil, T. (2018) · Frontiers in Neuroscience · doi:10.3389/fnins.2018.00774 · Cited by 818
Scaling Neural Machine Translation
Ott, M., Edunov, S., Grangier, D., & Auli, M. (2018) · Conference on Machine Translation (WMT) · doi:10.18653/v1/w18-6301 · Cited by 631
Distributed Learning in Wireless Networks: Recent Progress and Future Challenges
Chen, M., Gündüz, D., Huang, K., Saad, W., Bennis, M., Feljan, A. V., & Poor, H. V. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3118346 · Cited by 577
A comprehensive survey on model compression and acceleration
Choudhary, T., Mishra, V. K., Goswami, A., & Jagannathan, S. (2020) · Artificial Intelligence Review · doi:10.1007/s10462-020-09816-7 · Cited by 555
Recent advances in convolutional neural network acceleration
Zhang, Q., Zhang, M., Chen, T., Sun, Z., Ma, Y., & Yu, B. (2018) · Neurocomputing · doi:10.1016/j.neucom.2018.09.038 · Cited by 458
Computer vision algorithms and hardware implementations: A survey
Feng, X., Jiang, Y., Yang, X., Du, M., & Li, X. S. (2019) · Integration · doi:10.1016/j.vlsi.2019.07.005 · Cited by 380
More is Less: A More Complicated Network with Less Inference Complexity
Dong, X., Huang, J., Yang, Y., & Yan, S. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.205 · Cited by 305
Jia, X., Song, S., He, W., Wang, Y., Rong, H., Zhou, F., Xie, L., Guo, Z., Yang, Y., Yu, L.-W., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.11205 · Cited by 398
Hardware-oriented Approximation of Convolutional Neural Networks
Gysel, P., Motamedi, M., & Ghiasi, S. (2016) · arXiv · doi:10.48550/arxiv.1604.03168 · Cited by 287
Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges
Arivazhagan, N., Bapna, A., Firat, O., Lepikhin, D., Johnson, M., Krikun, M., Chen, M. X., Cao, Y., Foster, G., Cherry, C., et al. (2019) · arXiv · doi:10.48550/arxiv.1907.05019 · Cited by 279
A CNN Accelerator on FPGA Using Depthwise Separable Convolution
Bai, L., Zhao, Y., & Huang, X. (2018) · IEEE Transactions on Circuits & Systems II Express Briefs · doi:10.1109/tcsii.2018.2865896 · Cited by 253
Integer Quantization for Deep Learning Inference: Principles and Empirical Evaluation
Wu, H., Judd, P., Zhang, X., Isaev, M., & Micikevicius, P. (2020) · arXiv · doi:10.48550/arxiv.2004.09602 · Cited by 246
Khanam, R., Hussain, M., Hill, R. C., & Allen, P. (2024) · IEEE Access · doi:10.1109/access.2024.3425166 · Cited by 296
A Review of Binarized Neural Networks
Simons, T., & Lee, D.-J. (2019) · Electronics · doi:10.3390/electronics8060661 · Cited by 217
Quantization and Deployment of Deep Neural Networks on Microcontrollers
Novac, P.-E., Hacene, G. B., Pégatoquet, A., Miramond, B., & Gripon, V. (2021) · Sensors · doi:10.3390/s21092984 · Cited by 250
Gysel, P., Pimentel, J., Motamedi, M., & Ghiasi, S. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2808319 · Cited by 250
Bulat, A., & Tzimiropoulos, G. (2017) · Nottingham ePrints (University of Nottingham) · Cited by 199
Dynamic energy-accuracy trade-off using stochastic computing in deep neural networks
Kim, K., Kim, J., Yu, J., Seo, J., Lee, J., & Choi, K. (2016) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1145/2897937.2898011 · Cited by 193
Yao, S., Zhao, Y., Zhang, A., Su, L., & Abdelzaher, T. (2017) · Proceedings of the 15th ACM Conference on Embedded Network Sensor Systems · doi:10.1145/3131672.3131675 · Cited by 188
Imani, M., Gupta, S., Kim, Y., & Rosing, T. (2019) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1145/3307650.3322237 · Cited by 188
Compression of Neural Machine Translation Models via Pruning
See, A., Luong, M.-T., & Manning, C. D. (2016) · Conference on Computational Natural Language Learning · doi:10.18653/v1/k16-1029 · Cited by 220
An Overview of Machine Learning within Embedded and Mobile Devices–Optimizations and Applications
Ajani, T. S., Imoize, A. L., & Atayero, A. A. A. (2021) · Sensors · doi:10.3390/s21134412 · Cited by 195
Making AI Forget You: Data Deletion in Machine Learning
Ginart, A., Guan, M. Y., Valiant, G., & Zou, J. (2019) · arXiv · doi:10.48550/arxiv.1907.05012 · Cited by 175
Origami: A 803-GOp/s/W Convolutional Network Accelerator
Cavigelli, L., & Luca Benini, C. (2016) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2016.2592330 · Cited by 169
A Study of BFLOAT16 for Deep Learning Training
Kalamkar, D., Mudigere, D., Mellempudi, N., Das, D., Banerjee, K., Avancha, S., Vooturi, D. T., Jammalamadaka, N., Huang, J., Yuen, H., et al. (2019) · arXiv · doi:10.48550/arxiv.1905.12322 · Cited by 169
AdaComp : Adaptive Residual Gradient Compression for Data-Parallel Distributed Training
Chen, C., Choi, J., Brand, D., Agrawal, A., Zhang, W., & Gopalakrishnan, K. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11728 · Cited by 176
XNOR-Net++: Improved Binary Neural Networks
Bulat, A., & Tzimiropoulos, G. (2019) · arXiv · doi:10.48550/arxiv.1909.13863 · Cited by 165
Ando, K., Ueyoshi, K., Orimo, K., Yonekawa, H., Sato, S., Nakahara, H., Takamaeda-Yamazaki, S., Ikebe, M., Asai, T., Kuroda, T., et al. (2018) · IEEE Journal of Solid-State Circuits · doi:10.1109/jssc.2017.2778702 · Cited by 160
A Survey on Coarse-Grained Reconfigurable Architectures From a Performance Perspective
Podobas, A., Sano, K., & Matsuoka, S. (2020) · IEEE Access · doi:10.1109/access.2020.3012084 · Cited by 171
Improving the Accuracy and Hardware Efficiency of Neural Networks Using Approximate Multipliers
Ansari, M. S., Mrázek, V., Cockburn, B. F., Sekanina, L., Vašíček, Z., & Han, J. (2019) · IEEE Transactions on Very Large Scale Integration (VLSI) Systems · doi:10.1109/tvlsi.2019.2940943 · Cited by 155
Edge-Cloud Polarization and Collaboration: A Comprehensive Survey for AI
Yao, J., Zhang, S., Yao, Y., Wang, F., Ma, J., Zhang, J., Chu, Y., Ji, L., Jia, K., Shen, T., et al. (2022) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2022.3178211 · Cited by 161
Efficient Spike-Driven Learning With Dendritic Event-Based Processing
Yang, S., Gao, T. Y., Wang, J., Deng, B., Lansdell, B., & Linares-Barranco, B. (2021) · Frontiers in Neuroscience · doi:10.3389/fnins.2021.601109 · Cited by 150
Yuval, J., O’Gorman, P. A., & Hill, C. (2021) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 141
Quantized CNN: A Unified Approach to Accelerate and Compress Convolutional Networks
Cheng, J., Wu, J., Leng, C., Wang, Y., & Hu, Q. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2017.2774288 · Cited by 126
Chang, J.-W., Kang, K.-W., & Kang, S. (2018) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2018.2888898 · Cited by 126
DSConv: Efficient Convolution Operator
do Nascimento, M. G., Prisacariu, V. A., & Fawcett, R. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00525 · Cited by 163
Characterizing the deployment of deep neural networks on commercial edge devices
Hadidi, R., Cao, J., Xie, Y., Asgari, B., Krishna, T., & Kim, H. (2019) · 2019 IEEE International Symposium on Workload Characterization (IISWC) · doi:10.1109/iiswc47752.2019.9041955 · Cited by 118
Adaptive parallel execution of deep neural networks on heterogeneous edge devices
Zhou, L., Samavatian, M. H., Bacha, A., Majumdar, S., & Teodorescu, R. (2019) · ACM/IEEE Symposium on Edge Computing (SEC) · doi:10.1145/3318216.3363312 · Cited by 117
Li, H., Hu, G., Li, J., & Zhou, M. (2021) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2020.3048056 · Cited by 121
Compressing recurrent neural network with tensor train
Tjandra, A., Sakti, S., & Nakamura, S. (2017) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2017.7966420 · Cited by 111
Toward Collaborative Inferencing of Deep Neural Networks on Internet-of-Things Devices
Hadidi, R., Cao, J., Ryoo, M. S., & Kim, H. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.2972000 · Cited by 131
Design Flow of Accelerating Hybrid Extremely Low Bit-Width Neural Network in Embedded FPGA
Wang, J., Lou, Q., Zhang, X., Zhu, C., Lin, Y., & Chen, D. (2018) · International Conference on Field Programmable Logic and Applications (FPL) · doi:10.1109/fpl.2018.00035 · Cited by 109
Next Point-of-Interest Recommendation on Resource-Constrained Mobile Devices
Wang, Q., Yin, H., Chen, T., Huang, Z. H., Wang, H., Zhao, Y., & Nguyen, Q. V. H. (2020) · International World Wide Web Conference (WWW) · doi:10.1145/3366423.3380170 · Cited by 106
A Survey of Methods for Low-Power Deep Learning and Computer Vision
Goel, A., Tung, C., Lu, Y.-H., & Thiruvathukal, G. K. (2020) · IEEE World Forum on Internet of Things (WF-IoT) · doi:10.1109/wf-iot48130.2020.9221198 · Cited by 116
Towards Unified Data and Lifecycle Management for Deep Learning
Miao, H., Li, A., Davis, L. S., & Deshpande, A. (2017) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde.2017.112 · Cited by 123
Understanding and Optimizing Asynchronous Low-Precision Stochastic Gradient Descent
De, C., Feldman, M., Ré, C., & Olukotun, K. (2017) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1145/3079856.3080248 · Cited by 108
Zhang, X., Sun, J., Wang, J., Jin, Y., Wang, L., & Liu, Z. (2023) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2023.109605 · Cited by 98
Lee, Y.-L., Tsung, P.-K., & Wu, M. (2018) · International Symposium on VLSI Design, Automation and Test · doi:10.1109/vlsi-dat.2018.8373244 · Cited by 99
Fast and Memory-Efficient Neural Code Completion
Svyatkovskiy, A., Lee, S., Hadjitofi, A., Riechert, M., Franco, J. V., & Allamanis, M. (2021) · 2021 IEEE/ACM 18th International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr52588.2021.00045 · Cited by 95
Universal Deep Neural Network Compression
Choi, Y., El‐Khamy, M., & Lee, J. (2020) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2020.2975903 · Cited by 95
What Do Compressed Deep Neural Networks Forget?Preprint
Hooker, S., Courville, A. C., Clark, G., Dauphin, Y., & Frome, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.05248 · Cited by 233
BinFI: an efficient fault injector for safety-critical machine learning systems
Chen, Z., Li, G., Pattabiraman, K., & DeBardeleben, N. (2019) · Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis · doi:10.1145/3295500.3356177 · Cited by 91
Sensitive-Sample Fingerprinting of Deep Neural Networks
He, Z., Zhang, T., & Lee, R. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00486 · Cited by 90
Tiny Machine Learning for Resource-Constrained Microcontrollers
Immonen, R., & Hämäläinen, T. (2022) · Journal of Sensors · doi:10.1155/2022/7437023 · Cited by 85
GrapHD: Graph-Based Hyperdimensional Memorization for Brain-Like Cognitive Learning
Poduval, P., Alimohamadi, H., Zakeri, A., Imani, F., Najafi, M. H., Givargis, T. D., & Imani, M. (2022) · Frontiers in Neuroscience · doi:10.3389/fnins.2022.757125 · Cited by 82
Deep Model Compression and Architecture Optimization for Embedded Systems: A Survey
Berthelier, A., Château, T., Duffner, S., García, C., & Blanc, C. (2020) · Journal of Signal Processing Systems · doi:10.1007/s11265-020-01596-1 · Cited by 86
Performance Evaluation of cuDNN Convolution Algorithms on NVIDIA Volta GPUs
Jordà, M., Valero‐Lara, P., & Peña, A. J. (2019) · IEEE Access · doi:10.1109/access.2019.2918851 · Cited by 82
Topology of deep neural networksPreprint
Naitzat, G., Zhitnikov, A., & Lim, L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.06093 · Cited by 92
Two-Way Transpose Multibit 6T SRAM Computing-in-Memory Macro for Inference-Training AI Edge Chips
Su, J.-W., Si, X., Chou, Y.-C., Chang, T.-W., Huang, W.-H., Tu, Y.-N., Liu, R., Lu, P.-J., Liu, T.-W., Wang, J., et al. (2021) · IEEE Journal of Solid-State Circuits · doi:10.1109/jssc.2021.3108344 · Cited by 74
Scaling deep learning on GPU and knights landing clusters
You, Y., Buluç, A., & Demmel, J. W. (2017) · International Conference for High Performance Computing, Networking, Storage and Analysis (SC) · doi:10.1145/3126908.3126912 · Cited by 74
FPGA-Accelerated Dense Linear Machine Learning: A Precision-Convergence Trade-Off
Kara, K., Alistarh, D., Alonso, G., Mutlu, O., & Zhang, C. (2017) · 2017 IEEE 25th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM) · doi:10.1109/fccm.2017.39 · Cited by 77
FAST: DNN Training Under Variable Precision Block Floating Point with Stochastic Rounding
Zhang, S. Q., McDanel, B., & Kung, H. T. (2022) · IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca53966.2022.00067 · Cited by 82
Exploiting approximate computing for deep learning acceleration
Chen, C., Choi, J., Gopalakrishnan, K., Srinivasan, V., & Venkataramani, S. (2018) · Design, Automation, and Test in Europe · doi:10.23919/date.2018.8342119 · Cited by 67
Embracing Diversity: Enhanced DSP Blocks for Low-Precision Deep Learning on FPGAs
Boutros, A., Yazdanshenas, S., & Betz, V. (2018) · International Conference on Field Programmable Logic and Applications (FPL) · doi:10.1109/fpl.2018.00014 · Cited by 67
The cascading neural network: building the Internet of Smart Things
Leroux, S., Bohez, S., De Coninck, E., Verbelen, T., Vankeirsbilck, B., Simoens, P., & Dhoedt, B. (2017) · Knowledge and Information Systems · doi:10.1007/s10115-017-1029-1 · Cited by 59
Yuan, G., Ma, X., Ding, C., Lin, S., Zhang, T., Jalali, Z. S., Zhao, Y., Jiang, L., Soundarajan, S., & Wang, Y. (2019) · International Symposium on Low Power Electronics and Design (ISLPED) · doi:10.1109/islped.2019.8824944 · Cited by 63
A Survey on Approximate Multiplier Designs for Energy Efficiency: From Algorithms to Circuits
Wu, Y., Chen, C., Xiao, W., Wang, X., Wen, C., Han, J., Yin, X., Qian, W., & Zhuo, C. (2023) · ACM Transactions on Design Automation of Electronic Systems · doi:10.1145/3610291 · Cited by 76
Kurth, T., Zhang, J., Nadathur, S., Racah, E., Mitliagkas, I., Patwary, M. M. A., Malas, T. B., Sundaram, N., Bhimji, W., Smorkalov, M. E., et al. (2017) · International Conference for High Performance Computing, Networking, Storage and Analysis (SC) · doi:10.1145/3126908.3126916 · Cited by 82
Are CNNs Reliable Enough for Critical Applications? An Exploratory Study
Neggaz, M. A., Alouani, I., Niar, S., & Kurdahi, F. J. (2019) · IEEE Design and Test · doi:10.1109/mdat.2019.2952336 · Cited by 60
Accelerating Convolutional Neural Networks via Activation Map Compression
Georgiadis, G. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00725 · Cited by 61
Tensor Decomposition for Compressing Recurrent Neural Network
Tjandra, A., Sakti, S., & Nakamura, S. (2018) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2018.8489213 · Cited by 58
IMCE: Energy-efficient bit-wise in-memory convolution engine for deep neural network
Angizi, S., He, Z., Parveen, F., & Fan, D. (2018) · Asia and South Pacific Design Automation Conference (ASP-DAC) · doi:10.1109/aspdac.2018.8297291 · Cited by 56
Shafique, M., Hafiz, R., Javed, M. U., Abbas, S., Sekanina, L., Vašíček, Z., & Mrázek, V. (2017) · IEEE Computer Society Annual Symposium on VLSI · doi:10.1109/isvlsi.2017.124 · Cited by 55
Nguyen, D. T., Kim, H., Lee, H., & Chang, I. (2018) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas.2018.8351021 · Cited by 53
Resource-Efficient Neural Networks for Embedded SystemsPreprint
Roth, W., Schindler, G., Klein, B., Peharz, R., Tschiatschek, S., Fröning, H., Pernkopf, F., & Ghahramani, Z. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.03048 · Cited by 67
On the Design of Logarithmic Multiplier Using Radix-4 Booth Encoding
Pilipović, R., & Bulić, P. (2020) · IEEE Access · doi:10.1109/access.2020.2985345 · Cited by 51
ShiftCNN: Generalized Low-Precision Architecture for Inference of Convolutional Neural NetworksPreprint
Gudovskiy, D., & Rigazio, L. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.02393 · Cited by 51
An Approximate Memory Architecture for Energy Saving in Deep Learning Applications
Nguyen, D. T., Hung, N. H., Kim, H., & Lee, H. (2020) · IEEE Transactions on Circuits and Systems I Regular Papers · doi:10.1109/tcsi.2019.2962516 · Cited by 50
Chen, C., Qian, W., Imani, M., Yin, X., & Zhuo, C. (2021) · IEEE Transactions on Computers · doi:10.1109/tc.2021.3131850 · Cited by 50
8-bit Optimizers via Block-wise QuantizationPreprint
Dettmers, T., Lewis, M., Shleifer, S., & Zettlemoyer, L. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.02861 · Cited by 278
EDR-Net: Lightweight Deep Neural Network Architecture for Detecting Referable Diabetic Retinopathy
Aujih, A. B., Shapiai, M. I., Mériaudeau, F., & Tang, T. B. (2022) · IEEE Transactions on Biomedical Circuits and Systems · doi:10.1109/tbcas.2022.3182907 · Cited by 49
DSConv: Efficient Convolution OperatorPreprint
Gennari, M., Fawcett, R., & Prisacariu, V. A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.01928 · Cited by 47
Memristive Quantized Neural Networks: A Novel Approach to Accelerate Deep Learning On-Chip
Zhang, Y., Cui, M., Shen, L., & Zeng, Z. (2019) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2019.2912205 · Cited by 46
JPEG-ACT: Accelerating Deep Learning via Transform-based Lossy Compression
Evans, R. D., Liu, L., & Aamodt, T. M. (2020) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1109/isca45697.2020.00075 · Cited by 46
Effective Quantization Approaches for Recurrent Neural Networks
Alom, M. Z., Moody, A. T., Maruyama, N., Essen, B. C. V., & Taha, T. M. (2018) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2018.8489341 · Cited by 49
Efficient Layout Hotspot Detection via Binarized Residual Neural Network Ensemble
Jiang, Y., Yang, F., Yu, B., Zhou, D., & Zeng, X. (2020) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2020.3015918 · Cited by 45
RegHD: Robust and Efficient Regression in Hyper-Dimensional Learning System
Hernández-Cano, A., Zhuo, C., Yin, X., & Imani, M. (2021) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1109/dac18074.2021.9586284 · Cited by 43
Lee, S., & Nirjon, S. (2019) · ACM Conference on Embedded Networked Sensor Systems (SenSys) · doi:10.1145/3356250.3360030 · Cited by 45
A Survey of Numerical Methods Utilizing Mixed Precision ArithmeticPreprint
Abdelfattah, A., Anzt, H., Boman, E. G., Carson, E., Cojean, T., Dongarra, J. J., Gates, M., Grützmacher, T., Higham, N. J., Li, X. S., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.06674 · Cited by 40
DAQ: Channel-Wise Distribution-Aware Quantization for Deep Image Super-Resolution Networks
Hong, C., Kim, H., Baik, S., Oh, J., & Lee, K. M. (2022) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv51458.2022.00099 · Cited by 39
Weighted Quantization-Regularization in DNNs for Weight Memory Minimization Toward HW Implementation
Wess, M., Dinakarrao, S. M. P., & Jantsch, A. (2018) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2018.2857080 · Cited by 38
NITI: Training Integer Neural Networks Using Integer-Only Arithmetic
Wang, M., Rasoulinezhad, S. A., Leong, P. H. W., & So, H. K. (2022) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2022.3149787 · Cited by 54
Metadata
- Authors
- Courbariaux, M., Bengio, Y., & David, J.-P.
- Affiliations
- Polytechnique Montréal (CA)
- Université de Montréal (CA)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2014
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1412.7024
- OpenAlex
- W1825672851
- Cited by
- 407
- References
- 18
- Citations by year
- 2025: 12 · 2024: 17 · 2023: 23 · 2022: 30 · 2021: 65 · 2020: 64 · 2019: 80 · 2018: 62 · 2017: 36 · 2016: 15 · 2015: 2 · 2014: 1
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- PolyPublie (École Polytechnique de Montréal) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Canadian Institute for Advanced Research
- Compute Canada
- Canada Research Chairs
- Natural Sciences and Engineering Research Council of Canada
- Topics
- Physical Sciences › Computer Science › Computational Theory and Mathematics › Numerical Methods and Algorithms
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Low-power high-performance VLSI design
- Keywords
- low-precision arithmetic, deep neural network training, fixed-point arithmetic, floating-point precision, energy-efficient neural networks, MNIST, CIFAR-10, SVHN
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
