The DOI matches its DataCite record; 18 of the 18 references it cites were found in a registry or an index.
Training deep neural networks with low precision multiplications
Open accessAbstract
Multipliers are the most space and power-hungry arithmetic operators of the digital implementation of deep neural networks. We train a set of state-of-the-art neural networks (Maxout networks) on three benchmark datasets: MNIST, CIFAR-10 and SVHN. They are trained with three distinct formats: floating point, fixed point and dynamic fixed point. For each of those datasets and for each of those formats, we assess the impact of the precision of the multiplications on the final error after training. We find that very low precision is sufficient not just for running trained networks but also for training them. For example, it is possible to train Maxout networks with 10 bits multiplications.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Performing the backward pass with floating point is important, as accumulating the gradients in quantized precision can result in zerogradient or gradients that have high error, especially in low-precision [42,80,81,107,159,186,204,231].”
From A Survey of Quantization Methods for Efficient Neural Network Inference (2022) · cites this paper as [42]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“An important work here is BinaryConnect [42] which constrains the weights to either +1 or -1.”
From A Survey of Quantization Methods for Efficient Neural Network Inference (2022) · cites this paper as [42]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Self-citationLearning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,889
Improving neural networks by preventing co-adaptation of feature detectors
Hinton, G. E., Srivastava, N., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. R. (2012) · arXiv · doi:10.48550/arxiv.1207.0580 · Cited by 6,382
Reading digits in natural images with unsupervised feature learning
Netzer, Y. (2024) · doi:10.57702/e7wwo0zk · Cited by 4,261
Large Scale Distributed Deep Networks
Dean, J. B., Corrado, G. S., Monga, R., Chen, K., Devin, M., Mao, M., Ranzato, M., Senior, A., Tucker, P. A., Yang, K., et al. (2012) · Cited by 3,202
What is the best multi-stage architecture for object recognition?
Jarrett, K., Kavukcuoglu, K., Ranzato, M. A., & LeCun, Y. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459469 · Cited by 2,153
DaDianNao: A Machine-Learning Supercomputer
Chen, Y., Luo, T., Liu, S., Zhang, S., He, L., Wang, J., Li, L., Chen, T., Xu, Z., Sun, N., et al. (2014) · IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1109/micro.2014.58 · Cited by 1,547
Stochastic Pooling for Regularization of Deep Convolutional Neural NetworksPreprint
Zeiler, M. D., & Fergus, R. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.3557 · Cited by 1,010
Deep Learning with Limited Numerical Precision
Gupta, S., Agrawal, A., Gopalakrishnan, K., & Narayanan, P. (2015) · Cited by 757
(2011) · Cited by 713
Research works printed but not linked: 13
- Bastien, F., Lamblin, P., Pascanu, R., Bergstra, J., Goodfellow, I. J., Bergeron, A., Bouchard, N., and Bengio, Y. (2012). Theano: new features and speed improvements. Deep Learning and Unsupervised Feature Learning NIPS 2012 Workshop.
- Bergstra, J., Breuleux, O., Bastien, F., Lamblin, P., Pascanu, R., Desjardins, G., Turian, J., WardeFarley, D., and Bengio, Y. (2010). Theano: a CPU and GPU math expression compiler. In Proceedings of the Python for Scientific Computing Conference (SciPy) . Oral Presentation.
- Chen, T., Du, Z., Sun, N., Wang, J., Wu, C., Chen, Y., and Temam, O. (2014a). Diannao: A smallfootprint high-throughput accelerator for ubiquitous machine-learning. In Proceedings of the 19th international conference on Architectural support for programming languages and operating systems , pages 269-284. ACM.
- Farabet, C., Martini, B., Corda, B., Akselrod, P., Culurciello, E., and LeCun, Y. (2011). Neuflow: A runtime reconfigurable dataflow processor for vision. In Computer Vision and Pattern Recognition Workshops (CVPRW), 2011 IEEE Computer Society Conference on , pages 109-116. IEEE.
- Glorot, X., Bordes, A., and Bengio, Y. (2011). Deep sparse rectifier neural networks. In AISTATS'2011 .
- Goodfellow, I. J., Warde-Farley, D., Mirza, M., Courville, A., and Bengio, Y. (2013a). Maxout networks. Technical Report Arxiv report 1302.4389, Universit´ e de Montr´ eal.
- Goodfellow, I. J., Warde-Farley, D., Lamblin, P., Dumoulin, V., Mirza, M., Pascanu, R., Bergstra, J., Bastien, F., and Bengio, Y. (2013b). Pylearn2: a machine learning research library. arXiv preprint arXiv:1308.4214 . · arXiv
- Holt, J. L. and Baker, T. E. (1991). Back propagation simulations using limited precision calculations. In Neural Networks, 1991., IJCNN-91-Seattle International Joint Conference on , volume 2, pages 121-126. IEEE.
- Krizhevsky, A., Sutskever, I., and Hinton, G. (2012a). ImageNet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems 25 (NIPS'2012) .
- Krizhevsky, A., Sutskever, I., and Hinton, G. (2012b). ImageNet classification with deep convolutional neural networks. In NIPS'2012 .
- Nair, V. and Hinton, G. (2010). Rectified linear units improve restricted Boltzmann machines. In ICML'2010 .
- Pham, P.-H., Jelaca, D., Farabet, C., Martini, B., LeCun, Y., and Culurciello, E. (2012). Neuflow: dataflow vision processing system-on-a-chip. In Circuits and Systems (MWSCAS), 2012 IEEE 55th International Midwest Symposium on , pages 1044-1047. IEEE.
- Williamson, D. (1991//). Dynamically scaled fixed point arithmetic. pages 315 - 18, New York, NY, USA. dynamic scaling;iteration stages;digital filters;overflow probability;fixed point arithmetic;fixed-point filter;.
Cited by
Sorted by citations
Items 101–200 of 410
Hierarchical Binary CNNs for Landmark Localization with Limited Resources
Bulat, A., & Tzimiropoulos, G. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2866051 · Cited by 36
FPGA-Based Hybrid-Type Implementation of Quantized Neural Networks for Remote Sensing Applications
Wei, X. S., Liu, W., Chen, L., Ma, L., Chen, H., & Zhuang, Y. (2019) · Sensors · doi:10.3390/s19040924 · Cited by 39
Faster First-Order Methods for Stochastic Non-Convex Optimization on Riemannian Manifolds
Zhou, P., Yuan, X., Yan, S., & Feng, J. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2933841 · Cited by 35
Efficient Layout Hotspot Detection via Binarized Residual Neural Network
Jiang, Y., Yang, F., Zhu, H., Yu, B., Zhou, D., & Zeng, X. (2019) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1145/3316781.3317811 · Cited by 35
Automatic Generation of Multi-Precision Multi-Arithmetic CNN Accelerators for FPGAs
Zhao, Y., Gao, X., Guo, X., Liu, J., Wang, E., Mullins, R., Cheung, P. Y. K., Constantinides, G. A., & Xu, C. (2019) · International Conference on Field-Programmable Technology (FPT) · doi:10.1109/icfpt47387.2019.00014 · Cited by 37
Rotation Invariant Local Binary Convolution Neural Networks
Zhang, X., Liu, L., Xie, Y., Chen, J., Wu, L., & Pietikäinen, M. (2017) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2017.146 · Cited by 33
Improved training of binary networks for human pose estimation and image recognition
Bulat, A., Tzimiropoulos, G., Kossaifi, J., & Pantic, M. (2019) · arXiv · doi:10.48550/arxiv.1904.05868 · Cited by 34
Cascaded Projection: End-To-End Network Compression and Acceleration
Minnehan, B., & Savakis, A. E. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01097 · Cited by 33
Improving Extreme Low-Bit Quantization With Soft Threshold
Xu, W., Li, F., Jiang, Y., Yong, A., He, X., Wang, P., & Cheng, J. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3216389 · Cited by 32
On the Design of Iterative Approximate Floating-Point Multipliers
Towhidy, A., Omidi, R., & Mohammadi, K. (2022) · IEEE Transactions on Computers · doi:10.1109/tc.2022.3216465 · Cited by 31
An Overview of Neural Network CompressionPreprint
O’Neill, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.03669 · Cited by 149
Why TanH is a Hardware Friendly Activation Function for CNNs
Abdelouahab, K., Pelcat, M., & Berry, F. (2017) · International Conference on Distributed Smart Cameras · doi:10.1145/3131885.3131937 · Cited by 32
Na, T., & Mukhopadhyay, S. (2016) · International Symposium on Low Power Electronics and Design (ISLPED) · doi:10.1145/2934583.2934625 · Cited by 30
Challenges in Energy-Efficient Deep Neural Network Training with FPGA
Tao, Y., Ma, R., Shyu, M., & Chen, S. (2020) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00208 · Cited by 29
Motamedi, M., Gysel, P., & Ghiasi, S. (2017) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3131289 · Cited by 28
Alrawashdeh, K., & Purdy, C. (2017) · IEEE National Aerospace and Electronics Conference (NAECON) · doi:10.1109/naecon.2017.8268745 · Cited by 28
Exploring the Vision Processing Unit as Co-Processor for Inference
Rivas-Gomez, S., Peña, A. J., Moloney, D., Laure, E., & Markidis, S. (2018) · IEEE International Symposium on Parallel & Distributed Processing, Workshops and Phd Forum · doi:10.1109/ipdpsw.2018.00098 · Cited by 30
Yu, Y., & Jha, N. K. (2020) · IEEE Transactions on Emerging Topics in Computing · doi:10.1109/tetc.2020.3003328 · Cited by 31
Ferrarini, B., Milford, M. J., McDonald-Maier, K. D., & Ehsan, S. (2022) · IEEE Transactions on Robotics · doi:10.1109/tro.2022.3148908 · Cited by 27
Optimally approximated and unbiased floating-point multiplier with runtime configurability
Chen, C., Yang, S., Qian, W., Imani, M., Yin, X., & Zhuo, C. (2020) · doi:10.1145/3400302.3415702 · Cited by 27
Replacing Rules by Neural Networks A Framework for Agent-Based Modelling
Jäger, G. (2019) · Big Data and Cognitive Computing · doi:10.3390/bdcc3040051 · Cited by 30
Fine-grained energy profiling for deep convolutional neural networks on the Jetson TX1
Rodrigues, C. F., Riley, G. D., & Luján, M. (2017) · IEEE International Symposium on Workload Characterization · doi:10.1109/iiswc.2017.8167764 · Cited by 26
Training a Binary Weight Object Detector by Knowledge Transfer for Autonomous Driving
Xu, J., Nie, Y., Wang, P., & López, A. M. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8793743 · Cited by 25
Stage-Wise Magnitude-Based Pruning for Recurrent Neural Networks
Li, G., Yang, P., Qian, C., Hong, R., & Tang, K. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3184730 · Cited by 25
Manzoor, S., Kim, E., Joo, S.-H., Bae, S.-H., In, G.-G., Joo, K.-J., Choi, J.-H., & Kuc, T. (2022) · IEEE Access · doi:10.1109/access.2022.3190538 · Cited by 25
Wagner, K. H., & McComb, S. (2019) · IEEE Journal of Selected Topics in Quantum Electronics · doi:10.1109/jstqe.2019.2946655 · Cited by 25
Learning Low Precision Deep Neural Networks through Regularization.Preprint
Choi, Y., El‐Khamy, M., & Lee, J. (2018) · arXiv (Cornell University) · Cited by 25
An Optimized Recurrent Unit for Ultra-Low-Power Keyword Spotting
Amoh, J., & Odame, K. M. (2019) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3328907 · Cited by 28
Auth+Track: Enabling Authentication Free Interaction on Smartphone by Continuous User Tracking
Liang, C., Yu, C., Wei, X., Xu, X., Hu, Y., Wang, Y., & Shi, Y. (2021) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445624 · Cited by 24
The Low-Resource Double Bind: An Empirical Study of Pruning for Low-Resource Machine Translation
Ahia, O., Kreutzer, J., & Hooker, S. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.282 · Cited by 41
A portable, automatic data qantizer for deep neural networks
Oh, Y. H., Quan, Q., Kim, D.-Y., Kim, S., Heo, J., Jung, S., Jang, J.-Y., & Lee, J. W. (2018) · International Conference on Parallel Architectures and Compilation Techniques (PACT) · doi:10.1145/3243176.3243180 · Cited by 24
Ransomware Detection Using Limited Precision Deep Learning Structure in FPGA
Alrawashdeh, K., & Purdy, C. (2018) · IEEE National Aerospace and Electronics Conference (NAECON) · doi:10.1109/naecon.2018.8556824 · Cited by 24
Deep Learning at 15PF: Supervised and Semi-Supervised Classification for Scientific DataPreprint
Kurth, T., Zhang, J., Nadathur, S., Mitliagkas, I., Racah, E., Patwary, M. A., Malas, T. B., Sundaram, N., Bhimji, W., Smorkalov, M. E., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.05256 · Cited by 24
Lee, H., Kim, M., Min, D., Kim, J., Back, J., Yoo, H.-N., Lee, J., & Kim, J. (2022) · IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1109/micro56248.2022.00093 · Cited by 26
Gradient Descent Learning With Floats
Sun, T., Tang, K., & Li, D. (2020) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2020.2997399 · Cited by 25
Adaptive and Resilient Model-Distributed Inference in Edge Computing Systems
Li, P., Koyuncu, E., & Seferoğlu, H. (2023) · IEEE Open Journal of the Communications Society · doi:10.1109/ojcoms.2023.3280174 · Cited by 26
Mixed-precision quantization-aware training for photonic neural networks
Kirtas, M., Passalis, N., Oikonomou, A., Moralis-Pegios, M., Giamougiannis, G., Tsakyridis, A., Mourgias-Alexandris, G., Pleros, N., & Tefas, A. (2023) · Neural Computing and Applications · doi:10.1007/s00521-023-08848-8 · Cited by 25
A Fixed-Point Quantization Technique for Convolutional Neural Networks Based on Weight Scaling
Mitschke, N., Heizmann, M., Noffz, K.-H., & Wittmann, R. (2019) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2019.8803490 · Cited by 23
Cheng, T., Masuda, Y., Chen, J., Yu, J., & Hashimoto, M. (2020) · Integration · doi:10.1016/j.vlsi.2020.05.002 · Cited by 23
PruNet: Class-Blind Pruning Method For Deep Neural Networks
Marchisio, A., Hanif, M. A., Martina, M., & Shafique, M. (2018) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2018.8489764 · Cited by 21
A Review of FPGA‐Based Custom Computing Architecture for Convolutional Neural Network Inference
Xiyuan, P., Yu, J., Yao, B., Liansheng, L., & Peng, Y. (2021) · Chinese Journal of Electronics · doi:10.1049/cje.2020.11.002 · Cited by 23
High-Performance Acceleration of 2-D and 3-D CNNs on FPGAs Using Static Block Floating Point
Fan, H., Liu, S., Que, Z., Niu, X., & Luk, W. W. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3116302 · Cited by 23
Flexpoint: Predictive Numerics for Deep Learning
Popescu, V., Nassar, M., Wang, X., Tumer, E. C., & Webb, T. J. (2018) · IEEE Symposium on Computer Arithmetic (ARITH) · doi:10.1109/arith.2018.8464801 · Cited by 22
Design of Processing-“Inside”-Memory Optimized for DRAM Behaviors
Lee, W. J., Kim, C. H., Paik, Y., Park, J., Park, I., & Kim, S. W. (2019) · IEEE Access · doi:10.1109/access.2019.2924240 · Cited by 22
Goyal, R., Vanschoren, J., van Acht, V., & Nijssen, S. (2021) · TU/e Research Portal · doi:10.48550/arxiv.2102.02147 · Cited by 31
He, X. J., Lü, W., Yan, G., & Zhang, X. (2018) · IEEE Journal on Emerging and Selected Topics in Circuits and Systems · doi:10.1109/jetcas.2018.2845396 · Cited by 21
Convolution Accelerator Designs Using Fast Algorithms
Zhao, Y., Wang, D., & Wang, L. (2019) · Algorithms · doi:10.3390/a12050112 · Cited by 22
Quantizing Convolutional Neural Networks for Low-Power High-Throughput Inference EnginesPreprint
Settle, S. O., Bollavaram, M., D’Alberto, P., Delaye, E., Fernandez, O., Fraser, N. J., Ng, A. N., Sirasao, A., & Wu, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.07941 · Cited by 25
Attention-Based Neural Networks for Chroma Intra Prediction in Video Coding
Blanch, M. G., Blasi, S., Smeaton, A. F., O'Connor, N. E., & Mrak, M. (2020) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2020.3044482 · Cited by 18
Sateesan, A., Sinha, S., Smitha, K. G., & Vinod, A. P. (2021) · Neural Processing Letters · doi:10.1007/s11063-021-10458-1 · Cited by 21
Combining Arithmetic Approximation Techniques for Improved CNN Circuit Design
Lentaris, G., Chatzitsompanis, G., Leon, V., Pekmestzi, K., & Soudris, D. (2020) · IEEE International Conference on Electronics, Circuits and Systems (ICECS) · doi:10.1109/icecs49266.2020.9294869 · Cited by 18
Xiao, P., Zhang, C., Guo, Q. J., Xiao, X., & Wang, H. (2024) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2024.3452321 · Cited by 20
Collaborative Execution of Deep Neural Networks on Internet of Things DevicesPreprint
Hadidi, R., Cao, J., Ryoo, M. S., & Kim, H. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.02537 · Cited by 18
Mirage: An RNS-Based Photonic Accelerator for DNN Training
Demirkiran, C., Yang, G., Bunandar, D., & Joshi, A. (2024) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1109/isca59077.2024.00016 · Cited by 18
Quantization of deep neural networks for accumulator-constrained processors
de Bruin, B., Živković, Z., & Corporaal, H. (2019) · Microprocessors and Microsystems · doi:10.1016/j.micpro.2019.102872 · Cited by 17
Resource efficient activation functions for neural network accelerators
Wuraola, A., & Patel, N. D. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.11.032 · Cited by 18
FPChecker: Detecting Floating-Point Exceptions in GPU Applications
Laguna, I. (2019) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2019.00118 · Cited by 17
Towards secure and practical machine learning via secret sharing and random permutation
Zheng, F., Chen, C., Zheng, X., & Zhu, M. (2022) · Knowledge-Based Systems · doi:10.1016/j.knosys.2022.108609 · Cited by 16
Jiang, S., Peng, L., Li, L., Dai, Q., Pei, M., Wu, C., Su, J., Gu, D., Zhang, H., Guo, H., et al. (2024) · The Journal of Physical Chemistry Letters · doi:10.1021/acs.jpclett.4c00284 · Cited by 15
Timed Dataflow: Reducing Communication Overhead for Distributed Machine Learning Systems
Sun, P., Wen, Y., Duong, T. N. B., & Yan, S. (2016) · IEEE International Conference on Parallel and Distributed Systems (ICPADS) · doi:10.1109/icpads.2016.0146 · Cited by 16
Ibrahim, E. M., Mei, L., & Verhelst, M. (2022) · IEEE Transactions on Circuits and Systems I Regular Papers · doi:10.1109/tcsi.2022.3141519 · Cited by 16
The Impact of Data Representationson Hardware Based MLP Network Implementation
(2014) · Acta Polytechnica Hungarica · doi:10.12700/aph.15.1.2018.2.4 · Cited by 16
Training deep neural networks with discrete state transition
Li, G., Deng, L., Tian, L., Cui, H., Han, W., Pei, J., & Shi, L. (2017) · Neurocomputing · doi:10.1016/j.neucom.2017.06.058 · Cited by 15
Fixed-point iterative linear inverse solver with extended precision
Zhu, Z., Klein, A. B., Li, G., & Pang, S. (2023) · Scientific Reports · doi:10.1038/s41598-023-32338-5 · Cited by 15
DNN Feature Map Compression Using Learned Representation over GF(2)
Gudovskiy, D., Hodgkinson, A., & Rigazio, L. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-11018-5_41 · Cited by 15
Design and Analysis of a Neural Network Inference Engine Based on Adaptive Weight Compression
Ko, J. H., Kim, D.-H., Na, T., & Mukhopadhyay, S. (2018) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2018.2801228 · Cited by 15
Full deep neural network training on a pruned weight budgetPreprint
Golub, M., Lemieux, G. G. F., & Lis, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.06949 · Cited by 23
Optimizing Winograd-Based Convolution with Tensor Cores
Liu, J., Yang, D., & Lai, J. (2021) · Proceedings of the International Conference on Parallel Processing · doi:10.1145/3472456.3472473 · Cited by 14
Short floating-point representation for convolutional neural network inference
Kang, H. (2018) · IEICE Electronics Express · doi:10.1587/elex.15.20180909 · Cited by 14
Incremental Binarization on Recurrent Neural Networks for Single-channel Source Separation
Kim, S., Maity, M., & Kim, M. (2019) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2019.8682595 · Cited by 14
Quantized Memory-Augmented Neural Networks
Park, S., Kim, S., Lee, S.-I., Bae, H., & Yoon, S. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11704 · Cited by 14
Zhao, Y., Gao, X., Mullins, R., & Xu, C. (2018) · International Workshop on Embedded and Mobile Deep Learning (EMDL) · doi:10.1145/3212725.3212726 · Cited by 14
Rounding Error Analysis of Mixed Precision Block Householder QR Algorithms
Yang, L. M., Fox, A., & Sanders, G. (2021) · SIAM Journal on Scientific Computing · doi:10.1137/19m1296367 · Cited by 13
An FSCV Deep Neural Network: Development, Pruning, and Acceleration on an FPGA
Zhang, Z., Oh, Y., Adams, S. D., Bennet, K. E., & Kouzani, A. Z. (2020) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2020.3037366 · Cited by 14
Running sparse and low-precision neural network: When algorithm meets hardware
Li, B., Wen, W., Mao, J., Li, S., Chen, Y., & Li, H. (2018) · Asia and South Pacific Design Automation Conference (ASP-DAC) · doi:10.1109/aspdac.2018.8297378 · Cited by 13
A High-Performance Accelerator for Super-Resolution Processing on Embedded GPU
Zhao, W., Bai, Y., Sun, Q. L., Li, W., Zheng, H., Jiang, N., Lu, J., Yu, B., & Wong, M. D. F. (2023) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2023.3241110 · Cited by 13
Jiang, S., Peng, L., Hao, Z., Du, X., Gu, J., Su, J., Guo, H., Gu, D., Zhang, H., Wang, Q., et al. (2023) · ACS Applied Electronic Materials · doi:10.1021/acsaelm.3c00698 · Cited by 13
Wiedemann, S., Mehari, T., Kepp, K., & Samek, W. (2020) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00368 · Cited by 13
Deep Neural Networks Under Stress
Carvalho, M., Cord, M., Avila, S., Thome, N., & Junior, E. A. D. V. (2016) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2016.7533200 · Cited by 14
Dynamically Adapting Floating-Point Precision to Accelerate Deep Neural Network Training
Rios, J. H. O., Armejach, A., Petit, E., Henry, G., & Casas, M. (2021) · International Conference on Machine Learning and Applications · doi:10.1109/icmla52953.2021.00161 · Cited by 16
HOME: A Holistic GPU Memory Management Framework for Deep Learning
He, S., Chen, P., Chen, S., Li, Z., Yang, S., Chen, W., & Shou, L. (2022) · IEEE Transactions on Computers · doi:10.1109/tc.2022.3180991 · Cited by 12
Hardware-aware Quantization/Mapping Strategies for Compute-in-Memory Accelerators
Huang, S., Jiang, H., & Yu, S. (2022) · ACM Transactions on Design Automation of Electronic Systems · doi:10.1145/3569940 · Cited by 13
Fuzzy qualitative deep compression network
Tan, W. R., Chan, C. S., Aguirre, H. E., & Tanaka, K. (2017) · Neurocomputing · doi:10.1016/j.neucom.2017.04.023 · Cited by 12
512KiB RAM Is Enough! Live Camera Face Recognition DNN on MCU
Zemlyanikin, M., Smorkalov, A., Khanova, T., Petrovicheva, A., & Serebryakov, G. (2019) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2019.00305 · Cited by 12
Hybrid Fixed-Point/Binary Deep Neural Network Design Methodology for Low-Power Object Detection
Guo, J.-I., Tsai, C., Zeng, J.-L., Peng, S.-W., & Chang, E.-C. (2020) · IEEE Journal on Emerging and Selected Topics in Circuits and Systems · doi:10.1109/jetcas.2020.3015753 · Cited by 12
Dasbach, S., Tetzlaff, T., Diesmann, M., & Senk, J. (2021) · Frontiers in Neuroscience · doi:10.3389/fnins.2021.757790 · Cited by 12
AEP: An error-bearing neural network accelerator for energy efficiency and model protection
Zhao, L., Zhang, Y., & Yang, J. J. (2017) · International Conference on Computer Aided Design · doi:10.1109/iccad.2017.8203897 · Cited by 12
A review of machine learning techniques for ergonomic risk assessment based on human pose estimation
Deshpande, U. U., Araujo, S. D. C. S., Deshpande, S., Kangralkar, V., Patil, R., Chate, R. A. A., Birajdar, P., Singadi, S., Raikar, P., & Arjunwadkar, S. (2025) · Discover Artificial Intelligence · doi:10.1007/s44163-025-00566-5 · Cited by 11
Xu, J., Xie, C., Hu, S., Yu, J., Liu, X., & Meng, H. M. L. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9053483 · Cited by 11
Liu, Y., Dou, Y., & Qiao, P. (2020) · IET Computer Vision · doi:10.1049/iet-cvi.2018.5839 · Cited by 11
Reduced-Order Neural Network Synthesis With Robustness Guarantees
Drummond, R., Turner, M. C., & Duncan, S. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3182893 · Cited by 11
Survey of Deep Learning Neural Networks Implementation on FPGAs
Tourad, E. H. C., & Eleuldj, M. (2020) · International Conference on Cloud Computing Technologies and Applications (CloudTech) · doi:10.1109/cloudtech49835.2020.9365911 · Cited by 11
Stochastic Approximation Beyond Gradient for Signal Processing and Machine Learning
Dieuleveut, A., Fort, G., Moulines, É., & Wai, H.-T. (2023) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2023.3301121 · Cited by 11
Zhang, Y., Wang, M., Mai, Y., & Yu, Z. (2023) · IEEE Transactions on Very Large Scale Integration (VLSI) Systems · doi:10.1109/tvlsi.2023.3326741 · Cited by 11
Randomized Quantization: A Generic Augmentation for Data Agnostic Self-supervised Learning
Wu, H., Lei, C., Sun, X., Wang, P., Chen, Q., Cheng, K.-T. T., Lin, S. C.-F., & Wu, Z. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01494 · Cited by 12
Minimizing Power for Neural Network Training with Logarithm-Approximate Floating-Point Multiplier
Cheng, T., Yu, J., & Hashimoto, M. (2019) · International Workshop on Power and Timing Modeling, Optimization and Simulation (PATMOS) · doi:10.1109/patmos.2019.8862162 · Cited by 11
Term Quantization: Furthering Quantization at Run Time
Kung, H. T., McDanel, B., & Zhang, S. Q. (2020) · International Conference for High Performance Computing, Networking, Storage and Analysis (SC) · doi:10.1109/sc41405.2020.00100 · Cited by 13
CSWAP: A Self-Tuning Compression Framework for Accelerating Tensor Swapping in GPUs
Chen, P., He, S., Zhang, X., Chen, S., Hong, P., Yin, Y., Sun, X., & Chen, G. (2021) · IEEE International Conference on Cluster Computing (CLUSTER) · doi:10.1109/cluster48925.2021.00019 · Cited by 11
Seznec, M., Nicolas Gac, A., Ferrari, A., & Orieux, F. (2018) · SiPS ... design and implementation - IEEE Workshop on Signal Processing Systems · doi:10.1109/sips.2018.8598342 · Cited by 10
Overview of Circuits, Systems, and Applications of Spintronics
Ghosh, S., Iyengar, A. S., Motaman, S., Govindaraj, R., Jang, J.-W., Chung, J., Park, J., Li, X., Joshi, R., & Somasekhar, D. (2016) · IEEE Journal on Emerging and Selected Topics in Circuits and Systems · doi:10.1109/jetcas.2016.2601310 · Cited by 10
Metadata
- Authors
- Courbariaux, M., Bengio, Y., & David, J.-P.
- Affiliations
- Polytechnique Montréal (CA)
- Université de Montréal (CA)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2014
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1412.7024
- OpenAlex
- W1825672851
- Cited by
- 407
- References
- 18
- Citations by year
- 2025: 12 · 2024: 17 · 2023: 23 · 2022: 30 · 2021: 65 · 2020: 64 · 2019: 80 · 2018: 62 · 2017: 36 · 2016: 15 · 2015: 2 · 2014: 1
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- PolyPublie (École Polytechnique de Montréal) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Canadian Institute for Advanced Research
- Compute Canada
- Canada Research Chairs
- Natural Sciences and Engineering Research Council of Canada
- Topics
- Physical Sciences › Computer Science › Computational Theory and Mathematics › Numerical Methods and Algorithms
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Low-power high-performance VLSI design
- Keywords
- low-precision arithmetic, deep neural network training, fixed-point arithmetic, floating-point precision, energy-efficient neural networks, MNIST, CIFAR-10, SVHN
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
