The DOI matches its Crossref record; 7 of the 7 references it cites were found in a registry or an index.
DianNao family: energy-efficient hardware accelerators for machine learning
Abstract
Machine Learning (ML) tasks are becoming pervasive in a broad range of applications, and in a broad range of systems (from embedded systems to data centers). As computer architectures evolve toward heterogeneous multi-cores composed of a mix of cores and hardware accelerators, designing hardware accelerators for ML techniques can simultaneously achieve high efficiency and broad application scope. While efficient computational primitives are important for a hardware accelerator, inefficient memory transfers can potentially void the throughput, energy, or cost advantages of accelerators, that is, an Amdahl's law effect, and thus, they should become a first-order concern, just like in processors, rather than an element factored in accelerator design on a second step. In this article, we introduce a series of hardware accelerators (i.e., the DianNao family) designed for ML (especially neural networks), with a special emphasis on the impact of memory on accelerator design, performance, and energy. We show that, on a number of representative neural network layers, it is possible to achieve a speedup of 450.65x over a GPU, and reduce the energy by 150.31x on average for a 64-chip DaDianNao system (a member of the DianNao family).<!-- END_PAGE_1 -->
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Improving neural networks by preventing co-adaptation of feature detectors
Hinton, G. E., Srivastava, N., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. R. (2012) · arXiv · doi:10.48550/arxiv.1207.0580 · Cited by 6,382
Building high-level features using large scale unsupervised learning
Le, Q. V. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6639343 · Cited by 1,695
A dynamically configurable coprocessor for convolutional neural networks
Chakradhar, S., Sankaradas, M., Jakkula, V., & Cadambi, S. (2010) · Proceedings of the 37th annual international symposium on Computer architecture · doi:10.1145/1815961.1815993 · Cited by 343
Research works printed but not linked: 41
- 10.1109/FCCM.2009.34 · DOI
- 10.1145/1816038.1815993 · DOI
- Chan , E. Algorithmic Trading: Winning Strategies and Their Rationale . John Wiley & Sons , 2013 . Chan, E. Algorithmic Trading: Winning Strategies and Their Rationale. John Wiley & Sons, 2013. · doi:10.1002/9781118676998
- 10.1109/IISWC.2012.6402898 · DOI
- 10.1145/2541940.2541967 · DOI · Paper page
- 10.1109/MICRO.2014.58 · DOI · Paper page
- Coates , A. , Huval , B. , Wang , T. , Wu , D.J. , Ng , A.Y. Deep learning with cots HPC systems . In International Conference on Machine Learning , 2013: 1337--1345. Coates, A., Huval, B., Wang, T., Wu, D.J., Ng, A.Y. Deep learning with cots HPC systems. In International Conference on Machine Learning, 2013: 1337--1345.
- 10.1145/2749469.2750389 · DOI · Paper page
- 10.1145/2000064.2000108 · DOI · Paper page
- 10.1109/MICRO.2012.48 · DOI · Paper page
- 10.1109/CVPRW.2011.5981829 · DOI · Paper page
- 10.1109/CVPRW.2011.5981829 · DOI · Paper page
- Frery , A. , de Araujo , C. , Alice , H. , Cerqueira , J. , Loureiro , J.A. , de Lima , M.E. , Oliveira , M. , Horta , M. , Hyperspectral images clustering on reconfigurable hardware using the k-means algorithm . In Proceedings of the 16th Symposium on Integrated Circuits and Systems Design, 2003. SBCCI 2003 ( 2003 ). IEEE, 99--104. Frery, A., de Araujo, C., Alice, H., Cerqueira, J., Loureiro, J.A., de Lima, M.E., Oliveira, M., Horta, M., et al. Hyperspectral images clustering on reconfigurable hardware using the k-means algorithm. In Proceedings of the 16th Symposium on Integrated Circuits and Systems Design, 2003. SBCCI 2003 (2003). IEEE, 99--104.
- 10.1145/1816038.1815968 · DOI
- 10.1109/AHS.2011.5963944 · doi:10.1109/ahs.2011.5963944
- Keckler , S. Life after Dennard and how I learned to love the Picojoule (keynote) . In International Symposium on Microarchitecture, Keynote presentation , Sao Paolo , Dec. 2011 . Keckler, S. Life after Dennard and how I learned to love the Picojoule (keynote). In International Symposium on Microarchitecture, Keynote presentation, Sao Paolo, Dec. 2011.
- Kim , J.Y. , Kim , M. , Lee , S. , Oh , J. , Kim , K. , Yoo , H.-J.A. GOPS 496 mW real-time multi-object recognition processor with bio-inspired neural perception engine . IEEE Journal of Solid-State Circuits 45, 1 ( Jan. 2010 ), 32--45. Kim, J.Y., Kim, M., Lee, S., Oh, J., Kim, K., Yoo, H.-J.A. GOPS 496 mW real-time multi-object recognition processor with bio-inspired neural perception engine. IEEE Journal of Solid-State Circuits 45, 1 (Jan. 2010), 32--45.
- Krizhevsky A. Sutskever I. Hinton G. ImageNet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems (2012) 1--9. Krizhevsky A. Sutskever I. Hinton G. ImageNet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems (2012) 1--9.
- 10.5555/1986861.1987000 · doi:10.5555/1986861.1987000
- Le , Q.V. , Ranzato , M.A. , Monga , R. , Devin , M. , Chen , K. , Corrado , G.S. , Dean , J. , Ng , A.Y. Building high-level features using large scale unsupervised learning . In International Conference on Machine Learning , June 2012 . Le, Q.V., Ranzato, M.A., Monga, R., Devin, M., Chen, K., Corrado, G.S., Dean, J., Ng, A.Y. Building high-level features using large scale unsupervised learning. In International Conference on Machine Learning, June 2012.
- 10.1145/1669112.1669172 · DOI · Paper page
- 10.1145/2694344.2694358 · DOI · Paper page
- 10.1145/2228360.2228465 · DOI
- Maeda , N. , Komatsu , S. , Morimoto , M. , Shimazaki , Y. A 0. 41 µa standby leakage 32 kb embedded SRAM with low-voltage resume-standby utilizing all digital current comparator in 28 nm hkmg CMOS . In International Symposium on VLSI Circuits (VLSIC) , 2012 . Maeda, N., Komatsu, S., Morimoto, M., Shimazaki, Y. A 0.41 µa standby leakage 32 kb embedded SRAM with low-voltage resume-standby utilizing all digital current comparator in 28 nm hkmg CMOS. In International Symposium on VLSI Circuits (VLSIC), 2012.
- 10.1145/2133382.2133388 · DOI
- 10.1109/LES.2010.2100802 · DOI
- 10.1109/ISCAS.2010.5537602 · doi:10.1109/iscas.2010.5537602
- 10.1109/ICPR.2006.961 · DOI
- Muller , M . Dark silicon and the internet. In EE Times "Designing with ARM " Virtual Conference , 26 , 70(2010), 285 -- 288 . Muller, M. Dark silicon and the internet. In EE Times "Designing with ARM" Virtual Conference, 26, 70(2010), 285--288.
- 10.1109/FCCM.2010.39 · DOI
- 10.1145/2508148.2485925 · DOI
- Sermanet , P. , Chintala , S. , LeCun , Y. Convolutional neural networks applied to house numbers digit classification . In Pattern Recognition (ICPR), … , 2012 . Sermanet, P., Chintala, S., LeCun, Y. Convolutional neural networks applied to house numbers digit classification. In Pattern Recognition (ICPR), …, 2012.
- 10.1109/IJCNN.2011.6033589 · DOI · Paper page
- 10.1145/2514641.2514649 · doi:10.1145/2514641.2514649
- Swanson , S. , Michelson , K. , Schwerin , A. , Oskin , M. Wavescalar . In ACM/IEEE International Symposium on Microarchitecture (MICRO) (Dec 2003 ). IEEE Computer Society, 291. Swanson, S., Michelson, K., Schwerin, A., Oskin, M. Wavescalar. In ACM/IEEE International Symposium on Microarchitecture (MICRO) (Dec 2003). IEEE Computer Society, 291.
- 10.1145/1815961.1816008 · doi:10.1145/1815961.1816008
- 10.5555/2337159.2337200 · doi:10.5555/2337159.2337200
- Vanhoucke , V. , Senior , A. , Mao , M.Z. Improving the speed of neural networks on CPUs . In Deep Learning and Unsupervised Feature Learning Workshop (NIPS) ( 2011 ). Vol. 1 . Vanhoucke, V., Senior, A., Mao, M.Z. Improving the speed of neural networks on CPUs. In Deep Learning and Unsupervised Feature Learning Workshop (NIPS) (2011). Vol. 1.
- 10.1109/IEDM.2009.5424375 · doi:10.1109/iedm.2009.5424375
- 10.1162/neco.1996.8.7.1341 · DOI · Paper page
- Yeh , Y.-J. , Li , H.-Y. , Hwang , W.-J. , Fang , C.-Y. Fpga implementation of KNN classifier based on wavelet transform and partial distance search . In Image Analysis (June 2007 ). Springer Berlin Heidelberg , 512--521. Yeh, Y.-J., Li, H.-Y., Hwang, W.-J., Fang, C.-Y. Fpga implementation of KNN classifier based on wavelet transform and partial distance search. In Image Analysis (June 2007). Springer Berlin Heidelberg, 512--521.
Cited by
In-Datacenter Performance Analysis of a Tensor Processing Unit
Jouppi, N. P., Young, C., Patil, N., Patterson, D., Agrawal, G., Bajwa, R., Bates, S., Bhatia, S., Boden, N., Borchers, A., et al. (2017) · Proceedings of the 44th Annual International Symposium on Computer Architecture · doi:10.1145/3079856.3080246 · Cited by 4,537
Artificial intelligence: A powerful paradigm for scientific research
Xu, Y., Liu, X., Cao, X., Huang, C., Liu, E., Qian, S., Liu, X., Wu, Y., Dong, F., Qiu, C.-W., et al. (2021) · The Innovation · doi:10.1016/j.xinn.2021.100179 · Cited by 1,726
Deep Learning With Edge Computing: A Review
Chen, J., & Ran, X. (2019) · Proceedings of the IEEE · doi:10.1109/jproc.2019.2921977 · Cited by 1,540
In-Datacenter Performance Analysis of a Tensor Processing Unit
Jouppi, N. P., Young, C., Patil, N., Patterson, D., Agrawal, G., Bajwa, R., Bates, S., Bhatia, S., Boden, N., Borchers, A., et al. (2017) · ACM SIGARCH Computer Architecture News · doi:10.1145/3140659.3080246 · Cited by 1,240
Pruning and quantization for deep neural network acceleration: A survey
Liang, T., Glossner, J., Wang, L., Shi, S., & Zhang, X. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.07.045 · Cited by 943
Metadata
- Authors
- Chen, Y., Chen, T., Xu, Z., Sun, N., & Temam, O.
- Venue
- Communications of the ACM
- Published
- 2016-10-28
- Volume
- 59
- Issue
- 11
- Pages
- 105-112
- ISSN
- 0001-0782 (ISSN-L)
- 1557-7317
- OpenAlex
- W2540279855
- Cited by
- 205
- References
- 7
- FWCI
- 19.74
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.