The DOI matches its Crossref record; 39 of the 39 references it cites were found in a registry or an index.
DaDianNao: A Machine-Learning Supercomputer
Abstract
Many companies are deploying services, either for consumers or industry, which are largely based on machine-learning algorithms for sophisticated processing of large amounts of data. The state-of-the-art and most popular such machine-learning algorithms are Convolutional and Deep Neural Networks (CNNs and DNNs), which are known to be both computationally and memory intensive. A number of neural network accelerators have been recently proposed which can offer high computational capacity/area ratio, but which remain hampered by memory accesses. However, unlike the memory wall faced by processors on general-purpose workloads, the CNNs and DNNs memory footprint, while large, is not beyond the capability of the on chip storage of a multi-chip system. This property, combined with the CNN/DNN algorithmic characteristics, can lead to high internal bandwidth and low external communications, which can in turn enable high-degree parallelism at a reasonable area cost. In this article, we introduce a custom multi-chip machine-learning architecture along those lines. We show that, on a subset of the largest known neural network layers, it is possible to achieve a speedup of 450.65x over a GPU, and reduce the energy by 150.31x on average for a 64-chip system. We implement the node down to the place and route at 28nm, containing a combination of custom storage and computational units, with industry-grade interconnects.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Self-citationGradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Multi-column deep neural networks for image classification
Ciresan, D., Meier, U., & Schmidhuber, J. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248110 · Cited by 3,720
The PARSEC benchmark suite: characterization and architectural implications
Bienia, C., Kumar, S., Singh, J. P., & Li, K. (2008) · Proceedings of the 17th international conference on Parallel architectures and compilation techniques · doi:10.1145/1454115.1454128 · Cited by 3,444
Principles and Practices of Interconnection Networks
Dally, W. J., & Towles, B. (2004) · Cited by 3,260
Large Scale Distributed Deep Networks
Dean, J. B., Corrado, G. S., Monga, R., Chen, K., Devin, M., Mao, M., Ranzato, M., Senior, A., Tucker, P. A., Yang, K., et al. (2012) · Cited by 3,199
ImageNet: A large-scale hierarchical image database
Deng, J. H., Dong, W. F., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvprw.2009.5206848 · Cited by 2,714
Self-citationWhat is the best multi-stage architecture for object recognition?
Jarrett, K., Kavukcuoglu, K., Ranzato, M. A., & LeCun, Y. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459469 · Cited by 2,153
Learning deep structured semantic models for web search using clickthrough data
Huang, P.-S., He, X., Gao, J., Deng, L., Acero, A., & Heck, L. (2013) · Proceedings of the 22nd ACM international conference on Information & Knowledge Management · doi:10.1145/2505515.2505665 · Cited by 2,009
Building high-level features using large scale unsupervised learning
Le, Q. V. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6639343 · Cited by 1,695
Dark silicon and the end of multicore scaling
Esmaeilzadeh, H., Blem, E., St. Amant, R., Sankaralingam, K., & Burger, D. (2011) · Proceedings of the 38th annual international symposium on Computer architecture · doi:10.1145/2000064.2000108 · Cited by 1,532
Improving deep neural networks for LVCSR using rectified linear units and dropout
Dahl, G. E., Sainath, T. N., & Hinton, G. E. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6639346 · Cited by 1,310
DianNao: a small-footprint high-throughput accelerator for ubiquitous machine-learning
Chen, T., Du, Z., Sun, N., Wang, J., Wu, C., Chen, Y., & Temam, O. (2014) · Proceedings of the 19th international conference on Architectural support for programming languages and operating systems · doi:10.1145/2541940.2541967 · Cited by 1,297
Self-citation(2011) · doi:10.5591/978-1-57735-516-8/ijcai11-210 · Cited by 1,229
GradientBased Learning Applied to Document Recognition
(2009) · Intelligent Signal Processing · doi:10.1109/9780470544976.ch9 · Cited by 1,135
An empirical evaluation of deep architectures on problems with many factors of variation
Larochelle, H., Erhan, D., Courville, A., Bergstra, J., & Bengio, Y. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273556 · Cited by 992
(2011) · Cited by 713
(2013) · Cited by 651
Building high-level features using large scale unsupervised learning
Ranzato, M., Monga, R., Devin, M., Chen, K., Corrado, G. S., Dean, J., Le, Q. V., & Ng, A. Y. (2012) · International Conference on Machine Learning · Cited by 646
Neural Acceleration for General-Purpose Approximate Programs
Esmaeilzadeh, H., Sampson, A., Ceze, L., & Burger, D. (2012) · 2012 45th Annual IEEE/ACM International Symposium on Microarchitecture · doi:10.1109/micro.2012.48 · Cited by 626
Dark Silicon and the End of Multicore Scaling
Esmaeilzadeh, H., Blem, E., St. Amant, R., Sankaralingam, K., & Burger, D. (2012) · IEEE Micro · doi:10.1109/mm.2012.17 · Cited by 490
Introduction to “This is Watson”
Ferrucci, D. A. (2012) · IBM Journal of Research and Development · doi:10.1147/jrd.2012.2184356 · Cited by 484
An Efficient Learning Procedure for Deep Boltzmann Machines
Salakhutdinov, R., & Hinton, G. (2012) · Neural Computation · doi:10.1162/neco_a_00311 · Cited by 484
A digital neurosynaptic core using embedded crossbar memory with 45pJ per spike in 45nm
Merolla, P., Arthur, J., Akopyan, F., Imam, N., Manohar, R., & Modha, D. S. (2011) · 2011 IEEE Custom Integrated Circuits Conference (CICC) · doi:10.1109/cicc.2011.6055294 · Cited by 431
Understanding sources of inefficiency in general-purpose chips
Hameed, R., Qadeer, W., Wachs, M., Azizi, O., Solomatnikov, A., Lee, B. C., Richardson, S., Kozyrakis, C., & Horowitz, M. (2010) · Proceedings of the 37th annual international symposium on Computer architecture · doi:10.1145/1815961.1815968 · Cited by 391
NeuFlow: A runtime reconfigurable dataflow processor for vision
Farabet, C., Martini, B., Corda, B., Akselrod, P., Culurciello, E., & LeCun, Y. (2011) · CVPR 2011 WORKSHOPS · doi:10.1109/cvprw.2011.5981829 · Cited by 369
Learning to Label Aerial Images from Noisy Data
Mnih, V., & Hinton, G. E. (2012) · Cited by 364
Learning long‐range vision for autonomous off‐road driving
Hadsell, R., Sermanet, P., Ben, J., Erkan, A., Scoffier, M., Kavukcuoglu, K., Muller, U., & LeCun, Y. (2009) · Journal of Field Robotics · doi:10.1002/rob.20276 · Cited by 314
Wafer-scale integration of analog neural networks
Schemmel, J., Fieres, J., & Meier, K. (2008) · 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence) · doi:10.1109/ijcnn.2008.4633828 · Cited by 275
SpiNNaker: Mapping neural networks onto a massively-parallel chip multiprocessor
Khan, M. M., Lester, D. R., Plana, L. A., Rast, A., Jin, X., Painkras, E., & Furber, S. B. (2008) · 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence) · doi:10.1109/ijcnn.2008.4634199 · Cited by 270
Convolution engine: balancing efficiency & flexibility in specialized computing
Qadeer, W., Hameed, R., Shacham, O., Venkatesan, P., Kozyrakis, C., & Horowitz, M. A. (2013) · Proceedings of the 40th Annual International Symposium on Computer Architecture · doi:10.1145/2485922.2485925 · Cited by 168
Compute-and-Forward Strategies for Cooperative Distributed Antenna Systems
Hong, S.-N., & Caire, G. (2013) · IEEE Transactions on Information Theory · doi:10.1109/tit.2013.2265695 · Cited by 160
A defect-tolerant accelerator for emerging high-performance applications
Temam, O. (2012) · ACM SIGARCH Computer Architecture News · doi:10.1145/2366231.2337200 · Cited by 143
Self-citationA defect-tolerant accelerator for emerging high-performance applications
Temam, O. (2012) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1109/isca.2012.6237031 · Cited by 94
Self-citationContinuous real-world inputs can open up alternative accelerator designs
Belhadj, B., Joubert, A., Li, Z., Héliot, R., & Temam, O. (2013) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1145/2485922.2485923 · Cited by 66
Self-citationAutomatic abstraction and fault tolerance in cortical microachitectures
Hashmi, A. G., Berry, H., Temam, O., & Lipasti, M. H. (2011) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1145/2000064.2000066 · Cited by 53
Self-citationA Massively Parallel, Energy Efficient Programmable Accelerator for Learning and Classification
Majumdar, A., Cadambi, S., Becchi, M., Chakradhar, S., & Graf, H. P. (2012) · ACM Transactions on Architecture and Code Optimization · doi:10.1145/2133382.2133388 · Cited by 45
Hashmi, A. G., Nere, A., Thomas, J. J., & Lipasti, M. H. (2011) · International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) · doi:10.1145/1950365.1950385 · Cited by 40
Cited by
Efficient Processing of Deep Neural Networks: A Tutorial and Survey
Sze, V., Chen, Y.-H., Yang, T.-J., & Emer, J. S. (2017) · Proceedings of the IEEE · doi:10.1109/jproc.2017.2761740 · Cited by 4,215
“Another example of no local reuse dataflow is found in [93].”
InfluentialSelf-citationEyeriss: An Energy-Efficient Reconfigurable Accelerator for Deep Convolutional Neural Networks
Chen, Y.-H., Krishna, T., Emer, J. S., & Sze, V. (2017) · IEEE Journal of Solid-State Circuits · doi:10.1109/jssc.2016.2616357 · Cited by 3,247
Self-citationMemory devices and applications for in-memory computing
Sebastian, A., Le Gallo, M., Khaddam-Aljameh, R., & Eleftheriou, E. (2020) · Nature Nanotechnology · doi:10.1038/s41565-020-0655-z · Cited by 2,289
BinaryConnect: Training Deep Neural Networks with binary weights during propagations
Courbariaux, M., Bengio, Y., & David, J.-P. (2015) · arXiv · doi:10.48550/arxiv.1511.00363 · Cited by 2,180
Courbariaux, M., Hubara, I., Soudry, D., El-Yaniv, R., & Bengio, Y. (2016) · arXiv · doi:10.48550/arxiv.1602.02830 · Cited by 2,106
Metadata
- Authors
- Chen, Y., Luo, T., Liu, S., Zhang, S., He, L., Wang, J., Li, L., Chen, T., Xu, Z., Sun, N., & Temam, O.
- Venue
- 2014 47th Annual IEEE/ACM International Symposium on Microarchitecture
- Publisher
- IEEE
- Published
- 2014-12
- Pages
- 609-622
- ISSN
- 1072-4451 (ISSN-L)
- OpenAlex
- W2048266589
- Cited by
- 1,492
- References
- 39
- FWCI
- 57.80
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.