The DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
SCNN: An Accelerator for Compressed-sparse Convolutional Neural Networks
Abstract
Convolutional Neural Networks (CNNs) have emerged as a fundamental technology for machine learning. High performance and extreme energy efficiency are critical for deployments of CNNs, especially in mobile platforms such as autonomous vehicles, cameras, and electronic personal assistants. This paper introduces the Sparse CNN (SCNN) accelerator architecture, which improves performance and energy efficiency by exploiting the zero-valued weights that stem from network pruning during training and zero-valued activations that arise from the common ReLU operator. Specifically, SCNN employs a novel dataflow that enables maintaining the sparse weights and activations in a compressed encoding, which eliminates unnecessary data transfers and reduces storage requirements. Furthermore, the SCNN dataflow facilitates efficient delivery of those weights and activations to a multiplier array, where they are extensively reused; product accumulation is performed in a novel accumulator array. On contemporary neural networks, SCNN can improve both performance and energy by a factor of 2.7x and 2.3x, respectively, over a comparably provisioned dense CNN accelerator.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
Framewise phoneme classification with bidirectional LSTM and other neural network architectures
Graves, A., & Schmidhuber, J. (2005) · Neural Networks · doi:10.1016/j.neunet.2005.06.042 · Cited by 5,839
Natural Language Processing (almost) from Scratch
Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., & Kuksa, P. P. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1103.0398 · Cited by 4,962
Han, S., Mao, H., & Dally, W. J. (2015) · arXiv · doi:10.48550/arxiv.1510.00149 · Cited by 4,662
Self-citationDeep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1512.03385 · Cited by 4,196
Deep Speech 2: End-to-End Speech Recognition in English and Mandarin
Amodei, D., Anubhai, R., Battenberg, E., Case, C. T., Casper, J., Catanzaro, B., Chen, J., Chrzanowski, M., Coates, A., Diamos, G., et al. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1512.02595 · Cited by 2,110
Deep Networks with Stochastic Depth
Huang, G., Sun, Y., Liu, Z., Sedra, D., & Weinberger, K. Q. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_39 · Cited by 2,104
EIE: efficient inference engine on compressed deep neural network
Han, S., Liu, X., Mao, H., Pu, J., Pedram, A., Horowitz, M. A., & Dally, W. J. (2016) · ACM SIGARCH Computer Architecture News · doi:10.1145/3007787.3001163 · Cited by 1,937
Self-citationDianNao: a small-footprint high-throughput accelerator for ubiquitous machine-learning
Chen, T., Du, Z., Sun, N., Wang, J., Wu, C., Chen, Y., & Temam, O. (2014) · Proceedings of the 19th international conference on Architectural support for programming languages and operating systems · doi:10.1145/2541940.2541967 · Cited by 1,297
Eyeriss: a spatial architecture for energy-efficient dataflow for convolutional neural networks
Chen, Y.-H., Emer, J., & Sze, V. (2016) · ACM SIGARCH Computer Architecture News · doi:10.1145/3007787.3001177 · Cited by 1,182
Self-citationEIE: Efficient Inference Engine on Compressed Deep Neural Network
Han, S., Liu, X., Mao, H., Pu, J., Pedram, A., Horowitz, M. A., & Dally, W. J. (2016) · 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA) · doi:10.1109/isca.2016.30 · Cited by 1,077
Self-citationShiDianNao: shifting vision processing closer to the sensor
Du, Z., Fasthuber, R., Chen, T., Ienne, P., Li, L., Luo, T., Feng, X., Chen, Y., & Temam, O. (2015) · Proceedings of the 42nd Annual International Symposium on Computer Architecture · doi:10.1145/2749469.2750389 · Cited by 933
14.5 Eyeriss: An energy-efficient reconfigurable accelerator for deep convolutional neural networks
Chen, Y.-H., Krishna, T., Emer, J., & Sze, V. (2016) · 2016 IEEE International Solid-State Circuits Conference (ISSCC) · doi:10.1109/isscc.2016.7418007 · Cited by 713
Self-citationAlwani, M., Chen, H., Ferdman, M., & Milder, P. (2016) · 2016 49th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1109/micro.2016.7783725 · Cited by 662 · cited 1×
“propose to fuse adjacent layers in a dense CNN accelerator so that intermediate activations between layers can be kept on chip [2].”
BackgroundCnvlutin: ineffectual-neuron-free deep neural network computing
Albericio, J., Judd, P., Hetherington, T., Aamodt, T., Jerger, N. E., & Moshovos, A. (2016) · ACM SIGARCH Computer Architecture News · doi:10.1145/3007787.3001138 · Cited by 614
Cambricon-X: An accelerator for sparse neural networks
Zhang, S., Du, Z., Zhang, L., Lan, H., Liu, S., Li, L., Guo, Q., Chen, T., & Chen, Y. (2016) · 2016 49th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1109/micro.2016.7783723 · Cited by 603
Eyeriss: A Spatial Architecture for Energy-Efficient Dataflow for Convolutional Neural Networks
Chen, Y.-H., Emer, J., & Sze, V. (2016) · 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA) · doi:10.1109/isca.2016.40 · Cited by 534
Self-citationTETRIS: Scalable and Efficient Neural Network Acceleration with 3D Memory
Gao, M., Pu, J., Yang, X., Horowitz, M., & Kozyrakis, C. (2017) · Proceedings of the Twenty-Second International Conference on Architectural Support for Programming Languages and Operating Systems · doi:10.1145/3037697.3037702 · Cited by 421
Minimizing Computation in Convolutional Neural Networks
Cong, J., & Xiao, B. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-11179-7_36 · Cited by 372
vDNN: Virtualized deep neural networks for scalable, memory-efficient neural network design
Rhu, M., Gimelshein, N., Clemons, J., Zulfiqar, A., & Keckler, S. W. (2016) · 2016 49th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1109/micro.2016.7783721 · Cited by 338
Self-citationAutomatic performance tuning of sparse matrix kernels
Vuduc, R. W., & Demmel, J. W. (2003) · Cited by 288
Minerva: Enabling Low-Power, Highly-Accurate Deep Neural Network Accelerators
Reagen, B., Whatmough, P., Adolf, R., Rama, S., Lee, H., Lee, S. K., Hernandez-Lobato, J. M., Wei, G.-Y., & Brooks, D. (2016) · 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA) · doi:10.1109/isca.2016.32 · Cited by 246
Cnvlutin: Ineffectual-Neuron-Free Deep Neural Network Computing
Albericio, J., Judd, P., Hetherington, T., Aamodt, T., Jerger, N. E., & Moshovos, A. (2016) · 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA) · doi:10.1109/isca.2016.11 · Cited by 226
Accelerating Deep Convolutional Networks using low-precision and sparsity
Venkatesh, G., Nurvitadhi, E., & Marr, D. (2017) · 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2017.7952679 · Cited by 123 · cited 1×
“The Deep Learning Accelerator Core (DLAC) architecture paper mentions that it can skip computation on zero-valued operands to improve performance, but the architecture does not appear to employ zero compression [32].”
BackgroundPersistent RNNs: stashing recurrent weights on-chip
Diamos, G., Sengupta, S., Catanzaro, B., Chrzanowski, M., Coates, A., Elsen, E., Engel, J., Hannun, A., & Satheesh, S. (2016) · International Conference on Machine Learning · Cited by 101
(2014) · doi:10.4230/oasics.slate.2021.16 · Cited by 1
Cited by
Efficient Processing of Deep Neural Networks: A Tutorial and Survey
Sze, V., Chen, Y.-H., Yang, T.-J., & Emer, J. S. (2017) · Proceedings of the IEEE · doi:10.1109/jproc.2017.2761740 · Cited by 4,215
“SCNN [146] supports processing of convolutional”
Self-citationAMC: AutoML for Model Compression and Acceleration on Mobile Devices
He, Y., Lin, J., Liu, Z., Wang, H., Li, L.-J., & Han, S. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_48 · Cited by 1,400
Eyeriss v2: A Flexible Accelerator for Emerging Deep Neural Networks on Mobile Devices
Chen, Y.-H., Yang, T.-J., Emer, J. S., & Sze, V. (2019) · IEEE Journal on Emerging and Selected Topics in Circuits and Systems · doi:10.1109/jetcas.2019.2910232 · Cited by 1,106
Self-citationModel Compression and Hardware Acceleration for Neural Networks: A Comprehensive Survey
Deng, B. L., Li, G., Han, S., Shi, L., & Xie, Y. (2020) · Proceedings of the IEEE · doi:10.1109/jproc.2020.2976475 · Cited by 996
“Besides the output stationary dataflow [257], we can also see architectures with input stationary dataflow [262] or weight stationary dataflow [9], as shown in Fig.”
Method3-D Deep Learning Approach for Remote Sensing Image Classification
Ben Hamida, A., Benoit, A., Lambert, P., & Ben Amar, C. (2018) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2018.2818945 · Cited by 875
Metadata
- Authors
- Parashar, A., Rhu, M., Mukkara, A., Puglielli, A., Venkatesan, R., Khailany, B., Emer, J., Keckler, S. W., & Dally, W. J.
- Venue
- Proceedings of the 44th Annual International Symposium on Computer Architecture
- Publisher
- ACM
- Published
- 2017-06-24
- Pages
- 27-40
- ISSN
- 1063-6897 (ISSN-L)
- OpenAlex
- W2625457103
- Cited by
- 896
- References
- 29
- FWCI
- 37.21
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.