Sparse ReRAM engine: joint exploration of activation and weight sparsity in compressed neural networks
The DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
Abstract
Exploiting model sparsity to reduce ineffectual computation is a commonly used approach to achieve energy efficiency for DNN inference accelerators. However, due to the tightly coupled crossbar structure, exploiting sparsity for ReRAM-based NN accelerator is a less explored area. Existing architectural studies on ReRAM-based NN accelerators assume that an entire crossbar array can be activated in a single cycle. However, due to inference accuracy considerations, matrix-vector computation must be conducted in a smaller granularity in practice, called Operation Unit (OU). An OU-based architecture creates a new opportunity to exploit DNN sparsity. In this paper, we propose the first practical Sparse ReRAM Engine that exploits both weight and activation sparsity. Our evaluation shows that the proposed method is effective in eliminating ineffectual computation, and delivers significant performance improvement and energy savings.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 2×
“The neural networks used in our evaluation are LeNet [27] on MNIST, a CNN with three convolution layers and two fully-connected layers on CIFAR-10, and four large-scale CNNs ( CaffeNet [26], VGG-16 [41], GoogLeNet [43], and ResNet-50 [19]) on ImageNet.”
MethodDeep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1512.03385 · Cited by 4,196
Eyeriss: An Energy-Efficient Reconfigurable Accelerator for Deep Convolutional Neural Networks
Chen, Y.-H., Krishna, T., Emer, J. S., & Sze, V. (2017) · IEEE Journal of Solid-State Circuits · doi:10.1109/jssc.2016.2616357 · Cited by 3,247 · cited 3×
“One primary concern is how to efficiently exploit sparsity, which is commonly done in digital CMOS-based accelerators to improve energy efficiency [1, 8, 16, 36, 49].”
BackgroundOverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks
Sermanet, P., Eigen, D., Zhang, X., Mathieu, M., Fergus, R., & LeCun, Y. (2013) · arXiv · doi:10.48550/arxiv.1312.6229 · Cited by 2,680 · cited 2×
“Modern DNN models [26, 39, 41] are significantly larger than those used in the 1990s [10], requiring 138MB [41] for filter weight storage, 77MB [2, 41] memory accesses, and thousands to millions of computation operations per input activation.”
BackgroundMethodLearning both Weights and Connections for Efficient Neural Networks
Han, S., Pool, J., Tran, J., & Dally, W. J. (2015) · arXiv · doi:10.48550/arxiv.1506.02626 · Cited by 1,481 · cited 2×
“cant redundancy in filter weights and can be pruned dramatically during training without substantially affecting accuracy [18].”
BackgroundEIE: Efficient Inference Engine on Compressed Deep Neural Network
Han, S., Liu, X., Mao, H., Pu, J., Pedram, A., Horowitz, M. A., & Dally, W. J. (2016) · 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA) · doi:10.1109/isca.2016.30 · Cited by 1,077
ISAAC: A Convolutional Neural Network Accelerator with In-Situ Analog Arithmetic in Crossbars
Shafiee, A., Nag, A., Muralimanohar, N., Balasubramonian, R., Strachan, J. P., Hu, M., Williams, R. S., & Srikumar, V. (2016) · 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA) · doi:10.1109/isca.2016.12 · Cited by 775
Alwani, M., Chen, H., Ferdman, M., & Milder, P. (2016) · 2016 49th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1109/micro.2016.7783725 · Cited by 662 · cited 2×
“Modern DNN models [26, 39, 41] are significantly larger than those used in the 1990s [10], requiring 138MB [41] for filter weight storage, 77MB [2, 41] memory accesses, and thousands to millions of computation operations per input activation.”
Method
Cited by
Model Compression and Hardware Acceleration for Neural Networks: A Comprehensive Survey
Deng, B. L., Li, G., Han, S., Shi, L., & Xie, Y. (2020) · Proceedings of the IEEE · doi:10.1109/jproc.2020.2976475 · Cited by 996
“Yang et al. [296] argued that the entire crossbar cannot be activated at once if considering the computation accuracy on practical eNVM devices.”
BackgroundNeuro-inspired computing chips
Zhang, W., Gao, B., Tang, J., Yao, P., Yu, S., Chang, M.-F., Yoo, H.-J., Qian, H., & Wu, H. (2020) · Nature Electronics · doi:10.1038/s41928-020-0435-7 · Cited by 892
SIGMA: A Sparse and Irregular GEMM Accelerator with Flexible Interconnects for DNN Training
Qin, E., Samajdar, A., Kwon, H., Nadella, V., Srinivasan, S., Das, D., Kaul, B., & Krishna, T. (2020) · 2020 IEEE International Symposium on High Performance Computer Architecture (HPCA) · doi:10.1109/hpca47549.2020.00015 · Cited by 487
SpAtten: Efficient Sparse Attention Architecture with Cascade Token and Head Pruning
Wang, H., Zhang, Z., & Han, S. (2021) · 2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca51647.2021.00018 · Cited by 427
Machine Learning at the Network Edge: A Survey
Murshed, M. G. S., Murphy, C., Hou, D., Khan, N., Ananthanarayanan, G., & Hussain, F. (2022) · ACM Computing Surveys · doi:10.1145/3469029 · Cited by 402
Metadata
- Authors
- Yang, T.-H., Cheng, H.-Y., Yang, C.-L., Tseng, I.-C., Hu, H.-W., Chang, H.-S., & Li, H.-P.
- Venue
- Proceedings of the 46th International Symposium on Computer Architecture
- Publisher
- ACM
- Published
- 2019-06-22
- Pages
- 236-249
- ISSN
- 1063-6897 (ISSN-L)
- OpenAlex
- W2949674408
- Cited by
- 153
- References
- 15
- FWCI
- 35.30
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Sparse ReRAM engine: joint exploration of activation and weight sparsity in compressed neural networks. https://app.cochat.ai/papers/10.1145/3307650.3322271
- The paper
- Yang, T.-H., Cheng, H.-Y., Yang, C.-L., Tseng, I.-C., Hu, H.-W., Chang, H.-S., & Li, H.-P. (2019). Sparse ReRAM engine: joint exploration of activation and weight sparsity in compressed neural networks. Proceedings of the 46th International Symposium on Computer Architecture. https://doi.org/10.1145/3307650.3322271
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.