Design of Reliable DNN Accelerator with Un-reliable ReRAM
The DOI matches its Crossref record; 4 of the 4 references it cites were found in a registry or an index.
Abstract
This paper presents an algorithmic approach to design reliable ReRAM based Processing-in-Memory (PIM) architecture for Deep Neural Network (DNN) acceleration under intrinsic stochastic behavior of ReRAM devices. We employ the dynamical fixed point (DFP) data representation format to adaptively change the decimal point location based on the data range, minimizing the unused most significant bits (MSBs). Further, we propose a device variability aware (DVA) training methodology where stochastic noise is added to the parameters during training to enhance the robustness of network to the parameter's variation. Simulations indicate that, on average, the proposed algorithms improve the computing accuracy by more than 20% considering various benchmark DNNs (convolutional and recurrent). Moreover, the proposed approach enhances robustness of the DNN to noisy input data.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Speech recognition with deep recurrent neural networks
Graves, A., Mohamed, A.-R., & Hinton, G. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6638947 · Cited by 8,893 · cited 1×
“Deep neural networks (DNNs) have shown great promise for a wide range of applications, including image classification [1], speech recognition [2], language processing [3], and computer games [4], to name a few.”
BackgroundLearning long-term dependencies with gradient descent is difficult
Bengio, Y., Simard, P., & Frasconi, P. (1994) · IEEE Transactions on Neural Networks · doi:10.1109/72.279181 · Cited by 8,542 · cited 1×
“Long short term memory (LSTM) is developed to solve the gradient vanishing issues of basic RNN [14] via a gating mechanism (including input gate, forget gate and output gate), as shown in the right of Figure 1(b).”
BackgroundIn-Datacenter Performance Analysis of a Tensor Processing Unit
Jouppi, N. P., Young, C., Patil, N., Patterson, D., Agrawal, G., Bajwa, R., Bates, S., Bhatia, S., Boden, N., Borchers, A., et al. (2017) · Proceedings of the 44th Annual International Symposium on Computer Architecture · doi:10.1145/3079856.3080246 · Cited by 4,537 · cited 1×
“Dedicated application specific integrated circuit (ASIC) has been intensively investigated to improve the efficiency of DNN acceleration [5], [6].”
Background
Cited by
Compute-in-Memory Chips for Deep Learning: Recent Trends and Prospects
Yu, S., Jiang, H., Huang, S., Peng, X., & Lu, A. (2021) · IEEE Circuits and Systems Magazine · doi:10.1109/mcas.2021.3092533 · Cited by 391
Narayanan, P., Ambrogio, S., Okazaki, A., Hosokawa, K., Tsai, H., Nomura, A., Yasuda, T., Mackin, C., Lewis, S. C., Friz, A., et al. (2021) · IEEE Transactions on Electron Devices · doi:10.1109/ted.2021.3115993 · Cited by 85
On the Accuracy of Analog Neural Network Inference Accelerators
Xiao, T. P., Feinberg, B., Bennett, C. H., Prabhakar, V., Saxena, P., Agrawal, V., Agarwal, S., & Marinella, M. J. (2022) · IEEE Circuits and Systems Magazine · doi:10.1109/mcas.2022.3214409 · Cited by 76
“A common approach is to add noise to weights and activations during forward propagation [26, 30, 32, 34, 42].”
BackgroundNoise-Resilient DNN: Tolerating Noise in PCM-Based AI Accelerators via Noise-Aware Training
Kariyappa, S., Tsai, H., Spoon, K., Ambrogio, S., Narayanan, P., Mackin, C., Chen, A., Qureshi, M., & Burr, G. W. (2021) · IEEE Transactions on Electron Devices · doi:10.1109/ted.2021.3089987 · Cited by 45
Cao, T., Liu, C., Wang, W., Zhang, T., Lee, H. K., Li, M. H., Song, W., Chen, Z. X., Zhuo, V. Y.-Q., Wang, N., et al. (2022) · IEEE Journal on Emerging and Selected Topics in Circuits and Systems · doi:10.1109/jetcas.2022.3214334 · Cited by 34
Yang, X., Wu, C., Li, M., & Chen, Y. (2022) · Advanced Intelligent Systems · doi:10.1002/aisy.202200029 · Cited by 22
Meng, Z., Qian, W., Zhao, Y., Sun, Y., Yang, R., & Jiang, L. (2021) · 2021 Design, Automation & Test in Europe Conference & Exhibition (DATE) · doi:10.23919/date51398.2021.9474179 · Cited by 10
Gonugondla, S. K., Patil, A. D., & Shanbhag, N. R. (2020) · IEEE/ACM ICCAD · doi:10.1145/3400302.3415642 · Cited by 6
Metadata
- Authors
- Long, Y., She, X., & Mukhopadhyay, S.
- Venue
- 2019 Design, Automation & Test in Europe Conference & Exhibition (DATE)
- Publisher
- IEEE
- Published
- 2019-03
- Pages
- 1769-1774
- OpenAlex
- W2946047477
- Cited by
- 136
- References
- 4
- FWCI
- 33.27
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.