PANTHER: A Programmable Architecture for Neural Network Training Harnessing Energy-Efficient ReRAM
Open accessThe DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.
Abstract
The wide adoption of deep neural networks has been accompanied by ever-increasing energy and performance demands due to the expensive nature of training them. Numerous special-purpose architectures have been proposed to accelerate training: both digital and hybrid digital-analog using resistive RAM (ReRAM) crossbars. ReRAM-based accelerators have demonstrated the effectiveness of ReRAM crossbars at performing matrix-vector multiplication operations that are prevalent in training. However, they still suffer from inefficiency due to the use of serial reads and writes for performing the weight gradient and update step. A few works have demonstrated the possibility of performing outer products in crossbars, which can be used to realize the weight gradient and update step without the use of serial reads and writes. However, these works have been limited to low precision operations which are not sufficient for typical training workloads. Moreover, they have been confined to a limited set of training algorithms for fully-connected layers only. To address these limitations, we propose a bit-slicing technique for enhancing the precision of ReRAM-based outer products, which is substantially different from bit-slicing for matrix-vector multiplication only. We incorporate this technique into a crossbar architecture with three variants catered to different training algorithms. To evaluate our design on different types of layers in neural networks (fully-connected, convolutional, etc.) and training algorithms, we develop PANTHER, an ISA-programmable training accelerator with compiler support. Our design can also be integrated into other accelerators in the literature to enhance their efficiency. Our evaluation shows that PANTHER achieves up to 8.02×, 54.21×, and 103× energy reductions as well as 7.16×, 4.02×, and 16× execution time reductions compared to digital accelerators, ReRAM-based accelerators, and GPUs, respectively.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
In-Datacenter Performance Analysis of a Tensor Processing Unit
Jouppi, N. P., Young, C., Patil, N., Patterson, D., Agrawal, G., Bajwa, R., Bates, S., Bhatia, S., Boden, N., Borchers, A., et al. (2017) · Proceedings of the 44th Annual International Symposium on Computer Architecture · doi:10.1145/3079856.3080246 · Cited by 4,537 · cited 5×
“These accelerators have been built using various circuit technologies, including digital CMOS logic [2], [3] as well as hybrid digital-analog logic based on ReRAM crossbars [4], [5].”
BackgroundMethodInfluentialEfficient Processing of Deep Neural Networks: A Tutorial and Survey
Sze, V., Chen, Y.-H., Yang, T.-J., & Emer, J. S. (2017) · Proceedings of the IEEE · doi:10.1109/jproc.2017.2761740 · Cited by 4,215
Micikevicius, P., Narang, S., Alben, J., Diamos, G., Elsen, E., Garcia, D., Ginsburg, B., Houston, M., Kuchaiev, O., Venkatesh, G., et al. (2017) · arXiv · doi:10.48550/arxiv.1710.03740 · Cited by 1,179 · cited 3×
“DNN workloads require 16 to 32 bits of precision for training [12], [13].”
BackgroundMethodMemristor‐Based Analog Computation and Neural Network Classification with a Dot Product Engine
Hu, M., Graves, C. E., Li, C., Li, Y., Ge, N., Montgomery, E., Davila, N., Jiang, H., Williams, R. S., Yang, J. J., et al. (2018) · Advanced Materials · doi:10.1002/adma.201705914 · Cited by 751 · cited 2×
“To strike a balance, we choose p = 4, since p > 4 requires a device precision that exceeds ReRAM technology limits [15].”
BackgroundMethodSelf-citationHigh switching endurance in TaOx memristive devices
Yang, J. J., Zhang, M.-X., Strachan, J. P., Miao, F., Pickett, M. D., Kelley, R. D., Medeiros-Ribeiro, G., & Williams, R. S. (2010) · Applied Physics Letters · doi:10.1063/1.3524521 · Cited by 637
Self-citationTETRIS: Scalable and Efficient Neural Network Acceleration with 3D Memory
Gao, M., Pu, J., Yang, X., Horowitz, M., & Kozyrakis, C. (2017) · Proceedings of the Twenty-Second International Conference on Architectural Support for Programming Languages and Operating Systems · doi:10.1145/3037697.3037702 · Cited by 507
PUMA: A Programmable Ultra-efficient Memristor-based Accelerator for Machine Learning Inference
Ankit, A., Hajj, I. E., Chalamalasetti, S. R., Ndu, G., Foltin, M., Williams, R. S., Faraboschi, P., Hwu, W.-M. W., Strachan, J. P., Roy, K., et al. (2019) · Proceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems · doi:10.1145/3297858.3304049 · Cited by 424
Self-citationAccelerator-friendly neural-network training: Learning variations and defects in RRAM crossbar
Chen, L., Li, J., Chen, Y., Deng, Q., Shen, J., Liang, X., & Jiang, L. (2017) · Design, Automation & Test in Europe Conference & Exhibition (DATE), 2017 · doi:10.23919/date.2017.7926952 · Cited by 245
Saberi, M., Lotfi, R., Mafinezhad, K., & Serdijn, W. A. (2011) · IEEE Transactions on Circuits and Systems I: Regular Papers · doi:10.1109/tcsi.2011.2107214 · Cited by 236 · cited 1×
“[20].”
A Study of BFLOAT16 for Deep Learning Training
Kalamkar, D., Mudigere, D., Mellempudi, N., Das, D., Banerjee, K., Avancha, S., Vooturi, D. T., Jammalamadaka, N., Huang, J., Yuen, H., et al. (2019) · arXiv · doi:10.48550/arxiv.1905.12322 · Cited by 169
Cited by
Hardware implementation of memristor-based artificial neural networks
Aguirre, F., Sebastian, A., Le Gallo, M., Song, W., Wang, T., Yang, J. J., Lu, W., Chang, M.-F., Ielmini, D., Yang, Y., et al. (2024) · Nature Communications · doi:10.1038/s41467-024-45670-9 · Cited by 564
Chakraborty, I., Ali, M., Ankit, A., Jain, S., Roy, S., Sridharan, S., Agrawal, A., Raghunathan, A., & Roy, K. (2020) · Proceedings of the IEEE · doi:10.1109/jproc.2020.3003007 · Cited by 140
“To represent both positive and negative weights in the same device, the range of weight, RON to ROFF, is symmetrically centered around (RON + ROFF)/2 [102].”
BackgroundMethodInfluentialSelf-citationFusion of memristor and digital compute-in-memory processing for energy-efficient edge computing
Wen, T.-H., Hung, J.-M., Huang, W.-H., Jhang, C.-J., Lo, Y.-C., Hsu, H.-H., Ke, Z.-E., Chen, Y.-C., Chin, Y.-H., Su, C.-I., et al. (2024) · Science · doi:10.1126/science.adf5538 · Cited by 131
CMOS-integrated nanoscale memristive crossbars for CNN and optimization acceleration
Li, C., Ignowski, J., Sheng, X., Wessel, R., Jaffe, B., Ingemi, J., Graves, C., & Strachan, J. P. (2020) · 2020 IEEE International Memory Workshop (IMW) · doi:10.1109/imw48823.2020.9108112 · Cited by 37
Self-citation
Metadata
- Authors
- Ankit, A., Hajj, I. E., Chalamalasetti, S. R., Agarwal, S., Marinella, M., Foltin, M., Strachan, J. P., Milojicic, D., Hwu, W.-M., & Roy, K.
- Venue
- IEEE Transactions on Computers
- Published
- 2020-08-01
- Volume
- 69
- Issue
- 8
- Pages
- 1128-1142
- ISSN
- 0018-9340 (ISSN-L)
- 1557-9956
- 2326-3814
- OpenAlex
- W2997247951
- Cited by
- 88
- References
- 12
- FWCI
- 5.06
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: PANTHER: A Programmable Architecture for Neural Network Training Harnessing Energy-Efficient ReRAM. https://app.cochat.ai/papers/10.1109/tc.2020.2998456
- The paper
- Ankit, A., Hajj, I. E., Chalamalasetti, S. R., Agarwal, S., Marinella, M., Foltin, M., Strachan, J. P., Milojicic, D., Hwu, W.-M., & Roy, K. (2020). PANTHER: A Programmable Architecture for Neural Network Training Harnessing Energy-Efficient ReRAM. IEEE Transactions on Computers. https://doi.org/10.1109/tc.2020.2998456
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.