The DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.
Multiscale Co-Design Analysis of Energy, Latency, Area, and Accuracy of a ReRAM Analog Neural Training Accelerator
Open accessAbstract
Neural networks are an increasingly attractive algorithm for natural language processing and pattern recognition. Deep networks with >50 M parameters are made possible by modern graphics processing unit clusters operating at <;50 pJ per op and more recently, production accelerators are capable of <;5 pJ per operation at the board level. However, with the slowing of CMOS scaling, new paradigms will be required to achieve the next several orders of magnitude in performance per watt gains. Using an analog resistive memory (ReRAM) crossbar to perform key matrix operations in an accelerator is an attractive option. This paper presents a detailed design using the state-of-the-art 14/16 nm process development kit for of an analog crossbar circuit block designed to process three key kernels required in training and inference of neural networks. A detailed circuit and device-level analysis of energy, latency, area, and accuracy are given and compared with relevant designs using standard digital ReRAM and static random access memory (SRAM) operations. It is shown that the analog accelerator has 270× energy and 540× latency advantage over a similar block utilizing only digital ReRAM and takes only 11 fJ per multiply and accumulate. Compared with an SRAM-based accelerator, the energy is 430× better and latency is 34× better. Although training accuracy is degraded in the analog accelerator, several options to improve this are presented. The possible gains over a similar digital only version of this accelerator block suggest that continued optimization of analog resistive memories is valuable. This detailed circuit and device analysis of a training accelerator may serve as a foundation for further architecture-level studies.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Most compute-in-memory or near-memory approaches based on SRAMs and digital compute do not address the energy and time costs of reloading weights, making direct side-by-side comparisons against NVM-based weight-stationary approaches difficult; several NVM compute-in-memory studies have focused on the macro-level without accounting for data transport, control or chip infrastructure costs, and are,
“Although several compute-in-memory or near-memory approaches based on SRAMs and digital compute35–38 have been presented in the literature, most of these do not address the energy and time costs of reloading weights, thus making direct side-by-side comparisons against NVM-based weight-stationary approaches difficult.”
From An analog-AI chip for energy-efficient speech recognition and transcription (2023) · cites this paper as [41]
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Burr, G. W., Shelby, R. M., Sidler, S., di Nolfo, C., Jang, J., Boybat, I., Shenoy, R. S., Narayanan, P., Virwani, K., Giacometti, E. U., et al. (2015) · IEEE Transactions on Electron Devices · doi:10.1109/ted.2015.2439635 · Cited by 936 · cited 5×
“Reference [27] shows that as low as 2 bits × 2 bits can be used to achieve ideal numerical accuracy.”
BackgroundMethodInfluential
Flagged first, then by citations
Research works printed but not linked: 31
- 10.1145/2897937.2898010 · DOI
- 10.1109/IRPS.2013.6532101 · DOI
- xu, 2011, Design implications of memristor-based RRAM cross-point structures, Proc Design Autom Test Eur, 1
- 10.1109/IMW.2017.7939075 · DOI
- 10.23919/VLSIT.2017.7998164 · DOI
- 10.1109/IEDM.2014.7047135 · DOI
- 10.1109/ICCAD.2015.7372570 · DOI
- 10.1145/3007787.3001140 · DOI
- 10.1145/2744769.2744900 · DOI
- 10.1109/ICRC.2016.7738690 · DOI
- chakrabarti, 2017, A multiply-add engine with monolithically integrated 3D memristor crossbar/CMOS hybrid circuit, Sci Rep, 7
- 10.1147/rd.524.0449 · DOI
- 2015, The International Technology Roadmap for Semiconductors (ITRS)
- 10.1002/adma.201604310 · DOI
- 10.1149/05805.0059ecst · DOI
- 10.1116/1.4828701 · DOI
- lee, 2011, A fast, high-endurance and scalable non-volatile memory device made from asymmetric Ta2O5–x/TaO2–x bilayer structures, Nature Mater, 10, 625, 10.1038/nmat3070 · DOI · Paper page
- chen, 2015, Technology-design co-optimization of resistive cross-point array for accelerating learning algorithms on chip, Proc Design Autom Test Eur Conf Exhib, 854
- 10.1145/2408776.2408797 · DOI
- 10.3389/fnins.2013.00118 · DOI
- lecun, 2018, The MNIST database
- 10.1109/JEDS.2015.2442242 · DOI
- agarwal, 2017, CrossSim
- 10.1109/MICRO.2014.58 · DOI
- 10.1145/3079856.3080246 · DOI · Paper page
- 10.1145/3007787.3001139 · DOI
- park, 2012, A non-linear ReRAM cell with sub- $1~\mu \text{A}$ ultralow operating current for high density vertical resistive memory (VRRAM), IEDM Tech Dig, 20.8.1
- 10.1109/IJCNN.2016.7727298 · DOI · Paper page
- govoreanu, 2011, $10\times 10$ nm2 Hf/HfOx crossbar resistive RAM with excellent performance, reliability and low-energy operation, IEDM Tech Dig, 31.6.1
- 10.1109/IMW.2011.5873227 · DOI
- 10.5573/JSTS.2014.14.3.356 · DOI
Cited by
Sorted by citations
Efficient and self-adaptive in-situ learning in multilayer memristor neural networks
Li, C., Belkin, D., Li, Y., Yan, P., Hu, M., Ge, N., Jiang, H., Montgomery, E., Lin, P., Wang, Z., et al. (2018) · Nature Communications · doi:10.1038/s41467-018-04484-2 · Cited by 931
Parallel programming of an ionic floating-gate memory array for scalable neuromorphic computing
Fuller, E. J., Keene, S. T., Melianas, A., Wang, Z., Agarwal, S., Li, Y., Tuchman, Y., James, C. D., Marinella, M. J., Yang, J. J., et al. (2019) · Science · doi:10.1126/science.aaw5581 · Cited by 756
Self-citationBrain-inspired computing needs a master plan
Mehonic, A., & Kenyon, A. J. (2022) · Nature · doi:10.1038/s41586-021-04362-w · Cited by 632
An analog-AI chip for energy-efficient speech recognition and transcription
Ambrogio, S., Narayanan, P., Okazaki, A., Fasoli, A., Mackin, C., Hosokawa, K., Nomura, A., Yasuda, T., Chen, A., Friz, A., et al. (2023) · Nature · doi:10.1038/s41586-023-06337-5 · Cited by 269
“However, several NVM compute-in-memory studies have focused on the macro-level 32,34,39,40,41 , without accounting for data transport, control or chip infrastructure (such as clocking) costs.”
BackgroundAmirsoleimani, A., Alibart, F., Yon, V., Xu, J., Pazhouhandeh, M. R., Ecoffey, S., Beilliard, Y., Genov, R., & Drouin, D. (2020) · Advanced Intelligent Systems · doi:10.1002/aisy.202000115 · Cited by 216
Chakraborty, I., Ali, M., Ankit, A., Jain, S., Roy, S., Sridharan, S., Agrawal, A., Raghunathan, A., & Roy, K. (2020) · Proceedings of the IEEE · doi:10.1109/jproc.2020.3003007 · Cited by 140
Mixed-Precision Deep Learning Based on Computational Memory
Nandakumar, S. R., Le Gallo, M., Piveteau, C., Joshi, V., Mariani, G., Boybat, I., Karunaratne, G., Khaddam-Aljameh, R., Egger, U., Petropoulos, A., et al. (2020) · Frontiers in Neuroscience · doi:10.3389/fnins.2020.00406 · Cited by 120
On the Accuracy of Analog Neural Network Inference Accelerators
Xiao, T. P., Feinberg, B., Bennett, C. H., Prabhakar, V., Saxena, P., Agrawal, V., Agarwal, S., & Marinella, M. J. (2022) · IEEE Circuits and Systems Magazine · doi:10.1109/mcas.2022.3214409 · Cited by 76
Self-citationJain, S., Tsai, H., Chen, C.-T., Muralidhar, R., Boybat, I., Frank, M. M., Woźniak, S., Stanisavljevic, M., Adusumilli, P., Narayanan, P., et al. (2023) · IEEE Transactions on Very Large Scale Integration (VLSI) Systems · doi:10.1109/tvlsi.2022.3221390 · Cited by 76
Metadata
- Authors
- Marinella, M. J., Agarwal, S., Hsia, A., Richter, I., Jacobs-Gedrim, R., Niroula, J., Plimpton, S. J., Ipek, E., & James, C. D.
- Affiliations
- Sandia National Laboratories (US)
- Sandia National Laboratories California (US)
- University of Rochester (US)
- Massachusetts Institute of Technology (US)
- ORCID
- Venue
- IEEE Journal on Emerging and Selected Topics in Circuits and Systems
- Published
- 2018-03
- Volume
- 8
- Issue
- 1
- Pages
- 86-101
- ISSN
- 2156-3357 (ISSN-L)
- 2156-3365
- OpenAlex
- W2740220207
- Cited by
- 183
- References
- 3
- Citations by year
- 2026: 11 · 2025: 17 · 2024: 21 · 2023: 24 · 2022: 30 · 2021: 25 · 2020: 24 · 2019: 22 · 2018: 8
- FWCI
- 7.56
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Journal on Emerging and Selected Topics in Circuits and Systems (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Sandia National Laboratories Hardware Acceleration of Adaptive Neural Algorithms Grand Challenge Laboratory Directed Research and Development (LDRD) Project
- Sandia National Laboratories is a multimission laboratory
- National Technology and Engineering Solutions of Sandia, LLC., a wholly owned subsidiary of Honeywell International, Inc., for the U.S. Department of Energy’s National Nuclear Security Administration (DE-NA0003525)
- Topics
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Advanced Memory and Neural Computing
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Low-power high-performance VLSI design
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Analog and Mixed-Signal Circuit Design
- Keywords
- resistive random access memory, neural network training, energy efficiency, latency, area, training accuracy, multiply-accumulate operations, SRAM comparison, CMOS scaling, performance per watt
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
