Weight Programming in DNN Analog Hardware Accelerators in the Presence of NVM Variability
Abstract
Crossbar arrays of nonvolatile memory (NVM) can potentially accelerate development of deep neural networks (DNNs) by implementing crucial multiply–accumulate (MAC) operations at the location of data. Effective weight‐programming procedures can both minimize the performance impact during training and reduce the down time for inference, where new parameter sets may need to be loaded. Simultaneous weight programming along an entire dimension (e.g., row or column) of a crossbar array in the context of forward inference and training is shown to be important. A framework for determining the optimal hardware conditions in which to program weights is provided, and its efficacy in the presence of considerable NVM variability is explored through simulations. This strategy is shown capable of programming 98–99% of weights effectively, in a manner that is both largely independent of the target weight distribution and highly tolerant to variability in NVM conductance‐versus‐pulse characteristics. The probability that a weight fails to reach its target value, termed Pfail, is quantified and the fundamental trade‐off between Pfail and weight programming speed is explored. Lastly, the impact of imperfectly programmed weights on DNN test accuracies is examined for various networks, including multi‐layer perceptrons (MLPs) and long short‐term memory (LSTM).
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
These issues cause both MVM read-out and the writing or programming of the conductances to be erroneous and non-deterministic.
“These issues cause both MVM read-out31 and the writing or programming of the conductances32–34 to be erroneous and non-deterministic.”
From Hardware-aware training for large-scale and diverse deep learning inference workloads using in-memory computing-based accelerators (2023) · cites this paper as [34]
Cites
Equivalent-accuracy accelerated neural-network training using analogue memory
Ambrogio, S., Narayanan, P., Tsai, H., Shelby, R. M., Boybat, I., di Nolfo, C., Sidler, S., Giordano, M., Bodini, M., Farinha, N. C. P., et al. (2018) · Nature · doi:10.1038/s41586-018-0180-5 · Cited by 1,151 · see citing works
Self-citationBurr, G. W., Shelby, R. M., Sidler, S., di Nolfo, C., Jang, J., Boybat, I., Shenoy, R. S., Narayanan, P., Virwani, K., Giacometti, E. U., et al. (2015) · IEEE Transactions on Electron Devices · doi:10.1109/ted.2015.2439635 · Cited by 936 · see citing works
Self-citationNeuromorphic computing with multi-memristive synapses
Boybat, I., Le Gallo, M., Nandakumar, S. R., Moraitis, T., Parnell, T., Tuma, T., Rajendran, B., Leblebici, Y., Sebastian, A., & Eleftheriou, E. (2018) · Nature Communications · doi:10.1038/s41467-018-04933-y · Cited by 903 · see citing works
High-Performance Mixed-Signal Neurocomputing With Nanoscale Floating-Gate Memory Cell Arrays
Merrikh-Bayat, F., Guo, X., Klachko, M., Prezioso, M., Likharev, K. K., & Strukov, D. B. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2017.2778940 · Cited by 167 · see citing works
Programming algorithms for multilevel phase-change memory
Papandreou, N., Pozidis, H., Pantazi, A., Sebastian, A., Breitwisch, M., Lam, C., & Eleftheriou, E. (2011) · 2011 IEEE International Symposium of Circuits and Systems (ISCAS) · doi:10.1109/iscas.2011.5937569 · Cited by 124 · see citing works
ReRAM-based synaptic device for neuromorphic computing
Jang, J.-W., Park, S., Jeong, Y.-H., & Hwang, H. (2014) · 2014 IEEE International Symposium on Circuits and Systems (ISCAS) · doi:10.1109/iscas.2014.6865320 · Cited by 67 · see citing works
Improved Synaptic Behavior of CBRAM Using Internal Voltage Divider for Neuromorphic Systems
Lim, S., Kwak, M., & Hwang, H. (2018) · IEEE Transactions on Electron Devices · doi:10.1109/ted.2018.2857494 · Cited by 40 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Rasch, M. J., Mackin, C., Le Gallo, M., Chen, A., Fasoli, A., Odermatt, F., Li, N., Nandakumar, S. R., Narayanan, P., Tsai, H., et al. (2023) · Nature Communications · doi:10.1038/s41467-023-40770-4 · Cited by 178 · see citing works
“These issues cause both MVM read-out 31 and the writing or programming of the conductances 32 – 34 to be erroneous and non-deterministic.”
BackgroundSelf-citationSong, H., An, J., Son, S., Kim, Y. S., Park, J., Jeon, J. B., Kim, G., & Kim, K. M. (2020) · Advanced Intelligent Systems · doi:10.1002/aisy.202000014 · Cited by 4 · see citing works
- Authors
- Mackin, C., Tsai, H., Ambrogio, S., Narayanan, P., Chen, A., & Burr, G. W.
- Affiliations
- IBM Research - Almaden (US)
- IBM Research–Almaden 650 Harry Road San Jose CA 95120 USA
- Corresponding author
- Mackin, Charles
- Venue
- Advanced Electronic Materials
- Publisher
- Wiley
- Published
- 2019-04-23
- Volume
- 5
- Issue
- 9
- ISSN
- 2199-160X (ISSN-L)
- OpenAlex
- W2942077180
- Cited by
- 41
- References
- 7
- Citations by year
- 2025: 3 · 2024: 4 · 2023: 2 · 2022: 7 · 2021: 11 · 2020: 11 · 2019: 3
- FWCI
- 2.88
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Advanced Electronic Materials (publishedVersion)
- Topics
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Advanced Memory and Neural Computing
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Ferroelectric and Negative Capacitance Devices
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- non-volatile memory, crossbar arrays, deep neural network, multiply-accumulate operations, inference acceleration, multilayer perceptron, long short-term memory
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
