Perspective on training fully connected networks with resistive memories: Device requirements for multiple conductances of varying significance
Open accessThe DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
Abstract
Novel Deep Neural Network (DNN) accelerators based on crossbar arrays of non-volatile memories (NVMs)—such as Phase-Change Memory or Resistive Memory—can implement multiply-accumulate operations in a highly parallelized fashion. In such systems, computation occurs in the analog domain at the location of weight data encoded into the conductances of the NVM devices. This allows DNN training of fully-connected layers to be performed faster and with less energy. Using a mixed-hardware-software experiment, we recently showed that by encoding each weight into four distinct physical devices—a “Most Significant Conductance” pair (MSP) and a “Least Significant Conductance” pair (LSP)—we can train DNNs to software-equivalent accuracy despite the imperfections of real analog memory devices. We surmised that, by dividing the task of updating and maintaining weight values between the two conductance pairs, this approach should significantly relax the otherwise quite stringent device requirements. In this paper, we quantify these relaxed requirements for analog memory devices exhibiting a saturating conductance response, assuming either an immediate or a delayed steep initial slope in conductance change. We discuss requirements on the LSP imposed by the “Open Loop Tuning” performed after each training example and on the MSP due to the “Closed Loop Tuning” performed periodically for weight transfer between the conductance pairs. Using simulations to evaluate the final generalization accuracy of a trained four-neuron-layer fully-connected network, we quantify the required dynamic range (as controlled by the size of the steep initial jump), the tolerable device-to-device variability in both maximum conductance and maximum conductance change, the tolerable pulse-to-pulse variability in conductance change, and the tolerable device yield, for both the LSP and MSP devices. We also investigate various Closed Loop Tuning strategies and describe the impact of the MSP/LSP approach on device endurance.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892
Burr, G. W., Shelby, R. M., Sidler, S., di Nolfo, C., Jang, J., Boybat, I., Shenoy, R. S., Narayanan, P., Virwani, K., Giacometti, E. U., et al. (2015) · IEEE Transactions on Electron Devices · doi:10.1109/ted.2015.2439635 · Cited by 936
Self-citationFace classification using electronic synapses
Yao, P., Wu, H., Gao, B., Eryilmaz, S. B., Huang, X., Zhang, W., Zhang, Q., Deng, N., Shi, L., Wong, H.-S. P., et al. (2017) · Nature Communications · doi:10.1038/ncomms15199 · Cited by 909
Woo, J., Moon, K., Song, J., Lee, S., Kwak, M., Park, J., & Hwang, H. (2016) · IEEE Electron Device Letters · doi:10.1109/led.2016.2582859 · Cited by 512
Gokmen, T., & Vlasov, Y. (2016) · Frontiers in Neuroscience · doi:10.3389/fnins.2016.00333 · Cited by 491
Bayat, F. M., Prezioso, M., Chakrabarti, B., Nili, H., Kataeva, I., & Strukov, D. (2018) · Nature Communications · doi:10.1038/s41467-018-04482-4 · Cited by 377
Optimization of Conductance Change in Pr1–xCaxMnO3-Based Synaptic Devices for Neuromorphic Systems
Jang, J.-W., Park, S., Burr, G. W., Hwang, H., & Jeong, Y.-H. (2015) · IEEE Electron Device Letters · doi:10.1109/led.2015.2418342 · Cited by 310
Self-citation
Cited by
Ielmini, D., & Ambrogio, S. (2020) · Nanotechnology · doi:10.1088/1361-6528/ab554b · Cited by 321
Self-citationAI hardware acceleration with analog memory: Microarchitectures for low energy at high speed
Chang, H.-Y., Narayanan, P., Lewis, S. C., Farinha, N. C. P., Hosokawa, K., Mackin, C., Tsai, H., Ambrogio, S., Chen, A., & Burr, G. W. (2019) · IBM Journal of Research and Development · doi:10.1147/jrd.2019.2934050 · Cited by 62
Self-citationWeight Programming in DNN Analog Hardware Accelerators in the Presence of NVM Variability
Mackin, C., Tsai, H., Ambrogio, S., Narayanan, P., Chen, A., & Burr, G. W. (2019) · Advanced Electronic Materials · doi:10.1002/aelm.201900026 · Cited by 41
Self-citationNg, S., John, R. A., Yang, J.-T., & Mathews, N. (2020) · ACS Applied Electronic Materials · doi:10.1021/acsaelm.0c00002 · Cited by 18
Metadata
- Authors
- Cristiano, G., Giordano, M., Ambrogio, S., Romero, L. P., Cheng, C., Narayanan, P., Tsai, H., Shelby, R. M., & Burr, G. W.
- Venue
- Journal of Applied Physics
- Publisher
- AIP Publishing
- Published
- 2018-09-25
- Volume
- 124
- Issue
- 15
- ISSN
- 0021-8979 (ISSN-L)
- 1089-7550
- OpenAlex
- W2893064244
- Cited by
- 37
- References
- 9
- FWCI
- 2.54
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Perspective on training fully connected networks with resistive memories: Device requirements for multiple conductances of varying significance. https://app.cochat.ai/papers/10.1063/1.5042462
- The paper
- Cristiano, G., Giordano, M., Ambrogio, S., Romero, L. P., Cheng, C., Narayanan, P., Tsai, H., Shelby, R. M., & Burr, G. W. (2018). Perspective on training fully connected networks with resistive memories: Device requirements for multiple conductances of varying significance. Journal of Applied Physics. https://doi.org/10.1063/1.5042462
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.