Stochastic weight updates in phase-change memory-based synapses and their influence on artificial neural networks
Abstract
Artificial neural networks (ANN) have become a powerful tool for machine learning. Resistive memory devices can be used for the realization of a non-von Neumann computational platform for ANN training in an area-efficient way. For instance, the conductance values of phase-change memory (PCM) devices can be used to represent synaptic weights and can be updated in-situ according to learning rules. However, non-ideal device characteristics pose challenges to reach competitive classification accuracies. In this paper, we investigate the impact of granularity and stochasticity associated with the conductance changes on ANN performance. Using a PCM prototype chip fabricated in the 90 nm technology node, we present a detailed experimental characterization of the conductance changes. Simulations are done in order to quantify the effect of the experimentally observed conductance change granularity and stochasticity on classification accuracies in a fully connected ANN trained with backpropagation.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Large conductance change granularities and nonlinear conductance responses observed in PCM characterization degrade the performance of neural networks trained with memristive synapses.
“Large conductance change granularities, as well as nonlinear conductance responses, both observed in the PCM characterization performed here, have been shown to degrade the performance of neural networks trained with memristive synapses34,43.”
From Neuromorphic computing with multi-memristive synapses (2018) · cites this paper as [43]
Cites
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · see citing works
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892 · see citing works
Neuromorphic computing using non-volatile memory
Burr, G. W., Shelby, R. M., Sebastian, A., Kim, S., Kim, S., Sidler, S., Virwani, K. R., Ishii, M., Narayanan, P., Fumarola, A., et al. (2016) · Advances in Physics X · doi:10.1080/23746149.2016.1259585 · Cited by 1,158 · see citing works
Self-citationBurr, G. W., Shelby, R. M., Sidler, S., di Nolfo, C., Jang, J., Boybat, I., Shenoy, R. S., Narayanan, P., Virwani, K., Giacometti, E. U., et al. (2015) · IEEE Transactions on Electron Devices · doi:10.1109/ted.2015.2439635 · Cited by 936 · cited 5× · see citing works
“Burr et al. presented an experimental demonstration of a large-scale ANN using PCM devices fabricated in the 180 nm technology node, concluding that the performance of the neural network is quite robust to stochasticity of conductance changes [1].”
BackgroundMethodResultInfluentialSelf-citationRecent Progress in Phase-ChangeMemory Technology
Burr, G. W., BrightSky, M. J., Sebastian, A., Cheng, H.-Y., Wu, J.-Y., Kim, S., Sosa, N. E., Papandreou, N., Lung, H.-L., Pozidis, H., et al. (2016) · IEEE Journal on Emerging and Selected Topics in Circuits and Systems · doi:10.1109/jetcas.2016.2547718 · Cited by 393 · see citing works
Self-citationSuri, M., Bichler, O., Querlioz, D., Cueto, O., Perniola, L., Sousa, V., Vuillaume, D., Gamrat, C., & DeSalvo, B. (2011) · 2011 International Electron Devices Meeting · doi:10.1109/iedm.2011.6131488 · Cited by 314 · see citing works
Crystal growth within a phase change memory cell
Sebastian, A., Gallo, M. L., & Krebs, D. (2014) · Nature Communications · doi:10.1038/ncomms5314 · Cited by 242 · see citing works
Self-citationProjected phase-change memory devices
Koelmans, W. W., Sebastian, A., Jonnalagadda, V. P., Krebs, D., Dellmann, L., & Eleftheriou, E. S. (2015) · Nature Communications · doi:10.1038/ncomms9181 · Cited by 172 · see citing works
Self-citationA compound memristive synapse model for statistical learning through STDP in spiking neural networks
Bill, J., & Legenstein, R. A. (2014) · Frontiers in Neuroscience · doi:10.3389/fnins.2014.00412 · Cited by 104 · see citing works
Device, circuit and system-level analysis of noise in multi-bit phase-change memory
Close, G. F., Frey, U., Breitwisch, M., Lung, H. L., Lam, C., Hagleitner, C., & Eleftheriou, E. (2010) · 2010 International Electron Devices Meeting · doi:10.1109/iedm.2010.5703445 · Cited by 67 · see citing works
Self-citationA collective relaxation model for resistance drift in phase change memory cells
Sebastian, A., Krebs, D., Le Gallo, M., Pozidis, H., & Eleftheriou, E. (2015) · 2015 IEEE International Reliability Physics Symposium · doi:10.1109/irps.2015.7112808 · Cited by 36 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Neuromorphic computing with multi-memristive synapses
Boybat, I., Le Gallo, M., Nandakumar, S. R., Moraitis, T., Parnell, T., Tuma, T., Rajendran, B., Leblebici, Y., Sebastian, A., & Eleftheriou, E. (2018) · Nature Communications · doi:10.1038/s41467-018-04933-y · Cited by 903 · see citing works
Self-citationA phase-change memory model for neuromorphic computing
Nandakumar, S. R., Gallo, M. L., Boybat, I., Rajendran, B., Sebastian, A., & Eleftheriou, E. S. (2018) · Journal of Applied Physics · doi:10.1063/1.5042408 · Cited by 181 · see citing works
Self-citationSignal and noise extraction from analog memory elements for neuromorphic computing
Gong, N., Idé, T., Kim, S., Boybat, I., Sebastian, A., Narayanan, V., & Ando, T. (2018) · Nature Communications · doi:10.1038/s41467-018-04485-1 · Cited by 114 · see citing works
Self-citationEmerging memory technologies for neuromorphic computing
Kim, C.-H., Lim, S., Woo, S. Y., Kang, W.-M., Seo, Y.-T., Lee, S., Lee, S., Kwon, D., Oh, S., Noh, Y., et al. (2018) · Nanotechnology · doi:10.1088/1361-6528/aae975 · Cited by 82 · see citing works
Mixed-precision architecture based on computational memory for training deep neural networks
Nandakumar, S. R., Gallo, M. L., Boybat, I., Rajendran, B., Sebastian, A., & Eleftheriou, E. S. (2018) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas.2018.8351656 · Cited by 68 · see citing works
Self-citationLee, J.-H., Lim, D., Jeong, H., Ma, H., & Shi, L. (2019) · IEEE Transactions on Electron Devices · doi:10.1109/ted.2019.2906249 · Cited by 50 · see citing works
Adaptive Weight Quantization Method for Nonlinear Synaptic Devices
Kwon, D., Lim, S., Bae, J., Lee, S., Kim, H., Kim, C.-H., Park, B., & Lee, J. (2018) · IEEE Transactions on Electron Devices · doi:10.1109/ted.2018.2879821 · Cited by 41 · see citing works
Deep learning acceleration based on in-memory computing
Eleftheriou, E. S., Gallo, M. L., Nandakumar, S. R., Piveteau, C., Boybat, I., Joshi, V., Khaddam-Aljameh, R., Dazzi, M., Giannopoulos, I., Karunaratne, G., et al. (2019) · IBM Journal of Research and Development · doi:10.1147/jrd.2019.2947008 · Cited by 40 · see citing works
Self-citationPark, E.-S., Yoo, C., Kim, W., Ha, M., Jeon, J. W., Eom, T., Lee, Y., & Hwang, C. S. (2019) · Chemistry of Materials · doi:10.1021/acs.chemmater.9b02533 · Cited by 28 · see citing works
Zhao, R., He, M., Wang, L., Chen, Z., Cheng, X., Tong, H., & Miao, X. (2022) · Science China Materials · doi:10.1007/s40843-022-2028-7 · Cited by 21 · see citing works
Reliability Challenges with Materials for Analog Computing
Cartier, E., Kim, W., Gong, N., Gokmen, T., Frank, M., Bishop, D. M., Kim, Y., Kim, S., Ando, T., Wu, E., et al. (2019) · IEEE International Reliability Physics Symposium · doi:10.1109/irps.2019.8720599 · Cited by 17 · see citing works
Self-citationProgress and challenges of phase change memory for in-memory computing
Su, X., Guo, Z., Zhang, G., Tsai, H.-Y. S., & Li, N. (2025) · Current Opinion in Solid State and Materials Science · doi:10.1016/j.cossms.2025.101225 · Cited by 13 · see citing works
Multi-ReRAM Synapses for Artificial Neural Network Training
Boybat, I., Giovinazzo, C., Shahrabi, E., Krawczuk, I., Giannopoulos, I., Piveteau, C., Gallo, M. L., Ricciardi, C., Sebastian, A., Eleftheriou, E. S., et al. (2019) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas.2019.8702714 · Cited by 12 · see citing works
Self-citationEfficient Hybrid Training Method for Neuromorphic Hardware Using Analog Nonvolatile Memory
Kwon, D., Woo, S. Y., Hwang, J., Kim, H., Bae, J., Shin, W., Park, B., & Lee, J. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3327906 · Cited by 9 · see citing works
A Hybrid Weight Quantization Strategy for Memristive Neural Networks
Shen, S., Duan, S., & Wang, L. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.126778 · Cited by 8 · see citing works
An efficient synaptic architecture for artificial neural networks
Boybat, I., Gallo, M. L., Nandakumar, S. R., Moraitis, T., Tůma, T., Rajendran, B., Leblebici, Y., Sebastian, A., & Eleftheriou, E. S. (2017) · Non-Volatile Memory Technology Symposium · doi:10.1109/nvmts.2017.8171302 · Cited by 3 · see citing works
Self-citationPerformance Analysis of Spiking RBM with Measurement-Based Phase Change Memory Model
Ishii, M., Ito, M., Kim, W., Kim, S., Nomura, A., Okazaki, A., Okazawa, J., Hosokawa, K., BrightSky, M. J., & Hänsch, W. (2019) · Communications in computer and information science · doi:10.1007/978-3-030-36802-9_63 · Cited by 1 · see citing works
POD: PCM-Based Computing Platform for Object Detection in Biomedical Imaging Application
Chen, D., Amirsoleimani, A., Azghadi, M. R., Genov, R., & Ahmadi, M. A. (2024) · IEEE Latin American Symposium on Circuits and Systems (LASCAS) · doi:10.1109/lascas60203.2024.10506128 · Cited by 1 · see citing works
Baig, M. A., Lu, H., Tsai, C.-H., Hung, W.-C., Le, H., De, S., Chen, N.-Y., Lee, W., Lin, I.-C., Chang, D.-W., et al. (2023) · doi:10.36227/techrxiv.23787108.v1
Baig, M. A., Lu, H., Tsai, C.-H., Hung, W.-C., Le, H., De, S., Chen, N.-Y., Lee, W., Lin, I.-C., Chang, D.-W., et al. (2023) · doi:10.36227/techrxiv.23787108
- Authors
- Boybat, I., Le Gallo, M., Moraitis, T., Leblebici, Y., Sebastian, A., & Eleftheriou, E.
- Affiliations
- IBM Research - Zurich (CH)
- École Polytechnique Fédérale de Lausanne (CH)
- Venue
- 2017 13th Conference on Ph.D. Research in Microelectronics and Electronics (PRIME)
- Publisher
- IEEE
- Published
- 2017-06
- Pages
- 13-16
- OpenAlex
- W2734907865
- Cited by
- 20
- References
- 11
- Citations by year
- 2025: 1 · 2024: 1 · 2023: 4 · 2022: 1 · 2019: 6 · 2018: 6 · 2017: 1
- FWCI
- 5.03
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Conference on Ph.D. Research in Microelectronics and Electronics (publishedVersion)
- Funders
- Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung
- Topics
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Advanced Memory and Neural Computing
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Ferroelectric and Negative Capacitance Devices
- Physical Sciences › Materials Science › Materials Chemistry › Phase-change materials and chalcogenides
- Keywords
- phase-change memory, resistive memory devices, synaptic weights, conductance change, randomness, granularity, artificial neural networks, non-von Neumann computing, backpropagation training, classification accuracy, device characterization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
