Training a Probabilistic Graphical Model With Resistive Switching Electronic Synapses
Open accessThe DOI matches its Crossref record; 44 of the 44 references it cites were found in a registry or an index.
Abstract
Current large-scale implementations of deep learning and data mining require thousands of processors, massive amounts of off-chip memory, and consume gigajoules of energy. New memory technologies, such as nanoscale two-terminal resistive switching memory devices, offer a compact, scalable, and low-power alternative that permits on-chip colocated processing and memory in fine-grain distributed parallel architecture. Here, we report the first use of resistive memory devices for implementing and training a restricted Boltzmann machine (RBM), a generative probabilistic graphical model as a key component for unsupervised learning in deep networks. We experimentally demonstrate a 45-synapse RBM realized with 90 resistive phase change memory (PCM) elements trained with a bioinspired variant of the contrastive divergence algorithm, implementing Hebbian and anti-Hebbian weight updates. The resistive PCM devices show a twofold to tenfold reduction in error rate in a missing pixel pattern completion task trained over 30 epochs, compared with untrained case. Measured programming energy consumption is 6.1 nJ per epoch with the PCM devices, a factor of ~ 150 times lower than the conventional processor-memory systems. We analyze and discuss the dependence of learning performance on cycle-to-cycle variations and number of gradual levels in the PCM analog memory devices.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“the NeuRRAM chip achieves 1.6-times to 2.3-times lower EDP and 7-times to 13-times higher computational density (measured by throughput per million of RRAMs) at various MVM input and output bit-precisions than previous state-of-the-art RRAM-based CIM chips, despite being fabricated at an older technology node17–27,36.”
From A compute-in-memory chip based on resistive random-access memory (2022) · cites this paper as [25]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
The Internet of Things: A survey
Atzori, L., Iera, A., & Morabito, G. (2010) · Computer Networks · doi:10.1016/j.comnet.2010.05.010 · Cited by 15,613
Representation Learning: A Review and New Perspectives
Bengio, Y., Courville, A., & Vincent, P. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.50 · Cited by 13,001
Training Products of Experts by Minimizing Contrastive Divergence
Hinton, G. E. (2002) · Neural Computation · doi:10.1162/089976602760128018 · Cited by 4,879
A million spiking-neuron integrated circuit with a scalable communication network and interface
Merolla, P. A., Arthur, J. V., Alvarez-Icaza, R., Cassidy, A. S., Sawada, J., Akopyan, F., Jackson, B. L., Imam, N., Guo, C., Nakamura, Y., et al. (2014) · Science · doi:10.1126/science.1254642 · Cited by 4,147
Training and operation of an integrated neuromorphic network based on metal-oxide memristors
Prezioso, M., Merrikh-Bayat, F., Hoskins, B. D., Adam, G. C., Likharev, K. K., & Strukov, D. B. (2015) · Nature · doi:10.1038/nature14441 · Cited by 3,001
Human-level concept learning through probabilistic program induction
Lake, B. M., Salakhutdinov, R., & Tenenbaum, J. B. (2015) · Science · doi:10.1126/science.aab3050 · Cited by 2,817
EIE: efficient inference engine on compressed deep neural network
Han, S., Liu, X., Mao, H., Pu, J., Pedram, A., Horowitz, M. A., & Dally, W. J. (2016) · ACM SIGARCH Computer Architecture News · doi:10.1145/3007787.3001163 · Cited by 1,937
Short-term plasticity and long-term potentiation mimicked in single inorganic synapses
Ohno, T., Hasegawa, T., Tsuruoka, T., Terabe, K., Gimzewski, J. K., & Aono, M. (2011) · Nature Materials · doi:10.1038/nmat3054 · Cited by 1,793
Building high-level features using large scale unsupervised learning
Le, Q. V. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6639343 · Cited by 1,695
Nanoelectronic programmable synapses based on phase change materials for brain-inspired computing
Kuzum, D., Jeyasingh, R. G. D., Lee, B., & Wong, H.-S. P. (2012) · Nano Letters · doi:10.1021/nl201040y · Cited by 1,267
Self-citationNeurogrid: A Mixed-Analog-Digital Multichip System for Large-Scale Neural Simulations
Benjamin, B. V., Gao, P., McQuinn, E., Choudhary, S., Chandrasekaran, A. R., Bussat, J.-M., Alvarez-Icaza, R., Arthur, J. V., Merolla, P. A., & Boahen, K. (2014) · Proceedings of the IEEE · doi:10.1109/jproc.2014.2313565 · Cited by 1,266
Synaptic electronics: materials, devices and applications
Kuzum, D., Yu, S., & Philip Wong, H.-S. (2013) · Nanotechnology · doi:10.1088/0957-4484/24/38/382001 · Cited by 1,258
Chanthbouala, A., Garcia, V., Cherifi, R. O., Bouzehouane, K., Fusil, S., Moya, X., Xavier, S., Yamada, H., Deranlot, C., Mathur, N. D., et al. (2012) · Nature Materials · doi:10.1038/nmat3415 · Cited by 1,220
EIE: Efficient Inference Engine on Compressed Deep Neural Network
Han, S., Liu, X., Mao, H., Pu, J., Pedram, A., Horowitz, M. A., & Dally, W. J. (2016) · 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA) · doi:10.1109/isca.2016.30 · Cited by 1,077
Memory leads the way to better computing
Wong, H.-S. P., & Salahuddin, S. (2015) · Nature Nanotechnology · doi:10.1038/nnano.2015.29 · Cited by 956
Self-citationRepresentational Power of Restricted Boltzmann Machines and Deep Belief Networks
Le Roux, N., & Bengio, Y. (2008) · Neural Computation · doi:10.1162/neco.2008.04-07-510 · Cited by 821
Yu, S., Wu, Y., Jeyasingh, R., Kuzum, D., & Wong, H.-S. P. (2011) · IEEE Transactions on Electron Devices · doi:10.1109/ted.2011.2147791 · Cited by 805
Self-citation8 Gb 3-D DDR3 DRAM Using Through-Silicon-Via Technology
Kang, U., Chung, H.-J., Heo, S., Park, D.-H., Lee, H., Kim, J. H., Ahn, S.-H., Cha, S.-H., Ahn, J., Kwon, D., et al. (2010) · IEEE Journal of Solid-State Circuits · doi:10.1109/jssc.2009.2034408 · Cited by 485
An integrated GPU power and performance model
Hong, S., & Kim, H. (2010) · Proceedings of the 37th annual international symposium on Computer architecture · doi:10.1145/1815961.1815998 · Cited by 483
Energy management for commercial servers
Lefurgy, C., Rajamani, K., Rawson, F., Felter, W., Kistler, M., & Keller, T. W. (2003) · Computer · doi:10.1109/mc.2003.1250880 · Cited by 473
Hybrid memory cube new DRAM architecture increases density and performance
Jeddeloh, J., & Keeth, B. (2012) · 2012 Symposium on VLSI Technology (VLSIT) · doi:10.1109/vlsit.2012.6242474 · Cited by 444
On the quantitative analysis of deep belief networks
Salakhutdinov, R., & Murray, I. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390266 · Cited by 411
Write Strategies for 2 and 4-bit Multi-Level Phase-Change Memory
Nirschl, T., Chen, C.-F., Joseph, E., Lamorey, M., Cheek, R., Chen, S.-H., Zaidi, S., Raoux, S., Chen, Y. C., Zhu, Y., et al. (2007) · 2007 IEEE International Electron Devices Meeting · doi:10.1109/iedm.2007.4418973 · Cited by 277
Self-citationEnergy-Efficient Abundant-Data Computing: The N3XT 1,000x
Sabry Aly, M. M., Gao, M., Hills, G., Lee, C.-S., Pitner, G., Shulaker, M. M., Wu, T. F., Asheghi, M., Bokor, J., Franchetti, F., et al. (2015) · Computer · doi:10.1109/mc.2015.376 · Cited by 227
Self-citationLee, D.-H., Zhang, S., Fischer, A., & Bengio, Y. (2015) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-23528-8_31 · Cited by 216
Event-driven contrastive divergence for spiking neuromorphic systems
Neftci, E., Das, S., Pedroni, B., Kreutz-Delgado, K., & Cauwenberghs, G. (2014) · Frontiers in Neuroscience · doi:10.3389/fnins.2013.00272 · Cited by 214
Self-citationVisual Pattern Extraction Using Energy-Efficient “2-PCM Synapse” Neuromorphic Architecture
Bichler, O., Suri, M., Querlioz, D., Vuillaume, D., DeSalvo, B., & Gamrat, C. (2012) · IEEE Transactions on Electron Devices · doi:10.1109/ted.2012.2197951 · Cited by 212
Brain-like associative learning using a nanoscale non-volatile phase change synaptic device array
Eryilmaz, S. B., Kuzum, D., Jeyasingh, R., Kim, S., BrightSky, M., Lam, C., & Wong, H.-S. P. (2014) · Frontiers in Neuroscience · doi:10.3389/fnins.2014.00205 · Cited by 205
Self-citationA 1.2 V 12.8 GB/s 2 Gb Mobile Wide-I/O DRAM With 4 $\times$ 128 I/Os Using TSV Based Stacking
Kim, J.-S., Oh, C. S., Lee, H., Lee, D., Hwang, H. R., Hwang, S., Na, B., Moon, J., Kim, J.-G., Park, H., et al. (2012) · IEEE Journal of Solid-State Circuits · doi:10.1109/jssc.2011.2164731 · Cited by 204
Self-citationKim, S., Ishii, M., Lewis, S., Perri, T., BrightSky, M., Kim, W., Jordan, R., Burr, G. W., Sosa, N., Ray, A., et al. (2015) · 2015 IEEE International Electron Devices Meeting (IEDM) · doi:10.1109/iedm.2015.7409716 · Cited by 194
Self-citationA 45nm generation Phase Change Memory technology
Servalli, G. (2009) · 2009 IEEE International Electron Devices Meeting (IEDM) · doi:10.1109/iedm.2009.5424409 · Cited by 192
Electronic system with memristive synapses for pattern recognition
Park, S., Chu, M., Kim, J., Noh, J., Jeon, M., Hun Lee, B., Hwang, H., Lee, B., & Lee, B.-G. (2015) · Scientific Reports · doi:10.1038/srep10123 · Cited by 181
Monolithic 3D integration of logic and memory: Carbon nanotube FETs, resistive RAM, and silicon FETs
Shulaker, M. M., Wu, T. F., Pal, A., Zhao, L., Nishi, Y., Saraswat, K., Wong, H.-S. P., & Mitra, S. (2014) · 2014 IEEE International Electron Devices Meeting · doi:10.1109/iedm.2014.7047120 · Cited by 169
Self-citationNonvolatile 3D-FPGA with monolithically stacked RRAM-based configuration memory
Liauw, Y. Y., Zhang, Z., Kim, W., Gamal, A. E., & Wong, S. S. (2012) · 2012 IEEE International Solid-State Circuits Conference · doi:10.1109/isscc.2012.6177067 · Cited by 150
EIE: Efficient Inference Engine on Compressed Deep Neural NetworkPreprint
Han, S., Liu, X., Mao, H., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.01528 · Cited by 137
Kang, S., Cho, W. Y., Cho, B.-H., Lee, K., Lee, C., Oh, H.-R., Choi, B.-G., Wang, Q., Kim, H.-J., Park, M.-H., et al. (2007) · IEEE Journal of Solid-State Circuits · doi:10.1109/jssc.2006.888349 · Cited by 134
Self-citationLiu, B., Zhang, T., Xia, J., et al. (2004) · Semiconductor Science and Technology · doi:10.1088/0268-1242/19/6/l01 · Cited by 80
Eryilmaz, S. B., Kuzum, D., Yu, S., & Wong, H.-S. P. (2015) · 2015 IEEE International Electron Devices Meeting (IEDM) · doi:10.1109/iedm.2015.7409622 · Cited by 75
Self-citationKuzum, D., Jeyasingh, R. G. D., & Wong, H.-S. P. (2011) · 2011 International Electron Devices Meeting · doi:10.1109/iedm.2011.6131643 · Cited by 63
Self-citationChen, P.-Y., Kadetotad, D., Xu, Z., Mohanty, A., Lin, B., Ye, J., Vrudhula, S., Seo, J.-S., Cao, Y., & Yu, S. (2015) · Design, Automation & Test in Europe Conference & Exhibition (DATE), 2015 · doi:10.7873/date.2015.0620 · Cited by 61
Stacked chalcogenide layers used as multi-state storage medium for phase change memory
Lai, Y., Feng, J., Qiao, B., et al. (2006) · Applied Physics A · doi:10.1007/s00339-006-3571-7 · Cited by 46
Demonstration of CAM and TCAM Using Phase Change Devices
Rajendran, B., Cheek, R. W., Lastras, L., Franceschini, M., Breitwisch, M., Schrott, A. G., Li, J., Montoye, R. K., Chang, L., & Lam, C. H. (2011) · IEEE International Memory Workshop (IMW) · doi:10.1109/imw.2011.5873229 · Cited by 33
Self-citation
Research works printed but not linked: 9
- burr, 2014, Experimental demonstration and tolerancing of a large scale neural network, Proc IEEE IEDM, 29.5.1
- erhan, 2010, Why does unsupervised pre-training help deep learning?, J Mach Learn Res, 11, 625
- wong, 2016, Stanford Memory Trends
- shao, 2013, Energy characterization and instruction-level energy model of Intel’s Xeon Phi processor, Proc ISLPED, 389
- yu, 2012, 65k-neuron integrate and fire array transceiver with address-event reconfigurable synaptic routing, Proc IEEE Biomed Circuits Syst Conf (BioCAS), 21
- natarajan, 2014, A 14nm logic technology featuring 2nd-generation FinFET, air-gapped interconnects, self-aligned double patterning and a 0.0588 $\mu $ m2 SRAM cell size, Proc IEEE Int Electron Devices Meeting, 3.7.1
- 2016, 16-Channel Current-Input Analog-to-Digital Converter
- 2016, Dual current input 20-bit analog-to-digital converter
- mackay, 2003, Information Theory Inference and Learning Algorithms
Cited by
Neuro-Inspired Computing With Emerging Nonvolatile Memorys
Yu, S. (2018) · Proceedings of the IEEE · doi:10.1109/jproc.2018.2790840 · Cited by 1,170
A compute-in-memory chip based on resistive random-access memory
Wan, W., Kubendran, R., Schaefer, C., Eryilmaz, S. B., Zhang, W., Wu, D., Deiss, S., Raina, P., Qian, H., Gao, B., et al. (2022) · Nature · doi:10.1038/s41586-022-04992-8 · Cited by 943
Self-citationThe Future of Memristors: Materials Engineering and Neural Networks
Sun, K., Chen, J., & Yan, X. (2021) · Advanced Functional Materials · doi:10.1002/adfm.202006773 · Cited by 482
Large-Scale Neuromorphic Spiking Array Processors: A Quest to Mimic the Brain
Thakur, C. S., Molin, J. L., Cauwenberghs, G., Indiveri, G., Kumar, K., Qiao, N., Schemmel, J., Wang, R., Chicca, E., Olson Hasler, J., et al. (2018) · Frontiers in Neuroscience · doi:10.3389/fnins.2018.00891 · Cited by 288
Self-citationFrom biomaterial-based data storage to bio-inspired artificial synapse
Lv, Z., Zhou, Y., Han, S.-T., & Roy, V. A. L. (2018) · Materials Today · doi:10.1016/j.mattod.2017.12.001 · Cited by 261
Metadata
- Authors
- Eryilmaz, S. B., Neftci, E., Joshi, S., Kim, S., BrightSky, M., Lung, H.-L., Lam, C., Cauwenberghs, G., & Wong, H.-S. P.
- Affiliations
- Stanford University (US)
- University of California, Irvine (US)
- University of California San Diego (US)
- IBM (United States) (US)
- Macronix International (Taiwan) (TW)
- Venue
- IEEE Transactions on Electron Devices
- Published
- 2016-12
- Volume
- 63
- Issue
- 12
- Pages
- 5004-5011
- ISSN
- 0018-9383 (ISSN-L)
- 1557-9646
- OpenAlex
- W2526812109
- Cited by
- 34
- References
- 44
- Citations by year
- 2025: 1 · 2022: 5 · 2021: 3 · 2020: 6 · 2019: 8 · 2018: 8 · 2017: 3
- FWCI
- 2.92
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Transactions on Electron Devices (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- eScholarship (California Digital Library) (submittedVersion)
- arXiv (Cornell University)
- Funders
- SONIC, one of six centers of STARnet, a Semiconductor Research Corporation Program sponsored by MARCO and DARPA
- NSF Expedition on Computing under Visual Cortex on Silicon (1317470)
- Member Companies of the Stanford Non-Volatile Memory Technology Research Initiative and the Stanford SystemX Alliance
- Topics
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Advanced Memory and Neural Computing
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Ferroelectric and Negative Capacitance Devices
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Reservoir Computing
- Keywords
- resistive random access memory, phase-change memory, restricted Boltzmann machine, probabilistic graphical models, neuromorphic computing, contrastive divergence, Hebbian learning, unsupervised learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.