Immunity to Device Variations in a Spiking Neural Network With Memristive Nanodevices
Open accessThe DOI matches its Crossref record; 49 of the 49 references it cites were found in a registry or an index.
Abstract
Memristive nanodevices can feature a compact multilevel nonvolatile memory function, but are prone to device variability. We propose a novel neural network-based computing paradigm, which exploits their specific physics, and which has virtual immunity to their variability. Memristive devices are used as synapses in a spiking neural network performing unsupervised learning. They learn using a simplified and customized “spike timing dependent plasticity” rule. In the network, neurons' threshold is adjusted following a homeostasis-type rule. We perform system level simulations with an experimentally verified model of the memristive devices' behavior. They show, on the textbook case of character recognition, that performance can compare with traditional supervised networks of similar complexity. They also show that the system can retain functionality with extreme variations of various memristive devices' parameters (a relative standard dispersion of more than 50% is tolerated on all device parameters), thanks to the robustness of the scheme, its unsupervised nature, and the capability of homeostasis. Additionally the network can adjust to stimuli presented with different coding schemes, is particularly robust to read disturb effects and does not require unrealistic control on the devices' conductance. These results open the way for a novel design approach for ultraadaptive electronic systems.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Strukov, D. B., Snider, G. S., Stewart, D. R., & Williams, R. S. (2008) · Nature · doi:10.1038/nature06932 · Cited by 11,828
Nanoscale Memristor Device as Synapse in Neuromorphic Systems
Jo, S. H., Chang, T., Ebong, I., Bhadviya, B. B., Mazumder, P., & Lu, W. (2010) · Nano Letters · doi:10.1021/nl904092h · Cited by 4,174 · cited 10×
“It has been suggested [9], [10], [11], [12], and shown experimentally [13], [14], [15], [16], [17], [18], that such devices could reproduce a learning rule of biological synapses – spike timing dependent plasticity (STDP) [19], [20] – that is believed to be a foundation of learning in the brain [21].”
BackgroundInfluentialRegulation of Synaptic Efficacy by Coincidence of Postsynaptic APs and EPSPs
Markram, H., Lübke, J., Frotscher, M., & Sakmann, B. (1997) · Science · doi:10.1126/science.275.5297.213 · Cited by 3,832
A 128$\times$128 120 dB 15 $\mu$s Latency Asynchronous Temporal Contrast Vision Sensor
Lichtsteiner, P., Posch, C., & Delbruck, T. (2008) · IEEE Journal of Solid-State Circuits · doi:10.1109/jssc.2007.914337 · Cited by 2,485
‘Memristive’ switches enable ‘stateful’ logic operations via material implication
Borghetti, J., Snider, G. S., Kuekes, P. J., Yang, J. J., Stewart, D. R., & Williams, R. S. (2010) · Nature · doi:10.1038/nature08940 · Cited by 1,929 · cited 3×
“It should be noted that memristive currently devices developed in academic groups may have relative standard variations in Gmin, Gmax in the range between 50% or 100% ([4], [26] suppl.”
BackgroundSynaptic Modification by Correlated Activity: Hebb's Postulate Revisited
Bi, G.-Q., & Poo, M.-M. (2001) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.24.1.139 · Cited by 1,489
Complementary resistive switches for passive nanocrossbar memories
Linn, E., Rosezin, R., Kügeler, C., & Waser, R. (2010) · Nature Materials · doi:10.1038/nmat2748 · Cited by 1,274
Nanoelectronic programmable synapses based on phase change materials for brain-inspired computing
Kuzum, D., Jeyasingh, R. G. D., Lee, B., & Wong, H.-S. P. (2012) · Nano Letters · doi:10.1021/nl201040y · Cited by 1,267 · cited 3×
“It has been suggested [9], [10], [11], [12], and shown experimentally [13], [14], [15], [16], [17], [18], that such devices could reproduce a learning rule of biological synapses – spike timing dependent plasticity (STDP) [19], [20] – that is believed to be a foundation of learning in the brain [21].”
BackgroundVariability, compensation and homeostasis in neuron and network function
Marder, E., & Goaillard, J.-M. (2006) · Nature Reviews Neuroscience · doi:10.1038/nrn1949 · Cited by 1,212
Deep, Big, Simple Neural Nets for Handwritten Digit Recognition
Cireşan, D. C., Meier, U., Gambardella, L. M., & Schmidhuber, J. (2010) · Neural Computation · doi:10.1162/neco_a_00052 · Cited by 1,078
Spike Timing-Dependent Plasticity of Neural Circuits
Dan, Y., & Poo, M.-M. (2004) · Neuron · doi:10.1016/j.neuron.2004.09.007 · Cited by 1,040
Indiveri, G., Chicca, E., & Douglas, R. (2006) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2005.860850 · Cited by 1,009
Kim, K.-H., Gaba, S., Wheeler, D., Cruz-Albrecht, J. M., Hussain, T., Srinivasa, N., & Lu, W. (2012) · Nano Letters · doi:10.1021/nl203687n · Cited by 924
A Defect-Tolerant Computer Architecture: Opportunities for Nanotechnology
Heath, J. R., Kuekes, P. J., Snider, G. S., & Williams, R. S. (1998) · Science · doi:10.1126/science.280.5370.1716 · Cited by 889
Yu, S., Wu, Y., Jeyasingh, R., Kuzum, D., & Wong, H.-S. P. (2011) · IEEE Transactions on Electron Devices · doi:10.1109/ted.2011.2147791 · Cited by 805 · cited 3×
“It has been suggested [9], [10], [11], [12], and shown experimentally [13], [14], [15], [16], [17], [18], that such devices could reproduce a learning rule of biological synapses – spike timing dependent plasticity (STDP) [19], [20] – that is believed to be a foundation of learning in the brain [21].”
BackgroundHigh precision tuning of state for memristive devices by adaptable variation-tolerant algorithm
Alibart, F., Gao, L., Hoskins, B. D., & Strukov, D. B. (2012) · Nanotechnology · doi:10.1088/0957-4484/23/7/075201 · Cited by 557 · cited 2×
“resistive RAMs and memristors [1], [2], or adaptive transistors [8], [3].”
BackgroundA digital neurosynaptic core using embedded crossbar memory with 45pJ per spike in 45nm
Merolla, P., Arthur, J., Akopyan, F., Imam, N., Manohar, R., & Modha, D. S. (2011) · 2011 IEEE Custom Integrated Circuits Conference (CICC) · doi:10.1109/cicc.2011.6055294 · Cited by 431
Learning Real-World Stimuli in a Neural Network with Spike-Driven Synaptic Dynamics
Brader, J. M., Senn, W., & Fusi, S. (2007) · Neural Computation · doi:10.1162/neco.2007.19.11.2881 · Cited by 371
An Organic Nanoparticle Transistor Behaving as a Biological Spiking Synapse
Alibart, F., Pleutin, S., Guérin, D., Novembre, C., Lenfant, S., Lmimouni, K., Gamrat, C., & Vuillaume, D. (2010) · Advanced Functional Materials · doi:10.1002/adfm.200901335 · Cited by 352
Self-citationSpike-timing-dependent learning in memristive nanodevices
Snider, G. S. (2008) · 2008 IEEE International Symposium on Nanoscale Architectures · doi:10.1109/nanoarch.2008.4585796 · Cited by 341 · cited 17×
“This kind of robustness to spikes’ precise timing is consistent with computational neuroscience works on STDP [32].”
BackgroundMethodInfluentialModha, D. S., Ananthanarayanan, R., Esser, S. K., Ndirango, A., Sherbondy, A. J., & Singh, R. (2011) · Communications of the ACM · doi:10.1145/1978542.1978559 · Cited by 326
Suri, M., Bichler, O., Querlioz, D., Cueto, O., Perniola, L., Sousa, V., Vuillaume, D., Gamrat, C., & DeSalvo, B. (2011) · 2011 International Electron Devices Meeting · doi:10.1109/iedm.2011.6131488 · Cited by 314 · cited 6×
“It has been suggested [9], [10], [11], [12], and shown experimentally [13], [14], [15], [16], [17], [18], that such devices could reproduce a learning rule of biological synapses – spike timing dependent plasticity (STDP) [19], [20] – that is believed to be a foundation of learning in the brain [21].”
BackgroundInfluentialSelf-citationSeo, K., Kim, I., Jung, S., Jo, M., Park, S., Park, J., Shin, J., Biju, K. P., Kong, J., Lee, K., et al. (2011) · Nanotechnology · doi:10.1088/0957-4484/22/25/254023 · Cited by 288
AER EAR: A Matched Silicon Cochlea Pair With Address Event Representation Interface
Chan, V., Liu, S.-C., & van Schaik, A. (2007) · IEEE Transactions on Circuits and Systems I: Regular Papers · doi:10.1109/tcsi.2006.887979 · Cited by 285
Liang, J., & Wong, H.-S. P. (2010) · IEEE Transactions on Electron Devices · doi:10.1109/ted.2010.2062187 · Cited by 276
Self-organized computation with unreliable, memristive nanodevices
Snider, G. S. (2007) · Nanotechnology · doi:10.1088/0957-4484/18/36/365202 · Cited by 262
Neuromorphic, Digital, and Quantum Computation With Memory Circuit Elements
Pershin, Y. V., & Di Ventra, M. (2012) · Proceedings of the IEEE · doi:10.1109/jproc.2011.2166369 · Cited by 261
Engineering nonlinearity into memristors for passive crossbar applications
Joshua Yang, J., Zhang, M.-X., Pickett, M. D., Miao, F., Paul Strachan, J., Li, W.-D., Yi, W., Ohlberg, D. A. A., Joon Choi, B., Wu, W., et al. (2012) · Applied Physics Letters · doi:10.1063/1.3693392 · Cited by 214
Yu, S., Gao, B., Fang, Z., Yu, H., Kang, J., & Wong, H.-S. P. (2012) · 2012 International Electron Devices Meeting · doi:10.1109/iedm.2012.6479018 · Cited by 213
Visual Pattern Extraction Using Energy-Efficient “2-PCM Synapse” Neuromorphic Architecture
Bichler, O., Suri, M., Querlioz, D., Vuillaume, D., DeSalvo, B., & Gamrat, C. (2012) · IEEE Transactions on Electron Devices · doi:10.1109/ted.2012.2197951 · Cited by 212
Self-citationYu, S., Guan, X., & Wong, H.-S. P. (2011) · 2011 International Electron Devices Meeting · doi:10.1109/iedm.2011.6131572 · Cited by 204
Simulation of a memristor-based spiking neural network immune to device variations
Querlioz, D., Bichler, O., & Gamrat, C. (2011) · The 2011 International Joint Conference on Neural Networks · doi:10.1109/ijcnn.2011.6033439 · Cited by 198
Self-citationNeurons Tune to the Earliest Spikes Through STDP
Guyonneau, R., VanRullen, R., & Thorpe, S. J. (2005) · Neural Computation · doi:10.1162/0899766053429390 · Cited by 176
Bichler, O., Querlioz, D., Thorpe, S. J., Bourgoin, J.-P., & Gamrat, C. (2012) · Neural Networks · doi:10.1016/j.neunet.2012.02.022 · Cited by 160
Self-citationCantley, K. D., Subramaniam, A., Stiegler, H. J., Chapman, R. A., & Vogel, E. M. (2011) · IEEE Transactions on Nanotechnology · doi:10.1109/tnano.2011.2105887 · Cited by 159
Physical aspects of low power synapses based on phase change memory devices
Suri, M., Bichler, O., Querlioz, D., Traoré, B., Cueto, O., Perniola, L., Sousa, V., Vuillaume, D., Gamrat, C., & DeSalvo, B. (2012) · Journal of Applied Physics · doi:10.1063/1.4749411 · Cited by 142
Self-citationTwo‐Terminal Carbon Nanotube Programmable Devices for Adaptive Architectures
Agnus, G., Zhao, W. S., Derycke, V., Filoramo, A., Lhuillier, Y., Lenfant, S., Vuillaume, D., Gamrat, C., & Bourgoin, J. ‐. (2010) · Advanced Materials · doi:10.1002/adma.200902170 · Cited by 118
Self-citationLearning in Silicon: Timing is Everything
Arthur, J. V., & Boahen, K. A. (2005) · Cited by 117
Afifi, A., Ayatollahi, A., & Raissi, F. (2009) · European Conference on Circuit Theory and Design · doi:10.1109/ecctd.2009.5275035 · Cited by 113
(2009) · Cited by 107
From Synapses to Circuitry: Using Memristive Memory to Explore the Electronic Brain
Snider, G. S., Amerson, R., Carter, D., Abdalla, H., Qureshi, M. S., Léveillé, J., Versace, M., Ames, H., Patrick, S., Chandler, B., et al. (2011) · Computer · doi:10.1109/mc.2011.48 · Cited by 106
Robust neural logic block (NLB) based on memristor crossbar array
Chabi, D., Zhao, W. S., Querlioz, D., & Klein, J. (2011) · IEEE/ACM International Symposium on Nanoscale Architectures (NANOARCH) · doi:10.1109/nanoarch.2011.5941495 · Cited by 86
Self-citationOn neuromorphic spiking architectures for asynchronous STDP memristive systems
Pérez‐Carrasco, J. A., Zamarreño-Ramos, C., Serrano‐Gotarredona, T., & Linares-Barranco, B. (2010) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas.2010.5537484 · Cited by 78
Querlioz, D., Zhao, W. S., Dollfus, P., Klein, J., Bichler, O., & Gamrat, C. (2012) · IEEE/ACM International Symposium on Nanoscale Architectures (NANOARCH) · doi:10.1145/2765491.2765528 · Cited by 62
Self-citationLearning with memristive devices: How should we model their behavior?
Querlioz, D., Dollfus, P., Bichler, O., & Gamrat, C. (2011) · IEEE/ACM International Symposium on Nanoscale Architectures (NANOARCH) · doi:10.1109/nanoarch.2011.5941497 · Cited by 62
Self-citationDefect‐tolerant nanoelectronic pattern classifiers
Lee, J. H., & Likharev, K. K. (2007) · International Journal of Circuit Theory and Applications · doi:10.1002/cta.410 · Cited by 53
Instar and outstar learning with memristive nanodevices
Snider, G. S. (2010) · Nanotechnology · doi:10.1088/0957-4484/22/1/015201 · Cited by 35
Implementing homeostatic plasticity in VLSI networks of spiking neurons
Bartolozzi, C., Nikolayeva, O., & Indiveri, G. (2008) · IEEE International Conference on Electronics, Circuits and Systems (ICECS) · doi:10.1109/icecs.2008.4674945 · Cited by 32
Cited by
Unsupervised learning of digit recognition using spike-timing-dependent plasticity
Diehl, P. U., & Cook, M. (2015) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2015.00099 · Cited by 1,452
Deep learning in spiking neural networks
Tavanaei, A., Ghodrati, M., Kheradpisheh, S. R., Masquelier, T., & Maida, A. (2019) · Neural Networks · doi:10.1016/j.neunet.2018.12.002 · Cited by 1,386
Spatio-Temporal Backpropagation for Training High-Performance Spiking Neural Networks
Wu, Y., Deng, L., Li, G., Zhu, J., & Shi, L. (2018) · Frontiers in Neuroscience · doi:10.3389/fnins.2018.00331 · Cited by 1,196
Physics for neuromorphic computing
Marković, D., Mizrahi, A., Querlioz, D., & Grollier, J. (2020) · Nature Reviews Physics · doi:10.1038/s42254-020-0208-2 · Cited by 1,176
Self-citationNeuromorphic computing using non-volatile memory
Burr, G. W., Shelby, R. M., Sebastian, A., Kim, S., Kim, S., Sidler, S., Virwani, K., Ishii, M., Narayanan, P., Fumarola, A., et al. (2017) · Advances in Physics: X · doi:10.1080/23746149.2016.1259585 · Cited by 1,107
Metadata
- Authors
- Querlioz, D., Bichler, O., Dollfus, P., & Gamrat, C.
- Venue
- IEEE Transactions on Nanotechnology
- Published
- 2013-05
- Volume
- 12
- Issue
- 3
- Pages
- 288-295
- ISSN
- 1536-125X (ISSN-L)
- 1941-0085
- OpenAlex
- W2162827630
- Cited by
- 420
- References
- 49
- FWCI
- 9.79
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.