Learning Real-World Stimuli in a Neural Network with Spike-Driven Synaptic Dynamics
Open accessThe DOI matches its Crossref record; 80 of the 80 references it cites were found in a registry or an index.
Abstract
We present a model of spike-driven synaptic plasticity inspired by experimental observations and motivated by the desire to build an electronic hardware device that can learn to classify complex stimuli in a semisupervised fashion. During training, patterns of activity are sequentially imposed on the input neurons, and an additional instructor signal drives the output neurons toward the desired activity. The network is made of integrate-and-fire neurons with constant leak and a floor. The synapses are bistable, and they are modified by the arrival of presynaptic spikes. The sign of the change is determined by both the depolarization and the state of a variable that integrates the postsynaptic action potentials. Following the training phase, the instructor signal is removed, and the output neurons are driven purely by the activity of the input neurons weighted by the plastic synapses. In the absence of stimulation, the synapses preserve their internal state indefinitely. Memories are also very robust to the disruptive action of spontaneous activity. A network of 2000 input neurons is shown to be able to classify correctly a large number (thousands) of highly overlapping patterns (300 classes of preprocessed Latex characters, 30 patterns per class, and a subset of the NIST characters data set) and to generalize with performances that are better than or comparable to those of artificial neural networks. Finally we show that the synaptic dynamics is compatible with many of the experimental observations on the induction of long-term modifications (spike-timing-dependent plasticity and its dependence on both the postsynaptic depolarization and the frequency of pre- and postsynaptic neurons).
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Neural networks and physical systems with emergent collective computational abilities.
Hopfield, J. J. (1982) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.79.8.2554 · Cited by 19,270
The perceptron: A probabilistic model for information storage and organization in the brain.
Rosenblatt, F. (1958) · Psychological Review · doi:10.1037/h0042519 · Cited by 12,280
Introduction to the theory of neural computation
(1994) · Neural Networks · doi:10.1016/0893-6080(94)90014-0 · Cited by 6,401
Shape matching and object recognition using shape contexts
Belongie, S., Malik, J., & Puzicha, J. (2002) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.993558 · Cited by 6,306
Regulation of Synaptic Efficacy by Coincidence of Postsynaptic APs and EPSPs
Markram, H., Lübke, J., Frotscher, M., & Sakmann, B. (1997) · Science · doi:10.1126/science.275.5297.213 · Cited by 3,832
Marr, D. (1969) · The Journal of Physiology · doi:10.1113/jphysiol.1969.sp008820 · Cited by 3,253
A Short Introduction to Boosting
Freund, Y., & Schapire, R. E. (1999) · Cited by 3,141
Best practices for convolutional neural networks applied to visual document analysis
Simard, P. Y., Steinkraus, D., & Platt, J. C. (2005) · Seventh International Conference on Document Analysis and Recognition, 2003. Proceedings. · doi:10.1109/icdar.2003.1227801 · Cited by 2,741
A theory of cerebellar function
Albus, J. S. (1971) · Mathematical Biosciences · doi:10.1016/0025-5564(71)90051-4 · Cited by 2,539 · cited 2×
“Our output neurons are eseentially perceptrons which resemble the cerebellar Purkinje cells, as already proposed by Marr (Marr, 1969) and Albus (Albus, 1971) (see (Brunel et al.”
BackgroundSynaptic plasticity: taming the beast
Abbott, L. F., & Nelson, S. B. (2000) · Nature Neuroscience · doi:10.1038/81453 · Cited by 2,263 · cited 1×
“…of synaptic plasticity based on spike timing (STDP) have been studied both in recurrent neural networks and at the level of single synapses (see Abbott & Nelson, 2000; Kempter, Gerstner, & van Hemmen, 2001; Rubin, Lee, & Sompolinsky, 2001; Burkitt, Meffin, & Grayden, 2004), in which the authors…”
BackgroundModeling Brain Function: The World of Attractor Neural Networks
Amit, D. J. (1989) · Cambridge University Press eBooks · doi:10.1017/cbo9780511623257 · Cited by 2,227
Shape Quantization and Recognition with Randomized Trees
Amit, Y., & Geman, D. (1997) · Neural Computation · doi:10.1162/neco.1997.9.7.1545 · Cited by 1,275
Probabilistic decision making by slow reverberation in cortical circuits
Wang, X.-J. (2002) · Neuron · doi:10.1016/s0896-6273(02)01092-9 · Cited by 1,242
Rate, Timing, and Cooperativity Jointly Determine Cortical Synaptic Plasticity
Sjöström, P. J., Turrigiano, G. G., & Nelson, S. B. (2001) · Neuron · doi:10.1016/s0896-6273(01)00542-6 · Cited by 1,174
A neuronal learning rule for sub-millisecond temporal coding
Gerstner, W., Kempter, R., van Hemmen, J. L., & Wagner, H. (1996) · Nature · doi:10.1038/383076a0 · Cited by 1,110
A critical window for cooperation and competition among developing retinotectal synapses
Zhang, L. I., Tao, H. W., Holt, C. E., Harris, W. A., & Poo, M.-M. (1998) · Nature · doi:10.1038/25665 · Cited by 908
Neurocomputing: Foundations of Research
Anderson, J. A., & Rosenfeld, E. (1988) · Cited by 851
The tempotron: a neuron that learns spike timing–based decisions
Gütig, R., & Sompolinsky, H. (2006) · Nature Neuroscience · doi:10.1038/nn1643 · Cited by 815
Spike-timing-dependent synaptic modification induced by natural spike trains
Froemke, R. C., & Dan, Y. (2002) · Nature · doi:10.1038/416433a · Cited by 803
Hebb and homeostasis in neuronal plasticity
Turrigiano, G. G., & Nelson, S. B. (2000) · Current Opinion in Neurobiology · doi:10.1016/s0959-4388(00)00091-x · Cited by 683
Timing-Based LTP and LTD at Vertical Inputs to Layer II/III Pyramidal Cells in Rat Barrel Cortex
Feldman, D. E. (2000) · Neuron · doi:10.1016/s0896-6273(00)00008-8 · Cited by 658
A unified model of NMDA receptor-dependent bidirectional synaptic plasticity
Shouval, H. Z., Bear, M. F., & Cooper, L. N. (2002) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.152343099 · Cited by 607
Phenomenological models of synaptic plasticity based on spike timing
Morrison, A., Diesmann, M., & Gerstner, W. (2008) · Biological Cybernetics · doi:10.1007/s00422-008-0233-1 · Cited by 593
Cascade Models of Synaptically Stored Memories
Fusi, S., Drew, P. J., & Abbott, L. F. (2005) · Neuron · doi:10.1016/j.neuron.2005.02.001 · Cited by 569 · cited 1×
“If more than one output neuron is read, then nonlinearly separable patterns can also be classified, which is not possible with a simple perceptron.”
BackgroundSelf-citationFuster, J. M. (1999) · Cited by 451
The Perceptron: A Model for Brain Functioning. I
Block, H. D. (1962) · Reviews of Modern Physics · doi:10.1103/revmodphys.34.123 · Cited by 396
A single-transistor silicon synapse
Diorio, C., Hasler, P., Minch, A., & Mead, C. A. (1996) · IEEE Transactions on Electron Devices · doi:10.1109/16.543035 · Cited by 360 · cited 1×
“All the hardware implementations require negligible power consumption to stay in one of the two stable states (Fusi et al., 2000; Indiveri, 2000, 2001, 2002; Mitra et al., 2006; Badoni, Giulioni, Dante, & Del Giudice, 2006) and the bistable circuitry does not require nonstandard technology or high voltage, as for the…”
BackgroundAll-or-none potentiation at CA3-CA1 synapses
Petersen, C. C. H., Malenka, R. C., Nicoll, R. A., & Hopfield, J. J. (1998) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.95.8.4732 · Cited by 329
Equilibrium Properties of Temporally Asymmetric Hebbian Plasticity
Rubin, J., Lee, D. D., & Sompolinsky, H. (2001) · Physical Review Letters · doi:10.1103/physrevlett.86.364 · Cited by 301
Learning in Neural Networks with Material Synapses
Amit, D. J., & Fusi, S. (1994) · Neural Computation · doi:10.1162/neco.1994.6.5.957 · Cited by 290
Self-citationOptimal Information Storage and the Distribution of Synaptic Weights
Brunel, N., Hakim, V., Isope, P., Nadal, J.-P., & Barbour, B. (2004) · Neuron · doi:10.1016/j.neuron.2004.08.023 · Cited by 270
Neocortical Pyramidal Cells Respond as Integrate-and-Fire Neurons to In Vivo–Like Input Currents
Rauch, A., La Camera, G., Lüscher, H.-R., Senn, W., & Fusi, S. (2003) · Journal of Neurophysiology · doi:10.1152/jn.00293.2003 · Cited by 264
Self-citationReal-Time Classification of Complex Patterns Using Spike-Based Learning in Neuromorphic VLSI
Mitra, S., Fusi, S., & Indiveri, G. (2009) · IEEE Transactions on Biomedical Circuits and Systems · doi:10.1109/tbcas.2008.2005781 · Cited by 238
Self-citationSpike-Driven Synaptic Plasticity: Theory, Simulation, VLSI Implementation
Fusi, S., Annunziato, M., Badoni, D., Salamon, A., & Amit, D. J. (2000) · Neural Computation · doi:10.1162/089976600300014917 · Cited by 227
Self-citationSpike-Timing-Dependent Hebbian Plasticity as Temporal Difference Learning
Rao, R. P. N., & Sejnowski, T. J. (2001) · Neural Computation · doi:10.1162/089976601750541787 · Cited by 226
Limits on the memory storage capacity of bounded synapses
Fusi, S., & Abbott, L. F. (2007) · Nature Neuroscience · doi:10.1038/nn1859 · Cited by 224
Self-citationSenn, W., Markram, H., & Tsodyks, M. (2001) · Neural Computation · doi:10.1162/089976601300014628 · Cited by 219
Self-citationNetworks of Formal Neurons and Memory Palimpsests
Nadal, J. P., Toulouse, G., Changeux, J. P., & Dehaene, S. (1986) · Europhysics Letters (EPL) · doi:10.1209/0295-5075/1/10/008 · Cited by 212
What Can a Neuron Learn with Spike-Timing-Dependent Plasticity?
Legenstein, R., Naeger, C., & Maass, W. (2005) · Neural Computation · doi:10.1162/0899766054796888 · Cited by 207
Context-enabled learning in the human visual system
Adini, Y., Sagi, D., & Tsodyks, M. (2002) · Nature · doi:10.1038/415790a · Cited by 174
Intrinsic Stabilization of Output Rates by Spike-Based Hebbian Learning
Kempter, R., Gerstner, W., & Hemmen, J. L. V. (2001) · Neural Computation · doi:10.1162/089976601317098501 · Cited by 171
A Model of Spike-Timing Dependent Plasticity: One or Two Coincidence Detectors?
Karmarkar, U. R., & Buonomano, D. V. (2002) · Journal of Neurophysiology · doi:10.1152/jn.2002.88.1.507 · Cited by 169
A Pulse-Coded Communications Infrastructure for Neuromorphic Systems
Deiss, S. R., Douglas, R. J., & Whatley, A. M. (1998) · Pulsed Neural Networks · doi:10.7551/mitpress/5704.003.0011 · Cited by 168
Parisi, G. (1986) · Journal of Physics A: Mathematical and General · doi:10.1088/0305-4470/19/10/011 · Cited by 165
Collective Behavior of Networks with Linear (VLSI) Integrate-and-Fire Neurons
Fusi, S., & Mattia, M. (1999) · Neural Computation · doi:10.1162/089976699300016601 · Cited by 162
Self-citationCommunicating Neuronal Ensembles between Neuromorphic Chips
Boahen, K. A. (2007) · The Springer International Series in Engineering and Computer Science · doi:10.1007/978-0-585-28001-1_11 · Cited by 159
A Computational Model for Visual Selection
Amit, Y., & Geman, D. (1999) · Neural Computation · doi:10.1162/089976699300016197 · Cited by 158
Reversal and Stabilization of Synaptic Modifications in a Developing Visual System
Zhou, Q., Tao, H. W., & Poo, M.-M. (2003) · Science · doi:10.1126/science.1082212 · Cited by 148
Hebbian spike-driven synaptic plasticity for learning patterns of mean firing rates
Fusi, S. (2002) · Biological Cybernetics · doi:10.1007/s00422-002-0356-8 · Cited by 131
Self-citationPriming-Induced Shift in Synaptic Plasticity in the Rat Hippocampus
Wang, H., & Wagner, J. J. (1999) · Journal of Neurophysiology · doi:10.1152/jn.1999.82.4.2024 · Cited by 125
Dynamical model of long-term synaptic plasticity
Abarbanel, H. D. I., Huerta, R., & Rabinovich, M. I. (2002) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.132651299 · Cited by 115
Minimal Models of Adapted Neuronal Response to In Vivo–Like Input Currents
Camera, G. L., Rauch, A., Lüscher, H., Senn, W., & Fusi, S. (2004) · Neural Computation · doi:10.1162/0899766041732468 · Cited by 108
Self-citationBurkitt, A. N., Meffin, H., & Grayden, D. B. (2004) · Neural Computation · doi:10.1162/089976604773135041 · Cited by 98
Giudice, P. D., Fusi, S., & Mattia, M. (2003) · Journal of Physiology-Paris · doi:10.1016/j.jphysparis.2004.01.021 · Cited by 86
Self-citationModeling Selective Attention Using a Neuromorphic Analog VLSI Device
Indiveri, G. (2000) · Neural Computation · doi:10.1162/089976600300014755 · Cited by 81
Spike-Driven Synaptic Dynamics Generating Working Memory States
Amit, D. J., & Mongillo, G. (2003) · Neural Computation · doi:10.1162/089976603321192086 · Cited by 79
Slow stochastic Hebbian learning of classes of stimuli in a recurrent neural network
Brunel, N., Carusi, F., & Fusi, S. (1998) · Network: Computation in Neural Systems · doi:10.1088/0954-898x/9/1/007 · Cited by 78
Self-citationShouval, H. Z., & Kalantzis, G. (2005) · Journal of Neurophysiology · doi:10.1152/jn.00504.2004 · Cited by 77
How the Shape of Pre- and Postsynaptic Signals Can Influence STDP: A Biophysical Model
Saudargiene, A., Porr, B., & Wörgötter, F. (2004) · Neural Computation · doi:10.1162/089976604772744929 · Cited by 73
Convergence of stochastic learning in perceptrons with binary synapses
Senn, W., & Fusi, S. (2005) · Physical Review E · doi:10.1103/physreve.71.061907 · Cited by 67
Self-citationConstraints on learning in dynamic synapses
Amit, D. J., & Fusi, S. (1992) · Network Computation in Neural Systems · doi:10.1088/0954-898x_3_4_008 · Cited by 59
Self-citationYao, H., Shen, Y., & Dan, Y. (2004) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0302510101 · Cited by 55
ASSOCIATIVE MEMORY IN NEURAL NETWORKS WITH BINARY SYNAPSES
Tsodyks, M. V. (1990) · Modern Physics Letters B · doi:10.1142/s0217984990000891 · Cited by 55
A neuromorphic VLSI device for implementing 2D selective attention systems
Indiveri, G. (2001) · IEEE Transactions on Neural Networks · doi:10.1109/72.963780 · Cited by 53
Buchs, N. J., & Senn, W. (2002) · Journal of Computational Neuroscience · doi:10.1023/a:1020210230751 · Cited by 50
Self-citationLearning in realistic networks of spiking neurons and spike‐driven plastic synapses
Mongillo, G., Curti, E., Romani, S., & Amit, D. J. (2005) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2005.04087.x · Cited by 48
Spike-based learning in VLSI networks of integrate-and-fire neurons
Indiveri, G., & Fusi, S. (2007) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas.2007.378290 · Cited by 45
Self-citationSenn, W., & Fusi, S. (2005) · Neural Computation · doi:10.1162/0899766054615644 · Cited by 45
Self-citationLearning rules and network repair in spike-timing-based computation networks
Hopfield, J. J., & Brody, C. D. (2003) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2536316100 · Cited by 43
A VLSI spike-driven dynamic synapse which learns only when necessary
Mitra, S., Fusi, S., & Indiveri, G. (2006) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas.2006.1693200 · Cited by 35
Self-citationChicca, E., Indiveri, G., & Douglas, R. J. (2003) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas.2003.1205505 · Cited by 35
Spike-driven Synaptic Plasticity for Learning Correlated Patterns of Mean Firing Rates
Fusi, S. (2003) · Reviews in the Neurosciences · doi:10.1515/revneuro.2003.14.1-2.73 · Cited by 28
Self-citationStochastic synaptic plasticity in deterministic aVLSI networks of spiking neurons
Chicca, E., & Fusi, S. (2001) · PUB – Publications at Bielefeld University (Bielefeld University) · Cited by 26
Self-citationLong and short-term synaptic plasticity and the formation of working memory: A case study
Giudice, P. D., & Mattia, M. (2001) · Neurocomputing · doi:10.1016/s0925-2312(01)00557-4 · Cited by 26
Eluding oblivion with smart stochastic selection of synaptic updates
Fusi, S., & Senn, W. (2006) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/1.2213587 · Cited by 25
Self-citationAttractor Networks for Shape Recognition
Amit, Y., & Mascaro, M. (2001) · Neural Computation · doi:10.1162/08997660152002906 · Cited by 24
Constraints on learning in dynamic synapses
Amit, D. J., & Fusi, S. (1992) · Network Computation in Neural Systems · doi:10.1088/0954-898x/3/4/008 · Cited by 23
Self-citationSenn, W., & Buchs, N. J. (2003) · Journal of Computational Neuroscience · doi:10.1023/a:1021935100586 · Cited by 21
Self-citationCoath, M., Brader, J. M., Fusi, S., & Denham, S. L. (2005) · Network Computation in Neural Systems · doi:10.1080/09548980500290120 · Cited by 17
Self-citationLong term memory: Encoding and storing strategies of the brain
Fusi, S. (2001) · Neurocomputing · doi:10.1016/s0925-2312(01)00571-9 · Cited by 14
Self-citation
Cited by
Unsupervised learning of digit recognition using spike-timing-dependent plasticity
Diehl, P. U., & Cook, M. (2015) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2015.00099 · Cited by 1,452
Deep learning in spiking neural networks
Tavanaei, A., Ghodrati, M., Kheradpisheh, S. R., Masquelier, T., & Maida, A. (2019) · Neural Networks · doi:10.1016/j.neunet.2018.12.002 · Cited by 1,386
Nanoelectronic programmable synapses based on phase change materials for brain-inspired computing
Kuzum, D., Jeyasingh, R. G. D., Lee, B., & Wong, H.-S. P. (2012) · Nano Letters · doi:10.1021/nl201040y · Cited by 1,267
Synaptic electronics: materials, devices and applications
Kuzum, D., Yu, S., & Philip Wong, H.-S. (2013) · Nanotechnology · doi:10.1088/0957-4484/24/38/382001 · Cited by 1,258
Neuromorphic computing using non-volatile memory
Burr, G. W., Shelby, R. M., Sebastian, A., Kim, S., Kim, S., Sidler, S., Virwani, K., Ishii, M., Narayanan, P., Fumarola, A., et al. (2017) · Advances in Physics: X · doi:10.1080/23746149.2016.1259585 · Cited by 1,107
Metadata
- Authors
- Brader, J. M., Senn, W., & Fusi, S.
- Venue
- Neural Computation
- Publisher
- MIT Press - Journals
- Published
- 2007-11
- Volume
- 19
- Issue
- 11
- Pages
- 2881-2912
- ISSN
- 0899-7667 (ISSN-L)
- 1530-888X
- OpenAlex
- W2112408199
- Cited by
- 371
- References
- 80
- FWCI
- 7.88
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.