The DOI matches its Crossref record; 252 of the 252 references it cites were found in a registry or an index.
Deep learning in spiking neural networks
Open accessAbstract
In recent years, deep learning has revolutionized the field of machine learning, for computer vision in particular. In this approach, a deep (multilayer) artificial neural network (ANN) is trained, most often in a supervised manner using backpropagation. Vast amounts of labeled training examples are required, but the resulting classification accuracy is truly impressive, sometimes outperforming humans. Neurons in an ANN are characterized by a single, static, continuous-valued activation. Yet biological neurons use discrete spikes to compute and transmit information, and the spike times, in addition to the spike rates, matter. Spiking neural networks (SNNs) are thus more biologically realistic than ANNs, and are arguably the only viable option if one wants to understand how the brain computes at the neuronal description level. The spikes of biological neurons are sparse in time and space, and event-driven. Combined with bio-plausible local learning rules, this makes it easier to build low-power, neuromorphic hardware for SNNs. However, training deep SNNs remains a challenge. Spiking neurons' transfer function is usually non-differentiable, which prevents using backpropagation. Here we review recent supervised and unsupervised methods to train deep SNNs, and compare them in terms of accuracy and computational cost. The emerging picture is that SNNs still lag behind ANNs in terms of accuracy, but the gap is decreasing, and can even vanish on some tasks, while SNNs typically require many fewer operations and are the better candidates to process spatio-temporal data.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Items 201–252 of 252
Independent Component Analysis in Spiking Neurons
Savin, C., Joshi, P., & Triesch, J. (2010) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000757 · Cited by 105
Multi-distribution deep belief network for speech synthesis
Kang, S., Qian, X., & Meng, H. M. L. (2013) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2013.6639225 · Cited by 105
Stromatias, E., Neil, D., Galluppi, F., Pfeiffer, M., Liu, S.-C., & Furber, S. (2015) · 2015 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2015.7280625 · Cited by 97
Generating Facial Expressions with Deep Belief Nets
Joshua, M., Geoffrey, E., Javier, R. A. V., & Adam, K. (2008) · doi:10.5772/6167 · Cited by 95
Action Recognition Using a Bio-Inspired Feedforward Spiking Network
Escobar, M.-J., Masson, G. S., Viéville, T., & Kornprobst, P. (2009) · International Journal of Computer Vision · doi:10.1007/s11263-008-0201-1 · Cited by 94
Training spiking neural networks to associate spatio-temporal input–output spike patterns
Mohemmed, A. W., Schliebs, S., Matsuda, S., & Kasabov, N. K. (2012) · Neurocomputing · doi:10.1016/j.neucom.2012.08.034 · Cited by 94
Deep Spiking NetworksPreprint
O’Connor, P., & Welling, M. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.08323 · Cited by 93
Adriano Barra, A., Genovese, G., Sollich, P., & Tantari, D. (2018) · Physical review. E · doi:10.1103/physreve.97.022310 · Cited by 93
Effective sensor fusion with event-based sensors and deep network architectures
Neil, D., & Liu, S. (2016) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas.2016.7539039 · Cited by 92
Character Recognition using Spiking Neural Networks
Gupta, A., & Long, L. N. (2007) · IEEE International Conference on Neural Networks/IEEE ... International Conference on Neural Networks · doi:10.1109/ijcnn.2007.4370930 · Cited by 90
Multi-layer unsupervised learning in a spiking convolutional neural network
Tavanaei, A., & Maida, A. S. (2017) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2017.7966099 · Cited by 89
Self-citationExploration of rank order coding with spiking neural networks for speech recognition
Loiselle, S., Rouat, J., Pressnitzer, D., et al. (2006) · IEEE International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2005.1556220 · Cited by 88
Bio-Inspired Spiking Convolutional Neural Network using Layer-wise Sparse Coding and STDP LearningPreprint
Tavanaei, A., & Maida, A. S. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.03000 · Cited by 88
Self-citationOptimal Predictions in Everyday Cognition: The Wisdom of Individuals or Crowds?
Mozer, M. C., Pashler, H., & Homaei, H. (2008) · Cognitive Science · doi:10.1080/03640210802353016 · Cited by 86
Segmentation and Edge Detection Based on Spiking Neural Network Model
Meftah, B., Lézoray, O., & Benyettou, A. (2010) · Neural Processing Letters · doi:10.1007/s11063-010-9149-6 · Cited by 84
Perception science in the age of deep neural networks.
VanRullen, R. (2017) · Frontiers in Psychology · doi:10.3389/fpsyg.2017.00142 · Cited by 82 · cited 3×
“These recent advances in artificial intelligence (AI) have opened up new avenues for developing different engineering applications and understanding of howbiological brainswork (Hassabis et al., 2017; VanRullen, 2017).”
Mirrored STDP Implements Autoencoder Learning in a Network of Spiking Neurons
Burbank, K. S. (2015) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1004566 · Cited by 81
Neil, D., Pfeiffer, M., & Liu, S. (2016) · ACM Symposium on Applied Computing · doi:10.1145/2851613.2851724 · Cited by 80
Event-Based, Timescale Invariant Unsupervised Online Deep Learning With STDP
Thiele, J. C., Bichler, O., & Dupret, A. (2018) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2018.00046 · Cited by 77
Garbin, D., Bichler, O., Vianello, E., Rafhay, Q., Gamrat, C., Perniola, L., Ghibaudo, G., & DeSalvo, B. (2014) · Technical digest - International Electron Devices Meeting/Technical digest · doi:10.1109/iedm.2014.7047126 · Cited by 76
Cortical microcircuits as gated-recurrent neural networksPreprint
Costa, R. P., Assael, Y., Shillingford, B., de Freitas, N., & Vogels, T. P. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.02448 · Cited by 75
Phase transitions in restricted Boltzmann machines with generic priors
Adriano Barra, A., Genovese, G., Sollich, P., & Tantari, D. (2017) · Physical review. E · doi:10.1103/physreve.96.042156 · Cited by 73
Carrillo, S., Harkin, J., Mcdaid, L. J., Pande, S. D., Cawley, S., McGinley, B., & Morgan, F. (2012) · Neural Networks · doi:10.1016/j.neunet.2012.04.004 · Cited by 70
Learning to recognize objects using waves of spikes and Spike Timing-Dependent Plasticity
Masquelier, T., & Thorpe, S. J. (2010) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2010.5596934 · Cited by 62
Self-citationA spiking network that learns to extract spike signatures from speech signals
Tavanaei, A., & Maida, A. S. (2017) · Neurocomputing · doi:10.1016/j.neucom.2017.01.088 · Cited by 62
Self-citationTo Spike or Not to Spike: That Is the Question
Maass, W. (2015) · Proceedings of the IEEE · doi:10.1109/jproc.2015.2496679 · Cited by 61
Modelling Peri-Perceptual Brain Processes in a Deep Learning Spiking Neural Network Architecture
Doborjeh, Z., Kasabov, N. K., Doborjeh, M., & Sumich, A. L. (2018) · Scientific Reports · doi:10.1038/s41598-018-27169-8 · Cited by 58
Natural Movies Evoke Spike Trains with Low Spike Time Variability in Cat Primary Visual Cortex
Herikstad, R., Baker, J., Lachaux, J.-P., Gray, C. M., & Yen, S. (2011) · Journal of Neuroscience · doi:10.1523/jneurosci.5153-10.2011 · Cited by 56
Kasabov, N. K. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-30687-7_12 · Cited by 54
Paulun, L., Wendt, A., & Kasabov, N. K. (2018) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2018.00042 · Cited by 53
Multi-column Deep Neural Networks for Image Classification
Cireşan, D., Meier, U., & Schmidhuber, J. (2012) · arXiv · doi:10.48550/arxiv.1202.2745 · Cited by 53
NormAD - Normalized Approximate Descent based supervised learning rule for spiking neurons
Anwani, N., & Rajendran, B. (2015) · doi:10.1109/ijcnn.2015.7280618 · Cited by 52
Millisecond-Scale Motor Encoding in a Cortical Vocal Area
Tang, C., Chehayeb, D., Srivastava, K. H., Nemenman, I. M., & Sober, S. J. (2014) · PLoS Biology · doi:10.1371/journal.pbio.1002018 · Cited by 52
Training Spiking ConvNets by STDP and Gradient Descent
Tavanaei, A., Kirby, Z., & Maida, A. S. (2018) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2018.8489104 · Cited by 52
Self-citationSpatiotemporal Dynamics and Reliable Computations in Recurrent Spiking Neural Networks
Pyle, R., & Rosenbaum, R. (2017) · Physical Review Letters · doi:10.1103/physrevlett.118.018103 · Cited by 51
Acquisition of visual features through probabilistic spike-timing-dependent plasticity
Tavanaei, A., Masquelier, T., & Maida, A. S. (2016) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2016.7727213 · Cited by 44
Self-citationVariational Learning for Recurrent Spiking Networks
Rezende, D. J., Wierstra, D., & Wulfram Gerstner, B. (2011) · neural information processing systems · Cited by 40
Representation learning using event-based STDP
Tavanaei, A., Masquelier, T., & Maida, A. S. (2018) · Neural Networks · doi:10.1016/j.neunet.2018.05.018 · Cited by 39
Self-citationSequence learning with hidden units in spiking neural networks
Brea, J., Senn, W., & Pfister, J.-P. (2011) · Bern Open Repository and Information System (University of Bern) · Cited by 37
Humans and Deep Networks Largely Agree on Which Kinds of Variation Make Object Recognition Harder
Kheradpisheh, S. R., Ghodrati, M., Ganjtabesh, M., & Masquelier, T. (2016) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2016.00092 · Cited by 35
Self-citationDeep Belief Network for clustering and classification of a continuous data
Salama, M. A., Hassanien, A. E., & Fahmy, A. A. (2010) · doi:10.1109/isspit.2010.5711759 · Cited by 34
A forecast-based STDP rule suitable for neuromorphic implementation
Davies, S., Galluppi, F., Rast, A. D., & Furber, S. B. (2012) · Neural Networks · doi:10.1016/j.neunet.2012.02.018 · Cited by 31
Probabilistic Computation in Spiking Populations
Zemel, R. S., Huys, Q. J. M., Natarajan, R., & Dayan, P. (2004) · UCL Discovery (University College London) · Cited by 31
Hierarchical Models of the Visual System
Serre, T. (2015) · Springer eBooks · doi:10.1007/978-1-4614-6675-8_345 · Cited by 29
Training a Hidden Markov Model with a Bayesian Spiking Neural Network
Tavanaei, A., & Maida, A. S. (2016) · Journal of Signal Processing Systems · doi:10.1007/s11265-016-1153-2 · Cited by 29
Self-citationLearning Probabilistic Inference through Spike-Timing-Dependent Plasticity
Pecevski, D., & Maass, W. (2016) · eNeuro · doi:10.1523/eneuro.0048-15.2016 · Cited by 24
A spike-based long short-term memory on a neurosynaptic processor
Shrestha, A., Ahmed, K., Wang, Y., Widemann, D. P., Moody, A. T., Essen, B. C. V., & Qiu, Q. (2017) · International Conference on Computer Aided Design · doi:10.1109/iccad.2017.8203836 · Cited by 24
Tavanaei, A., & Anthony, S. (2015) · INTERNATIONAL JOURNAL OF ADVANCED RESEARCH IN ARTIFICIAL INTELLIGENCE · doi:10.14569/ijarai.2015.040701 · Cited by 23
Self-citationTowards recognition of protein function based on its structure using deep convolutional networks
Tavanaei, A., Maida, A. S., Kaniymattam, A., & Loganantharaj, R. (2016) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm.2016.7822509 · Cited by 23
Self-citationTemporal Precision of Spike Trains in Extrastriate Cortex of the Behaving Macaque Monkey
Bair, W., & Koch, C. (1999) · The MIT Press eBooks · doi:10.7551/mitpress/4924.003.0016 · Cited by 14
Hierarchical Models of the Visual System
Ricci, M. R., & Serre, T. (2022) · Springer eBooks · doi:10.1007/978-1-0716-1006-0_345 · Cited by 9
Studying the interaction of a hidden Markov model with a Bayesian spiking neural network
Tavanaei, A., & Maida, A. S. (2015) · Machine learning for signal processing ... · doi:10.1109/mlsp.2015.7324350 · Cited by 8
Self-citation
Cited by
Neftci, E. O., Mostafa, H., & Zenke, F. (2019) · IEEE Signal Processing Magazine · doi:10.1109/msp.2019.2931595 · Cited by 1,538
Resistive switching materials for information processing
Wang, Z., Wu, H., Burr, G. W., Hwang, C. S., Wang, K. L., Xia, Q., & Yang, J. J. (2020) · Nature Reviews Materials · doi:10.1038/s41578-019-0159-3 · Cited by 1,214
Tang, J., Yuan, F., Shen, X., Wang, Z., Rao, M., He, Y., Sun, Y., Li, X., Zhang, W., Li, Y., et al. (2019) · Advanced Materials · doi:10.1002/adma.201902761 · Cited by 834
Deep Learning With Spiking Neurons: Opportunities and Challenges
Pfeiffer, M., & Pfeil, T. (2018) · Frontiers in Neuroscience · doi:10.3389/fnins.2018.00774 · Cited by 766
Direct Training for Spiking Neural Networks: Faster, Larger, Better
Wu, Y., Deng, L., Li, G., Zhu, J., Xie, Y., & Shi, L. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33011311 · Cited by 670
Metadata
- Authors
- Tavanaei, A., Ghodrati, M., Kheradpisheh, S. R., Masquelier, T., & Maida, A.
- Venue
- Neural Networks
- Publisher
- Elsevier BV
- Published
- 2019-03
- Volume
- 111
- Pages
- 47-63
- ISSN
- 0893-6080 (ISSN-L)
- OpenAlex
- W2798878556
- Cited by
- 1,386
- References
- 252
- FWCI
- 34.53
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.