The DOI matches its Crossref record; 80 of the 80 references it cites were found in a registry or an index.
The Heidelberg Spiking Data Sets for the Systematic Evaluation of Spiking Neural Networks
Open accessAbstract
Spiking neural networks are the basis of versatile and power-efficient information processing in the brain. Although we currently lack a detailed understanding of how these networks compute, recently developed optimization techniques allow us to instantiate increasingly complex functional spiking neural networks in-silico. These methods hold the promise to build more efficient non-von-Neumann computing hardware and will offer new vistas in the quest of unraveling brain circuit function. To accelerate the development of such methods, objective ways to compare their performance are indispensable. Presently, however, there are no widely accepted means for comparing the computational performance of spiking neural networks. To address this issue, we introduce two spike-based classification data sets, broadly applicable to benchmark both software and neuromorphic hardware implementations of spiking neural networks. To accomplish this, we developed a general audio-to-spiking conversion procedure inspired by neurophysiology. Furthermore, we applied this conversion to an existing and a novel speech data set. The latter is the free, high-fidelity, and word-level aligned Heidelberg digit data set that we created specifically for this study. By training a range of conventional and spiking classifiers, we show that leveraging spike timing information within these data sets is essential for good classification accuracy. These results serve as the first reference for future performance comparisons of spiking neural networks.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Scikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,395
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.123 · Cited by 19,741
(2017) · Cited by 11,625
“Librispeech: an ASR corpus based on public domain audio books,”
Panayotov, V., Chen, G., Povey, D., & Khudanpur, S. (2015) · 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2015.7178964 · Cited by 6,072
Reading digits in natural images with unsupervised feature learning
Netzer, Y. (2024) · doi:10.57702/e7wwo0zk · Cited by 4,149
Loihi: A Neuromorphic Manycore Processor with On-Chip Learning
Davies, M., Srinivasa, N., Lin, T.-H., Chinya, G., Cao, Y., Choday, S. H., Dimou, G., Joshi, P., Imam, N., Jain, S., et al. (2018) · IEEE Micro · doi:10.1109/mm.2018.112130359 · Cited by 3,897
Spiking Neuron Models: Single Neurons, Populations, Plasticity
Gerstner, W., & Kistler, W. M. (2002) · Cambridge University Press eBooks · doi:10.1017/cbo9780511815706 · Cited by 3,080
A 128$\times$128 120 dB 15 $\mu$s Latency Asynchronous Temporal Contrast Vision Sensor
Lichtsteiner, P., Posch, C., & Delbruck, T. (2008) · IEEE Journal of Solid-State Circuits · doi:10.1109/jssc.2007.914337 · Cited by 2,485 · cited 1×
“The majority of these datasets are based on the output of neuromorphic sensors like, for instance, the dynamic vision sensor (DVS) [Lichtsteiner et al., 2008] or the silicon cochlea [Anumula et al., 2018].”
BackgroundTowards spike-based machine intelligence with neuromorphic computing
Roy, K., Jaiswal, A., & Panda, P. (2019) · Nature · doi:10.1038/s41586-019-1677-2 · Cited by 2,190
Spiking Neuron Models: Single Neurons, Populations, Plasticity
Wulfram Gerstner, B., & Kistler, W. M. (2002) · Cited by 2,028
Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
Bengio, Y., Léonard, N., & Courville, A. (2013) · arXiv · doi:10.48550/arxiv.1308.3432 · Cited by 1,910
Spoken Language Processing: A Guide to Theory, Algorithm, and System Development
Huang, X., Acero, A., Hon, H.-W., & Reddy, R. (2001) · Cited by 1,593
Deep learning in spiking neural networks
Tavanaei, A., Ghodrati, M., Kheradpisheh, S. R., Masquelier, T., & Maida, A. (2019) · Neural Networks · doi:10.1016/j.neunet.2018.12.002 · Cited by 1,386
(2011) · Validated Numerics · doi:10.2307/j.ctvcm4g18.8 · Cited by 1,187
Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition
Warden, P. (2018) · arXiv · doi:10.48550/arxiv.1804.03209 · Cited by 1,033
Error-backpropagation in temporally encoded networks of spiking neurons
Bohte, S. M., Kok, J. N., & La Poutré, H. (2002) · Neurocomputing · doi:10.1016/s0925-2312(01)00658-0 · Cited by 1,021
A Low Power, Fully Event-Based Gesture Recognition System
Amir, A., Taba, B., Berg, D., Melano, T., McKinstry, J., Di Nolfo, C., Nayak, T., Andreopoulos, A., Garreau, G., Mendoza, M., et al. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.781 · Cited by 1,017
Converting Static Image Datasets to Spiking Neuromorphic Datasets Using Saccades
Orchard, G., Jayawant, A., Cohen, G. K., & Thakor, N. (2015) · Frontiers in Neuroscience · doi:10.3389/fnins.2015.00437 · Cited by 882
The tempotron: a neuron that learns spike timing–based decisions
Gütig, R., & Sompolinsky, H. (2006) · Nature Neuroscience · doi:10.1038/nn1643 · Cited by 815
STDP-based spiking deep convolutional neural networks for object recognition
Kheradpisheh, S. R., Ganjtabesh, M., Thorpe, S. J., & Masquelier, T. (2018) · Neural Networks · doi:10.1016/j.neunet.2017.12.005 · Cited by 768
Deep Learning With Spiking Neurons: Opportunities and Challenges
Pfeiffer, M., & Pfeil, T. (2018) · Frontiers in Neuroscience · doi:10.3389/fnins.2018.00774 · Cited by 766
A wafer-scale neuromorphic hardware system for large-scale neural modeling
Schemmel, J., Briiderle, D., Griibl, A., Hock, M., Meier, K., & Millner, S. (2010) · Proceedings of 2010 IEEE International Symposium on Circuits and Systems · doi:10.1109/iscas.2010.5536970 · Cited by 682
Self-citationMoradi, S., Qiao, N., Stefanini, F., & Indiveri, G. (2018) · IEEE Transactions on Biomedical Circuits and Systems · doi:10.1109/tbcas.2017.2759700 · Cited by 650
Overview of the SpiNNaker System Architecture
Furber, S. B., Lester, D. R., Plana, L. A., Garside, J. D., Painkras, E., Temple, S., & Brown, A. D. (2013) · IEEE Transactions on Computers · doi:10.1109/tc.2012.142 · Cited by 645
Ponulak, F., & Kasiński, A. (2010) · Neural Computation · doi:10.1162/neco.2009.11-08-901 · Cited by 573
Neural Engineering: Computation, Representation, and Dynamics in Neurobiological Systems
(2004) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.826381 · Cited by 543
SuperSpike: Supervised Learning in Multilayer Spiking Neural Networks
Zenke, F., & Ganguli, S. (2018) · Neural Computation · doi:10.1162/neco_a_01086 · Cited by 515
Self-citationA solution to the learning dilemma for recurrent networks of spiking neurons
Bellec, G., Scherr, F., Subramoney, A., Hajek, E., Salaj, D., Legenstein, R., & Maass, W. (2020) · Nature Communications · doi:10.1038/s41467-020-17236-y · Cited by 506
Efficient codes and balanced networks
Denève, S., & Machens, C. K. (2016) · Nature Neuroscience · doi:10.1038/nn.4243 · Cited by 471 · cited 2×
“The latter are typically defined as linear combinations of low-pass filtered spike trains [Gilra and Gerstner, 2017; Nicola and Clopath, 2017; Eliasmith and Anderson, 2004; Denève and Machens, 2016; Abbott et al., 2016].”
BackgroundSLAYER: Spike Layer Error Reassignment in Time
Shrestha, S. B., & Orchard, G. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.08646 · Cited by 444
Enabling Spike-Based Backpropagation for Training Deep Neural Network Architectures
Lee, C., Sarwar, S. S., Panda, P., Srinivasan, G., & Roy, K. (2020) · Frontiers in Neuroscience · doi:10.3389/fnins.2020.00119 · Cited by 434
Neural engineering: Computation, representation, and dynamics in neurobiological systems
(2003) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(03)90181-7 · Cited by 399
Supervised Learning Based on Temporal Coding in Spiking Neural Networks
Mostafa, H. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2017.2726060 · Cited by 331
Spiking Neuron Models: Single Neurons, Populations, Plasticity
Andrew, A. M. (2003) · Kybernetes · doi:10.1108/k.2003.06732gae.003 · Cited by 298
Synaptic Plasticity Dynamics for Deep Continuous Local Learning (DECOLLE)
Kaiser, J., Mostafa, H., & Neftci, E. (2020) · Frontiers in Neuroscience · doi:10.3389/fnins.2020.00424 · Cited by 288
SPAN: SPIKE PATTERN ASSOCIATION NEURON FOR LEARNING SPATIO-TEMPORAL SPIKE PATTERNS
MOHEMMED, A., SCHLIEBS, S., MATSUDA, S., & KASABOV, N. (2012) · International Journal of Neural Systems · doi:10.1142/s0129065712500128 · Cited by 270
Zenke, F., & Vogels, T. P. (2021) · Neural Computation · doi:10.1162/neco_a_01367 · Cited by 261
Self-citationOptimal Spike-Timing-Dependent Plasticity for Precise Action Potential Firing in Supervised Learning
Pfister, J.-P., Toyoizumi, T., Barber, D., & Gerstner, W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.6.1318 · Cited by 245
Event-Driven Random Back-Propagation: Enabling Neuromorphic Deep Learning Machines
Neftci, E. O., Augustine, C., Paul, S., & Detorakis, G. (2017) · Frontiers in Neuroscience · doi:10.3389/fnins.2017.00324 · Cited by 237
Stöckl, C., & Maass, W. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00311-4 · Cited by 213
Supervised learning in spiking neural networks with FORCE training
Nicola, W., & Clopath, C. (2017) · Nature Communications · doi:10.1038/s41467-017-01827-3 · Cited by 211
The chronotron: a neuron that learns to fire temporally-precise spike patterns
Florian, R. (2010) · Nature Precedings · doi:10.1038/npre.2010.5190.1 · Cited by 204
TED-LIUM: an Automatic Speech Recognition dedicated corpus
Rousseau, A., Deléglise, P., & Estève, Y. (2012) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/48xafxbrk3qa · Cited by 204
Building functional networks of spiking model neurons
Abbott, L. F., DePasquale, B., & Memmesheimer, R.-M. (2016) · Nature Neuroscience · doi:10.1038/nn.4241 · Cited by 204
Spiking Deep Networks with LIF Neurons
Hunsberger, E., & Eliasmith, C. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1510.08829 · Cited by 203
Zylberberg, J., Murphy, J. T., & DeWeese, M. R. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002250 · Cited by 201
Effective and Efficient Computation with Multiple-timescale Spiking Recurrent Neural Networks
Yin, B., Corradi, F., & Bohté, S. M. (2020) · International Conference on Neuromorphic Systems (ICONS) · doi:10.1145/3407197.3407225 · Cited by 172
Temporal Coding in Spiking Neural Networks with Alpha Synaptic Function
Comsa, I. M., Potempa, K., Versari, L., Fischbacher, T., Gesmundo, A., & Alakuijala, J. (2020) · ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp40776.2020.9053856 · Cited by 170
Deep learning incorporating biologically inspired neural dynamics and in-memory computing
Woźniak, S., Pantazi, A., Bohnstingl, T., & Eleftheriou, E. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-020-0187-0 · Cited by 160
Rothman, J. S., Young, E. D., & Manis, P. B. (1993) · Journal of Neurophysiology · doi:10.1152/jn.1993.70.6.2562 · Cited by 156
Learning Precisely Timed Spikes
Memmesheimer, R.-M., Rubin, R., Ölveczky, B. P., & Sompolinsky, H. (2014) · Neuron · doi:10.1016/j.neuron.2014.03.026 · Cited by 145
Surrogate Gradient Learning in Spiking Neural NetworksPreprint
Neftci, E., Mostafa, H., & Zenke, F. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.09948 · Cited by 144
Self-citationAuditory physics. Physical principles in hearing theory. II
de Boer, E. (1984) · Physics Reports · doi:10.1016/0370-1573(84)90108-x · Cited by 142
Feature Representations for Neuromorphic Audio Spike Streams
Anumula, J., Neil, D., Delbrück, T., & Liu, S. (2018) · Frontiers in Neuroscience · doi:10.3389/fnins.2018.00023 · Cited by 131
Demonstrating Hybrid Learning in a Flexible Neuromorphic Hardware System
Friedmann, S., Schemmel, J., Gruebl, A., Hartel, A., Hock, M., & Meier, K. (2016) · IEEE Transactions on Biomedical Circuits and Systems · doi:10.1109/tbcas.2016.2579164 · Cited by 126
Self-citationEvent-driven random backpropagation: Enabling neuromorphic deep learning machines
Neftci, E. O., Augustine, C., Paul, S., & Detorakis, G. (2017) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas.2017.8050529 · Cited by 118
Boahen, K. A. (2017) · Computing in Science & Engineering · doi:10.1109/mcse.2017.33 · Cited by 104
Jang, H., Simeone, O., Gardner, B., & Gruning, A. (2019) · IEEE Signal Processing Magazine · doi:10.1109/msp.2019.2935234 · Cited by 101
Predicting non-linear dynamics by stable local learning in a recurrent spiking neural network
Gilra, A., & Gerstner, W. (2017) · eLife · doi:10.7554/elife.28295 · Cited by 100
Benchmarks for progress in neuromorphic computing
Davies, M. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0097-1 · Cited by 100
Comșa, I. M., Potempa, K., Versari, L., Fischbacher, T., Gesmundo, A., & Alakuijala, J. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3071976 · Cited by 100
Learning Universal Computations with Spikes
Thalmeier, D., Uhlmann, M., Kappen, H. J., & Memmesheimer, R.-M. (2016) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1004895 · Cited by 99
Gütig, R. (2014) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2014.01.004 · Cited by 93
Stochastic variational learning in recurrent spiking networks
Rezende, D. J., & Wulfram Gerstner, B. (2014) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2014.00038 · Cited by 78
A Plasticity-Centric Approach to Train the Non-Differential Spiking Neural Networks
Zhang, T., Zeng, Y., Zhao, D., & Shi, M. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11317 · Cited by 71
Supervised Learning in Spiking Neural Networks for Precise Temporal Encoding
Brian, G., & Grüning, A. (2016) · PLoS ONE · doi:10.1371/journal.pone.0161335 · Cited by 54
Error-Backpropagation in Networks of Fractionally Predictive Spiking Neurons
Bohté, S. M. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-21735-7_8 · Cited by 50
Jakobovski/free-spoken-digit-dataset v1.0.4
Jackson, Z., Souza, C., Flaks, J., Pan, Y., Nicolas, H., & Thite, A. (2017) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.1000741 · Cited by 47
Enabling Spike-based Backpropagation in State-of-the-art Deep Neural Network ArchitecturesPreprint
Lee, C., Sarwar, S. S., & Roy, K. (2019) · arXiv (Cornell University) · Cited by 43
Mining the Spoken Wikipedia for Speech Data and Beyond
Köhn, A., Stegen, F., & Baumann, T. (2016) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/5dnpnbxgjfh6 · Cited by 36
Closing the Accuracy Gap in an Event-Based Visual Recognition TaskPreprint
Rueckauer, B., Känzig, N., Liu, S., Delbrück, T., & Sandamirskaya, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.08859 · Cited by 28
Fast and deep: energy-efficient neuromorphic learning with first-spike timesPreprint
Göltz, J., Kriener, L., Baumbach, A., Billaudelle, S., Breitwieser, O., Cramer, B., Dold, D., Kungl, Á. F., Senn, W., Schemmel, J., et al. (2019) · arXiv (Cornell University) · Cited by 19
Self-citationEfficient Computation in Adaptive Artificial Spiking Neural NetworksPreprint
Zambrano, D., Nusselder, R., Scholte, H. S., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.04838 · Cited by 18
Fast and deep neuromorphic learning with time-to-first-spike codingPreprint
Göltz, J., Baumbach, A., Billaudelle, S., Breitwieser, O., Dold, D., Kriener, L., Kungl, Á. F., Senn, W., Schemmel, J., Meier, K., et al. (2019) · arXiv (Cornell University) · Cited by 17
Self-citationLi, Y., Kim, R., & B, T. J. S. A. (2021) · Neural Computation · doi:10.1162/neco_a_01409 · Cited by 7
Classifying Images with Few Spikes per NeuronPreprint
Stöckl, C., & Maass, W. (2020) · arXiv (Cornell University) · Cited by 6
Cited by
Opportunities for neuromorphic computing algorithms and applications
Schuman, C. D., Kulkarni, S. R., Parsa, M., Mitchell, J. P., Date, P., & Kay, B. (2022) · Nature Computational Science · doi:10.1038/s43588-021-00184-y · Cited by 1,117
“As surveyed by Zenke and Neftci 40 , approaches such as backpropagation through time and real-time recurrent learning have been demonstrated on neuromorphic datasets, such as the Spiking Heidelberg Digits (SHD) and the Spiking Speech Command (SSC) dataset 41 .”
BackgroundTraining Spiking Neural Networks Using Lessons From Deep Learning
Eshraghian, J. K., Ward, M., Neftci, E. O., Wang, X., Lenz, G., Dwivedi, G., Bennamoun, M., Jeong, D. S., & Lu, W. D. (2023) · Proceedings of the IEEE · doi:10.1109/jproc.2023.3308088 · Cited by 782
SpikingJelly: An open-source machine learning infrastructure platform for spike-based intelligence
Fang, W., Chen, Y., Ding, J., Yu, Z., Masquelier, T., Chen, D., Huang, L., Zhou, H., Li, G., & Tian, Y. (2023) · Science Advances · doi:10.1126/sciadv.adi1480 · Cited by 444
Neuromorphic computing at scale
Kudithipudi, D., Schuman, C., Vineyard, C. M., Pandit, T., Merkel, C., Kubendran, R., Aimone, J. B., Orchard, G., Mayr, C., Benosman, R., et al. (2025) · Nature · doi:10.1038/s41586-024-08253-8 · Cited by 357
Neural heterogeneity promotes robust learning
Perez-Nieves, N., Leung, V. C. H., Dragotti, P. L., & Goodman, D. F. M. (2021) · Nature Communications · doi:10.1038/s41467-021-26022-3 · Cited by 267
Metadata
- Authors
- Cramer, B., Stradmann, Y., Schemmel, J., & Zenke, F.
- Venue
- IEEE Transactions on Neural Networks and Learning Systems
- Published
- 2022-07
- Volume
- 33
- Issue
- 7
- Pages
- 2744-2757
- ISSN
- 2162-237X (ISSN-L)
- 2162-2388
- OpenAlex
- W2980513133
- Cited by
- 288
- References
- 80
- FWCI
- 7.34
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.