The DOI matches its Crossref record; 66 of the 66 references it cites were found in a registry or an index.
Real-time classification and sensor fusion with a spiking deep belief network
Open accessAbstract
Deep Belief Networks (DBNs) have recently shown impressive performance on a broad range of classification problems. Their generative properties allow better understanding of the performance, and provide a simpler solution for sensor fusion tasks. However, because of their inherent need for feedback and parallel update of large numbers of units, DBNs are expensive to implement on serial computers. This paper proposes a method based on the Siegert approximation for Integrate-and-Fire neurons to map an offline-trained DBN onto an efficient event-driven spiking neural network suitable for hardware implementation. The method is demonstrated in simulation and by a real-time implementation of a 3-layer network with 2694 neurons used for visual classification of MNIST handwritten digits with input from a 128 × 128 Dynamic Vision Sensor (DVS) silicon retina, and sensory-fusion using additional input from a 64-channel AER-EAR silicon cochlea. The system is implemented through the open-source software in the jAER project and runs in real-time on a laptop computer. It is demonstrated that the system can recognize digits in the presence of distractions, noise, scaling, translation and rotation, and that the degradation of recognition performance by using an event-based approach is less than 1%. Recognition is achieved in an average of 5.8 ms after the onset of the presentation of a digit. By cue integration from both silicon retina and cochlea outputs we show that the system can be biased to select the correct digit from otherwise ambiguous input.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Two new Later Stone Age sites from the Final Pleistocene in the Falémé Valley, eastern Senegal
Ndiaye, M., Lespez, L., Tribolo, C., Rassé, M., Hadjas, I., Davidoux, S., Huysecom, É., & Douze, K. (2024) · Monash University Research Portal (Monash University) · doi:10.3929/ethz-b-000667478 · Cited by 23,868
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336
(2010) · International Conference on Machine Learning · Cited by 13,995
Hinton, G., Deng, L., Yu, D., Dahl, G., Mohamed, A.-R., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T., et al. (2012) · IEEE Signal Processing Magazine · doi:10.1109/msp.2012.2205597 · Cited by 10,260
The free-energy principle: a unified brain theory?
Friston, K. (2010) · Nature Reviews Neuroscience · doi:10.1038/nrn2787 · Cited by 8,120 · cited 2×
“The hierarchical organization of sensory cortices would then naturally correspond to a hierarchy of prior distributions from higher to lower areas that can be optimally adapted to statistics of the real world in order to minimize surprise (Friston, 2010).”
BackgroundDistributed hierarchical processing in the primate cerebral cortex
Felleman, D. J., & Van Essen, D. C. (1991) · Cerebral Cortex · doi:10.1093/cercor/1.1.1 · Cited by 7,783
(2023) · doi:10.1126/science · Cited by 6,063
Rao, R. P. N., & Ballard, D. H. (1999) · Nature Neuroscience · doi:10.1038/4580 · Cited by 5,483 · cited 3×
“…1983; Riesenhuber and Poggio, 1999), and DBNs provide a first step toward linking state-of-the-art machine learning techniques and modern models of Bayesian inference and predictive coding in the brain (Rao and Ballard, 1999; Hochstein and Ahissar, 2002; Friston, 2010; Markov and Kennedy, 2013).”
BackgroundGreedy layer-wise training of deep networks
Bengio, Y., Lamblin, P., Popovici, D., & Larochelle, H. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0024 · Cited by 4,841
Colman (2022) · doi:10.1073/pnas · Cited by 4,370
Hierarchical models of object recognition in cortex
Riesenhuber, M., & Poggio, T. (1999) · Nature Neuroscience · doi:10.1038/14819 · Cited by 3,309 · cited 3×
“Feed-back connections are rarely used in models of biologically inspired vision, e.g., HMAX (Riesenhuber and Poggio, 1999), but as we show e.g., in Figure 7 , feed-back and recurrency are essential for implementing general probabilistic inference, e.g., to infer missing, ambiguous, or noisy values in the input.”
BackgroundMethodSpiking Neuron Models: Single Neurons, Populations, Plasticity
Gerstner, W., & Kistler, W. M. (2002) · Cambridge University Press eBooks · doi:10.1017/cbo9780511815706 · Cited by 3,080
Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition
Dahl, G. E., Yu, D., Deng, L., & Acero, A. (2012) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2011.2134090 · Cited by 3,011
Why Does Unsupervised Pre-training Help Deep Learning?
Erhan, D., Bengio, Y., Courville, A. C., Manzagol, P.-A., Vincent, P., & Bengio, S. (2010) · Journal of Machine Learning Research · Cited by 2,631
The Bayesian brain: The role of uncertainty in neural coding and computation
Knill, D. C., & Pouget, A. (2004) · Trends in Neurosciences · doi:10.1016/j.tins.2004.10.007 · Cited by 2,588 · cited 2×
“This is consistent with a variety of theories that the brain encodes Bayesian generative models of its natural environment (Kersten and Yuille, 2003; Knill and Pouget, 2004).”
BackgroundA 128$\times$128 120 dB 15 $\mu$s Latency Asynchronous Temporal Contrast Vision Sensor
Lichtsteiner, P., Posch, C., & Delbruck, T. (2008) · IEEE Journal of Solid-State Circuits · doi:10.1109/jssc.2007.914337 · Cited by 2,485 · cited 6×
“…digital hardware solutions and spike-based convolution chips (Camuñas Mesa et al., 2010; Farabet et al., 2012), which through the use of the Address Event Representation (AER) protocol, can also directly process events coming from event-based dynamic vision sensors (Lichtsteiner et al., 2008).”
BackgroundMethodInfluentialLee, H., Grosse, R., Ranganath, R., & Ng, A. Y. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553453 · Cited by 2,412
Neuromorphic Silicon Neuron Circuits
Indiveri, G., Linares-Barranco, B., Hamilton, T. J., Schaik, A. V., Etienne-Cummings, R., Delbruck, T., Liu, S.-C., Dudek, P., Häfliger, P., Renaud, S., et al. (2011) · Frontiers in Neuroscience · doi:10.3389/fnins.2011.00073 · Cited by 1,821
Self-citationAcoustic Modeling Using Deep Belief Networks
Mohamed, A.-R., Dahl, G. E., & Hinton, G. (2012) · IEEE Transactions on Audio, Speech, and Language Processing · doi:10.1109/tasl.2011.2109382 · Cited by 1,727
Building high-level features using large scale unsupervised learning
Le, Q. V. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6639343 · Cited by 1,695
Multisensory integration: current issues from the perspective of the single neuron
Stein, B. E., & Stanford, T. R. (2008) · Nature Reviews Neuroscience · doi:10.1038/nrn2331 · Cited by 1,598
Visual properties of neurons in inferotemporal cortex of the Macaque.
Gross, C. G., Rocha-Miranda, C. E., & Bender, D. B. (1972) · Journal of Neurophysiology · doi:10.1152/jn.1972.35.1.96 · Cited by 1,483
Stimulus-selective properties of inferior temporal neurons in the macaque
Desimone, R., Albright, T., Gross, C., & Bruce, C. (1984) · The Journal of Neuroscience · doi:10.1523/jneurosci.04-08-02051.1984 · Cited by 1,423
Hochstein (2002) · doi:10.1016/s0896-6273(02)01091-7 · Cited by 1,397
Gradient Flow in Recurrent Nets: The Difficulty of Learning LongTerm Dependencies
(2009) · A Field Guide to Dynamical Recurrent Networks · doi:10.1109/9780470544037.ch14 · Cited by 1,246
Gerstner, W. (2009) · Encyclopedia of Neuroscience · doi:10.1016/b978-008045046-9.01405-4 · Cited by 1,199
GradientBased Learning Applied to Document Recognition
(2009) · Intelligent Signal Processing · doi:10.1109/9780470544976.ch9 · Cited by 1,135
Deep, Big, Simple Neural Nets for Handwritten Digit Recognition
Cireşan, D. C., Meier, U., Gambardella, L. M., & Schmidhuber, J. (2010) · Neural Computation · doi:10.1162/neco_a_00052 · Cited by 1,078
Learning and relearning in Boltzmann machines
Hinton, G. E., & Sejnowski, T. J. (2001) · Graphical Models · doi:10.7551/mitpress/3349.003.0005 · Cited by 1,037
Hawkins, J., & Blakeslee, S. (2004) · Cited by 1,012
Integrated model of visual processing
Bullier, J. (2001) · Brain Research Reviews · doi:10.1016/s0165-0173(01)00085-6 · Cited by 1,011
An empirical evaluation of deep architectures on problems with many factors of variation
Larochelle, H., Erhan, D., Courville, A., Bergstra, J., & Bengio, Y. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273556 · Cited by 992
Hierarchical organization and functional streams in the visual cortex
Van Essen, D. C., & Maunsell, J. H. R. (1983) · Trends in Neurosciences · doi:10.1016/0166-2236(83)90167-4 · Cited by 962
Anatomical evidence of multimodal integration in primate striate cortex
Falchier, A., Clavagnier, S., Barone, P., & Kennedy, H. (2002) · The Journal of Neuroscience · doi:10.1523/jneurosci.22-13-05749.2002 · Cited by 892 · cited 1×
“Recent studies have revealed the existence of anatomical connections that would enable cross-modal interactions also at lower levels (Falchier et al., 2002; Markov et al., 2012), and functional studies have provided some (but not conclusive) evidence of co-activations of early sensory areas by…”
BackgroundGradient Flow in Recurrent Nets: the Difficulty of Learning Long-Term Dependencies
Hochreiter, S., Bengio, Y., Informatik, F. F., Bengio, Y., Frasconi, P., & Schmidhuber, J. (2001) · Cited by 863
A Weighted and Directed Interareal Connectivity Matrix for Macaque Cerebral Cortex
Markov, N. T., Ercsey-Ravasz, M. M., Ribeiro Gomes, A. R., Lamy, C., Magrou, L., Vezoli, J., Misery, P., Falchier, A., Quilodran, R., Gariel, M. A., et al. (2014) · Cerebral Cortex · doi:10.1093/cercor/bhs270 · Cited by 857
Conversational speech transcription using context-dependent deep neural networks
Seide, F., Li, G., & Yu, D. (2011) · Interspeech 2011 · doi:10.21437/interspeech.2011-169 · Cited by 852
(2007) · Cited by 849
Training restricted Boltzmann machines using approximations to the likelihood gradient
Tieleman, T. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390290 · Cited by 837
Building high-level features using large scale unsupervised learning
Ranzato, M., Monga, R., Devin, M., Chen, K., Corrado, G. S., Dean, J., Le, Q. V., & Ng, A. Y. (2012) · International Conference on Machine Learning · Cited by 823
The Role of Area 17 in Visual Imagery: Convergent Evidence from PET and rTMS
Kosslyn, S. M., Pascual-Leone, A., Felician, O., Camposano, S., Keenan, J. P., L., W., Thompson, Ganis, G., Sukel, K. E., & Alpert, N. M. (1999) · Science · doi:10.1126/science.284.5411.167 · Cited by 747
Modeling Human Motion Using Binary Latent Variables
Taylor, G. W., Hinton, G. E., & Roweis, S. T. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0173 · Cited by 689
Feedforward, horizontal, and feedback processing in the visual cortex
Lamme, V. A., Supèr, H., & Spekreijse, H. (1998) · Current Opinion in Neurobiology · doi:10.1016/s0959-4388(98)80042-1 · Cited by 621
Shape perception reduces activity in human primary visual cortex
Murray, S. O., Kersten, D., Olshausen, B. A., Schrater, P., & Woods, D. L. (2002) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.192579399 · Cited by 496 · cited 3×
“…and Martin, 2009; Douglas and Martin, 2011; Markov et al., 2012), and in vivo experiments (Lamme et al., 1998; Kosslyn et al., 1999; Bullier, 2001; Murray et al., 2002), but the precise role of feed-back processing is still debated (Lamme et al., 1998; Bullier, 2001; Kersten and Yuille, 2003).”
BackgroundResultOn the First Passage Time Probability Problem
Siegert, A. J. F. (1951) · Physical Review · doi:10.1103/physrev.81.617 · Cited by 448
Buesing, L., Bill, J., Nessler, B., & Maass, W. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002211 · Cited by 384
Perez-Carrasco, J. A., Zhao, B., Serrano, C., Acha, B., Serrano-Gotarredona, T., Chen, S., & Linares-Barranco, B. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.71 · Cited by 324
Learning long‐range vision for autonomous off‐road driving
Hadsell, R., Sermanet, P., Ben, J., Erkan, A., Scoffier, M., Kavukcuoglu, K., Muller, U., & LeCun, Y. (2009) · Journal of Field Robotics · doi:10.1002/rob.20276 · Cited by 314
Bayesian models of object perception
Kersten, D., & Yuille, A. (2003) · Current Opinion in Neurobiology · doi:10.1016/s0959-4388(03)00042-4 · Cited by 307
Do early sensory cortices integrate cross-modal information?
Kayser, C., & Logothetis, N. K. (2007) · Brain Structure and Function · doi:10.1007/s00429-007-0154-0 · Cited by 293
Nessler, B., Pfeiffer, M., Buesing, L., & Maass, W. (2013) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003037 · Cited by 281 · cited 1×
“Nessler et al. (2013) have shown that learning via STDP in cortical microcircuits can lead to the emergence of Bayesian computation for the detection of hidden causes of inputs.”
BackgroundSelf-citationUsing fast weights to improve persistent contrastive divergence
Tieleman, T., & Hinton, G. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553506 · Cited by 274 · cited 3×
“This modification is inspired by the idea of using auxiliary “fast-weights” for learning (Tieleman and Hinton, 2009), which transiently raise the energy of any state that the network is currently in, thereby slightly pushing it out of that state.”
BackgroundMethodInfluentialThe importance of being hierarchical
Markov, N. T., & Kennedy, H. (2013) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2012.12.008 · Cited by 204
An optimal frequency domain filter for edge detection in digital pictures
(1979) · NCHU Institutional Repository CRIS · doi:10.1109/tpami · Cited by 198
Event-based 64-channel binaural silicon cochlea with Q enhancement mechanisms
Liu, S., André, V. S., Mincti, B. A., & Delbrück, T. (2010) · doi:10.1109/iscas.2010.5537164 · Cited by 91
Self-citationFarabet, C., Paz, R., Pérez‐Carrasco, J. A., Zamarreño-Ramos, C., Linares-Barranco, A., LeCun, Y., Culurciello, E., Serrano‐Gotarredona, T., & Linares-Barranco, B. (2012) · Frontiers in Neuroscience · doi:10.3389/fnins.2012.00032 · Cited by 74
Live search for mobile:Web services by voice on the cellphone
Acero, A., Bernstein, N., Chambers, R., Ju, Y.-C., Li, X., Odell, J. J., Nguyen, P., Scholz, O., & Zweig, G. (2008) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2008.4518845 · Cited by 68
da Costa, N. M., & Martin, K. A. C. (2009) · The Journal of Comparative Neurology · doi:10.1002/cne.22133 · Cited by 53
Seo, S. N. (2017) · Elsevier eBooks · doi:10.1016/b978-0-12-811874-0.00005-2 · Cited by 38
Biasing Restricted Boltzmann Machines to Manipulate Latent Selectivity and Sparsity
Goh, H., Thome, N., & Cord, M. (2010) · HAL · Cited by 36
The thermodynamic temperature of a rhythmic spiking networkPreprint
Merolla, P. A., Ursell, T. S., & Arthur, J. V. (2010) · arXiv (Cornell University) · doi:10.48550/arxiv.1009.5473 · Cited by 13
Recurrent competitive networks can learn locally excitatory topologies
Jug, F., Cook, M., & Steger, A. (2012) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2012.6252786 · Cited by 13
What’s Black and White About the Grey Matter?
Douglas, R. J., & Martin, K. A. C. (2011) · Neuroinformatics · doi:10.1007/s12021-011-9106-1 · Cited by 11
On scalable spiking convnet hardware for cortex-like visual sensory processing systems
Camuñas-Mesa, L. A., Pérez‐Carrasco, J. A., Zamarreño-Ramos, C., et al. (2010) · doi:10.1109/iscas.2010.5537918 · Cited by 9
Cited by
Deep learning in neural networks: an overview
Schmidhuber, J. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.09.003 · Cited by 18,570
Gallego, G., Delbruck, T., Orchard, G., Bartolozzi, C., Taba, B., Censi, A., Leutenegger, S., Davison, A. J., Conradt, J., Daniilidis, K., et al. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3008413 · Cited by 2,566
Unsupervised learning of digit recognition using spike-timing-dependent plasticity
Diehl, P. U., & Cook, M. (2015) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2015.00099 · Cited by 1,452
Deep learning in spiking neural networks
Tavanaei, A., Ghodrati, M., Kheradpisheh, S. R., Masquelier, T., & Maida, A. (2019) · Neural Networks · doi:10.1016/j.neunet.2018.12.002 · Cited by 1,386
Spatio-Temporal Backpropagation for Training High-Performance Spiking Neural Networks
Wu, Y., Deng, L., Li, G., Zhu, J., & Shi, L. (2018) · Frontiers in Neuroscience · doi:10.3389/fnins.2018.00331 · Cited by 1,196
Metadata
- Authors
- O'Connor, P., Neil, D., Liu, S.-C., Delbruck, T., & Pfeiffer, M.
- Venue
- Frontiers in Neuroscience
- Publisher
- Frontiers Media SA
- Published
- 2013
- Volume
- 7
- Pages
- 178-178
- ISSN
- 1662-453X (ISSN-L)
- OpenAlex
- W2008008156
- Cited by
- 408
- References
- 66
- FWCI
- 15.58
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.