The DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
HOTS: A Hierarchy of Event-Based Time-Surfaces for Pattern Recognition
Abstract
This paper describes novel event-based spatio-temporal features called time-surfaces and how they can be used to create a hierarchical event-based pattern recognition architecture. Unlike existing hierarchical architectures for pattern recognition, the presented model relies on a time oriented approach to extract spatio-temporal features from the asynchronously acquired dynamics of a visual scene. These dynamics are acquired using biologically inspired frameless asynchronous event-driven vision sensors. Similarly to cortical structures, subsequent layers in our hierarchy extract increasingly abstract features using increasingly large spatio-temporal windows. The central concept is to use the rich temporal information provided by events to create contexts in the form of time-surfaces which represent the recent temporal activity within a local spatial neighborhood. We demonstrate that this concept can robustly be used at all stages of an event-based hierarchical model. First layer feature units operate on groups of pixels, while subsequent layer feature units operate on the output of lower level feature units. We report results on a previously published 36 class character recognition task and a four class canonical dynamic card pip task, achieving near 100 percent accuracy on each. We introduce a new seven class moving face recognition task, achieving 79 percent accuracy.This paper describes novel event-based spatio-temporal features called time-surfaces and how they can be used to create a hierarchical event-based pattern recognition architecture. Unlike existing hierarchical architectures for pattern recognition, the presented model relies on a time oriented approach to extract spatio-temporal features from the asynchronously acquired dynamics of a visual scene. These dynamics are acquired using biologically inspired frameless asynchronous event-driven vision sensors. Similarly to cortical structures, subsequent layers in our hierarchy extract increasingly abstract features using increasingly large spatio-temporal windows. The central concept is to use the rich temporal information provided by events to create contexts in the form of time-surfaces which represent the recent temporal activity within a local spatial neighborhood. We demonstrate that this concept can robustly be used at all stages of an event-based hierarchical model. First layer feature units operate on groups of pixels, while subsequent layer feature units operate on the output of lower level feature units. We report results on a previously published 36 class character recognition task and a four class canonical dynamic card pip task, achieving near 100 percent accuracy on each. We introduce a new seven class moving face recognition task, achieving 79 percent accuracy.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Distinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589
Emergence of simple-cell receptive field properties by learning a sparse code for natural images
Olshausen, B. A., & Field, D. J. (1996) · Nature · doi:10.1038/381607a0 · Cited by 5,694
Large Scale Distributed Deep Networks
Dean, J. B., Corrado, G. S., Monga, R., Chen, K., Devin, M., Mao, M., Ranzato, M., Senior, A., Tucker, P. A., Yang, K., et al. (2012) · Cited by 3,199
A 128$\times$128 120 dB 15 $\mu$s Latency Asynchronous Temporal Contrast Vision Sensor
Lichtsteiner, P., Posch, C., & Delbruck, T. (2008) · IEEE Journal of Solid-State Circuits · doi:10.1109/jssc.2007.914337 · Cited by 2,485 · cited 4×
“Over the past few years, a variety of these event-based devices has been developed, including temporal contrast vision sensors that are sensitive to relative luminance change [22], [23], [24], gradient-based sensors sensitive to static edges [25], and optical-flow sensors [26].”
BackgroundMethodInfluentialNeuromorphic electronic systems
Mead, C. (1990) · Proceedings of the IEEE · doi:10.1109/5.58356 · Cited by 2,368
A Novel Connectionist System for Unconstrained Handwriting Recognition
Graves, A., Liwicki, M., Fernandez, S., Bertolami, R., Bunke, H., & Schmidhuber, J. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.137 · Cited by 1,983
Robust Object Recognition with Cortex-Like Mechanisms
Serre, T., Wolf, L., Bileschi, S., Riesenhuber, M., & Poggio, T. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.56 · Cited by 1,667
High Dynamic Range Imaging: Acquisition, Display, and Image-Based Lighting
Reinhard, E., Ward, G., Pattanaik, S., Debevec, P. E., Heidrich, W., & Myszkowski, K. (2010) · Cited by 1,359
Furber, S. B., Galluppi, F., Temple, S., & Plana, L. A. (2014) · Proceedings of the IEEE · doi:10.1109/jproc.2014.2304638 · Cited by 1,350
Self-citationSpiking Deep Convolutional Neural Networks for Energy-Efficient Object Recognition
Cao, Y., Chen, Y., & Khosla, D. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-014-0788-3 · Cited by 917
Eye Smarter than Scientists Believed: Neural Computations in Circuits of the Retina
Gollisch, T., & Meister, M. (2010) · Neuron · doi:10.1016/j.neuron.2009.12.009 · Cited by 795
Posch, C., Matolin, D., & Wohlgenannt, R. (2011) · IEEE Journal of Solid-State Circuits · doi:10.1109/jssc.2010.2085952 · Cited by 793
A Biologically Inspired System for Action Recognition
Jhuang, H., Serre, T., Wolf, L., & Poggio, T. (2007) · 2007 IEEE 11th International Conference on Computer Vision · doi:10.1109/iccv.2007.4408988 · Cited by 723
Point-to-point connectivity between neuromorphic chips using address events
Boahen, K. A. (2000) · IEEE Transactions on Circuits and Systems II: Analog and Digital Signal Processing · doi:10.1109/82.842110 · Cited by 654
Unsupervised Learning of Visual Features through Spike Timing Dependent Plasticity
Masquelier, T., & Thorpe, S. J. (2007) · PLoS Computational Biology · doi:10.1371/journal.pcbi.0030031 · Cited by 548
Image and Video Compression Standards
Bhaskaran, V., & Konstantinides, K. (1997) · Springer eBooks · doi:10.1007/978-1-4615-6199-6 · Cited by 483
Retinomorphic Event-Based Vision Sensors: Bioinspired Cameras With Spiking Output
Posch, C., Serrano-Gotarredona, T., Linares-Barranco, B., & Delbruck, T. (2014) · Proceedings of the IEEE · doi:10.1109/jproc.2014.2346153 · Cited by 471
Real-time classification and sensor fusion with a spiking deep belief network
O'Connor, P., Neil, D., Liu, S.-C., Delbruck, T., & Pfeiffer, M. (2013) · Frontiers in Neuroscience · doi:10.3389/fnins.2013.00178 · Cited by 408
Serrano-Gotarredona, T., & Linares-Barranco, B. (2013) · IEEE Journal of Solid-State Circuits · doi:10.1109/jssc.2012.2230553 · Cited by 334
Serrano-Gotarredona, R., Oster, M., Lichtsteiner, P., Linares-Barranco, A., Paz-Vicente, R., Gomez-Rodriguez, F., Camunas-Mesa, L., Berner, R., Rivas-Perez, M., Delbruck, T., et al. (2009) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2009.2023653 · Cited by 325
Perez-Carrasco, J. A., Zhao, B., Serrano, C., Acha, B., Serrano-Gotarredona, T., Chen, S., & Linares-Barranco, B. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.71 · Cited by 324
HFirst: A Temporal Approach to Object Recognition
Orchard, G., Meyer, C., Etienne-Cummings, R., Posch, C., Thakor, N., & Benosman, R. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2392947 · Cited by 312
Self-citationMahowald, M. A., & Mead, C. (1991) · Scientific American · doi:10.1038/scientificamerican0591-76 · Cited by 270
A 3.6 $\mu$s Latency Asynchronous Frame-Free Event-Driven Dynamic-Vision-Sensor
Lenero-Bardallo, J. A., Serrano-Gotarredona, T., & Linares-Barranco, B. (2011) · IEEE Journal of Solid-State Circuits · doi:10.1109/jssc.2011.2118490 · Cited by 202
A Contrast Sensitive Silicon Retina with Reciprocal Synapses
Boahen, K. A., & Andreou, A. G. (1991) · CaltechAUTHORS (California Institute of Technology) · Cited by 185
Poker-DVS and MNIST-DVS. Their History, How They Were Made, and Other Details
Serrano-Gotarredona, T., & Linares-Barranco, B. (2015) · Frontiers in Neuroscience · doi:10.3389/fnins.2015.00481 · Cited by 161
An asynchronous time-based image sensor
Posch, C., Matolin, D., & Wohlgenannt, R. (2008) · 2008 IEEE International Symposium on Circuits and Systems (ISCAS) · doi:10.1109/iscas.2008.4541871 · Cited by 132
Synthesis of neural networks for spatio-temporal spike pattern recognition and processing
Tapson, J. C., Cohen, G., Afshar, S., Stiefel, K. M., Buskila, Y., Wang, R. M., Hamilton, T. J., & André, V. S. (2013) · Frontiers in Neuroscience · doi:10.3389/fnins.2013.00153 · Cited by 78
Spatiotemporal features for asynchronous event-based data
Lagorce, X., Ieng, S.-H., Clady, X., Pfeiffer, M., & Benosman, R. B. (2015) · Frontiers in Neuroscience · doi:10.3389/fnins.2015.00046 · Cited by 55
Self-citationWhat Can Neuromorphic Event-Driven Precise Timing Add to Spike-Based Pattern Recognition?
Akolkar, H., Meyer, C., Clady, X., Marre, O., Bartolozzi, C., Panzeri, S., & Benosman, R. B. (2015) · Neural Computation · doi:10.1162/neco_a_00703 · Cited by 55
Self-citationEfficient Feedforward Categorization of Objects and Human Postures with Address-Event Image Sensors
Chen, S., Akselrod, P., Zhao, B., Pérez‐Carrasco, J. A., Linares-Barranco, B., & Culurciello, E. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2011.120 · Cited by 55
Analog Integrated 2-D Optical Flow Sensor
Stocker, A. A. (2005) · Analog Integrated Circuits and Signal Processing · doi:10.1007/s10470-005-0439-2 · Cited by 25
Folowosele, F. O., Vogelstein, R. J., & Etienne‐Cummings, R. (2011) · IEEE Journal on Emerging and Selected Topics in Circuits and Systems · doi:10.1109/jetcas.2012.2183409 · Cited by 22
Dynamic Coding of Signed Quantities in Cortical Feedback Circuits
Ballard, D. H., & Jehee, J. F. M. (2012) · Frontiers in Psychology · doi:10.3389/fpsyg.2012.00254 · Cited by 18
Cited by
Gallego, G., Delbruck, T., Orchard, G., Bartolozzi, C., Taba, B., Censi, A., Leutenegger, S., Davison, A. J., Conradt, J., Daniilidis, K., et al. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3008413 · Cited by 2,566
Self-citationDeep learning in spiking neural networks
Tavanaei, A., Ghodrati, M., Kheradpisheh, S. R., Masquelier, T., & Maida, A. (2019) · Neural Networks · doi:10.1016/j.neunet.2018.12.002 · Cited by 1,386
Deep Learning With Spiking Neurons: Opportunities and Challenges
Pfeiffer, M., & Pfeil, T. (2018) · Frontiers in Neuroscience · doi:10.3389/fnins.2018.00774 · Cited by 766
Direct Training for Spiking Neural Networks: Faster, Larger, Better
Wu, Y., Deng, L., Li, G., Zhu, J., Xie, Y., & Shi, L. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33011311 · Cited by 670
Moradi, S., Qiao, N., Stefanini, F., & Indiveri, G. (2018) · IEEE Transactions on Biomedical Circuits and Systems · doi:10.1109/tbcas.2017.2759700 · Cited by 650
Metadata
- Authors
- Lagorce, X., Orchard, G., Galluppi, F., Shi, B. E., & Benosman, R. B.
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2017-07-01
- Volume
- 39
- Issue
- 7
- Pages
- 1346-1359
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- OpenAlex
- W2469278928
- Cited by
- 612
- References
- 35
- FWCI
- 11.43
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.