The DOI matches its Crossref record; 67 of the 67 references it cites were found in a registry or an index.
End-to-End Learning of Representations for Asynchronous Event-Based Data
Open accessAbstract
Event cameras are vision sensors that record asynchronous streams of per-pixel brightness changes, referred to as "events”. They have appealing advantages over frame based cameras for computer vision, including high temporal resolution, high dynamic range, and no motion blur. Due to the sparse, non-uniform spatio-temporal layout of the event signal, pattern recognition algorithms typically aggregate events into a grid-based representation and subsequently process it by a standard vision pipeline, e.g., Convolutional Neural Network (CNN). In this work, we introduce a general framework to convert event streams into grid-based representations by means of strictly differentiable operations. Our framework comes with two main advantages: (i) allows learning the input event representation together with the task dedicated network in an end to end manner, and (ii) lays out a taxonomy that unifies the majority of extant event representations in the literature and identifies novel ones. Empirically, we show that our approach to learning the event representation end-to-end yields an improvement of approximately 12% on optical flow estimation and object recognition over state-of-the-art methods.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Distinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589
You Only Look Once: Unified, Real-Time Object Detection
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.91 · Cited by 53,803
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
(2014) · Cited by 33,419
The pascal visual object classes (VOC) challenge
Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2010) · International Journal of Computer Vision · doi:10.1007/s11263-009-0275-4 · Cited by 19,599
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204
Robust Real-Time Face Detection
Viola, P., & Jones, M. J. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000013087.49260.fb · Cited by 14,097
TensorFlow: A system for large-scale machine learning
Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.08695 · Cited by 11,822
(2017) · Cited by 11,625
XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks
Rastegari, M., Ordonez, V., Redmon, J., & Farhadi, A. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_32 · Cited by 4,408
Flownet: Learning optical flow with convolutional networks
Dosovitskiy, A., Fischer, P., Ilg, E., Hausser, P., Hazirbas, C., Golkov, V., Smagt, P. V. D., Cremers, D., & Brox, T. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.316 · Cited by 4,313
BRISK: Binary Robust invariant scalable keypoints
Leutenegger, S., Chli, M., & Siegwart, R. Y. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126542 · Cited by 3,450
Pedestrian Detection: An Evaluation of the State of the Art
Dollar, P., Wojek, C., Schiele, B., & Perona, P. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2011.155 · Cited by 3,190
FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks
Ilg, E., Mayer, N., Saikia, T., Keuper, M., Dosovitskiy, A., & Brox, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.179 · Cited by 3,152
One-shot learning of object categories
Fei-Fei, L., Fergus, R., & Perona, P. (2006) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2006.79 · Cited by 3,029
NetVLAD: CNN Architecture for Weakly Supervised Place Recognition
Arandjelovic, R., Gronat, P., Torii, A., Pajdla, T., & Sivic, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.572 · Cited by 2,584
Gallego, G., Delbruck, T., Orchard, G., Bartolozzi, C., Taba, B., Censi, A., Leutenegger, S., Davison, A. J., Conradt, J., Daniilidis, K., et al. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3008413 · Cited by 2,566
Self-citationA 128$\times$128 120 dB 15 $\mu$s Latency Asynchronous Temporal Contrast Vision Sensor
Lichtsteiner, P., Posch, C., & Delbruck, T. (2008) · IEEE Journal of Solid-State Circuits · doi:10.1109/jssc.2007.914337 · Cited by 2,485
NetVLAD: CNN architecture for weakly supervised place recognitionPreprint
Arandjelović, R., Gronát, P., Torii, A., Pajdla, T., & Šivic, J. (2015) · arXiv (Cornell University) · Cited by 1,790
A 240 × 180 130 dB 3 µs Latency Global Shutter Spatiotemporal Vision Sensor
Brandli, C., Berner, R., Yang, M., Liu, S.-C., & Delbruck, T. (2014) · IEEE Journal of Solid-State Circuits · doi:10.1109/jssc.2014.2342715 · Cited by 1,144
On optimization methods for deep learning
Ngiam, J., Coates, A., Lahiri, A., Prochnow, B., Le, Q. V., & Ng, A. Y. (2011) · International Conference on Machine Learning · Cited by 1,076
A Low Power, Fully Event-Based Gesture Recognition System
Amir, A., Taba, B., Berg, D., Melano, T., McKinstry, J., Di Nolfo, C., Nayak, T., Andreopoulos, A., Garreau, G., Mendoza, M., et al. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.781 · Cited by 1,017
Fast-classifying, high-accuracy spiking deep networks through weight and threshold balancing
Diehl, P. U., Neil, D., Binas, J., Cook, M., Liu, S.-C., & Pfeiffer, M. (2015) · 2015 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2015.7280696 · Cited by 991
Training Deep Spiking Neural Networks Using Backpropagation
Lee, J. H., Delbruck, T., & Pfeiffer, M. (2016) · Frontiers in Neuroscience · doi:10.3389/fnins.2016.00508 · Cited by 976
Converting Static Image Datasets to Spiking Neuromorphic Datasets Using Saccades
Orchard, G., Jayawant, A., Cohen, G. K., & Thakor, N. (2015) · Frontiers in Neuroscience · doi:10.3389/fnins.2015.00437 · Cited by 882
Posch, C., Matolin, D., & Wohlgenannt, R. (2011) · IEEE Journal of Solid-State Circuits · doi:10.1109/jssc.2010.2085952 · Cited by 793
Sun, D., Roth, S., & Black, M. J. (2014) · International Journal of Computer Vision · doi:10.1007/s11263-013-0644-x · Cited by 614
HOTS: A Hierarchy of Event-Based Time-Surfaces for Pattern Recognition
Lagorce, X., Orchard, G., Galluppi, F., Shi, B. E., & Benosman, R. B. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2574707 · Cited by 612
Gallego, G., Delbrück, T., Orchard, G., Bartolozzi, C., Taba, B., Censi, A., Leutenegger, S., Davison, A. J., Conradt, J., Daniilidis, K., et al. (2020) · Zurich Open Repository and Archive (University of Zurich) · doi:10.3929/ethz-b-000521357 · Cited by 591
Self-citationUnsupervised Event-Based Learning of Optical Flow, Depth, and Egomotion
Zhu, A. Z., Yuan, L., Chaney, K., & Daniilidis, K. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00108 · Cited by 571
The Multivehicle Stereo Event Camera Dataset: An Event Camera Dataset for 3D Perception
Zhu, A. Z., Thakur, D., Ozaslan, T., Pfrommer, B., Kumar, V., & Daniilidis, K. (2018) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2018.2800793 · Cited by 567
Vidal, A. R., Rebecq, H., Horstschaefer, T., & Scaramuzza, D. (2018) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2018.2793357 · Cited by 541
Self-citationUnFlow: Unsupervised Learning of Optical Flow With a Bidirectional Census Loss
Meister, S., Hur, J., & Roth, S. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12276 · Cited by 520
Efficient Visual Search of Videos Cast as Text Retrieval
Sivic, J., & Zisserman, A. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.111 · Cited by 499
Event-Based Vision Meets Deep Learning on Steering Prediction for Self-Driving Cars
Maqueda, A. I., Loquercio, A., Gallego, G., Garcia, N. N., & Scaramuzza, D. (2018) · Zurich Open Repository and Archive (University of Zurich) · doi:10.5167/uzh-175994 · Cited by 493
Self-citationHATS: Histograms of Averaged Time Surfaces for Robust Event-Based Object Classification
Sironi, A., Brambilla, M., Bourdis, N., Lagorce, X., & Benosman, R. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00186 · Cited by 486
EV-FlowNet: Self-Supervised Optical Flow Estimation for Event-based Cameras
Zhu, A., Yuan, L., Chaney, K., & Daniilidis, K. (2018) · Robotics: Science and Systems XIV · doi:10.15607/rss.2018.xiv.062 · Cited by 470
Real-Time 3D Reconstruction and 6-DoF Tracking with an Event Camera
Kim, H., Leutenegger, S., & Davison, A. J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46466-4_21 · Cited by 438
Benosman, R., Clercq, C., Lagorce, X., Ieng, S.-H., & Bartolozzi, C. (2014) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2013.2273537 · Cited by 419
EVO: A Geometric Approach to Event-Based 6-DOF Parallel Tracking and Mapping in Real Time
Rebecq, H., Horstschaefer, T., Gallego, G., & Scaramuzza, D. (2017) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2016.2645143 · Cited by 410
Self-citationJOINT 3D ESTIMATION OF VEHICLES AND SCENE FLOW
Menze, M., Heipke, C., & Geiger, A. (2015) · ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences · doi:10.5194/isprsannals-ii-3-w5-427-2015 · Cited by 341
Perez-Carrasco, J. A., Zhao, B., Serrano, C., Acha, B., Serrano-Gotarredona, T., Chen, S., & Linares-Barranco, B. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.71 · Cited by 324
Phased LSTM: Accelerating Recurrent Network Training for Long or Event-based Sequences
Neil, D., Pfeiffer, M., & Liu, S. (2016) · Zurich Open Repository and Archive (University of Zurich) · doi:10.48550/arxiv.1610.09513 · Cited by 319
HFirst: A Temporal Approach to Object Recognition
Orchard, G., Meyer, C., Etienne-Cummings, R., Posch, C., Thakor, N., & Benosman, R. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2392947 · Cited by 312
Simultaneous Optical Flow and Intensity Estimation from an Event Camera
Bardow, P., Davison, A. J., & Leutenegger, S. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.102 · Cited by 295
EMVS: Event-Based Multi-View Stereo—3D Reconstruction with an Event Camera in Real-Time
Rebecq, H., Gallego, G., Mueggler, E., & Scaramuzza, D. (2018) · International Journal of Computer Vision · doi:10.1007/s11263-017-1050-6 · Cited by 278
Self-citationGallego, G., Rebecq, H., & Scaramuzza, D. (2018) · Zurich Open Repository and Archive (University of Zurich) · doi:10.5167/uzh-176051 · Cited by 271
Self-citationReal-time Visual-Inertial Odometry for Event Cameras using Keyframe-based Nonlinear Optimization
Rebecq, H., Horstschaefer, T., & Scaramuzza, D. (2017) · Procedings of the British Machine Vision Conference 2017 · doi:10.5244/c.31.16 · Cited by 263
Self-citationEvent-based, 6-DOF pose tracking for high-speed maneuvers
Mueggler, E., Huber, B., & Scaramuzza, D. (2014) · 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2014.6942940 · Cited by 258
Self-citationEvent-Based Visual Inertial Odometry
Zhu, A. Z., Atanasov, N., & Daniilidis, K. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.616 · Cited by 249
Asynchronous frameless event-based optical flow
Benosman, R., Ieng, S.-H., Clercq, C., Bartolozzi, C., & Srinivasan, M. (2012) · Neural Networks · doi:10.1016/j.neunet.2011.11.001 · Cited by 228
Zhao, B., Ding, R., Chen, S., Linares-Barranco, B., & Tang, H. (2015) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2014.2362542 · Cited by 216
Event-Based, 6-DOF Camera Tracking from Photometric Depth Maps
Gallego, G., Lund, J. E. A., Mueggler, E., Rebecq, H., Delbruck, T., & Scaramuzza, D. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2769655 · Cited by 211
Self-citationEvent-based feature tracking with probabilistic data association
Zhu, A. Z., Atanasov, N., & Daniilidis, K. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989517 · Cited by 174
Asynchronous, Photometric Feature Tracking Using Events and Frames
Gehrig, D., Rebecq, H., Gallego, G., & Scaramuzza, D. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01258-8_46 · Cited by 157
Self-citationDirect face detection and video reconstruction from event cameras
Miyatani, Y., Barua, S., & Veeraraghavan, A. (2016) · 2016 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv.2016.7477561 · Cited by 133 · cited 1×
“From this line of research, typical high-level applications are gesture recognition [31], object recognition [29,45,59] or face detection [6].”
BackgroundAsynchronous event‐based high speed vision for microparticle tracking
Ni, Z., Pacoret, C., Benosman, R. B., Ieng, S.-H., & Régnier, S. (2011) · Journal of Microscopy · doi:10.1111/j.1365-2818.2011.03565.x · Cited by 120
Realtime Time Synchronized Event-Based Stereo
Zhu, A. Z., Chen, Y., & Daniilidis, K. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01231-1_27 · Cited by 93
A Low Power, High Throughput, Fully Event-Based Stereo System
Andreopoulos, A., Kashyap, H. J., Nayak, T. K., Amir, A., & Flickner, M. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00786 · Cited by 65
Asynchronous Event-Based Hebbian Epipolar Geometry
Benosman, R. B., Ieng, S.-H., Rogister, P., & Posch, C. (2011) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2011.2167239 · Cited by 58
Live demonstration: Gesture-based remote control using stereo pair of dynamic vision sensors
Lee, J., Delbrück, T., Park, P. K. J., Pfeiffer, M., Shin, C.-W., Ryu, H., & Kang, B. C. (2012) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas.2012.6272144 · Cited by 37
Park, P. K. J., Lee, K., Lee, J. H., et al. (2015) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2015.7350936 · Cited by 11
Efficient neuromorphic optomotor heading regulation
Censi, A. (2015) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.1109/acc.2015.7171931 · Cited by 10
Cited by
Gallego, G., Delbruck, T., Orchard, G., Bartolozzi, C., Taba, B., Censi, A., Leutenegger, S., Davison, A. J., Conradt, J., Daniilidis, K., et al. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3008413 · Cited by 2,566
Self-citationGallego, G., Delbrück, T., Orchard, G., Bartolozzi, C., Taba, B., Censi, A., Leutenegger, S., Davison, A. J., Conradt, J., Daniilidis, K., et al. (2020) · Zurich Open Repository and Archive (University of Zurich) · doi:10.3929/ethz-b-000521357 · Cited by 591
Self-citationv2e: From Video Frames to Realistic DVS Events
Hu, Y., Liu, S.-C., & Delbruck, T. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00144 · Cited by 408
Chen, G., Cao, H., Conradt, J., Tang, H., Rohrbein, F., & Knoll, A. (2020) · IEEE Signal Processing Magazine · doi:10.1109/msp.2020.2985815 · Cited by 304
Event-driven Re-Id: A New Benchmark and Method Towards Privacy-Preserving Person Re-Identification
Ahmad, S., Scarpellini, G., Morerio, P., & Bue, A. D. (2022) · 2022 IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW) · doi:10.1109/wacvw54805.2022.00052 · Cited by 54
Metadata
- Authors
- Gehrig, D., Loquercio, A., Derpanis, K., & Scaramuzza, D.
- Venue
- 2019 IEEE/CVF International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2019-10
- Pages
- 5632-5642
- OpenAlex
- W3010268791
- Cited by
- 386
- References
- 67
- FWCI
- 11.72
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.