The DOI matches its Crossref record; 40 of the 40 references it cites were found in a registry or an index.
RMP-SNN: Residual Membrane Potential Neuron for Enabling Deeper High-Accuracy and Low-Latency Spiking Neural Network
Open accessAbstract
Spiking Neural Networks (SNNs) have recently attracted significant research interest as the third generation of artificial neural networks that can enable low-power event-driven data analytics. The best performing SNNs for image recognition tasks are obtained by converting a trained Analog Neural Network (ANN), consisting of Rectified Linear Units (ReLU), to SNN composed of integrate-and-fire neurons with "proper" firing thresholds. The converted SNNs typically incur loss in accuracy compared to that provided by the original ANN and require sizable number of inference time-steps to achieve the best accuracy. We find that performance degradation in the converted SNN stems from using "hard reset" spiking neuron that is driven to fixed reset potential once its membrane potential exceeds the firing threshold, leading to information loss during SNN inference. We propose ANN-SNN conversion using "soft reset" spiking neuron model, referred to as Residual Membrane Potential (RMP) spiking neuron, which retains the "residual" membrane potential above threshold at the firing instants. We demonstrate near loss-less ANN-SNN conversion using RMP neurons for VGG-16, ResNet-20, and ResNet-34 SNNs on challenging datasets including CIFAR-10 (93.63% top-1), CIFAR-100 (70.93% top-1), and ImageNet (73.09% top-1 accuracy). Our results also show that RMP-SNN surpasses the best inference accuracy provided by the converted SNN with "hard reset" spiking neurons using 2-8 times fewer inference time-steps across network architectures and datasets.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
You Only Look Once: Unified, Real-Time Object Detection
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.91 · Cited by 53,803
(2010) · International Conference on Machine Learning · Cited by 13,995
A million spiking-neuron integrated circuit with a scalable communication network and interface
Merolla, P. A., Arthur, J. V., Alvarez-Icaza, R., Cassidy, A. S., Sawada, J., Akopyan, F., Jackson, B. L., Imam, N., Guo, C., Nakamura, Y., et al. (2014) · Science · doi:10.1126/science.1254642 · Cited by 4,147
Loihi: A Neuromorphic Manycore Processor with On-Chip Learning
Davies, M., Srinivasa, N., Lin, T.-H., Chinya, G., Cao, Y., Choday, S. H., Dimou, G., Joshi, P., Imam, N., Jain, S., et al. (2018) · IEEE Micro · doi:10.1109/mm.2018.112130359 · Cited by 3,897
Networks of spiking neurons: The third generation of neural network models
Maass, W. (1997) · Neural Networks · doi:10.1016/s0893-6080(97)00011-7 · Cited by 3,481
(2011) · Cited by 2,752
Google’s Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation
Johnson, M., Schuster, M., Le, Q. V., Krikun, M., Wu, Y., Chen, Z., Thorat, N., Viégas, F., Wattenberg, M., Corrado, G., et al. (2017) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00065 · Cited by 1,747
Unsupervised learning of digit recognition using spike-timing-dependent plasticity
Diehl, P. U., & Cook, M. (2015) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2015.00099 · Cited by 1,452
Going Deeper in Spiking Neural Networks: VGG and Residual Architectures
Sengupta, A., Ye, Y., Wang, R., Liu, C., & Roy, K. (2019) · Frontiers in Neuroscience · doi:10.3389/fnins.2019.00095 · Cited by 1,211
Self-citationSpatio-Temporal Backpropagation for Training High-Performance Spiking Neural Networks
Wu, Y., Deng, L., Li, G., Zhu, J., & Shi, L. (2018) · Frontiers in Neuroscience · doi:10.3389/fnins.2018.00331 · Cited by 1,196
Rueckauer, B., Lungu, I.-A., Hu, Y., Pfeiffer, M., & Liu, S.-C. (2017) · Frontiers in Neuroscience · doi:10.3389/fnins.2017.00682 · Cited by 1,072
Fast-classifying, high-accuracy spiking deep networks through weight and threshold balancing
Diehl, P. U., Neil, D., Binas, J., Cook, M., Liu, S.-C., & Pfeiffer, M. (2015) · 2015 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2015.7280696 · Cited by 991
Training Deep Spiking Neural Networks Using Backpropagation
Lee, J. H., Delbruck, T., & Pfeiffer, M. (2016) · Frontiers in Neuroscience · doi:10.3389/fnins.2016.00508 · Cited by 976
Spiking Deep Convolutional Neural Networks for Energy-Efficient Object Recognition
Cao, Y., Chen, Y., & Khosla, D. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-014-0788-3 · Cited by 917
Convolutional networks for fast, energy-efficient neuromorphic computing
Esser, S. K., Merolla, P. A., Arthur, J. V., Cassidy, A. S., Appuswamy, R., Andreopoulos, A., Berg, D. J., McKinstry, J. L., Melano, T., Barch, D. R., et al. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1604850113 · Cited by 781
STDP-based spiking deep convolutional neural networks for object recognition
Kheradpisheh, S. R., Ganjtabesh, M., Thorpe, S. J., & Masquelier, T. (2018) · Neural Networks · doi:10.1016/j.neunet.2017.12.005 · Cited by 768
Long short-term memory and learning-to-learn in networks of spiking neuronsPreprint
Bellec, G., Salaj, D., Subramoney, A., Legenstein, R. A., & Maass, W. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.09574 · Cited by 683
Unsupervised Learning of Visual Features through Spike Timing Dependent Plasticity
Masquelier, T., & Thorpe, S. J. (2007) · PLoS Computational Biology · doi:10.1371/journal.pcbi.0030031 · Cited by 548
SLAYER: Spike Layer Error Reassignment in Time
Shrestha, S. B., & Orchard, G. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.08646 · Cited by 444
Identity Matters in Deep LearningPreprint
Hardt, M., & Ma, T. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.04231 · Cited by 333
Perez-Carrasco, J. A., Zhao, B., Serrano, C., Acha, B., Serrano-Gotarredona, T., Chen, S., & Linares-Barranco, B. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.71 · Cited by 324
Zhao, B., Ding, R., Chen, S., Linares-Barranco, B., & Tang, H. (2015) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2014.2362542 · Cited by 216
Spiking Deep Networks with LIF Neurons
Hunsberger, E., & Eliasmith, C. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1510.08829 · Cited by 203
Diehl, P. U., Zarrella, G., Cassidy, A., Pedroni, B. U., & Neftci, E. (2016) · 2016 IEEE International Conference on Rebooting Computing (ICRC) · doi:10.1109/icrc.2016.7738691 · Cited by 202
Benchmarking Keyword Spotting Efficiency on Neuromorphic Hardware
Blouw, P., Choo, X., Hunsberger, E., & Eliasmith, C. (2019) · Proceedings of the 7th Annual Neuro-inspired Computational Elements Workshop · doi:10.1145/3320288.3320304 · Cited by 192
Training Spiking Deep Networks for Neuromorphic HardwarePreprint
Hunsberger, E., & Eliasmith, C. (2016) · arXiv (Cornell University) · doi:10.13140/rg.2.2.10967.06566 · Cited by 185
Theory and Tools for the Conversion of Analog to Spiking Convolutional Neural NetworksPreprint
Rueckauer, B., Lungu, I.-A., Hu, Y., & Pfeiffer, M. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.04052 · Cited by 149
Lee, C., Srinivasan, G., Panda, P., & Roy, K. (2018) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2018.2833071 · Cited by 144
Self-citationSurrogate Gradient Learning in Spiking Neural NetworksPreprint
Neftci, E., Mostafa, H., & Zenke, F. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.09948 · Cited by 144
Panda, P., & Roy, K. (2016) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2016.7727212 · Cited by 129
Self-citationUnsupervised Feature Learning With Winner-Takes-All Based STDP
Ferré, T. P. A., Mamalet, F., & Thorpe, S. J. (2018) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2018.00024 · Cited by 110
Event-Based, Timescale Invariant Unsupervised Online Deep Learning With STDP
Thiele, J. C., Bichler, O., & Dupret, A. (2018) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2018.00046 · Cited by 77
Srinivasan, G., & Roy, K. (2019) · Frontiers in Neuroscience · doi:10.3389/fnins.2019.00189 · Cited by 72
Self-citationTraining Spiking ConvNets by STDP and Gradient Descent
Tavanaei, A., Kirby, Z., & Maida, A. S. (2018) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2018.8489104 · Cited by 52
Srinivasan, G., Panda, P., & Roy, K. (2018) · ACM Journal on Emerging Technologies in Computing Systems · doi:10.1145/3266229 · Cited by 44
Self-citationEnabling Spike-based Backpropagation in State-of-the-art Deep Neural Network ArchitecturesPreprint
Lee, C., Sarwar, S. S., & Roy, K. (2019) · arXiv (Cornell University) · Cited by 43
Self-citationMozafari, M., Ganjtabesh, M., Nowzari-Dalini, A., Thorpe, S. J., & Masquelier, T. (2018) · Cited by 39
Cited by
Spiking Neural Networks and Their Applications: A Review
Yamazaki, K., Vo-Ho, V.-K., Bulsara, D., & Le, N. (2022) · Brain Sciences · doi:10.3390/brainsci12070863 · Cited by 626
Incorporating Learnable Membrane Time Constant to Enhance Learning of Spiking Neural Networks
Fang, W., Yu, Z., Chen, Y., Masquelier, T., Huang, T., & Tian, Y. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00266 · Cited by 605
Going Deeper With Directly-Trained Larger Spiking Neural Networks
Zheng, H., Wu, Y., Deng, L., Hu, Y., & Li, G. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i12.17320 · Cited by 506
Attention Spiking Neural Networks
Yao, M., Zhao, G., Zhang, H., Hu, Y., Deng, L., Tian, Y., Xu, B., & Li, G. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3241201 · Cited by 266
Rathi, N., & Roy, K. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3111897 · Cited by 264
Self-citation
Metadata
- Authors
- Han, B., Srinivasan, G., & Roy, K.
- Venue
- 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2020-06
- Pages
- 13555-13564
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W3035644810
- Cited by
- 330
- References
- 40
- FWCI
- 60.61
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.