Enabling deep learning at the IoT edge
The DOI matches its Crossref record; 12 of the 13 references it cites were found in a registry or an index.
Abstract
Deep learning algorithms have demonstrated super-human capabilities in many cognitive tasks, such as image classification and speech recognition. As a result, there is an increasing interest in deploying neural networks (NNs) on low-power processors found in always-on systems, such as those based on Arm Cortex-M microcontrollers. In this paper, we discuss the challenges of deploying neural networks on microcontrollers with limited memory, compute resources and power budgets. We introduce CMSIS-NN, a library of optimized software kernels to enable deployment of NNs on Cortex-M cores. We also present techniques for NN algorithm exploration to develop light-weight models suitable for resource constrained systems, using keyword spotting as an example.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., & Adam, H. (2017) · arXiv · doi:10.48550/arxiv.1704.04861 · Cited by 19,015
TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., et al. (2016) · arXiv · doi:10.48550/arxiv.1603.04467 · Cited by 9,408
Edge computing: Vision and challenges
Shi, W., Cao, J., Zhang, Q., Li, Y., & Xu, L. (2016) · IEEE Internet of Things Journal · doi:10.1109/jiot.2016.2579198 · Cited by 8,733
Fixed Point Quantization of Deep Convolutional Networks
Lin, D. D., Talathi, S. S., & Annapureddy, V. S. (2015) · arXiv · doi:10.48550/arxiv.1511.06393 · Cited by 684
Throughput-Optimized OpenCL-based FPGA Accelerator for Large-Scale Convolutional Neural Networks
Suda, N., Chandra, V., Dasika, G., Mohanty, A., Ma, Y., Vrudhula, S., Seo, J.-S., & Cao, Y. (2016) · Proceedings of the 2016 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays · doi:10.1145/2847263.2847276 · Cited by 553
Self-citationSmall-footprint keyword spotting using deep neural networks
Chen, G., Parada, C., & Heigold, G. (2014) · 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2014.6854370 · Cited by 545
Convolutional neural networks for small-footprint keyword spotting
Sainath, T. N., & Parada, C. (2015) · Interspeech 2015 · doi:10.21437/interspeech.2015-352 · Cited by 521
Hello Edge: Keyword Spotting on Microcontrollers
Zhang, Y., Suda, N., Lai, L., & Chandra, V. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.07128 · Cited by 334
Self-citationCMSIS-NN: Efficient Neural Network Kernels for Arm Cortex-M CPUs
Oliveira, A., Costa, D., Moreira, G., Martins, J., & Pinto, S. (2018) · arXiv (Cornell University) · doi:10.4230/oasics.ng-res.2025.4 · Cited by 288
Self-citationConvolutional Recurrent Neural Networks for Small-Footprint Keyword Spotting
Arık, S. Ö., Kliegl, M., Child, R., Hestness, J., Gibiansky, A., Fougner, C., Prenger, R., & Coates, A. (2017) · Interspeech 2017 · doi:10.21437/interspeech.2017-1737 · Cited by 148
Max-pooling loss training of long short-term memory networks for small-footprint keyword spotting
Sun, M., Raju, A., Tucker, G., Panchapagesan, S., Fu, G., Mandal, A., Matsoukas, S., Strom, N., & Vitaladevuni, S. N. P. (2016) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt.2016.7846306 · Cited by 142
Optimizing Memory Efficiency for Deep Convolutional Neural Networks on GPUs
Li, C., Yang, Y., Feng, M., Chakradhar, S., & Zhou, H. (2016) · International Conference for High Performance Computing, Networking, Storage and Analysis (SC) · doi:10.1109/sc.2016.53 · Cited by 130
Research works printed but not linked: 4
- Liangzhen Lai, Naveen Suda, and Vikas Chandra. 2017. Deep Convolutional Neural Network Inference with Floating-point Weights and Fixed-point Activations. arXiv preprint arXiv:1703.03073 (2017). · arXiv
- 10.5555/3014904.3014977 · doi:10.5555/3014904.3014977
- 10.5555/3045390.3045690 · doi:10.5555/3045390.3045690
- Pete Warden. 2017. Speech Commands: A public dataset for single-word speech recognition. Dataset available from http://download.tensorflow.org/data/speech_commands_v0.01.tar.gz (2017).
Cited by
Deep Learning With Edge Computing: A Review
Chen, J., & Ran, X. (2019) · Proceedings of the IEEE · doi:10.1109/jproc.2019.2921977 · Cited by 1,540
Convergence of Edge Computing and Deep Learning: A Comprehensive Survey
Wang, X., Han, Y., Leung, V. C. M., Niyato, D., Yan, X., & Chen, X. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.2970550 · Cited by 1,498
“A software framework is valuable if it 1) provides a library of optimized software kernels to enable deployment of DL [150]; 2) automatically compresses DL models into smaller dense matrices by finding the minimum number of non-redundant hidden elements [151]; 3) performs quantization and coding on”
MethodEfficient Acceleration of Deep Learning Inference on Resource-Constrained Edge Devices: A Review
Shuvo, M. M. H., Islam, S. K., Cheng, J., & Morshed, B. I. (2023) · Proceedings of the IEEE · doi:10.1109/jproc.2022.3226481 · Cited by 436
Machine Learning at the Network Edge: A Survey
Murshed, M. G. S., Murphy, C., Hou, D., Khan, N., Ananthanarayanan, G., & Hussain, F. (2022) · ACM Computing Surveys · doi:10.1145/3469029 · Cited by 402
Quantization and Deployment of Deep Neural Networks on Microcontrollers
Novac, P.-E., Boukli Hacene, G., Pegatoquet, A., Miramond, B., & Gripon, V. (2021) · Sensors · doi:10.3390/s21092984 · Cited by 210
Metadata
- Authors
- Lai, L., & Suda, N.
- Affiliations
- Arm Inc.
- ORCID
- Lai, Liangzhen
- Venue
- Proceedings of the International Conference on Computer-Aided Design
- Publisher
- ACM
- Published
- 2018-11-05
- Pages
- 1-6
- OpenAlex
- W2900071409
- Cited by
- 49
- References
- 13
- Citations by year
- 2026: 2 · 2025: 3 · 2024: 9 · 2023: 10 · 2022: 5 · 2021: 4 · 2020: 12 · 2019: 4
- FWCI
- 1.51
- Type
- conference paper
- Language
- en
- Open access
- closed
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Networks and Communications › IoT and Edge/Fog Computing
- Physical Sciences › Computer Science › Hardware and Architecture › Embedded Systems Design Techniques
- Keywords
- deep learning deployment, edge computing, Internet of Things, neural network inference, resource-constrained systems, keyword spotting, lightweight neural networks, low-power processors
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.