FANN-on-MCU: An Open-Source Toolkit for Energy-Efficient Neural Network Inference at the Edge of the Internet of Things
Open accessThe DOI matches its Crossref record; 58 of the 59 references it cites were found in a registry or an index.
Abstract
The growing number of low-power smart devices in the Internet of Things is coupled with the concept of “edge computing” that is moving some of the intelligence, especially machine learning, toward the edge of the network. Enabling machine learning algorithms to run on resource-constrained hardware, typically on low-power smart devices, is challenging in terms of hardware (optimized and energy-efficient integrated circuits), algorithmic, and firmware implementations. This article presents a FANN-on-MCU, an open-source toolkit built upon the fast artificial neural network (FANN) library to run lightweight and energy-efficient neural networks on microcontrollers based on both the ARM Cortex-M series and the novel RISC-V-based parallel ultralow-power (PULP) platform. The toolkit takes multilayer perceptrons trained with FANN and generates code targeted to low-power microcontrollers. This article also presents detailed analyses of energy efficiency across the different cores, and the optimizations to handle different network sizes. Moreover, it provides a detailed analysis of parallel speedups and degradations due to parallelization overhead and memory transfers. Further evaluations include experimental results for three different applications using a self-sustainable wearable multisensor bracelet. The experimental results show a measured latency in the order of only a few microseconds and power consumption of a few milliwatts while keeping the memory requirements below the limitations of the targeted microcontrollers. In particular, the parallel implementation on the octa-core RISC-V platform reaches a speedup of 22× and a 69% reduction in energy consumption with respect to a single-core implementation on Cortex-M4 for continuous real-time classification.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The text we found for this paper looks like another document, so the self-check wasn’t run.
Citations
Cites
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.123 · Cited by 19,741 · cited 2×
“Recent results on machine learning models demonstrate impressive classification accuracy, in some cases even outperforming humans [1].”
BackgroundInternet of Things for Smart Cities
Zanella, A., Bui, N., Castellani, A., Vangelista, L., & Zorzi, M. (2014) · IEEE Internet of Things Journal · doi:10.1109/jiot.2014.2306328 · Cited by 6,292 · cited 1×
“One essential feature of machine learning and in particular neural networks (NNs) is their flexibility that makes them suitable for a wide range of applications, including computer vision [2], [3], natural language processing [4], [5], biomedical [6], [7], and several others [8]–[11].”
BackgroundEdge Computing for the Internet of Things: A Case Study
Premsankar, G., Di Francesco, M., & Taleb, T. (2018) · IEEE Internet of Things Journal · doi:10.1109/jiot.2018.2805263 · Cited by 697 · cited 1×
“The reasons are the massive workload on the IoT network, the cost of the communication infrastructure, the required energy for data transmission, and, more generally, reliability, latency, and privacy concerns [15], [16].”
BackgroundNear-Threshold RISC-V Core With DSP Extensions for Scalable IoT Endpoint Devices
Gautschi, M., Schiavone, P. D., Traber, A., Loi, I., Pullini, A., Rossi, D., Flamand, E., Gurkaynak, F. K., & Benini, L. (2017) · IEEE Transactions on Very Large Scale Integration (VLSI) Systems · doi:10.1109/tvlsi.2017.2654506 · Cited by 507
BLE Beacons for Internet of Things Applications: Survey, Challenges, and Opportunities
Jeon, K. E., She, J., Soonsawad, P., & Ng, P. C. (2018) · IEEE Internet of Things Journal · doi:10.1109/jiot.2017.2788449 · Cited by 391 · cited 1×
“Thus, bringing intelligence to the edge is creating fascinating challenges for industrial and academic researchers [10], [12], [21].”
BackgroundCan, Y. S., Chalabianloo, N., Ekiz, D., & Ersoy, C. (2019) · Sensors · doi:10.3390/s19081849 · Cited by 349 · cited 1×
“Many studies identified MLPs to be the best or one of the best algorithms to solve tasks in the IoT domain using wearable devices [57]–[60].”
BackgroundCMSIS-NN: Efficient Neural Network Kernels for Arm Cortex-M CPUs
Oliveira, A., Costa, D., Moreira, G., Martins, J., & Pinto, S. (2018) · arXiv (Cornell University) · doi:10.4230/oasics.ng-res.2025.4 · Cited by 288 · cited 1×
“In parallel, ARM has been developing CMSIS-NN [48], an additional library for their Cortex Microcontroller Software Interface Standard (CMSIS), in order to provide optimal-performance implementations of layers commonly present in deep NNs with speedups in the order of 4.6 × over a simple baseline…”
MethodInfluentialA Robust and Energy-Efficient Classifier Using Brain-Inspired Hyperdimensional Computing
Rahimi, A., Kanerva, P., & Rabaey, J. M. (2016) · Proceedings of the 2016 International Symposium on Low Power Electronics and Design · doi:10.1145/2934583.2934624 · Cited by 276 · cited 1×
“Today’s IoT devices host microcontrollers (MCUs), especially from the ARM Cortex-M family, which are able to achieve power consumption in the order of mW and computational resources in the order of hundreds of MOPS [1], [26], [27].”
BackgroundYodaNN: An Architecture for Ultralow Power Binary-Weight CNN Acceleration
Andri, R., Cavigelli, L., Rossi, D., & Benini, L. (2018) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2017.2682138 · Cited by 247 · cited 1×
“The most promising approaches to improve the performance of ultra-low-power processors are parallelism, low-power fixed-function hardware accelerators [32], [33], memory access optimization [34], and near-threshold technology [35], [36].”
BackgroundSelf-citationFrom Cloud Down to Things: An Overview of Machine Learning in Internet of Things
Samie, F., Bauer, L., & Henkel, J. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2019.2893866 · Cited by 240 · cited 1×
“to the sensors where the data are collected [13].”
Background
Research works printed but not linked: 3
- 2019, MYO Gesture Control Armband—Wearable Technology by Thalmic Labs
- 2017, CMSIS-Cortex Microcontroller Software Interface Standard
- kuyumcu, 2007, FANN Tool
Cited by
Convergence of Edge Computing and Deep Learning: A Comprehensive Survey
Wang, X., Han, Y., Leung, V. C. M., Niyato, D., Yan, X., & Chen, X. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.2970550 · Cited by 1,498
Self-citationA review on TinyML: State-of-the-art and prospects
Ray, P. P. (2022) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2021.11.019 · Cited by 460
Ingolfsson, T. M., Hersche, M., Wang, X., Kobayashi, N., Cavigelli, L., & Benini, L. (2020) · 2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC) · doi:10.1109/smc42975.2020.9283028 · Cited by 353
“However, accurate networks such as the TPCT model have 7.78 M train-able parameters and require 1.73 billion multiply-accumulate (MAC) operations per inference, which is out of reach of a typical low-power MCU with few MB of Flash and few hundreds of kB of RAM [18].”
BackgroundInfluentialSelf-citationKhalil, R. A., Saeed, N., Masood, M., Fard, Y. M., Alouini, M.-S., & Al-Naffouri, T. Y. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3051414 · Cited by 318
Edge Computing on IoT for Machine Signal Processing and Fault Diagnosis: A Review
Lu, S., Lu, J., An, K., Wang, X., & He, Q. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3239944 · Cited by 295
“One of the main reasons is that edge computing is highly depended on both hardware and algorithm, and the implementation of fault diagnosis methods on the specific processors is still a difficulty and challenge [7].”
BackgroundSelf-citation
Metadata
- Authors
- Wang, X., Magno, M., Cavigelli, L., & Benini, L.
- Affiliations
- ETH Zurich (CH)
- Venue
- IEEE Internet of Things Journal
- Published
- 2020-05
- Volume
- 7
- Issue
- 5
- Pages
- 4403-4417
- ISSN
- 2327-4662 (ISSN-L)
- 2372-2541
- OpenAlex
- W3007729836
- Cited by
- 162
- References
- 59
- Citations by year
- 2026: 12 · 2025: 18 · 2024: 18 · 2023: 32 · 2022: 33 · 2021: 34 · 2020: 15
- FWCI
- 13.39
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Internet of Things Journal (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Archivio istituzionale della ricerca (Alma Mater Studiorum Università di Bologna) (submittedVersion, other-oa)
- arXiv (Cornell University)
- Funders
- Swiss Data Science Center Ph.D. Fellowship (ID P18-04)
- U.S. Office of Naval Research Global “Zero-Power Sensing for Underwater Monitoring,” (ONRG - NICOP - N62909-19-1-2018)
- Armasuisse Science and Technology
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › IoT and Edge/Fog Computing
- Physical Sciences › Computer Science › Hardware and Architecture › Parallel Computing and Optimization Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- edge computing, Internet of Things, microcontroller, ARM Cortex-M, RISC-V, multilayer perceptron, code generation, parallelization overhead, memory transfer, real-time classification
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: FANN-on-MCU: An Open-Source Toolkit for Energy-Efficient Neural Network Inference at the Edge of the Internet of Things. https://app.cochat.ai/papers/10.1109/jiot.2020.2976702
- The paper
- Wang, X., Magno, M., Cavigelli, L., & Benini, L. (2020). FANN-on-MCU: An Open-Source Toolkit for Energy-Efficient Neural Network Inference at the Edge of the Internet of Things. IEEE Internet of Things Journal. https://doi.org/10.1109/jiot.2020.2976702
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.