Tiny Machine Learning: Progress and Futures [Feature]
Open accessThe DOI matches its Crossref record; 61 of the 61 references it cites were found in a registry or an index.
Abstract
Tiny machine learning (TinyML) is a new frontier of machine learning. By squeezing deep learning models into billions of IoT devices and microcontrollers (MCUs), we expand the scope of applications and enable ubiquitous intelligence. However, TinyML is challenging due to the hardware constraints: the tiny memory resource is difficult hold deep learning models designed for cloud and mobile platforms. There is also limited compiler and inference engine support for bare-metal devices. Therefore, we need to co-design the algorithm and system stack to enable TinyML. In this review, we will first discuss the definition, challenges, and applications of TinyML. We then survey the recent progress in TinyML and deep learning on MCUs. Next, we will introduce MCUNet, showing how we can achieve ImageNet-scale AI applications on IoT devices with system-algorithm co-design. We will further extend the solution from inference to training and introduce tiny on-device training techniques. Finally, we present future directions in this area. Today’s “large” model might be tomorrow’s “tiny” model. The scope of TinyML should evolve and adapt over time.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
PyTorch: An Imperative Style, High-Performance Deep Learning Library
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al. (2019) · arXiv · doi:10.48550/arxiv.1912.01703 · Cited by 31,473
MobileNetV2: Inverted Residuals and Linear Bottlenecks
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L.-C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00474 · Cited by 28,755
The pascal visual object classes (VOC) challenge
Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2010) · International Journal of Computer Vision · doi:10.1007/s11263-009-0275-4 · Cited by 19,599
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., & Adam, H. (2017) · arXiv · doi:10.48550/arxiv.1704.04861 · Cited by 19,015
Self-citationYOLOv3: An Incremental Improvement
Redmon, J., & Farhadi, A. (2018) · arXiv · doi:10.48550/arxiv.1804.02767 · Cited by 17,141
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
Tan, M., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1905.11946 · Cited by 17,072
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204
ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
Zhang, X., Zhou, X., Lin, M., & Sun, J. (2017) · arXiv · doi:10.48550/arxiv.1707.01083 · Cited by 9,982
ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
Zhang, X., Zhou, X., Lin, M., & Sun, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00716 · Cited by 9,982
Howard, A., Sandler, M., Chen, B., Wang, W., Chen, L.-C., Tan, M., Chu, G., Vasudevan, V., Zhu, Y., Pang, R., et al. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00140 · Cited by 9,708
Self-citationShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design
Ma, N., Zhang, X., Zheng, H.-T., & Sun, J. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01264-9_8 · Cited by 7,356 · cited 3×
“There has been a lot of effort optimizing deep learning models for mobile platforms like MobileNets [5, 4] and ShuffleNet [6].”
XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks
Rastegari, M., Ordonez, V., Redmon, J., & Farhadi, A. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_32 · Cited by 4,408
Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference
Jacob, B., Kligys, S., Chen, B., Zhu, M., Tang, M., Howard, A., Adam, H., & Kalenichenko, D. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00286 · Cited by 4,114
DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
Donahue, J., Jia, Y., Vinyals, O., Hoffman, J., Zhang, N., Tzeng, E., & Darrell, T. (2013) · arXiv · doi:10.48550/arxiv.1310.1531 · Cited by 4,019 · cited 1×
“Then, only the last layer needs to be fine-tuned on a smaller, task-specific dataset [119, 120, 121, 122].”
Neural Architecture Search with Reinforcement Learning
Zoph, B., & Le, Q. V. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.01578 · Cited by 3,974
MnasNet: Platform-Aware Neural Architecture Search for Mobile
Tan, M., Chen, B., Pang, R., Vasudevan, V., Sandler, M., Howard, A., & Le, Q. V. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00293 · Cited by 3,416
DARTS: Differentiable architecture search
Liu, H., Simonyan, K., & Yang, Y. (2018) · arXiv · doi:10.48550/arxiv.1806.09055 · Cited by 3,248
Learning Efficient Convolutional Networks through Network Slimming
Liu, Z., Li, J., Shen, Z., Huang, G., Yan, S., & Zhang, C. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.298 · Cited by 2,761
Channel Pruning for Accelerating Very Deep Neural Networks
He, Y., Zhang, X., & Sun, J. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.155 · Cited by 2,647 · cited 2×
“Compressing off-the-shelf networks by removing redundancy and reducing complexity through pruning [57, 58, 59, 60, 61, 62] and quantization [63, 64, 65, 66, 67, 68, 69, 70] are two popular methods to improve network efficiency.”
PaLM: Scaling Language Modeling with Pathways
Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.02311 · Cited by 2,590
Designing Network Design Spaces
Radosavovic, I., Kosaraju, R. P., Girshick, R., He, K., & Dollar, P. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01044 · Cited by 2,084
Towards Total Recall in Industrial Anomaly Detection
Roth, K., Pemula, L., Zepeda, J., Scholkopf, B., Brox, T., & Gehler, P. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01392 · Cited by 1,810 · cited 1×
“TinyML is also capable of predicting and recognizing sign languages [28]. with the capability to perform anomaly detection to reduce human efforts [32, 33, 34].”
DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
Zhou, S., Wu, Y., Ni, Z., Zhou, X., Wen, H., & Zou, Y. (2016) · arXiv · doi:10.48550/arxiv.1606.06160 · Cited by 1,690
AMC: AutoML for Model Compression and Acceleration on Mobile Devices
He, Y., Lin, J., Liu, Z., Wang, H., Li, L.-J., & Han, S. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_48 · Cited by 1,400
Self-citationDo Better ImageNet Models Transfer Better?
Kornblith, S., Shlens, J., & Le, Q. V. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00277 · Cited by 1,325
TVM: an automated end-to-end optimizing compiler for deep learning
(2018) · arXiv (Cornell University) · doi:10.5555/3291168.3291211 · Cited by 1,170
Big Transfer (BiT): General Visual Representation Learning
Kolesnikov, A., Beyer, L., Zhai, X., Puigcerver, J., Yung, J., Gelly, S., & Houlsby, N. (2019) · arXiv · doi:10.48550/arxiv.1912.11370 · Cited by 1,039
Big Transfer (BiT): General Visual Representation Learning
Kolesnikov, A., Beyer, L., Zhai, X., Puigcerver, J., Yung, J., Gelly, S., & Houlsby, N. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58558-7_29 · Cited by 1,039 · cited 1×
“In transfer learning, a neural network is first pre-trained on a large-scale dataset, such as ImageNet [115], and used as a feature extractor [116, 117, 118].”
An Analysis of Deep Neural Network Models for Practical Applications
Canziani, A., Paszke, A., & Culurciello, E. (2016) · arXiv · doi:10.48550/arxiv.1605.07678 · Cited by 969 · cited 1×
“The intuition is that, within the same model family, the accuracy is usually positively related to the computation [128, 61].”
Once-for-All: Train One Network and Specialize it for Efficient Deployment
Cai, H., Gan, C., Wang, T., Zhang, Z., & Han, S. (2019) · arXiv · doi:10.48550/arxiv.1908.09791 · Cited by 845 · cited 2×
“Specifically, it follows Once-For-All (OFA) NAS [129] to perform network specialization (Figure 6).”
Self-citationPACT: Parameterized Clipping Activation for Quantized Neural Networks
Choi, J., Wang, Z., Venkataramani, S., Chuang, P. I.-J., Srinivasan, V., & Gopalakrishnan, K. (2018) · arXiv · doi:10.48550/arxiv.1805.06085 · Cited by 703
Alwani, M., Chen, H., Ferdman, M., & Milder, P. (2016) · 2016 49th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1109/micro.2016.7783725 · Cited by 662 · cited 2×
“With a similar goal of reducing peak memory, other researchers further propose computing partial spatial regions across multiple layers [103, 104, 105].”
TensorFlow Lite Micro: Embedded Machine Learning on TinyML Systems
David, R., Duke, J., Jain, A., Reddi, V. J., Jeffries, N., Li, J., Kreeger, N., Nappier, I., Natraj, M., Regev, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.08678 · Cited by 470 · cited 2×
“…., latency, peak memory, and flash usage) of four representative related studies ( i.e ., CMSIS-NN [41], X-Cube-AI [42], TinyEngine [8], and TF-Lite Micro [47]) on an identical MCU (STM32H743) and identical datasets (VWW and Imagenet), in order to provide a more accurate and transparent comparison.”
Test-Time Training with Self-Supervision for Generalization under Distribution Shifts
Sun, Y., Wang, X., Liu, Z., Miller, J., Efros, A. A., & Hardt, M. (2019) · arXiv · doi:10.48550/arxiv.1909.13231 · Cited by 393
LLM.int8(): 8-Bit Matrix Multiplication for Transformers at Scale
Dettmers, T., Lewis, M., Belkada, Y., & Zettlemoyer, L. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-2198 · Cited by 305
LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale
Dettmers, T., Lewis, M., Belkada, Y., & Zettlemoyer, L. (2022) · arXiv · doi:10.48550/arxiv.2208.07339 · Cited by 305
SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models
Xiao, G., Lin, J., Seznec, M., Wu, H., Demouth, J., & Han, S. (2022) · arXiv · doi:10.48550/arxiv.2211.10438 · Cited by 247 · cited 1×
“For example, quantization techniques have been effective in both TinyML [8, 9] and LargeML settings [152, 153], and the concept of sparse learning has been used in both scenarios to run models efficiently with limited resources [154, 56].”
Self-citationTinyOL: TinyML with Online-Learning on Microcontrollers
Ren, H., Anicic, D., & Runkler, T. A. (2021) · 2021 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9533927 · Cited by 185
Banbury, C., Zhou, C., Fedorov, I., Navarro, R. M., Thakker, U., Gope, D., Reddi, V. J., Mattina, M., & Whatmough, P. N. (2020) · arXiv · doi:10.48550/arxiv.2010.11267 · Cited by 183
Detailed comparison of communication efficiency of split learning and federated learning
Singh, A., Vepakomma, P., Gupta, O., & Raskar, R. (2019) · arXiv · doi:10.48550/arxiv.1909.09145 · Cited by 148
CMix-NN: Mixed Low-Precision CNN Library for Memory-Constrained Edge Devices
Capotondi, A., Rusci, M., Fariselli, M., & Benini, L. (2020) · IEEE Transactions on Circuits and Systems II: Express Briefs · doi:10.1109/tcsii.2020.2983648 · Cited by 135
Improved Knowledge Distillation via Teacher Assistant
Mirzadeh, S. I., Farajtabar, M., Li, A., Levine, N., Matsukawa, A., & Ghasemzadeh, H. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.5963 · Cited by 101 · cited 1×
“In order to enhance network efficiency, knowledge distillation is also a method to transfer information learned from one teacher model to another student model [74, 75, 76, 77, 78, 79, 80, 81].”
A Review of Machine Learning and TinyML in Healthcare
Tsoukas, V., Boumpa, E., Giannakas, G., & Kakarountas, A. (2021) · 25th Pan-Hellenic Conference on Informatics · doi:10.1145/3503823.3503836 · Cited by 79
A Tiny CNN Architecture for Medical Face Mask Detection for Resource-Constrained Endpoints
Mohan, P., Paul, A. J., & Chirania, A. (2021) · Lecture Notes in Electrical Engineering · doi:10.1007/978-981-16-0749-3_52 · Cited by 70
Robustifying the Deployment of tinyML Models for Autonomous mini-vehicles
de Prado, M., Rusci, M., Capotondi, A., Donze, R., Benini, L., & Pazos, N. (2021) · Sensors · doi:10.3390/s21041339 · Cited by 57
TinyML Smart Sensor for Energy Saving in Internet of Things Precision Agriculture platform
Nicolas, C., Naila, B., & Amar, R.-C. (2022) · 2022 Thirteenth International Conference on Ubiquitous and Future Networks (ICUFN) · doi:10.1109/icufn55119.2022.9829675 · Cited by 53
A Battery-Free Long-Range Wireless Smart Camera for Face Detection
Giordano, M., Mayer, P., & Magno, M. (2020) · Proceedings of the 8th International Workshop on Energy Harvesting and Energy-Neutral Sensing Systems · doi:10.1145/3417308.3430273 · Cited by 38 · cited 1×
“• Smart home: TinyML can enable object detection, image recognition, and face detection on IoT devices to build smart environments, such as smart homes and hospitals [19, 20, 21, 22, 23].”
Capacitive Sensing Based On-board Hand Gesture Recognition with TinyML
Bian, S., & Lukowicz, P. (2021) · Adjunct Proceedings of the 2021 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2021 ACM International Symposium on Wearable Computers · doi:10.1145/3460418.3479287 · Cited by 37 · cited 1×
“• Human-machine interface: TinyML can enable human-machine interface applications, like hand gesture recognition [24, 25, 26, 27].”
SmallSats: a new technological frontier in ecology and conservation?
Curnick, D. J., Davies, A. J., Duncan, C., Freeman, R., Jacoby, D. M. P., Shelley, H. T. E., Rossi, C., Wearn, O. R., Williamson, M. J., & Pettorelli, N. (2022) · Remote Sensing in Ecology and Conservation · doi:10.1002/rse2.239 · Cited by 34 · cited 1×
“• Ecology and agriculture: TinyML can also help with ecological, agricultural, environmental, and phenomics applications so as to conserve endangered species or forecast weather activities [35, 36, 37, 38, 39, 40].”
Paul, A. J., Mohan, P., & Sehgal, S. (2020) · 2020 IEEE Recent Advances in Intelligent Computational Systems (RAICS) · doi:10.1109/raics51191.2020.9332480 · Cited by 32
Nakhle, F., & Harfouche, A. L. (2021) · Patterns · doi:10.1016/j.patter.2021.100323 · Cited by 28 · cited 1×
“• Ecology and agriculture: TinyML can also help with ecological, agricultural, environmental, and phenomics applications so as to conserve endangered species or forecast weather activities [35, 36, 37, 38, 39, 40].”
TinySpeech: Attention Condensers for Deep Speech Recognition Neural Networks on Edge Devices
Wong, A., Famouri, M., Pavlova, M., & Surana, S. (2020) · arXiv · doi:10.48550/arxiv.2008.04245 · Cited by 27 · cited 1×
“However, these frameworks typically rely on a host language ( e.g ., Python) and various runtime systems, which adds significant overhead and makes them incompatible with tiny edge devices.”
BackgroundAdaptive Traffic Control With TinyML
Roshan, A. N., Gokulapriyan, B., Siddarth, C., & Kokil, P. (2021) · 2021 Sixth International Conference on Wireless Communications, Signal Processing and Networking (WiSPNET) · doi:10.1109/wispnet51692.2021.9419472 · Cited by 27 · cited 1×
“• Ecology and agriculture: TinyML can also help with ecological, agricultural, environmental, and phenomics applications so as to conserve endangered species or forecast weather activities [35, 36, 37, 38, 39, 40].”
AttendNets: Tiny Deep Image Recognition Neural Networks for the Edge via Visual Attention Condensers
Wong, A., Famouri, M., & Shafiee, M. J. (2020) · arXiv · doi:10.48550/arxiv.2009.14385 · Cited by 20 · cited 1×
“• Smart home: TinyML can enable object detection, image recognition, and face detection on IoT devices to build smart environments, such as smart homes and hospitals [19, 20, 21, 22, 23].”
Memory-Efficient, Limb Position-Aware Hand Gesture Recognition using Hyperdimensional Computing
Zhou, A., Muller, R., & Rabaey, J. (2021) · arXiv · doi:10.48550/arxiv.2103.05267 · Cited by 17
Enabling Large Neural Networks on Tiny Microcontrollers with Swapping
Miao, H., & Lin, F. X. (2021) · arXiv · doi:10.48550/arxiv.2101.08744 · Cited by 12
Hardy, E., & Badets, F. (2021) · arXiv · doi:10.48550/arxiv.2103.04792 · Cited by 6
Cough Activated Dynamic Face Visor
Luukkonen, T., Colley, A., Seppänen, T., & Häkkilä, J. (2021) · Augmented Humans Conference 2021 · doi:10.1145/3458709.3459000 · Cited by 5 · cited 1×
“• Smart home: TinyML can enable object detection, image recognition, and face detection on IoT devices to build smart environments, such as smart homes and hospitals [19, 20, 21, 22, 23].”
Cited by
Lamaakal, I., Ouahbi, I., El Makkaoui, K., Maleh, Y., Pławiak, P., & Alblehai, F. (2024) · IEEE Access · doi:10.1109/access.2024.3406631 · Cited by 34
“The emergence of TinyML [1], [2], [3], [4], [55], [56], [58], [60] represents a significant breakthrough in technology, blending the ever-growing needs of data-driven applications with the constraints of low-power, embedded devices.”
BackgroundEfficient Gesture-Based Recognition of Tifinagh Characters in Air Handwriting with a TinyDL Model
Lamaakal, I., El Mourabit, N., El Makkaoui, K., Ouahbi, I., & Maleh, Y. (2024) · 2024 Sixth International Conference on Intelligent Computing in Data Sciences (ICDS) · doi:10.1109/icds62089.2024.10756483 · Cited by 14
Indongo, N. N., Smith, R. C., Winschiers-Theophilus, H., Kandjengo, S., & Mbinge, U. (2026) · Proceedings of the 19th Participatory Design Conference 2026, Vol. 1: Full Papers · doi:10.1145/3796624.3796667 · Cited by 1
“To automate the chatbots’ functionalities, a lightweight large language model, Tiny Machine Learning (TinyML) [27], was integrated, ensuring responsive and efficient interactions.”
Metadata
- Authors
- Lin, J., Zhu, L., Chen, W.-M., Wang, W.-C., & Han, S.
- Venue
- IEEE Circuits and Systems Magazine
- Published
- 2023
- Volume
- 23
- Issue
- 3
- Pages
- 8-34
- ISSN
- 0163-6812 (ISSN-L)
- 1531-636X
- 1558-0830
- OpenAlex
- W4387609230
- Cited by
- 186
- References
- 61
- FWCI
- 16.91
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.