Deep Learning on Mobile and Embedded Devices: State-of-the-art, Challenges, and Future Directions
Abstract
Recent years have witnessed an exponential increase in the use of mobile and embedded devices. With the great success of deep learning in many fields, there is an emerging trend to deploy deep learning on mobile and embedded devices to better meet the requirement of real-time applications and user privacy protection. However, the limited resources of mobile and embedded devices make it challenging to fulfill the intensive computation and storage demand of deep learning models. In this survey, we conduct a comprehensive review on the related issues for deep learning on mobile and embedded devices. We start with a brief introduction of deep learning and discuss major challenges of implementing deep learning models on mobile and embedded devices. We then conduct an in-depth survey on important compression and acceleration techniques that help adapt deep learning models to mobile and embedded devices, which we specifically classify as pruning, quantization, model distillation, network design strategies, and low-rank factorization. We elaborate on the hardware-based solutions, including mobile GPU, FPGA, and ASIC, and describe software frameworks for mobile deep learning models, especially the development of frameworks based on OpenCL and RenderScript. After that, we present the application of mobile deep learning in a variety of areas, such as navigation, health, speech recognition, and information security. Finally, we discuss some future directions for deep learning on mobile and embedded devices to inspire further research in this area.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Inference of DL in edge devices is not yet prepared enough to be fully executed.
“However, inference of DL in edge devices is not yet prepared enough to be fully executed [3,4,6].”
From TinyML: Enabling of Inference Deep Learning Models on Ultra-Low-Power IoT Edge Devices for AI Applications (2022) · cites this paper as [4]
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design
Ma, N., Zhang, X., Zheng, H.-T., & Sun, J. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01264-9_8 · Cited by 7,356 · cited 1× · see citing works
“[117] argued that the effect of methods should be evaluated by direct metrics or targeted platforms, not by indirect metrics (i.”
BackgroundRecurrent neural network based language model
Mikolov, T., Karafiát, M., Burget, L., Černocký, J., & Khudanpur, S. (2010) · Interspeech 2010 · doi:10.21437/interspeech.2010-343 · Cited by 5,390 · see citing works
Membership Inference Attacks Against Machine Learning Models
Shokri, R., Stronati, M., Song, C., & Shmatikov, V. (2017) · 2017 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2017.41 · Cited by 4,754 · see citing works
Efficient Processing of Deep Neural Networks: A Tutorial and Survey
Sze, V., Chen, Y.-H., Yang, T.-J., & Emer, J. S. (2017) · Proceedings of the IEEE · doi:10.1109/jproc.2017.2761740 · Cited by 4,215 · cited 1× · see citing works
“Reference [168] mainly focused on hardware solutions, such as hardware platforms, circuits design, and the use of new memory technologies.”
BackgroundSelf-citationChannel Pruning for Accelerating Very Deep Neural Networks
He, Y., Zhang, X., & Sun, J. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.155 · Cited by 2,647 · see citing works
Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning
Hitaj, B., Ateniese, G., & Perez-Cruz, F. (2017) · Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3133956.3134012 · Cited by 1,377 · see citing works
Cheng, Y., Wang, D., Zhou, P., & Zhang, T. (2018) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2765695 · Cited by 817 · see citing works
Designing Energy-Efficient Convolutional Neural Networks Using Energy-Aware Pruning
Yang, T.-J., Chen, Y.-H., & Sze, V. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.643 · Cited by 732 · see citing works
Self-citationDeepMon: Mobile GPU-based Deep Learning Framework for Continuous Vision Applications
Huynh, L. N., Lee, Y., & Balan, R. K. (2017) · Proceedings of the 15th Annual International Conference on Mobile Systems, Applications, and Services · doi:10.1145/3081333.3081360 · Cited by 329 · see citing works
Yang, Z., Moczulski, M., Denil, M., De Freitas, N., Song, L., & Wang, Z. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.173 · Cited by 215 · see citing works
Accelerating Deep Convolutional Networks using low-precision and sparsity
Venkatesh, G., Nurvitadhi, E., & Marr, D. (2017) · 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2017.7952679 · Cited by 123 · see citing works
Characterizing the deployment of deep neural networks on commercial edge devices
Hadidi, R., Cao, J., Xie, Y., Asgari, B., Krishna, T., & Kim, H. (2019) · 2019 IEEE International Symposium on Workload Characterization (IISWC) · doi:10.1109/iiswc47752.2019.9041955 · Cited by 118 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Emotion recognition in EEG signals using deep learning methods: A review
Jafari, M., Shoeibi, A., Khodatars, M., Bagherzadeh, S., Shalbaf, A., García, D. L., Gorriz, J. M., & Acharya, U. R. (2023) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2023.107450 · Cited by 240 · see citing works
Alajlan, N. N., & Ibrahim, D. M. (2022) · Micromachines · doi:10.3390/mi13060851 · Cited by 224 · see citing works
Efficient Gesture-Based Recognition of Tifinagh Characters in Air Handwriting with a TinyDL Model
Lamaakal, I., El Mourabit, N., El Makkaoui, K., Ouahbi, I., & Maleh, Y. (2024) · 2024 Sixth International Conference on Intelligent Computing in Data Sciences (ICDS) · doi:10.1109/icds62089.2024.10756483 · Cited by 14 · see citing works
- Authors
- Chen, Y., Zheng, B., Zhang, Z., Wang, Q., Shen, C., & Zhang, Q.
- Affiliations
- Wuhan University (CN)
- Xi'an Jiaotong University (CN)
- Hong Kong University of Science and Technology (HK)
- Venue
- ACM Computing Surveys
- Published
- 2020-08-20
- Volume
- 53
- Issue
- 4
- Pages
- 1-37
- ISSN
- 0360-0300 (ISSN-L)
- 1557-7341
- OpenAlex
- W3080404596
- Cited by
- 159
- References
- 13
- Citations by year
- 2026: 21 · 2025: 41 · 2024: 34 · 2023: 24 · 2022: 16 · 2021: 19 · 2020: 3 · 2017: 1
- FWCI
- 6.03
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- ACM Computing Surveys (publishedVersion)
- Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
- Funders
- National Natural Science Foundation of China (61972296 and 61702380, 61822207 and U1636219)
- Equipment Pre-research Joint Fund (6141A02033327)
- Outstanding Youth Foundation of Hubei Province (2017CFA047)
- Wuhan Advanced Application Project (2019010701011419)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Networks and Communications › IoT and Edge/Fog Computing
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- mobile deep learning, embedded deep learning, model compression, model acceleration, neural network pruning, quantization, knowledge distillation, low-rank factorization, mobile GPUs, FPGA, ASIC, OpenCL, real-time applications, user privacy
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
