AutoScale: Energy Efficiency Optimization for Stochastic Edge Inference Using Reinforcement Learning
The DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index.
Abstract
Deep learning inference is increasingly run at the edge. As the programming and system stack support becomes mature, it enables acceleration opportunities in a mobile system, where the system performance envelope is scaled up with a plethora of programmable co-processors. Thus, intelligent services designed for mobile users can choose between running inference on the CPU or any of the co-processors in the mobile system, and exploiting connected systems such as the cloud or a nearby, locally connected mobile system. By doing so, these services can scale out the performance and increase the energy efficiency of edge mobile systems. This gives rise to a new challenge—deciding when inference should run where. Such execution scaling decision becomes more complicated with the stochastic nature of mobile-cloud execution environment, where signal strength variation in the wireless networks and resource interference can affect real-time inference performance and system energy efficiency. To enable energy efficient deep learning inference at the edge, this paper proposes AutoScale, an adaptive and lightweight execution scaling engine built on the custom-designed reinforcement learning algorithm. It continuously learns and selects the most energy efficient inference execution target by considering characteristics of neural networks and available systems in the collaborative cloud-edge execution environment while adapting to stochastic runtime variance. Real system implementation and evaluation, considering realistic execution scenarios, demonstrate an average of 9.8x and 1.6x energy efficiency improvement over the baseline mobile CPU and cloud offloading, respectively, while meeting the real-time performance and accuracy requirements.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Howard, A., Sandler, M., Chen, B., Wang, W., Chen, L.-C., Tan, M., Chu, G., Vasudevan, V., Zhu, Y., Pang, R., et al. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00140 · Cited by 9,708
Learning Transferable Architectures for Scalable Image Recognition
Zoph, B., Vasudevan, V., Shlens, J., & Le, Q. V. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00907 · Cited by 7,831
Large-Scale Video Classification with Convolutional Neural Networks
Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., & Fei-Fei, L. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.223 · Cited by 6,354 · cited 1×
“DNNs connect numerous functional layers to extract features from inputs at multiple levels of abstraction [54], [66].”
BackgroundQuantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference
Jacob, B., Kligys, S., Chen, B., Zhu, M., Tang, M., Howard, A., Adam, H., & Kalenichenko, D. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00286 · Cited by 4,114
Lee, H., Grosse, R., Ranganath, R., & Ng, A. Y. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553453 · Cited by 2,412
MAUI: making smartphones last longer with code offload
Cuervo, E., Balasubramanian, A., Cho, D.-K., Wolman, A., Saroiu, S., Chandra, R., & Bahl, P. (2010) · Proceedings of the 8th international conference on Mobile systems, applications, and services · doi:10.1145/1814433.1814441 · Cited by 2,281 · cited 1×
“Various offloading techniques that execute all or some computations on remote servers while considering the variability of the mobile environment have been also proposed [1], [2], [16], [48], [56], [61], [87], [91], [113], [117].”
BackgroundTVM: an automated end-to-end optimizing compiler for deep learning
(2018) · arXiv (Cornell University) · doi:10.5555/3291168.3291211 · Cited by 1,170
Benchmark analysis of representative deep neural network architectures
Bianco, S., Cadene, R., Celona, L., & Napoletano, P. (2018) · IEEE Access · doi:10.1109/access.2018.2877890 · Cited by 838
MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices
Sun, Z., Yu, H., Song, X., Liu, R., Yang, Y., & Zhou, D. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.195 · Cited by 757
Neurosurgeon: Collaborative Intelligence Between the Cloud and Mobile Edge
Kang, Y., Hauswald, J., Gao, C., Rovinski, A., Mudge, T., Mars, J., & Tang, L. (2017) · Proceedings of the Twenty-Second International Conference on Architectural Support for Programming Languages and Operating Systems · doi:10.1145/3037697.3037698 · Cited by 673
DeepX: A Software Accelerator for Low-Power Deep Learning Inference on Mobile Devices
Lane, N. D., Bhattacharya, S., Georgiev, P., Forlivesi, C., Jiao, L., Qendro, L., & Kawsar, F. (2016) · 2016 15th ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN) · doi:10.1109/ipsn.2016.7460664 · Cited by 450
AI Benchmark: Running Deep Neural Networks on Android Smartphones
Ignatov, A., Timofte, R., Chou, W., Wang, K., Wu, M., Hartley, T., & Van Gool, L. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-11021-5_19 · Cited by 347
DeepMon: Mobile GPU-based Deep Learning Framework for Continuous Vision Applications
Huynh, L. N., Lee, Y., & Balan, R. K. (2017) · Proceedings of the 15th Annual International Conference on Mobile Systems, Applications, and Services · doi:10.1145/3081333.3081360 · Cited by 329 · cited 2×
“As mobile systems become powerful [31], [41], [50], [107], there has been increasing push to execute DNN inference at the edge [10], [25], [42], [53], [55], [65], [106], [107], [111], [121].”
BackgroundImproving Neural Network Quantization without Retraining using Outlier Channel Splitting
Zhao, R., Hu, Y., Dotzel, J., De Sa, C., & Zhang, Z. (2019) · arXiv · doi:10.48550/arxiv.1901.09504 · Cited by 231
MLPerf: An Industry Standard Benchmark Suite for Machine Learning Performance
Mattson, P., Reddi, V. J., Cheng, C., Coleman, C., Diamos, G., Kanter, D., Micikevicius, P., Patterson, D., Schmuelling, G., Tang, H., et al. (2020) · IEEE Micro · doi:10.1109/mm.2020.2974843 · Cited by 171 · cited 1×
“Varying co-running apps [76], [93], [108].”
Self-citationLi, G., Liu, L., Wang, X., Dong, X., Zhao, P., & Feng, X. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01418-6_40 · Cited by 80
Cited by
Sustainable AI: Environmental Implications, Challenges and Opportunities
Wu, C.-J., Raghavendra, R., Gupta, U., Acun, B., Ardalani, N., Maeng, K., Chang, G., Behram, F. A., Huang, J., Bai, C., et al. (2021) · arXiv · doi:10.48550/arxiv.2111.00364 · Cited by 405
“Optimizing the overall energy efficiency of FL and on-device AI is an important first step [89], [90], [91], [92], [93].”
BackgroundSelf-citationA systematic review of Green AI
Verdecchia, R., Sallou, J., & Cruz, L. (2023) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1507 · Cited by 375
DynamoLLM: Designing LLM Inference Clusters for Performance and Energy Efficiency
Stojkovic, J., Zhang, C., Goiri, Í., Torrellas, J., & Choukse, E. (2025) · 2025 IEEE International Symposium on High Performance Computer Architecture (HPCA) · doi:10.1109/hpca61900.2025.00102 · Cited by 112
“Unlike existing energy frameworks for GPUs, we need to use all available parameters simultaneously, rather than relying solely on GPU frequency [42], [79] or scaling in/out [23].”
Metadata
- Authors
- Kim, Y. G., & Wu, C.-J.
- Venue
- 2020 53rd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO)
- Publisher
- IEEE
- Published
- 2020-10
- Pages
- 1082-1096
- ISSN
- 1072-4451 (ISSN-L)
- OpenAlex
- W3105381414
- Cited by
- 75
- References
- 18
- FWCI
- 8.98
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.