Resource Usage and Performance Trade-offs for Machine Learning Models in Smart Environments
Open accessThe DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
Abstract
The application of artificial intelligence enhances the ability of sensor and networking technologies to realize smart systems that sense, monitor and automatically control our everyday environments. Intelligent systems and applications often automate decisions based on the outcome of certain machine learning models. They collaborate at an ever increasing scale, ranging from smart homes and smart factories to smart cities. The best performing machine learning model, its architecture and parameters for a given task are ideally automatically determined through a hyperparameter tuning process. At the same time, edge computing is an emerging distributed computing paradigm that aims to bring computation and data storage closer to the location where they are needed to save network bandwidth or reduce the latency of requests. The challenge we address in this work is that hyperparameter tuning does not take into consideration resource trade-offs when selecting the best model for deployment in smart environments. The most accurate model might be prohibitively expensive to computationally evaluate on a resource constrained node at the edge of the network. We propose a multi-objective optimization solution to find acceptable trade-offs between model accuracy and resource consumption to enable the deployment of machine learning models in resource constrained smart environments. We demonstrate the feasibility of our approach by means of an anomaly detection use case. Additionally, we evaluate the extent that transfer learning techniques can be applied to reduce the amount of training required by reusing previous models, parameters and trade-off points from similar settings.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Edge computing: Vision and challenges
Shi, W., Cao, J., Zhang, Q., Li, Y., & Xu, L. (2016) · IEEE Internet of Things Journal · doi:10.1109/jiot.2016.2579198 · Cited by 8,733
Practical Bayesian Optimization of Machine Learning Algorithms
Snoek, J., Larochelle, H., & Adams, R. P. (2012) · arXiv · doi:10.48550/arxiv.1206.2944 · Cited by 6,691
Federated Machine Learning: Concept and Applications
Yang, Q., Liu, Y., Chen, T., & Tong, Y. (2019) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3298981 · Cited by 6,322
Toward Generating a New Intrusion Detection Dataset and Intrusion Traffic Characterization
Sharafaldin, I., Habibi Lashkari, A., & Ghorbani, A. A. (2018) · Proceedings of the 4th International Conference on Information Systems Security and Privacy · doi:10.5220/0006639801080116 · Cited by 5,318 · cited 1×
“• CICIDS 2017: CICIDS 2017 [44] is a network intrusion detection dataset—available for download at https://www.”
BackgroundMnasNet: Platform-Aware Neural Architecture Search for Mobile
Tan, M., Chen, B., Pang, R., Vasudevan, V., Sandler, M., Howard, A., & Le, Q. V. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00293 · Cited by 3,416
Sequential Model-Based Optimization for General Algorithm Configuration
Hutter, F., Hoos, H. H., & Leyton-Brown, K. (2011) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-25566-3_40 · Cited by 2,469
Deep learning for sensor-based activity recognition: A survey
Wang, J., Chen, Y., Hao, S., Peng, X., & Hu, L. (2019) · Pattern Recognition Letters · doi:10.1016/j.patrec.2018.02.010 · Cited by 2,014
Federated Optimization: Distributed Machine Learning for On-Device Intelligence
Konečný, J., McMahan, H. B., Ramage, D., & Richtárik, P. (2016) · arXiv · doi:10.48550/arxiv.1610.02527 · Cited by 1,563
Deep Learning for IoT Big Data and Streaming Analytics: A Survey
Mohammadi, M., Al-Fuqaha, A., Sorour, S., & Guizani, M. (2018) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2018.2844341 · Cited by 1,471
Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning
Hitaj, B., Ateniese, G., & Perez-Cruz, F. (2017) · Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3133956.3134012 · Cited by 1,377 · cited 1×
“However, recent research has shown that this assumption may not always hold [20,21].”
BackgroundFederated Learning for Mobile Keyboard Prediction
Hard, A., Rao, K., Mathews, R., Ramaswamy, S., Beaufays, F., Augenstein, S., Eichner, H., Kiddon, C., & Ramage, D. (2018) · arXiv · doi:10.48550/arxiv.1811.03604 · Cited by 1,148 · cited 1×
“The technique was used amongst others on smartphones [19] to enhance the next-word prediction for virtual keyboards.”
MethodPareto Multi Objective Optimization
Ngatchou, P., Zarei, A., & El-Sharkawi, A. (2006) · Proceedings of the 13th International Conference on, Intelligent Systems Application to Power Systems · doi:10.1109/isap.2005.1599245 · Cited by 664
An Empirical Evaluation of Deep Learning on Highway Driving
Huval, B., Wang, T., Tandon, S., Kiske, J., Song, W., Pazhayampallil, J., Andriluka, M., Rajpurkar, P., Migimatsu, T., Cheng-Yue, R., et al. (2015) · arXiv · doi:10.48550/arxiv.1504.01716 · Cited by 537
[DL] A Survey of FPGA-based Neural Network Inference Accelerators
Guo, K., Zeng, S., Yu, J., Wang, Y., & Yang, H. (2019) · ACM Transactions on Reconfigurable Technology and Systems · doi:10.1145/3289185 · Cited by 292 · cited 1×
“…hardware solutions—such as neuromorphic computing chips [51] that emulate the electrical behavior of neurons in the brain, or FPGA-based accelerators [52]—will enable the evaluation of neural networks at a very low energy cost and can turn into a tipping-point for deep learning at the edge.”
BackgroundDeepCare: A Deep Dynamic Memory Model for Predictive Medicine
Pham, T., Tran, T., Phung, D., & Venkatesh, S. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-31750-2_3 · Cited by 269
Cited by
A survey on deep transfer learning to edge computing for mitigating the COVID-19 pandemic
Sufian, A., Ghosh, A., Sadiq, A. S., & Smarandache, F. (2020) · Journal of Systems Architecture · doi:10.1016/j.sysarc.2020.101830 · Cited by 156
“al proposed a resource and performance trade-off strategy for a smart environment [87].”
BackgroundIoT Anomaly Detection to Strengthen Cybersecurity in the Critical Infrastructure of Smart Cities
Villegas-Ch, W., Govea, J., & Jaramillo-Alcazar, A. (2023) · Applied Sciences · doi:10.3390/app131910977 · Cited by 48
Lessons and tips for designing a machine learning study using EHR data
Arbet, J., Brokamp, C., Meinzen-Derr, J., Trinkley, K. E., & Spratt, H. M. (2021) · Journal of Clinical and Translational Science · doi:10.1017/cts.2020.513 · Cited by 46
“Hyperparameters are parameters that cannot be automatically learned from themodel building process, and rather, the user needs to try a grid of values and use cross-validation or some other method to find the optimized value [41].”
Metadata
- Authors
- Preuveneers, D., Tsingenopoulos, I., & Joosen, W.
- Venue
- Sensors
- Publisher
- MDPI AG
- Published
- 2020-02-20
- Volume
- 20
- Issue
- 4
- Pages
- 1176
- ISSN
- 1424-8220 (ISSN-L)
- OpenAlex
- W3007322587
- Cited by
- 58
- References
- 15
- FWCI
- 3.30
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.