Hardware-aware machine learning: modeling and optimization
The DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.
Abstract
Recent breakthroughs in Machine Learning (ML) applications, and especially in Deep Learning (DL), have made DL models a key component in almost every modern computing system. The increased popularity of DL applications deployed on a wide-spectrum of platforms (from mobile devices to datacenters) have resulted in a plethora of design challenges related to the constraints introduced by the hardware itself. “What is the latency or energy cost for an inference made by a Deep Neural Network (DNN)?” “Is it possible to predict this latency or energy consumption before a model is even trained?” “If yes, how can machine learners take advantage of these models to design the hardware-optimal DNN for deployment?” From lengthening battery life of mobile devices to reducing the runtime requirements of DL models executing in the cloud, the answers to these questions have drawn significant attention. One cannot optimize what isn't properly modeled. Therefore, it is important to understand the hardware efficiency of DL models during serving for making an inference, before even training the model. This key observation has motivated the use of predictive models to capture the hardware performance or energy efficiency of ML applications. Furthermore, ML practitioners are currently challenged with the task of designing the DNN model, i.e., of tuning the hyper-parameters of the DNN architecture, while optimizing for both accuracy of the DL model and its hardware efficiency. Therefore, state-of-the-art methodologies have proposed hardware-aware hyper-parameter optimization techniques. In this paper, we provide a comprehensive assessment of state-of-the-art work and selected results on the hardware-aware modeling and optimization for ML applications. We also highlight several open questions that are poised to give rise to novel hardware-aware designs in the next few years, as DL applications continue to significantly impact associated hardware systems and platforms.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference
Jacob, B., Kligys, S., Chen, B., Zhu, M., Tang, M., Howard, A., Adam, H., & Kalenichenko, D. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00286 · Cited by 4,114
MnasNet: Platform-Aware Neural Architecture Search for Mobile
Tan, M., Chen, B., Pang, R., Vasudevan, V., Sandler, M., Howard, A., & Le, Q. V. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00293 · Cited by 3,416 · cited 6×
“For instance, state-of-the-art hardware-aware methodologies from Google Brain consider DL optimization and design techniques that optimize for both the accuracy of DNNs for image classification and the runtime of the model on Google Pixel smartphones [37].”
BackgroundMethodInfluentialEyeriss: An Energy-Efficient Reconfigurable Accelerator for Deep Convolutional Neural Networks
Chen, Y.-H., Krishna, T., Emer, J. S., & Sze, V. (2017) · IEEE Journal of Solid-State Circuits · doi:10.1109/jssc.2016.2616357 · Cited by 3,247 · cited 4×
“As opposed to Eyeriss, Paleo has been developed with general purpose platforms (GPUs) in mind.”
BackgroundMethodInfluentialCourbariaux, M., Hubara, I., Soudry, D., El-Yaniv, R., & Bengio, Y. (2016) · arXiv · doi:10.48550/arxiv.1602.02830 · Cited by 2,106
Learning both Weights and Connections for Efficient Neural Networks
Han, S., Pool, J., Tran, J., & Dally, W. J. (2015) · arXiv · doi:10.48550/arxiv.1506.02626 · Cited by 1,481 · cited 1×
“Pruning: Prior art has investigated pruning-based methods that aim at reducing the ML model complexity by gradually removing weights of the model and by retraining it [8, 19, 38].”
MethodTETRIS: Scalable and Efficient Neural Network Acceleration with 3D Memory
Gao, M., Pu, J., Yang, X., Horowitz, M., & Kozyrakis, C. (2017) · Proceedings of the Twenty-Second International Conference on Architectural Support for Programming Languages and Operating Systems · doi:10.1145/3037697.3037702 · Cited by 507
Minerva: enabling low-power, highly-accurate deep neural network accelerators
Reagen, B., Whatmough, P., Adolf, R., Rama, S., Lee, H., Lee, S. K., Hernández-Lobato, J. M., Wei, G.-Y., & Brooks, D. (2016) · ACM SIGARCH Computer Architecture News · doi:10.1145/3007787.3001165 · Cited by 446 · cited 5×
“In this paper, we investigate the main tools for employing hardware-aware hyper-parameter optimization, such as methodologies based on hardware-aware Bayesian optimization [34, 35], multi-level co-optimization [30] and Neural”
MethodInfluentialNeST: A Neural Network Synthesis Tool Based on a Grow-and-Prune Paradigm
Dai, X., Yin, H., & Jha, N. K. (2019) · IEEE Transactions on Computers · doi:10.1109/tc.2019.2914438 · Cited by 233
Cited by
Challenges in deploying machine learning: A survey of case studies
Paleyes, A., Urma, R.-G., & Lawrence, N. D. (2023) · ACM Computing Surveys · doi:10.1145/3533378 · Cited by 638
“[43] in the context of hardware-aware ML.”
ChamNet: Towards Efficient Network Design Through Platform-Aware Model Adaptation
Dai, X., Zhang, P., Wu, B., Yin, H., Sun, F., Wang, Y., Dukhan, M., Hu, Y., Wu, Y., Jia, Y., et al. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01166 · Cited by 279
“NAS and SMBO: Platform-aware NAS and SMBO have emerged as a promising direction for automating the synthesis flow of a model based on direct metrics, making it more suitable for deployment [14, 22, 3, 19, 20, 4].”
On-device machine learning An algorithms and learning theory perspective
Dhar, S., Guo, J., Liu, J., Tripathi, S., Kurup, U., & Shah, M. (2019) · arXiv · doi:10.48550/arxiv.1911.00623 · Cited by 47
FLASH: F ast Neura l A rchitecture S earch with H ardware Optimization
Li, G., Mandal, S. K., Ogras, U. Y., & Marculescu, R. (2021) · ACM Transactions on Embedded Computing Systems · doi:10.1145/3476994 · Cited by 15
“Authors in [37] propose a hardware-aware NAS framework to design convolutional neural networks.”
Method
Metadata
- Authors
- Marculescu, D., Stamoulis, D., & Cai, E.
- Venue
- Proceedings of the International Conference on Computer-Aided Design
- Publisher
- ACM
- Published
- 2018-11-05
- Pages
- 1-8
- OpenAlex
- W2890590696
- Cited by
- 49
- References
- 12
- FWCI
- 1.85
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.