Designing adaptive neural networks for energy-constrained image classification
Open accessThe DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index.
Abstract
As convolutional neural networks (CNNs) enable state-of-the-art computer vision applications, their high energy consumption has emerged as a key impediment to their deployment on embedded and mobile devices. Towards efficient image classification under hardware constraints, prior work has proposed adaptive CNNs, i.e., systems of networks with different accuracy and computation characteristics, where a selection scheme adaptively selects the network to be evaluated for each input image. While previous efforts have investigated different network selection schemes, we find that they do not necessarily result in energy savings when deployed on mobile systems. The key limitation of existing methods is that they learn only how data should be processed among the CNNs and not the network architectures, with each network being treated as a blackbox. To address this limitation, we pursue a more powerful design paradigm where the architecture settings of the CNNs are treated as hyper-parameters to be globally optimized. We cast the design of adaptive CNNs as a hyper-parameter optimization problem with respect to energy, accuracy, and communication constraints imposed by the mobile device. To efficiently solve this problem, we adapt Bayesian optimization to the properties of the design space, reaching near-optimal configurations in few tens of function evaluations. Our method reduces the energy consumed for image classification on a mobile device by up to 6×, compared to the best previously published work that uses CNNs as blackboxes. Finally, we evaluate two image classification practices, i.e., classifying all images locally versus over the cloud under energy and communication constraints.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Stamoulis et al. [106] cast the adaptive DNN model selection issue as a hyperparameter optimization problem by taking into account the accuracy and communication constraints imposed by the devices.Then, a Bayesian optimization (BO) is adopted to solve this problem, achieving an improvement by up to 6× in terms of minimum energy per image under accuracy constraints.”
From Edge Intelligence: Paving the Last Mile of Artificial Intelligence With Edge Computing (2019) · cites this paper as [106]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Practical Bayesian Optimization of Machine Learning Algorithms
Snoek, J., Larochelle, H., & Adams, R. P. (2012) · arXiv · doi:10.48550/arxiv.1206.2944 · Cited by 6,691
Caffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. J. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1408.5093 · Cited by 3,752
Eyeriss: An Energy-Efficient Reconfigurable Accelerator for Deep Convolutional Neural Networks
Chen, Y.-H., Krishna, T., Emer, J. S., & Sze, V. (2017) · IEEE Journal of Solid-State Circuits · doi:10.1109/jssc.2016.2616357 · Cited by 3,247 · cited 1×
“, hardware accelerators [4]).”
Learning both Weights and Connections for Efficient Neural Networks
Han, S., Pool, J., Tran, J., & Dally, W. J. (2015) · arXiv · doi:10.48550/arxiv.1506.02626 · Cited by 1,481 · cited 1×
“adaptive CNNs, there are two key approaches that address energy efficiency based on CNN parameter reduction. First, prior art has investigated pruning-based methods to reduce the network connections [6, 11, 33]. Second, other works reduce directly the computational complexity of CNNs by quantizing the network weights…”
MethodDistributed Deep Neural Networks Over the Cloud, the Edge and End Devices
Teerapittayanon, S., McDanel, B., & Kung, H. T. (2017) · 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS) · doi:10.1109/icdcs.2017.226 · Cited by 843
Designing Energy-Efficient Convolutional Neural Networks Using Energy-Aware Pruning
Yang, T.-J., Chen, Y.-H., & Sze, V. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.643 · Cited by 732 · cited 2×
“First, prior art has investigated pruning-based methods to reduce the network connections [6, 11, 33].”
BackgroundMulti-digit Number Recognition from Street View Imagery using Deep Convolutional Neural Networks
Goodfellow, I. J., Bulatov, Y., Ibarz, J., Arnoud, S., & Shet, V. (2013) · arXiv · doi:10.48550/arxiv.1312.6082 · Cited by 582 · cited 1×
“A large body of work from both industry [9] and academia [17–20, 32] exploits the insight that a large portion of an image dataset can be correctly classified by simpler CNN architectures.”
BackgroundMorphNet: Fast & Simple Resource-Constrained Structure Learning of Deep Networks
Gordon, A., Eban, E., Nachum, O., Chen, B., Wu, H., Yang, T.-J., & Choi, E. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00171 · Cited by 341
Bayesian Optimization with Unknown Constraints
Gelbart, M. A., Snoek, J., & Adams, R. P. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1403.5607 · Cited by 299
NeST: A Neural Network Synthesis Tool Based on a Grow-and-Prune Paradigm
Dai, X., Yin, H., & Jha, N. K. (2019) · IEEE Transactions on Computers · doi:10.1109/tc.2019.2914438 · Cited by 233
Ko, J. H., Na, T., Amir, M. F., & Mukhopadhyay, S. (2018) · 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS) · doi:10.1109/avss.2018.8639121 · Cited by 153
Conditional Deep Learning for Energy-Efficient and Enhanced Pattern Recognition
Panda, P., Sengupta, A., & Roy, K. (2016) · Proceedings of the 2016 Design, Automation & Test in Europe Conference & Exhibition (DATE) · doi:10.3850/9783981537079_0819 · Cited by 142
Big/little deep neural network for ultra low power inference
Park, E., Kim, D., Kim, S., Kim, Y.-D., Kim, G., Yoon, S., & Yoo, S. (2015) · 2015 International Conference on Hardware/Software Codesign and System Synthesis (CODES+ISSS) · doi:10.1109/codesisss.2015.7331375 · Cited by 130
A General Framework for Constrained Bayesian Optimization using Information-based Search
Hernández-Lobato, J. M., Gelbart, M. A., Adams, R. P., Hoffman, M. W., & Ghahramani, Z. (2015) · arXiv (Cornell University) · doi:10.17863/cam.6477 · Cited by 118
Scalable-effort classifiers for energy-efficient machine learning
Venkataramani, S., Raghunathan, A., Liu, J., & Shoaib, M. (2015) · Proceedings of the 52nd Annual Design Automation Conference · doi:10.1145/2744769.2744904 · Cited by 107
A case for efficient accelerator design space exploration via Bayesian optimization
Reagen, B., Hernández-Lobato, J. M., Adolf, R., Gelbart, M. A., Whatmough, P. N., Wei, G.-Y., & Brooks, D. M. (2017) · International Symposium on Low Power Electronics and Design (ISLPED) · doi:10.1109/islped.2017.8009208 · Cited by 98
NeuralPower: Predict and Deploy Energy-Efficient Convolutional Neural Networks
Cai, E., Juan, D.-C., Stamoulis, D. S., & Marculescu, D. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.05420 · Cited by 87
Self-citationNeural networks designing neural networks: multi-objective hyper-parameter optimization
Smithson, S. C., Yang, G., Gross, W. J., & Meyer, B. H. (2016) · doi:10.1145/2966986.2967058 · Cited by 84
Big/little deep neural network for ultra low power inference
Park, E., Kim, D., Kim, S., et al. (2015) · Seoul National University Open Repository (Seoul National University) · doi:10.5555/2830840.2830854 · Cited by 67
HyperPower: Power- and memory-constrained hyper-parameter optimization for neural networks
Stamoulis, D., Cai, E., Juan, D.-C., & Marculescu, D. (2018) · 2018 Design, Automation & Test in Europe Conference & Exhibition (DATE) · doi:10.23919/date.2018.8341973 · Cited by 67
Self-citationThe cascading neural network: building the Internet of Smart Things
Leroux, S., Bohez, S., De Coninck, E., Verbelen, T., Vankeirsbilck, B., Simoens, P., & Dhoedt, B. (2017) · Knowledge and Information Systems · doi:10.1007/s10115-017-1029-1 · Cited by 59
Quantized deep neural networks for energy efficient hardware-based inference
Ding, R., Liu, Z., Blanton, R. D., & Marculescu, D. (2018) · Asia and South Pacific Design Automation Conference (ASP-DAC) · doi:10.1109/aspdac.2018.8297274 · Cited by 59
Self-citationNeST: A Neural Network Synthesis Tool Based on a Grow-and-Prune ParadigmPreprint
Dai, X., Yin, H., & Jha, N. K. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.02017 · Cited by 50
DeLight: Adding Energy Dimension To Deep Neural Networks
Rouhani, B. D., Mirhoseini, A., & Koushanfar, F. (2016) · Proceedings of the 2016 International Symposium on Low Power Electronics and Design · doi:10.1145/2934583.2934599 · Cited by 43
FALCON: Feature driven selective classification for energy-efficient image recognition
Panda, P., Ankit, A., Wijesinghe, P., & Roy, K. (2017) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2017.2681075 · Cited by 21 · cited 1×
“asi et al. extend the formulation of network-level adaptive systems to multiple networks [1]. Finally, other works extend the cascading execution to more tree-like structures for image classification [17, 23]. Nevertheless, existing approaches rely on off-the-shelf network architectures without studying their architec…”
BackgroundExploring aging deceleration in FinFET-based multi-core systems
Cai, E., Stamoulis, D. S., & Marculescu, D. (2016) · doi:10.1145/2966986.2967039 · Cited by 20
Self-citationProfit: Priority and Power/Performance Optimization for Many-Core Systems
Chen, Z., Stamoulis, D. S., & Marculescu, D. (2017) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2017.2772822 · Cited by 18
Self-citationCan We Guarantee Performance Requirements under Workload and Process Variations?
Stamoulis, D. S., & Marculescu, D. (2016) · International Symposium on Low Power Electronics and Design (ISLPED) · doi:10.1145/2934583.2934641 · Cited by 17
Self-citationEnergy-Efficient Adaptive Classifier Design for Mobile Systems
Takhirov, Z., Wang, J., Saligrama, V., & Joshi, A. (2016) · International Symposium on Low Power Electronics and Design (ISLPED) · doi:10.1145/2934583.2934615 · Cited by 17
Tree-CNN: A Deep Convolutional Neural Network for Lifelong LearningPreprint
Roy, D., Panda, P., & Roy, K. (2018) · arXiv (Cornell University) · Cited by 16
Efficient Reliability Analysis of Processor Datapath using Atomistic BTI Variability Models
Stamoulis, D. S., Rodopoulos, D., Meyer, B. H., Soudris, D., Catthoor, F., & Žilić, Ž. (2015) · ACM Great Lakes Symposium on VLSI (GLSVLSI) · doi:10.1145/2742060.2742079 · Cited by 9
Self-citationEnergy-Efficient Object Detection Using Semantic Decomposition
Panda, P., Venkataramani, S., Sengupta, A., Raghunathan, A., & Roy, K. (2017) · IEEE Transactions on Very Large Scale Integration (VLSI) Systems · doi:10.1109/tvlsi.2017.2707077 · Cited by 8
Research works printed but not linked: 3
- T. Bolukbasi et al. 2017. Adaptive Neural Networks for Efficient Inference. In International Conference on Machine Learning. 527--536.
- J. M. Hernández-Lobato et al. 2016. Designing neural network hardware accelerators with decoupled objective evaluations. In NIPS.
- 10.5555/2946645.3053442 · doi:10.5555/2946645.3053442
Cited by
Edge Intelligence: Paving the Last Mile of Artificial Intelligence With Edge Computing
Zhou, Z., Chen, X., Li, E., Zeng, L., Luo, K., & Zhang, J. (2019) · Proceedings of the IEEE · doi:10.1109/jproc.2019.2918951 · Cited by 2,455
“[106] cast the adaptive DNN model selection issue as a hyperparameter optimization problem by taking into account the accuracy and communication constraints imposed by the devices.”
MethodSustainable AI: Environmental Implications, Challenges and Opportunities
Wu, C.-J., Raghavendra, R., Gupta, U., Acun, B., Ardalani, N., Maeng, K., Chang, G., Behram, F. A., Huang, J., Bai, C., et al. (2021) · arXiv · doi:10.48550/arxiv.2111.00364 · Cited by 405
“Optimizing the overall energy efficiency of FL and on-device AI is an important first step [92], [93], [94], [95], [96].”
BackgroundA systematic review of Green AI
Verdecchia, R., Sallou, J., & Cruz, L. (2023) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1507 · Cited by 375
Autonomous Vehicles Enabled by the Integration of IoT, Edge Intelligence, 5G, and Blockchain
Biswas, A., & Wang, H.-C. (2023) · Sensors · doi:10.3390/s23041963 · Cited by 274
Trustworthy ai: A computational perspective
Liu, H., Wang, Y., Fan, W., Liu, X., Li, Y., Jain, S., Liu, Y., Jain, A., & Tang, J. (2023) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3546872 · Cited by 261
Metadata
- Authors
- Stamoulis, D., Chin, T.-W. (., Prakash, A. K., Fang, H., Sajja, S., Bognar, M., & Marculescu, D.
- Affiliations
- Carnegie Mellon University (US)
- Venue
- Proceedings of the International Conference on Computer-Aided Design
- Publisher
- ACM
- Published
- 2018-11-05
- Pages
- 1-8
- OpenAlex
- W2885527679
- Cited by
- 63
- References
- 33
- Citations by year
- 2026: 2 · 2025: 6 · 2024: 3 · 2023: 8 · 2022: 10 · 2021: 13 · 2020: 14 · 2019: 6 · 2018: 1
- FWCI
- 2.52
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (1564022)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Physical Sciences › Computer Science › Computer Networks and Communications › IoT and Edge/Fog Computing
- Keywords
- adaptive neural networks, convolutional neural networks, network selection, hyperparameter optimization, Bayesian optimization, mobile devices, embedded systems, communication constraints, CNN architecture design
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.