The DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
A Hierarchical Framework of Cloud Resource Allocation and Power Management Using Deep Reinforcement Learning
Open accessAbstract
Automatic decision-making approaches, such as reinforcement learning (RL), have been applied to (partially) solve the resource allocation problem adaptively in the cloud computing system. However, a complete cloud resource allocation framework exhibits high dimensions in state and action spaces, which prohibit the usefulness of traditional RL techniques. In addition, high power consumption has become one of the critical concerns in design and control of cloud computing systems, which degrades system reliability and increases cooling cost. An effective dynamic power management (DPM) policy should minimize power consumption while maintaining performance degradation within an acceptable level. Thus, a joint virtual machine (VM) resource allocation and power management framework is critical to the overall cloud computing system. Moreover, novel solution framework is necessary to address the even higher dimensions in state and action spaces. In this paper, we propose a novel hierarchical framework for solving the overall resource allocation and power management problem in cloud computing systems. The proposed hierarchical framework comprises a global tier for VM resource allocation to the servers and a local tier for distributed power management of local servers. The emerging deep reinforcement learning (DRL) technique, which can deal with complicated control problems with large state space, is adopted to solve the global tier problem. Furthermore, an autoencoder and a novel weight sharing structure are adopted to handle the high-dimensional state space and accelerate the convergence speed. On the other hand, the local tier of distributed server power managements comprises an LSTM based workload predictor and a model-free RL based power manager, operating in a distributed manner. Experiment results using actual Google cluster traces show that our proposed hierarchical framework significantly saves the power consumption and energy usage than the baseline while achieving no severe latency degradation. Meanwhile, the proposed framework can achieve the best trade-off between latency and power/energy consumption in a server cluster.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Playing Atari with Deep Reinforcement Learning
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013) · arXiv · doi:10.48550/arxiv.1312.5602 · Cited by 9,471
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083
Learning Deep Architectures for AI
Bengio, Y. (2009) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000006 · Cited by 6,938
Convolutional networks for images, speech, and time series
LeCun, Y., & Bengio, Y. (1998) · HAL (Le Centre pour la Communication Scientifique Directe) · doi:10.5555/303568.303704 · Cited by 4,231
Power provisioning for a warehouse-sized computer
Fan, X., Weber, W.-D., & Barroso, L. A. (2007) · Proceedings of the 34th annual international symposium on Computer architecture · doi:10.1145/1250662.1250665 · Cited by 1,797
Recurrent neural networks and robust time series prediction
Connor, J. T., Martin, R. D., & Atlas, L. E. (1994) · IEEE Transactions on Neural Networks · doi:10.1109/72.279188 · Cited by 1,316
A survey of design techniques for system-level dynamic power management
Benini, L., Bogliolo, A., & De Micheli, G. (2000) · IEEE Transactions on Very Large Scale Integration (VLSI) Systems · doi:10.1109/92.845896 · Cited by 1,241
Fundamentals of Queueing Theory
Gross, D., Shortle, J., Thompson, J. M., & Harris, C. M. (2008) · Wiley series in probability and statistics · doi:10.1002/9781118625651 · Cited by 1,123
Generalization in Reinforcement Learning: Successful Examples Using Sparse Coarse Coding
Sutton, R. S. (1995) · Cited by 1,072
Srivastava, M. B., Chandrakasan, A. P., & Brodersen, R. W. (1996) · IEEE Transactions on Very Large Scale Integration (VLSI) Systems · doi:10.1109/92.486080 · Cited by 427
Towards characterizing cloud backend workloads: insights from Google compute clusters
Mishra, A. K., Hellerstein, J. L., Cirne, W., & Das, C. R. (2010) · ACM SIGMETRICS Performance Evaluation Review · doi:10.1145/1773394.1773400 · Cited by 374
Workload characterization and prediction in the cloud: A multiple time series approach
Khan, A., Yan, X., Tao, S., & Anerousis, N. (2012) · 2012 IEEE Network Operations and Management Symposium · doi:10.1109/noms.2012.6212065 · Cited by 305
Kalchbrenner, N., Danihelka, I., & Graves, A. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1507.01526 · Cited by 267
Reinforcement Learning Methods for Continuous-Time Markov Decision Problems
Bradtke, S. J., & Duff, M. O. (1994) · Cited by 263
Barrett, E., Howley, E., & Duggan, J. (2013) · Concurrency and Computation: Practice and Experience · doi:10.1002/cpe.2864 · Cited by 220
VCONF: a reinforcement learning approach to virtual machines auto-configuration
Rao, J., Bu, X., Xu, C.-Z., Wang, L., & Yin, G. (2009) · Proceedings of the 6th international conference on Autonomic computing · doi:10.1145/1555228.1555263 · Cited by 218
Simunic, T., Benini, L., Glynn, P. W., & Micheli, G. D. (2001) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/43.931003 · Cited by 196
Meisner, D., Gold, B. T., & Wenisch, T. F. (2009) · ACM SIGPLAN Notices · doi:10.1145/1508284.1508269 · Cited by 171
Managing Power Consumption and Performance of Computing Systems Using Reinforcement Learning
Tesauro, G. J., Das, R., Chan, H., Kephart, J. O., Levine, D., Rawson, F. L., & Lefurgy, C. R. (2007) · Cited by 118
Resource Allocation in the Grid Using Reinforcement Learning
Galstyan, A., Czajkowski, K., & Lerman, K. (2004) · doi:10.1109/aamas.2004.232 · Cited by 110
A predictive system shutdown method for energy saving of event-driven computation
Hwang, C.-H., & ., W. (1997) · International Conference on Computer Aided Design · doi:10.1109/iccad.1997.643266 · Cited by 110
Dynamic power management using machine learning
Dhiman, G., & Simunic, T. (2006) · Digest of technical papers/Digest of technical papers - IEEE/ACM International Conference on Computer-Aided Design · doi:10.1145/1233501.1233656 · Cited by 82
Dynamic Power Management Using Machine Learning
Dhiman, G., & Rosing, T. (2006) · Digest of technical papers/Digest of technical papers - IEEE/ACM International Conference on Computer-Aided Design · doi:10.1109/iccad.2006.320115 · Cited by 48
A Reinforcement Learning-Based Power Management Framework for Green Computing Data Centers
Lin, X., Wang, Y., & Pedram, M. (2016) · IEEE International Conference on Cloud Engineering · doi:10.1109/ic2e.2016.33 · Cited by 44
Self-citation
Cited by
A State-of-the-Art Survey on Deep Learning Theory and Architectures
Alom, M. Z., Taha, T. M., Yakopcic, C., Westberg, S., Sidike, P., Nasrin, M. S., Hasan, M., Van Essen, B. C., Awwal, A. A. S., & Asari, V. K. (2019) · Electronics · doi:10.3390/electronics8030292 · Cited by 1,620
Deep Reinforcement Learning: An Overview
Li, Y. (2017) · arXiv · doi:10.48550/arxiv.1701.07274 · Cited by 762
Hussain, F., Hassan, S. A., Hussain, R., & Hossain, E. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.2964534 · Cited by 395
Qi, Q., Wang, J., Ma, Z., Sun, H., Cao, Y., Zhang, L., & Liao, J. (2019) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2019.2894437 · Cited by 269
A Gentle Introduction to Reinforcement Learning and its Application in Different Fields
Naeem, M., Rizvi, S. T. H., & Coronato, A. (2020) · IEEE Access · doi:10.1109/access.2020.3038605 · Cited by 257
Metadata
- Authors
- Liu, N., Li, Z., Xu, J., Xu, Z., Lin, S., Qiu, Q., Tang, J., & Wang, Y.
- Venue
- 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS)
- Publisher
- IEEE
- Published
- 2017-06
- Pages
- 372-382
- OpenAlex
- W2597068831
- Cited by
- 299
- References
- 29
- FWCI
- 24.99
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.