The DOI matches its Crossref record; 22 of the 22 references it cites were found in a registry or an index.
Resource Central: Understanding and Predicting Workloads for Improved Resource Management in Large Cloud Platforms
Abstract
Cloud research to date has lacked data on the characteristics of the production virtual machine (VM) workloads of large cloud providers. A thorough understanding of these characteristics can inform the providers' resource management systems, e.g. VM scheduler, power manager, server health manager. In this paper, we first introduce an extensive characterization of Microsoft Azure's VM workload, including distributions of the VMs' lifetime, deployment size, and resource consumption. We then show that certain VM behaviors are fairly consistent over multiple lifetimes, i.e. history is an accurate predictor of future behavior. Based on this observation, we next introduce Resource Central (RC), a system that collects VM telemetry, learns these behaviors offline, and provides predictions online to various resource managers via a general client-side library. As an example of RC's online use, we modify Azure's VM scheduler to leverage predictions in oversubscribing servers (with oversubscribable VM types), while retaining high VM performance. Using real VM traces, we then show that the prediction-informed schedules increase utilization and prevent physical resource exhaustion. We conclude that providers can exploit their workloads' characteristics and machine learning to improve resource management substantially.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
TensorFlow: A system for large-scale machine learning
Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.08695 · Cited by 11,822
Caffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654889 · Cited by 11,077
Heterogeneity and dynamicity of clouds at scale: Google trace analysis
Reiss, C., Tumanov, A., Ganger, G. R., Katz, R. H., & Kozuch, M. A. (2012) · Proceedings of the Third ACM Symposium on Cloud Computing · doi:10.1145/2391229.2391236 · Cited by 1,063
CloudCmp: comparing public cloud providers
Li, A., Yang, X., Kandula, S., & Zhang, M. (2010) · Proceedings of the 10th ACM SIGCOMM conference on Internet measurement · doi:10.1145/1879141.1879143 · Cited by 882
Dynamic Placement of Virtual Machines for Managing SLA Violations
Bobroff, N., Kochut, A., & Beaty, K. (2007) · 2007 10th IFIP/IEEE International Symposium on Integrated Network Management · doi:10.1109/inm.2007.374776 · Cited by 808
Energy Efficient Resource Management in Virtualized Cloud Data Centers
Beloglazov, A., & Buyya, R. (2010) · 2010 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing · doi:10.1109/ccgrid.2010.46 · Cited by 760
Quasar: resource-efficient and QoS-aware cluster management
Delimitrou, C., & Kozyrakis, C. (2014) · Proceedings of the 19th international conference on Architectural support for programming languages and operating systems · doi:10.1145/2541940.2541941 · Cited by 702
pMapper: Power and Migration Cost Aware Application Placement in Virtualized Systems
Verma, A., Ahuja, P., & Neogi, A. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-89856-6_13 · Cited by 649
Empirical prediction models for adaptive resource provisioning in the cloud
Islam, S., Keung, J., Lee, K., & Liu, A. (2012) · Future Generation Computer Systems · doi:10.1016/j.future.2011.05.027 · Cited by 631
Workload Prediction Using ARIMA Model and Its Impact on Cloud Applications’ QoS
Calheiros, R. N., Masoumi, E., Ranjan, R., & Buyya, R. (2015) · IEEE Transactions on Cloud Computing · doi:10.1109/tcc.2014.2350475 · Cited by 617
Efficient Autoscaling in the Cloud Using Predictive Models for Workload Forecasting
Roy, N., Dubey, A., & Gokhale, A. (2011) · 2011 IEEE 4th International Conference on Cloud Computing · doi:10.1109/cloud.2011.42 · Cited by 591
PRESS: PRedictive Elastic ReSource Scaling for cloud systems
Gong, Z., Gu, X., & Wilkes, J. (2010) · 2010 International Conference on Network and Service Management · doi:10.1109/cnsm.2010.5691343 · Cited by 544
Heracles: improving resource efficiency at scale
Lo, D., Cheng, L., Govindaraju, R., Ranganathan, P., & Kozyrakis, C. (2015) · Proceedings of the 42nd Annual International Symposium on Computer Architecture · doi:10.1145/2749469.2749475 · Cited by 466
Clipper: A Low-Latency Online Prediction Serving System
Crankshaw, D., Wang, X., Zhou, G., Franklin, M. J., Gonzalez, J. E., & Stoica, I. (2016) · arXiv · doi:10.48550/arxiv.1612.03079 · Cited by 415
Towards characterizing cloud backend workloads: insights from Google compute clusters
Mishra, A. K., Hellerstein, J. L., Cirne, W., & Das, C. R. (2010) · ACM SIGMETRICS Performance Evaluation Review · doi:10.1145/1773394.1773400 · Cited by 374
Bubble-flux: precise online QoS management for increased utilization in warehouse scale computers
Yang, H., Breslow, A., Mars, J., & Tang, L. (2013) · Proceedings of the 40th Annual International Symposium on Computer Architecture · doi:10.1145/2485922.2485974 · Cited by 345
Workload characterization and prediction in the cloud: A multiple time series approach
Khan, A., Yan, X., Tao, S., & Anerousis, N. (2012) · 2012 IEEE Network Operations and Management Symposium · doi:10.1109/noms.2012.6212065 · Cited by 305
Novaković, D., Vasić, N., Novaković, S., Kostić, D., & Bianchini, R. (2013) · IMDEA Networks Institute Digital Repository · Cited by 230
Self-citationCharacterization and Comparison of Cloud versus Grid Workloads
Di, S., Kondo, D., & Cirne, W. (2012) · 2012 IEEE International Conference on Cluster Computing · doi:10.1109/cluster.2012.35 · Cited by 170
Crankshaw, D., Bailis, P. D., Gonzalez, J. E., Li, H., Zhang, Z., Franklin, M. J. M., Ghodsi, A., & Jordan, M. I. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1409.3809 · Cited by 84
Next stop, the cloud: understanding modern web service deployment in EC2 and azure
He, K., Fisher, A., Wang, L., Gember-Jacobson, A., Akella, A., & Ristenpart, T. (2013) · ACM Internet Measurement Conference (IMC) · doi:10.1145/2504730.2504740 · Cited by 75
Usage Patterns and the Economics of the Public Cloud
Kilcioglu, C., Rao, J. M., Kannan, A., & McAfee, R. P. (2017) · International World Wide Web Conference (WWW) · doi:10.1145/3038912.3052707 · Cited by 53
Cited by
Shahrad, M., Fonseca, R., Goiri, Í., Chaudhry, G., Batum, P., Cooke, J., Laureano, E., Tresness, C., Russinovich, M., & Bianchini, R. (2020) · arXiv · doi:10.48550/arxiv.2003.03423 · Cited by 391
“For example, data from Azure [12] shows that 63% of all VM allocations last longer than 15 minutes, and only less than 8% of the VMs last less 5 minutes or less.”
MethodSelf-citationPond: CXL-Based Memory Pooling Systems for Cloud Platforms
Li, H., Berger, D. S., Hsu, L., Ernst, D., Zardoshti, P., Novakovic, S., Shah, M., Rajadnya, S., Lee, S., Agarwal, I., et al. (2023) · Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2 · doi:10.1145/3575693.3578835 · Cited by 341
Self-citationTirmazi, M., Barker, A., Deng, N., Haque, M. E., Qin, Z. G., Hand, S., Harchol-Balter, M., & Wilkes, J. (2020) · Proceedings of the Fifteenth European Conference on Computer Systems · doi:10.1145/3342195.3387517 · Cited by 325
Characterizing Microservice Dependency and Performance: Alibaba Trace Analysis
Luo, S., Xu, H., Lu, C., Ye, K., Xu, G., Zhang, L., Ding, Y., He, J., & Xu, C. (2021) · Proceedings of the ACM Symposium on Cloud Computing · doi:10.1145/3472883.3487003 · Cited by 293
Who limits the resource efficiency of my datacenter: an analysis of Alibaba datacenter traces
Guo, J., Chang, Z., Wang, S., Ding, H., Feng, Y., Mao, L., & Bao, Y. (2019) · Proceedings of the International Symposium on Quality of Service · doi:10.1145/3326285.3329074 · Cited by 287
Metadata
- Authors
- Cortez, E., Bonde, A., Muzio, A., Russinovich, M., Fontoura, M., & Bianchini, R.
- Venue
- Proceedings of the 26th Symposium on Operating Systems Principles
- Publisher
- ACM
- Published
- 2017-10-14
- Pages
- 153-167
- OpenAlex
- W2764100055
- Cited by
- 633
- References
- 22
- FWCI
- 34.73
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.