The DOI matches its Crossref record; 20 of the 20 references it cites were found in a registry or an index.
Heterogeneity and dynamicity of clouds at scale: Google trace analysis
Abstract
To better understand the challenges in developing effective cloud-based resource schedulers, we analyze the first publicly available trace data from a sizable multi-purpose cluster. The most notable workload characteristic is heterogeneity: in resource types (e.g., cores:RAM per machine) and their usage (e.g., duration and resources needed). Such heterogeneity reduces the effectiveness of traditional slot- and core-based scheduling. Furthermore, some tasks are constrained as to the kind of machine types they can use, increasing the complexity of resource assignment and complicating task migration. The workload is also highly dynamic, varying over time and most workload features, and is driven by many short jobs that demand quick scheduling decisions. While few simplifying assumptions apply, we find that many longer-running jobs have relatively stable resource utilizations, which can help adaptive resource schedulers.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
MapReduce: simplified data processing on large clusters
Dean, J., & Ghemawat, S. (2008) · Communications of the ACM · doi:10.1145/1327452.1327492 · Cited by 18,476
Power-Law Distributions in Empirical Data
Clauset, A., Shalizi, C. R., & Newman, M. E. J. (2009) · SIAM Review · doi:10.1137/070710111 · Cited by 8,019
Power-law distributions in empirical data
Clauset, A. J., Shalizi, C. R., & Newman, M. E. J. (2018) · Figshare · doi:10.1184/r1/6586835.v1 · Cited by 6,504
Dryad: distributed data-parallel programs from sequential building blocks
Isard, M., Budiu, M., Yu, Y., Birrell, A., & Fetterly, D. (2007) · Proceedings of the 2nd ACM SIGOPS/EuroSys European Conference on Computer Systems 2007 · doi:10.1145/1272996.1273005 · Cited by 2,432
What Supercomputers Say: A Study of Five System Logs
Oliner, A., & Stearley, J. (2007) · 37th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN'07) · doi:10.1109/dsn.2007.103 · Cited by 620
The workload on parallel supercomputers: modeling the characteristics of rigid jobs
Lublin, U., & Feitelson, D. G. (2003) · Journal of Parallel and Distributed Computing · doi:10.1016/s0743-7315(03)00108-4 · Cited by 441
The Case for Evaluating MapReduce Performance Using Workload Suites
Chen, Y., Ganapathi, A., Griffith, R., & Katz, R. (2011) · 2011 IEEE 19th Annual International Symposium on Modelling, Analysis, and Simulation of Computer and Telecommunication Systems · doi:10.1109/mascots.2011.12 · Cited by 428
Self-citationAn Analysis of Traces from a Production MapReduce Cluster
Kavulya, S., Tan, J., Gandhi, R., & Narasimhan, P. (2010) · 2010 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing · doi:10.1109/ccgrid.2010.112 · Cited by 315
Dynamic energy-aware capacity provisioning for cloud computing environments
Zhang, Q., Zhani, M. F., Zhang, S., Zhu, Q., Boutaba, R., & Hellerstein, J. L. (2012) · Proceedings of the 9th international conference on Autonomic computing · doi:10.1145/2371536.2371562 · Cited by 225
Characterizing, modeling, and generating workload spikes for stateful services
Bodik, P., Fox, A., Franklin, M. J., Jordan, M. I., & Patterson, D. A. (2010) · Proceedings of the 1st ACM symposium on Cloud computing · doi:10.1145/1807128.1807166 · Cited by 207
Modeling and synthesizing task placement constraints in Google compute clusters
Sharma, B., Chudnovsky, V., Hellerstein, J. L., Rifaat, R., & Das, C. R. (2011) · Proceedings of the 2nd ACM Symposium on Cloud Computing · doi:10.1145/2038916.2038919 · Cited by 184
Characterization and Comparison of Cloud versus Grid Workloads
Di, S., Kondo, D., & Cirne, W. (2012) · 2012 IEEE International Conference on Cluster Computing · doi:10.1109/cluster.2012.35 · Cited by 170
Characterizing Machines and Workloads on a Google Cluster
Liu, Z., & Cho, S. (2012) · International Conference on Parallel Processing Workshops (ICPPW) · doi:10.1109/icppw.2012.57 · Cited by 169
Co-analysis of RAS Log and Job Log on Blue Gene/P
Zheng, Z., Yu, L., Tang, W., Lan, Z., Gupta, R., Desai, N., Coghlan, S., & Buettner, D. (2011) · IEEE International Parallel and Distributed Processing Symposium (IPDPS) · doi:10.1109/ipdps.2011.83 · Cited by 79
alsched: algebraic scheduling of mixed workloads in heterogeneous clouds
Tumanov, A., Cipar, J., Ganger, G. R., & Kozuch, M. A. (2012) · ACM Symposium on Cloud Computing (SoCC) · doi:10.1145/2391229.2391254 · Cited by 61
Self-citationThe seven deadly sins of cloud computing research
Schwarzkopf, M., Murray, D. G., & Hand, S. (2012) · USENIX conference on Hot Topics in Cloud Ccomputing · Cited by 57
Design Insights for MapReduce from Diverse Production Workloads
Chen, Y., Alspaugh, S., & Katz, R. H. (2012) · Defense Technical Information Center (DTIC) · doi:10.21236/ada555881 · Cited by 56
Self-citationKozuch, M. A., Ryan, M. P., Gass, R., Schlosser, S. W., O’Hallaron, D. R., Cipar, J., Krevat, E., López, J., Stroucken, M., & Ganger, G. R. (2009) · doi:10.1145/1555271.1555282 · Cited by 56
Self-citationObfuscatory obscanturism: Making workload traces of commercially-sensitive systems safe to release
Reiss, C., Wilkes, J., & Hellerstein, J. L. (2012) · IEEE/IFIP Network Operations and Management Symposium (NOMS) · doi:10.1109/noms.2012.6212064 · Cited by 42
Self-citationThe datacenter needs an operating system
Zaharia, M., Hindman, B., Konwinski, A., Ghodsi, A., Joesph, A. D., Katz, R. H., Shenker, S. J., & Stoica, I. G. (2011) · IEEE International Conference on Cloud Computing Technology and Science · doi:10.5555/2170444.2170461 · Cited by 39
Self-citation
Cited by
Large-scale cluster management at Google with Borg
Verma, A., Pedrosa, L., Korupolu, M., Oppenheimer, D., Tune, E., & Wilkes, J. (2015) · Proceedings of the Tenth European Conference on Computer Systems · doi:10.1145/2741948.2741964 · Cited by 1,468
Quasar: resource-efficient and QoS-aware cluster management
Delimitrou, C., & Kozyrakis, C. (2014) · Proceedings of the 19th international conference on Architectural support for programming languages and operating systems · doi:10.1145/2541940.2541941 · Cited by 702
Omega: flexible, scalable schedulers for large compute clusters
Schwarzkopf, M., Konwinski, A., Abd-El-Malek, M., & Wilkes, J. (2013) · Proceedings of the 8th ACM European Conference on Computer Systems · doi:10.1145/2465351.2465386 · Cited by 650
Cortez, E., Bonde, A., Muzio, A., Russinovich, M., Fontoura, M., & Bianchini, R. (2017) · Proceedings of the 26th Symposium on Operating Systems Principles · doi:10.1145/3132747.3132772 · Cited by 633
Resource Management in Clouds: Survey and Research Challenges
Jennings, B., & Stadler, R. (2015) · Journal of Network and Systems Management · doi:10.1007/s10922-014-9307-7 · Cited by 548
Metadata
- Authors
- Reiss, C., Tumanov, A., Ganger, G. R., Katz, R. H., & Kozuch, M. A.
- Venue
- Proceedings of the Third ACM Symposium on Cloud Computing
- Publisher
- ACM
- Published
- 2012-10-14
- Pages
- 1-13
- OpenAlex
- W2129542763
- Cited by
- 1,063
- References
- 20
- FWCI
- 50.03
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.