The DOI matches its Crossref record; 20 of the 20 references it cites were found in a registry or an index.
Heterogeneity and dynamicity of clouds at scale: Google trace analysis
Abstract
To better understand the challenges in developing effective cloud-based resource schedulers, we analyze the first publicly available trace data from a sizable multi-purpose cluster. The most notable workload characteristic is heterogeneity: in resource types (e.g., cores:RAM per machine) and their usage (e.g., duration and resources needed). Such heterogeneity reduces the effectiveness of traditional slot- and core-based scheduling. Furthermore, some tasks are constrained as to the kind of machine types they can use, increasing the complexity of resource assignment and complicating task migration. The workload is also highly dynamic, varying over time and most workload features, and is driven by many short jobs that demand quick scheduling decisions. While few simplifying assumptions apply, we find that many longer-running jobs have relatively stable resource utilizations, which can help adaptive resource schedulers.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
MapReduce: simplified data processing on large clusters
Dean, J., & Ghemawat, S. (2008) · Communications of the ACM · doi:10.1145/1327452.1327492 · Cited by 18,476
Power-Law Distributions in Empirical Data
Clauset, A., Shalizi, C. R., & Newman, M. E. J. (2009) · SIAM Review · doi:10.1137/070710111 · Cited by 8,019
Power-law distributions in empirical data
Clauset, A. J., Shalizi, C. R., & Newman, M. E. J. (2018) · Figshare · doi:10.1184/r1/6586835.v1 · Cited by 6,504
Dryad: distributed data-parallel programs from sequential building blocks
Isard, M., Budiu, M., Yu, Y., Birrell, A., & Fetterly, D. (2007) · Proceedings of the 2nd ACM SIGOPS/EuroSys European Conference on Computer Systems 2007 · doi:10.1145/1272996.1273005 · Cited by 2,432
What Supercomputers Say: A Study of Five System Logs
Oliner, A., & Stearley, J. (2007) · 37th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN'07) · doi:10.1109/dsn.2007.103 · Cited by 620
The workload on parallel supercomputers: modeling the characteristics of rigid jobs
Lublin, U., & Feitelson, D. G. (2003) · Journal of Parallel and Distributed Computing · doi:10.1016/s0743-7315(03)00108-4 · Cited by 441
The Case for Evaluating MapReduce Performance Using Workload Suites
Chen, Y., Ganapathi, A., Griffith, R., & Katz, R. (2011) · 2011 IEEE 19th Annual International Symposium on Modelling, Analysis, and Simulation of Computer and Telecommunication Systems · doi:10.1109/mascots.2011.12 · Cited by 428
Self-citationAn Analysis of Traces from a Production MapReduce Cluster
Kavulya, S., Tan, J., Gandhi, R., & Narasimhan, P. (2010) · 2010 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing · doi:10.1109/ccgrid.2010.112 · Cited by 315
Dynamic energy-aware capacity provisioning for cloud computing environments
Zhang, Q., Zhani, M. F., Zhang, S., Zhu, Q., Boutaba, R., & Hellerstein, J. L. (2012) · Proceedings of the 9th international conference on Autonomic computing · doi:10.1145/2371536.2371562 · Cited by 225
Characterizing, modeling, and generating workload spikes for stateful services
Bodik, P., Fox, A., Franklin, M. J., Jordan, M. I., & Patterson, D. A. (2010) · Proceedings of the 1st ACM symposium on Cloud computing · doi:10.1145/1807128.1807166 · Cited by 207
Cited by
Large-scale cluster management at Google with Borg
Verma, A., Pedrosa, L., Korupolu, M., Oppenheimer, D., Tune, E., & Wilkes, J. (2015) · Proceedings of the Tenth European Conference on Computer Systems · doi:10.1145/2741948.2741964 · Cited by 1,468
Quasar: resource-efficient and QoS-aware cluster management
Delimitrou, C., & Kozyrakis, C. (2014) · Proceedings of the 19th international conference on Architectural support for programming languages and operating systems · doi:10.1145/2541940.2541941 · Cited by 702
Omega: flexible, scalable schedulers for large compute clusters
Schwarzkopf, M., Konwinski, A., Abd-El-Malek, M., & Wilkes, J. (2013) · Proceedings of the 8th ACM European Conference on Computer Systems · doi:10.1145/2465351.2465386 · Cited by 650
Cortez, E., Bonde, A., Muzio, A., Russinovich, M., Fontoura, M., & Bianchini, R. (2017) · Proceedings of the 26th Symposium on Operating Systems Principles · doi:10.1145/3132747.3132772 · Cited by 633
Resource Management in Clouds: Survey and Research Challenges
Jennings, B., & Stadler, R. (2015) · Journal of Network and Systems Management · doi:10.1007/s10922-014-9307-7 · Cited by 548
Heracles: improving resource efficiency at scale
Lo, D., Cheng, L., Govindaraju, R., Ranganathan, P., & Kozyrakis, C. (2015) · Proceedings of the 42nd Annual International Symposium on Computer Architecture · doi:10.1145/2749469.2749475 · Cited by 466
Tirmazi, M., Barker, A., Deng, N., Haque, M. E., Qin, Z. G., Hand, S., Harchol-Balter, M., & Wilkes, J. (2020) · Proceedings of the Fifteenth European Conference on Computer Systems · doi:10.1145/3342195.3387517 · Cited by 325
Autopilot: workload autoscaling at Google
Rzadca, K., Findeisen, P., Swiderski, J., Zych, P., Broniek, P., Kusmierek, J., Nowak, P., Strack, B., Witusowski, P., Hand, S., et al. (2020) · Proceedings of the Fifteenth European Conference on Computer Systems · doi:10.1145/3342195.3387524 · Cited by 303
Characterizing Microservice Dependency and Performance: Alibaba Trace Analysis
Luo, S., Xu, H., Lu, C., Ye, K., Xu, G., Zhang, L., Ding, Y., He, J., & Xu, C. (2021) · Proceedings of the ACM Symposium on Cloud Computing · doi:10.1145/3472883.3487003 · Cited by 293
Who limits the resource efficiency of my datacenter: an analysis of Alibaba datacenter traces
Guo, J., Chang, Z., Wang, S., Ding, H., Feng, Y., Mao, L., & Bao, Y. (2019) · Proceedings of the International Symposium on Quality of Service · doi:10.1145/3326285.3329074 · Cited by 287
Statistical Characterization of Business-Critical Workloads Hosted in Cloud Datacenters
Shen, S., Van Beek, V., & Iosup, A. (2015) · 2015 15th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing · doi:10.1109/ccgrid.2015.60 · Cited by 286
Xu, F., Liu, F., Jin, H., & Vasilakos, A. V. (2014) · Proceedings of the IEEE · doi:10.1109/jproc.2013.2287711 · Cited by 285
CPI 2: CPU performance isolation for shared compute clusters
Zhang, X., Tune, E., Hagmann, R., Jnagal, R., Gokhale, V., & Wilkes, J. (2013) · Proceedings of the 8th ACM European Conference on Computer Systems · doi:10.1145/2465351.2465388 · Cited by 285
Experience with using the Parallel Workloads Archive
Feitelson, D. G., Tsafrir, D., & Krakov, D. (2014) · Journal of Parallel and Distributed Computing · doi:10.1016/j.jpdc.2014.06.013 · Cited by 276
Mann, Z. Á. (2015) · ACM Computing Surveys · doi:10.1145/2797211 · Cited by 274
Seer: Leveraging Big Data to Navigate the Complexity of Performance Debugging in Cloud Microservices
Gan, Y., Zhang, Y., Hu, K., Cheng, D., He, Y., Pancholi, M., & Delimitrou, C. (2019) · Proceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems · doi:10.1145/3297858.3304004 · Cited by 269
Systems Approaches to Tackling Configuration Errors: A Survey
Xu, T., & Zhou, Y. (2015) · ACM Computing Surveys · doi:10.1145/2791577 · Cited by 139
A Large-scale Analysis of Hundreds of In-memory Key-value Cache Clusters at Twitter
Yang, J., Yue, Y., & Rashmi, K. V. (2021) · ACM Transactions on Storage · doi:10.1145/3468521 · Cited by 109
(2019) · Cited by 62
Resource and Deadline-Aware Job Scheduling in Dynamic Hadoop Clusters
Cheng, D., Rao, J., Jiang, C., & Zhou, X. (2015) · 2015 IEEE International Parallel and Distributed Processing Symposium · doi:10.1109/ipdps.2015.36 · Cited by 61
Metadata
- Authors
- Reiss, C., Tumanov, A., Ganger, G. R., Katz, R. H., & Kozuch, M. A.
- Venue
- Proceedings of the Third ACM Symposium on Cloud Computing
- Publisher
- ACM
- Published
- 2012-10-14
- Pages
- 1-13
- OpenAlex
- W2129542763
- Cited by
- 1,063
- References
- 20
- FWCI
- 50.03
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.