The DOI matches its Crossref record; 7 of the 7 references it cites were found in a registry or an index.
Towards characterizing cloud backend workloads: insights from Google compute clusters
Abstract
The advent of cloud computing promises highly available, efficient, and flexible computing services for applications such as web search, email, voice over IP, and web search alerts. Our experience at Google is that realizing the promises of cloud computing requires an extremely scalable backend consisting of many large compute clusters that are shared by application tasks with diverse service level requirements for throughput, latency, and jitter. These considerations impact (a) capacity planning to determine which machine resources must grow and by how much and (b) task scheduling to achieve high machine utilization and to meet service level objectives. Both capacity planning and task scheduling require a good understanding of task resource consumption (e.g., CPU and memory usage). This in turn demands simple and accurate approaches to workload classification-determining how to form groups of tasks (workloads) with similar resource demands. One approach to workload classification is to make each task its own workload. However, this approach scales poorly since tens of thousands of tasks execute daily on Google compute clusters. Another approach to workload classification is to view all tasks as belonging to a single workload. Unfortunately, applying such a coarse-grain workload classification to the diversity of tasks running on Google compute clusters results in large variances in predicted resource consumptions. This paper describes an approach to workload classification and its application to the Google Cloud Backend, arguably the largest cloud backend on the planet. Our methodology for workload classification consists of: (1) identifying the workload dimensions; (2) constructing task classes using an off-the-shelf algorithm such as k-means; (3) determining the break points for qualitative coordinates within the workload dimensions; and (4) merging adjacent task classes to reduce the number of workloads. We use the foregoing, especially the notion of qualitative coordinates, to glean several insights about the Google Cloud Backend: (a) the duration of task executions is bimodal in that tasks either have a short duration or a long duration; (b) most tasks have short durations; and (c) most resources are consumed by a few tasks with long duration that have large demands for CPU and memory.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Generating representative Web workloads for network and server performance evaluation
Barford, P., & Crovella, M. (1998) · Proceedings of the 1998 ACM SIGMETRICS joint international conference on Measurement and modeling of computer systems · doi:10.1145/277851.277897 · Cited by 1,431
Generating representative Web workloads for network and server performance evaluation
Barford, P., & Crovella, M. (1998) · ACM SIGMETRICS Performance Evaluation Review · doi:10.1145/277858.277897 · Cited by 442
PROBABILITY AND MATHEMATICAL STATISTICS
(1999) · Elsevier eBooks · doi:10.1016/b978-012642350-1/50033-3 · Cited by 425
Job characteristics of a production parallel scientific workload on the NASA Ames iPSC/860
Feitelson, D. G., & Nitzberg, B. (1995) · Lecture Notes in Computer Science · doi:10.1007/3-540-60153-8_38 · Cited by 199
Characterizing Network Traffic in a Cluster-based, Multi-tier Data Center
Ersoz, D., Yousif, M. S., & Das, C. R. (2007) · International Conference on Distributed Computing Systems · doi:10.1109/icdcs.2007.90 · Cited by 111
Self-citationParallel Computer Workload Modeling with Markov Chains
Song, B., Ernemann, C., & Yahyapour, R. (2005) · Lecture notes in computer science · doi:10.1007/11407522_3 · Cited by 66
A Co-Plot analysis of logs and models of parallel workloads
Talby, D., Feitelson, D. G., & Raveh, A. (2007) · ACM Transactions on Modeling and Computer Simulation · doi:10.1145/1243991.1243993 · Cited by 12
Cited by
Large-scale cluster management at Google with Borg
Verma, A., Pedrosa, L., Korupolu, M., Oppenheimer, D., Tune, E., & Wilkes, J. (2015) · Proceedings of the Tenth European Conference on Computer Systems · doi:10.1145/2741948.2741964 · Cited by 1,468
Bubble-Up: increasing utilization in modern warehouse scale computers via sensible co-locations
Mars, J., Tang, L., Hundt, R., Skadron, K., & Soffa, M. L. (2011) · Proceedings of the 44th Annual IEEE/ACM International Symposium on Microarchitecture · doi:10.1145/2155620.2155650 · Cited by 607
Interactive analytical processing in big data systems: a cross-industry study of MapReduce workloads
Chen, Y., Alspaugh, S., & Katz, R. (2012) · Proceedings of the VLDB Endowment · doi:10.14778/2367502.2367519 · Cited by 469
Bubble-flux: precise online QoS management for increased utilization in warehouse scale computers
Yang, H., Breslow, A., Mars, J., & Tang, L. (2013) · Proceedings of the 40th Annual International Symposium on Computer Architecture · doi:10.1145/2485922.2485974 · Cited by 345
Tirmazi, M., Barker, A., Deng, N., Haque, M. E., Qin, Z. G., Hand, S., Harchol-Balter, M., & Wilkes, J. (2020) · Proceedings of the Fifteenth European Conference on Computer Systems · doi:10.1145/3342195.3387517 · Cited by 325
Metadata
- Authors
- Mishra, A. K., Hellerstein, J. L., Cirne, W., & Das, C. R.
- Venue
- ACM SIGMETRICS Performance Evaluation Review
- Published
- 2010-03-27
- Volume
- 37
- Issue
- 4
- Pages
- 34-41
- ISSN
- 0163-5999 (ISSN-L)
- OpenAlex
- W2111556044
- Cited by
- 374
- References
- 7
- FWCI
- 58.73
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Towards characterizing cloud backend workloads: insights from Google compute clusters. https://app.cochat.ai/papers/10.1145/1773394.1773400
- The paper
- Mishra, A. K., Hellerstein, J. L., Cirne, W., & Das, C. R. (2010). Towards characterizing cloud backend workloads: insights from Google compute clusters. ACM SIGMETRICS Performance Evaluation Review. https://doi.org/10.1145/1773394.1773400
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.