The DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index.
Quincy: fair scheduling for distributed computing clusters
Abstract
This paper addresses the problem of scheduling concurrent jobs on clusters where application data is stored on the computing nodes. This setting, in which scheduling computations close to their data is crucial for performance, is increasingly common and arises in systems such as MapReduce, Hadoop, and Dryad as well as many grid-computing environments. We argue that data-intensive computation benefits from a fine-grain resource sharing model that differs from the coarser semi-static resource allocations implemented by most existing cluster computing architectures. The problem of scheduling with locality and fairness constraints has not previously been extensively studied under this resource-sharing model.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
MapReduce: simplified data processing on large clusters
Dean, J., & Ghemawat, S. (2008) · Communications of the ACM · doi:10.1145/1327452.1327492 · Cited by 18,476
Ghemawat, S., Gobioff, H., & Leung, S.-T. (2003) · ACM SIGOPS Operating Systems Review · doi:10.1145/1165389.945450 · Cited by 4,984
Ford, D. R., & Fulkerson, D. R. (1962) · Cited by 4,072
Approximation Algorithms for NP-Hard Problems
(1997) · ACM SIGACT News · doi:10.1145/261342.571216 · Cited by 3,103
Dryad: distributed data-parallel programs from sequential building blocks
Isard, M., Budiu, M., Yu, Y., Birrell, A., & Fetterly, D. (2007) · Proceedings of the 2nd ACM SIGOPS/EuroSys European Conference on Computer Systems 2007 · doi:10.1145/1272996.1273005 · Cited by 2,432
Self-citationDistributed computing in practice: the Condor experience
Thain, D., Tannenbaum, T., & Livny, M. (2005) · Concurrency and Computation: Practice and Experience · doi:10.1002/cpe.938 · Cited by 1,886
Improving MapReduce performance in heterogeneous environments
Zaharia, M., Konwinski, A., Joseph, A. D., Katz, R. H., & Stoica, I. G. (2008) · doi:10.5555/1855741.1855744 · Cited by 1,588
Ghemawat, S., Gobioff, H., & Leung, S.-T. (2003) · Proceedings of the nineteenth ACM symposium on Operating systems principles · doi:10.1145/945445.945450 · Cited by 1,423
Matchmaking: distributed resource management for high throughput computing
Raman, R., Livny, M., & Solomon, M. (2002) · Proceedings. The Seventh International Symposium on High Performance Distributed Computing (Cat. No.98TB100244) · doi:10.1109/hpdc.1998.709966 · Cited by 747
Yu, Y., Isard, M., Fetterly, D., Budiu, M., Erlingsson, Ú., Gunda, P. K., & Currey, J. (2008) · doi:10.5555/1855741.1855742 · Cited by 689
Self-citation
Cited by
(2011) · doi:10.5555/1972457.1972488 · Cited by 1,619
Large-scale cluster management at Google with Borg
Verma, A., Pedrosa, L., Korupolu, M., Oppenheimer, D., Tune, E., & Wilkes, J. (2015) · Proceedings of the Tenth European Conference on Computer Systems · doi:10.1145/2741948.2741964 · Cited by 1,468
Delay scheduling: a simple technique for achieving locality and fairness in cluster scheduling
Zaharia, M., Borthakur, D., Sen Sarma, J., Elmeleegy, K., Shenker, S., & Stoica, I. (2010) · Proceedings of the 5th European conference on Computer systems · doi:10.1145/1755913.1755940 · Cited by 1,393
Resource Management with Deep Reinforcement Learning
Mao, H., Alizadeh, M., Menache, I., & Kandula, S. (2016) · Proceedings of the 15th ACM Workshop on Hot Topics in Networks · doi:10.1145/3005745.3005750 · Cited by 1,353
Dominant resource fairness: fair allocation of multiple resource types
Ghodsi, A., Zaharia, M., Hindman, B., Konwinski, A., Shenker, S. J., & Stoica, I. G. (2011) · doi:10.5555/1972457.1972490 · Cited by 1,049
Dynamic Resource Allocation Using Virtual Machines for Cloud Computing Environment
Xiao, Z., Song, W., & Chen, Q. (2013) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2012.283 · Cited by 858
Learning scheduling algorithms for data processing clusters
Mao, H., Schwarzkopf, M., Venkatakrishnan, S. B., Meng, Z., & Alizadeh, M. (2019) · Proceedings of the ACM Special Interest Group on Data Communication · doi:10.1145/3341302.3342080 · Cited by 723
Omega: flexible, scalable schedulers for large compute clusters
Schwarzkopf, M., Konwinski, A., Abd-El-Malek, M., & Wilkes, J. (2013) · Proceedings of the 8th ACM European Conference on Computer Systems · doi:10.1145/2465351.2465386 · Cited by 650
Reining in the outliers in map-reduce clusters using Mantri
Ananthanarayanan, G., Kandula, S., Greenberg, A. G., Stoica, I. G., Lu, Y., Saha, B., & Harris, E. F. (2010) · Operating Systems Design and Implementation · doi:10.5555/1924943.1924962 · Cited by 636
Sparrow: distributed, low latency scheduling
Ousterhout, K., Wendell, P., Zaharia, M., & Stoica, I. (2013) · Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles · doi:10.1145/2517349.2522716 · Cited by 586
Managing data transfers in computer clusters with orchestra
Chowdhury, M., Zaharia, M., Ma, J., Jordan, M. I., & Stoica, I. (2011) · Proceedings of the ACM SIGCOMM 2011 conference · doi:10.1145/2018436.2018448 · Cited by 543
Interactive analytical processing in big data systems: a cross-industry study of MapReduce workloads
Chen, Y., Alspaugh, S., & Katz, R. (2012) · Proceedings of the VLDB Endowment · doi:10.14778/2367502.2367519 · Cited by 469
Multi-resource packing for cluster schedulers
Grandl, R., Ananthanarayanan, G., Kandula, S., Rao, S., & Akella, A. (2014) · Proceedings of the 2014 ACM conference on SIGCOMM · doi:10.1145/2619239.2626334 · Cited by 421
Occupy the cloud: distributed computing for the 99%
Jonas, E., Pu, Q., Venkataraman, S., Stoica, I., & Recht, B. (2017) · Proceedings of the 2017 Symposium on Cloud Computing · doi:10.1145/3127479.3128601 · Cited by 411
ARIA: automatic resource inference and allocation for mapreduce environments
Verma, A., Cherkasova, L., & Campbell, R. H. (2011) · Proceedings of the 8th ACM international conference on Autonomic computing · doi:10.1145/1998582.1998637 · Cited by 410
Shark: SQL and rich analytics at scale
Xin, R. S., Rosen, J., Zaharia, M., Franklin, M. J., Shenker, S., & Stoica, I. (2013) · Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data · doi:10.1145/2463676.2465288 · Cited by 369
Sharing the data center network
Shieh, A., Kandula, S., Greenberg, A. G., Kim, C., & Saha, B. (2011) · doi:10.5555/1972457.1972489 · Cited by 322
An Analysis of Traces from a Production MapReduce Cluster
Kavulya, S., Tan, J., Gandhi, R., & Narasimhan, P. (2010) · 2010 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing · doi:10.1109/ccgrid.2010.112 · Cited by 315
Augmenting data center networks with multi-gigabit wireless links
Halperin, D., Kandula, S., Padhye, J., Bahl, P., & Wetherall, D. (2011) · Proceedings of the ACM SIGCOMM 2011 conference · doi:10.1145/2018436.2018442 · Cited by 312
Managing data transfers in computer clusters with orchestra
Chowdhury, M., Zaharia, M., Ma, J., Jordan, M. I., & Stoica, I. (2011) · ACM SIGCOMM Computer Communication Review · doi:10.1145/2043164.2018448 · Cited by 305
Apollo: scalable and coordinated scheduling for cloud-scale computing
Boutin, É., Ekanayake, J., Lin, W., Shi, B., Zhou, J., Qian, Z., Wu, M., & Zhou, L. (2014) · Operating Systems Design and Implementation · doi:10.5555/2685048.2685071 · Cited by 300
Jockey: guaranteed job latency in data parallel clusters
Ferguson, A. D., Bodik, P., Kandula, S., Boutin, E., & Fonseca, R. (2012) · Proceedings of the 7th ACM european conference on Computer Systems · doi:10.1145/2168836.2168847 · Cited by 286
PARTIES: QoS-Aware Resource Partitioning for Multiple Interactive Services
Chen, S., Delimitrou, C., & Martínez, J. F. (2019) · Proceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems · doi:10.1145/3297858.3304005 · Cited by 267
Piccolo: building fast, distributed programs with partitioned tables
Power, R., & Li, J. (2010) · doi:10.5555/1924943.1924964 · Cited by 252
Low Latency Geo-distributed Data Analytics
Pu, Q., Ananthanarayanan, G., Bodik, P., Kandula, S., Akella, A., Bahl, P., & Stoica, I. (2015) · Proceedings of the 2015 ACM Conference on Special Interest Group on Data Communication · doi:10.1145/2785956.2787505 · Cited by 246
A Novel Multi-Satellite and Multi-Task Scheduling Method Based on Task Network Graph Aggregation
Fan, H., Yang, Z., Zhang, X., Wu, S., Long, J., & Liu, L. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.117565 · Cited by 37
(2021) · doi:10.1145/3450268.3453521 · Cited by 19
Locality-aware Fair Scheduling in LLM Serving
Cao, S., Wang, Y., Mao, Z., Hsu, P.-L., Yin, L., Xia, T., Li, D., Liu, S., Zhang, Y., Zhou, Y., et al. (2025) · arXiv · doi:10.48550/arxiv.2501.14312
Metadata
- Authors
- Isard, M., Prabhakaran, V., Currey, J., Wieder, U., Talwar, K., & Goldberg, A.
- Venue
- Proceedings of the ACM SIGOPS 22nd symposium on Operating systems principles
- Publisher
- ACM
- Published
- 2009-10-11
- Pages
- 261-276
- OpenAlex
- W2022678927
- Cited by
- 856
- References
- 18
- FWCI
- 39.95
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.