The DOI matches its Crossref record; 63 of the 63 references it cites were found in a registry or an index.
Large-scale cluster management at Google with Borg
Open accessAbstract
Google's Borg system is a cluster manager that runs hundreds of thousands of jobs, from many thousands of different applications, across a number of clusters each with up to tens of thousands of machines.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Borg [25], the Kubernetes lineage [52], Tetris [26], Hawk [27], and Sparrow [28] pack stateless jobs; Gaia [22] optimizes geo-distributed training .”
From Janus: Joint Prefill/Decode Disaggregation with KV-Cache-Aware Multi-Cloud Routing for Edge-Adjacent LLM Serving (2026) · cites this paper as [25]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Google's Borg [27] and Kubernetes [28] handle container orchestration with resource limits and quotas.”
From AgentRM: An OS-Inspired Resource Manager for LLM Agent Systems (2026) · cites this paper as [27]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
MapReduce: simplified data processing on large clusters
Dean, J., & Ghemawat, S. (2008) · Communications of the ACM · doi:10.1145/1327452.1327492 · Cited by 18,476
Malewicz, G. G., Austern, M. H., Bik, A. J. C., Dehnert, J. C., Horn, I., Leiser, N., & Czajkowski, G. (2010) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/1807167.1807184 · Cited by 3,518
Chang, F. W., Dean, J., Ghemawat, S., Hsieh, W. C.-Y., Wallach, D. A., Burrows, M., Chandra, T., Fikes, A., & Gruber, R. E. (2008) · ACM Transactions on Computer Systems · doi:10.1145/1365815.1365816 · Cited by 3,461
Lamport, L. (1998) · ACM Transactions on Computer Systems · doi:10.1145/279227.279229 · Cited by 2,748
Ghemawat, S., Gobioff, H., & Leung, S.-T. A. (2003) · ACM SIGOPS Operating Systems Review · doi:10.1145/1165389.945450 · Cited by 2,665
Condor-a hunter of idle workstations
Litzkow, M., Livny, M., & Mutka, M. W. (2003) · Proceedings of the ... International Conference on Distributed Computing Systems/Proceedings of the International Conference on Distributed Computing Systems · doi:10.1109/dcs.1988.12507 · Cited by 2,243
Dean, J., & Barroso, L. A. (2013) · Communications of the ACM · doi:10.1145/2408776.2408794 · Cited by 1,852
Apache Hadoop YARN: yet another resource negotiator
Vavilapalli, V. K., Murthy, A. C., Douglas, C., Agarwal, S., Konar, M., Evans, R., Graves, T., Lowe, J., Shah, H., Seth, S., et al. (2013) · Proceedings of the 4th annual Symposium on Cloud Computing · doi:10.1145/2523616.2523633 · Cited by 1,848
The Datacenter as a Computer: An Introduction to the Design of Warehouse-Scale Machines
Barroso, L. A., & Hölzle, U. (2009) · Synthesis lectures on computer architecture · doi:10.2200/s00193ed1v01y200905cac006 · Cited by 1,711
(2011) · doi:10.5555/1972457.1972488 · Cited by 1,619
Research works printed but not linked: 26
- Adaptive Computing Enterprises Inc., 2011, Maui Scheduler Administrator's Guide, 3.2 edition
- Aurora Apache, 2014, http://aurora.incubator.apache.org/
- Amazon Web AWS., 2014, http://aws.amazon.com/ec2/instance-types/
- Baker J., 2011, Proc. Conference on Innovative Data Systems Research (CIDR), 223
- Bokharouss I., 2008, Eindhoven Univ. of Technology
- Boutin E., 2014, Proc. USENIX Symp. on Operating Systems Design and Implementation (OSDI)
- Burrows M., 2006, Proc. USENIX Symp. on Operating Systems Design and Implementation (OSDI), 335
- CFS, 2013, http://lwn.net/Articles/531853
- Project Docker, 2014, https://www.docker.io/
- 2014, http://www.elasticsearch.org
- 2014, http://www.fluentd.org/
- Ghodsi A., 2011, Proc. USENIX Symp. on Networked Systems Design and Implementation (NSDI), 323
- Engine Google App, 2014, http://cloud.google.com/AppEngine
- Google Container, 2015, https://cloud.google.com/container-engine/
- Project Apache Hadoop, 2009, http://hadoop.apache.org/
- Hamilton J., 2007, Proc. Large Installation System Administration Conf. (LISA), 231
- Hindman B., 2011, Proc. USENIX Symp. on Networked Systems Design and Implementation (NSDI)
- Iqbal S., 2005, Dell Power Solutions
- 10.5555/646382.689682 · DOI
- 2014, Aug.
- 10.5555/1413370.1413424 · DOI
- Project Apache Spark, 2014, http://spark.apache.org/
- Turner P., 2010, Proc. Linux Symposium, 245
- Wilkes J., 2011, Nov.
- Zhai Y., 2014, Proc. USENIX Annual Technical Conf. (USENIX ATC), 211
- Zhang Q., 2011, Proc. Int'l Workshop on Large-Scale Distributed Systems and Middleware (LADIS)
Cited by
Sorted by citations
TensorFlow: A system for large-scale machine learningPreprint
Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., et al. (2016) · arXiv (Cornell University) · doi:10.5555/3026877.3026899 · Cited by 16,548
TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., et al. (2016) · arXiv · doi:10.48550/arxiv.1603.04467 · Cited by 9,408
“In our distributed environment, these different tasks are containers in jobs managed by a cluster scheduling sys-tem [51].”
Network Function Virtualization: State-of-the-Art and Research Challenges
Mijumbi, R., Serrat, J., Gorricho, J.-L., Bouten, N., De Turck, F., & Boutaba, R. (2016) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2015.2477041 · Cited by 1,933
“In addition, existing scheduling tools such as Google’s Borg [174] and Apache Mesos [175] may be considered for scheduling of VNFs. Borg uses task-packing, over-commitment, and machine sharing with process-level performance isolation to run multiple jobs, from many applications, across a number of clusters.”
MethodLearning scheduling algorithms for data processing clusters
Mao, H., Schwarzkopf, M., Venkatakrishnan, S. B., Meng, Z., & Alizadeh, M. (2019) · ACM Special Interest Group on Data Communication · doi:10.1145/3341302.3342080 · Cited by 831
“A good scheduling policy packs work tightly to reduce fragmentation [18; 19; 46], prioritizes jobs according to high-level metrics such as user-perceived latency [47], and avoids inefficiencies due to incorrect job configurations [14].”
BackgroundInfluentialBurns, B., Grant, B., Oppenheimer, D., Brewer, E., & Wilkes, J. (2016) · Communications of the ACM · doi:10.1145/2890784 · Cited by 690
Self-citationGan, Y., Zhang, Y., Cheng, D., Shetty, A., Rathi, P., Katarki, N., Bruno, A., Hu, J., Ritchken, B., Jackson, B., et al. (2019) · Proceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems · doi:10.1145/3297858.3304013 · Cited by 665
Burns, B., Grant, B., Oppenheimer, D., Brewer, E. A., & Wilkes, J. (2016) · Queue · doi:10.1145/2898442.2898444 · Cited by 446
Self-citationOptimus: an efficient dynamic resource scheduler for deep learning clusters
Peng, Y.-H., Bao, Y., Chen, Y.-R., Wu, C., & Guo, C.-X. (2018) · European Conference on Computer Systems · doi:10.1145/3190508.3190517 · Cited by 544
“, Google uses Borg [63], Microsoft, Tencent and Baidu use YARN-like schedulers [61]) allocate a fixed amount of resources to each job upon its submission, according to resource requirements specified by the job owner.”
BackgroundMethodProfiling a warehouse-scale computer
Kanev, S., Darago, J. P., Hazelwood, K., Ranganathan, P., Moseley, T., Wei, G.-Y., & Brooks, D. M. (2015) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1145/2749469.2750392 · Cited by 409
“extensive tuning [47], the scheduler alone accounts for more than 5% of all datacenter cycles.”
BackgroundErnest: efficient performance prediction for large-scale advanced analytics
Venkataraman, S., Yang, Z., Franklin, M. J. M., Recht, B., & Stoica, I. G. (2016) · Networked Systems Design and Implementation · Cited by 373
A petavoxel fragment of human cerebral cortex reconstructed at nanoscale resolution
Shapson-Coe, A., Januszewski, M., Berger, D. R., Pope, A., Wu, Y., Blakely, T., Schalek, R. L., Li, P. H., Wang, S., Maitin-Shepard, J., et al. (2024) · Science · doi:10.1126/science.adk4858 · Cited by 361
Tirmazi, M., Barker, A., Deng, N., Haque, M. E., Qin, Z. G., Hand, S., Harchol‐Balter, M., & Wilkes, J. (2020) · European Conference on Computer Systems · doi:10.1145/3342195.3387517 · Cited by 340
Self-citationCarbon-aware computing for datacenters
Radovanović, A., Koningstein, R., Schneider, I., Chen, B., Duarte, A., Roy, B., Xiao, D., Haridasan, M., Hung, P., Care, N., et al. (2023) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2022.3173250 · Cited by 353
“Compute jobs at Google are managed by a distributed clusterlevel operating system (known as Borg [28], [29]).”
Rzaḑca, K., Findeisen, P., Swiderski, J., Zych, P., Broniek, P., Kusmierek, J., Nowak, P. K., Strack, B., Witusowski, P., Hand, S., et al. (2020) · European Conference on Computer Systems · doi:10.1145/3342195.3387524 · Cited by 306
“Borg [34], Omega [29] and Kubernetes [3] all require their users to set such limits when submitting jobs (Borg, Omega) or pods (Kubernetes).”
BackgroundMethodSelf-citationWho limits the resource efficiency of my datacenter
Guo, J., Chang, Z., Wang, S., et al. (2019) · doi:10.1145/3326285.3329074 · Cited by 288
“A series of state-of-the-art CMSes like Borg [28] and Quasar [7] adopt an elaborate centralized scheduler for resource allocation and job placement.”
BackgroundVideoEdge: Processing Camera Streams using Hierarchical Clusters
Hung, C.-C., Ananthanarayanan, G., Bodik, P., Golubchik, L., Yu, M., Bahl, P., & Philipose, M. (2018) · 2018 IEEE/ACM Symposium on Edge Computing (SEC) · doi:10.1109/sec.2018.00016 · Cited by 287
“, Yarn [19], Mesos [34], Apollo [24], Borg [64]) has been an important research direction.”
BackgroundChen, S., Delimitrou, C., & Martínez, J. F. (2019) · International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) · doi:10.1145/3297858.3304005 · Cited by 269
“The irst approach is to simply disallow interactive services from sharing resources with other applications to avoid interference [42, 43, 51, 53].”
BackgroundSAND: Towards High-Performance Serverless Computing.
Akkuş, İ. E., Chen, R., Rimac, I., Stein, M., Satzke, K., Beck, A. T., Aditya, P., & Hilt, V. (2018) · Cited by 263
YouTube UGC Dataset for Video Compression Research
Wang, Y., Inguva, S., & Adsumilli, B. (2019) · 2019 IEEE 21st International Workshop on Multimedia Signal Processing (MMSP) · doi:10.1109/mmsp.2019.8901772 · Cited by 282
“LIVE datasets [10]-[ 12] are also quite popular.”
Kubernetes Scheduling: Taxonomy, Ongoing Issues and Challenges
Carrión, M. C. (2022) · ACM Computing Surveys · doi:10.1145/3539606 · Cited by 252
“It is the evolution of Google’s Borg [126] project, which had more than a decade of experience running scalable applications in production.”
BackgroundInfluentialAnalysis of Large-Scale Multi-Tenant GPU Clusters for DNN Training WorkloadsPreprint
Jeon, M., Venkataraman, S., Phanishayee, A., Qian, J., Xiao, W., & Yang, F. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.05758 · Cited by 241
“Similar to clusters setup for big data analysis [13, 48], using shared clusters can facilitate better utilization and reduce development overheads.”
BackgroundTiresias: A GPU Cluster Manager for Distributed Deep Learning
Gu, J., Chowdhury, M., Shin, K. G., et al. (2019) · Scholarworks@UNIST (Ulsan National Institute of Science and Technology) · Cited by 239
Sinan: ML-based and QoS-aware resource management for cloud microservices
Zhang, Y., Hua, W., Zhou, Z., Suh, G. E., & Delimitrou, C. (2021) · International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) · doi:10.1145/3445814.3446693 · Cited by 230
“Most cluster managers focus on CPU and memory management [14, 42, 51].”
BackgroundLegoOS: a disseminated, distributed OS for hardware resource disaggregation
Shan, Y., Huang, Y., Chen, Y., et al. (2018) · Operating Systems Design and Implementation · doi:10.5555/3291168.3291175 · Cited by 214
Serverless Computing: State-of-the-Art, Challenges and Opportunities
Li, Y., Lin, Y., Wang, Y., Ye, K., & Xu, C. (2022) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2022.3166553 · Cited by 218
“In comparison, for others, say Brog [135] and Quasar [42], they all could calculate the best parameters based on the requirements of users (e.g., running time) to manage the resources for the scheduled VM instances.”
BackgroundEfficient memory disaggregation with INFINISWAP
Gu, J., Lee, Y., Zhang, Y., Chowdhury, M., & Shin, K. G. (2017) · Networked Systems Design and Implementation · Cited by 209
Imbalance in the cloud: An analysis on Alibaba cluster trace
Lu, C., Ye, K., Xu, G., et al. (2017) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata.2017.8258257 · Cited by 199
“∗Corresponding author Co-locating online service and offline batch jobs on the same cluster is shown to be an efficient approach to improve cluster utilization in modern cloud data centers [14][15][2].”
BackgroundScalable Deep Learning on Distributed Infrastructures
Mayer, R., & Jacobsen, H. (2020) · ACM Computing Surveys · doi:10.1145/3363554 · Cited by 232
“There is a large variety of general purpose resource schedulers such as Mesos [64], YARN [178], and Borg [179].”
BackgroundPocket: elastic ephemeral storage for serverless analytics
Klimovic, A., Wang, Y., Stuedi, P., et al. (2018) · Operating Systems Design and Implementation · doi:10.5555/3291168.3291200 · Cited by 187
Optimized container scheduling for data-intensive serverless edge computing
Rausch, T., Rashed, A., & Dustdar, S. (2020) · Future Generation Computer Systems · doi:10.1016/j.future.2020.07.017 · Cited by 199
“To compare our system in non-edge computing scenarios, we also consider a cloud computing configuration where there are no edge devices and less heterogeneity than in edge scenarios [42].”
MethodMorpheus: towards automated SLOs for enterprise clusters
Jyothi, S. A., Curino, C., Menache, I., Narayanamurthy, S., Tumanov, A., Yaniv, J., Mavlyutov, R., Goiri, Í., Krishnan, S., Kulkarni, J., et al. (2016) · Operating Systems Design and Implementation · doi:10.5555/3026877.3026887 · Cited by 179
Sage: practical and scalable ML-driven performance debugging in microservices
Gan, Y., Liang, M., Dev, S., Lo, D., & Delimitrou, C. (2021) · International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) · doi:10.1145/3445814.3446700 · Cited by 177
Tumanov, A., Zhu, T., Park, J. W., Kozuch, M. A., Harchol‐Balter, M., & Ganger, G. R. (2016) · European Conference on Computer Systems · doi:10.1145/2901318.2901355 · Cited by 170
“An exception is Google’s Borg [39], which supports soft constraints on diverse resources, but lacks support for estimated runtimes or time-based SLOs.”
BackgroundFirmament: fast, centralized cluster scheduling at scale
Gog, I., Schwarzkopf, M., Gleave, A., Watson, R. N. M., & Hand, S. (2016) · Operating Systems Design and Implementation · doi:10.5555/3026877.3026886 · Cited by 167
Integrated NFV/SDN Architectures
Bonfim, M., Dias, K., & Fernandes, S. (2019) · ACM Computing Surveys · doi:10.1145/3172866 · Cited by 168
“Tools such as Google’s Borg [125], Omega [126], and Apache Mesos [127] may be considered for scheduling of VNFs.”
MethodCan far memory improve job throughput?
Amaro, E., Branner-Augmon, C., Luo, Z., Ousterhout, A., Aguilera, M. K., Panda, A., Ratnasamy, S., & Shenker, S. J. (2020) · European Conference on Computer Systems · doi:10.1145/3342195.3387522 · Cited by 168
“Our scheduler is simple and follows conventional designs [61], except when dealing with memory.”
BackgroundInfluentialA versatile data-intensive computing platform for information retrieval from big geospatial data
Soille, P., Burger, A., Marchi, D. D., Kempeneers, P., Rodriguez, D., Syrris, V., & Vasilev, V. E. (2017) · Future Generation Computer Systems · doi:10.1016/j.future.2017.11.007 · Cited by 185
Hawk: Hybrid Datacenter Scheduling
Delgado, P., Dinu, F., Kermarrec, A.-M., & Willy Zwaenepoel, Y. (2015) · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · Cited by 163
Slacker: fast distribution with lazy docker containers
Harter, T., Salmon, B., Liu, R., et al. (2016) · Cited by 157
Graphene: packing and dependency-aware scheduling for data-parallel clusters
Grandl, R., Kandula, S., Rao, S., Akella, A., & Kulkarni, J. (2016) · Operating Systems Design and Implementation · doi:10.5555/3026877.3026885 · Cited by 155
A Survey of Distributed Data Stream Processing Frameworks
Isah, H., Abughofa, T., Mahfuz, S., et al. (2019) · IEEE Access · doi:10.1109/access.2019.2946884 · Cited by 155
“Cutting-edge resource management tools include ZooKeeper [40], Yet Another Resource Negotiator (YARN), Mesos, Tupperware, andmanaged services such as Borg [39].”
BackgroundOtto, B., ten Hompel, M., & Wrobel, S. (2022) · Springer eBooks · doi:10.1007/978-3-030-93975-5 · Cited by 183
Prakash, C., Lee, J., Turner, Y. F., Kang, J., Akella, A., Banerjee, S., Clark, C., Ma, Y., Sharma, P. K., & Zhang, Y. (2015) · ACM Special Interest Group on Data Communication · doi:10.1145/2785956.2787506 · Cited by 155
“Logical labels have been advocated or supported in both networking [6, 23, 14] and nonnetworking [40, 9, 1] contexts.”
BackgroundDeep Learning-based Job Placement in Distributed Machine Learning Clusters
Bao, Y., Peng, Y., & Wu, C. (2019) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom.2019.8737460 · Cited by 153
“, consolidating workloads on the least number of machines) [5] [20].”
BackgroundMethodInfluentialFIRM: An Intelligent Fine-Grained Resource Management Framework for SLO-Oriented MicroservicesPreprint
Qiu, H., Banerjee, S. S., Jha, S., Kalbarczyk, Z., & Iyer, R. K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.08509 · Cited by 146
Venkataraman, S., Panda, A., Ousterhout, K., et al. (2017) · Symposium on Operating Systems Principles · doi:10.1145/3132747.3132750 · Cited by 145
“Systems like Borg [61], YARN [7] and Mesos [32] schedule jobs from different frameworks and implement resource sharing policies [28].”
BackgroundEmerging Trends, Techniques and Open Issues of Containerization: A Review
Watada, J., Roy, A., Kadikar, R., et al. (2019) · IEEE Access · doi:10.1109/access.2019.2945930 · Cited by 145
“Dupont et al. [108] have developed one such platform (called Cloud4IoT) for performing horizontal (roaming) and vertical (offloading) migration of IoT functions using a three-tier (cloud, edge and IoT gateways) Kubernetes cluster.”
BackgroundMethodInfluentialZhu, Y., Zhang, W., Chen, Y., & Gao, H. (2019) · EURASIP Journal on Wireless Communications and Networking · doi:10.1186/s13638-019-1605-z · Cited by 151
Machine Learning-based Orchestration of Containers: A Taxonomy and Future Directions
Zhong, Z., Xu, M., Rodriguez, M. A., Xu, C., & Buyya, R. (2022) · ACM Computing Surveys · doi:10.1145/3510415 · Cited by 143
“The relevant articles have been broadly searched in the mainstream academic sources, including ACM Digital Library, IEEEXplore, Springer, Elsevier, ScienceDirect, Wiley Interscience, and Google Scholar.”
BackgroundInfluentialAltruistic scheduling in multi-resource clusters
Grandl, R., Chowdhury, M., Akella, A., & Ananthanarayanan, G. (2016) · Operating Systems Design and Implementation · doi:10.5555/3026877.3026884 · Cited by 139
Mercury: hybrid centralized and distributed scheduling in large shared clusters
Karanasos, K., Rao, S., Curino, C., Douglas, C., Chaliparambil, K., Fumarola, G. M., Heddaya, S., Ramakrishnan, R., & Sakalanaga, S. (2015) · USENIX Annual Technical Conference · Cited by 130
Container‐based cluster orchestration systems: A taxonomy and future directions
Rodriguez, M. A., & Buyya, R. (2018) · Software Practice and Experience · doi:10.1002/spe.2660 · Cited by 130
Klimovic, A., Kozyrakis, C., Thereska, E., et al. (2016) · European Conference on Computer Systems · doi:10.1145/2901318.2901337 · Cited by 126
“This is a typical bin packing problem that all cluster managers address [16, 29, 62, 71].”
BackgroundSoftware-Defined Far Memory in Warehouse-Scale Computers
Lagar-Cavilla, A., Ahn, J., Souhlal, S., Agarwal, N., Burny, R., Butt, S., Chang, J., Chaugule, A., Deng, N., Shahid, J., et al. (2019) · International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) · doi:10.1145/3297858.3304053 · Cited by 125
“Our WSC control plane [40] offers an eviction SLO to users, which has never been breached in 18 months in production while we realized memory savings.”
BackgroundOnline Job Scheduling in Distributed Machine Learning Clusters
Bao, Y., Peng, Y., Wu, C., & Li, Z. (2018) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom.2018.8486422 · Cited by 128
“Most adopt Borg or YARN-like schedulers for ML cluster management [12][13].”
BackgroundMethodInfluentialDocker Cluster Management for the Cloud - Survey Results and Own Solution
Peinl, R., Holzschuher, F., & Pfitzer, F. (2016) · Journal of Grid Computing · doi:10.1007/s10723-016-9366-y · Cited by 121
“The need for such a solution first arose in large companies running hundreds and thousands of containers like Google [33].”
BackgroundCloud-Native Computing: A Survey From the Perspective of Services
Deng, S., Zhao, H., Huang, B., Zhang, C., Chen, F., Deng, Y., Yin, J., Dustdar, S., & Zomaya, A. Y. (2024) · Proceedings of the IEEE · doi:10.1109/jproc.2024.3353855 · Cited by 141
“K8s [9], originated from Google’s Borg cluster manager [10], is the most popular open-source container orchestration software.”
BackgroundMethodMobility-Aware and Delay-Sensitive Service Provisioning in Mobile Edge-Cloud Networks
Ma, Y., Liang, W., Li, J., et al. (2020) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2020.3006507 · Cited by 116
“Also, The implementation of this process consists of several stages [2], [29], such as resource availability check, file package building, testing and deploying, configu-ration setup, and dependency installations.”
MethodAI-Based Resource Provisioning of IoE Services in 6G: A Deep Reinforcement Learning Approach
Sami, H., Otrok, H., Bentahar, J., & Mourad, A. (2021) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2021.3066625 · Cited by 114
“A series of experiments using the Google Cluster Usage Trace dataset [26] are conducted.”
MethodA Survey on FPGA Virtualization
Vaishnav, A., Pham, K. D., & Koch, D. (2018) · International Conference on Field Programmable Logic and Applications (FPL) · doi:10.1109/fpl.2018.00031 · Cited by 114
“An example of this is the Borg cluster management system which Google uses internally to manage most of its clusters [84].”
MethodAutoFDO: automatic feedback-directed optimization for warehouse-scale applications
Chen, D., Li, X. D., & Moseley, T. (2016) · International Symposium on Code Generation and Optimization (CGO) · doi:10.1145/2854038.2854044 · Cited by 113
“Each machine in a Borg[28] cluster has a standard set of daemons installed to support monitoring and system man-14 agement.”
BackgroundScalable system scheduling for HPC and big data
Reuther, A., Byun, C., Arcand, W., et al. (2017) · Journal of Parallel and Distributed Computing · doi:10.1016/j.jpdc.2017.06.009 · Cited by 111
“Google Borg is a software container management system [Verma et al. (2015), Burns et al. (2016)] that evolved from the diversification of workloads at Google that expanded past the map-reduce paradigm of parallel computation. The default execution environment through Borg is Linux containers with static library linkin…”
MethodInfluentialDL2: A Deep Learning-Driven Scheduler for Deep Learning Clusters
Peng, Y., Bao, Y., Chen, Y., Wu, C., Meng, C., & Lin, W. (2021) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2021.3052895 · Cited by 120
“For example, Google uses Borg [8] as its DL cluster scheduler; Microsoft, Tencent, and Baidu use custom versions of YARN-like schedulers [9] for managing DL jobs.”
MethodA Brief History of Cloud Application Architectures
Kratzke, N. (2018) · Applied Sciences · doi:10.3390/app8081368 · Cited by 100
The Elasticity and Plasticity in Semi-Containerized Co-locating Cloud Workload
Liu, Q., & Yu, Z. (2018) · ACM Symposium on Cloud Computing (SoCC) · doi:10.1145/3267809.3267830 · Cited by 98
MEMTIS: Efficient Memory Tiering with Dynamic Page Classification and Page Size Determination
Lee, T., Monga, S. K., Min, C., & Eom, Y. I. (2023) · ACM SIGOPS Symposium on Operating Systems Principles (SOSP) · doi:10.1145/3600006.3613167 · Cited by 97
“In this case, its node agent, Borglet, and the cluster-level Borg scheduler [77] may evict applications to maintain the total hot working set size below the fast tier capacity on a machine (and thereby protect the performance SLOs of applications).”
BackgroundEfficient queue management for cluster scheduling
Rasley, J., Karanasos, K., Kandula, S., Fonseca, R., Vojnović, M., & Rao, S. (2016) · European Conference on Computer Systems · doi:10.1145/2901318.2901354 · Cited by 96
“Cluster schedulers such as YARN [28] and Borg [29] have a logically centralized service, often called the resource manager (RM), which serves as a matchmaker between the resource needs of various jobs and the available resources on worker machines.”
BackgroundMethodInfluentialCOSE: Configuring Serverless Functions using Statistical Learning
Akhtar, N., Raza, A., Ishakian, V., & Matta, I. (2020) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom41043.2020.9155363 · Cited by 96
“Cloud provider’s cluster managing systems, such as Google’s Borg [26], Mesos [27], Paragon [28] and Quasar [29], allow the user to specify the need for the application and the system finds the best con-figuration.”
MethodMali, S., Zeng, F., Adhikari, D., Ullah, I., Al‐Khasawneh, M. A., Alfarraj, O., & Alblehai, F. (2025) · Sensors · doi:10.3390/s25072197 · Cited by 96
“Emphasizing the significance of Borg cells in large-scale cluster management in [36,37], our project leverages Google Cluster Traces, a dataset replete with detailed information.”
NetMARKS: Network Metrics-AwaRe Kubernetes Scheduler Powered by Service Mesh
Wojciechowski, Ł., Opasiak, K., Latusek, J., Wereski, M., Morales, V., Kim, T., & Hong, M. (2021) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom42981.2021.9488670 · Cited by 95
“In 2014, Google decided to open source Kubernetes [2], a cluster manager for containers based on their internal system called Borg [3] that they used to run Google Applications like Gmail.”
MethodBlock-Diagonal and LT Codes for Distributed Computing With Straggling Servers
Severinson, A., Amat, A. G. I., & Rosnes, E. (2018) · IEEE Transactions on Communications · doi:10.1109/tcomm.2018.2877391 · Cited by 93
“These include transmission and queuing delays during initialization as well as contention for the local disk and slow-downs due to higher priority tasks being assigned to the same server [19].”
BackgroundOn the diversity of cluster workloads and its impact on research results
Amvrosiadis, G., Park, J. W., Ganger, G. R., Gibson, G. A., Baseman, E., & DeBardeleben, N. (2018) · USENIX Annual Technical Conference · Cited by 89
ENSURE: Efficient Scheduling and Autonomous Resource Management in Serverless Environments
Suresh, A., Somashekar, G., Varadarajan, A., Kakarla, V. R., Upadhyay, H., & Gandhi, A. (2020) · IEEE International Conference on Autonomic Computing and Self-Organizing Systems (ACSOS) · doi:10.1109/acsos49614.2020.00020 · Cited by 92
“While container schedulers, such as Borg, may appear to be well suited for serverless workloads, they are not necessarily designed for short-lived functions, and can have task placement latencies as high as 25s [23]; by”
BackgroundLoad Balancing Across Microservices
Niu, Y., Liu, F., & Li, Z. (2018) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom.2018.8486300 · Cited by 91
Yan, Y., Gao, Y., Chen, Y., et al. (2016) · ACM Symposium on Cloud Computing (SoCC) · doi:10.1145/2987550.2987576 · Cited by 90
“Given these extraordinary investments, it is worthwhile to note that even with the latest state-of-the-art cluster management and scheduling techniques (e.g. [16, 24, 25, 29, 32, 34]) the average resource utilization of servers in data centers is very low.”
BackgroundLiu, W., Yan, Y., Sun, Y., Mao, H., Cheng, M., Wang, P., & Ding, Z. (2023) · Applied Energy · doi:10.1016/j.apenergy.2023.120918 · Cited by 90
Self-managing cloud-native applications: Design, implementation, and experience
Toffetti, G., Brunner, S., Blöchlinger, M., et al. (2016) · Future Generation Computer Systems · doi:10.1016/j.future.2016.09.002 · Cited by 89
“Due to the economy of scale, infrastructure installations are large and typically built of commodity hardware so that failures are the norm rather than the exception [Verma et al. (2015)].”
BackgroundHarlap, A., Tumanov, A., Chung, A., Ganger, G. R., & Gibbons, P. B. (2017) · European Conference on Computer Systems · doi:10.1145/3064176.3064182 · Cited by 90
“But, these resources may subsequently be revoked if higher priority workloads intensify or additional jobs arrive [13, 33, 34].”
BackgroundResearch challenges in nextgen service orchestration
Vaquero, L. M., Cuadrado, F., Elkhatib, Y., Bernabé, J. B., Srirama, S. N., & Zhani, M. F. (2018) · Future Generation Computer Systems · doi:10.1016/j.future.2018.07.039 · Cited by 88
“Some works use delegated workloads that are then analysed and scored before their allocation [162].”
BackgroundTariq, A., Pahl, A., Nimmagadda, S., Rozner, E., & Lanka, S. (2020) · ACM Symposium on Cloud Computing (SoCC) · doi:10.1145/3419111.3421306 · Cited by 88
“Many architectures have been designed, from distributed [68, 75, 78, 78, 100] to centralized [48, 97, 98] techniques.”
BackgroundThemis: Fair and Efficient GPU Cluster SchedulingPreprint
Mahajan, K., Balasubramanian, A., Singhvi, A., Venkataraman, S., Akella, A., Phanishayee, A., & Chawla, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.01484 · Cited by 91
“We build on rich literature on cluster scheduling disciplines [8, 10–12] and two level schedulers [17, 31, 37].”
MethodState machine replication in containers managed by Kubernetes
Netto, H. V., Lung, L. C., Correia, M. P., Luiz, A. F., & de Souza, L. M. S. (2016) · Journal of Systems Architecture · doi:10.1016/j.sysarc.2016.12.007 · Cited by 84
Selecting subexpressions to materialize at datacenter scale
Jindal, A., Karanasos, K., Rao, S., et al. (2018) · Proceedings of the VLDB Endowment · doi:10.14778/3192965.3192971 · Cited by 82
Le, T. N., Sun, X., Chowdhury, M., & Liu, Z. (2020) · European Conference on Computer Systems · doi:10.1145/3342195.3387547 · Cited by 82
Cloud-based storage and computing for remote sensing big data: a technical review
Xu, C., Du, X., Fan, X., Giuliani, G., Hu, Z., Wang, W., Liu, J., Wang, T., Yan, Z., Zhu, J., et al. (2022) · International Journal of Digital Earth · doi:10.1080/17538947.2022.2115567 · Cited by 88
“Borg (Verma et al. 2015), the internal Google version of Kubernetes, is used for resource scheduling and load balancing within Earth Engine.”
MethodChung, A., Park, J. W., & Ganger, G. R. (2018) · ACM Symposium on Cloud Computing (SoCC) · doi:10.1145/3267809.3267819 · Cited by 78
“Job scheduling for clusters of computers has a rich history, with innovation still occurring as new systems address larger scale and emerging work patterns [16, 19, 22, 24, 31, 32, 41, 46, 55, 56].”
BackgroundA hybrid CNN-LSTM model for predicting server load in cloud computing
Patel, E., & Kushwaha, D. S. (2022) · The Journal of Supercomputing · doi:10.1007/s11227-021-04234-0 · Cited by 83
“On the other hand, Borg Cluster Management System at Google production center reported 60% and 85% usage of allocated CPU and memory, respectively [2], with further improvement in CPU utilization by 20–40% and in memory utilization by 3–30%, as compared to that in trace 2011 [3].”
BackgroundResultAn energy, performance efficient resource consolidation scheme for heterogeneous cloud datacenters
Khan, A. A., Zakarya, M., Khan, R., Rahman, I. U., Khan, M., & Khan, A. U. R. (2019) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2019.102497 · Cited by 77
The Datacenter as a Computer: Designing Warehouse-Scale Machines, Third Edition
Barroso, L. A., Hölzle, U., & Ranganathan, P. (2018) · Synthesis lectures on computer architecture · doi:10.2200/s00874ed3v01y201809cac046 · Cited by 93
Suresh, A., & Gandhi, A. (2019) · International Workshop on Serverless Computing (WoSC) · doi:10.1145/3366623.3368136 · Cited by 75
“…such as Borg, may appear to be well suited for serverless workloads, they are not necessarily designed for short-lived functions, and can have task placement latencies as high as 25s [17]; by contrast, serverless functions typically have latencies ranging from millisec-onds to few seconds [10, 13].”
BackgroundMelnik, S., Gubarev, A., Long, J. J., et al. (2020) · Proceedings of the VLDB Endowment · doi:10.14778/3415478.3415568 · Cited by 75
“Dremel was migrated to a cluster management system called Borg [45].”
MethodTowards an Adaptable Systems Architecture for Memory Tiering at Warehouse-Scale
Duraisamy, P., Xü, W., Hare, S., et al. (2023) · International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) · doi:10.1145/3582016.3582031 · Cited by 74
“Large scale cluster managers such as Mesos [25], Twine [47], YARN [49] and Borg [50] can schedule jobs across a large number of heterogeneous machines to optimize the efficiency of data centers.”
BackgroundPerfIso: Performance Isolation for Commercial Latency-Sensitive Services
Iorgulescu, C., Azimi, R., Kwon, Y., et al. (2018) · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · Cited by 73
Nebula: Distributed Edge Cloud for Data Intensive Computing
Jonathan, A., Ryden, M., Oh, K., Chandra, A., & Weissman, J. (2017) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2017.2717883 · Cited by 73
“The Resource Manager allows multiple applications from different ComputePools to run concurrently which is similar to those in the traditional centralized cloud [21], [22], [23], [24].”
MethodDynamic Resource Management for Efficient Utilization of Multitasking GPUs
Park, J. J. K., Park, Y., & Mahlke, S. A. (2017) · International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) · doi:10.1145/3037697.3037707 · Cited by 72
“Super-computers, cloud services, and data centers with GPUs will benefit significantly with shared GPUs because resource sharing is critical to efficient resource utilization in these environments [29].”
BackgroundSinghvi, A., Balasubramanian, A., Houck, K., et al. (2021) · ACM Symposium on Cloud Computing (SoCC) · doi:10.1145/3472883.3486981 · Cited by 72
“Borg [61] uses random sampling while calculating scores and thus trades o scheduling optimality for scalability.”
MethodPrakash, C., Lee, J., Turner, Y. F., Kang, J., Akella, A., Banerjee, S., Clark, C., Ma, Y., Sharma, P. K., & Zhang, Y. (2015) · ACM SIGCOMM Computer Communication Review · doi:10.1145/2829988.2787506 · Cited by 72
History-based harvesting of spare cycles and storage in large-scale datacenters
Zhang, Y., Prekas, G., Fumarola, G. M., da Fontoura, M. F. M. C., Goiri, Í., & Bianchini, R. (2016) · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · doi:10.5555/3026877.3026935 · Cited by 70
Shen, H., & Chen, L. (2017) · IEEE Transactions on Cloud Computing · doi:10.1109/tcc.2017.2737628 · Cited by 70
“In today’s cloud datacenter, VMs may generate transient workload spikes in PMs [26], which are sharp rises in the resource utilization that immediately followed by decreases.”
BackgroundSotiriadis, S., Bessis, N., Amza, C., & Buyya, R. (2016) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2016.2634024 · Cited by 70
Metadata
- Authors
- Verma, A., Pedrosa, L., Korupolu, M., Oppenheimer, D., Tune, E., & Wilkes, J.
- Affiliations
- Google (United States) (US)
- Venue
- Proceedings of the Tenth European Conference on Computer Systems
- Publisher
- ACM
- Published
- 2015-04-17
- Pages
- 1-17
- OpenAlex
- W2141992894
- Semantic Scholar
- 3a043714354fe498752b45e4cf429dbae0fb2558
- Cited by
- 1,468
- References
- 63
- Citations by year
- 2026: 154 · 2025: 139 · 2024: 110 · 2023: 115 · 2022: 114 · 2021: 156 · 2020: 129 · 2019: 152 · 2018: 141 · 2017: 114 · 2016: 119 · 2015: 25
- FWCI
- 115.57
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- European Conference on Computer Systems (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Information Systems › Cloud Computing and Resource Management
- Physical Sciences › Computer Science › Computer Networks and Communications › Distributed and Parallel Computing Systems
- Physical Sciences › Computer Science › Computer Networks and Communications › Distributed systems and fault tolerance
- Keywords
- cluster management, job scheduling, large-scale distributed systems, Google, cluster computing, resource management
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
