The DOI matches its Crossref record; 46 of the 46 references it cites were found in a registry or an index.
Optimus: an efficient dynamic resource scheduler for deep learning clusters
Abstract
Deep learning workloads are common in today's production clusters due to the proliferation of deep learning driven AI services (e.g., speech recognition, machine translation). A deep learning training job is resource-intensive and time-consuming. Efficient resource scheduling is the key to the maximal performance of a deep learning cluster. Existing cluster schedulers are largely not tailored to deep learning jobs, and typically specifying a fixed amount of resources for each job, prohibiting high resource efficiency and job performance. This paper proposes Optimus, a customized job scheduler for deep learning clusters, which minimizes job training time based on online resource-performance models. Optimus uses online fitting to predict model convergence during training, and sets up performance models to accurately estimate training speed as a function of allocated resources in each job. Based on the models, a simple yet effective method is designed and used for dynamically allocating resources and placing deep learning tasks to minimize job completion time. We implement Optimus on top of Kubernetes, a cluster manager for container orchestration, and experiment on a deep learning cluster with 7 CPU servers and 6 GPU servers, running 9 training jobs using the MXNet framework. Results show that Optimus outperforms representative cluster schedulers by about 139% and 63% in terms of job completion time and makespan, respectively.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Rethinking the Inception Architecture for Computer Vision
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.308 · Cited by 32,138
Convolutional Neural Networks for Sentence Classification
Kim, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1181 · Cited by 14,831
Aggregated Residual Transformations for Deep Neural Networks
Xie, S., Girshick, R., Dollar, P., Tu, Z., & He, K. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.634 · Cited by 12,465
TensorFlow: A system for large-scale machine learning
Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.08695 · Cited by 11,822
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
Wu, Y., Schuster, M., Chen, Z., Le, Q. V., Norouzi, M., Macherey, W., Krikun, M., Cao, Y., Gao, Q., Macherey, K., et al. (2016) · arXiv · doi:10.48550/arxiv.1609.08144 · Cited by 5,319
Resilient distributed datasets: a fault-tolerant abstraction for in-memory cluster computing
Zaharia, M., Chowdhury, M., Das, T., Dave, A., Ma, J. H., McCauley, M., Franklin, M. J. M., Shenker, S. J., & Stoica, I. G. (2012) · UC Berkeley · Cited by 4,891
Large Scale Distributed Deep Networks
Dean, J. B., Corrado, G. S., Monga, R., Chen, K., Devin, M., Mao, M., Ranzato, M., Senior, A., Tucker, P. A., Yang, K., et al. (2012) · neural information processing systems · Cited by 4,489
Convolutional Sequence to Sequence Learning
Gehring, J., Auli, M., Grangier, D., Yarats, D., & Dauphin, Y. N. (2017) · arXiv · doi:10.48550/arxiv.1705.03122 · Cited by 2,850 · cited 1×
“An example of model fitting when training the Seq2Seq model [33] in Table 1 is given in Fig.”
MethodAccurate, Large Minibatch SGD: Training ImageNet in 1 Hour
Goyal, P., Dollár, P., Girshick, R., Noordhuis, P., Wesolowski, L., Kyrola, A., Tulloch, A., Jia, Y., & He, K. (2017) · arXiv · doi:10.48550/arxiv.1706.02677 · Cited by 2,294 · cited 1×
“This guarantees that the same training result (model) can be achieved while varying the number of workers [36].”
BackgroundDeep Speech 2: End-to-End Speech Recognition in English and Mandarin
Amodei, D., Anubhai, R., Battenberg, E., Case, C. T., Casper, J., Catanzaro, B., Chen, J., Chrzanowski, M., Coates, A., Diamos, G., et al. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1512.02595 · Cited by 2,110
Apache Hadoop YARN: yet another resource negotiator
Vavilapalli, V. K., Murthy, A. C., Douglas, C., Agarwal, S., Konar, M., Evans, R., Graves, T., Lowe, J., Shah, H., Seth, S., et al. (2013) · Proceedings of the 4th annual Symposium on Cloud Computing · doi:10.1145/2523616.2523633 · Cited by 1,848
MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems
Chen, T., Li, M., Li, Y., Lin, M., Wang, N., Wang, M., Xiao, T., Xu, B., Zhang, C., & Zhang, Z. (2015) · arXiv · doi:10.48550/arxiv.1512.01274 · Cited by 1,819
(2011) · doi:10.5555/1972457.1972488 · Cited by 1,619
Large-scale cluster management at Google with Borg
Verma, A., Pedrosa, L., Korupolu, M., Oppenheimer, D., Tune, E., & Wilkes, J. (2015) · Proceedings of the Tenth European Conference on Computer Systems · doi:10.1145/2741948.2741964 · Cited by 1,468 · cited 2×
“, Google uses Borg [63], Microsoft, Tencent and Baidu use YARN-like schedulers [61]) allocate a fixed amount of resources to each job upon its submission, according to resource requirements specified by the job owner.”
BackgroundMethodDominant resource fairness: fair allocation of multiple resource types
Ghodsi, A., Zaharia, M., Hindman, B., Konwinski, A., Shenker, S. J., & Stoica, I. G. (2011) · doi:10.5555/1972457.1972490 · Cited by 1,049
Scaling distributed machine learning with the parameter server
Li, M., Andersen, D. G., Park, J. W., Smola, A. J., Ahmed, A. A., Josifovski, V., Long, J. E., Shekita, E. J., & Su, B.-Y. (2014) · Operating Systems Design and Implementation · doi:10.5555/2685048.2685095 · Cited by 1,021
A Latent Semantic Model with Convolutional-Pooling Structure for Information Retrieval
Shen, Y., He, X., Gao, J., Deng, L., & Mesnil, G. (2014) · Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management · doi:10.1145/2661829.2661935 · Cited by 661
Ernest: efficient performance prediction for large-scale advanced analytics
Venkataraman, S., Yang, Z., Franklin, M. J. M., Recht, B., & Stoica, I. G. (2016) · Networked Systems Design and Implementation · Cited by 559
Revisiting Distributed Synchronous SGD
Pan, X., Chen, J., Monga, R., Bengio, S., & Józefowicz, R. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.05800 · Cited by 548
Multi-resource packing for cluster schedulers
Grandl, R., Ananthanarayanan, G., Kandula, S., Rao, S., & Akella, A. (2014) · Proceedings of the 2014 ACM conference on SIGCOMM · doi:10.1145/2619239.2626334 · Cited by 421
An architecture for parallel topic models
Smola, A., & Narayanamurthy, S. (2010) · Proceedings of the VLDB Endowment · doi:10.14778/1920841.1920931 · Cited by 399
Cherrypick: adaptively unearthing the best cloud configurations for big data analytics
Alipourfard, O., Liu, H. H., Chen, J., Venkataraman, S., Yu, M., & Zhang, M. (2017) · Networked Systems Design and Implementation · Cited by 374
The microsoft 2016 conversational speech recognition system
Xiong, W., Droppo, J., Huang, X., Seide, F., Seltzer, M., Stolcke, A., Yu, D., & Zweig, G. (2017) · 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2017.7953159 · Cited by 326
Ketkar, N. (2017) · Deep Learning with Python · doi:10.1007/978-1-4842-2766-4_8 · Cited by 319
Device Placement Optimization with Reinforcement Learning
Mirhoseini, A., Pham, H., Le, Q. V., Steiner, B., Larsen, R., Zhou, Y., Kumar, N., Norouzi, M., Bengio, S., & Dean, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.04972 · Cited by 299
Jockey: guaranteed job latency in data parallel clusters
Ferguson, A. D., Bodik, P., Kandula, S., Boutin, E., & Fonseca, R. (2012) · Proceedings of the 7th ACM european conference on Computer Systems · doi:10.1145/2168836.2168847 · Cited by 286
Firmament: fast, centralized cluster scheduling at scale
Gog, I., Schwarzkopf, M., Gleave, A., Watson, R. N. M., & Hand, S. (2016) · Operating Systems Design and Implementation · doi:10.5555/3026877.3026886 · Cited by 277
Fuxi: a fault-tolerant resource management and job scheduling system at internet scale
Zhang, Z., Li, C., Tao, Y., Yang, R., Tang, H., & Xu, J. (2014) · Proceedings of the VLDB Endowment · doi:10.14778/2733004.2733012 · Cited by 182
TetriSched: global rescheduling with adaptive plan-ahead in dynamic heterogeneous clusters
Tumanov, A., Zhu, T., Park, J. W., Kozuch, M. A., Harchol-Balter, M., & Ganger, G. R. (2016) · Proceedings of the Eleventh European Conference on Computer Systems · doi:10.1145/2901318.2901355 · Cited by 170
SLAQ: quality-driven scheduling for distributed machine learning
Zhang, H., Stafman, L., Or, A., & Freedman, M. J. (2017) · Symposium on Cloud Computing · doi:10.1145/3127479.3127490 · Cited by 168
Automating model search for large scale machine learning
Sparks, E., Talwalkar, A., Haas, D., Franklin, M., Jordan, M. I., & Kraska, T. (2015) · ACM Symposium on Cloud Computing · doi:10.1145/2806777.2806945 · Cited by 165
Network-Aware Scheduling for Data-Parallel Jobs
Jalaparti, V., Bodík, P., Menache, I., Rao, S., Makarychev, K., & Caesar, M. (2015) · ACM Special Interest Group on Data Communication · doi:10.1145/2785956.2787488 · Cited by 157
Petuum: A New Platform for Distributed Machine Learning on Big Data
Xing, E. P., Ho, Q., Dai, W., Kim, J.-K., Wei, J., Lee, S., Zheng, X., Xie, P., Kumar, A., & Yu, Y. (2015) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2783258.2783323 · Cited by 153
Job-aware Scheduling in Eagle: Divide and Stick to Your Probes
Delgado, P. O., Didona, D., Dinu, F., & Zwaenepoel, W. (2016) · ACM Symposium on Cloud Computing (SoCC) · doi:10.1145/2987550.2987563 · Cited by 118
Proteus: agile ML elasticity through tiered reliability in dynamic resource markets
Harlap, A., Tumanov, A., Chung, A., Ganger, G. R., & Gibbons, P. B. (2017) · European Conference on Computer Systems · doi:10.1145/3064176.3064182 · Cited by 118
Performance Modeling and Scalability Optimization of Distributed Deep Learning Systems
Yan, F., Ruwase, O., He, Y., & Chilimbi, T. M. (2015) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2783258.2783270 · Cited by 116
BOAT: Building Auto-Tuners with Structured Bayesian Optimization
Dalibard, V., Schaarschmidt, M., & Yoneki, E. (2017) · International World Wide Web Conference (WWW) · doi:10.1145/3038912.3052662 · Cited by 98
STRADS: a distributed framework for scheduled model parallel machine learning
Kim, J. K., Ho, Q., Lee, S., Zheng, X., Dai, W., Gibson, G. A., & Xing, E. P. (2016) · European Conference on Computer Systems · doi:10.1145/2901318.2901331 · Cited by 85
Angel: a new large-scale machine learning system
Jiang, J., Yu, L., Jiang, J., Liu, Y., & Cui, B. (2017) · National Science Review · doi:10.1093/nsr/nwx018 · Cited by 80
PerfOrator: eloquent performance models for Resource Optimization
Rajan, K., Kakadia, D., Curino, C., & Krishnan, S. (2016) · ACM Symposium on Cloud Computing (SoCC) · doi:10.1145/2987550.2987566 · Cited by 76
Resource Elasticity for Large-Scale Machine Learning
Huang, B., Böehm, M., Tian, Y., Reinwald, B., Tatikonda, S., & Reiss, F. R. (2015) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/2723372.2749432 · Cited by 66
Popescu, A. D., Balmin, A., Ercegovac, V., & Ailamaki, A. (2013) · Proceedings of the VLDB Endowment · doi:10.14778/2556549.2556553 · Cited by 58
Towards Distributed Machine Learning in Shared Clusters: A Dynamically-Partitioned Approach
Sun, P., Wen, Y., Ta, N. B. D., & Yan, S. (2017) · IEEE International Conference on Smart Computing · doi:10.1109/smartcomp.2017.7947053 · Cited by 41
Towards distributed machine learning in shared clusters: A dynamically-partitioned approach
Sun, P., Wen, Y., Nguyen Binh Duong, T. A., et al. (2017) · Singapore Management University Institutional Knowledge (InK) (Singapore Management University) · Cited by 34
Cited by
A Learning-Based Incentive Mechanism for Federated Learning
Zhan, Y., Li, P., Qu, Z., Zeng, D., & Guo, S. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.2967772 · Cited by 602
A generic communication scheduler for distributed DNN training acceleration
Peng, Y., Zhu, Y., Chen, Y., Bao, Y., Yi, B., Lan, C., Wu, C., & Guo, C. (2019) · Proceedings of the 27th ACM Symposium on Operating Systems Principles · doi:10.1145/3341301.3359642 · Cited by 321
Self-citationLazy Batching: An SLA-aware Batching System for Cloud Machine Learning Inference
Choi, Y., Kim, Y., & Rhu, M. (2021) · 2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca51647.2021.00049 · Cited by 69
LoongServe: Efficiently Serving Long-Context Large Language Models with Elastic Sequence Parallelism
Wu, B., Liu, S., Zhong, Y., Sun, P., Liu, X., & Jin, X. (2024) · Proceedings of the ACM SIGOPS 30th Symposium on Operating Systems Principles · doi:10.1145/3694715.3695948 · Cited by 62
Federated Threat-Hunting Approach for Microservice-Based Industrial Cyber-Physical System
Abdel-Basset, M., Hawash, H., & Sallam, K. (2022) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2021.3091150 · Cited by 47
Metadata
- Authors
- Peng, Y., Bao, Y., Chen, Y., Wu, C., & Guo, C.
- Venue
- Proceedings of the Thirteenth EuroSys Conference
- Publisher
- ACM
- Published
- 2018-04-23
- Pages
- 1-14
- OpenAlex
- W2798515322
- Cited by
- 409
- References
- 46
- FWCI
- 34.08
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.