Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Cited by 232

The DOI matches its Crossref record; 7 of the 7 references it cites were found in a registry or an index.

Nexus: a GPU cluster engine for accelerating DNN-based video analysis

Open access
Shen, Haichen, Chen, Lequn, Jin, Yuchen, Zhao, Liangyu, Kong, Bingyu, Philipose, Matthai, Krishnamurthy, Arvind, Sundaram, Ravi · 2019 · Proceedings of the 27th ACM Symposium on Operating Systems Principles

Abstract

We address the problem of serving Deep Neural Networks (DNNs) efficiently from a cluster of GPUs. In order to realize the promise of very low-cost processing made by accelerators such as GPUs, it is essential to run them at sustained high utilization. Doing so requires cluster-scale resource management that performs detailed scheduling of GPUs, reasoning about groups of DNN invocations that need to be co-scheduled, and moving from the conventional whole-DNN execution model to executing fragments of DNNs. Nexus is a fully implemented system that includes these innovations. In large-scale case studies on 16 GPUs, when required to stay within latency constraints at least 99% of the time, Nexus can process requests at rates 1.8-12.7X higher than state of the art systems can. A long-running multi-application deployment stays within 84% of optimal utilization and, on a 100-GPU cluster, violates latency SLOs on 0.27% of requests.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3341301.3359658

Cites

7 of 7 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 7 references

Research works printed but not linked: 27

  1. Martín Abadi , Paul Barham , Jianmin Chen , Zhifeng Chen , Andy Davis , Jeffrey Dean , Matthieu Devin , Sanjay Ghemawat , Geoffrey Irving , Michael Isard , Manjunath Kudlur , Josh Levenberg , Rajat Monga , Sherry Moore , Derek Gordon Murray , Benoit Steiner , Paul A. Tucker , Vijay Vasudevan , Pete Warden , Martin Wicke , Yuan Yu , and Xiaoqiang Zheng . 2016 . TensorFlow: A System for Large-Scale Machine Learning. In 12th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2016 , Savannah, GA, USA, November 2--4 , 2016. USENIX Association, 265--283. Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, Manjunath Kudlur, Josh Levenberg, Rajat Monga, Sherry Moore, Derek Gordon Murray, Benoit Steiner, Paul A. Tucker, Vijay Vasudevan, Pete Warden, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. 2016. TensorFlow: A System for Large-Scale Machine Learning. In 12th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2016, Savannah, GA, USA, November 2--4, 2016. USENIX Association, 265--283.
  2. 10.1016/j.jocs.2016.12.009 · DOI
  3. Atul Adya , Daniel Myers , Jon Howell , Jeremy Elson , Colin Meek , Vishesh Khemani , Stefan Fulger , Pan Gu , Lakshminath Bhuvanagiri , Jason Hunter , Roberto Peon , Larry Kai , Alexander Shraer , Arif Merchant , and Kfir Lev-Ari . 2016 . Slicer: Auto-Sharding for Datacenter Applications. In 12th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2016 , Savannah, GA, USA, November 2--4 , 2016. USENIX Association, 739--753. Atul Adya, Daniel Myers, Jon Howell, Jeremy Elson, Colin Meek, Vishesh Khemani, Stefan Fulger, Pan Gu, Lakshminath Bhuvanagiri, Jason Hunter, Roberto Peon, Larry Kai, Alexander Shraer, Arif Merchant, and Kfir Lev-Ari. 2016. Slicer: Auto-Sharding for Datacenter Applications. In 12th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2016, Savannah, GA, USA, November 2--4, 2016. USENIX Association, 739--753.
  4. Daniel Crankshaw , Xin Wang , Giulio Zhou , Michael J. Franklin , Joseph E. Gonzalez , and Ion Stoica . 2017 . Clipper: A Low-Latency Online Prediction Serving System. In 14th USENIX Symposium on Networked Systems Design and Implementation, NSDI 2017 , Boston, MA, USA, March 27--29 , 2017. USENIX Association, 613--627. Daniel Crankshaw, Xin Wang, Giulio Zhou, Michael J. Franklin, Joseph E. Gonzalez, and Ion Stoica. 2017. Clipper: A Low-Latency Online Prediction Serving System. In 14th USENIX Symposium on Networked Systems Design and Implementation, NSDI 2017, Boston, MA, USA, March 27--29, 2017. USENIX Association, 613--627.
  5. Michael R. Garey and David S . Johnson . 1979 . Computers and Intractability: A Guide to the Theory of NP-Completeness. W. H. Freeman & Co. , New York, NY, USA. Michael R. Garey and David S. Johnson. 1979. Computers and Intractability: A Guide to the Theory of NP-Completeness. W. H. Freeman & Co., New York, NY, USA.
  6. Google. 2014 . Kubernetes: Production-Grade Container Orchestration. https://kubernetes.io/ Google. 2014. Kubernetes: Production-Grade Container Orchestration. https://kubernetes.io/
  7. Juncheng Gu , Mosharaf Chowdhury , Kang G Shin , Yibo Zhu , Myeongjae Jeon , Junjie Qian , Hongqiang Liu , and Chuanxiong Guo . 2019 . Tiresias: A GPU Cluster Manager for Distributed Deep Learning. In 16th USENIX Symposium on Networked Systems Design and Implementation (NSDI 19) . 485--500. Juncheng Gu, Mosharaf Chowdhury, Kang G Shin, Yibo Zhu, Myeongjae Jeon, Junjie Qian, Hongqiang Liu, and Chuanxiong Guo. 2019. Tiresias: A GPU Cluster Manager for Distributed Deep Learning. In 16th USENIX Symposium on Networked Systems Design and Implementation (NSDI 19). 485--500.
  8. 10.1145/2906388.2906396 · DOI · Paper page
  9. Benjamin Hindman , Andy Konwinski , Matei Zaharia , Ali Ghodsi , Anthony D. Joseph , Randy H. Katz , Scott Shenker , and Ion Stoica . 2011 . Mesos: A Platform for Fine-Grained Resource Sharing in the Data Center . In Proceedings of the 8th USENIX Symposium on Networked Systems Design and Implementation, NSDI 2011 , Boston, MA, USA, March 30 - April 1, 2011. USENIX Association. Benjamin Hindman, Andy Konwinski, Matei Zaharia, Ali Ghodsi, Anthony D.Joseph, Randy H. Katz, Scott Shenker, and Ion Stoica. 2011. Mesos: A Platform for Fine-Grained Resource Sharing in the Data Center. In Proceedings of the 8th USENIX Symposium on Networked Systems Design and Implementation, NSDI 2011, Boston, MA, USA, March 30 - April 1, 2011. USENIX Association.
  10. Kevin Hsieh , Ganesh Ananthanarayanan , Peter Bodík , Shivaram Venkataraman , Paramvir Bahl , Matthai Philipose , Phillip B. Gibbons , and Onur Mutlu . 2018 . Focus: Querying Large Video Datasets with Low Latency and Low Cost. In 13th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2018 , Carlsbad, CA, USA, October 8--10 , 2018. USENIX Association, 269--286. Kevin Hsieh, Ganesh Ananthanarayanan, Peter Bodík, Shivaram Venkataraman, Paramvir Bahl, Matthai Philipose, Phillip B. Gibbons, and Onur Mutlu. 2018. Focus: Querying Large Video Datasets with Low Latency and Low Cost. In 13th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2018, Carlsbad, CA, USA, October 8--10, 2018. USENIX Association, 269--286.
  11. 10.1145/2647868.2654889 · DOI · Paper page
  12. Angela H Jiang , Daniel L-K Wong , Christopher Canel , Lilia Tang , Ishan Misra , Michael Kaminsky , Michael A Kozuch , Padmanabhan Pillai , David G Andersen , and Gregory R Ganger . 2018 . Mainstream: Dynamic stem-sharing for multi-tenant video processing . In 2018 USENIX Annual Technical Conference (USENIX ATC 18) . 29--42. Angela H Jiang, Daniel L-K Wong, Christopher Canel, Lilia Tang, Ishan Misra, Michael Kaminsky, Michael A Kozuch, Padmanabhan Pillai, David G Andersen, and Gregory R Ganger. 2018. Mainstream: Dynamic stem-sharing for multi-tenant video processing. In 2018 USENIX Annual Technical Conference (USENIX ATC 18). 29--42.
  13. 10.1007/978-3-540-71844-4_18 · doi:10.1007/978-3-540-71844-4_18
  14. Christopher Olston , Noah Fiedel , Kiril Gorovoy , Jeremiah Harmsen , Li Lao , Fangwei Li , Vinu Rajashekhar , Sukriti Ramesh , and Jordan Soyke . 2017 . Tensorflow-serving: Flexible, high-performance ml serving. arXiv preprint arXiv:1712.06139 (2017). Christopher Olston, Noah Fiedel, Kiril Gorovoy, Jeremiah Harmsen, Li Lao, Fangwei Li, Vinu Rajashekhar, Sukriti Ramesh, and Jordan Soyke. 2017. Tensorflow-serving: Flexible, high-performance ml serving. arXiv preprint arXiv:1712.06139 (2017). · arXiv
  15. 10.1145/2517349.2522716 · DOI · Paper page
  16. Vivek S Pai , Peter Druschel , and Willy Zwaenepoel . 1999 . Flash: An efficient and portable Web server . In USENIX Annual Technical Conference, General Track. 199--212 . Vivek S Pai, Peter Druschel, and Willy Zwaenepoel. 1999. Flash: An efficient and portable Web server. In USENIX Annual Technical Conference, General Track. 199--212.
  17. 10.5244/C29.41 · doi:10.5244/c29.41
  18. 10.1145/3190508.3190517 · DOI · Paper page
  19. 10.1007/978-3-319-46493-0_32 · DOI · Paper page
  20. 10.1145/2465351.2465386 · DOI · Paper page
  21. 10.1109/CVPRW.2014.131 · DOI · Paper page
  22. 10.1109/CVPR.2017.236 · DOI
  23. Karen Simonyan and Andrew Zisserman. 2014. Two-stream convolutional networks for action recognition in videos. In Advances in neural information processing systems. 568--576. Karen Simonyan and Andrew Zisserman. 2014. Two-stream convolutional networks for action recognition in videos. In Advances in neural information processing systems. 568--576.
  24. Wencong Xiao , Romil Bhardwaj , Ramachandran Ramjee , Muthian Sivathanu , Nipun Kwatra , Zhenhua Han , Pratyush Patel , Xuan Peng , Hanyu Zhao , Quanlu Zhang , Fan Yang , and Lidong Zhou . 2018 . Gandiva: Introspective Cluster Scheduling for Deep Learning. In 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18) . USENIX Association, Carlsbad, CA, 595--610. Wencong Xiao, Romil Bhardwaj, Ramachandran Ramjee, Muthian Sivathanu, Nipun Kwatra, Zhenhua Han, Pratyush Patel, Xuan Peng, Hanyu Zhao, Quanlu Zhang, Fan Yang, and Lidong Zhou. 2018. Gandiva: Introspective Cluster Scheduling for Deep Learning. In 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18). USENIX Association, Carlsbad, CA, 595--610.
  25. 10.1109/CVPR.2015.7299023 · DOI · Paper page
  26. 10.1109/ICASSP.2012.6288897 · DOI
  27. Haoyu Zhang , Ganesh Ananthanarayanan , Peter Bodik , Matthai Philipose , Paramvir Bahl , and Michael J Freedman . 2017 . Live video analytics at scale with approximation and delay-tolerance . In 14th USENIX Symposium on Networked Systems Design and Implementation (NSDI 17) . 377--392. Haoyu Zhang, Ganesh Ananthanarayanan, Peter Bodik, Matthai Philipose, Paramvir Bahl, and Michael J Freedman. 2017. Live video analytics at scale with approximation and delay-tolerance. In 14th USENIX Symposium on Networked Systems Design and Implementation (NSDI 17). 377--392.

Cited by

See all 232

Metadata

Authors
Shen, H., Chen, L., Jin, Y., Zhao, L., Kong, B., Philipose, M., Krishnamurthy, A., & Sundaram, R.
Venue
Proceedings of the 27th ACM Symposium on Operating Systems Principles
Publisher
ACM
Published
2019-10-27
Pages
322-337
OpenAlex
W2982157693
Cited by
232
References
7
FWCI
14.10
Type
conference paper
Language
en
Open access
gold
Funders
National Science Foundation (CNS-1614717)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Nexus: a GPU cluster engine for accelerating DNN-based video analysis. https://app.cochat.ai/papers/10.1145/3341301.3359658
The paper
Shen, H., Chen, L., Jin, Y., Zhao, L., Kong, B., Philipose, M., Krishnamurthy, A., & Sundaram, R. (2019). Nexus: a GPU cluster engine for accelerating DNN-based video analysis. Proceedings of the 27th ACM Symposium on Operating Systems Principles. https://doi.org/10.1145/3341301.3359658

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.