Serving DNNs like Clockwork: Performance Predictability from the Bottom Up
Open accessThe DOI matches its OpenAlex record; 13 of the 13 references it cites were found in a registry or an index.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Squeeze-and-Excitation Networks
Hu, J., Shen, L., Albanie, S., Sun, G., & Wu, E. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2913372 · Cited by 12,157
Identity mappings in deep residual networks
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_38 · Cited by 10,741
Identity Mappings in Deep Residual Networks
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · arXiv · doi:10.48550/arxiv.1603.05027 · Cited by 10,665
Howard, A., Sandler, M., Chen, B., Wang, W., Chen, L.-C., Tan, M., Chu, G., Vasudevan, V., Zhu, Y., Pang, R., et al. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00140 · Cited by 9,708
Energy and Policy Considerations for Deep Learning in NLP
Strubell, E., Ganesh, A., & McCallum, A. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1355 · Cited by 3,684 · cited 1×
“SpecializedML hardware is necessary to achieve interactive latencies [34], but it is comparatively expensive to procure and operate, and must thus be used efficiently [50, 55].”
BackgroundDean, J., & Barroso, L. A. (2013) · Communications of the ACM · doi:10.1145/2408776.2408794 · Cited by 1,852 · cited 3×
“To tame it, the system designer can either seek to (quoting Dean and Barrosso [13]) “create a predictably responsive whole out of less-predictable parts”, or to expend significant effort to systematically unshroud and mitigate the performance variability of these underlying components.”
BackgroundResNeSt: Split-Attention Networks
Zhang, H., Wu, C., Zhang, Z., Zhu, Y., Lin, H., Zhang, Z., Sun, Y., He, T., Mueller, J., Manmatha, R., et al. (2020) · arXiv · doi:10.48550/arxiv.2004.08955 · Cited by 1,809 · cited 4×
“We compiled ResNet50v2 [78] with TVM 0.”
BackgroundInfluentialTVM: an automated end-to-end optimizing compiler for deep learning
(2018) · arXiv (Cornell University) · doi:10.5555/3291168.3291211 · Cited by 1,170
Cited by
Efficient Memory Management for Large Language Model Serving with PagedAttention
Kwon, W., Li, Z., Zhuang, S., Sheng, Y., Zheng, L., Yu, C. H., Gonzalez, J., Zhang, H., & Stoica, I. (2023) · Proceedings of the 29th Symposium on Operating Systems Principles · doi:10.1145/3600006.3613165 · Cited by 1,817
Splitwise: Efficient Generative LLM Inference Using Phase Splitting
Patel, P., Choukse, E., Zhang, C., Shah, A., Goiri, Í., Maleki, S., & Bianchini, R. (2024) · 2024 ACM/IEEE 51st Annual International Symposium on Computer Architecture (ISCA) · doi:10.1109/isca59077.2024.00019 · Cited by 298
“Since LLM requests are much longer than typical ML requests [37], [38], they incur lower scheduling overhead for similar cluster sizes.”
BackgroundDemystifying TensorRT: Characterizing Neural Network Inference Engine on Nvidia Edge Devices
Shafi, O., Rai, C., Sen, R., & Ananthanarayanan, G. (2021) · 2021 IEEE International Symposium on Workload Characterization (IISWC) · doi:10.1109/iiswc53511.2021.00030 · Cited by 51
Helix: Serving Large Language Models over Heterogeneous GPUs and Network via Max-Flow
Mei, Y., Zhuang, Y., Miao, X., Yang, J., Jia, Z., & Vinayak, R. (2025) · Proceedings of the 30th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 1 · doi:10.1145/3669940.3707215 · Cited by 35
“In this evaluation, we run Helix to find the optimal model placement for serving LLaMA 30B in a cluster with 4 L4 and 6 T4 machines.”
Cache-Craft: Managing Chunk-Caches for Efficient Retrieval-Augmented Generation
Agarwal, S., Sundaresan, S., Mitra, S., Mahapatra, D., Gupta, A., Sharma, R., Kapu, N. J., Yu, T., & Saini, S. (2025) · Proceedings of the ACM on Management of Data · doi:10.1145/3725273 · Cited by 21
Towards End-to-End Optimization of LLM-based Applications with Ayo
Tan, X., Jiang, Y., Yang, Y., & Xu, H. (2025) · Proceedings of the 30th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2 · doi:10.1145/3676641.3716278 · Cited by 13
“When applications are co-located in the same infrastructure [35, 78], they are orchestrated by Ayo independently.”
Metadata
- Authors
- Gujarati, A., Karimi, R. J., Alzayat, S., Hao, W., Kaufmann, A., Vigfússon, Ý., & Mace, J.
- Venue
- arXiv (Cornell University)
- Publisher
- Cornell University
- Published
- 2020-06-03
- Pages
- 443-462
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2006.02464
- OpenAlex
- W3033527233
- Cited by
- 70
- References
- 13
- Type
- posted content
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.