CoChat
Sign Up
Cited by 49Distributed systems and fault toleranceCloud Computing and Resource ManagementAdvanced Database Systems and Queries

Parity models: erasure-coded resilience for prediction serving systems

Open access
Kosaian, Jack, Rashmi, K. V., Venkataraman, Shivaram·2019·Proceedings of the 27th ACM Symposium on Operating Systems Principles
DOI 10.1145/3341301.3359654
Open this paper in CoChat — free accountOpen PDF

Abstract

Machine learning models are becoming the primary work-horses for many applications. Services deploy models through prediction serving systems that take in queries and return predictions by performing inference on models. Prediction serving systems are commonly run on many machines in cluster settings, and thus are prone to slowdowns and failures that inflate tail latency. Erasure coding is a popular technique for achieving resource-efficient resilience to data unavailability in storage and communication systems. However, existing approaches for imparting erasure-coded resilience to distributed computation apply only to a severely limited class of functions, precluding their use for many serving workloads, such as neural network inference.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3341301.3359654

Cites

77 of 77 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

4 of the 81 references couldn’t be matched to a known paper.

Research works printed but not linked: 21

  1. Agarwal, D., Long, B., Traupman, J., Xin, D., and Zhang, L. LASER: A Scalable Response Prediction Platform for Online Advertising. In Proceedings of the 7th ACM International Conference on Web Search and Data Mining (WSDM 14) (2014).
  2. Ananthanarayanan, G., Kandula, S., Greenberg, A. G., Stoica, I., Lu, Y., Saha, B., and Harris, E. Reining in the Outliers in Map-Reduce Clusters using Mantri. In 9th USENIX Symposium on Operating Systems Design and Implementation (OSDI 10) (2010).
  3. Aoudia, F. A., and Hoydis, J. End-to-End Learning of Com- munications Systems Without a Channel Model. arXiv preprint arXiv:1804.02276 (2018). · arXiv
  4. Chung, E., Fowers, J., Ovtcharov, K., Papamichael, M., Caulfield, A., Massengill, T., Liu, M., Lo, D., Alkalay, S., Haselman, M., et al. Serving DNNs in Real Time at Datacenter Scale with Project Brainwave. IEEE Micro 38, 2 (2018), 8-20.
  5. Crankshaw, D., Wang, X., Zhou, G., Franklin, M. J., Gonzalez, J. E., and Stoica, I. Clipper: A Low-Latency Online Prediction Serving System. In 14th USENIX Symposium on Networked Systems Design and Implementation (NSDI 17) (2017).
  6. Gujarati, A., Elnikety, S., He, Y., McKinley, K. S., and Branden- burg, B. B. Swayam: Distributed Autoscaling to Meet SLAs of Machine Learning Inference Services with Resource Efficiency. In Proceedings of the 18th ACM/IFIP/USENIX Middleware Conference (Middleware 17) (2017).
  7. Hao, M., Li, H., Tong, M. H., Pakha, C., Suminto, R. O., Stuardo, C. A., Chien, A. A., and Gunawi, H. S. MittOS: Supporting Millisecond Tail Tolerance with Fast Rejecting SLO-Aware OS Interface. In Proceedings of the 26th Symposium on Operating Systems Principles (SOSP 17) (2017).
  8. Hu, H., Dey, D., Bagnell, J. A., and Hebert, M. Learning Anytime Predictions in Neural Networks via Adaptive Loss Balancing. arXiv preprint arXiv:1708.06832 (2018). · arXiv
  9. Iorgulescu, C., Azimi, R., Kwon, Y., Elnikety, S., Syamala, M., Narasayya, V., Herodotou, H., Tomita, P., Chen, A., Zhang, J., and Wang, J. PerfIso: Performance Isolation for Commercial Latency- Sensitive Services. In 2018 USENIX Annual Technical Conference (USENIX ATC 18) (2018).
  10. Kosaian, J., Rashmi, K. V., and Venkataraman, S. Learning a Code: Machine Learning for Approximate Non-Linear Coded Computation. arXiv preprint arXiv:1806.01259 (2018). · arXiv
  11. Lee, Y., Scolari, A., Chun, B.-G., Santambrogio, M. D., Weimer, M., and Interlandi, M. PRETZEL: Opening the Black Box of Machine Learning Prediction Serving Systems. In 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18) (2018).
  12. Li, Z. L., Liang, C.-J. M., He, W., Zhu, L., Dai, W., Jiang, J., and Sun, G. Metis: Robustly Tuning Tail Latencies of Cloud Systems. In 2018 USENIX Annual Technical Conference (USENIX ATC 18) (2018).
  13. Mace, J., Bodik, P., Musuvathi, M., Fonseca, R., and Varadarajan, K. 2DFQ: Two-Dimensional Fair Queuing for Multi-Tenant Cloud Ser- vices. In Proceedings of the 2016 ACM SIGCOMM Conference (SIGCOMM 16) (2016).
  14. Mallick, A., Chaudhari, M., and Joshi, G. Rateless Codes for Near- Perfect Load Balancing in Distributed Matrix-Vector Multiplication. arXiv preprint arXiv:1804.10331 (2018). · arXiv
  15. Olston, C., Fiedel, N., Gorovoy, K., Harmsen, J., Lao, L., Li, F., Rajashekhar, V., Ramesh, S., and Soyke, J. TensorFlow-Serving: Flexible, High-Performance ML Serving. NIPS ML Systems Workshop (2017).
  16. Park, J., Naumov, M., Basu, P., Deng, S., Kalaiah, A., Khudia, D., Law, J., Malani, P., Malevich, A., Nadathur, S., et al. Deep Learning In- ference in Facebook Data Centers: Characterization, Performance Opti- mizations and Hardware Implications. arXiv preprint arXiv:1811.09886 (2018). · arXiv
  17. Reed, I. S., and Solomon, G. Polynomial Codes Over Certain Finite Fields. Journal of the society for industrial and applied mathematics 8, 2 (1960), 300-304.
  18. Wang, W., Gao, J., Zhang, M., Wang, S., Chen, G., Ng, T. K., Ooi, B. C., Shao, J., and Reyad, M. Rafiki: Machine Learning as an Analytics Service System. Proceedings of the VLDB Endowment 12, 2 (2018), 128-140.
  19. Warden, P. Speech commands: A Dataset for Limited-Vocabulary Speech Recognition. arXiv preprint arXiv:1804.03209 (2018). · arXiv
  20. Yadwadkar, N. J., Ananthanarayanan, G., and Katz, R. Wrangler: Predictable and Faster Jobs using Fewer Resources. In Proceedings of the ACM Symposium on Cloud Computing (SoCC 14) (2014).
  21. Yu, Q., Maddah-Ali, M., and Avestimehr, S. Polynomial Codes: An Optimal Design for High-Dimensional Coded Matrix Multiplication. In Advances in Neural Information Processing Systems (NIPS 17) (2017).

Cited by

Sorted by citations

Related

Authors
Kosaian, J., Rashmi, K. V., & Venkataraman, S.
Affiliations
  • Carnegie Mellon University (US)
  • University of Wisconsin–Madison (US)
ORCID
Venue
Proceedings of the 27th ACM Symposium on Operating Systems Principles
Publisher
ACM
Published
2019-10-27
Pages
30-46
DOI
10.1145/3341301.3359654
OpenAlex
W2981449041
Cited by
48
References
81
Citations by year
2026: 5 · 2025: 3 · 2024: 6 · 2023: 5 · 2022: 5 · 2021: 12 · 2020: 11 · 2019: 1
FWCI
6.27
Type
conference paper
Language
en
Open access
Licence
www.acm.org/publications/policies/copyright_policy#Background (vor)
Locations
ACM SIGOPS Symposium on Operating Systems Principles (SOSP) (publishedVersion)
Funders
  • Office of the Vice Chancellor for Research and Graduate Education at the University of Wisconsin
  • Wisconsin Alumni Research Foundation
  • National Science Foundation (DGE-1745016,DGE-1252522,CNS-1850483,CNS-1838733)
Topics
  • Physical Sciences › Computer Science › Computer Networks and Communications › Distributed systems and fault tolerance
  • Physical Sciences › Computer Science › Information Systems › Cloud Computing and Resource Management
  • Physical Sciences › Computer Science › Computer Networks and Communications › Advanced Database Systems and Queries
Keywords
erasure coding, distributed computing, tail latency, fault tolerance, neural network inference, distributed learning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review