Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 32 of the 35 references it cites were found in a registry or an index.

Sifter: Scalable Sampling for Distributed Traces, without Feature Engineering

Open access
Las-Casas, Pedro, Papakerashvili, Giorgi, Anand, Vaastav, Mace, Jonathan · 2019 · Proceedings of the ACM Symposium on Cloud Computing

Abstract

Distributed tracing is a core component of cloud and datacenter systems, and provides visibility into their end-to-end runtime behavior. To reduce computational and storage overheads, most tracing frameworks do not keep all traces, but sample them uniformly at random. While effective at reducing overheads, uniform random sampling inevitably captures redundant, common-case execution traces, which are less useful for analysis and troubleshooting tasks. In this work we present Sifter, a general-purpose framework for biased trace sampling. Sifter captures qualitatively more diverse traces, by weighting sampling decisions towards edge-case code paths, infrequent request types, and anomalous events. Sifter does so by using the incoming stream of traces to build an unbiased low-dimensional model that approximates the system's common-case behavior. Sifter then biases sampling decisions towards traces that are poorly captured by this model. We have implemented Sifter, integrated it with several open-source tracing systems, and evaluate with traces from a range of open-source and production distributed systems. Our evaluation shows that Sifter effectively biases towards anomalous and outlier executions, is robust to noisy and heterogeneous traces, is efficient and scalable, and adapts to changes in workloads over time.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3357223.3362736

Cites

32 of 32 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig. 1 Tracing pipeline described in §2.2.
Fig. 1 Tracing pipeline described in §2.2.CC-BY · From the PDF
Fig. 2 Trace sampling ( ✓ ✗ ) can occur before (a,b) or after (c,d) request execution. For non-sampled requests ( ✗ ), head-based sampling avoids all overheads (a), while tail-based sampling avoids only backend processing and storage overheads (c).
Fig. 2 Trace sampling ( ✓ ✗ ) can occur before (a,b) or after (c,d) request execution. For non-sampled requests ( ✗ ), head-based sampling avoids all overheads (a), while tail-based sampling avoids only backend processing and storage overheads (c).CC-BY · From the PDF
Image 3
CC-BY · From the PDF
Fig. 4 Sifter's internal pipeline that makes sampling decisions based on an internal probabilistic model of paths in the system. Each sampling decision updates the model.
Fig. 4 Sifter's internal pipeline that makes sampling decisions based on an internal probabilistic model of paths in the system. Each sampling decision updates the model.CC-BY · From the PDF
Fig. 5 Loss and sampling probability for a workload of 1000 HDFS read traces. We intermittently introduce 5 write traces, correspondingtothefivespikeswithhighsamplingprobability.
Fig. 5 Loss and sampling probability for a workload of 1000 HDFS read traces. We intermittently introduce 5 write traces, correspondingtothefivespikeswithhighsamplingprobability.CC-BY · From the PDF
Fig. 6 Loss and sampling probability for ComposePost API calls. Five exception traces are introduced, corresponding to five spikes with high sampling probability.
Fig. 6 Loss and sampling probability for ComposePost API calls. Five exception traces are introduced, corresponding to five spikes with high sampling probability.CC-BY · From the PDF
Image 7
CC-BY · From the PDF
Fig. 8 Proportion of request types sampled by three different schemes, for three different workloads. Note that uniform random sampling reflects the original workload distribution.
Fig. 8 Proportion of request types sampled by three different schemes, for three different workloads. Note that uniform random sampling reflects the original workload distribution.CC-BY · From the PDF
Image 9
CC-BY · From the PDF
Fig. 10 Sifter requires no bootstrapping or pre-training.
Fig. 10 Sifter requires no bootstrapping or pre-training.CC-BY · From the PDF
Image 11
CC-BY · From the PDF
Image 12
CC-BY · From the PDF
Fig. 12 Sampling latency for Hierarchical Clustering grows proportional to the number of sampled traces.
Fig. 12 Sampling latency for Hierarchical Clustering grows proportional to the number of sampled traces.CC-BY · From the PDF
Fig. 13 Distribution of Sifter sampling latency for datasets.
Fig. 13 Distribution of Sifter sampling latency for datasets.CC-BY · From the PDF
Fig. 14 PCA projection shows clusters of different trace types.
Fig. 14 PCA projection shows clusters of different trace types.CC-BY · From the PDF

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 35 references

Research works printed but not linked: 17

  1. Gan Y., 24th ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS '19). (§1 and 7.1).
  2. Gan Y., 24th ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS '19)
  3. Kasick M. P., Black-Box Problem Diagnosis in Parallel File Systems. In 8th USENIX Conference on File and Storage Technologies (FAST '10)
  4. Leavitt J., 2014, Thesis
  5. Lou J.-G., USENIX Annual Technical Conference (ATC '10)
  6. Mace J., 2013, Project
  7. Mann G., Modeling the Parallel Execution of Black-Box Services. In 3rd USENIX Workshop on Hot Topics in Cloud Computing (HotCloud '11)
  8. Mikolov T., 27th Conference on Neural Information Processing Systems (NIPS '13)
  9. Nagaraj K., 9th USENIX Conference on Networked Systems Design and Implementation (NSDI '12)
  10. Ostrowski K., Diagnosing Latency in Multi-Tier Black-Box Services. In 5th Workshop on Large Scale Distributed Systems and Middleware (LADIS '11)
  11. Sambasivan R. R., 8th USENIX Symposium on Networked Systems Design and Implementation (NSDI '11)
  12. Thereska E., 2006 ACM International Conference on Measurement and Modeling of Computer Systems (SIGMETRICS '06)
  13. Tian H., Low-Latency Continual Learning. In 10th ACM Symposium on Cloud Computing (SOCC '18)
  14. Xu W., 22nd ACM Symposium on Operating Systems Principles (SOSP '09)
  15. Zhao X., 26th ACM Symposium on Operating Systems Principles (SOSP '17)
  16. Zhao X., 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI '16)
  17. Zhao X., 11th USENIX Symposium on Operating Systems Design and Implementation (OSDI '14)

Cited by

See all 51

Metadata

Authors
Las-Casas, P., Papakerashvili, G., Anand, V., & Mace, J.
Affiliations
Max Planck Institute for Software Systems (DE)
Venue
Proceedings of the ACM Symposium on Cloud Computing
Publisher
ACM
Published
2019-11-20
Pages
312-324
OpenAlex
W2988883278
Cited by
42
References
35
Citations by year
2026: 3 · 2025: 12 · 2024: 4 · 2023: 10 · 2022: 5 · 2021: 7 · 2020: 1
FWCI
2.81
Type
conference paper
Language
en
Open access
Locations
ACM Symposium on Cloud Computing (SoCC) (publishedVersion, cc-by)
Topics
  • Physical Sciences › Computer Science › Computer Networks and Communications › Software System Performance and Reliability
  • Physical Sciences › Computer Science › Information Systems › Cloud Computing and Resource Management
  • Physical Sciences › Computer Science › Information Systems › Advanced Database Systems and Queries
Keywords
distributed tracing, uniform random sampling, anomaly detection, low-dimensional models, SIFTER

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Sifter: Scalable Sampling for Distributed Traces, without Feature Engineering. https://app.cochat.ai/papers/10.1145/3357223.3362736
The paper
Las-Casas, P., Papakerashvili, G., Anand, V., & Mace, J. (2019). Sifter: Scalable Sampling for Distributed Traces, without Feature Engineering. Proceedings of the ACM Symposium on Cloud Computing. https://doi.org/10.1145/3357223.3362736

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.