Cited by 39Software System Performance and ReliabilityParallel Computing and Optimization TechniquesSoftware Testing and Debugging Techniques

tprof: Performance profiling via structural aggregation and automated analysis of distributed systems traces

Open access
Huang, Lexiang, Zhu, Timothy·2021·Proceedings of the ACM Symposium on Cloud Computing

Abstract

The traditional approach for performance debugging relies upon performance profilers (e.g., gprof, VTune) that provide average function runtime information. These aggregate statistics help identify slow regions affecting the entire workload, but they are ill-suited for identifying slow regions that only impact a fraction of the workload, such as tail latency effects. This paper takes a new approach to performance profiling by utilizing distributed tracing systems (e.g., Dapper, Zipkin, Jaeger). Since traces provide detailed timing information on a per-request basis, it is possible to group and aggregate tracing data in many different ways to identify the slow parts of the system. Our new approach to trace aggregation uses the structure embedded within traces to hierarchically group similar traces and calculate increasingly detailed aggregate statistics based on how the traces are grouped. We also develop an automated tool for analyzing the hierarchy of statistics to identify the most likely performance issues. Our case study across two complex distributed systems illustrates how our tool is able to find multiple performance issues that lead to 10x and 28x performance improvements in terms of average and tail latency, respectively. Our comparison with a state-of-the-art industry tool shows that our tool can pinpoint performance slowdowns more accurately than current approaches.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3472883.3486994

Cites

56 of 56 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • tprof organizes traces into hierarchical groups based on request types, span structures, and parent-child relationships, enabling aggregate trace representations for scalable analysis.

    “tprof [84] organizes traces into hierarchical groups based on request types, span structures, and parent-child relationships, enabling aggregate trace representations for scalable analysis.”

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

1 of the 57 references couldn’t be matched to a known paper.

Research works printed but not linked: 2

  1. Charlie Curtsinger and Emery D. Berger. 2016. COZ: Finding Code that Counts with Causal Profiling. In 2016 USENIX Annual Tech- nical Conference (USENIX ATC 16). USENIX Association, Denver, CO. https://www.usenix.org/conference/atc16/technical-sessions/ presentation/curtsinger
  2. A. Parker, D. Spoonhower, J. Mace, B. Sigelman, and R. Isaacs. 2020. Distributed Tracing in Practice: Instrumenting, Analyzing, and Debug- ging Microservices. O'Reilly Media, Incorporated. https://books.google. com/books?id=fgfIyAEACAAJ

Cited by

Sorted by citations

Authors
Huang, L., & Zhu, T.
Affiliations
Pennsylvania State University (US)
Venue
Proceedings of the ACM Symposium on Cloud Computing
Publisher
ACM
Published
2021-11
Pages
76-91
OpenAlex
W3208273031
Cited by
37
References
57
Citations by year
2026: 6 · 2025: 7 · 2024: 9 · 2023: 12 · 2022: 3
FWCI
5.55
Type
conference paper
Language
en
Open access
Locations
  • ACM Symposium on Cloud Computing (SoCC) (publishedVersion)
  • ScholarSphere (Penn State Libraries) (submittedVersion)
Topics
  • Physical Sciences › Computer Science › Computer Networks and Communications › Software System Performance and Reliability
  • Physical Sciences › Computer Science › Hardware and Architecture › Parallel Computing and Optimization Techniques
  • Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
Keywords
performance profiling, distributed tracing, tail latency, performance debugging, distributed systems, automated performance analysis

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review