Cited by 299Advanced Database Systems and QueriesParallel Computing and Optimization TechniquesEmbedded Systems Design Techniques

A catalog of stream processing optimizations

Open access
Hirzel, Martin, Soulé, Robert, Schneider, Scott, Gedik, Buğra, Grimm, Robert·2014·ACM Computing Surveys

Abstract

Various research communities have independently arrived at stream processing as a programming model for efficient and parallel computing. These communities include digital signal processing, databases, operating systems, and complex event processing. Since each community faces applications with challenging performance requirements, each of them has developed some of the same optimizations, but often with conflicting terminology and unstated assumptions. This article presents a survey of optimizations for stream processing. It is aimed both at users who need to understand and guide the system’s optimizer and at implementers who need to make engineering tradeoffs. To consolidate terminology, this article is organized as a catalog, in a style similar to catalogs of design patterns or refactorings. To make assumptions explicit and help understand tradeoffs, each optimization is presented with its safety constraints (when does it preserve correctness?) and a profitability experiment (when does it improve performance?). We hope that this survey will help future streaming system builders to stand on the shoulders of giants from not just their own community.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/2528412

Cites

97 of 97 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Various research communities have independently arrived at stream processing as a programming model for efficient and parallel computing.

    “Various research communities have independently arrived at stream processing as a programming model for efficient and parallel computing (Hirzel, Soulé, Schneider, Gedik, & Grimm, 2014).”

    From Prescriptive analytics: Literature review and research challenges (2020) · cites this paper as Hirzel, Soulé, Schneider, Gedik, & Grimm, 2014

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 12

  1. Joshua S. Auerbach, David F. Bacon, Perry Cheng, and Rodric M. Rabbah. 2010. Lime: A Java-compatible and Synthesizable Language for Heterogeneous Architectures. In Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA). 89-108.
  2. Ron Avnur and Joseph M. Hellerstein. 2000. Eddies: Continuously Adaptive Query Processing. In Interna- tional Conference on Management of Data (SIGMOD). 261-272.
  3. Irina Botan, Roozbeh Derakhshan, Nihal Dindar, Laura Haas, Renée J. Miller, and Nesime Tatbul. 2010. SECRET: A Model for Analysis of the Execution Semantics of Stream Processing Systems. In Conference on Very Large Data Bases (VLDB). 232-243.
  4. Jianjun Chen, David J. DeWitt, Feng Tian, and Yuan Wang. 2000. NiagaraCQ: A Scalable Continuous Query System for Internet Databases. In International Conference on Management of Data (SIGMOD). 379-390.
  5. Corinna Cortes, Kathleen Fisher, Daryl Pregibon, Anne Rogers, and Frederick Smith. 2004. Hancock: A Language for Analyzing Transactional Data Streams. ACM Transactions on Programming Languages and Systems 26, 2 (March 2004), 301-338.
  6. Bu gra Gedik, Henrique Andrade, Kun-Lung Wu, Philip S. Yu, and Myungcheol Doo. 2008a. SPADE: The System S Declarative Stream Processing Engine. In International Conference on Management of Data (SIGMOD). 1123-1134.
  7. Amir H. Hormati, Yoonseo Choi, Mark Woh, Manjunath Kudlur, Rodric Rabbah, Trevor Mudge, and Scott Mahlke. 2010. MacroSS: Macro-SIMDization of Streaming Applications. In Architectural Support for Programming Languages and Operating Systems (ASPLOS). 285-296.
  8. Michael Isard, Mihai Budiuand Yuan Yu, Andrew Birrell, and Dennis Fetterly. 2007. Dryad: Distributed Data-Parallel Program from Sequential Building Blocks. In European Conference on Computer Systems (EuroSys). 59-72.
  9. Scott Schneider, Bu gra Gedik, and Martin Hirzel. 2013. Tutorial: Stream Processing Optimizations. In Conference on Distributed Event-Based Systems (DEBS). 249-258.
  10. William Thies, Vikram Chandrasekhar, and Saman Amarasinghe. 2007. A Practical Approach to Exploiting Coarse-Grained Pipeline Parallelism in C Programs. In International Symposium on Microarchitecture (MICRO). 314-327.
  11. Matt Welsh, David Culler, and Eric Brewer. 2001. SEDA: An Architecture for Well-Conditioned, Scalable Internet Services. In Symposium on Operating Systems Principles (SOSP). 230-243.
  12. Jianxin Xiong, Jeremy Johnson, Robert Johson, and David Padua. 2001. SPL: A Language and Compiler for DSP Algorithms. In Programming Language Design and Implementation (PLDI). 298-308.

Cited by

Sorted by citations

Authors
Hirzel, M., Soulé, R., Schneider, S., Gedik, B., & Grimm, R.
Affiliations
  • IBM (United States) (US)
  • Università della Svizzera italiana (CH)
  • Bilkent University (TR)
  • New York University (US)
Venue
ACM Computing Surveys
Published
2014-03
Volume
46
Issue
4
Pages
1-34
ISSN
  • 0360-0300 (ISSN-L)
  • 1557-7341
OpenAlex
W2148222209
Cited by
299
References
97
Citations by year
2026: 5 · 2025: 9 · 2024: 12 · 2023: 17 · 2022: 23 · 2021: 30 · 2020: 24 · 2019: 44 · 2018: 31 · 2017: 32 · 2016: 37 · 2015: 21 · 2014: 10 · 2013: 3 · 2012: 1
FWCI
38.31
Type
journal article
Language
en
Open access
Locations
  • ACM Computing Surveys (publishedVersion)
  • Bilkent University Institutional Repository (Bilkent University) (submittedVersion)
Funders
  • National Science Foundation (1162444, 1111698)
  • Office of Naval Research (N00014-12-1-0757, N00014-12-1, N00014-12, N00014)
Topics
  • Physical Sciences › Computer Science › Computer Networks and Communications › Advanced Database Systems and Queries
  • Physical Sciences › Computer Science › Hardware and Architecture › Parallel Computing and Optimization Techniques
  • Physical Sciences › Computer Science › Hardware and Architecture › Embedded Systems Design Techniques
Keywords
stream processing, stream processing systems, correctness preservation, performance trade-offs, parallel computing, digital signal processing, complex event processing

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review