Adaptive ordering of pipelined stream filters
The DOI matches its Crossref record.
Abstract
We consider the problem of pipelined filters, where a continuous stream of tuples is processed by a set of commutative filters. Pipelined filters are common in stream applications and capture a large class of multiway stream joins. We focus on the problem of ordering the filters adaptively to minimize processing cost in an environment where stream and filter characteristics vary unpredictably over time. Our core algorithm, A-Greedy (for Adaptive Greedy), has strong theoretical guarantees: If stream and filter characteristics were to stabilize, A-Greedy would converge to an ordering within a small constant factor of optimal. (In experiments A-Greedy usually converges to the optimal ordering.) One very important feature of A-Greedy is that it monitors and responds to selectivities that are correlated across filters (i.e., that are nonindependent), which provides the strong quality guarantee but incurs run-time overhead. We identify a three-way tradeoff among provable convergence to good orderings, run-time overhead, and speed of adaptivity. We develop a suite of variants of A-Greedy that lie at different points on this tradeoff spectrum. We have implemented all our algorithms in the STREAM prototype Data Stream Management System and a thorough performance evaluation is presented.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cited by
The CQL continuous query language: semantic foundations and query execution
Arasu, A., Babu, S., & Widom, J. (2006) · The VLDB Journal · doi:10.1007/s00778-004-0147-z · Cited by 1,170
Self-citationSubmodular Function Maximization
Krause, A., & Golovin, D. (2014) · Tractability · doi:10.1017/cbo9781139177801.004 · Cited by 751
STREAM: The Stanford Data Stream Management System
Arasu, A., Babcock, B., Babu, S., Cieslewicz, J., Datar, M., Ito, K., Motwani, R., Srivastava, U., & Widom, J. (2016) · Data-Centric Systems and Applications · doi:10.1007/978-3-540-28608-0_16 · Cited by 406
Self-citationNoScope: optimizing neural network queries over video at scale
Kang, D., Emmons, J., Abuzaid, F., Bailis, P., & Zaharia, M. (2017) · Proceedings of the VLDB Endowment · doi:10.14778/3137628.3137664 · Cited by 367
“[11]): one specialized model may complement one difference detector but not another, and the viable combinations of thresholds for each pair of models will differ across scenes.”
BackgroundMethodA catalog of stream processing optimizations
Hirzel, M., Soulé, R., Schneider, S., Gedik, B., & Grimm, R. (2014) · ACM Computing Surveys · doi:10.1145/2528412 · Cited by 299
Metadata
- Authors
- Babu, S., Motwani, R., Munagala, K., Nishizawa, I., & Widom, J.
- Venue
- Proceedings of the 2004 ACM SIGMOD international conference on Management of data
- Publisher
- ACM
- Published
- 2004-06-13
- Pages
- 407-418
- ISSN
- 0730-8078 (ISSN-L)
- OpenAlex
- W2101207848
- Cited by
- 229
- References
- 0
- FWCI
- 11.42
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Adaptive ordering of pipelined stream filters. https://app.cochat.ai/papers/10.1145/1007568.1007615
- The paper
- Babu, S., Motwani, R., Munagala, K., Nishizawa, I., & Widom, J. (2004). Adaptive ordering of pipelined stream filters. Proceedings of the 2004 ACM SIGMOD international conference on Management of data. https://doi.org/10.1145/1007568.1007615
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.