The DOI matches its Crossref record; 20 of the 20 references it cites were found in a registry or an index.
Processing complex aggregate queries over data streams
Abstract
Recent years have witnessed an increasing interest in designing algorithms for querying and analyzing streaming data (i.e., data that is seen only once in a fixed order) with only limited memory. Providing (perhaps approximate) answers to queries over such continuous data streams is a crucial requirement for many application environments; examples include large telecom and IP network installations where performance data from different parts of the network needs to be continuously collected and analyzed.In this paper, we consider the problem of approximately answering general aggregate SQL queries over continuous data streams with limited memory. Our method relies on randomizing techniques that compute small "sketch" summaries of the streams that can then be used to provide approximate answers to aggregate queries with provable guarantees on the approximation error. We also demonstrate how existing statistical information on the base data (e.g., histograms) can be used in the proposed framework to improve the quality of the approximation provided by our algorithms. The key idea is to intelligently partition the domain of the underlying attribute(s) and, thus, decompose the sketching problem in a way that provably tightens our guarantees. Results of our experimental study with real-life as well as synthetic data streams indicate that sketches provide significantly more accurate answers compared to histograms for aggregate queries. This is especially true when our domain partitioning methods are employed to further boast the accuracy of the final estimates.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Classification and Regression Trees.
Gordon, A. D., Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1984) · Biometrics · doi:10.2307/2530946 · Cited by 23,522
Classification and Regression Trees.
Ryzin, J. V., Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1986) · Journal of the American Statistical Association · doi:10.2307/2288003 · Cited by 20,825
Classification And Regression Trees
Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (2017) · doi:10.1201/9781315139470 · Cited by 11,417
An overview of data warehousing and OLAP technology
Chaudhuri, S., & Dayal, U. (1997) · ACM SIGMOD Record · doi:10.1145/248603.248616 · Cited by 2,424
Mining high-speed data streams
Domingos, P., & Hulten, G. (2000) · Proceedings of the sixth ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/347090.347107 · Cited by 2,185
Random sampling with a reservoir
Vitter, J. S. (1985) · ACM Transactions on Mathematical Software · doi:10.1145/3147.3165 · Cited by 1,758
The Space Complexity of Approximating the Frequency Moments
Alon, N., Matias, Y., & Szegedy, M. (1999) · Journal of Computer and System Sciences · doi:10.1006/jcss.1997.1545 · Cited by 1,140
Clustering data streams: theory and practice
Guha, S., Meyerson, A., Mishra, N., Motwani, R., & O'Callaghan, L. (2003) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2003.1198387 · Cited by 882
Maintaining Stream Statistics over Sliding Windows
Datar, M., Gionis, A., Indyk, P., & Motwani, R. (2002) · SIAM Journal on Computing · doi:10.1137/s0097539701398363 · Cited by 835
The space complexity of approximating the frequency moments
Alon, N., Matias, Y., & Szegedy, M. (1996) · Proceedings of the twenty-eighth annual ACM symposium on Theory of computing - STOC '96 · doi:10.1145/237814.237823 · Cited by 822
Continuous queries over data streams
Babu, S., & Widom, J. (2001) · ACM SIGMOD Record · doi:10.1145/603867.603884 · Cited by 721
Guha, S., Mishra, N., Motwani, R., & O'Callaghan, L. (2002) · Proceedings 41st Annual Symposium on Foundations of Computer Science · doi:10.1109/sfcs.2000.892124 · Cited by 623
Space-efficient online computation of quantile summaries
Greenwald, M., & Khanna, S. (2001) · Proceedings of the 2001 ACM SIGMOD international conference on Management of data · doi:10.1145/375663.375670 · Cited by 451
Surfing Wavelets on Streams: One-Pass Summaries for Approximate Aggregate Queries
Gilbert, A. C., Kotidis, Y., Muthukrishnan, S., & Strauss, M. J. (2001) · Cited by 376
Ripple joins for online aggregation
Haas, P. J., & Hellerstein, J. M. (1999) · ACM SIGMOD Record · doi:10.1145/304181.304208 · Cited by 369
On computing correlated aggregates over continual data streams
Gehrke, J., Korn, F., & Srivastava, D. (2001) · Proceedings of the 2001 ACM SIGMOD international conference on Management of data · doi:10.1145/375663.375665 · Cited by 278
Self-citationMaintaining stream statistics over sliding windows: (extended abstract)
Datar, M., Gionis, A., Indyk, P., & Motwani, R. (2002) · doi:10.5555/545381.545466 · Cited by 253
Tracking join and self-join sizes in limited storage
Alon, N., Gibbons, P. B., Matias, Y., & Szegedy, M. (1999) · doi:10.1145/303976.303978 · Cited by 211
Approximate Query Processing: Taming the TeraBytes
Garofalakis, M., & Gibbon, P. B. (2001) · Very Large Data Bases · Cited by 174
Self-citationAn Approximate L p-Difference Algorithm for Massive Data Streams: Extended Abstract
Fong, J. H., & Strauss, M. J. (2000) · Lecture Notes in Computer Science · doi:10.1007/3-540-46541-3_16 · Cited by 163
Cited by
Models and issues in data stream systems
Babcock, B., Babu, S., Datar, M., Motwani, R., & Widom, J. (2002) · Proceedings of the twenty-first ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems · doi:10.1145/543613.543615 · Cited by 2,495
An improved data stream summary: the count-min sketch and its applications
Cormode, G., & Muthukrishnan, S. (2005) · Journal of Algorithms · doi:10.1016/j.jalgor.2003.12.001 · Cited by 1,743
Gossip-based computation of aggregate information
Kempe, D., Dobra, A., & Gehrke, J. (2004) · 44th Annual IEEE Symposium on Foundations of Computer Science, 2003. Proceedings. · doi:10.1109/sfcs.2003.1238221 · Cited by 1,480
Self-citationThe CQL continuous query language: semantic foundations and query execution
Arasu, A., Babu, S., & Widom, J. (2006) · The VLDB Journal · doi:10.1007/s00778-004-0147-z · Cited by 1,170
Clock synchronization for wireless sensor networks: a survey
Sundararaman, B., Buy, U., & Kshemkalyani, A. D. (2005) · Ad Hoc Networks · doi:10.1016/j.adhoc.2005.01.002 · Cited by 988
Metadata
- Authors
- Dobra, A., Garofalakis, M., Gehrke, J., & Rastogi, R.
- Venue
- Proceedings of the 2002 ACM SIGMOD international conference on Management of data
- Publisher
- ACM
- Published
- 2002-06-03
- Pages
- 61-72
- ISSN
- 0730-8078 (ISSN-L)
- OpenAlex
- W2148706674
- Cited by
- 319
- References
- 20
- FWCI
- 19.09
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.