The DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
Adaptive stream resource management using Kalman Filters
Abstract
To answer user queries efficiently, a stream management system must handle continuous, high-volume, possibly noisy, and time-varying data streams. One major research area in stream management seeks to allocate resources (such as network bandwidth and memory) to query plans, either to minimize resource usage under a precision requirement, or to maximize precision of results under resource constraints. To date, many solutions have been proposed; however, most solutions are ad hoc with hard-coded heuristics to generate query plans. In contrast, we perceive stream resource management as fundamentally a filtering problem, in which the objective is to filter out as much data as possible to conserve resources, provided that the precision standards can be met. We select the Kalman Filter as a general and adaptive filtering solution for conserving resources. The Kalman Filter has the ability to adapt to various stream characteristics, sensor noise, and time variance. Furthermore, we realize a significant performance boost by switching from traditional methods of caching static data (which can soon become stale) to our method of caching dynamic procedures that can predict data reliably at the server without the clients' involvement. In this work we focus on minimization of communication overhead for both synthetic and real-world streams. Through examples and empirical studies, we demonstrate the flexibility and effectiveness of using the Kalman Filter as a solution for managing trade-offs between precision of results and resources in satisfying stream queries.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A New Approach to Linear Filtering and Prediction Problems
Kalman, R. E. (1960) · Journal of Basic Engineering · doi:10.1115/1.3662552 · Cited by 31,751
Russell, E. L., Chiang, L. H., & Braatz, R. D. (2000) · Advances in Industrial Control · doi:10.1007/978-1-4471-0409-4_3 · Cited by 7,862
Stochastic models, estimation, and control
(1982) · Mathematics in Science and Engineering · doi:10.1016/s0076-5392(08)x6186-7 · Cited by 3,229
Models and issues in data stream systems
Babcock, B., Babu, S., Datar, M., Motwani, R., & Widom, J. (2002) · Proceedings of the twenty-first ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems · doi:10.1145/543613.543615 · Cited by 2,495
Pattern Classification (2nd Edition)
Duda, R. O., Hart, P. E., & Stork, D. G. (2000) · Wiley-Interscience eBooks · Cited by 2,070
Energy-aware wireless microsensor networks
Raghunathan, V., Schurgers, C., Park, S., & Srivastava, M. B. (2002) · IEEE Signal Processing Magazine · doi:10.1109/79.985679 · Cited by 1,706
Stochastic models, estimation, and control
(1979) · Mathematics in Science and Engineering · doi:10.1016/s0076-5392(08)x6125-9 · Cited by 1,624
Introduction to applied mathematics.
(1986) · Mathematics and Computers in Simulation · doi:10.1016/0378-4754(86)90036-4 · Cited by 1,495
Issues in data stream management
Golab, L., & ÖZSU, M. T. (2003) · ACM SIGMOD Record · doi:10.1145/776985.776986 · Cited by 964
TelegraphCQ: Continuous Dataflow Processing for an Uncertain World.
Chandrasekaran, S., Cooper, O., Deshpande, A., Franklin, M. J. M., Hellerstein, J. M., Hong, W., Krishnamurthy, S., Madden, S. R., Raman, V., Reiss, F. R., et al. (2003) · Conference on Innovative Data Systems Research · Cited by 947
Introduction to Applied Mathematics
Strang, G., & Freund, L. B. (1986) · Journal of Applied Mechanics · doi:10.1115/1.3171799 · Cited by 664
Introduction to random signal analysis and Kalman filtering
Brown, R. G. (1983) · Medical Entomology and Zoology · Cited by 634
Load Shedding in a Data Stream Manager
Tatbul, N., Çetintemel, U., Zdonik, S., Cherniack, M., & Stonebraker, M. (2003) · Proceedings 2003 VLDB Conference · doi:10.1016/b978-012722442-8/50035-5 · Cited by 577
Evaluating probabilistic queries over imprecise data
Cheng, R. C. K., Kalashnikov, D. V., & Prabhakar, S. (2003) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/872757.872823 · Cited by 570
Stochastic models, estimation and control, Volume 1
Maybeck, P. S. (1979) · Cited by 452
Adaptive filters for continuous queries over distributed data streams
Olston, C., Jiang, J., & Widom, J. (2003) · Proceedings of the 2003 ACM SIGMOD international conference on Management of data · doi:10.1145/872757.872825 · Cited by 447
STREAM: The Stanford Stream Data Manager.
Arasu, A., Babcock, B., Babu, S., Datar, M., Ito, K. A., Motwani, R., Nishizawa, I., Srivastava, U., Thomas, D., Varma, R., et al. (2003) · IEEE Data(base) Engineering Bulletin · Cited by 410
An adaptive query execution system for data integration
Ives, Z. G., Florescu, D., Friedman, M., Levy, A. Y., & Weld, D. S. (1999) · ACM SIGMOD Record · doi:10.1145/304181.304209 · Cited by 396
Query Processing, Resource Management, and Approximation ina Data Stream Management System
Motwani, R., Widom, J., Arasu, A., Babcock, B., Babu, S., Datar, M., Manku, G. S., Olston, C., Rosenstein, J., & Varma, R. (2002) · Cited by 342
Load shedding for aggregation queries over data streams
Babcock, B., Datar, M., & Motwani, R. (2004) · Proceedings. 20th International Conference on Data Engineering · doi:10.1109/icde.2004.1320010 · Cited by 311
Aurora: a data stream management system
Abadi, D., Carney, D., Çetintemel, U., Cherniack, M., Convey, C., Erwin, C., Galvez, E., Hatoun, M., Maskey, A., Rasin, A., et al. (2003) · Proceedings of the 2003 ACM SIGMOD international conference on Management of data · doi:10.1145/872757.872855 · Cited by 201
Capturing sensor-generated time series with quality guarantees
Lazaridis, I., & Mehrotra, S. (2004) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde.2003.1260811 · Cited by 178
Online data mining for co-evolving time sequences
Yi, B.-K., Sidiropoulos, N. D., Johnson, T., Jagadish, H. V., Faloutsos, C., & Biliris, A. (2002) · Proceedings of 16th International Conference on Data Engineering (Cat. No.00CB37073) · doi:10.1109/icde.2000.839383 · Cited by 169
Operator scheduling in data stream systems
Babcock, B., Babu, S., Datar, M., Motwani, R., & Thomas, D. (2004) · The VLDB Journal · doi:10.1007/s00778-004-0132-6 · Cited by 161
Neural Decoding of Cursor Motion Using a Kalman Filter
Wu, W. L., Black, M. J., Gao, Y., Bienenstock, E., Serruya, M. D., Shaikhouni, A., & Donoghue, J. P. (2003) · Cited by 156
Characterizing memory requirements for queries over continuous data streams
Arasu, A., Babcock, B., Babu, S., McAlister, J., & Widom, J. (2002) · ACM SIGACT-SIGMOD-SIGART Symposium on Principles of Database Systems (PODS) · doi:10.1145/543613.543642 · Cited by 107
Energy efficient design of portable wireless systems
Simunic, T., Vikalo, H., Glynn, P. W., & Micheli, G. D. (2000) · International Symposium on Low Power Electronics and Design (ISLPED) · doi:10.1145/344166.344197 · Cited by 63
Multi-camera spatio-temporal fusion and biased sequence-data learning for security surveillance
Wu, G., Wu, Y., Jiao, L., Wang, Y.-F., & Chang, E. Y. (2003) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/957013.957126 · Cited by 63
Self-citation
Cited by
Energy conservation in wireless sensor networks: A survey
Anastasi, G., Conti, M., Di Francesco, M., & Passarella, A. (2009) · Ad Hoc Networks · doi:10.1016/j.adhoc.2008.06.003 · Cited by 2,583
Information fusion for wireless sensor networks: Methods, models, and classifications
Nakamura, E. F., Loureiro, A. A. F., & Frery, A. C. (2007) · ACM Computing Surveys · doi:10.1145/1267070.1267073 · Cited by 537
Never Walk Alone: Uncertainty for Anonymity in Moving Objects Databases
Abul, O., Bonchi, F., & Nanni, M. (2008) · 2008 IEEE 24th International Conference on Data Engineering · doi:10.1109/icde.2008.4497446 · Cited by 506
Rise and fall patterns of information diffusion: model and implications
Matsubara, Y., Sakurai, Y., Prakash, B. A., Li, L., & Faloutsos, C. (2012) · Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/2339530.2339537 · Cited by 306
Metadata
- Authors
- Jain, A., Chang, E. Y., & Wang, Y.-F.
- Venue
- Proceedings of the 2004 ACM SIGMOD international conference on Management of data
- Publisher
- ACM
- Published
- 2004-06-13
- Pages
- 11-22
- ISSN
- 0730-8078 (ISSN-L)
- OpenAlex
- W2042961981
- Cited by
- 307
- References
- 28
- FWCI
- 10.47
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.