TimeStream: reliable stream computation in the cloud
The DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.
Abstract
TimeStream is a distributed system designed specifically for low-latency continuous processing of big streaming data on a large cluster of commodity machines. The unique characteristics of this emerging application domain have led to a significantly different design from the popular MapReduce-style batch data processing. In particular, we advocate a powerful new abstraction called resilient substitution that caters to the specific needs in this new computation model to handle failure recovery and dynamic reconfiguration in response to load changes. Several real-world applications running on our prototype have been shown to scale robustly with low latency while at the same time maintaining the simple and concise declarative programming model. TimeStream handles an on-line advertising aggregation pipeline at a rate of 700,000 URLs per second with a 2-second delay, while performing sentiment analysis of Twitter data at a peak rate close to 10,000 tweets per second, with approximately 2-second delay.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
MapReduce: simplified data processing on large clusters
Dean, J., & Ghemawat, S. (2008) · Communications of the ACM · doi:10.1145/1327452.1327492 · Cited by 18,476
ZooKeeper: wait-free coordination for internet-scale systems
Hunt, P. G., Konar, M., Junqueira, F., & Reed, B. C. (2010) · USENIX Annual Technical Conference · Cited by 1,968
Pig latin: a not-so-foreign language for data processing
Olston, C., Reed, B., Srivastava, U., Kumar, R., & Tomkins, A. (2008) · Proceedings of the 2008 ACM SIGMOD international conference on Management of data · doi:10.1145/1376616.1376726 · Cited by 1,730
Hive: a warehousing solution over a map-reduce framework
Thusoo, A., Sarma, J. S., Jain, N., Shao, Z., Chakka, P., Anthony, S., Liu, H., Wyckoff, P., & Murthy, R. (2009) · Proceedings of the VLDB Endowment · doi:10.14778/1687553.1687609 · Cited by 1,556
S4: Distributed Stream Computing Platform
Neumeyer, L., Robbins, B., Nair, A., & Kesari, A. (2010) · 2010 IEEE International Conference on Data Mining Workshops · doi:10.1109/icdmw.2010.172 · Cited by 900 · cited 1×
“In contrast to many other systems that deal with off-line batch processing, TimeStream focuses on large-scale lowlatency stream computation, as with a number of recent systems including Storm [2], S4 [20], D-Streams [28], Puma [19] and Flume [14].”
BackgroundSCOPE: easy and efficient parallel processing of massive data sets
Chaiken, R., Jenkins, B., Larson, P.-Å., Ramsey, B., Shakib, D., Weaver, S., & Zhou, J. (2008) · Proceedings of the VLDB Endowment · doi:10.14778/1454159.1454166 · Cited by 727
Yu, Y., Isard, M., Fetterly, D., Budiu, M., Erlingsson, Ú., Gunda, P. K., & Currey, J. (2008) · doi:10.5555/1855741.1855742 · Cited by 689
Self-citationDiscretized streams: an efficient and fault-tolerant model for stream processing on large clusters
Zaharia, M., Das, T., Li, H., Shenker, S. J., & Stoica, I. G. (2012) · USENIX conference on Hot Topics in Cloud Ccomputing · Cited by 522
LINQ: reconciling object, relations and XML in the .NET framework
Meijer, E., Beckman, B., & Bierman, G. M. (2006) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/1142473.1142552 · Cited by 381
FlumeJava: easy, efficient data-parallel pipelines
Chambers, C., Raniwala, A., Perry, F., Adams, S., Henry, R. R., Bradshaw, R., & Weizenbaum, N. (2010) · Proceedings of the 31st ACM SIGPLAN Conference on Programming Language Design and Implementation · doi:10.1145/1806596.1806638 · Cited by 331 · cited 5×
“[14] CH AMB E R S , C., RA NI WA L A , A., PE R RY, F., ADA M S, S., HE N RY, R. R., BR A DSHAW, R., A N D WE I ZE NBAUM , N. FlumeJava: Easy, ef.cient data-parallel pipelines.”
BackgroundInfluential
Cited by
A Survey of Fog Computing: Concepts, Applications and Issues
Yi, S., Li, C., & Li, Q. (2015) · Proceedings of the 2015 Workshop on Mobile Big Data · doi:10.1145/2757384.2757397 · Cited by 1,339
Discretized streams: fault-tolerant streaming computation at scale
Zaharia, M., Das, T., Li, H., Hunter, T., Shenker, S., & Stoica, I. (2013) · Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles · doi:10.1145/2517349.2522737 · Cited by 982
Baccarelli, E., Naranjo, P. G. V., Scarpiniti, M., Shojafar, M., & Abawajy, J. H. (2017) · IEEE Access · doi:10.1109/access.2017.2702013 · Cited by 354
Dias de Assunção, M., da Silva Veith, A., & Buyya, R. (2018) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2017.12.001 · Cited by 304
Big data stream analysis: a systematic literature review
Kolajo, T., Daramola, O., & Adebiyi, A. (2019) · Journal of Big Data · doi:10.1186/s40537-019-0210-7 · Cited by 259
A survey on the evolution of stream processing systems
Fragkoulis, M., Carbone, P., Kalavri, V., & Katsifodimos, A. (2024) · The VLDB Journal · doi:10.1007/s00778-023-00819-8 · Cited by 207
Baccarelli, E., Cordeschi, N., Mei, A., Panella, M., Shojafar, M., & Stefa, J. (2016) · IEEE Network · doi:10.1109/mnet.2016.7437025 · Cited by 200
“Dynamic adaptation of the available computing resources to the ever changing rate of the input streams is, indeed, provided by the more recent Time Stream [6] and PLAstiCC [7] management frameworks.”
MethodConsistency and Completeness: Rethinking Distributed Stream Processing in Apache Kafka
Wang, G., Chen, L., Dikshit, A., Gustafson, J., Chen, B., Sax, M. J., Roesler, J., Blee-Goldman, S., Cadonna, B., Mehta, A., et al. (2021) · Proceedings of the 2021 International Conference on Management of Data · doi:10.1145/3448016.3457556 · Cited by 74
DSPBench: A suite of benchmark applications for distributed data stream processing systems
Bordin, M. V., Griebler, D., Mencagli, G., Geyer, C. F. R., & Fernandes, L. G. L. (2020) · IEEE Access · doi:10.1109/access.2020.3043948 · Cited by 63
Hazelcast jet: low-latency stream processing at the 99.99 th percentile
Gencer, C., Topolnik, M., Ďurina, V., Demirci, E., Kahveci, E. B., Gürbüz, A., Lukáš, O., Bartók, J., Gierlach, G., Hartman, F., et al. (2021) · Proceedings of the VLDB Endowment · doi:10.14778/3476311.3476387 · Cited by 38
IoT streaming data integration from multiple sources
Tu, D. Q., Kayes, A. S. M., Rahayu, W., & Nguyen, K. (2020) · Computing · doi:10.1007/s00607-020-00830-9 · Cited by 28
“In the literature, several techniques for processing streaming data [9,10] have been proposed, including a windowing technique [11], however, these approaches are not able to integrate IoT data from multiple sources in real time.”
Background
Metadata
- Authors
- Qian, Z., He, Y., Su, C., Wu, Z., Zhu, H., Zhang, T., Zhou, L., Yu, Y., & Zhang, Z.
- Venue
- Proceedings of the 8th ACM European Conference on Computer Systems
- Publisher
- ACM
- Published
- 2013-04-15
- Pages
- 1-14
- OpenAlex
- W1984220394
- Cited by
- 227
- References
- 19
- FWCI
- 27.45
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.