Incoop: MapReduce for incremental computations
The DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index.
Abstract
Many online data sets evolve over time as new entries are slowly added and existing entries are deleted or modified. Taking advantage of this, systems for incremental bulk data processing, such as Google's Percolator, can achieve efficient updates. To achieve this efficiency, however, these systems lose compatibility with the simple programming models offered by non-incremental systems, e.g., MapReduce, and more importantly, requires the programmer to implement application-specific dynamic algorithms, ultimately increasing algorithm and code complexity.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
MapReduce: simplified data processing on large clusters
Dean, J., & Ghemawat, S. (2008) · Communications of the ACM · doi:10.1145/1327452.1327492 · Cited by 18,476
Pig latin: a not-so-foreign language for data processing
Olston, C., Reed, B., Srivastava, U., Kumar, R., & Tomkins, A. (2008) · Proceedings of the 2008 ACM SIGMOD international conference on Management of data · doi:10.1145/1376616.1376726 · Cited by 1,730
Improving MapReduce performance in heterogeneous environments
Zaharia, M., Konwinski, A., Joseph, A. D., Katz, R. H., & Stoica, I. G. (2008) · doi:10.5555/1855741.1855744 · Cited by 1,588
A low-bandwidth network file system
Muthitacharoen, A., Chen, B., & Mazières, D. (2001) · Proceedings of the eighteenth ACM symposium on Operating systems principles · doi:10.1145/502034.502052 · Cited by 784
HaLoop: efficient iterative data processing on large clusters
Bu, Y., Howe, B., Balazinska, M., & Ernst, M. D. (2010) · Proceedings of the VLDB Endowment · doi:10.14778/1920841.1920881 · Cited by 693
Condie, T., Conway, N., Alvaro, P., Hellerstein, J. M., Elmeleegy, K., & Sears, R. (2010) · doi:10.5555/1855711.1855732 · Cited by 673
Large-scale incremental processing using distributed transactions and notifications
Peng, D. N., & Dabek, F. (2010) · doi:10.5555/1924943.1924961 · Cited by 402
Deriving Production Rules for Incremental View Maintenance
Ceri, S., & Widom, J. (1999) · Materialized Views · doi:10.7551/mitpress/4472.003.0035 · Cited by 330 · cited 1×
“techniques used by these systems can either directly operate on the database internals to perform these incremental updates, or rely on SQL queries that efficiently compute the modifications to the database view, and that are issued upon the execution of a database trigger [5].”
MethodNectar: automatic management of data and computation in datacenters
Gunda, P. K., Ravindranath, L., Thekkath, R. A., Yu, Y., & Li, Z. (2010) · Cited by 202
Stateful bulk processing for incremental analytics
Logothetis, D., Olston, C., Reed, B. C., Webb, K. C., & Yocum, K. (2010) · ACM Symposium on Cloud Computing (SoCC) · doi:10.1145/1807128.1807138 · Cited by 172
Comet: batched stream processing for data intensive distributed computing
He, B., Yang, M., Guo, Z., Chen, R., Su, B., Lin, W., & Zhou, L. (2010) · Proceedings of the 1st ACM symposium on Cloud computing · doi:10.1145/1807128.1807139 · Cited by 140
Nova: continuous Pig/Hadoop workflows
Olston, C., Chiou, G., Chitnis, L., Liu, F., Han, Y., Larsson, M., Neumann, A., Rao, V. B. N., Sankarasubramanian, V., Seth, S., et al. (2011) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/1989323.1989439 · Cited by 137
Adaptive functional programming
Acar, U. A., Blelloch, G. E., & Harper, R. W. (2006) · ACM Transactions on Programming Languages and Systems · doi:10.1145/1186632.1186634 · Cited by 120
Self-citationDryadInc: reusing work in large-scale computations
Popa, L., Budiu, M., Yu, Y., & Isard, M. (2009) · Cited by 89
Cited by
Chen, M., Mao, S., & Liu, Y. (2014) · Mobile Networks and Applications · doi:10.1007/s11036-013-0489-0 · Cited by 2,913
Toward Scalable Systems for Big Data Analytics: A Technology Tutorial
Hu, H., Wen, Y., Chua, T.-S., & Li, X. (2014) · IEEE Access · doi:10.1109/access.2014.2332453 · Cited by 1,099
“In addition, other research has focused on programming models for more specific tasks, such as cross joining two sets [173], iterative computation [174], [175], in-memory computation with fault-tolerance [176], incremental computation [177]–[180], and data-dependent flow control decision [181].”
BackgroundDiscretized streams: fault-tolerant streaming computation at scale
Zaharia, M., Das, T., Li, H., Hunter, T., Shenker, S., & Stoica, I. (2013) · Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles · doi:10.1145/2517349.2522737 · Cited by 982
Kineograph: taking the pulse of a fast-changing and connected world
Cheng, R., Hong, J., Kyrola, A., Miao, Y., Weng, X., Wu, M., Yang, F., Zhou, L., Zhao, F., & Chen, E. (2012) · Proceedings of the 7th ACM european conference on Computer Systems · doi:10.1145/2168836.2168846 · Cited by 247
“Incoop [2], Haloop [3], DryadInc [25], Yahoo continuous bulk processing (CBP) [16], Comet [11], and Nectar [10] achieve this by allowing their applications to reuse existing computation results.”
BackgroundTimeStream: reliable stream computation in the cloud
Qian, Z., He, Y., Su, C., Wu, Z., Zhu, H., Zhang, T., Zhou, L., Yu, Y., & Zhang, Z. (2013) · Proceedings of the 8th ACM European Conference on Computer Systems · doi:10.1145/2465351.2465353 · Cited by 227
“[11] BH ATOT I A , P., WIEDE R , A., RO D RI G U E S , R., ACA R , U. A., A N D PASQUI N , R. Incoop: MapReduce for incremental computations.”
MethodInfluential
Metadata
- Authors
- Bhatotia, P., Wieder, A., Rodrigues, R., Acar, U. A., & Pasquin, R.
- Venue
- Proceedings of the 2nd ACM Symposium on Cloud Computing
- Publisher
- ACM
- Published
- 2011-10-26
- Pages
- 1-14
- OpenAlex
- W2010929544
- Cited by
- 298
- References
- 14
- FWCI
- 32.85
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.