The DOI matches its Crossref record; 20 of the 20 references it cites were found in a registry or an index.
SkewTune: mitigating skew in mapreduce applications
Abstract
We present an automatic skew mitigation approach for user-defined MapReduce programs and present SkewTune, a system that implements this approach as a drop-in replacement for an existing MapReduce implementation. There are three key challenges: (a) require no extra input from the user yet work for all MapReduce applications, (b) be completely transparent, and (c) impose minimal overhead if there is no skew. The SkewTune approach addresses these challenges and works as follows: When a node in the cluster becomes idle, SkewTune identifies the task with the greatest expected remaining processing time. The unprocessed input data of this straggling task is then proactively repartitioned in a way that fully utilizes the nodes in the cluster and preserves the ordering of the input data so that the original output can be reconstructed by concatenation. We implement SkewTune as an extension to Hadoop and evaluate its effectiveness using several real applications. The results show that SkewTune can significantly reduce job runtime in the presence of skew and adds little to no overhead in the absence of skew.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
MapReduce: simplified data processing on large clusters
Dean, J., & Ghemawat, S. (2008) · Communications of the ACM · doi:10.1145/1327452.1327492 · Cited by 18,476
The anatomy of a large-scale hypertextual Web search engine
Brin, S., & Page, L. (1998) · Computer Networks and ISDN Systems · doi:10.1016/s0169-7552(98)00110-x · Cited by 16,004
Dryad: distributed data-parallel programs from sequential building blocks
Isard, M., Budiu, M., Yu, Y., Birrell, A., & Fetterly, D. (2007) · Proceedings of the 2nd ACM SIGOPS/EuroSys European Conference on Computer Systems 2007 · doi:10.1145/1272996.1273005 · Cited by 2,432
Improving MapReduce performance in heterogeneous environments
Zaharia, M., Konwinski, A., Joseph, A. D., Katz, R. H., & Stoica, I. G. (2008) · doi:10.5555/1855741.1855744 · Cited by 1,588
A comparison of approaches to large-scale data analysis
Pavlo, A., Paulson, E., Rasin, A., Abadi, D. J., DeWitt, D. J., Madden, S., & Stonebraker, M. (2009) · Proceedings of the 2009 ACM SIGMOD International Conference on Management of data · doi:10.1145/1559845.1559865 · Cited by 1,071
Hellerstein, J. M., Haas, P. J., & Wang, H. J. (1997) · Proceedings - ACM-SIGMOD International Conference on Management of Data · doi:10.1145/253260.253291 · Cited by 902
HaLoop: efficient iterative data processing on large clusters
Bu, Y., Howe, B., Balazinska, M., & Ernst, M. D. (2010) · Proceedings of the VLDB Endowment · doi:10.14778/1920841.1920881 · Cited by 693
Self-citationCondie, T., Conway, N., Alvaro, P., Hellerstein, J. M., Elmeleegy, K., & Sears, R. (2010) · doi:10.5555/1855711.1855732 · Cited by 673
Reining in the outliers in map-reduce clusters using Mantri
Ananthanarayanan, G., Kandula, S., Greenberg, A. G., Stoica, I. G., Lu, Y., Saha, B., & Harris, E. F. (2010) · Operating Systems Design and Implementation · doi:10.5555/1924943.1924962 · Cited by 636
CloudBurst: highly sensitive read mapping with MapReduce
Schatz, M. C. (2009) · Bioinformatics · doi:10.1093/bioinformatics/btp236 · Cited by 626
Cited by
Shark: SQL and rich analytics at scale
Xin, R. S., Rosen, J., Zaharia, M., Franklin, M. J., Shenker, S., & Stoica, I. (2013) · Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data · doi:10.1145/2463676.2465288 · Cited by 369
Health Big Data Analytics: A Technology Survey
Harerimana, G., Jang, B., Kim, J. W., & Park, H. K. (2018) · IEEE Access · doi:10.1109/access.2018.2878254 · Cited by 119
Metadata
- Authors
- Kwon, Y., Balazinska, M., Howe, B., & Rolia, J.
- Venue
- Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data
- Publisher
- ACM
- Published
- 2012-05-20
- Pages
- 25-36
- ISSN
- 0730-8078 (ISSN-L)
- OpenAlex
- W2140509629
- Cited by
- 400
- References
- 20
- FWCI
- 40.68
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: SkewTune: mitigating skew in mapreduce applications. https://app.cochat.ai/papers/10.1145/2213836.2213840
- The paper
- Kwon, Y., Balazinska, M., Howe, B., & Rolia, J. (2012). SkewTune: mitigating skew in mapreduce applications. Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data. https://doi.org/10.1145/2213836.2213840
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.