The DOI matches its Crossref record.
A comparison of approaches to large-scale data analysis
Abstract
There is currently considerable enthusiasm around the MapReduce (MR) paradigm for large-scale data analysis [17]. Although the basic control flow of this framework has existed in parallel SQL database management systems (DBMS) for over 20 years, some have called MR a dramatically new computing model [8, 17]. In this paper, we describe and compare both paradigms. Furthermore, we evaluate both kinds of systems in terms of performance and development complexity. To this end, we define a benchmark consisting of a collection of tasks that we have run on an open source version of MR as well as on two parallel DBMSs. For each task, we measure each system's performance for various degrees of parallelism on a cluster of 100 nodes. Our results reveal some interesting trade-offs. Although the process to load data into and tune the execution of parallel DBMSs took much longer than the MR system, the observed performance of these DBMSs was strikingly better. We speculate about the causes of the dramatic performance difference and consider implementation concepts that future systems should take from both kinds of architectures.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cited by
Data-intensive applications, challenges, techniques and technologies: A survey on Big Data
Philip Chen, C. L., & Zhang, C.-Y. (2014) · Information Sciences · doi:10.1016/j.ins.2014.01.015 · Cited by 3,047
Toward Scalable Systems for Big Data Analytics: A Technology Tutorial
Hu, H., Wen, Y., Chua, T.-S., & Li, X. (2014) · IEEE Access · doi:10.1109/access.2014.2332453 · Cited by 1,099
Low Latency Geo-distributed Data Analytics
Pu, Q., Ananthanarayanan, G., Bodik, P., Kandula, S., Akella, A., Bahl, P., & Stoica, I. (2015) · Proceedings of the 2015 ACM Conference on Special Interest Group on Data Communication · doi:10.1145/2785956.2787505 · Cited by 246
Computational Health Informatics in the Big Data Age: A Survey
Fang, R., Pouyanfar, S., Yang, Y., Chen, S.-C., & Iyengar, S. S. (2017) · ACM Computing Surveys · doi:10.1145/2932707 · Cited by 210
What Goes Around Comes Around... And Around...
Stonebraker, M., & Pavlo, A. (2024) · ACM SIGMOD Record · doi:10.1145/3685980.3685984 · Cited by 29
Self-citation
Metadata
- Authors
- Pavlo, A., Paulson, E., Rasin, A., Abadi, D. J., DeWitt, D. J., Madden, S., & Stonebraker, M.
- Venue
- Proceedings of the 2009 ACM SIGMOD International Conference on Management of data
- Publisher
- ACM
- Published
- 2009-06-29
- Pages
- 165-178
- ISSN
- 0730-8078 (ISSN-L)
- OpenAlex
- W2114303224
- Cited by
- 1,071
- References
- 0
- FWCI
- 76.45
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: A comparison of approaches to large-scale data analysis. https://app.cochat.ai/papers/10.1145/1559845.1559865
- The paper
- Pavlo, A., Paulson, E., Rasin, A., Abadi, D. J., DeWitt, D. J., Madden, S., & Stonebraker, M. (2009). A comparison of approaches to large-scale data analysis. Proceedings of the 2009 ACM SIGMOD International Conference on Management of data. https://doi.org/10.1145/1559845.1559865
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
