The DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
RCFile: A fast and space-efficient data placement structure in MapReduce-based warehouse systems
Abstract
MapReduce-based data warehouse systems are playing important roles of supporting big data analytics to understand quickly the dynamics of user behavior trends and their needs in typical Web service providers and social network sites (e.g., Facebook). In such a system, the data placement structure is a critical factor that can affect the warehouse performance in a fundamental way. Based on our observations and analysis of Facebook production systems, we have characterized four requirements for the data placement structure: (1) fast data loading, (2) fast query processing, (3) highly efficient storage space utilization, and (4) strong adaptivity to highly dynamic workload patterns. We have examined three commonly accepted data placement structures in conventional databases, namely row-stores, column-stores, and hybrid-stores in the context of large data analysis using MapReduce. We show that they are not very suitable for big data processing in distributed systems. In this paper, we present a big data placement structure called RCFile (Record Columnar File) and its implementation in the Hadoop system. With intensive experiments, we show the effectiveness of RCFile in satisfying the four requirements. RCFile has been chosen in Facebook data warehouse system as the default option. It has also been adopted by Hive and Pig, the two most widely used data analysis systems developed in Facebook and Yahoo!
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
MapReduce: simplified data processing on large clusters
Dean, J., & Ghemawat, S. (2008) · Communications of the ACM · doi:10.1145/1327452.1327492 · Cited by 18,476
Bigtable: a distributed storage system for structured data
Chang, F. W., Dean, J. B., Ghemawat, S., Hsieh, W. C., Wallach, D. A., Burrows, M., Chandra, T., Fikes, A., & Gruber, R. E. (2006) · doi:10.5555/1298455.1298475 · Cited by 1,928
Hive: a warehousing solution over a map-reduce framework
Thusoo, A., Sarma, J. S., Jain, N., Shao, Z., Chakka, P., Anthony, S., Liu, H., Wyckoff, P., & Murthy, R. (2009) · Proceedings of the VLDB Endowment · doi:10.14778/1687553.1687609 · Cited by 1,556
Self-citation(2014) · Elsevier eBooks · doi:10.1016/b978-0-12-799957-9.00002-x · Cited by 1,313
A comparison of approaches to large-scale data analysis
Pavlo, A., Paulson, E., Rasin, A., Abadi, D. J., DeWitt, D. J., Madden, S., & Stonebraker, M. (2009) · Proceedings of the 2009 ACM SIGMOD International Conference on Management of data · doi:10.1145/1559845.1559865 · Cited by 1,071
Hive - a petabyte scale data warehouse using Hadoop
Thusoo, A., Sarma, J. S., Jain, N., Shao, Z., Chakka, P., Zhang, N., Antony, S., Liu, H., & Murthy, R. (2010) · 2010 IEEE 26th International Conference on Data Engineering (ICDE 2010) · doi:10.1109/icde.2010.5447738 · Cited by 929
Self-citationHadoopDB: an architectural hybrid of MapReduce and DBMS technologies for analytical workloads
Abouzeid, A., Bajda-Pawlikowski, K., Abadi, D., Silberschatz, A., & Rasin, A. (2009) · Proceedings of the VLDB Endowment · doi:10.14778/1687627.1687731 · Cited by 857
Leon, A. (2003) · Cited by 536
Column-stores vs. row-stores: how different are they really?
Abadi, D. J., Madden, S. R., & Hachem, N. (2008) · Proceedings of the 2008 ACM SIGMOD international conference on Management of data · doi:10.1145/1376616.1376712 · Cited by 436
Data warehousing and analytics infrastructure at facebook
Thusoo, A., Shao, Z., Anthony, S., Borthakur, D., Jain, N., Sen Sarma, J., Murthy, R., & Liu, H. (2010) · Proceedings of the 2010 ACM SIGMOD International Conference on Management of data · doi:10.1145/1807167.1807278 · Cited by 425
Self-citation
Cited by
Parallel data processing with MapReduce: a survey
Lee, K.-H., Lee, Y.-J., Choi, H., Chung, Y. D., & Moon, B. (2012) · ACM SIGMOD Record · doi:10.1145/2094114.2094118 · Cited by 558
Big Data Analytics: A Literature Review Paper
Elgendy, N., & Elragal, A. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-08976-8_16 · Cited by 457
Understandable Big Data: A survey
Kacfah Emani, C., Cullot, N., & Nicolle, C. (2015) · Computer Science Review · doi:10.1016/j.cosrev.2015.05.002 · Cited by 299
Understanding Human-Machine Networks: A Cross-Disciplinary Survey
Tsvetkova, M., Yasseri, T., Meyer, E. T., Pickering, J. B., Engen, V., Walland, P., Lüders, M., Følstad, A., & Bravos, G. (2018) · ACM Computing Surveys · doi:10.1145/3039868 · Cited by 67
Metadata
- Authors
- He, Y., Lee, R., Huai, Y., Shao, Z., Jain, N., Zhang, X., & Xu, Z.
- Venue
- 2011 IEEE 27th International Conference on Data Engineering
- Publisher
- IEEE
- Published
- 2011-04
- Pages
- 1199-1208
- ISSN
- 1084-4627 (ISSN-L)
- OpenAlex
- W2172220707
- Cited by
- 260
- References
- 15
- FWCI
- 24.52
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: RCFile: A fast and space-efficient data placement structure in MapReduce-based warehouse systems. https://app.cochat.ai/papers/10.1109/icde.2011.5767933
- The paper
- He, Y., Lee, R., Huai, Y., Shao, Z., Jain, N., Zhang, X., & Xu, Z. (2011). RCFile: A fast and space-efficient data placement structure in MapReduce-based warehouse systems. 2011 IEEE 27th International Conference on Data Engineering. https://doi.org/10.1109/icde.2011.5767933
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.