The DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index.
Benchmarking cloud serving systems with YCSB
Abstract
While the use of MapReduce systems (such as Hadoop) for large scale data analysis has been widely recognized and studied, we have recently seen an explosion in the number of systems developed for cloud data serving. These newer systems address "cloud OLTP" applications, though they typically do not support ACID transactions. Examples of systems proposed for cloud serving use include BigTable, PNUTS, Cassandra, HBase, Azure, CouchDB, SimpleDB, Voldemort, and many others. Further, they are being applied to a diverse range of applications that differ considerably from traditional (e.g., TPC-C like) serving workloads. The number of emerging cloud serving systems and the wide range of proposed applications, coupled with a lack of apples-to-apples performance comparisons, makes it difficult to understand the tradeoffs between systems and the workloads for which they are suited. We present the "Yahoo! Cloud Serving Benchmark" (YCSB) framework, with the goal of facilitating performance comparisons of the new generation of cloud data serving systems. We define a core set of benchmarks and report results for four widely used systems: Cassandra, HBase, Yahoo!'s PNUTS, and a simple sharded MySQL implementation. We also hope to foster the development of additional cloud benchmark suites that represent other classes of applications by making our benchmark tool available via open source. In this regard, a key feature of the YCSB framework/tool is that it is extensible--it supports easy definition of new workloads, in addition to making it easy to benchmark new systems.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Bigtable: A Distributed Storage System for Structured Data
Chang, F., Dean, J., Ghemawat, S., Hsieh, W. C., Wallach, D. A., Burrows, M., Chandra, T., Fikes, A., & Gruber, R. E. (2008) · ACM Transactions on Computer Systems · doi:10.1145/1365815.1365816 · Cited by 3,460
(2007) · doi:10.1145/1294261.1294281 · Cited by 3,447 · cited 2×
“…model similar to that of Google’s BigTable [16], their underlying implementations are quite different—HBase’s architecture is similar to BigTable (using synchronous updates to multiple copies of data chunks), while Cassandra’s is similar to Dynamo [18] (e.g., using gossip and eventual consistency).”
BackgroundMethodBrewer's conjecture and the feasibility of consistent, available, partition-tolerant web services
Gilbert, S., & Lynch, N. (2002) · ACM SIGACT News · doi:10.1145/564585.564601 · Cited by 1,727
An integrated experimental environment for distributed systems and networks
White, B., Lepreau, J., Stoller, L. B., Ricci, R., Guruprasad, S., Newbold, M., Hibler, M., Barb, C., & Joglekar, A. A. (2002) · ACM SIGOPS Operating Systems Review · doi:10.1145/844128.844152 · Cited by 1,302
A comparison of approaches to large-scale data analysis
Pavlo, A., Paulson, E., Rasin, A., Abadi, D. J., DeWitt, D. J., Madden, S., & Stonebraker, M. (2009) · Proceedings of the 2009 ACM SIGMOD International Conference on Management of data · doi:10.1145/1559845.1559865 · Cited by 1,071
PNUTS: Yahoo!'s hosted data serving platform
Cooper, B. F., Ramakrishnan, R., Srivastava, U., Silberstein, A., Bohannon, P., Jacobsen, H.-A., Puz, N., Weaver, D., & Yerneni, R. (2008) · Proceedings of the VLDB Endowment · doi:10.14778/1454159.1454167 · Cited by 958
Self-citation(2008) · Developing with Google App Engine · doi:10.1007/978-1-4302-1832-6_1 · Cited by 610
Arasu, A., Maskey, A. S., Cherniack, M., Ryvkina, E., Galvez, E., Stonebraker, M., Maier, D., & Tibbetts, R. (2004) · Proceedings 2004 VLDB Conference · doi:10.1016/b978-012088469-8.50044-9 · Cited by 322
Cassandra: structured storage system on a P2P network
Lakshman, A., & Malik, P. (2009) · Proceedings of the 28th ACM symposium on Principles of distributed computing · doi:10.1145/1582716.1582722 · Cited by 231
Belazzougui, D., Botelho, F. C., & Dietzfelbinger, M. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-04128-0_61 · Cited by 125
Cited by
Applications of Deep Reinforcement Learning in Communications and Networking: A Survey
Luong, N. C., Hoang, D. T., Gong, S., Niyato, D., Wang, P., Liang, Y.-C., & Kim, D. I. (2019) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2916583 · Cited by 2,060
Scalable SQL and NoSQL data stores
Cattell, R. (2011) · ACM SIGMOD Record · doi:10.1145/1978915.1978919 · Cited by 1,358
Untangling Blockchain: A Data Processing View of Blockchain Systems
Dinh, T. T. A., Liu, R., Zhang, M., Chen, G., Ooi, B. C., & Wang, J. (2018) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2017.2781227 · Cited by 1,186
Toward Scalable Systems for Big Data Analytics: A Technology Tutorial
Hu, H., Wen, Y., Chua, T.-S., & Li, X. (2014) · IEEE Access · doi:10.1109/access.2014.2332453 · Cited by 1,099
Borkar, S., & Chien, A. A. (2011) · Communications of the ACM · doi:10.1145/1941487.1941507 · Cited by 966
A framework for ranking of cloud computing services
Garg, S. K., Versteeg, S., & Buyya, R. (2013) · Future Generation Computer Systems · doi:10.1016/j.future.2012.06.006 · Cited by 887
CloudCmp: comparing public cloud providers
Li, A., Yang, X., Kandula, S., & Zhang, M. (2010) · Proceedings of the 10th ACM SIGCOMM conference on Internet measurement · doi:10.1145/1879141.1879143 · Cited by 882
Next generation cloud computing: New trends and research directions
Varghese, B., & Buyya, R. (2018) · Future Generation Computer Systems · doi:10.1016/j.future.2017.09.020 · Cited by 857
Tsai, C.-W., Lai, C.-F., Chao, H.-C., & Vasilakos, A. V. (2015) · Journal of Big Data · doi:10.1186/s40537-015-0030-3 · Cited by 818
Clearing the clouds: a study of emerging scale-out workloads on modern hardware
Ferdman, M., Adileh, A., Kocberber, O., Volos, S., Alisafaee, M., Jevdjic, D., Kaynak, C., Popescu, A. D., Ailamaki, A., & Falsafi, B. (2012) · Proceedings of the seventeenth international conference on Architectural Support for Programming Languages and Operating Systems · doi:10.1145/2150976.2150982 · Cited by 814
BLOCKBENCH: A Framework for Analyzing Private Blockchains
Dinh, T. T. A., Wang, J., Chen, G., Liu, R., Ooi, B. C., & Tan, K.-L. (2017) · Proceedings of the 2017 ACM International Conference on Management of Data · doi:10.1145/3035918.3064033 · Cited by 814
Runtime measurements in the cloud: observing, analyzing, and reducing variance
Schad, J., Dittrich, J., & Quiané-Ruiz, J.-A. (2010) · Proceedings of the VLDB Endowment · doi:10.14778/1920841.1920902 · Cited by 670
SCONE: secure Linux containers with Intel SGX
(2016) · Operating Systems Design and Implementation · doi:10.5555/3026877.3026930 · Cited by 582
Automatic Database Management System Tuning Through Large-scale Machine Learning
Van Aken, D., Pavlo, A., Gordon, G. J., & Zhang, B. (2017) · Proceedings of the 2017 ACM International Conference on Management of Data · doi:10.1145/3035918.3064029 · Cited by 573
Challenges and opportunities in edge computing
Varghese, B., Wang, N., Barbhuiya, S., Kilpatrick, P., & Nikolopoulos, D. S. (2016) · 2016 IEEE International Conference on Smart Cloud (SmartCloud) · doi:10.1109/smartcloud.2016.18 · Cited by 565
NetCache: Balancing Key-Value Stores with Fast In-Network Caching
Jin, X., Li, X., Zhang, H., Soulé, R., Lee, J., Foster, N., Kim, C., & Stoica, I. (2017) · Proceedings of the 26th Symposium on Operating Systems Principles · doi:10.1145/3132747.3132764 · Cited by 556
Schism: a workload-driven approach to database replication and partitioning
Curino, C., Jones, E., Zhang, Y., & Madden, S. (2010) · Proceedings of the VLDB Endowment · doi:10.14778/1920841.1920853 · Cited by 539
Cache craftiness for fast multicore key-value storage
Mao, Y., Kohler, E., & Morris, R. T. (2012) · Proceedings of the 7th ACM european conference on Computer Systems · doi:10.1145/2168836.2168855 · Cited by 467
Data-intensive text processing with MapReduce
Lin, J., & Dyer, C. (2009) · Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Companion Volume: Tutorial Abstracts on - NAACL '09 · doi:10.3115/1620950.1620951 · Cited by 445
The Case for Evaluating MapReduce Performance Using Workload Suites
Chen, Y., Ganapathi, A., Griffith, R., & Katz, R. (2011) · 2011 IEEE 19th Annual International Symposium on Modelling, Analysis, and Simulation of Computer and Telecommunication Systems · doi:10.1109/mascots.2011.12 · Cited by 428
A Survey of Large Scale Data Management Approaches in Cloud Environments
Sakr, S., Liu, A., Batista, D. M., & Alomari, M. (2011) · IEEE Communications Surveys & Tutorials · doi:10.1109/surv.2011.032211.00087 · Cited by 424
BigDataBench: A big data benchmark suite from internet services
Wang, L., Zhan, J., Luo, C., Zhu, Y., Yang, Q., He, Y., Gao, W., Jia, Z., Shi, Y., Zhang, S., et al. (2014) · 2014 IEEE 20th International Symposium on High Performance Computer Architecture (HPCA) · doi:10.1109/hpca.2014.6835958 · Cited by 422
Using RDMA efficiently for key-value services
Kalia, A., Kaminsky, M., & Andersen, D. G. (2014) · Proceedings of the 2014 ACM conference on SIGCOMM · doi:10.1145/2619239.2626299 · Cited by 395
OLTP-Bench: an extensible testbed for benchmarking relational databases
Difallah, D. E., Pavlo, A., Curino, C., & Cudre-Mauroux, P. (2013) · Proceedings of the VLDB Endowment · doi:10.14778/2732240.2732246 · Cited by 392
MICA: A Holistic Approach to Fast In-Memory Key-Value Storage
Lim, H., Han, D., Andersen, D. G., & Kaminsky, M. (2018) · Figshare · doi:10.1184/r1/6607280 · Cited by 361
Consistent and durable data structures for non-volatile byte-addressable memory
Venkataraman, S., Tolia, N. H., Ranganathan, P., & Campbell, R. H. (2011) · doi:10.5555/1960475.1960480 · Cited by 356
Bubble-flux: precise online QoS management for increased utilization in warehouse scale computers
Yang, H., Breslow, A., Mars, J., & Tang, L. (2013) · Proceedings of the 40th Annual International Symposium on Computer Architecture · doi:10.1145/2485922.2485974 · Cited by 345
BigBench: towards an industry standard benchmark for big data analytics
Ghazal, A., Rabl, T., Hu, M., Raab, F., Poess, M., Crolotte, A., & Jacobsen, H.-A. (2013) · Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data · doi:10.1145/2463676.2463712 · Cited by 343
Data management in cloud environments: NoSQL and NewSQL data stores
Grolinger, K., Higashino, W. A., Tiwari, A., & Capretz, M. A. (2013) · Journal of Cloud Computing: Advances, Systems and Applications · doi:10.1186/2192-113x-2-22 · Cited by 318
LinkBench: a database benchmark based on the Facebook social graph
Armstrong, T. G., Ponnekanti, V., Borthakur, D., & Callaghan, M. (2013) · Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data · doi:10.1145/2463676.2465296 · Cited by 313
Security, Performance, and Applications of Smart Contracts: A Systematic Survey
Rouhani, S., & Deters, R. (2019) · IEEE Access · doi:10.1109/access.2019.2911031 · Cited by 309
WiscKey: Separating Keys from Values in SSD-Conscious Storage
Lu, L., Pillai, T. S., Gopalakrishnan, H., Arpaci-Dusseau, A. C., & Arpaci-Dusseau, R. H. (2017) · ACM Transactions on Storage · doi:10.1145/3033273 · Cited by 301
MemC3: Compact and Concurrent MemCache with Dumber Caching and Smarter Hashing
Fan, B., Andersen, D. G., & Kaminsky, M. (2018) · doi:10.1184/r1/6607223 · Cited by 296
Basic Performance Measurements of the Intel Optane DC Persistent Memory Module
Izraelevitz, J., Yang, J., Zhang, L., Kim, J., Liu, X., Memaripour, A., Soh, Y. J., Wang, Z., Xu, Y., Dulloor, S. R., et al. (2019) · arXiv · doi:10.48550/arxiv.1903.05714 · Cited by 295
SILT: a memory-efficient, high-performance key-value store
Lim, H., Fan, B., Andersen, D. G., & Kaminsky, M. (2011) · Proceedings of the Twenty-Third ACM Symposium on Operating Systems Principles · doi:10.1145/2043556.2043558 · Cited by 294
Performance Evaluation of Blockchain Systems: A Systematic Survey
Fan, C., Ghaemi, S., Khazaei, H., & Musilek, P. (2020) · IEEE Access · doi:10.1109/access.2020.3006078 · Cited by 294
Characterizing Microservice Dependency and Performance: Alibaba Trace Analysis
Luo, S., Xu, H., Lu, C., Ye, K., Xu, G., Zhang, L., Ding, Y., He, J., & Xu, C. (2021) · Proceedings of the ACM Symposium on Cloud Computing · doi:10.1145/3472883.3487003 · Cited by 293
Ousterhout, J., Gopalan, A., Gupta, A., Kejriwal, A., Lee, C., Montazeri, B., Ongaro, D., Park, S. J., Qin, H., Rosenblum, M., et al. (2015) · ACM Transactions on Computer Systems · doi:10.1145/2806887 · Cited by 268
PARTIES: QoS-Aware Resource Partitioning for Multiple Interactive Services
Chen, S., Delimitrou, C., & Martínez, J. F. (2019) · Proceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems · doi:10.1145/3297858.3304005 · Cited by 267
A performance comparison of SQL and NoSQL databases
Li, Y., & Manoharan, S. (2013) · 2013 IEEE Pacific Rim Conference on Communications, Computers and Signal Processing (PACRIM) · doi:10.1109/pacrim.2013.6625441 · Cited by 262
CockroachDB: The Resilient Geo-Distributed SQL Database
Taft, R., Sharif, I., Matei, A., VanBenschoten, N., Lewis, J., Grieger, T., Niemi, K., Woods, A., Birzin, A., Poss, R., et al. (2020) · Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data · doi:10.1145/3318464.3386134 · Cited by 254
(2015) · doi:10.1145/2815400.2815415 · Cited by 202
A Large-scale Analysis of Hundreds of In-memory Key-value Cache Clusters at Twitter
Yang, J., Yue, Y., & Rashmi, K. V. (2021) · ACM Transactions on Storage · doi:10.1145/3468521 · Cited by 109
Bergman, S., Cassel, N., Bjørling, M., & Silberstein, M. (2023) · ACM Transactions on Storage · doi:10.1145/3582434 · Cited by 38
Yuan, L., He, Q., Tan, S., Li, B., Yu, J., Chen, F., & Yang, Y. (2023) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2022.3231296 · Cited by 33
Metadata
- Authors
- Cooper, B. F., Silberstein, A., Tam, E., Ramakrishnan, R., & Sears, R.
- Venue
- Proceedings of the 1st ACM symposium on Cloud computing
- Publisher
- ACM
- Published
- 2010-06-10
- Pages
- 143-154
- OpenAlex
- W1985229168
- Cited by
- 3,716
- References
- 10
- FWCI
- 88.60
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.