The DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
OLTP-Bench: an extensible testbed for benchmarking relational databases
Abstract
Benchmarking is an essential aspect of any database management system (DBMS) effort. Despite several recent advancements, such as pre-configured cloud database images and database-as-a-service (DBaaS) offerings, the deployment of a comprehensive testing platform with a diverse set of datasets and workloads is still far from being trivial. In many cases, researchers and developers are limited to a small number of workloads to evaluate the performance characteristics of their work. This is due to the lack of a universal benchmarking infrastructure, and to the difficulty of gaining access to real data and workloads. This results in lots of unnecessary engineering efforts and makes the performance evaluation results difficult to compare. To remedy these problems, we present OLTP-Bench, an extensible "batteries included" DBMS benchmarking testbed. The key contributions of OLTP-Bench are its ease of use and extensibility, support for tight control of transaction mixtures, request rates, and access distributions over time, as well as the ability to support all major DBMSs and DBaaS platforms. Moreover, it is bundled with fifteen workloads that all differ in complexity and system demands, including four synthetic workloads, eight workloads from popular benchmarks, and three workloads that are derived from real-world applications. We demonstrate through a comprehensive set of experiments conducted on popular DBMS and DBaaS offerings the different features provided by OLTP-Bench and the effectiveness of our testbed in characterizing the performance of database services.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Benchmarking cloud serving systems with YCSB
Cooper, B. F., Silberstein, A., Tam, E., Ramakrishnan, R., & Sears, R. (2010) · Proceedings of the 1st ACM symposium on Cloud computing · doi:10.1145/1807128.1807152 · Cited by 3,716
Measuring user influence in twitter: The million follower fallacy
Cha, M., Haddadi, H., Benevenuto, F., & Gummadi, K. (2010) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v4i1.14033 · Cited by 3,022 · cited 1×
“In order to provide a realistic benchmark, we obtained an anonymized snapshot of the Twitter social graph from August 2009 that contains 51 million users and almost 2 billion “follows” relationships [14].”
BackgroundScalable SQL and NoSQL data stores
Cattell, R. (2011) · ACM SIGMOD Record · doi:10.1145/1978915.1978919 · Cited by 1,358
Runtime measurements in the cloud: observing, analyzing, and reducing variance
Schad, J., Dittrich, J., & Quiané-Ruiz, J.-A. (2010) · Proceedings of the VLDB Endowment · doi:10.14778/1920841.1920902 · Cited by 670
Wikipedia workload analysis for decentralized hosting
Urdaneta, G., Pierre, G., & van Steen, M. (2009) · Computer Networks · doi:10.1016/j.comnet.2009.02.019 · Cited by 368
Controversial users demand local trust metrics: An experimental study on epinions.com community
Massa, P., & Avesani, P. (2005) · Cited by 341
Arasu, A., Maskey, A. S., Cherniack, M., Ryvkina, E., Galvez, E., Stonebraker, M., Maier, D., & Tibbetts, R. (2004) · Proceedings 2004 VLDB Conference · doi:10.1016/b978-012088469-8.50044-9 · Cited by 322
LinkBench: a database benchmark based on the Facebook social graph
Armstrong, T. G., Ponnekanti, V., Borthakur, D., & Callaghan, M. (2013) · Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data · doi:10.1145/2463676.2465296 · Cited by 313
Morgan Kaufmann Publishers, Inc.
(1986) · IEEE Software · doi:10.1109/ms.1986.233092 · Cited by 311
Serializable isolation for snapshot databases
Cahill, M. J., Röhm, U., & Fekete, A. D. (2009) · ACM Transactions on Database Systems · doi:10.1145/1620585.1620587 · Cited by 195
YCSB++: benchmarking and performance debugging advanced features in scalable table stores
Patil, S., Polte, M., Ren, K., Tantisiriroj, W., Xiao, L., López, J., Gibson, G., Fuchs, A., & Rinaldi, B. (2011) · Proceedings of the 2nd ACM Symposium on Cloud Computing · doi:10.1145/2038916.2038925 · Cited by 178
Serializable isolation for snapshot databases
Cahill, M. J., Röhm, U., & Fekete, A. D. (2008) · Proceedings of the 2008 ACM SIGMOD international conference on Management of data · doi:10.1145/1376616.1376690 · Cited by 120
From Active Data Management to Event-Based Systems and More
Sachs, K., Guerrero, P., & Petrov, I. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-17226-7 · Cited by 104
Cited by
BLOCKBENCH: A Framework for Analyzing Private Blockchains
Dinh, T. T. A., Wang, J., Chen, G., Liu, R., Ooi, B. C., & Tan, K.-L. (2017) · Proceedings of the 2017 ACM International Conference on Management of Data · doi:10.1145/3035918.3064033 · Cited by 814
Automatic Database Management System Tuning Through Large-scale Machine Learning
Van Aken, D., Pavlo, A., Gordon, G. J., & Zhang, B. (2017) · Proceedings of the 2017 ACM International Conference on Management of Data · doi:10.1145/3035918.3064029 · Cited by 573
Self-citationPerformance Evaluation of Blockchain Systems: A Systematic Survey
Fan, C., Ghaemi, S., Khazaei, H., & Musilek, P. (2020) · IEEE Access · doi:10.1109/access.2020.3006078 · Cited by 294
A machine learning approach to live migration modeling
Jo, C., Cho, Y., & Egger, B. (2017) · Proceedings of the 2017 Symposium on Cloud Computing · doi:10.1145/3127479.3129262 · Cited by 66
GPTuner: An LLM-Based Database Tuning System
Lao, J., Wang, Y., Li, Y., Wang, J., Zhang, Y., Cheng, Z., Chen, W., Tang, M., & Wang, J. (2025) · ACM SIGMOD Record · doi:10.1145/3733620.3733641 · Cited by 17
Metadata
- Authors
- Difallah, D. E., Pavlo, A., Curino, C., & Cudre-Mauroux, P.
- Venue
- Proceedings of the VLDB Endowment
- Published
- 2013-12
- Volume
- 7
- Issue
- 4
- Pages
- 277-288
- ISSN
- 2150-8097 (ISSN-L)
- OpenAlex
- W2240667924
- Cited by
- 392
- References
- 13
- FWCI
- 22.91
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.