Let's Talk About Storage & Recovery Methods for Non-Volatile Memory Database Systems
The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Abstract
The advent of non-volatile memory (NVM) will fundamentally change the dichotomy between memory and durable storage in database management systems (DBMSs). These new NVM devices are almost as fast as DRAM, but all writes to it are potentially persistent even after power loss. Existing DBMSs are unable to take full advantage of this technology because their internal architectures are predicated on the assumption that memory is volatile. With NVM, many of the components of legacy DBMSs are unnecessary and will degrade the performance of data intensive applications.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Strukov, D. B., Snider, G. S., Stewart, D. R., & Williams, R. S. (2008) · Nature · doi:10.1038/nature06932 · Cited by 11,828 · cited 4×
“NVM technologies, like phase-change memory [55] and memristors [60], remove these tuple transformation and propagation costs through byte-addressable loads and stores with low latency.”
BackgroundInfluentialSpace/time trade-offs in hash coding with allowable errors
Bloom, B. H. (1970) · Communications of the ACM · doi:10.1145/362686.362692 · Cited by 7,492 · cited 2×
“The Log engine also constructs a Bloom filter [12] for each SSTable to quickly determine at runtime whether it contains entries associated with a tuple to avoid unnecessary index look-ups.”
MethodBenchmarking cloud serving systems with YCSB
Cooper, B. F., Silberstein, A., Tam, E., Ramakrishnan, R., & Sears, R. (2010) · Proceedings of the 1st ACM symposium on Cloud computing · doi:10.1145/1807128.1807152 · Cited by 3,716 · cited 2×
“YCSB: This is a widely-used key-value store workload from Yahoo! [20].”
MethodThe log-structured merge-tree (LSM-tree)
O’Neil, P., Cheng, E., Gawlick, D., & O’Neil, E. (1996) · Acta Informatica · doi:10.1007/s002360050048 · Cited by 1,317 · cited 3×
“The LSM tree reduces write amplification by batching the updates in MemTable and periodically cascading the changes to durable storage [50].”
BackgroundMethodThe design and implementation of a log-structured file system
Rosenblum, M., & Ousterhout, J. K. (1992) · ACM Transactions on Computer Systems · doi:10.1145/146941.146943 · Cited by 1,268
System R: relational approach to database management
Astrahan, M. M., Blasgen, M. W., Chamberlin, D. D., Eswaran, K. P., Gray, J. N., Griffiths, P. P., King, W. F., Lorie, R. A., McJones, P. R., Mehl, J. W., et al. (1976) · ACM Transactions on Database Systems · doi:10.1145/320455.320457 · Cited by 1,044 · cited 1×
“The former is exemplified by the first DBMSs, such as IBM’s System R [7], where the system is predicated on the management of blocks of tuples on disk using an in-memory cache; the latter by IBM’s IMS/VS Fast Path [32], where the system performs updates on in-memory data and relies on the disk to ensure durability.”
MethodMohan, C., Haderle, D., Lindsay, B., Pirahesh, H., & Schwarz, P. (1992) · ACM Transactions on Database Systems · doi:10.1145/128765.128770 · Cited by 1,043 · cited 3×
“With ARIES, the engine periodically takes checkpoints that are stored on the filesystem to bound recovery latency and reduce the storage space consumed by the log.”
BackgroundPhase-change random access memory: A scalable technology
Raoux, S., Burr, G. W., Breitwisch, M. J., Rettner, C. T., Chen, Y.-C., Shelby, R. M., Salinga, M., Krebs, D., Chen, S.-H., Lung, H.-L., et al. (2008) · IBM Journal of Research and Development · doi:10.1147/rd.524.0465 · Cited by 1,010 · cited 5×
“NVM technologies, like phase-change memory [55] and memristors [60], remove these tuple transformation and propagation costs through byte-addressable loads and stores with low latency.”
BackgroundInfluentialThe design and implementation of a log-structured file system
Rosenblum, M., & Ousterhout, J. K. (1991) · Proceedings of the thirteenth ACM symposium on Operating systems principles · doi:10.1145/121132.121137 · Cited by 970 · cited 1×
“This approach originated from log-structured filesystems [57], and then it was adapted to DBMSs as log-structured merge (LSM) trees [50] for write-intensive workloads.”
MethodImplementation techniques for main memory database systems
DeWitt, D. J., Katz, R. H., Olken, F., Shapiro, L. D., Stonebraker, M. R., & Wood, D. (1984) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/602259.602261 · Cited by 879 · cited 1×
“This means that they can be used for efficient architectures that are used in memory-oriented DBMSs [25].”
MethodBetter I/O through byte-addressable, persistent memory
Condit, J., Nightingale, E. B., Frost, C., Ipek, E., Lee, B., Burger, D., & Coetzee, D. (2009) · Proceedings of the ACM SIGOPS 22nd symposium on Operating systems principles · doi:10.1145/1629575.1629589 · Cited by 824 · cited 1×
“BPFS is a filesystem designed for persistent memory [19].”
BackgroundMnemosyne: lightweight persistent memory
Volos, H., Tack, A. J., & Swift, M. M. (2011) · Proceedings of the sixteenth international conference on Architectural support for programming languages and operating systems · doi:10.1145/1950365.1950379 · Cited by 711 · cited 1×
“Mnemosyne [64] and NVheaps [18] use software transactional memory to support transactional updates to data stored on NVM.”
MethodOverview of candidate device technologies for storage-class memory
Burr, G. W., Kurdi, B. N., Scott, J. C., Lam, C. H., Gopalakrishnan, K., & Shenoy, R. S. (2008) · IBM Journal of Research and Development · doi:10.1147/rd.524.0449 · Cited by 687 · cited 9×
“Shrinking SSDs to smaller sizes also degrades their reliability and increases interference effects.”
BackgroundInfluentialNV-Heaps: making persistent objects fast and safe with next-generation, non-volatile memories
Coburn, J., Caulfield, A. M., Akel, A., Grupp, L. M., Gupta, R. K., Jhala, R., & Swanson, S. (2011) · Proceedings of the sixteenth international conference on Architectural support for programming languages and operating systems · doi:10.1145/1950365.1950380 · Cited by 646 · cited 1×
“Mnemosyne [64] and NVheaps [18] use software transactional memory to support transactional updates to data stored on NVM.”
MethodSystem software for persistent memory
Dulloor, S. R., Kumar, S., Keshavamurthy, A., Lantz, P., Reddy, D., Sankaran, R., & Jackson, J. (2014) · Proceedings of the Ninth European Conference on Computer Systems · doi:10.1145/2592798.2592814 · Cited by 617 · cited 9×
“For this reason, we use a NVM hardware emulator developed by Intel Labs [27] in this paper.”
BackgroundMethodInfluentialSelf-citationThe end of an architectural era: (it's time for a complete rewrite)
Stonebraker, M. R., Madden, S. R., Abadi, D. J., Harizopoulos, S., Hachem, N. I., & Helland, P. (2007) · Cited by 558
Main memory database systems: an overview
García-Molina, H., & Salem, K. (1992) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/69.180602 · Cited by 545 · cited 1×
“This has involved optimizing the layout of data for each storage layer depending on how fast it can perform random accesses [31].”
MethodEnergy management for commercial servers
Lefurgy, C., Rajamani, K., Rawson, F., Felter, W., Kistler, M., & Keller, T. W. (2003) · Computer · doi:10.1109/mc.2003.1250880 · Cited by 473 · cited 1×
“Stop-gap solutions, such as battery-backed DRAM caches, help mitigate the performance difference but do not resolve these other problems [11].”
BackgroundConsistent and durable data structures for non-volatile byte-addressable memory
Venkataraman, S., Tolia, N. H., Ranganathan, P., & Campbell, R. H. (2011) · doi:10.5555/1960475.1960480 · Cited by 356
OLTP through the looking glass, and what we found there
Harizopoulos, S., Abadi, D. J., Madden, S. R., & Stonebraker, M. R. (2008) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/1376616.1376713 · Cited by 316 · cited 1×
“Previous studies have shown that the overhead of managing this data movement for OLTP workloads is considerable [35].”
BackgroundThe Computer science and engineering handbook
(1997) · Choice Reviews Online · doi:10.5860/choice.35-2165 · Cited by 313
Skew-aware automatic database partitioning in shared-nothing, parallel OLTP systems
Pavlo, A., Curino, C., & Zdonik, S. (2012) · Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data · doi:10.1145/2213836.2213844 · Cited by 291 · cited 1×
“The tables in each database are partitioned in such way that there are only single-partition transactions [52].”
BackgroundSelf-citationChallenges and future directions for the scaling of dynamic random-access memory (DRAM)
Mandelman, J. A., Dennard, R. H., Bronner, G. B., DeBrosse, J. K., Divakaruni, R., Li, Y., & Radens, C. J. (2002) · IBM Journal of Research and Development · doi:10.1147/rd.462.0187 · Cited by 270 · cited 1×
“This is problematic for OLTP applications that make many small changes to the database because these devices only support a limited number of writes per address [66].”
BackgroundMemory Performance and Cache Coherency Effects on an Intel Nehalem Multiprocessor System
Molka, D., Hackenberg, D., Schöne, R., & Müller, M. (2009) · International Conference on Parallel Architectures and Compilation Techniques (PACT) · doi:10.1109/pact.2009.22 · Cited by 269 · cited 1×
“But the changes of uncommitted transactions may be present in the database because the memory controller can evict cache lines containing those changes to NVM at any time [48].”
BackgroundShore-MT: a scalable storage manager for the multicore era
Johnson, R., Pandis, I., Hardavellas, N., Ailamaki, A., & Falsafi, B. (2009) · International Conference on Extending Database Technology (EDBT) · doi:10.1145/1516360.1516365 · Cited by 214 · cited 1×
“This work is based on the Shore-MT engine [37], which means that the DBMS records page-level before-images in the undo logs before performing in-place updates.”
MethodScalable logging through emerging non-volatile memory
Wang, T., & Johnson, R. (2014) · Proceedings of the VLDB Endowment · doi:10.14778/2732951.2732960 · Cited by 200 · cited 6×
“This work is based on the Shore-MT engine [37], which means that the DBMS records page-level before-images in the undo logs before performing in-place updates.”
BackgroundMethodInfluentialHigh performance database logging using storage class memory
Fang, R., Hsiao, H.-I., He, B., Mohan, C., & Wang, Y. (2011) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde.2011.5767918 · Cited by 117
From ARIES to MARS: transaction support for next-generation, solid-state drives
Coburn, J., Bunker, T., Schwarz, M., Gupta, R. K., & Swanson, S. (2013) · ACM SIGOPS Symposium on Operating Systems Principles (SOSP) · doi:10.1145/2517349.2522724 · Cited by 98 · cited 3×
“Others have developed new recovery mechanisms for DBMSs that are using a NVM + DRAM storage hierarchy [53, 30, 17, 65].”
MethodOukid, I., Booss, D., Lehner, W., et al. (2014) · International Workshop on Data Management on New Hardware (DaMoN) · doi:10.1145/2619228.2619236 · Cited by 64 · cited 3×
“The engine is designed to not perform any logging and uses MVCC. Similar to SOFORT, we make use of non-volatile pointers [58].”
BackgroundMethodA Prolegomenon on OLTP Database Systems for Non-Volatile Memory
DeBrabant, J. A., Arulraj, J. J. P., Pavlo, A., et al. (2015) · Cited by 49
Self-citationA limits study of benefits from nanostore-based future data-centric system architectures
Chang, J., Ranganathan, P., Mudge, T., et al. (2012) · ACM International Conference on Computing Frontiers (CF) · doi:10.1145/2212908.2212915 · Cited by 19 · cited 1×
“There are several aspects of NVM that make existing DBMS architectures inappropriate for them [14, 23].”
BackgroundConcurrency Control and Recovery
Franklin, M. J. M. (2014) · Chapman & Hall/CRC eBooks · doi:10.1201/9780429171529-105 · Cited by 5
Metadata
- Authors
- Arulraj, J., Pavlo, A., & Dulloor, S. R.
- Venue
- Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data
- Publisher
- ACM
- Published
- 2015-05-27
- Pages
- 707-722
- ISSN
- 0730-8078 (ISSN-L)
- OpenAlex
- W1988852271
- Cited by
- 199
- References
- 32
- FWCI
- 21.47
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.