P-HNSW: Crash-Consistent HNSW for Vector Databases on Persistent Memory
Open accessThe DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Abstract
The rapid growth of Large Language Models (LLMs) has generated massive amounts of high-dimensional feature vectors extracted from diverse datasets. Efficient storage and retrieval of such data are critical for enabling accurate and fast query responses. Vector databases (Vector DBs) provide efficient storage and retrieval for high-dimensional vectors. These systems rely on Approximate Nearest Neighbor Search (ANNS) indexes, such as HNSW, to handle large-scale data efficiently. However, the original HNSW is implemented on DRAM, which is both costly and vulnerable to crashes. Therefore, we propose P-HNSW, a crash-consistent HNSW on persistent memory. To guarantee crash consistency, P-HNSW introduces two logs, NLog and NlistLog. We describe the logging process during the operation and the recovery process in the event of system crashes. Our experimental results demonstrate that the overhead of the proposed logging mechanism is negligible, while P-HNSW achieves superior performance compared with SSD-based recovery mechanisms.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures












Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Multidimensional binary search trees used for associative searching
Bentley, J. L. (1975) · Communications of the ACM · doi:10.1145/361002.361007 · Cited by 7,520
Approximate nearest neighbors: towards removing the curse of dimensionality
Indyk, P., & Motwani, R. (1998) · Proceedings of the thirtieth annual ACM symposium on Theory of computing - STOC '98 · doi:10.1145/276698.276876 · Cited by 4,188
Product Quantization for Nearest Neighbor Search
Jégou, H., Douze, M., & Schmid, C. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.57 · Cited by 2,956
Malkov, Y. A., & Yashunin, D. A. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2889473 · Cited by 1,904
Fast approximate nearest neighbor search with the navigating spreading-out graph
Fu, C., Xiang, C., Wang, C., & Cai, D. (2019) · Proceedings of the VLDB Endowment · doi:10.14778/3303753.3303754 · Cited by 379
Current applications and challenges in large language models for patient care: a systematic review
Busch, F., Hoffmann, L., Rueger, C., van Dijk, E. H., Kader, R., Ortiz-Prado, E., Makowski, M. R., Saba, L., Hadamitzky, M., Kather, J. N., et al. (2025) · Communications Medicine · doi:10.1038/s43856-024-00717-2 · Cited by 358
Large Language Models: A Survey
Minaee, S., Mikolov, T., Nikzad, N., Chenaghlu, M., Socher, R., Amatriain, X., & Gao, J. (2024) · arXiv · doi:10.48550/arxiv.2402.06196 · Cited by 325
Searching in one billion vectors: Re-rank with source coding
Jegou, H., Tavenard, R., Douze, M., & Amsaleg, L. (2011) · 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2011.5946540 · Cited by 313
Wang, M., Xu, X., Yue, Q., & Wang, Y. (2021) · Proceedings of the VLDB Endowment · doi:10.14778/3476249.3476255 · Cited by 264
FreshDiskANN: A Fast and Accurate Graph-Based ANN Index for Streaming Similarity SearchPreprint
Singh, A., Subramanya, S. J., Krishnaswamy, R., & Simhadri, H. V. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.09613 · Cited by 194
Survey of Vector Database Management Systems
Pan, J. J., Wang, J., & Li, G. (2023) · arXiv · doi:10.48550/arxiv.2310.14021 · Cited by 184
Survey of vector database management systems
Pan, J. J., Wang, J., & Li, G. (2024) · The VLDB Journal · doi:10.1007/s00778-024-00864-x · Cited by 173
Industrial applications of large language models
Raza, M., Jahangir, Z., Riaz, M. B., Saeed, M. J., & Sattar, M. A. (2025) · Scientific Reports · doi:10.1038/s41598-025-98483-1 · Cited by 163
Jung, M. (2022) · ACM Workshop on Hot Topics in Storage and File Systems (HotStorage) · doi:10.1145/3538643.3539745 · Cited by 83
The Faiss libraryPreprint
Douze, M., Guzhva, A., Deng, C., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2401.08281 · Cited by 75
Kim, W.-H., Krishnan, R. M., Fu, X., Kashyap, S., & Min, C. (2021) · ACM SIGOPS Symposium on Operating Systems Principles (SOSP) · doi:10.1145/3477132.3483589 · Cited by 63
Self-citationDB-LSH 2.0: Locality-Sensitive Hashing With Query-Based Dynamic Bucketing
Tian, Y., Zhao, X., & Zhou, X. (2023) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2023.3295831 · Cited by 48
Halo: A Hybrid PMem-DRAM Persistent Hash Index with Fast Recovery
Hu, D., Chen, Z., Che, W., Sun, J., & Chen, H. (2022) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/3514221.3517884 · Cited by 35
A Comprehensive Survey on Vector Database: Storage and Retrieval Technique, Challenge
Ma, L., Zhang, R., Han, Y., Yu, S., Wang, Z., Ning, Z., Zhang, J., Xu, P., Li, P., Qiao, Z., et al. (2023) · arXiv · doi:10.48550/arxiv.2310.11703 · Cited by 34
Lightweight-Yet-Efficient: Revitalizing Ball-Tree for Point-to-Hyperplane Nearest Neighbor Search
Huang, Q., & Tung, A. K. H. (2023) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde55515.2023.00040 · Cited by 24
OmniSearchSage: Multi-Task Multi-Entity Embeddings for Pinterest Search
Agarwal, P., Islam, S. M., Pancha, N., Hazra, K. S., Xu, J., & Rosenberg, C. (2024) · International World Wide Web Conference (WWW) · doi:10.1145/3589335.3648309 · Cited by 23
Performance Characterizations and Usage Guidelines of Samsung CXL Memory Module Hybrid PrototypePreprint
Zeng, J., Pei, S., Zhang, D., Zhou, Y., Beygi, A., Yao, X., Kachare, R., Zhang, T., Li, Z., Nguyen, M., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2503.22017 · Cited by 23
Xu, Q., Zhang, F., Li, C., Cao, L., Chen, Z., Zhai, J., & Du, X. (2025) · Proceedings of the ACM on Management of Data · doi:10.1145/3749167 · Cited by 21
Parallel kd-tree with Batch Updates
Men, Z., Shen, Z., Gu, Y., & Sun, Y. (2025) · Proceedings of the ACM on Management of Data · doi:10.1145/3709712 · Cited by 19
Performance Characterizations and Usage Guidelines of Samsung CMM-H
Zeng, J., Pei, S., Zhang, D., Zhou, Y., Beygi, A., Yao, X., Kachare, R., Zhang, T., Li, Z., Nguyen, M., et al. (2025) · IEEE Micro · doi:10.1109/mm.2025.3591577 · Cited by 8
Scalable Graph Representation Learning via Locality-Sensitive Hashing
Chen, X., Jiang, J., & Wang, W. W. (2022) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3511808.3557689 · Cited by 4
ZBTree: A Fast and Scalable B+-Tree for Persistent Memory
Che, W., Chen, Z., Hu, D., Sun, J., & Chen, H. (2024) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2024.3421232 · Cited by 3
Wang, H., Meng, J., Gong, L., Xu, J. J., & Ogihara, M. (2021) · Proceedings of the VLDB Endowment · doi:10.14778/3484224.3484226 · Cited by 2
Jin, H., Lee, J., Piao, S., Seo, S., Kwon, S., & Park, S. (2025) · Design, Automation, and Test in Europe · doi:10.23919/date64628.2025.10992903 · Cited by 2
Xu, Q., Zhang, F., Li, C., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2506.14707 · Cited by 1
Dual-Branch HNSW Approach with Skip Bridges and LID-Driven OptimizationPreprint
Nguyen, H., Nguyen, N. H., Nguyen, N. L. B., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2501.13992
Wang, M., Xu, X., Yue, Q., & Wang, Y. (2021) · arXiv · doi:10.48550/arxiv.2101.12631
Research works printed but not linked: 5
- Subramanya, S.J., Kadekodi, R., Krishaswamy, R., and Simhadri, H.V. (2019, January 8–14). DiskANN: Fast Accurate Billion-point Nearest Neighbor Search on a Single Node. Proceedings of the 33rd Conference and Workshop on Neural Information Processing Systems (NeurIPS), Vancouver, BC, Canada.
- Li, S., Zhou, Y.E., Ren, H., and Huang, J. (April, January 30). ByteFS: System Support for (CXL-based) Memory-Semantic Solid-State Drives. Proceedings of the 30th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS ’25, Rotterdam, The Netherlands.
- Nam, M., Cha, H., Choi, Y.r., Noh, S.H., and Nam, B. (2019, January 25–28). Write-Optimized Dynamic Hashing for Persistent Memory. Proceedings of the 17th USENIX Conference on File and Storage Technologies (FAST), Boston, MA, USA.
- Ma, S., Chen, K., Chen, S., Liu, M., Zhu, J., Kang, H., and Wu, Y. (2021, January 23–25). ROART: Range-query optimized persistent ART. Proceedings of the 19th USENIX Conference on File and Storage Technologies (FAST 21), Virtual Event.
- Lu, B., Hao, X., Wang, T., and Lo, E. (September, January 31). Dash: Scalable Hashing on Persistent Memory. Proceedings of the 46th International Conference on Very Large Data Bases (VLDB), Tokyo, Japan.
Cited by
Yang, C., & Kim, M. (2026) · Applied Sciences · doi:10.3390/app16073391
Boltaboyeva, A., Baigarayeva, Z., Иманбек, Б., Amangeldy, B., Tasmurzayev, N., Ozhikenov, K., Alimbayeva, Z. N., Alimbayev, C. A., & Karymsakova, N. (2026) · Algorithms · doi:10.3390/a19020099 · Cited by 1
Metadata
- Authors
- Lee, H., Park, T., Na, Y., & Kim, W.-H.
- Affiliations
- Konkuk University (KR)
- Corresponding author
- Kim, Wook-Hee
- Venue
- Applied Sciences
- Publisher
- MDPI AG
- Published
- 2025-09-29
- Volume
- 15
- Issue
- 19
- Pages
- 10554
- ISSN
- 2076-3417 (ISSN-L)
- OpenAlex
- W4414669346
- Cited by
- 1
- References
- 23
- Citations by year
- 2026: 1
- FWCI
- 0.42
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Applied Sciences (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Institute of Information and Communications Technology Planning and Evaluation (IITP) under the Metaverse Support Program to Nurture the Best Talents (IITP-2025-RS-2023-00256615)
- Topics
- Physical Sciences › Computer Science › Information Systems › Advanced Database Systems and Queries
- Physical Sciences › Computer Science › Computer Networks and Communications › Advanced Data Storage Technologies
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Keywords
- HNSW index, persistent memory, crash consistency, approximate nearest neighbor search, vector databases, crash recovery
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.