vAttention: Dynamic Memory Management for Serving LLMs without PagedAttention
Open accessThe DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
Abstract
PagedAttention is a popular approach for dynamic memory allocation in LLM serving systems. It enables on-demand allocation of GPU memory to mitigate KV cache fragmentation - a phenomenon that crippled the batch size (and consequently throughput) in prior systems. However, in trying to allocate physical memory at runtime, PagedAttention ends up changing the virtual memory layout of the KV cache from contiguous to non-contiguous. Such a design leads to non-trivial programming and performance overheads.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
PaLM: Scaling Language Modeling with Pathways
Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.02311 · Cited by 2,590
Longformer: The Long-Document Transformer
Beltagy, I., Peters, M. E., & Cohan, A. (2020) · arXiv · doi:10.48550/arxiv.2004.05150 · Cited by 2,281 · cited 1×
“This section elaborates on issues that arise with such an approach.”
Efficient Memory Management for Large Language Model Serving with PagedAttention
Kwon, W., Li, Z., Zhuang, S., Sheng, Y., Zheng, L., Yu, C. H., Gonzalez, J., Zhang, H., & Stoica, I. (2023) · Proceedings of the 29th Symposium on Operating Systems Principles · doi:10.1145/3600006.3613165 · Cited by 1,817
Efficient Memory Management for Large Language Model Serving with PagedAttention
Kwon, W., Li, Z., Zhuang, S., Sheng, Y., Zheng, L., Yu, C. H., Gonzalez, J. E., Zhang, H., & Stoica, I. (2023) · arXiv · doi:10.48550/arxiv.2309.06180 · Cited by 1,802
Generating Long Sequences with Sparse Transformers
Child, R., Gray, S., Radford, A., & Sutskever, I. (2019) · arXiv · doi:10.48550/arxiv.1904.10509 · Cited by 1,063 · cited 1×
“This section elaborates on issues that arise with such an approach.”
Splitwise: Efficient Generative LLM Inference Using Phase Splitting
Patel, P., Choukse, E., Zhang, C., Shah, A., Goiri, Í., Maleki, S., & Bianchini, R. (2024) · 2024 ACM/IEEE 51st Annual International Symposium on Computer Architecture (ISCA) · doi:10.1109/isca59077.2024.00019 · Cited by 298 · cited 5×
“This is called the KV cache [36, 63, 78] which accounts for a majority of GPU memory usage during inference.”
Influential(2023) · doi:10.52202/075280-1506 · Cited by 113
FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision
Shah, J., Bikshandi, G., Zhang, Y., Thakkar, V., Ramani, P., & Dao, T. (2024) · arXiv · doi:10.48550/arxiv.2407.08608 · Cited by 104
Fast Transformer Decoding: One Write-Head is All You Need
Shazeer, N. (2019) · arXiv · doi:10.48550/arxiv.1911.02150 · Cited by 101
POD-A ttention : Unlocking Full Prefill-Decode Overlap for Faster LLM Inference
Kamath, A. K., Prabhu, R., Mohan, J., Peter, S., Ramjee, R., & Panwar, A. (2025) · Proceedings of the 30th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2 · doi:10.1145/3676641.3715996 · Cited by 26 · cited 3×
“Given the size and scale of modern LLM deployments, optimizing inference has become extremely important [36, 46, 48, 49, 51, 63, 78, 82].”
Self-citationSARATHI: Efficient LLM Inference by Piggybacking Decodes with Chunked Prefills
Agrawal, A., Panwar, A., Mohan, J., Kwatra, N., Gulavani, B. S., & Ramjee, R. (2023) · arXiv · doi:10.48550/arxiv.2308.16369 · Cited by 20
Self-citationDeepSpeed-FastGen: High-throughput Text Generation for LLMs via MII and DeepSpeed-Inference
Holmes, C., Tanaka, M., Wyatt, M., Awan, A. A., Rasley, J., Rajbhandari, S., Aminabadi, R. Y., Qin, H., Bakhtiari, A., Kurilenko, L., et al. (2024) · arXiv · doi:10.48550/arxiv.2401.08671 · Cited by 10 · cited 3×
“8 × slower than FlashAttention-2 (Table 7).”
InfluentialInference without Interference: Disaggregate LLM Inference for Mixed Downstream Workloads
Hu, C., Huang, H., Xu, L., Chen, X., Xu, J., Chen, S., Feng, H., Wang, C., Wang, S., Bao, Y., et al. (2024) · arXiv · doi:10.48550/arxiv.2401.11181 · Cited by 8
Splitwise: Efficient generative LLM inference using phase splitting
Patel, P., Choukse, E., Zhang, C., Shah, A., Goiri, Í., Maleki, S., & Bianchini, R. (2023) · arXiv · doi:10.48550/arxiv.2311.18677 · Cited by 6
InfiniGen: Efficient Generative Inference of Large Language Models with Dynamic KV Cache Management
Lee, W., Lee, J., Seo, J., & Sim, J. (2024) · arXiv · doi:10.48550/arxiv.2406.19707 · Cited by 4 · cited 1×
“Optimizing LLM inference is an active area of research [44, 50, 53–57, 71, 72, 81, 83].”
Cited by
POD-A ttention : Unlocking Full Prefill-Decode Overlap for Faster LLM Inference
Kamath, A. K., Prabhu, R., Mohan, J., Peter, S., Ramjee, R., & Panwar, A. (2025) · Proceedings of the 30th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2 · doi:10.1145/3676641.3715996 · Cited by 26
“FI_Serial is mostly on par with FA_Serial but FlashInfer’s prefill kernel is somewhat slower than that of FlashAttention [42].”
Self-citationQoServe: Breaking the Silos of LLM Inference Serving
Goel, K., Mohan, J., Kwatra, N., Anupindi, R. S., & Ramjee, R. (2026) · Proceedings of the 31st ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2 · doi:10.1145/3779212.3790206 · Cited by 3
“Request arrival times are generated using a Poisson distribution, [4, 13], while maintaining the prefill and decode token counts of the respective traces.”
Self-citationEfficient LLM Serving for Agentic Workflows: A Data Systems Perspective
Wadlom, N., Shen, J., & Lu, Y. (2026) · Proceedings of the ACM on Management of Data · doi:10.1145/3802046
Metadata
- Authors
- Prabhu, R., Nayak, A., Mohan, J., Ramjee, R., & Panwar, A.
- Venue
- Proceedings of the 30th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 1
- Publisher
- ACM
- Published
- 2025-03-30
- Pages
- 1133-1150
- OpenAlex
- W4407217670
- Cited by
- 40
- References
- 15
- FWCI
- 35.38
- Type
- conference paper
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.