The DOI matches its Crossref record; 46 of the 46 references it cites were found in a registry or an index; 20 of the 35 sources its claims cite were checked for support.
Efficient Memory Management for Large Language Model Serving with PagedAttention
Open accessAbstract
High throughput serving of large language models (LLMs) requires batching sufficiently many requests at a time. However, existing systems struggle because the key-value cache (KV cache) memory for each request is huge and grows and shrinks dynamically. When managed inefficiently, this memory can be significantly wasted by fragmentation and redundant duplication, limiting the batch size. To address this problem, we propose PagedAttention, an attention algorithm inspired by the classical virtual memory and paging techniques in operating systems. On top of it, we build vLLM, an LLM serving system that achieves (1) near-zero waste in KV cache memory and (2) flexible sharing of KV cache within and across requests to further reduce memory usage. Our evaluations show that vLLM improves the throughput of popular LLMs by 2--4× with the same level of latency compared to the state-of-the-art systems, such as FasterTransformer and Orca. The improvement is more pronounced with longer sequences, larger models, and more complex decoding algorithms. vLLM's source code is publicly available at https://github.com/vllm-project/vllm.
Overview
CoChat Summary
TL;DR
The paper introduces PagedAttention, an attention algorithm that stores KV cache in non-contiguous fixed-size blocks inspired by OS virtual memory and paging, and builds vLLM, an LLM serving system with paged KV cache management and sharing. Evaluations show vLLM improves LLM serving throughput by 2–4× over FasterTransformer and Orca at similar latency, with larger gains for longer sequences, larger models, and complex decoding.
Key findings
- vLLM improves throughput by 2–4× over state-of-the-art systems FasterTransformer and Orca with the same latency; on ShareGPT it sustains 1.7–2.7× higher request rates than Orca (Oracle) and 2.7–8× than Orca (Max), and up to 22× vs FasterTransformer.
- PagedAttention enables memory sharing: on Alpaca, 6.1–9.8% memory saving for parallel sampling and 37.6–55.2% for beam search; on ShareGPT, 16.2–30.5% and 44.3–66.3% respectively. Shared prefix gives 1.67× higher throughput than Orca (Oracle) with a one-shot prefix and 3.58× with a few-shot prefix; chatbot workload sustains 2× higher request rates.
- Existing systems waste KV cache memory: only 20.4–38.2% of KV cache memory is used to store actual token states, while vLLM achieves near-zero waste. PagedAttention adds 20–26% higher attention kernel latency vs FasterTransformer, but vLLM still outperforms end-to-end.
- vLLM uses default block size 16; recomputation overhead is never higher than 20% of swapping’s latency, and for medium block sizes 16–64 the two recovery methods show comparable end-to-end performance.
Methods
- Design: PagedAttention partitions each sequence’s KV cache into fixed-size KV blocks, stores them in non-contiguous GPU memory, and uses block tables mapping logical to physical blocks; vLLM adds a centralized scheduler, KV cache manager, copy-on-write, preemption via recomputation or swapping, and distributed execution with Megatron-LM style tensor parallelism.
- Evaluation: OPT 13B/66B/175B and LLaMA 13B on NVIDIA A100 GPUs (A2 instances on Google Cloud Platform); synthetic workloads from ShareGPT and Alpaca with Poisson request arrivals; baselines are FasterTransformer with a custom dynamic-batching scheduler and three Orca variants (Oracle, Pow2, Max); metrics are normalized latency and throughput.
- Ablations: kernel microbenchmark, block size impact, and recomputation vs swapping; most experiments use 1-hour traces, except OPT-175B uses 15-minute traces due to cost.
Limitations
- Stated: virtual memory and paging are effective for LLM serving because the workload requires dynamic memory allocation and is memory-bound, but this does not generally hold for every GPU workload; DNN training has static shapes, and non-LLM serving may be compute-bound, where vLLM’s techniques may degrade performance due to memory indirection and non-contiguous block memory.
- Stated/evident: Orca is not publicly available, so the authors implement their own Orca baselines; Orca (Oracle) assumes knowledge of actual output lengths and is infeasible in practice. Evident: evaluation uses synthetic request arrivals (Poisson) from ShareGPT and Alpaca without timestamps, and OPT-175B uses 15-minute traces due to cost.
- Evident: the chatbot evaluation does not store KV cache between different conversation rounds, which may limit applicability to multi-turn caching scenarios.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
3 Valid · 16 Not valid · 6 Not checked · 111 Own results and interpretations
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“for the 13B parameter OPT model [62], the KV cache of a single token demands 800 KB of space, calculated as 2 (key and value vectors) × 5120 (hidden state size) × 40 (number of layers) × 2 (bytes per FP16).”
§ Efficient Memory Management for Large Language Model Serving with PagedAttention > 3 Memory Challenges in LLM Serving
What the sources say
- Could not verifycites [62] sourceExperimental — accuracy not yet measured
The model specifications (Appendix B, p. 19) list GPT-3 models, not OPT-13B; Table 6 (p. 19) gives a 5120-wide GPT-3 model with 48 layers, not the claimed OPT configuration. Searches for OPT-13B and per-token key–value cache memory found no passage addressing the 800 KB figure or its calculation. (LLM, conf=0.76)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“First, the existing systems [31,60] suffer from internal and external memory fragmentation.”
§ Efficient Memory Management for Large Language Model Serving with PagedAttention > 1 Introduction
What the sources say
- Could not verifycites [31] 31. Christopher Olston, Noah Fiedel, Kiril Gorovoy, Jeremiah Harmsen, Li Lao, Fangwei Li, Vinu Rajashekhar, Sukriti Ramesh, and Jordan Soyke. 2017. Tensorflow-serving: Flexible, high-performance ml serving. arXiv preprint arXiv:1712.06139 (2017). sourceExperimental — accuracy not yet measured
The Introduction (p. 1) discusses managing server RAM as models grow, and §2.1.2 (p. 4) describes unloading models to free memory. Neither passage addresses internal or external memory fragmentation, and searches for those terms found no passage that does. The copy also does not discuss LLM serving systems. (LLM, conf=0.72)
- Not matchedcites [60]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“In more sophisticated algorithms like beam search [49], different request beams can share larger portions (up to 55% memory saving, see §6.3) of their KV cache”
§ Efficient Memory Management for Large Language Model Serving with PagedAttention > 3 Memory Challenges in LLM Serving
What the sources say
- Could not verifycites [49] 49. Jing Wang, Youyou Lu, Qing Wang, Minhui Xie, Keji Huang, and Jiwu Shu. 2022. Pacman: An Efficient Compaction Approach for {Log- Structured} {Key-Value} Store on Persistent Memory. In 2022 USENIX Annual Technical Conference (USENIX ATC 22). 773-788. sourceExperimental — accuracy not yet measured
The passages from our copy discuss persistent-memory key-value stores, not beam search or language-model KV caches. Searches for beam sharing, KV-cache sharing, and the claimed 55% memory saving found no passage addressing the claim; §6.3 (p. 12) instead reports storage-system performance. (LLM, conf=0.76)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“Due to the unpredictable output lengths from the LLM, they statically allocate a chunk of memory for a request based on the request's maximum possible sequence length, irrespective of the actual input or eventual output length of the request.”
§ Efficient Memory Management for Large Language Model Serving with PagedAttention > 3.1 Memory Management in Existing Systems
What the sources say
- Could not verifycites [31] 31. Christopher Olston, Noah Fiedel, Kiril Gorovoy, Jeremiah Harmsen, Li Lao, Fangwei Li, Vinu Rajashekhar, Sukriti Ramesh, and Jordan Soyke. 2017. Tensorflow-serving: Flexible, high-performance ml serving. arXiv preprint arXiv:1712.06139 (2017). sourceExperimental — accuracy not yet measured
Section 2.2.1 (p. 5) discusses batching inference requests, while the Introduction (p. 1) discusses managing memory for loaded models. Neither addresses static memory allocation per request based on maximum sequence length or unpredictable LLM output length. Searches for those terms and their paraphrases found no supporting passage in our copy. (LLM, conf=0.75)
- Not matchedcites [60]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“However, these general systems fail to take into account the autoregressive property and token state of LLM inference, resulting in missed opportunities for optimization.”
§ Efficient Memory Management for Large Language Model Serving with PagedAttention > 8 Discussion
What the sources say
- Partly supportedcites [11] 11. Daniel Crankshaw, Xin Wang, Guilio Zhou, Michael J Franklin, Joseph E Gonzalez, and Ion Stoica. 2017. Clipper: A Low-Latency Online Prediction Serving System. In 14th USENIX Symposium on Networked Systems Design and Implementation (NSDI 17). 613-627. sourceExperimental — accuracy not yet measured
Section 7 (Limitations; page not available) says Clipper treats models as black boxes and does not optimize their execution, supporting the general point that its serving architecture misses model-specific optimizations. The paper does not address LLM inference, autoregressive generation, or token state, and it does not establish the claim about general serving systems as a group. (LLM, conf=0.78)
- Could not verifycites [33] 33. OpenAI. 2022. https://openai.com/blog/chatgpt sourceExperimental — accuracy not yet measured
The Inference section (p. 4) discusses planned APIs for sequence models, and Inter-Request Batching (p. 5) describes batching within a sequence model’s loop. Neither passage addresses LLM token state or concludes that general serving systems miss optimizations because they ignore it (searched: autoregressive token state, sequence-model decoding and batching). (LLM, conf=0.76)
- Partly supportedcites [45] 45. Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to se- quence learning with neural networks. Advances in neural information processing systems 27 (2014). sourceExperimental — accuracy not yet measured
Related Work says that inference systems inheriting offloading techniques from training ignore the special computational properties of generative inference and miss opportunities to schedule I/O efficiently. The Generative Inference section describes token-by-token dependencies and the KV cache. The paper makes this criticism of offloading-based inference systems, not general model-serving systems as a whole, and does not specifically say those systems ignore token state. (LLM, conf=0.82)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
72 more claims
Create a free account to see all 136 claims and to verify the 1 reference still open.
- Probably not a substantial claim. 13
- It may state two separate claims as one. 48
- The quote only partly states this claim. 2
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“vLLM [23] and ChunkedAttention [58] explore some simple reuse cases (e.g., system prompt sharing) but do not cover multi-level tree-structured sharing or LRU caching.”
From SGLang: Efficient Execution of Structured Language Model Programs (2023) · cites this paper as [23]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“vLLM [30] reserves a fixed set of physical memory blocks for shared prefixes, enabling reuse of prefixes that align with predefined block sizes.”
From HotPrefix: Hotness-Aware KV Cache Scheduling for Efficient Prefix Sharing in LLM Inference Systems (2025) · cites this paper as [30]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Existing methods [30,72] often fail to effectively manage GPU memory, resulting in frequent eviction of high-hotness caches in favor of low-hotness ones.”
From HotPrefix: Hotness-Aware KV Cache Scheduling for Efficient Prefix Sharing in LLM Inference Systems (2025) · cites this paper as [30]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“While this mechanism simplifies memory allocation, it suffers from coarse granularity.If the prefix length does not align with block sizes, some parts of the prefix cannot be effectively reused, leading to inefficiencies.”
From HotPrefix: Hotness-Aware KV Cache Scheduling for Efficient Prefix Sharing in LLM Inference Systems (2025) · cites this paper as [30]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Parallel efforts target memory efficiency, such as PagedAttention's approach to KV cache management, which reduces fragmentation and overhead [289].”
From A Survey of Large Language Models (2026) · cites this paper as [289]
Experimental — accuracy not yet measured
72 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 1×
“4, all other components in the Transformer model, including the embedding layer, feed-forward layer, layer normalization [2], residual connection [22], output logit computation, and the query, key, and value transformation in Eq.”
BackgroundVaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
PyTorch: An Imperative Style, High-Performance Deep Learning Library
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al. (2019) · arXiv · doi:10.48550/arxiv.1912.01703 · Cited by 31,473
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Transformers: State-of-the-Art Natural Language Processing
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations · doi:10.18653/v1/2020.emnlp-demos.6 · Cited by 8,741
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
Wu, Y., Schuster, M., Chen, Z., Le, Q. V., Norouzi, M., Macherey, W., Krikun, M., Cao, Y., Gao, Q., Macherey, K., et al. (2016) · arXiv · doi:10.48550/arxiv.1609.08144 · Cited by 5,319 · cited 1×
“In LLM tasks like machine translation [59], the users expect the top-𝑘 most appropriate translations output by the LLM. Beam search [49] is widely used to decode the most probable output sequence from an LLM, as it mitigates the computational complexity of fully traversing the Unlike parallel…”
LLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602
HuggingFace's Transformers: State-of-the-art Natural Language Processing
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.03771 · Cited by 3,347
The Power of Scale for Parameter-Efficient Prompt Tuning
Lester, B., Al-Rfou, R., & Constant, N. (2021) · arXiv · doi:10.48550/arxiv.2104.08691 · Cited by 3,092
2017 USENIX Annual Technical Conference (USENIX ATC'17)
Quoc, D. L., Beck, M. S., Bhatotia, P., Chen, R., Fetzer, C. W., & Strufe, T. (2017) · Cited by 3,091
Research works printed but not linked: 8
- Yoshua Bengio , Réjean Ducharme , and Pascal Vincent . 2000. A neural probabilistic language model. Advances in neural information processing systems 13 ( 2000 ). Yoshua Bengio, Réjean Ducharme, and Pascal Vincent. 2000. A neural probabilistic language model. Advances in neural information processing systems 13 (2000).
- Weihao Cui , Han Zhao , Quan Chen , Hao Wei , Zirui Li , Deze Zeng , Chao Li , and Minyi Guo . 2022 . DVABatch: Diversity-aware Multi-Entry Multi-Exit Batching for Efficient Processing of DNN Services on GPUs. In 2022 USENIX Annual Technical Conference (USENIX ATC 22) . 183--198. Weihao Cui, Han Zhao, Quan Chen, Hao Wei, Zirui Li, Deze Zeng, Chao Li, and Minyi Guo. 2022. DVABatch: Diversity-aware Multi-Entry Multi-Exit Batching for Efficient Processing of DNN Services on GPUs. In 2022 USENIX Annual Technical Conference (USENIX ATC 22). 183--198.
- Mingcong Han , Hanze Zhang , Rong Chen , and Haibo Chen . 2022 . Microsecond-scale Preemption for Concurrent {GPU-accelerated}{DNN} Inferences . In 16th USENIX Symposium on Operating Systems Design and Implementation (OSDI 22) . 539--558. Mingcong Han, Hanze Zhang, Rong Chen, and Haibo Chen. 2022. Microsecond-scale Preemption for Concurrent {GPU-accelerated}{DNN} Inferences. In 16th USENIX Symposium on Operating Systems Design and Implementation (OSDI 22). 539--558.
- Ilya Sutskever , Oriol Vinyals , and Quoc V Le. 2014. Sequence to sequence learning with neural networks. Advances in neural information processing systems 27 ( 2014 ). Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. Advances in neural information processing systems 27 (2014).
- Jing Wang , Youyou Lu , Qing Wang , Minhui Xie , Keji Huang , and Jiwu Shu . 2022 . Pacman: An Efficient Compaction Approach for {Log-Structured} {Key-Value} Store on Persistent Memory . In 2022 USENIX Annual Technical Conference (USENIX ATC 22) . 773--788. Jing Wang, Youyou Lu, Qing Wang, Minhui Xie, Keji Huang, and Jiwu Shu. 2022. Pacman: An Efficient Compaction Approach for {Log-Structured} {Key-Value} Store on Persistent Memory. In 2022 USENIX Annual Technical Conference (USENIX ATC 22). 773--788.
- Gyeong-In Yu , Joo Seong Jeong , Geon-Woo Kim , Soojeong Kim , and Byung-Gon Chun . 2022 . Orca: A Distributed Serving System for {Transformer-Based} Generative Models . In 16th USENIX Symposium on Operating Systems Design and Implementation (OSDI 22) . 521--538. Gyeong-In Yu, Joo Seong Jeong, Geon-Woo Kim, Soojeong Kim, and Byung-Gon Chun. 2022. Orca: A Distributed Serving System for {Transformer-Based} Generative Models. In 16th USENIX Symposium on Operating Systems Design and Implementation (OSDI 22). 521--538.
- Hong Zhang , Yupeng Tang , Anurag Khandelwal , and Ion Stoica . 2023 . SHEPHERD: Serving DNNs in the Wild. In 20th USENIX Symposium on Networked Systems Design and Implementation (NSDI 23) . USENIX Association, Boston, MA, 787--808. https://www.usenix.org/conference/nsdi23/presentation/zhang-hong Hong Zhang, Yupeng Tang, Anurag Khandelwal, and Ion Stoica. 2023. SHEPHERD: Serving DNNs in the Wild. In 20th USENIX Symposium on Networked Systems Design and Implementation (NSDI 23). USENIX Association, Boston, MA, 787--808. https://www.usenix.org/conference/nsdi23/presentation/zhang-hong
- Zhe Zhou , Xuechao Wei , Jiejing Zhang , and Guangyu Sun . 2022 . PetS: A Unified Framework for Parameter-Efficient Transformers Serving. In 2022 USENIX Annual Technical Conference (USENIX ATC 22) . 489--504. Zhe Zhou, Xuechao Wei, Jiejing Zhang, and Guangyu Sun. 2022. PetS: A Unified Framework for Parameter-Efficient Transformers Serving. In 2022 USENIX Annual Technical Conference (USENIX ATC 22). 489--504.
Cited by
A Survey of Large Language Models
Zhao, W. X., Zhou, K., Li, J., Tang, T., Dong, Z., Hou, Y., Zhang, B., Min, Y., Zhang, J., Liu, P., et al. (2026) · Frontiers of Computer Science · doi:10.1007/s11704-026-60308-3 · Cited by 2,093
A Comprehensive Overview of Large Language Models
Naveed, H., Khan, A. U., Qiu, S., Saqib, M., Anwar, S., Usman, M., Akhtar, N., Barnes, N., & Mian, A. (2023) · arXiv · doi:10.48550/arxiv.2307.06435 · Cited by 1,259
A Comprehensive Overview of Large Language Models
Naveed, H., Khan, A. U., Qiu, S., Saqib, M., Anwar, S., Usman, M., Akhtar, N., Barnes, N., & Mian, A. (2025) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3744746 · Cited by 1,257
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
DeepSeek-AI, Guo, D., Yang, D., Zhang, H., Song, J., Wang, P., Zhu, Q., Xu, R., Zhang, R., Ma, S., et al. (2025) · arXiv · doi:10.48550/arxiv.2501.12948 · Cited by 18,918
Self-citationJiang, A. Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D. S., Casas, D. D. L., Bressand, F., Lengyel, G., Lample, G., Saulnier, L., et al. (2023) · arXiv · doi:10.48550/arxiv.2310.06825 · Cited by 340
Metadata
- Authors
- Kwon, W., Li, Z., Zhuang, S., Sheng, Y., Zheng, L., Yu, C. H., Gonzalez, J., Zhang, H., & Stoica, I.
- Affiliations
- University of California, Berkeley (US)
- Independent Researcher, Berkeley, United States of America
- University of California San Diego (US)
- ORCID
- Venue
- Proceedings of the 29th Symposium on Operating Systems Principles
- Publisher
- ACM
- Published
- 2023-10-23
- Pages
- 611-626
- OpenAlex
- W4387321091
- Semantic Scholar
- 83b90f4a0ae4cc214eb3cc140ccfef9cd99fac05
- Cited by
- 1,817
- References
- 19
- Citations by year
- 2026: 1017 · 2025: 623 · 2024: 169 · 2023: 8
- FWCI
- 321.20
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM SIGOPS Symposium on Operating Systems Principles (SOSP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- VMware
- Zayed University
- Samsung
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Hardware and Architecture › Parallel Computing and Optimization Techniques
- Keywords
- large language model serving, memory management, key-value cache, request batching, memory fragmentation, vLLM, inference throughput, dynamic memory allocation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-09-28, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-09-28 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Efficient Memory Management for Large Language Model Serving with PagedAttention. https://app.cochat.ai/papers/10.1145/3600006.3613165
- The paper
- Kwon, W., Li, Z., Zhuang, S., Sheng, Y., Zheng, L., Yu, C. H., Gonzalez, J., Zhang, H., & Stoica, I. (2023). Efficient Memory Management for Large Language Model Serving with PagedAttention. Proceedings of the 29th Symposium on Operating Systems Principles. https://doi.org/10.1145/3600006.3613165
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.