CoChat
Sign Up
PreprintCited by 11Parallel Computing and Optimization TechniquesNatural Language Processing TechniquesTopic Modeling

InfiniGen: Efficient Generative Inference of Large Language Models with Dynamic KV Cache Management

Open access
Lee, Wonbeom, Lee, Jungi, Seo, Junghwan, Sim, Jaewoong·2024·arXiv
DOI 10.48550/arxiv.2406.19707arXiv 2406.19707
Open this paper in CoChat — free accountOpen PDF

Abstract

Transformer-based large language models (LLMs) demonstrate impressive performance across various natural language processing tasks. Serving LLM inference for generating long contents, however, poses a challenge due to the enormous memory footprint of the transient state, known as the key-value (KV) cache, which scales with the sequence length and batch size. In this paper, we present InfiniGen, a novel KV cache management framework tailored for long-text generation, which synergistically works with modern offloading-based inference systems. InfiniGen leverages the key insight that a few important tokens that are essential for computing the subsequent attention layer in the Transformer can be speculated by performing a minimal rehearsal with the inputs of the current layer and part of the query weight and key cache of the subsequent layer. This allows us to prefetch only the essential KV cache entries (without fetching them all), thereby mitigating the fetch overhead from the host memory in offloading-based LLM serving systems. Our evaluation on several representative LLMs shows that InfiniGen improves the overall performance of a modern offloading-based system by up to 3.00x compared to prior KV cache management methods while offering substantially better model accuracy.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2406.19707

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • KV caching is a fundamental optimization that mitigates the quadratic computational complexity of LLMs in token generation by storing computed KV pairs for past tokens and reusing them for future queries.

    “KV caching is a fundamental optimization that mitigates this complexity by storing the computed KV pairs for past tokens and reusing them for future queries (Kwon et al., 2023; Lee et al., 2024; Sheng et al., 2024).”

    From Randomization Boosts KV Caching, Learning Balances Query Load: A Joint Perspective (2026) · cites this paper as Lee et al., 2024

    Section 2.2 (page unavailable) says KV caching stores keys and values from previous tokens so they need not be recomputed at each decoding iteration. It does not say caching mitigates quadratic computational complexity in token generation; it describes the avoided recomputation overhead as growing linearly with the number of previous tokens. The paper discusses quadratic complexity separately in relation to the prefill stage. (LLM, conf=0.85)

    Experimental — accuracy not yet measured

  • HCache, InfiniGen, CacheGen, MoonCake, and CachedAttention reduce the cost of long-context serving through recomputation, offloading, compression, or disaggregation.

    “HCache [5], InfiniGen [13], CacheGen [19], MoonCake [30], and CachedAttention [4] reduce the cost of long-context serving through recomputation, offloading, compression, or disaggregation.”

    From TokenDance: Scaling Multi-Agent LLM Serving via Collective KV Cache Sharing (2026) · cites this paper as [13]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

  • Rethinking Attention with Performers

    Choromanski, K., Likhosherstov, V., Dohan, D., Song, X., Gane, A., Sarlos, T., Hawkins, P., Davis, J., Mohiuddin, A., Kaiser, L., et al. (2020) · arXiv · doi:10.48550/arxiv.2009.14794 · Cited by 695 · cited 1× · see citing works

    “It is widely recognized that the keys and values of certain tokens are more important than others in attention computation [9,10,14,33,63].”
    Background

Flagged first, then by citations

Research works printed but not linked: 68

  1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023. · arXiv
  2. Tyler Allen and Rong Ge. In-depth analyses of unified virtual memory system for gpu accelerated computing. In International Conference for High Performance Com- puting, Networking, Storage and Analysis (SC), 2021.
  3. Reza Yazdani Aminabadi, Samyam Rajbhandari, Am- mar Ahmad Awan, Cheng Li, Du Li, Elton Zheng, Olatunji Ruwase, Shaden Smith, Minjia Zhang, Jeff Rasley, et al. Deepspeed-inference: enabling efficient inference of transformer models at unprecedented scale. In International Conference for High Performance Com- puting, Networking, Storage and Analysis (SC), 2022.
  4. Benjamin Berg, Daniel S. Berger, Sara McAllister, Isaac Grosof, Sathya Gunasekar, Jimmy Lu, Michael Uhlar, Jim Carrig, Nathan Beckmann, Mor Harchol-Balter, and Gregory R. Ganger. The CacheLib caching engine: Design and experiences at scale. In Proceedings of the USENIX Symposium on Operating Systems Design and Implementation (OSDI), 2020.
  5. Yonatan Bisk, Rowan Zellers, Ronan bras, Jianfeng Gao, and Choi Yejin. Piqa: Reasoning about physical commonsense in natural language. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), 2020.
  6. Beidi Chen, Tri Dao, Eric Winsor, Zhao Song, Atri Rudra, and Christopher Ré. Scatterbrain: Unifying sparse and low-rank attention. In Advances in Neural Information Processing Systems (NeurIPS), 2021.
  7. Beidi Chen, Zichang Liu, Binghui Peng, Zhaozhuo Xu, Jonathan Lingjie Li, Tri Dao, Zhao Song, Anshumali Shrivastava, and Christopher Re. Mongoose: A learn- able lsh framework for efficient neural network train- ing. In Proceedings of the International Conference on Learning Representations (ICLR), 2021.
  8. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374, 2021. · arXiv
  9. Shouyuan Chen, Sherman Wong, Liangjian Chen, and Yuandong Tian. Extending context window of large language models via positional interpolation. arXiv preprint arXiv:2306.15595, 2023. · arXiv
  10. Rewon Child, Scott Gray, Alec Radford, and Ilya Sutskever. Generating long sequences with sparse trans- formers. arXiv preprint arXiv:1904.10509, 2019. · arXiv
  11. Daniel Crankshaw, Gur-Eyal Sela, Xiangxi Mo, Corey Zumar, Ion Stoica, Joseph Gonzalez, and Alexey Tu- manov. Inferline: latency-aware provisioning and scal- ing for prediction serving pipelines. In Proceedings of the ACM Symposium on Cloud Computing (SoCC), 2020.
  12. Daniel Crankshaw, Xin Wang, Guilio Zhou, Michael J Franklin, Joseph E Gonzalez, and Ion Stoica. Clipper: A low-latency online prediction serving system. In Proceedings of the USENIX Symposium on Networked Systems Design and Implementation (NSDI), 2017.
  13. Weihao Cui, Han Zhao, Quan Chen, Hao Wei, Zirui Li, Deze Zeng, Chao Li, and Minyi Guo. Dvabatch: Diversity-aware multi-entry multi-exit batching for effi- cient processing of dnn services on gpus. In Proceedings of the USENIX Annual Technical Conference (USENIX ATC), 2022.
  14. Tri Dao, Dan Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. Flashattention: Fast and memory- efficient exact attention with io-awareness. In Advances in Neural Information Processing Systems (NeurIPS), 2022.
  15. Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. Gpt3.int8(): 8-bit matrix multiplication for transformers at scale. In Advances in Neural Informa- tion Processing Systems (NeurIPS), 2022.
  16. Angela Fan, Mike Lewis, and Yann Dauphin. Hierarchi- cal neural story generation. In Annual Meeting of the Association for Computational Linguistics (ACL), 2018.
  17. Jiarui Fang, Yang Yu, Chengduo Zhao, and Jie Zhou. Turbotransformers: an efficient gpu serving system for transformer models. In Proceedings of the Symposium on Principles and Practice of Parallel Programming (PPoPP), 2021.
  18. Elias Frantar and Dan Alistarh. Sparsegpt: Massive language models can be accurately pruned in one-shot. In Proceedings of the International Conference on Ma- chine Learning (ICML), 2023.
  19. Arpan Gujarati, Reza Karimi, Safya Alzayat, Wei Hao, Antoine Kaufmann, Ymir Vigfusson, and Jonathan Mace. Serving dnns like clockwork: Performance pre- dictability from the bottom up. In Proceedings of the USENIX Symposium on Operating Systems Design and Implementation (OSDI), 2020.
  20. Cong Guo, Jiaming Tang, Weiming Hu, Jingwen Leng, Chen Zhang, Fan Yang, Yunxin Liu, Minyi Guo, and Yuhao Zhu. Olive: Accelerating large language models via hardware-friendly outlier-victim pair quantization. In Proceedings of the International Symposium on Com- puter Architecture (ISCA), 2023.
  21. Tae Jun Ham, Yejin Lee, Seong Hoon Seo, Soosung Kim, Hyunji Choi, Sung Jun Jung, and Jae W Lee. Elsa: Hardware-software co-design for efficient, lightweight self-attention mechanism in neural networks. In Pro- ceedings of the International Symposium on Computer Architecture (ISCA), 2021.
  22. Mingcong Han, Hanze Zhang, Rong Chen, and Haibo Chen. Microsecond-scale preemption for concurrent gpu-accelerated dnn inferences. In Proceedings of the USENIX Symposium on Operating Systems Design and Implementation (OSDI), 2022.
  23. Mark Hildebrand, Jawad Khan, Sanjeev Trika, Jason Lowe-Power, and Venkatesh Akella. Autotm: Automatic tensor movement in heterogeneous memory systems using integer linear programming. In Proceedings of the International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), 2020.
  24. Chien-Chin Huang, Gu Jin, and Jinyang Li. Swapadvi- sor: Pushing deep learning beyond the gpu memory limit via smart swapping. In Proceedings of the International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), 2020.
  25. Jaehoon Jung, Jinpyo Kim, and Jaejin Lee. Deepum: Tensor migration and prefetching in unified memory. In Proceedings of the International Conference on Ar- chitectural Support for Programming Languages and Operating Systems (ASPLOS), 2023.
  26. Sheng-Chun Kao, Suvinay Subramanian, Gaurav Agrawal, Amir Yazdanbakhsh, and Tushar Krishna. Flat: An optimized dataflow for mitigating attention bottlenecks. In Proceedings of the International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), 2023.
  27. Nikita Kitaev, Lukasz Kaiser, and Anselm Levskaya. Re- former: The efficient transformer. In Proceedings of the International Conference on Learning Representations (ICLR), 2020.
  28. Woosuk Kwon, Sehoon Kim, Michael W Mahoney, Joseph Hassoun, Kurt Keutzer, and Amir Gholami. A fast post-training pruning framework for transformers. In Advances in Neural Information Processing Systems (NeurIPS), 2022.
  29. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory manage- ment for large language model serving with pagedatten- tion. In Proceedings of the Symposium on Operating Systems Principles (SOSP), 2023.
  30. Jungi Lee, Wonbeom Lee, and Jaewoong Sim. Tender: Accelerating large language models via tensor decom- position and runtime requantization. In Proceedings of the International Symposium on Computer Architecture (ISCA), 2024.
  31. Zichang Liu, Aditya Desai, Fangshuo Liao, Weitao Wang, Victor Xie, Zhaozhuo Xu, Anastasios Kyrillidis, and Anshumali Shrivastava. Scissorhands: Exploiting the persistence of importance hypothesis for llm kv cache compression at test time. In Advances in Neural Information Processing Systems (NeurIPS), 2023.
  32. Mitch Marcus, Grace Kim, Mary Ann Marcinkiewicz, Robert MacIntyre, Ann Bies, Mark Ferguson, Karen Katz, and Britta Schasberger. The penn treebank: Anno- tating predicate argument structure. In Proceedings of the Workshop on Human Language Technology, 1994.
  33. Pak Markthub, Mehmet E. Belviranli, Seyong Lee, Jef- frey S. Vetter, and Satoshi Matsuoka. Dragon: Breaking gpu memory capacity limits with direct nvm access. In International Conference for High Performance Com- puting, Networking, Storage and Analysis (SC), 2018.
  34. Pierre-Emmanuel Mazare, Samuel Humeau, Martin Rai- son, and Antoine Bordes. Training millions of person- alized dialogue agents. In Conference on Empirical Methods in Natural Language Processing (EMNLP), 2018.
  35. Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models. In Pro- ceedings of the International Conference on Learning Representations (ICLR), 2017.
  36. Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. Can a suit of armor conduct electricity? a new dataset for open book question answering. In Conference on Empirical Methods in Natural Language Processing (EMNLP), 2018.
  37. Erik Nijkamp, Bo Pang, Hiroaki Hayashi, Lifu Tu, Huan Wang, Yingbo Zhou, Silvio Savarese, and Caim- ing Xiong. Codegen: An open large language model for code with multi-turn program synthesis. In Pro- ceedings of the International Conference on Learning Representations (ICLR), 2023.
  38. Christopher Olston, Noah Fiedel, Kiril Gorovoy, Jeremiah Harmsen, Li Lao, Fangwei Li, Vinu Rajashekhar, Sukriti Ramesh, and Jordan Soyke. Tensorflow-serving: Flexible, high-performance ml serving. In Advances in Neural Information Processing Systems (NeurIPS), 2017.
  39. Xuan Peng, Xuanhua Shi, Hulin Dai, Hai Jin, Weiliang Ma, Qian Xiong, Fan Yang, and Xuehai Qian. Capuchin: Tensor-based gpu memory management for deep learn- ing. In Proceedings of the International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), 2020.
  40. Reiner Pope, Sholto Douglas, Aakanksha Chowdhery, Jacob Devlin, James Bradbury, Jonathan Heek, Kefan Xiao, Shivani Agrawal, and Jeff Dean. Efficiently scal- ing transformer inference. In Proceedings of the Ma- chine Learning and Systems (MLSys), 2023.
  41. Zheng Qu, Liu Liu, Fengbin Tu, Zhaodong Chen, Yufei Ding, and Yuan Xie. Dota: detect and omit weak atten- tions for scalable transformer acceleration. In Proceed- ings of the International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), 2022.
  42. Jack W Rae, Anna Potapenko, Siddhant M Jayakumar, Chloe Hillier, and Timothy P Lillicrap. Compressive transformers for long-range sequence modelling. In Pro- ceedings of the International Conference on Learning Representations (ICLR), 2020.
  43. Machel Reid, Nikolay Savinov, Denis Teplyashin, Dmitry Lepikhin, Timothy Lillicrap, Jean-baptiste Alayrac, Radu Soricut, Angeliki Lazaridou, Orhan Fi- rat, Julian Schrittwieser, et al. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprint arXiv:2403.05530, 2024. · arXiv
  44. Melissa Roemmele, Cosmin Bejan, and Andrew Gordon. Choice of plausible alternatives: An evaluation of com- monsense causal reasoning. In AAAI Spring Symposium Series, 2011.
  45. Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhaga- vatula, and Yejin Choi. Winogrande: An adversarial winograd schema challenge at scale. Communications of the ACM, 2021.
  46. Haichen Shen, Lequn Chen, Yuchen Jin, Liangyu Zhao, Bingyu Kong, Matthai Philipose, Arvind Krishnamurthy, and Ravi Sundaram. Nexus: A gpu cluster engine for accelerating dnn-based video analysis. In Proceed- ings of the Symposium on Operating Systems Principles (SOSP), 2019.
  47. Ying Sheng, Lianmin Zheng, Binhang Yuan, Zhuohan Li, Max Ryabinin, Beidi Chen, Percy Liang, Christo- pher Re, Ion Stoica, and Ce Zhang. FlexGen: High- throughput generative inference of large language mod- els with a single gpu. In Proceedings of International Conference on Machine Learning (ICML), 2023.
  48. Yining Shi, Zhi Yang, Jilong Xue, Lingxiao Ma, Yuqing Xia, Ziming Miao, Yuxiao Guo, Fan Yang, and Lidong Zhou. Welder: Scheduling deep learning memory access via tile-graph. In Proceedings of the USENIX Sympo- sium on Operating Systems Design and Implementation (OSDI), 2023.
  49. Ilya Sutskever, Oriol Vinyals, and Quoc V Le. Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems (NeurIPS), 2014.
  50. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023. · arXiv
  51. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems (NeurIPS), 2017.
  52. Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. Glue: A multi- task benchmark and analysis platform for natural lan- guage understanding. In EMNLP Workshop Black- boxNLP, 2018.
  53. Sinong Wang, Belinda Z. Li, Madian Khabsa, Han Fang, and Hao Ma. Linformer: Self-attention with linear com- plexity. arXiv preprint arXiv:2006.04768, 2020. · arXiv
  54. Yiming Wang, Zhuosheng Zhang, and Rui Wang. Element-aware summarization with large language mod- els: Expert-aligned evaluation and chain-of-thought method. In Annual Meeting of the Association for Com- putational Linguistics (ACL), 2023.
  55. Xiuying Wei, Yunchen Zhang, Xiangguo Zhang, Ruihao Gong, Shanghang Zhang, Qi Zhang, Fengwei Yu, and Xianglong Liu. Outlier suppression: Pushing the limit of low-bit transformer language models. In Advances in Neural Information Processing Systems (NeurIPS), 2022.
  56. Guangxuan Xiao, Ji Lin, Mickael Seznec, Hao Wu, Julien Demouth, and Song Han. Smoothquant: Accu- rate and efficient post-training quantization for large language models. In International Conference on Ma- chine Learning (ICML), 2023.
  57. Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. Efficient streaming language models with attention sinks. In Proceedings of the In- ternational Conference on Learning Representations (ICLR), 2024.
  58. Haoran Xu, Young Jin Kim, Amr Sharaf, and Hany Has- san Awadalla. A paradigm shift in machine transla- tion: Boosting translation performance of large language models. In Proceedings of the International Conference on Learning Representations (ICLR), 2024.
  59. Juncheng Yang, Yazhuo Zhang, Ziyue Qiu, Yao Yue, and Rashmi Vinayak. Fifo queues are all you need for cache eviction. In Proceedings of the Symposium on Operating Systems Principles (SOSP), 2023.
  60. Tzu-Wei Yang, Seth Pollen, Mustafa Uysal, Arif Mer- chant, and Homer Wolfmeister. CacheSack: Admission optimization for google datacenter flash caches. In Pro- ceedings of the USENIX Annual Technical Conference (USENIX ATC), 2022.
  61. Gyeong-In Yu, Joo Seong Jeong, Geon-Woo Kim, Soo- jeong Kim, and Byung-Gon Chun. Orca: A distributed serving system for Transformer-Based generative mod- els. In Proceedings of the USENIX Symposium on Op- erating Systems Design and Implementation (OSDI), 2022.
  62. Yuan Yu, Martín Abadi, Paul Barham, Eugene Brevdo, Mike Burrows, Andy Davis, Jeff Dean, Sanjay Ghe- mawat, Tim Harley, Peter Hawkins, et al. Dynamic control flow in large-scale machine learning. In Proceed- ings of the Thirteenth EuroSys Conference (EuroSys), 2018.
  63. Haoyang Zhang, Yirui Eric Zhou, Yu Xue, Yiqi Liu, and Jian Huang. G10: Enabling an efficient unified gpu memory and storage architecture with smart tensor mi- grations. In Proceedings of the International Symposium on Microarchitecture (MICRO), 2023.
  64. Haoyu Zhang, Jingjing Cai, Jianjun Xu, and Ji Wang. Pretraining-based natural language generation for text summarization. In Conference on Computational Natu- ral Language Learning (CoNLL), 2019.
  65. Hong Zhang, Yupeng Tang, Anurag Khandelwal, and Ion Stoica. Shepherd: Serving dnns in the wild. In Proceedings of the Symposium on Networked Systems Design and Implementation (NSDI), 2023.
  66. Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. Personalizing dialogue agents: I have a dog, do you have pets too? In Annual Meeting of the Association for Computational Linguistics (ACL), 2018.
  67. Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher De- wan, Mona Diab, Xian Li, Xi Victoria Lin, et al. Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205.01068, 2022. · arXiv
  68. Zhenyu Zhang, Ying Sheng, Tianyi Zhou, Tianlong Chen, Lianmin Zheng, Ruisi Cai, Zhao Song, Yuandong Tian, Christopher Ré, Clark Barrett, et al. H 2 O: Heavy- hitter oracle for efficient generative inference of large language models. In Advances in Neural Information Processing Systems (NeurIPS), 2023.

Cited by

Sorted by citations

Authors
Lee, W., Lee, J., Seo, J., & Sim, J.
Affiliations
Seoul National University (KR)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2024
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2406.19707
arXiv
2406.19707
OpenAlex
W4400222625
Cited by
4
References
1
Citations by year
2025: 3 · 2024: 1
Type
posted content
Language
en
Open access
Licence
Locations
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Funders
  • Samsung Advanced Institute of Technology
  • Seoul National University
  • Samsung
  • Institute for Information and Communications Technology Promotion
  • Institute of Engineering Research, Seoul National University
Topics
  • Physical Sciences › Computer Science › Hardware and Architecture › Parallel Computing and Optimization Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
Keywords
large language models, long-form text generation, inference offloading, speculative decoding, transformer attention, large language model inference

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review