MemoryBench: A Benchmark for Memory and Continual Learning in LLM Systems
Open accessAbstract
Scaling up data, parameters, and test-time computation has been the mainstream methods to improve LLM systems (LLMsys), but their upper bounds are almost reached due to the gradual depletion of high-quality data and marginal gains obtained from larger computational resource consumption. Inspired by the abilities of human and traditional AI systems in learning from practice, constructing memory and continual learning frameworks for LLMsys has become an important and popular research direction in recent literature. Yet, existing benchmarks for LLM memory often focus on evaluating the system on homogeneous reading comprehension tasks with long-form inputs rather than testing their abilities to learn from accumulated user feedback in service time. Therefore, we propose a user feedback simulation framework and a comprehensive benchmark covering multiple domains, languages, and types of tasks to evaluate the continual learning abilities of LLMsys. Experiments show that the effectiveness and efficiency of state-of-the-art baselines are far from satisfying, and we hope this benchmark could pave the way for future studies on LLM memory and optimization algorithms. Website: https://memorybench.thuir.cn Code: https://github.com/THUIR/MemoryBench Data: https://huggingface.co/datasets/THUIR/MemoryBench Data-Full: https://huggingface.co/datasets/THUIR/MemoryBench-Full
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
MemoryBench found no advanced system consistently outperforms RAG baselines, suggesting current benchmarks may not capture what differentiates memory architectures.
“MemoryBench [1] found no advanced system consistently outperforms RAG baselines, suggesting current benchmarks may not capture what differentiates memory architectures.”
From SuperLocalMemory: Privacy-Preserving Multi-Agent Memory with Bayesian Trust Defense Against Memory Poisoning (2026) · cites this paper as [1]
Section 3.3 directly reports that none of the advanced memory-based systems consistently outperformed the RAG baselines on MemoryBench. Sections 2.1 and 3.3 also describe limitations of previous benchmarks, but the paper does not conclude that benchmarks generally fail to capture what differentiates memory architectures; it presents MemoryBench as addressing some of those limitations. The passages provide no page numbers. (LLM, conf=0.82)
Experimental — accuracy not yet measuredExisting memory benchmarks predominantly evaluate retention of personalized information in multi-turn chat, revealing little about memory behavior within agents that interleave perception, reasoning, and tool use.
“Existing memory benchmarks [29, 48, 18, 1] predominantly evaluate retention of personalized information in multi-turn chat, revealing little about memory behavior within agents that interleave perception, reasoning, and tool use.”
From MemGym: a Long-Horizon Memory Environment for LLM Agents (2026) · cites this paper as [1]
The Introduction (page unavailable) says existing memory benchmarks mostly test long-context handling, including user profiles and conversation histories, often through question answering. Section 2.1 (page unavailable) likewise says previous benchmarks mainly test factual answers drawn from supplied context. The paper does not specifically assess whether those benchmarks reveal memory behavior in agents that interleave perception, reasoning, and tool use. (LLM, conf=0.84)
Experimental — accuracy not yet measuredLoCoMo comparability is limited.
“LoCoMo comparability is limited [1]”
From SuperLocalMemory: Privacy-Preserving Multi-Agent Memory with Bayesian Trust Defense Against Memory Poisoning (2026) · cites this paper as [1]
MemoryBench uses simulated user feedback to drive continual learning across sessions.
“MemoryBench [1] uses simulated user feedback to drive continual learning across sessions.”
From MemGym: a Long-Horizon Memory Environment for LLM Agents (2026) · cites this paper as [1]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 51
- Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Ale- man, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023. · arXiv
- Mariam Arain, Maliha Haque, Lina Johal, Puja Mathur, Wynand Nel, Afsha Rais, Ranbir Sandhu, and Sushil Sharma. Maturation of the adolescent brain. Neuropsychiatric disease and treatment, pp. 449-461, 2013.
- Richard C Atkinson and Richard M Shiffrin. Human memory: A proposed system and its control processes. In Psychology of learning and motivation, volume 2, pp. 89-195. Elsevier, 1968.
- Satanjeev Banerjee and Alon Lavie. METEOR: An automatic metric for MT evaluation with im- proved correlation with human judgments. In Jade Goldstein, Alon Lavie, Chin-Yew Lin, and Clare Voss (eds.), Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Mea- sures for Machine Translation and/or Summarization, pp. 65-72, Ann Arbor, Michigan, June 2005. Association for Computational Linguistics. URL https://aclanthology.org/ W05-0909/.
- Junjie Chen, Haitao Li, Zhumin Chu, Yiqun Liu, and Qingyao Ai. Overview of the ntcir-18 au- tomatic evaluation of llms (aeollm) task, 2025. URL https://arxiv.org/abs/2503. 13038.
- Prateek Chhikara, Dev Khant, Saket Aryan, Taranjeet Singh, and Deshraj Yadav. Mem0: Building production-ready ai agents with scalable long-term memory. arXiv preprint arXiv:2504.19413, 2025. · arXiv
- Michael D Cohen and Paul Bacdayan. Organizational routines are stored as procedural memory: Evidence from a laboratory study. Organization science, 5(4):554-568, 1994.
- Meri Coleman and Ta Lin Liau. A computer readability formula designed for machine scoring. Journal of Applied Psychology, 60(2):283, 1975.
- Nick Craswell, Onno Zoeter, Michael Taylor, and Bill Ramsey. An experimental comparison of click position-bias models. In Proceedings of the 2008 international conference on web search and data mining, pp. 87-94, 2008.
- W Bruce Croft, Donald Metzler, Trevor Strohman, et al. Search engines: Information retrieval in practice, volume 520. Addison-Wesley Reading, 2010.
- Edgar Dale and Jeanne S Chall. A formula for predicting readability: Instructions. Educational research bulletin, pp. 37-54, 1948.
- Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers), pp. 4171-4186, 2019.
- Yiming Du, Hongru Wang, Zhengyi Zhao, Bin Liang, Baojun Wang, Wanjun Zhong, Zezhong Wang, and Kam-Fai Wong. Perltqa: A personal long-term memory dataset for memory classification, re- trieval, and synthesis in question answering, 2024. URL https://arxiv.org/abs/2402. 16288.
- Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025a. · arXiv
- Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3, 2022.
- Yuanzhe Hu, Yu Wang, and Julian McAuley. Evaluating memory in llm agents via incremental multi-turn interactions, 2025. URL https://arxiv.org/abs/2507.05257. · arXiv
- Gongyao Jiang, Xinran Shi, and Qiong Luo. JRE-L: Journalist, reader, and editor LLMs in the loop for science journalism for the general audience. In Luis Chiruzzo, Alan Ritter, and Lu Wang (eds.), Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pp. 6579-6594, Albuquerque, New Mexico, April 2025. Association for Compu- tational Linguistics. ISBN 979-8-89176-189-6. doi: 10.18653/v1/2025.naacl-long.335. URL https://aclanthology.org/2025.naacl-long.335/. · DOI
- Jiazheng Kang, Mingming Ji, Zhe Zhao, and Ting Bai. Memory os of ai agent. arXiv preprint arXiv:2506.06326, 2025. · arXiv
- Jiho Kim, Woosog Chay, Hyeonji Hwang, Daeun Kyung, Hyunseung Chung, Eunbyeol Cho, Yohan Jo, and Edward Choi. Dialsim: A real-time simulator for evaluating long-term multi-party dialogue understanding of conversation systems, 2025. URL https://arxiv.org/abs/ 2406.13144.
- Haitao Li, You Chen, Qingyao Ai, Yueyue Wu, Ruizhe Zhang, and Yiqun Liu. Lexeval: A comprehensive chinese legal benchmark for evaluating large language models. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang (eds.), Advances in Neural Information Processing Systems, volume 37, pp. 25061-25094.
- Curran Associates, Inc., 2024a. URL https://proceedings.neurips.cc/paper_files/paper/2024/file/ 2cb40fc022ca7bdc1a9a78b793661284-Paper-Datasets_and_Benchmarks_ Track.pdf.
- Haitao Li, Qian Dong, Junjie Chen, Huixue Su, Yujia Zhou, Qingyao Ai, Ziyi Ye, and Yiqun Liu. Llms-as-judges: A comprehensive survey on llm-based evaluation methods, 2024b. URL https://arxiv.org/abs/2412.05579. · arXiv
- Lei Liu, Xiaoyan Yang, Yue Shen, Binbin Hu, Zhiqiang Zhang, Jinjie Gu, and Guannan Zhang. Think-in-memory: Recalling and post-thinking enable llms with long-term memory. arXiv preprint arXiv:2311.08719, 2023. · arXiv
- Junru Lu, Siyu An, Mingbao Lin, Gabriele Pergola, Yulan He, Di Yin, Xing Sun, and Yunsheng Wu. Memochat: Tuning llms to use memos for consistent long-range open-domain conversation. arXiv preprint arXiv:2308.08239, 2023. · arXiv
- Adyasha Maharana, Dong-Ho Lee, Sergey Tulyakov, Mohit Bansal, Francesco Barbieri, and Yuwei Fang. Evaluating very long-term conversational memory of llm agents, 2024. URL https: //arxiv.org/abs/2402.17753.
- Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to fol- low instructions with human feedback. Advances in neural information processing systems, 35: 27730-27744, 2022.
- Joon Sung Park, Joseph O'Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th annual acm symposium on user interface software and technology, pp. 1-22, 2023.
- Haoran Que, Feiyu Duan, Liqun He, Yutao Mou, Wangchunshu Zhou, Jiaheng Liu, Wenge Rong, Zekun Moore Wang, Jian Yang, Ge Zhang, Junran Peng, Zhaoxiang Zhang, Songyang Zhang, and Kai Chen. Hellobench: Evaluating long text generation capabilities of large language models, 2024. URL https://arxiv.org/abs/2409.16191. · arXiv
- Lianlei Shan, Shixian Luo, Zezhou Zhu, Yu Yuan, and Yong Wu. Cognitive memory in large lan- guage models, 2025. URL https://arxiv.org/abs/2504.02441. · arXiv
- Haizhou Shi, Zihao Xu, Hengyi Wang, Weiyi Qin, Wenyuan Wang, Yibin Wang, Zifeng Wang, Sayna Ebrahimi, and Hao Wang. Continual learning of large language models: A comprehensive survey. ACM Computing Surveys, 2024.
- Kurt Shuster, Jing Xu, Mojtaba Komeili, Da Ju, Eric Michael Smith, Stephen Roller, Megan Ung, Moya Chen, Kushal Arora, Joshua Lane, et al. Blenderbot 3: a deployed conversational agent that continually learns to responsibly engage. arXiv preprint arXiv:2208.03188, 2022. · arXiv
- Endel Tulving and Hans J Markowitsch. Episodic and declarative memory: role of the hippocampus. Hippocampus, 8(3):198-204, 1998.
- Chao Wang, Yiqun Liu, Min Zhang, Shaoping Ma, Meihong Zheng, Jing Qian, and Kuo Zhang. In- corporating vertical results into search click models. In Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval, pp. 503-512, 2013.
- Zhenduo Wang, Zhichao Xu, Vivek Srikumar, and Qingyao Ai. An in-depth investigation of user response simulation for conversational search. In Proceedings of the ACM Web Conference 2024, pp. 1407-1418, 2024.
- Ryen W. White, Joemon M. Jose, and Ian Ruthven. An implicit feedback approach for interactive information retrieval. Information Processing & Management, 42(1):166-190, 2006. ISSN 0306-4573. doi: https://doi.org/10.1016/j.ipm.2004.08.010. URL https:// www.sciencedirect.com/science/article/pii/S0306457304001001. For- mal Methods for Information Retrieval. · DOI
- Di Wu, Hongwei Wang, Wenhao Yu, Yuwei Zhang, Kai-Wei Chang, and Dong Yu. Longmemeval: Benchmarking chat assistants on long-term interactive memory. In The Thirteenth Interna- tional Conference on Learning Representations, 2025a. URL https://openreview.net/ forum?id=pZiyCaVuti.
- Yangzhen Wu, Zhiqing Sun, Shanda Li, Sean Welleck, and Yiming Yang. Inference scaling laws: An empirical analysis of compute-optimal inference for problem-solving with language models, 2025b. URL https://arxiv.org/abs/2408.00724. · arXiv
- Yaxiong Wu, Sheng Liang, Chen Zhang, Yichao Wang, Yongyue Zhang, Huifeng Guo, Ruiming Tang, and Yong Liu. From human memory to ai memory: A survey on memory mechanisms in the era of llms, 2025c. URL https://arxiv.org/abs/2504.15965. · arXiv
- Yuning Wu, Jiahao Mei, Ming Yan, Chenliang Li, Shaopeng Lai, Yuran Ren, Zijia Wang, Ji Zhang, Mengyue Wu, Qin Jin, and Fei Huang. Writingbench: A comprehensive benchmark for generative writing, 2025d. URL https://arxiv.org/abs/2503.05244. · arXiv
- Wujiang Xu, Kai Mei, Hang Gao, Juntao Tan, Zujie Liang, and Yongfeng Zhang. A-mem: Agentic memory for llm agents. arXiv preprint arXiv:2502.12110, 2025a. · arXiv
- Zhijian Xu, Yilun Zhao, Manasi Patwardhan, Lovekesh Vig, and Arman Cohan. Can LLMs iden- tify critical limitations within scientific research? a systematic evaluation on AI research papers. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (eds.), Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Vol- ume 1: Long Papers), pp. 20652-20706, Vienna, Austria, July 2025b. Association for Compu- tational Linguistics. ISBN 979-8-89176-251-0. doi: 10.18653/v1/2025.acl-long.1009. URL https://aclanthology.org/2025.acl-long.1009/. · DOI
- Ming Yan, Ruihao Li, Hao Zhang, Hao Wang, Zhilan Yang, and Ji Yan. Larp: Language-agent role play for open-world games. arXiv preprint arXiv:2312.17653, 2023. · arXiv
- An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025a. · arXiv
- Zhe Yang, Yichang Zhang, Yudong Wang, Ziyao Xu, Junyang Lin, and Zhifang Sui. A probabilistic inference scaling theory for llm self-correction, 2025b. URL https://arxiv.org/abs/ 2508.16456.
- Erhan Zhang, Xingzhu Wang, Peiyuan Gong, Zixuan Yang, and Jiaxin Mao. Exploring human-like thinking in search simulations with large language models. In Proceedings of the 48th Inter- national ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 2669-2673, 2025a.
- Kaiyuan Zhang, Jiaqi Li, Yueyue Wu, Haitao Li, Cheng Luo, Shaokun Zou, Yujia Zhou, Weihang Su, Qingyao Ai, and Yiqun Liu. Chinese court simulation with llm-based agent system, 2025b. URL https://arxiv.org/abs/2508.17322. · arXiv
- Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. Bertscore: Evalu- ating text generation with bert, 2020. URL https://arxiv.org/abs/1904.09675. · arXiv
- Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang, Pengjun Xie, An Yang, Dayiheng Liu, Junyang Lin, et al. Qwen3 embedding: Advancing text embedding and reranking through foundation models. arXiv preprint arXiv:2506.05176, 2025c. · arXiv
- Yinan Zhang, Xueqing Liu, and ChengXiang Zhai. Information retrieval evaluation as search simu- lation: A general formal framework for ir evaluation. In Proceedings of the ACM SIGIR Interna- tional Conference on Theory of Information Retrieval, pp. 193-200, 2017.
- Zeyu Zhang, Quanyu Dai, Luyu Chen, Zeren Jiang, Rui Li, Jieming Zhu, Xu Chen, Yi Xie, Zhenhua Dong, and Ji-Rong Wen. Memsim: A bayesian simulator for evaluating memory of llm-based personal assistants, 2024. URL https://arxiv.org/abs/2409.20163. · arXiv
- Wanjun Zhong, Lianghong Guo, Qiqi Gao, He Ye, and Yanlin Wang. Memorybank: Enhancing large language models with long-term memory. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 19724-19731, 2024.
Cited by
Sorted by citations
From Storage to Experience: A Survey on the Evolution of LLM Agent Memory Mechanisms
Luo, J., Tian, Y., Cao, C., Luo, Z., Lin, H., Li, K., Kong, C., Yang, R., & Ma, J. (2026) · Findings of the Association for Computational Linguistics: ACL 2026 · doi:10.18653/v1/2026.findings-acl.2069 · Cited by 3 · see citing works
MemTrace: Probing What Final Accuracy Misses in Long-Term Memory
Long, X., Chen, Z., Zeng, S., Wang, S., Guo, K., & Tang, J. (2026) · arXiv · doi:10.48550/arxiv.2606.17328
“…Bian et al., 2026), agent memory (Hu et al 2025; Tavakoli et al., 2026), dynamic profiles and evolving preferences (Li et al., 2026; Uddin et al., 2026), stale or hallucinated memories (Chao et al., 2026; Tan et al., 2025), and missing or conflicting evidence (Chen et al., 2026a; Ai et al., 2026).”
Zhang, W., Wei, X., Huang, W.-C., Hui, Z., Wang, C., Gong, M., & Yu, P. S. (2026) · arXiv · doi:10.48550/arxiv.2603.25973
“First, the majority of memory benchmarks (Maharana et al., 2024; Ai et al., 2025; Tan et al., 2025a; Zhao et al., 2025; Wu et al., 2025a; Bei et al., 2026) are synthetic and constructed from LLM simulation that fail to provide user feedback from authentic users.”
Self-citationHindsight: Structured Agent Memory that Retains, Recalls, and Reflects
Latimer, C., Boschi, N., Neeser, A., Bartholomew, C., Srivastava, G., Wang, X., & Ramakrishnan, N. (2026) · Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations) · doi:10.18653/v1/2026.acl-demo.27
“This component also supports Mem-oryBench (Ai et al., 2025) questions, providing additional coverage of memory and continual learning evaluation scenarios (Tavakoli et al., 2025).”
GroupMemBench: Benchmarking LLM Agent Memory in Multi-Party Conversations
Yang, J., Lai, K.-H., Wang, X., Chang, S., Harari, Y., & Gabrilovich, E. (2026) · arXiv · doi:10.48550/arxiv.2605.14498
“…[13] expand conversations to hundreds of sessions and test multi-session recall; MemBench [30], MemoryAgentBench [21], MemoryArena [10], AMA-Bench [11], and MemoryBench [12] broaden the question taxonomy to include multi-hop reasoning, temporal alignment, knowledge updates, and continual learning.”
MemGym: a Long-Horizon Memory Environment for LLM Agents
Xu, W., Wang, Y., Mei, K., Liang, K., Wang, Z., Jin, M., Zhang, H., Zhang, S.-X., Hua, W., Sahu, S., et al. (2026) · arXiv · doi:10.48550/arxiv.2605.20833
“…✗ ✗ Low ✗ 9K LongMemEval [48] Long-term dialogue 1 ✗ ✗ Medium ✗ 115K MemoryAgentBench [18] Multi-turn dialogue 1 ✗ ✗ Medium ✗ 100K–300K MemoryBench [1] Continual dialogue 1 Long-Horizon Agent Benchmarks SWE-Gym [32] Repository coding 1 ✓ ✗ High ✓ Task-dep. τ 2 -bench [5] Tool-agent-user dialogue 1…”
MemoryArena: Benchmarking Agent Memory in Interdependent Multi-Session Agentic Tasks
He, Z., Wang, Y., Zhi, C., Hu, Y., Chen, T.-P., Yin, L., Chen, Z., Wu, T. A., Ouyang, S., Wang, Z., et al. (2026) · arXiv · doi:10.48550/arxiv.2602.16313
“LoCoMo (Maharana et al., 2024), LongMemEval (Wu et al., 2025), MemoryAgent-Bench (Hu et al., 2025a), MemoryBench (Ai et al., 2025), and EvoMem (Wei et al., 2025b) assess whether models can retain and recall information introduced in the previous interactions.”
A Survey of Agent Memory in the Second Half: Towards Self-Evolving and Long-Horizon Agents
Huang, W.-C., Zhang, W., Liang, Y., Bei, Y., Chen, Y., Feng, T., Pan, X., Tan, Z., Wang, Y., Wei, T., et al. (2026) · arXiv · doi:10.48550/arxiv.2602.06052
Self-citationAMA-Bench: Evaluating Long-Horizon Memory for Agentic Applications
Zhao, Y., Yuan, B., Huang, J., Yuan, H., Yu, Z., Xu, H., Hu, L., Shankarampeta, A., Huang, Z., Ni, W., et al. (2026) · arXiv · doi:10.48550/arxiv.2602.22769
Agentic Memory Management Is the Next Frontier of Information Retrieval
Ai, Q. (2026) · arXiv · doi:10.48550/arxiv.2610.32464
Self-citationMeta Context Engineering via Agentic Skill Evolution
Ye, H., He, X., Arak, V., Dong, H., & Song, G. (2026) · arXiv · doi:10.48550/arxiv.2601.21557 · Cited by 1
- Authors
- Ai, Q., Tang, Y., Wang, C., Long, J., Su, W., & Liu, Y.
- Affiliations
- Tsinghua University (CN)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2025
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2510.17281
- OpenAlex
- W4415963275
- Cited by
- 0
- References
- 0
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- LLM memory, continual learning, benchmark, large language models, learning from practice
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
