Lost in the Middle: How Language Models Use Long Contexts
Open accessThe DOI matches its Crossref record; 48 of the 59 references it cites were found in a registry or an index; 15 of the 23 sources its claims cite were checked for support.
Abstract
While recent language models have the ability to take long contexts as input, relatively little is known about how well they use longer context. We analyze the performance of language models on two tasks that require identifying relevant information in their input contexts: multi-document question answering and key-value retrieval. We find that performance can degrade significantly when changing the position of relevant information, indicating that current language models do not robustly make use of information in long input contexts. In particular, we observe that performance is often highest when relevant information occurs at the beginning or end of the input context, and significantly degrades when models must access relevant information in the middle of long contexts, even for explicitly long-context models. Our analysis provides a better understanding of how language models use their input context and provides new evaluation protocols for future long-context language models.
Overview
CoChat Summary
TL;DR
Language models’ performance on tasks requiring access to long input contexts is strongly affected by where relevant information appears, with a U-shaped curve: best at the beginning or end and worst in the middle. Extended-context models are not necessarily better at using context, and reader performance saturates before retriever recall in open-domain QA.
Key findings
- In multi-document QA, performance is highest when the answer-containing document is at the start or end of the context and degrades in the middle; GPT-3.5-Turbo can drop by more than 20%, and in 20- and 30-document settings middle-position performance is lower than its closed-book performance of 56.1% (oracle: 88.3%).
- Extended-context models are not necessarily better at using input context: when the input fits both windows, GPT-3.5-Turbo and GPT-3.5-Turbo(16K) perform nearly identically as a function of relevant-information position.
- In synthetic key-value retrieval, Claude-1.3 and Claude-1.3(100K) do nearly perfectly, while other models struggle especially with 140 or 300 key-value pairs; query-aware contextualization gives GPT-3.5-Turbo(16K) perfect performance at 300 pairs, versus worst-case 45.6% without it.
- In open-domain QA, reader performance saturates long before retriever recall: using 50 instead of 20 retrieved documents only marginally improves performance (∼1.5% for GPT-3.5-Turbo and ∼1% for Claude-1.3).
Methods
- Controlled multi-document QA on NaturalQuestions-Open: 2,655 queries whose annotated long answer is a paragraph; each context has exactly one answer-containing Wikipedia passage plus k−1 Contriever/MS-MARCO distractor passages; the paper varies answer position and context length (10, 20, 30 documents) and measures accuracy.
- Synthetic key-value retrieval: inputs are serialized JSON objects with unique random UUID key-value pairs; models must return the value for a specified key; experiments use 75, 140, and 300 key-value pairs (500 examples each), varying key position and number of pairs, with accuracy as the metric.
- Models include open MPT-30B-Instruct and LongChat-13B(16K), closed GPT-3.5-Turbo, GPT-3.5-Turbo(16K), Claude-1.3, and Claude-1.3(100K), plus additional architecture/instruction experiments with Flan-T5-XXL, Flan-UL2, MPT-30B, Llama-2, and GPT-4; an open-domain QA case study uses a Contriever retriever-reader setup.
Limitations
- Stated: the paper uses greedy decoding and leaves exploration of other decoding methods to future work; the investigations into why models are not robust are described as preliminary; GPT-4 is evaluated only on a subset of 500 multi-document QA examples because full evaluation would cost upwards of $6000.
- Stated: NaturalQuestions-Open has ambiguity, so the authors run an unambiguous subset and also test random distractors and randomized distractor order, finding similar trends.
- Evident: the controlled multi-document QA setting always contains exactly one document that answers the question, unlike open-domain QA; the synthetic key-value task removes natural language semantics; results may not generalize to all tasks, models, or decoding settings.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
7 Valid · 0 Not valid · 5 Not checked · 13 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Language models have become an important and flexible building block in a variety of user-facing language technologies, including conversational interfaces, search and summarization, and collaborative writing (Shuster et al., 2022;Thoppilan et al., 2022;Lee et al., 2022, inter alia).”
§ Lost in the Middle: How Language Models Use Long Contexts > 1 Introduction
What the sources say
- Partly supportedcites Shuster et al., 2022 sourceExperimental — accuracy not yet measured
Section 3.1.1 (p. 4) describes a conversational agent built on a language model that generates internet search queries, supporting conversational interfaces and search as part of dialogue. Section 2 (p. 3) mentions summarization only as related work, and the passages do not establish collaborative writing as an application of this paper’s model. (LLM, conf=0.85)
- Supportedcites Thoppilan et al., 2022 source
The Abstract (p. 1) identifies LaMDA as a family of language models specialized for dialog. In this sentence's list of applications, Thoppilan et al. supports the conversational-interface example; the other cited works need not each cover every example. (LLM, conf=0.92)
- Supportedcites Lee et al., 2022 source
Lee et al. is cited alongside other works, and its contribution to the sentence is the collaborative-writing example. Section 4.2.2 describes writers receiving and selecting model-generated suggestions in a writing interface; Section 2.2.1 also describes language models as applicable to dialogue and summarization. The paper need not establish every example in the jointly cited list. (LLM, conf=0.89)
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“These observations complement prior work, which found that non-instruction fine-tuned language models are biased towards recent tokens (i.e., the end of the input context; Khandelwal et al., 2018;Press et al., 2021).”
§ Lost in the Middle: How Language Models Use Long Contexts > 4.3 Effect of Instruction Fine-Tuning
What the sources say
- Partly supportedcites Khandelwal et al., 2018 sourceExperimental — accuracy not yet measured
The Introduction and Section 5 report that the studied LSTM is much more sensitive to recent word order than to distant word order, supporting a recency effect. The paper studies a standard LSTM on two datasets, however, rather than establishing this tendency across non-instruction-fine-tuned language models generally; it also finds that distant words remain relevant. (LLM, conf=0.82)
- Partly supportedcites Press et al., 2021 sourceExperimental — accuracy not yet measured
Section 3.1 (page not available in our copy) finds that very short contexts hurt performance, while increasing subsequence length from 1,024 to 3,072 tokens gives negligible improvement. That is consistent with limited benefit from more distant context, but the paper does not establish that the models are biased toward recent tokens within an input context. (LLM, conf=0.78)
- Partly supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“We instantiate this task with data from NaturalQuestions-Open (Lee et al., 2019;Kwiatkowski et al., 2019), which contains historical queries issued to the Google search engine, coupled with human-annotated answers extracted from Wikipedia.”
§ Lost in the Middle: How Language Models Use Long Contexts > 2.1 Experimental Setup
What the sources say
- Supportedcites Lee et al., 2019 source
Section 7.1 says the paper creates an open version of Natural Questions and uses it for training and evaluation. The Introduction describes the task as answering questions by retrieving evidence from an open corpus, supporting the cited use of NaturalQuestions-Open for this task. (LLM, conf=0.91)
- Not checked — no full text of the sourcecites Kwiatkowski et al., 2019 source
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Following Kandpal et al. (2022) and Mallen et al. (2023), we use accuracy as our primary evaluation metric, judging whether any of the cor- rect answers (as taken from the NaturalQuestions annotations) appear in the predicted output.”
§ Lost in the Middle: How Language Models Use Long Contexts > 2.1 Experimental Setup
What the sources say
- Supportedcites Kandpal et al. (2022) source
Section 3 (p. 4) says the paper evaluates answers using exact match against a ground-truth answer set, and Section 3.1 (p. 4) reports QA accuracies as its evaluation results. This supports citing Kandpal et al. for using accuracy to evaluate question answering. (LLM, conf=0.90)
- Supportedcites Mallen et al. (2023) source
Section 3.1 (‘Focus and Task’) explicitly names accuracy as the metric and says a prediction is correct when it contains an exact match to any gold answer. This supports the evaluation approach attributed to Mallen et al.; the NaturalQuestions annotations describe the citing authors’ answer source. (LLM, conf=0.94)
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“or when language models are augmented with external information (relevant documents from a search engine, database query results, etc.; Petroni et al., 2020;Ram et al., 2023;Shi et al., 2023;Mallen et al., 2023;Schick et al., 2023, inter alia).”
§ Lost in the Middle: How Language Models Use Long Contexts > 1 Introduction
What the sources say
- Partly supportedcites Petroni et al., 2020 sourceExperimental — accuracy not yet measured
The Abstract (p. 1) says the paper integrates information from a retrieval system with a pre-trained language model, and §3.4.2 (p. 5) specifies a retrieved paragraph as context. This supports augmentation with retrieved documents, but our copy does not show language models being augmented with database query results. (LLM, conf=0.84)
- Partly supportedcites Ram et al., 2023 sourceExperimental — accuracy not yet measured
The Introduction describes augmenting language models with relevant documents retrieved from an external knowledge source, and Section 5 describes using a BM25 retriever to find those documents. The passages do not establish database query results as another type of external information used by this paper. (LLM, conf=0.82)
- Partly supportedcites Shi et al., 2023 sourceExperimental — accuracy not yet measured
Section 3.1 (page unavailable) describes retrieving relevant documents from a corpus, and the Conclusion says REPLUG uses those documents to augment a language model. The paper supports the document-retrieval example, but our copy does not describe augmenting models with database query results or documents from a search engine specifically. (LLM, conf=0.82)
- Partly supported
20 more claims · 2 with their cited sources checked
Create a free account to see all 25 claims and every source verdict.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In vanilla multi-document or NIAH-based benchmarks (Kamradt, 2023; Liu et al., 2024), models perform consistently well when the needle (or gold document) appears at the very beginning or end of the context window, with minimal impact from context length.”
From NoLiMa: Long-Context Evaluation Beyond Literal Matching (2025) · cites this paper as Liu et al., 2024
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Even within a single long sequence, attention effectiveness degrades with distance due to attention dilution , positional encoding limitations, and token interference, leading to the well-known 'lost-in-the-middle' and context-decay phenomena (Liu et al., 2024; Press et al., 2021a).”
From MAGMA: A Multi-Graph based Agentic Memory Architecture for AI Agents (2026) · cites this paper as Liu et al., 2024
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Current large language models (LLMs) exhibit significant deficiencies in episodic memory tasks including encoding, storing, and retrieving specific information from temporally dependent events over a long period of time.”
From ARTEM: Enhancing Large Language Model Agents with Spatial-Temporal Episodic Memory (2026) · cites this paper as Liu et al. 2024
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Recent approaches to handle memory tasks in LLMs, such as in-context learning, retrieval-augmented generation (RAG), and fine-tuning, may resolve the long-term retention issues, but are still inadequate to handle tasks requiring chronological awareness of the stored information.”
From ARTEM: Enhancing Large Language Model Agents with Spatial-Temporal Episodic Memory (2026) · cites this paper as Liu et al. 2024
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“If not ranked at the top, the key information may be neglected by the generative module, resulting in incomplete or inaccurate responses to the user queries24.”
From Leveraging long context in retrieval augmented language models for medical question answering (2025) · cites this paper as [24]
Experimental — accuracy not yet measured
86 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929
LLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602
Transformer-XL: Attentive language models beyond a fixed-length context
Dai, Z., Yang, Z., Yang, Y., Carbonell, J., Le, Q., & Salakhutdinov, R. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1285 · Cited by 3,169 · cited 2×
“Recent improvements in hardware (e.g., faster GPUs with more memory) and algorithms (Dai et al., 2019; Dao et al., 2022; Poli et al., 2023; Rubin and Berant, 2023, inter alia ) have resulted in language models with larger context windows (e.g., 4096, 32K, and even 100K tokens), but it remains…”
BackgroundLlama 2: Open Foundation and Fine-Tuned Chat Models
Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.09288 · Cited by 2,604
Longformer: The Long-Document Transformer
Beltagy, I., Peters, M. E., & Cohan, A. (2020) · arXiv · doi:10.48550/arxiv.2004.05150 · Cited by 2,281
Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering
Izacard, G., & Grave, E. (2020) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.48550/arxiv.2007.01282 · Cited by 2,263
Natural Questions: A benchmark for question answering research
Kwiatkowski, T., Palomaki, J., Redfield, O., Collins, M., Parikh, A., Alberti, C., Epstein, D., Polosukhin, I., Devlin, J., Lee, K., et al. (2019) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00276 · Cited by 2,069 · cited 1×
“We instantiate this task with data from NaturalQuestions-Open (Lee et al., 2019; Kwiatkowski et al., 2019), which contains historical queries issued to the Google search engine, coupled with human-annotated answers extracted from Wikipedia.”
MethodThe serial position effect of free recall.
Murdock, B. B. (1962) · Journal of Experimental Psychology · doi:10.1037/h0045106 · Cited by 1,717 · cited 1×
“The U-shaped curve we observe in this work has a connection in psychology known as the serial-position effect (Ebbinghaus, 1913; Murdock Jr, 1962), that states that in free-association recall of elements from a list, humans tend to best re-member the first and last elements of the list.”
MethodMemory: A contribution to experimental psychology.
Ebbinghaus, H. (1913) · doi:10.1037/10011-000 · Cited by 1,645 · cited 1×
“The U-shaped curve we observe in this work has a connection in psychology known as the serial-position effect (Ebbinghaus, 1913; Murdock Jr, 1962), that states that in free-association recall of elements from a list, humans tend to best re-member the first and last elements of the list.”
Method
Research works printed but not linked: 1
- Sharan, 2018, Prediction with a short memory, 10.18653/v1/K18-3013 · DOI
Cited by
Items 1,501–1,594 of 1,594
KERKIS: a Modeling Language for Multi-Agent System Interactions in AI-Native Software Engineering
Kalliokoski, K., Christophe, F., Kärkkäinen, H., Hämäläinen, I., Hasan, M. T., Hasan, M. M., Zhang, Z., Abrahamsson, P., & Mikkonen, T. (2026) · ACM/IEEE International Conference on Model Driven Engineering Languages and Systems (MODELS) · doi:10.1145/3837062.3839380
Lu, C., Zhang, P., Hao, J., Luo, X., Li, K., Lu, Y. C., & Cui, X. (2026) · Colloids and Surfaces A Physicochemical and Engineering Aspects · doi:10.1016/j.colsurfa.2026.141995
Karaarslan, E., Alan, A. Y., Kumbalı, S., & Aydın, Ö. (2026) · İzmir Sosyal Bilimler Dergisi · doi:10.47899/izsbd.1974910 · Cited by 1
Against mandatory prompt disclosure in AI-assisted scholarship
Earp, B. D., Schüklenk, U., Savulescu, J., & Mann, S. P. (2026) · AI & Society · doi:10.1007/s00146-026-03326-w · Cited by 1
Sulaima, M. F., Jyotsna, A., Herout, L., Gálvez, I. P., Krejcar, O., & Namazi, H. (2026) · Energy Nexus · doi:10.1016/j.nexus.2026.100870
Latifaj, M. (2026) · ACM/IEEE International Conference on Model Driven Engineering Languages and Systems (MODELS) · doi:10.1145/3837062.3838892
LIBER: Lifelong User Behavior Modeling Based on Large Language Models
Shan, R., Zhu, C., Quan, S., Chen, B., Lin, J., Cai, X., Zhu, H., Li, X., Xi, Y., Zhang, W., et al. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37682-4_24
CM-USR: Critical-Missing Calibrated User-State Regeneration for Lost-in-Conversation
Xu, Y., & Li, X. (2026) · International Conference on Computer Science and Blockchain (CCSB) · doi:10.1109/ccsb71176.2026.11716468
“Long-context studies also find that evidence position affects retrieval and use [8].”
Lin, P., Xu, H., & Yang, J. (2026) · Research Square · doi:10.21203/rs.3.rs-11009751/v2
Venkatachalapathy, R. (2026) · International Conference on Soft Computing for Security Applications (ICSCSA) · doi:10.1109/icscsa69367.2026.11708424
“Retrieval-augmented generation and long-context reasoning further enable models to reason over external evidence, although context selection and evidence grounding remain important challenges [10]-[12].”
Retrieval-Augmented Reasoning for Knowledge-Intensive AI Systems Using Dynamic Evidence Acquisition
Venkatachalapathy, R. (2026) · International Conference on Soft Computing for Security Applications (ICSCSA) · doi:10.1109/icscsa69367.2026.11707973
“Adding more documents can increase recall but may also introduce distracting context, especially when evidence is buried in long prompts [9].”
Aksyonov, K. A., Sun, L., & Aksyonova, O. P. (2026) · International Russian Automation Conference (RusAutoCon) · doi:10.1109/rusautocon69876.2026.11707467
“Pure prompt-based LLMs exhibit instability with long JSON inputs, repeated fields, exhaustive rule pair enumeration, and strict format output [8-9]; even when models can provide natural language explanations, they cannot guarantee consistent conflict identification across runs.”
Bashir, A., & Peng, R. (2026) · Data & Knowledge Engineering · doi:10.1016/j.datak.2026.102656
EmphaBot: Enhancing accessibility in mental health support using a CoT-RAG-based language model
Nwokoye, C. H., & Iloka, B. O. (2026) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2026.101361
Effective Context Management for LLM Agents via Hybrid Chat History Optimization
Songa, S., Bayyavarapu, S. R. K. V., Gulkotwar, N., Pasupuleti, V., Allala, S. R., & Tyagi, S. (2026) · IEEE World Conference on Applied Intelligence and Computing (AIC) · doi:10.1109/aic70896.2026.11717333
PROTEUS: Privacy-Preserving On-Premise RAG for Biomedical Text Understanding
Antar, S. S., & Ador, S. S. (2026) · International Conference on Technology Innovation and Its Applications (ICTIIA) · doi:10.1109/ictiia70925.2026.11717316
“LLMs exhibit a U-shaped positional bias [24].”
CARE-KGQA: Refusal-Aware Answerability Recovery for Knowledge Graph Question Answering
Kong, K., & Zhou, Z. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-5696-9_20
On-Device GitHub Contributions Audit utilizing Recursive Language Models
Effendy, C. L., Djayadi, D., Wati, P. M., & Saputri, T. R. D. (2026) · International Conference on Technology Innovation and Its Applications (ICTIIA) · doi:10.1109/ictiia70925.2026.11717373
“To overcome the context window bottleneck [5], this system utilizes Recursive Language Models (RLMs) [6].”
Kővári, A. (2026) · Discover Artificial Intelligence · doi:10.1007/s44163-026-02312-x
“The shift in focus in such an event changes the primary design challenge from determining how much context to include to selecting which context to include and how to control it [9–11].”
InfluentialA Semantic-Adaptive Hybrid Retrieval Framework for the Education Domain
Xiao, W., Wang, K., Feng, A., Chen, K., & Yu, T. (2026) · International Conference on Artificial Intelligence, Digital Media and Education · doi:10.1145/3839592.3839618
“Liu et al. [14] experimentally showed that large language models are highly sensitive to the positions of key information in long contexts.”
Cognitive evolution enables de novo design of untethered millimeter-scale robotsPreprint
Li, H., Zhao, W., Tao, Z., Li, P., Wang, X., Wang, Q., Wang, B., Zhai, Y., Wei, W., Wang, S., et al. (2026) · Research Square · doi:10.21203/rs.3.rs-10770716/v1
From Interaction Topology to Realized Context: Diagnosing Exposure in LLM-Agent Social Simulation
Xu, Y., Han, X., & Ma, C. (2026) · ACM Turing Celebration Conference - China (ACM TURC) · doi:10.1145/3833142.3833202
SafeDep: Detecting State Mutation Unsoundness in Rust via Dependency Slicing and LLM Reasoning
Rao, Z., Chen, Y., Chen, C., & Xu, H. (2026) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3832783.3837481
What Breaks When LLMs Code? Characterizing Operational Safety Failures of Agentic Code Assistants
Hasan, A. A., & Biswas, S. (2026) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3832783.3834393 · Cited by 7
“As an agent engages in extended sessions or crosses session boundaries, it loses the ability to retrieve the true system state from its expanding context window [56].”
TraceGate: A Framework for Policy-Controlled Failure Evidence in LLM-Assisted Program Repair
Schuler, N., Scotti, V., & Mirandola, R. (2026) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3832783.3834596 · Cited by 1
“However, their utility is context-dependent: evidence that aids one repair attempt may distract the model, increase context length, or raise costs in another [4, 7, 8, 11].”
CtxFuzz: Context-Aware Directed Fuzzing via Runtime Fuzzer-Agent Cooperation
Kim, J., Kim, D., Han, H., Kim, Y., Park, S., Ji, S., Wang, J., Jeong, D. R., & Kim, T. (2026) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3832783.3834365
“However, LLMs degrade on long contexts, particularly for generation rather than retrieval or classification [28].”
LLM-Assisted Joint Ticket and Log Analysis for Incident Triage in Intelligent and Connected Vehicles
Fu, R., Zhang, S., Gu, W., Li, W., Sun, Y., & Pei, D. (2026) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3832783.3834521
“When applying LLMs to log analysis, the limited context length becomes a critical constraint that must be carefully addressed [21].”
Prompts Matter: On the Impact of Code Smell Remediation with Large Language Models
Hagatulah, N., Borg, M., & Söderberg, E. (2026) · doi:10.1145/3843779.3844637
Stealth of Guidance Injection: How Malicious OpenClaw Skills Evade Existing Detectors
Liu, F., Chen, Z., Lan, T., Tan, H., Xu, Z., Li, X., Chen, G., Meng, Y., & Zhu, H. (2026) · ACM Turing Celebration Conference - China (ACM TURC) · doi:10.1145/3833142.3833171
“Due to LLMs’ primacy effect[1], information early in the context window strongly shapes later interpretations.”
Ly, D., Aydemir, F. B., & Dalpiaz, F. (2026) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3832783.3837558
GPT-4o Translation of Pediatric Discharge Instructions Across Languages
Kats, D. J., Ray, M., Moorkens, J., Rai, D., Brewster, R., Khan, A., Rader, B. M., Brownstein, J. S., & Hron, J. D. (2026) · Hospital Pediatrics · doi:10.1542/hpeds.2026-009476
When Comments Mislead: An Empirical Study of Outdated Comments in LLM-Based Code Completion
Xia, Z., Huang, Y., Chen, X., & Zheng, Z. (2026) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3832783.3834359
“This observation is consistent with recent work emphasizing that context quality and alignment, rather than quantity alone, are critical for reliable LLM behavior [30].”
Roh, J., & Bae, K. (2026) · ETRI Journal · doi:10.4218/etrij.2026-0242 · Cited by 2
Ranking Is Not Authority: Executable Context-Routing Contracts for Coding Agents
Mahesh, N. (2026) · Proceedings of the 1st International Workshop on PromptOps and Vibe Coding · doi:10.1145/3843779.3844639
Let’s Chat about Design: A Transformation-Oriented Viewpoint for LLM-Native Software
Braberman, V. A., & Bonomo, F. (2026) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3832783.3834545
“Motifs in this group explore large-scale entities (e.g., codebases, repositories) where bulk processing for analysis or information retrieval fails [13].”
Geometry-Aware Prompt Context for Model Consistency Management
Ojha, M., Gupta, S., & Sharma, R. (2026) · doi:10.1145/3843779.3844638
“Paying that cost on every query is wasteful when the answer depends on a handful of classifiers, and models attend unevenly to long contexts, degrading on material buried in the middle of the window [7].”
Park, S., Kim, L., Park, D., & Lee, S. (2026) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2026.124895
Bakhtary, P. (2026) · BenchCouncil Transactions on Benchmarks Standards and Evaluations · doi:10.66834/0azb3w20
A legal support system for entrepreneurs based on a Retrieval-Augmented Generation architecture
БОЙКОВ, Е. С., Kolyeva, N. S., & Kortenko, L. (2026) · Digital models and solutions · doi:10.29141/2949-477x-2026-5-3-3
Zhang, W., Wei, X., Huang, W.-C., Hui, Z., Wang, C., Gong, M., & Yu, P. S. (2026) · arXiv · doi:10.48550/arxiv.2603.25973
“During inference, LoCoMo performs semantic retrieval to bring relevant memories into the prompt, improving response accuracy compared to the Long-Context baseline by reducing reliance on position biases and effectively foregrounding past events even when distant in the session.”
InfluentialLahmy, M., & Yozevitch, R. (2025) · arXiv · doi:10.48550/arxiv.2512.10787 · Cited by 1
“The k = 5 results empirically validate the “Context Dilution” hypothesis [19].”
InfluentialAnatomy of Agentic Memory: Taxonomy and Empirical Analysis of Evaluation and System Limitations
Jiang, D., Li, Y., Wei, S., Yang, J., Kishore, A., Zhao, A., Kang, D., Hu, X., Chen, F., Li, Q., et al. (2026) · arXiv · doi:10.48550/arxiv.2602.19320
“However, fixed context windows fundamentally limit their ability to retain and manipulate persistent state (Brown et al., 2020; Beltagy et al., 2020; Liu et al., 2024; Press et al., 2021).”
FadeMem: Biologically-Inspired Forgetting for Efficient Agent Memory
Wei, L., Peng, X., Dong, X., Xie, N., & Wang, B. (2026) · arXiv
“…categories of memory management approaches: (1) Fixed-window methods : including context windows of 4K, 8K, and 16K tokens with FIFO eviction [10, 2]; (2) RAG-based systems : LangChain Memory with default configurations; (3) Specialized agent memory : Mem0 [11] as our primary baseline,…”
Cherif, A. (2026) · arXiv · doi:10.48550/arxiv.2608.00009 · Cited by 1
“However, studies consistently show that LLMs suffer from lost-in-the-middle degradation [20]: retrieval accuracy drops for information placed far from the start or end of the context, making raw window extension insufficient for production deployments.”
Behrouz, A., Razaviyayn, M., Zhong, P., & Mirrokni, V. (2025) · arXiv · doi:10.48550/arxiv.2504.13173 · Cited by 65
Lindenbauer, T., Slinko, I., Felder, L., Bogomolov, E., & Zharov, Y. (2025) · arXiv · doi:10.48550/arxiv.2508.21433
“More critically, even with context windows exceeding 1M tokens, LLMs suffer from the "lost in the middle" problem [16].”
Weighted Memory Tree: Remembering What Matters for Long-Horizon LLM Agents
Dao, Q., Kathalkar, P., & Eaton, K. (2026) · arXiv · doi:10.48550/arxiv.2608.20631 · Cited by 1
“…studies have shown that language models struggle to utilize relevant information within lengthy inputs, with performance often degrading well below the nominal context-window limit (Liu et al., 2024; Levy et al., 2024; Fraga, 2024; He et al., 2024; Hsieh et al., 2024; Tian et al., 2025).”
Activity Frames: Deterministic Screen-Activity Compilation for Agent Memory and Replay
Iyamu, N. (2026) · arXiv
“Context-engineering and long-context degradation results both argue for compact structured context over raw dumps [9,37].”
Cao, S., Zou, K., & Wang, L. (2025) · Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2025.emnlp-main.1707
“For comparisons, we include the extraction of corresponding keys with given values from JSON file (JSON KV) (Liu et al., 2023b; Yen et al., 2024).”
Make Each Token Count: Towards Improving Long-Context Performance with KV Cache Eviction
Bui, N., Nguyen, H. T., Cohan, A., & Ying, R. (2026) · arXiv · doi:10.48550/arxiv.2605.09649
“Long-context failures can arise even when the relevant evidence is present in the cache [Liu et al., 2024c; Yang et al., 2025a] (see Figure 1a).”
AriadneMem: Threading the Maze of Lifelong Memory for LLM Agents
Zhu, W., Chen, X., Wang, Z., Wang, J., Dong, X., Huang, M., Cai, R., Sang, H., Wang, H., Qiu, P., et al. (2026) · arXiv · doi:10.48550/arxiv.2603.03290 · Cited by 4
“While large context windows have expanded, they remain insufficient for lifelong interaction and suffer from ”lost-in-the-middle” phenomena Liu et al. (2024).”
EvolMem: A Cognitive-Driven Benchmark for Multi-Session Dialogue Memory
Shen, Y., Pei, D., Guo, Y., Wang, J., Guo, Y., Zhang, Z., Jia, Q., Zhou, J., & Zhai, G. (2026) · arXiv · doi:10.48550/arxiv.2601.03543
“As large language models (LLMs) are increasingly deployed in interactive and long-horizon applications (Xi et al., 2025; Packer et al., 2023; Liu et al., 2024a; Ouyang et al., 2022), memory plays a central role in enabling models to preserve user preferences, accumulate knowledge, and support…”
Ai, Q., Zhan, J., & Liu, Y. (2025) · arXiv · doi:10.48550/arxiv.2501.02842
“For example, retrieved documents could be lengthy, and LLMs are usually not good at processing long input context [101].”
Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization
Hsieh, C.-Y., Chuang, Y.-S., Li, C.-L., Wang, Z., Le, L. T., Kumar, A., Glass, J., Ratner, A., Lee, C.-Y., Krishna, R., et al. (2024) · arXiv · doi:10.48550/arxiv.2406.16008
“Unfortunately, re-ranking usually requires additional supervision or dedicated finetuning for performant RAG performance (Karpukhin et al., 2020; Shi et al., 2023c; Sun et al., 2023).”
InfluentialHelwig, M. (2026) · arXiv · doi:10.48550/arxiv.2609.05510
“Model-side evidence that irrelevant context degrades output [Shi 2023; Liu 2024; Chroma 2025] argues for injecting less; the reader-side skim reflex argues for silence as the default .”
MEM1: Learning to Synergize Memory and Reasoning for Efficient Long-Horizon Agents
Zhou, Z., Qu, A., Wu, Z., Kim, S., Prakash, A., Rus, D., Zhao, J., Low, B. K. H., & Liang, P. P. (2025) · arXiv · doi:10.48550/arxiv.2506.15841 · Cited by 218
“This reduces its ability to reason effectively, even when relevant information is technically still present within the prompt [2, 30, 56].”
Self-citationATLAS: Learning to Optimally Memorize the Context at Test Time
Behrouz, A., Li, Z., Kacham, P., Daliri, M., Deng, Y., Zhong, P., Razaviyayn, M., & Mirrokni, V. (2025) · arXiv · doi:10.48550/arxiv.2505.23735
“Computing this pairwise dependencies, however, while accurate, causes quadratic space and time complexity, limiting their applicability in long context understanding, memorization, or modeling (Dalal et al. 2025; Li, Huang, et al. 2024; Liu, Lin, et al. 2024).”
Shared Selective Persistent Memory for Agentic LLM Systems
Pedada, S., Dhavala, A., & Patil, N. (2026) · arXiv · doi:10.48550/arxiv.2607.09493 · Cited by 1
“This selective forgetting is motivated by the “lost in the middle” finding (Liu et al., 2024): injecting irrelevant context into LLM prompts degrades performance.”
InfluentialAccurate and Efficient Long-Term Memory for LLM Agents
Zhao, Z., Guo, X., Lv, L., Wang, M., Li, M., & Li, S. (2026) · arXiv · doi:10.48550/arxiv.2607.16211
“Questions requiring temporal reasoning (e.g., “What did the user say about X before mentioning Y?”) benefit from the graph’s ability to traverse temporal edges directly rather than reconstructing temporal order from retrieved text fragments [Liu et al., 2024].”
Re-Invoke: Tool Invocation Rewriting for Zero-Shot Tool Retrieval
Chen, Y., Yoon, J., Sachan, D. S., Wang, Q., Cohen-Addad, V., Bateni, M., Lee, C.-Y., & Pfister, T. (2024) · arXiv · doi:10.48550/arxiv.2408.01875
“Moreover, LLMs can struggle to effectively process relevant information from lengthy input contexts (Liu et al., 2023).”
SealQA: Raising the Bar for Reasoning in Search-Augmented Language Models
Pham, T., Nguyen, N., Zunjare, P., Chen, W., Tseng, Y.-M., & Vu, T. (2025) · arXiv · doi:10.48550/arxiv.2506.01062
Remember When It Matters: Proactive Memory Agent for Long-Horizon Agents
Wu, Y., Zhang, L., Zhou, Y., Wang, M., Peng, B., Li, S., Fan, X., & Zhao, Z. (2026) · arXiv · doi:10.48550/arxiv.2607.08716 · Cited by 20
VADAOrchestra: Neurosymbolic Orchestration of Adaptive Reasoning Workflows
Baldazzi, T., Bellomarini, L., Coletta, A., Iezzi, M., Maple, C., Pesare, A., & Sallinger, E. (2026) · arXiv · doi:10.48550/arxiv.2606.22485
FOCAL: Filtered On-device Continuous Activity Logging for Efficient Personal Desktop Summarization
Yin, H., Wen, Z., Cao, J., Yuan, B., & Yang, R. (2026) · arXiv · doi:10.48550/arxiv.2604.19541
Dynamic Cheatsheet: Test-Time Learning with Adaptive Memory
Suzgun, M., Yuksekgonul, M., Bianchi, F., Jurafsky, D., & Zou, J. (2025) · arXiv · doi:10.48550/arxiv.2504.07952
Token Reduction Is Not Cost Reduction
Weinberger, S., & Hozez, A. (2026) · arXiv · doi:10.48550/arxiv.2607.12161 · Cited by 1
“[7, 23].”
Pollertlam, N., & Kornsuwannawit, W. (2026) · arXiv · doi:10.48550/arxiv.2603.04814
ARTEM: Enhancing Large Language Model Agents with Spatial-Temporal Episodic Memory
Tan, C. H.-M., Subagdja, B., & Tan, A.-H. (2026) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v40i30.39773
Grounding Agent Memory in Contextual Intent
Yang, R., Jiang, Y., Jiang, Y., Kargupta, P., Zhang, Y., & Han, J. (2026) · Findings of the Association for Computational Linguistics: ACL 2026 · doi:10.18653/v1/2026.findings-acl.584 · Cited by 1
Transferable End-to-End Optimization for Indirect Long-Term Memory Poisoning in LLM Agents
Zang, C., Wang, J., Chen, W., Meng, X., Wang, L., Gao, X., Li, Z., & Guo, S. (2026) · arXiv
Chain-of-Memory: Lightweight Memory Construction with Dynamic Evolution for LLM Agents
Xu, X., Xu, B., Xueyun, T., Huang, Z., Chen, R., Yunfan, L., & Shen, H. (2026) · Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2026.acl-long.534
When Context Overwhelms: A Study of Long-Context vs. Retrieval-Based QA Under Noise
Akbar, Q. U. N., & Akbar, Q. M. (2026) · doi:10.33774/coe-2026-tvxc6
MS-Eval: Benchmarking Sub-Agent Delegation and Orchestration in MCP-Intensive Working Scenarios
Ding, B., Wang, Q., Ruan, L., Lu, W., Ge, Q., Fang, R., & Lin, T. (2026) · arXiv · doi:10.48550/arxiv.2610.22869
“More critically, exploratory sub-tasks often retrieve information-intensive chunks that quickly bloat the context window, leading to severe decision distortion in subsequent steps (Liu et al., 2024; Xue et al., 2026).”
Tran, D., & Kiela, D. (2026) · arXiv · doi:10.48550/arxiv.2604.02460 · Cited by 47
“…including context degradation and long-context stress testing (Du et al., 2025; Li et al., 2025c; Lu et al., 2024), failures that are sensitive to their position such as the lost in the middle effect (Liu et al., 2024), and broader reliability decay as input length grows (Hong et al., 2025).”
All-Mem: Agentic Lifelong Memory via Dynamic Topology Evolution
Lv, C., Chang, H., Tao, S., Chen, M., Fan, Z., Zhang, Z., Guo, Y., & Zhou, S. (2026) · arXiv · doi:10.48550/arxiv.2603.19595 · Cited by 11
Does Rank Still Matter? Position Bias When AI Agents Shop on Our Behalf
Wadi, D., & Ma, Y. (2026) · arXiv · doi:10.48550/arxiv.2608.22697
“AI agents’ position bias is concentrated in the middle rather than at the bottom of the page (Ursu, 2018; Liu et al., 2024).”
InfluentialShopping by algorithm: How agentic AI deploys human heuristics as a surrogate consumer
Wadi, D., & Ma, Y. (2026) · arXiv · doi:10.48550/arxiv.2609.28372
Not All Flips Are Conformity: Decomposing Stance Convergence in Multi-Agent LLM Debate
Hao, X., Wu, Z., Qiu, Y.-X., Xiao, C., Xu, R., Zheng, S., & Qin, J. (2026) · arXiv · doi:10.48550/arxiv.2606.00820
Zhang, Y., Liu, F., Shan, Y., Huang, X., Yang, X., Zhu, Y., Cheng, X., Liu, C., Zeng, K., Zhang, T. J., et al. (2026) · arXiv · doi:10.48550/arxiv.2603.01045
“…recent advances have extended context lengths to millions of to-kens (Reid et al., 2024; Liu et al., 2024a), the quadratic computational complexity of attention mechanisms makes centralized processing increasingly resource-intensive and prone to “lost-in-the-middle” phenomena (Liu et al., 2024b).”
Diagnosing the limits of large language models for structured scientific evidence extraction
Tan, Z., D’Souza, J., & Auer, S. (2026) · International Journal on Digital Libraries · doi:10.1007/s00799-026-00449-9
Eliciting In-context Retrieval and Reasoning for Long-context Large Language Models
Qiu, Y., Embar, V. R., Zhang, Y., Jaitly, N., Cohen, S. B., & Han, B. (2025) · Findings of the Association for Computational Linguistics: ACL 2025 · doi:10.18653/v1/2025.findings-acl.165
Baba in Wonderland: Online Self-Supervised Dynamics Discovery for Executable World Models
Seo, S., Han, D., Noh, S., & Kang, H. (2026) · arXiv · doi:10.48550/arxiv.2605.16725
Cartridges: Lightweight and general-purpose long context representations via self-study
Eyuboglu, S., Ehrlich, R., Arora, S., Guha, N., Zinsley, D., Liu, E., Tennien, W., Rudra, A., Zou, J., Mirhoseini, A., et al. (2025) · arXiv · doi:10.48550/arxiv.2506.06266
A Survey of Agent Memory in the Second Half: Towards Self-Evolving and Long-Horizon Agents
Huang, W.-C., Zhang, W., Liang, Y., Bei, Y., Chen, Y., Feng, T., Pan, X., Tan, Z., Wang, Y., Wei, T., et al. (2026) · arXiv · doi:10.48550/arxiv.2602.06052 · Cited by 70
FreshStack: Building Realistic Benchmarks for Evaluating Retrieval on Technical Documents
Thakur, N., Lin, J., Havens, S., Carbin, M., Khattab, O., & Drozdov, A. (2025) · arXiv · doi:10.48550/arxiv.2504.13128
Xu, N., & Ma, X. (2024) · arXiv · doi:10.48550/arxiv.2410.14166
The Effect of State Representation on LLM Agent Behavior in Dynamic Routing Games
Goodyear, L., Guo, R., & Johari, R. (2025) · arXiv · doi:10.48550/arxiv.2506.15624
Ghost in the Context: Policy-Carriage Integrity in LLM Agents
Santos-Grueiro, I. (2026) · arXiv · doi:10.48550/arxiv.2605.12535
Harnessing the Reasoning Economy: A Survey of Efficient Reasoning for Large Language Models
Wang, R., Wang, H., Xue, B., Li, Y., Pang, J., Liu, S., Chen, Y., Qiu, J., Wong, D. F., Chen, G., et al. (2025) · arXiv · doi:10.48550/arxiv.2503.24377
Reducing Token Usage of State-in-Context Agents using Minification
Hrubec, N., & Cito, J. (2026) · arXiv · doi:10.48550/arxiv.2606.01326
“Additionally, studies have shown that LLMs can struggle to fully utilize long contexts, leading to performance degradation [3, 6, 8, 15, 16].”
Prefix Sharing Is a Sorting Problem
He, R. (2026) · arXiv · doi:10.48550/arxiv.2609.13692
Liu, R., Geng, J., Wu, A. J., Sucholutsky, I., Lombrozo, T., & Griffiths, T. L. (2024) · arXiv · doi:10.48550/arxiv.2410.21333 · Cited by 6
Metadata
- Authors
- Liu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., & Liang, P.
- Affiliations
- Stanford University (US)
- University of California, Berkeley (US)
- Samaya AI, UK
- Corresponding author
- Liu, Nelson F.
- Venue
- Transactions of the Association for Computational Linguistics
- Publisher
- MIT Press
- Published
- 2024
- Volume
- 12
- Pages
- 157-173
- ISSN
- 2307-387X (ISSN-L)
- OpenAlex
- W4391876619
- Semantic Scholar
- 1733eb7792f7a43dd21f51f4d1017a1bffd217b5
- Cited by
- 1,612
- References
- 59
- Citations by year
- 2026: 901 · 2025: 516 · 2024: 172
- FWCI
- 233.95
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- Locations
- Transactions of the Association for Computational Linguistics (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- long-context language models, multi-document question answering, position bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-09-27 (prompt extract-v2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.