Lost in the Middle: How Language Models Use Long Contexts
Open accessThe DOI matches its Crossref record; 48 of the 59 references it cites were found in a registry or an index; 15 of the 23 sources its claims cite were checked for support.
Abstract
While recent language models have the ability to take long contexts as input, relatively little is known about how well they use longer context. We analyze the performance of language models on two tasks that require identifying relevant information in their input contexts: multi-document question answering and key-value retrieval. We find that performance can degrade significantly when changing the position of relevant information, indicating that current language models do not robustly make use of information in long input contexts. In particular, we observe that performance is often highest when relevant information occurs at the beginning or end of the input context, and significantly degrades when models must access relevant information in the middle of long contexts, even for explicitly long-context models. Our analysis provides a better understanding of how language models use their input context and provides new evaluation protocols for future long-context language models.
Overview
CoChat Summary
TL;DR
Language models’ performance on tasks requiring access to long input contexts is strongly affected by where relevant information appears, with a U-shaped curve: best at the beginning or end and worst in the middle. Extended-context models are not necessarily better at using context, and reader performance saturates before retriever recall in open-domain QA.
Key findings
- In multi-document QA, performance is highest when the answer-containing document is at the start or end of the context and degrades in the middle; GPT-3.5-Turbo can drop by more than 20%, and in 20- and 30-document settings middle-position performance is lower than its closed-book performance of 56.1% (oracle: 88.3%).
- Extended-context models are not necessarily better at using input context: when the input fits both windows, GPT-3.5-Turbo and GPT-3.5-Turbo(16K) perform nearly identically as a function of relevant-information position.
- In synthetic key-value retrieval, Claude-1.3 and Claude-1.3(100K) do nearly perfectly, while other models struggle especially with 140 or 300 key-value pairs; query-aware contextualization gives GPT-3.5-Turbo(16K) perfect performance at 300 pairs, versus worst-case 45.6% without it.
- In open-domain QA, reader performance saturates long before retriever recall: using 50 instead of 20 retrieved documents only marginally improves performance (∼1.5% for GPT-3.5-Turbo and ∼1% for Claude-1.3).
Methods
- Controlled multi-document QA on NaturalQuestions-Open: 2,655 queries whose annotated long answer is a paragraph; each context has exactly one answer-containing Wikipedia passage plus k−1 Contriever/MS-MARCO distractor passages; the paper varies answer position and context length (10, 20, 30 documents) and measures accuracy.
- Synthetic key-value retrieval: inputs are serialized JSON objects with unique random UUID key-value pairs; models must return the value for a specified key; experiments use 75, 140, and 300 key-value pairs (500 examples each), varying key position and number of pairs, with accuracy as the metric.
- Models include open MPT-30B-Instruct and LongChat-13B(16K), closed GPT-3.5-Turbo, GPT-3.5-Turbo(16K), Claude-1.3, and Claude-1.3(100K), plus additional architecture/instruction experiments with Flan-T5-XXL, Flan-UL2, MPT-30B, Llama-2, and GPT-4; an open-domain QA case study uses a Contriever retriever-reader setup.
Limitations
- Stated: the paper uses greedy decoding and leaves exploration of other decoding methods to future work; the investigations into why models are not robust are described as preliminary; GPT-4 is evaluated only on a subset of 500 multi-document QA examples because full evaluation would cost upwards of $6000.
- Stated: NaturalQuestions-Open has ambiguity, so the authors run an unambiguous subset and also test random distractors and randomized distractor order, finding similar trends.
- Evident: the controlled multi-document QA setting always contains exactly one document that answers the question, unlike open-domain QA; the synthetic key-value task removes natural language semantics; results may not generalize to all tasks, models, or decoding settings.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
7 Valid · 0 Not valid · 5 Not checked · 13 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Language models have become an important and flexible building block in a variety of user-facing language technologies, including conversational interfaces, search and summarization, and collaborative writing (Shuster et al., 2022;Thoppilan et al., 2022;Lee et al., 2022, inter alia).”
§ Lost in the Middle: How Language Models Use Long Contexts > 1 Introduction
What the sources say
- Partly supportedcites Shuster et al., 2022 matched to “BlenderBot 3: a deployed conversational agent that continually learns to responsibly engage” (2022) sourceExperimental — accuracy not yet measured
Section 3.1.1 (p. 4) describes a conversational agent built on a language model that generates internet search queries, supporting conversational interfaces and search as part of dialogue. Section 2 (p. 3) mentions summarization only as related work, and the passages do not establish collaborative writing as an application of this paper’s model. (LLM, conf=0.85)
- Supportedcites Thoppilan et al., 2022 matched to “LaMDA: Language Models for Dialog Applications” (2022) source
The Abstract (p. 1) identifies LaMDA as a family of language models specialized for dialog. In this sentence's list of applications, Thoppilan et al. supports the conversational-interface example; the other cited works need not each cover every example. (LLM, conf=0.92)
- Supportedcites Lee et al., 2022 matched to “CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilities” (2022) source
Lee et al. is cited alongside other works, and its contribution to the sentence is the collaborative-writing example. Section 4.2.2 describes writers receiving and selecting model-generated suggestions in a writing interface; Section 2.2.1 also describes language models as applicable to dialogue and summarization. The paper need not establish every example in the jointly cited list. (LLM, conf=0.89)
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“These observations complement prior work, which found that non-instruction fine-tuned language models are biased towards recent tokens (i.e., the end of the input context; Khandelwal et al., 2018;Press et al., 2021).”
§ Lost in the Middle: How Language Models Use Long Contexts > 4.3 Effect of Instruction Fine-Tuning
What the sources say
- Partly supportedcites Khandelwal et al., 2018 matched to “Sharp nearby, fuzzy far away: How neural language models use context” (2018) sourceExperimental — accuracy not yet measured
The Introduction and Section 5 report that the studied LSTM is much more sensitive to recent word order than to distant word order, supporting a recency effect. The paper studies a standard LSTM on two datasets, however, rather than establishing this tendency across non-instruction-fine-tuned language models generally; it also finds that distant words remain relevant. (LLM, conf=0.82)
- Partly supportedcites Press et al., 2021 matched to “Shortformer: Better language modeling using shorter inputs” (2021) sourceExperimental — accuracy not yet measured
Section 3.1 (page not available in our copy) finds that very short contexts hurt performance, while increasing subsequence length from 1,024 to 3,072 tokens gives negligible improvement. That is consistent with limited benefit from more distant context, but the paper does not establish that the models are biased toward recent tokens within an input context. (LLM, conf=0.78)
- Partly supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“We instantiate this task with data from NaturalQuestions-Open (Lee et al., 2019;Kwiatkowski et al., 2019), which contains historical queries issued to the Google search engine, coupled with human-annotated answers extracted from Wikipedia.”
§ Lost in the Middle: How Language Models Use Long Contexts > 2.1 Experimental Setup
What the sources say
- Supportedcites Lee et al., 2019 matched to “Latent retrieval for weakly supervised open domain question answering” (2019) source
Section 7.1 says the paper creates an open version of Natural Questions and uses it for training and evaluation. The Introduction describes the task as answering questions by retrieving evidence from an open corpus, supporting the cited use of NaturalQuestions-Open for this task. (LLM, conf=0.91)
- Not checked — no full text of the sourcecites Kwiatkowski et al., 2019 matched to “Natural Questions: A benchmark for question answering research” (2019) source
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Following Kandpal et al. (2022) and Mallen et al. (2023), we use accuracy as our primary evaluation metric, judging whether any of the cor- rect answers (as taken from the NaturalQuestions annotations) appear in the predicted output.”
§ Lost in the Middle: How Language Models Use Long Contexts > 2.1 Experimental Setup
What the sources say
- Supportedcites Kandpal et al. (2022) matched to “Large Language Models Struggle to Learn Long-Tail Knowledge” (2022) source
Section 3 (p. 4) says the paper evaluates answers using exact match against a ground-truth answer set, and Section 3.1 (p. 4) reports QA accuracies as its evaluation results. This supports citing Kandpal et al. for using accuracy to evaluate question answering. (LLM, conf=0.90)
- Supportedcites Mallen et al. (2023) matched to “When not to trust language models: Investigating effectiveness of parametric and non-parametric memories” (2023) source
Section 3.1 (‘Focus and Task’) explicitly names accuracy as the metric and says a prediction is correct when it contains an exact match to any gold answer. This supports the evaluation approach attributed to Mallen et al.; the NaturalQuestions annotations describe the citing authors’ answer source. (LLM, conf=0.94)
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“or when language models are augmented with external information (relevant documents from a search engine, database query results, etc.; Petroni et al., 2020;Ram et al., 2023;Shi et al., 2023;Mallen et al., 2023;Schick et al., 2023, inter alia).”
§ Lost in the Middle: How Language Models Use Long Contexts > 1 Introduction
What the sources say
- Partly supportedcites Petroni et al., 2020 matched to “How Context Affects Language Models' Factual Predictions” (2020) sourceExperimental — accuracy not yet measured
The Abstract (p. 1) says the paper integrates information from a retrieval system with a pre-trained language model, and §3.4.2 (p. 5) specifies a retrieved paragraph as context. This supports augmentation with retrieved documents, but our copy does not show language models being augmented with database query results. (LLM, conf=0.84)
- Partly supportedcites Ram et al., 2023 matched to “In-Context Retrieval-Augmented Language Models” (2023) sourceExperimental — accuracy not yet measured
The Introduction describes augmenting language models with relevant documents retrieved from an external knowledge source, and Section 5 describes using a BM25 retriever to find those documents. The passages do not establish database query results as another type of external information used by this paper. (LLM, conf=0.82)
- Partly supportedcites Shi et al., 2023 matched to “REPLUG: Retrieval-augmented black-box language models” (2023) sourceExperimental — accuracy not yet measured
Section 3.1 (page unavailable) describes retrieving relevant documents from a corpus, and the Conclusion says REPLUG uses those documents to augment a language model. The paper supports the document-retrieval example, but our copy does not describe augmenting models with database query results or documents from a search engine specifically. (LLM, conf=0.82)
- Partly supported
20 more claims · 2 with their cited sources checked
Create a free account to see all 25 claims and every source verdict.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In vanilla multi-document or NIAH-based benchmarks (Kamradt, 2023; Liu et al., 2024), models perform consistently well when the needle (or gold document) appears at the very beginning or end of the context window, with minimal impact from context length.”
From NoLiMa: Long-Context Evaluation Beyond Literal Matching (2025) · cites this paper as Liu et al., 2024
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Even within a single long sequence, attention effectiveness degrades with distance due to attention dilution , positional encoding limitations, and token interference, leading to the well-known 'lost-in-the-middle' and context-decay phenomena (Liu et al., 2024; Press et al., 2021a).”
From MAGMA: A Multi-Graph based Agentic Memory Architecture for AI Agents (2026) · cites this paper as Liu et al., 2024
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Current large language models (LLMs) exhibit significant deficiencies in episodic memory tasks including encoding, storing, and retrieving specific information from temporally dependent events over a long period of time.”
From ARTEM: Enhancing Large Language Model Agents with Spatial-Temporal Episodic Memory (2026) · cites this paper as Liu et al. 2024
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Recent approaches to handle memory tasks in LLMs, such as in-context learning, retrieval-augmented generation (RAG), and fine-tuning, may resolve the long-term retention issues, but are still inadequate to handle tasks requiring chronological awareness of the stored information.”
From ARTEM: Enhancing Large Language Model Agents with Spatial-Temporal Episodic Memory (2026) · cites this paper as Liu et al. 2024
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“If not ranked at the top, the key information may be neglected by the generative module, resulting in incomplete or inaccurate responses to the user queries24.”
From Leveraging long context in retrieval augmented language models for medical question answering (2025) · cites this paper as [24]
Experimental — accuracy not yet measured
86 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929
LLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602
Transformer-XL: Attentive language models beyond a fixed-length context
Dai, Z., Yang, Z., Yang, Y., Carbonell, J., Le, Q., & Salakhutdinov, R. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1285 · Cited by 3,169 · cited 2×
“Recent improvements in hardware (e.g., faster GPUs with more memory) and algorithms (Dai et al., 2019; Dao et al., 2022; Poli et al., 2023; Rubin and Berant, 2023, inter alia ) have resulted in language models with larger context windows (e.g., 4096, 32K, and even 100K tokens), but it remains…”
BackgroundLlama 2: Open Foundation and Fine-Tuned Chat Models
Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.09288 · Cited by 2,604
Longformer: The Long-Document Transformer
Beltagy, I., Peters, M. E., & Cohan, A. (2020) · arXiv · doi:10.48550/arxiv.2004.05150 · Cited by 2,281
Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering
Izacard, G., & Grave, E. (2020) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.48550/arxiv.2007.01282 · Cited by 2,281
Natural Questions: A benchmark for question answering research
Kwiatkowski, T., Palomaki, J., Redfield, O., Collins, M., Parikh, A., Alberti, C., Epstein, D., Polosukhin, I., Devlin, J., Lee, K., et al. (2019) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00276 · Cited by 2,069 · cited 1×
“We instantiate this task with data from NaturalQuestions-Open (Lee et al., 2019; Kwiatkowski et al., 2019), which contains historical queries issued to the Google search engine, coupled with human-annotated answers extracted from Wikipedia.”
MethodThe serial position effect of free recall.
Murdock, B. B. (1962) · Journal of Experimental Psychology · doi:10.1037/h0045106 · Cited by 1,717 · cited 1×
“The U-shaped curve we observe in this work has a connection in psychology known as the serial-position effect (Ebbinghaus, 1913; Murdock Jr, 1962), that states that in free-association recall of elements from a list, humans tend to best re-member the first and last elements of the list.”
MethodMemory: A contribution to experimental psychology.
Ebbinghaus, H. (1913) · doi:10.1037/10011-000 · Cited by 1,645 · cited 1×
“The U-shaped curve we observe in this work has a connection in psychology known as the serial-position effect (Ebbinghaus, 1913; Murdock Jr, 1962), that states that in free-association recall of elements from a list, humans tend to best re-member the first and last elements of the list.”
Method
Research works printed but not linked: 1
- Sharan, 2018, Prediction with a short memory, 10.18653/v1/K18-3013 · DOI
Cited by
Items 701–800 of 1,597
Chen, W., Ou, D., Zhu, Y., et al. (2026) · Sustainability · doi:10.3390/su18136727 · Cited by 1
Across Images and Graphs for Question Answering
Wen, Z., Qian, J., Qian, B., Qin, Y., Qin, J., Xuan, Q., & Yuan, Y. (2024) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde60146.2024.00112 · Cited by 2
“To address the challenge of performing question-and-answer tasks while avoiding forgetting effects [57], our next step involves combining a formalized knowledge base, specifically a Knowledge Graph, with a parameterized knowledge base represented by a Large Language Model.”
MethodSong, Z., Kong, X., Bao, X., Zhou, Y., Jiao, J., Liu, S., Zhou, Y., & Qi, H. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.131627 · Cited by 8
“This limits the framework’s effectiveness (Liu et al., 2024).”
Gaze2Prompt: Turning Eye-Tracking Data into Visual Prompts for Multimodal LLMs
Choi, J. Y., Kim, S. G., Jeong, J. H., Rossi, R. A., Kil, J., & Lee, T. Y. (2025) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/3714394.3754401 · Cited by 1
Yan, M., Wang, Y., Jiang, X., Zhou, H. L., & Li, J. (2025) · Frontiers of Computer Science · doi:10.1007/s11704-025-41143-4 · Cited by 5
Verified Telcorag: A Verified Local-Rag for Telecommunication Uses
Salajeghe, Y., Kopaee, A. M., Najjari, S., Ahmadi, I., Dehmolaee, M. A., & Khalaj, B. H. (2025) · Joint European Conference on Networks and Communications & 6G Summit (EuCNC/6G Summit) · doi:10.1109/eucnc/6gsummit63408.2025.11037160 · Cited by 1
Local Explanations and Self-Explanations for Assessing Faithfulness in black-box LLMs
Fragkathoulas, C., & Chlapanis, O. S. (2024) · Hellenic Conference on Artificial Intelligence · doi:10.1145/3688671.3688775 · Cited by 6
Automating the semantic validation of SysML v2models through a multi-agent frameworkPreprint
Olivert-Iserte, J., Cibrián, E., Mendieta, R., & Álvarez-Rodríguez, J. M. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6553831 · Cited by 1
Masri, Y., Ma, E., Wang, Z., Rogers, J., & Yang, C. P. (2026) · Computers · doi:10.3390/computers15070430 · Cited by 4
TRACE: Topical Reasoning with Adaptive Contextual Experts
Ye, J., Xin, Q., Zhang, C., & Qi, H. (2026) · Big Data and Cognitive Computing · doi:10.3390/bdcc10010031 · Cited by 1
Liveness in the Age of Agents: Are We Back to Compile-and-Run Cycles?
Mattis, T., Taeumel, M., Böhme, L., & Hirschfeld, R. (2026) · International Conference on Art, Science, and Engineering of Programming (<Programming>) · doi:10.1145/3801119.3801131 · Cited by 1
Tung, Y. H., Rezvani, S., Asgharzadeh, F., et al. (2025) · Journal of Physics Conference Series · doi:10.1088/1742-6596/3140/16/162007 · Cited by 1
“This can lead to redundant text snippets [10], a lack of global context, and an inability to utilize IFC hierarchies effectively [11].”
Uncertainty-aware extraction of clinical findings from Finnish EHRs using open large language modelsPreprint
Leinonen, J., Knuuttila, J., Pamilo, S., Kurki, S. H., & Koskinen, M. (2026) · medRxiv · doi:10.64898/2026.07.07.26355248 · Cited by 1
EVINCE: Optimizing Adversarial LLM Dialogues via Conditional Statistics and Information Theory
Chang, E. Y. (2025) · ACM eBooks · doi:10.1145/3749421.3749431 · Cited by 3
Amari, A., & Ammar, M. A. B. (2025) · IFIP advances in information and communication technology · doi:10.1007/978-3-031-96231-8_29 · Cited by 1
“However, these models are resource-intensive and may encounter issues like "lost in the middle" [11], particularly in maintaining context across lengthy documents.”
Exploring the application of large language models in coding the experiencing scale (EXP)
Yim, B., Muran, J. C., Ren, Q., & Gorman, B. S. (2026) · Cogent Mental Health · doi:10.1080/28324765.2026.2664163 · Cited by 1
Beyond the Algorithm: A Field Guide to Deploying AI Agents in Clinical PracticePreprint
Gallifant, J., Kellogg, K. C., Butler, M., Centi, A., Chen, S., Doyle, P., Dutta, S., Guo, J., Hadfield, M., Kim, E., et al. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5732262 · Cited by 2
Multi-Hop Arabic LLM Reasoning in Complex QA
Abdelazim, H., Begemy, T., Galal, A., Sedki, H., & Mohamed, A. (2024) · Procedia Computer Science · doi:10.1016/j.procs.2024.10.179 · Cited by 1
DSAɪVE - Dynamic Systematic AI Vector Engine - Literature Review in Multidisciplinary Context
Oliveira, J. P., Mendes, A., & Ferreira, M. C. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-10728-2_11 · Cited by 1
UKRAG: A Unified Knowledge Graph to Enhance Retrieval Augmented Generation Performance
Alkouz, A., Al-Saleh, M. I., Alarabeyyat, A., & Bouchahma, M. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-82931-4_1 · Cited by 2
ALAS: A Stateful Multi-LLM Agent Framework for Disruption-Aware Planning
Chang, E. Y. (2025) · ACM eBooks · doi:10.1145/3749421.3749436 · Cited by 2
“ALAS excels in dynamic environments where reactive decision-making, incre mental adjustments, and local consistency deliver superior results compared to global recomputation approaches.”
Chen, J. K., Xie, Z., Wang, R., Jin, M., & Lyu, Y. (2026) · Journal of King Saud University - Computer and Information Sciences · doi:10.1007/s44443-026-00999-7 · Cited by 2
Vibe Design: Human-in-the-loop Agentic Framework for UI Design with Large Language Models
Hwang, J., & Kang, J. (2026) · doi:10.2139/ssrn.6297816 · Cited by 1
AI-driven risk estimation: a GPT-based approach to news monitoring for manufacturing resilience
Jacob, A., Achour, A. B., & Teicher, U. (2026) · The International Journal of Advanced Manufacturing Technology · doi:10.1007/s00170-025-17260-6 · Cited by 1
SIS: Leveraging Semantically-Informed Similarity of Text Embeddings for Enhanced Ontology Alignment
Macilenti, G., Stellato, A., & Fiorelli, M. (2025) · Procedia Computer Science · doi:10.1016/j.procs.2025.09.169 · Cited by 2
Ngo-Ho, A.-K., Vo, K.-D., & Ho, A. K. N. (2025) · Signals and communication technology · doi:10.1007/978-3-031-94973-9_19 · Cited by 2
Kou, J., Lu, X., Yin, A., Zhang, W., Fang, X., Li, Y., Pan, T., Zhang, L., Cheng, G., Yuan, H., et al. (2024) · International Conference on Universal Village (UV) · doi:10.1109/uv63228.2024.11189154 · Cited by 7
HELMET: How to Evaluate Long-Context Language Models Effectively and Thoroughly
Yen, H., Gao, T., Hou, M., Ding, K., Fleischer, D., Izsak, P., Wasserblat, M., & Chen, D. (2024) · arXiv · Cited by 2
“Synthetic tasks are often used to evaluate LCLMs since they can be proce-durally generated, thereby enabling arbitrarily long input lengths and controlled “needle” placement (Tay et al., 2021; Liu et al., 2023).”
BackgroundMethodChunkAttention: Efficient Self-Attention with Prefix-Aware KV Cache and Two-Phase Partition
Ye, L., Tao, Z., Huang, Y., & Li, Y. (2024) · arXiv · doi:10.48550/arxiv.2402.15220 · Cited by 34
“Liu et al. (2023) reveals that language model performance degrades significantly when changing the position of relevant information, indicating that models struggle to access and use information in long input contexts robustly.”
BackgroundSWE-Pruner: Self-Adaptive Context Pruning for Coding Agents
Wang, Y., Shi, Y., Yang, M., Zhang, R., He, S., Lian, H., Chen, Y., Ye, S., Cai, K., & Gu, X. (2026) · arXiv · doi:10.48550/arxiv.2601.16746 · Cited by 2
“…models have emerged (Achiam et al., 2023; Team et al., 2024), blindly ingesting large volumes of code incurs prohibitive inference costs (Jiang et al., 2023) and introduces severe noise, leading to attention dilution and hallucinations (Liu et al., 2023a; Li et al., 2023a; Wang et al., 2025b).”
LongCodeZip: Compress Long Context for Code Language Models
Shi, Y., Qian, Y., Zhang, H., Shen, B., & Gu, X. (2025) · arXiv · Cited by 7
“Despite increasingly longer context windows, LLMs exhibit significant limitations in long context scenarios, especially when relevant information appears in the middle of a prompt [19], [20] or when code completion requires cross-function or structural dependencies [21], [61].”
Time, Identity and Consciousness in Language Model Agents
Perrier, E., & Bennett, M. T. (2026) · Proceedings of the AAAI Symposium Series · doi:10.1609/aaaiss.v8i1.42561 · Cited by 6
“Empirically, long context performance is uneven and position dependent (Liu et al. 2024).”
Shen, Y., Li, K., Zhou, W., & Hu, S. (2026) · Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2026.acl-long.370 · Cited by 2
Externalization in LLM Agents: A Unified Review of Memory, Skills, Protocols and Harness Engineering
Zhou, C., Chai, H., Chen, W., Guo, Z., Shan, R., Song, Y., Xu, T., Yang, Y., Yu, A., Zhang, W., et al. (2026) · arXiv · doi:10.48550/arxiv.2604.08224 · Cited by 1
TiMem: Temporal-Hierarchical Memory Consolidation for Long-Horizon Conversational Agents
Li, K., Yu, X., Ni, Z., Zeng, Y., Xu, Y., Zhang, Z., Li, X., Sang, J., Duan, X., Wang, X., et al. (2026) · arXiv · doi:10.48550/arxiv.2601.02845 · Cited by 1
CodexGraph: Bridging Large Language Models and Code Repositories via Code Graph Databases
Liu, X., Lan, B., Hu, Z., Liu, Y., Zhang, Z., Wang, F., Shieh, M., & Zhou, W. (2024) · arXiv · doi:10.48550/arxiv.2408.03910 · Cited by 1
Mittal, S. (2026) · arXiv · doi:10.48550/arxiv.2609.01660 · Cited by 1
SCBench: A KV Cache-Centric Analysis of Long-Context Methods
Li, Y., Jiang, H., Wu, Q., Luo, X., Ahn, S., Zhang, C., Abdi, A. H., Li, D., Gao, J., Yang, Y., et al. (2024) · arXiv · doi:10.48550/arxiv.2412.10319 · Cited by 1
ThreadWeaver: Plan-and-Restate Dialog over Ultra-Long Histories
Praneethreddy, B., & Baddipadiga, S. R. P. R. (2026) · International Conference on Emerging Smart Computing and Informatics (ESCI) · doi:10.1109/esci68015.2026.11493358
“…lies in the inherent limitations of transformer-based architectures when processing ultra-long contexts, particularly the well-documented ”lost-in-the-middle” phenomenon where critical information positioned in the middle of lengthy inputs receives insufficient attention during generation [1].”
Gong, Y., Zhang, J., Lin, Y.-F. F., & Wang, H. (2025) · International Conference on Computer Supported Cooperative Work in Design (CSCWD) · doi:10.1109/cscwd64889.2025.11033332
WeSEAL: We ll-calibrated S earch for E liminating A ttention-sink L eakage
Wang, J., Wang, C., Li, A., Hu, H., Fang, Y., Wu, H., Xu, J., & Liao, D. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3808406
“It causes the first document to accumulate disproportionate weight regardless of actual relevance [10], severely distorting ranking results as shown in Figure 1(a).”
Ponduru, P. S., Nandanavanam, P. P. V., & Ponduru, S. K. K. (2024) · International Journal of Scientific Research in Science Engineering and Technology · doi:10.32628/ijsrset2613385
LLM-Based AI Agent for Virtual Network Function Deployment
Nam, S., Tu, N. V., & Hong, J. W. (2026) · Journal of Network and Systems Management · doi:10.1007/s10922-026-10078-x · Cited by 2
Kalinich, M., Luccarelli, J., Maria, J. P. S., Flathers, M., Nguyen, A., Song, S. H., Makhoul, K. G., Criado, M. J. R., Ginapp, C. M., Hill, B., et al. (2026) · medRxiv · doi:10.64898/2026.07.10.26357132
Zhao, Z., Zhang, S., Jin, X., et al. (2026) · Applied Energy · doi:10.1016/j.apenergy.2026.128819
Dynamic Asymmetric Attention for Enhanced Reasoning and Interpretability in LLMs
Wen, F., Lu, X., Yu, H., Lu, C., Li, H., & Shi, X. (2025) · Symmetry · doi:10.3390/sym17081303 · Cited by 1
Dhakal, A., Neupane, S. S., Dhakal, A., & Paudel, M. (2026) · International Journal of Scientific Research in Science Engineering and Technology · doi:10.32628/ijsrset261313
Information Diffusion and Preferential Attachment in a Network of Large Language Models
Jain, A., Krishnamurthy, V., & Zhang, Y. (2025) · Proceedings of the IEEE Conference on Decision & Control, including the Symposium on Adaptive Processes · doi:10.1109/cdc57313.2025.11312386
“There are a few very long context window LLMs, but testing them exhaustively on their reliability is a challenging task [11].”
Investigating the Impact of API Design Techniques for Tools in Agentic Applications
Bernardino, K. D., Queiroz, P. G. G., Guerra, E., Gonçalves, E., & Praxedes, L. B. F. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-39143-8_21
Ding, Y., & Inoue, T. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-10156-3_12 · Cited by 1
Zhang, F., Fan, Z., Xia, X., et al. (2026) · doi:10.1109/ccssta69471.2026.11635600
“We also design source-side chunking, target-side batching, and cross-chunk anchor aggregation mechanisms to alleviate the lost-in-the-middle problem of large models in long-context scenarios [4].”
Cai, R., Holmes, G., & Bowen, J. (2026) · doi:10.2196/preprints.106989
Zhang, Y., Li, Z., Kong, Z., & Xie, H. (2026) · Research Square · doi:10.21203/rs.3.rs-10925119/v1
Towards a Relevance Posterior in Neural Information Access
Parry, A., Lionis, E. G., Ganguly, D., & MacAvaney, S. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3808541
“In retrieval-augmented generation systems, for example, ranking may primarily serve to surface plausible contexts rather than to fully resolve relevance [40, 66].”
Oliveira, V. T., Lima, M. R., de Oliveira, S. S. T., & Dias, E. S. (2025) · Congresso Latino-Americano de Software Livre e Tecnologias Abertas (Latinoware) · doi:10.5753/latinoware.2025.16269 · Cited by 1
“Still, traditional RAG fails to capture structural relationships between documents (e.g., citations and hierarchies), suffers from the “lost in the middle” effect in long contexts [6], and retrieves only textual subsets, lacking a global view of the normative structure.”
Per-Attribute Privacy in Large Language Models Using Matrix-Variate Gaussian Mechanism
Monir, I. A., & Ghinita, G. (2025) · Conference on Privacy, Security and Trust · doi:10.1109/pst65910.2025.11268877
“Existing approaches for text data [15] consider that the central region of the input carries more sensitive information than the edges, thus necessitating stronger privacy protection.”
Review of Graph Retrieval-Augmented Generation Research
周, 俭. (2025) · Artificial Intelligence and Robotics Research · doi:10.12677/airr.2025.142040
Synth: Code as the Control Plane for Long-Horizon Agents
Gandhi, S., Jain, S., Agarwal, G., Singh, T., & Prajwala, T. R. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-35078-7_18
Uncertainty-aware multi-agent system for automated and real-time geotechnical analysis in tunneling
Qian, Z., & Shi, C. (2026) · Computer-Aided Civil and Infrastructure Engineering · doi:10.1016/j.cacaie.2026.100079 · Cited by 3
Perera, K. J. P. G., Nanayakkara, S., & Huff, E. W. (2026) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3816046.3816225
“Parallel work on LLM failure modes examines hallucination [14, 15], refusal behavior [3], and semantic drift [22], typically at the model or prompt level.”
A large language model-based question-answering system for crack information
Zhang, L., Lu, X., Gong, F., & Sun, L. (2026) · Advances in Structural Engineering · doi:10.1177/13694332261475266
Rethinking Model Selection for Zero-Shot Biomedical STS: Architecture, Scale, and Similarity Metrics
Malhi, M. E., Barbori, S. E., Bahmadi, A. E., & Kerzazi, N. (2025) · IEEE Congress on Information Science and Technology (CiSt) · doi:10.1109/cist65886.2025.11224208
“The evolution of embedding models and semantic similarity techniques has significantly advanced the field of LLMs and Natural Language Processing (NLP) [8], [9] in particular, enabling machines to understand and compare textual content [10], [11].”
Guljajeva, V., & Solà, M. C. (2025) · doi:10.1145/3773699.3776641
Yeerpan, T., Zhu, Y., Hu, Y., Lu, S., & Wang, Z. (2026) · Applied Sciences · doi:10.3390/app16052546
Wang, M., Li, X., Zeng, Q., Liu, X., Yang, M., & Jia, Z. (2025) · Information · doi:10.3390/info16100900
GPT-C: Generative PrompT Compression
Liu, L., Wang, R., Jing, L., Lv, F., & Zhu, Z. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10888801 · Cited by 2
“4, we conduct ablation studies on the performance of multi-document QA task and the adherence to length instructions using the Natural Questions dataset [33].”
Zhang, J., Guo, P., Jiang, G., Zhou, M., Wei, G., & Ni, T. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.05.16.720484
Using Mutation-Analysis to Examine an LLM's Ability to Summarize Code
Khatib, L., Pu, M., Vasilescu, B., & Nagappan, M. (2026) · ACM International Conference on AI-Powered Software (AIware) · doi:10.1145/3805760.3814892 · Cited by 2
Palicki, S.-K., Bravo, I., & Claessen, C. (2026) · Social Science Computer Review · doi:10.1177/08944393261473230
Do Agents Think Deeper? A Mechanistic Investigation of Layer-Wise Dynamics in Sequential Planning
Cui, Z., & Luo, X. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3403-5_12
Can AI Match Professional Analysts? Evidence from a Multi-Agent SystemPreprint
Francis, A., & Zhang, C. X. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5584131
An experiment on an automated literature survey of data-driven speech enhancement methods
dos Santos, A., Pereira, J., Nogueira, R. F., Masiero, B. S., Sander-Tavallaey, S., & Zea, E. (2024) · Acta Acustica · doi:10.1051/aacus/2023067 · Cited by 1
“However, it has been seen that generated responses are often partially (sometimes entirely) fake [Alkaissi and McFarlane, 2023], and the answering accuracy can deteriorate when the answer to the query lies in the middle of the context [Liu et al., 2023].”
BackgroundSequential and parallel prompt engineering on research article topic mining
Lou, W., Tang, W., & Gao, Y. (2026) · Information Processing & Management · doi:10.1016/j.ipm.2026.105110
Медианные значения критериев качества генерации обзоров при сравнении с альтернативными подходами
Кузнецов, И. (2026) · Cifra LLC Journals · doi:10.60797/irj.2026.171.79.4
Query-Aware Adaptive Chunking for RAG in Mental Health and Well-Being Policy Question Answering
Kondapalli, P., Saimon, M. Z. R., Polok, S. D., Chakraborty, G. S., Abrar, M. A., & Batra, S. (2026) · International Conference on Innovative Practices in Technology and Management (ICIPTM) · doi:10.1109/iciptm69057.2026.11465692 · Cited by 1
“While studies about the use of long context modeling have shown that often language models are ineffective at using long inputs, especially when the information of interest is scattered or far away, the research has indicated that in other cases, it is the best method to use in query processing [4].”
Smart-PCLib: A LLM-based Multi-Agent Framework for Automated PCB Component Library Generation
Di, Z., Tu, J., He, Z., et al. (2026) · doi:10.23919/date69613.2026.11539310
“Simply feeding an entire document to a general MLLM can lead to high computational costs and a loss of focus on critical details [8].”
Large language models in the patent lifecycle: A systematic review
Jiang, P., Arena, A. P., Sorce, S., et al. (2026) · World Patent Information · doi:10.1016/j.wpi.2026.102506
MedGuard-KG: Dual-Stream Knowledge Graph Verification for Safe Clinical Reasoning in Medical LLMs
Yitao, W., Kexin, M., Jiawen, D., & Fuji, R. (2025) · International Computer Conference on Wavelet Active Media Technology and Information Processing (ICCWAMTIP) · doi:10.1109/iccwamtip68645.2025.11352596
“Early RAG variants are sensitive to retrieval noise and “lost in the middle” effects [16].”
Diagnosing and Mitigating Mid-Sequence Degradation in Recommender Systems
Guo, L., Bisht, N., Wu, S., Huo, H., Wang, X., & Xu, G. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3809738
AI Based Meeting Summarizer using Large Language Models
Kavan Anselm, A., Leenus Bosco, S., & Manikandan, G. (2026) · International Conference on Visual Analytics and Data Visualization (ICVADV) · doi:10.1109/icvadv67766.2026.11469774
“The results show that this approach effectively solves the long context issue [13] of the meeting transcripts while still maintains coherence and prevents information loss through iterative, recursive summarization providing improved performance especially context based content generation while…”
Yin, Y., Wang, S., Koopman, B., & Zuccon, G. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3808553 · Cited by 3
“As a result, an injected prompt placed at the back is more likely to influence the final output than one placed at the front [14].”
ParaMind: Parallelized Adaptive Memory Integration for Long-Context Large Language Models
Basrur, S. (2025) · International Conference on Intelligent Systems and Embedded Design (ISED) · doi:10.1109/ised67359.2025.11405146
“Lifelong LLMs [27] and context windows with learned compression [28] attempt to preserve salient information across sessions.”
Toward Trustworthy Chatbots: A Protocol for Red Teaming for Health Related ConversationsPreprint
Hussain, S.-A., Jackson, D. I., Lewis, A., Fosler-Lussier, E., & Sezgin, E. (2025) · medRxiv · doi:10.64898/2025.12.15.25342297 · Cited by 2
“Finally, our RAG pipeline, which selectively retrieves a limited number of targeted resources per turn, avoids issues seen in very long contexts where models can struggle to locate relevant information [38].”
SagaLLM: Persistent Context Management, Constraint Validation, and Transaction Guarantees
Chang, E. Y. (2025) · ACM eBooks · doi:10.1145/3749421.3749435
“Furthermore, context loss in long conversations [Hsieh et al. 2024, Liu et al. 2024a, Xiao et al. 2024, Modarressi et al. 2025] can cause LLMs to forget earlier steps, leading to contradictory decisions.”
Reasoning About Humor: It’s No Laughing Matter
Sung, I., & Hu, R. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-20897-2_13
Progressive knowledge-enhanced instruction generation for tactical UAV swarms
Tan, C., Li, B., Wei, Z., Ma, G., Zhu, W., & Zhang, B. (2026) · Defence Technology · doi:10.1016/j.dt.2026.06.004
Event Level Incremental State Modeling for Long Context Meeting Understanding
Huang, T.-C., Yang, W.-C., Chang, Y. J., Tao, H.-S., & Chang, C. (2026) · IEEE International Conference on Consumer Electronics - Taiwan (ICCE-Taiwan) · doi:10.1109/icce-taiwan71481.2026.11652797
“Even with extended context windows, language models exhibit context drift and positional bias under long context inputs [4], and semantic retrieval alone cannot ensure alignment with the evolving decision framework.”
CAFE(S): Your Agent Is Only As Good As Its Context
Houck, B., Kanat-Alexander, M., Kalliamvakou, E., Storey, M.-A., & Forsgren, N. (2026) · Queue · doi:10.1145/3847288 · Cited by 1
Медианные значения критериев качества генерации обзоров для рассматриваемых моделей и подходов
Кузнецов, И. (2026) · Cifra LLC Journals · doi:10.60797/irj.2026.171.79.3
: Large language model enhanced hierarchical script extraction from multiple documents
Liao, Z., Liao, J., Hu, S., Huang, P., & Zhao, X. H. (2026) · Knowledge-Based Systems · doi:10.1016/j.knosys.2026.115608
Mathevet, E., & Cailleux, L. (2026) · Policy Studies Journal · doi:10.1111/psj.70161
AI-Powered Patent Classification: A Hybrid Approach Merging Algorithms and LLMs
Giuntelli, G., Spreafico, C., Precorvi, A., et al. (2025) · IFIP advances in information and communication technology · doi:10.1007/978-3-032-08851-2_10
A Novel Dual-Loop Causality-Traceable Retrieval Framework for Long-Horizon Conversational Agents
Chan, D. Y., Cheng, C.-Y., Wang, J.-F., & Tseng, S.-P. (2026) · Electronics · doi:10.3390/electronics15112373 · Cited by 1
An, D., Wang, S., Liu, B., & Zhu, L. (2026) · Mathematics · doi:10.3390/math14183297
Aphorisms for Collaborative Intelligence
Chang, E. Y. (2025) · ACM eBooks · doi:10.1145/3749421.3749441
“Effects such as effective long-context loss [Liu et al. 2024], attention narrowing and distraction [Zhu et al. 2025], and context override through state transitions [Yen et al. 2024] can cause the model to diverge from earlier logic or contradict prior statements without awareness.”
ConCise: Training-Free Conclusion-Chain State Compression for Cost-Efficient Multi-Step RAG Services
Yan, K., Tang, Z., Wang, T., & Weijia Jia, B. (2026) · IEEE International Conference on Web Services (ICWS) · doi:10.1109/icws72778.2026.00092
“Under the full-context paradigm, all retrieved documents and reasoning traces are carried into every subsequent round, causing cumulative input tokens to grow as roughly O ( N 2 ) while noise density keeps increasing [6](Fig.”
Kong, D., Funabiki, N., Kyaw, H. H. S., Kotama, I. N. D., & Zhu, Z. (2026) · Future Internet · doi:10.3390/fi18080442
“A prior work has shown that language models may not use long contexts uniformly and can miss or underuse relevant information depending on its position in the context [45].”
Otimização de Inferência em LLMs na CPU: Análise do Cenário Atual
Cattai, P., Baldassin, A., & Dantas, A. B. D. O. (2025) · doi:10.5753/eradsp.2025.9731
Load Balancing for LLM Traffic via Flow Block
Li, X., Zhang, Y., Lyu, S., et al. (2025) · Conference on Local Computer Networks · doi:10.1109/lcn65610.2025.11146353
Metadata
- Authors
- Liu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., & Liang, P.
- Affiliations
- Stanford University (US)
- University of California, Berkeley (US)
- Samaya AI, UK
- Corresponding author
- Liu, Nelson F.
- Venue
- Transactions of the Association for Computational Linguistics
- Publisher
- MIT Press
- Published
- 2024
- Volume
- 12
- Pages
- 157-173
- ISSN
- 2307-387X (ISSN-L)
- OpenAlex
- W4391876619
- Semantic Scholar
- 1733eb7792f7a43dd21f51f4d1017a1bffd217b5
- Cited by
- 1,612
- References
- 59
- Citations by year
- 2026: 901 · 2025: 516 · 2024: 172
- FWCI
- 233.95
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- Locations
- Transactions of the Association for Computational Linguistics (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- long-context language models, multi-document question answering, position bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-09-27 (prompt extract-v2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.