Lost in the Middle: How Language Models Use Long Contexts
Open accessThe DOI matches its Crossref record; 48 of the 59 references it cites were found in a registry or an index; 15 of the 23 sources its claims cite were checked for support.
Abstract
While recent language models have the ability to take long contexts as input, relatively little is known about how well they use longer context. We analyze the performance of language models on two tasks that require identifying relevant information in their input contexts: multi-document question answering and key-value retrieval. We find that performance can degrade significantly when changing the position of relevant information, indicating that current language models do not robustly make use of information in long input contexts. In particular, we observe that performance is often highest when relevant information occurs at the beginning or end of the input context, and significantly degrades when models must access relevant information in the middle of long contexts, even for explicitly long-context models. Our analysis provides a better understanding of how language models use their input context and provides new evaluation protocols for future long-context language models.
Overview
CoChat Summary
TL;DR
Language models’ performance on tasks requiring access to long input contexts is strongly affected by where relevant information appears, with a U-shaped curve: best at the beginning or end and worst in the middle. Extended-context models are not necessarily better at using context, and reader performance saturates before retriever recall in open-domain QA.
Key findings
- In multi-document QA, performance is highest when the answer-containing document is at the start or end of the context and degrades in the middle; GPT-3.5-Turbo can drop by more than 20%, and in 20- and 30-document settings middle-position performance is lower than its closed-book performance of 56.1% (oracle: 88.3%).
- Extended-context models are not necessarily better at using input context: when the input fits both windows, GPT-3.5-Turbo and GPT-3.5-Turbo(16K) perform nearly identically as a function of relevant-information position.
- In synthetic key-value retrieval, Claude-1.3 and Claude-1.3(100K) do nearly perfectly, while other models struggle especially with 140 or 300 key-value pairs; query-aware contextualization gives GPT-3.5-Turbo(16K) perfect performance at 300 pairs, versus worst-case 45.6% without it.
- In open-domain QA, reader performance saturates long before retriever recall: using 50 instead of 20 retrieved documents only marginally improves performance (∼1.5% for GPT-3.5-Turbo and ∼1% for Claude-1.3).
Methods
- Controlled multi-document QA on NaturalQuestions-Open: 2,655 queries whose annotated long answer is a paragraph; each context has exactly one answer-containing Wikipedia passage plus k−1 Contriever/MS-MARCO distractor passages; the paper varies answer position and context length (10, 20, 30 documents) and measures accuracy.
- Synthetic key-value retrieval: inputs are serialized JSON objects with unique random UUID key-value pairs; models must return the value for a specified key; experiments use 75, 140, and 300 key-value pairs (500 examples each), varying key position and number of pairs, with accuracy as the metric.
- Models include open MPT-30B-Instruct and LongChat-13B(16K), closed GPT-3.5-Turbo, GPT-3.5-Turbo(16K), Claude-1.3, and Claude-1.3(100K), plus additional architecture/instruction experiments with Flan-T5-XXL, Flan-UL2, MPT-30B, Llama-2, and GPT-4; an open-domain QA case study uses a Contriever retriever-reader setup.
Limitations
- Stated: the paper uses greedy decoding and leaves exploration of other decoding methods to future work; the investigations into why models are not robust are described as preliminary; GPT-4 is evaluated only on a subset of 500 multi-document QA examples because full evaluation would cost upwards of $6000.
- Stated: NaturalQuestions-Open has ambiguity, so the authors run an unambiguous subset and also test random distractors and randomized distractor order, finding similar trends.
- Evident: the controlled multi-document QA setting always contains exactly one document that answers the question, unlike open-domain QA; the synthetic key-value task removes natural language semantics; results may not generalize to all tasks, models, or decoding settings.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
7 Valid · 0 Not valid · 5 Not checked · 13 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Language models have become an important and flexible building block in a variety of user-facing language technologies, including conversational interfaces, search and summarization, and collaborative writing (Shuster et al., 2022;Thoppilan et al., 2022;Lee et al., 2022, inter alia).”
§ Lost in the Middle: How Language Models Use Long Contexts > 1 Introduction
What the sources say
- Partly supportedcites Shuster et al., 2022 sourceExperimental — accuracy not yet measured
Section 3.1.1 (p. 4) describes a conversational agent built on a language model that generates internet search queries, supporting conversational interfaces and search as part of dialogue. Section 2 (p. 3) mentions summarization only as related work, and the passages do not establish collaborative writing as an application of this paper’s model. (LLM, conf=0.85)
- Supportedcites Thoppilan et al., 2022 source
The Abstract (p. 1) identifies LaMDA as a family of language models specialized for dialog. In this sentence's list of applications, Thoppilan et al. supports the conversational-interface example; the other cited works need not each cover every example. (LLM, conf=0.92)
- Supportedcites Lee et al., 2022 source
Lee et al. is cited alongside other works, and its contribution to the sentence is the collaborative-writing example. Section 4.2.2 describes writers receiving and selecting model-generated suggestions in a writing interface; Section 2.2.1 also describes language models as applicable to dialogue and summarization. The paper need not establish every example in the jointly cited list. (LLM, conf=0.89)
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“These observations complement prior work, which found that non-instruction fine-tuned language models are biased towards recent tokens (i.e., the end of the input context; Khandelwal et al., 2018;Press et al., 2021).”
§ Lost in the Middle: How Language Models Use Long Contexts > 4.3 Effect of Instruction Fine-Tuning
What the sources say
- Partly supportedcites Khandelwal et al., 2018 sourceExperimental — accuracy not yet measured
The Introduction and Section 5 report that the studied LSTM is much more sensitive to recent word order than to distant word order, supporting a recency effect. The paper studies a standard LSTM on two datasets, however, rather than establishing this tendency across non-instruction-fine-tuned language models generally; it also finds that distant words remain relevant. (LLM, conf=0.82)
- Partly supportedcites Press et al., 2021 sourceExperimental — accuracy not yet measured
Section 3.1 (page not available in our copy) finds that very short contexts hurt performance, while increasing subsequence length from 1,024 to 3,072 tokens gives negligible improvement. That is consistent with limited benefit from more distant context, but the paper does not establish that the models are biased toward recent tokens within an input context. (LLM, conf=0.78)
- Partly supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“We instantiate this task with data from NaturalQuestions-Open (Lee et al., 2019;Kwiatkowski et al., 2019), which contains historical queries issued to the Google search engine, coupled with human-annotated answers extracted from Wikipedia.”
§ Lost in the Middle: How Language Models Use Long Contexts > 2.1 Experimental Setup
What the sources say
- Supportedcites Lee et al., 2019 source
Section 7.1 says the paper creates an open version of Natural Questions and uses it for training and evaluation. The Introduction describes the task as answering questions by retrieving evidence from an open corpus, supporting the cited use of NaturalQuestions-Open for this task. (LLM, conf=0.91)
- Not checked — no full text of the sourcecites Kwiatkowski et al., 2019 source
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Following Kandpal et al. (2022) and Mallen et al. (2023), we use accuracy as our primary evaluation metric, judging whether any of the cor- rect answers (as taken from the NaturalQuestions annotations) appear in the predicted output.”
§ Lost in the Middle: How Language Models Use Long Contexts > 2.1 Experimental Setup
What the sources say
- Supportedcites Kandpal et al. (2022) source
Section 3 (p. 4) says the paper evaluates answers using exact match against a ground-truth answer set, and Section 3.1 (p. 4) reports QA accuracies as its evaluation results. This supports citing Kandpal et al. for using accuracy to evaluate question answering. (LLM, conf=0.90)
- Supportedcites Mallen et al. (2023) source
Section 3.1 (‘Focus and Task’) explicitly names accuracy as the metric and says a prediction is correct when it contains an exact match to any gold answer. This supports the evaluation approach attributed to Mallen et al.; the NaturalQuestions annotations describe the citing authors’ answer source. (LLM, conf=0.94)
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“or when language models are augmented with external information (relevant documents from a search engine, database query results, etc.; Petroni et al., 2020;Ram et al., 2023;Shi et al., 2023;Mallen et al., 2023;Schick et al., 2023, inter alia).”
§ Lost in the Middle: How Language Models Use Long Contexts > 1 Introduction
What the sources say
- Partly supportedcites Petroni et al., 2020 sourceExperimental — accuracy not yet measured
The Abstract (p. 1) says the paper integrates information from a retrieval system with a pre-trained language model, and §3.4.2 (p. 5) specifies a retrieved paragraph as context. This supports augmentation with retrieved documents, but our copy does not show language models being augmented with database query results. (LLM, conf=0.84)
- Partly supportedcites Ram et al., 2023 sourceExperimental — accuracy not yet measured
The Introduction describes augmenting language models with relevant documents retrieved from an external knowledge source, and Section 5 describes using a BM25 retriever to find those documents. The passages do not establish database query results as another type of external information used by this paper. (LLM, conf=0.82)
- Partly supportedcites Shi et al., 2023 sourceExperimental — accuracy not yet measured
Section 3.1 (page unavailable) describes retrieving relevant documents from a corpus, and the Conclusion says REPLUG uses those documents to augment a language model. The paper supports the document-retrieval example, but our copy does not describe augmenting models with database query results or documents from a search engine specifically. (LLM, conf=0.82)
- Partly supported
20 more claims · 2 with their cited sources checked
Create a free account to see all 25 claims and every source verdict.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In vanilla multi-document or NIAH-based benchmarks (Kamradt, 2023; Liu et al., 2024), models perform consistently well when the needle (or gold document) appears at the very beginning or end of the context window, with minimal impact from context length.”
From NoLiMa: Long-Context Evaluation Beyond Literal Matching (2025) · cites this paper as Liu et al., 2024
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Even within a single long sequence, attention effectiveness degrades with distance due to attention dilution , positional encoding limitations, and token interference, leading to the well-known 'lost-in-the-middle' and context-decay phenomena (Liu et al., 2024; Press et al., 2021a).”
From MAGMA: A Multi-Graph based Agentic Memory Architecture for AI Agents (2026) · cites this paper as Liu et al., 2024
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Current large language models (LLMs) exhibit significant deficiencies in episodic memory tasks including encoding, storing, and retrieving specific information from temporally dependent events over a long period of time.”
From ARTEM: Enhancing Large Language Model Agents with Spatial-Temporal Episodic Memory (2026) · cites this paper as Liu et al. 2024
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Recent approaches to handle memory tasks in LLMs, such as in-context learning, retrieval-augmented generation (RAG), and fine-tuning, may resolve the long-term retention issues, but are still inadequate to handle tasks requiring chronological awareness of the stored information.”
From ARTEM: Enhancing Large Language Model Agents with Spatial-Temporal Episodic Memory (2026) · cites this paper as Liu et al. 2024
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“If not ranked at the top, the key information may be neglected by the generative module, resulting in incomplete or inaccurate responses to the user queries24.”
From Leveraging long context in retrieval augmented language models for medical question answering (2025) · cites this paper as [24]
Experimental — accuracy not yet measured
86 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929
LLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602
Transformer-XL: Attentive language models beyond a fixed-length context
Dai, Z., Yang, Z., Yang, Y., Carbonell, J., Le, Q., & Salakhutdinov, R. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1285 · Cited by 3,169 · cited 2×
“Recent improvements in hardware (e.g., faster GPUs with more memory) and algorithms (Dai et al., 2019; Dao et al., 2022; Poli et al., 2023; Rubin and Berant, 2023, inter alia ) have resulted in language models with larger context windows (e.g., 4096, 32K, and even 100K tokens), but it remains…”
BackgroundLlama 2: Open Foundation and Fine-Tuned Chat Models
Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.09288 · Cited by 2,604
Longformer: The Long-Document Transformer
Beltagy, I., Peters, M. E., & Cohan, A. (2020) · arXiv · doi:10.48550/arxiv.2004.05150 · Cited by 2,281
Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering
Izacard, G., & Grave, E. (2020) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.48550/arxiv.2007.01282 · Cited by 2,263
Natural Questions: A benchmark for question answering research
Kwiatkowski, T., Palomaki, J., Redfield, O., Collins, M., Parikh, A., Alberti, C., Epstein, D., Polosukhin, I., Devlin, J., Lee, K., et al. (2019) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00276 · Cited by 2,069 · cited 1×
“We instantiate this task with data from NaturalQuestions-Open (Lee et al., 2019; Kwiatkowski et al., 2019), which contains historical queries issued to the Google search engine, coupled with human-annotated answers extracted from Wikipedia.”
MethodThe serial position effect of free recall.
Murdock, B. B. (1962) · Journal of Experimental Psychology · doi:10.1037/h0045106 · Cited by 1,717 · cited 1×
“The U-shaped curve we observe in this work has a connection in psychology known as the serial-position effect (Ebbinghaus, 1913; Murdock Jr, 1962), that states that in free-association recall of elements from a list, humans tend to best re-member the first and last elements of the list.”
MethodMemory: A contribution to experimental psychology.
Ebbinghaus, H. (1913) · doi:10.1037/10011-000 · Cited by 1,645 · cited 1×
“The U-shaped curve we observe in this work has a connection in psychology known as the serial-position effect (Ebbinghaus, 1913; Murdock Jr, 1962), that states that in free-association recall of elements from a list, humans tend to best re-member the first and last elements of the list.”
Method
Research works printed but not linked: 1
- Sharan, 2018, Prediction with a short memory, 10.18653/v1/K18-3013 · DOI
Cited by
Items 1,401–1,500 of 1,592
Dynamic Spectral Clustering for Multi-Document Summarization of Social Media Discourse
Wang, G., Wu, W., & Du, D. (2026) · ACM Transactions on Social Computing · doi:10.1145/3821404
“Even long-context transformers may suffer from positional bias and reduced effectiveness in integrating information distributed across documents [29].”
Confirmation, Framing, and Position Biases in LLM Responses
Магдановна, Е. Л., Firsov, A., & Kamps, J. (2026) · Conference on Human Information Interaction and Retrieval · doi:10.1145/3786304.3787879 · Cited by 4
“Liu et al. [9] reveals how results at the beginning or the end of a long context may get priority over those in the middle.”
Expansion-Contraction: A Multi-Agent Graph Traversal Pattern for Compound AI Systems
Taleb, A., Afolabi, Z., Sousa, J., & Seidel, M. (2026) · ACM Conference on AI and Agentic Systems (CAIS) · doi:10.1145/3786335.3813170
NarrativeXL: a Large-scale Dataset for Long-Term Memory Models
Moskvichev, A., & Mai, K.-V. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.1005 · Cited by 6
EDITMEM: Enhancing Multi-Hop Fact Verification Via Editable Memory
Lan, Y., Liu, Z. B., Gu, Y., & Yu, G. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11463827
MACRDR: Enhancing Interest Discovery and Diversity in News Recommendation via Multi-Agent ReflectionPreprint
Zhang, X., & Feng, S. (2026) · Research Square · doi:10.21203/rs.3.rs-10567677/v1
A Review of Retrieval-Augmented Generation Technology
Jiang, P., & Cai, X. (2026) · Symmetry · doi:10.3390/sym18091431
HAAS: Holistic Attention-free Animation from Speech using Mamba
Venkatesan, L. N. N., Singh, H. V., Sinha, V., Madhavan, G. S., Lahiri, S., R., R. B., Josyula, V., Jayagopi, D. B., Tumuluri, R., Revang, M., et al. (2026) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3776574.3831201
How Model Choice and Memory Shape Preference Consistency in Large Language Models
Chung, Y. (2026) · Sociological Methods & Research · doi:10.1177/00491241261492203
Grounding large language models in hydrologic modelling
Li, Y., Hu, S., Zhang, F., Yu, X., Luo, W., Liu, D., Xu, B., & Zhao, J. (2026) · Journal of Hydrology · doi:10.1016/j.jhydrol.2026.136358 · Cited by 1
Vu, T.-T. D., Tran, G.-N., Pham, B.-N., & Trung, H. D. (2026) · International Conference on Multimedia Analysis and Pattern Recognition (MAPR) · doi:10.1109/mapr72750.2026.11685711
“This disparity highlights a critical limitation of RAG systems: even with advanced models, they struggle to retrieve the exact legal clause among similar penalties due to the ”lost-in-the-middle” phenomenon [22].”
Teaching an Agent to Sketch One Part at a Time
Du, X., Xu, R., Yunis, D., Vinker, Y., & Shakhnarovich, G. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37032-7_14 · Cited by 3
NOAH: A System For Analyzing Non-relational Tables With Natural Language
Zheng, L., Lei, C., & Cao, L. (2026) · Proceedings of the ACM on Management of Data · doi:10.1145/3837120
Character History in the Context Design of LLM-Based AI
Matsumoto, T. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-30038-6_26
Enabling large language model with plug-and-play symbolic reasoning components
Galitsky, B. A. (2025) · Elsevier eBooks · doi:10.1016/b978-0-443-30046-2.00014-4
The Stability Trap: Evaluating the Reliability of LLM-Based Instruction Adherence Auditing
Shergadwala, M. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3806536 · Cited by 1
“Future work should evaluate the degradation of adherence stability in long-context scenarios (such as rubrics with exceeding 10 criteria), where “lost-in-the-middle” [34] phenomena might further exacerbate the instability we observed.”
Sakai, T., Fang, H., Kato, M. P., Bi, K., Joho, H., Clarke, C. L. A., Tao, S., Wang, J., Shi, H., Hiramatsu, R., et al. (2026) · ACM SIGIR Forum · doi:10.1145/3841298.3841301
“…all the input tokens it is given, and it can be sensitive to ordering and surface-level presentation cues, exhibiting positional bias (e.g., “Lost in the Middle” [Liu et al., 2024a]) as well as cognitive biases (e.g., threshold priming [Chen et al., 2024] and recency bias [Fang et al., 2025]).”
Moya-Sánchez, D. N. (2026) · AI in neuroscience. · doi:10.1177/2997979x261465520
Zhang, B.-L. (2026) · Journal of Future Artificial Intelligence and Technologies · doi:10.62411/faith.3048-3719-377 · Cited by 1
If-Qsr: Boosting Large Language Models’Reasoning in Intricate NarrativesPreprint
Fan, F., Liu, F., & Chi, L. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5364155
Bridging Knowledge Gaps: Fine-Tuned RAG Frameworks for Biomedical Evidence-Based Question Answering
Wang, X., You, Z.-H., Li, Y., Yan, Y.-H., & Li, Y.-C. (2025) · Lecture notes in computer science · doi:10.1007/978-981-95-0014-7_7
Ellerm, A., Adams, B., & GAHEGAN, M. N. (2025) · Open Conference Proceedings · doi:10.52825/ocp.v8i.3174 · Cited by 1
Tian, Z. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3417-2_11
Liu, W., Shi, F., & Wang, X. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6830575
Sanna, L., Bolpagni, M., Fietta, V., Carli, S. D., Franzin, M., Gavioli, G., Gios, L., Pardini, S., Nicoletti, A. E., Rizzi, S., et al. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-16708-8_38
Kaseteiam, P., & Vanijja, V. (2026) · doi:10.1145/3816713.3818221
“Our framework synthesizes these insights, combining markdown-preserving chunking with a post-retrieval Chain-of-Verification (CoVe) loop to bypass long-context degradation [7, 21].”
Gangapuram, R. R., & Andreopoulos, W. (2026) · IEEE International Conference on Big Data Computing Service and Applications (BigDataService) · doi:10.1109/bigdataservice70481.2026.00015
“Robustness work has also shown that small prompt changes can steer RAG systems toward wrong outputs, and long-context studies show that evidence position can affect model behavior [16], [17].”
Ho, A. K. N., Ho, A. K. N., & Khuong-Duy, V. (2024) · International Conference on Advanced Technologies for Communications · doi:10.1109/atc63255.2024.10908291 · Cited by 2
“Note that even the powerful LLMs such as GPT-4 or Llama-2 still face the issue of losing information in the middle of a long content (Liu et al., 2024).”
Data Foundations of Long-Context Language Models: A Survey
Sun, Z., Sun, Y., Su, Z., Tang, Z., Li, J., Zhou, A., Chen, W., & Zhang, M. (2026) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl.a.775 · Cited by 2
Lai, X., Pan, X., Su, Y., & Zhang, G. (2025) · International Conference on Artificial Intelligence, Robotics, and Communication (ICAIRC) · doi:10.1109/icairc68035.2025.11384878
“Since LLMs are sensitive to the end of the input [28], the question was consistently placed last.”
Near-Zero Cost KV Cache Compression for Large Language Model Inference
Zhang, B., Zhou, D., Huang, Y., Song, S., Feng, H., Jia, J., Zhang, C., & Zhang, Z. (2026) · IEEE International Parallel and Distributed Processing Symposium (IPDPS) · doi:10.1109/ipdps65963.2026.00058 · Cited by 1
“LLM inference tasks such as long-document summarization, retrieval-augmented generation, and multi-turn dialogue rely heavily on large context windows to maintain coherence and factual consistency across extended sequences [3]–[6].”
Park, K., Baik, M., Hwang, Y., Shin, Y., Lee, H., Lee, R., Lee, S. M., Sun, J. Y. H., Lee, A. R., Yoon, S. Y., et al. (2025) · JMIR Medical Informatics · doi:10.2196/73725 · Cited by 2
“We investigated the formulation of a task description document with diverse and extensive information into instructions because instruction construction significantly influences LLM performance [42-44].”
Park, J., & Han, S. (2025) · Journal of Construction Automation and Robotics · doi:10.55785/jcar.4.4.9
Yonathan, M. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5361016
Niu, Z., Du, C., Wang, B., Song, Y., Xiong, J., & Li, L. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3426-4_36
Long-Term Memory Mechanism of Large Language Models for Personalized Medical Inquiry Service
Xu, F. (2026) · ITM Web of Conferences · doi:10.1051/itmconf/20268801023
“However, the traditional LLM still has some shortcomings in memory preservation and retrieval, such as the loss of key information in the long-term interaction [1], the lack of macro capture of event trends and reasoning ability, which hinder the development of medical inquiry system based on LLM.”
Haq, S. U., Castilho, S., & Graham, Y. (2026) · Information · doi:10.3390/info17070689
“These include reproducibility concerns associated with proprietary models [89], inconsistent adherence to task instructions [84], potential knowledge dilution in long prompts [98], and relatively weak correlations with human judgments at the segment level [84,87].”
A bird-inspired artificial intelligence framework for advanced large text summarization
Huang, B., & Bari, A. (2026) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2026.1703769
Eger, B.-L., Biller, L., Rose, D., Bontrup, F., Dinter, B., & Kollwitz, C. (2026) · HMD Praxis der Wirtschaftsinformatik · doi:10.1365/s40702-026-01301-z
OCEAN-Weighted Event Gating Preserves Recall in LLM NPC Memory Pipelines
Damastuti, F. A., Gumelar, A. B., Firmansyah, K., Fananda, I. Y., Hakkun, R. Y., & Akbar, Z. F. (2026) · International Electronics Symposium (IES) · doi:10.1109/ies71033.2026.11702684
“Important facts can still be missed when buried inside long prompts, a pattern often described as “lost in the middle” [2].”
Beyond Relevance: On the Relationship Between Retrieval and RAG Information Coverage
Samuel, S., Martin, A., Yang, E., Yates, A., Lawrie, D., Soboroff, I. M., Dietz, L., & Durme, B. V. (2026) · ACM SIGIR International Conference on Theory of Information Retrieval (ICTIR) · doi:10.1145/3805713.3820424 · Cited by 5
“To date, these LLMs are not able to faithfully leverage all advertised context window, exhibiting information loss [34] and hallucination [26] when exceeding certain effective limit.”
Explainability of Text Processing and Retrieval Methods: A Survey
Saha, S., Majumdar, D., & Mitra, M. (2026) · ACM Computing Surveys · doi:10.1145/3801957 · Cited by 10
“A Supplementary Tables (Electronic Version Only)”
StructEval: A Systematic Evaluation Framework for Structural Awareness in Large Language Models
Wang, R., Chen, Y., Fu, H., & Bian, Y. (2026) · IEEE International Symposium on Intelligent Transportation and Smart City (ITASC) · doi:10.1109/itasc70892.2026.00018
“However, empirical studies have identified the “Lost in the Middle” phenomenon [10], where models struggle to utilize mid-document information.”
KORAL: Knowledge Graph Guided LLM Reasoning for SSD Operational Analysis
Akewar, M., Madireddy, S., Luo, D., & Bhimani, J. (2026) · IEEE International Parallel and Distributed Processing Symposium (IPDPS) · doi:10.1109/ipdps65963.2026.00085 · Cited by 4
“…applying LLMs to storage telemetry poses challenges: free-form generation can hallucinate [60], [61], correct reasoning requires rich context on device design, firmware, and operational constraints [62], and long time series must preserve ordering, units, and align events across devices and sites.”
Query-Adaptive Graph Compression for LLM-Based Service Reasoning and Composition
Zare, M., Beheshti, A., Li, M., & Asgari, P. (2026) · IEEE International Conference on Web Services (ICWS) · doi:10.1109/icws72778.2026.00171
“Liu et al. [6] demonstrate that performance degrades when relevant information is positioned mid-context, regardless of window size.”
Impact of Ensemble of Vector Embeddings on Speculative Retrieval Augmented Generation
Kukreja, S., Kumar, T., Bharate, V., Singh, R. K., Dasgupta, A., & Guha, D. (2024) · International Conference on Data Intelligence and Cognitive Informatics (ICDICI) · doi:10.1109/icdici62993.2024.10810945 · Cited by 2
“It affects the computational efficiency and bias for the extended context retrieved from a vector database [7].”
Mikkelsen, Y. (2026) · SSRN Electronic Journal · doi:10.21203/rs.3.rs-9237602/v1
Reconfirm Instruction Prompting for LLM in In-context Learning
Yoshioka, D., & Matoishi, R. (2026) · Journal of Information Processing · doi:10.2197/ipsjjip.34.607
ProRAG: Towards Reliable and Proficient AIGC-Based Digital Avatar
Zhou, Y., Yan, M., Yao, J., & Xu, G. (2026) · Lecture notes in computer science · doi:10.1007/978-981-95-4158-4_29
Liao, C.-T., Xiao, X., Meng, C., Chen, Z., Qiao, Y., Zhou, W., Wang, T., Xu, Z., & Cao, X. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37235-2_25 · Cited by 13
STALL + : Boosting LLM-based Repository-level Code Completion with Static Analysis
Jun-wei, L., Chen, Y., Liu, M., Peng, X., & Lou, Y. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3846180 · Cited by 48
“Having been pre-trained on massive code corpora, code LLMs exhibit strong effectiveness in code completion.”
Friend, J., Chow, C., Sideman, A. B., Pinheiro‐Chagas, P., Brooks-Smith-Lowe, S., Dulaney, S., Sawyer, R. J., Hess, M., Lum, H. D., Olney, N. T., et al. (2025) · doi:10.2196/preprints.88543
Zhao, Y., Miao, Y., Luo, Y., Guo, R., Wang, H., & Wu, Y. N. (2026) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2026.106513 · Cited by 3
Wang, H., Wang, Z., Zhou, W., & Deng, Y. (2026) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2026.3675784 · Cited by 1
“Earlieranalysesthatinitiallyidentified andcharacterizedpositionalbiaseswereconductedusingolder modelsandrelativelyshortercontextsthatnolongerqualify as“longcontext”bycurrentstandards[12],[19],[20].”
InfluentialCostas, S. M., Diaz, F. M. R., Orchanski, D., Pastore, P. G., Fernández, E. A., & Jaureguiberry, P. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6381322 · Cited by 1
Heo, S. (2026) · Scientific Reports · doi:10.1038/s41598-026-50127-8
“Fifth, the Verifier itself is imperfect, so some solver or report inconsistencies may still escape detection, which is a known limitation of LLM-based judging more broadly 5 .”
Dynamic Information Sub-Selection for Adaptive Decision Support
Huang, H.-T., Lennon, M., Brahmavar, S. B., Sylvia, S., & Oliva, J. (2026) · ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics (ACM-BCB) · doi:10.1145/3807503.3819468
“For LLM-based tools in health-care and bioinformatics, performance depends strongly on prompt composition and available context, and long-context reasoning can degrade depending on where relevant information appears [35].”
Execution Control Harness for Governed LLM-Based Code Generation
Takeda, M., Funaya, Y., Harriet, O., Mibe, R., & Kanuka, H. (2026) · International Conference on Information, Intelligence, Systems & Applications (IISA) · doi:10.1109/iisa71656.2026.11705535
“Long and complex business requirements often lead to specification omissions (e.g., the Lost in the Middle problem [4]), unintended regressions caused by localized fixes, and difficulties in enforcing organization-level quality standards.”
XPipe: Explainable Multi-Stage LLM Pipeline Evaluation via Causal Attribution for Telecommunications
Messou, F. J. A., Zhang, S., Wang, W., Yu, T., Liu, T., Chen, J., & Yu, K. (2026) · ITU Kaleidoscope Academic Conference (ITU K) · doi:10.23919/ituk70038.2026.11632701
Kuang, E., Shen, L., Soure, E. J., Fan, M., & Shinohara, K. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2026.2638554
AgentTCP: A Collaborative Multi-Agent Framework for Change-Aware Test Case Prioritization
Chen, J., Li, X., Wang, J., Xie, H., Liu, C., Wu, Z., & Lei, Y. (2025) · Asia-Pacific Software Engineering Conference (APSEC) · doi:10.1109/apsec66846.2025.00066
“The Monolithic-LLM approach, while semantically aware, suffers from the well-documented "lost in the middle" problem[19].”
A Hybrid RAG Framework for AI Powered Personalized Document Summarization and Contextual Querying
Prasad, D. R., Yegireddi, R., & Anjaneyulu, P. (2026) · International Conference on Data Science and Business Systems (ICDSBS) · doi:10.1109/icdsbs69077.2026.11634727
“Liu et al. (2023) proved the "lost in the middle" problem, where LLMs do not effectively use contextual information placed far from either end of their window, motivating our 1,000 token chunking strategy with 100 token overlaps [6].”
Ocak, A., Diri, S., & Kilimci, Z. H. (2026) · Süleyman Demirel Üniversitesi Fen Bilimleri Enstitüsü Dergisi · doi:10.19113/sdufenbed.1912302
Kumar, A., Kumar, P., Mishra, S. P., & Gupta, S. (2025) · 2025 International Conference on Digital Innovations for Sustainable Solutions (ICDISS) · doi:10.1109/icdiss68238.2025.11320587
Efficient GraphRAG via Community Embedding for Global and Reasoning-Intensive Question Answering *
Yang, Q., Chen, K., Liu, X., Zhang, T., & Huang, D. (2025) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc58881.2025.11342988
Lu, M., Gao, F., Tang, X., & Chen, L. (2023) · ChemRxiv · doi:10.26434/chemrxiv-2023-gxmxr
Epistemological Contours of Large Language ModelsPreprint
Wong, M. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6354698
A Heterogeneous Agentic RAG Architecture for Administrative Knowledge Management
Gałązkiewicz, A. (2026) · Lecture notes in business information processing · doi:10.1007/978-3-032-26363-6_16
Suntwal, S., & Brown, S. A. (2026) · Information Systems Research · doi:10.1287/isre.2023.0457
ReRead: Retrieval-Guided Efficient Reading for Long-Context Question Answering
Ling, J., Bao, W., Zhong, Z., Zhou, T., & Wang, J. (2026) · International Conference on Big Data and Information Analytics (BigDIA) · doi:10.1109/bigdia71496.2026.11684612
“Prior studies have shown that long-context models can still be affected by evidence position, context length, and the density of distracting information when processing ultra-long inputs, which may cause critical information in the middle of the context to be underutilized [4].”
Agentic Retrieval for Conversational Memory in LLM-Based Chatbots
Turgut, V., & Mahmoud, Q. H. (2026) · IEEE Annual Computing and Communication Workshop and Conference (CCWC) · doi:10.1109/ccwc67433.2026.11393694 · Cited by 1
Calantoc, D. J. R., Dasig, D. D., Baltazar, A. J. D. C., Gatpandan, P. H., Singh, S. K., Peng, S.-L., Mazo, P. N., & Enriquez, S. (2026) · International Conference on Computational Intelligence and Communication Networks (CICN) · doi:10.1109/cicn70047.2026.11594370
“This distinction is central in the present benchmark because CHED-MCP Assisted generation shifts token use toward input-side curriculum grounding, while Zero-Shot Iterative refinement increases cumulative token cost through repeated correction prompts [14] .”
Chen, S. (2026) · arXiv · doi:10.48550/arxiv.2606.22528 · Cited by 21
“Long-context degradation is a known confound: models under-use mid-context information [17], exhibit “context rot” as distractors grow [13], and show unstable refusal in long-context agents [11].”
SmoothAgent: Efficient Long-Horizon LLM-Based Agent Serving with Lookahead Context Engineering
Pan, Z., Wang, Q., Hu, Z., Chen, C., Guan, Y., Zhou, Y., Swanson, S., & Ding, Y. (2026) · arXiv · doi:10.48550/arxiv.2607.00151
“This approach is simple yet effective, as recent interactions are typically the most relevant for next-step reasoning in agent workloads [40].”
From Recall to Forgetting: Benchmarking Long-Term Memory for Personalized Agents
Uddin, M. N., Shubham, K., Blanco, E., Baral, C., & Wang, G. (2026) · arXiv · doi:10.48550/arxiv.2604.20006 · Cited by 32
“In realistic settings, placing the entire interaction history into the prompt is impractical (Lewis et al., 2020; Packer et al., 2023) and often counterproductive (Liu et al., 2024; Du et al., 2025).”
In Prospect and Retrospect: Reflective Memory Management for Long-term Personalized Dialogue Agents
Tan, Z., Yan, J., Hsu, I.-H., Han, R., Wang, Z., Le, L. T., Song, Y., Chen, Y., Palangi, H., Lee, G., et al. (2025) · arXiv · Cited by 54
“Second, the agent must accurately retrieve relevant past information from the memory, as incorporating irrelevant context can distract the LLM and degrade response quality (Liu et al., 2024b; Shi et al., 2023).”
SE-Blackboard: A Shared-State Architecture for Multi-Agent Software Engineering Pipelines
Liu, E., Zhu, Q., & Dong, X. (2026) · IEEE Access · doi:10.1109/access.2026.3695435
“When presented with this larger context—consistent with prior findings that LLMs struggle to make effective use of very long inputs [28]—the Coder often produces no diff output.”
Let’s read the log: root cause analysis of railway test execution logs with large language models
Hermawan, R., Bucaioni, A., Enoiu, E., Afzal, W., Saadatmand, M., Zaimovic, N., & Ibtasham, M. S. (2026) · Innovations in Systems and Software Engineering · doi:10.1007/s11334-026-00657-1
“Prior work shows that LLMs often struggle to reliably select and integrate the most relevant evidence when multiple competing signals are present, particularly in long or complex contexts [36].”
SEMA-RAG: A Self-Evolving Multi-Agent Retrieval-Augmented Generation Framework for Medical Reasoning
Huang, Y., Chen, R., & Cheng, J. (2026) · Findings of the Association for Computational Linguistics: ACL 2026 · doi:10.18653/v1/2026.findings-acl.917
“…evidence exploration, and answer adjudication are tightly coupled, the cognitive load increases and the steps become interdependent, making it hard for the model to promptly adjust retrieval and reasoning when evidence is insufficient or conflicting (Wang et al., 2023; Liu et al., 2024).”
An, Y., Lu, C., Zhu, K., Yu, T., Zhao, C., Wu, H., Tang, M., & Wang, J. (2026) · arXiv · doi:10.48550/arxiv.2605.08840
“…and (2) RULER (Hsieh et al., 2024), a challenging long-context benchmark consisting of 4 categories and 13 complex tasks; (3) Needle-in-a-Haystack (Liu et al., 2024a), designed to assess the ability of models to retrieve key information from long sequences; (4) InfiniteBench (Zhang et al., 2024b),…”
InfluentialD-MEM: Dopamine-Gated Agentic Memory via Reward Prediction Error Routing
Song, Y., & Xin, Q. (2026) · arXiv · doi:10.48550/arxiv.2603.14597 · Cited by 1
“Extended-context approaches [Bai et al., 2024] partially address recall depth but are computationally prohibitive for lifelong agents and remain susceptible to the “lost in the middle” phenomenon [Liu et al., 2024] under noisy, interleaved dialogue logs.”
RecMem: Recurrence-based Memory Consolidation for Efficient and Effective Long-Running LLM Agents
Dai, Z., Deng, S., Guan, S., Tian, Y., Yao, X., Yan, X., & Cheng, J. (2026) · Findings of the Association for Computational Linguistics: ACL 2026 · doi:10.18653/v1/2026.findings-acl.1619
“In practice, this is hindered by two critical limitations: current LLMs cannot retain information beyond their limited context windows (Liu et al., 2025), and they often under-utilize relevant evidences even if they are present in long inputs due to the lost-in-the-middle effect (Liu et al., 2023).”
Oracle Agent Memory as an Enterprise Memory Substrate for Long-Horizon AI Agents
Alake, R., Bernardis, C., Cayet, P., Engel, L., Hilloulin, D., Hong, S., Hosler, A., Kavantzas, N., Kossyk, I., Le, S., et al. (2026) · arXiv · doi:10.48550/arxiv.2607.13157
“However, empirical work on long-context use shows that models can fail to use information reliably when relevant evidence appears in the middle of long inputs [13].”
What Deserves Memory: Adaptive Memory Distillation for LLM Agents
Ma, W., Nan, J., & Wu, W. (2026) · Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2026.acl-long.1607 · Cited by 89
“…long-term behavioral consistency in Large Language Model (LLM)- based agents stems from a fundamental conflict: the reliance of stateless LLMs on linearly expanding interaction trajectories versus the constraints of finite context window and the Lost in the Middle phenomenon (Liu et al., 2024).”
Hippocampus: An Efficient and Scalable Memory Module for Agentic AI
Li, Y., Cao, L., Ahmed, F., Sharma, P., & Li, B. (2026) · arXiv · doi:10.48550/arxiv.2602.13594
“Empirical studies, including the “ Lost-in-Middle ” effect (Liu et al., 2023a), show that reasoning accuracy degrades sharply as prompts grow longer.”
Just-In-Time Reinforcement Learning: Continual Learning in LLM Agents Without Gradient Updates
Li, Y., Lin, Z., Deng, A., Zhang, X., He, Y., Ji, S., Cao, T., & Hooi, B. (2026) · arXiv · doi:10.48550/arxiv.2601.18510
“However, such approaches tend to struggle as the required context length grows, especially since agentic tasks often involve long sequences of interactions (Liu et al., 2024).”
Harnessing Large Language Models for Scientific Novelty Detection
Liu, Y., Yang, Z., Poria, S., Nguyen, T.-S., & Cambria, E. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-38410-2_39 · Cited by 8
A Survey of Long-Document Retrieval in the PLM and LLM Era
Li, M., Zhang, Y., Lv, T., Zhao, S., Luo, M., Nie, E., & Zhou, G. (2026) · ACM Transactions on Information Systems · doi:10.1145/3849382 · Cited by 8
RAG-LER: Ranking adapted generation with language-model enabled regulation
Zhai, F., Tang, W., & Jin, J. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.131514 · Cited by 1
Ideuchi, M., Oida, Y., Higashiyama, S., Tanaka, H., Utiyama, M., & Watanabe, T. (2025) · Journal of Natural Language Processing · doi:10.5715/jnlp.32.918
DR.SIMON: Domain-Wise Rewrite for Segment-Informed Medical Oversight Network
Lee, S., Choe, S., Choi, J., et al. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-07845-2_10
Large Language Models and Knowledge Graphs Synergy to Support the Public Administration
Ghizzota, E. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-99554-5_37
Braberman, V. A., Bonomo, F., Charalambous, Y., Colonna, J. G., Cordeiro, L. C., & de Freitas, R. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5624902 · Cited by 8
Technischer Hintergrund: Große Sprachmodelle (LLM)
Lang, F. (2025) · Springer eBooks · doi:10.1007/978-3-662-71542-0_2
AEG-Edit: Bridging API Evolution and Code Semantics for Accurate API Usage in Code Generation
Lin, Y., Tian, H., Zhao, D., Ni, C., Yang, X., & Ren, X. (2026) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3832783.3834415
“Prompt-based methods supply API documentation through the input context without updating model parameters [44, 46, 51, 57], but LLMs are prone to context neglect and the lost-in-the-middle problem [30], reverting to memorized deprecated patterns.”
Maintaining Control States for Initialization-Aware GRPO in Industrial LLM Workflows
Liu, C., Yuan, Z., 范奉媛, Che, L., Qian, Q., Liang, Q., Sun, Z., & Zhang, L. (2026) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3832783.3834459
“Second, the optimization target must match downstream GRPO: a starting state that improves immediate outputs may still generate early samples with weak reward differences, providing little useful signal for GRPO updates [24, 27].”
NEAT: Agentic Interpretation: Lattice-Structured Evidence for LLM-Based Program Analysis
Mitchell, J., & Wang, C. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-40374-2_15
KERKIS: a Modeling Language for Multi-Agent System Interactions in AI-Native Software Engineering
Kalliokoski, K., Christophe, F., Kärkkäinen, H., Hämäläinen, I., Hasan, M. T., Hasan, M. M., Zhang, Z., Abrahamsson, P., & Mikkonen, T. (2026) · ACM/IEEE International Conference on Model Driven Engineering Languages and Systems (MODELS) · doi:10.1145/3837062.3839380
Metadata
- Authors
- Liu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., & Liang, P.
- Affiliations
- Stanford University (US)
- University of California, Berkeley (US)
- Samaya AI, UK
- Corresponding author
- Liu, Nelson F.
- Venue
- Transactions of the Association for Computational Linguistics
- Publisher
- MIT Press
- Published
- 2024
- Volume
- 12
- Pages
- 157-173
- ISSN
- 2307-387X (ISSN-L)
- OpenAlex
- W4391876619
- Semantic Scholar
- 1733eb7792f7a43dd21f51f4d1017a1bffd217b5
- Cited by
- 1,612
- References
- 59
- Citations by year
- 2026: 901 · 2025: 516 · 2024: 172
- FWCI
- 233.95
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- Locations
- Transactions of the Association for Computational Linguistics (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- long-context language models, multi-document question answering, position bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-09-27 (prompt extract-v2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.