Lost in the Middle: How Language Models Use Long Contexts
Open accessThe DOI matches its Crossref record; 48 of the 59 references it cites were found in a registry or an index; 15 of the 23 sources its claims cite were checked for support.
Abstract
While recent language models have the ability to take long contexts as input, relatively little is known about how well they use longer context. We analyze the performance of language models on two tasks that require identifying relevant information in their input contexts: multi-document question answering and key-value retrieval. We find that performance can degrade significantly when changing the position of relevant information, indicating that current language models do not robustly make use of information in long input contexts. In particular, we observe that performance is often highest when relevant information occurs at the beginning or end of the input context, and significantly degrades when models must access relevant information in the middle of long contexts, even for explicitly long-context models. Our analysis provides a better understanding of how language models use their input context and provides new evaluation protocols for future long-context language models.
Overview
CoChat Summary
TL;DR
Language models’ performance on tasks requiring access to long input contexts is strongly affected by where relevant information appears, with a U-shaped curve: best at the beginning or end and worst in the middle. Extended-context models are not necessarily better at using context, and reader performance saturates before retriever recall in open-domain QA.
Key findings
- In multi-document QA, performance is highest when the answer-containing document is at the start or end of the context and degrades in the middle; GPT-3.5-Turbo can drop by more than 20%, and in 20- and 30-document settings middle-position performance is lower than its closed-book performance of 56.1% (oracle: 88.3%).
- Extended-context models are not necessarily better at using input context: when the input fits both windows, GPT-3.5-Turbo and GPT-3.5-Turbo(16K) perform nearly identically as a function of relevant-information position.
- In synthetic key-value retrieval, Claude-1.3 and Claude-1.3(100K) do nearly perfectly, while other models struggle especially with 140 or 300 key-value pairs; query-aware contextualization gives GPT-3.5-Turbo(16K) perfect performance at 300 pairs, versus worst-case 45.6% without it.
- In open-domain QA, reader performance saturates long before retriever recall: using 50 instead of 20 retrieved documents only marginally improves performance (∼1.5% for GPT-3.5-Turbo and ∼1% for Claude-1.3).
Methods
- Controlled multi-document QA on NaturalQuestions-Open: 2,655 queries whose annotated long answer is a paragraph; each context has exactly one answer-containing Wikipedia passage plus k−1 Contriever/MS-MARCO distractor passages; the paper varies answer position and context length (10, 20, 30 documents) and measures accuracy.
- Synthetic key-value retrieval: inputs are serialized JSON objects with unique random UUID key-value pairs; models must return the value for a specified key; experiments use 75, 140, and 300 key-value pairs (500 examples each), varying key position and number of pairs, with accuracy as the metric.
- Models include open MPT-30B-Instruct and LongChat-13B(16K), closed GPT-3.5-Turbo, GPT-3.5-Turbo(16K), Claude-1.3, and Claude-1.3(100K), plus additional architecture/instruction experiments with Flan-T5-XXL, Flan-UL2, MPT-30B, Llama-2, and GPT-4; an open-domain QA case study uses a Contriever retriever-reader setup.
Limitations
- Stated: the paper uses greedy decoding and leaves exploration of other decoding methods to future work; the investigations into why models are not robust are described as preliminary; GPT-4 is evaluated only on a subset of 500 multi-document QA examples because full evaluation would cost upwards of $6000.
- Stated: NaturalQuestions-Open has ambiguity, so the authors run an unambiguous subset and also test random distractors and randomized distractor order, finding similar trends.
- Evident: the controlled multi-document QA setting always contains exactly one document that answers the question, unlike open-domain QA; the synthetic key-value task removes natural language semantics; results may not generalize to all tasks, models, or decoding settings.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
7 Valid · 0 Not valid · 5 Not checked · 13 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Language models have become an important and flexible building block in a variety of user-facing language technologies, including conversational interfaces, search and summarization, and collaborative writing (Shuster et al., 2022;Thoppilan et al., 2022;Lee et al., 2022, inter alia).”
§ Lost in the Middle: How Language Models Use Long Contexts > 1 Introduction
What the sources say
- Partly supportedcites Shuster et al., 2022 sourceExperimental — accuracy not yet measured
Section 3.1.1 (p. 4) describes a conversational agent built on a language model that generates internet search queries, supporting conversational interfaces and search as part of dialogue. Section 2 (p. 3) mentions summarization only as related work, and the passages do not establish collaborative writing as an application of this paper’s model. (LLM, conf=0.85)
- Supportedcites Thoppilan et al., 2022 source
The Abstract (p. 1) identifies LaMDA as a family of language models specialized for dialog. In this sentence's list of applications, Thoppilan et al. supports the conversational-interface example; the other cited works need not each cover every example. (LLM, conf=0.92)
- Supportedcites Lee et al., 2022 source
Lee et al. is cited alongside other works, and its contribution to the sentence is the collaborative-writing example. Section 4.2.2 describes writers receiving and selecting model-generated suggestions in a writing interface; Section 2.2.1 also describes language models as applicable to dialogue and summarization. The paper need not establish every example in the jointly cited list. (LLM, conf=0.89)
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“These observations complement prior work, which found that non-instruction fine-tuned language models are biased towards recent tokens (i.e., the end of the input context; Khandelwal et al., 2018;Press et al., 2021).”
§ Lost in the Middle: How Language Models Use Long Contexts > 4.3 Effect of Instruction Fine-Tuning
What the sources say
- Partly supportedcites Khandelwal et al., 2018 sourceExperimental — accuracy not yet measured
The Introduction and Section 5 report that the studied LSTM is much more sensitive to recent word order than to distant word order, supporting a recency effect. The paper studies a standard LSTM on two datasets, however, rather than establishing this tendency across non-instruction-fine-tuned language models generally; it also finds that distant words remain relevant. (LLM, conf=0.82)
- Partly supportedcites Press et al., 2021 sourceExperimental — accuracy not yet measured
Section 3.1 (page not available in our copy) finds that very short contexts hurt performance, while increasing subsequence length from 1,024 to 3,072 tokens gives negligible improvement. That is consistent with limited benefit from more distant context, but the paper does not establish that the models are biased toward recent tokens within an input context. (LLM, conf=0.78)
- Partly supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“We instantiate this task with data from NaturalQuestions-Open (Lee et al., 2019;Kwiatkowski et al., 2019), which contains historical queries issued to the Google search engine, coupled with human-annotated answers extracted from Wikipedia.”
§ Lost in the Middle: How Language Models Use Long Contexts > 2.1 Experimental Setup
What the sources say
- Supportedcites Lee et al., 2019 source
Section 7.1 says the paper creates an open version of Natural Questions and uses it for training and evaluation. The Introduction describes the task as answering questions by retrieving evidence from an open corpus, supporting the cited use of NaturalQuestions-Open for this task. (LLM, conf=0.91)
- Not checked — no full text of the sourcecites Kwiatkowski et al., 2019 source
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Following Kandpal et al. (2022) and Mallen et al. (2023), we use accuracy as our primary evaluation metric, judging whether any of the cor- rect answers (as taken from the NaturalQuestions annotations) appear in the predicted output.”
§ Lost in the Middle: How Language Models Use Long Contexts > 2.1 Experimental Setup
What the sources say
- Supportedcites Kandpal et al. (2022) source
Section 3 (p. 4) says the paper evaluates answers using exact match against a ground-truth answer set, and Section 3.1 (p. 4) reports QA accuracies as its evaluation results. This supports citing Kandpal et al. for using accuracy to evaluate question answering. (LLM, conf=0.90)
- Supportedcites Mallen et al. (2023) source
Section 3.1 (‘Focus and Task’) explicitly names accuracy as the metric and says a prediction is correct when it contains an exact match to any gold answer. This supports the evaluation approach attributed to Mallen et al.; the NaturalQuestions annotations describe the citing authors’ answer source. (LLM, conf=0.94)
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“or when language models are augmented with external information (relevant documents from a search engine, database query results, etc.; Petroni et al., 2020;Ram et al., 2023;Shi et al., 2023;Mallen et al., 2023;Schick et al., 2023, inter alia).”
§ Lost in the Middle: How Language Models Use Long Contexts > 1 Introduction
What the sources say
- Partly supportedcites Petroni et al., 2020 sourceExperimental — accuracy not yet measured
The Abstract (p. 1) says the paper integrates information from a retrieval system with a pre-trained language model, and §3.4.2 (p. 5) specifies a retrieved paragraph as context. This supports augmentation with retrieved documents, but our copy does not show language models being augmented with database query results. (LLM, conf=0.84)
- Partly supportedcites Ram et al., 2023 sourceExperimental — accuracy not yet measured
The Introduction describes augmenting language models with relevant documents retrieved from an external knowledge source, and Section 5 describes using a BM25 retriever to find those documents. The passages do not establish database query results as another type of external information used by this paper. (LLM, conf=0.82)
- Partly supportedcites Shi et al., 2023 sourceExperimental — accuracy not yet measured
Section 3.1 (page unavailable) describes retrieving relevant documents from a corpus, and the Conclusion says REPLUG uses those documents to augment a language model. The paper supports the document-retrieval example, but our copy does not describe augmenting models with database query results or documents from a search engine specifically. (LLM, conf=0.82)
- Partly supported
20 more claims · 2 with their cited sources checked
Create a free account to see all 25 claims and every source verdict.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In vanilla multi-document or NIAH-based benchmarks (Kamradt, 2023; Liu et al., 2024), models perform consistently well when the needle (or gold document) appears at the very beginning or end of the context window, with minimal impact from context length.”
From NoLiMa: Long-Context Evaluation Beyond Literal Matching (2025) · cites this paper as Liu et al., 2024
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Even within a single long sequence, attention effectiveness degrades with distance due to attention dilution , positional encoding limitations, and token interference, leading to the well-known 'lost-in-the-middle' and context-decay phenomena (Liu et al., 2024; Press et al., 2021a).”
From MAGMA: A Multi-Graph based Agentic Memory Architecture for AI Agents (2026) · cites this paper as Liu et al., 2024
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Current large language models (LLMs) exhibit significant deficiencies in episodic memory tasks including encoding, storing, and retrieving specific information from temporally dependent events over a long period of time.”
From ARTEM: Enhancing Large Language Model Agents with Spatial-Temporal Episodic Memory (2026) · cites this paper as Liu et al. 2024
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Recent approaches to handle memory tasks in LLMs, such as in-context learning, retrieval-augmented generation (RAG), and fine-tuning, may resolve the long-term retention issues, but are still inadequate to handle tasks requiring chronological awareness of the stored information.”
From ARTEM: Enhancing Large Language Model Agents with Spatial-Temporal Episodic Memory (2026) · cites this paper as Liu et al. 2024
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“If not ranked at the top, the key information may be neglected by the generative module, resulting in incomplete or inaccurate responses to the user queries24.”
From Leveraging long context in retrieval augmented language models for medical question answering (2025) · cites this paper as [24]
Experimental — accuracy not yet measured
86 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929
LLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602
Transformer-XL: Attentive language models beyond a fixed-length context
Dai, Z., Yang, Z., Yang, Y., Carbonell, J., Le, Q., & Salakhutdinov, R. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1285 · Cited by 3,169 · cited 2×
“Recent improvements in hardware (e.g., faster GPUs with more memory) and algorithms (Dai et al., 2019; Dao et al., 2022; Poli et al., 2023; Rubin and Berant, 2023, inter alia ) have resulted in language models with larger context windows (e.g., 4096, 32K, and even 100K tokens), but it remains…”
BackgroundLlama 2: Open Foundation and Fine-Tuned Chat Models
Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.09288 · Cited by 2,604
Longformer: The Long-Document Transformer
Beltagy, I., Peters, M. E., & Cohan, A. (2020) · arXiv · doi:10.48550/arxiv.2004.05150 · Cited by 2,281
Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering
Izacard, G., & Grave, E. (2020) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.48550/arxiv.2007.01282 · Cited by 2,263
Natural Questions: A benchmark for question answering research
Kwiatkowski, T., Palomaki, J., Redfield, O., Collins, M., Parikh, A., Alberti, C., Epstein, D., Polosukhin, I., Devlin, J., Lee, K., et al. (2019) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00276 · Cited by 2,069 · cited 1×
“We instantiate this task with data from NaturalQuestions-Open (Lee et al., 2019; Kwiatkowski et al., 2019), which contains historical queries issued to the Google search engine, coupled with human-annotated answers extracted from Wikipedia.”
MethodThe serial position effect of free recall.
Murdock, B. B. (1962) · Journal of Experimental Psychology · doi:10.1037/h0045106 · Cited by 1,717 · cited 1×
“The U-shaped curve we observe in this work has a connection in psychology known as the serial-position effect (Ebbinghaus, 1913; Murdock Jr, 1962), that states that in free-association recall of elements from a list, humans tend to best re-member the first and last elements of the list.”
MethodMemory: A contribution to experimental psychology.
Ebbinghaus, H. (1913) · doi:10.1037/10011-000 · Cited by 1,645 · cited 1×
“The U-shaped curve we observe in this work has a connection in psychology known as the serial-position effect (Ebbinghaus, 1913; Murdock Jr, 1962), that states that in free-association recall of elements from a list, humans tend to best re-member the first and last elements of the list.”
Method
Research works printed but not linked: 1
- Sharan, 2018, Prediction with a short memory, 10.18653/v1/K18-3013 · DOI
Cited by
Items 1,301–1,400 of 1,592
Adaptive Mixing of Embeddings from Multiple Code Language Models for Fault Localization
Yang, J., Park, E. Y., & Ko, I. (2026) · IEEE International Conference on Software Testing, Verification and Validation (ICST) · doi:10.1109/icst69053.2026.00044
“First, it aims to alleviate the technical challenge of language models, which tend to suffer performance degradation as input token length increases [42].”
A Pragmatist Robot: Learning to Plan Tasks by Experiencing the Real World
Qu, K., Lan, G., Zurbrügg, R., Chen, C., Mower, C. E., Ammar, H. B., & Hutter, M. (2026) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2026.3669804 · Cited by 15
“This aligns with prior findings that excessively long or noisy contexts can degrade model performance, as LLMs may struggle to focus on the most relevant content [38], [39].”
Agentic AI workflows in anatomic pathology: Opportunities and challenges
Ngo, A., Tian, R., Mohammad, A., Weon, J. L., Arvisais‐Anhalt, S., Minor, J., & Youens, K. (2026) · Journal of Pathology Informatics · doi:10.1016/j.jpi.2026.100718
Dimension-Constrained Organizational Relay for Long-Context LLMs
Wang, X., Li, Z., Hu, C., & Wei, S. (2026) · Applied Sciences · doi:10.3390/app16167903
ОСОБЛИВОСТІ СИСТЕМ ГЕНЕРАЦІЇ З ДОПОВНЕНИМ ПОШУКОМ НА ОСНОВІ ВЕЛИКИХ МОВНИХ МОДЕЛЕЙ
Коломійцев, О., Вірич, Д. А., Комаров, В. О., Katunin, A., Собора, А. І., Закутін, К. В., Лільчицький, В., Коломієць, О., Соловйов, А. М., Кобець, Ю. В., et al. (2026) · ГРААЛЬ НАУКИ · doi:10.36074/grail-of-science.18.09.2026.033 · Cited by 1
“Явище «lost in the middle», розкрите у [15], полягає у тому, що велика мовна модель надійно видобуває факти, розміщені на початку або у кінці вхідного контексту та суттєво гірше – факти, розміщені в його середині.”
Li, B., Xu, Z., Zhang, S., & Ye, W. (2026) · Neurocomputing · doi:10.1016/j.neucom.2026.134582
Detecting Contradictions Within and Across Documents
Chen, S., Fan, W., Jin, R., & Ma, K. (2026) · Proceedings of the ACM on Management of Data · doi:10.1145/3837105
Hartmann, S., Bardakcioglu, R., & Schaal, S. (2025) · Frontiers in Psychology · doi:10.3389/fpsyg.2025.1696783 · Cited by 2
“Although the Gemini Models employed have a large enough context length to potentially screen hundreds of titles and abstracts with only one well-crafted prompt, context confusion could affect screening results significantly (Liu et al., 2024).”
Jiang, F., Yang, Q., Zheng, X., Yin, N., Zhang, J., Lan, T., Wu, Y., Lin, Y., Jiang, K., & Chen, Y. (2026) · Journal of Medical Internet Research · doi:10.2196/92663
“The value of k=20 was selected as a pragmatic balance between evidence coverage and noise control: a smaller k may miss complementary evidence, whereas a larger k may introduce weakly relevant fragments and distract the model from the most relevant information [16-18].”
Nezhad, S. B., & Agrawal, A. (2026) · Neurosymbolic Artificial Intelligence · doi:10.1177/29498732261443192
Comparative Evaluation of Encoder- and Decoder-Based Models for Actionable Findings in CT Reports
Fukui, Y., Harada, N., Nagatani, Y., Iguchi, H., & Watanabe, Y. (2026) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-026-01888-1
Fact-Centric Knowledge Web for Information Retrieval
Sinha, R., & Shiramatsu, S. (2024) · IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology (WI-IAT) · doi:10.1109/wi-iat62293.2024.00067 · Cited by 1
Improving Retrieval-Augmented Generation for Medical Literature: A Case Study on Pertussis Research
Takabatake, H., Martono, N. P., Kuwae, A., Iuchi, T., & Ohwada, H. (2025) · International Conference on Information & Communication Technology and System (ICTS) · doi:10.1109/icts67612.2025.11369608
“Moreover, when large number of documents including irrelevant content to LLMs can cause efficiency and performance issues, even with state-of-the-art models [18].”
From Similarity Ranking to Definitive Verdict: LLM-Enhanced Source-to-Binary Function Localization
Shi, J., Liu, C., Xu, Z., Xiao, Y., Chen, X., Li, Y., Huo, W., & Liu, Y. (2026) · Proceedings of the ACM on Programming Languages · doi:10.1145/3839539
“…it achieves the highest effectiveness among LLM-based ranking strategies: pointwise scoring suffers from unreliable calibration across invocations, while listwise ranking is undermined by the “lost in the middle” phenomenon [31] where candidates are overlooked due to position rather than content.”
MatKV: Trading Compute for Flash Storage in LLM Inference
Shin, K.-W., Park, J. H., Oh, M., Jo, Y., Do, J., & Lee, S.-W. (2026) · 2026 IEEE 42nd International Conference on Data Engineering (ICDE) · doi:10.1109/icde65706.2026.00071
“One possible reason for MatKV’s improved accuracy is that vanilla inference processes multiple documents within a single context window, potentially leading to token loss, position bias, or diluted attention [27].”
Question Answering for Multi-Release Systems: A Case Study at Ciena
Khamsepour, P., Cole, M., Ashraf, I., Puri, S., Sabetzadeh, M., & Nejati, S. (2026) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner67736.2026.00120 · Cited by 1
“As noted earlier, large context chunks, even when within an LLM’s token limit, can create a “needle in a haystack” problem, where irrelevant details obscure key information and hinder accurate answer generation by LLMs [23], [22], [24], [11].”
Zheng, Q., Zhang, J., Stebner, A., & Zhang, X. (2026) · JOM · doi:10.1007/s11837-026-08612-y
MRM: Improving Multihop Reasoning Capability for LLMs With Multistage QA Retrieval
Li, W., Zhang, Y., Sun, G., Ma, C., Qiang, J., & Shen, J. (2025) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2025.3642285
“Since each reasoning step needs different knowledge, they might retrieve many irrelevant documents to LLM to reason answer [20], thus generating the undesired answer.”
The Hitchhikers Guide to Production-ready Trustworthy Foundation Model Powered Software (FMware)
Vasilevski, K., Rajbahadur, G. K., Oliva, G. A., Rombaut, B., Gallaba, K., Cogo, F. R., Lin, J., Lin, D., Zhang, H., Chen, B., et al. (2025) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3711896.3736572 · Cited by 3
“While structured retrieval can improve precision, it requires carefully curated datasets and filtering mechanisms to maintain relevance [26, 58].”
MARVEL: a multi-agent research validator and enabler using large language models
Mukund, N., Luo, Y., Zhang, F., Barsotti, L., & Katsavounidis, E. (2026) · Machine Learning Science and Technology · doi:10.1088/2632-2153/ae68ba · Cited by 3
KGA-LM: Representation-Level Grounding for Conversational Search over Knowledge Graphs
Li, Y., Chengfei Liu, A., Zhou, R. H., Xu, Z., Zhang, S., & Wei, J. (2026) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3770855.3817824
“External evidence must constantly compete with contextual history for self-attention, triggering severe information decay—such as the "lost-in-the-middle" effect and catastrophic performance drops during long-context reasoning [18].”
InfluentialSmall Transformer Architectures for Task Switching
Gros, C. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-04558-4_10 · Cited by 1
Tool retrieval bridge: Aligning vague instructions with retriever preferences via bridge model
Chen, K., Zhuang, L., Liao, F., Liu, J., Wang, J., & Du, B. (2026) · Neural Networks · doi:10.1016/j.neunet.2026.109037 · Cited by 5
“Recent works have proposed various methods for tool retrieval, e.g. , using sparse retrieval techniques like BM25 (Patil et al., 2024; Liu et al., 2024) or dense methods (Qin et al., 2023; Kong et al., 2024; Gao et al., 2023a) that rely on sentence embeddings of the Sentence-BERT transformer model…”
Nguyen, S. T., Tulabandhula, T., & Watson‐Manheim, M. B. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4662955
Yu, Y., Shen, H., Lyu, Z., & He, Q. (2025) · ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing (SNPD) · doi:10.1109/snpd65828.2025.11254671 · Cited by 1
“Additionally, Liu et al. [16] noted that performance is typically highest when key information appears at the beginning or end of the input context, implying that simply increasing the number of prompt samples does not necessarily enhance the model’s fact-check-worthiness estimation performance.”
Comparative evaluation of ambient digital scribe systems in clinical documentation
Weiss, E., Zsenits, B., Nasar, F., Phillips, M., Mudgundi, V., Salhab, T., Muth, A., Zhu, K., Kolosko, J., Hansen, S., et al. (2026) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocag157
VulWeaver : Weaving Broken Semantics for Grounded Vulnerability Detection
Cao, Y., Chen, Y., Hu, X., Chen, B., Deng, J., Zhou, Z., Wu, S., Huang, Y., Du, X., Chen, X., et al. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3846196 · Cited by 1
“Therefore, we utilize Qwen2.”
Accelerating Language Model Workflows with Prompt Choreography
Bai, T., & Eisner, J. M. (2026) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl.a.643
Akkaya, A. (2026) · Akıllı Ulaşım Sistemleri ve Uygulamaları Dergisi · doi:10.51513/jitsa.1848120 · Cited by 1
Nyilas, G., Fleiner, R., & Kail, E. (2026) · IEEE International Conference on Intelligent Engineering Systems (INES) · doi:10.1109/ines69513.2026.11661061 · Cited by 1
“[ 15] This, ”lost in the middle” sate must be addressed by additional mechanisms.”
Consistency-driven evidence reconstruction for retrieval-augmented generation
Shao, H., Zhang, S., Wang, F., & Xie, Z. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.133704
Designing Reliable LLM Systems for AI-Assisted Engineering Education
Venkatachalapathy, R. (2026) · IEEE International Conference on Innovation, Ethics & Emerging Tech in Engineering and Computing Education (IE2C) · doi:10.1109/ie2c69620.2026.11663036 · Cited by 3
LLMA4OpenMP: A Large Language Model Based Agent for Automatic OpenMP Parallelization
Zhu, H., & Wang, W. (2025) · International Symposium on Image and Signal Processing and Analysis · doi:10.1109/ispa67752.2025.00165
“Unlike tools such as AutoPar-Clava [4], which require a complete codebase for static analysis, LLMA4OpenMP focuses on individual for loops (aligning with LLM limitations on long contexts [18]).”
Fang, Q., & Liu, W. (2026) · Computers and Education Artificial Intelligence · doi:10.1016/j.caeai.2026.100664 · Cited by 2
Extracting Fix Ingredients using Language Models
Prenner, J. A., & Robbes, R. (2025) · IEEE/ACM International Conference on AI Foundation Models and Software Engineering (FORGE) · doi:10.1109/forge66646.2025.00028 · Cited by 2
“Until recently, Transformer-based LLMs made much better use of information that appeared at either the beginning or the end of the input window [50], for retrieval and question answering tasks.”
Bridging LLMs and KGs: Data and Knowledge Adaptive Medical Reasoning for Med-VQA
Huang, W., He, Y., & Yang, Z. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-0369-7_6
Towards a Multi-agent LLM-Based Tutoring Tool for Mathematical Argumentation Skills
Koulani, H., Marchionne, L., Schmidtpott-Schulz, H., Eichler, A., Bley, A., & Söllner, M. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-28570-6_6
Plan-Execute-Generate-Judge: A Self Verifying Multi Agent LLM Framework for Complex NoSQL Querying
Kaplunovich, A. (2025) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata66926.2025.11400819
“More recent advances, like IRNet [31] and RAT-SQL [32], integrate schema representations and attention mechanisms to handle increasingly complex SQL generation tasks, achieving state-of-the-art results.”
Reproducibility in Information Retrieval
Maistro, M., Breuer, T., & Ferro, N. (2026) · Foundations and Trends® in Information Retrieval · doi:10.1108/ftinr-05-2025-0087 · Cited by 1
Diagnosing Evidence Utilization in Multimodal Document Question Answering
Basak, D., Bose, D., Goswami, K., & Desarkar, M. S. (2026) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3770855.3817956
“For our study, we use three long multimodal multi-document QA benchmarks: PaperPDF [26], MMLongBench-Doc [18], and Long-DocURL [9], which provide gold evidence, allowing us to evaluate evidence usage in a controlled setting without retrieval errors [16].”
Structure-Aware Generative Information Extraction via Feature Space Alignment
Li, Y., Tao, C., Zhang, B., & Zhang, W. (2026) · Information · doi:10.3390/info17050409
Dynamic Retrieval Strategy for Summarizing Doctor–Patient Dialogues with RAG
Shi, Y., & Hu, X. T. (2025) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata66926.2025.11402171
“Research such as ”Lost in the Middle” [12] reveals that LLMs often struggle to retain and utilize information from the middle portions of lengthy input.”
Wang, J., Song, F., Zhang, M., Du, B., & Pei, R. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3832222
“Given that the integration of multi-source data yields complex prompts that often trigger cognitive overload, leading to unintended information loss [68], we decompose the monolithic detection task into a three-stage pipeline.”
Coverage-Aware Evidence Packing for Budgeted Multi-Hop RAG on Small Language Models
Hamidi, C., Gaou, S., Mohamed, B., Mohamed, B., Boutahir, M. K., & Tribak, H. (2026) · International Conference on Circuit, Systems and Communication (ICCSC) · doi:10.1109/iccsc71566.2026.11650301
“Longer prompts are not a complete answer either, since models can fail to use relevant text when it appears in less useful positions [3].”
Bauerdick, J., Streibel, L., Milde, M., et al. (2026) · Zeitschrift für wirtschaftlichen Fabrikbetrieb · doi:10.1515/zwf-2026-1010
Zhao, Z., Ruiyu, Y., Zhou, J., Luo, J., Zheng, L., Lv, L., & Zhao, Y. (2026) · Scientometrics · doi:10.1007/s11192-026-05820-w
Improving security vulnerability assessment with attention-guided hierarchical code simplification
Chen, X., Liang, S., Shen, Z., Li, Z., & Chen, S. S. (2026) · Software Quality Journal · doi:10.1007/s11219-026-09776-1
“Long-context models can accept longer inputs, but they usually introduce higher computational cost and may still attend to large amounts of irrelevant code (Liu et al., 2024b).”
A hybrid quantitative–qualitative RAG approach for territorial well-being decision support
Comiti, G., Bisgambiglia, P.-A., & Lameta, N. (2026) · Machine Learning with Applications · doi:10.1016/j.mlwa.2026.101005
DSL Script Generation via Task Decomposition with LLMs
Hirano, T., Inoue, H., & Fujiwara, T. (2026) · IEICE Communications Express · doi:10.23919/comex.2026xbl0074
CLAIRE: A Controllable LLM Tutoring Framework for Reading Comprehension
Magurno, C., Contalbo, M. L., Paganelli, M., Giliberti, E., Bertolini, C., & Guerra, F. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-29755-6_4
LANCER: LLM Reranking for Nugget Coverage
Ju, J.-H., Landry, F.-G., Yang, E., Verberne, S., & Yates, A. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-21300-6_12 · Cited by 3
LLM-Based Grid-World Path Planning With Probabilistic Model Checking
Arya, A., Babikian, A. A., Jit, S. S., & Chećhik, M. (2026) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3803437.3806714 · Cited by 1
Widhiarto, E. D., & Girsang, A. S. (2026) · Procedia Computer Science · doi:10.1016/j.procs.2026.08.258
Jing, W., Chen, X., Jiao, Z., & Jiang, X. (2026) · International Symposium on Aerospace Engineering and Systems (ISAES) · doi:10.1109/isaes70982.2026.11669212
“The results support a prompt-budget interpretation: under structured prompting, the full 3D object list can overload the VLM with peripheral context [27], whereas relevance filtering concentrates decision-critical spatial evidence and recovers trajectory accuracy.”
A Zero-Shot Explainable Doctor Ranking Framework with Large Language Models
Zeng, Z., Li, D., & Yang, Y. (2026) · Big Data Mining and Analytics · doi:10.26599/bdma.2025.9020098 · Cited by 4
“Additionally, both pairwise and listwise methods are susceptible to position bias [13,30,31] , where the order of candidates can influence the ranking outcomes.”
Validating GenAI Work: Expert Quality Assurance in Contract AnalysisPreprint
Tseng, A., & Chang, C. J. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6188540
Structured Prompt and Agent AI Framework for Reliable LLM-Based RC Beam Design
Heo, S., Choi, W., & Lee, S. (2026) · International Journal of Concrete Structures and Materials · doi:10.1186/s40069-026-00950-0
“They also show why engineering inputs and code assumptions must be stated explicitly and why some tasks may require coupling LLMs (Liu et al., 2024).”
LLM-driven behavior generation and negotiation for collective robotic construction
Slezak, S., Siriwardena, L., Leder, S., & Menges, A. (2026) · Construction Robotics · doi:10.1007/s41693-026-00216-x
“The 200,000-token context window allows extended reasoning across multiple simulation runs, though longer contexts risk accuracy degradation (Liu et al. 2024).”
Designing Proactive Memory Privacy Visualization and Management in LLM-based ChatbotsPreprint
Zhang, S., Ye, L., Li, S., Xing, H., Jingyu, T., Shui, B., Nie, K., liu, P., Yi, X., & Li, H. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7288197
Lan, J., Li, F., & Zhang, L. N. (2026) · International Conference on Communications, Information System and Computer Engineering (CISCE) · doi:10.1109/cisce69494.2026.11504788
Nadaf, S., & Chavan, U. B. (2025) · International Conference for Advancement in Technology (ICONAT) · doi:10.1109/iconat66879.2025.11362770
Divide and Conquer: A Cascaded Architecture for Relation Extraction on Long-Tailed Datasets
Rahimi, P., & Bidgoli, B. M. (2026) · International Interdisciplinary Conference on Artificial Intelligence: Engineering, Health, Finance and Humanities (IICAI) · doi:10.1109/iicai70155.2026.11620810
Kim, H., Moon, J., & Baik, N. (2026) · Electronics · doi:10.3390/electronics15163643 · Cited by 1
RAGMoE: Retrieval Augmented Generation with Parameter Updating based on Mixture of ExpertsPreprint
Liu, X., Qiao, S., & Na, S. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5420653
Guo, C., Zheng, J., Su, P., Tian, Z., Tang, J., Li, S., & Wang, T. (2026) · Information Processing & Management · doi:10.1016/j.ipm.2026.105112
Podstawski, M., & Кamola, M. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-03711-4_12
Financialized ESG InformationPreprint
Chen, Y.-C., Lin, T.-C., & Zhang, Q. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6728902
NSync: Automated Cloud Infrastructure-as-Code Reconciliation with AI Agents
Yang, Z., Guan, H., Nicolet, V., Paulsen, B., Dodds, J., Kroening, D., & Chen, A. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3832121 · Cited by 10
“The learning process is agent-driven : the agent invokes the tools knowledge_update and knowledge_retrieval when it judges that the current context is complete and accurate enough to warrant persistence for reuse, as opposed to always logging every intermediate action.”
Mirhosseini, N., Shojaei, D., & Sabri, S. (2026) · Buildings · doi:10.3390/buildings16173432 · Cited by 2
Evaluating Visual Prompts with Eye-Tracking Data for MLLM-Based Human Activity Recognition
Choi, J. Y., Kim, S. G., Yoon, H., Lee, T., Lee, D., Chung, J., Kil, J., Rossi, R., Lee, S.-J., & Lee, T. Y. (2026) · IEEE Pacific Visualization Conference (PacificVis) · doi:10.1109/pacificvis68791.2026.00023
“While providing extended temporal context is beneficial for these complex activities, textual prompting suffers from the “lost in the middle” [8] and reduced computational efficiency.”
ConfigTransLE: Large Language Models Enhanced Network Configuration Translation
Li, F., Zheng, N., Liang, B., Yang, Y., Liu, C., Gao, C., Wang, X., & Cao, J. (2026) · IEEE Transactions on Networking · doi:10.1109/ton.2026.3716703
“However, LLMs have limitations on input text length [14], and their performance noticeably declines as the input text becomes longer [15].”
Guan, W., Liu, H., Li, X., Zhou, J. J., & Yan, Y. (2025) · International Conference on Computer Supported Cooperative Work in Design (CSCWD) · doi:10.1109/cscwd64889.2025.11033484 · Cited by 1
“…in ascending order; b) sorting the text chunks by importance in descending order; c) sorting the text chunks according to their positions in the original long text ; d) interleaving the text chunks at the front and back ends of the sequence alternately by importance in ascending order [21].”
SAGE-QA: Shared-context agentic graph-enhanced question answering for scientific reasoning
Hsu, Y.-C., & Buehler, M. J. (2026) · MRS Communications · doi:10.1557/s43579-026-01036-4 · Cited by 1
S2-CAR: Segmentation-Supervised Complexity-Adaptive RecommendationPreprint
Guo, L., Bisht, N., Wu, S., et al. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6989767
OSAgent: Copiloting Operating System with LLM-based Agent
Xu, J., Guo, K., Gong, W., & Shi, R. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10650304 · Cited by 11
“In fact, it is difficult to effectively improve model performance by drastically increasing the context length [19].”
BackgroundCopy That? The Influence of Process Model Representation Format on LLM Comprehension
Rybinski, F., Kruse, N., Wilhelm, A., & Fritsch, A. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-40377-3_3
Patient-Facing Radiology Communication with LLMs: Calibration Deficit and the Metadata Paradox
Shin, C., Park, J. H., Kim, S. J., Lee, Y. H., & Lee, H. S. (2026) · Healthcare · doi:10.3390/healthcare14111490 · Cited by 2
“Our secondary aims were: (1) to quantify the impact of patient metadata [22]—a phenomenon we term the ‘Metadata Paradox’—and (2) to investigate the calibration deficit between self-confidence and accuracy to propose strategies for safe clinical deployment [23,24].”
Zheng, H., Wu, H., & He, Z. (2025) · Proceedings - ACM IEEE Design Automation Conference · doi:10.1109/dac63849.2025.11132969
“However, LLMs have significant limitations when processing long-context inputs, which affects their effectiveness in analyzing large codebases [14] and executing lengthy reasoning sequences [15].”
Liu, M., Zheng, D., Su, J., & Wu, C. (2026) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3770855.3817484
“(1) InstanceNorm: the known variables are ’A’ with the dtype torch.float16 and shape [67, 27, 77, 10], ’D’ with the dtype torch.float16 and shape [27], ’E’ with the dtype torch.float16 and shape [27], and ’C’ with the dtype torch.float16 and shape [67, 27, 77, 10].”
Vantaku, C., & Avakian, A. (2026) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-026-02326-y
Detecting Code-Comment Inconsistencies in Smart Contracts by Combining LLM and Program Analysis
Zhang, J., Chen, J., Zhang, T., Li, Y., Wu, D., Wang, Y., Gao, J., Chen, T., & Chen, Z. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3808112
“Specifically, the complexity of real-world smart contracts and the scattered information related to a specific detection task can cause LLMs to become “lost in the middle” [24], leading to high false negatives.”
Saßmannshausen, T. M., & Wagener, S. (2026) · Qeios · doi:10.32388/gag6kd
Triple-S: A Collaborative Multi-LLM Framework for Solving Long-Horizon Implicative Tasks in Robotics
Jia, Z., Gao, H., Li, F., Liu, J., Li, H., & Liu, Q. (2025) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros60139.2025.11246810 · Cited by 1
“In-Context Learning (ICL) offers a training-free paradigm [17] that enables LLMs to generalize by leveraging semantic similarity [11], [18] and environmental feedback [19], [20].”
SmartGen: Synthesizing Context-Aware User Behavior Data for Adaptive Smart Home Intelligence
Xu, Z., Zhao, D., Zou, Q., Li, Q., Jiang, Y., Wang, Y., & Xiao, J. (2026) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3770854.3780267 · Cited by 1
“Second, lengthy inputs may cause LLMs to lose focus on salient information—a phenomenon known as the “lost-in-the-middle” effect [20]—which can undermine the plausibility and coherence of the generated behaviors.”
Lim, H. (2026) · Journal of Youth Impact · doi:10.66245/jyi.v1.i1.002 · Cited by 1
Internalizing persona knowledge in small language models via multi-objective contrastive learning
Cho, S., Park, S., & Kim, J. H. (2026) · Knowledge-Based Systems · doi:10.1016/j.knosys.2026.116335
MedConf-RAG: Reliable Medical LLMs with Gated Graph Integration and Conformal Factuality Guarantees
Huang, X., Bu, Y., Zhang, N., Zhang, Y., Sheng, H., & Li, Y. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-2856-0_39 · Cited by 1
Generative AI Technologies, Techniques and Tensions: A Primer
Behrens, J. T. (2026) · Springer eBooks · doi:10.1007/978-3-032-24819-0_2 · Cited by 2
Knowledge Editing of Large Language Models Unconstrained by Word Order
Ishigaki, R., Suzuki, J., Shuzo, M., & Maeda, E. (2025) · Journal of Natural Language Processing · doi:10.5715/jnlp.32.1062
Ding, R., Li, Q., & Li, X. (2026) · SAE technical papers on CD-ROM/SAE technical paper series · doi:10.4271/2026-99-0750
Joint Modeling of Typicality and Uncertainty for Sot-Based Few-Shot LLM Reasoning
Wang, B., Wei, P., Ren, Z., Li, S., Zhang, J., & Chen, S. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11464177
Antichi, L., Sajno, E., Rossi, C., Frisone, F., Salve, F. D., Oasi, O., & Riva, G. (2026) · Current Psychology · doi:10.1007/s12144-026-09685-3
“Additionally, LLMs pay less attention to information in the middle of long inputs, and prompt compression methods, which make prompts denser with relevant information, can maintain or even improve performance on shorter texts (Jiang et al., 2023; Liu et al., 2024).”
Labeling and Feature Engineering
HAWKINS, J. (2026) · doi:10.1002/9781394406432.ch7
Cheng, Z., Lai, L., Liu, Y., & Sun, Y. (2026) · Research Square · doi:10.2139/ssrn.6698538
HMT: Hierarchical Mamba-Transformer for Efficient Long-Sequence Text Classification
Xie, L., Wu, Y., Cui, T., He, L., & Du, Z. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3426-4_10
Latent Retrieval Augmented Generation
Zhou, S., Chen, J., Ling, R., Fan, T., & Wang, H. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3809870
Rojas, H., Larraín, A. F., Sanhueza, D., Mery, D., & Dunstan, J. (2026) · Tapuya Latin American Science Technology and Society · doi:10.1080/25729861.2026.2698211 · Cited by 3
Large Language Models for Optimizing Patient Recruitment Decisions in Voice Data Generation Projects
Anibal, J., Nama, G. K. C., Ganesh, S., Nafii, Y., Cruz, S. S., Daoud, V., Zanin, M., Le, T., Khorrami, P., Wood, B. J., et al. (2026) · Laryngoscope Investigative Otolaryngology · doi:10.1002/lio2.70519
“Model uncertainties could then be used as low-dimensional representations of complex health states, reducing the risk of downstream LLM performance degradation caused by overly lengthy inputs [76].”
Liu, X., Sun, B., Yu, H., Li, P., Zhao, R., Ji, H., Song, G., Li, P., & Wang, C. (2026) · Applied Energy · doi:10.1016/j.apenergy.2026.128417 · Cited by 1
Metadata
- Authors
- Liu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., & Liang, P.
- Affiliations
- Stanford University (US)
- University of California, Berkeley (US)
- Samaya AI, UK
- Corresponding author
- Liu, Nelson F.
- Venue
- Transactions of the Association for Computational Linguistics
- Publisher
- MIT Press
- Published
- 2024
- Volume
- 12
- Pages
- 157-173
- ISSN
- 2307-387X (ISSN-L)
- OpenAlex
- W4391876619
- Semantic Scholar
- 1733eb7792f7a43dd21f51f4d1017a1bffd217b5
- Cited by
- 1,612
- References
- 59
- Citations by year
- 2026: 901 · 2025: 516 · 2024: 172
- FWCI
- 233.95
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- Locations
- Transactions of the Association for Computational Linguistics (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- long-context language models, multi-document question answering, position bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-09-27 (prompt extract-v2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.