Lost in the Middle: How Language Models Use Long Contexts
Open accessThe DOI matches its Crossref record; 48 of the 59 references it cites were found in a registry or an index; 15 of the 23 sources its claims cite were checked for support.
Abstract
While recent language models have the ability to take long contexts as input, relatively little is known about how well they use longer context. We analyze the performance of language models on two tasks that require identifying relevant information in their input contexts: multi-document question answering and key-value retrieval. We find that performance can degrade significantly when changing the position of relevant information, indicating that current language models do not robustly make use of information in long input contexts. In particular, we observe that performance is often highest when relevant information occurs at the beginning or end of the input context, and significantly degrades when models must access relevant information in the middle of long contexts, even for explicitly long-context models. Our analysis provides a better understanding of how language models use their input context and provides new evaluation protocols for future long-context language models.
Overview
CoChat Summary
TL;DR
Language models’ performance on tasks requiring access to long input contexts is strongly affected by where relevant information appears, with a U-shaped curve: best at the beginning or end and worst in the middle. Extended-context models are not necessarily better at using context, and reader performance saturates before retriever recall in open-domain QA.
Key findings
- In multi-document QA, performance is highest when the answer-containing document is at the start or end of the context and degrades in the middle; GPT-3.5-Turbo can drop by more than 20%, and in 20- and 30-document settings middle-position performance is lower than its closed-book performance of 56.1% (oracle: 88.3%).
- Extended-context models are not necessarily better at using input context: when the input fits both windows, GPT-3.5-Turbo and GPT-3.5-Turbo(16K) perform nearly identically as a function of relevant-information position.
- In synthetic key-value retrieval, Claude-1.3 and Claude-1.3(100K) do nearly perfectly, while other models struggle especially with 140 or 300 key-value pairs; query-aware contextualization gives GPT-3.5-Turbo(16K) perfect performance at 300 pairs, versus worst-case 45.6% without it.
- In open-domain QA, reader performance saturates long before retriever recall: using 50 instead of 20 retrieved documents only marginally improves performance (∼1.5% for GPT-3.5-Turbo and ∼1% for Claude-1.3).
Methods
- Controlled multi-document QA on NaturalQuestions-Open: 2,655 queries whose annotated long answer is a paragraph; each context has exactly one answer-containing Wikipedia passage plus k−1 Contriever/MS-MARCO distractor passages; the paper varies answer position and context length (10, 20, 30 documents) and measures accuracy.
- Synthetic key-value retrieval: inputs are serialized JSON objects with unique random UUID key-value pairs; models must return the value for a specified key; experiments use 75, 140, and 300 key-value pairs (500 examples each), varying key position and number of pairs, with accuracy as the metric.
- Models include open MPT-30B-Instruct and LongChat-13B(16K), closed GPT-3.5-Turbo, GPT-3.5-Turbo(16K), Claude-1.3, and Claude-1.3(100K), plus additional architecture/instruction experiments with Flan-T5-XXL, Flan-UL2, MPT-30B, Llama-2, and GPT-4; an open-domain QA case study uses a Contriever retriever-reader setup.
Limitations
- Stated: the paper uses greedy decoding and leaves exploration of other decoding methods to future work; the investigations into why models are not robust are described as preliminary; GPT-4 is evaluated only on a subset of 500 multi-document QA examples because full evaluation would cost upwards of $6000.
- Stated: NaturalQuestions-Open has ambiguity, so the authors run an unambiguous subset and also test random distractors and randomized distractor order, finding similar trends.
- Evident: the controlled multi-document QA setting always contains exactly one document that answers the question, unlike open-domain QA; the synthetic key-value task removes natural language semantics; results may not generalize to all tasks, models, or decoding settings.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
7 Valid · 0 Not valid · 5 Not checked · 13 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Language models have become an important and flexible building block in a variety of user-facing language technologies, including conversational interfaces, search and summarization, and collaborative writing (Shuster et al., 2022;Thoppilan et al., 2022;Lee et al., 2022, inter alia).”
§ Lost in the Middle: How Language Models Use Long Contexts > 1 Introduction
What the sources say
- Partly supportedcites Shuster et al., 2022 sourceExperimental — accuracy not yet measured
Section 3.1.1 (p. 4) describes a conversational agent built on a language model that generates internet search queries, supporting conversational interfaces and search as part of dialogue. Section 2 (p. 3) mentions summarization only as related work, and the passages do not establish collaborative writing as an application of this paper’s model. (LLM, conf=0.85)
- Supportedcites Thoppilan et al., 2022 source
The Abstract (p. 1) identifies LaMDA as a family of language models specialized for dialog. In this sentence's list of applications, Thoppilan et al. supports the conversational-interface example; the other cited works need not each cover every example. (LLM, conf=0.92)
- Supportedcites Lee et al., 2022 source
Lee et al. is cited alongside other works, and its contribution to the sentence is the collaborative-writing example. Section 4.2.2 describes writers receiving and selecting model-generated suggestions in a writing interface; Section 2.2.1 also describes language models as applicable to dialogue and summarization. The paper need not establish every example in the jointly cited list. (LLM, conf=0.89)
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“These observations complement prior work, which found that non-instruction fine-tuned language models are biased towards recent tokens (i.e., the end of the input context; Khandelwal et al., 2018;Press et al., 2021).”
§ Lost in the Middle: How Language Models Use Long Contexts > 4.3 Effect of Instruction Fine-Tuning
What the sources say
- Partly supportedcites Khandelwal et al., 2018 sourceExperimental — accuracy not yet measured
The Introduction and Section 5 report that the studied LSTM is much more sensitive to recent word order than to distant word order, supporting a recency effect. The paper studies a standard LSTM on two datasets, however, rather than establishing this tendency across non-instruction-fine-tuned language models generally; it also finds that distant words remain relevant. (LLM, conf=0.82)
- Partly supportedcites Press et al., 2021 sourceExperimental — accuracy not yet measured
Section 3.1 (page not available in our copy) finds that very short contexts hurt performance, while increasing subsequence length from 1,024 to 3,072 tokens gives negligible improvement. That is consistent with limited benefit from more distant context, but the paper does not establish that the models are biased toward recent tokens within an input context. (LLM, conf=0.78)
- Partly supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“We instantiate this task with data from NaturalQuestions-Open (Lee et al., 2019;Kwiatkowski et al., 2019), which contains historical queries issued to the Google search engine, coupled with human-annotated answers extracted from Wikipedia.”
§ Lost in the Middle: How Language Models Use Long Contexts > 2.1 Experimental Setup
What the sources say
- Supportedcites Lee et al., 2019 source
Section 7.1 says the paper creates an open version of Natural Questions and uses it for training and evaluation. The Introduction describes the task as answering questions by retrieving evidence from an open corpus, supporting the cited use of NaturalQuestions-Open for this task. (LLM, conf=0.91)
- Not checked — no full text of the sourcecites Kwiatkowski et al., 2019 source
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Following Kandpal et al. (2022) and Mallen et al. (2023), we use accuracy as our primary evaluation metric, judging whether any of the cor- rect answers (as taken from the NaturalQuestions annotations) appear in the predicted output.”
§ Lost in the Middle: How Language Models Use Long Contexts > 2.1 Experimental Setup
What the sources say
- Supportedcites Kandpal et al. (2022) source
Section 3 (p. 4) says the paper evaluates answers using exact match against a ground-truth answer set, and Section 3.1 (p. 4) reports QA accuracies as its evaluation results. This supports citing Kandpal et al. for using accuracy to evaluate question answering. (LLM, conf=0.90)
- Supportedcites Mallen et al. (2023) source
Section 3.1 (‘Focus and Task’) explicitly names accuracy as the metric and says a prediction is correct when it contains an exact match to any gold answer. This supports the evaluation approach attributed to Mallen et al.; the NaturalQuestions annotations describe the citing authors’ answer source. (LLM, conf=0.94)
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“or when language models are augmented with external information (relevant documents from a search engine, database query results, etc.; Petroni et al., 2020;Ram et al., 2023;Shi et al., 2023;Mallen et al., 2023;Schick et al., 2023, inter alia).”
§ Lost in the Middle: How Language Models Use Long Contexts > 1 Introduction
What the sources say
- Partly supportedcites Petroni et al., 2020 sourceExperimental — accuracy not yet measured
The Abstract (p. 1) says the paper integrates information from a retrieval system with a pre-trained language model, and §3.4.2 (p. 5) specifies a retrieved paragraph as context. This supports augmentation with retrieved documents, but our copy does not show language models being augmented with database query results. (LLM, conf=0.84)
- Partly supportedcites Ram et al., 2023 sourceExperimental — accuracy not yet measured
The Introduction describes augmenting language models with relevant documents retrieved from an external knowledge source, and Section 5 describes using a BM25 retriever to find those documents. The passages do not establish database query results as another type of external information used by this paper. (LLM, conf=0.82)
- Partly supportedcites Shi et al., 2023 sourceExperimental — accuracy not yet measured
Section 3.1 (page unavailable) describes retrieving relevant documents from a corpus, and the Conclusion says REPLUG uses those documents to augment a language model. The paper supports the document-retrieval example, but our copy does not describe augmenting models with database query results or documents from a search engine specifically. (LLM, conf=0.82)
- Partly supported
20 more claims · 2 with their cited sources checked
Create a free account to see all 25 claims and every source verdict.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In vanilla multi-document or NIAH-based benchmarks (Kamradt, 2023; Liu et al., 2024), models perform consistently well when the needle (or gold document) appears at the very beginning or end of the context window, with minimal impact from context length.”
From NoLiMa: Long-Context Evaluation Beyond Literal Matching (2025) · cites this paper as Liu et al., 2024
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Even within a single long sequence, attention effectiveness degrades with distance due to attention dilution , positional encoding limitations, and token interference, leading to the well-known 'lost-in-the-middle' and context-decay phenomena (Liu et al., 2024; Press et al., 2021a).”
From MAGMA: A Multi-Graph based Agentic Memory Architecture for AI Agents (2026) · cites this paper as Liu et al., 2024
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Current large language models (LLMs) exhibit significant deficiencies in episodic memory tasks including encoding, storing, and retrieving specific information from temporally dependent events over a long period of time.”
From ARTEM: Enhancing Large Language Model Agents with Spatial-Temporal Episodic Memory (2026) · cites this paper as Liu et al. 2024
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Recent approaches to handle memory tasks in LLMs, such as in-context learning, retrieval-augmented generation (RAG), and fine-tuning, may resolve the long-term retention issues, but are still inadequate to handle tasks requiring chronological awareness of the stored information.”
From ARTEM: Enhancing Large Language Model Agents with Spatial-Temporal Episodic Memory (2026) · cites this paper as Liu et al. 2024
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“If not ranked at the top, the key information may be neglected by the generative module, resulting in incomplete or inaccurate responses to the user queries24.”
From Leveraging long context in retrieval augmented language models for medical question answering (2025) · cites this paper as [24]
Experimental — accuracy not yet measured
86 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929
LLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602
Transformer-XL: Attentive language models beyond a fixed-length context
Dai, Z., Yang, Z., Yang, Y., Carbonell, J., Le, Q., & Salakhutdinov, R. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1285 · Cited by 3,169 · cited 2×
“Recent improvements in hardware (e.g., faster GPUs with more memory) and algorithms (Dai et al., 2019; Dao et al., 2022; Poli et al., 2023; Rubin and Berant, 2023, inter alia ) have resulted in language models with larger context windows (e.g., 4096, 32K, and even 100K tokens), but it remains…”
BackgroundLlama 2: Open Foundation and Fine-Tuned Chat Models
Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.09288 · Cited by 2,604
Longformer: The Long-Document Transformer
Beltagy, I., Peters, M. E., & Cohan, A. (2020) · arXiv · doi:10.48550/arxiv.2004.05150 · Cited by 2,281
Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering
Izacard, G., & Grave, E. (2020) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.48550/arxiv.2007.01282 · Cited by 2,263
Natural Questions: A benchmark for question answering research
Kwiatkowski, T., Palomaki, J., Redfield, O., Collins, M., Parikh, A., Alberti, C., Epstein, D., Polosukhin, I., Devlin, J., Lee, K., et al. (2019) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00276 · Cited by 2,069 · cited 1×
“We instantiate this task with data from NaturalQuestions-Open (Lee et al., 2019; Kwiatkowski et al., 2019), which contains historical queries issued to the Google search engine, coupled with human-annotated answers extracted from Wikipedia.”
MethodThe serial position effect of free recall.
Murdock, B. B. (1962) · Journal of Experimental Psychology · doi:10.1037/h0045106 · Cited by 1,717 · cited 1×
“The U-shaped curve we observe in this work has a connection in psychology known as the serial-position effect (Ebbinghaus, 1913; Murdock Jr, 1962), that states that in free-association recall of elements from a list, humans tend to best re-member the first and last elements of the list.”
MethodMemory: A contribution to experimental psychology.
Ebbinghaus, H. (1913) · doi:10.1037/10011-000 · Cited by 1,645 · cited 1×
“The U-shaped curve we observe in this work has a connection in psychology known as the serial-position effect (Ebbinghaus, 1913; Murdock Jr, 1962), that states that in free-association recall of elements from a list, humans tend to best re-member the first and last elements of the list.”
Method
Research works printed but not linked: 1
- Sharan, 2018, Prediction with a short memory, 10.18653/v1/K18-3013 · DOI
Cited by
Items 301–400 of 1,597
Meta-Cognitive Memory Policy Optimization for Long-Horizon LLM Agents
Liu, Z., Hao, Z., Chen, Y., Wang, H., Hou, J., Ding, R., Yang, Y., Ji, W., Xia, W., & Liu, F. (2026) · arXiv · Cited by 12
In-context Autoencoder for Context Compression in a Large Language Model
Ge, T., Hu, J., Wang, L., Wang, X., Chen, S.-Q., & Wei, F. (2023) · arXiv · doi:10.48550/arxiv.2307.06945 · Cited by 11
“, 2023) attempts to tackle the long context issue through architectural innovations of an LLM, which still poses difficulties: While the complexity of computation and memory can be much reduced by some approximate methods, overcoming the notable decline in performance on long contexts remains a challenge (Liu et al.,…”
BackgroundATOM: AdapTive and OptiMized dynamic temporal knowledge graph construction using LLMs
Lairgi, Y., Moncla, L., Benabdeslem, K., Cazabet, R., & Cléau, P. (2026) · Findings of the Association for Computational Linguistics: EACL 2026 · doi:10.18653/v1/2026.findings-eacl.49 · Cited by 7
Large language models for biological sequence analysis in infectious disease research
Luo, J., Cai, X., & Li, Y. (2025) · Biosafety and Health · doi:10.1016/j.bsheal.2025.09.007 · Cited by 9
Bootstrapping Cognitive Agents with a Large Language Model
Zhu, F., & Simmons, R. G. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i1.27822 · Cited by 23
“…two benefits: 1) it allows each query to the LLM to be of reasonable length ( ∼ 5 k tokens), preventing LLMs from losing focus on lengthy prompts (Liu et al. 2023b); and 2) it facilitates a modular design, which enables generating code from English descriptions generated from other sources,…”
BackgroundUsing large language models to analyze political texts through natural language understanding
Benoit, K. R., de Marchi, S., Laver, C., Laver, M., & Ma, J. (2026) · American Journal of Political Science · doi:10.1111/ajps.70050 · Cited by 37
METIS: Fast Quality-Aware RAG Systems with Configuration Adaptation
Ray, S., Pan, R., Gu, Z., Du, K., Feng, S., Ananthanarayanan, G., Netravali, R., & Jiang, J. (2025) · ACM SIGOPS Symposium on Operating Systems Principles (SOSP) · doi:10.1145/3731569.3764855 · Cited by 52
Empowering Social Service with AI: Insights from a Participatory Design Study with Practitioners
Tan, Y., Soh, K. X., Zhang, R., Lee, J., Meng, H., Sen, B., & Lee, Y. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3719736 · Cited by 22
SelfCP: Compressing over-limit prompt via the frozen large language model itself
Gao, J., Cao, Z., & Li, W. (2024) · Information Processing & Management · doi:10.1016/j.ipm.2024.103873 · Cited by 20
Zeng, T., Wu, L., Shi, L., Zhou, D., & Guo, F. (2025) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3711896.3737396 · Cited by 21
Resume Screening Using Large Language Models
Salakar, E., Rai, J., Salian, A., Shah, Y., & Wadmare, J. G. (2023) · International Conference on Advances in Science and Technology (ICAST) · doi:10.1109/icast59062.2023.10454984 · Cited by 9
An Advanced Retrieval-Augmented Generation System for Manufacturing Quality ControlPreprint
Heredia, J. A., & Barreda, J. G. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4828863 · Cited by 6
FinPersona: An LLM-Driven Conversational Agent for Personalized Financial Advising
Takayanagi, T., Suzuki, M., Izumi, K., Sanz-Cruzado, J., McCreadie, R., & Ounis, I. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-88720-8_3 · Cited by 6
“Additionally, incorporating multiple stocks would lead to longer prompts for the LLM, leading to difficulties in its ability to handle them effectively [9].”
A Chat about Boring Problems: Studying GPT-Based Text Normalization
Zhang, Y., Bartley, T., Graterol-Fuenmayor, M., Lavrukhin, V., Bakhturina, E., & Ginsburg, B. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10447169 · Cited by 13
“We surmise this stems from ’lost in the middle’ phenomenon [23]: context knowledge for normalization may be lost over critically large token spans.”
BackgroundSilico-driven drug discovery: A paradigm shift for nanomedicine science and industry
Jin, Y., Zhou, Y., Xu, Z., Jin, Z., Meng, H., Li, S., Yan, L., Wang, H., Zheng, J., Gao, X., et al. (2025) · Nano Today · doi:10.1016/j.nantod.2025.102918 · Cited by 8
Embodied LLM Agents Learn to Cooperate in Organized Teams
Guo, X., Huang, K., Liu, J., Fan, W., Vélez, N., Wu, Q., Wang, H., Griffiths, T. L., & Wang, M. (2026) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2025.3637527 · Cited by 123
“Modern LLMs are prone to over-reporting and obeying instructions, as a by-product of RLHF finetuning [2], and they can ignore critical information [28] or be distracted by irrelevant information [46], especially when the context is long (see Figure 1 for examples).”
BackgroundLLM4SZZ: Enhancing SZZ Algorithm with Context-Enhanced Assessment on Large Language Models
Tang, L., Liu, J., Liu, Z., Yang, X., & Bao, L. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3728885 · Cited by 24
“Previous studies [35] reveal that LLMs tend to ignore content in the middle of text when handling long texts.”
Garín, D., Cook, S., Ferry, C., Bennar, W., Togni, M., Meier, P., Wenaweser, P., Puricel, S., & Arroyo, D. (2025) · European Heart Journal - Digital Health · doi:10.1093/ehjdh/ztaf068 · Cited by 11
“33,34 Removing or adding guided-thinking in prompts containing nine examples produced no discernible difference.”
Thakur, N., Pradeep, R., Upadhyay, S., Campos, D., Craswell, N., Soboroff, I. M., Dang, H. T., & Lin, J. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730165 · Cited by 16
“RAG systems improve factuality and verifiability, reducing hallucinations observed in “closed-book” LLM generation [7, 10, 11, 13, 21].”
ChatGPT on campus: how top U.S. universities govern generative AI across higher education
Alba, C., McIlwain, C. D., & An, R. (2026) · Policy Reviews in Higher Education · doi:10.1080/23322969.2026.2622674 · Cited by 15
“The goal of this iterative shuffling is to mitigate the lost-in-the-middle phenomenon (Liu et al. 2024), where LLMs tend to ignore content in the middle of lengthy sequences.”
Unveiling Challenges for LLMs in Enterprise Data Engineering
Bodensohn, J.-M., Brackmann, U., Vogel, L., Sanghi, A., & Binnig, C. (2025) · Proceedings of the VLDB Endowment · doi:10.14778/3773749.3773758 · Cited by 23
“First, we observe a drastic decline in performance as the number of customers increases, suggesting that the model struggles with long-range dependencies and finding matches in large datasets.”
Give LLMs a Security Course: Securing Retrieval-Augmented Code Generation via Knowledge Injection
Lin, B., Wang, S., Qin, Y., Chen, L., & Mao, X. (2025) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3719027.3765049 · Cited by 41
“(a) Vulnerable code generated when a poisoned example is retrieved.”
Extracting Information from Brazilian Legal Documents with Retrieval Augmented Generation
de Aquino, I. V., dos Santos, M. M., Dorneles, C. F., & Carvalho, J. T. (2024) · Simpósio Brasileiro de Bancos de Dados (SBBD) · doi:10.5753/sbbd_estendido.2024.244241 · Cited by 14
RaDiT: A Differential Transformer-Based Hybrid Deep Learning Model for Radar Echo Extrapolation
Zhu, W., Lu, Z., Zhang, Y., Zhao, Z., Lu, B., & Li, R. (2025) · Remote Sensing · doi:10.3390/rs17121976 · Cited by 7
Jiang, C., Zhang, P., Ni, Y., Wang, X., Peng, H., Liu, S., Fei, M., He, Y., Xiao, Y., Huang, J., et al. (2025) · The Visual Computer · doi:10.1007/s00371-025-03829-5 · Cited by 16
“This limitation results in the problem of information loss, leading to the “lost in the middle” phenomenon where central content can be overlooked or misinterpreted due to attention limitations [7].”
DomainLynx: Advancing LLM Techniques for Robust Domain Squatting Detection
Chiba, D., Nakano, H., & Koide, T. (2025) · IEEE Access · doi:10.1109/access.2025.3542036 · Cited by 11
“For placement, these must-pass domains are strategically inserted into each input chunk to ensure they are evenly distributed and to mitigate the ‘‘lost in the middle’’ phenomenon [28], where LLMs may overlook information in the center of large inputs.”
How is language intelligence evolving? A multi-dimensional survey of large language models
Chen, X., Zhou, L., Chen, J., Wang, G., & Li, X. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.130637 · Cited by 11
Barno, E., & Phelps, G. (2025) · Education Sciences · doi:10.3390/educsci15101273 · Cited by 6
“…aim to initiate the development of specific requirements needed to create a theoretically sound and pedagogically supportive rehearsal for students, we now introduce a framework that applies this expertise to be digestible beyond interpretations within a single prompt (e.g., Liu et al., 2024).”
MCeT: Behavioral Model Correctness Evaluation using Large Language Models
Ahmed, K., Song, J., Chen, B.-Q., Wei, O., & Zheng, B. (2025) · ACM/IEEE International Conference on Model Driven Engineering Languages and Systems · doi:10.1109/models67397.2025.00014 · Cited by 10
“When the diagram and requirements are long, the underlying LLM often overlooks details in the prompt which includes both the diagram and the requirements, a phenomenon common for LLMs [31], and thus falsely reports that the diagram is not complete.”
L-Eval: Instituting Standardized Evaluation for Long Context Language Models
An, C., Gong, S., Zhong, M., Zhao, X., Li, M., Zhang, J., Kong, L., & Qiu, X. (2023) · arXiv · doi:10.48550/arxiv.2307.11088 · Cited by 6
“Previous work (Li et al., 2023a; Liu et al., 2023) has used retrieval tasks to test the ability of modeling long context dependency via retrieving something over lengthy context.”
BackgroundMethodSnapKV: LLM Knows What You are Looking for Before Generation
Li, Y., Huang, Y., Yang, B., Venkitesh, B., Locatelli, A., Ye, H., Cai, T., Lewis, P., & Chen, D. (2024) · arXiv · doi:10.48550/arxiv.2404.14469 · Cited by 74
“Lost-in-the-middle is aimed to analyze whether the performance of the model varies when altering the position of ground-truth information in the context [24].”
BackgroundA Simple and Effective $L_2$ Norm-Based Strategy for KV Cache Compression
Devoto, A., Zhao, Y., Scardapane, S., & Minervini, P. (2024) · arXiv · doi:10.48550/arxiv.2406.11430 · Cited by 6
U-NIAH: Unified RAG and LLM Evaluation for Long Context Needle-In-A-Haystack
Gao, Y., Xiong, Y., Wu, W., Huang, Z., Li, B., & Wang, H. (2025) · arXiv · doi:10.48550/arxiv.2503.00353 · Cited by 6
Lin, T.-L., Chen, W.-C., Hsiao, T.-F., Liu, H.-I., Yeh, Y.-H., Chan, Y.-K., Lien, W.-S., Kuo, P.-Y., Yu, P. S., & Shuai, H.-H. (2025) · Findings of the Association for Computational Linguistics: EMNLP 2025 · doi:10.18653/v1/2025.findings-emnlp.259 · Cited by 6
Fang, J., Peng, Y., Zhang, X., Wang, Y., Yi, X., Zhang, G., Xu, Y., Wu, B., Liu, S., Li, Z., et al. (2025) · arXiv · doi:10.48550/arxiv.2508.07407 · Cited by 6
What Does YouTube Advise Students About Bypassing AI-Text Detection Tools? A Pragmatic Analysis
Foltýnek, T., & Newton, P. M. (2025) · Journal of Academic Ethics · doi:10.1007/s10805-025-09675-3 · Cited by 7
“(Liu et al., 2023 ) and found that model performance degrades significantly when relevant information is in the middle of long contexts, with better performance for information at the beginning or end.”
Gilani, K., Verket, M., Peters, C., Dumontier, M., Rocca, H.-P. B.-L., & Urovi, V. (2025) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2025.110745 · Cited by 5
Preference-Guided Refactored Tuning for Retrieval Augmented Code Generation
Gao, X., Xiong, Y., Wang, D., Guan, Z., Shi, Z., Wang, H., & Li, S. (2024) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3691620.3694987 · Cited by 29
“This not only increases inference costs and risks getting lost in middle [25], but also results in the loss of critical information due to exceeding the context length limit for smaller models.”
How the Risk of Exploitation in Human-Ai Relationships Depends on Relationship TypePreprint
Earp, B. D., Feroz, F., Mann, S. P., Voinea, C., Chalson, S., Jurčys, P., Reinecke, M. G., Savulescu, J., Singh, I., & Clark, M. S. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5288102 · Cited by 7
Hierarchical Indexing for Retrieval-Augmented Opinion Summarization
Hosking, T., Tang, H., & Lapata, M. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00703 · Cited by 13
“Models with long context windows have been proposed (Beltagy et al., 2020; Gu et al., 2022) but these are not currently instruction-tuned, and it has been shown that LLMs are biased toward information at the start and end of the input (Liu et al., 2023).”
Duan, Z., Gu, H., Ke, Y., & Zhou, D. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.112118 · Cited by 9
Enhancing Self-Regulated Learning Using Generative AI: Development and Evaluation of SRLMentor
Wang, S., Luo, X., & Hew, K. F. (2025) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2025.3634216 · Cited by 8
“Despite the capabilities of newer LLMs to handle larger data volumes, overloading them can degrade performance and cause errors or hallucinations [44].”
Integração de Modelos de Linguagem e RAG na Criação de Chatbots Oftalmológicos
Passinato, E. B., Rios, W. S. R., & Filho, A. R. G. (2024) · doi:10.5753/sbcas.2024.2228 · Cited by 9
Zhang, W., Chen, Y., Liu, X., & Zhang, H. (2025) · Land Use Policy · doi:10.1016/j.landusepol.2025.107807 · Cited by 10
SumBlogger: Abstractive Summarization of Large Collections of Scientific Articles
Zakkas, P., Verberne, S., & Zavrel, J. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-56027-9_23 · Cited by 16
Towards Explainability in Retrieval-Augmented LLMs
Rorseth, J., Godfrey, P., Golab, L., Srivastava, D., & Szlichta, J. (2024) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde60146.2024.00466 · Cited by 6
“Furthermore, recent research has uncovered a “lost in the middle” bias in many popular LLMs [3], where the consideration given to sources can be influenced by their number and relative order in the prompt.”
BackgroundAutomatically Fixing Dependency Breaking Changes
Fruntke, L., & Krinke, J. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3729366 · Cited by 27
“Since the prompt produces rather high input token usage, common LLM issues such as confusion due to long contexts [43, 59] could artificially deflate performance.”
Rank-Without-GPT: Building GPT-Independent Listwise Rerankers on Open-Source Large Language Models
Zhang, C., Hofstätter, S., Lewis, P., Tang, R., & Lin, J. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-88711-6_15 · Cited by 6
Improving the Readability of Automatically Generated Tests Using Large Language Models
Biagiola, M., Ghislotti, G., & Tonella, P. (2025) · IEEE International Conference on Software Testing, Verification and Validation (ICST) · doi:10.1109/icst62969.2025.10989020 · Cited by 20
“Moreover, long prompts can be affected by lost-in-the-middle effects [25], where the model’s performance degrades when it needs to access information in the middle of long prompts; (2) LLMs have a limited context window size, measured in the number of tokens making up the input prompt.”
P rome F uzz : A Knowledge-Driven Approach to Fuzzing Harness Generation with Large Language Models
Liu, Y., Deng, J., Jia, X., Wang, Y., Wang, M., Huang, L., Wei, T., & Su, P. (2025) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3719027.3765222 · Cited by 26
Menke, J. D., Ming, S., Radhakrishna, S., Kilicoglu, H., & Smalheiser, N. R. (2025) · medRxiv · doi:10.1101/2025.04.23.25326300 · Cited by 4
“This is important, as models often struggle with long-context memorization in summarization, which can result in hallucinations or the generation of irrelevant content [45].”
Wang, S., Zhang, Y., Gao, Y., He, X., Deng, G., & Du, J. (2025) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaf219 · Cited by 6
“These results were supported by LLM reviewers (Supplementary Tables 11 and 12).”
Wang, G., Liu, Y., Liu, S., Zhang, L., & Yang, L. (2025) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-025-02570-8 · Cited by 16
“A recall length that is too long may increase the model’s difficulty in understanding, while a recall length that is too short may result in insufficient information [21].”
Liu, Z., Kou, J., Zhang, W., Gu, C., Fang, X., Huang, Z., Hao, Y., Li, H., Lu, X., Yin, A., et al. (2024) · International Conference on Universal Village (UV) · doi:10.1109/uv63228.2024.11189137 · Cited by 12
Cue RAG: Dynamic multi-output cue memory under H framework for retrieval-augmented generation
Fu, Y., Liu, D., Zhang, B., Jiang, Z., Mei, H., & Guan, J. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.130235 · Cited by 12
Zeng, Y., Ding, X., Cai, B., Qin, B., & Liu, T. (2026) · Information Processing & Management · doi:10.1016/j.ipm.2026.104659 · Cited by 10
Ali, F., Tan, A. S.-C., & Wang, S. J.-W. (2025) · Research Synthesis Methods · doi:10.1017/rsm.2024.16 · Cited by 9
“18 21 However, there is a lack of systematic investigation using various datasets while variances in performance have been unexplained.”
U-NIAH: Unified RAG and LLM Evaluation for Long Context Needle-in-a-Haystack
Gao, Y., Xiong, Y., Wu, W., Li, B., Zhong, Y., & Wang, H. (2026) · ACM Transactions on Information Systems · doi:10.1145/3786609 · Cited by 6
“…windows faces critical challenges in computational efficiency and information retrieval accuracy, especially when handling complex queries in lengthy documents, and research shows that model performance drops rapidly in ultra-long contexts, with the“Lost in the middle" phenomenon occurring [19].”
Schindler, C., & Rausch, A. (2024) · Electronics · doi:10.3390/electronics13050816 · Cited by 6
LLM-Enriched Finite-State Chatbots for Mental Health Support: A Case Study on Self-Help+
Sanna, L., Bolpagni, M., Fietta, V., Gavioli, G., Franzin, M., Dragoni, M., & Gabrielli, S. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-95838-0_38 · Cited by 7
Harnessing Generative AI to Facilitate Epistemic Understanding of Students in Scientific Inquiry
Chang, J., Park, J., & Sim, J. Y. (2026) · Journal of Science Education and Technology · doi:10.1007/s10956-026-10298-5 · Cited by 9
“Since model performance may degrade as the length of the input context increases (Liu et al., 2023), student responses were input in batches of 15 responses at a time to generate scoring results and feedback.”
Sharfuddin, A., & Breaux, T. D. (2025) · IEEE International Requirements Engineering Conference (RE) · doi:10.1109/re63999.2025.00019 · Cited by 8
“One concern, however, is that transformer-based models appear to lose attention in the middle of large input contexts [22] (see Section II), called a lead and/or trailing bias [23], which could result in goals being missed in the extracted goal list.”
RAG Powered LLMs for QA: Evolution, Challenges, Applications, and Future Directions
Zeeshan, H. M. A., Faizan, M., Zia, U., & Gohar, A. S. (2025) · ComTech · doi:10.1109/comtech65062.2025.11034531 · Cited by 9
“Inputting all the content directly copied to the LLM is not a good idea because the content may be redundant but there’s a mistake that it may ignore the middle, this is what is called “Lost in the middle” in [10].”
Manufacturing Domain QA with Integrated Term Enhanced RAG
Bei, Y., Fang, Z., Mao, S., Yu, S., Jiang, Y., Tong, Y., & Cai, W. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10649905 · Cited by 11
“When the context is very long, the model will ignore the useful knowledge in the middle part of the context and only pay attention to the beginning and end [35].”
BackgroundFINCH: Prompt-guided Key-Value Cache Compression for Large Language Models
Corallo, G., & Papotti, P. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00716 · Cited by 5
IOAgent: Democratizing Trustworthy HPC I/O Performance Diagnosis Capability via LLMs
Egersdoerfer, C., Sareen, A., Bez, J. L., Byna, S., Xu, D., & Dai, D. (2025) · IEEE International Parallel and Distributed Processing Symposium (IPDPS) · doi:10.1109/ipdps64566.2025.00036 · Cited by 11
“Such cases demonstrate IOAgent’s unique potential to effectively assist domain scientists at scale.”
Retrieval-Augmented Chain-of-Thought in Semi-structured Domains
Mavi, V., Saparov, A., & Zhao, C. (2023) · Natural Legal Language Processing Workshop (NLLP) · doi:10.18653/v1/2023.nllp-1.18 · Cited by 15
SCM: Enhancing Large Language Model with Self-Controlled Memory Framework
Wang, B., Liang, X., Yang, J., Huang, H., Wu, S., Wu, P., Lu, L., Ma, Z., & Li, Z. (2023) · arXiv · doi:10.48550/arxiv.2304.13343 · Cited by 5
Large language models for peer review in biotechnology
Li, W., Zhao, R., Sun, P.-T., Lee, J., Lu, O., Yuan, S., Sureshkumar, P., Chen, Y., Xiao, Z., & Tang, Y. J. (2026) · New Biotechnology · doi:10.1016/j.nbt.2026.03.007 · Cited by 7
UniPar: A Unified LLM-Based Framework for Parallel and Accelerated Code Translation in HPC
Bitan, T., Kadosh, T., Kaplan, E., Meiri, S., Chen, L., Morales, P., Hasabnis, N., & Oren, G. (2025) · IEEE High Performance Extreme Computing Conference · doi:10.1109/hpec67600.2025.11196677 · Cited by 8
“Prior studies have shown that LLM performance may degrade when processing long or unbalanced inputs, as the context window becomes saturated or loses focus [45].”
(2025) · doi:10.1145/3737412.3743489 · Cited by 4
Proceedings of the Workshop on Edge and Mobile Foundation Models
(2024) · doi:10.1145/3662006 · Cited by 4
MUNet: A lightweight Mamba-based Under-Display Camera restoration network
Wang, W., Li, B., Liu, W., Peng, X., & Gou, Y. (2025) · Image and Vision Computing · doi:10.1016/j.imavis.2025.105486 · Cited by 6
Knowledge-augmented Methods for Natural Language Generation
Jiang, M., Lin, B. Y., Wang, S., Yi‐chong, X., Yu, W., & Zhu, C. (2024) · SpringerBriefs in computer science · doi:10.1007/978-981-97-0747-8_4 · Cited by 7
Li, Z., Husung, S., & Wang, H. (2025) · IEEE International Symposium on Systems Engineering · doi:10.1109/isse65546.2025.11369983 · Cited by 8
“Restrictions apply. consistency, but can increase processing latency, cognitive overload on the LLM, and output instability due to attention degradation and token limits [42].”
HyperG: Hypergraph-Enhanced LLMs for Structured Knowledge
Huang, S., Li, H., Gu, Y., Hu, X., Li, Q., & Xu, G. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730002 · Cited by 16
“Scalability is a critical concern, as large tables pose significant challenges for LLMs, which often struggle to interpret and integrate context from lengthy prompts [22, 44].”
Music Recommendation with Large Language Models: Challenges, Opportunities, and Evaluation
Epure, E., Deldjoo, Y., Sguerra, B., Schedl, M., & Moussallam, M. (2026) · ACM Transactions on Recommender Systems · doi:10.1145/3844509 · Cited by 13
Tuna: Instruction Tuning using Feedback from Large Language Models
Li, H., Liu, Y., Zhang, X., Lu, W., & Wei, F. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.1011 · Cited by 19
Querying Templatized Document Collections with Large Language Models
Lin, Y., Hulsebos, M., Ma, R., Shankar, S., Zeighami, S., Parameswaran, A., & Wu, E. (2025) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde65448.2025.00183 · Cited by 24
“The error-prone nature of LLMs is not surprising given that LLMs can’t effectively handle large contexts [9], [10], or complex data processing tasks [11], [12].”
COMET: Generating commit messages using delta graph context representation
Mandli, A. R., Rajput, S., & Sharma, T. (2024) · Journal of Systems and Software · doi:10.1016/j.jss.2024.112307 · Cited by 9
Quality Evaluation of Summarization Models for Patent Documents
Ding, J., Chen, H., Kolapudi, S. T., Pobbathi, L., & Nguyen, H. (2023) · IEEE International Conference on Software Quality, Reliability and Security (QRS) · doi:10.1109/qrs60937.2023.00033 · Cited by 9
“For example, if an abstract and detailed description of the abstract are included in one document, then the detailed description could be totally ignored in the summarization [18].”
InfluentialSheik, R., Reji, S. A., Sharon, A., Avisha, M., & Nirmala, S. J. (2025) · Artificial Intelligence and Law · doi:10.1007/s10506-025-09459-5 · Cited by 5
Okadome, Y., Yuguchi, A., Fukui, R., & Matsumoto, Y. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72356-8_3 · Cited by 5
A Survey on Retrieval-Augmented Generation Technology in Large Language Models
Guo, J., Yang, H., & Liu, Q. (2026) · Communications in computer and information science · doi:10.1007/978-981-95-4788-3_23 · Cited by 8
Giuliani, C., Benadi, G., Engel, F., Werner, J., Watter, M., Schwarzer, G., Groß, O., Zeiser, R., Binder, H., & Kaier, K. (2025) · JMIR Formative Research · doi:10.2196/73822 · Cited by 6
Saleh, A. O. M., Tür, G., & Saygın, Y. (2025) · Machine Learning and Knowledge Extraction · doi:10.3390/make7030074 · Cited by 9
Detrimental Contexts in Open-Domain Question Answering
Oh, P., & Thorne, J. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.776 · Cited by 16
Bianchi, M. G., D'addario, A. M. V., Junior, P. F. G.-B., Machado, B. S., Agondi, R. C., Almeida, S. K. A., Junior, W. X. A., Armelin, L. M., Aun, M. V., Bordignon, N., et al. (2024) · Journal of Allergy and Clinical Immunology Global · doi:10.1016/j.jacig.2024.100373 · Cited by 9
Enhancing Document-Level Translation of Large Language Model Via Translation Mixed-InstructionsPreprint
Li, Y., Li, J., Jiang, J. L., & Zhang, M. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4871785 · Cited by 4
Zhang, Y., Dong, Z., & Xu, W. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.130441 · Cited by 10
Aldana-Martín, J. F., Durillo, J. J., Roldán‐García, M. D. M., & Nebro, A. J. (2025) · Engineering Optimization · doi:10.1080/0305215x.2025.2544215 · Cited by 4
“This result highlights a known limitation of LLMs: the probability of errors increases as the length of the input and generated output grows (N. F. Liu et al. 2024).”
LLM Augmentations to support Analytical Reasoning over Multiple Documents
Yousuf, R. B., Defelice, N., Sharma, M., Xu, S., & Ramakrishnan, N. (2024) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata62323.2024.10826114 · Cited by 12
“Our finding is consistent with those of recent research, e.g., needle in the haystack [41], lost in the middle [42] studies.”
de Wynter, A. (2024) · IEEE Transactions on Games · doi:10.1109/tg.2024.3497601 · Cited by 22
“Missing enemies could be due in part to the long history fed into the Agent and the lost-in-the-middle problem [42], where LLMs place too much emphasis on the first and last sections of the prompt, and ignore (forget) the centre.”
Yuan, C., Wei, T., Li, C., Yi, X., Liu, S., Zhang, Z., Cai, Y., & Du, X. (2026) · Journal of King Saud University - Computer and Information Sciences · doi:10.1007/s44443-026-00708-4 · Cited by 7
Malo in the Code Jungle: Explainable Fault Localization for Decentralized Applications
Zhang, H., Wu, J., Wu, Z., Chen, Z., Lin, D., Chen, J., Zhou, Y., & Zheng, Z. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3578816 · Cited by 5
“This decline may be related to two factors: 1) excessive context exploration introduces irrelevant function information; 2) long contexts negatively affect the performance of the LLM [35].”
InfluentialHao, J., Chen, L., & Meng, L. (2025) · Drones · doi:10.3390/drones9050361 · Cited by 10
Shah, S. T. U., Hussein, M., Barcomb, A., & Moshirpour, M. (2026) · Automated Software Engineering · doi:10.1007/s10515-025-00587-5 · Cited by 6
“This range balances pro - viding sufficient context for informed decision-making while avoiding information overload that can degrade LLM performance (Liu et al. 2024).”
Automating Database-Native Function Code Synthesis with LLMs
Zhou, W., Zhou, X., He, Q., Li, G., He, B., Xu, Q., & Wu, F. (2026) · Proceedings of the ACM on Management of Data · doi:10.1145/3802018 · Cited by 5
StoryWriter: A Multi-Agent Framework for Long Story Generation
Xia, H., Peng, H., Qi, Y., Xu, B., Li, J., Lei, H., & Wang, X. (2025) · Proceedings of the 34th ACM International Conference on Information and Knowledge Management · doi:10.1145/3746252.3761616 · Cited by 4
Metadata
- Authors
- Liu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., & Liang, P.
- Affiliations
- Stanford University (US)
- University of California, Berkeley (US)
- Samaya AI, UK
- Corresponding author
- Liu, Nelson F.
- Venue
- Transactions of the Association for Computational Linguistics
- Publisher
- MIT Press
- Published
- 2024
- Volume
- 12
- Pages
- 157-173
- ISSN
- 2307-387X (ISSN-L)
- OpenAlex
- W4391876619
- Semantic Scholar
- 1733eb7792f7a43dd21f51f4d1017a1bffd217b5
- Cited by
- 1,612
- References
- 59
- Citations by year
- 2026: 901 · 2025: 516 · 2024: 172
- FWCI
- 233.95
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- Locations
- Transactions of the Association for Computational Linguistics (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- long-context language models, multi-document question answering, position bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-09-27 (prompt extract-v2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.