Lost in the Middle: How Language Models Use Long Contexts
Open accessThe DOI matches its Crossref record; 48 of the 59 references it cites were found in a registry or an index; 15 of the 23 sources its claims cite were checked for support.
Abstract
While recent language models have the ability to take long contexts as input, relatively little is known about how well they use longer context. We analyze the performance of language models on two tasks that require identifying relevant information in their input contexts: multi-document question answering and key-value retrieval. We find that performance can degrade significantly when changing the position of relevant information, indicating that current language models do not robustly make use of information in long input contexts. In particular, we observe that performance is often highest when relevant information occurs at the beginning or end of the input context, and significantly degrades when models must access relevant information in the middle of long contexts, even for explicitly long-context models. Our analysis provides a better understanding of how language models use their input context and provides new evaluation protocols for future long-context language models.
Overview
CoChat Summary
TL;DR
Language models’ performance on tasks requiring access to long input contexts is strongly affected by where relevant information appears, with a U-shaped curve: best at the beginning or end and worst in the middle. Extended-context models are not necessarily better at using context, and reader performance saturates before retriever recall in open-domain QA.
Key findings
- In multi-document QA, performance is highest when the answer-containing document is at the start or end of the context and degrades in the middle; GPT-3.5-Turbo can drop by more than 20%, and in 20- and 30-document settings middle-position performance is lower than its closed-book performance of 56.1% (oracle: 88.3%).
- Extended-context models are not necessarily better at using input context: when the input fits both windows, GPT-3.5-Turbo and GPT-3.5-Turbo(16K) perform nearly identically as a function of relevant-information position.
- In synthetic key-value retrieval, Claude-1.3 and Claude-1.3(100K) do nearly perfectly, while other models struggle especially with 140 or 300 key-value pairs; query-aware contextualization gives GPT-3.5-Turbo(16K) perfect performance at 300 pairs, versus worst-case 45.6% without it.
- In open-domain QA, reader performance saturates long before retriever recall: using 50 instead of 20 retrieved documents only marginally improves performance (∼1.5% for GPT-3.5-Turbo and ∼1% for Claude-1.3).
Methods
- Controlled multi-document QA on NaturalQuestions-Open: 2,655 queries whose annotated long answer is a paragraph; each context has exactly one answer-containing Wikipedia passage plus k−1 Contriever/MS-MARCO distractor passages; the paper varies answer position and context length (10, 20, 30 documents) and measures accuracy.
- Synthetic key-value retrieval: inputs are serialized JSON objects with unique random UUID key-value pairs; models must return the value for a specified key; experiments use 75, 140, and 300 key-value pairs (500 examples each), varying key position and number of pairs, with accuracy as the metric.
- Models include open MPT-30B-Instruct and LongChat-13B(16K), closed GPT-3.5-Turbo, GPT-3.5-Turbo(16K), Claude-1.3, and Claude-1.3(100K), plus additional architecture/instruction experiments with Flan-T5-XXL, Flan-UL2, MPT-30B, Llama-2, and GPT-4; an open-domain QA case study uses a Contriever retriever-reader setup.
Limitations
- Stated: the paper uses greedy decoding and leaves exploration of other decoding methods to future work; the investigations into why models are not robust are described as preliminary; GPT-4 is evaluated only on a subset of 500 multi-document QA examples because full evaluation would cost upwards of $6000.
- Stated: NaturalQuestions-Open has ambiguity, so the authors run an unambiguous subset and also test random distractors and randomized distractor order, finding similar trends.
- Evident: the controlled multi-document QA setting always contains exactly one document that answers the question, unlike open-domain QA; the synthetic key-value task removes natural language semantics; results may not generalize to all tasks, models, or decoding settings.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
7 Valid · 0 Not valid · 5 Not checked · 13 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Language models have become an important and flexible building block in a variety of user-facing language technologies, including conversational interfaces, search and summarization, and collaborative writing (Shuster et al., 2022;Thoppilan et al., 2022;Lee et al., 2022, inter alia).”
§ Lost in the Middle: How Language Models Use Long Contexts > 1 Introduction
What the sources say
- Partly supportedcites Shuster et al., 2022 sourceExperimental — accuracy not yet measured
Section 3.1.1 (p. 4) describes a conversational agent built on a language model that generates internet search queries, supporting conversational interfaces and search as part of dialogue. Section 2 (p. 3) mentions summarization only as related work, and the passages do not establish collaborative writing as an application of this paper’s model. (LLM, conf=0.85)
- Supportedcites Thoppilan et al., 2022 source
The Abstract (p. 1) identifies LaMDA as a family of language models specialized for dialog. In this sentence's list of applications, Thoppilan et al. supports the conversational-interface example; the other cited works need not each cover every example. (LLM, conf=0.92)
- Supportedcites Lee et al., 2022 source
Lee et al. is cited alongside other works, and its contribution to the sentence is the collaborative-writing example. Section 4.2.2 describes writers receiving and selecting model-generated suggestions in a writing interface; Section 2.2.1 also describes language models as applicable to dialogue and summarization. The paper need not establish every example in the jointly cited list. (LLM, conf=0.89)
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“These observations complement prior work, which found that non-instruction fine-tuned language models are biased towards recent tokens (i.e., the end of the input context; Khandelwal et al., 2018;Press et al., 2021).”
§ Lost in the Middle: How Language Models Use Long Contexts > 4.3 Effect of Instruction Fine-Tuning
What the sources say
- Partly supportedcites Khandelwal et al., 2018 sourceExperimental — accuracy not yet measured
The Introduction and Section 5 report that the studied LSTM is much more sensitive to recent word order than to distant word order, supporting a recency effect. The paper studies a standard LSTM on two datasets, however, rather than establishing this tendency across non-instruction-fine-tuned language models generally; it also finds that distant words remain relevant. (LLM, conf=0.82)
- Partly supportedcites Press et al., 2021 sourceExperimental — accuracy not yet measured
Section 3.1 (page not available in our copy) finds that very short contexts hurt performance, while increasing subsequence length from 1,024 to 3,072 tokens gives negligible improvement. That is consistent with limited benefit from more distant context, but the paper does not establish that the models are biased toward recent tokens within an input context. (LLM, conf=0.78)
- Partly supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“We instantiate this task with data from NaturalQuestions-Open (Lee et al., 2019;Kwiatkowski et al., 2019), which contains historical queries issued to the Google search engine, coupled with human-annotated answers extracted from Wikipedia.”
§ Lost in the Middle: How Language Models Use Long Contexts > 2.1 Experimental Setup
What the sources say
- Supportedcites Lee et al., 2019 source
Section 7.1 says the paper creates an open version of Natural Questions and uses it for training and evaluation. The Introduction describes the task as answering questions by retrieving evidence from an open corpus, supporting the cited use of NaturalQuestions-Open for this task. (LLM, conf=0.91)
- Not checked — no full text of the sourcecites Kwiatkowski et al., 2019 source
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Following Kandpal et al. (2022) and Mallen et al. (2023), we use accuracy as our primary evaluation metric, judging whether any of the cor- rect answers (as taken from the NaturalQuestions annotations) appear in the predicted output.”
§ Lost in the Middle: How Language Models Use Long Contexts > 2.1 Experimental Setup
What the sources say
- Supportedcites Kandpal et al. (2022) source
Section 3 (p. 4) says the paper evaluates answers using exact match against a ground-truth answer set, and Section 3.1 (p. 4) reports QA accuracies as its evaluation results. This supports citing Kandpal et al. for using accuracy to evaluate question answering. (LLM, conf=0.90)
- Supportedcites Mallen et al. (2023) source
Section 3.1 (‘Focus and Task’) explicitly names accuracy as the metric and says a prediction is correct when it contains an exact match to any gold answer. This supports the evaluation approach attributed to Mallen et al.; the NaturalQuestions annotations describe the citing authors’ answer source. (LLM, conf=0.94)
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“or when language models are augmented with external information (relevant documents from a search engine, database query results, etc.; Petroni et al., 2020;Ram et al., 2023;Shi et al., 2023;Mallen et al., 2023;Schick et al., 2023, inter alia).”
§ Lost in the Middle: How Language Models Use Long Contexts > 1 Introduction
What the sources say
- Partly supportedcites Petroni et al., 2020 sourceExperimental — accuracy not yet measured
The Abstract (p. 1) says the paper integrates information from a retrieval system with a pre-trained language model, and §3.4.2 (p. 5) specifies a retrieved paragraph as context. This supports augmentation with retrieved documents, but our copy does not show language models being augmented with database query results. (LLM, conf=0.84)
- Partly supportedcites Ram et al., 2023 sourceExperimental — accuracy not yet measured
The Introduction describes augmenting language models with relevant documents retrieved from an external knowledge source, and Section 5 describes using a BM25 retriever to find those documents. The passages do not establish database query results as another type of external information used by this paper. (LLM, conf=0.82)
- Partly supportedcites Shi et al., 2023 sourceExperimental — accuracy not yet measured
Section 3.1 (page unavailable) describes retrieving relevant documents from a corpus, and the Conclusion says REPLUG uses those documents to augment a language model. The paper supports the document-retrieval example, but our copy does not describe augmenting models with database query results or documents from a search engine specifically. (LLM, conf=0.82)
- Partly supported
20 more claims · 2 with their cited sources checked
Create a free account to see all 25 claims and every source verdict.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In vanilla multi-document or NIAH-based benchmarks (Kamradt, 2023; Liu et al., 2024), models perform consistently well when the needle (or gold document) appears at the very beginning or end of the context window, with minimal impact from context length.”
From NoLiMa: Long-Context Evaluation Beyond Literal Matching (2025) · cites this paper as Liu et al., 2024
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Even within a single long sequence, attention effectiveness degrades with distance due to attention dilution , positional encoding limitations, and token interference, leading to the well-known 'lost-in-the-middle' and context-decay phenomena (Liu et al., 2024; Press et al., 2021a).”
From MAGMA: A Multi-Graph based Agentic Memory Architecture for AI Agents (2026) · cites this paper as Liu et al., 2024
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Current large language models (LLMs) exhibit significant deficiencies in episodic memory tasks including encoding, storing, and retrieving specific information from temporally dependent events over a long period of time.”
From ARTEM: Enhancing Large Language Model Agents with Spatial-Temporal Episodic Memory (2026) · cites this paper as Liu et al. 2024
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Recent approaches to handle memory tasks in LLMs, such as in-context learning, retrieval-augmented generation (RAG), and fine-tuning, may resolve the long-term retention issues, but are still inadequate to handle tasks requiring chronological awareness of the stored information.”
From ARTEM: Enhancing Large Language Model Agents with Spatial-Temporal Episodic Memory (2026) · cites this paper as Liu et al. 2024
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“If not ranked at the top, the key information may be neglected by the generative module, resulting in incomplete or inaccurate responses to the user queries24.”
From Leveraging long context in retrieval augmented language models for medical question answering (2025) · cites this paper as [24]
Experimental — accuracy not yet measured
86 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929
LLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602
Transformer-XL: Attentive language models beyond a fixed-length context
Dai, Z., Yang, Z., Yang, Y., Carbonell, J., Le, Q., & Salakhutdinov, R. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1285 · Cited by 3,169 · cited 2×
“Recent improvements in hardware (e.g., faster GPUs with more memory) and algorithms (Dai et al., 2019; Dao et al., 2022; Poli et al., 2023; Rubin and Berant, 2023, inter alia ) have resulted in language models with larger context windows (e.g., 4096, 32K, and even 100K tokens), but it remains…”
BackgroundLlama 2: Open Foundation and Fine-Tuned Chat Models
Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.09288 · Cited by 2,604
Longformer: The Long-Document Transformer
Beltagy, I., Peters, M. E., & Cohan, A. (2020) · arXiv · doi:10.48550/arxiv.2004.05150 · Cited by 2,281
Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering
Izacard, G., & Grave, E. (2020) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.48550/arxiv.2007.01282 · Cited by 2,263
Natural Questions: A benchmark for question answering research
Kwiatkowski, T., Palomaki, J., Redfield, O., Collins, M., Parikh, A., Alberti, C., Epstein, D., Polosukhin, I., Devlin, J., Lee, K., et al. (2019) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00276 · Cited by 2,069 · cited 1×
“We instantiate this task with data from NaturalQuestions-Open (Lee et al., 2019; Kwiatkowski et al., 2019), which contains historical queries issued to the Google search engine, coupled with human-annotated answers extracted from Wikipedia.”
MethodThe serial position effect of free recall.
Murdock, B. B. (1962) · Journal of Experimental Psychology · doi:10.1037/h0045106 · Cited by 1,717 · cited 1×
“The U-shaped curve we observe in this work has a connection in psychology known as the serial-position effect (Ebbinghaus, 1913; Murdock Jr, 1962), that states that in free-association recall of elements from a list, humans tend to best re-member the first and last elements of the list.”
MethodMemory: A contribution to experimental psychology.
Ebbinghaus, H. (1913) · doi:10.1037/10011-000 · Cited by 1,645 · cited 1×
“The U-shaped curve we observe in this work has a connection in psychology known as the serial-position effect (Ebbinghaus, 1913; Murdock Jr, 1962), that states that in free-association recall of elements from a list, humans tend to best re-member the first and last elements of the list.”
Method
Research works printed but not linked: 1
- Sharan, 2018, Prediction with a short memory, 10.18653/v1/K18-3013 · DOI
Cited by
Items 401–500 of 1,597
AI chatbots in higher education: Comparing expectations to evidence
Thoeni, A., & Fryer, L. K. (2026) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2026.101061 · Cited by 4
Reddy, V., & Veeranjaneyulu, N. (2025) · Journal of Intelligent Information Systems · doi:10.1007/s10844-025-00959-z · Cited by 4
“…models (LLMs), enhance forecasting, sentiment analysis, and intent classification (Bahoo et al., 2024), but face scalability issues under multi-user loads, compliance challenges due to inadequate validation, and limited real-world validation (Liu et al., 2024; Parimi, 2024; Riyazahmed, 2021).”
Toward Sustainable Agentic AI Systems: A Survey of Architectures and Methodologies
Lee, Y., & Park, E. (2026) · Sustainable Development · doi:10.1002/sd.70942 · Cited by 13
LENS: Layers of Evaluation of Hallucination in GenAI Systems
Gu, C., Zhang, W., Huang, Z., Kou, J., Liu, Z., Zhao, C., Liu, C., Zhang, L., Lin, W., Wang, Z., et al. (2024) · International Conference on Universal Village (UV) · doi:10.1109/uv63228.2024.11189150 · Cited by 36
Action Detail Matters: Refining Video Recognition with Local Action Queries
Wang, M., Huang, Z., Kong, X., Shen, G., Dai, G., Wang, J., & Liu, Y. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01782 · Cited by 20
“However, recent studies [24, 46] have shown that Transformers struggle to accurately retrieve key information due to “attention noise,” which reduces the efficiency of key feature extraction and hinders performance.”
In-Context Learning Enables Robot Action Prediction in LLMs
Yin, Y., Wang, Z., Sharma, Y., Niu, D., Darrell, T., & Herzig, R. (2025) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra55743.2025.11128807 · Cited by 43
“Moreover, as shown in [10], [11], simply adding a large number of inputs across a long context might lead to hallucinations and degraded performance.”
ChatNVD: Advancing Cybersecurity Vulnerability Assessment With Large Language Models
Chopra, S., Ahmad, H., Goel, D., & Szabo, C. (2026) · IEEE Access · doi:10.1109/access.2026.3659205 · Cited by 26
“This pattern aligns with what recent studies refer to as the “lost-in-the-middle” phenomenon [85], wherein transformer-based models struggle to retain and prioritize information positioned in the center of long inputs.”
LongCodeZip: Compress Long Context for Code Language Models
Shi, Y., Qian, Y., Zhang, H., Shen, B., & Gu, X. (2025) · 2025 40th IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase63991.2025.00020 · Cited by 7
“Despite increasingly longer context windows, LLMs exhibit significant limitations in long context scenarios, especially when relevant information appears in the middle of a prompt [19], [20] or when code completion requires cross-function or structural dependencies [21], [61].”
Palaniappan, S., Mali, R., Cuomo, L., Vitale, M., Youssef, A., Madathil, A. P., Murugesan, M., Bettini, A., Magistris, G. D., & Veneri, G. (2024) · ADIPEC · doi:10.2118/222032-ms · Cited by 4
Cents: A Flexible and Cost-Effective Framework for LLM-Based Table Understanding
Xiao, G., Dong, H., Wang, J., & Bałazińska, M. (2025) · Proceedings of the VLDB Endowment · doi:10.14778/3749646.3749714 · Cited by 10
Masoud, A. A., Arazzi, M., & Nocera, A. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.133154 · Cited by 17
Szoke, D. R., Hutzler, I., Liu, J., Addante, S., Akhtar, Z., Smith, D. L., Dickens, K., Small, C., Pridgen, S., & Held, P. (2026) · JMIR Mental Health · doi:10.2196/91367 · Cited by 5
“Short exchanges may lack the information needed to disambiguate risk from figurative language, whereas long conversations can obscure critical risk signals in a “needle-in-a-haystack” problem [27] or be exploited through multiturn adversarial attacks [18].”
Şahin, M. Z., Derdiyok, F. B., Kılıç, S. A., Serbest, K., & Nas, K. (2026) · Bioengineering · doi:10.3390/bioengineering13010125 · Cited by 5
“A chunk size of 3000 characters is sufficiently large to encapsulate a complete clinical concept or argument, yet small enough to operate efficiently within the context window of modern LLMs, mitigating issues related to the model’s attenuated attention over very long contexts [32].”
Beyond Topic Modeling: Analyzing Historical Text with Large Language Models
Kuznetsov, A. V. (2024) · Историческая информатика · doi:10.7256/2585-7797.2024.4.72560 · Cited by 7
Li, Y., Fu, X., Verma, G., Buitelaar, P., & Liu, M. (2025) · arXiv · doi:10.48550/arxiv.2510.24476 · Cited by 4
“Document reranking aims to optimize the selection and ordering of retrieved results within the context window, often prioritizing the most relevant content at the beginning or end of the input to maximize its impact on generation [74].”
ChunkKV: Semantic-Preserving KV Cache Compression for Efficient Long-Context LLM Inference
Liu, X., Tang, Z., Dong, P., Li, Z., Liu, Y., Li, B., Hu, X., & Chu, X. (2025) · arXiv · doi:10.48550/arxiv.2502.00299 · Cited by 4
The Compaction Cliff in Long-Running AI Agent Memory
Zerhoudi, S., Mitrović, J., & Granitzer, M. (2026) · ACM · doi:10.1145/3799682.3840567 · Cited by 4
NoLiMa: Long-Context Evaluation Beyond Literal Matching
Modarressi, A., Deilamsalehy, H., Dernoncourt, F., Bui, T., Rossi, R. A., Yoon, S., & Schütze, H. (2025) · arXiv · doi:10.48550/arxiv.2502.05167 · Cited by 3
CiteME: Can Language Models Accurately Cite Scientific Claims?
Press, O., Hochlehnert, A., Prabhu, A., Udandarao, V., Press, O., & Bethge, M. (2024) · arXiv · doi:10.48550/arxiv.2407.12861 · Cited by 4
“This could stem from LMs loss of important details when given a long context window [52].”
BackgroundInfluentialAgentic Trading: When LLM Agents Meet Financial Markets
Xia, Y., You, P., Wang, T., Liu, F., Qi, H., Wu, X., & Zhang, S. (2026) · arXiv · doi:10.48550/arxiv.2605.19337 · Cited by 4
GEM-RAG: Graphical Eigen Memories for Retrieval Augmented Generation
Rappazzo, B., Wang, Y., Ferber, A., & Gomes, C. P. (2024) · International Conference on Machine Learning and Applications · doi:10.1109/icmla61862.2024.00196 · Cited by 5
“Our first observation from psychology is the so called “testing effect”, which observes that if humans are tested on subject material, they are more likely to accurately remember it [16].”
Multi-Grained Patch Training for Efficient LLM-based Recommendation
Liao, J., Xie, R., Li, S., Wang, X., Sun, X., Kang, Z., & He, X. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730042 · Cited by 19
LABIIUM: AI-Enhanced Zero-configuration Measurement Automation System
Olowe, E. A., & Chitnis, D. (2025) · IEEE International Instrumentation and Measurement Technology Conference · doi:10.1109/i2mtc62753.2025.11079200 · Cited by 4
“A key challenge when working with LLM APIs can be working within the confines of their effective context windows, as LLMs can forget information deep within long contexts [17] and LLM performance may decline as large contexts are used [18].”
Speiser, K., Maciocci, G., Boukamp, F., & Teizer, J. (2026) · Advanced Engineering Informatics · doi:10.1016/j.aei.2026.104681 · Cited by 8
Design Process for Retrieval Augmented Generation Systems
Xu, X., Zhang, D., Zhang, W., Lu, Q., & Zhu, L. (2025) · IEEE International Conference on Software Architecture Companion (ICSA-C) · doi:10.1109/icsa-c65153.2025.00072 · Cited by 10
Leveraging LLMs for predictive insights in food policy and behavioral interventions
Kaiser, M., Lohmann, P. M., Ochieng, P., Shi, B., Sunstein, C. R., & Reisch, L. A. (2025) · Discover Food · doi:10.1007/s44187-025-00552-x · Cited by 6
“We attribute this behavior to two key factors: first, large language models may inefficiently utilize longer or noisier context distributions [48]; second, continual fine-tuning with small datasets may trigger catastrophic forgetting [47], ultimately degrading performance.”
InfluentialA Comparative Analysis of Linguistic and Retrieval Diversity in LLM-Generated Search Queries
Zendel, O., Lawati, S. F. D. A., Rashidi, L., Scholer, F., & Sanderson, M. (2025) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3746252.3761382 · Cited by 11
“Although GPT-4o can process inputs up to 128,000 tokens, we truncate each document to 1,500 words to reduce computation time and minimize context degradation issues, such as “lost in the middle” [28, 32].”
Integrating External Knowledge with LLMs: A Systematic Review of RAG Approaches
Mikulić, I., Vlaić, M., Delač, G., Šilić, M., & Vladimir, K. (2025) · International Convention on Information and Communication Technology, Electronics and Microelectronics · doi:10.1109/mipro65660.2025.11131735 · Cited by 6
“In addition to standard prompting techniques, such as using markdown for content separation, studies have highlighted the "lost in the middle" problem [88], where information placed in the middle of the context window is less effectively utilized than content at the beginning or end.”
Enhancing Large Language Model Inference Efficiency via Lookahead Cache Filtering
Ou, J., Chen, Y., Jiang, S., & Tian, W. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10888646 · Cited by 3
“We selected the two datasets, Multi-document Question Answering (MDQA) and Key-Value Retrieval (KV retrieval) tasks from Liu et al. [16].”
Pang, R. Y., Feng, K. J. K., Feng, S., Li, C., Shi, W., Tsvetkov, Y., Heer, J., & Reinecke, K. (2026) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3742413.3789091 · Cited by 40
Predicting Retrieval Utility and Answer Quality in Retrieval-Augmented Generation
Tian, F., Ganguly, D., & Macdonald, C. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-21289-4_24 · Cited by 15
Boolzen, C., Kühn, J., Flegr, S., Rott, E.-M., Stausberg, N., & Küchemann, S. (2026) · Artificial Intelligence Review · doi:10.1007/s10462-026-11665-9 · Cited by 17
Utilizing Past User Feedback for More Accurate Text-to-SQL
Urban, M., Ding, J., Kernert, D., Vaidya, K., & Kraska, T. (2025) · Workshop on Human-In-the-Loop Data Analytics (HILDA) · doi:10.1145/3736733.3736739 · Cited by 8
SkillScope: A Tool to Predict Fine-Grained Skills Needed to Solve Issues on GitHub
Carter, B. C., Contreras, J. R., Villegas, C. A. L., Acharya, P., Utzerath, J., Farner, A. O., Jenkins, H., Johnson, D., Penney, J., Steinmacher, I., et al. (2025) · IEEE/ACM International Workshop on Natural Language-Based Software Engineering (NLBSE) · doi:10.1109/nlbse66842.2025.00007 · Cited by 5
“Large Language Models (LLMs) have demonstrated strong capabilities in NLP and data classification tasks [7], [8], even though their performance can vary by task and domain [9], [10], prompt [11], and the context size provided [12].”
Chondamrongkul, N., Kyaw, M. P. P., Ko, S. M., Paing, P. P., Swe, M. K. T., & Hongthong, T. (2026) · Science of Computer Programming · doi:10.1016/j.scico.2026.103477 · Cited by 3
Enhancing Large Language Models for Automated Homework Assessment in Undergraduate Circuit Analysis
Chen, L., Xie, H., Qin, Z., Guo, Y., Rohde, J. A., & Zhang, Y. (2025) · IEEE Frontiers in Education Conference (FIE) · doi:10.1109/fie63693.2025.11328374 · Cited by 9
“However, a key drawback of this approach is that GPT-4o must consider all aspects of the assessment simultaneously, which can lead to a loss of focus and an increased likelihood of hallucinations [22].”
Analyzing and Adapting Large Language Models for Few-Shot Multilingual NLU: Are We There Yet?
Razumovskaia, E., Vulić, I., & Korhonen, A. (2025) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl.a.33 · Cited by 25
Xia, S.-J., Zhang, H., Jiang, Y., Chen, X., Chen, Y., Li, Z., Wan, Z., & Sun, A. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5731427 · Cited by 4
RAG-VR: Leveraging Retrieval-Augmented Generation for 3D Question Answering in VR Environments
Ding, S., & Chen, Y. (2025) · IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW) · doi:10.1109/vrw66409.2025.00034 · Cited by 7
“However, these methods [9, 23] use context data as the input (e.g., large amounts of texts, images) to LLMs, where relevant data may not be effectively and efficiently accessed within long input [16].”
Xu, Y., Chen, J., Peng, Z., Chen, Z., Lin, Q., Ma, L., Shi, B., & Dong, B. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3754821 · Cited by 10
DeepWind: A foundation model for zero-shot wind power forecasting
Wang, H., Zhou, T., Jia, C., Liu, Y., & Wang, L. (2026) · Energy · doi:10.1016/j.energy.2026.141793 · Cited by 4
IaraMed: A Women’s Healthcare Chatbot for Portuguese Speakers
Färber, F. B., Dollis, J. S., Ribeiro, P. S. F. B., Brito, I. A., Sousa, R. T., & Filho, A. R. G. (2025) · doi:10.5753/sbcas.2025.7897 · Cited by 6
Babaeipour, R., Charest, F., & Wright, M. (2026) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2026.105036 · Cited by 5
“We classify the information to be extracted into six broad categories: general information, inclusion/exclusion criteria, adverse event definitions, intervention, site requirements and schedule of events.”
When Retrieval Augmentation Meets API Documentation: Can LLMs Code with Less-Common Libraries?
Chen, J., Chen, S., Cao, J., Shen, J., & Cheung, S.-C. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3821424 · Cited by 3
“To learn the optimal RAG configurations, researchers evaluated RAG under diverse document types [11], document positions [36], and retriever choices [55].”
Multiscale Temporal-Variable Patches Network for Long-Term Prediction of Industrial Key Parameters
Lv, C., Zhou, X., Hu, G., Li, S., Liu, S., & Li, M. (2024) · IEEE Sensors Journal · doi:10.1109/jsen.2024.3453435 · Cited by 3
Wardle, G., & Sušnjak, T. (2025) · Big Data and Cognitive Computing · doi:10.3390/bdcc9060149 · Cited by 11
Integrating LLMs and Chatbots Technologies - A Case Study on Brazilian Transit Law
Rocho, R. S. M., Perez, A. L. F., Farias, G. P., & Panisson, A. R. (2024) · doi:10.5753/eniac.2024.245203 · Cited by 3
Li, T., Wang, J., Zhang, Y., Li, S., & Chen, L. (2025) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3711896.3736829 · Cited by 8
A First Look at AI Trends in Value-Aligned Software Engineering Publications: Human-LLM Insights
Mougouei, D., Azarnik, A., Fahmideh, M., Mougouei, E., Dam, H. K., Khan, A. A., Rafi, S., Khan, J. A., & Ahmad, A. (2025) · IEEE/ACM International Conference on Software Engineering: Software Engineering in Society (ICSE-SEIS) · doi:10.1109/icse-seis66351.2025.00014 · Cited by 6
(Generative) AI in Financial EconomicsPreprint
Mo, H., & Ouyang, S. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5287110 · Cited by 3
GraphiT: Efficient Node Classification on Text-Attributed Graphs with Prompt Optimized LLMs
Khoshraftar, S., Abedini, N., & Hajian, A. (2025) · International World Wide Web Conference (WWW) · doi:10.1145/3701716.3717805 · Cited by 3
“• We present GraphiT, a novel technique for graph encoding”
DR.FIX: Automatically Fixing Data Races at Industry Scale
Behrang, F., Zhang, Z., Saioc, G.-V., Liu, P., & Chabbi, M. (2025) · Proceedings of the ACM on Programming Languages · doi:10.1145/3729265 · Cited by 7
“Finally, the 193 fixes generated by Dr.Fix add up to producing 2.1K lines of fixes.”
Ghosh, S., & Shah, C. S. (2024) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.24974 · Cited by 4
“While recent advancements in large language models (LLMs) and conversational agents, such as ChatGPT, have significantly enhanced conversational capabilities, they still face challenges in ensuring alignment with user needs and context during complex or multi-step tasks (Liu et al., 2024).”
Evaluating Reinforcement Learning Based Models for Test Time Enhancement in RAG
Kamra, V., Gupta, L., Arora, D., & Yadav, A. K. (2025) · International Conference on Cognitive Robotics and Intelligent Systems (ICC-ROBINS) · doi:10.1109/icc-robins64345.2025.11086322 · Cited by 3
“This comprehensive evaluation setup is critical for understanding the strengths and limitations of reinforcement learning-based test time enhancement in RAG systems [23].”
Lan, Z., Sun, Y., Ren, J., Gao, H., & Xiao, G. (2025) · Nature Communications · doi:10.1038/s41467-025-66599-7 · Cited by 9
Ihalage, A. A., Taheri, S., Faris, M., & Al-Raweshidy, H. S. (2025) · IEEE Access · doi:10.1109/access.2025.3587029 · Cited by 5
“Recent findings indicate that even the long-context LLMs may exhibit suboptimal performance, particularly when relevant information is situated in the middle of the context window rather than at the beginning or end [39].”
Vision-as-Inverse-Graphics Agent via Interleaved Multimodal Reasoning
Yin, S., Ge, J., Wang, Z. Z., Wang, C., Li, X., Black, M. J., Darrell, T., Kanazawa, A., & Feng, H. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37464-6_30 · Cited by 69
Less is More for Long Document Summary Evaluation by LLMs
Iso, H. (2024) · Journal of Natural Language Processing · doi:10.5715/jnlp.31.1389 · Cited by 17
Reverse Engineering User Stories from Code using Large Language Models
Ouf, M., Li, H., Zhang, M., & Guizani, M. (2025) · IEEE International Conference on Collaborative Advances in Software and Computing (CASCON) · doi:10.1109/cascon66301.2025.00080 · Cited by 8
“The decreasing F1 in the 200+ NLOC range could be attributed to the “lost-in-the-middle” effect, where tokens in the middle of long prompts have less attention [34].”
InfluentialTableZoomer: a collaborative agent framework for large-scale table question answering
Xiong, S., He, Z., He, Z., Zhao, Y., Pan, C., Zhang, J., Song, S., & Li, Y. (2025) · Vicinagearth. · doi:10.1007/s44336-025-00016-x · Cited by 9
“The limited context window size of LLMs and Loss in the Middle [19] phenomenon make processing large-scale tables difficult or even infeasible.”
Multi-Source Knowledge Pruning for Retrieval-Augmented Generation: A Benchmark and Empirical Study
Yu, S., Cheng, M., Liu, Q., Wang, D., Yang, J., Ouyang, J., Luo, Y., Lei, C., & Chen, E. (2025) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3746252.3761340 · Cited by 24
Giuliani, C., Benadi, G., Engel, F., Werner, J., Watter, M., Schwarzer, G., Groß, O., Zeiser, R., Binder, H., & Kaier, K. (2025) · medRxiv · doi:10.1101/2025.03.04.25323310 · Cited by 3
ClassInvGen: Class Invariant Synthesis Using Large Language Models
Sun, C., Agashe, V., Chakraborty, S., Taneja, J., Barrett, C., Dill, D. L., Qiu, X., & Lahiri, S. K. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-99991-8_4 · Cited by 17
Relationalizing Tables with Large Language Models: The Promise and Challenges
Huang, Z., & Wu, E. (2024) · IEEE International Conference on Data Engineering Workshops (ICDEW) · doi:10.1109/icdew61823.2024.00045 · Cited by 9
“This complexity could be too overwhelming for LLMs [5], [6].”
BackgroundEnhancing RAG Performance by Representing Hierarchical Nodes in Headers for Tabular Data
Song, M. (2025) · IEEE Access · doi:10.1109/access.2025.3569872 · Cited by 6
“Prior research [36] found that increasing the number of retrieved documents from the vector database can enhance retrieval recall; however, sending an excessive number of documents directly to the LLM can also negatively impact its recall performance.”
Macilenti, G., Stellato, A., & Fiorelli, M. (2024) · Procedia Computer Science · doi:10.1016/j.procs.2024.09.557 · Cited by 5
Episodic Memory Verbalization Using Hierarchical Representations of Life-Long Robot Experience
Bärmann, L., DeChant, C., Plewnia, J., Peller-Konrad, F., Bauer, D., Asfour, T., & Waibel, A. (2025) · IEEE-RAS International Conference on Humanoid Robots (Humanoids) · doi:10.1109/humanoids65713.2025.11203101 · Cited by 15
“Although recent LLMs offer increasingly long context windows (i. e., the maximum number of tokens they can process), up to 2M tokens [11], previous studies [12], [13] have shown that these models have difficulty in using all information contained in such long contexts.”
ChatHPC: Building the Foundations for a Productive and Trustworthy AI-Assisted HPC Ecosystem
Valero‐Lara, P., Young, A. R., Vetter, J. S., Jin, Z., Pophale, S., Monil, M. A. H., Teranishi, K., & Godoy, W. F. (2025) · International Conference for High Performance Computing, Networking, Storage and Analysis (SC) · doi:10.1145/3712285.3759787 · Cited by 6
Beyond pip Install: Evaluating LLM Agents for the Automated Installation of Python Projects
Milliken, L., Kang, S., & Yoo, S. (2025) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner64311.2025.00009 · Cited by 34
“The additional cost can vary greatly depending on the contents of the target repository; extensive documentations or ambiguous naming can lead to the agent searching through more files than necessary, and thus consuming more tokens [26], [28], [29], resulting in higher cost.”
Wang, C., Chai, S., Xu, T., Adil, M., & Qiu, T. (2026) · IEEE Internet of Things Journal · doi:10.1109/jiot.2026.3652422 · Cited by 6
“Prior studies have shown that LLMs tend to exhibit stronger attention focus, memory encoding, and utilization capabilities for information located at the start and end of input sequences, whereas information in the middle positions is more likely to be overlooked or underutilized [32].”
Motoki, F. Y. S., Monteiro, J. J., Malagueño, R., & Rodrigues, V. R. D. S. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4595896 · Cited by 4
Dementia risk prediction using decision-focused content selection from medical notes
Li, S., Dexter, P. R., Miled, Z. B., & Boustani, M. A. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.109144 · Cited by 3
Less Is More: Failing Test Generation with Large Language Models
Li, T.-O., Yan, X., Zong, W., Wang, Y., Tian, H., Xu, C., Wang, Y., Cheung, S.-C., & Kramer, J. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3793675 · Cited by 3
CROP: Circuit Retrieval and Optimization with Parameter Guidance using LLMs
Pan, J., Jacobson, I., Zhao, Z., Chen, T.-C., Zhou, G.-S., Chang, C.-C., Rashingkar, V., & Chen, Y. (2025) · doi:10.1109/iccad66269.2025.11240643 · Cited by 8
“The context window size of LLMs is crucial to their ability to craft coherent, accurate, and relevant responses for code analysis [31]– [33].”
Enhancing LLM’s Reliability by Iterative Verification Attributions with Keyword Fronting
Sui, Y., Ren, J., Tan, H., Chen, H., Li, Z., & Wang, J. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-70365-2_15 · Cited by 5
Nordman, J., Najjuuko, C., Jeschke, N., Snyders, R. E., Guillamet, C. V., Shapiro, J., Leone, C. L., Dethloff, M., Babcock, H. M., Reich, P., et al. (2026) · Clinical Infectious Diseases · doi:10.1093/cid/ciag301 · Cited by 3
Explainable Optimization: Leveraging Large Language Models for User-Friendly Explanations
Biemans, B., Troubil, P., Grau, I., & Nuijten, W. (2025) · Communications in computer and information science · doi:10.1007/978-3-032-08327-2_3 · Cited by 9
Kim, M.-W., Park, H.-B., Ahn, H.-J., Park, W.-R., Jeon, J.-W., Lee, K.-H., Lee, R., & Choi, D. (2025) · Electronics · doi:10.3390/electronics14061175 · Cited by 9
Ba, H., Du, H., Cheng, C., Zhang, Y., Ruan, L., & Chen, Z. (2025) · JMIR Formative Research · doi:10.2196/70602 · Cited by 4
“…when handling long text tasks, this study reconstructs the dialogue interface with every 15 questions, reloads the task context and rules, and repeats the processes of in-context learning, chain-of-thought, and choice shuffling ensemble until all cases have completed syndrome differentiation [43].”
Systematization of Knowledge: Security and Safety in the Model Context Protocol Ecosystem
Gaire, S., Gyawali, S., Mishra, S., Niroula, S., Thakur, D., & Yadav, U. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-32726-0_29 · Cited by 36
LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory
Wu, D., Wang, H., Yu, W., Zhang, Y., Chang, K.-W., & Yu, D. (2024) · arXiv · doi:10.48550/arxiv.2410.10813 · Cited by 493
“Although the retrieval accuracy can be enhanced through the preceding designs, it does not guarantee that the LLM can effectively reason over the extensive context (Shi et al., 2023; Liu et al., 2024).”
FORGE: An LLM-driven Framework for Large-Scale Smart Contract Vulnerability Dataset Construction
Chen, J., Shen, Y., Zhang, J., Li, Z., Grundy, J., SHAO, Z., Wang, Y., Wang, J., Chen, T., & Zheng, Z. (2026) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3744916.3764529 · Cited by 20
Dong, Z., Tang, T., Li, J., Zhao, W. X., & Wen, J.-R. (2023) · arXiv · doi:10.48550/arxiv.2309.13345 · Cited by 7
Evaluating Very Long-Term Conversational Memory of LLM Agents
Maharana, A., Lee, D.-H., Tulyakov, S., Bansal, M., Barbieri, F., & Fang, Y. (2024) · arXiv · doi:10.48550/arxiv.2402.17753 · Cited by 3
“…process (Mallen et al., 2023; Gao et al., 2023b; Liu et al., 2023b); (2) Challenges arise in reasoning over retrieved documents, especially when the model struggles to identify the correct context among the retrieved data (Liu et al., 2023a); (3) Reasoning over time intervals presents challenges.”
BackgroundMRAG: a modular retrieval framework for time-sensitive question answering
Zhang, S., Xue, Y., Zhang, Y., Wu, X., Luu, A. T., & Zhao, C. (2025) · Findings of the Association for Computational Linguistics: EMNLP 2025 · doi:10.18653/v1/2025.findings-emnlp.167 · Cited by 3
Attribute or Abstain: Large Language Models as Long Document Assistants
Buchmann, J., Liu, X., & Gurevych, I. (2024) · Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2024.emnlp-main.463 · Cited by 3
Larimar: Large Language Models with Episodic Memory Control
Das, P., Chaudhury, S., Nelson, E., Melnyk, I., Swaminathan, S., Dai, S., Lozano, A., Kollias, G., Chenthamarakshan, V., Jiří, et al. (2024) · arXiv · doi:10.48550/arxiv.2403.11901 · Cited by 3
AI in the city: impact assessment of artificial intelligence uses in earth observation
Šćepanović, S., Bogucka, E. P., & Quercia, D. (2025) · Urban Informatics · doi:10.1007/s44212-025-00074-y · Cited by 4
“We opted for five examples to strike a balance between providing a diverse range of examples and maintaining a manageable context length in the prompt (Liu et al., 2024; Brown et al., 2020).”
Hitl, M., Babac, M. B., & Mornar, V. (2026) · Data Technologies and Applications · doi:10.1108/dta-07-2025-0609 · Cited by 4
Language Modelling Techniques for Analysing the Impact of Human Genetic Variation
Hegde, M., Nebel, J., & Rahman, F. (2025) · Bioinformatics and Biology Insights · doi:10.1177/11779322251358314 · Cited by 7
“101 The attention Markov models are often used to construct grammars.”
Improving OpenDevin: Boosting code generation LLM through advanced memory management
He, R., Ying, A., & Hu, X. (2024) · Applied and Computational Engineering · doi:10.54254/2755-2721/68/20241506 · Cited by 1
Lenz, S., Ustjanzew, A., Jeray, M., Ressing, M., & Panholzer, T. (2025) · BioData Mining · doi:10.1186/s13040-025-00463-8 · Cited by 8
“In addition to the computational costs for using the complete documents as inputs, there is also the problem of degradation if relevant information is embedded in very long inputs [16].”
Yan, F., Liu, J., Guo, L. H., Liu, L., & Geng, X. (2026) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2026.01.016 · Cited by 5
SRSA: A Cost-Efficient Strategy-Router Search Agent for Real-world Human-Machine Interactions
Wang, Y.-Q., & Xu, H.-P. (2024) · 2024 IEEE International Conference on Data Mining Workshops (ICDMW) · doi:10.1109/icdmw65004.2024.00046 · Cited by 1
“Using too many examples in one prompt will confuse the referee model, and the middle examples may be forgotten as in the case of long prompts, the middle prompt gets [41].”
Dong, Y., Zhang, T., & Tian, X.-Y. (2026) · Journal of Hospitality and Tourism Technology · doi:10.1108/jhtt-11-2025-1080 · Cited by 3
LLM-Powered Interactive Graph Search: A Scalable and Practical Approach
Linghu, H., Cong, Q., Huang, Y., Lu, S., Feng, L., & Tang, J. (2025) · Proceedings of the ACM on Management of Data · doi:10.1145/3769804 · Cited by 3
Zhang, W., Lim, W. S., & Pavlo, A. (2026) · Proceedings of the ACM on Management of Data · doi:10.1145/3786704 · Cited by 4
“We discuss each part separately.”
Metadata
- Authors
- Liu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., & Liang, P.
- Affiliations
- Stanford University (US)
- University of California, Berkeley (US)
- Samaya AI, UK
- Corresponding author
- Liu, Nelson F.
- Venue
- Transactions of the Association for Computational Linguistics
- Publisher
- MIT Press
- Published
- 2024
- Volume
- 12
- Pages
- 157-173
- ISSN
- 2307-387X (ISSN-L)
- OpenAlex
- W4391876619
- Semantic Scholar
- 1733eb7792f7a43dd21f51f4d1017a1bffd217b5
- Cited by
- 1,612
- References
- 59
- Citations by year
- 2026: 901 · 2025: 516 · 2024: 172
- FWCI
- 233.95
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- Locations
- Transactions of the Association for Computational Linguistics (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- long-context language models, multi-document question answering, position bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-09-27 (prompt extract-v2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.