Lost in the Middle: How Language Models Use Long Contexts
Open accessThe DOI matches its Crossref record; 48 of the 59 references it cites were found in a registry or an index; 15 of the 23 sources its claims cite were checked for support.
Abstract
While recent language models have the ability to take long contexts as input, relatively little is known about how well they use longer context. We analyze the performance of language models on two tasks that require identifying relevant information in their input contexts: multi-document question answering and key-value retrieval. We find that performance can degrade significantly when changing the position of relevant information, indicating that current language models do not robustly make use of information in long input contexts. In particular, we observe that performance is often highest when relevant information occurs at the beginning or end of the input context, and significantly degrades when models must access relevant information in the middle of long contexts, even for explicitly long-context models. Our analysis provides a better understanding of how language models use their input context and provides new evaluation protocols for future long-context language models.
Overview
CoChat Summary
TL;DR
Language models’ performance on tasks requiring access to long input contexts is strongly affected by where relevant information appears, with a U-shaped curve: best at the beginning or end and worst in the middle. Extended-context models are not necessarily better at using context, and reader performance saturates before retriever recall in open-domain QA.
Key findings
- In multi-document QA, performance is highest when the answer-containing document is at the start or end of the context and degrades in the middle; GPT-3.5-Turbo can drop by more than 20%, and in 20- and 30-document settings middle-position performance is lower than its closed-book performance of 56.1% (oracle: 88.3%).
- Extended-context models are not necessarily better at using input context: when the input fits both windows, GPT-3.5-Turbo and GPT-3.5-Turbo(16K) perform nearly identically as a function of relevant-information position.
- In synthetic key-value retrieval, Claude-1.3 and Claude-1.3(100K) do nearly perfectly, while other models struggle especially with 140 or 300 key-value pairs; query-aware contextualization gives GPT-3.5-Turbo(16K) perfect performance at 300 pairs, versus worst-case 45.6% without it.
- In open-domain QA, reader performance saturates long before retriever recall: using 50 instead of 20 retrieved documents only marginally improves performance (∼1.5% for GPT-3.5-Turbo and ∼1% for Claude-1.3).
Methods
- Controlled multi-document QA on NaturalQuestions-Open: 2,655 queries whose annotated long answer is a paragraph; each context has exactly one answer-containing Wikipedia passage plus k−1 Contriever/MS-MARCO distractor passages; the paper varies answer position and context length (10, 20, 30 documents) and measures accuracy.
- Synthetic key-value retrieval: inputs are serialized JSON objects with unique random UUID key-value pairs; models must return the value for a specified key; experiments use 75, 140, and 300 key-value pairs (500 examples each), varying key position and number of pairs, with accuracy as the metric.
- Models include open MPT-30B-Instruct and LongChat-13B(16K), closed GPT-3.5-Turbo, GPT-3.5-Turbo(16K), Claude-1.3, and Claude-1.3(100K), plus additional architecture/instruction experiments with Flan-T5-XXL, Flan-UL2, MPT-30B, Llama-2, and GPT-4; an open-domain QA case study uses a Contriever retriever-reader setup.
Limitations
- Stated: the paper uses greedy decoding and leaves exploration of other decoding methods to future work; the investigations into why models are not robust are described as preliminary; GPT-4 is evaluated only on a subset of 500 multi-document QA examples because full evaluation would cost upwards of $6000.
- Stated: NaturalQuestions-Open has ambiguity, so the authors run an unambiguous subset and also test random distractors and randomized distractor order, finding similar trends.
- Evident: the controlled multi-document QA setting always contains exactly one document that answers the question, unlike open-domain QA; the synthetic key-value task removes natural language semantics; results may not generalize to all tasks, models, or decoding settings.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
7 Valid · 0 Not valid · 5 Not checked · 13 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Language models have become an important and flexible building block in a variety of user-facing language technologies, including conversational interfaces, search and summarization, and collaborative writing (Shuster et al., 2022;Thoppilan et al., 2022;Lee et al., 2022, inter alia).”
§ Lost in the Middle: How Language Models Use Long Contexts > 1 Introduction
What the sources say
- Partly supportedcites Shuster et al., 2022 sourceExperimental — accuracy not yet measured
Section 3.1.1 (p. 4) describes a conversational agent built on a language model that generates internet search queries, supporting conversational interfaces and search as part of dialogue. Section 2 (p. 3) mentions summarization only as related work, and the passages do not establish collaborative writing as an application of this paper’s model. (LLM, conf=0.85)
- Supportedcites Thoppilan et al., 2022 source
The Abstract (p. 1) identifies LaMDA as a family of language models specialized for dialog. In this sentence's list of applications, Thoppilan et al. supports the conversational-interface example; the other cited works need not each cover every example. (LLM, conf=0.92)
- Supportedcites Lee et al., 2022 source
Lee et al. is cited alongside other works, and its contribution to the sentence is the collaborative-writing example. Section 4.2.2 describes writers receiving and selecting model-generated suggestions in a writing interface; Section 2.2.1 also describes language models as applicable to dialogue and summarization. The paper need not establish every example in the jointly cited list. (LLM, conf=0.89)
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“These observations complement prior work, which found that non-instruction fine-tuned language models are biased towards recent tokens (i.e., the end of the input context; Khandelwal et al., 2018;Press et al., 2021).”
§ Lost in the Middle: How Language Models Use Long Contexts > 4.3 Effect of Instruction Fine-Tuning
What the sources say
- Partly supportedcites Khandelwal et al., 2018 sourceExperimental — accuracy not yet measured
The Introduction and Section 5 report that the studied LSTM is much more sensitive to recent word order than to distant word order, supporting a recency effect. The paper studies a standard LSTM on two datasets, however, rather than establishing this tendency across non-instruction-fine-tuned language models generally; it also finds that distant words remain relevant. (LLM, conf=0.82)
- Partly supportedcites Press et al., 2021 sourceExperimental — accuracy not yet measured
Section 3.1 (page not available in our copy) finds that very short contexts hurt performance, while increasing subsequence length from 1,024 to 3,072 tokens gives negligible improvement. That is consistent with limited benefit from more distant context, but the paper does not establish that the models are biased toward recent tokens within an input context. (LLM, conf=0.78)
- Partly supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“We instantiate this task with data from NaturalQuestions-Open (Lee et al., 2019;Kwiatkowski et al., 2019), which contains historical queries issued to the Google search engine, coupled with human-annotated answers extracted from Wikipedia.”
§ Lost in the Middle: How Language Models Use Long Contexts > 2.1 Experimental Setup
What the sources say
- Supportedcites Lee et al., 2019 source
Section 7.1 says the paper creates an open version of Natural Questions and uses it for training and evaluation. The Introduction describes the task as answering questions by retrieving evidence from an open corpus, supporting the cited use of NaturalQuestions-Open for this task. (LLM, conf=0.91)
- Not checked — no full text of the sourcecites Kwiatkowski et al., 2019 source
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Following Kandpal et al. (2022) and Mallen et al. (2023), we use accuracy as our primary evaluation metric, judging whether any of the cor- rect answers (as taken from the NaturalQuestions annotations) appear in the predicted output.”
§ Lost in the Middle: How Language Models Use Long Contexts > 2.1 Experimental Setup
What the sources say
- Supportedcites Kandpal et al. (2022) source
Section 3 (p. 4) says the paper evaluates answers using exact match against a ground-truth answer set, and Section 3.1 (p. 4) reports QA accuracies as its evaluation results. This supports citing Kandpal et al. for using accuracy to evaluate question answering. (LLM, conf=0.90)
- Supportedcites Mallen et al. (2023) source
Section 3.1 (‘Focus and Task’) explicitly names accuracy as the metric and says a prediction is correct when it contains an exact match to any gold answer. This supports the evaluation approach attributed to Mallen et al.; the NaturalQuestions annotations describe the citing authors’ answer source. (LLM, conf=0.94)
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“or when language models are augmented with external information (relevant documents from a search engine, database query results, etc.; Petroni et al., 2020;Ram et al., 2023;Shi et al., 2023;Mallen et al., 2023;Schick et al., 2023, inter alia).”
§ Lost in the Middle: How Language Models Use Long Contexts > 1 Introduction
What the sources say
- Partly supportedcites Petroni et al., 2020 sourceExperimental — accuracy not yet measured
The Abstract (p. 1) says the paper integrates information from a retrieval system with a pre-trained language model, and §3.4.2 (p. 5) specifies a retrieved paragraph as context. This supports augmentation with retrieved documents, but our copy does not show language models being augmented with database query results. (LLM, conf=0.84)
- Partly supportedcites Ram et al., 2023 sourceExperimental — accuracy not yet measured
The Introduction describes augmenting language models with relevant documents retrieved from an external knowledge source, and Section 5 describes using a BM25 retriever to find those documents. The passages do not establish database query results as another type of external information used by this paper. (LLM, conf=0.82)
- Partly supportedcites Shi et al., 2023 sourceExperimental — accuracy not yet measured
Section 3.1 (page unavailable) describes retrieving relevant documents from a corpus, and the Conclusion says REPLUG uses those documents to augment a language model. The paper supports the document-retrieval example, but our copy does not describe augmenting models with database query results or documents from a search engine specifically. (LLM, conf=0.82)
- Partly supported
20 more claims · 2 with their cited sources checked
Create a free account to see all 25 claims and every source verdict.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In vanilla multi-document or NIAH-based benchmarks (Kamradt, 2023; Liu et al., 2024), models perform consistently well when the needle (or gold document) appears at the very beginning or end of the context window, with minimal impact from context length.”
From NoLiMa: Long-Context Evaluation Beyond Literal Matching (2025) · cites this paper as Liu et al., 2024
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Even within a single long sequence, attention effectiveness degrades with distance due to attention dilution , positional encoding limitations, and token interference, leading to the well-known 'lost-in-the-middle' and context-decay phenomena (Liu et al., 2024; Press et al., 2021a).”
From MAGMA: A Multi-Graph based Agentic Memory Architecture for AI Agents (2026) · cites this paper as Liu et al., 2024
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Current large language models (LLMs) exhibit significant deficiencies in episodic memory tasks including encoding, storing, and retrieving specific information from temporally dependent events over a long period of time.”
From ARTEM: Enhancing Large Language Model Agents with Spatial-Temporal Episodic Memory (2026) · cites this paper as Liu et al. 2024
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Recent approaches to handle memory tasks in LLMs, such as in-context learning, retrieval-augmented generation (RAG), and fine-tuning, may resolve the long-term retention issues, but are still inadequate to handle tasks requiring chronological awareness of the stored information.”
From ARTEM: Enhancing Large Language Model Agents with Spatial-Temporal Episodic Memory (2026) · cites this paper as Liu et al. 2024
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“If not ranked at the top, the key information may be neglected by the generative module, resulting in incomplete or inaccurate responses to the user queries24.”
From Leveraging long context in retrieval augmented language models for medical question answering (2025) · cites this paper as [24]
Experimental — accuracy not yet measured
86 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929
LLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602
Transformer-XL: Attentive language models beyond a fixed-length context
Dai, Z., Yang, Z., Yang, Y., Carbonell, J., Le, Q., & Salakhutdinov, R. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1285 · Cited by 3,169 · cited 2×
“Recent improvements in hardware (e.g., faster GPUs with more memory) and algorithms (Dai et al., 2019; Dao et al., 2022; Poli et al., 2023; Rubin and Berant, 2023, inter alia ) have resulted in language models with larger context windows (e.g., 4096, 32K, and even 100K tokens), but it remains…”
BackgroundLlama 2: Open Foundation and Fine-Tuned Chat Models
Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.09288 · Cited by 2,604
Longformer: The Long-Document Transformer
Beltagy, I., Peters, M. E., & Cohan, A. (2020) · arXiv · doi:10.48550/arxiv.2004.05150 · Cited by 2,281
Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering
Izacard, G., & Grave, E. (2020) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.48550/arxiv.2007.01282 · Cited by 2,263
Natural Questions: A benchmark for question answering research
Kwiatkowski, T., Palomaki, J., Redfield, O., Collins, M., Parikh, A., Alberti, C., Epstein, D., Polosukhin, I., Devlin, J., Lee, K., et al. (2019) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00276 · Cited by 2,069 · cited 1×
“We instantiate this task with data from NaturalQuestions-Open (Lee et al., 2019; Kwiatkowski et al., 2019), which contains historical queries issued to the Google search engine, coupled with human-annotated answers extracted from Wikipedia.”
MethodThe serial position effect of free recall.
Murdock, B. B. (1962) · Journal of Experimental Psychology · doi:10.1037/h0045106 · Cited by 1,717 · cited 1×
“The U-shaped curve we observe in this work has a connection in psychology known as the serial-position effect (Ebbinghaus, 1913; Murdock Jr, 1962), that states that in free-association recall of elements from a list, humans tend to best re-member the first and last elements of the list.”
MethodMemory: A contribution to experimental psychology.
Ebbinghaus, H. (1913) · doi:10.1037/10011-000 · Cited by 1,645 · cited 1×
“The U-shaped curve we observe in this work has a connection in psychology known as the serial-position effect (Ebbinghaus, 1913; Murdock Jr, 1962), that states that in free-association recall of elements from a list, humans tend to best re-member the first and last elements of the list.”
Method
Research works printed but not linked: 1
- Sharan, 2018, Prediction with a short memory, 10.18653/v1/K18-3013 · DOI
Cited by
Items 101–200 of 1,597
Application of large language models to intelligently analyze long construction contract texts
Gao, Y., Gan, Y., Chen, Y., & Chen, Y. (2024) · Construction Management and Economics · doi:10.1080/01446193.2024.2415676 · Cited by 41
“Secondly, the intrinsic mechanisms by which LLMs use given context information are not fully clear, and a model’s memory capacity does not directly correlate with its semantic understanding abilities (Liu et al .”
Ray, M., Kats, D. J., Moorkens, J., Rai, D., Shaar, N., Quinones, D., Vermeulen, A., Mateo, C. M., Brewster, R., Khan, A., et al. (2025) · Archives of Pediatrics and Adolescent Medicine · doi:10.1001/jamapediatrics.2025.1729 · Cited by 40
A psychometric framework for evaluating and shaping personality traits in large language models
Serapio-García, G., Safdari, M., Crepy, C., Sun, L., Fitz, S., Romero, P., Abdulhai, M., Faust, A., & Matarić, M. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-025-01115-6 · Cited by 30
Large language model trained on clinical oncology data predicts cancer progression
Zhu, M., Lin, H., Jiang, J., Jinia, A. J., Jee, J., Pichotta, K., Waters, M., Rose, D., Schultz, N., Chalise, S., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01780-2 · Cited by 30
“This decline may be attributed to the model nearing its maximum context size, causing it to lose focus on content in the middle of the sequence 47 .”
DocETL: Agentic Query Rewriting and Evaluation for Complex Document Processing
Shankar, S., Chambers, T., Shah, T., Parameswaran, A., & Wu, E. (2025) · Proceedings of the VLDB Endowment · doi:10.14778/3746405.3746426 · Cited by 188
“During optimization, DocETL decomposes the equijoin into a map-equijoin.”
Laskar, M. T. R., Fu, X.-Y., Chen, C., & Shashi Bhushan, T. N. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-industry.33 · Cited by 99
Wang, Z., Liu, Z., Lu, J. W. Z., & Lu, J. H. (2025) · Journal of Building Engineering · doi:10.1016/j.jobe.2025.112189 · Cited by 49
Membership Inference Attacks Against In-Context Learning
Wen, R., Li, Z., Backes, M., & Zhang, Y. (2024) · Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3658644.3690306 · Cited by 27
Codebook LLMs: Evaluating LLMs as Measurement Tools for Political Science Concepts
Halterman, A., & Keith, K. A. (2025) · Political Analysis · doi:10.1017/pan.2025.10017 · Cited by 119
Exploring RAG Solutions to Reduce Hallucinations in LLMs
AboulEla, S., Zabihitari, P., Ibrahim, N., Afshar, M., & Kashef, R. F. (2025) · IEEE International Systems Conference (SysCon) · doi:10.1109/syscon64521.2025.11014810 · Cited by 39
“Large amounts of text can exceed the context window limits of LLMs, and merely increasing window size may not prevent information from being ”Lost-in-the-Middle” [33].”
Li, D., & Lutfi, S. L. (2025) · JMIR Medical Informatics · doi:10.2196/79039 · Cited by 45
Survey on Evaluation of LLM-based Agents
Yehudai, A., Eden, L., Li, A., Uziel, G., Zhao, Y., Bar-Haim, R., Cohan, A., & Shmueli-Scheuer, M. (2025) · arXiv · doi:10.48550/arxiv.2503.16416 · Cited by 26
“Similarly, A-MEM introduces an advanced memory architecture evaluated using the Lo-CoMo benchmark (Maharana et al., 2024), while MemGPT manages a tiered memory system tested on NaturalQuestions-Open (Liu et al., 2024b) and multi-session chat datasets (Xu et al., 2021).”
Statler: State-Maintaining Language Models for Embodied Reasoning
Yoneda, T., Fang, J., Li, P., Zhang, H., Jiang, T., Lin, S., Picker, B., Yunis, D., Mei, H., & Walter, M. R. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10610634 · Cited by 92
Parametric Retrieval Augmented Generation
Su, W., Tang, Y., Ai, Q., Yan, J., Wang, C., Wang, H., Ye, Z., Zhou, Y., & Liu, Y. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3729957 · Cited by 129
Guardian: A Runtime Framework for LLM-Based UI Exploration
Ran, D., Wang, H., Song, Z., Wu, M. S., Cao, Y., Zhang, Y., Yang, W. M., & Xie, T. (2024) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3650212.3680334 · Cited by 68
“This limitation becomes even more significant in scenarios involving long contexts, where the models’ ability to accurately follow instructions tends to decline [26, 32].”
Dissecting Corporate Culture Using Generative AI
Li, K., Mai, F., Shen, R., Yang, C., & Zhang, T. (2023) · Review of Financial Studies · doi:10.1093/rfs/hhaf081 · Cited by 97
Fostering effective hybrid human-LLM reasoning and decision making
Passerini, A., Gema, A. P., Minervini, P., Sayin, B., & Tentori, K. (2025) · Frontiers Artif. Intell. · doi:10.3389/frai.2024.1464690 · Cited by 32
“However, LLMs can still disregard provided information and rely on their parametric knowledge due to intrinsic mechanisms (Jin et al., 2024; Xu et al., 2024a) or sensitivity to prompts (Liu et al., 2024).”
Technological folie à deux: Feedback Loops Between AI Chatbots and Mental Illness
Dohnány, S., Kurth-Nelson, Z., Spens, E., Luettgau, L., Reid, A., Gabriel, I., Summerfield, C., Shanahan, M., & Nour, M. M. (2025) · arXiv · doi:10.48550/arxiv.2507.19218 · Cited by 26
“In line with recent work, we argue that the psychological and mental health risks of chatbot use cannot be explained by a narrow consideration of chatbot limitations, such as falsehoods ( hallucinations, see Box 1 ), socially harmful biases, or failures in pragmatic language understanding 26–32 .”
∞Bench: Extending Long Context Evaluation Beyond 100K Tokens
Zhang, X., Chen, Y., Hu, S., Xu, Z., Chen, J., Hao, M., Han, X., Thai, Z., Wang, S., Liu, Z., et al. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.814 · Cited by 27
Incremental accumulation of linguistic context in artificial and biological neural networks
Tikochinski, R., Goldstein, A., Meiri, Y., Hasson, U., & Reichart, R. (2025) · Nature Communications · doi:10.1038/s41467-025-56162-9 · Cited by 24
A Survey of Reasoning with Foundation Models: Concepts, Methodologies, and Outlook
Sun, J., Zheng, C., Xie, E., Liu, Z., Chu, R., Qiu, J., Xu, J., Ding, M., Li, H., Geng, M., et al. (2025) · ACM Computing Surveys · doi:10.1145/3729218 · Cited by 110
Ibrahimzada, A. R., Ke, K., Pawagi, M., Abid, M. S., Pan, R., Sinha, S., & Jabbarvand, R. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3729379 · Cited by 110
“Assuming an unlimited context window, LLMs still suffer from short attention span [38], preventing them from properly capturing the intra-and inter-procedural dependencies in real-world projects.”
Balog, K., Metzler, D., & Qin, Z. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730348 · Cited by 56
ChatGPT for Textual Analysis? How to use Generative LLMs in Accounting ResearchPreprint
de Kok, T. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4429658 · Cited by 23
Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention
Munkhdalai, T., Faruqui, M., & Gopal, S. (2024) · arXiv · doi:10.48550/arxiv.2404.07143 · Cited by 23
“The attention mechanism is also prone to the issues of attention sink (Xiao et al., 2023) and lost-in-the-middle (Liu et al., 2024).”
BackgroundPatel, L., Jha, S., Pan, M., Gupta, H., Asawa, P., Guestrin, C., & Zaharia, M. (2025) · Proceedings of the VLDB Endowment · doi:10.14778/3749646.3749685 · Cited by 23
RTLRewriter: Methodologies for Large Models aided RTL Code Optimization
Yao, X., Wang, Y., Li, X., Lian, Y., Chen, R., Chen, L., Yuan, M., Xu, H. L., & Yu, B. (2024) · doi:10.1145/3676536.3676775 · Cited by 71
“The challenge of comprehending long contexts remains an unsolved problem in transformer-based large models [39] due to their quadratic complexity, and we have observed this phenomenon in RTL code optimization as well.”
Genovese, A., Prabha, S., Borna, S., Gomez-Cabello, C. A., Haider, S. A., Trabilsy, M., Tao, C., Aziz, K. T., Murray, P. M., & Forte, A. J. (2025) · Aesthetic Surgery Journal · doi:10.1093/asj/sjaf038 · Cited by 26
Can Large Language Models Understand Real-World Complex Instructions?
He, Q., Zeng, J., Huang, W., Chen, L., Xiao, J., He, Q., Zhou, X., Liang, J., & Xiao, Y. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i16.29777 · Cited by 125
Memolet: Reifying the Reuse of User-AI Conversational Memories
Yen, R., & Zhao, J. (2024) · Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3654777.3676388 · Cited by 22
“However, current LLMs often are limited to handling long-term memory [37, 72, 77] and remain opaque about how they use longer contexts in downstream tasks [50, 92].”
Li, H., Huang, J., Ji, M., Yang, Y., & An, R. (2025) · Journal of Medical Internet Research · doi:10.2196/66098 · Cited by 46
“Despite its effectiveness, the naïve RAG has limitations, such as the “lost in the middle” phenomenon, where the LLM’s performance deteriorates when handling information located in the middle of lengthy contexts [16].”
OntoGenix: Leveraging Large Language Models for enhanced ontology engineering from datasets
Val-Calvo, M., Aranguren, M. E., Mulero-Hernández, J., Almagro-Hernández, G., Deshmukh, P., Bernabé-Díaz, J. A., Espinoza-Arias, P., Sánchez-Fernández, J. L., Mueller, J., & Fernández‐Breis, J. T. (2024) · Information Processing & Management · doi:10.1016/j.ipm.2024.104042 · Cited by 36
Karakurt, E., & Akbulut, A. (2025) · Applied Sciences · doi:10.3390/app16010368 · Cited by 58
Agentic Workflows for Improving Large Language Model Reasoning in Robotic Object-Centered Planning
Moncada-Ramirez, J., Matez-Bandera, J.-L., González-Jiménez, J., & Ruiz-Sarmiento, J.-R. (2025) · Robotics · doi:10.3390/robotics14030024 · Cited by 27
Ke, W., Zheng, Y., Li, Y., Xu, H., Nie, D., Wang, P., & He, Y. (2025) · ACM Transactions on Information Systems · doi:10.1145/3768156 · Cited by 73
Cache-Craft: Managing Chunk-Caches for Efficient Retrieval-Augmented Generation
Agarwal, S., Sundaresan, S., Mitra, S., Mahapatra, D., Gupta, A., Sharma, R., Kapu, N. J., Yu, T., & Saini, S. (2025) · arXiv · doi:10.48550/arxiv.2502.15734 · Cited by 21
Feedback by Design: Understanding and Overcoming User Feedback Barriers in Conversational Agents
Sharma, N., Zhang, Z., Lee, D., Krishnan, N., Ren, G.-J., Xiao, Z., & Li, Y. (2026) · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791600 · Cited by 21
Cache-Craft: Managing Chunk-Caches for Efficient Retrieval-Augmented Generation
Agarwal, S., Sundaresan, S., Mitra, S., Mahapatra, D., Gupta, A., Sharma, R., Kapu, N. J., Yu, T., & Saini, S. (2025) · Proceedings of the ACM on Management of Data · doi:10.1145/3725273 · Cited by 21
Chung, P., Swaminathan, A., Goodell, A. J., Kim, Y., Reincke, S. M., Han, L., Deverett, B., Sadeghi, M. A., Ariss, A.-B., Ghanem, M., et al. (2025) · NEJM AI · doi:10.1056/aidbp2500418 · Cited by 79
(2025) · doi:10.14778/3750601.3750611 · Cited by 20
Can GPT-4 Replicate Empirical Software Engineering Research?
Liang, J. T., Badea, C., Bird, C., DeLine, R., Ford, D., Forsgren, N., & Zimmermann, T. (2024) · Proceedings of the ACM on Software Engineering · doi:10.1145/3660767 · Cited by 19
Nanua, S., Steward, R., Neely, B., Datto, M., & Youens, K. E. (2025) · Journal of Pathology Informatics · doi:10.1016/j.jpi.2025.100520 · Cited by 25
Cox, S. R., & Ooi, W. T. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-54975-5_10 · Cited by 57
“For example, LLM memory is an on-going research area [25] with recent implementations using interaction logs [27] or summarised conversations 9 in subsequent prompting, as well as investigations of how size of conversational memory impacts model behaviour [19].”
Backgroundvan Duijn, M., van Dijk, B., Kouwenhoven, T., de Valk, W., Spruit, M., & vanderPutten, P. (2023) · Proceedings of the 27th Conference on Computational Natural Language Learning (CoNLL) · doi:10.18653/v1/2023.conll-1.25 · Cited by 24
Task Planning for a Factory Robot Using Large Language Model
Tsushima, Y., Yamamoto, S., Ravankar, A. A., Luces, J. V. S., & Hirata, Y. (2025) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2025.3531153 · Cited by 30
“This is based on research findings that LLM tends to pay more attention to information closer to the beginning and end of the input [19].”
FlexFL: Flexible and Effective Fault Localization With Open-Source Large Language Models
Xu, C.-Y., Liu, Z.-X., Ren, X.-X., Zhang, G., Liang, M., & Lo, D. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3553363 · Cited by 35
“However, it is costly and ineffective to directly feed the whole large program into LLMs for processing since the computational complexity of existing LLMs is quadratic and the performance of LLMs degrades as the length of input contexts increases [28].”
Hu, X., Chen, S., Chen, L., Wang, H., Zhang, X., & Zhou, Z. (2025) · National Science Review · doi:10.1093/nsr/nwaf372 · Cited by 19
“While the DeepSeek-v3 model exhibited strong instruction-following capabilities, it occasionally encountered ‘lost in the middle’ phenomena [30 ], resulting in parameter extraction failures.”
Barrit, S., Torcida, N., Mazeraud, A., Boulogne, S., Benoit, J. B., Carette, T., Carron, T., Delsaut, B., Diab, E., Kermorvant, H., et al. (2025) · Brain Sciences · doi:10.3390/brainsci15040347 · Cited by 21
“Furthermore, while modern LLMs have expanded context windows, they still face attention degradation across long contexts, potentially limiting their effectiveness in complex, data-rich environments typical of healthcare and research [11,12].”
TokenRec: Learning to Tokenize ID for LLM-Based Generative Recommendations
Qu, H., Fan, W., Zhao, Z., & Li, Q. (2025) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2025.3599265 · Cited by 127
“This can be attributed to our generative retrieval paradigm, bypassing the most time-consuming auto-regressive decoding and beam search processes of LLMs [29].”
BioMaster: Multi-agent system for automated bioinformatics analysis workflow
Su, H., Feng, J., Lu, Y., Xu, Y., Yang, J., Lu, H., Yang, J., Yang, X., Xie, S., Long, W., et al. (2026) · Patterns · doi:10.1016/j.patter.2026.101611 · Cited by 37
Automating quantum computing laboratory experiments with an agent-based AI framework
Cao, S., Zhang, Z., Alghadeer, M., Fasciati, S. D., Piscitelli, M., Bakr, M. S., Leek, P., & Aspuru‐Guzik, A. (2025) · Patterns · doi:10.1016/j.patter.2025.101372 · Cited by 75
“Coaxmon qubits, a variant of transmon qubits with coaxial geometry and off-chip wiring, have demonstrated high coherence, low crosstalk, and stable higher excited states.”
Developing a Llama-Based Chatbot for CI/CD Question Answering: A Case Study at Ericsson
Chaudhary, D., Vadlamani, S. L., Thomas, D., Nejati, S., & Sabetzadeh, M. (2024) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme58944.2024.00075 · Cited by 29
“This approach was inspired by recent research, which indicates that LLMs utilize context most effectively when it is located at the beginning or end, with a decline in performance when the relevant context is situated in the middle of long contexts [31].”
Titans: Learning to Memorize at Test Time
Behrouz, A., Zhong, P., & Mirrokni, V. (2025) · arXiv · doi:10.48550/arxiv.2501.00663 · Cited by 24
“In complex real-world tasks (e.g., language modeling [4], video understanding [5], long-term time series forecasting [6]), the context window can become extremely large, making the applicability of Transformers challenging in these downstream tasks.”
L-Eval: Instituting Standardized Evaluation for Long Context Language Models
An, C., Gong, S., Zhong, M., Zhao, X., Li, M., Zhang, J., Kong, L., & Qiu, X. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.776 · Cited by 18
Towards lifelong dialogue agents via timeline-based memory management
Ong, K. T.-I., Kim, N., Gwak, M., Chae, H., Kwon, T., Jo, Y., Hwang, S.-W., Lee, D., & Yeo, J. (2025) · Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) · doi:10.18653/v1/2025.naacl-long.435 · Cited by 18
“While using the large context windows of recent LLMs to process all dialogue history/memories is an op-a tion to prevent such information loss, 1 this often leads to biased attention toward the latest user input (Figure 1 (b)), ignoring relevant contexts from the past (Liu et al., 2024).”
BioMaster: Multi-agent System for Automated Bioinformatics Analysis Workflow
Su, H., Long, W., & Zhang, Y. (2025) · doi:10.1101/2025.01.23.634608 · Cited by 18
Describing Differences in Image Sets with Natural Language
Dunlap, L., Zhang, Y., Wang, X., Zhong, R., Darrell, T., Steinhardt, J., Gonzalez, J. E., & Yeung, S. Y. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02284 · Cited by 80
ChatRule: Mining Logical Rules with Large Language Models for Knowledge Graph Reasoning
Luo, L., Ju, J., Xiong, B., Li, Y.-F., Haffari, G., & Pan, S. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-8173-0_25 · Cited by 95
Wang, C., Lee, B., Drucker, S. M., Marshall, D., & Gao, J. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713296 · Cited by 70
“To address these iterative chart authoring challenges, our frst key insight is to design a multi-modal chart builder that blends the shelf-confguration UI [45, 58] with natural language (NL) input to enhance users’ ability to structurally specify their chart designs.”
A Comprehensive Review on the Application of Large Language Models (LLMs) in Power Systems
Ghafari, M., Sami, A., Rezapour, H., Ghayour, S. S., Lin, F., Başaran, K., Lăzăroiu, G. C., & Siano, P. (2025) · IEEE Access · doi:10.1109/access.2025.3637226 · Cited by 29
“Models are also more sensitive to the instructions happening in the beginning and at the end of the prompts [21].”
TimeRAG: Boosting LLM Time Series Forecasting via Retrieval-Augmented Generation
Yang, S., Wang, D., Zheng, H., & Jin, R. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10889933 · Cited by 98
“Instead of storing and retrieving complete raw sequences, our sequence-segmentation approach preserves the local information of the sequence, avoids long sequences where LLMs tend to miss key information [16], and improves the retrieval efficiency.”
Prompting is all you need: LLMs for systematic review screening
(2024) · doi:10.1101/2024.06.01.24308323 · Cited by 72
Socrates, V., Wright, D. S., Huang, T., Fereydooni, S., Dien, C., Chi, L., Albano, J., Patterson, B., Sasidhar Kanaparthy, N., Wright, C. X., et al. (2025) · JMIR Aging · doi:10.2196/69504 · Cited by 17
“Separating the process of criteria filtering and application both reduces confusion due to large input context sizes [22] and ensures that extraneous context does not distract the LLM [23].”
Myers, S., Miller, T. A., Gao, Y., Churpek, M. M., Mayampurath, A. M., Dligach, D., & Afshar, M. (2024) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocae308 · Cited by 32
“Furthermore, the utilization of the full context window can cause LLMs to suffer from the lost-in-the-middle effect [13], where their ability to utilize information towards the middle of the context decreases as the length of the text increases.”
Let’s Fix this Together: Conversational Debugging with GitHub Copilot
Bajpai, Y., Chopra, B., Biyani, P., Aslan, C., Coleman, D., Gulwani, S., Parnin, C., Radhakrishna, A., & Soares, G. (2024) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vl/hcc60511.2024.00011 · Cited by 34
“In parallel, software comprehension, an important sub-task for automated software engineering tools has also seen improvements, from using longer contexts for LLMs [44], to improved code retrieval strategies [45], [46] and code execution feedback [47]–[49].”
Guidelines For Rigorous Evaluation of Clinical LLMs For Conversational ReasoningPreprint
Johri, S., Jeong, J., Tran, B. A., Schlessinger, D. I., Wongvibulsin, S., Cai, Z. R., Daneshjou, R., & Rajpurkar, P. (2023) · medRxiv · doi:10.1101/2023.09.12.23295399 · Cited by 17
“One explanation for this lack of improvement could be the models' struggle to effectively process and understand information presented over longer textual contexts 22 .”
BackgroundOctoPack: Instruction Tuning Code Large Language Models
Muennighoff, N., Liu, Q., Zebaze, A., Zheng, Q., Hui, B., Zhuo, T. Y., Singh, S., Tang, X., von Werra, L., & Longpre, S. (2023) · arXiv · doi:10.48550/arxiv.2308.07124 · Cited by 245
SciSciGPT: advancing human–AI collaboration in the science of science
Shao, E., Wang, Y., Qian, Y., Pan, Z., Liu, H., & Wang, D. (2025) · Nature Computational Science · doi:10.1038/s43588-025-00906-6 · Cited by 76
“…SciSciGPT employs several mechanisms to compress the context, optimize prompt quality, minimize redundancy and improve computational efficiency 60,61 by pruning content less relevant to ongoing reasoning: (1) The Research-Manager acts as the system’s coordinator and maintains a complete…”
Large language models for causal hypothesis generation in science
Cohrs, K.-H., Díaz, E., Sitokonstantinou, V., Varando, G., & Camps‐Valls, G. (2024) · Machine Learning Science and Technology · doi:10.1088/2632-2153/ada47f · Cited by 37
“Nonetheless, aggregating over all permutations is a sensible approach [32, 143], mitigating the‘lost in the middle’ problem, describing that LLMs give more importance to information at the start and the end of a prompt than to information in between [92].”
Deng, Y., Tu, Z., Yao, J.-H., Zhang, M.-S., Zhang, T., & Zheng, X. (2023) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3569086 · Cited by 30
“Each module generates concise prompts to reduce the length of input context [25], [26] and mitigate hallucination risks [27], [28], [29].”
Enhancing RAG Performance Through Chunking and Text Splitting Techniques
Kshirsagar, A. (2024) · International Journal of Scientific Research in Computer Science Engineering and Information Technology · doi:10.32628/cseit2410593 · Cited by 24
A Survey of Reasoning with Foundation ModelsPreprint
Sun, J., Zheng, C., Xie, E., Liu, Z., Chu, R., Liu, J., Xu, J., Ding, M., Li, H., Geng, M., et al. (2023) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/ac4sp · Cited by 158
A Survey on Speech Large Language Models for Understanding
Peng, J., Wang, Y., Li, B., Guo, Y., Wang, H., Fang, Y., Xi, Y., Li, H., Li, X., Zhang, K., et al. (2025) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2025.3640535 · Cited by 152
“Instruction sensitivity in LLMs has been extensively explored within natural language processing (NLP), where it is widely regarded as a key indicator of model robustness [175], [176].”
Lu, M., Gao, F., Tang, X., & Chen, L. (2024) · iScience · doi:10.1016/j.isci.2024.109451 · Cited by 23
“28 For using CoTs to infer experimental performance, the initial interaction of UM3 or UM4 (as depicted in Figures S2 and S3, respectively) in the front-user message for context-aware adaptation employed tactics I , II , IV , and a .”
MethodIn Prospect and Retrospect: Reflective Memory Management for Long-term Personalized Dialogue Agents
Tan, Z., Yan, J., Hsu, I.-H., Han, R., Wang, Z., Le, L., Song, Y., Chen, Y., Palangi, H., Lee, G., et al. (2025) · Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2025.acl-long.413 · Cited by 219
Cheng, Y., Bajaber, O., Tsegai, S. A., Song, D. X., & Gao, P. (2025) · IEEE European Symposium on Security and Privacy · doi:10.1109/eurosp63326.2025.00057 · Cited by 83
“These methods were chosen to explore the impact of recency bias in LLMs, which suggests that models give more weight to examples placed nearer to the query [60].”
End-to-end Task-oriented Dialogue: A Survey of Tasks, Methods, and Future Directions
Qin, L., Pan, W., Chen, Q., Liao, L., Zhou, Y., Zhang, Y., Che, W., & Li, M. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.363 · Cited by 77
Płonka, M., Kocot, K., Hołda, K., Daniec, K., & Nawrat, A. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.126711 · Cited by 32
The Fact Selection Problem in LLM-Based Program Repair
Parasaram, N., Yan, H., Yang, B., Flahy, Z., Qudsi, A., Ziaber, D., Barr, E. T., & Mechtaev, S. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00162 · Cited by 46
“This finding is non-obvious, because each fact may contain crucial information for fixing the bug, and although previous research showed that LLMs do not robustly make use of information in long input contexts [11], Fig.”
Learning long sequences in spiking neural networks
Stan, M.-I., & Rhodes, O. (2024) · Scientific Reports · doi:10.1038/s41598-024-71678-8 · Cited by 55
“…aplications ranging from processing biosignals such as electroencephalograms spanning tens of thousands of time steps [Tang et al., 2023], to comprehending and potentially writing large documents (e.g., novels, scientific papers) using large language models [Zhou et al., 2023, Liu et al., 2023].”
MethodWhy Do Multi-Agent LLM Systems Fail?
Cemri, M., Pan, M. Z., Yang, S., Agrawal, L. A., Chopra, B., Tiwari, R., Keutzer, K., Parameswaran, A., Klein, D., Ramchandran, K., et al. (2025) · arXiv · doi:10.48550/arxiv.2503.13657 · Cited by 630
ContextCite: Attributing Model Generation to Context
Cohen-Wang, B., Shah, H., Georgiev, K., & Madry, A. (2024) · arXiv · doi:10.48550/arxiv.2409.00729 · Cited by 16
Harvel, N., Haiek, F. B., Ankolekar, A., & Brunner, D. J. (2024) · Proceedings of the AAAI Symposium Series · doi:10.1609/aaaiss.v3i1.31191 · Cited by 21
“Unfortunately, LLMs do not attend to the entirety of the context equally and tend to focus more on information located at the beginning and end of the context (Liu et al. 2023).”
BackgroundAutoIOT: LLM-Driven Automated Natural Language Programming for AIoT Applications
Shen, L., Yang, Q., Zheng, Y., & Li, M. (2025) · ACM Annual International Conference on Mobile Computing and Networking (MobiCom) · doi:10.1145/3680207.3723486 · Cited by 48
“.”
MoRSE: Bridging the Gap in Cybersecurity Expertise with Retrieval Augmented Generation
Simoni, M., Saracino, A., Vinod, P., & Conti, M. (2025) · ACM Symposium on Applied Computing · doi:10.1145/3672608.3707898 · Cited by 40
“𝑖 𝑖 𝑖”
Clinicians must participate in the development of multimodal AI
Banerji, C. R. S., Shah, A. B., Dabson, B., Chakraborti, T., Hellon, V., Harbron, C. G., & MacArthur, B. D. (2025) · EClinicalMedicine · doi:10.1016/j.eclinm.2025.103252 · Cited by 24
“32 Recent advances have been made in addressing this problem, such as Google ’ s Gemini 1.5 11 and retrieval-augmented generation models, 33 however they still signi fi cantly underperform compared to human long context gap retrieval.”
Influential(Generative) AI in Financial Economics
Mo, H., & Ouyang, S. (2025) · Journal of Chinese Economic and Business Studies · doi:10.1080/14765284.2025.2569006 · Cited by 46
Can artificial intelligence diagnose seizures based on patients' descriptions? A study of GPT ‐4
Ford, J., Pevy, N., Grünewald, R. A., Howell, S. J., & Reuber, M. (2025) · Epilepsia · doi:10.1111/epi.18322 · Cited by 17
“30 The quality of the training examples (which were randomly selected) could also have had an impact, although we did exclusively train the model with examples that all neurologists had diagnosed correctly.”
TourRank: Utilizing Large Language Models for Documents Ranking with a Tournament-Inspired Strategy
Chen, Y., Liu, Q., Zhang, Y., Sun, W., Ma, X., Yang, W., Shi, D., Mao, J., & Yin, D. (2025) · International World Wide Web Conference (WWW) · doi:10.1145/3696410.3714863 · Cited by 85
Query Rewriting for Retrieval-Augmented Large Language Models
Ma, X., Gong, Y., He, P., Zhao, H., & Duan, N. (2023) · arXiv · doi:10.48550/arxiv.2305.14283 · Cited by 14
“This is consistent with the findings in related search (Mallen et al., 2022; Liu et al., 2023).”
ResultCompAct: Compressing Retrieved Documents Actively for Question Answering
Yoon, C., Lee, T., Hwang, H., Jeong, M., & Kang, J. (2024) · arXiv · doi:10.48550/arxiv.2407.09014 · Cited by 14
Instruction Position Matters in Sequence Generation with Large Language Models
Liu, Y., Zeng, X., Meng, F., & Zhou, J. (2023) · arXiv · doi:10.48550/arxiv.2308.12097 · Cited by 14
Li, Y., Qi, S., Gao, C., Peng, Y., Lo, D., Lyu, M. R.-T., & Xu, Z. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3524461 · Cited by 21
“This observation aligns with the concept of Primacy Bias, as discussed in [68], which refers to the tendency of models to better retain and utilize information appearing earlier in the input while struggling to effectively process information in the middle of long contexts.”
Towards Fair RAG: On the Impact of Fair Ranking in Retrieval-Augmented Generation
Kim, T. E., & Díaz, F. (2025) · ACM SIGIR International Conference on Theory of Information Retrieval (ICTIR) · doi:10.1145/3731120.3744599 · Cited by 45
Gomez-Cabello, C. A., Prabha, S., Haider, S. A., Genovese, A., Collaço, B. G., Wood, N. G., Bagaria, S. P., & Forte, A. J. (2025) · Bioengineering · doi:10.3390/bioengineering12111194 · Cited by 38
Simplifying software compliance: AI technologies in drafting technical documentation for the AI Act
Sovrano, F., Hine, E., Anzolut, S., & Bacchelli, A. (2025) · Empirical Software Engineering · doi:10.1007/s10664-025-10645-x · Cited by 35
“…ChatGPT, are capable of processing extensive inputs, they often struggle to fully leverage information within a lengthy context, a challenge referred to as the lost-in-the-middle or lost-in-the-end issue (Liu et al. 2024; Sovrano et al. 2025; An et al. 2024; Xu et al. 2024; Levy et al. 2024).”
Knowing You Don't Know: Learning When to Continue Search in Multi-round RAG through Self-Practicing
Yang, D., Zeng, L., Rao, J., & Zhang, Y. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730018 · Cited by 41
AU-RAG: Agent-based Universal Retrieval Augmented Generation
Jang, J., & Li, W.-S. (2024) · Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region (SIGIR-AP) · doi:10.1145/3673791.3698416 · Cited by 22
“Liu et al. [10] show that model performance decreases when relevant information is in the middle of its input context.”
Xu, S., Chen, M., & Chen, S. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-5678-0_34 · Cited by 23
JavaBench: A Benchmark of Object-Oriented Code Generation for Evaluating Large Language Models
Cao, J., Chen, Z., Wu, J., Cheung, S.-C., & Xu, C. (2024) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3691620.3695470 · Cited by 62
“A code project has a long context (documentation and codes), which usually exceeds LLMs’ context limit or exposes performance degradation [43].”
Metadata
- Authors
- Liu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., & Liang, P.
- Affiliations
- Stanford University (US)
- University of California, Berkeley (US)
- Samaya AI, UK
- Corresponding author
- Liu, Nelson F.
- Venue
- Transactions of the Association for Computational Linguistics
- Publisher
- MIT Press
- Published
- 2024
- Volume
- 12
- Pages
- 157-173
- ISSN
- 2307-387X (ISSN-L)
- OpenAlex
- W4391876619
- Semantic Scholar
- 1733eb7792f7a43dd21f51f4d1017a1bffd217b5
- Cited by
- 1,612
- References
- 59
- Citations by year
- 2026: 901 · 2025: 516 · 2024: 172
- FWCI
- 233.95
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- Locations
- Transactions of the Association for Computational Linguistics (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- long-context language models, multi-document question answering, position bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-09-27 (prompt extract-v2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.