The DOI matches its DataCite record; 59 of the 59 references it cites were found in a registry or an index; 10 of the 15 sources its claims cite were checked for support.
Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization
Open accessAbstract
Large language models (LLMs), even when specifically trained to process long input contexts, struggle to capture relevant information located in the middle of their input. This phenomenon has been known as the lost-in-the-middle problem. In this work, we make three contributions. First, we set out to understand the factors that cause this phenomenon. In doing so, we establish a connection between lost-in-the-middle to LLMs' intrinsic attention bias: LLMs exhibit a U-shaped attention bias where the tokens at the beginning and at the end of its input receive higher attention, regardless of their relevance. Second, we mitigate this positional bias through a calibration mechanism, found-in-the-middle, that allows the model to attend to contexts faithfully according to their relevance, even though when they are in the middle. Third, we show found-in-the-middle not only achieves better performance in locating relevant information within a long context, but also eventually leads to improved retrieval-augmented generation (RAG) performance across various tasks, outperforming existing methods by up to 15 percentage points. These findings open up future directions in understanding LLM attention bias and its potential consequences.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
33 Own result · 10 Own interpretation · 6 Own method · 3 Supported · 3 Partly supported · 2 No full text of the sources · 1 Couldn’t confirm · 1 No basis given · accuracy not yet measured
Every source the paper cites for this claim was checked and supports it.
In the paper
“This correlates well with the U-shaped RAG performance observed in prior literature (Liu et al., 2023).”
§ Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization > 1 Introduction
What the sources say
- Supportedcites Liu et al., 2023 source
The Introduction reports a U-shaped performance curve: performance is highest when relevant information is at the beginning or end of the input and degrades when it is in the middle. The multi-document question-answering results (Appendix C) show the same pattern with retrieved search results. The comparison with an attention bias is the citing authors’ own framing, not a finding attributed to Liu et al. (LLM, conf=0.91)
boostedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Large language models (LLMs), even when specifically trained to process long input contexts, struggle to capture relevant information located in the middle of their input.”
- Also stated in: § 1 Introduction · plain
“LLMs struggle to locate relevant documents when they are placed in the middle of their input prompts (Liu et al., 2023;Li et al., 2023a).”
§ Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization > Abstract
What the sources say
- Supportedcites Liu et al., 2023 source
The Abstract reports that performance significantly degrades when relevant information is in the middle of a long context, even for explicitly long-context models. This supports the citing sentence’s point about models designed for long inputs. (LLM, conf=0.92)
- Not matchedcites Li et al., 2023a
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 1 Introduction · plain
Every source the paper cites for this claim was checked and supports it.
In the paper
“Interestingly, this focus on the beginning and end also extends to content utilization: models preferentially use information from the beginning and end of their prompts (Ravaut et al., 2023;Peysakhovich and Lerer, 2023).”
§ Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization > 1 Introduction
What the sources say
- Supportedcites Ravaut et al., 2023 source
Section 3.2 reports that sentences from the first and last 10% of the input are much more represented in LLM summaries than sentences elsewhere. Section 4's controlled experiments likewise find a performance drop when salient information is placed away from the first or final position. The claim is a fair paraphrase of these findings about summarization prompts. (LLM, conf=0.93)
- Supportedcites Peysakhovich and Lerer, 2023 source
Section 3 reports that accuracy varies with the relevant document’s position and describes a preference for information at the beginning and end of the context. It qualifies this pattern: in longer contexts, the beginning-of-context preference weakens and recency bias predominates. (LLM, conf=0.91)
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Nonetheless, even models specifically trained for long-context suffer lost-in-the-middle problem (Liu et al., 2023;Li et al., 2023a).”
§ Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization > Long-context utilization in language models.
What the sources say
- Supportedcites Liu et al., 2023 source
The Abstract states that performance degrades when relevant information is in the middle of long contexts, even for explicitly long-context models. The Introduction also reports that extended-context models are not necessarily better at using their input context. (LLM, conf=0.94)
- Not matchedcites Li et al., 2023a
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“While these existing solutions lead to improved model performances by manipulating the input contexts, they do not fundamentally improve LLMs' underlying long-context utilization capability.”
§ Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization > Long-context utilization in language models.
What the sources say
- Partly supportedcites Chen et al., 2023 sourceExperimental — accuracy not yet measured
The Introduction describes approaches that extend or select input context, then says they remain limited by design; the Context window scaling section identifies further limits in long-sequence attention and recurrence. This supports the concern about persistent long-context limitations, but the paper does not establish the broader claim that these methods improve performance without fundamentally improving LLMs’ underlying long-context utilization capability. (LLM, conf=0.82)
- Partly supportedcites Jiang et al., 2023Experimental — accuracy not yet measured
The Abstract and Section 7 (Conclusion; page unavailable) report that prompt compression improves performance, and Section 5 (Experiments; page unavailable) compares methods that manipulate input contexts. The paper does not establish that such methods fail to fundamentally improve LLMs’ underlying long-context utilization capability; it also reports that some existing compression methods perform poorly. (LLM, conf=0.82)
- Partly supportedcites Peysakhovich and Lerer, 2023 sourceExperimental — accuracy not yet measured
The Conclusion and Limitations (page unavailable) describes attention sorting as an inference-time method that modifies a model’s context to improve task performance, and suggests joint fine-tuning as a more principled solution. This supports the distinction for attention sorting, but the paper does not establish that existing context-manipulation methods generally fail to improve underlying long-context utilization capability. (LLM, conf=0.83)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This bias persists whether the input documents retain their original order or are randomly shuffled.”
§ Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization > 2.1 U-shaped attention bias
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This calibrated attention proves to be better correlated to the ground truth relevance of the document to a user query.”
§ Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization > 1 Introduction
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- The authors state this as an assumption: A dummy document x_dum has a constant relevance rel(x_dum).
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Observations indicate that model performance significantly decreases when x gold is placed within the middle of the input prompt (i.e., x doc ⌊k/2⌋ ), compared to scenarios where x gold is placed at the beginning or end.”
§ Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization > 2 Positional attention bias overpowers mid-sequence context
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This dependence persists even after randomly shuffling the document order, irrespective of its relevance to the query.”
§ Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization > 2 Positional attention bias overpowers mid-sequence context
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We show that calibrating the attention leads to improvements across two popular LLMs with different context window lengths on two RAG tasks.”
§ Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization > 1 Introduction
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Probably not a substantial claim. 11
- It may state two separate claims as one. 15
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“However, the relationship between the lost-inthe-middle problem and LLM's self-attention has been under-explored.”
§ Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization > Long-context utilization in language models.
What the sources say
- Could not verifycites Peysakhovich and Lerer, 2023 sourceExperimental — accuracy not yet measured
The Introduction discusses the lost-in-the-middle effect and self-attention, and Section 3 examines attention weights alongside that effect. Neither passage says whether the relationship had been under-explored in prior research; searches for that characterization and for discussion of prior work did not establish it. (LLM, conf=0.72)
- Not matchedcites He et al., 2023
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
- Could not verify
Every source the paper cites for this claim was checked and supports it.
In the paper
“All the documents in SynthWiki are GPT-4 generated Wikipedia paragraphs for fictional people, thus it can minimize the knowledge contamination issue from pre-training and ensure the LLMs can only use information from the provided context.”
§ Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization > Additional information
What the sources say
- Supportedcites Peysakhovich and Lerer, 2023 source
Section 2 says SynthWiki uses GPT-4 to generate one-paragraph Wikipedia-style articles about fictional people and was designed to minimize pre-training knowledge contamination. The Introduction says that, because the task data is generated, models cannot use prior knowledge to help answer its questions and must rely on the supplied documents. (LLM, conf=0.91)
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
- Supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Quantitatively, the model's response strongly depends on the document at the first position.This dependence persists even after randomly shuffling the document order, irrespective of its relevance to the query.”
§ Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization > Ethical Statement
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- The authors state this as an assumption: The positional attention bias can be modeled as a simple additive combination of document relevance and position bias (Eq. 1).
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In the experiments, we find attention calibration to be robust to the temperature term t in Eq. 5.”
§ Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization > Additional information
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- The authors state this as an assumption: The positional attention bias can be modeled as a simple additive combination of document relevance and position bias (Eq. 1).
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We see that the proposed calibrated attention consistently outperforms vanilla attention by a large margin, and also shows superior performances when compared to the other two re-ranking metrics.”
§ Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization > 3.2 Disentangling positional attention bias
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- The authors state this as an assumption: A dummy document x_dum has a constant relevance rel(x_dum).
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Models do attend to relevant contexts even when placed in the middle, but are eventually distracted by leading/ending contexts”
§ Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization > Ethical Statement
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- The authors state this as an assumption: The positional attention bias can be modeled as a simple additive combination of document relevance and position bias (Eq. 1).
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Experiments show that attention calibration improves the performance compared to its uncalibrated counterpart especially when relevant context occurs in the middle of the input.”
- Also stated in: § Ethical Statement · plain
“Attention calibration effectively improves models' context utilization ability, with its performance curves lying almost entirely above standard vanilla attention (on 22 out of 24 cases).”
- Also stated in: § Ethical Statement · plain
“On the most challenging settings where the gold documents are placed in the middle, attention calibration provides 6-15 points improvements.”
§ Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization > 6 Discussion
plaindirectn = 24- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- The authors state this as an assumption: The positional attention bias can be modeled as a simple additive combination of document relevance and position bias (Eq. 1).
- Also stated in: § Ethical Statement · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Qualitatively, the model's response exhibits a strong bias towards the document at the first position (red).This persists whether the input documents retain their original order (left: gold document at the 10th position) or are randomly shuffled (middle: gold document at the 13th position).”
§ Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization > Ethical Statement
boostedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- The authors state this as an assumption: The positional attention bias can be modeled as a simple additive combination of document relevance and position bias (Eq. 1).
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In Appendix C, we also explore log-linear models, which results in competitive 0.75 rank correlation.”
- Also stated in: § Additional information · plain
“The log-linear model and linear are competitive to each other, which all result in rank correlation above 0.75.”
§ Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization > Ethical Statement
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- The authors state this as an assumption: The positional attention bias can be modeled as a simple additive combination of document relevance and position bias (Eq. 1).
- Also stated in: § Additional information · plain
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We thus conjecture that lost-in-the-middle happens because of the dominating force of positional bias.”
§ Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization > Observation 2: Model prioritizes highlyweighted documents for generation.
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This contrasts with instances where x doc 10 is uncontrolled, suggesting that apart from U-shaped positional bias, the model exhibits an ability to prioritize relevant context.”
§ Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization > 2.2 Does attention favor relevant context?
hedgedcorrelationalSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In Sec. 2, we find that there are two main forces driving the model attention assigned to different documents of an input prompt: (a) where the document locates within the entire input, and (b) the relevance of the document.”
§ Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization > 3.1 Two main factors in model attention
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Our method is a pure inference method, so there is no need to do training or hyperparameter searching.”
§ Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization > Additional information
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“We ensure that the use of these two datasets was consistent with their intended purpose for academic research and in accordance with their specified licensing agreements.”
§ Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization > Additional information
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“ensuring that no personal or sensitive data was collected or utilized.”
§ Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization > Ethical Statement
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These results suggest that attention calibration can more fundamentally improve models' context utilization, providing a complementary way to re-ordering methods to further improve current RAG pipeline.”
§ Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization > 4.3 Attention calibration in practice
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“We run our experiments with all greedy decoding without any nondeterministic factor, so we only need to run the experiments for once.”
§ Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization > Additional information
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“These two dataset should not contain any information that names or uniquely identifies individual people or offensive content.”
§ Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization > Additional information
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Unlike the method, which necessitate significant investment in data collection and LLM tuning, our method offers an efficient solution by mitigating lost-in-the-middle problem with off-theshelf LLMs.”
§ Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization > Long-context utilization in language models.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We thus consider modeling the observable attention weights allocated to the k-th document of an input x prompt as: Attn(x prompt , k) = f (rel(x doc k ), bias(k)), (1 ) where rel(•) measures the relevance of an input document, bias(•) characterizes the positional attention bias, and f (•) is some unknown monotonically increasing function w.r.t. to both rel(x doc k ) and bias(k).”
§ Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization > Our hypothesis.
plaindirect- Asserted: fails — The authors may not put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The authors may not put it forward as true. The paper may give no reason why it holds.
- The authors state this as an assumption: The function f is monotonically increasing in both relevance and positional bias.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“However, our work does not definitively pinpoint the root cause of attention bias in LLMs.”
§ Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization > Computational overhead.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- The authors state this as an assumption: The positional attention bias can be modeled as a simple additive combination of document relevance and position bias (Eq. 1).
The paper relies on this without support.
Re-ranking methods do not fundamentally improve LLMs' ability to utilize and capture relevant information from the provided input contexts.
“Worse, re-ranking methods do not fundamentally improve LLMs' ability to utilize and capture relevant information from the provided input contexts.”
The authors state this as an assumption.
Model attention can be modeled as Attn(x_doc, k) = rel(x_doc) + bias(k) + ε.
“Attn(x doc , k) = rel(x doc ) + bias(k) + ϵ, (2) where ϵ is a noise.”
The authors state this as an assumption.
The function f is monotonically increasing in both relevance and positional bias.
“f (•) is some unknown monotonically increasing function w.r.t. to both rel(x doc k ) and bias(k).”
The authors state this as an assumption.
A dummy document x_dum has a constant relevance rel(x_dum).
“Consider using a consistent dummy document x dum which has a constant rel(x dum )”
The authors state this as an assumption.
The positional attention bias can be modeled as a simple additive combination of document relevance and position bias (Eq. 1).
“We proposed a simple hypothesis to model the positional attention bias, as shown in Eq. 1.”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization > Additional information
“The log-linear model and linear are competitive to each other, which all result in rank correlation above 0.75.”
§ Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization > Additional information
“| Log-linear | 0.75 |”
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 5
- [6]6. Sebastian Borgeaud, Arthur Mensch, Jordan Hoff- mann, Trevor Cai, Eliza Rutherford, Katie Milli- can, George Bm Van Den Driessche, Jean-Baptiste Lespiau, Bogdan Damoc, Aidan Clark, et al. 2022. Improving language models by retrieving from tril- lions of tokens. In International conference on ma- chine learning, pages 2206-2240. PMLR.
- [16]16. Junqing He, Kunhao Pan, Xiaoqun Dong, Zhuoyang Song, Yibo Liu, Yuxin Liang, Hao Wang, Qianguo Sun, Songxin Zhang, Zejian Xie, et al. 2023. Never lost in the middle: Improving large language mod- els via attention strengthening question answering. arXiv e-prints, pages arXiv-2311.
- [21]21. Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023. Survey of halluci- nation in natural language generation. ACM Comput- ing Surveys, 55(12):1-38.
- [42]42. Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang. 2023. Hugging- gpt: Solving ai tasks with chatgpt and its friends in huggingface. In Advances in Neural Information Processing Systems.
- [44]44. PMLR.
Cited by
Sorted by citations
Items 101–131 of 131
Dual-Latent Memory Routing for Vision-Language Reasoning
Ma, H.-X., Qi, J.-F., Xiao, Y.-C., et al. (2026) · doi:10.48550/arxiv.2609.05539
“…growing context, early visual evidence becomes progressively harder to reuse, which aligns with long-context retrieval failures (Liu et al., 2024; Hsieh et al., 2024) and the empirically observed visual grounding decay in MLLMs, where attention to image tokens weakens as generation proceeds and…”
@skills: Attention is all you have
Yin, L., Li, Z., Shi, Z., et al. (2026) · doi:10.48550/arxiv.2608.12610
“Subsequent work strengthened the finding: the bias is an intrinsic, U-shaped attention artifact present regardless of content relevance [37], mirrors human serial-position (primacy/recency) effects and resists prompt-level mitigation [36], and effective context is far smaller than advertised…”
Self-citationDillitzer, M., Sohn, T. S., Corso, J. J., et al. (2026) · doi:10.48550/arxiv.2608.03711
“[20] propose positional calibration to subtract a learned baseline from positional attention bias.”
An Adjoint-Sensitivity Framework for Lost-in-the-Middle Phenomena in Causal Residual Transformers
Huan, C., & Yuan, H.-W. (2026) · arXiv.org · doi:10.48550/arxiv.2607.17696
“Empirical work documents substantial position sensitivity in retrieval and question answering, often with stronger performance near the beginning or end of a context than in its middle [11, 16].”
Diagnosing and Mitigating Mid-Sequence Degradation in Recommender Systems
Guo, L., Bisht, N., Wu, S., Huo, H., Wang, X., & Xu, G. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3809738
Chen, Y., Zeng, Z., & Hassan, M. S. (2026) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2026.1784200
“Previous research indicates that when the input context becomes long, models tend to focus more on information at the beginning or end of the text, while their ability to utilize information in the middle significantly declines (Hsieh et al., 2024).”
Tekale, A., & Masum, M. (2026) · IEEE Annual International Computer Software and Applications Conference (COMPSAC) · doi:10.1109/compsac69091.2026.00037
“Techniques to improve long-context utilization remain an active area of research [13].”
Stochastic Retrieval for Fairness–Accuracy Trade-offs in Clinical RAG
Lunardi, R., Mea, V. D., Eickhoff, C., & Roitero, K. (2026) · ACM International Conference on Bioinformatics, Computational Biology and Biomedicine · doi:10.1145/3807503.3819374
Position Bias Correction is Insufficient for One-Pass Attention SortingPreprint
Tang, Q., Hu, X., Liu, X., Chen, Y., & Shao, Y. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.27793
“This U-shaped attention pattern has been attributed to intrinsic attention bias, where tokens at extreme positions receive disproportionately high attention regardless of their semantic relevance (Hsieh et al., 2024).”
Humanizing Automatically Generated Unit Test Suites with LLM-Based Refactoring
Ouédraogo, W. C., Li, Y.-H., Dang, X.-Q., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2606.28229
“This design mitigates known limitations of long-context prompting—such as the “lost-in-the-middle” effect and position-dependent performance degradation [26, 34]—while preserving essential semantics.”
CoreMem: Riemannian Retrieval and Fisher-Guided Distillation for Long-Term Memory in Dialogue Agents
Chen, J.-Q., Zeng, Y.-Q., Chen, S.-S., et al. (2026) · arXiv.org · doi:10.48550/arxiv.2606.18406
“Beyond inflating API costs linearly with conversation length, excessive context exacerbates position bias and “lost-in-the-middle” hallucinations (Liu et al., 2024; Hsieh et al., 2024).”
Dense Contexts Are Hard Contexts: Lexical Density Limits Effective Context in LLMsPreprint
Dettori, G., Boffa, M., Giordano, D., Drago, I., & Mellia, M. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.06203
FocuSFT: Bilevel Optimization for Dilution-Aware Long-Context Fine-TuningPreprint
Pei, Z., Zhen, H.-L., Yu, X., Pan, S. J., Yuan, M., & Yu, B. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.09932
“Existing remedies overwhelmingly target inference (positional calibration [16], dynamic scaling [39], test-time training [4, 32]) or require pretraining from scratch [8, 38].”
MemReread: Enhancing Agentic Long-Context Reasoning via Memory-Guided RereadingPreprint
Ji, B., Weng, X., Li, J., Tang, Z., Lou, Y., & Zhang, M. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.10268
“This performance degradation stems primarily from attention dilution in extensive contexts, which impairs the identification of critical facts and undermines coherent reasoning chains [4, 5].”
The Cost of Context: Mitigating Textual Bias in Multimodal Retrieval-Augmented Generation
Jung, H., & Wang, X.-Q. (2026) · arXiv.org · doi:10.48550/arxiv.2605.05594
“These include using dummy documents for attention calibration [8], scaling positional indices to mitigate decay [28], or reassigning positions based on semantic relevance [24].”
Jiang, S., Wang, Y., & Yuan, X. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.132623
Investigating Large Language Model-Based Decision Making for Deep Space Habitat Systems
Manikkan, S., Reimer, M., Mirfarah, M., Chu, Z., Chi, M., Silva, C. E., Dongare, L., Bilionis, I., & Dyke, S. J. (2026) · IEEE Aerospace Conference (AERO) · doi:10.1109/aero66936.2026.11520161
Lehmann, S., Wilhelmy, F., Schwaebe, F. V., von Dercks, N., Güresir, E., & Wach, J. (2026) · Applied Clinical Informatics · doi:10.1055/a-2913-8450
From Single to Multi-Agent Reasoning: Advancing GeneGPT for Genomics QA
Abedini, K., Shami, F., & Silvello, G. (2026) · European Conference on Information Retrieval · doi:10.48550/arxiv.2601.10581
“The system’s rigid dependency on specific API formats makes it fragile when interfacing with evolving tools, while its reliance on extensive context windows can lead to attention dilution and reduced focus on the original query [13,9].”
The Road of Adaptive AI for Precision in Cybersecurity
Garg, S. (2025) · arXiv.org · doi:10.48550/arxiv.2512.06048
“LLMs offer the ability to “find the needle in the haystack” [Kam-radt, 2023, Liu et al., 2023, Hsieh et al., 2024], extracting precise answers from fragmented and pressured knowledge at scale.”
MHA-RAG: Improving Efficiency, Accuracy, and Consistency by Encoding Exemplars as Soft PromptsPreprint
Jain, A., Yao, X., Reps, T., & Jermaine, C. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.05363
“Beyond these points, additional exemplars degrade effective accuracy, likely due to the inclusion of less relevant or noisy samples, consistent with findings of [23, 57, 14].”
Working Document - Formalising Software Requirements with Large Language Models
Beg, A., O'Donoghue, D., & Monahan, R. (2025) · arXiv.org · doi:10.48550/arxiv.2506.14627
“[42] reviewed the evolution of prompt engineering in LLMs, including discussions on self-consistency and multimodal prompt learning.”
InfluentialGopalakrishnan, S., & Patra, S. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2505.04732
“There is research that shows transformer attention maybe biased towards information near the start and end of the context, especially in long contexts (Lu et al. 2024; Hsieh et al. 2024).”
Guan, Z., Liang, X., & Zhang, S. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10890215
“…between lost-in-the-middle to LLMs’ intrinsic attention bias, then propose a mechanism to disentangle positional bias from model’s attention [10]. they fianlly show that calibrating the attention leads to improvements across two popular LLMs with different context window *Corresponding…”
Wang, H., Feng, Y., Xie, X.-K., et al. (2025) · arXiv.org · doi:10.48550/arxiv.2503.05203
“Due to the position encoding, LLMs exhibit significant sensitivity to token positions, a phenomenon known as position bias (Tang et al., 2023; Hsieh et al., 2024; Zhang et al., 2024a).”
Parallel Key-Value Cache Fusion for Position Invariant RAGPreprint
Oh, P., Shin, J., & Thorne, J. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2501.07523
“Token-Level Consistency Previous research (He et al., 2024; Hsieh et al., 2024b; Yu et al., 2024a) on positional bias primarily focuses on metrics like accuracy to assess position-agnostic behavior.”
DoubleDipper: Improving Long-Context LLMs via Context Recycling
Cattan, A., Jacovi, A., Fabrikant, A., et al. (2024) · doi:10.48550/arxiv.2406.13632
“…data, e.g., to incorporate longer inputs and more challenging long-context scenarios into training (Chen et al., 2024b; He et al., 2024; Chen et al., 2024a); via attention intervention (Hsieh et al., 2024) or by considering question likelihood as a signal for prompt reordering (Liu et al., 2024b).”
GRAD: Generalizing RAG Adaptation with Decoding
Lee, Y., Hwang, S.-W., Yao, Z., et al. (2026) · Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2026.acl-long.2099
“Other inference-time strategies include Hsieh et al. (2024), which tracked the average attention weights to calibrate the effect of position bias.”
Domain Adapted Text Summarization with Self-Generated Guidelines
Michail, A., Rudnikowicz, B., Fragkogiannis, P., et al. (2025) · Proceedings of the Natural Legal Language Processing Workshop 2025 · doi:10.18653/v1/2025.nllp-1.16
DoubleDipper: Recycling Contexts for Efficient and Attributed In-Context Learning
Cattan, A., Jacovi, A., Fabrikant, A., et al. (2025) · IJCNLP-AACL · doi:10.18653/v1/2025.findings-ijcnlp.42
“…data, e.g., to incorporate longer inputs and more challenging long-context scenarios into training (Chen et al., 2024b; He et al., 2024; Chen et al., 2024a); via attention intervention (Hsieh et al., 2024) or by considering question likelihood as a signal for prompt reordering (Liu et al., 2024b).”
SagaLLM: Persistent Context Management, Constraint Validation, and Transaction Guarantees
Chang, E. Y. (2025) · ACM eBooks · doi:10.1145/3749421.3749435
“Furthermore, context loss in long conversations [Hsieh et al. 2024, Liu et al. 2024a, Xiao et al. 2024, Modarressi et al. 2025] can cause LLMs to forget earlier steps, leading to contradictory decisions.”
Metadata
- Authors
- Hsieh, C.-Y., Chuang, Y.-S., Li, C.-L., Wang, Z., Le, L. T., Kumar, A., Glass, J., Ratner, A., Lee, C.-Y., Krishna, R., & Pfister, T.
- Affiliations
- University of Washington (US)
- Massachusetts Institute of Technology (US)
- Google (United States) (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2024
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2406.16008
- OpenAlex
- W4400023197
- Cited by
- 0
- References
- 0
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- long-context modeling, retrieval-augmented generation, large language models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
