Found in the middle: Calibrating Positional Attention Bias Improves Long Context Utilization
Open accessAbstract
Cheng-Yu Hsieh, Yung-Sung Chuang, Chun-Liang Li, Zifeng Wang, Long Le, Abhishek Kumar, James Glass, Alexander Ratner, Chen-Yu Lee, Ranjay Krishna, Tomas Pfister. Findings of the Association for Computational Linguistics: ACL 2024. 2024.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Model performance drops significantly when N (top N positions) becomes larger than 3.
“The threshold N is set to 3 since model performance drops significantly when N becomes larger than 3, according to earlier studies24,27,28.”
From Leveraging long context in retrieval augmented language models for medical question answering (2025) · cites this paper as [28]
The "lost-in-the-middle" issue is attributed to positional attention bias, where more attention weights are allocated to information at spotlight positions than others.
“Current studies attribute this issue to positional attention bias, i.e., more attention weights are allocated more to information at spotlight positions than others27–29.”
From Leveraging long context in retrieval augmented language models for medical question answering (2025) · cites this paper as [28]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 21
- Dzmitry Bahdanau, Kyung Hyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In 3rd Inter- national Conference on Learning Representations, ICLR 2015.
- Sebastian Borgeaud, Arthur Mensch, Jordan Hoff- mann, Trevor Cai, Eliza Rutherford, Katie Milli- can, George Bm Van Den Driessche, Jean-Baptiste Lespiau, Bogdan Damoc, Aidan Clark, et al. 2022. Improving language models by retrieving from tril- lions of tokens. In International conference on ma- chine learning, pages 2206-2240. PMLR.
- Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901.
- Tri Dao, Dan Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. 2022. Flashattention: Fast and memory-efficient exact attention with io-awareness. Advances in Neural Information Processing Systems, 35:16344-16359.
- Yaru Hao, Li Dong, Furu Wei, and Ke Xu. 2021. Self-attention attribution: Interpreting information interactions inside transformer. In Proceedings of the AAAI Conference on Artificial Intelligence, vol- ume 35, pages 12963-12971.
- Junqing He, Kunhao Pan, Xiaoqun Dong, Zhuoyang Song, Yibo Liu, Yuxin Liang, Hao Wang, Qianguo Sun, Songxin Zhang, Zejian Xie, et al. 2023. Never lost in the middle: Improving large language mod- els via attention strengthening question answering. arXiv e-prints, pages arXiv-2311.
- Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023. Survey of hallu- cination in natural language generation. ACM Com- puting Surveys, 55(12):1-38.
- Urvashi Khandelwal, Omer Levy, Dan Jurafsky, Luke Zettlemoyer, and Mike Lewis. 2020. Generalization through memorization: Nearest neighbor language models. In International Conference on Learning Representations.
- Tom Kwiatkowski, Jennimaria Palomaki, Olivia Red- field, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, et al. 2019. Natural questions: a bench- mark for question answering research. Transactions of the Association for Computational Linguistics, 7:453-466.
- Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Ku- mar, Benjamin Newman, Binhang Yuan, Bobby Yan, Ce Zhang, Christian Alexander Cosgrove, Christo- pher D Manning, Christopher Re, Diana Acosta- Navas, Drew Arad Hudson, Eric Zelikman, Esin Durmus, Faisal Ladhak, Frieda Rong, Hongyu Ren, Huaxiu Yao, Jue WANG, Keshav Santhanam, Laurel Orr, Lucia Zheng, Mert Yuksekgonul, Mirac Suzgun, Nathan Kim, Neel Guha, Niladri S. Chatterji, Omar Khattab, Peter Henderson, Qian Huang, Ryan An- drew Chi, Sang Michael Xie, Shibani Santurkar, Surya Ganguli, Tatsunori Hashimoto, Thomas Icard, Tianyi Zhang, Vishrav Chaudhary, William Wang, Xuechen Li, Yifan Mai, Yuhui Zhang, and Yuta Ko- reeda. 2023. Holistic evaluation of language models. Transactions on Machine Learning Research. Fea- tured Certification, Expert Certification.
- Minh-Thang Luong, Hieu Pham, and Christopher D Manning. 2015. Effective approaches to attention- based neural machine translation. In Proceedings of the 2015 Conference on Empirical Methods in Natu- ral Language Processing, pages 1412-1421.
- Sewon Min, Suchin Gururangan, Eric Wallace, Han- naneh Hajishirzi, Noah A. Smith, and Luke Zettle- moyer. 2024. SILO language models: Isolating le- gal risk in a nonparametric datastore. In ICLR.
- Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Wei Koh, Mohit Iyyer, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2023. FActScore: Fine-grained atomic evaluation of fac- tual precision in long form text generation. In EMNLP.
- Ofir Press, Noah Smith, and Mike Lewis. 2021. Train short, test long: Attention with linear biases enables input length extrapolation. In International Confer- ence on Learning Representations.
- Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang. 2023. Hugging- gpt: Solving ai tasks with chatgpt and its friends in huggingface. In Advances in Neural Information Processing Systems.
- Freda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales, David Dohan, Ed H Chi, Nathanael Schärli, and Denny Zhou. 2023a. Large language models can be easily distracted by irrelevant context. In In- ternational Conference on Machine Learning, pages 31210-31227.
- PMLR.
- Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information process- ing systems, 30.
- Yizhong Wang, Hamish Ivison, Pradeep Dasigi, Jack Hessel, Tushar Khot, Khyathi Chandu, David Wad- den, Kelsey MacMillan, Noah A Smith, Iz Beltagy, et al. 2023. How far can camels go? exploring the state of instruction tuning on open resources. Ad- vances in Neural Information Processing Systems, 36:74764-74786.
- Zilong Wang, Hao Zhang, Chun-Liang Li, Julian Mar- tin Eisenschlos, Vincent Perot, Zifeng Wang, Lesly Miculicich, Yasuhisa Fujii, Jingbo Shang, Chen- Yu Lee, and Tomas Pfister. 2024. Chain-of-table: Evolving tables in the reasoning chain for table un- derstanding. In International Conference on Learn- ing Representations.
- Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. 2023. Efficient streaming lan- guage models with attention sinks. arXiv.
Cited by
Sorted by citations
Leveraging long context in retrieval augmented language models for medical question answering
Zhang, G., Xu, Z., Jin, Q., Chen, F., Fang, Y., Liu, Y., Rousseau, J. F., Xu, Z., Lu, Z., Weng, C., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01651-w · Cited by 81 · see citing works
Xu, B., Wu, G., Li, Z., Xu, G., Zeng, H., Tong, R., & Ng, H. Y. (2025) · npj Clean Water · doi:10.1038/s41545-025-00509-8 · Cited by 23 · see citing works
Merola, C., & Singh, J. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-02899-0_1 · Cited by 19 · see citing works
Sammour, F., Xu, J., Wang, X., Hu, M., & Zhang, Z. (2025) · Journal of Construction Engineering and Management · doi:10.1061/jcemd4.coeng-16639 · Cited by 26 · see citing works
Towards Fair RAG: On the Impact of Fair Ranking in Retrieval-Augmented Generation
Kim, T. E., & Díaz, F. (2025) · ACM SIGIR International Conference on Theory of Information Retrieval (ICTIR) · doi:10.1145/3731120.3744599 · Cited by 20 · see citing works
Zabihi, P., Nawara, D., Ibrahim, A., & Kashef, R. F. (2026) · Applied Sciences · doi:10.3390/app16073410 · Cited by 2 · see citing works
ALAS: A Stateful Multi-LLM Agent Framework for Disruption-Aware Planning
Chang, E. Y. (2025) · ACM eBooks · doi:10.1145/3749421.3749436 · Cited by 1 · see citing works
Understanding Hallucinations in Large Visual and Language Models
Ho, Z. Y., Liang, S., & Tao, D. (2026) · ACM Computing Surveys · doi:10.1145/3811409 · Cited by 3 · see citing works
Weighted Memory Tree: Remembering What Matters for Long-Horizon LLM Agents
Dao, Q., Kathalkar, P., & Eaton, K. (2026) · arXiv · doi:10.48550/arxiv.2608.20631
Guan, Z., Liang, X., & Zhang, S. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10890215
LLM-Augmented Spatio-Temporal Graph Learning for POI Recommendation
Wang, J., Wang, Z., Lu, T., & Yan, C. (2025) · Lecture notes in computer science · doi:10.1007/978-981-95-3462-3_38
Self-citationSagaLLM: Persistent Context Management, Constraint Validation, and Transaction Guarantees
Chang, E. Y. (2025) · ACM eBooks · doi:10.1145/3749421.3749435
ATACompressor: Adaptive Task-Aware Compression for Efficient Long-Context Processing in LLMs
Li, X., Li, H., Zhou, Y., Ai, Q., & Liu, Y. (2025) · Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region (SIGIR-AP) · doi:10.1145/3767695.3769499
Who Benefits from RAG? The Role of Exposure, Utility and Attribution Bias
Dehghan, M., & McDonald, G. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-21289-4_19
From Single to Multi-agent Reasoning: Advancing GeneGPT for Genomics QA
Abedini, K., Shami, F., & Silvello, G. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-21300-6_49
Self-Consistency Falls Short! The Adverse Effects of Positional Bias on Long-Context Problems
Byerly, A., & Khashabi, D. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl.a.625 · Cited by 10 · see citing works
Brevik, A., Jerpseth, H., & Lafontan, S. R. (2026) · Frontiers in Education · doi:10.3389/feduc.2026.1904452
Investigating Large Language Model-Based Decision Making for Deep Space Habitat Systems
Manikkan, S., Reimer, M., Mirfarah, M., Chu, Z., Chi, M., Silva, C. E., Dongare, L., Bilionis, I., & Dyke, S. J. (2026) · IEEE Aerospace Conference (AERO) · doi:10.1109/aero66936.2026.11520161
Shi, F., Xu, W., Shi, J., Ji, X., & Wang, J. (2026) · Communications in computer and information science · doi:10.1007/978-981-95-9507-5_12
Chen, Y., Zeng, Z., & Hassan, M. S. (2026) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2026.1784200
Purifying Multimodal Retrieval: Fragment-Level Evidence Selection for RAG
Wang, X., Wang, Z., Huang, C., Liu, C., Zeng, K., Sheng, Q. Z., & Yao, L. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3809692
Self-citationDiagnosing and Mitigating Mid-Sequence Degradation in Recommender Systems
Guo, L., Bisht, N., Wu, S., Huo, H., Wang, X., & Xu, G. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3809738
LLM-Oriented Information Retrieval: A Denoising-First Perspective
Dai, L., Sun, L., Cao, F., Rao, Z., Yang, C., Liu, H., & Xiong, H. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3808544
RetailSQL: A Chinese Text-to-SQL Dataset for Retail Industry
Zhang, W., Ji, B., Li, Q., & Li, J. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3435-6_25
Stochastic Retrieval for Fairness–Accuracy Trade-offs in Clinical RAG
Lunardi, R., Mea, V. D., Eickhoff, C., & Roitero, K. (2026) · ACM International Conference on Bioinformatics, Computational Biology and Biomedicine · doi:10.1145/3807503.3819374
Tekale, A., & Masum, M. (2026) · IEEE Annual International Computer Software and Applications Conference (COMPSAC) · doi:10.1109/compsac69091.2026.00037
Task-Aligned Contrastive Learning for Filtering Noise in Multi-label Text Classification
Metzger, A., Dantu, R., Quach, V., & Blackwell, A. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-35078-7_6
Николаевич, Г. В. (2026) · Modelirovanie, optimizaciâ i informacionnye tehnologii. · doi:10.26102/2310-6018/2026.60.9.006
Making Large Vision–Language Models Better Few-Shot Learners
Zhang, C., Liu, F., Xu, Y., An, Y., Cao, S., & Deng, C. D. (2026) · IEEE Transactions on Image Processing · doi:10.1109/tip.2026.3732763
The Fusion Tax: A Diagnostic Analysis of When Hybrid Retrieval Hurts
Chauhan, A., & Agarwal, A. (2026) · IEEE Access · doi:10.1109/access.2026.3743502
CAUS-RAG: Algorithmic Interceptor for Mitigating Positional Bias in Technical Code Generation
Martin, M., Roy, A., & Shalini, T. (2026) · International Conference on Networks, Computers and Communications (ICNCCom) · doi:10.1109/icnccom69117.2026.11736252
Yavuz, Ö. T., Coskun, E., & Beşen, İ. (2026) · International Artificial Intelligence and Data Processing Symposium (IDAP) · doi:10.1109/idap71869.2026.11728004
- Authors
- Hsieh, C.-Y., Chuang, Y.-S., Li, C.-L., Wang, Z., Le, L., Kumar, A., Glass, J., Ratner, A., Lee, C.-Y., Krishna, R., & Pfister, T.
- Affiliations
- University of Washington (US)
- Massachusetts Institute of Technology (US)
- Google (United States) (US)
- Venue
- Findings of the Association for Computational Linguistics ACL 2024
- Published
- 2024
- Pages
- 14982-14995
- OpenAlex
- W4402684028
- Cited by
- 31
- References
- 0
- Citations by year
- 2026: 21 · 2025: 10
- FWCI
- 9.15
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Findings of the Association for Computational Linguistics: ACL (Findings of ACL) (publishedVersion, cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- transformer models, positional encoding, long-range dependencies
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
