The DOI matches its DataCite record; 1 of the 6 references it cites were found in a registry or an index, and 5 are not checked yet.
Extending Context Window of Large Language Models via Positional Interpolation
Open accessAbstract
We present Position Interpolation (PI) that extends the context window sizes of RoPE-based pretrained LLMs such as LLaMA models to up to 32768 with minimal fine-tuning (within 1000 steps), while demonstrating strong empirical results on various tasks that require long context, including passkey retrieval, language modeling, and long document summarization from LLaMA 7B to 65B. Meanwhile, the extended model by Position Interpolation preserve quality relatively well on tasks within its original context window. To achieve this goal, Position Interpolation linearly down-scales the input position indices to match the original context window size, rather than extrapolating beyond the trained context length which may lead to catastrophically high attention scores that completely ruin the self-attention mechanism. Our theoretical study shows that the upper bound of interpolation is at least $\sim 600 \times$ smaller than that of extrapolation, further demonstrating its stability. Models extended via Position Interpolation retain its original architecture and can reuse most pre-existing optimization and infrastructure.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“Recent works extend context windows to 100K+ tokens [18], [19], [23].”
From From Goldfish to Elephant: Long-Term Memory for Large Language Models via Selective Storage and Hierarchical Compression (2026) · cites this paper as [18]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“unlimited context remains computationally infeasible and doesn't address selective retention or consistency.”
From From Goldfish to Elephant: Long-Term Memory for Large Language Models via Selective Storage and Hierarchical Compression (2026) · cites this paper as [18]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, many model developments do not incorporate instruction tuning (Chen et al., 2023; Fu et al., 2024), leaving these models reliant on perplexity-based evaluation or synthetic tasks.”
From HELMET: How to Evaluate Long-Context Language Models Effectively and Thoroughly (2024) · cites this paper as Chen et al., 2023
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“the fundamental tension persists: more capacity does not eliminate the need for selective curation.”
From Externalization in LLM Agents: A Unified Review of Memory, Skills, Protocols and Harness Engineering (2026) · cites this paper as Chen et al., 2023
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Extending context windows to 100k+ tokens [Chen et al., 2023] delays the problem but cannot eliminate it, and longer contexts incur quadratic cost increases in attention.”
From Memory for Autonomous LLM Agents:Mechanisms, Evaluation, and Emerging Frontiers (2026) · cites this paper as Chen et al., 2023
Experimental — accuracy not yet measured
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Despite context windows stretching to 200k tokens [Chen et al., 2023], long-context models consistently underperform purpose-built memory systems on tasks requiring selective retrieval and active management.”
From Memory for Autonomous LLM Agents:Mechanisms, Evaluation, and Emerging Frontiers (2026) · cites this paper as Chen et al., 2023
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Some methods, such as PI [103] and SelfExtend [104], reorganize positional indices to ensure they remain within context lengths.”
From A Survey of Large Language Models (2026) · cites this paper as [103]
The cited source says what the claim attributes to it.
In the citing paper
“they address extrapolation of the positional code, a different concern from the entropy growth we analyse”
From Curvature-Corrected Rotary Position Embeddings: An Entropy-Invariant Temperature for Mitigating Numerical-Rank Collapse in Long-Context Attention (2026) · cites this paper as [4]
The cited source says what the claim attributes to it.
In the citing paper
“Position Interpolation [4] and YaRN [5] modify the RoPE frequencies to keep evaluation positions within the trained distribution”
From Curvature-Corrected Rotary Position Embeddings: An Entropy-Invariant Temperature for Mitigating Numerical-Rank Collapse in Long-Context Attention (2026) · cites this paper as [4]
The cited source says what the claim attributes to it.
In the citing paper
“Position interpolation techniques enable models to process sequences beyond original context window limitations by intelligently rescaling position indices rather than extrapolating to unseen positions [153].”
From A Survey of Context Engineering for Large Language Models (2025) · cites this paper as [153]
The cited source says what the claim attributes to it.
In the citing paper
“Despite this growing importance, LLMs consistently face challenges in processing long contexts, primarily due to the substantial computational resources required for long sequence training (Dao et al., 2022; Dao, 2023) as well as the apparent inability to generalize to sequences longer than those encountered during training (Chen et al., 2023; Peng et al., 2023b).”
From ∞Bench: Extending Long Context Evaluation Beyond 100K Tokens (2024) · cites this paper as Chen et al., 2023
The cited source says what the claim attributes to it.
In the citing paper
“Most models uses RoPE (Su et al., 2021), PI (Chen et al., 2023), LongRoPE (Ding et al., 2024), or YaRN (Peng et al., 2024).”
From HELMET: How to Evaluate Long-Context Language Models Effectively and Thoroughly (2024) · cites this paper as Chen et al., 2023
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Recent large language models (LLMs) support long contexts ranging from 128K to 1M tokens.”
From NoLiMa: Long-Context Evaluation Beyond Literal Matching (2025) · cites this paper as Chen et al., 2023
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 19
- Krzysztof Marcin Choromanski, Valerii Likhosherstov, David Dohan, Xingyou Song, Andreea Gane, Tamás Sarlós, Peter Hawkins, Jared Quincy Davis, Afroz Mohiuddin, Lukasz Kaiser, David Benjamin Belanger, Lucy J. Colwell, and Adrian Weller. Rethinking attention with per- formers. In 9th International Conference on Learning Representations, ICLR 2021. OpenRe- view.net, May 2021.
- Tri Dao, Daniel Y. Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. FlashAttention: Fast and memory-efficient exact attention with IO-awareness. In Advances in Neural Information Process- ing Systems, 2022.
- Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszko- reit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recogni- tion at scale. In International Conference on Learning Representations, 2021. URL https: //openreview.net/forum?id=YicbFdNTTy.
- Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, Shawn Presser, and Connor Leahy. The Pile: An 800gb dataset of diverse text for language modeling. arXiv preprint arXiv:2101.00027, 2020. · arXiv
- Adi Haviv, Ori Ram, Ofir Press, Peter Izsak, and Omer Levy. Transformer language models without positional encodings still learn positional information. 2022.
- Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021. · arXiv
- Omar Khattab, Christopher Potts, and Matei Zaharia. Relevance-guided supervision for openqa with colbert. Transactions of the Association for Computational Linguistics, 9:929-944, 2021. doi: 10.1162/tacl a 00405. · DOI
- Nikita Kitaev, Lukasz Kaiser, and Anselm Levskaya. Reformer: The efficient transformer. In 8th International Conference on Learning Representations, ICLR 2020. OpenReview.net, April 2020.
- Chin-Yew Lin. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pp. 74-81, Barcelona, Spain, July 2004. Association for Computational Linguis- tics. URL https://aclanthology.org/W04-1013.
- Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Confer- ence on Learning Representations, 2019. URL https://openreview.net/forum?id= Bkg6RiCqY7.
- Amirkeivan Mohtashami and Martin Jaggi. Landmark attention: Random-access infinite context length for transformers. arXiv preprint arXiv:2305.16300, 2023. · arXiv
- Ofir Press, Noah Smith, and Mike Lewis. Train short, test long: Attention with linear biases enables input length extrapolation. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=R8sQPpGCv0.
- Jack W. Rae, Anna Potapenko, Siddhant M. Jayakumar, Chloe Hillier, and Timothy P. Lilli- crap. Compressive transformers for long-range sequence modelling. In International Confer- ence on Learning Representations, 2020. URL https://openreview.net/forum?id= SylKikSYDH.
- Hongyu Ren, Hanjun Dai, Zihang Dai, Mengjiao Yang, Jure Leskovec, Dale Schuurmans, and Bo Dai. Combiner: Full attention transformer with sparse computation cost. In Marc'Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan (eds.), Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, pp. 22470-22482.
- Uri Shaham, Elad Segal, Maor Ivgi, Avia Efrat, Ori Yoran, Adi Haviv, Ankit Gupta, Wenhan Xiong, Mor Geva, Jonathan Berant, and Omer Levy. SCROLLS: Standardized CompaRison over long language sequences. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 12007-12021, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguistics. URL https://aclanthology.org/2022. emnlp-main.823.
- Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Ad- vances in Neural Information Processing Systems, volume 30.
- Curran Associates, Inc., 2017. URL https://proceedings.neurips.cc/paper_files/paper/2017/ file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf.
- Yuhuai Wu, Markus Norman Rabe, DeLesley Hutchins, and Christian Szegedy. Memorizing trans- formers. In The Tenth International Conference on Learning Representations, ICLR 2022. Open- Review.net, April 2022.
- Manzil Zaheer, Guru Guruganesh, Kumar Avinava Dubey, Joshua Ainslie, Chris Alberti, Santi- ago Ontañón, Philip Pham, Anirudh Ravula, Qifan Wang, Li Yang, and Amr Ahmed. Big bird: Transformers for longer sequences. In Hugo Larochelle, Marc'Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (eds.), Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020.
Cited by
Sorted by citations
Items 201–271 of 271
Re3Syn: A Dependency-Based Data Synthesis Framework for Long-Context Post-training
Zhang, Z., Liu, Z., Wang, H., Shan, R., Kuang, L., Wang, L., & Soh, D. W. (2025) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2025.acl-long.1518
Wang, Z., Wei, X., Du, K., Zhang, Y., Qin, L., Xia, Y., & Kuang, L. (2025) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2025.acl-long.1286
Boosting Long-Context Information Seeking via Query-Guided Activation Refilling
Qian, H., Liu, Z., Zhang, P., Dou, Z., & Lian, D. (2025) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2025.acl-long.465 · Cited by 1
IN-CONTEXT LEARNING AS GENERAL-PURPOSE LEARNING: A COMPREHENSIVE SURVEY AND NEW PERSPECTIVESPreprint
Wang, F., Bo, Y., Shao, P., Chen, Z., Chou, H.-C., Kang, L., Shao, S., Xia, X., Ding, N., Xiong, H., et al. (2025) · doi:10.36227/techrxiv.175492111.15449662/v1
The Integration of CFD and Cutting-edge Multivariate Modeling in Future Forest Fire Research
Xie, D. (2025) · Theoretical and Natural Science · doi:10.54254/2753-8818/2025.dl26772
In-Context Learning as General-Purpose Learning: a Comprehensive Survey and New PerspectivesPreprint
Wang, F., Bo, Y., Shao, P., Chen, Z., Chou, H.-C., Kang, L., Shao, S., Xia, X., Ding, N., Xiong, H., et al. (2025) · doi:10.36227/techrxiv.175492111.15449662/v2
Large language models – the future of fundamental physics?
Heneka, C. S., Nieser, F., Ore, A., Plehn, T., & Schiller, D. (2026) · SciPost Physics · doi:10.21468/scipostphys.20.3.070 · Cited by 13
OkraLong: A Flexible Retrieval-Augmented Framework for Long-Text Question Answering
Hui, Y.-L., Liu, Y.-H., Lu, Y., & Zhang, H.-C. (2025) · Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2025.findings-emnlp.890
Hu, J., Guo, D., Liu, Y., Ai, Q., Wang, L., Sun, X., Zhang, Q., Zhou, Q., & Luo, C. (2025) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2025.findings-emnlp.1149 · Cited by 1
You Only Use Reactive Attention Slice When Retrieving From Long Context
Soh, Y. J., Huang, H., Tian, Y., & Zhao, J. (2025) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2025.findings-emnlp.1125
Self-citationHAWK: Highlighting Entity-aware Knowledge for Alleviating Information Sparsity in Long Contexts
Koo, S., Kim, J., Park, C., & Lim, H. (2025) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2025.findings-emnlp.708
SWAN: An Efficient and Scalable Approach for Long-Context Language Modeling
Puvvada, K. C., Ladhak, F., Serrano, S. A., Hsieh, C.-P., Acharya, S., Majumdar, S., Jia, F., Kriman, S., Sun, S., Rekesh, D., et al. (2025) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2025.emnlp-main.123 · Cited by 3
What is the Best Sequence Length for BabyLM?
Salhan, S., Martinez, R. D., Goriely, Z., & Buttery, P. J. (2025) · BabyLM Workshop · doi:10.18653/v1/2025.babylm-main.10
In-Context Learning as General-Purpose Learning: a Comprehensive Survey and New PerspectivesPreprint
Wang, F., Bo, Y., Shao, P., Chen, Z., Chou, H.-C., Kang, L., Shao, S., Xia, X., Ding, N., Xiong, H., et al. (2025) · doi:10.36227/techrxiv.175492111.15449662/v3
Bayesian Optimization in High‐Dimensional Mixed Spaces: A Complex‐Valued Field Perspective
Xue, C., Wang, Z., Li, J., & Li, C. (2025) · Electronics Letters · doi:10.1049/ell2.70487
Sule, M. (2025) · International Journal of Science and Research (IJSR) · doi:10.21275/sr251215184109
Zhao, W. X., Zhou, K., Li, J., Tang, T., & Wen, J.-R. (2025) · Artificial intelligence: foundations, theory, and algorithms/Artificial intelligence: Foundations, theory, and algorithms · doi:10.1007/978-981-96-6259-3_5
Dev, A., Asakura, T., & Sætre, R. (2024) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/3pq3nw4vzona
Report on scipost_202507_00080v1
(2025) · SciPost Physics · doi:10.21468/scipost.report.12046
Report on scipost_202507_00080v1
(2025) · SciPost Physics · doi:10.21468/scipost.report.12045
Report on scipost_202507_00080v1
(2025) · SciPost Physics · doi:10.21468/scipost.report.12088
MemRoPE: Training-Free Infinite Video Generation via Evolving Memory Tokens
Kim, Y., Hu, Q., Kuo, C. J., & Beerel, P. A. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37356-4_14 · Cited by 25
Boosting Resolution Generalization of Diffusion Transformers with Randomized Positional Encodings
Hou, L., Liu, C., Zheng, M., Tao, X., Wan, P., ZHANG, D., & Gai, K. (2026) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v40i6.42478
RetroLM: Retrieval-Augmented KVs for Long-Context Processing
Luo, K., Liu, Z., Xiao, S., Chen, J., Qian, H., Zhang, P., Jiang, S., Dong, B., Zhao, J., & Liu, K. (2026) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v40i38.40511
Can Calibration of Positional Encodings Enhance Long Context Utilization?
Zehle, T., & Aßenmacher, M. (2026) · Findings of the Association for Computational Linguistics: EACL · doi:10.18653/v1/2026.findings-eacl.120
Extending Audio Context for Long-Form Understanding in Large Audio-Language Models
Chaichana, Y., Taveekitworachai, P., Sirichotedumrong, W., Manakul, P., & Pipatanakul, K. (2026) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2026.eacl-long.286 · Cited by 2
Retrieval‐augmented Generation for Materials Large Language Models
Jiang, X., & Su, Y. (2026) · Wiley eBooks · doi:10.1002/9783527855841.ch12
Fine‐tuning and Application for Materials Large Language Models
Jiang, X., & Su, Y. (2026) · Wiley eBooks · doi:10.1002/9783527855841.ch13
ScrollScape: Unlocking 32K Image Generation With Video Diffusion Priors
Yu, H., Zhang, Y., Di, D., Zhang, R., & Zuo, W. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37553-7_15
Wang, X., Zhang, T., Cui, Z., Guo, X., Wang, F., Wang, Y., Cai, X., & Zheng, W. (2026) · Nature Machine Intelligence · doi:10.1038/s42256-026-01210-2
PI-EnLLM: Personalized Interactive Healthcare Assistance via Ensembling Large Language Models
Dwibedi, A. K., Bhattacharjee, S. D., Chang, Y.-P., & Yuan, J. (2026) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-032-16099-7_20
Wang, Z., Chen, W., Liu, R., Yang, K., Zhou, W., Lin, C., Liu, Q., Sim, M. Y., Zhang, G., Chen, X., et al. (2026) · Synthesis lectures on human language technologies · doi:10.1007/978-3-032-06264-2_13
Low Rank Adaptation Enables Efficient Domain Transfer in Billion Parameter Language Models
Yang, J. (2026) · Jisuanji shenghuojia. · doi:10.54097/6w0gxa44
LongRanker: Efficient One-Pass Document Reranking with Long-Context Large Language Models
Zhou, C., Zhang, R., Guo, J., de Rijke, M., Yixing, F., & Cheng, X. (2026) · International World Wide Web Conference (WWW) · doi:10.1145/3774904.3792179
A survey of large language models: techniques, applications, and challenges
Ji, J., Zhao, X., & Shi, X. (2026) · Multimedia Systems · doi:10.1007/s00530-026-02288-9
Majumder, S., & Sarkar, S. C. (2026) · Research Square · doi:10.21203/rs.3.rs-9986413/v1
Zhang, S., Gao, G., 甘紫云, Yuan, C., Lin, Z., Peng, H., Li, B., & Hu, W. (2026) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2026.findings-acl.1608
Resonating with RoPE: Spectral Quantization for High-Fidelity Key Cache Compression
Wang, X., Tang, H., Liang, T., Wang, Z., Hu, Y., & Guan, W. (2026) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2026.acl-long.1732
Failure Modes in Multi-Hop QA: The Weakest Link Effect and the Recognition Bottleneck
Zhang, M., Meng, Z., & Collier, N. (2026) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2026.acl-long.1937
Wang, Y., Wang, Y., Zhang, B., Zhang, Y., Guo, D., Chen, J., Zhang, H., & Zheng, Z. (2026) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2026.acl-long.1189
Liu, S., Wang, G., Wang, S., Zeng, J., Xie, H., Lou, S., Yang, J., Yu, D., Wang, H., & Yang, C. (2026) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2026.acl-long.1199
Mitigating Tokenization-Induced Distance Distortion in Long-Context Multilingual Machine Translation
Selialia, K., Nzeyimana, A., & Anwar, F. M. (2026) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2026.acl-long.1696
TiMem: Temporal-Hierarchical Memory Consolidation for Long-Horizon Conversational Agents
Li, K., Yu, X., Ni, Z., Zeng, Y., Xu, Y., Zhang, Z., Li, X., Sang, J., Duan, X., Wang, X., et al. (2026) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2026.findings-acl.1091 · Cited by 5
Improving Long-Context Translation via Self-Supervised Dual Learning
Cheng, S., She, S., Bao, Y., Zhang, J., Chen, J., & Huang, S. (2026) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2026.acl-long.1257
SCOPE: Boosting LLM Efficiency with Scoped Position Encoding
Qi, Q., Chen, H., & Li, Z. (2026) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2026.acl-long.1650
PPA-Plan: Proactive Pitfall Avoidance for Reliable Planning in Long-Context LLM Reasoning
Kim, B., Kim, G., & Park, S. Y. (2026) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2026.acl-long.2032
Yin, X., Ding, Z., Zhang, Y., Wang, Q., Wang, R., Ni, C., & Cui, Z. (2026) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2026.findings-acl.217 · Cited by 2
Twos All the Way Down: A Hidden Condition Every Deployed RoPE Schedule SatisfiesPreprint
Ryu, S. (2026) · Research Square · doi:10.21203/rs.3.rs-10251537/v1
Video Generation Models Are General-Purpose Vision Learners
Wang, L., Zhang, C., Kabra, R., Uijlings, J., Waslander, S. L., Zisserman, A., Carreira, J., He, K., Andriluka, M., Băzăvan, E. G., et al. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37369-4_33 · Cited by 9
Learning to Route Queries to Heads for Attention-based Re-ranking with Large Language Models
Tian, Y., Mo, F., Huang, Z., Zhang, W., & Nie, J. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3809945
Self-citationEMS-GL: Adaptive Evict-then-Merge Strategy for KV Cache Compression Based on Global-Local Importance
Li, Y., Li, Y., Meng, Y., Ma, X., Geng, Z., Xia, S., & Wang, Z. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-2759-4_32
RepoFuse: A Dual-Context Approach to Repository-Level Code Completion at Industrial Scale
Liang, M., Xie, X., Zhang, G., Zheng, X., Jiang, W., Wang, C., Fan, G., & Di, P. (2026) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3803437.3805230
HMT: Hierarchical Mamba-Transformer for Efficient Long-Sequence Text Classification
Xie, L., Wu, Y., Cui, T., He, L., & Du, Z. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3426-4_10
Wu, J. (2026) · Applied and Computational Engineering · doi:10.54254/2755-2721/2026.35523
Optimizing information retrieval tasks with large language model for data enhancement
Vaidya, C. D., Amudhavel, J., Mishra, P. K., Palekar, V. R., Tikhe, G. N., & Bhure, K. (2026) · Journal of Nonlinear Complex and Data Science · doi:10.1515/jncds-2025-0124
Task-Adaptive Calibration for Multimodal Long-Context ExtensionPreprint
Li, X., Guo, Z., & Liu, R. (2026) · Research Square · doi:10.21203/rs.3.rs-10769418/v1
Entity-Driven Knowledge Compression for Question Answering on Long Contexts
Wang, Q., Cheng, Y., & Li, W. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37667-1_3
WARP: Wide Attention with Rich Projections for Image Super-Resolution
Kim, J., & Lee, K. M. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37261-1_4
On the Effectiveness of Wavelets for Positional Encoding
Oka, Y., Saito, I., & Nishida, K. (2026) · Journal of Natural Language Processing · doi:10.5715/jnlp.33.1333
Wang, C., Zhang, Z., Xu, X., Zheng, M., Wei, G., & Zhang, X. (2026) · Foundations and Trends® in Programming Languages · doi:10.1108/ftpgl-11-2025-0068
Development of Trusted AI for Effective Digital Government Document Summarization
Sampath, D., Nivisha, S., Kharnika, G. R., & Prasanna, M. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-9129-8.ch012
Shi, J., Liu, T., Wang, Y., Liu, P., Wang, X., & Ji, W. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3858419
TRONTOR: To RoPE or Not to RoPE for Long-Context Modeling
Son, Y., Park, J., & Lee, J. (2026) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/2026.findings-emnlp.836
Short-Context Gains, Long-Context Drift: Negative Results for Token-Conditioned RoPE
Kazakov, A. (2026) · Workshop on Insights from Negative Results in NLP (Insights) · doi:10.18653/2026.insights-1.13
PrimeTTT: Priming Test-Time Training for LLMs via Meta-Learned Initialization
Wang, X., Aishan, D., Zhou, Y., Huang, Y., & Zhang, X. (2026) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/2026.emnlp-main.271
ECO-Mem: An Event-Centric Organization Framework for Visibility-Controlled Long-Term Memory
Pei, X., Song, D., Wu, Z., Tian, Y., Yang, J., Ma, Y., Li, Y., Li, X., Ma, H., & Xu, Y. (2026) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/2026.findings-emnlp.1817
Self-citationSurrogateKV: Representation-Preserving KV Cache Compression for Long-Context LLMs
Kim, J., Ryu, J., Talibli, F., So, J., & Kim, Y. (2026) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/2026.findings-emnlp.1580
A Survey of Large Language Models
Zhao, W. X., Zhou, K., Li, J., Tang, T., Wang, X., Hou, Y., Min, Y., Zhang, B., Zhang, J., Dong, Z., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.18223 · Cited by 2,090
Metadata
- Authors
- Chen, S., Wong, S., Chen, L., & Tian, Y.
- Affiliations
- Meta (United States) (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2023
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2306.15595
- OpenAlex
- W4382603041
- Cited by
- 50
- References
- 0
- Citations by year
- 2026: 3 · 2025: 19 · 2024: 22 · 2023: 6
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- context window extension, rope, large language models, self-attention, long-context modeling, long document summarization, llamas
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
