The DOI matches its Crossref record; 29 of the 30 references it cites were found in a registry or an index.
Same Task, More Tokens: the Impact of Input Length on the Reasoning Performance of Large Language Models
Open accessAbstract
This paper explores the impact of extending input lengths on the capabilities of Large Language Models (LLMs).Despite LLMs advancements in recent times, their performance consistency across different input lengths is not well understood.We investigate this aspect by introducing a novel QA reasoning framework, specifically designed to assess the impact of input length.We isolate the effect of input length using multiple versions of the same sample, each being extended with padding of different lengths, types and locations.Our findings show a notable degradation in LLMs' reasoning performance at much shorter input lengths than their technical maximum.We show that the degradation trend appears in every version of our dataset, although at different intensities.Additionally, our study reveals that the traditional metric of next word prediction correlates negatively with performance of LLMs' on our reasoning dataset.We analyse our results and identify failure modes that can serve as useful guides for future research, potentially informing strategies to address the limitations observed in LLMs.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, the viability of this future remains largely uncertain given the many engineering hurdles involved and the apparent limitations of long-context LLMs, even within current context lengths [41, 45, 96, 21].”
From HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models (2024) · cites this paper as [41]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Longcontext studies have shown that language models struggle to utilize relevant information within lengthy inputs, with performance often degrading well below the nominal context-window limit (Liu et al., 2024; Levy et al., 2024; Fraga, 2024; He et al., 2024; Hsieh et al., 2024; Tian et al., 2025).”
From Weighted Memory Tree: Remembering What Matters for Long-Horizon LLM Agents (2026) · cites this paper as Levy et al., 2024
The cited source says what the claim attributes to it.
In the citing paper
“Models can be sensitive to the position of relevant evidence and can exhibit reasoning degradation as input length increases (Liu et al., 2024; Levy et al., 2024).”
From Weighted Memory Tree: Remembering What Matters for Long-Horizon LLM Agents (2026) · cites this paper as Levy et al., 2024
The cited source says what the claim attributes to it.
In the citing paper
“Long-context studies show that increasing the available context does not ensure reliable use of all included information.”
From Weighted Memory Tree: Remembering What Matters for Long-Horizon LLM Agents (2026) · cites this paper as Levy et al., 2024
The cited source says what the claim attributes to it.
In the citing paper
“Third, it possible that such very long context windows will continue to lead to degraded performance, as has been seen in current models [33].”
From Episodic Memory in AI Agents Poses Risks that Should be Studied and Mitigated (2025) · cites this paper as [33]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 6
- Hong Liu, Sang Michael Xie, Zhiyuan Li, and Tengyu Ma. 2023a. Same pre-training loss, better down- stream: Implicit bias matters for language models. In International Conference on Machine Learning, pages 22188-22214.
- PMLR.
- Yuning Mao, Pengcheng He, Xiaodong Liu, Yelong Shen, Jianfeng Gao, Jiawei Han, and Weizhu Chen. 2020. Generation-augmented retrieval for open- domain question answering. In Annual Meeting of the Association for Computational Linguistics.
- Freda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales, David Dohan, Ed H Chi, Nathanael Schärli, and Denny Zhou. 2023. Large language models can be easily distracted by irrelevant context. In Inter- national Conference on Machine Learning, pages 31210-31227.
- PMLR.
- Jason Weston, Antoine Bordes, Sumit Chopra, Alexan- der M Rush, Bart Van Merriënboer, Armand Joulin, and Tomas Mikolov. 2016. Towards ai-complete question answering: A set of prerequisite toy tasks. In 4th International Conference on Learning Repre- sentations, ICLR 2016.
Cited by
Sorted by citations
RULER: What's the Real Context Size of Your Long-Context Language Models?
Hsieh, C.-P., Sun, S., Kriman, S., Acharya, S., Rekesh, D., Jia, F., Zhang, Y., & Ginsburg, B. (2024) · arXiv · Cited by 1,068
Clinical entity augmented retrieval for clinical information extraction
López, I., Swaminathan, A., Vedula, K. S., Narayanan, S., Haredasht, F. N., Ma, S. P., Liang, A. S., Tate, S., Maddali, M. V., Gallo, R. J., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-024-01377-1 · Cited by 97
Du, X., Zhou, Z., Wang, Y., Chuang, Y.-W., Li, Y., Yang, R. C., Zhang, W., Wang, X., Chen, X., Guan, H., et al. (2025) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaf233 · Cited by 56
DocETL: Agentic Query Rewriting and Evaluation for Complex Document Processing
Shankar, S., Chambers, T., Shah, T., Parameswaran, A., & Wu, E. (2025) · Proceedings of the VLDB Endowment · doi:10.14778/3746405.3746426 · Cited by 147
Accelerating knowledge graph and ontology engineering with large language models
Shimizu, C., & Hitzler, P. (2025) · Journal of Web Semantics · doi:10.1016/j.websem.2025.100862 · Cited by 50
Retrieval Augmented Generation or Long-Context LLMs? A Comprehensive Study and Hybrid Approach
Li, Z., Li, C., Zhang, M., Mei, Q., & Bendersky, M. (2024) · Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track · doi:10.18653/v1/2024.emnlp-industry.66 · Cited by 49
A strategy for cost-effective large language model use at health system-scale
Klang, E., Apakama, D. U., Abbott, E. E., Vaid, A., Lampert, J. M., Sakhuja, A., Freeman, R., Charney, A. W., Reich, D. L., Kraft, M., et al. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01315-1 · Cited by 56
Complex Claim Verification with Evidence Retrieved in the Wild
Chen, J., Kim, G., Sriram, A., Durrett, G., & Choi, E. (2024) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2024.naacl-long.196 · Cited by 85
Schwieger, A., Angst, K., de Bardeci, M., Burrer, A., Cathomas, F., Ferrea, S., Grätz, F., Knorr, M. S., Kronenberg, G., Spiller, T. R., et al. (2024) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2024.105654 · Cited by 49
The Human‐ GenAI Value Loop in Human‐Centered Innovation: Beyond the Magical Narrative
Grange, C., Demazure, T., Ringeval, M., Bourdeau, S., & Martineau, C. (2025) · Information Systems Journal · doi:10.1111/isj.12602 · Cited by 53
Parametric Retrieval Augmented Generation
Su, W., Tang, Y., Ai, Q., Yan, J., Wang, C., Wang, H., Ye, Z., Zhou, Y., & Liu, Y. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3729957 · Cited by 74
Zhang, X., Qin, H., Wang, F., Dong, Y., & Li, J. (2025) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra55743.2025.11127951 · Cited by 27
One Thousand and One Pairs: A “novel” challenge for long-context language models
Karpinska, M., Thai, K., Lo, K., Goyal, T., & Iyyer, M. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.948 · Cited by 71
Lyo, S. K., Mohan, S., Hassankhani, A., Noor, A., Dako, F., & Cook, T. S. (2024) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-024-01233-4 · Cited by 25
Large Language Model Powered Symbolic Execution
Li, Y., Meng, R., & Duck, G. J. (2025) · Proceedings of the ACM on Programming Languages · doi:10.1145/3763163 · Cited by 38
HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models
Gutiérrez, B. J., Shu, Y., Gu, Y., Yasunaga, M., & Su, Y. (2024) · arXiv · doi:10.48550/arxiv.2405.14831 · Cited by 152
Policy-as-Prompt: Rethinking Content Moderation in the Age of Large Language Models
Palla, K., Redondo-García, J. L., Hauff, C., Fabbri, F., Damianou, A., Lindström, H., Taber, D., & Lalmas, M. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732054 · Cited by 41
Simplifying software compliance: AI technologies in drafting technical documentation for the AI Act
Sovrano, F., Hine, E., Anzolut, S., & Bacchelli, A. (2025) · Empirical Software Engineering · doi:10.1007/s10664-025-10645-x · Cited by 26
Language Repository for Long Video Understanding
Kahatapitiya, K., Ranasinghe, K., Park, J., & Ryoo, M. S. (2025) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2025.findings-acl.294 · Cited by 48
A Hitchhiker's Guide to Good Prompting Practices for Large Language Models in Radiology
Tripathi, S., Alkhulaifat, D., Lyo, S. K., Sukumaran, R., Li, B., Acharya, V., McBeth, R., & Cook, T. S. (2025) · Journal of the American College of Radiology · doi:10.1016/j.jacr.2025.02.051 · Cited by 18
Li, J., Li, X., Steinberg, J., Choube, A., Yao, B., Xu, X., Wang, D., Mynatt, E. D., & Mishra, V. (2025) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3749508 · Cited by 28
Roeschl, T., Hoffmann, M., Hashemi, D., Rarreck, F., Hinrichs, N., Trippel, T. D., Gröschel, M. I., Unbehaun, A., Klein, C., Kempfert, J., et al. (2025) · JMIRx Med · doi:10.2196/74899 · Cited by 17
Context Is What You Need: The Maximum Effective Context Window for Real World Limits of LLMs
Paulsen, N. (2026) · Advances in Artificial Intelligence and Machine Learning · doi:10.54364/aaiml.2026.61268 · Cited by 28
Harnessing the Power of Large Language Models for Automated Code Generation and Verification
Antero, U., Blanco, F. O., Oñativia, J., Sallé, D., & Sierra, B. (2024) · Robotics · doi:10.3390/robotics13090137 · Cited by 14
Large Language Models for In-File Vulnerability Localization Can Be “Lost in the End”
Sovrano, F., Bauer, A., & Bacchelli, A. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3715758 · Cited by 12
Wang, H., Shi, H., Tan, S., Qin, W., Wang, W., Zhang, T., Nambi, A. U., Ganu, T., & Wang, H. (2025) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2025.naacl-long.166 · Cited by 12
Statically Contextualizing Large Language Models with Typed Holes
Blinn, A., Li, X., Kim, J. H., & Omar, C. (2024) · Proceedings of the ACM on Programming Languages · doi:10.1145/3689728 · Cited by 10
Retrieval augmented generation for building datasets from scientific literature
Maharana, P. R., Verma, A., & Joshi, K. (2025) · Journal of Physics Materials · doi:10.1088/2515-7639/ade1fa · Cited by 12
Chain-of-Thought Matters: Improving Long-Context Language Models with Reasoning Path Supervision
Zhu, D., Wei, X., Zhao, G., Wu, W., Zou, H., Ran, J., XWang, Sun, L., Zhang, X., & Li, S. (2025) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2025.findings-emnlp.170 · Cited by 23
Is It Really Long Context if All You Need Is Retrieval? Towards Genuinely Difficult Long Context NLP
Goldman, O., Jacovi, A., Slobodkin, A., Maimon, A., Dagan, I., & Tsarfaty, R. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.924 · Cited by 25
Dynamic and Parametric Retrieval-Augmented Generation
Su, W., Ai, Q., Zhan, J., Dong, Q., & Liu, Y. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3731692 · Cited by 11
PromptPilot: Exploring User Experience of Prompting with AI-Enhanced Initiative in LLMs
Kim, S., Eun, J., Park, Y. E., Lee, K., Lee, G., & Lee, J. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2489030 · Cited by 11
Do We Need a Detailed Rubric for Automated Essay Scoring Using Large Language Models?
Yoshida, L. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-98465-5_8 · Cited by 9
Lee, H., Kim, M., Baek, S., Zhou, W., Debbah, M. A., & Lee, I. (2025) · IEEE Communications Magazine · doi:10.1109/mcom.001.2400577 · Cited by 8
Wardle, G., & Sušnjak, T. (2025) · Big Data and Cognitive Computing · doi:10.3390/bdcc9060149 · Cited by 10
On the Robustness of Agentic Function Calling
Rabinovich, E., & Tavor, A. A. (2025) · Workshop on Trustworthy NLP (TrustNLP) · doi:10.18653/v1/2025.trustnlp-main.20 · Cited by 8
Conversational Process Model Redesign
Klievtsova, N., Kampik, T., Mangler, J., & Rinderle‐Ma, S. (2026) · International Journal of Cooperative Information Systems · doi:10.1142/s0218843026500048 · Cited by 8
General scales unlock AI evaluation with explanatory and predictive power
Zhou, L., Pacchiardi, L., Martínez‐Plumed, F., Collins, K. M., Moros-Daval, Y., Zhang, S., Zhao, Q., Huang, Y., Sun, L., Prunty, J. E., et al. (2026) · Nature · doi:10.1038/s41586-026-10303-2 · Cited by 21
Chen, R., Wang, C., Sun, Y., Zhao, X., & Xu, S. (2025) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2025.acl-long.1438 · Cited by 14
Yuan, J., Liu, H., Zhong, S., Chuang, Y.-N., Li, S., Wang, G., Le, D., Jin, H., Chaudhary, V., Xu, Z., et al. (2024) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2024.findings-emnlp.266 · Cited by 14
Efficient Usage of RAG Systems in the World of LLMs
Rathod, P. J., & Anurag (2024) · International Journal For Multidisciplinary Research · doi:10.36948/ijfmr.2024.v06i04.36364 · Cited by 7
Zeng, Y., Brown, C., Byari, M., Raymond, J., Schmitt, T. M., & Rounsevell, M. (2025) · Geoscientific model development · doi:10.5194/gmd-18-4983-2025 · Cited by 7
Measuring the Influence of Incorrect Code on Test Generation
Huang, D., Zhang, J. M., Harman, M., Du, M., & Cui, H. (2026) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3744916.3764532 · Cited by 21
LongFaith: Enhancing Long-Context Reasoning in LLMs with Faithful Synthetic Data
Yang, C., Lin, X., Xu, C., Jiang, X., Ma, S., Liu, A. Q., Xiong, H., & Guo, J. G. (2025) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2025.findings-acl.169 · Cited by 19
Automatic Evaluation of Programming Tasks Supported by Language Models
Karsa, Z. I., & Goldschmidt, B. (2025) · IEEE Access · doi:10.1109/access.2025.3601448 · Cited by 6
Incorporating Causality Into Deep Learning Architectures to Improve Flash Drought Forecasts
Tang, S., Wang, S., Jiang, J., & Zheng, Y. (2025) · Water Resources Research · doi:10.1029/2024wr039470 · Cited by 7
Using GPT-4 to Generate Failure Logic
Clegg, K. D., Habli, I., & McDermid, J. A. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-68738-9_11 · Cited by 6
Han, B., Yun, W., Seo, B.-S., & Kim, J. (2025) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra55743.2025.11128120 · Cited by 10
Episodic Memory in AI Agents Poses Risks that Should be Studied and Mitigated
DeChant, C. (2025) · 2025 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) · doi:10.1109/satml64287.2025.00024 · Cited by 5
Leveraging Large Language Models for Use Case Model Generation from Software Requirements
Eisenreich, T., Friedlaender, N., & Wagner, S. (2025) · IEEE/ACM International Conference on Automated Software Engineering Workshops (ASEW) · doi:10.1109/asew67777.2025.00050 · Cited by 5
GeoVisA11y: An AI-based Geovisualization Question-Answering System for Screen-Reader Users
Li, C., Pang, R. Y., Chheda-Kothary, A., Sharif, A., Assalif, H., Heer, J., & Froehlich, J. E. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790334 · Cited by 5
Large Language Models-Aided Program Debloating
Lin, B., Wang, S., Qin, Y., Chen, L., & Mao, X. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3594673 · Cited by 5
Context-Aware Membership Inference Attacks against Pre-trained Large Language Models
Chang, H., Shamsabadi, A. S., Katevas, K., Haddadi, H., & Shokri, R. (2025) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2025.emnlp-main.370 · Cited by 19
Markov Chain of Thought for Efficient Mathematical Reasoning
Yang, W., Liao, M., & Fan, K. B. (2025) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2025.naacl-long.365 · Cited by 4
LETS-C: Leveraging Text Embedding for Time Series Classification
Kaur, R., Zeng, Z., Balch, T. R., & Veloso, M. (2025) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2025.acl-long.1557 · Cited by 7
Wu, X., Wang, M., Liu, Y., Shi, X., Yan, H., Xiangju, L., Zhu, J., & Zhang, W. (2025) · Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2025.acl-long.803 · Cited by 4
Ge, Y., Guo, Y., Das, S., & Sarker, A. (2026) · npj Artificial Intelligence · doi:10.1038/s44387-025-00062-2 · Cited by 6
LLM Augmentations to support Analytical Reasoning over Multiple Documents
Yousuf, R. B., Defelice, N., Sharma, M., Xu, S., & Ramakrishnan, N. (2024) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata62323.2024.10826114 · Cited by 6
Zhou, Y., Kang, J., & Nguyen, H. (2025) · Proceedings. · doi:10.22318/icls2025.704957 · Cited by 3
ChatTCAD: Leveraging Large Language Models for Automated TCAD Simulation File Generation
Zhang, Z., Nassif, S., & Tahoori, M. B. (2025) · IEEE Computer Society Annual Symposium on VLSI · doi:10.1109/isvlsi65124.2025.11130247 · Cited by 3
Han, T., Hyun, J., Jeong, W. K., Shin, J., Roh, Y. H., & Kim, M. J. (2026) · European Journal of Radiology · doi:10.1016/j.ejrad.2026.112677 · Cited by 3
EditCoT: A Stepwise Chain-of-Thought Reasoning Framework for Multi-Intent Text Revision
Li, X., Wei, C., & Chen, W. (2026) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3797483 · Cited by 3
Evaluating Long-Horizon Memory for Multi-Party Collaborative Dialogues
Hu, C., Li, T., Gao, X., Chen, H., Bai, Y., Xu, D., Lin, T., Li, X., Han, Y., Pei, J., et al. (2026) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3770855.3817589 · Cited by 37
LLM-Based Detection of Tangled Code Changes for Higher-Quality Method-Level Bug Datasets
Opu, M. N. I., Wang, S., & Chowdhury, S. (2026) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/3793302.3793351 · Cited by 8
Bauer, M., Dada, A., Seibold, C., & Kleesiek, J. (2024) · Clinical Natural Language Processing Workshop (ClinicalNLP) · doi:10.18653/v1/2024.clinicalnlp-1.44 · Cited by 3
L-CiteEval: A Suite for Evaluating Fidelity of Long-context Models
Tang, Z., Zhou, K., Li, J., Ji, B., Hou, J., & Zhang, M. (2025) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2025.acl-long.263 · Cited by 11
Robust Pronoun Fidelity with English LLMs: Are they Reasoning, Repeating, or Just Biased?
Gautam, V., Bingert, E., Zhu, D., Lauscher, A., & Klakow, D. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00719 · Cited by 6
Object Positions Interpretation System for Service Robots Through Targeted Object Marking
Yamao, K., Kanaoka, D., Isomoto, K., & Tamukoh, H. (2025) · IEEE/SICE International Symposium on System Integration (SII) · doi:10.1109/sii59315.2025.10870994 · Cited by 2
Poglitsch, C., Szakács, F., & Pirker, J. (2025) · International Conference on the Foundations of Digital Games (FDG) · doi:10.1145/3723498.3723702 · Cited by 7
Fan, K. S., Gan, J., Zou, I. X., Kaladjiska, M., Inguanez, M. B., & Garden, G. L. (2025) · Cureus · doi:10.7759/cureus.93960 · Cited by 2
Yaacoub, J.-P. A., Noura, H., Salman, O., & Pujolle, G. (2025) · Annals of Telecommunications · doi:10.1007/s12243-025-01134-9 · Cited by 3
Object Recognition and Conversational AI in Real-World Contexts
Ortíz, A. R., Illana, Á., & Salas, M. (2025) · Street Art & Urban Creativity · doi:10.62161/sauc.v11.6010 · Cited by 2
Onishi, S., Shibata, D., Tsujikawa, M., Kubo, M., Nakagawa, A., Ishii, R., Furukawa, H., Tominaga, J., & Ota, H. (2025) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc58623.2025.11252841 · Cited by 2
Large Language Models in Radiologic Numerical Tasks: A Thorough Evaluation and Error Analysis
Nowroozi, A., Bondarenko, M., Serapio, A., Schnitzler, T., Brar, S. S., & Sohn, J. H. (2026) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-025-01824-9 · Cited by 2
Can the current generation of large language models (LLMs) produce an adequate book index?
Bartmess, E., & Combs, M. R. (2026) · The Indexer · doi:10.3828/index.2026.4 · Cited by 2
HAFix: history-augmented large language models for bug fixing
Shi, Y., Bangash, A. A., Fallahzadeh, E., Adams, B., & Hassan, A. E. (2026) · Empirical Software Engineering · doi:10.1007/s10664-026-10833-3 · Cited by 8
Liu, Y., John, J., Sarkar, S., Zakkar, A., Kinkopf, P., Teo, P. T., & Abazeed, M. E. (2026) · JCO Clinical Cancer Informatics · doi:10.1200/cci-25-00284 · Cited by 2
Wang, L., Sun, S., Wang, X., Gu, J., & Liu, K. (2026) · Target International Journal of Translation Studies · doi:10.1075/target.25081.wan · Cited by 2
PortGPT: Towards Automated Backporting Using Large Language Models
Li, Z. L., Yu, Z., Song, J., Xu, M., Luo, Y., & Mu, D. (2026) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp63933.2026.00034 · Cited by 2
RankFlow: A Multi-Role Collaborative Reranking Workflow Utilizing Large Language Models
Jin, C., Peng, H., Zhang, A., Chen, N., Zhao, J., Xie, X., Li, K., Feng, S., Zhong, K.-W., Ding, C., et al. (2025) · International World Wide Web Conference (WWW) · doi:10.1145/3701716.3717575 · Cited by 4
FlexEmu: Towards Flexible MCU Peripheral Emulation
Lei, C., Ling, Z., Xu, X., Li, S., Liu, G., Dong, K., & Luo, J. (2025) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3719027.3765086 · Cited by 2
Bayesian Optimization for Biochemical Discovery with LLMsPreprint
Akke, M., Yang, S., Ruža, J., Song, J., Pan, E., & Gómez‐Bombarelli, R. (2025) · ChemRxiv · doi:10.26434/chemrxiv-2025-w1wsh · Cited by 2
Microservices Performance Testing with Causality-enhanced Large Language Models
Mascia, C., Guerriero, A., Giamattei, L., Pietrantuono, R., & Russo, S. (2025) · IEEE/ACM International Conference on AI Foundation Models and Software Engineering (FORGE) · doi:10.1109/forge66646.2025.00022 · Cited by 1
Kahn, T. M., & Brandt, A. (2025) · Elsevier eBooks · doi:10.1016/b978-0-443-34073-4.00011-3 · Cited by 1
Two Shots are Enough: Reliable Constrained Generation with LLMs
Mondal, M., Dolamic, L., Cudré-Mauroux, P., & Audiffren, J. (2025) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata66926.2025.11402199 · Cited by 1
Jensen, T. V. R., Tahavori, M., Mirshekali, H., & Shaker, H. R. (2026) · International Informatics and Software Engineering Conference (IISEC) · doi:10.1109/iisec69317.2026.11418403 · Cited by 1
Vázquez, S. G., Bertolini, L., Ceresa, M., Consoli, S., & Acosta, M. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-21480-5_17 · Cited by 1
Taira, K., Itaya, T., Yada, S., Hiyama, K., & Hanada, A. (2025) · JMIR Nursing · doi:10.2196/67197 · Cited by 1
Piazza, T. D., Lazarus, C., Nempont, O., & Boussel, L. (2025) · SPIE Medical Imaging: Computer-Aided Diagnosis · doi:10.1117/12.3046901 · Cited by 1
Towards A “Novel” Benchmark: Evaluating Literary Fiction with Large Language Models
Wang, W., Gao, M., Hu, X., & Wan, X. (2025) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2025.findings-acl.1114 · Cited by 2
Elliott, T. J., Levitt, S. P., Nixon, K. J., & Bekker, M. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-96262-2_15 · Cited by 1
Dietrich, N. T., Bradbury, N., & Loh, C. (2025) · American Journal of Roentgenology · doi:10.2214/ajr.25.33947 · Cited by 1
Dynamic and Parametric Retrieval Augmented Generation
Su, W., Dong, Q., Ai, Q., & Liu, Y. (2025) · Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region (SIGIR-AP) · doi:10.1145/3767695.3769672 · Cited by 1
ESG Classification by Implicit Rule Learning via GPT-4
Hyojeong, Y., Chanyoung, K., Hahm, M., Kim, K., & Son, G. (2024) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/54fti295dzx6 · Cited by 1
From jargon to clarity: bridging understanding through graded simplification of legal data
Adibhatla, H. S., Mukherjee, A., & Shrivastava, M. (2026) · Artificial Intelligence and Law · doi:10.1007/s10506-026-09503-y · Cited by 1
Matching Patients With Cell Surface–Targeted Clinical Trials Using Large Language Models
Callahan, S. C., Chrostek, M. R., Rydzewski, N. R., Bootsma, M. L., Helzer, K. T., Bakhtiar, H., Morris, Z. S., Kosoff, D., Lang, J. M., Ma, V. T.-L., et al. (2026) · JCO Precision Oncology · doi:10.1200/po-25-00757 · Cited by 1
Efficient Input Data Strategies for LLMs (Large Language Models)-Based Bearing Fault Diagnosis
Jeong, P.-G., Hu, Y., Kong, J., Song, K., Yoo, B., & Kang, K.-T. (2026) · International Journal of Precision Engineering and Manufacturing-Green Technology · doi:10.1007/s40684-026-00880-y · Cited by 1
SERA: Self-referential assessment framework for bidirectional generative commonsense reasoning
Seo, J., Moon, H., Jang, Y., & Lim, H. (2026) · Knowledge-Based Systems · doi:10.1016/j.knosys.2026.116152 · Cited by 1
Shen, M.-T., & Joung, Y.-J. (2026) · International Workshop on Agentic Engineering · doi:10.1145/3786167.3788424 · Cited by 1
Metadata
- Authors
- Levy, M., Jacoby, A., & Goldberg, Y.
- Affiliations
- Bar-Ilan University (IL)
- Venue
- Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
- Published
- 2024
- Pages
- 15339-15353
- ISSN
- 0736-587X (ISSN-L)
- OpenAlex
- W4402683949
- Cited by
- 119
- References
- 0
- Citations by year
- 2026: 48 · 2025: 57 · 2024: 14
- FWCI
- 35.12
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- European Commission (HORIZON2020)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- large language models, reasoning performance, question answering, padding, next-word prediction, performance degradation, failure modes
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
