Characterizing LLM Abstention Behavior in Science QA with Context Perturbations
Open accessAbstract
The correct model response in the face of uncertainty is to abstain from answering a question so as not to mislead the user. In this work, we study the ability of LLMs to abstain from answering context-dependent science questions when provided insufficient or incorrect context. We probe model sensitivity in several settings: removing gold context, replacing gold context with irrelevant context, and providing additional context beyond what is given. In experiments on four QA datasets with six LLMs, we show that performance varies greatly across models, across the type of context provided, and also by question type; in particular, many LLMs seem unable to abstain from answering boolean questions using standard QA prompts. Our analysis also highlights the unexpected impact of abstention performance on QA task accuracy. Counter-intuitively, in some settings, replacing gold context with irrelevant context or adding irrelevant context to gold context can improve abstention performance in a way that results in improvements in task performance. Our results imply that changes are needed in QA dataset design and evaluation to more effectively assess the correctness and downstream impacts of model abstention.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
If the model has the necessary information in neither its parametric knowledge nor the user-provided context, the model should abstain from answering to avoid misinforming a user.
“If the model has the necessary information in neither its parametric knowledge nor the user-provided context, the model should abstain from answering to avoid misinforming a user [68].”
From Laboratory-Scale AI: Open-Weight Models are Competitive with ChatGPT Even in Low-Resource Settings (2024) · cites this paper as [68]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 19
- Sishuo Chen, Wenkai Yang, Xiaohan Bi, and Xu Sun. 2023. Fine-tuning deteriorates general textual out- of-distribution detection by distorting task-agnostic features. In Findings of the Association for Com- putational Linguistics: EACL 2023, pages 564-579, Dubrovnik, Croatia. Association for Computational Linguistics.
- John Giorgi, Luca Soldaini, Bo Wang, Gary Bader, Kyle Lo, Lucy Lu Wang, and Arman Cohan. 2022. Open domain multi-document summarization: A compre- hensive study of model brittleness under retrieval. In EMNLP Findings.
- Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yu- taka Matsuo, and Yusuke Iwasawa. 2024. Large language models are zero-shot reasoners. In Pro- ceedings of the 36th International Conference on Neural Information Processing Systems, NIPS '22, Red Hook, NY, USA.
- Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Ku- mar, et al. 2022. Holistic evaluation of language models. arXiv preprint arXiv:2211.09110. · arXiv
- Jinzhi Liao, Xiang Zhao, Jianming Zheng, Xinyi Li, Fei Cai, and Jiuyang Tang. 2022. Ptau: Prompt tuning for attributing unanswerable questions. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 1219-1229.
- Yi Liu, Lianzhe Huang, Shicheng Li, Sishuo Chen, Hao Zhou, Fandong Meng, Jie Zhou, and Xu Sun. 2023. Recall: A benchmark for llms robustness against external counterfactual knowledge. arXiv preprint arXiv:2311.08147. · arXiv
- Anastasios Nentidis, Georgios Katsimpras, Eirini Van- dorou, Anastasia Krithara, Luis Gasco, Martin Krallinger, and Georgios Paliouras. 2021. Overview of bioasq 2021: The ninth bioasq challenge on large- scale biomedical semantic indexing and question an- swering. In Experimental IR Meets Multilinguality, Multimodality, and Interaction: 12th International Conference of the CLEF Association, CLEF 2021, Virtual Event, September 21-24, 2021, Proceedings 12, pages 239-263. Springer.
- Harsha Nori, Yin Tat Lee, Sheng Zhang, Dean Carig- nan, Richard Edgar, Nicolo Fusi, Nicholas King, Jonathan Larson, Yuanzhi Li, Weishung Liu, et al. 2023. Can generalist foundation models outcom- pete special-purpose tuning? case study in medicine. arXiv preprint arXiv:2311.16452. · arXiv
- Jayr Pereira, Robson Fidalgo, Roberto Lotufo, and Ro- drigo Nogueira. 2023. Visconde: Multi-document qa with gpt-3 and neural reranking. In European Conference on Information Retrieval, pages 534-543. Springer.
- Jie Ren, Jiaming Luo, Yao Zhao, Kundan Krishna, Mo- hammad Saleh, Balaji Lakshminarayanan, and Pe- ter J Liu. 2022. Out-of-distribution detection and selective generation for conditional language mod- els. In The Eleventh International Conference on Learning Representations.
- Freda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales, David Dohan, Ed H. Chi, Nathanael Schärli, and Denny Zhou. 2023. Large language models can be easily distracted by irrelevant context. In Proceed- ings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 31210-31227.
- PMLR.
- Ross Taylor, Marcin Kardas, Guillem Cucurull, Thomas Scialom, Anthony Hartshorn, Elvis Saravia, Andrew Poulton, Viktor Kerkez, and Robert Stojnic. 2022. Galactica: A large language model for science. arXiv preprint arXiv:2211.09085. · arXiv
- Boxin Wang, Chejian Xu, Shuohang Wang, Shuohang Wang, Zhe Gan, Yu Cheng, Jianfeng Gao, Ahmed Awadallah, and Bo Li. 2021. Adversarial glue: A multi-task benchmark for robustness evaluation of language models. In Proceedings of the Neural In- formation Processing Systems Track on Datasets and Benchmarks, volume 1.
- Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2022. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171. · arXiv
- Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, An- drew M Dai, and Quoc V Le. 2021. Finetuned lan- guage models are zero-shot learners. arXiv preprint arXiv:2109.01652. · arXiv
- Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2024. Chain-of-thought prompt- ing elicits reasoning in large language models. In Proceedings of the 36th International Conference on Neural Information Processing Systems, NIPS '22, Red Hook, NY, USA.
- Denny Zhou, Nathanael Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Olivier Bousquet, Quoc Le, and Ed Chi. 2022. Least-to-most prompting enables complex reason- ing in large language models. arXiv preprint arXiv:2205.10625. · arXiv
- Wei Zou, Runpeng Geng, Binghui Wang, and Jinyuan Jia. 2024. Poisonedrag: Knowledge poisoning at- tacks to retrieval-augmented generation of large lan- guage models. arXiv preprint arXiv:2402.07867. · arXiv
Cited by
Sorted by citations
Know Your Limits: A Survey of Abstention in Large Language Models
Wen, B., Yao, J., Feng, S., Xu, C., Tsvetkov, Y., Howe, B., & Wang, L. L. (2025) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00754 · Cited by 70 · see citing works
Self-citationLaboratory-Scale AI: Open-Weight Models are Competitive with ChatGPT Even in Low-Resource Settings
Wolfe, R., Slaughter, I., Han, B., Wen, B., Yang, Y., Rosenblatt, L., Herman, B., Brown, E., Qu, Z., Weber, N., et al. (2024) · The 2024 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3630106.3658966 · Cited by 17 · see citing works
Self-citationPatientSafeBench: Evaluating the Safety of Medical LLMs for Patient Use
Kim, M., Park, H., Kim, W., Choi, S. H., Kim, H. E., Sohn, H., Park, J., Kim, S., Yu, S.-C., & Oh, Y. (2025) · IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI ...) · doi:10.1109/bhi67747.2025.11269553 · Cited by 2 · see citing works
Cho, A.-Y., Eo, S., Jeon, M., Kim, J., Ihm, J., & Seo, D. (2026) · JMIR Medical Education · doi:10.2196/91809 · Cited by 2 · see citing works
Causal evidence that language models use confidence to drive behaviour
Kumaran, D., Daw, N. D., Osindero, S., Veličković, P., & Pătrăucean, V. (2026) · Nature Machine Intelligence · doi:10.1038/s42256-026-01293-x · Cited by 1 · see citing works
Exploring the Abstention Ability of LLMs using Scientific Reasoning in Critical Fields
Summers, L., Yoon, H., Zheng, W., & Feng, Y. (2026) · ACM/IEEE Joint Conference on Digital Libraries · doi:10.1145/3805696.3847298
- Authors
- Wen, B., Howe, B., & Wang, L. L.
- Affiliations
- University of Washington (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2024
- Pages
- 3437-3450
- arXiv
- 2404.12452
- OpenAlex
- W4404782525
- Cited by
- 5
- References
- 0
- Citations by year
- 2026: 3 · 2025: 2
- FWCI
- 1.44
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Keywords
- large language models, uncertainty, model sensitivity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
