Navigating the Grey Area: How Expressions of Uncertainty and Overconfidence Affect Language Models
Open accessAbstract
The increased deployment of LMs for realworld tasks involving knowledge and facts makes it important to understand model epistemology: what LMs think they know, and how their attitudes toward that knowledge are affected by language use in their inputs.Here, we study an aspect of model epistemology: how epistemic markers of certainty, uncertainty, or evidentiality like "I'm sure it's", "I think it's", or "Wikipedia says it's" affect models, and whether they contribute to model failures.We develop a typology of epistemic markers and inject 50 markers into prompts for question answering.We find that LMs are highly sensitive to epistemic markers in prompts, with accuracies varying more than 80%.Surprisingly, we find that expressions of high certainty result in a 7% decrease in accuracy as compared to low certainty expressions; similarly, factive verbs hurt performance, while evidentials benefit performance.Our analysis of a popular pretraining dataset shows that these markers of uncertainty are associated with answers on question-answering websites, while markers of certainty are associated with questions.These associations may suggest that the behavior of LMs is based on mimicking observed language use, rather than truly reflecting epistemic uncertainty.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Mitigating indirect action and language by outputting confident advice can backfire if the recommendations are wrong.
“Mitigating indirect action and language by outputting confident advice can backfire if the recommendations are wrong, echoing concerns on overconfidence and miscalibration (Kim et al., 2024;Zhou et al., 2023).”
From Sycophantic AI decreases prosocial intentions and promotes dependence (2026) · cites this paper as Zhou et al., 2023
The research community has called for LLMs and LLM-infused applications to express the uncertainty of outputs to end users.
“As one approach to reduce overreliance, the research community has called for LLMs and LLM-infused applications to express the uncertainty of outputs to end users [6, 56, 57, 73, 101, 120].”
From "I'm Not Sure, But...": Examining the Impact of Large Language Models' Uncertainty Expression on User Reliance and Trust (2024) · cites this paper as [120]
Zhou et al. taught OpenAI's GPT3 model to express (un)certainty through prompt engineering, but found that the generated expressions were not well calibrated, especially those suggesting high certainty.
“Zhou et al. [120] 'taught' OpenAI's GPT3 [18] model to express (un)certainty through prompt engineering, but also found that the generated expressions were not well calibrated, especially those suggesting high certainty.”
From "I'm Not Sure, But...": Examining the Impact of Large Language Models' Uncertainty Expression on User Reliance and Trust (2024) · cites this paper as Zhou et al. [120]
Cites
Natural Questions: A benchmark for question answering research
Kwiatkowski, T., Palomaki, J., Redfield, O., Collins, M., Parikh, A., Alberti, C., Epstein, D., Polosukhin, I., Devlin, J., Lee, K., et al. (2019) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00276 · Cited by 2,069 · see citing works
TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension
Joshi, M., Choi, E., Weld, D., & Zettlemoyer, L. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1147 · Cited by 1,690 · see citing works
Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity
Lu, Y., Bartolo, M., Moore, A., Riedel, S., & Stenetorp, P. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.556 · Cited by 610 · cited 1× · see citing works
“Recent work has shown the drastic differences that appear based on simple changes to prompting setup (Suzgun et al., 2022; Lu et al., 2022).”
BackgroundHolistic Evaluation of Language Models
Bommasani, R., Liang, P., & Lee, T. (2023) · Annals of the New York Academy of Sciences · doi:10.1111/nyas.15007 · Cited by 480 · see citing works
The Role of Explanations on Trust and Reliance in Clinical Decision Support Systems
Bussone, A., Stumpf, S., & O'Sullivan, D. (2015) · 2015 International Conference on Healthcare Informatics · doi:10.1109/ichi.2015.26 · Cited by 441 · see citing works
Jacobs, M., Pradier, M. F., McCoy, T. H., Perlis, R. H., Doshi-Velez, F., & Gajos, K. Z. (2021) · Translational Psychiatry · doi:10.1038/s41398-021-01224-x · Cited by 247 · see citing works
Calibration of Pre-trained Transformers
Desai, S., & Durrett, G. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.21 · Cited by 209 · see citing works
Language Models (Mostly) Know What They Know
Kadavath, S., Conerly, T., Askell, A., Henighan, T., Drain, D., Perez, E., Schiefer, N., Hatfield-Dodds, Z., DasSarma, N., Tran-Johnson, E., et al. (2022) · arXiv · doi:10.48550/arxiv.2207.05221 · Cited by 187 · cited 1× · see citing works
“Prior work has focused on one aspect of this: linguistic calibration, particularly on learning the mapping between the internal probabilities of a model and an ordinal output (Kadavath et al., 2022; Lin et al., 2022; Mielke et al., 2022).”
BackgroundDemystifying Prompts in Language Models via Perplexity Estimation
Gonen, H., Iyer, S., Blevins, T., Smith, N., & Zettlemoyer, L. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.679 · Cited by 109 · see citing works
Reducing conversational agents' overconfidence through linguistic calibration
Mielke, S. J., Szlam, A., Dinan, E., & Boureau, Y.-L. (2020) · arXiv · doi:10.48550/arxiv.2012.14983 · Cited by 72 · see citing works
Reducing Conversational Agents’ Overconfidence Through Linguistic Calibration
Mielke, S. J., Szlam, A., Dinan, E., & Boureau, Y.-L. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00494 · Cited by 72 · see citing works
Kuhn, L., Gal, Y., & Farquhar, S. (2023) · arXiv · doi:10.48550/arxiv.2302.09664 · Cited by 52 · see citing works
“Was it “stated” or was it “claimed”?: How linguistic bias affects generative language models
Patel, R., & Pavlick, E. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.790 · Cited by 11 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
What large language models know and what people think they know
Steyvers, M., Tejeda, H., Kumar, A., Belem, C., Karny, S., Hu, X., Mayer, L. W., & Smyth, P. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-024-00976-7 · Cited by 209 · see citing works
Kim, S. S. Y., Liao, Q. V., Vorvoreanu, M., Ballard, S., & Vaughan, J. W. (2024) · The 2024 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3630106.3658941 · Cited by 147 · see citing works
“Zhou et al. [110] “taught” OpenAI’s GPT-3 [18] model to express (un)certainty through prompt engineering, but also found that the generated expressions were not well calibrated, especially those suggesting high certainty.”
BackgroundInfluentialSycophantic AI decreases prosocial intentions and promotes dependence
Cheng, M., Lee, C., Khadpe, P., Yu, S., Han, D., & Jurafsky, D. (2026) · Science · doi:10.1126/science.aec8352 · Cited by 185 · see citing works
Self-citationSavage, T., Wang, J., Gallo, R., Boukil, A., Patel, V., Safavi-Naini, S. A. A., Soroush, A., & Chen, J. H. (2025) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocae254 · Cited by 92 · see citing works
A Survey of Confidence Estimation and Calibration in Large Language Models . -
Geng, J., Cai, F., Wang, Y., Koeppl, H., Nakov, P., & Gurevych, I. (2024) · Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) · doi:10.18653/v1/2024.naacl-long.366 · Cited by 88 · see citing works
Public attitudes toward chatgpt on twitter: sentiments, topics, and occupations
Koonchanok, R., Pan, Y., & Jang, H. (2024) · Social Network Analysis and Mining · doi:10.1007/s13278-024-01260-7 · Cited by 25 · see citing works
Xu, Z., Song, T., & Lee, Y.-C. (2025) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2025.103455 · Cited by 18 · see citing works
Large Language Model Uncertainty Measurement and Calibration for Medical Diagnosis and Treatment
Savage, T., Wang, J., Gallo, R., Boukil, A., Patel, V., Ahmad Safavi-Naini, S. A., Soroush, A., & Chen, J. H. (2024) · doi:10.1101/2024.06.06.24308399 · Cited by 11 · see citing works
- Authors
- Zhou, K., Jurafsky, D., & Hashimoto, T.
- Affiliations
- Stanford University (US)
- Venue
- Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing
- Published
- 2023
- Pages
- 5506-5524
- OpenAlex
- W4389523793
- Cited by
- 38
- References
- 13
- Citations by year
- 2026: 6 · 2025: 16 · 2024: 13 · 2023: 3
- FWCI
- 6.73
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (2128145)
- Open Philanthropy Project
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Keywords
- epistemic markers, epistemic uncertainty, overconfidence, language models, evidentials, factive verbs, question answering, prompt injection, pre-training data, factual knowledge, model failure
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
