How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answering
Open accessThe DOI matches its Crossref record; 72 of the 72 references it cites were found in a registry or an index.
Abstract
Recent works have shown that language models (LM) capture different types of knowledge regarding facts or common sense. However, because no model is perfect, they still fail to provide appropriate answers in many cases. In this paper, we ask the question, “How can we know when language models know, with confidence, the answer to a particular query?” We examine this question from the point of view of calibration, the property of a probabilistic model’s predicted probabilities actually being well correlated with the probabilities of correctness. We examine three strong generative models—T5, BART, and GPT-2—and study whether their probabilities on QA tasks are well calibrated, finding the answer is a relatively emphatic no. We then examine methods to calibrate such models to make their confidence scores correlate better with the likelihood of correctness through fine-tuning, post-hoc probability modification, or adjustment of the predicted outputs or inputs. Experiments on a diverse range of datasets demonstrate the effectiveness of our methods. We also perform analysis to study the strengths and limitations of these methods, shedding light on further improvements that may be made in methods for calibrating LMs. We have released the code at https://github.com/jzbjyb/lm-calibration.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“Another line of work that sheds light on the nature of hallucinations shows that LMs know when they’re hallucinating [1,3,18,20].”
From Calibrated Language Models Must Hallucinate (2024) · cites this paper as [18]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe cited source doesn’t support the claim as written.
In the citing paper
“Jiang et al. [51] observed the probabilities of pre-trained LMs (e.g., BART, T5, and GPT-2) on QA tasks are well calibrated.”
From Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing (2023) · cites this paper as [51]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Past work on uncertainty for LLMs has focused on simpler settings, such as classifiers16,17 and regressors18,19, whereas the most exciting applications of LLMs relate to free-form generations.”
From Detecting hallucinations in large language models using semantic entropy (2024) · cites this paper as [16]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“find only a weak correlation between the log-likelihood models assign to their answer and the answer's correctness.”
From Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation (2023) · cites this paper as Jiang et al. (2021)
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“More complex decision-making is currently bottlenecked by issues such as over-confidence and miscalibration (Jiang et al., 2021; Braverman et al., 2020; Chen et al., 2022), misalignment with human values or bias (Liang et al., 2021; Feng et al., 2023), hallucinations in self-evaluation (Shinn et al., 2023), and lack of human-in-the-loop mechanisms in face of uncertainties (Nguyen et al., 2022a; R”
From Cognitive Architectures for Language Agents (2023) · cites this paper as Jiang et al., 2021
8 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 2×
“We found that larger LMs usually achieve both higher accuracy and better calibration performance, which is contradictory to the observation in image clas-sification (Guo et al., 2017) where larger models such as ResNet (He et al., 2016) are no longer well calibrated compared to smaller models like LeNet (Lecun et al.,…”
MethodVaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 1×
“The T5 model is a sequence-to-sequence model with both encoder and decoder using trans-formers (Vaswani et al., 2017), and the largest version has 11 billion parameters, allowing it to realize state-of-the-art performance on tasks such as question answering and natural language understanding…”
BackgroundGradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 2×
“We found that larger LMs usually achieve both higher accuracy and better calibration performance, which is contradictory to the observation in image clas-sification (Guo et al., 2017) where larger models such as ResNet (He et al., 2016) are no longer well calibrated compared to smaller models like LeNet (Lecun et al.,…”
MethodXGBoost: A Scalable Tree Boosting System
Chen, T., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939785 · Cited by 54,569 · cited 2×
“We follow previous works (Jagannatha and Yu, 2020; Dong et al., 2018) and use gradient boosted decision trees (Chen and Guestrin, 2016) as our regres-sor to estimate the confidence based on features.”
MethodInfluentialBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 3×
“LMs are now a ubiquitous tool in not only natu-ral language generation, but also natural language understanding (NLU), where they are largely used for unsupervised representation learning in pre-trained models such as BERT (Devlin et al., 2019).”
MethodInfluentialLanguage Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Kaugeranna, R., Kaugeranna, E., & Aion, (. S. 4. 6. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.giscience.2023.43 · Cited by 13,690
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · cited 3×
“No-tably, large models trained on massive crawls of internet text (such as T5 (Raffel et al., 2019) and GPT-3 (Brown et al., 2020)) have been shown to be able to perform quite sophisticated knowledge-based tasks simply through prompting the model to predict the next words given a particular cue.”
BackgroundMethodScaling Laws for Neural Language Models
Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., & Amodei, D. (2020) · arXiv · doi:10.48550/arxiv.2001.08361 · Cited by 9,550 · cited 1×
“Unlike ResNet trained on CIFAR-100, the training of LMs is not bottlenecked by the dataset, and larger LMs have a stronger capacity to model text distribution and memorize facts, which leads to better calibration performance overall (Kaplan et al., 2020).”
MethodLewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.703 · Cited by 8,821
Research works printed but not linked: 4
- Hewitt, 2019, A structural probe for finding syntax in word representations, 4129
- Ng, 2019, Facebook fair’s WMT19 news translation task submission, 314
- Quach, 2020, Researchers made an OpenAI GPT-3 medical chatbot as an experiment. It told a mock patient to kill themselves, The Register
- Radford, 2019, Language models are unsupervised multitask learners, OpenAI Blog, 1
Cited by
Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., & Neubig, G. (2023) · ACM Computing Surveys · doi:10.1145/3560815 · Cited by 4,361
“• Answer Paraphrasing Thesemethods start with an initial answer spaceZ, and then use paraphrasing to expand this answer space to broaden its coverage [51].”
BackgroundMethodInfluentialSelf-citationRecent Advances in Natural Language Processing via Large Pre-trained Language Models: A Survey
Min, B., Ross, H., Sulem, E., Veyseh, A. P. B., Nguyen, T. H., Sainz, O., Agirre, E., Heintz, I., & Roth, D. (2024) · ACM Computing Surveys · doi:10.1145/3605943 · Cited by 1,387
“The prompts can be combined using a uniform average [75, 166, 210] or a weighted average [75, 143, 166, 167].”
BackgroundMethodInfluentialDetecting hallucinations in large language models using semantic entropy
Farquhar, S., Kossen, J., Kuhn, L., & Gal, Y. (2024) · Nature · doi:10.1038/s41586-024-07421-0 · Cited by 934
Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models
Zhang, Y., Li, Y., Cui, L., Cai, D., Liu, L., Fu, T., Huang, X., Zhao, E., Zhang, Y., Chen, Y., et al. (2025) · Computational Linguistics · doi:10.1162/coli.a.16 · Cited by 534
Calibrate Before Use: Improving Few-Shot Performance of Language Models
Zhao, T. Z., Wallace, E., Feng, S., Klein, D., & Singh, S. (2021) · arXiv · doi:10.48550/arxiv.2102.09690 · Cited by 513
Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., & Neubig, G. (2021) · arXiv · doi:10.48550/arxiv.2107.13586 · Cited by 4,352
“• Answer Paraphrasing Thesemethods start with an initial answer spaceZ, and then use paraphrasing to expand this answer space to broaden its coverage [51].”
BackgroundMethodInfluentialSelf-citationActive Retrieval Augmented Generation
Jiang, Z., Xu, F., Gao, L., Sun, Z., Liu, Q., Dwivedi-Yu, J., Yang, Y., Callan, J., & Neubig, G. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.495 · Cited by 497
Self-citationGenerating with Confidence: Uncertainty Quantification for Black-box Large Language Models
Lin, Z., Trivedi, S., & Sun, J. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.19187 · Cited by 35
A systematic survey of prompt engineering on vision-language foundation models
Gu, J., Han, Z., Chen, S., Beirami, A., He, B., Zhang, G., Liao, R., Qin, Y., Tresp, V., & Torr, P. (2023) · arXiv · doi:10.48550/arxiv.2307.12980 · Cited by 280
“Prompt engineering has been widely adopted in various natural language processing applications including question answering [174, 175], text classification [61, 176], text generation [36, 23, 177], and information extraction [178, 179], etc .”
BackgroundRetrieval-Augmented Generation for AI-Generated Content: A Survey
Zhao, P., Zhang, H., Yu, Q., Wang, Z., Geng, Y., Fu, F., Yang, L., Zhang, W., Jiang, J., & Cui, B. (2024) · arXiv · doi:10.48550/arxiv.2402.19473 · Cited by 240
“Jiang et al. [212] studied Model Uncertainty, Input Uncertainty, and Input Statistics to comprehensively assess the confidence level of the model.”
Large Language Models for Information Retrieval: A Survey
Zhu, Y., Yuan, H., Wang, S., Liu, J., Liu, W., Deng, C., Chen, H., Liu, Z., Dou, Z., & Wen, J.-R. (2026) · ACM Transactions on Information Systems · doi:10.1145/3748304 · Cited by 225
“Since the probability can serve as an indicator of LLMs’ confidence during text generation [186, 187], a low probability for a generated term could suggest that LLMs require additional knowledge.”
What large language models know and what people think they know
Steyvers, M., Tejeda, H., Kumar, A., Belem, C., Karny, S., Hu, X., Mayer, L. W., & Smyth, P. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-024-00976-7 · Cited by 209
“This method for reading out model confidence allows a direct computation of the relative probabilities of different possible answers in multiple-choice questions 12,13,18,19,23 and the probability that the answer to an open-ended question is correct 13,17 .”
InfluentialHadi, M. U., tashi, Q. A., Qureshi, R., Shah, A., muneer, A., Irfan, M., Zafar, A., Shaikh, M. B., Akhtar, N., Wu, J., et al. (2023) · doi:10.36227/techrxiv.23589741.v4 · Cited by 103
Language Models (Mostly) Know What They Know
Kadavath, S., Conerly, T., Askell, A., Henighan, T., Drain, D., Perez, E., Schiefer, N., Hatfield-Dodds, Z., DasSarma, N., Tran-Johnson, E., et al. (2022) · arXiv · doi:10.48550/arxiv.2207.05221 · Cited by 187
PAQ: 65 Million Probably-Asked Questions and What You Can Do With Them
Lewis, P., Wu, Y., Liu, L., Minervini, P., Küttler, H., Piktus, A., Stenetorp, P., & Riedel, S. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00415 · Cited by 146
Know Your Limits: A Survey of Abstention in Large Language Models
Wen, B., Yao, J., Feng, S., Xu, C., Tsvetkov, Y., Howe, B., & Wang, L. L. (2025) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00754 · Cited by 56
“Label smoothing (Zhao et al., 2022; Xiao et al., 2022; Jiang et al., 2021) Prompting-based methods In-context examples and hints can enhance model performance on abst-ention.”
A survey on knowledge-oriented retrieval-augmented generation
Cheng, M., Luo, Y., Ouyang, J., Liu, Q., Liu, H., Li, L., Yu, S., Zhang, B., Cao, J., Ma, J., et al. (2025) · arXiv · doi:10.48550/arxiv.2503.10677 · Cited by 140
“Calibration techniques [105] help models identify the most crucial information, filtering out less relevant retrievals before reasoning takes place.”
A Survey of Reasoning with Foundation ModelsPreprint
Sun, J., Zheng, C., Xie, E., Liu, Z., Chu, R., Liu, J., Xu, J., Ding, M., Li, H., Geng, M., et al. (2023) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/ac4sp · Cited by 157
GPT-3-Driven Pedagogical Agents to Train Children’s Curious Question-Asking Skills
Abdelghani, R., Wang, Y.-H., Yuan, X., Wang, T., Lucas, P., Sauzéon, H., & Oudeyer, P.-Y. (2024) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-023-00340-7 · Cited by 129
“This technique has been applied in a wide range of applications such as knowledge-based question-answering [25] but, up to our knowledge has not yet been applied for curiosity-based questions generation.”
Surface Form Competition: Why the Highest Probability Answer Isn’t Always Right
Holtzman, A., West, P., Shwartz, V., Choi, Y., & Zettlemoyer, L. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.564 · Cited by 127
The “Problem” of Human Label Variation: On Ground Truth in Data, Modeling and Evaluation
Plank, B. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.731 · Cited by 121
“, 2017) is increasing (Desai and Durrett, 2020; Kong et al., 2020; Jiang et al., 2021) to counter overconfidence of neural classifiers (Meister et al.”
BackgroundLe Guellec, B., Lefèvre, A., Geay, C., Shorten, L., Bruge, C., Hacein-Bey, L., Amouyel, P., Pruvo, J.-P., Kuchcinski, G., & Hamroun, A. (2024) · Radiology: Artificial Intelligence · doi:10.1148/ryai.230364 · Cited by 115
“Other methods exist to assess the level of certainty for LLM ratings, including examining the probability associated with each output token, a proxy for confidence level (32).”
MethodQuach, V., Fisch, A., Schuster, T., Yala, A., Sohn, J. H., Jaakkola, T., & Barzilay, R. (2023) · International Conference on Learning Representations · doi:10.48550/arxiv.2306.10193 · Cited by 162
“Most current mitigation approaches focus on introducing linguistic cues [39; 80] or empirical post-hoc logit calibration [25; 29; 44; 78].”
BackgroundCalibration and Correctness of Language Models for Code
Spieß, C. K., Gros, D., Pai, K. S., Pradel, M., Rabin, M. R. I., Alipour, M. A., Jha, S., Dévanbu, P., & Ahmed, T. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00040 · Cited by 120
“More recent works evaluated LLM calibration on a wide variety of settings [7], [18], [25], [59].”
Calibrated Language Models Must Hallucinate
Kalai, A. T., & Vempala, S. S. (2024) · Proceedings of the 56th Annual ACM Symposium on Theory of Computing · doi:10.1145/3618260.3649777 · Cited by 91
“" The Good-Turing estimator was shown to be within | 𝑝 ( 𝑈 ) − (cid:100) 𝑀𝐹 | = ˜ 𝑂 ( √︁ 1 / )”
A Survey of Confidence Estimation and Calibration in Large Language Models . -
Geng, J., Cai, F., Wang, Y., Koeppl, H., Nakov, P., & Gurevych, I. (2024) · Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) · doi:10.18653/v1/2024.naacl-long.366 · Cited by 88
“Calibration methods Jiang et al. (2021) proposed various fine-tuning loss functions and temperature scaling for calibrating the performance of MQCA datasets.”
BackgroundMethodInfluentialLM vs LM: Detecting Factual Errors via Cross Examination
Cohen, R., Hamri, M., Geva, M., & Globerson, A. (2023) · arXiv · doi:10.48550/arxiv.2305.13281 · Cited by 85
True Few-Shot Learning with Prompts—A Real-World Perspective
Schick, T., & Schütze, H. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00485 · Cited by 111
“…know a priori how well it performs for a given task; we therefore believe an important next step is to investigate methods for estimating performance without access to large test sets – for example, through model calibration (Desai and Durrett, 2020; Jiang et al., 2021) – in real-world settings.”
BackgroundUnderstanding LLMs: A comprehensive overview from training to inference
Liu, Y., He, H., Han, T., Zhang, X., Liu, M., Tian, J., Zhang, Y., Wang, J., Gao, X., Zhong, T., et al. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.129190 · Cited by 106
LoRA dropout as a sparsity regularizer for overfitting reduction
Lin, Y., Ma, X., Chu, X., Jin, Y., Yang, Z., & Wang, Y. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.114241 · Cited by 85
Cognitive Architectures for Language Agents
Sumers, T. R., Yao, S., Narasimhan, K., & Griffiths, T. L. (2023) · arXiv · doi:10.48550/arxiv.2309.02427 · Cited by 798
“On the other hand, LLM development might be influenced or even shaped by the increased usage of reasoning toward complex decision making, and better support decision making by solving known issues such as over-confidence and miscalibration (Jiang et al., 2021; Braverman et al., 2020; Chen et al., 2022), misalignment w…”
BackgroundReducing Conversational Agents’ Overconfidence Through Linguistic Calibration
Mielke, S. J., Szlam, A., Dinan, E., & Boureau, Y.-L. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00494 · Cited by 72
“Rather than us-ing LMs as target predictors on classification tasks like NLI and NER, Jiang et al. (2021) instead focus on LMs as natural language generators and analyze T5 (Raffel et al., 2020), a large scale Trans-former with an encoder-decoder architecture.”
BackgroundReducing conversational agents' overconfidence through linguistic calibration
Mielke, S. J., Szlam, A., Dinan, E., & Boureau, Y.-L. (2020) · arXiv · doi:10.48550/arxiv.2012.14983 · Cited by 72
“Rather than us-ing LMs as target predictors on classification tasks like NLI and NER, Jiang et al. (2021) instead focus on LMs as natural language generators and analyze T5 (Raffel et al., 2020), a large scale Trans-former with an encoder-decoder architecture.”
Background”My AI is Lying to Me”: User-reported LLM hallucinations in AI mobile apps reviews
Massenon, R., Gambo, I., Khan, J. A., Agbonkhese, C., & Alwadain, A. S. (2025) · Scientific Reports · doi:10.1038/s41598-025-15416-8 · Cited by 86
Language models, like humans, show content effects on reasoning tasks
Lampinen, A. K., Dasgupta, I., Chan, S. C. Y., Sheahan, H. R., Creswell, A., Kumaran, D., McClelland, J. L., & Hill, F. (2024) · PNAS Nexus · doi:10.1093/pnasnexus/pgae233 · Cited by 69
“Thus, LM representations often reflect human semantic cognition; e.g. language models re-produce patterns like association and typicality effects (30, 31), and LM predictions can reproduce human knowledge and beliefs (32, 33).”
Hadi, M. U., tashi, Q. A., Qureshi, R., Shah, A., muneer, A., Irfan, M., Zafar, A., Shaikh, M. B., Akhtar, N., Wu, J., et al. (2023) · doi:10.36227/techrxiv.23589741.v3 · Cited by 53
Generation Probabilities Are Not Enough: Uncertainty Highlighting in AI Code Completions
Vasconcelos, H., Bansal, G., Fourney, A., Liao, Q. V., & Vaughan, J. W. (2024) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3702320 · Cited by 90
Introduction to large language models (LLMs) for dementia care and research
Treder, M. S., Lee, S., & Tsvetanov, K. A. (2024) · Frontiers in Dementia · doi:10.3389/frdem.2024.1385303 · Cited by 63
Crawling The Internal Knowledge-Base of Language Models
Cohen, R., Geva, M., Berant, J., & Globerson, A. (2023) · Findings of the Association for Computational Linguistics: EACL · doi:10.18653/v1/2023.findings-eacl.139 · Cited by 112
Uncertainty-Based Abstention in LLMs Improves Safety and Reduces Hallucinations
Tomani, C., Chaudhuri, K., Evtimov, I., Cremers, D., & Ibrahim, M. (2024) · arXiv · doi:10.48550/arxiv.2404.10960
“Jiang et al. [2021] and Malinin and Gales [2021] compute predictive entropy by focusing on token-wise conditional probabilities, thereby measuring lexical confidence.”
LM vs LM: Detecting Factual Errors via Cross Examination
Cohen, R., Hamri, M., Geva, M., & Globerson, A. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.778 · Cited by 60
Teaching Models to Express Their Uncertainty in Words
Lin, S., Hilton, J., & Evans, O. (2022) · arXiv · doi:10.48550/arxiv.2205.14334 · Cited by 59
“In the specific case of natural language generation, Jiang et al. (2021) study calibration by framing multiple-choice and extractive QA as generative tasks, where a language model’s uncertainty can be extracted from its logits over all tokens in an answer sequence.”
BackgroundA Comprehensive Evaluation of Large Language Models on Legal Judgment Prediction
Shui, R., Cao, Y., Wang, X., & Chua, T. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.490 · Cited by 93
Varshney, N., Mishra, S., & Baral, C. (2022) · Findings of the Association for Computational Linguistics: ACL 2022 · doi:10.18653/v1/2022.findings-acl.158 · Cited by 32
Uncertainty Quantification and Confidence Calibration in Large Language Models: A Survey
Liu, X., Chen, T., Da, L., Chen, C., Lin, Z., & Wei, H. (2025) · Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2 · doi:10.1145/3711896.3736569 · Cited by 53
Kuhn, L., Gal, Y., & Farquhar, S. (2023) · arXiv · doi:10.48550/arxiv.2302.09664 · Cited by 700
“Jiang et al. (2021) do examine calibration in generative question answering and find only a weak correlation between the log-likelihood models assign to their answer and the answer’s correctness.”
BackgroundCan LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs
Xiong, M., Hu, Z., Lu, X., Li, Y., Fu, J., He, J., & Hooi, B. (2023) · arXiv · doi:10.48550/arxiv.2306.13063 · Cited by 618
“We then benchmark these methods on two key tasks—confidence calibration and failure prediction—across five types of tasks (Commonsense, Arithmetic, Symbolic, Ethics and Professional Knowledge) and five widely-used LLMs, i.e., GPT-3 (Brown et al., 2020), GPT-3.”
BackgroundInfluentialCan AI Keep You Safe? A Study of Large Language Models for Phishing Detection
Chataut, R., Gyawali, P., & Usman, Y. (2024) · IEEE Annual Computing and Communication Workshop and Conference (CCWC) · doi:10.1109/ccwc60891.2024.10427626 · Cited by 65
“5) and the higher confidence scores in the more accurate models (GPT-4 and CyberGPT) suggests that these LLMs are well-calibrated [11].”
BackgroundOn the Calibration of Large Language Models and Alignment
Zhu, C., Xu, B., Wang, Q., Zhang, Y., & Mao, Z. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.654 · Cited by 109
The Art of SOCRATIC QUESTIONING: Recursive Thinking with Large Language Models
Qi, J., Xu, Z., Shen, Y., Liu, M., Jin, D., Wang, Q., & Huang, L. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.255 · Cited by 80
“From this example, we see that LLMs, such as GPT-3 or ChatGPT, have strong capabilities not only in reasoning but also self-questioning.”
BackgroundInfluentialSlobodkin, A., Goldman, O., Caciularu, A., Dagan, I., & Ravfogel, S. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.220 · Cited by 143
NORMSAGE: Multi-Lingual Multi-Cultural Norm Discovery from Conversations On-the-Fly
Fung, Y. R., Chakrabarty, T., Guo, H., Rambow, O., Muresan, S., & Ji, H. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.941 · Cited by 91
STraTA: Self-Training with Task Augmentation for Better Few-shot Learning
Vu, T., Luong, M.-T., Le, Q. V., Simon, G., & Iyyer, M. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.462 · Cited by 59
“This can be problematic as state-of-the-art language models like BERT are overconfident and poorly calibrated (Jiang et al., 2021).”
BackgroundStop Measuring Calibration When Humans Disagree
Baan, J., Aziz, W. F., Plank, B., & Fernández, R. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.124 · Cited by 109
Relying on the Unreliable: The Impact of Language Models' Reluctance to Express Uncertainty
Zhou, K., Hwang, J. D., Ren, X., & Sap, M. (2024) · arXiv · doi:10.48550/arxiv.2401.06730 · Cited by 44
“In machine learning, work has focused on improving model calibration (Jiang et al., 2021; De-sai and Durrett, 2020; Jagannatha and Yu, 2020; Kamath et al., 2020; Kong et al., 2020) by calibrating the confidence value assigned by a model and model accuracy through a measure called ECE (Naeini et…”
MethodAutomatic Model Selection with Large Language Models for Reasoning
Zhao, J., Xie, Y., Kawaguchi, K., He, J., & Xie, M. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.55 · Cited by 58
Navigating the Grey Area: How Expressions of Uncertainty and Overconfidence Affect Language Models
Zhou, K., Jurafsky, D., & Hashimoto, T. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.335 · Cited by 38
Large Language Models Are Unreliable for Cyber Threat Intelligence
Mezzi, E., Massacci, F., & Tuma, K. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-00627-1_17 · Cited by 45
“From Jiang et al. [57], given an input X and true output Y , a model output ˆ Y , and a probability PN ( ˆ Y | X ) calculated over this output, a perfectly calibrated model satisfies the condition: which is that the confidence p of the model, corresponding to the calculated probability of its…”
SimBench: Benchmarking the Ability of Large Language Models to Simulate Human Behaviors
Hu, T., Baumann, J., Lupo, L., Collier, N., Hovy, D., & Röttger, P. (2025) · International Conference on Learning Representations · doi:10.48550/arxiv.2510.17516 · Cited by 52
Predictable artificial intelligence
Zhou, L., Casares, P. A. M., Martínez‐Plumed, F., Burden, J., Burnell, R., Cheke, L. G., Ferri, C. S., Marcoci, A., Mehrbakhsh, B., Moros-Daval, Y., et al. (2026) · Artificial Intelligence · doi:10.1016/j.artint.2026.104491 · Cited by 41
Linguistic Calibration of Long-Form GenerationsPreprint
Band, N., Li, X., Ma, T., & Hashimoto, T. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2404.00474 · Cited by 56
“However, both classic calibration methods like temperature scaling (Guo et al., 2017) and methods for LMs (Jiang et al., 2021, Kadavath et al., 2022, Kuhn et al., 2023, Lin et al., 2022, Mielke et al., 2022, Tian et al., 2023) are restricted to classification or short outputs and hence cannot…”
Getting MoRE out of Mixture of Language Model Reasoning Experts
Si, C., Shi, W., Zhao, C., Zettlemoyer, L., & Boyd-Graber, J. L. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.552 · Cited by 67
“However, confidence scores of the generated answers can be poorly calibrated (Jiang et al., 2020; Si et al., 2022) for this purpose.”
BackgroundBuilding trustworthy NeuroSymbolic AI Systems: Consistency, reliability, explainability, and safety
Gaur, M., & Sheth, A. (2024) · AI Magazine · doi:10.1002/aaai.12149 · Cited by 64
Misra, K., Rayz, J., & Ettinger, A. (2023) · Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.213 · Cited by 15
“Similarly, standard cloze-testing of PLMs (Petroni et al., 2019; Weir et al., 2020; Jiang et al., 2021) also faces multiple limitations.”
BackgroundA psychometric framework for evaluating and shaping personality traits in large language models
Serapio-García, G., Safdari, M., Crepy, C., Sun, L., Fitz, S., Romero, P., Abdulhai, M., Faust, A., & Matarić, M. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-025-01115-6 · Cited by 30
Li, J., Yang, Y., Zhang, R., Liao, Q. V., Song, T., Xu, Z., & Lee, Y.-C. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2402.07632 · Cited by 34
“Addressing the miscalibration issue of AI models is a common focus in the fields of machine learning and AI [23, 27, 70].”
Self-Evaluation Guided Beam Search for Reasoning
Xie, Y., Kawaguchi, K., Zhao, Y., Zhao, J. X., Kan, M.-Y., He, J., & Xie, M. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-1802 · Cited by 27
“…shows that current LLMs’ probabilistic predictions correspond well with actual token occurrence frequencies, leading to well-calibrated predictions for specific tasks (Rae et al., 2021; Kadavath et al., 2022; Guo et al., 2017; Kadavath et al., 2022; Jiang et al., 2021; Kuhn et al., 2023).”
BackgroundJoffe, I., Felobes, G., Elgouhari, Y., Talebi-Kalaleh, M., Mei, Q., & Chui, Y. H. (2025) · Journal of Computing in Civil Engineering · doi:10.1061/jccee5.cpeng-6037 · Cited by 34
Utilizing large language models for gastroenterology research: a conceptual framework
Berry, P., Dhanakshirur, R. R., & Khanna, S. (2025) · Therapeutic Advances in Gastroenterology · doi:10.1177/17562848251328577 · Cited by 29
“62 Temperature scaling is a key calibration method that refines confidence levels by correcting systematic overconfidence.”
Taking Advice from ChatGPTPreprint
Zhang, P. (2023) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/b53vn · Cited by 26
Zhang, D., Zhang, X., Bansal, C., Las-Casas, P., Fonseca, R., & Rajmohan, S. (2024) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3663529.3663858 · Cited by 41
Multi-lingual and Multi-cultural Figurative Language Understanding
Kabra, A., Liu, E., Khanuja, S., Aji, A. F., Winata, G. I., Cahyawijaya, S., Aremu, A., Ogayo, P., & Neubig, G. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.525 · Cited by 24
Self-citationStrength in Numbers: Estimating Confidence of Large Language Models by Prompt Agreement
Portillo-Wightman, G., DeLucia, A., & Dredze, M. H. (2023) · Workshop on Trustworthy Natural Language Processing (TrustNLP) · doi:10.18653/v1/2023.trustnlp-1.28 · Cited by 71
“Log probability of the generation is a common method for confidence estimation (Jiang et al., 2021; Nguyen and O’Connor, 2015; Dong et al., 2018).”
BackgroundMethodInfluentialMerging Generated and Retrieved Knowledge for Open-Domain QA
Zhang, Y., Khalifa, M., Logeswaran, L., Lee, M., Lee, H., & Wang, L. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.286 · Cited by 62
Rethinking Inference-Time Scaling: Efficiency Limits and Linguistic SignalsPreprint
Wang, J., Zhu, S., Saad-Falcon, J., Athiwaratkun, B., Wu, Q., Wang, J., Song, S. L., Zhang, C., Dhingra, B., & Zou, J. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2504.14047 · Cited by 25
“Early calibration studies examined whether token probabilities align with answer accuracy [Jiang et al., 2021, Tian et al., 2023].”
Generation Probabilities Are Not Enough: Uncertainty Highlighting in AI Code Completions
Vasconcelos, H., Bansal, G., Fourney, A., Liao, Q. V., & Vaughan, J. W. (2023) · arXiv · doi:10.48550/arxiv.2302.07248 · Cited by 23
“Post hoc calibration methods and auxiliary models can also be used to compute and improve uncertainty scores [21, 26, 40].”
MethodChatGPT Incorrectness Detection in Software Reviews
Tanzil, M. H., Khan, J. Y., & Uddin, G. (2024) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3597503.3639194 · Cited by 35
Zero‐shot animal behaviour classification with vision‐language foundation models
Dussert, G., Miele, V., Reeth, C. V., Delestrade, A., Dray, S., & Chamaillé‐Jammes, S. (2025) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.70059 · Cited by 26
“Following Jiang et al. ( 2021 ), we normalize the pre - dicted behaviour's score by the sum of the scores of the candidate's words only (the three behaviours words).”
Selecting Better Samples from Pre-trained LLMs: A Case Study on Question Generation
Yuan, X., Wang, T., Wang, Y.-H., Fine, E., Abdelghani, R., Sauzéon, H., & Oudeyer, P.-Y. (2023) · Findings of the Association for Computational Linguistics: ACL 2023 · doi:10.18653/v1/2023.findings-acl.820 · Cited by 21
Leave It to Large Language Models! Correction and Planning with Memory Integration
Zhang, Y., Wang, C., Qi, J., & Peng, Y. (2023) · Cyborg and Bionic Systems · doi:10.34133/cbsystems.0087 · Cited by 24
“However, expanding or replacing LLMs can introduce derivative issues, such as hallucination [26,27], which will result in unstable output.”
BackgroundSantoso, J., Ishizuka, K., & Hashimoto, T. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10448316 · Cited by 43
“These models have revolutionized the field of natural language processing (NLP) through their ability to comprehend context, semantics, and nuances in text, resulting in various applications, including dialog summarization [2] and question-answering [3].”
BackgroundGarcía-Olano, D., Onoe, Y., & Ghosh, J. (2022) · International World Wide Web Conference (WWW) · doi:10.1145/3487553.3524648 · Cited by 20
“(Desai and Durrett 2020; Jiang et al. 2021) E-BERTs effect on VQA explainability We extract visual and text explanations using BM-GAE and TRF on our KVQA models and Table 4 shows accuracy results for when these explanation methods find E-BERT enhanced entities to be in the top 5 most important…”
Selectively Answering Ambiguous Questions
Cole, J., Zhang, M., Gillick, D., Eisenschlos, J., Dhingra, B., & Eisenstein, J. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.35 · Cited by 19
Evaluation of LLMs for mathematical problem solving
Wang, R., Wang, R., Shen, Y., Wu, C., Zhou, Q., & Chandra, R. (2026) · Next research. · doi:10.1016/j.nexres.2026.101705 · Cited by 22
LMSanitator: Defending Prompt-Tuning Against Task-Agnostic Backdoors
Wei, C., Meng, W., Zhang, Z., Chen, M., Zhao, M., Fang, W., Wang, L., Zhang, Z., & Chen, W. (2024) · Network and Distributed System Security Symposium · doi:10.14722/ndss.2024.23238 · Cited by 31
“Prompt paradigm in NLP freezes all parameters of a pretrained model and uses a natural language prompt to query a language model [50, 27, 5, 60, 63, 59, 22].”
BackgroundSelf-Supervised Query Reformulation for Code Search
Mao, Y., Wan, C., Jiang, Y., & Gu, X. (2023) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3611643.3616306 · Cited by 23
“By pre-training on large-scale text corpora, T5 achieves state-of-the-art performance in a variety of NLP tasks, such as sentence acceptability judgment [54], sentiment analysis [41], paraphrasing similarity calculation [39], and question answering [27].”
BackgroundA Question and Answering Service of Typhoon Disasters Based on the T5 Large Language Model
Xia, Y., Huang, Y., Qiu, Q., Zhang, X., Miao, L., & Chen, Y. (2024) · ISPRS International Journal of Geo-Information · doi:10.3390/ijgi13050165 · Cited by 28
“Open-source LLMs are used as a foundation to train domain-specific models [42].”
Selectively Answering Ambiguous Questions
Cole, J. R., Zhang, M. J. Q., Gillick, D., Eisenschlos, J. M., Dhingra, B., & Eisenstein, J. (2023) · arXiv · doi:10.48550/arxiv.2305.14613 · Cited by 19
Xu, Z., Song, T., & Lee, Y.-C. (2025) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2025.103455 · Cited by 18
Are Large Language Models More Honest in Their Probabilistic or Verbalized Confidence?
Ni, S., Bi, K., Yu, L., & Guo, J. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-1710-4_10 · Cited by 19
Feasibility Study for Supporting Static Malware Analysis Using LLM
Fujii, S., & Yamagishi, R. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-82362-6_1 · Cited by 30
Active Retrieval Augmented Generation
Jiang, Z., Xu, F. F., Gao, L., Sun, Z., Liu, Q., Dwivedi-Yu, J., Yang, Y., Callan, J., & Neubig, G. (2023) · arXiv · doi:10.48550/arxiv.2305.06983 · Cited by 18
“Given the observation that large LMs tend to be well-calibrated and low probability/confidence often indicates a lack of knowledge (Jiang et al., 2021; Kadavath et al., 2022), we adopt an active retrieval strategy that only retrieves when LMs generate low-probability tokens.”
BackgroundMethodSelf-citationProbabilistic Tree-of-thought Reasoning for Answering Knowledge-intensive Complex Questions
Cao, S., Zhang, J., Shi, J., Lv, X., Yao, Z., Tian, Q., Hou, L., & Li, J. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.835 · Cited by 17
Few-shot Reranking for Multi-hop QA via Language Model Prompting
Khalifa, M., Logeswaran, L., Lee, M., Lee, H., & Wang, L. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.885 · Cited by 37
“This can be seen as an instance of model calibration (Guo et al., 2017; Desai and Durrett, 2020; Jiang et al., 2021) with the goal of improving the reranking scores.”
BackgroundQuantifying uncert-AI-nty: Testing the accuracy of LLMs’ confidence judgments
Cash, T. N., Oppenheimer, D. M., Christie, S., & Devgan, M. (2025) · Memory & Cognition · doi:10.3758/s13421-025-01755-4 · Cited by 31
Ulmer, D., Frellsen, J., & Hardmeier, C. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.198 · Cited by 46
Understanding Knowledge Drift in LLMs Through Misinformation
Fastowski, A., & Kasneci, G. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-82346-6_5 · Cited by 26
Answer-level Calibration for Free-form Multiple Choice Question Answering
Kumar, S. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.49 · Cited by 16
“Finally, Jiang et al. (2021) explore supervised methods, including finetuning as well as post-hoc methods, to improve calibration using training examples.”
MethodPromptGen: Automatically Generate Prompts using Generative Models
Zhang, Y., Fei, H., Li, D., & Li, P. (2022) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2022.findings-naacl.3 · Cited by 35
“Recently, prompt learning has been used in applications such as knowledge probing (Petroni et al., 2019; Zhong et al., 2021; Jiang et al., 2021), text classification (Gao et al.”
MethodAutomated Identification of Eviction Status From Electronic Health Record Notes
Yao, Z., Tsai, J., Liu, W., Levy, D. A., Druhl, E., Reisman, J. I., & Yu, H. (2023) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocad081 · Cited by 15
“In addition, it is well known that modern neural networks including language models are poorly calibrated, especially in the extremely imbalanced setting [33, 34, 35].”
Background
Metadata
- Authors
- Jiang, Z., Araki, J., Ding, H., & Neubig, G.
- Affiliations
- Carnegie Mellon University (US)
- Robert Bosch (United States) (US)
- Corresponding author
- Jiang, Zhengbao
- Araki, Jun
- Ding, Haibo
- Neubig, Graham
- Venue
- Transactions of the Association for Computational Linguistics
- Publisher
- MIT Press
- Published
- 2021
- Volume
- 9
- Pages
- 962-977
- ISSN
- 2307-387X (ISSN-L)
- OpenAlex
- W3199958362
- Cited by
- 177
- References
- 71
- Citations by year
- 2026: 43 · 2025: 49 · 2024: 32 · 2023: 37 · 2022: 14 · 2021: 2
- FWCI
- 7.48
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Locations
- Transactions of the Association for Computational Linguistics (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- question answering, confidence calibration, probabilistic reasoning, T5, BART, GPT-2, fine-tuning, knowledge capture, factual knowledge, commonsense knowledge
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.