CoChat
Sign Up
Cited by 91Topic ModelingNatural Language Processing TechniquesAdversarial Robustness in Machine Learning

A Survey of Confidence Estimation and Calibration in Large Language Models . -

Open access
Geng, Jiahui, Cai, Fengyu, Wang, Yuxia, Koeppl, Heinz, Nakov, Preslav, Gurevych, Iryna·2024·Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)
DOI 10.18653/v1/2024.naacl-long.366
Open this paper in CoChat — free accountOpen PDF

Abstract

Jiahui Geng, Fengyu Cai, Yuxia Wang, Heinz Koeppl, Preslav Nakov, Iryna Gurevych. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2024.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.18653/v1/2024.naacl-long.366

Cites

0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Recent work on long-form generation shows that calibration is particularly challenging when answers are partially correct rather than strictly true or false.

    “Recent work on long-form generation shows that calibration is particularly challenging when answers are partially correct rather than strictly true or false [9]”

    From Machine Psychometrics: A Mathematical Psychology of Artificial Intelligence (2026) · cites this paper as [9]

    The Model section reports work that treats correctness as a distribution rather than a binary value, and §6 Future Directions calls for fine-grained annotations of long-form responses. These passages support the relevance of nonbinary correctness to calibration, but they do not say calibration is particularly more challenging for partially correct answers than for strictly true-or-false ones. Page numbers are unavailable in our copy. (LLM, conf=0.83)

    Experimental — accuracy not yet measured

  • Studies on hallucination, calibration, sycophancy, and cognitive bias indicate that failures often result not from a lack of capability but from unstable behavioral dispositions under uncertainty or pressure.

    “Studies on hallucination, calibration, sycophancy, and cognitive bias indicate that failures often result not from a lack of capability but from unstable behavioral dispositions under uncertainty or pressure [8, 9, 10, 12, 15].”

    From Machine Psychometrics: A Mathematical Psychology of Artificial Intelligence (2026) · cites this paper as [9]

    The survey’s section on black-box methods links low confidence and inconsistent responses with hallucinations, and Section 5 discusses using uncertainty to detect them (page numbers unavailable). It does not say that failures generally stem from unstable behavioral dispositions rather than limited capability, or address sycophancy and pressure as causes of failure. (LLM, conf=0.85)

    Experimental — accuracy not yet measured

  • Surveys of confidence estimation and hallucination detection show that calibration, uncertainty quantification, and abstention are central to trustworthy behavior in large language models.

    “Surveys of confidence estimation and hallucination detection show that calibration, uncertainty quantification, and abstention are central to trustworthy behavior in large language models [9, 10].”

    From Machine Psychometrics: A Mathematical Psychology of Artificial Intelligence (2026) · cites this paper as [9]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

The publisher hasn’t deposited this paper’s reference list.

Research works printed but not linked: 86

  1. Alfonso Amayuelas, Liangming Pan, Wenhu Chen, and William Wang. 2023. Knowledge of knowledge: Ex- ploring known-unknowns uncertainty with large lan- guage models. ArXiv preprint, abs/2305.13712. · arXiv
  2. Rohan Anil, Andrew M. Dai, Orhan Firat, Melvin John- son, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, Eric Chu, Jonathan H. Clark, Laurent El Shafey, Yanping Huang, Kathy Meier-Hellstern, Gau- rav Mishra, Erica Moreira, Mark Omernick, Kevin Robinson, Sebastian Ruder, Yi Tay, Kefan Xiao, Yuanzhong Xu, Yujing Zhang, Gustavo Hernández Ábrego, Junwhan Ahn, Jacob Austin, Paul Barham, Jan A. Botha, James Bradbury, Siddhartha Brahma, Kevin Brooks, Michele Catasta, Yong Cheng, Colin Cherry, Christopher A. Choquette-Choo, Aakanksha Chowdhery, Clément Crepy, Shachi Dave, Mostafa Dehghani, Sunipa Dev, Jacob Devlin, Mark Díaz, Nan Du, Ethan Dyer, Vladimir Feinberg, Fangxi- aoyu Feng, Vlad Fienber, Markus Freitag, Xavier Garcia, Sebastian Gehrmann, Lucas Gonzalez, and et al. 2023. PaLM 2 technical report. ArXiv preprint, abs/2305.10403. · arXiv
  3. Joris Baan, Nico Daheim, Evgenia Ilia, Dennis Ul- mer, Haau-Sing Li, Raquel Fernández, Barbara Plank, Rico Sennrich, Chrysoula Zerva, and Wilker Aziz. 2023. Uncertainty in natural language gener- ation: From theory to applications. ArXiv preprint, abs/2307.15703. · arXiv
  4. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Ben- gio. 2015. Neural machine translation by jointly learning to align and translate. In Proceedings of the 3rd International Conference on Learning Represen- tations, ICLR'2015, San Diego, CA, USA.
  5. Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. 2022. Training a helpful and harmless assistant with reinforcement learning from human feedback. ArXiv preprint, abs/2204.05862. · arXiv
  6. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Pro- cessing Systems 2020, NeurIPS'2020.
  7. Collin Burns, Haotian Ye, Dan Klein, and Jacob Stein- hardt. 2023. Discovering latent knowledge in lan- guage models without supervision. In Proceedings of the 11th International Conference on Learning Representations, ICLR'2023, Kigali, Rwanda.
  8. Jiuhai Chen and Jonas Mueller. 2023. Quantifying uncertainty in answers from any language model and enhancing their trustworthiness. ArXiv preprint, abs/2308.16175. · arXiv
  9. Charles Corbière, Nicolas Thome, Avner Bar-Hen, Matthieu Cord, and Patrick Pérez. 2019. Addressing failure prediction by learning model confidence. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Pro- cessing Systems, NeurIPS'2019, pages 2898-2909, Vancouver, BC, Canada.
  10. Wenliang Dai, Junnan Li, Dongxu Li, Anthony Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi. 2023. InstructBLIP: Towards general-purpose vision-language models with instruc- tion tuning. In Advances in Neural Information Pro- cessing Systems 36: Annual Conference on Neural Information Processing Systems, NeurIPS'2023, New Orleans, LA, USA.
  11. Terrance DeVries and Graham W Taylor. 2018. Learn- ing confidence for out-of-distribution detection in neural networks. ArXiv preprint, abs/1802.04865. · arXiv
  12. Jinhao Duan, Hao Cheng, Shiqi Wang, Chenan Wang, Alex Zavalny, Renjing Xu, Bhavya Kailkhura, and Kaidi Xu. 2023. Shifting attention to relevance: To- wards the uncertainty estimation of large language models. ArXiv preprint, abs/2307.01379. · arXiv
  13. Ekaterina Fadeeva, Aleksandr Rubashevskii, Artem Shelmanov, Sergey Petrakov, Haonan Li, Hamdy Mubarak, Evgenii Tsymbalov, Gleb Kuzmin, Alexan- der Panchenko, Timothy Baldwin, Preslav Nakov, and Maxim Panov. 2024. Fact-checking the output of large language models via token-level uncertainty quantification. arXiv preprint arXiv:2403.04696. · arXiv
  14. Yarin Gal and Zoubin Ghahramani. 2016. Dropout as a Bayesian approximation: representing model uncer- tainty in deep learning. In Proceedings of the 33rd International Conference on International Confer- ence on Machine Learning -Volume 48, ICML'16, page 1050-1059. JMLR.org.
  15. Jiahui Geng, Yova Kementchedjhieva, Preslav Nakov, and Iryna Gurevych. 2024. Multimodal large lan- guage models to support real-world fact-checking. arXiv preprint arXiv:2403.03627. · arXiv
  16. Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. 2017. On calibration of modern neu- ral networks. In Proceedings of the 34th Interna- tional Conference on Machine Learning -Volume 70, ICML'17, page 1321-1330, Sydney, NSW, Australia. JMLR.org.
  17. Zhixiong Han, Yaru Hao, Li Dong, Yutao Sun, and Furu Wei. 2023. Prototypical calibration for few- shot learning of language models. In Proceedings of the Eleventh International Conference on Learning Representations, ICLR'2023, Kigali, Rwanda.
  18. Marton Havasi, Rodolphe Jenatton, Stanislav Fort, Jeremiah Zhe Liu, Jasper Snoek, Balaji Lakshmi- narayanan, Andrew Mingbo Dai, and Dustin Tran. 2021. Training independent subnetworks for ro- bust prediction. In Proceedings of the 9th Inter- national Conference on Learning Representations, ICLR'2021.
  19. Pengcheng He, Jianfeng Gao, and Weizhu Chen. 2023. DeBERTaV3: Improving DeBERTa us- ing ELECTRA-style pre-training with gradient- disentangled embedding sharing. In Proceedings of the Eleventh International Conference on Learning Representations, ICLR'2023, Kigali, Rwanda.
  20. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Stein- hardt. 2021. Measuring massive multitask language understanding. In Proceedings of the 9th Inter- national Conference on Learning Representations, ICLR'2021.
  21. Bairu Hou, Yujian Liu, Kaizhi Qian, Jacob Andreas, Shiyu Chang, and Yang Zhang. 2023. Decompos- ing uncertainty for large language models through input clarification ensembling. ArXiv preprint, abs/2311.08718. · arXiv
  22. Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, et al. 2023a. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. ArXiv preprint, abs/2311.05232. · arXiv
  23. Yuheng Huang, Jiayang Song, Zhijie Wang, Huam- ing Chen, and Lei Ma. 2023b. Look before you leap: An exploratory study of uncertainty measure- ment for large language models. ArXiv preprint, abs/2307.10236. · arXiv
  24. Yukun Huang, Yixin Liu, Raghuveer Thirukovalluru, Arman Cohan, and Bhuwan Dhingra. 2024. Cali- brating long-form generations from large language models. ArXiv preprint, abs/2402.06544. · arXiv
  25. Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield Dodds, Nova DasSarma, Eli Tran-Johnson, et al. 2022. Language models (mostly) know what they know. ArXiv preprint, abs/2207.05221. · arXiv
  26. Alex Kendall and Yarin Gal. 2017. What uncertainties do we need in Bayesian deep learning for computer vision? In Advances in Neural Information Process- ing Systems 30: Annual Conference on Neural Infor- mation Processing Systems 2017, pages 5574-5584, Long Beach, CA, USA.
  27. Jaeyoung Kim, Dongbin Na, Sungchul Choi, and Sung- bin Lim. 2023. Bag of tricks for in-distribution cali- bration of pretrained transformers. In Findings of the Association for Computational Linguistics: EACL 2023, pages 551-563, Dubrovnik, Croatia. Associa- tion for Computational Linguistics.
  28. Lorenz Kuhn, Yarin Gal, and Sebastian Farquhar. 2023. Semantic uncertainty: Linguistic invariances for un- certainty estimation in natural language generation. In Proceedings of the Eleventh International Con- ference on Learning Representations, ICLR'2023, Kigali, Rwanda.
  29. Meelis Kull, Miquel Perelló-Nieto, Markus Kängsepp, Telmo de Menezes e Silva Filho, Hao Song, and Peter A. Flach. 2019. Beyond temperature scaling: Obtaining well-calibrated multi-class probabilities with Dirichlet calibration. In Advances in Neural Information Processing Systems 32: Annual Confer- ence on Neural Information Processing Systems 2019, NeurIPS'2019, pages 12295-12305, Vancouver, BC, Canada.
  30. Aviral Kumar and Sunita Sarawagi. 2019. Calibration of encoder decoder models for neural machine trans- lation. ArXiv preprint, abs/1903.00802. · arXiv
  31. Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. 2017. Simple and scalable pre- dictive uncertainty estimation using deep ensembles. In Advances in Neural Information Processing Sys- tems 30: Annual Conference on Neural Information Processing Systems 2017, pages 6402-6413, Long Beach, CA, USA.
  32. Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. 2018. A simple unified framework for detecting out- of-distribution samples and adversarial attacks. In Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Pro- cessing Systems 2018, NeurIPS'2018, pages 7167- 7177, Montréal, Canada. 6588
  33. Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg. 2023. Inference- time intervention: Eliciting truthful answers from a language model. In Advances in Neural Infor- mation Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS'2023, New Orleans, LA, USA.
  34. Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Ku- mar, Benjamin Newman, Binhang Yuan, Bobby Yan, Ce Zhang, Christian Alexander Cosgrove, Christo- pher D Manning, Christopher Re, Diana Acosta- Navas, Drew Arad Hudson, Eric Zelikman, Esin Durmus, Faisal Ladhak, Frieda Rong, Hongyu Ren, Huaxiu Yao, Jue WANG, Keshav Santhanam, Laurel Orr, Lucia Zheng, Mert Yuksekgonul, Mirac Suzgun, Nathan Kim, Neel Guha, Niladri S. Chatterji, Omar Khattab, Peter Henderson, Qian Huang, Ryan An- drew Chi, Sang Michael Xie, Shibani Santurkar, Surya Ganguli, Tatsunori Hashimoto, Thomas Icard, Tianyi Zhang, Vishrav Chaudhary, William Wang, Xuechen Li, Yifan Mai, Yuhui Zhang, and Yuta Ko- reeda. 2023. Holistic evaluation of language models. Transactions on Machine Learning Research.
  35. Stephanie Lin, Jacob Hilton, and Owain Evans. 2022. Teaching models to express their uncertainty in words. Transactions on Machine Learning Research.
  36. Zhen Lin, Shubhendu Trivedi, and Jimeng Sun. 2023. Generating with confidence: Uncertainty quantifi- cation for black-box large language models. ArXiv preprint, abs/2305.19187. · arXiv
  37. Genglin Liu, Xingyao Wang, Lifan Yuan, Yangyi Chen, and Hao Peng. 2023a. Prudent silence or foolish bab- ble? Examining large language models' responses to the unknown. ArXiv preprint, abs/2311.09731. · arXiv
  38. Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023b. Visual instruction tuning. In Advances in Neural Information Processing Systems 36: An- nual Conference on Neural Information Processing Systems 2023, NeurIPS'2023, New Orleans, LA, USA.
  39. Yang Liu, Yuanshun Yao, Jean-Francois Ton, Xiaoying Zhang, Ruocheng Guo Hao Cheng, Yegor Klochkov, Muhammad Faaiz Taufiq, and Hang Li. 2023c. Trust- worthy LLMs: a survey and guideline for evaluating large language models' alignment. ArXiv preprint, abs/2308.05374. · arXiv
  40. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Man- dar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A robustly optimized BERT pretraining approach. ArXiv preprint, abs/1907.11692. · arXiv
  41. Andrey Malinin and Mark Gales. 2021. Uncertainty estimation in autoregressive structured prediction. In Proceedings of the 9th International Conference on Learning Representations, ICLR2021, Virtual Event, Austria.
  42. Johannes Mario Meissner, Napat Thumwanit, Saku Sug- awara, and Akiko Aizawa. 2021. Embracing ambi- guity: Shifting the training target of NLI models. In Proceedings of the 59th Annual Meeting of the Asso- ciation for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pages 862-869, Online. Association for Computational Linguistics. · DOI
  43. Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettle- moyer. 2022. Rethinking the role of demonstrations: What makes in-context learning work? In Proceed- ings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 11048-11064, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  44. Jooyoung Moon, Jihyo Kim, Younghak Shin, and Sangheum Hwang. 2020. Confidence-aware learning for deep neural networks. In Proceedings of the 37th International Conference on Machine Learning, vol- ume 119 of ICML'2020, pages 7034-7044.
  45. PMLR.
  46. Jishnu Mukhoti, Viveka Kulharia, Amartya Sanyal, Stu- art Golodetz, Philip H. S. Torr, and Puneet K. Doka- nia. 2020. Calibrating deep neural networks using focal loss. In Advances in Neural Information Pro- cessing Systems 33: Annual Conference on Neural In- formation Processing Systems 2020, NeurIPS'2020.
  47. Feng Nie, Meixi Chen, Zhirui Zhang, and Xu Cheng. 2022. Improving few-shot performance of language models via nearest neighbor calibration. ArXiv preprint, abs/2212.02216. · arXiv
  48. Jeremy Nixon, Michael W. Dusenberry, Linchuan Zhang, Ghassen Jerfel, and Dustin Tran. 2019. Mea- suring calibration in deep learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pages 38-41, Long Beach, CA, USA. Computer Vision Foundation / IEEE.
  49. Tim Pearce, Alexandra Brintrup, and Jun Zhu. 2021. Understanding softmax confidence and uncertainty. ArXiv preprint, abs/2106.04972. · arXiv
  50. Guilherme Penedo, Quentin Malartic, Daniel Hesslow, Ruxandra Cojocaru, Hamza Alobeidli, Alessandro Cappelli, Baptiste Pannier, Ebtesam Almazrouei, and Julien Launay. 2023. The RefinedWeb dataset for Falcon LLM: Outperforming curated corpora with web data only. In Proceedings of the Thirty-seventh Conference on Neural Information Processing Sys- tems Datasets and Benchmarks Track, New Orleans, LA, USA.
  51. Gabriel Pereyra, George Tucker, Jan Chorowski, Lukasz Kaiser, and Geoffrey E. Hinton. 2017. Regularizing neural networks by penalizing confident output dis- tributions. In Workshop Track Proceedings of the 5th International Conference on Learning Representa- tions, Toulon, France.
  52. Barbara Plank. 2022. The "problem" of human label variation: On ground truth in data, modeling and evaluation. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Process- ing, pages 10671-10682, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  53. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sas- try, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learn- ing transferable visual models from natural language supervision. In Proceedings of the 38th International Conference on Machine Learning, volume 139 of ICML'2021, pages 8748-8763. PMLR.
  54. Jie Ren, Jiaming Luo, Yao Zhao, Kundan Krishna, Mo- hammad Saleh, Balaji Lakshminarayanan, and Pe- ter J Liu. 2023. Out-of-distribution detection and selective generation for conditional language models. In Proceedings of the Eleventh International Con- ference on Learning Representations, ICLR'2023, Kigali, Rwanda.
  55. Vaishnavi Shrivastava, Percy Liang, and Ananya Ku- mar. 2023. Llamas know what GPTs don't show: Surrogate models for confidence estimation. ArXiv preprint, abs/2311.08877. · arXiv
  56. Chenglei Si, Chen Zhao, Sewon Min, and Jordan Boyd- Graber. 2022. Re-examining calibration: The case of question answering. In Findings of the Associa- tion for Computational Linguistics: EMNLP 2022, pages 2814-2829, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  57. Hongjin SU, Jungo Kasai, Chen Henry Wu, Weijia Shi, Tianlu Wang, Jiayi Xin, Rui Zhang, Mari Ostendorf, Luke Zettlemoyer, Noah A. Smith, and Tao Yu. 2023. Selective annotation makes language models better few-shot learners. In Proceedings of the Eleventh In- ternational Conference on Learning Representations, ICLR'2023, Kigali, Rwanda.
  58. Hao Sun, Boris van Breugel, Jonathan Crabbe, Nabeel Seedat, and Mihaela van der Schaar. 2022. What is flagged in uncertainty quantification? Latent den- sity models for uncertainty categorization. ArXiv preprint, abs/2207.05161. · arXiv
  59. Alex Tamkin, Kunal Handa, Avash Shrestha, and Noah Goodman. 2023. Task ambiguity in humans and language models. In Proceedings of the Eleventh In- ternational Conference on Learning Representations, ICLR'2023, Kigali, Rwanda.
  60. Katherine Tian, Eric Mitchell, Huaxiu Yao, Christo- pher D Manning, and Chelsea Finn. 2024. Fine- tuning language models for factuality. In Proceed- ings of the Twelfth International Conference on Learning Representations, ICLR'2024, Vienna, Aus- tria.
  61. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurélien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023a. LLaMA: Open and efficient foundation language models. ArXiv preprint, abs/2302.13971. · arXiv
  62. Hugo Touvron, Louis Martin, Kevin Stone, Peter Al- bert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023b. LLaMA 2: Open founda- tion and fine-tuned chat models. arXiv preprint arXiv:2307.09288. · arXiv
  63. Liam van der Poel, Ryan Cotterell, and Clara Meis- ter. 2022. Mutual information alleviates hallucina- tions in abstractive summarization. In Proceedings of the 2022 Conference on Empirical Methods in Nat- ural Language Processing, pages 5956-5965, Abu Dhabi, United Arab Emirates. Association for Com- putational Linguistics.
  64. Neeraj Varshney, Wenlin Yao, Hongming Zhang, Jian- shu Chen, and Dong Yu. 2023. A stitch in time saves nine: Detecting and mitigating hallucinations of LLMs by validating low-confidence generation. ArXiv preprint, abs/2307.03987. · arXiv
  65. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Pro- cessing Systems 30: Annual Conference on Neural Information Processing Systems, ICLR'2017, pages 5998-6008, Long Beach, CA, USA.
  66. Vikas Verma, Alex Lamb, Christopher Beckham, Amir Najafi, Ioannis Mitliagkas, David Lopez-Paz, and Yoshua Bengio. 2019. Manifold Mixup: Better rep- resentations by interpolating hidden states. In Pro- ceedings of the 36th International Conference on Machine Learning, volume 97 of ICML'2019, pages 6438-6447, Long Beach, California, USA. PMLR.
  67. Cunxiang Wang, Xiaoze Liu, Yuanhao Yue, Xiangru Tang, Tianhang Zhang, Jiayang Cheng, Yunzhi Yao, Wenyang Gao, Xuming Hu, Zehan Qi, Yidong Wang, Linyi Yang, Jindong Wang, Xing Xie, Zheng Zhang, and Yue Zhang. 2023a. Survey on factuality in large language models: Knowledge, retrieval and domain- specificity. ArXiv preprint, abs/2310.07521. · arXiv
  68. Xiaosu Wang, Yun Xiong, Beichen Kang, Yao Zhang, Philip S. Yu, and Yangyong Zhu. 2023c. Reduc- ing negative effects of the biases of language mod- els in zero-shot setting. In Proceedings of the Six- teenth ACM International Conference on Web Search and Data Mining, WSDM'2023, page 904-912, New York, NY, USA. Association for Computing Machin- ery. · DOI
  69. Yuxia Wang, Haonan Li, Xudong Han, Preslav Nakov, and Timothy Baldwin. 2024a. Do-Not-Answer: Eval- uating safeguards in LLMs. In Findings of the Asso- ciation for Computational Linguistics: EACL 2024, pages 896-911, St. Julian's, Malta. Association for Computational Linguistics.
  70. Yuxia Wang, Minghan Wang, Muhammad Arslan Man- zoor, Georgi Georgiev, Rocktim Jyoti Das, and Preslav Nakov. 2024b. Factuality of large lan- guage models in the year 2024. arXiv preprint arXiv:2402.02420. · arXiv
  71. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits rea- soning in large language models. In Proceedings of the Annual Conference on Neural Information Pro- cessing, volume 35 of NeurIPS 2022, pages 24824- 24837, New Orleans, LA, USA.
  72. Yuxin Xiao, Paul Pu Liang, Umang Bhatt, Willie Neiswanger, Ruslan Salakhutdinov, and Louis- Philippe Morency. 2022. Uncertainty quantification with pre-trained language models: A large-scale em- pirical analysis. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 7273-7284, Abu Dhabi, United Arab Emirates. As- sociation for Computational Linguistics.
  73. Miao Xiong, Zhiyuan Hu, Xinyang Lu, YIFEI LI, Jie Fu, Junxian He, and Bryan Hooi. 2024. Can LLMs express their uncertainty? An empirical evaluation of confidence elicitation in LLMs. In Proceeding of The Twelfth International Conference on Learning Representations, ICLR'2024, Vienna, Austria.
  74. Hiyori Yoshikawa and Naoaki Okazaki. 2023. Selective- LAMA: Selective prediction for confidence-aware evaluation of language models. In Findings of the As- sociation for Computational Linguistics: EACL 2023, pages 2017-2028, Dubrovnik, Croatia. Association for Computational Linguistics.
  75. Yue Yu, Rongzhi Zhang, Ran Xu, Jieyu Zhang, Jiaming Shen, and Chao Zhang. 2023. Cold-start data selec- tion for better few-shot language model fine-tuning: A prompt-based uncertainty propagation approach. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2499-2521, Toronto, Canada. Association for Computational Linguistics. · DOI
  76. Hongyi Zhang, Moustapha Cissé, Yann N. Dauphin, and David Lopez-Paz. 2018. mixup: Beyond empiri- cal risk minimization. In Proceeding of Sixth Inter- national Conference on Learning Representations, ICLR'2018, Vancouver, BC, Canada.
  77. Jingqing Zhang, Yao Zhao, Mohammad Saleh, and Pe- ter J. Liu. 2020a. PEGASUS: pre-training with ex- tracted gap-sentences for abstractive summarization. In Proceedings of the 37th International Conference on Machine Learning, volume 119 of ICML'2020, pages 11328-11339, Virtual Event. PMLR.
  78. Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher De- wan, Mona Diab, Xian Li, Xi Victoria Lin, et al. 2022. OPT: Open pre-trained transformer language models. ArXiv preprint, abs/2205.01068. · arXiv
  79. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2020b. BERTScore: Evaluating text generation with BERT. In Proceed- ings of Eighth International Conference on Learning Representations, ICLR'2020, Addis Ababa, Ethiopia.
  80. Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, et al. 2023b. Siren's song in the AI ocean: A survey on hallucination in large language models. ArXiv preprint, abs/2309.01219. · arXiv
  81. Yao Zhao, Rishabh Joshi, Tianqi Liu, Misha Khalman, Mohammad Saleh, and Peter J Liu. 2023a. SliC-HF: Sequence likelihood calibration with human feed- back. ArXiv preprint, abs/2305.10425. · arXiv
  82. Yao Zhao, Misha Khalman, Rishabh Joshi, Shashi Narayan, Mohammad Saleh, and Peter J. Liu. 2023b. Calibrating sequence likelihood improves conditional language generation. In Proceedings of the Eleventh International Conference on Learning Representa- tions, ICLR'2024, Kigali, Rwanda.
  83. Yukun Zhao, Lingyong Yan, Weiwei Sun, Guoliang Xing, Chong Meng, Shuaiqiang Wang, Zhicong Cheng, Zhaochun Ren, and Dawei Yin. 2023c. Knowing what LLMs do not know: A simple yet effective self-detection method. ArXiv preprint, abs/2310.17918. · arXiv
  84. Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. 2021. Calibrate before use: Improv- ing few-shot performance of language models. In Proceedings of the 38th International Conference on Machine Learning, volume 139 of ICML'2021, pages 12697-12706, Virtual Event. PMLR.
  85. Chujie Zheng, Hao Zhou, Fandong Meng, Jie Zhou, and Minlie Huang. 2024. Large language models are not robust multiple choice selectors. In Proceedings of the Twelfth International Conference on Learning Representations, ICLR'2024, Vienna, Austria.
  86. Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. 2024. MiniGPT-4: Enhancing vision-language understanding with advanced large language models. In Proceedings of the Twelfth In- ternational Conference on Learning Representations, ICLR'2024, Vienna, Austria.

Cited by

Sorted by citations

Related

Authors
Geng, J., Cai, F., Wang, Y., Koeppl, H., Nakov, P., & Gurevych, I.
Affiliations
Technische Universität Darmstadt (DE)
ORCID
Venue
Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)
Publisher
Association for Computational Linguistics
Published
2024
Pages
6577-6595
DOI
10.18653/v1/2024.naacl-long.366
OpenAlex
W4401042444
Cited by
88
References
0
Citations by year
2026: 54 · 2025: 30 · 2024: 4
FWCI
25.97
Type
conference paper
Language
en
Open access
Licence
cc-by
Locations
  • Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) (publishedVersion, cc-by)
  • TUbilio (Technical University of Darmstadt) (submittedVersion)
Funders
Bundesministerium für Bildung und Forschung
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
Keywords
confidence estimation, calibration, large language models, uncertainty estimation, probabilistic forecasting

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review