CoChat
Sign Up
PreprintCited by 19Topic ModelingNatural Language Processing TechniquesAdversarial Robustness in Machine Learning

Calibrating LLM-Based Evaluator

Open access
Liu, Yuxuan, Yang, Tianchi, Huang, Shaohan, Zhang, Zihan, Huang, Haizhen, Wei, Furu, Deng, Weiwei, Sun, Feng +1 more·2023·arXiv
DOI 10.48550/arxiv.2309.13308arXiv 2309.13308
Open this paper in CoChat — free accountOpen PDF

Abstract

Recent advancements in large language models (LLMs) on language modeling and emergent capabilities make them a promising reference-free evaluator of natural language generation quality, and a competent alternative to human evaluation. However, hindered by the closed-source or high computational demand to host and tune, there is a lack of practice to further calibrate an off-the-shelf LLM-based evaluator towards better human alignment. In this work, we propose AutoCalibrate, a multi-stage, gradient-free approach to automatically calibrate and align an LLM-based evaluator toward human preference. Instead of explicitly modeling human preferences, we first implicitly encompass them within a set of human labels. Then, an initial set of scoring criteria is drafted by the language model itself, leveraging in-context learning on different few-shot examples. To further calibrate this set of criteria, we select the best performers and re-draft them with self-refinement. Our experiments on multiple text quality evaluation datasets illustrate a significant improvement in correlation with expert evaluation through calibration. Our comprehensive qualitative analysis conveys insightful intuitions and observations on the essence of effective scoring criteria.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2309.13308

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Liu et al. demonstrated using annotated LLM outputs judged on criteria like consistency and relevance as few-shot examples for calibrating LLM-based evaluators.

    “Liu et al. [34] demonstrated using annotated LLM outputs-judged on criteria like consistency and relevance-as 'few-shot examples' for calibrating LLM-based evaluators.”

    From Who Validates the Validators? Aligning LLM-Assisted Evaluation of LLM Outputs with Human Preferences (2024) · cites this paper as [34]

    Section 2.3 describes using human-labeled sample–score pairs as few-shot exemplars to infer scoring criteria for calibrating an LLM-based evaluator. Section 4.1 evaluates criteria including consistency and relevance. The copy does not establish that the annotated outputs used as examples were specifically generated by LLMs, rather than other text-generation systems. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • AutoCalibrate is a method to calibrate LLM evaluators with human preferences that requires large expert-labelled datasets with settled criteria.

    “For instance, AutoCalibrate is a method to calibrate LLM evaluators with human preferences that requires large expert-labelled datasets with settled (i.e., established upfront) criteria [34].”

    From Who Validates the Validators? Aligning LLM-Assisted Evaluation of LLM Outputs with Human Preferences (2024) · cites this paper as [34]

    Sections 2.1 and 2.3 describe AutoCalibrate as aligning LLM evaluators with human expert labels. But the Effect of Few-Shot Example Size section says 8–12 examples suffice for inferring effective criteria, rather than establishing a need for large labelled datasets. Section 2.3 and Criteria Revisiting say the method drafts and refines criteria from examples, rather than requiring criteria settled upfront. Page numbers are unavailable in our copy. (LLM, conf=0.89)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Still being checked.

Research works printed but not linked: 32

  1. Markus Freitag, Ricardo Rei, Nitika Mathur, Chi-kiu Lo, Craig Stewart, Eleftherios Avramidis, Tom Kocmi, George Foster, Alon Lavie, and André FT Martins. 2022. Results of wmt22 metrics shared task: Stop using bleu-neural metrics are better and more robust. In Proceedings of the Sev- enth Conference on Machine Translation (WMT), pages 46-68.
  2. Jinlan Fu, See-Kiong Ng, Zhengbao Jiang, and Pengfei Liu. 2023. Gptscore: Evaluate as you desire. arXiv preprint arXiv:2302.04166. · arXiv
  3. Max Grusky, Mor Naaman, and Yoav Artzi. 2018. Newsroom: A dataset of 1.3 million summaries with diverse extractive strategies. In Proceedings of NAACL-HLT, pages 708-719.
  4. Lishan Huang, Zheng Ye, Jinghui Qin, Liang Lin, and Xiaodan Liang. 2020. Grade: Automatic graph-enhanced coherence metric for evaluating open-domain dialogue systems. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 9230-9240.
  5. Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, An- drea Madotto, and Pascale Fung. 2023. Survey of hallucination in natural language generation. ACM Computing Surveys, 55(12):1-38.
  6. Tom Kocmi and Christian Federmann. 2023. Large language models are state-of-the-art evalu- ators of translation quality. arXiv preprint arXiv:2302.14520. · arXiv
  7. Chin-Yew Lin. 2004. Rouge: A package for auto- matic evaluation of summaries. In Text summa- rization branches out, pages 74-81.
  8. Pengfei Liu, Jinlan Fu, Yang Xiao, Weizhe Yuan, Shuaichen Chang, Junqi Dai, Yixin Liu, Zihui- wen Ye, and Graham Neubig. 2021. Explain- aboard: An explainable leaderboard for nlp. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing: System Demonstrations, pages 280-289.
  9. Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. 2023. G-eval: Nlg evaluation using gpt-4 with better human alignment. arXiv preprint arXiv:2303.16634. · arXiv
  10. Qingyu Lu, Baopu Qiu, Liang Ding, Liping Xie, and Dacheng Tao. 2023. Error analysis prompting en- ables human-like translation evaluation in large language models: A case study on chatgpt. arXiv preprint arXiv:2303.13809. · arXiv
  11. Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yim- ing Yang, et al. 2023. Self-refine: Iterative refinement with self-feedback. arXiv preprint arXiv:2303.17651. · arXiv
  12. Shikib Mehri and Maxine Eskenazi. 2020. Usr: An unsupervised and reference free evaluation metric for dialog generation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 681-707.
  13. Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Wei Koh, Mohit Iyyer, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2023. Factscore: Fine-grained atomic evaluation of fac- tual precision in long form text generation. arXiv preprint arXiv:2305.14251. · arXiv
  14. OpenAI. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774. · arXiv
  15. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. Ad- vances in Neural Information Processing Sys- tems, 35:27730-27744.
  16. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for auto- matic evaluation of machine translation. In Pro- ceedings of the 40th annual meeting of the As- sociation for Computational Linguistics, pages 311-318.
  17. Ricardo Rei, Craig Stewart, Ana C Farinha, and Alon Lavie. 2020. Comet: A neural framework for mt evaluation. In Proceedings of the 2020 Confer- ence on Empirical Methods in Natural Language Processing (EMNLP), pages 2685-2702.
  18. Brian Thompson and Matt Post. 2020. Auto- matic machine translation evaluation in many lan- guages via zero-shot paraphrasing. In Proceed- ings of the 2020 Conference on Empirical Meth- ods in Natural Language Processing (EMNLP), pages 90-121.
  19. Ramakrishna Vedantam, C Lawrence Zitnick, and Devi Parikh. 2015. Cider: Consensus-based image description evaluation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4566-4575.
  20. Alex Wang, Kyunghyun Cho, and Mike Lewis. 2020a. Asking and answering questions to eval- uate the factual consistency of summaries. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5008-5020.
  21. Alex Wang, Kyunghyun Cho, and Mike Lewis. 2020b. Asking and answering questions to eval- uate the factual consistency of summaries. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5008-5020.
  22. Jiaan Wang, Yunlong Liang, Fandong Meng, Haox- iang Shi, Zhixu Li, Jinan Xu, Jianfeng Qu, and Jie Zhou. 2023a. Is chatgpt a good nlg eval- uator? a preliminary study. arXiv preprint arXiv:2303.04048. · arXiv
  23. Peiyi Wang, Lei Li, Liang Chen, Dawei Zhu, Binghuai Lin, Yunbo Cao, Qi Liu, Tianyu Liu, and Zhifang Sui. 2023b. Large language mod- els are not fair evaluators. arXiv preprint arXiv:2305.17926. · arXiv
  24. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Pro- cessing Systems, 35:24824-24837.
  25. Tsung-Hsien Wen, Milica Gasic, Nikola Mrkšić, Pei- Hao Su, David Vandyke, and Steve Young. 2015. Semantically conditioned lstm-based natural lan- guage generation for spoken dialogue systems. In Proceedings of the 2015 Conference on Em- pirical Methods in Natural Language Processing, pages 1711-1721.
  26. Weizhe Yuan, Graham Neubig, and Pengfei Liu. 2021. Bartscore: Evaluating generated text as text generation. Advances in Neural Information Processing Systems, 34:27263-27277.
  27. Tianyi Zhang*, Varsha Kishore*, Felix Wu*, Kilian Q. Weinberger, and Yoav Artzi. 2020. Bertscore: Evaluating text generation with bert. In Interna- tional Conference on Learning Representations.
  28. Wei Zhao, Maxime Peyrard, Fei Liu, Yang Gao, Christian M Meyer, and Steffen Eger. 2019. Moverscore: Text generation evaluating with con- textualized embeddings and earth mover dis- tance. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Process- ing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 563-578.
  29. Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. 2021. Calibrate before use: Im- proving few-shot performance of language mod- els. In International Conference on Machine Learning, pages 12697-12706.
  30. PMLR.
  31. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena. arXiv preprint arXiv:2306.05685. · arXiv
  32. Ming Zhong, Yang Liu, Da Yin, Yuning Mao, Yizhu Jiao, Pengfei Liu, Chenguang Zhu, Heng Ji, and Jiawei Han. 2022. Towards a unified multi- dimensional evaluator for text generation. In Pro- ceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 2023-2038.

Cited by

Sorted by citations

Related

Authors
Liu, Y., Yang, T., Huang, S., Zhang, Z., Huang, H., Wei, F., Deng, W., Sun, F., & Zhang, Q.
Affiliations
  • Peking University (CN)
  • Microsoft (United States) (US)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2023
Pages
2638-2656
DOI
10.48550/arxiv.2309.13308
arXiv
2309.13308
OpenAlex
W7131789684
Cited by
12
References
0
Citations by year
2026: 6 · 2025: 3 · 2024: 2 · 2023: 1
FWCI
3.25
Type
conference paper
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • Language Resources and Evaluation Conference (LREC) (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
Keywords
large language models, LLM-based evaluators, calibration, assessment reliability

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review