CoChat
Sign Up
PreprintCited by 4

The DOI matches its DataCite record; 1 of the 1 references it cites were found in a registry or an index.

The ICL Consistency Test

Weber, Lucas, Bruni, Elia, Hupkes, Dieuwke·2023·arXiv
DOI 10.48550/arxiv.2312.04945arXiv 2312.04945
Open this paper in CoChat — free account

Abstract

Just like the previous generation of task-tuned models, large language models (LLMs) that are adapted to tasks via prompt-based methods like in-context-learning (ICL) perform well in some setups but not in others. This lack of consistency in prompt-based learning hints at a lack of robust generalisation. We here introduce the ICL consistency test -- a contribution to the GenBench collaborative benchmark task (CBT) -- which evaluates how consistent a model makes predictions across many different setups while using the same data. The test is based on different established natural language inference tasks. We provide preprocessed data constituting 96 different 'setups' and a metric that estimates model consistency across these setups. The metric is provided on a fine-grained level to understand what properties of a setup render predictions unstable and on an aggregated level to compare overall model consistency. We conduct an empirical analysis of eight state-of-the-art models, and our consistency metric reveals how all tested LLMs lack robust generalisation.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2312.04945

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Figures

Image 1
CC-BY · From the PDF
Figure 2 The following performance estimates help to put the results of our consistency metric into perspective: (a) A kernel density estimation of the accuracy scores of all eight models across all of our setups. We see how the spread of accuracy scores is high, meaning that the accuracy of a model is very different depending on the setup in which it is evaluated. (b) The β -values of the main effects of each individual factor across many different runs. The values can be directly interpreted as ' expected accuracy gain/loss ' when a factor is present compared to when it is absent.
Figure 2 The following performance estimates help to put the results of our consistency metric into perspective: (a) A kernel density estimation of the accuracy scores of all eight models across all of our setups. We see how the spread of accuracy scores is high, meaning that the accuracy of a model is very different depending on the setup in which it is evaluated. (b) The β -values of the main effects of each individual factor across many different runs. The values can be directly interpreted as ' expected accuracy gain/loss ' when a factor is present compared to when it is absent.CC-BY · From the PDF
Figure 3 The consistency values comparing predictions when a specific factor λ is present or not. A value of 0 indicates a complete change of predictions while a value of 1 indicates perfect agreement. Hence, a low value indicates that a model is not robust to a change in a specific λ .
Figure 3 The consistency values comparing predictions when a specific factor λ is present or not. A value of 0 indicates a complete change of predictions while a value of 1 indicates perfect agreement. Hence, a low value indicates that a model is not robust to a change in a specific λ .CC-BY · From the PDF
Figure 4 Average kappa per model. A high value reveals a model's robustness to setup changes.
Figure 4 Average kappa per model. A high value reveals a model's robustness to setup changes.CC-BY · From the PDF
Image 5
CC-BY · From the PDF
Image 6
CC-BY · From the PDF
Figure 7 An illustration of the meaning of every digit in a setup_ID: A factor is either absent (0), present (1) or irrelevant for the setup (2).
Figure 7 An illustration of the meaning of every digit in a setup_ID: A factor is either absent (0), present (1) or irrelevant for the setup (2).CC-BY · From the PDF
Figure 8 The diversity of model predictions across the whole dataset measured through entropy. A value close to 0 indicates that a model is strongly biased to always predict the same label. A value close to the entropy of the true label distribution H ( Y ) indicates that the model has no bias towards any specific label.
Figure 8 The diversity of model predictions across the whole dataset measured through entropy. A value close to 0 indicates that a model is strongly biased to always predict the same label. A value close to the entropy of the true label distribution H ( Y ) indicates that the model has no bias towards any specific label.CC-BY · From the PDF

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 1 references

Research works printed but not linked: 27

  1. Ekin Akyürek, Dale Schuurmans, Jacob Andreas, Tengyu Ma, and Denny Zhou. 2022. What learn- ing algorithm is in-context learning? investigations with linear models. In The Eleventh International Conference on Learning Representations.
  2. Anas Awadalla, Mitchell Wortsman, Gabriel Ilharco, Sewon Min, Ian Magnusson, Hannaneh Hajishirzi, and Ludwig Schmidt. 2022. Exploring the landscape of distributional robustness for question answering models. In Findings of the Association for Computa- tional Linguistics: EMNLP 2022, pages 5971-5987, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  3. Stephen Bach, Victor Sanh, Zheng Xin Yong, Albert Webson, Colin Raffel, Nihal V. Nayak, Abheesht Sharma, Taewoon Kim, M Saiful Bari, Thibault Fevry, Zaid Alyafeai, Manan Dey, Andrea Santilli, Zhiqing Sun, Srulik Ben-david, Canwen Xu, Gun- jan Chhablani, Han Wang, Jason Fries, Maged Al- shaibani, Shanya Sharma, Urmish Thakker, Khalid Almubarak, Xiangru Tang, Dragomir Radev, Mike Tian-jian Jiang, and Alexander Rush. 2022. Prompt- Source: An integrated development environment and repository for natural language prompts. In Proceed- ings of the 60th Annual Meeting of the Association for Computational Linguistics: System Demonstra- tions, pages 93-104, Dublin, Ireland. Association for Computational Linguistics. · DOI · Paper page
  4. Yejin Bang, Samuel Cahyawijaya, Nayeon Lee, Wen- liang Dai, Dan Su, Bryan Wilie, Holy Lovenia, Ziwei Ji, Tiezheng Yu, Willy Chung, et al. 2023. A multi- task, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity. ArXiv preprint, abs/2302.04023. · arXiv
  5. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners. In Ad- vances in Neural Information Processing Systems 33: Annual Conference on Neural Information Process- ing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual.
  6. Jacob Cohen. 1960. A coefficient of agreement for nominal scales. Educational and psychological mea- surement, 20(1):37-46.
  7. Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. 2022. Llm. int8 (): 8-bit matrix multi- plication for transformers at scale. ArXiv preprint, abs/2208.07339. · arXiv
  8. Shivam Garg, Dimitris Tsipras, Percy S Liang, and Gre- gory Valiant. 2022. What can transformers learn in-context? a case study of simple function classes. Advances in Neural Information Processing Systems, 35:30583-30598.
  9. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. Lora: Low-rank adaptation of large language models. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net.
  10. D. Hupkes, M. Giulianelli, V. Dankers, et al. 2023. A taxonomy and review of generalization research in nlp. Nature Machine Intelligence, 5:1161-1174. · DOI
  11. Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Ku- mar, et al. 2022. Holistic evaluation of language models. ArXiv preprint, abs/2211.09110. · arXiv
  12. Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettle- moyer. 2022. Rethinking the role of demonstrations: What makes in-context learning work? In Proceed- ings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 11048-11064, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  13. Swaroop Mishra, Daniel Khashabi, Chitta Baral, Yejin Choi, and Hannaneh Hajishirzi. 2022. Reframing instructional prompts to GPTk's language. In Find- ings of the Association for Computational Linguistics: ACL 2022, pages 589-612, Dublin, Ireland. Associa- tion for Computational Linguistics. · DOI
  14. Yixin Nie, Adina Williams, Emily Dinan, Mohit Bansal, Jason Weston, and Douwe Kiela. 2020. Adversarial NLI: A new benchmark for natural language under- standing. In Proceedings of the 58th Annual Meet- ing of the Association for Computational Linguistics, pages 4885-4901, Online. Association for Computa- tional Linguistics. · DOI · Paper page
  15. Xenia Ohmer, Elia Bruni, and Dieuwke Hupkes. 2023. Evaluating task understanding through multilingual consistency: A chatgpt case study. ArXiv preprint, abs/2305.11662. · arXiv
  16. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9.
  17. Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. SQuAD: 100,000+ questions for machine comprehension of text. In Proceedings of the 2016 Conference on Empirical Methods in Natu- ral Language Processing, pages 2383-2392, Austin, Texas. Association for Computational Linguistics. · DOI · Paper page
  18. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and effi- cient foundation language models. ArXiv preprint, abs/2302.13971. · arXiv
  19. Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Al- isa Liu, Noah A Smith, Daniel Khashabi, and Han- naneh Hajishirzi. 2022. Self-instruct: Aligning lan- guage model with self generated instructions. ArXiv preprint, abs/2212.10560. · arXiv
  20. Zhiguo Wang, Wael Hamza, and Radu Florian. 2017. Bilateral multi-perspective matching for natural lan- guage sentences. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelli- gence, IJCAI 2017, Melbourne, Australia, August 19-25, 2017, pages 4144-4150. ijcai.org. · DOI
  21. Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, An- drew M. Dai, and Quoc V. Le. 2022. Finetuned language models are zero-shot learners. In The Tenth International Conference on Learning Representa- tions, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net.
  22. Jerry Wei, Jason Wei, Yi Tay, Dustin Tran, Albert Webson, Yifeng Lu, Xinyun Chen, Hanxiao Liu, Da Huang, Denny Zhou, et al. 2023. Larger language models do in-context learning differently. ArXiv preprint, abs/2303.03846. · arXiv
  23. Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. A broad-coverage challenge corpus for sen- tence understanding through inference. In Proceed- ings of the 2018 Conference of the North American Chapter of the Association for Computational Lin- guistics: Human Language Technologies, Volume 1 (Long Papers), pages 1112-1122, New Orleans, Louisiana. Association for Computational Linguis- tics. · DOI · Paper page
  24. Kang Min Yoo, Junyeob Kim, Hyuhng Joon Kim, Hyun- soo Cho, Hwiyeol Jo, Sang-Woo Lee, Sang-goo Lee, and Taeuk Kim. 2022. Ground-truth labels matter: A deeper look into input-label demonstrations. In Pro- ceedings of the 2022 Conference on Empirical Meth- ods in Natural Language Processing, pages 2422- 2437, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  25. Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. 2021. Calibrate before use: Improv- ing few-shot performance of language models. In Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Vir- tual Event, volume 139 of Proceedings of Machine Learning Research, pages 12697-12706.
  26. PMLR.
  27. Ruiqi Zhong, Kristy Lee, Zheng Zhang, and Dan Klein. 2021. Adapting language models for zero-shot learn- ing by meta-tuning on dataset and prompt collections. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 2856-2878, Punta Cana, Dominican Republic. Association for Compu- tational Linguistics. · DOI

Cited by

Sorted by citations

See all 4

Metadata

Authors
Weber, L., Bruni, E., & Hupkes, D.
Venue
arXiv
Publisher
arXiv
Published
2023
DOI
10.48550/arxiv.2312.04945
arXiv
2312.04945
References
1
Type
Preprint
Licence
creativecommons.org/licenses/by/4.0/legalcode

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: The ICL Consistency Test. https://app.cochat.ai/papers/10.48550/arxiv.2312.04945
The paper
Weber, L., Bruni, E., & Hupkes, D. (2023). The ICL Consistency Test. arXiv. https://doi.org/10.48550/arxiv.2312.04945