CoChat
Sign Up
Cited by 47Natural Language Processing TechniquesSyntax, Semantics, Linguistic VariationTopic Modeling

The Goldilocks of Pragmatic Understanding: Fine-Tuning Strategy Matters for Implicature Resolution by LLMs

Open access
Ruis, Laura, Khan, Akbir, Biderman, Stella, Hooker, Sara, Rocktäschel, Tim, Grefenstette, Edward·2023·Advances in Neural Information Processing Systems 36
DOI 10.52202/075280-0913
Open this paper in CoChat — free accountOpen PDF

Abstract

Despite widespread use of LLMs as conversational agents, evaluations of performance fail to capture a crucial aspect of communication: interpreting language in context -- incorporating its pragmatics. Humans interpret language using beliefs and prior knowledge about the world. For example, we intuitively understand the response "I wore gloves" to the question "Did you leave fingerprints?" as meaning "No". To investigate whether LLMs have the ability to make this type of inference, known as an implicature, we design a simple task and evaluate four categories of widely used state-of-the-art models. We find that, despite only evaluating on utterances that require a binary inference (yes or no), models in three of these categories perform close to random. However, LLMs instruction-tuned at the example-level perform significantly better. These results suggest that certain fine-tuning strategies are far better at inducing pragmatic understanding in models. We present our findings as the starting point for further research into evaluating how LLMs interpret language in context and to drive the development of more pragmatic and useful models of human discourse.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.52202/075280-0913

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 56

  1. T., Kilcher, Y., Rothe, S., Sessa, P. G., and Sestorain, L. (2022). Boosting search engines with interactive agents. Transactions on Machine Learning Research.
  2. American Psychiatric Association, A. P. A. (2013). Diagnostic and statistical manual of mental disorders : DSM-5. American Psychiatric Association Arlington, VA, 5th ed. edition.
  3. Askell, A., Bai, Y., Chen, A., Drain, D., Ganguli, D., Henighan, T., Jones, A., Joseph, N., Mann, B., DasSarma, N., Elhage, N., Hatfield-Dodds, Z., Hernandez, D., Kernion, J., Ndousse, K., Olsson, C., Amodei, D., Brown, T. B., Clark, J., McCandlish, S., Olah, C., and Kaplan, J. (2021). A general language assistant as a laboratory for alignment. CoRR, abs/2112.00861. · arXiv
  4. Austin, J., Odena, A., Nye, M., Bosma, M., Michalewski, H., Dohan, D., Jiang, E., Cai, C., Terry, M., Le, Q., et al. (2021). Program synthesis with large language models. arXiv preprint arXiv:2108.07732. · arXiv
  5. Bach, K. (1999). The myth of conventional implicature. Linguistics and Philosophy, 22(4):327–366.
  6. Bai, Y., Jones, A., Ndousse, K., Askell, A., Chen, A., DasSarma, N., Drain, D., Fort, S., Ganguli, D., Henighan, T., et al. (2022). Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862. · arXiv
  7. Bender, E. M., Gebru, T., McMillan-Major, A., and Shmitchell, S. (2021). On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, pages 610–623.
  8. Biderman, S. and Raff, E. (2022). Fooling moss detection with pretrained language models. arXiv preprint arXiv:2201.07406. · arXiv
  9. Black, S., Biderman, S., Hallahan, E., Anthony, Q., Gao, L., Golding, L., He, H., Leahy, C., McDonell, K., Phang, J., Pieler, M., Prashanth, U. S., Purohit, S., Reynolds, L., Tow, J., Wang, B., and Weinbach, S. (2022). GPT-NeoX-20B: An open-source autoregressive language model. In Proceedings of the ACL Workshop on Challenges & Perspectives in Creating Large Language Models.
  10. Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., and Amodei, D. (2020). Language models are few-shot learners. In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., and Lin, H., editors, Advances in Neural Information Processing Systems, volume 33, pages 1877–1901. Curran Associates, Inc.
  11. Cianflone, A., Feng, Y., Kabbara, J., and Cheung, J. C. K. (2018). Let's do it "again": A first computational approach to detecting adverbial presupposition triggers. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2747–2755, Melbourne, Australia. Association for Computational Linguistics.
  12. Davis, W. (2019). Implicature. In Zalta, E. N., editor, The Stanford Encyclopedia of Philosophy. Metaphysics Research Lab, Stanford University, fall 2019 edition.
  13. Davis, W. A. (1998). Implicature : intention, convention, and principle in the failure of Gricean theory / Wayne A. Davis. Cambridge studies in philosophy. Cambridge University Press, Cambridge England ; New York.
  14. Devlin, J., Chang, M., Lee, K., and Toutanova, K. (2018). BERT: pre-training of deep bidirectional transformers for language understanding. CoRR, abs/1810.04805. · arXiv
  15. Efrat, A. and Levy, O. (2020). The turking test: Can language models understand instructions? CoRR, abs/2010.11982. · arXiv
  16. Gehman, S., Gururangan, S., Sap, M., Choi, Y., and Smith, N. A. (2020). RealToxicityPrompts: Evaluating neural toxic degeneration in language models. In Findings of the Association for Com- putational Linguistics: EMNLP 2020, pages 3356–3369, Online. Association for Computational Linguistics.
  17. George, E. J. and Mamidi, R. (2020). Conversational implicatures in english dialogue: Annotated dataset. Procedia Computer Science, 171:2316–2323. https://doi.org/10.1016/j.procs.2020.04.251. · DOI
  18. Greene, S. and Resnik, P. (2009). More than words: Syntactic packaging and implicit sentiment. In Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 503–511, Boulder, Colorado. Association for Computational Linguistics.
  19. Grice, H. P. (1975). Logic and conversation. In Cole, P. and Morgan, J. L., editors, Syntax and Semantics: Vol. 3: Speech Acts, pages 41–58. Academic Press, New York.
  20. Hosseini, M. J., Radlinski, F., Pareti, S., and Louis, A. (2023). Resolving indirect referring expressions for entity selection. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12313–12335, Toronto, Canada. Association for Computational Linguistics.
  21. Huang, W., Abbeel, P., Pathak, D., and Mordatch, I. (2022). Language models as zero-shot planners: Extracting actionable knowledge for embodied agents. arXiv preprint arXiv:2201.07207. · arXiv
  22. Huang, Y. (2017). The Oxford Handbook of Pragmatics. Oxford handbooks in linguistics. Oxford University Press.
  23. Jeretic, P., Warstadt, A., Bhooshan, S., and Williams, A. (2020). Are natural language inference models IMPPRESsive? Learning IMPlicature and PRESupposition. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8690–8705, Online. Association for Computational Linguistics.
  24. Katsos, N., Roqueta, C. A., Estevan, R. A. C., and Cummins, C. (2011). Are children with specific language impairment competent with the pragmatics and logic of quantification? Cognition, 119(1):43–57.
  25. Kenton, Z., Everitt, T., Weidinger, L., Gabriel, I., Mikulik, V., and Irving, G. (2021). Alignment of language agents. CoRR, abs/2103.14659. · arXiv
  26. Kiela, D., Bartolo, M., Nie, Y., Kaushik, D., Geiger, A., Wu, Z., Vidgen, B., Prasad, G., Singh, A., Ringshia, P., Ma, Z., Thrush, T., Riedel, S., Waseem, Z., Stenetorp, P., Jia, R., Bansal, M., Potts, C., and Williams, A. (2021). Dynabench: Rethinking benchmarking in nlp. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguis- tics: Human Language Technologies, pages 4110–4124, Online. Association for Computational Linguistics.
  27. Kim, N., Pavlick, E., Karagol Ayan, B., and Ramachandran, D. (2021). Which linguist invented the lightbulb? presupposition verification for question-answering. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 3932–3945, Online. Association for Computational Linguistics.
  28. Kim, S. Y., Park, H., Shin, K., and Kim, K.-M. (2022). Ask me what you need: Product retrieval using knowledge from gpt-3. arXiv preprint arXiv:2207.02516. · arXiv
  29. Lepore, E. and Stone, M. (2014). Imagination and Convention: Distinguishing Grammar and Inference in Language. Oxford University Press.
  30. Levinson, S. C. (1983). Pragmatics. Cambridge University Press, Cambridge, U.K.
  31. Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-t., Rocktäschel, T., et al. (2020). Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Processing Systems, 33:9459–9474.
  32. Li, E., Schuster, S., and Degen, J. (2021). Predicting scalar inferences from "or" to "not both" using neural sentence encoders. In Proceedings of the Society for Computation in Linguistics 2021, pages 446–450, Online. Association for Computational Linguistics.
  33. Lin, S., Hilton, J., and Evans, O. (2022). TruthfulQA: Measuring how models mimic human false- hoods. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3214–3252, Dublin, Ireland. Association for Computational Lin- guistics.
  34. Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov, V. (2019). Roberta: A robustly optimized BERT pretraining approach. CoRR, abs/1907.11692. · arXiv
  35. Louis, A., Roth, D., and Radlinski, F. (2020). "I'd rather just go to bed": Understanding indirect answers. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7411–7425, Online. Association for Computational Linguistics.
  36. Lu, Y., Bartolo, M., Moore, A., Riedel, S., and Stenetorp, P. (2022). Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity. In ACL.
  37. Ng, N., Yee, K., Baevski, A., Ott, M., Auli, M., and Edunov, S. (2019). Facebook FAIR's WMT19 news translation task submission. In Proceedings of the Fourth Conference on Machine Trans- lation (Volume 2: Shared Task Papers, Day 1), pages 314–319, Florence, Italy. Association for Computational Linguistics.
  38. Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., Schulman, J., Hilton, J., Kelton, F., Miller, L. E., Simens, M., Askell, A., Welinder, P., Christiano, P. F., Leike, J., and Lowe, R. J. (2022). Training language models to follow instructions with human feedback. ArXiv, abs/2203.02155. · arXiv
  39. Parrish, A., Schuster, S., Warstadt, A., Agha, O., Lee, S.-H., Zhao, Z., Bowman, S. R., and Linzen, T. (2021). NOPE: A corpus of naturally-occurring presuppositions in English. In Proceedings of the 25th Conference on Computational Natural Language Learning, pages 349–366, Online. Association for Computational Linguistics.
  40. Patel, R. and Pavlick, E. (2021). "was it "stated" or was it "claimed"?: How linguistic bias affects generative language models. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 10080–10095, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  41. Potts, C. (2005). The Logic of Conventional Implicatures. Oxford University Press UK.
  42. Potts, C. (2006). Conversational implicatures via general pragmatic pressures. In Washio, T., Satoh, K., Takeda, H., and Inokuchi, A., editors, New Frontiers in Artificial Intelligence, pages 205–218, Berlin, Heidelberg. Springer Berlin Heidelberg.
  43. Recasens, M., Danescu-Niculescu-Mizil, C., and Jurafsky, D. (2013). Linguistic models for analyzing and detecting biased language. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1650–1659, Sofia, Bulgaria. Association for Computational Linguistics.
  44. Reynolds, L. and McDonell, K. (2021b). Prompt programming for large language models: Beyond the few-shot paradigm. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems, CHI EA '21, New York, NY, USA. Association for Computing Machinery.
  45. Sanh, V., Webson, A., Raffel, C., Bach, S., Sutawika, L., Alyafeai, Z., Chaffin, A., Stiegler, A., Raja, A., Dey, M., Bari, M. S., Xu, C., Thakker, U., Sharma, S. S., Szczechla, E., Kim, T., Chhablani, G., Nayak, N., Datta, D., Chang, J., Jiang, M. T.-J., Wang, H., Manica, M., Shen, S., Yong, Z. X., Pandey, H., Bawden, R., Wang, T., Neeraj, T., Rozen, J., Sharma, A., Santilli, A., Fevry, T., Fries, J. A., Teehan, R., Scao, T. L., Biderman, S., Gao, L., Wolf, T., and Rush, A. M. (2022). Multitask prompted training enables zero-shot task generalization. In International Conference on Learning Representations.
  46. Schuster, S., Chen, Y., and Degen, J. (2020). Harnessing the linguistic signal to predict scalar inferences. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5387–5403, Online. Association for Computational Linguistics.
  47. Sperber, D. and Wilson, D. (1986). Relevance: Communication and Cognition. Language and thought series. Harvard University Press.
  48. Tan, Z., Zhang, X., Wang, S., and Liu, Y. (2021). Msp: Multi-stage prompting for making pre-trained language models better translators. arXiv preprint arXiv:2110.06609. · arXiv
  49. Kuzmina, V., Fenton, J., Cohen, A., Bernstein, R., Kurzweil, R., Aguera-Arcas, B., Cui, C., Croak, M., Chi, E. H., and Le, Q. (2022). Lamda: Language models for dialog applications. CoRR, abs/2201.08239. · arXiv
  50. Volden, J. (2017). Autism Spectrum Disorder, pages 59–83. Springer International Publishing, Cham.
  51. Webson, A. and Pavlick, E. (2021). Do prompt-based models really understand the meaning of their prompts? arXiv preprint arXiv:2109.01247. · arXiv
  52. Wei, J., Bosma, M., Zhao, V., Guu, K., Yu, A. W., Lester, B., Du, N., Dai, A. M., and Le, Q. V. (2022). Finetuned language models are zero-shot learners. In International Conference on Learning Representations.
  53. Wittgenstein, L. (1921). Tractatus logico-philosophicus. London: Routledge, 1981.
  54. Wittgenstein, L. (1953). Philosophical Investigations. Basil Blackwell, Oxford.
  55. Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., Davison, J., Shleifer, S., von Platen, P., Ma, C., Jernite, Y., Plu, J., Xu, C., Le Scao, T., Gugger, S., Drame, M., Lhoest, Q., and Rush, A. (2020). Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38–45, Online. Association for Computational Linguistics.
  56. Zheng, Z., Qiu, S., Fan, L., Zhu, Y., and Zhu, S.-C. (2021). GRICE: A grammar-based dataset for recovering implicature and conversational rEasoning. In Findings of the Association for Computa- tional Linguistics: ACL-IJCNLP 2021, pages 2074–2085, Online. Association for Computational Linguistics. https://github.com/kingoflolz/mesh-transformer-jax

Cited by

Sorted by citations

Authors
Ruis, L., Khan, A., Biderman, S., Hooker, S., Rocktäschel, T., & Grefenstette, E.
ORCID
Venue
Advances in Neural Information Processing Systems 36
Publisher
Neural Information Processing Systems Foundation, Inc. (NeurIPS)
Published
2023
Pages
20827-20905
ISSN
1049-5258 (ISSN-L)
DOI
10.52202/075280-0913
OpenAlex
W7133187004
Cited by
47
References
0
Citations by year
2026: 7 · 2025: 17 · 2024: 13 · 2023: 10
FWCI
7.46
Type
conference paper
Language
en
Open access
Licence
cc-by
Locations
  • neural information processing systems (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • UCL Discovery (University College London) (submittedVersion, cc-by)
  • arXiv (Cornell University)
Funders
Engineering and Physical Sciences Research Council (EP/S021566/1)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
  • Social Sciences › Arts and Humanities › Language and Linguistics › Syntax, Semantics, Linguistic Variation
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
Keywords
pragmatic understanding, large language models, fine-tuning strategies, pragmatic reasoning, conversational implicature

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review