CoChat
Sign Up
Cited by 127Ethics and Social Impacts of AISpeech and dialogue systemsLanguage, Discourse, Communication Strategies

In Conversation with Artificial Intelligence: Aligning language Models with Human Values

Open access
Kasirzadeh, Atoosa, Gabriel, Iason·2023·Philosophy & Technology
DOI 10.1007/s13347-023-00606-x
Open this paper in CoChat — free accountOpen PDF

Abstract

Large-scale language technologies are increasingly used in various forms of communication with humans across different contexts. One particular use case for these technologies is conversational agents, which output natural language text in response to prompts and queries. This mode of engagement raises a number of social and ethical questions. For example, what does it mean to align conversational agents with human norms or values? Which norms or values should they be aligned with? And how can this be accomplished? In this paper, we propose a number of steps that help answer these questions. We start by developing a philosophical analysis of the building blocks of linguistic communication between conversational agents and human interlocutors. We then use this analysis to identify and formulate ideal norms of conversation that can govern successful linguistic communication between humans and conversational agents. Furthermore, we explore how these norms can be used to align conversational agents with human values across a range of different discursive domains. We conclude by discussing the practical implications of our proposal for the design of conversational agents that are aligned with these norms and values.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1007/s13347-023-00606-x

Cites

10 of 10 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Large language models can perform performative-like statements that can change aspects of the world.

    “Large language models not only produce declarativelike statements but also can perform performative-like statements that can change aspects of the world (Kasirzadeh and Gabriel 2023).”

    From Epistemic Injustice in Generative AI (2024) · cites this paper as Kasirzadeh and Gabriel 2023

    Section 3.1 defines performatives as utterances that change part of reality and discusses performatives issued by conversational agents. But it says the majority of those utterances do not bring about the change because the agent lacks the necessary authority. The claim omits that important limitation. (LLM, conf=0.85)

    Experimental — accuracy not yet measured

  • Generative models such as large language models can power conversational agents that understand and respond to patient questions and concerns.

    “Generative models such as LLMs can power conversational agents that understand and respond to patient questions and concerns [23].”

    From Generative AI in medical practice: in-depth exploration of privacy and security challenges (2024) · cites this paper as [23]

    The Introduction says large language models underpin conversational agents that respond to prompts and queries. Section 5 gives a therapy agent responding to a patient as an example, but treats contextual understanding as a design goal rather than establishing that such agents understand and address patient questions and concerns generally. Page numbers are unavailable in our copy. (LLM, conf=0.83)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 95

  1. Abid, A., Farooqi, M., & Zou, J. (2021). Persistent anti-muslim bias in large language models. In Proceedings of the 2021 AAAI/ACM conference on AI ethics, and society (pp. 298-306).
  2. Ackerly, B. A. (2000). Political theory and feminist social criticism. New York: Cambridge University Press.
  3. Allan, K. (2013). What is common ground? In Perspectives on linguistic pragmatics (pp. 285-310).
  4. Anderson, E. (2004). Uses of value judgments in science: a general argument, with lessons from a case study of feminist research on divorce. Hypatia, 19(1), 1-24.
  5. Asher, N., & Lascarides, A. (2003). Logics of conversation. Cambridge: Cambridge University Press.
  6. Bakker, M. A., Chadwick, M. J., Sheahan, H. R., Tessler, M. H., Campbell-Gillingham, L., Balaguer, J., & et al, (2022). Fine-tuning language models to find agreement among humans with diverse preferences. arXiv: 2211.15006. · arXiv
  7. Bender, E. M. (2013). Linguistic fundamentals for natural language processing: 100 essentials from morphology and syntax. Synthesis Lectures on Human Language Technologies, 6(3), 1-184.
  8. Boden, M. A. (1998). Creativity and artificial intelligence. Artificial Intelligence, 103(1-2), 347-356.
  9. Bondi, E., Xu, L., Acosta-Navas, D., & Killian, J.A. (2021). Envisioning communities: a participatory approach towards ai for social good. In Proceedings of the 2021 aaai/acm conference on ai, ethics, and society (pp. 425-436).
  10. Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., & et al. (2020). Language models are few-shot learners. arXiv: 2005.14165. · arXiv
  11. Bunge, M. (2012). The correspondence theory of truth. Semiotica, 2012(188), 65-75.
  12. Burleson, B. R., & Kline, S. L. (1979). Habermas' theory of communication: a critical explication. Quarterly Journal of Speech, 65(4), 412-428.
  13. Buss, S. (1999). Appearing respectful: The moral significance of manners. Ethics, 109(4), 795-826.
  14. Calhoun, C. (2000). The virtue of civility. Philosophy & Public Affairs, 29(3), 251-275.
  15. Cameron, D. (1992). Feminism and linguistic theory. Berlin: Springer.
  16. Carston, R. (2008). Linguistic communication and the semantics/pragmatics distinction. Synthese, 165(3), 321-345.
  17. Chierchia, G., & McConnell-Ginet, S. (2000). Meaning and grammar: an introduction to semantics. Boston: MIT press.
  18. Chomsky, N. (1957). Syntactic structures. Berlin: De Gruyter Mouton.
  19. Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., & et al. (2022). Palm: Scaling language modeling with pathways. arXiv: 2204.02311. · arXiv
  20. Dale, R., & Reiter, E. (1995). Computational interpretations of the gricean maxims in the generation of referring expressions. Cognitive science, 19(2), 233-263.
  21. Delvin, J., Chang, M. W., Lee, K., & Toutanova, K. (2019). Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of naacl-hlt (pp. 4171-4186).
  22. Díaz, M., Amironesei, R., Weidinger, L., & Gabriel, I. (2022). Accounting for offensive speech as a practice of resistance. In Proceedings of the sixth workshop on online abuse and harms (woah) (pp. 192-202).
  23. Douglas, H. (2000). Inductive risk and values in science. Philosophy of science, 67(4), 559-579.
  24. Douglas, H. (2009). Science, policy and the value-free ideal. Pittsburgh: University of Pittsburgh Press.
  25. Elliott, K. C., & McKaughan, D. J. (2014). Nonepistemic values and the multiple goals of science. Philosophy of Science, 81(1), 1-21.
  26. Evans, O., Cotton-Barratt, O., Finnveden, L., Bales, A., Balwit, A., & Wills, P. (2021). Truthful ai: Developing and governing ai that does not lie. arXiv: 2110.06674. · arXiv
  27. Fortuna, P., & Nunes, S. (2018). A survey on automatic detection of hate speech in text. ACM Computing Surveys (CSUR), 51(4), 1-30.
  28. Gabriel, I. (2022). Toward a theory of justice for artificial intelligence. Daedalus, 151(2), 218-231.
  29. Gabriel, I., & Ghazavi, V. (2021). The challenge of value alignment: From fairer algorithms to ai safety. arXiv: 2101.06060. · arXiv
  30. Gallie, W. B. (1955). Essentially contested concepts. In Proceedings of the aristotelian society, (Vol. 56 pp. 167-198).
  31. Goodman, N. D., & Lassiter, D. (2015). Probabilistic semantics and pragmatics: Uncertainty in language and thought. In The handbook of contemporary semantic theory. 2nd edn. Hoboken: Wiley-Blackwell.
  32. Grice, H. P. (1968). Utterer's meaning, sentence-meaning, and word-meaning. In Philosophy, language, and artificial intelligence: Resources for processing natural language (pp. 49-66). Dordrecht: Kluwer Academic Publishers.
  33. Grice, H. P. (1981). Presupposition and conversational implicature. In C. Peter (Ed.) Radical pragmatics. New York: Academic Press.
  34. Grice, H. P. (1989). Studies in the way of words. Boston: Harvard University Press.
  35. Haack, S. (1976). The pragmatist theory of truth. The British Journal for the Philosophy of Science, 27(3), 231-249.
  36. Habermas, J. (1984). The theory of communicative action. In Reason and the rationalization of society, (Vol. I p. 1981). Boston: Beacon Press. Translated by T. McCarthy, German.
  37. Habermas, J. (1987). The theory of communicative action. In Lifeworld and System, (Vol. II p. 1981). Boston: Beacon Press. Translated by T. McCarthy, German.
  38. Habermas, J. (2003). Truth and justification, (p. 1999). cambridge: Polity Press.
  39. Habermas, J. (2016). Three normative models of democracy. In Constitutionalism and democracy (pp. 277-286). New York: Routledge.
  40. Heim, I., & Kratzer, A. (1185). Semantics In generative grammar. Oxford: Blackwell Publishing.
  41. Held, D. (2006). Models of democracy. Cambridge: Polity Press.
  42. Henderson, P., Sinha, K., Angelard-Gontier, N., Ke, N. R., Fried, G., Lowe, R., & Pineau, J. (2018). Ethical challenges in data-driven dialogue systems. In Proceedings of the 2018 aaai/acm conference on ai, ethics, and society (pp. 123-129).
  43. Horn, L., & Ward, G. (2008). The handbook of pragmatics Vol. 26. London: Blackwell Publishing.
  44. Kapetanios, E., Tatar, D., & Sacarea, C. (2013). Natural language processing: semantic aspects. Boca Raton: CRC Press.
  45. Kaplan, D. (1979). On the logic of demonstratives. Journal of Philosophical Logic, 8(1), 81-98.
  46. Kasirzadeh, A. (2021). A new role for mathematics in empirical sciences. Philosophy of Science, 88(4), 686-706.
  47. Kasirzadeh, A. (2022). Algorithmic fairness and structural injustice: Insights from feminist political philosophy. In Proceedings of the 2022 aaai/acm conference on ai, ethics, and society (pp. 349-356).
  48. Kasirzadeh, A., & Klein, C. (2021). The ethical gravity thesis: Marrian levels and the persistence of bias in automated decision-making systems. In Proceedings of the 2021 aaai/acm conference on ai, ethics, and society (pp. 618-626).
  49. Keenan, E. O. (1976). The universality of conversational postulates. Language in Society, 5(1), 67-80.
  50. Kirkham, R. L. (1992). Theories of truth: a critical introduction. Boston: MIT Press.
  51. Kiros, R., Salakhutdinov, R., & Zemel, R. (2014). Multimodal neural language models. In International conference on machine learning (pp. 595-603).
  52. Kuhn, T. (1977). Objectivity, value judgment, and theory choice. In The essential tension: Selected studies in scientific tradition and change (pp. 320-39). Chicago: Chicago University Press.
  53. Kurdi, M. Z. (2016). Natural language processing and computational linguistics: speech, morphology and syntax Vol. 1. London: Wiley.
  54. Ladegaard, H. J. (2009). Pragmatic cooperation revisited: Resistance and non-cooperation as a discursive strategy in asymmetrical discourses. Journal of Pragmatics, 41(4), 649-666.
  55. Latour, B. (2007). Reassembling the social: An introduction to actor-network-theory. Oxford: Oup Oxford.
  56. Lee, M. K., Kusbit, D., Kahng, A., Kim, J. T., Yuan, X., Chan, A., & et al. (2019). WEbuildai: Participatory framework for algorithmic governance. Proceedings of the ACM on human-computer interaction, 3(CSCW), 1-35.
  57. Longino, H. E. (1990). Science as social knowledge. Princeton: Princeton University Press.
  58. Longino, H. E. (1996). Cognitive and non-cognitive values in science: Rethinking the dichotomy. In Feminism, science, and the philosophy of science (pp. 39-58). Dordrecht: Springer.
  59. Luhmann, N. (1995). Social systems. Redwood: Stanford University Press.
  60. Marsh, G. P. (2003). Man and nature. Washington: University of Washington Press.
  61. Maulud, D. H., Zeebaree, S. R., Jacksi, K., Sadeeq, M. A. M., & Sharif, K.H. (2021). State of art for semantic analysis of natural language processing. Qubahan Academic Journal, 1(2), 21-28.
  62. McMullin, E. (1982). Values in science. In Psa: Proceedings of the biennial meeting of the philosophy of science association, (Vol. 1982 pp. 3-28).
  63. Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2021). A survey on bias and fairness in machine learning. ACM Computing Surveys (CSUR), 54(6), 1-35.
  64. Merton, R. K. (1942). A note on science and democracy. Journal of Legal & Political Sociology, 1, 115- 126.
  65. Mitchell, S., Potash, E., Barocas, S., D'Amour, A., & Lum, K. (2021). Algorithmic fairness: choices, assumptions, and definitions. Annual Review of Statistics and Its Application, 8, 141-163.
  66. Montague, R. (1938). Pragmatics. In Contemporary philosophy. a survey (pp. 102-122). Florence: La Nuova Italia Editrice.
  67. Morris, C. W. (1938). Foundations of the theory of signs. In International encyclopedia of unified science (pp. 1-59). Chicago: Chicago University Press.
  68. Neff, G. (2016). Talking to bots: Symbiotic agency and the case of tay. International Journal of Communication.
  69. Perez-Marin, D., & Pascual-Nieto, I. (2011). Conversational agents and natural language interaction: Techniques and effective practices: Techniques and effective practices. IGI Global, Illustrated edition.
  70. Popper, K., & Bartley, W. W. III.. (1985). Realism and the aim of science: From the postscript to the logic of scientific discovery. London: Routledge.
  71. Rabinowitz, N., Perbet, F., Song, F., Zhang, C., Eslami, S. A., & Botvinick, M. (2018). Machine theory of mind. In International conference on machine learning (pp. 4218-4227).
  72. Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., & et al. (2021). Learning transferable visual models from natural language supervision. arXiv: 2103.00020. · arXiv
  73. Rae, J. W., Borgeaud, S., Cai, T., Millican, K., Hoffmann, J., Song, F., & et al. (2021). Scaling language models: Methods, analysis & insights from training gopher. arXiv: 2112.11446. · arXiv
  74. Rooney, P. (1992). On values in science: is the epistemic/non-epistemic distinction useful? In Psa: Proceedings of the biennial meeting of the philosophy of science association, (Vol. 1992 pp. 13-22).
  75. Rudner, R. (1953). The scientist qua scientist makes value judgments. Philosophy of Science, 20(1), 1-6.
  76. Salmon, W. C. (2006). Four decades of scientific explanation. Pittsburgh: University of Pittsburgh press.
  77. Sarewitz, D., & Pielke, J.r. R. (1999). Prediction in science and policy. Technology in Society, 21(2), 121-133.
  78. Searle, J. R. (1976). A classification of illocutionary acts. Language in Society, 5(1), 1-23.
  79. Silverstein, M. (1972). Linguistic theory: syntax, semantics, pragmatics. Annual Review of Anthropology, 1(1), 349-382.
  80. Simonton, D. K. (2000). Creativity: cognitive, personal, developmental, and social aspects. American Psychologist, 55(1), 151.
  81. Sperber, D., & Wilson, D. (1986). Relevance: Communication and cognition Vol. 142. Cambridge: Blackwell Publishers Inc.
  82. Stalnaker, R. (2014). Context. Oxford: Oxford University Press.
  83. Tamkin, A., Brundage, M., Clark, J., & Ganguli, D. (2021). Understanding the capabilities, limitations, and societal impact of large language models. arXiv: 2102.02503. · arXiv
  84. Taylor, R., Kardas, M., Cucurull, G., Scialom, T., Hartshorn, A., Saravia, E., & Stojnic, R (2022). Galactica: A large language model for science. arXiv: 2211.09085. · arXiv
  85. Thomas, J. A. (1995). Meaning in interaction: an introduction to pragmatics. London: Routledge.
  86. Trout, J. D. (2002). Scientific explanation and the sense of understanding. Philosophy of Science, 69(2), 212-233.
  87. van Fraassen, B. C. (1980). The scientific image. Oxford: Oxford University Press.
  88. Van Valin, R. D., & LaPolla, R. J. (1997). Syntax: Structure, meaning and function. Cambridge: Cambridge University Press.
  89. Walker, R. C. (2018). The coherence theory of truth. InThe oxford handbook of truth, (pp. 219-237). Oxford: Oxford University Press.
  90. Walzer, M. (1993). Interpretation and social criticism Vol. 1. Cambridge: Harvard University Press.
  91. Waseem, Z. (2016). Are you a racist or am i seeing things? annotator influence on hate speech detection on twitter. In Proceedings of the first workshop on nlp and computational social science (pp. 138-142).
  92. Weidinger, L., Mellor, J., Rauh, M., Griffin, C., Uesato, J., Huang, P. S., & et al. (2021). Ethical and social risks of harm from language models. arXiv: 2112.04359. · arXiv
  93. Welbl, J., Glaese, A., Uesato, J., Dathathri, S., Mellor, J., Hendricks, L. A., & Huang, P.S. (2021). Chal- lenges In detoxifying language models. In Findings of the association for computational linguistics: Emnlp (pp. 2447-2469).
  94. Wen, T. H., Vandyke, D., Mrksic, N., Gasic, M., Rojas-Barahona, L. M., Su, P. H., & Young, S. (2016). A network-based end-to-end trainable task-oriented dialogue system. arXiv: 1604.04562. · arXiv
  95. Wierzbicka, A. (1991). Cross-cultural pragmatics: The semantics of human interaction. In Cross-cultural pragmatics. Berlin: De Gruyter Mouton.

Cited by

Sorted by citations

Authors
Kasirzadeh, A., & Gabriel, I.
Affiliations
  • University of Edinburgh (GB)
  • Google DeepMind (United Kingdom) (GB)
Corresponding author
  • Kasirzadeh, Atoosa
  • Gabriel, Iason
ORCID
Venue
Philosophy & Technology
Publisher
Springer Science and Business Media LLC
Published
2023-04-19
Volume
36
Issue
2
ISSN
  • 2210-5433 (ISSN-L)
  • 2210-5441
DOI
10.1007/s13347-023-00606-x
OpenAlex
W4366390744
Cited by
127
References
10
Citations by year
2026: 26 · 2025: 40 · 2024: 45 · 2023: 15 · 2022: 1
FWCI
31.53
Type
journal article
Language
en
Open access
Licence
Locations
  • Philosophy & Technology (publishedVersion, cc-by)
  • PhilSci-Archive (University of Pittsburgh) (acceptedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • PhilPapers (PhilPapers Foundation) (submittedVersion, cc-by)
  • PhilPapers (PhilPapers Foundation) (submittedVersion)
  • Edinburgh Research Explorer (University of Edinburgh) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Funders
DeepMind
Topics
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
  • Social Sciences › Arts and Humanities › Language and Linguistics › Language, Discourse, Communication Strategies
Keywords
chatbots, large language models, value alignment, linguistic communication, human-agent interaction, AI ethics, agent design

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review