CoChat
Sign Up
Cited by 83Speech and dialogue systemsSocial Robot Interaction and HRIAI in Service Interactions

Mirages. On Anthropomorphism in Dialogue Systems

Open access
Abercrombie, Gavin, Curry, Amanda, Dinkar, Tanvi, Rieser, Verena, Talat, Zeerak·2023·Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing
DOI 10.18653/v1/2023.emnlp-main.290
Open this paper in CoChat — free accountOpen PDF

Abstract

Automated dialogue or conversational systems are anthropomorphised by developers and personified by users.While a degree of anthropomorphism may be inevitable due to the choice of medium, conscious and unconscious design choices can guide users to personify such systems to varying degrees.Encouraging users to relate to automated systems as if they were human can lead to high risk scenarios caused by over-reliance on their outputs.As a result, natural language processing researchers have investigated the factors that induce personification and develop resources to mitigate such effects.However, these efforts are fragmented, and many aspects of anthropomorphism have yet to be explored.In this paper, we discuss the linguistic factors that contribute to the anthropomorphism of dialogue systems and the harms that can arise, including reinforcing gender stereotypes and notions of acceptable language.We recommend that future efforts towards developing dialogue systems take particular care in their design, development, release, and description; and attend to the many linguistic cues that can elicit personification by users.

AI snapshot

Written from the paper’s full text.

TL;DR

This position paper argues that gratuitous anthropomorphic features in dialogue systems create “mirages of humanity” that can mislead users and cause harms such as misplaced trust, gender stereotyping, and language policing. It maps psychological and linguistic contributors to anthropomorphism and recommends that designers, developers, and researchers reassess anthropomorphic design, release, and description.

Key findings

  • Anthropomorphism is driven by human factors—mindless cue-based processing, effectance, and sociality—and by agent factors such as interactivity, language use, humanlike roles, personas, names, and preferences.
  • Linguistic cues that can increase personification include voice qualities, prosody, disfluencies, accent, self-referential pronouns, expressions of thought/sentience, pseudo-empathy, claims of humanlike activities, phatic expressions, hedging, personas, subservience, and anthropomorphic terminology.
  • The paper cites prior findings that crowd-workers rated 20–30% of utterances from nine different systems as machine-impossible, with only MultiWoz not appearing anthropomorphic; Abercrombie et al. (2021) found systems prone to claiming uniquely human abilities in a quarter of responses; and Gros et al. (2021) found that neither end-to-end research systems nor commercial voice assistants could answer human/machine identity queries truthfully the majority of the time.
  • Consequences include unjustified trust and deception, especially for vulnerable populations; reinforcement of gender stereotypes through gendered or genderless-but-gendered systems; and erasure of marginalised communities through standardisation, accent prestige, and code-switching pressures.

Methods

  • Design: normative position paper making an argument against gratuitous anthropomorphic features, grounded in psychology, linguistics, and human-computer interaction.
  • Data or sample: Not stated in the paper; it synthesizes prior literature, examples, and cited studies rather than presenting a new dataset or experiment.
  • Main analysis: conceptual taxonomy and discussion of psychological mechanisms, linguistic factors, consequences, and recommendations for design, development, release, and description.

Limitations

  • The paper states its list of linguistic factors is not exhaustive, anthropomorphism varies person-to-person, and people may react differently to different design aspects.
  • The paper states it focuses primarily on English-language dialogue systems due to the authors’ Indo-European language backgrounds and English dominance in NLP, while noting other languages have animacy and gender features that could influence personification.
  • Evident from the text: it is a position paper without systematic quantification or empirical validation, and the authors note that the field lacks an approach that quantifies linguistic factors and relates them to possible harms and risks.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.18653/v1/2023.emnlp-main.290

Cites

88 of 88 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 26 cited sources checked · 0 supported · 0 flagged

  • Premisecitation

    In an analysis of reviews of commercial voice assistants, the big five personality traits do not adequately describe user expectations of systems' personalities.

    “In an analysis of reviews of commercial voice assistants, Völkel et al. (2020) find that the big five personality traits (De Raad, 2000) do not adequately describe user expectations of systems' 'personalities'.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Völkel et al., 2020

      Not checked in the free run.

  • Interpretationreasoning

    Users who are members of vulnerable populations, such as the very young, the elderly, or people with illnesses or disabilities, or who lack subject matter expertise, may be even more susceptible to trust issues.

    “This may be even more problematic when users are members of vulnerable populations, such as the very young, the elderly, or people with illnesses or disabilities, or simply lack subject matter expertise.”
    Quote in the paperSubstantial claim
  • Interpretationreasoning

    First person pronoun use may contribute to anthropomorphism, either by design or due to human-produced training data.

    “First person pronoun use may therefore contribute to anthropomorphism, either by design or due to their human-produced training data, for symbolic and data driven dialogue systems, respectively.”
    Quote in the paperSubstantial claim
  • Premisecitation

    There has already been a case of a person committing suicide allegedly as a consequence of interaction with a high-performing dialogue system.

    “While high performing dialogue systems have only recently been introduced to the public domain, there has already been a case of a person committing suicide, allegedly as a consequence of interaction with such a system(Lovens, 2023).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Lovens, 2023

      Not checked in the free run.

  • Premisecitation

    The tendency of people who know how technologies are built to use anthropomorphic language represents a significant failure in scientific communication and engagement.

    “The tendency of people who do know how technologies are built to use anthropomorphic language represents, according to Salles et al. (2020, p. 93), 'a significant failure in scientific communication and engagement'.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Salles et al., 2020

      Not checked in the free run.

  • May combine two claims. 11
  • Classified as probably not a substantial claim. 19

  • May combine two claims.

    Premisecitation

    Automated dialogue systems are anthropomorphised by developers and personified by users.

    “Automated dialogue or conversational systems are anthropomorphised by developers and personified by users.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites UNESCO, 2019

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given

    Conscious and unconscious design choices can guide users to personify dialogue systems to varying degrees.

    “While a degree of anthropomorphism may be inevitable due to the choice of medium, conscious and unconscious design choices can guide users to personify such systems to varying degrees.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given

    Whether or not it is a part of a dialogue system's deliberate design to use specific linguistic forms, listeners will invariably perceive communicative intent.

    “Whether or not it is a part of a dialogue system's deliberate design to use specific linguistic forms (e.g. the cues outlined in §3), listeners will invariably perceive communicative intent.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationreasoning

    The personification of dialogue systems could exacerbate the erasure of marginalised communities, e.g., through limiting diverse language data.

    “The personification of such systems could exacerbate the erasure of marginalised communities, e.g. through limiting diverse language data.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationcitation

    The anthropomorphism of dialogue systems can induce a number of adverse societal effects, e.g., they can generate unreliable information and reinforce social roles, language norms, and stereotypes.

    “The anthropomorphism of dialogue systems can induce a number of adverse societal effects, e.g. they can generate unreliable information and reinforce social roles, language norms, and stereotypes.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    The longstanding goal of anthropomorphic systems is in part responsible for the small body of work that does not seek anthropomorphism, despite growing awareness of its harms.

    “It is in part due to this longstanding goal of anthropomorphic systems that there only exists a small body of work that does not seek anthropomorphism, despite growing awareness of its harms.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Traditionally, the goal of Artificial Intelligence research has been to create systems that would exhibit intelligence indistinguishable from humans.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Premisecitation

    There exist significant motivations to continue to create hype around products and research.

    “while there exist significant motivations to continue to create hype around products and research (Hunger, 2023)”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Hunger, 2023

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    As people conform to language variants that are recognised by dialogue systems, they also conform to whiteness and the continued erasure of marginalised communities.

    “Consequently, as people conform to language variants that are recognised by dialogue systems, they also conform to whiteness and the continued erasure of marginalised communities.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    In reviews of commercial voice assistants, the only consistently desired personality trait was agreeableness, because users expect prompt and reliable responses to queries.

    “The only consistently desired trait was agreeableness, as users expect prompt and reliable responses to queries (Völkel et al., 2020).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Völkel et al., 2020

      Not checked in the free run.

  • May combine two claims.

    Interpretationnone given

    While computing systems do not have thoughts, the language they output can give the appearance of thought by indicating that they hold opinions and morals or sentience.

    “While computing systems do not have thoughts, the language that they output can give the appearance of thought by indicating that they hold opinions and morals or sentience.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Computing systems do not have thoughts.

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

    Interpretationnone given

    Efforts to investigate and mitigate anthropomorphism are fragmented, and many aspects of anthropomorphism have yet to be explored.

    “However, these efforts are fragmented, and many aspects of anthropomorphism have yet to be explored.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Each persona in Personachat consists of a list of statements such as 'I am a vegetarian. I like swimming. My father used to work for Ford. My favorite band is Maroon5. I got a new job last month, which is about advertising design.'

    “For example, each persona in Personachat(Zhang et al., 2018) consists of a list of statements such as 'I am a vegetarian.I like swimming.My father used to work for Ford.My favorite band is Maroon5.I got a new job last month, which is about advertising design.'”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Zhang et al., 2018

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    It is only within a given context that one can make judgement on whether anthropomorphism is a concern.

    “Indeed, it is only within a given context that one can make judgement on whether anthropomorphism is a concern.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given

    Dialogue systems often produce responses that blur the lines between animate and inanimate things, for example by expressing preferences or opinions.

    “However, dialogue systems often produce responses that blur these lines, for example, by expressing preferences or opinions.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Dialogue systems are not capable of experiencing empathy, and are unable to correctly recognise emotions.

    “However, dialogue systems are not capable of experiencing empathy, and are unable to correctly recognise emotions (Véliz, 2021).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Véliz, 2021

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Public confusion about current language technologies extends from children to adults, including journalists, policymakers, and business people, who are convinced either of humanity's imminent enslavement to super-intelligent artificial agents or that such systems provide super-human solutions to the world's problems.

    “This confusion extends from children (Andries and Robertson, 2023) to adults (including some journalists, policymakers, and business people) who are convinced, on the one hand, of humanity's imminent enslavement to 'super-intelligent artificial agents' (to the neglect of actual harms already propagated by technological systems), or, on the other, that such systems provide super-human solutions to”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites Andries and Robertson, 2023

      Not checked in the free run.

    • cites Hunger, 2023

      Not checked in the free run.

    • cites Klein, 2023

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The increased realism of voice is a primary factor contributing to anthropomorphism of dialogue assistants.

    “Indeed, West et al. (2019) argue that the increased realism of voice is a primary factor contributing to anthropomorphism of dialogue assistants.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites West et al., 2019

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    By carefully considering how a system may be anthropomorphised and deliberately selecting the attributes that are appropriate for each context, developers and designers can avoid creating mirages of humanity.

    “By carefully considering how a system may be anthropomorphised and deliberately selecting the attributes that are appropriate for each context, developers and designers can avoid falling into the trap of creating mirages of humanity.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationreasoning

    The studies that do not seek anthropomorphism exist in isolation, and the field lacks an approach that quantifies linguistic factors and relates them to possible harms and risks.

    “Furthermore, these studies exist in isolation, and the taxonomy introduced in this paper highlights that we lack an approach that quantifies linguistic factors and relates them to possible harms and risks.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationnone given

    Anthropomorphism varies from person to person, and people may react differently to different aspects of a system's design.

    “As we describe in section 2, anthropomorphism varies from person-to-person and people may react differently to different aspects of a system's design.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Other languages have features such as grammatical ways of denoting animacy and gender that could influence users' personification of systems.

    “However, it should be noted that other languages have features such as grammatical ways of denoting animacy (Yamamoto, 1999) and gender that could influence users personification of systems, and which developers should consider if they wish to limit anthropomorphism.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Yamamoto, 1999

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Work in this area has historically been cast as imbuing agents with beliefs, desires, and intentions (BDI).

    “Work in this area has historically been cast as imbuing 'agents' with 'beliefs', 'desires', and 'intentions' (BDI) (e.g.Pulman, 1997;Traum and Larsson, 2003).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Pulman, 1997

      Not checked in the free run.

    • cites Traum and Larsson, 2003

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    There is no necessary and sufficient condition for a system to be anthropomorphic; anthropomorphism exists on a spectrum.

    “There is no necessary and sufficient condition for a system to be anthropomorphic, i.e. there exist no particular threshold that affords a binary classification of whether a system is anthropomorphic or not, instead anthropomorphism exists on a spectrum.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationreasoning

    By designing anthropomorphic systems or even simply leaving space for their (gendered) personification by users, developers risk enabling propagating stereotypes and associated harms.

    “That is, by designing anthropomorphic systems or even simply leaving space for their (gendered) personification by users, developers risk enabling propagating stereotypes and associated harms.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given

    The roles that dialogue systems are given by designers and users can shift them from the realm of tools towards humanlike roles.

    “The roles that dialogue systems are unconsciously and consciously given by their designers and users can shift dialogue systems from the realm of tools towards one of humanlike roles.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationreasoning

    The more anthropomorphic elements a system has, the more humanlike it appears.

    “The more of such elements a system has, the more humanlike it appears.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given

    Even if developers and designers attempt to avoid including any anthropomorphic signals, humans may still personify systems and perceive them as anthropomorphic entities.

    “Even if developers and designers attempt to avoid including any anthropomorphic signals, humans may still personify systems and perceive them as anthropomorphic entities.”
    Quote in the paperNot a substantial claim
27 of 27

  • The paper relies on this without support.

    Computing systems do not have thoughts.

    “While computing systems do not have thoughts, the language that they output can give the appearance of thought by indicating that they hold opinions and morals or sentience.”

  • The paper relies on this without support.

    Computing systems cannot have values or morals.

    “While computing systems cannot have values or morals, there have been attempts to align the output of dialogue systems with expressed human moral values.”

  • The paper relies on this without support.

    Being an agent requires intentionality and animacy.

    “However, being an agent, i.e. having agency, requires intentionality and animacy.”

  • The paper relies on this without support.

    Traditionally, the goal of Artificial Intelligence research has been to create systems that would exhibit intelligence indistinguishable from humans.

    “Traditionally, the goal of Artificial Intelligence research has been to create systems that would exhibit intelligence indistinguishable from humans.”

  • The paper relies on this without support.

    Anthropomorphic terminology is deeply rooted in the argot of computer scientists, particularly those working in artificial intelligence.

    “Although anthropomorphic terminology is deeply rooted in the argot of computer scientists, particularly those working in 'artificial intelligence'”

  • The paper relies on this without support.

    Anthropomorphising dialogue systems can be attractive for researchers in order to drive user engagement.

    “Anthropomorphising dialogue systems can be attractive for researchers in order to drive user engagement.”

  • The paper relies on this without support.

    Systems such as OpenAI's ChatGPT and Microsoft's Sydney have high surface form language generation performance.

    “systems such as OpenAI's ChatGPT and Microsoft's Sydney, which have high surface form language generation performance.”

  • The paper relies on this without support.

    Data sources are often limited to specific populations, and thus only represent the morals or values of some people.

    “The data sources are often limited to specific populations, and thus only represent the morals or values of some people.”

  • The paper relies on this without support.

    It is infeasible to comprehensively map which linguistic cues to use or avoid.

    “while it is infeasible to comprehensively map which linguistic cues to use or avoid”
9 of 9

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • There are concerns around harms from anthropomorphism of AI systems that may stem from over-trust, deception, threats to human agency, and propagation of stereotypes.

    “There are also concerns around harms from anthropomorphism of AI systems that may stem from over-trust, deception, threats to human agency, and propagation of stereotypes [1].”

    From "I'm Not Sure, But...": Examining the Impact of Large Language Models' Uncertainty Expression on User Reliance and Trust (2024) · cites this paper as [1]

    Section 4, “Consequences of Anthropomorphism” (page not provided), links anthropomorphism to unwarranted trust, deception, and reinforced stereotypes. The Introduction also warns that people may follow a system’s harmful instructions. The paper discusses a shift of agency from humans to machines in Section 3.2, but presents it as a way of anthropomorphising systems, not clearly as a resulting threat to human agency. (LLM, conf=0.83)

    Experimental — accuracy not yet measured

  • The tendency of users to anthropomorphize clinical AI agents can lead to over-reliance and high-risk scenarios.

    “The tendency of users to anthropomorphize these agents can lead to over-reliance and high-risk scenarios, necessitating careful design to mitigate these risks in healthcare settings (Abercrombie et al. 2023).”

    From Human-Clinical AI Agent Collaboration (2025) · cites this paper as Abercrombie et al. 2023

    Section 4 says anthropomorphic behaviour can increase unwarranted trust in dialogue systems, and the Introduction warns that following their advice can cause harm. The paper also mentions high-risk medical diagnoses and treatment plans under “Unqualified Expertise” and recommends careful design, but it does not specifically establish over-reliance on anthropomorphized clinical AI agents in healthcare settings. (LLM, conf=0.85)

    Experimental — accuracy not yet measured

  • Recent work has expressed concern around anthropomorphism leading to over-trust.

    “Recent work has expressed concern around anthropomorphism leading to over-trust [1, 91], a potential path for uncertainty expression to backfire.”

    From "I'm Not Sure, But...": Examining the Impact of Large Language Models' Uncertainty Expression on User Reliance and Trust (2024) · cites this paper as [1]

Cites

Flagged first, then by citations

Research works printed but not linked: 6

  1. Scott H. Fraundorf, Jennifer Arnold, and Valerie J. Langlois. 2018. Disfluency. https://www.oxfordbibliographies. com/view/document/obo-9780199772810/ obo-9780199772810-0189.xml. Oxford Univer- sity Press. Accessed: 2023-05-12.
  2. Tua Korhonen. 2019. Anthropomorphism and the ae- sopic animal fables. Animals and their Relation to Gods, Humans and Things in the Ancient World, pages 211-231.
  3. Byron Reeves and Clifford Nass. 1996. The Media Equation: How People Treat Computers, Television, and New Media like Real People. Cambridge univer- sity press Cambridge, UK.
  4. Roger Scruton. 2017. On human nature. In On Human Nature. Princeton University Press.
  5. Toby Walsh. 2016. Turing's red flag. Communications of the ACM, 59(7):34-37.
  6. Sarah Wilson and Roger K. Moore. 2017. Robot, alien and cartoon voices: Implications for speech-enabled systems. In 1st International Workshop on Vocal Interactivity in-and-between Humans, Animals and Robots (VIHAR), page 42-46.

Cited by

Sorted by citations

Related

Authors
Abercrombie, G., Curry, A., Dinkar, T., Rieser, V., & Talat, Z.
Affiliations
Heriot-Watt University (GB)
ORCID
Venue
Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing
Publisher
Association for Computational Linguistics
Published
2023
Pages
4776-4790
DOI
10.18653/v1/2023.emnlp-main.290
OpenAlex
W4389523717
Semantic Scholar
ed73c55634624beaa6ff97fe5c277c3ae1800a96
Cited by
83
References
88
Citations by year
2026: 44 · 2025: 27 · 2024: 10 · 2023: 2
FWCI
14.73
Type
conference paper
Language
en
Open access
Licence
cc-by
Locations
  • Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • Research Explorer (The University of Manchester) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Funders
  • Leverhulme Trust
  • European Commission (HORIZON2020, 201100)
  • Engineering and Physical Sciences Research Council (EP/W025493/1, EP/T026952/1, EP/T023767/1)
  • H2020 European Research Council (949944)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
  • Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
  • Physical Sciences › Computer Science › Artificial Intelligence › AI in Service Interactions
Keywords
anthropomorphism, personification, chatbots, dialogue systems, linguistic cues, gender stereotypes, overreliance on AI

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-10.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-10.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review