Cited by 92Social Robot Interaction and HRIInnovative Human-Technology InteractionDigital Mental Health Interventions

Empathy Toward Artificial Intelligence Versus Human Experiences and the Role of Transparency in Mental Health and Social Support Chatbot Design: Comparative Study

Open access
Shen, Jocelyn, DiPaola, Daniella, Ali, Safinah, Sap, Maarten, Park, Hae Won, Breazeal, Cynthia·2024·JMIR Mental Health

Published version of a preprint

Abstract

Background: Empathy is a driving force in our connection to others, our mental well-being, and resilience to challenges. With the rise of generative artificial intelligence (AI) systems, mental health chatbots, and AI social support companions, it is important to understand how empathy unfolds toward stories from human versus AI narrators and how transparency plays a role in user emotions. Objective: We aim to understand how empathy shifts across human-written versus AI-written stories, and how these findings inform ethical implications and human-centered design of using mental health chatbots as objects of empathy. Methods: We conducted crowd-sourced studies with 985 participants who each wrote a personal story and then rated empathy toward 2 retrieved stories, where one was written by a language model, and another was written by a human. Our studies varied disclosing whether a story was written by a human or an AI system to see how transparent author information affects empathy toward the narrator. We conducted mixed methods analyses: through statistical tests, we compared user’s self-reported state empathy toward the stories across different conditions. In addition, we qualitatively coded open-ended feedback about reactions to the stories to understand how and why transparency affects empathy toward human versus AI storytellers. Results: We found that participants significantly empathized with human-written over AI-written stories in almost all conditions, regardless of whether they are aware (t196=7.07, P<.001, Cohen d=0.60) or not aware (t298=3.46, P<.001, Cohen d=0.24) that an AI system wrote the story. We also found that participants reported greater willingness to empathize with AI-written stories when there was transparency about the story author (t494=–5.49, P<.001, Cohen d=0.36). Conclusions: Our work sheds light on how empathy toward AI or human narrators is tied to the way the text is presented, thus informing ethical considerations of empathetic artificial social support or mental health chatbots.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.2196/62679

Cites

45 of 45 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The anthropomorphic design of chatbots and virtual companions encourages users to attribute human-like intentionality and empathy to algorithms.

    “the anthropomorphic design of chatbots and virtual companions encourages users to attribute human-like intentionality and empathy to algorithms, a dynamic reminiscent of the “ELIZA effect” described in early human-computer interaction literature [20,21].”

    From Delusional Experiences Emerging From AI Chatbot Interactions or “AI Psychosis” (2025) · cites this paper as [20]

    The Introduction says commercial chatbots adopt identities or experiences and that human-like appearance and behavior can lead users to form empathic relationships with AI agents. It does not establish that users attribute human-like intentionality or empathy to the algorithms themselves. The Discussion (Principal Results) also cautions that more anthropomorphic representation may not increase trust in the storytelling setting studied. (LLM, conf=0.83)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 3

  1. [10]Paiva A Dias J Sobral D Aylett R Sobreperez P Woods S Zoll C Hall L Caring for agents and agents that care: building empathic relations with synthetic agents 2005 Proceedings of the 3rd International Joint Conference on Autonomous Agents and Multiagent Systems, 2004 (AAMAS 2004) 2004 New York, NY
  2. [25]Giorgi S Markowitz DM Soni N Varadarajan V Mangalik S Schwartz HA “I slept like a baby”: using human traits to characterize deceptive ChatGPT and human text 2023 1st International Workshop on Implicit Author Characterization from Texts for Search and Retrieval (IACT’23) 2023 Taipei, Taiwan 23 37
  3. [42]Yampolskiy RV Taxonomy of pathways to dangerous artificial intelligence 2016 The Workshops of the 30th AAAI Conference on Artificial Intelligence March 29, 2016 San Jose, CA 143 148

Cited by

Sorted by citations

Authors
Shen, J., DiPaola, D., Ali, S., Sap, M., Park, H. W., & Breazeal, C.
Affiliations
  • MIT Media Lab (US)
  • Carnegie Mellon University (US)
Corresponding author
Shen, Jocelyn
Venue
JMIR Mental Health
Published
2024-09-25
Volume
11
Pages
e62679
ISSN
2368-7959 (ISSN-L)
OpenAlex
W4402155818
Cited by
92
References
42
Citations by year
2026: 60 · 2025: 32
FWCI
43.36
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • JMIR Mental Health (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DSpace@MIT (Massachusetts Institute of Technology) (submittedVersion, cc-by-nc)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
Funders
  • National Science Foundation (2141064)
  • Ministry of Science and ICT, South Korea (2020-0-00842)
  • Institute for Information and Communications Technology Promotion (2020-0-00842)
Topics
  • Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
  • Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
  • Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
Keywords
AI-generated text, transparency, mental health chatbots, author disclosure, generative AI, human-centered design, AI ethics, user emotions
MeSH terms
Adult, Artificial Intelligence, Empathy, Female, Humans, Male, Mental Health, Middle Aged, Social Support, Narration, Young Adult

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review