Cited by 216Digital Mental Health InterventionsEthics and Social Impacts of AIArtificial Intelligence in Healthcare and Education

Your robot therapist is not your therapist: understanding the role of AI-powered mental health chatbots

Open access
Khawaja, Zoha, Bélisle-Pipon, Jean-Christophe·2023·Frontiers in Digital Health

Abstract

Artificial intelligence (AI)-powered chatbots have the potential to substantially increase access to affordable and effective mental health services by supplementing the work of clinicians. Their 24/7 availability and accessibility through a mobile phone allow individuals to obtain help whenever and wherever needed, overcoming financial and logistical barriers. Although psychological AI chatbots have the ability to make significant improvements in providing mental health care services, they do not come without ethical and technical challenges. Some major concerns include providing inadequate or harmful support, exploiting vulnerable populations, and potentially producing discriminatory advice due to algorithmic bias. However, it is not always obvious for users to fully understand the nature of the relationship they have with chatbots. There can be significant misunderstandings about the exact purpose of the chatbot, particularly in terms of care expectations, ability to adapt to the particularities of users and responsiveness in terms of the needs and resources/treatments that can be offered. Hence, it is imperative that users are aware of the limited therapeutic relationship they can enjoy when interacting with mental health chatbots. Ignorance or misunderstanding of such limitations or of the role of psychological AI chatbots may lead to a therapeutic misconception (TM) where the user would underestimate the restrictions of such technologies and overestimate their ability to provide actual therapeutic support and guidance. TM raises major ethical concerns that can exacerbate one's mental health contributing to the global mental health crisis. This paper will explore the various ways in which TM can occur particularly through inaccurate marketing of these chatbots, forming a digital therapeutic alliance with them, receiving harmful advice due to bias in the design and algorithm, and the chatbots inability to foster autonomy with patients.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3389/fdgth.2023.1278186

Cites

49 of 49 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The risk of LLM-CBs validating or reinforcing harmful beliefs such as suicidal ideations or delusions can occur from underrepresentation of severe mental health cases in training data, training on stigmatizing data, or LLM-CBs reflecting users' biases.

    “This risk can occur from the underrepresentation of severe mental health cases in training data, training on stigmatizing data, or by LLM-CBs reflecting users’ biases41.”

    From A scoping review on the mental health harms of LLM-based chatbots (2026) · cites this paper as [41]

    The Introduction and “Bias in AI algorithms” section say that biased or unrepresentative training data can lead mental-health chatbots to give inappropriate or harmful responses. The paper does not specifically connect this risk to underrepresentation of severe mental-health cases, training on stigmatizing data, or chatbots reflecting a user’s biases; nor does it describe validation of suicidal ideation or delusions. (LLM, conf=0.86)

    Experimental — accuracy not yet measured
  • As general-purpose LLMs are not medical applications, entered data is not protected with the standards typical for medical data, and data protection often fails HIPAA standards.

    “As general-purpose LLMs are not medical applications, entered data is not protected with the standards typical for medical data, and data protection often fails HIPAA standards41,48.”

    From A scoping review on the mental health harms of LLM-based chatbots (2026) · cites this paper as [41]

    In “Chatbot, friend or foe: forming a digital therapeutic alliance” (page unavailable), the paper says mental-health chatbots are not considered medical devices and are not bound by doctors’ confidentiality obligations. “Meet your AI self-help expert: marketing chatbots” (page unavailable) adds that many mental-health apps fall outside HIPAA’s scope. These passages support the privacy concern, but they do not establish that general-purpose LLMs as a class fail HIPAA standards; falling outside HIPAA’s scope is not the same as failing its standards. (LLM, conf=0.84)

    Experimental — accuracy not yet measured
  • Two articles mentioned that therapeutic misconception may arise when LLM-CB behaviors lead to users overestimating its abilities to provide psychotherapeutic support while underestimating its limitations.

    “Mentioned by two articles, therapeutic misconception may arise when LLM-CB behaviors (e.g., simulated empathy, validation) lead to users overestimating its abilities to provide psychotherapeutic support while underestimating its limitations41.”

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

1 of the 50 references couldn’t be matched to a known paper.

Research works printed but not linked: 2

  1. [7]Sepahpour T. Ethical considerations of chatbot use for mental health support [master’s thesis]. John Hopkins University. (2020).
  2. [40]Wintermeyer SA. Positive psychology apps: a systematic review of the quality and characteristics of a selection of current free-of-charge positive psychological apps aiming to enhance resilience available in the Google Play Store [Master's thesis]. Enschede: University of Twente (2021). https://purl.utwente.nl/essays/89052

Cited by

Sorted by citations

Authors
Khawaja, Z., & Bélisle-Pipon, J.-C.
Affiliations
Simon Fraser University (CA)
Venue
Frontiers in Digital Health
Published
2023-11-08
Volume
5
Pages
1278186-1278186
ISSN
2673-253X (ISSN-L)
OpenAlex
W4388524014
Cited by
214
References
50
Citations by year
2026: 95 · 2025: 87 · 2024: 32
FWCI
55.24
Type
journal article
Language
en
Open access
Locations
  • Frontiers in Digital Health (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics
  • Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
Keywords
mental health chatbots, therapeutic misconception, artificial intelligence in mental health care, algorithmic bias, chatbot marketing, patient autonomy, vulnerable populations, access to mental health care

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review