Human-Human vs Human-AI Therapy: An Empirical Study
Open accessThe DOI matches its Crossref record; 27 of the 27 references it cites were found in a registry or an index.
Abstract
In many nations, demand for mental health services currently outstrips supply, especially in the area of talk-based psychological interventions. Within this context, chatbots (software applications designed to simulate conversations with human users) are increasingly explored as potential adjuncts to traditional mental healthcare service delivery with a view to improving accessibility and reducing waiting times. However, the effectiveness and acceptability of such chatbots remains under-researched. This study evaluates mental health professionals’ perceptions of Pi, a relational Artificial Intelligence (AI) chatbot, in the early stages of the psychotherapeutic process (problem exploration). We asked 63 therapists to assess therapy transcripts between a human client and Pi (human-AI) versus traditional therapy transcripts between therapists and clients (human-human). Therapists were unable to reliably discriminate between human-AI and human-human therapy transcripts. Therapists were accurate only 53.9% of the time, no better than chance, and rated the human-AI transcripts as higher quality on average. This study has potentially profound implications for the treatment of mental health problems, adding tentative support for the use of relational AI chatbots in providing initial assistance for mild to moderate psychological issues, especially when access to human therapists is constrained.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Structural Equation Modeling With AMOS: Basic Concepts, Applications, and Programming, Third Edition
Byrne, B. M. (2016) · doi:10.4324/9781315757421 · Cited by 4,653
Beyond the Turk: Alternative platforms for crowdsourcing behavioral research
Peer, E., Brandimarte, L., Samat, S., & Acquisti, A. (2017) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2017.01.006 · Cited by 2,957 · cited 1×
“Prolific is considered one of the most widely recommended data collection websites due to its high data quality (Peer et al., 2017).”
Fitzpatrick, K. K., Darcy, A., & Vierhile, M. (2017) · JMIR Mental Health · doi:10.2196/mental.7785 · Cited by 2,730 · cited 12×
“Within the mental health field, chatbots that use CBT, “Wysa” and "Woebot," stand out as notable examples.”
InfluentialThe ethics of AI in health care: A mapping review
Morley, J., Machado, C. C. V., Burr, C., Cowls, J., Joshi, I., Taddeo, M., & Floridi, L. (2020) · Social Science & Medicine · doi:10.1016/j.socscimed.2020.113172 · Cited by 1,105 · cited 1×
“Yet, this personalization brings forth privacy concerns that necessitate careful consideration (Morley et al., 2020).”
Chatbots and Conversational Agents in Mental Health: A Review of the Psychiatric Landscape
Vaidyam, A. N., Wisniewski, H., Halamka, J. D., Kashavan, M. S., & Torous, J. B. (2019) · The Canadian Journal of Psychiatry · doi:10.1177/0706743719828977 · Cited by 1,048 · cited 3×
“To this end, a few studies have been undertaken to determine the effectiveness of chatbots in the mental health domain (Vaidyam et al., 2019).”
Adoption of AI-based chatbots for hospitality and tourism
Pillai, R., & Sivathanu, B. (2020) · International Journal of Contemporary Hospitality Management · doi:10.1108/ijchm-04-2020-0259 · Cited by 986 · cited 1×
“Chatbots have been employed across various business sectors, including retail (Chung et al., 2020), hospitality (Pillai & Sivathanu, 2020), real estate (Quan et al., 2018), finance (Quah & Chua, 2019), healthcare (Chow et al., 2023a, 2023b), and education (Kuhail et al., 2023).”
Interacting with educational chatbots: A systematic review
Kuhail, M. A., Alturki, N., Alramlawi, S., & Alhejori, K. (2023) · Education and Information Technologies · doi:10.1007/s10639-022-11177-3 · Cited by 841
Self-citationAn overview of the features of chatbots in mental health: A scoping review
Abd-alrazaq, A. A., Alajlani, M., Alalwan, A. A., Bewick, B. M., Gardner, P., & Househ, M. (2019) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2019.103978 · Cited by 605 · cited 4×
“As such, it is imperative for ongoing research to thoroughly evaluate and summarize the evidence concerning their effectiveness and acceptability (Abd-Alrazaq et al., 2019).”
InfluentialXu, L., Sanders, L., Li, K., & Chow, J. C. L. (2021) · JMIR Cancer · doi:10.2196/27850 · Cited by 571 · cited 1×
“In physical healthcare, chatbots can provide patient support, diagnosis, and work-flow efficiency and encourage a healthy lifestyle for cancer patients (Xu et al., 2021).”
Exploring relationship development with social chatbots: A mixed-method study of replika
Pentina, I., Hancock, T., & Xie, T. (2023) · Computers in Human Behavior · doi:10.1016/j.chb.2022.107600 · Cited by 522 · cited 2×
“Chatbots provide a range of benefits, including companion-ship and entertainment to support (Klaus & Zaichkowsky, 2020), as well as physical (Chow et al., 2023a) and mental health care (Pentina et al., 2023).”
Cited by
Moore, J., Grabb, D., Agnew, W., Klyman, K., Chancellor, S., Ong, D. C., & Haber, N. (2025) · Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3715275.3732039 · Cited by 204
Herbener, A. B., & Damholdt, M. F. (2025) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2024.103409 · Cited by 107
Warning: AI chatbots will soon dominate psychotherapy
Frances, A. (2026) · The British Journal of Psychiatry · doi:10.1192/bjp.2025.10380 · Cited by 33
Kuhail, M. A., Al-Shamaileh, O., Farooq, S., Shahin, H., Abdelzaher, F., & Thomas, J. (2025) · Human Behavior and Emerging Technologies · doi:10.1155/hbe2/9942295 · Cited by 11
Self-citationMcFadyen, J., Habicht, J., Dina, L.-M., Harper, R., Hauser, T. U., & Rollwage, M. (2024) · doi:10.1101/2024.11.01.24316565 · Cited by 10
Metadata
- Authors
- Kuhail, M. A., Alturki, N., Thomas, J., Alkhalifa, A. K., & Alshardan, A.
- Venue
- International Journal of Human–Computer Interaction
- Publisher
- Informa UK Limited
- Published
- 2024-08-16
- Volume
- 41
- Issue
- 11
- Pages
- 6841-6852
- ISSN
- 1044-7318 (ISSN-L)
- 1532-7590
- OpenAlex
- W4401648069
- Cited by
- 61
- References
- 27
- FWCI
- 22.15
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Human-Human vs Human-AI Therapy: An Empirical Study. https://app.cochat.ai/papers/10.1080/10447318.2024.2385001
- The paper
- Kuhail, M. A., Alturki, N., Thomas, J., Alkhalifa, A. K., & Alshardan, A. (2025). Human-Human vs Human-AI Therapy: An Empirical Study. International Journal of Human–Computer Interaction. https://doi.org/10.1080/10447318.2024.2385001
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.