Comparing the Perspectives of Generative AI, Mental Health Experts, and the General Public on Schizophrenia Recovery: Case Vignette Study
Open accessThe DOI matches its Crossref record; 17 of the 17 references it cites were found in a registry or an index.
Abstract
Background: The current paradigm in mental health care focuses on clinical recovery and symptom remission. This model’s efficacy is influenced by therapist trust in patient recovery potential and the depth of the therapeutic relationship. Schizophrenia is a chronic illness with severe symptoms where the possibility of recovery is a matter of debate. As artificial intelligence (AI) becomes integrated into the health care field, it is important to examine its ability to assess recovery potential in major psychiatric disorders such as schizophrenia. Objective: This study aimed to evaluate the ability of large language models (LLMs) in comparison to mental health professionals to assess the prognosis of schizophrenia with and without professional treatment and the long-term positive and negative outcomes. Methods: Vignettes were inputted into LLMs interfaces and assessed 10 times by 4 AI platforms: ChatGPT-3.5, ChatGPT-4, Google Bard, and Claude. A total of 80 evaluations were collected and benchmarked against existing norms to analyze what mental health professionals (general practitioners, psychiatrists, clinical psychologists, and mental health nurses) and the general public think about schizophrenia prognosis with and without professional treatment and the positive and negative long-term outcomes of schizophrenia interventions. Results: For the prognosis of schizophrenia with professional treatment, ChatGPT-3.5 was notably pessimistic, whereas ChatGPT-4, Claude, and Bard aligned with professional views but differed from the general public. All LLMs believed untreated schizophrenia would remain static or worsen without professional treatment. For long-term outcomes, ChatGPT-4 and Claude predicted more negative outcomes than Bard and ChatGPT-3.5. For positive outcomes, ChatGPT-3.5 and Claude were more pessimistic than Bard and ChatGPT-4. Conclusions: The finding that 3 out of the 4 LLMs aligned closely with the predictions of mental health professionals when considering the “with treatment” condition is a demonstration of the potential of this technology in providing professional clinical prognosis. The pessimistic assessment of ChatGPT-3.5 is a disturbing finding since it may reduce the motivation of patients to start or persist with treatment for schizophrenia. Overall, although LLMs hold promise in augmenting health care, their application necessitates rigorous validation and a harmonious blend with human expertise.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The alliance in adult psychotherapy: A meta-analytic synthesis
Flückiger, C., Del Re, A. C., Wampold, B. E., & Horvath, A. O. (2018) · Psychotherapy · doi:10.1037/pst0000172 · Cited by 1,763 · cited 1×
“A meta-analysis of over 30,000 participants showed the therapeutic alliance was highly correlated with outcomes, regardless of therapy type [29].”
BackgroundMcCutcheon, R. A., Reis Marques, T., & Howes, O. D. (2020) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2019.3360 · Cited by 1,481
Uses and abuses of recovery: implementing recovery-oriented practices in mental health systems
Slade, M., Amering, M., Farkas, M., Hamilton, B., O'Hagan, M., Panther, G., Perkins, R., Shepherd, G., Tse, S., & Whitley, R. (2014) · World Psychiatry · doi:10.1002/wps.20084 · Cited by 949
Correll, C. U., & Schooler, N. R. (2020) · Neuropsychiatric Disease and Treatment · doi:10.2147/ndt.s225643 · Cited by 911 · cited 1×
“It has a major impact on life expectancy and quality of life, and its repercussions extend to family and caregivers [2].”
BackgroundThe Role of Common Factors in Psychotherapy Outcomes
Cuijpers, P., Reijnders, M., & Huibers, M. J. H. (2019) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-050718-095424 · Cited by 748 · cited 1×
“One salient finding is the positive correlation between a strong therapeutic alliance and enhanced outcomes [28].”
BackgroundAli, O., Abdelbaki, W., Shrestha, A., Elbasi, E., Alryalat, M. A. A., & Dwivedi, Y. K. (2023) · Journal of Innovation & Knowledge · doi:10.1016/j.jik.2023.100333 · Cited by 568
Patient apprehensions about the use of artificial intelligence in healthcare
Richardson, J. P., Smith, C., Curtis, S., Watson, S., Zhu, X., Barry, B., & Sharp, R. R. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00509-1 · Cited by 424
(2023) · Journal of Applied Learning & Teaching · doi:10.37074/jalt.2023.6.1.23 · Cited by 421 · cited 1×
“These algorithms were trained on different amounts and qualities of data, underwent different processes of elimination, and use distinct strategies for receiving feedback from system users [55].”
MethodCuijpers, P., Quero, S., Noma, H., Ciharova, M., Miguel, C., Karyotaki, E., Cipriani, A., Cristea, I. A., & Furukawa, T. A. (2021) · World Psychiatry · doi:10.1002/wps.20860 · Cited by 388
ChatGPT outperforms humans in emotional awareness evaluations
Elyoseph, Z., Hadar-Shoval, D., Asraf, K., & Lvovsky, M. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1199058 · Cited by 324
Self-citation
Cited by
Evaluating Generative AI in Mental Health: Systematic Review of Capabilities and Limitations
Wang, L., Bhanushali, T., Huang, Z., Yang, J., Badami, S., & Hightow-Weidman, L. (2025) · JMIR Mental Health · doi:10.2196/70014 · Cited by 73
“The study compared the AI model’s responses with mental health professionals, treating mental health professionals as the golden criterion [24].”
InfluentialSolaiman, B. (2024) · International Journal of Law and Psychiatry · doi:10.1016/j.ijlp.2024.102028 · Cited by 24
Shinan-Altman, S., Elyoseph, Z., & Levkovich, I. (2024) · PeerJ · doi:10.7717/peerj.17468 · Cited by 21
Self-citationA scoping review on the mental health harms of LLM-based chatbots
Diel, A., Torous, J., Cuijpers, P., Kleesiek, J., Nensa, F., Weber, N., Faust, F., Lalgi, T. J., Mellis, F. S., Teufel, M., et al. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-03054-x · Cited by 4
Metadata
- Authors
- Elyoseph, Z., & Levkovich, I.
- Venue
- JMIR Mental Health
- Publisher
- JMIR Publications Inc.
- Published
- 2024-03-18
- Volume
- 11
- Pages
- e53043-e53043
- ISSN
- 2368-7959 (ISSN-L)
- OpenAlex
- W4393231947
- Cited by
- 58
- References
- 17
- FWCI
- 4.65
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Comparing the Perspectives of Generative AI, Mental Health Experts, and the General Public on Schizophrenia Recovery: Case Vignette Study. https://app.cochat.ai/papers/10.2196/53043
- The paper
- Elyoseph, Z., & Levkovich, I. (2024). Comparing the Perspectives of Generative AI, Mental Health Experts, and the General Public on Schizophrenia Recovery: Case Vignette Study. JMIR Mental Health. https://doi.org/10.2196/53043
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.