The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
The health risks of generative AI-based wellness apps
Abstract
Artificial intelligence (AI)-enabled chatbots are increasingly being used to help people manage their mental health. Chatbots for mental health and particularly 'wellness' applications currently exist in a regulatory 'gray area'. Indeed, most generative AI-powered wellness apps will not be reviewed by health regulators. However, recent findings suggest that users of these apps sometimes use them to share mental health problems and even to seek support during crises, and that the apps sometimes respond in a manner that increases the risk of harm to the user, a challenge that the current US regulatory structure is not well equipped to address. In this Perspective, we discuss the regulatory landscape and potential health risks of AI-enabled wellness apps. Although we focus on the United States, there are similar challenges for regulators across the globe. We discuss the problems that arise when AI-based wellness apps cross into medical territory and the implications for app developers and regulatory bodies, and we outline outstanding priorities for the field.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The deployment of lightweight AI models on edge devices revolutionizes medical AI by enabling real-time, localized data processing with minimal latency.132,133”
From A foundational architecture for AI agents in healthcare (2025) · cites this paper as [132]
In the citing paper
“Complementary experimental work shows that AI companions can reduce self-reported loneliness as effectively as brief human interaction, and more than solitary digital activities such as browsing or watching videos (De Freitas and Cohen 2024).”
From Toward an Ethic of Synthetic Relationality: Identity, Intimacy, and Risk in AI-Mediated Roleplay Environments (2025) · cites this paper as De Freitas and Cohen 2024
In the citing paper
“Despite calls for clear guidelines on the use of LLMs in mental health [39], these bots are currently available in public-facing platforms used by millions, potentially interacting with people in mental health crises.”
From Expressing stigma and inappropriate responses prevents LLMs from safely replacing mental health providers. (2025) · cites this paper as [39]
In the citing paper
“These applications of LLMs are unregulated in the U.S., whereas therapists and mental health care providers have strict training and clinical licensing requirements [39].”
From Expressing stigma and inappropriate responses prevents LLMs from safely replacing mental health providers. (2025) · cites this paper as [39]
In the citing paper
“These applications of LLMs are unregulated in the U.S., whereas therapists and mental health care providers have strict training and clinical licensing requirements [39].”
From Expressing stigma and inappropriate responses prevents LLMs from safely replacing mental health providers. (2025) · cites this paper as [39]
4 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Artificial hallucinations in ChatGPT: implications in scientific writing
Alkaissi, H., & McFarlane, S. I. (2023) · Cureus · doi:10.7759/cureus.35179 · Cited by 1,540
Chatbots and Conversational Agents in Mental Health: A Review of the Psychiatric Landscape
Vaidyam, A. N., Wisniewski, H., Halamka, J. D., Kashavan, M. S., & Torous, J. B. (2019) · The Canadian Journal of Psychiatry · doi:10.1177/0706743719828977 · Cited by 1,048 · cited 2×
“Unfortunately, we are at the nascent stage of building that evidence base and there is relatively little empirical work on the issue 17–19 .”
BackgroundHuman resources for mental health care: current situation and strategies for action
Kakuma, R., Minas, H., van Ginneken, N., Dal Poz, M. R., Desiraju, K., Morris, J. E., Saxena, S., & Scheffler, R. M. (2011) · The Lancet · doi:10.1016/s0140-6736(11)61093-3 · Cited by 979
Stigma about depression and its impact on help-seeking intentions
Barney, L. J., Griffiths, K. M., Jorm, A. F., & Christensen, H. (2006) · Australian & New Zealand Journal of Psychiatry · doi:10.1080/j.1440-1614.2006.01741.x · Cited by 844
Eysenbach, G. (2023) · JMIR Medical Education · doi:10.2196/46885 · Cited by 840
An overview of the features of chatbots in mental health: A scoping review
Abd-alrazaq, A. A., Alajlani, M., Alalwan, A. A., Bewick, B. M., Gardner, P., & Househ, M. (2019) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2019.103978 · Cited by 605
Abd-Alrazaq, A. A., Rababeh, A., Alajlani, M., Bewick, B. M., & Househ, M. (2020) · Journal of Medical Internet Research · doi:10.2196/16021 · Cited by 503 · cited 2×
“Unfortunately, we are at the nascent stage of building that evidence base and there is relatively little empirical work on the issue 17–19 .”
BackgroundInfluentialArtificially intelligent chatbots in digital mental health interventions: a review
Boucher, E. M., Harake, N. R., Ward, H. E., Stoeckl, S. E., Vargas, J., Minkel, J., Parks, A. C., & Zilca, R. (2021) · Expert Review of Medical Devices · doi:10.1080/17434440.2021.2013200 · Cited by 440
AI-Generated Medical Advice—GPT and Beyond
Haupt, C. E., & Marks, M. (2023) · JAMA · doi:10.1001/jama.2023.5321 · Cited by 346 · cited 1×
“Moreover, there may be complex questions relating to First Amendment freedom-of-speech protections (in the US) against liability for some forms of chatbot advice outside of a medical relationship 33 .”
BackgroundWhat works for whom: Tailoring psychotherapy to the person
Norcross, J. C., & Wampold, B. E. (2011) · Journal of Clinical Psychology · doi:10.1002/jclp.20764 · Cited by 338
Research works printed but not linked: 3
- Simon, D. A., Shachar, C. & Cohen, I. G. Skating the line between general wellness products and regulated devices: strategies and implications. J. Law Biosci. 9, lsac015 (2022).
- De Freitas, J., Uğuralp, A. K., Uğuralp, Z.-O. U. & Puntoni, S. Chatbots and mental health: insights into the safety of generative AI. J. Consum. Psychol. 00, 1–11 (2023).
- Opel, D. J., Kious, B. M. & Cohen, I. G. AI as a mental health therapist for adolescents. JAMA Pediatr. 177, 1253–1254 (2023).
Cited by
Randomized Trial of a Generative AI Chatbot for Mental Health Treatment
Heinz, M. V., Mackin, D. M., Trudeau, B. M., Bhattacharya, S., Wang, Y., Banta, H. A., Jewett, A. D., Salzhauer, A. J., Griffin, T. Z., & Jacobson, N. C. (2025) · NEJM AI · doi:10.1056/aioa2400802 · Cited by 287
“It happened to be the perfect thing”: experiences of generative AI chatbots for mental health
Siddals, S., Torous, J., & Coxon, A. (2024) · npj Mental Health Research · doi:10.1038/s44184-024-00097-4 · Cited by 241
Moore, J., Grabb, D., Agnew, W., Klyman, K., Chancellor, S., Ong, D. C., & Haber, N. (2025) · Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3715275.3732039 · Cited by 204
AI, Health, and Health Care Today and Tomorrow: The JAMA Summit Report on Artificial Intelligence
Angus, D. C., Khera, R., Lieu, T., Liu, V., Ahmad, F. S., Anderson, B., Bhavani, S. V., Bindman, A., Brennan, T., Celi, L. A., et al. (2025) · JAMA · doi:10.1001/jama.2025.18490 · Cited by 195
Self-citationThe Application and Ethical Implication of Generative AI in Mental Health: Systematic Review
Wang, X., Zhou, Y., & Zhou, G. (2025) · JMIR Mental Health · doi:10.2196/70610 · Cited by 126
Metadata
- Authors
- De Freitas, J., & Cohen, I. G.
- Affiliations
- Harvard University (US)
- Corresponding author
- Cohen, I. Glenn
- ORCID
- Cohen, I. Glenn
- Venue
- Nature Medicine
- Published
- 2024-04-29
- Volume
- 30
- Issue
- 5
- Pages
- 1269-1275
- ISSN
- 1078-8956 (ISSN-L)
- 1546-170X
- OpenAlex
- W4395959984
- Cited by
- 95
- References
- 21
- Citations by year
- 2026: 38 · 2025: 45 · 2024: 12
- FWCI
- 5.79
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springernature.com/gp/researchers/text-and-data-mining (tdm)
- www.springernature.com/gp/researchers/text-and-data-mining (vor)
- Locations
- Nature Medicine (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Glenn Cohen was supported in part by a Novo Nordisk Foundation Grant for a scientifically independent International Collaborative Bioscience Innovation & Law Programme
- Topics
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Keywords
- generative AI, wellness apps, chatbots, mental health, health regulation, health risks, crisis support, app developers, United States
- MeSH terms
- Artificial Intelligence, Health Promotion, Humans, Mental Health, United States, Telemedicine, Mobile Applications
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The health risks of generative AI-based wellness apps. https://app.cochat.ai/papers/10.1038/s41591-024-02943-6
- The paper
- De Freitas, J., & Cohen, I. G. (2024). The health risks of generative AI-based wellness apps. Nature Medicine. https://doi.org/10.1038/s41591-024-02943-6
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.