Evaluating generative AI responses to real-world drug-related questions
Open accessAbstract
Generative Artificial Intelligence (AI) systems such as OpenAI's ChatGPT, capable of an unprecedented ability to generate human-like text and converse in real time, hold potential for large-scale deployment in clinical settings such as substance use treatment. Treatment for substance use disorders (SUDs) is particularly high stakes, requiring evidence-based clinical treatment, mental health expertise, and peer support. Thus, promises of AI systems addressing deficient healthcare resources and structural bias are relevant within this domain, especially in an anonymous setting. This study explores the effectiveness of generative AI in answering real-world substance use and recovery questions. We collect questions from online recovery forums, use ChatGPT and Meta's LLaMA-2 for responses, and have SUD clinicians rate these AI responses. While clinicians rated the AI-generated responses as high quality, we discovered instances of dangerous disinformation, including disregard for suicidal ideation, incorrect emergency helplines, and endorsement of home detox. Moreover, the AI systems produced inconsistent advice depending on question phrasing. These findings indicate a risky mix of seemingly high-quality, accurate responses upon initial inspection that contain inaccurate and potentially deadly medical advice. Consequently, while generative AI shows promise, its real-world application in sensitive healthcare domains necessitates further safeguards and clinical validation.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
These factors are especially important given the potential harm of incorrect medical information and that LLMs have been known to add irrelevant or sometimes dangerous text.
“These factors are especially important given the potential harm of incorrect medical information and that LLMs have been known to add irrelevant or sometimes dangerous text (Giorgi et al., 2024;Mei et al., 2023).”
From MedScore: Generalizable Factuality Evaluation of Open-ended Long-form Medical Answers by Domain-adapted Claim Decomposition and Verification (2026) · cites this paper as Giorgi et al., 2024
The Results (Fact checking; page unavailable) document nonfactual and dangerous LLM responses, and the Conclusions warn that an inaccurate answer could have serious health consequences. The cited passages do not establish that the models added irrelevant text, so that part of the sentence is not supported by this source. (LLM, conf=0.84)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · see citing works
LLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602 · see citing works
Large language models encode clinical knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2023) · Nature · doi:10.1038/s41586-023-06291-2 · Cited by 4,083 · see citing works
Baumgartner, J., Zannettou, S., Keegan, B., Squire, M., & Blackburn, J. (2020) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v14i1.7347 · Cited by 881 · see citing works
Mental Health Discourse on reddit: Self-Disclosure, Social Support, and Anonymity
De Choudhury, M., & De, S. (2014) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v8i1.14526 · Cited by 836 · see citing works
Studying Reddit: A Systematic Overview of Disciplines, Approaches, Methods, and Ethics
Proferes, N., Jones, N., Gilbert, S., Fiesler, C., & Zimmer, M. (2021) · Social Media + Society · doi:10.1177/20563051211019004 · Cited by 537 · cited 1× · see citing works
“Per the recommendations of Proferes et al. (2021), we refrain from directly quoting the Reddit questions and instead show rephrased and summarized examples in Table 1.”
Zack, T., Lehman, E., Suzgun, M., Rodriguez, J. A., Celi, L. A., Gichoya, J., Jurafsky, D., Szolovits, P., Bates, D. W., Abdulnour, R.-E. E., et al. (2024) · The Lancet Digital Health · doi:10.1016/s2589-7500(23)00225-x · Cited by 510 · see citing works
Sharma, A., Lin, I. W., Miner, A. S., Atkins, D. C., & Althoff, T. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-022-00593-2 · Cited by 500 · see citing works
Large language models propagate race-based medicine
Omiye, J. A., Lester, J. C., Spichak, S., Rotemberg, V., & Daneshjou, R. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00939-z · Cited by 417 · see citing works
Generative AI in Medicine and Healthcare: Promises, Opportunities and Challenges
Zhang, P., & Kamel Boulos, M. N. (2023) · Future Internet · doi:10.3390/fi15090286 · Cited by 411 · cited 1× · see citing works
“The potential of generative AI has already been noted across several fields, including health (Korn-giebel and Mooney, 2021, Varghese and Chapiro, 2023, Zhang and Kamel Boulos, 2023), psychology (Demszky et al., 2023, Kjell et al., 2023), and psychotherapy (Stade et al., 2024).”
Stade, E. C., Stirman, S. W., Ungar, L. H., Boland, C. L., Schwartz, H. A., Yaden, D. B., Sedoc, J., DeRubeis, R. J., Willer, R., & Eichstaedt, J. C. (2024) · npj Mental Health Research · doi:10.1038/s44184-024-00056-z · Cited by 391 · cited 4× · see citing works
“Next, generative AI systems can be tailored (Stade et al., 2024) or in-clude humans “ in the loop ” (Sharma et al., 2023) to provide personalized feedback within a given domain.”
InfluentialSelf-citationUsing large language models in psychology
Demszky, D., Yang, D., Yeager, D. S., Bryan, C. J., Clapper, M., Chandhok, S., Eichstaedt, J. C., Hecht, C., Jamieson, J., Johnson, M., et al. (2023) · Nature Reviews Psychology · doi:10.1038/s44159-023-00241-5 · Cited by 369 · see citing works
A Survey on Conversational Agents/Chatbots Classification and Design Techniques
Hussain, S., Ameri Sianaki, O., & Ababneh, N. (2019) · Advances in Intelligent Systems and Computing · doi:10.1007/978-3-030-15035-8_93 · Cited by 279 · see citing works
Korngiebel, D. M., & Mooney, S. D. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00464-x · Cited by 259 · see citing works
Barriers to Medications for Addiction Treatment: How Stigma Kills
Wakeman, S. E., & Rich, J. D. (2017) · Substance Use & Misuse · doi:10.1080/10826084.2017.1363238 · Cited by 221 · see citing works
Stigma and the Toll of Addiction
Volkow, N. D. (2020) · New England Journal of Medicine · doi:10.1056/nejmp1917360 · Cited by 206 · see citing works
Examination of Stigmatizing Language in the Electronic Health Record
Himmelstein, G., Bates, D. W., & Zhou, L. (2022) · JAMA Network Open · doi:10.1001/jamanetworkopen.2021.44967 · Cited by 201 · see citing works
Substance use, recovery, and linguistics: The impact of word choice on explicit and implicit bias
Ashford, R. D., Brown, A. M., & Curtis, B. L. (2018) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2018.05.005 · Cited by 196 · see citing works
Self-citation12-Step Interventions and Mutual Support Programs for Substance Use Disorders: An Overview
Donovan, D. M., Ingalsbe, M. H., Benbow, J., & Daley, D. C. (2013) · Social Work in Public Health · doi:10.1080/19371918.2013.774663 · Cited by 194 · see citing works
Proceedings of the 2008 ACM conference on Computer supported cooperative work
Begole, B., & McDonald, D. W. (2008) · doi:10.1145/1460563 · Cited by 191 · see citing works
AI chatbots not yet ready for clinical use.
Au Yeung, J., Kraljevic, Z., Luintel, A., Balston, A., Idowu, E., Dobson, R. J., & Teo, J. T. (2023) · Frontiers in Digital Health · doi:10.3389/fdgth.2023.1161098 · Cited by 176 · see citing works
Convergence of speech rate in conversation predicts cooperation
Manson, J. H., Bryant, G. A., Gervais, M. M., & Kline, M. A. (2013) · Evolution and Human Behavior · doi:10.1016/j.evolhumbehav.2013.08.001 · Cited by 167 · see citing works
Lord, S. P., Sheng, E., Imel, Z. E., Baer, J., & Atkins, D. C. (2015) · Behavior Therapy · doi:10.1016/j.beth.2014.11.002 · Cited by 144 · see citing works
Primary care physicians’ perspectives on the prescription opioid epidemic
Kennedy‐Hendricks, A., Busch, S. H., McGinty, E. E., Bachhuber, M. A., Niederdeppe, J., Gollust, S. E., Webster, D. W., Fiellin, D. A., & Barry, C. L. (2016) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2016.05.010 · Cited by 133 · see citing works
Systemic Racism and Substance Use Disorders
Farahmand, P., Arshed, A., & Bradley, M. V. (2020) · Psychiatric Annals · doi:10.3928/00485713-20201008-01 · Cited by 129 · see citing works
Evaluating Artificial Intelligence Responses to Public Health Questions
Ayers, J. W., Zhu, Z., Poliak, A., Leas, E. C., Dredze, M. H., Hogarth, M. A., & Smith, D. M. (2023) · JAMA Network Open · doi:10.1001/jamanetworkopen.2023.17517 · Cited by 115 · see citing works
Bian, N., Han, X., Sun, L., Lin, H., Lu, Y., He, B., Jiang, S., & Dong, B. (2024) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/32y85i5g9gso · Cited by 98 · see citing works
ChatGPT: The transformative influence of generative AI on science and healthcare
Varghese, J., & Chapiro, J. (2023) · Journal of Hepatology · doi:10.1016/j.jhep.2023.07.028 · Cited by 94 · see citing works
Linguistic mimicry and trust in text-based CMC
Scissors, L. E., Gill, A. J., & Gergle, D. (2008) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/1460563.1460608 · Cited by 93 · see citing works
Mirages. On Anthropomorphism in Dialogue Systems
Abercrombie, G., Curry, A., Dinkar, T., Rieser, V., & Talat, Z. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.290 · Cited by 83 · see citing works
The Unified and Holistic Method Gamma (γ) for Inter-Annotator Agreement Measure and Alignment
Mathet, Y., Widlöcher, A., & Métivier, J.-P. (2015) · Computational Linguistics · doi:10.1162/coli_a_00227 · Cited by 70 · see citing works
Studies of Depression and Anxiety Using Reddit as a Data Source: Scoping Review
Boettcher, N. (2021) · JMIR Mental Health · doi:10.2196/29487 · Cited by 70 · see citing works
“Abusing Addiction”: Our Language Still Isn’t Good Enough
Ashford, R. D., Brown, A. M., & Curtis, B. L. (2018) · Alcoholism Treatment Quarterly · doi:10.1080/07347324.2018.1513777 · Cited by 54 · see citing works
Self-citationKiang, M. V., Barnett, M. L., Wakeman, S. E., Humphreys, K. N., & Tsai, A. C. (2021) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2021.109081 · Cited by 54 · see citing works
Ogilvie, L., Prescott, J., & Carson, J. (2022) · European Addiction Research · doi:10.1159/000525959 · Cited by 51 · see citing works
Chat GPT-4 significantly surpasses GPT-3.5 in drug information queries
He, N., Yan, Y., Wu, Z., Cheng, Y., Liu, F., Li, X., & Zhai, S. (2023) · Journal of Telemedicine and Telecare · doi:10.1177/1357633x231181922 · Cited by 43 · see citing works
Diaz-Asper, C., Hauglid, M. K., Chandler, C., Cohen, A. S., Foltz, P. W., & Elvevåg, B. (2024) · American Psychologist · doi:10.1037/amp0001195 · Cited by 35 · see citing works
Snell‐Rood, C., Pollini, R. A., & Willging, C. E. (2021) · Psychiatric Services · doi:10.1176/appi.ps.202000312 · Cited by 33 · see citing works
Potential Value of the Insights and Lived Experiences of Addiction Researchers With Addiction
Stull, S. W., Smith, K. E., Vest, N. A., Effinger, D. P., & Epstein, D. H. (2021) · Journal of Addiction Medicine · doi:10.1097/adm.0000000000000867 · Cited by 31 · see citing works
Matthews, S. (2019) · Springer eBooks · doi:10.1007/978-3-030-02580-9_2 · Cited by 31 · see citing works
Valdez, D., & Patterson, M. S. (2022) · PLOS Digital Health · doi:10.1371/journal.pdig.0000143 · Cited by 30 · see citing works
Examining stigma relating to substance use and contextual factors in social media discussions
Chen, A., Johnny, S., & Conway, M. (2022) · Drug and Alcohol Dependence Reports · doi:10.1016/j.dadr.2022.100061 · Cited by 29 · see citing works
NLP for Maternal Healthcare: Perspectives and Guiding Principles in the Age of LLMs
Antoniak, M., Naik, A., Alvarado, C. S., Wang, L. L., & Chen, I. Y. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3658982 · Cited by 23 · see citing works
Miller‐Rosales, C., Morden, N. E., Brunette, M. F., Busch, S. H., Torous, J. B., & Meara, E. R. (2023) · JAMA Network Open · doi:10.1001/jamanetworkopen.2023.23741 · Cited by 22 · see citing works
Kjell, O., Kjell, K., & Schwartz, H. A. (2023) · doi:10.31234/osf.io/yfd8g · Cited by 15 · see citing works
Evaluating Risk Progression in Mental Health Chatbots Using Escalating PromptsPreprint
Heston, T. F. (2023) · medRxiv · doi:10.1101/2023.09.10.23295321 · Cited by 11 · see citing works
Giorgi, S., Bellew, D., Habib, D. R. S., Sedoc, J., Smitterberg, C. W., Devoto, A., Himelein-Wachowiak, M., & Curtis, B. L. (2024) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v18i1.31328 · Cited by 10 · see citing works
Self-citationExploring the ChatGPT platform with scenario-specific prompts for vaping cessation
Amin, S., Kawamoto, C. T., & Pokhrel, P. (2023) · Tobacco Control · doi:10.1136/tc-2023-058009 · Cited by 8 · see citing works
A linguistic analysis of dehumanization toward substance use across three decades of news articles
Giorgi, S., Habib, D. R. S., Bellew, D., Sherman, G. T., & Curtis, B. (2023) · Frontiers in Public Health · doi:10.3389/fpubh.2023.1275975 · Cited by 4 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 4
- Antoniak M, Naik A, Alvarado CS, Wang LL, Chen IY, 2023. Designing Guiding Principles For Nlp For healthcare: A case Study of Maternal Health. arXiv.:2312.11803. · arXiv
- Berglund L, Tong M, Kaufmann M, Balesni M, Stickland AC, Korbak T, Evans O, The Reversal curse: Llms trained on” a is b” Fail to learn” b is a”, arXiv preprint arXiv:2309.12288 (2023). · arXiv
- National Academies of Sciences, 2016. Division of Behavioral, Social Sciences, Board on Behavioral, Sensory Sciences, and Committee on the Science of Changing Behavioral Health Social Norms. Ending discrimination against people with mental and substance use disorders: The evidence for stigma change. National Academies Press. 27631043
- OpenAI, Introducing chatgpt. URL https://openai.com/blog/chatgpt.OpenAI,2023.Gpt-4TechnicalReport.arXiv.:2303.08774.
Cited by
Sorted by citations
The Application and Ethical Implication of Generative AI in Mental Health: Systematic Review
Wang, X., Zhou, Y., & Zhou, G. (2025) · JMIR Mental Health · doi:10.2196/70610 · Cited by 95 · see citing works
ChatGPT Clinical Use in Mental Health Care: Scoping Review of Empirical Evidence
Balan, R., & Gumpel, T. P. (2025) · JMIR Mental Health · doi:10.2196/81204 · Cited by 22 · see citing works
Large language models in global health
Ong, J. C. L., Ning, Y., Yang, R., Bitterman, D. S., Liu, X., Tham, Y. C., Collins, P. G. S., Jiménez, M., Mateen, B. A., Amissah-Arthur, K. N., et al. (2026) · Nature Health · doi:10.1038/s44360-025-00024-7 · Cited by 16 · see citing works
AI in substance use recovery: Innovation or risk? A call for caution
Ramadhani, E., Putri, R. D., Delano, V., Akhbar, M. T., Erdiansyah, E., & Kurniawan, M. (2024) · Psychiatry Research · doi:10.1016/j.psychres.2024.116193 · Cited by 5 · see citing works
Graham, C. M. (2025) · Journal of Health Organization and Management · doi:10.1108/jhom-02-2025-0100 · Cited by 4 · see citing works
SISTEM TANYA JAWAB BERBASIS CHATBOT WEBSITE MENGGUNAKAN GEMINI AI PADA DATA FIQIH KONTEMPORER
Helviansyah, T., Harahap, N. S., Irsyad, M., & Negara, B. S. (2025) · Journal of Information System Management (JOISM) · doi:10.24076/joism.2025v7i1.2082 · Cited by 4 · see citing works
Assessing Generative AI Chatbots for Alcohol Misuse Support: A Longitudinal Simulation Study
Uscher‐Pines, L., Sousa, J. L., Raja, P. V., Ayer, L. A., Mehrotra, A., Huskamp, H. A., & Busch, A. B. (2026) · NEJM AI · doi:10.1056/aics2500676 · Cited by 4 · see citing works
A Scoping Review of Large Language Model Chatbot Use for Alcohol and Other Drug Health Information
Harding, N., Bovopoulos, N., Caspi, D., Martin, C., McPhie, S., Bouadjenek, M. R., Aryal, S., & Hobbs, M. (2025) · Drug and Alcohol Review · doi:10.1111/dar.70068 · Cited by 3 · see citing works
de Vries, M., & Schaub, M. P. (2026) · Current Opinion in Psychiatry · doi:10.1097/yco.0000000000001088 · Cited by 3 · see citing works
Huang, H., DeLucia, A., Tiyyala, V. M., & Dredze, M. (2026) · Findings of the Association for Computational Linguistics: ACL 2026 · doi:10.18653/v1/2026.findings-acl.693 · Cited by 2 · see citing works
“These factors are especially important given the potential harm of incorrect medical information and that LLMs have been known to add irrelevant or sometimes dangerous text (Giorgi et al., 2024; Mei et al., 2023).”
Prevention of Alcohol-Associated Liver Disease
Burton, R., Leggio, L., Louvet, A., Sheron, N., & Singal, A. K. (2025) · The American Journal of Gastroenterology · doi:10.14309/ajg.0000000000003427 · Cited by 2 · see citing works
Umer, F., Mr, S., & Rahman, A. J. (2026) · BDJ Open · doi:10.1038/s41405-026-00462-9 · Cited by 2 · see citing works
Skin Disease Classification using Hybrid Framework and Automated Clinical Insights using LLM
Mounish, S. S., & Thanka, M. R. (2025) · International Conference on Augmented Intelligence and Sustainable Systems (ICAISS) · doi:10.1109/icaiss61471.2025.11042071 · Cited by 1 · see citing works
Phenotypes of stigma expressed by people who use drugs on Reddit
Bouzoubaa, L. A., Aghakhani, E., & Rezapour, R. (2025) · Social Science & Medicine · doi:10.1016/j.socscimed.2025.118889 · Cited by 1 · see citing works
Editorial: 8th biennial international drug and alcohol research society conference 2022
Onaivi, E. S. (2025) · Advances in Drug and Alcohol Research · doi:10.3389/adar.2024.13706
8th Biennial International Drug and Alcohol Research Society (IDARS) Conference 2022
(2025) · Frontiers Media SA eBooks · doi:10.3389/978-2-8325-6231-4
The Future of Pharma Training: AI-Driven Conversation for Assessment & Professional Growth
Surya, V., Tyagi, R., Rampally, V. S. K., Gonala, D., & Gonala, S. K. (2025) · doi:10.38124/ijisrt/25apr1781
Balan, R., & Gumpel, T. P. (2025) · doi:10.2196/preprints.81204
Possible risks of turning to tea-bots based on artificial intelligence in difficult life situations
Enikolopov, S. N., Boyko, O. M., Медведева, Т. Е., & Воронцова, О. Ю. (2025) · Медицинская психология в России · doi:10.24884/2219-8245-2025-17-4-69-82
A Scientometric Analysis of Generative AI in the Educational Context
Vázquez, J. P. G., Astorga-Vargas, M. A., Corral, L. E. V., Zatarain-Aceves, H., & Vega, J. E. S. (2026) · Springer eBooks · doi:10.1007/978-981-95-4871-2_18
Decker, M., Kamm, C., Burgoa, S., Rao, M., Mejia, M. C., Ramdin, C., Dean, A. F., Nasr, M., Nelson, L. S., & Sacca, L. (2026) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2026.113074
Yoo, D. W., Lalka, A., Peddi, J., Kim, H., & Hong, S. (2026) · doi:10.2196/preprints.92712
Generative AI Chatbot Responses to Suicide and Self-Harm: A Systematic Review (Preprint)
Kelly, D. A., Franz, P. J., Gutman, A., Sleiman, M., Dorsky, J., Singh, H., Appel, W., Wyatt, S. M., Low, D. M., Stirman, S. W., et al. (2026) · Journal of Medical Internet Research · doi:10.2196/103517
Zhou, S. C., Wang, Q., Hou, H., Bai, N., Lu, X., Zou, H., Luo, D., Liu, Q., Wang, X. Q., Wei, X., et al. (2026) · doi:10.2196/preprints.104685
Exploring the Role of AI as a Recovery Companion for People Who Use Drugs: Potential and Boundaries
Hsu, S.-C., Chiu, C.-H., Wang, H.-C., & Lin, X.-R. (2026) · Smart innovation, systems and technologies · doi:10.1007/978-3-032-23104-8_20
Evaluating Large Language Models in Response to Questions on Substance Use: Helpful or Harmful?
Maddams, S., Chen, S., Bitterman, D. S., Sharma, S., Hathaway, D. B., McGuire, P., Johnson, I. G., Wu, A. C., Prostko, S., & Suzuki, J. (2026) · Substance Use & Misuse · doi:10.1080/10826084.2026.2716917
Evaluating Harm Reduction Advice From Large Language Models
Choo, K., Smith, N., Ward, B., & Lundin, R. M. (2026) · Drug and Alcohol Review · doi:10.1111/dar.70272
Towards Reliable Evaluation of Factuality in Long-Form Text Generated by LLMs
Pham, T., Le, T., & Nguyen, H. T. (2026) · Procedia Computer Science · doi:10.1016/j.procs.2026.09.252
- Authors
- Giorgi, S., Isman, K., Liu, T., Fried, Z., Sedoc, J., & Curtis, B.
- Affiliations
- National Institute on Drug Abuse (US)
- New York University (US)
- Corresponding author
- Curtis, Brenda
- Venue
- Psychiatry Research
- Publisher
- Elsevier BV
- Published
- 2024-09
- Volume
- 339
- Pages
- 116058
- ISSN
- 0165-1781 (ISSN-L)
- PMC
- PMC11705880
- OpenAlex
- W4400037188
- Cited by
- 24
- References
- 50
- Citations by year
- 2026: 12 · 2025: 11 · 2024: 1
- FWCI
- 2.24
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Psychiatry Research (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion, cc-by-nc-nd)
- Funders
- National Institutes of Health
- National Institute on Drug Abuse
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Health Sciences › Medicine › Epidemiology › Substance Abuse Treatment and Outcomes
- Keywords
- generative AI, ChatGPT, Llama 2, substance use disorders, AI-generated medical advice, disinformation, suicidal ideation, clinical validation of AI, addiction treatment
- MeSH terms
- Artificial Intelligence, Humans, Substance-Related Disorders, Suicidal Ideation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
