Clinical decision support for bipolar depression using large language models
Open accessThe DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
Abstract
Management of depressive episodes in bipolar disorder remains challenging for clinicians despite the availability of treatment guidelines. In other contexts, large language models have yielded promising results for supporting clinical decisionmaking. We developed 50 sets of clinical vignettes reflecting bipolar depression and presented them to experts in bipolar disorder, who were asked to identify 5 optimal next-step pharmacotherapies and 5 poor or contraindicated choices. The same vignettes were then presented to a large language model (GPT4-turbo; gpt-4-1106-preview), with or without augmentation by prompting with recent bipolar treatment guidelines, and asked to identify the optimal next-step pharmacotherapy. Overlap between model output and gold standard was estimated. The augmented model prioritized the expert-designated optimal choice for 508/1000 vignettes (50.8%, 95% CI 47.7–53.9%; Cohen’s kappa = 0.31, 95% CI 0.28–0.35). For 120 vignettes (12.0%), at least one model choice was among the poor or contraindicated treatments. Results were not meaningfully different when gender or race of the vignette was permuted to examine risk for bias. By comparison, an un-augmented model identified the optimal treatment for 234 (23.0%, 95% CI 20.8–26.0%; McNemar’s p < 0.001 versus augmented model) of the vignettes. A sample of community clinicians scoring the same vignettes identified the optimal choice for 23.1% (95% CI 15.7–30.5%) of vignettes, on average; McNemar’s p < 0.001 versus augmented model. Large language models prompted with evidence-based guidelines represent a promising, scalable strategy for clinical decision support. In addition to prospective studies of efficacy, strategies to avoid clinician overreliance on such models, and address the possibility of bias, will be needed.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Yatham, L. N., Kennedy, S. H., Parikh, S. V., Schaffer, A., Bond, D. J., Frey, B. N., Sharma, V., Goldstein, B. I., Rej, S., Beaulieu, S., et al. (2018) · Bipolar Disorders · doi:10.1111/bdi.12609 · Cited by 1,852
Pacchiarotti, I., Bond, D. J., Baldessarini, R. J., Nolen, W. A., Grunze, H., Licht, R. W., Post, R. M., Berk, M., Goodwin, G. M., Sachs, G. S., et al. (2013) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2013.13020185 · Cited by 770
Self-citationMalhi, G. S., Bell, E., Bassett, D., Boyce, P., Bryant, R., Hazell, P., Hopwood, M., Lyndon, B., Mulder, R., Porter, R., et al. (2021) · Australian & New Zealand Journal of Psychiatry · doi:10.1177/0004867420979353 · Cited by 614
Zack, T., Lehman, E., Suzgun, M., Rodriguez, J. A., Celi, L. A., Gichoya, J., Jurafsky, D., Szolovits, P., Bates, D. W., Abdulnour, R.-E. E., et al. (2024) · The Lancet Digital Health · doi:10.1016/s2589-7500(23)00225-x · Cited by 510 · cited 1×
“An evolving literature suggests that when LLM ’ s are asked to perform clinical tasks, they may exhibit subtle biases – for example, in associating race with diagnoses when asked to generate clinical vignettes [16].”
BackgroundCreation and Adoption of Large Language Models in Medicine
Shah, N. H., Entwistle, D., & Pfeffer, M. A. (2023) · JAMA · doi:10.1001/jama.2023.14217 · Cited by 435 · cited 1×
“Large language models have demonstrated pro fi ciency across medicine [11], from responding to clinical examination questions to identifying rare or dif fi cult-to-solve clinical scenarios [12].”
BackgroundAlmanac — Retrieval-Augmented Language Models for Clinical Medicine
Zakka, C., Shad, R., Chaurasia, A., Dalal, A. R., Kim, J. L., Moor, M., Fong, R., Phillips, C., Alexander, K., Ashley, E., et al. (2024) · NEJM AI · doi:10.1056/aioa2300068 · Cited by 424
Jacobs, M., Pradier, M. F., McCoy, T. H., Perlis, R. H., Doshi-Velez, F., & Gajos, K. Z. (2021) · Translational Psychiatry · doi:10.1038/s41398-021-01224-x · Cited by 247
Self-citationUse of gpt-4 to diagnose complex clinical cases
Eriksen, A. V., Möller, S., & Ryg, J. (2024) · NEJM AI · doi:10.1056/aip2300031 · Cited by 200 · cited 1×
“Large language models have demonstrated pro fi ciency across medicine [11], from responding to clinical examination questions to identifying rare or dif fi cult-to-solve clinical scenarios [12].”
BackgroundAntidepressants in bipolar depression: An enduring controversy
Gitlin, M. J. (2018) · International Journal of Bipolar Disorders · doi:10.1186/s40345-018-0133-9 · Cited by 116 · cited 2×
“The discordance between community clinician performance and expert choices likely re fl ects the continuing dif fi culty in treatment decision-making for this aspect of bipolar disorder [2], with heterogeneity in expert opinion [3] as well as clinical practice [4, 5].”
Background
Cited by
Torous, J., Linardon, J., Goldberg, S. B., Sun, S., Bell, I., Nicholas, J., Hassan, L., Hua, Y., Milton, A., & Firth, J. (2025) · World Psychiatry · doi:10.1002/wps.21299 · Cited by 323
Large Language Models for Mental Health Applications: Systematic Review
Guo, Z., Lai, A., Thygesen, J. H., Farrington, J., Keen, T., & Li, K. (2024) · JMIR Mental Health · doi:10.2196/57400 · Cited by 269
“…focused on ChatGPT, exploring its utility across various scenarios such as clinical diagnosis [100,106,111], treatment planning [106,128,131], medication guidance [105,109,129], patient management [106], psychiatry examinations [118], and psychology education [102], among others [107,110,127,130].”
BackgroundA scoping review of large language models for generative tasks in mental health care
Hua, Y., Na, H., Li, Z., Liu, F., Fang, X., Clifton, D., & Torous, J. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01611-4 · Cited by 151
Luo, X., Tham, Y. C., Giuffrè, M., Ranisch, R., Daher, M., Lam, K., Eriksen, A. V., Hsu, C.-W., Ozaki, A., Moraes, F. Y. D., et al. (2025) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2025-113825 · Cited by 97
The Applications of Large Language Models in Mental Health: Scoping Review
Jin, Y., Liu, J., Li, P., Wang, B., Yan, Y., Zhang, H., Ni, C., Wang, J., Li, Y., Bu, Y., et al. (2025) · Journal of Medical Internet Research · doi:10.2196/69284 · Cited by 94
“These enhanced capabilities posit LLMs as potential tools for the detection and prediction of mental disorders through the analysis of extensive datasets, including social media content [30,32,33], EMRs [21,86], and counseling notes [87,88].”
Metadata
- Authors
- Perlis, R. H., Goldberg, J. F., Ostacher, M. J., & Schneck, C. D.
- Venue
- Neuropsychopharmacology
- Published
- 2024-03-13
- Volume
- 49
- Issue
- 9
- Pages
- 1412-1416
- ISSN
- 0893-133X (ISSN-L)
- 1740-634X
- OpenAlex
- W4392739577
- Cited by
- 67
- References
- 9
- FWCI
- 18.04
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Clinical decision support for bipolar depression using large language models. https://app.cochat.ai/papers/10.1038/s41386-024-01841-2
- The paper
- Perlis, R. H., Goldberg, J. F., Ostacher, M. J., & Schneck, C. D. (2024). Clinical decision support for bipolar depression using large language models. Neuropsychopharmacology. https://doi.org/10.1038/s41386-024-01841-2
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.