CoChat

Foundation models for generalist medical artificial intelligence

Open access

The DOI matches its Crossref record; 70 of the 70 references it cites were found in a registry or an index; 23 of the 33 sources its claims cite were checked for support.

Abstract

The exceptionally rapid development of highly flexible, reusable artificial intelligence (AI) models is likely to usher in newfound capabilities in medicine. We propose a new paradigm for medical AI, which we refer to as generalist medical AI (GMAI). GMAI models will be capable of carrying out a diverse set of tasks using very little or no task-specific labelled data. Built through self-supervision on large, diverse datasets, GMAI will flexibly interpret different combinations of medical modalities, including data from imaging, electronic health records, laboratory results, genomics, graphs or medical text. Models will in turn produce expressive outputs such as free-text explanations, spoken recommendations or image annotations that demonstrate advanced medical reasoning abilities. Here we identify a set of high-impact potential applications for GMAI and lay out specific technical capabilities and training datasets necessary to enable them. We expect that GMAI-enabled applications will challenge current strategies for regulating and validating AI devices for medicine and will shift practices associated with the collection of large medical datasets. This review discusses generalist medical artificial intelligence, identifying potential applications and setting out specific technical capabilities and training datasets necessary to enable them, as well as highlighting challenges to its implementation.

Overview

CoChat Summary

TL;DR

The paper proposes generalist medical AI (GMAI), a class of foundation models that would use self-supervision on large, diverse medical data to perform dynamically specified tasks, flexibly combine modalities, and reason with formal medical knowledge. It outlines six high-impact use cases and argues that GMAI will require new approaches to validation, regulation, bias auditing, privacy, and data collection.

Key findings

  • Defines GMAI by three capabilities: dynamic task specification, flexible multimodal inputs/outputs, and formal representation of medical knowledge for advanced reasoning; existing models such as GPT-3 or PaLM show some medical reasoning but are not multimodal and do not reliably generate factual statements.
  • Proposes six use cases: grounded radiology reports, augmented procedures, bedside decision support, chatbots for patients, text-to-protein generation, and interactive note-taking.
  • Notes current medical AI is narrow: of more than 500 FDA-approved AI models for clinical medicine, most were approved for only 1 or 2 narrow tasks; CheXzero can detect dozens of diseases in chest X-rays without explicit labels; GPT-3 powered more than 300 apps within months of release.
  • Flags scale and cost: PaLM, a 540-billion-parameter model, required an estimated 8.4 million hours’ worth of tensor processing unit v4 chips, using roughly 3,000 to 6,000 chips at a time, amounting to millions of dollars; training each model has been estimated to generate up to hundreds of tons of CO₂ equivalent; one study recommended 20 times more tokens than parameters for optimal performance.

Methods

  • Perspective/review article; no primary data, sample, or experimental analysis.
  • Synthesizes recent foundation-model advances and proposes the GMAI paradigm, identifying technical capabilities, training-data needs, use cases, and challenges.
  • Draws on cited prior work and examples (e.g., GPT-3, Gato, CheXzero, PaLM) rather than new experiments.

Limitations

  • Stated: GMAI will be uniquely difficult to validate because it can carry out previously unseen tasks, making failure modes hard to anticipate; social bias can increase with model scale; privacy risks include memorization of training data and prompt attacks; scale brings high computational, financial, and environmental costs; verification may require multidisciplinary panels; data collection/access and regulatory/liability frameworks are unresolved.
  • Evident from the text: no GMAI model is built or empirically evaluated here; the use cases are prospective and illustrative; many capabilities depend on future advances in multimodal self-supervision, in-context learning, and medical knowledge integration.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1038/s41586-023-05881-4

Cites

70 of 70 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Regulations: Application approval; validation; audits; community-based challenges; analyses of biases, fairness and diversity
Regulations: Application approval; validation; audits; community-based challenges; analyses of biases, fairness and diversityCC-BY · From the PDF
Fig. 2 Illustration of three potential applications of GMAI. a , GMAI could enable versatile and self-explanatory bedside decision support. b , Grounded radiology reports are equipped with clickable links for visualizing each finding. c , GMAI has the potential to classify phenomena that were never encountered before during model development. In augmented procedures, a rare outlier finding is explained with step-by-step reasoning by leveraging medical domain knowledge and topographic context. The presented example is inspired by a case report 58 . Image of the fistula in panel c adapted from ref. 58, CC BY 3.0.
Fig. 2 Illustration of three potential applications of GMAI. a , GMAI could enable versatile and self-explanatory bedside decision support. b , Grounded radiology reports are equipped with clickable links for visualizing each finding. c , GMAI has the potential to classify phenomena that were never encountered before during model development. In augmented procedures, a rare outlier finding is explained with step-by-step reasoning by leveraging medical domain knowledge and topographic context. The presented example is inspired by a case report 58 . Image of the fistula in panel c adapted from ref. 58, CC BY 3.0.CC-BY · From the PDF

Verification

Claims the paper makes

17 Valid · 3 Not valid · 5 Not checked · 59 Own results and interpretations

32 more claims · 2 with their cited sources checked

Create a free account to see all 84 claims and every source verdict.

See all claims — free account
  • Probably not a substantial claim. 35
  • It may state two separate claims as one. 25

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

5 more claims about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

1 of 1

Citations

Cites

Show all 70 references

Research works printed but not linked: 2

  1. Reed, S. et al. A generalist agent. In Transactions on Machine Learning Research (2022). This study presented Gato, a generalist model that can carry out a variety of tasks across modalities such as chatting, captioning images, playing video games and controlling a robot arm.
  2. Brown, T. et al. Language models are few-shot learners. In Advances in Neural Information Processing Systems (eds Larochelle, H. et al.) 33, 1877–1901 (2020). This study presented the language model GPT-3 and discovered that large language models can carry out in-context learning.

Cited by

See all 2,028

Metadata

Authors
Moor, M., Banerjee, O., Abad, Z. S. H., Krumholz, H. M., Leskovec, J., Topol, E. J., & Rajpurkar, P.
Affiliations
  • Stanford University (US)
  • Harvard University (US)
  • University of Toronto (CA)
  • Yale New Haven Hospital (US)
  • Scripps Research Institute (US)
Corresponding author
  • Topol, Eric J.
  • Rajpurkar, Pranav
Venue
Nature
Published
2023-04-12
Volume
616
Issue
7956
Pages
259-265
ISSN
  • 0028-0836 (ISSN-L)
  • 1476-4687
OpenAlex
W4365143687
Cited by
2,028
References
50
Citations by year
2026: 753 · 2025: 645 · 2024: 445 · 2023: 181
FWCI
23.78
Type
journal article
Language
en
Open access
Licence
Locations
  • Nature (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
Funders
  • National Science Foundation (1934578, 1835598, UL1TR001114, 1918940)
  • National Institutes of Health (UL1TR001114)
  • Defense Advanced Research Projects Agency (HR00112190039, N660011924033)
  • Advanced Research Projects Agency
  • Multidisciplinary University Research Initiative
  • Wu Tsai Neurosciences Institute, Stanford University
  • National Human Genome Research Institute (U54HG010426)
  • National Institute of Neurological Disorders and Stroke (R61NS11865)
  • National Center for Advancing Translational Sciences (grant UL1TR001114)
  • Army Research Office (W911NF-16-1-0342, W911NF-16-1-, W911NF)
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
Keywords
foundation models, self-supervised learning, multimodal medical data, medical reasoning, explainable AI, health data collection, artificial intelligence in healthcare
MeSH terms
Unsupervised Machine Learning, Artificial Intelligence, Diagnostic Imaging, Humans, Medicine, Genomics, Electronic Health Records, Datasets as Topic

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Foundation models for generalist medical artificial intelligence. https://app.cochat.ai/papers/10.1038/s41586-023-05881-4
The paper
Moor, M., Banerjee, O., Abad, Z. S. H., Krumholz, H. M., Leskovec, J., Topol, E. J., & Rajpurkar, P. (2023). Foundation models for generalist medical artificial intelligence. Nature. https://doi.org/10.1038/s41586-023-05881-4

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.