Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 175 of the 182 references it cites were found in a registry or an index; 96 of the 129 sources its claims cite were checked for support.

Multimodal biomedical AI

Open access

Abstract

The increasing availability of biomedical data from large biobanks, electronic health records, medical imaging, wearable and ambient biosensors, and the lower cost of genome and microbiome sequencing have set the stage for the development of multimodal artificial intelligence solutions that capture the complexity of human health and disease. In this Review, we outline the key applications enabled, along with the technical and analytical challenges. We explore opportunities in personalized medicine, digital clinical trials, remote monitoring and care, pandemic surveillance, digital twin technology and virtual health assistants. Further, we survey the data, modeling and privacy challenges that must be overcome to realize the full potential of multimodal artificial intelligence in health. Multimodal artificial intelligence models could unlock many exciting applications in health and medicine; this Review outlines the most promising uses and the technical pitfalls to avoid.

Overview

CoChat Summary

TL;DR

This Review surveys how multimodal biomedical AI—integrating omics, EHR, imaging, wearables, ambient sensors, and other data—could enable precision health, digital clinical trials, remote monitoring, pandemic surveillance, digital twins, and virtual health assistants. It also catalogs technical, data, and privacy challenges that must be overcome before these applications can be validated and deployed.

Key findings

  • Multimodal data can improve prediction over single modalities: a DETECT Health study follow-up found that jointly using self-reported symptoms and wearable sensor metrics reached an AUC of 0.80 (95% CI 0.73–0.86) for classifying COVID-19-positive versus COVID-19-negative status.
  • Large multimodal resources are expanding: UK Biobank enrolled over 500,000 participants with plans to follow for at least 30 years; All of Us enrolled about 400,000 diverse participants toward a 1 million target; MIMIC contains de-identified data from thousands of critical care patients and about 380,000 records in the table.
  • Fusion of modalities can outperform single-modality models: imaging plus EHR data improved pulmonary embolism detection, and a DeepMind EHR-based model projected 620,000 dimensions into 800 dimensions over a 6-h time frame to predict acute kidney injury.
  • The field faces major unresolved challenges: high dimensionality and “dataset blind spots,” missing data, selection and racial bias, privacy/re-identification risks, and the need for randomized trials to validate remote monitoring and virtual health assistants.

Methods

  • Design: narrative Review article; the authors state they outline key applications, technical and analytical challenges, and promising strategies, and explicitly do not discuss basic AI/machine learning concepts.
  • Data or sample: no primary dataset or systematic sample; the paper synthesizes published literature, examples, and existing multimodal resources such as UK Biobank, China Kadoorie Biobank, Biobank Japan, Million Veteran Program, TOPMed, All of Us, Project Baseline, American Gut Project, MIMIC, MIPACT, and the North American Prodrome Longitudinal Study.
  • Main analysis: conceptual and narrative synthesis of opportunities (personalized omics, digital clinical trials, remote monitoring, pandemic surveillance, digital twins, virtual health assistants) and challenges (modeling, data collection/harmonization, bias, privacy, data ownership).

Limitations

  • The paper states that many multimodal AI use cases require further validation and replication, that remote patient monitoring for hospital-at-home is “not yet validated” and needs randomized trials versus hospital admission, and that most virtual health assistant applications have been tested only in small observational studies and need randomized clinical trials.
  • The paper states that current methods are unlikely to be sufficient to overcome all major challenges and that further innovation is required; it also notes that false positives and clinician interpretation of risks were outside the scope of the Review.
  • Evident from the text: as a narrative Review, it does not report a systematic search strategy, formal inclusion/exclusion criteria, or quantitative meta-analysis, so its claims are illustrative rather than systematically synthesized.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1038/s41591-022-01981-2

Cites

175 of 175 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

80 Valid · 0 Not valid · 17 Not checked · 42 Own results and interpretations

40 more claims · 17 with their cited sources checked

Create a free account to see all 139 claims and every source verdict.

See all claims — free account
  • It may state two separate claims as one. 51
  • Probably not a substantial claim. 61

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

16 more claims about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Citations

Cites

Show all 182 references

Research works printed but not linked: 3

  1. Zhang, X., Zeman, M., Tsiligkaridis, T. & Zitnik, M. Graph-guided network for irregularly sampled multivariate time series. In International Conference on Learning Representation (ICLR, 2022).
  2. Luo, Z. et al. Computer vision-based descriptive analytics of seniors’ daily activities for long-term health monitoring. In Proc. Machine Learning Research Vol. 85, 1–18 (PMLR, 2018).
  3. Guu, K., Lee, K., Tung, Z., Pasupat, P. & Chang, M. Retrieval augmented language model pre-training. In Proc. 37th International Conference on Machine Learning (eds. Iii, H. D. & Singh, A.) vol. 119, 3929–3938 (PMLR, 13–18 July 2020).

Cited by

See all 1,365

Metadata

Authors
Acosta, J. N., Falcone, G. J., Rajpurkar, P., & Topol, E. J.
Affiliations
  • Yale University (US)
  • Harvard University (US)
  • Scripps Research Institute (US)
Corresponding author
  • Rajpurkar, Pranav
  • Topol, Eric J.
Venue
Nature Medicine
Published
2022-09
Volume
28
Issue
9
Pages
1773-1784
ISSN
  • 1078-8956 (ISSN-L)
  • 1546-170X
OpenAlex
W4295951577
Cited by
1,365
References
182
Citations by year
2026: 474 · 2025: 387 · 2024: 306 · 2023: 186 · 2022: 12
FWCI
15.28
Type
journal article
Language
en
Open access
Licence
Locations
  • Nature Medicine (publishedVersion, other-oa)
  • PubMed (publishedVersion)
Funders
National Center for Advancing Translational Science (NCATS)/National Institutes of Health
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
Keywords
multimodal AI, biomedical data integration, biobanks, electronic health records, diagnostic imaging, wearable biosensors, microbiome sequencing, personalized medicine, remote patient monitoring, pandemic surveillance, digital twin, virtual health assistants, data privacy
MeSH terms
Artificial Intelligence, Humans, Privacy, Electronic Health Records, Pandemics

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-12, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-12 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Multimodal biomedical AI. https://app.cochat.ai/papers/10.1038/s41591-022-01981-2
The paper
Acosta, J. N., Falcone, G. J., Rajpurkar, P., & Topol, E. J. (2022). Multimodal biomedical AI. Nature Medicine. https://doi.org/10.1038/s41591-022-01981-2

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.