The DOI matches its Crossref record; 202 of the 202 references it cites were found in a registry or an index; 44 of the 73 sources its claims cite were checked for support.
Multimodal Large Language Models in Health Care: Applications, Challenges, and Future Outlook.
Open accessPublished version of a preprint
Abstract
In the complex and multidimensional field of medicine, multimodal data are prevalent and crucial for informed clinical decisions. Multimodal data span a broad spectrum of data types, including medical images (eg, MRI and CT scans), time-series data (eg, sensor data from wearable devices and electronic health records), audio recordings (eg, heart and respiratory sounds and patient interviews), text (eg, clinical notes and research articles), videos (eg, surgical procedures), and omics data (eg, genomics and proteomics). While advancements in large language models (LLMs) have enabled new applications for knowledge retrieval and processing in the medical field, most LLMs remain limited to processing unimodal data, typically text-based content, and often overlook the importance of integrating the diverse data modalities encountered in clinical practice. This paper aims to present a detailed, practical, and solution-oriented perspective on the use of multimodal LLMs (M-LLMs) in the medical field. Our investigation spanned M-LLM foundational principles, current and potential applications, technical and ethical challenges, and future research directions. By connecting these elements, we aimed to provide a comprehensive framework that links diverse aspects of M-LLMs, offering a unified vision for their future in health care. This approach aims to guide both future research and practical implementations of M-LLMs in health care, positioning them as a paradigm shift toward integrated clinical AI systems.
Overview
CoChat Summary
## TL;DR
This paper is a narrative review and perspective on multimodal large language models (M-LLMs) in health care, covering their principles, applications, challenges, and future directions. It argues that M-LLMs can extend unimodal LLMs by integrating text, images, audio, video, temporal, and omics data, but real-world use still faces major technical, ethical, and evaluation hurdles.
## Key findings
- The paper identifies 4 core stages in an M-LLM architecture: modality-specific encoding, embedding alignment and fusion, contextual understanding and cross-modal interactions, and decision-making/output generation.
- It argues that current unimodal LLMs are insufficient for medicine because clinical decision-making often combines multiple data types over time, including images, EHRs, laboratory results, audio, and omics data.
- The review highlights wide application areas for M-LLMs, including medical imaging, temporal monitoring, audio analysis, video interpretation, text/EHR analysis, and omics-driven personalized medicine.
- It reports that a general-domain M-LLM study found that such models can “inherently understand medical images” and, in some contexts, even outperform specialized task-specific pretraining methods; the paper uses CONCH as a use case for zero-shot cross-modal retrieval in histopathology.
## Methods
- Narrative review/perspective article synthesizing foundational principles, current and potential applications, technical and ethical challenges, and future outlook for medical M-LLMs.
- The paper organizes the discussion by modality and task: medical images, temporal data, audio, text, video, omics data, and any-to-any M-LLMs.
- It includes a use case example using the CONCH model, run with pretrained weights from Hugging Face on a Linux machine with an NVIDIA GeForce GTX 1080 Ti, in a Flask-based demo for zero-shot cross-modal retrieval on histopathology slides.
## Limitations
- The paper states that recent studies on M-LLMs in health care are limited by a narrow range of modalities, largely theoretical discussion, and limited exploration of solutions to technical barriers.
- The conclusions note that M-LLMs still require further evaluation and benchmarking in real-world medical settings, and that progress toward sophisticated multimodal outputs remains comparatively slow.
- Evident from the text: this is a conceptual review rather than an empirical study, so it does not report original clinical outcome data or a systematic quantitative comparison across models.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures






Verification
Claims the paper makes
29 Valid · 13 Not valid · 12 Not checked · 69 Own results and interpretations
Some cited sources support this claim and some don’t — read each source’s verdict below.
In the paper
“Third, although the challenges of integrating diverse data types into M-LLMs are recognized, there is limited exploration of possible solutions or ongoing research aimed at overcoming these technical barriers [38,39].”
§ Multimodal Large Language Models in Health Care: Applications, Challenges, and Future Outlook > Introduction
What the sources say
- Partly supportedcites [38] sourceExperimental — accuracy not yet measured
The ‘Barriers and Limitations’ section identifies significant technical challenges in integrating text, images, video, and sound into M-LLMs. It does not assess whether recent studies have explored solutions or ongoing research to a limited extent; the Conclusions even suggest developing new tokenization strategies as a possible response. No page numbers are available in our copy. (LLM, conf=0.84)
- Disputedcites [39] sourceExperimental — accuracy not yet measured
The “Multi-modal LLM” section (page unavailable) describes ongoing research combining images, audio, and text, while acknowledging limited data availability. The “Cross-modal dental diagnosis with LLMs” section (page unavailable) also explores a way to combine text and images for dental diagnosis. The paper therefore does not support the attributed lack of exploration of solutions or ongoing research. (LLM, conf=0.85)
Checked by a reasoning model — may be wrong.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“While this unimodal focus on text-based operation has been transformative in the medical field, it does not fully capture the complex and diverse nature of health care practice [8].”
§ Multimodal Large Language Models in Health Care: Applications, Challenges, and Future Outlook > Introduction
What the sources say
- Partly supportedcites [8] sourceExperimental — accuracy not yet measured
The main text says clinical practice requires more than responding accurately to queries and includes practical tasks that LLMs are not equipped to replace. That supports the claim that text-based operation does not capture the full scope of healthcare. But the paper does not describe LLMs as uniformly unimodal: it says GPT-4 accepts images as well as text. It also discusses potential benefits in medicine rather than establishing that text-based operation has already been transformative there. (LLM, conf=0.86)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: unclear — It is unclear whether it could be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“First, the range of data modalities examined is often restricted to text, images, videos, and audio [38], with some studies focusing narrowly on a limited number of clinical applications [39].”
§ Multimodal Large Language Models in Health Care: Applications, Challenges, and Future Outlook > Introduction
What the sources say
- Partly supportedcites [38] sourceExperimental — accuracy not yet measured
The ‘Multimodal Analysis’ section (page unavailable) lists text, images, audio, and video as data types that future M-LLMs could analyze. The ‘Barriers and Limitations’ section says multimodal training data sets are limited, but the paper does not establish that recent studies often restrict the modalities they examine to those four types. (LLM, conf=0.82)
- Partly supportedcites [39] sourceExperimental — accuracy not yet measured
In “Cross-modal dental diagnosis with LLMs” (page unavailable), the paper says most dental AI studies have used image-only approaches, supporting the point that modalities studied have been limited. “Multi-modal LLM” (page unavailable) discusses text, images, and audio in medical learning, but the paper does not establish that health-care M-LLM studies are often restricted to the four listed modalities, including video. (LLM, conf=0.82)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“However, there is a gap in the development of large audio models specifically tailored for medical applications [103].”
§ Multimodal Large Language Models in Health Care: Applications, Challenges, and Future Outlook > Applications > Audio
What the sources say
- Could not verifycites [103] sourceExperimental — accuracy not yet measured
The Introduction says large models have advanced more gradually in audio than in language, images, and video. The Related Surveys and Differences section mentions healthcare surveys but does not identify a gap in developing large audio models specifically for medical applications. Searches for medical and healthcare applications found no passage making that claim; page numbers were unavailable in our copy. (LLM, conf=0.73)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Moreover, nontextual data types play a crucial role in diagnosis, effective treatment planning, research, and patient care [9-11].”
§ Multimodal Large Language Models in Health Care: Applications, Challenges, and Future Outlook > Introduction
What the sources say
- Supportedcites [9] source
The Introduction (page not provided) identifies medical images and omics data as relevant to diagnostics, therapy planning, and patient monitoring. “Towards Future Human-AI Interfaces for Multi-Modal Causability” (page not provided) says that integrating images, omics, and other data is needed to understand diseases and biological processes. Together, these passages support the broad claim about nontextual data in research and patient care. (LLM, conf=0.85)
- Partly supportedcites [10] sourceExperimental — accuracy not yet measured
The Introduction (page unavailable) describes using patients’ genomic data to predict early warning signals of cancer, and the Biologic significance section (page unavailable) identifies potential drug targets for early intervention. This supports a narrower point about nontextual molecular data in cancer prediction and treatment research, but not the broad claim about nontextual data types across diagnosis, effective treatment planning, research, and patient care. The paper’s “multi-modal” refers to alternative biomarker configurations, not different data types. (LLM, conf=0.83)
- Not checked — no full text of the sourcecites [11] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
34 more claims · 13 with their cited sources checked
Create a free account to see all 123 claims and every source verdict.
- It may state two separate claims as one. 26
- Probably not a substantial claim. 74
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“LLMs, such as GPT and BERT, are trained on massive datasets and have the unique capability of understanding both structured and unstructured text (11,81).”
From Generative artificial intelligence in predictive analysis of diabetes and its complications: a narrative review (2025) · cites this paper as [81]
Experimental — accuracy not yet measured
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 1×
“For example, image encoders (eg, transformers [54] and convolutional neural networks (CNNs) [55,56]) are designed to capture fine-grained patterns or anomalies crucial for diagnosis, whereas text encoders (BERT [15]) aim to comprehend complex medical terminology and patient histories.”
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 1×
“For example, image encoders (eg, transformers [54] and convolutional neural networks (CNNs) [55,56]) are designed to capture fine-grained patterns or anomalies crucial for diagnosis, whereas text encoders (BERT [15]) aim to comprehend complex medical terminology and patient histories.”
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 2×
“Time-series encoders (such as transformer-based models [15,59-61] and LSTM [12]) are intended to detect critical changes over time, signaling the need for urgent medical intervention.”
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472 · cited 1×
“For example, image encoders (eg, transformers [54] and convolutional neural networks (CNNs) [55,56]) are designed to capture fine-grained patterns or anomalies crucial for diagnosis, whereas text encoders (BERT [15]) aim to comprehend complex medical terminology and patient histories.”
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204 · cited 1×
“Knowledge distillation is another approach that involves training a smaller “student” model to replicate the behavior of a larger “teacher” model, requiring less computational power while retaining performance [161].”
LoRA: Low-Rank Adaptation of Large Language Models
Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., & Chen, W. (2021) · arXiv · doi:10.48550/arxiv.2106.09685 · Cited by 13,453 · cited 1×
“Parameter-efficient fine-tuning methods such as adapter layers help reduce the computational load by fine-tuning only a subset of the model’s parameters [157,158].”
BioBERT: a pre-trained biomedical language representation model for biomedical text mining
Lee, J., Yoon, W., Kim, S., Kim, D., Kim, S., So, C. H., & Kang, J. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btz682 · Cited by 7,218 · cited 1×
“Examples of biomedical text LLMs include BiMediX [104], BioBERT [105], PubMedBERT [106], and ClinicalBERT [20].”
Han, S., Mao, H., & Dally, W. J. (2015) · arXiv · doi:10.48550/arxiv.1510.00149 · Cited by 4,662 · cited 1×
“In addition, quantization approaches can address the scalability and computational demands by shifting toward quantized versions of existing models using curated, domain-specific data rather than pretraining from scratch [159].”
In-Datacenter Performance Analysis of a Tensor Processing Unit
Jouppi, N. P., Young, C., Patil, N., Patterson, D., Agrawal, G., Bajwa, R., Bates, S., Bhatia, S., Boden, N., Borchers, A., et al. (2017) · Proceedings of the 44th Annual International Symposium on Computer Architecture · doi:10.1145/3079856.3080246 · Cited by 4,537 · cited 1×
“Additional optimization techniques, such as model compression [165] and hardware acceleration using graphics processing units (GPUs) or tensor processing units (TPUs) [166], further enhance efficiency.”
Large language models encode clinical knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2023) · Nature · doi:10.1038/s41586-023-06291-2 · Cited by 4,083 · cited 1×
“Subsequently, a series of medical BERT models were proposed to accelerate medical research [6,17-20].”
Research works printed but not linked: 25
- VaswaniAShazeerNParmarNUszkoreitJJonesLGomezANKaiserLPolosukhinIAttention is all you needProceedings of the 31st International Conference on Neural Information Processing Systems2017NIPS'17December 4-9, 2017Long Beach, CA
- Devlin, J, arXiv
- Touvron, H, arXiv
- Chung, HW, arXiv
- Huang, W, arXiv
- Zhu, D, arXiv
- Su, Y, arXiv
- Koh, JY, arXiv
- Alayrac, JB, arXiv
- Rasheed, H, arXiv
- WangZZhangQYuHHuSJinXGongZChenHMulti-level protein structure pre-training with prompt learningProceedings of the Eleventh International Conference on Learning Representations2023ICLR 2023May 1-5, 2023Kigali, Rwanda
- Zhou, H, arXiv
- Yordanov, Y, arXiv
- HewittJManningCDA structural probe for finding syntax in word representationsProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies2019NAACL-HLT 2019June 2-7, 2019Minneapolis, MN
- Gerke, S, 2020, Artificial Intelligence in Healthcare, 295
- Lee, N, arXiv
- Reddy, AG, arXiv
- Carlini, N, arXiv
- Aghajanyan, A, arXiv
- Huang, Y, arXiv
- Wang, X, arXiv
- Fu, C, arXiv
- Yang, X, arXiv
- Ge, W, arXiv
- Li, S, arXiv
Cited by
Items 301–330 of 330
Campos, C. G. D. C., de Araújo, C. M., Luiz, L. B. D. P., Bubna, D. P., Schröder, Â. G. D., Baratto‐Filho, F., Perin, C. P., de Araujo, B. M. D. M., & Matos, T. D. P. (2026) · Odontology · doi:10.1007/s10266-026-01534-w
Al-Azzam, Q. H. A., Balachandran, W., & Hunaiti, Z. (2026) · AI in Medicine · doi:10.3390/aimed1030024 · Cited by 1
Multimodal large language models for ADOS-M1 behavioral assessment
Liu, W., Pan, Y., Zhang, D., Deng, H., Zou, X., & Li, M. (2026) · Neurocomputing · doi:10.1016/j.neucom.2026.135024
Cross-Modal Multi-Level Backdoor Poisioning in Multimodal Federated Learning
Ibrar, K., Xhafa, F., & Palmieri, F. (2026) · Lecture notes on data engineering and communications technologies · doi:10.1007/978-3-032-29561-3_15
Cheng, P., Lyu, M., Chen, Z., & Wu, Y. (2026) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocag146 · Cited by 1
Modeling Clinician Preferences in AI-Assisted Drafting: A Human-in-the-Loop Adaptation Theory
Rahman, S., Zaki, A., & Huda, N. (2024) · Journal of Health Informatics and Digital Systems · doi:10.68159/z142477894 · Cited by 1
Beyond ChatGPT: A Roadmap for Agentic AI in Mineral Processing
Chelgani, S. C. (2026) · Mineral Processing and Extractive Metallurgy Review · doi:10.1080/08827508.2026.2731556
“In medicine, similar systems integrate medical images, laboratory results, and physicians’ notes into a single reasoning process to support diagnosis (Acosta et al. 2022; AlSaad et al. 2024).”
Multimodal Artificial Intelligence in Lung Cancer: From Data Integration to Precision Oncology
Chakrabarti, T., Mansour, A. G., Wu, X., Arias-Romero, J., Mambetsariev, I., Chang, N., Santos, S. D., Mirzapoiazova, T., Fricke, J., Kim, J., et al. (2026) · Cancers · doi:10.3390/cancers18182953
“Techniques such as prompt-based multimodal fusion, hybrid fusion, gated fusion, and tensor fusion can assist in alignment [88].”
Singhania, R., & Chikop, S. A. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7459817
Large Language Model Benchmarks in Medical Tasks
Yan, L. K. Q., Niu, Q., Li, M., Zhang, Y., Yin, C. H., Fei, C., Peng, B., Bi, Z., Feng, P., Chen, K., et al. (2026) · Medicine Advances · doi:10.1002/med4.70085 · Cited by 1
“These include audio, video, ECG data, and various omics datasets[163].”
Kazmina, A., Venhoff, C., Wong, B., Hong, S., Xie, Y., & Yi, M. Y. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-38059-3_47
From Prompts To Masks: Assessing Gemini Image Models for Tooth Segmentation on Panoramic Radiographs
Mine, Y., Taji, T., Okazaki, S., Peng, T.-Y., & Kakimoto, N. (2026) · International Dental Journal · doi:10.1016/j.identj.2026.111171
Jeffs, L. P., & Preziosi, P. (2026) · Nursing Outlook · doi:10.1016/j.outlook.2026.102898
Li, Z., Chen, K., Ji, Y., Wu, H., & Lu, Q. (2026) · Skin · doi:10.2738/skin.2026.0011
Camlet, A., Kusiak, A., & Świetlik, D. (2026) · Journal of Clinical Medicine · doi:10.3390/jcm15197387
Villarreal‐Zegarra, D., Paredes-Gonzales, Y., & García-Serna, J. (2026) · PLOS mental health. · doi:10.1371/journal.pmen.0000702 · Cited by 1
“…process natural language, summarize unstructured clinical information, generate candidate items or prompts, assist with linguistic refinement, classify open-text responses, extract structured information from text, and support integration across text, audio, image, and sensor-derived data [1–4].”
InfluentialTRACE-PSG: Traceable Sleep Diagnosis Support with Pointer-Grounded Evidence
Wu, S., Tang, S., Liu, C., et al. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-38098-2_61
Wang, G., Bi, H., Zhang, Y., Wang, W., Zhu, Y., Wang, C., & Yang, X. (2026) · Diagnostics · doi:10.3390/diagnostics16193100
Oliveira, H. S., & Oliveira, H. P. (2026) · Machine Learning with Applications · doi:10.1016/j.mlwa.2026.101013
Li, C., Yu, W., & Zhang, W. (2026) · Journal of Health Informatics and Digital Systems · doi:10.68159/k304401829
Caimi, E., Vaccari, S., Lupacchini, A. M., Cavallero, M. F., Furlan, S., Giuli, R. D., Bandi, V., Catania, B., Veronesi, A., Klinger, M. E. A., et al. (2026) · Aesthetic Plastic Surgery · doi:10.1007/s00266-026-06283-3
Facial Pigmented Lesion Classification with SwiGLU-Enhanced Swin Transformer V2
Rubaye, M. A., Memiş, A., & Naser, B. M. (2026) · World Congress on Medical Physics and Biomedical Engineering, September 7 - 12, 2009, Munich, Germany · doi:10.1007/978-3-032-39674-7_23
Chen, J., Xu, C., Zhang, Y., Zhou, T., & Shao, Q. (2026) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3832014 · Cited by 1
“Despite these advances, most systems function as general-purpose chat interfaces or remain weakly grounded in the physical and interactional context of caregiving [7].”
Rahman, M. R., Haque, A. U., & Rahman, M. S. (2026) · Research Square · doi:10.21203/rs.3.rs-10083317/v1
Reyed, R. M., & Prabhakar, P. K. (2026) · Elsevier eBooks · doi:10.1016/b978-0-443-27722-1.00012-2
Rossi, D. D., Fiore, L., Citarella, A. A., Marco, F. D., Wang, H., Cleland, I., Zheng, H., & Tortora, G. (2026) · Companion Publication of the International Conference on Multimodal Interaction (ICMI Companion) · doi:10.1145/3776591.3836643
Swinckels, L., Rabelo, K. A., Delamare, E. L., Loos, B. G., Lahoud, P., Bijwaard, H., de Keijzer, A., Kim, J., & Bruers, J. J. M. (2026) · Journal of Medical Internet Research · doi:10.2196/87781
“Multimodal LLMs (M-LLMs) are designed to integrate and interpret complex datasets from multiple modalities on which clinical decisions in health care often rely [38].”
Chai, Z., Zhou, Y., & Yu, H. (2026) · International Conference on Automation and Computing (ICAC) · doi:10.1109/icac70325.2026.11712318
“…[5], [10], [16] Domain gap; tokenization and temporal representation mismatch; hallucination; limited physiological grounding [5], [19], [20], [22] Integration complexity; multimodal alignment and standardisation constraints; edge–cloud deployment burden; no dementia benchmarks [5],…”
YAPAY ZEKA ÇAĞINDA DİJİTAL SAĞLIK OKURYAZARLIĞI VE HASTA HAKLARI
Deniz, E. (2026) · Sağlıkta Performans ve Kalite Dergisi · doi:10.70915/spkd.1854900
Yan, Y., Li, G., Chen, T., Yan, J., Wang, C., Yuan, A., Zhang, J., & Pu, J. (2026) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2026.1962796
Metadata
- Authors
- AlSaad, R., Abd-alrazaq, A., Boughorbel, S., Ahmed, A., Renault, M.-A., Damseh, R., & Sheikh, J.
- Affiliations
- Weill Cornell Medical College in Qatar (QA)
- Hamad bin Khalifa University (QA)
- United Arab Emirates University (AE)
- Corresponding author
- AlSaad, Rawan
- ORCID
- Venue
- Journal of medical Internet research
- Publisher
- JMIR Publications Inc.
- Published
- 2024-09-25
- Volume
- 26
- Pages
- e59505
- ISSN
- 1438-8871 (ISSN-L)
- PMC
- PMC11464944
- OpenAlex
- W4402155831
- Cited by
- 349
- References
- 175
- Citations by year
- 2026: 202 · 2025: 139 · 2024: 8
- FWCI
- 32.47
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Journal of Medical Internet Research (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Keywords
- multimodal large language models, multimodal data integration, health care applications, medical images, wearable sensor data, electronic health records, clinical notes, audio recordings, surgical video, omics data, clinical decision support, ethical considerations, multimodal fusion
- MeSH terms
- Delivery of Health Care, Humans, Natural Language Processing, Electronic Health Records
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-09, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-09 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.