Multimodal Large Language Models in Health Care: Applications, Challenges, and Future Outlook.
Open accessPublished version of a preprint
The DOI matches its Crossref record; 202 of the 202 references it cites were found in a registry or an index; 44 of the 73 sources its claims cite were checked for support.
Abstract
In the complex and multidimensional field of medicine, multimodal data are prevalent and crucial for informed clinical decisions. Multimodal data span a broad spectrum of data types, including medical images (eg, MRI and CT scans), time-series data (eg, sensor data from wearable devices and electronic health records), audio recordings (eg, heart and respiratory sounds and patient interviews), text (eg, clinical notes and research articles), videos (eg, surgical procedures), and omics data (eg, genomics and proteomics). While advancements in large language models (LLMs) have enabled new applications for knowledge retrieval and processing in the medical field, most LLMs remain limited to processing unimodal data, typically text-based content, and often overlook the importance of integrating the diverse data modalities encountered in clinical practice. This paper aims to present a detailed, practical, and solution-oriented perspective on the use of multimodal LLMs (M-LLMs) in the medical field. Our investigation spanned M-LLM foundational principles, current and potential applications, technical and ethical challenges, and future research directions. By connecting these elements, we aimed to provide a comprehensive framework that links diverse aspects of M-LLMs, offering a unified vision for their future in health care. This approach aims to guide both future research and practical implementations of M-LLMs in health care, positioning them as a paradigm shift toward integrated clinical AI systems.
Overview
CoChat Summary
## TL;DR
This paper is a narrative review and perspective on multimodal large language models (M-LLMs) in health care, covering their principles, applications, challenges, and future directions. It argues that M-LLMs can extend unimodal LLMs by integrating text, images, audio, video, temporal, and omics data, but real-world use still faces major technical, ethical, and evaluation hurdles.
## Key findings
- The paper identifies 4 core stages in an M-LLM architecture: modality-specific encoding, embedding alignment and fusion, contextual understanding and cross-modal interactions, and decision-making/output generation.
- It argues that current unimodal LLMs are insufficient for medicine because clinical decision-making often combines multiple data types over time, including images, EHRs, laboratory results, audio, and omics data.
- The review highlights wide application areas for M-LLMs, including medical imaging, temporal monitoring, audio analysis, video interpretation, text/EHR analysis, and omics-driven personalized medicine.
- It reports that a general-domain M-LLM study found that such models can “inherently understand medical images” and, in some contexts, even outperform specialized task-specific pretraining methods; the paper uses CONCH as a use case for zero-shot cross-modal retrieval in histopathology.
## Methods
- Narrative review/perspective article synthesizing foundational principles, current and potential applications, technical and ethical challenges, and future outlook for medical M-LLMs.
- The paper organizes the discussion by modality and task: medical images, temporal data, audio, text, video, omics data, and any-to-any M-LLMs.
- It includes a use case example using the CONCH model, run with pretrained weights from Hugging Face on a Linux machine with an NVIDIA GeForce GTX 1080 Ti, in a Flask-based demo for zero-shot cross-modal retrieval on histopathology slides.
## Limitations
- The paper states that recent studies on M-LLMs in health care are limited by a narrow range of modalities, largely theoretical discussion, and limited exploration of solutions to technical barriers.
- The conclusions note that M-LLMs still require further evaluation and benchmarking in real-world medical settings, and that progress toward sophisticated multimodal outputs remains comparatively slow.
- Evident from the text: this is a conceptual review rather than an empirical study, so it does not report original clinical outcome data or a systematic quantitative comparison across models.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures






Verification
Claims the paper makes
29 Valid · 13 Not valid · 12 Not checked · 69 Own results and interpretations
Some cited sources support this claim and some don’t — read each source’s verdict below.
In the paper
“Third, although the challenges of integrating diverse data types into M-LLMs are recognized, there is limited exploration of possible solutions or ongoing research aimed at overcoming these technical barriers [38,39].”
§ Multimodal Large Language Models in Health Care: Applications, Challenges, and Future Outlook > Introduction
What the sources say
- Partly supportedcites [38] sourceExperimental — accuracy not yet measured
The ‘Barriers and Limitations’ section identifies significant technical challenges in integrating text, images, video, and sound into M-LLMs. It does not assess whether recent studies have explored solutions or ongoing research to a limited extent; the Conclusions even suggest developing new tokenization strategies as a possible response. No page numbers are available in our copy. (LLM, conf=0.84)
- Disputedcites [39] sourceExperimental — accuracy not yet measured
The “Multi-modal LLM” section (page unavailable) describes ongoing research combining images, audio, and text, while acknowledging limited data availability. The “Cross-modal dental diagnosis with LLMs” section (page unavailable) also explores a way to combine text and images for dental diagnosis. The paper therefore does not support the attributed lack of exploration of solutions or ongoing research. (LLM, conf=0.85)
Checked by a reasoning model — may be wrong.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“While this unimodal focus on text-based operation has been transformative in the medical field, it does not fully capture the complex and diverse nature of health care practice [8].”
§ Multimodal Large Language Models in Health Care: Applications, Challenges, and Future Outlook > Introduction
What the sources say
- Partly supportedcites [8] sourceExperimental — accuracy not yet measured
The main text says clinical practice requires more than responding accurately to queries and includes practical tasks that LLMs are not equipped to replace. That supports the claim that text-based operation does not capture the full scope of healthcare. But the paper does not describe LLMs as uniformly unimodal: it says GPT-4 accepts images as well as text. It also discusses potential benefits in medicine rather than establishing that text-based operation has already been transformative there. (LLM, conf=0.86)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: unclear — It is unclear whether it could be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“First, the range of data modalities examined is often restricted to text, images, videos, and audio [38], with some studies focusing narrowly on a limited number of clinical applications [39].”
§ Multimodal Large Language Models in Health Care: Applications, Challenges, and Future Outlook > Introduction
What the sources say
- Partly supportedcites [38] sourceExperimental — accuracy not yet measured
The ‘Multimodal Analysis’ section (page unavailable) lists text, images, audio, and video as data types that future M-LLMs could analyze. The ‘Barriers and Limitations’ section says multimodal training data sets are limited, but the paper does not establish that recent studies often restrict the modalities they examine to those four types. (LLM, conf=0.82)
- Partly supportedcites [39] sourceExperimental — accuracy not yet measured
In “Cross-modal dental diagnosis with LLMs” (page unavailable), the paper says most dental AI studies have used image-only approaches, supporting the point that modalities studied have been limited. “Multi-modal LLM” (page unavailable) discusses text, images, and audio in medical learning, but the paper does not establish that health-care M-LLM studies are often restricted to the four listed modalities, including video. (LLM, conf=0.82)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“However, there is a gap in the development of large audio models specifically tailored for medical applications [103].”
§ Multimodal Large Language Models in Health Care: Applications, Challenges, and Future Outlook > Applications > Audio
What the sources say
- Could not verifycites [103] sourceExperimental — accuracy not yet measured
The Introduction says large models have advanced more gradually in audio than in language, images, and video. The Related Surveys and Differences section mentions healthcare surveys but does not identify a gap in developing large audio models specifically for medical applications. Searches for medical and healthcare applications found no passage making that claim; page numbers were unavailable in our copy. (LLM, conf=0.73)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Moreover, nontextual data types play a crucial role in diagnosis, effective treatment planning, research, and patient care [9-11].”
§ Multimodal Large Language Models in Health Care: Applications, Challenges, and Future Outlook > Introduction
What the sources say
- Supportedcites [9] source
The Introduction (page not provided) identifies medical images and omics data as relevant to diagnostics, therapy planning, and patient monitoring. “Towards Future Human-AI Interfaces for Multi-Modal Causability” (page not provided) says that integrating images, omics, and other data is needed to understand diseases and biological processes. Together, these passages support the broad claim about nontextual data in research and patient care. (LLM, conf=0.85)
- Partly supportedcites [10] sourceExperimental — accuracy not yet measured
The Introduction (page unavailable) describes using patients’ genomic data to predict early warning signals of cancer, and the Biologic significance section (page unavailable) identifies potential drug targets for early intervention. This supports a narrower point about nontextual molecular data in cancer prediction and treatment research, but not the broad claim about nontextual data types across diagnosis, effective treatment planning, research, and patient care. The paper’s “multi-modal” refers to alternative biomarker configurations, not different data types. (LLM, conf=0.83)
- Not checked — no full text of the sourcecites [11] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
34 more claims · 13 with their cited sources checked
Create a free account to see all 123 claims and every source verdict.
- It may state two separate claims as one. 26
- Probably not a substantial claim. 74
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“LLMs, such as GPT and BERT, are trained on massive datasets and have the unique capability of understanding both structured and unstructured text (11,81).”
From Generative artificial intelligence in predictive analysis of diabetes and its complications: a narrative review (2025) · cites this paper as [81]
Experimental — accuracy not yet measured
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 1×
“For example, image encoders (eg, transformers [54] and convolutional neural networks (CNNs) [55,56]) are designed to capture fine-grained patterns or anomalies crucial for diagnosis, whereas text encoders (BERT [15]) aim to comprehend complex medical terminology and patient histories.”
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 1×
“For example, image encoders (eg, transformers [54] and convolutional neural networks (CNNs) [55,56]) are designed to capture fine-grained patterns or anomalies crucial for diagnosis, whereas text encoders (BERT [15]) aim to comprehend complex medical terminology and patient histories.”
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 2×
“Time-series encoders (such as transformer-based models [15,59-61] and LSTM [12]) are intended to detect critical changes over time, signaling the need for urgent medical intervention.”
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472 · cited 1×
“For example, image encoders (eg, transformers [54] and convolutional neural networks (CNNs) [55,56]) are designed to capture fine-grained patterns or anomalies crucial for diagnosis, whereas text encoders (BERT [15]) aim to comprehend complex medical terminology and patient histories.”
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204 · cited 1×
“Knowledge distillation is another approach that involves training a smaller “student” model to replicate the behavior of a larger “teacher” model, requiring less computational power while retaining performance [161].”
LoRA: Low-Rank Adaptation of Large Language Models
Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., & Chen, W. (2021) · arXiv · doi:10.48550/arxiv.2106.09685 · Cited by 13,453 · cited 1×
“Parameter-efficient fine-tuning methods such as adapter layers help reduce the computational load by fine-tuning only a subset of the model’s parameters [157,158].”
BioBERT: a pre-trained biomedical language representation model for biomedical text mining
Lee, J., Yoon, W., Kim, S., Kim, D., Kim, S., So, C. H., & Kang, J. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btz682 · Cited by 7,218 · cited 1×
“Examples of biomedical text LLMs include BiMediX [104], BioBERT [105], PubMedBERT [106], and ClinicalBERT [20].”
Han, S., Mao, H., & Dally, W. J. (2015) · arXiv · doi:10.48550/arxiv.1510.00149 · Cited by 4,662 · cited 1×
“In addition, quantization approaches can address the scalability and computational demands by shifting toward quantized versions of existing models using curated, domain-specific data rather than pretraining from scratch [159].”
In-Datacenter Performance Analysis of a Tensor Processing Unit
Jouppi, N. P., Young, C., Patil, N., Patterson, D., Agrawal, G., Bajwa, R., Bates, S., Bhatia, S., Boden, N., Borchers, A., et al. (2017) · Proceedings of the 44th Annual International Symposium on Computer Architecture · doi:10.1145/3079856.3080246 · Cited by 4,537 · cited 1×
“Additional optimization techniques, such as model compression [165] and hardware acceleration using graphics processing units (GPUs) or tensor processing units (TPUs) [166], further enhance efficiency.”
Large language models encode clinical knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2023) · Nature · doi:10.1038/s41586-023-06291-2 · Cited by 4,083 · cited 1×
“Subsequently, a series of medical BERT models were proposed to accelerate medical research [6,17-20].”
Research works printed but not linked: 25
- VaswaniAShazeerNParmarNUszkoreitJJonesLGomezANKaiserLPolosukhinIAttention is all you needProceedings of the 31st International Conference on Neural Information Processing Systems2017NIPS'17December 4-9, 2017Long Beach, CA
- Devlin, J, arXiv
- Touvron, H, arXiv
- Chung, HW, arXiv
- Huang, W, arXiv
- Zhu, D, arXiv
- Su, Y, arXiv
- Koh, JY, arXiv
- Alayrac, JB, arXiv
- Rasheed, H, arXiv
- WangZZhangQYuHHuSJinXGongZChenHMulti-level protein structure pre-training with prompt learningProceedings of the Eleventh International Conference on Learning Representations2023ICLR 2023May 1-5, 2023Kigali, Rwanda
- Zhou, H, arXiv
- Yordanov, Y, arXiv
- HewittJManningCDA structural probe for finding syntax in word representationsProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies2019NAACL-HLT 2019June 2-7, 2019Minneapolis, MN
- Gerke, S, 2020, Artificial Intelligence in Healthcare, 295
- Lee, N, arXiv
- Reddy, AG, arXiv
- Carlini, N, arXiv
- Aghajanyan, A, arXiv
- Huang, Y, arXiv
- Wang, X, arXiv
- Fu, C, arXiv
- Yang, X, arXiv
- Ge, W, arXiv
- Li, S, arXiv
Cited by
Large Language Models in Healthcare and Medical Applications: A Review
Maity, S., & Saikia, M. J. (2025) · Bioengineering · doi:10.3390/bioengineering12060631 · Cited by 249
“The integration of LLMs with medical imaging represents a significant advancement in diagnostic capabilities, particularly through multimodal models that can process both visual and textual information [4,11,56–58].”
Zhou, M., Pan, Y., Zhang, Y., Song, X., & Zhou, Y. (2025) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2025.105871 · Cited by 142
Multimodal Large Language Models in Medical Imaging: Current State and Future Directions
Nam, Y., Kim, D., Kyung, S., Seo, J., Song, J. M., Kwon, J., Kim, J., Jo, W., Park, H. J., Sung, J., et al. (2025) · Korean Journal of Radiology · doi:10.3348/kjr.2025.0599 · Cited by 154
“The defining feature is their ability to integrate and align information across modalities, often mapping them into a shared representational space [8].”
InfluentialAbbas, S. R., Seol, H., Abbas, Z., & Lee, S. W. (2025) · Healthcare · doi:10.3390/healthcare13141642 · Cited by 80
“Multimodal AI Models: Research is underway to integrate imaging, EHR data, genomic profiles, and behavioral metrics into unified decision-making tools [103].”
A Current Review of Generative AI in Medicine: Core Concepts, Applications, and Current Limitations
Rouzrokh, P., Khosravi, B., Faghani, S., Moassefi, M., Shariatnia, M. M., Rouzrokh, P., & Erickson, B. J. (2025) · Current Reviews in Musculoskeletal Medicine · doi:10.1007/s12178-025-09961-y · Cited by 66
“Audio and video generation models in medicine are enabling new ways to enhance clinical training, patient communication, and medical education [ 64 ].”
An overview of methods and techniques in multimodal data fusion with application to healthcare
Chaabene, S., Boudaya, A., Bouaziz, B., & Chaâri, L. (2025) · International Journal of Data Science and Analytics · doi:10.1007/s41060-025-00715-0 · Cited by 68
A Comprehensive Review of Multimodal Deep Learning for Enhanced Medical Diagnostics
Al‐Zoghby, A. M., Ebada, A. I., Saleh, A. S., Abdelhay, M., & Awad, W. A. (2025) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2025.065571 · Cited by 87
From screens to scenes: A survey of embodied AI in healthcare
Liu, Y., Cao, X., Chen, T., Jiang, Y., You, J., Wu, M., Wang, X., Feng, M., Jin, Y., & Chen, J. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103033 · Cited by 92
AI in Breast Cancer Imaging: An Update and Future Trends
Chen, Y., Shao, X., Shi, K., Rominger, A., & Caobelli, F. (2025) · Seminars in Nuclear Medicine · doi:10.1053/j.semnuclmed.2025.01.008 · Cited by 50
Leading artificial intelligence–driven drug discovery platforms: 2025 landscape and global outlook
Dharmasivam, M., Kaya, B., Akinware, A., Azad, M. G., & Richardson, D. R. (2025) · Pharmacological Reviews · doi:10.1016/j.pharmr.2025.100102 · Cited by 51
An AI ‐Enabled Nursing Future With no Documentation Burden: A Vision for a New Reality
Michalowski, M., Topaz, M., & Peltonen, L. (2025) · Journal of Advanced Nursing · doi:10.1111/jan.16911 · Cited by 52
Agentic AI and large language models in radiology: opportunities and hallucination challenges
Salehi, S., Singh, Y., Horst, K. K., Hathaway, Q. A., & Erickson, B. J. (2025) · Bioengineering · doi:10.3390/bioengineering12121303 · Cited by 32
“Radiologists now routinely employ LLMs for structured reporting, automatic measurement extraction, and differential diagnosis generation, with studies showing 40% time reduction in report creation and 25% improvement in diagnostic consistency [2,3].”
Sohn, J.-S., Lee, E., Kim, J., Oh, H.-K., & Kim, E. J. (2025) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2025.1628216 · Cited by 49
Saadh, M. J., Ahmed, H. H., Kareem, R. A., Yadav, A., Ganesan, S., Shankhyan, A., Sharma, G. C., Naidu, K. S., Rakhmatullaev, A., Sameer, H. N., et al. (2025) · Discover Oncology · doi:10.1007/s12672-025-02111-3 · Cited by 30
“This approach exemplifies how state-of-the-art deep learning methodologies are increasingly becoming indispensable tools in genomic research [63].”
Neuro-symbolic AI for auditable cognitive information extraction from medical reports
Prenosil, G., Weitzel, T., Bello, S. C., Mingels, C., Manzini, G., Meier, L. P., Shi, K., Rominger, A., & Afshar‐Oromieh, A. (2025) · Communications Medicine · doi:10.1038/s43856-025-01194-x · Cited by 48
Du, X., Zhou, Z., Wang, Y., Chuang, Y.-W., Li, Y., Yang, R., Hong, P., Bates, D. W., & Zhou, L. (2026) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2025.106091 · Cited by 26
Leveraging large language models for automated depression screening
Teferra, B. G., Perivolaris, A., Hsiang, W.-N., Sidharta, C. K., Rueda, A., Parkington, K. B., Wu, Y., Soni, A., Samavi, R., Jetly, R., et al. (2025) · PLOS Digital Health · doi:10.1371/journal.pdig.0000943 · Cited by 34
“This strategy may enhance clinical decision-making by ensuring a more nuanced and inclusive assessment of mental health conditions, similar to how multimodal diagnostics are utilized in radiology and oncology [28].”
Fleurence, R., Dawoud, D., Bian, J., Higashi, M. K., Wang, X., Xu, H., Chhatwal, J., & Ayer, T. (2025) · Value in Health · doi:10.1016/j.jval.2025.06.018 · Cited by 41
Xu, X., & Sankar, R. (2025) · Information · doi:10.3390/info16100894 · Cited by 20
“Agents must also handle missing data gracefully, which calls for models explicitly designed for partial or sparse input [81].”
Rapid review: Growing usage of Multimodal Large Language Models in healthcare
Gupta, P., Zhang, Z., Song, M., Michalowski, M., Hu, X., Štiglic, G., & Topaz, M. (2025) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2025.104875 · Cited by 29
Ortiz-Pérez, D., Benavent-Lledó, M., Rodriguez-Juan, J., García‐Rodríguez, J., & Tomás, D. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.114264 · Cited by 29
“This expressiveness makes them ideal for tasks involving subtle cues, such as those in clinical speech data [57], [58].”
Mese, I., D’Antonoli, T. A., Bluethgen, C., Bressem, K., Cuocolo, R., Chaudhari, A., Tejani, A. S., Isaac, A., Ponsiglione, A., Meddeb, A., et al. (2026) · Diagnostic and Interventional Radiology · doi:10.4274/dir.2026.263812 · Cited by 17
A personalized periodontitis risk based on nonimage electronic dental records by machine learning
Swinckels, L., de Keijzer, A., Loos, B. G., Applegate, R. J., Kookal, K. K., Kalenderian, E., Bijwaard, H., & Bruers, J. J. M. (2024) · Journal of Dentistry · doi:10.1016/j.jdent.2024.105469 · Cited by 23
Kwan, H. Y., Shell, J., Fahy, C., Yang, S., & Xing, Y. (2025) · Systems · doi:10.3390/systems13040281 · Cited by 20
Evaluation of deepseek, gemini, ChatGPT-4o, and perplexity in responding to salivary gland cancer
Bashah, A., Salem, A., Al-waqeerah, A., Ghaleb, E., Wahan, N., Awad, A. M., Altos, O., & Chen, G. (2025) · BMC Oral Health · doi:10.1186/s12903-025-06726-4 · Cited by 26
“Despite their promise, M-LLMs have seen limited adoption in medical and dental research, signaling a need for deeper exploration to overcome existing technical, ethical, and practical challenges [69, 70].”
Transformers for Multi-Modal Image Analysis in Healthcare
Sagheer, S. V. M., Meghana, K. H., Ameer, P. M., Parayangat, M., & Abbas, M. A. E. (2025) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2025.063726 · Cited by 30
“Additionally, Alsaad et al. [34] highlighted that the flexible architecture of transformers facilitates the seamless integration of various data types, including medical histories of the patients, laboratory test results, and genetic information. adaptability opens up possibilities for developing…”
Mascarenhas, M., Ribeiro, T., Agudo, B., Afonso, J. P. L., Mendes, F., Martins, M., Cardoso, P., Mota, J., Almeida, M. J., da Costa, A. M. P., et al. (2025) · Journal of Clinical Medicine · doi:10.3390/jcm14020572 · Cited by 15
Alsaedi, S. B., Gao, X., & Gojobori, T. (2025) · FEBS Open Bio · doi:10.1002/2211-5463.70003 · Cited by 21
Medical multimodal large language models: A systematic review
Hu, Y., Xu, C., Lin, B., Yang, W., & Tang, Y. Y. (2025) · Intelligent Oncology · doi:10.1016/j.intonc.2025.09.005 · Cited by 14
Multimodal large language models for medical image diagnosis: Challenges and opportunities
Zhang, A., Zhao, E., Wang, R., Zhang, X., Wang, J., & Chen, E. (2025) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2025.104895 · Cited by 21
Exploring the application boundaries of LLMs in mental health: a systematic scoping review
Yang, J., Liu, T., Luo, Y. T., Niu, T., Pang, P. C., Ao, X., & Yang, Q. (2026) · Frontiers in Psychology · doi:10.3389/fpsyg.2025.1715306 · Cited by 31
“For instance, they can analyze electronic health records (EHRs), handwritten clinical notes, and initial consultation transcripts to extract key symptoms, medical history, and behavioral patterns (AlSaad et al., 2024).”
Ni, H.-P., Liu, C., Paul, F., Chong, W. O., & Chou, J. (2025) · Applied Energy · doi:10.1016/j.apenergy.2025.126420 · Cited by 12
Mashekova, A., Zhao, Y., Zarikas, V., Mukhmetov, O., Aidossov, N., Ng, E. Y. K., Wei, D., & Shapatova, M. (2025) · Bioengineering · doi:10.3390/bioengineering12101110 · Cited by 22
Bridging modalities with AI: a review of AI advances in multimodal biomedical imaging
Doan, L. M. T., Shahhosseini, K., Verma, S., Marefat, A., Locicero, G., Verma, S., Angione, C., & Occhipinti, A. (2026) · Communications Engineering · doi:10.1038/s44172-026-00602-x · Cited by 31
“…on extensive multimodal datasets incorporating biomedical imaging, clinical notes and EHRs, have demonstrated the capacity to generalise across a wide range of downstream tasks (e.g., diagnosis, prognosis and biomarker identi fi cation) without the need for task-speci fi c architectures 8,144,145 .”
A review of medical text analysis: Theory and practice
Chen, Y., Zhang, C., Bai, R., Sun, T., Ding, W., & Wang, R. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103024 · Cited by 13
Abugabah, A., Shukla, P. K., Shukla, P. K., & Pandey, A. (2025) · BioData Mining · doi:10.1186/s13040-025-00487-0 · Cited by 11
“Transformers are more adaptable, have better feature representation, and better ability to incorporate heterogeneous data, which makes them especially suitable for multimodal healthcare applications compared to CNNs [10].”
Wongsuwan, J., Tubtawee, T., Nirattisaikul, S., Danpanichkul, P., Cheungpasitporn, W., Chaichulee, S., & Kaewdech, A. (2025) · BMJ Open Gastroenterology · doi:10.1136/bmjgast-2025-001832 · Cited by 16
“32 Benefits of AI for HCC surveillance DL models, particularly CNNs, have shown considerable promise in analysing medical images and detecting abnormalities in modalities such as X-rays, US, CT and MRI.”
Ibáñez, M. B., Estrada, M. L. B., & Cabada, R. Z. (2025) · Computer Applications in Engineering Education · doi:10.1002/cae.70080 · Cited by 19
“While initial applications of MLLMs have been reported in domains such as healthcare [15], autonomous driving [16], and even in financial services [17], their ability to accurately interpret and generate UML diagrams remains largely unexplored [18].”
Multimodal language models in agriculture: A tutorial and survey
Haghighat, M., Saleh, A., & Azghadi, M. R. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.104042 · Cited by 25
Haider, S. A., Prabha, S., Gomez-Cabello, C. A., Borna, S., Genovese, A., Trabilsy, M., Elegbede, A. I., Yang, J. F., Galvao, A., Tao, C., et al. (2025) · Journal of Clinical Medicine · doi:10.3390/jcm14103484 · Cited by 15
“Unlike traditional computer vision techniques, which often rely on predefined features, MLLMs can learn complex patterns and relationships directly from data, potentially capturing subtle aesthetic cues that may be missed by human observers or simpler algorithms [19].”
From Traditional Machine Learning Models to Multimodal Large Models: A Review of Aquaculture
Liu, S., Du, Z., Wang, G., Zhang, P., Xu, W., Yu, J., & Li, D. (2025) · Reviews in Aquaculture · doi:10.1111/raq.70111 · Cited by 19
Battle of the Bots: Solving Clinical Cases in Osteoarticular Infections With Large Language Models
Borgonovo, F., Matsuo, T., Petri, F., Amin Alavi, S. M., Mazudie Ndjonko, L. C., Gori, A., & Berbari, E. F. (2025) · Mayo Clinic Proceedings: Digital Health · doi:10.1016/j.mcpdig.2025.100230 · Cited by 10
Using large language models for semantic interoperability: A systematic literature review
Abu-Salih, B., Alotaibi, S., Alanazi, A. L., Abukhurma, R., Al-Shboul, B. A., Khoury, A., & Aljaafari, M. A. H. (2025) · ICT Express · doi:10.1016/j.icte.2025.06.011 · Cited by 17
Yazdi, N. A., Ahari, U. Z., KhadiviMaleki, H., & Hasanabadi, P. (2025) · BMC Medical Education · doi:10.1186/s12909-025-07621-w · Cited by 16
Deep Learning in Biomedical Image and Signal Processing: A Survey
Omarov, B. S. (2025) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2025.064799 · Cited by 26
Baumann, S., Stone, R., & Abdelall, E. S. (2025) · Sensors · doi:10.3390/s25196067 · Cited by 14
“Recent work by Alsaad et al. [56] supports this need for personalization by demonstrating how multimodal large language models (MLLMs) can sustain engagement through conversational interfaces that evolve with the user’s needs over time.”
Anh, N. V., Vuong, T. Q. T., & Nguyen, V. H. (2025) · PLoS ONE · doi:10.1371/journal.pone.0335775 · Cited by 15
“Large-language models (LLMs) have progressed rapidly from text-only transformers to multimodal engines capable of parsing images, audio, and code in real time [1].”
Nazar, W., Nazar, G., Kamińska, A., & Daniłowicz‐Szymanowicz, L. (2025) · Journal of Medical Internet Research · doi:10.2196/70481 · Cited by 20
“Recent research shows that the following metadata can be useful for text-based IFT datasets in medicine (Figure 5) [8,9,15,16,21,36,37].”
Navarro-Garcia, D., Marcos, A. V., Beets‐Tan, R. G. H., Blomqvist, L., Bodalal, Z., Deandreis, D., Crispin‐Ortuzar, M., Gallagher, F. A., Giandini, T., Graves, M. J., et al. (2025) · ESMO Real World Data and Digital Oncology · doi:10.1016/j.esmorw.2025.100120 · Cited by 11
“25”
AI-Driven Personalized Healthcare: Leveraging Multimodal Data for Precision Medicine
Lin, R. (2024) · IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology (WI-IAT) · doi:10.1109/wi-iat62293.2024.00112 · Cited by 18
“Furthermore, there is a risk of AI systems introducing or perpetuating biases, particularly if trained on biased datasets [19], [34].”
Advanced AI-based technologies for early detection and personalized management of cervical cancer
Zhang, Y., Xu, Y., Wang, C., Zhang, Z., Zhou, K., Zhu, Y., & Yu, X. (2025) · Biomedical Technology · doi:10.1016/j.bmt.2025.100100 · Cited by 15
Artificial intelligence-enabled wearable microgrids for self-sustained energy management
Ding, S., Bian, Y., Saha, T., Khan, M. I., Chang, A., Yin, L., Xu, S. H., & Wang, J. (2025) · Nature Reviews Electrical Engineering · doi:10.1038/s44287-025-00206-1 · Cited by 11
“However, there are relevant insights into the advancements in wearable energy harvesting and management technologies, as large language models can help interpret the wearable data and deliver suggestions to users for the best outcome 29,34,35 .”
Mine, Y., Iwamoto, Y., Okazaki, S., Nishimura, T., Tabata, E., Takeda, S., Peng, T., Nomura, R., Kakimoto, N., & MURAYAMA, T. (2025) · International Journal of Paediatric Dentistry · doi:10.1111/ipd.70029 · Cited by 12
“The Mann–Whitney U - test was performed to compare binary groups using IBM SPSS Statistics 27 (IBM SPSS Inc., Armonk, NY, USA).”
Clinical Management of Wasp Stings Using Large Language Models: Cross-Sectional Evaluation Study
Pan, W., Zhang, S., Wang, Y., Quan, Z., Zhu, Y., Fang, Z., & Yang, X. (2025) · Journal of Medical Internet Research · doi:10.2196/67489 · Cited by 7
“In addition, the multimodal applications of LLMs in health care management have shown unprecedented diversity [23].”
Enhancing Ultrahigh Resolution Remote Sensing Imagery Analysis With ImageRAG: A new framework
Zhang, Z., Shen, H., Zhao, T., Guan, Z., Chen, B., Wang, Y., Xu, J. W., Cai, Y., Shang, Y., & Yin, J. (2025) · IEEE Geoscience and Remote Sensing Magazine · doi:10.1109/mgrs.2025.3574742 · Cited by 22
“MLLMs tend to exhibit suboptimal performance in specialized professional domains, such as RS [5], [52] and medical imaging [53], [54].”
Pan, X., Bai, Y., Zou, K., Zhou, Y., Zhou, J., Fu, H., Tham, Y. C., & Liu, Y. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103631 · Cited by 10
“…Large Language Models (MLLMs) have gained attention for their ability to process diverse medical data, including imaging modalities (e.g., MRI, CT scans), time-series signals (e.g., outputs from electroencephalogram), audio recordings (e.g., patient interviews), and clinical documentation [2, 3].”
Zhao, J., Shangguan, S., & Wang, Y. (2025) · Journal of Computer Assisted Learning · doi:10.1111/jcal.70110 · Cited by 14
Magouliotis, D. E., Sicouri, N., Ramlawi, L., Baudo, M., Androutsopoulou, V. A., & Sicouri, S. (2026) · Journal of Personalized Medicine · doi:10.3390/jpm16020069 · Cited by 12
“Unlike CNNs, transformers use attention mechanisms to dynamically weight the importance of different input features over time [16].”
InfluentialKottlors, J., Iuga, A.-I., Blüthgen, C., Bressem, K. K., Kather, J. N., Moy, L. P., Wald, C., Wang, W., Liu, T., Ranschaert, E. R., et al. (2026) · Radiology · doi:10.1148/radiol.250913 · Cited by 16
Exploring large language models for summarizing and interpreting an online brain tumor support forum
Muasher-Kerwin, C., Hughes, M. C., Foster, M. L., Azher, I. A., & Alhoori, H. (2025) · Digital Health · doi:10.1177/20552076251337345 · Cited by 9
“9 – 11 Models such as GPT-3.”
Shokouhmand, S., Bhatt, S., & Faezipour, M. (2025) · Electronics · doi:10.3390/electronics14101994 · Cited by 9
Oleribe, O. O. (2025) · Healthcare · doi:10.3390/healthcare13151898 · Cited by 9
“This may facilitate early identification of biomarkers, uncover previously unrecognized subtypes, and guide precision medicine strategies tailored to individual and population-level needs [28,40].”
InfluentialArdila, C. M., Pineda‐Vélez, E., & Vivares‐Builes, A. M. (2025) · Dentistry Journal · doi:10.3390/dj13110489 · Cited by 12
“Multimodal/vision–language models that combine visual encoders with LLM backbones can bridge imaging and text but introduce additional variance due to connector design and fine-tuning data [48].”
Cheng, P., Zhang, K., Lyu, M., Liu, H., Sun, L., & Wu, Y. (2025) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2025.104946 · Cited by 19
Li, S., Lin, T., Lin, L., Zhang, W., Liu, J., Yang, X., Li, J., He, Y., Song, X., Xiao, J., et al. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3755229 · Cited by 21
Li, P., Xu, Y., Liu, X., Shen, Z., Wang, Y., Lv, X., Lu, Z., Wu, H., Zhuang, J., & Chen, Y. (2026) · JMIR Medical Informatics · doi:10.2196/81422 · Cited by 9
“Despite their promise, the application of LLMs in health care—especially in clinical decision-making—remains constrained by several limitations [5,6].”
CancerLLM: a large language model in cancer domain
Li, M., Zhan, Z., Huang, J., Yeung, J., Ding, K., Blaes, A. H., Johnson, S., Liu, H., Xu, H., & Zhang, R. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-02441-8 · Cited by 61
“…reviewofsystems(nursingros)records,laboratorytestresults,patientself-descriptions, medical history, treatment modalities, assessments, and patient outcomes, LLMs can furnish healthcare professionals with more precise and personalized recommendations 22 .”
Systematic Literature Review on Medical Image Captioning Using CNN-LSTM and Transformer-Based Models
Fadhilah, H., & Utama, N. P. (2025) · JURNAL MASYARAKAT INFORMATIKA · doi:10.14710/jmasif.16.1.73127 · Cited by 9
Rhee, J. Y., Paulino, M. T., Finnemore, A., Tentor, Z., & Cashman, C. R. (2025) · Current Neurology and Neuroscience Reports · doi:10.1007/s11910-025-01429-3 · Cited by 7
“The impact of large language models and artifi - cial intelligence to review and summarize large amounts of healthcare data [109] or quantify elements of the neuromus - cular examination [110] is only just beginning to be under - stood, with great potential to help identify the best treatment…”
Matsumoto, K., Suzuki, M., Ishihara, K., Tokunaga, K., Matsuda, K., Chen, J., Yamashiro, S., Soejima, H., Nakashima, N., & Kamouchi, M. (2025) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2025.105989 · Cited by 4
Aydin, C., Duygu, Ö. B., Karakaş, A. B., Er, E., Gökmen, G., Öztürk, A. M., & Gövsa, F. (2025) · Medicina · doi:10.3390/medicina61081342 · Cited by 8
Artificial Intelligence-Driven Nephrology: The Role of Large Language Models in Kidney Care
Zoccali, C., Floyd, L., Cseprekál, O., Eisenga, M. F., Mirioğlu, Ş., Caravaca‐Fontán, F., & Mallamaci, F. (2025) · American Journal of Nephrology · doi:10.1159/000548208 · Cited by 9
“Furthermore, they aid in differential diagnosis by evaluating patient history, symptoms, and test results, helping predict disease progression while recommending personalized treatment plans [21].”
Moglia, A., Leccardi, M., Cavicchioli, M., Maccarini, A., Marcon, M., Mainardi, L. T., & Cerveri, P. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103709 · Cited by 19
Multimodal large language models for women’s reproductive mental health
AlSaad, R., Youssef, A., Kashani, S., Alabdulla, M., Abd-Alrazaq, A. A., Khaled, S. M., Ahmed, A., & Sheikh, J. I. (2025) · Archives of Women s Mental Health · doi:10.1007/s00737-025-01633-7 · Cited by 8
“…applications in reproduc - tive medicine, most existing models focus narrowly on predicting clinical outcomes, such as embryo quality or fer - tilization success, without incorporating the patient’s emo - tional or psychological state (Orovou et al. 2025; AlSaad et al. 2024b; AlSaad et al. 2025a).”
InfluentialSelf-citationTeles, A. S., Chaturvedi, J., Wang, T., Scazufca, M., Msosa, Y. J., Stahl, D. R., & Roberts, A. (2025) · PLOS mental health. · doi:10.1371/journal.pmen.0000488 · Cited by 8
Adaptive Scheduling of Multimodal Large Language Model in Intelligent Edge Computing
Yuan, X., Li, H., Dong, M., & Ota, K. (2025) · ACM Transactions on Autonomous and Adaptive Systems · doi:10.1145/3774908 · Cited by 9
Mai, H., Lee, D., Kaenploy, J., Kim, J., & Cho, S. (2025) · Journal of Esthetic and Restorative Dentistry · doi:10.1111/jerd.70064 · Cited by 8
A survey on privacy issues and mitigation strategies for LLMs in healthcare
Alissa, K. A. (2025) · The Journal of Supercomputing · doi:10.1007/s11227-025-08146-1 · Cited by 8
Imran, M., & Lee, Y. (2026) · IEEE Access · doi:10.1109/access.2026.3656368 · Cited by 15
“…prior work [2], [3] by providing: (1) comprehensive RAG-enhanced VLM analysis, (2) quantitative performance synthesis showing 33.1% improvement from domain specialization, and (3) clinical deployment analysis with FDA-approved systems. transition to multimodal methods [3], [15], [16], [17], [18].”
InfluentialEvaluating cognitive depth of AI-generated multiple-choice questions with Bloom’s Taxonomy
Nguyen, T. T., Nguyen, L., Nguyệt, H. T., Nguyen, H. T. T., & Tong, S. M. (2026) · PLoS ONE · doi:10.1371/journal.pone.0341317 · Cited by 8
“Although no previous studies have directly compared LLMs’ capabilities to generate MCQs aligned with Bloom’s Taxonomy, several investigations have demonstrated that LLMs can produce dental and medical questions with high levels of accuracy, relevance, and complexity [14,20].”
Knott, M., Krebs, M., & Kerscher, A. (2026) · Frontiers in Digital Health · doi:10.3389/fdgth.2026.1761641 · Cited by 11
“The evolution toward multimodal LLMs—systems capable of processing text, images, and other data types simultaneously— presents additional opportunities for quality management applications (51).”
Chen, Z., Chambara, N., Liu, S. Y., Chow, T. C. M., Lai, C. M. S., & Ying, M. T. C. (2025) · Cancers · doi:10.3390/cancers17132068 · Cited by 5
“Additionally, LLMs offer the advantage of generating human-readable reasoning, which may enhance interpretability and transparency in diagnostic processes [15].”
MIRA: A Novel Framework for Fusing Modalities in Medical RAG
Wang, J., Ashraf, T., Han, Z., Laaksonen, J. T., & Anwer, R. M. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3755760 · Cited by 15
Li, J., Yuan, R., Tian, Y., & Li, J. (2024) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm62325.2024.10822624 · Cited by 6
Khalifa, A., & Hussein, R. (2026) · Journal of Medical Systems · doi:10.1007/s10916-026-02391-4 · Cited by 5
Benchmarking large multimodal models for ophthalmic visual question answering with OphthalWeChat
Xu, P., Gong, X., Chen, X., Zhang, W., Yang, J., Yan, B., Yuan, M., Yalin, Z., He, M., & Shi, D. (2025) · Advances in Ophthalmology Practice and Research · doi:10.1016/j.aopr.2025.10.006 · Cited by 8
“3,4 12 However, the development and automated evaluation of VLMs rely on access to diverse, high-quality datasets and standardized benchmarks to ensure robust performance comparison and validation.”
Wolber, J., Samadi, M. E., Sellin, J., & Schuppert, A. (2025) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2025.104945 · Cited by 11
Choi, J. Y., & Yoo, T. K. (2026) · Discover Health Systems · doi:10.1007/s44250-026-00343-w · Cited by 14
“Although unimodal language models based solely on text demonstrate strong reasoning capabilities, a previous article noted that they do not fully capture the complex and varied nature of clinical practice, highlighting the need for multimodal integration [3].”
Shih, Y.-C., Wu, C.-Y., Huang, S., & Tsai, C.-Y. (2026) · Journal of Medical Internet Research · doi:10.2196/87193 · Cited by 6
“Against this background, multimodal large language models (MM-LLMs) hold substantial potential [15].”
Hack, S., Craig, J. R., Lin, C.-F., Fu, C., Kwiatkowska, M. A., Kocum, P., Allevi, F., & Saibene, A. M. (2026) · European Archives of Oto-Rhino-Laryngology · doi:10.1007/s00405-026-10040-2 · Cited by 4
Mavrych, V., Yousef, E. M., Yaqinuddin, A., Shaikh, A. A., & Bolgova, O. (2026) · PLOS Digital Health · doi:10.1371/journal.pdig.0001306 · Cited by 5
“This cross-modal consistency in challenging tissue types indicates that certain histological concepts present fundamental recognition difficulties regardless of input modality [11].”
Gelen, M. A., Barua, P. D., Taşçı, İ., Taşçı, G., Aydemir, E., Doğan, Ş., Tuncer, T., & Acharya, U. R. (2025) · Scientific Reports · doi:10.1038/s41598-025-00071-w · Cited by 8
Chang, L., Huang, C., Yu, H., Hung, L., Lin, H., Liao, L., & Lai, I. (2025) · International Nursing Review · doi:10.1111/inr.70076 · Cited by 6
Lin, Y., Cheng, M., Wu, C., Huang, Y., Zhu, T., Li, J., Gao, H., & Wang, K. (2024) · The Lancet Regional Health - Western Pacific · doi:10.1016/j.lanwpc.2024.101254 · Cited by 6
“127 For example, CheXzero, a zero-shot chest X-ray classi fi er, has achieved radiologist-level performance without prior exposure to speci fi c disease labels.”
Improving Psychiatry Services with Artificial Intelligence: Opportunities and Challenges
Eser, H. Y., BALLI, M., & DOĞAN, A. E. (2024) · Turkish Journal of Psychiatry · doi:10.5080/u27604 · Cited by 10
Osonuga, A., Olawade, D. B., Gore, M., Rotifa, O. B., Egbon, E., Chinwah, V., & Boussios, S. (2025) · Annals of Translational Medicine · doi:10.21037/atm-25-62 · Cited by 8
Lingshu: A Generalist Foundation Model for Unified Multimodal Medical Understanding and Reasoning
Xu, W., Chan, H. P., Li, L., Aljunied, M., Yuan, R., Wang, J., Xiao, C., Chen, G., Liu, C., Li, Z., et al. (2025) · arXiv · doi:10.48550/arxiv.2506.07044 · Cited by 4
Characterizing the Flaws of Image-Based AI-Generated Content
Vasir, G., & Huh, J. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3720004 · Cited by 6
A bibliometric analysis of large language model-based AI chatbots in surgery
Wang, Z., Zhou, H., & Song, T. (2025) · Annals of Medicine and Surgery · doi:10.1097/ms9.0000000000003234 · Cited by 3
Liu, Y., & Li, C.-X. (2025) · Journal of Cardiothoracic and Vascular Anesthesia · doi:10.1053/j.jvca.2025.06.043 · Cited by 3
Metadata
- Authors
- AlSaad, R., Abd-alrazaq, A., Boughorbel, S., Ahmed, A., Renault, M.-A., Damseh, R., & Sheikh, J.
- Affiliations
- Weill Cornell Medical College in Qatar (QA)
- Hamad bin Khalifa University (QA)
- United Arab Emirates University (AE)
- Corresponding author
- AlSaad, Rawan
- ORCID
- Venue
- Journal of medical Internet research
- Publisher
- JMIR Publications Inc.
- Published
- 2024-09-25
- Volume
- 26
- Pages
- e59505
- ISSN
- 1438-8871 (ISSN-L)
- PMC
- PMC11464944
- OpenAlex
- W4402155831
- Cited by
- 349
- References
- 175
- Citations by year
- 2026: 202 · 2025: 139 · 2024: 8
- FWCI
- 32.47
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Journal of Medical Internet Research (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Keywords
- multimodal large language models, multimodal data integration, health care applications, medical images, wearable sensor data, electronic health records, clinical notes, audio recordings, surgical video, omics data, clinical decision support, ethical considerations, multimodal fusion
- MeSH terms
- Delivery of Health Care, Humans, Natural Language Processing, Electronic Health Records
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-09, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-09 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.