Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Sign Up
Cited by 229

The DOI matches its Crossref record; 70 of the 70 references it cites were found in a registry or an index.

Vision-language models for medical report generation and visual question answering: a review

Open access
Hartsock, Iryna, Rasool, Ghulam·2024·Frontiers in Artificial Intelligence
DOI 10.3389/frai.2024.1430984
Open this paper in CoChat — free account

Abstract

Medical vision-language models (VLMs) combine computer vision (CV) and natural language processing (NLP) to analyze visual and textual medical data. Our paper reviews recent advancements in developing VLMs specialized for healthcare, focusing on publicly available models designed for medical report generation and visual question answering (VQA). We provide background on NLP and CV, explaining how techniques from both fields are integrated into VLMs, with visual and language data often fused using Transformer-based architectures to enable effective learning from multimodal data. Key areas we address include the exploration of 18 public medical vision-language datasets, in-depth analyses of the architectures and pre-training strategies of 16 recent noteworthy medical VLMs, and comprehensive discussion on evaluation metrics for assessing VLMs' performance in medical report generation and VQA. We also highlight current challenges facing medical VLM development, including limited data availability, concerns with data privacy, and lack of proper evaluation metrics, among others, while also proposing future directions to address these obstacles. Overall, our review summarizes the recent progress in developing VLMs to harness multimodal medical data for improved healthcare applications.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3389/frai.2024.1430984

Cites

70 of 70 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 70 references

Research works printed but not linked: 126

  1. Abacha, 2020, “Overview of the VQA-Med task at ImageCLEF 2020: visual question answering and generation in the medical domain,”
  2. Abacha, 2019, “VQA-Med: overview of the medical visual question answering task at imageclef 2019,”
  3. Ahmed, 2023, Transformers in time-series analysis: a tutorial, Circ. Syst. Sign. Process, 42, 7433, 10.48550/arXiv.2205.01138 · arXiv · doi:10.48550/arxiv.2205.01138
  4. Alayrac, 2022, Flamingo: a visual language model for few-shot learning, Adv. Neural Inform. Process. Syst, 35, 23716, 10.48550/arXiv.2204.14198 · DOI · Paper page
  5. Allan, 2020, 2018 robotic scene segmentation challenge, arXiv Preprint arXiv:2001.11190, 10.48550/arXiv.2001.11190 · DOI
  6. Allan, 2019, 2017 robotic instrument segmentation challenge, arXiv Preprint arXiv:1902.06426, 10.48550/arXiv.1902.06426 · DOI
  7. Bai, Qwen-VL: a versatile vision-language model for understanding, localization, text reading, and beyond, arXiv Preprint arXiv:2308.12966, 10.48550/arXiv.2308.12966 · DOI · Paper page
  8. Bai, “CAT-ViL: co-attention gated vision-language embedding for visual question localized-answering in robotic surgery,”, 397
  9. Bai, 2018, A survey on automatic image caption generation, Neurocomputing, 311, 80, 10.1016/j.neucom.2018.05.080 · DOI · Paper page
  10. Bajwa, 2021, Artificial intelligence in healthcare: transforming the practice of medicine, Fut. Healthc. J, 8, e188, 10.7861/fhj.2021-0095 · DOI · Paper page
  11. Baldi, 2021, Deep Learning in Science, 10.1017/9781108955652 · DOI
  12. Banerjee, 2005, “METEOR: an automatic metric for MT evaluation with improved correlation with human judgments,”, ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, 65
  13. Barhoumi, 2023, Efficient scopeformer: toward scalable and rich feature extraction for intracranial hemorrhage detection, IEEE Access, 11, 81656, 10.48550/arXiv.2302.00220 · arXiv · doi:10.48550/arxiv.2302.00220
  14. Bazi, 2023, Vision—language model for visual question answering in medical imagery, Bioengineering, 10, 380, 10.3390/bioengineering10030380 · DOI
  15. Beam, 2020, Clinical concept embeddings learned from massive sources of multimodal medical data, Pacif. Symp. Biocomput, 25, 295, 10.1142/9789811215636_0027 · DOI · Paper page
  16. Ben Abacha, 2019, “Overview of the MEDIQA 2019 shared task on textual inference, question entailment and question answering,”, BioNLP Workshop and Shared Task, 370, 10.18653/v1/W19-5039 · DOI
  17. Bigolin Lanfredi, 2022, Reflacx, a dataset of reports and eye-tracking data for localization of abnormalities in chest x-rays, Sci. Data, 9, 1441, 10.1038/s41597-022-01441-z · DOI
  18. Boecking, 2022, Making the most of text semantics to improve biomedical vision–language processing, Comput. Vis, 5, 1, 10.1007/978-3-031-20059-5 · doi:10.1007/978-3-031-20059-5
  19. Cai, 2024, Dynamic transformer architecture for continual learning of multimodal tasks, arXiv Preprint arXiv:2401.15275, 10.48550/arXiv.2401.15275 · arXiv · doi:10.48550/arxiv.2401.15275
  20. Chen, 2023, VLP: a survey on vision-language pre-training, Machine Intell. Res, 20, 38, 10.1007/s11633-022-1369-5 · DOI · Paper page
  21. Chen, A simple framework for contrastive learning of visual representations, arXiv Preprint arXiv:2002.05709, 10.48550/arXiv.2002.05709 · DOI · Paper page
  22. Cho, 2021, “Unifying vision-and-language tasks via text generation,”, 1931
  23. Cho, 2014, “Learning phrase representations using rnn encoder—decoder for statistical machine translation,”, Conference on Empirical Methods in Natural Language Processing, 1724
  24. Clunie, 2024, Summary of the National Cancer Institute 2023 virtual workshop on medical image de-identification—part 2: pathology whole slide image de-identification, de-facing, the role of AI in image de-identification, and the NCI MIDI datasets and pipeline, J. Imag. Informat. Med, 24, 1183, 10.1007/s10278-024-01183-x · DOI
  25. Dai, 2023, InstructBLIP: towards general-purpose vision-language models with instruction tuning, arXiv Preprint arXiv:2305.06500, 10.48550/arXiv.2305.06500 · arXiv · doi:10.48550/arxiv.2305.06500 · Paper page
  26. Dosovitskiy, 2021, “An image is worth 16x16 words: transformers for image recognition at scale,”
  27. Dou, 2022, “An empirical study of training end-to-end vision-and-language transformers,”, 18145
  28. Dwork, 2006, “Differential privacy,”, 1, 10.1007/11787006_1 · DOI · Paper page
  29. Eslami, 2023, PubmedCLIP: how much does clip benefit visual question answering in the medical domain?, Find. Assoc. Comput. Linguist, 88, 1181, 10.18653/v1/2023.findings-eacl.88 · DOI · Paper page
  30. Gan, 2022, Vision-language pre-training: basics, recent advances, and future trends, arXiv Preprint arXiv:2210.09263, 10.48550/arXiv.2210.09263 · DOI
  31. Goodfellow, 2016, Deep Learning
  32. Han, 2023, MedAlpaca—an open-source collection of medical conversational AI models and training data, arXiv Preprint arXiv:2304.08247, 10.48550/arXiv.2304.08247 · DOI
  33. Hao, 2020, Probabilistic predictions of people perusing: evaluating metrics of language model performance for psycholinguistic modeling, arXiv Preprint arXiv:2009.03954, 10.18653/v1/2020.cmcl-1.10 · DOI
  34. He, SA-Attack: improving adversarial transferability of vision-language pre-training models via self-augmentation, arXiv Preprint arXiv:2312.04913, 10.48550/arXiv.2312.04913 · DOI
  35. He, 2020, PathVQA: 30000+ questions for medical visual question answering, arXiv Preprint arXiv:2003.10286, 10.48550/arXiv.2003.10286 · DOI · Paper page
  36. Hu, 2022, “LoRA: low-rank adaptation of large language models,”
  37. Huang, 2022, Convolutional networks with dense connectivity, IEEE Trans. Pat. Anal. Machine Intell, 44, 8704, 10.1109/TPAMI.2019.2918284 · DOI · Paper page
  38. Huang, 2021, “What makes multimodal learning better than single (provably),”
  39. Ionescu, 2021, Overview of the imageclef 2021: multimedia retrieval in medical, nature, internet and social media applications, Exp. IR Meets Multilingual. Multimodal. Interact, 23, 345, 10.1007/978-3-030-85251-1_23 · DOI
  40. Irvin, 2019, CheXpert: a large chest radiograph dataset with uncertainty labels and expert comparison, AAAI Conf. Artif. Intell, 33, 590, 10.1609/aaai.v33i01.3301590 · DOI · Paper page
  41. Jeong, 2023, Multimodal image-text matching improves retrieval-based chest x-ray report generation, arXiv Preprint arXiv:2303.17579, 10.48550/arXiv.2303.17579 · DOI
  42. Ji, 2020, 5—computer vision applications, Comput. Vis. Pat. Recogn, 10, 191, 10.1016/B978-0-12-803467-5.00010-1 · DOI
  43. Jin, 2019, PubMedQA: a dataset for biomedical research question answering, Conference on Empirical Methods in Natural Language Processing, 2567
  44. Johnson, MIMIC-CXR, a de-identified publicly available database of chest radiographs with free-text reports, Sci. Data, 6, 317, 10.1038/s41597-019-0322-0 · DOI · Paper page
  45. Kayser, 2022, Explaining chest x-ray pathologies in natural language, Int. Conf. Med. Image Comput. Computer-Assist. Interv, 13435, 701, 10.1007/978-3-031-16443-9_67 · DOI · Paper page
  46. Khan, 2023, “The importance of robust features in mitigating catastrophic forgetting,”, 752, 10.1109/ISCC58397.2023.10218203 · DOI
  47. Khan, 2024, Brain-inspired continual learning: robust feature distillation and re-consolidation for class incremental learning, IEEE Access, 2024, 14588, 10.48550/arXiv.2404.14588 · arXiv · doi:10.48550/arxiv.2404.14588
  48. Kim, 2018, Bilinear attention networks, Adv. Neural Inform. Process. Syst, 31, 1564, 10.48550/arXiv.1805.07932 · DOI · Paper page
  49. Kingma, 2014, “ADAM: a method for stochastic optimization,”
  50. Koutsoubis, Future-Proofing Medical Imaging with Privacy-Preserving Federated Learning and Uncertainty Quantification: A Review
  51. Koutsoubis, Privacy Preserving Federated Learning in Medical Imaging with Uncertainty Estimation
  52. Kwon, 2023, Masked vision and language modeling for multi-modal representation learning, arXiv Preprint arXiv:2208.02131, 10.48550/arXiv.2208.02131 · DOI
  53. Lau, 2018, A dataset of clinically generated visual questions and answers about radiology images, Sci. Data, 5, 180251, 10.1038/sdata.2018.251 · DOI · Paper page
  54. Li, 2022, “Cross-modal clinical graph transformer for ophthalmic report generation,”, 20624
  55. Li, “Masked vision and language pre-training with unimodal and multimodal contrastive losses for medical visual question answering,”, Medical Image Computing and Computer Assisted Intervention (MICCAI), 374
  56. Lin, 2004, “ROUGE: a package for automatic evaluation of summaries,”, 74
  57. Lin, PMC-CLIP: contrastive language-image pre-training using biomedical documents, arXiv Preprint arXiv:2303.07240, 10.48550/arXiv.2303.07240 · DOI
  58. Lin, Medical visual question answering: a survey, Artif. Intell. Med, 143, 102611, 10.1016/j.artmed.2023.102611 · DOI · Paper page
  59. Liu, SLAKE: a semantically-labeled knowledge-enhanced dataset for medical visual question answering, IEEE 18th International Symposium on Biomedical Imaging (ISBI), 1650
  60. Liu, Medical visual question answering via conditional reasoning and contrastive learning, IEEE Trans. Med. Imag, 42, 1532, 10.1109/TMI.2022.3232411 · DOI
  61. Liu, A systematic review of deep learning-based research on radiology report generation, arXiv Preprint arXiv:2311.14199, 10.48550/arXiv.2311.14199 · DOI
  62. Liu, 2022, DePlot: one-shot visual language reasoning by plot-to-table translation, arXiv Preprint arXiv:2212.10505, 10.48550/arXiv.2212.10505 · DOI
  63. Liu, 2024, A survey on hallucination in large vision-language models, arXiv Preprint arXiv:2402.00253, 10.48550/arXiv.2402.00253 · DOI · Paper page
  64. Liu, “Swin transformer: hierarchical vision transformer using shifted windows,”, 9992
  65. Lu, 2019, ViLBERT: pretraining task-agnostic visiolinguistic representations for vision-and-language tasks, Advances in Neural Information Processing Systems, 13
  66. Mabotuwana, 2020, Framework for extracting critical findings in radiology reports, J. Digit. Imag, 33, 988, 10.1007/s10278-020-00349-7 · doi:10.1007/s10278-020-00349-7
  67. Manzari, 2023, MedViT: a robust vision transformer for generalized medical image classification, Comput. Biol. Med, 157, 106791, 10.1016/j.compbiomed.2023.106791 · DOI · Paper page
  68. Masci, 2011, “Stacked convolutional auto-encoders for hierarchical feature extraction,”, 52
  69. Mishra, 2021, “VT-ADL: a vision transformer network for image anomaly detection and localization,”, 01
  70. Mohsan, 2023, Vision transformer and language model based radiology report generation, IEEE Access, 11, 1814, 10.1109/ACCESS.2022.3232719 · DOI
  71. Monshi, 2020, Deep learning in generating radiology reports: a survey, Artif. Intell. Med, 106, 101878, 10.1016/j.artmed.2020.101878 · DOI · Paper page
  72. Moon, 2022, Multi-modal understanding and generation for medical images and text via vision-language pre-training, IEEE J. Biomed. Health Informat, 26, 6070, 10.1109/JBHI.2022.3207502 · DOI · Paper page
  73. Moor, 2023, Med-Flamingo: a multimodal medical few-shot learner, arXiv Preprint arXiv:2307.15189, 10.48550/arXiv.2307.15189 · DOI
  74. Nadkarni, 2011, Natural language processing: an introduction, J. Am. Med. Informat. Assoc, 18, 544, 10.1136/amiajnl-2011-000464 · DOI · Paper page
  75. Norgeot, 2020, Protected health information filter (philter): accurately and securely de-identifying free-text clinical notes, NPJ Digit. Med, 3, 258, 10.1038/s41746-020-0258-y · DOI
  76. Papineni, 2002, “BLEU: a method for automatic evaluation of machine translation,”, Annual Meeting of the Association for Computational Linguistics, 311
  77. Pelka, 2018, Radiology objects in context (ROCO): a multimodal image dataset, Intravasc. Imag. Comput. Assist. Stent. Large-Scale Annot. Biomed. Data Expert Label Synth, 11043, 180, 10.1007/978-3-030-01364-6_20 · DOI · Paper page
  78. Pellegrini, 2023, Radialog: a large vision-language model for radiology report generation and conversational assistance, arXiv Preprint arXiv:2311.18681, 10.48550/arXiv.2311.18681 · DOI
  79. Pennington, 2014, Glove: global vectors for word representation, Empir. Methods Natur. Lang. Process, 14, 1532, 10.3115/v1/D14-1162 · DOI · Paper page
  80. Rai, 2021, Study of various methods for tokenization, Appl. Internet Things, 18, 193, 10.1007/978-981-15-6198-6_18 · DOI
  81. Ramesh, 2022, Improving radiology report generation systems by removing hallucinated references to non-existent priors, Machine Learn. Res, 193, 456, 10.48550/arXiv.2210.06340 · DOI · Paper page
  82. Ranftl, 2021, “Vision transformers for dense prediction,”, 12159
  83. Ranjit, 2023, Retrieval augmented chest X-ray report generation using openAI GPT models, arXiv Preprint arXiv:2305.03660, 10.48550/arXiv.2305.03660 · DOI
  84. Ren, 2024, Learning or self-aligning? rethinking instruction fine-tuning, arXiv Preprint arXiv:2402.18243, 10.48550/arXiv.2402.18243 · arXiv · doi:10.48550/arxiv.2402.18243
  85. Rezatofighi, 2019, “Generalized intersection over union: a metric and a loss for bounding box regression,”, IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 658
  86. Romanov, 2018, “Lessons from natural language inference in the clinical domain,”, 1586
  87. Rückert, 2022, “Overview of imageclefmedical 2022—caption prediction and concept detection,”, CEUR Workshop Proceedings, Vol. 3180, 1294
  88. Schmidt, 2019, Recurrent neural networks (RNNS): a gentle introduction and overview, arXiv Preprint arXiv:1912.05911, 10.48550/arXiv.1912.05911 · DOI
  89. Seenivasan, 2022, “Surgical-VQA: visual question answering in surgical scenes using transformer,”, Medical Image Computing and Computer Assisted Intervention—MICCAI, 33
  90. Sengupta, 2023, Automatic report generation for histopathology images using pre-trained vision transformers and BERT, arXiv Preprint arXiv:2312.01435, 10.48550/arXiv.2312.01435 · arXiv · doi:10.48550/arxiv.2312.01435
  91. Sharma, 2023, Evolution of visual data captioning methods, datasets, and evaluation metrics: a comprehensive survey, Expert Syst. Appl, 221, 119773, 10.1016/j.eswa.2023.119773 · DOI
  92. Shrestha, 2023, Medical vision language pretraining: a survey, arXiv Preprint arXiv:2312.06224, 10.48550/arXiv.2312.06224 · DOI
  93. Soviany, 2021, Curriculum learning: a survey, Int. J. Comput. Vis, 130, 1526, 10.1007/s11263-022-01611-x · DOI · Paper page
  94. Stripelis, 2021, Secure neuroimaging analysis using federated learning with homomorphic encryption, SPIE Med. Imag, 22, 1611, 10.48550/arXiv.2108.03437 · arXiv · doi:10.48550/arxiv.2108.03437
  95. Subramanian, 2020, “MediCAT: a dataset of medical images, captions, and textual references,”, 2112, 10.18653/v1/2020.findings-emnlp.191 · DOI
  96. Sun, 2023, Aligning large multimodal models with factually augmented RLHF, arXiv Preprint arXiv:2309.14525, 10.48550/arXiv.2309.14525 · DOI
  97. Tanwani, 2022, “RepsNet: combining vision with language for automated medical reports,”, Medical Image Computing and Computer Assisted Intervention (MICCAI), 714
  98. Taylor, 1953, “Cloze procedure”: a new tool for measuring readability, J. Mass Commun. Quart, 30, 415, 10.1177/107769905303000401 · DOI · Paper page
  99. Thawkar, 2023, XrayGPT: chest radiographs summarization using medical vision-language models, arXiv Preprint arXiv:2306.07971, 10.48550/arXiv.2306.07971 · DOI
  100. Ting, 2023, A survey on automatic generation of medical imaging reports based on deep learning, BioMed. Eng. OnL, 22, 1113, 10.1186/s12938-023-01113-y · DOI
  101. Touvron, LLaMA 2: open foundation and fine-tuned chat models, arXiv Preprint arXiv:2307.09288, 10.48550/arXiv.2307.09288 · DOI · Paper page
  102. Tripathi, Building flexible, scalable, and machine learning-ready multimodal oncology datasets, Sensors, 24, 51634, 10.3390/s24051634 · DOI
  103. Tripathi, HoneyBee: A Scalable Modular Framework for Creating Multimodal Oncology Datasets with Foundational Embedding Models
  104. Tyagi, 2021, “Detecting pneumonia using vision transformer and comparing with other techniques,”, International Conference on Electronics, Communication and Aerospace Technology (ICECA), 12
  105. Vaswani, 2017, Attention is all you need, Adv. Neural Inform. Process. Syst, 30, 5998, 10.48550/arXiv.1706.03762 · DOI · Paper page
  106. Verspoor, 2013, Encyclopedia of Systems Biology, Chapter Natural Language Processing, 1495, 10.1007/978-1-4419-9863-7_158 · doi:10.1007/978-1-4419-9863-7_158
  107. Wang, GIT: a generative image-to-text transformer for vision and language, arXiv Preprint arXiv:2205.14100, 10.48550/arXiv.2205.14100 · arXiv · doi:10.48550/arxiv.2205.14100 · Paper page
  108. Wang, MedCLIP: contrastive learning from unpaired medical images and text, arXiv Preprint arXiv:2210.10163, 10.48550/arXiv.2210.10163 · DOI
  109. Wang, “SimVLM: simple visual language model pretraining with weak supervision,”, International Conference on Learning Representations (ICLR)
  110. Waqas, 2023, Revolutionizing digital pathology with the power of generative artificial intelligence and foundation models, Lab. Invest, 103, 100255, 10.1016/j.labinv.2023.100255 · DOI
  111. Waqas, Digital pathology and multimodal learning on oncology data, Artif. Intell, 1, 1, 10.1093/bjrai/ubae014 · DOI
  112. Waqas, Embedding-based Multimodal Learning on Pan-Squamous Cell Carcinomas for Improved Survival Outcomes
  113. Xie, 2016, “Aggregated residual transformations for deep neural networks,”, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 5987
  114. Xie, 2022, “SimMIM: a simple framework for masked image modeling,”, IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 9643
  115. Xin, 2022, An improved transformer network for skin cancer classification, Comput. Biol. Med, 149, 105939, 10.1016/j.compbiomed.2022.105939 · DOI · Paper page
  116. Xu, 2021, “Learning domain adaptation with model calibration for surgical report generation in robotic surgery,”, 2021 IEEE International Conference on Robotics and Automation (ICRA), 12350, 10.1109/ICRA48506.2021.9561569 · DOI
  117. Yang, 2022, A large language model for electronic health records, NPJ Digit. Med, 5, 9, 10.1038/s41746-022-00742-2 · DOI · Paper page
  118. Yuan, 2023, RAMM: retrieval-augmented biomedical visual question answering with multi-modal pre-training, arXiv Preprint arXiv:2303.00534, 10.48550/arXiv.2303.00534 · arXiv · doi:10.48550/arxiv.2303.00534
  119. Zeng, 2020, “MedDialog: large-scale medical dialogue datasets,”, Conference on Empirical Methods in Natural Language Processing (EMNLP), 9241, 10.18653/v1/2020.emnlp-main.743 · DOI · Paper page
  120. Zhan, 2020, “Medical visual question answering via conditional reasoning,”, The 28th ACM International Conference on Multimedia, 2345
  121. Zhang, 2021, A survey on federated learning, Knowl. Based Syst, 216, 106775, 10.1016/j.knosys.2021.106775 · DOI · Paper page
  122. Zhang, 2018, “Grounding referring expressions in images by variational context,”, IEEE/CVF Conference on Computer Vision and Pattern Recognition, 4158
  123. Zhang, Large-scale domain-specific pretraining for biomedical vision-language processing, arXiv Preprint arXiv:2303.00915, 10.48550/arXiv.2303.00915 · DOI
  124. Zhang, Adapter learning in pretrained feature extractor for continual learning of diseases, arXiv Preprint arXiv:2304.09042, 10.48550/arXiv.2304.09042 · arXiv · doi:10.48550/arxiv.2304.09042
  125. Zhao, 2023, Retrieving multimodal information for augmented generation: a survey, arXiv Preprint arXiv:2303.10868, 10.48550/arXiv.2303.10868 · DOI
  126. Zhou, Learning without forgetting for vision-language models, arXiv Preprint arXiv:2305.19270, 10.48550/arXiv.2305.19270 · arXiv · doi:10.48550/arxiv.2305.19270

Cited by

See all 229

Metadata

Authors
Hartsock, I., & Rasool, G.
Venue
Frontiers in Artificial Intelligence
Publisher
Frontiers Media SA
Published
2024-11-19
Volume
7
Pages
1430984-1430984
ISSN
2624-8212 (ISSN-L)
DOI
10.3389/frai.2024.1430984
OpenAlex
W4404509535
Cited by
229
References
70
FWCI
32.35
Type
journal article
Language
en
Open access
gold
Licence
creativecommons.org/licenses/by/4.0/ (vor)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Vision-language models for medical report generation and visual question answering: a review. https://app.cochat.ai/papers/10.3389/frai.2024.1430984
The paper
Hartsock, I., & Rasool, G. (2024). Vision-language models for medical report generation and visual question answering: a review. Frontiers in Artificial Intelligence. https://doi.org/10.3389/frai.2024.1430984

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.