Vision-language models for medical report generation and visual question answering: a review
Open accessThe DOI matches its Crossref record; 93 of the 93 references it cites were found in a registry or an index.
Abstract
Medical vision-language models (VLMs) combine computer vision (CV) and natural language processing (NLP) to analyze visual and textual medical data. Our paper reviews recent advancements in developing VLMs specialized for healthcare, focusing on publicly available models designed for medical report generation and visual question answering (VQA). We provide background on NLP and CV, explaining how techniques from both fields are integrated into VLMs, with visual and language data often fused using Transformer-based architectures to enable effective learning from multimodal data. Key areas we address include the exploration of 18 public medical vision-language datasets, in-depth analyses of the architectures and pre-training strategies of 16 recent noteworthy medical VLMs, and comprehensive discussion on evaluation metrics for assessing VLMs' performance in medical report generation and VQA. We also highlight current challenges facing medical VLM development, including limited data availability, concerns with data privacy, and lack of proper evaluation metrics, among others, while also proposing future directions to address these obstacles. Overall, our review summarizes the recent progress in developing VLMs to harness multimodal medical data for improved healthcare applications.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · cited 1×
“While RAG usually involves a frozen model during training, exploring the pre-training of VLMs within the RAG framework opens up a new avenue of research [Zha+23d].”
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
End-to-End Object Detection with Transformers
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. (2020) · arXiv · doi:10.48550/arxiv.2005.12872 · Cited by 18,253
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
Tan, M., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1905.11946 · Cited by 17,072
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293
Enriching Word Vectors with Subword Information
Bojanowski, P., Grave, E., Joulin, A., & Mikolov, T. (2017) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00051 · Cited by 9,645 · cited 1×
“Machine translation is a fundamental NLP task in breaking down language barriers by automatically translating text from one language to another.”
A Stochastic Approximation Method
Robbins, H., & Monro, S. (1951) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177729586 · Cited by 9,643
Generalized Intersection Over Union: A Metric and a Loss for Bounding Box Regression
Rezatofighi, H., Tsoi, N., Gwak, J., Sadeghian, A., Reid, I., & Savarese, S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00075 · Cited by 6,612
Representation Learning with Contrastive Predictive Coding
Oord, A. V. D., Li, Y., & Vinyals, O. (2018) · arXiv · doi:10.48550/arxiv.1807.03748 · Cited by 6,242 · cited 1×
“Retrieval augmented generation (RAG) is a form of prompt engineering that involves strategically crafting prompts for both retrieval and generation phases, allowing for an adaptive and efficient process that leverages external knowledge sources to enhance generative tasks.”
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
Wu, Y., Schuster, M., Chen, Z., Le, Q. V., Norouzi, M., Macherey, W., Krikun, M., Cao, Y., Gao, Q., Macherey, K., et al. (2016) · arXiv · doi:10.48550/arxiv.1609.08144 · Cited by 5,319
Convolutional neural networks: an overview and application in radiology
Yamashita, R., Nishio, M., Do, R. K. G., & Togashi, K. (2018) · Insights into Imaging · doi:10.1007/s13244-018-0639-9 · Cited by 4,682
LLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602
VQA: Visual Question Answering
Antol, S., Agrawal, A., Lu, J., Mitchell, M., Batra, D., Zitnick, C. L., & Parikh, D. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.279 · Cited by 4,487
Training language models to follow instructions with human feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.02155 · Cited by 4,208
Large Language Models Encode Clinical Knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2212.13138 · Cited by 4,083
Large language models encode clinical knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2023) · Nature · doi:10.1038/s41586-023-06291-2 · Cited by 4,083 · cited 3×
“Pre-trained LLMs that are part of medical VLMs architecture are often fine-tuned using LoRA (e.g., Visual Med-Alpaca [Shu+23] and RaDialog [Pel+23]).”
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-T., Rocktäschel, T., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.11401 · Cited by 3,101
The Power of Scale for Parameter-Efficient Prompt Tuning
Lester, B., Al-Rfou, R., & Constant, N. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.243 · Cited by 3,096
The Power of Scale for Parameter-Efficient Prompt Tuning
Lester, B., Al-Rfou, R., & Constant, N. (2021) · arXiv · doi:10.48550/arxiv.2104.08691 · Cited by 3,092
BERTScore: Evaluating Text Generation with BERT
Zhang, T., Kishore, V., Wu, F., Weinberger, K. Q., & Artzi, Y. (2019) · arXiv · doi:10.48550/arxiv.1904.09675 · Cited by 2,865
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
Llama 2: Open Foundation and Fine-Tuned Chat Models
Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.09288 · Cited by 2,604
PaLM: Scaling Language Modeling with Pathways
Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.02311 · Cited by 2,590
Taming Transformers for High-Resolution Image Synthesis
Esser, P., Rombach, R., & Ommer, B. (2020) · arXiv · doi:10.48550/arxiv.2012.09841 · Cited by 2,514
Taming Transformers for High-Resolution Image Synthesis
Esser, P., Rombach, R., & Ommer, B. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01268 · Cited by 2,514 · cited 1×
“The model tokenizes chest X-rays leveraging VQGAN [ERO21], a generative model that amalgamates generative adversarial networks (GANs) with vector quantization (VQ) techniques.”
Prefix-Tuning: Optimizing Continuous Prompts for Generation
Li, X. L., & Liang, P. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.353 · Cited by 2,473
“Cloze Procedure”: A New Tool for Measuring Readability
Taylor, W. L. (1953) · Journalism Quarterly · doi:10.1177/107769905303000401 · Cited by 2,332
Domain-specific language model pretraining for biomedical natural language processing
Gu, Y., Tinn, R., Cheng, H., Lucas, M., Usuyama, N., Liu, X., Naumann, T., Gao, J., & Poon, H. (2022) · ACM Transactions on Computing for Healthcare · doi:10.1145/3458754 · Cited by 2,170
ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks
Lu, J., Batra, D., Parikh, D., & Lee, S. (2019) · arXiv · doi:10.48550/arxiv.1908.02265 · Cited by 2,109
UNITER: UNiversal Image-TExt Representation Learning
Chen, Y.-C., Li, L., Yu, L., El Kholy, A., Ahmed, F., Gan, Z., Cheng, Y., & Liu, J. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58577-8_7 · Cited by 2,055
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision
Jia, C., Yang, Y., Xia, Y., Chen, Y.-T., Parekh, Z., Pham, H., Le, Q. V., Sung, Y., Li, Z., & Duerig, T. (2021) · arXiv · doi:10.48550/arxiv.2102.05918 · Cited by 1,977 · cited 2×
“A prevalent strategy involves intensive pre-training on datasets where images/videos are paired with their corresponding textual descriptions.”
A survey on federated learning
Zhang, C., Xie, Y., Bai, H., Yu, B., Li, W., & Gao, Y. (2021) · Knowledge-Based Systems · doi:10.1016/j.knosys.2021.106775 · Cited by 1,899
Artificial intelligence in healthcare: transforming the practice of medicine
Bajwa, J., Munir, U., Nori, A., & Williams, B. (2021) · Future Healthcare Journal · doi:10.7861/fhj.2021-0095 · Cited by 1,856
MIMIC-CXR, a de-identified publicly available database of chest radiographs with free-text reports
Johnson, A. E. W., Pollard, T. J., Berkowitz, S. J., Greenbaum, N. R., Lungren, M. P., Deng, C.-Y., Mark, R. G., & Horng, S. (2019) · Scientific Data · doi:10.1038/s41597-019-0322-0 · Cited by 1,816
Towards a general-purpose foundation model for computational pathology
Chen, R. J., Ding, T., Lu, M. Y., Williamson, D. F. K., Jaume, G., Song, A. H., Chen, B., Zhang, A., Shao, D., Shaban, M., et al. (2024) · Nature Medicine · doi:10.1038/s41591-024-02857-3 · Cited by 1,560
Flamingo: a Visual Language Model for Few-Shot Learning
Alayrac, J.-B., Donahue, J., Luc, P., Miech, A., Barr, I., Hasson, Y., Lenc, K., Mensch, A., Millican, K., Reynolds, M., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.14198 · Cited by 1,555
Acosta, J. N., Falcone, G. J., Rajpurkar, P., & Topol, E. J. (2022) · Nature Medicine · doi:10.1038/s41591-022-01981-2 · Cited by 1,365
Self-Instruct: Aligning Language Models with Self-Generated Instructions
Wang, Y., Kordi, Y., Mishra, S., Liu, A., Smith, N. A., Khashabi, D., & Hajishirzi, H. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.754 · Cited by 1,243
VisualBERT: A Simple and Performant Baseline for Vision and Language
Li, L. H., Yatskar, M., Yin, D., Hsieh, C.-J., & Chang, K.-W. (2019) · arXiv · doi:10.48550/arxiv.1908.03557 · Cited by 1,198 · cited 1×
“Following the retrieval phase, the model returns a set of contexts related to the given input.”
Preparing a collection of radiology examinations for distribution and retrieval
Demner-Fushman, D., Kohli, M. D., Rosenman, M. B., Shooshan, S. E., Rodriguez, L., Antani, S., Thoma, G. R., & McDonald, C. J. (2016) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocv080 · Cited by 1,164
A Comprehensive Survey of Continual Learning: Theory, Method and Application
Wang, L., Zhang, X., Su, H., & Zhu, J. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3367329 · Cited by 1,113
Align before Fuse: Vision and Language Representation Learning with Momentum Distillation
Li, J., Selvaraju, R. R., Gotmare, A. D., Joty, S., Xiong, C., & Hoi, S. (2021) · arXiv · doi:10.48550/arxiv.2107.07651 · Cited by 1,078
LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day
Li, C., Wong, C., Zhang, S., Usuyama, N., Liu, H., Yang, J., Naumann, T., Poon, H., & Gao, J. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-1240 · Cited by 986
A visual-language foundation model for computational pathology
Lu, M. Y., Chen, B., Williamson, D. F. K., Chen, R. J., Liang, I., Ding, T., Jaume, G., Odintsov, I., Le, L. P., Gerber, G., et al. (2024) · Nature Medicine · doi:10.1038/s41591-024-02856-4 · Cited by 966
Neural Machine Translation of Rare Words with Subword Units
Sennrich, R., Haddow, B., & Birch, A. (2015) · arXiv · doi:10.48550/arxiv.1508.07909 · Cited by 950
A large language model for electronic health records
Yang, X., Chen, A., PourNejatian, N., Shin, H. C., Smith, K. E., Parisien, C., Compas, C., Martin, C., Costa, A. B., Flores, M. G., et al. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00742-2 · Cited by 847 · cited 1×
“Noteworthy examples of such LLMs include GatorTron [Yan+22], ChatDoctor [Li+23c], Med-PaLM (Medical Pathways Language Model) [Sin+23] and Med-Alpaca [Han+23].”
From Recognition to Cognition: Visual Commonsense Reasoning
Zellers, R., Bisk, Y., Farhadi, A., & Choi, Y. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00688 · Cited by 769
Jin, D., Pan, E., Oufattole, N., Weng, W.-H., Fang, H., & Szolovits, P. (2021) · Applied Sciences · doi:10.3390/app11146421 · Cited by 718
Jin, D., Pan, E., Oufattole, N., Weng, W.-H., Fang, H., & Szolovits, P. (2021) · doi:10.20944/preprints202105.0498.v1 · Cited by 718
PubMedQA: A Dataset for Biomedical Research Question Answering
Jin, Q., Dhingra, B., Liu, Z., Cohen, W., & Lu, X. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1259 · Cited by 685
Liu, H., Li, C., Wu, Q., & Lee, Y. J. (2023) · arXiv · doi:10.48550/arxiv.2304.08485 · Cited by 682
Reproducible Scaling Laws for Contrastive Language-Image Learning
Cherti, M., Beaumont, R., Wightman, R., Wortsman, M., Ilharco, G., Gordon, C., Schuhmann, C., Schmidt, L., & Jitsev, J. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00276 · Cited by 673
Harnessing multimodal data integration to advance precision oncology
Boehm, K. M., Khosravi, P., Vanguri, R., Gao, J., & Shah, S. P. (2022) · Nature Reviews Cancer · doi:10.1038/s41568-021-00408-3 · Cited by 662
LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day
Li, C., Wong, C., Zhang, S., Usuyama, N., Liu, H., Yang, J., Naumann, T., Poon, H., & Gao, J. (2023) · arXiv · doi:10.48550/arxiv.2306.00890 · Cited by 590
Reddy, S. (2024) · Implementation Science · doi:10.1186/s13012-024-01357-9 · Cited by 554
Li, Y., Li, Z., Zhang, K., Dan, R., Jiang, S., & Zhang, Y. (2023) · arXiv · doi:10.48550/arxiv.2303.14070 · Cited by 512
Deep Supervised Cross-Modal Retrieval
Zhen, L., Hu, P., Wang, X., & Peng, D. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01064 · Cited by 478
A multimodal generative AI copilot for human pathology
Lu, M. Y., Chen, B., Williamson, D. F. K., Chen, R. J., Zhao, M., Chow, A. K., Ikemura, K., Kim, A., Pouli, D., Patel, A., et al. (2024) · Nature · doi:10.1038/s41586-024-07618-3 · Cited by 461
S2ORC: The Semantic Scholar Open Research Corpus
Lo, K., Wang, L. L., Neumann, M., Kinney, R., & Weld, D. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.447 · Cited by 457
BioWordVec, improving biomedical word embeddings with subword information and MeSH
Zhang, Y., Chen, Q., Yang, Z., Lin, H., & Lu, Z. (2019) · Scientific Data · doi:10.1038/s41597-019-0055-0 · Cited by 455
Fine-Tuning Language Models from Human Preferences
Ziegler, D. M., Stiennon, N., Wu, J., Brown, T. B., Radford, A., Amodei, D., Christiano, P., & Irving, G. (2019) · arXiv · doi:10.48550/arxiv.1909.08593 · Cited by 413
Jiang, A. Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D. S., Casas, D. D. L., Bressand, F., Lengyel, G., Lample, G., Saulnier, L., et al. (2023) · arXiv · doi:10.48550/arxiv.2310.06825 · Cited by 340 · cited 1×
“This motivated the development of VLMs, which can handle and understand both NLP and CV data simultaneously.”
Reinforcement learning for intelligent healthcare applications: A survey
Coronato, A., Naeem, M., De Pietro, G., & Paragliola, G. (2020) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2020.101964 · Cited by 320
Transformers in Time-Series Analysis: A Tutorial
Ahmed, S., Nielsen, I. E., Tripathi, A., Siddiqui, S., Ramachandran, R. P., & Rasool, G. (2023) · Circuits, Systems, and Signal Processing · doi:10.1007/s00034-023-02454-8 · Cited by 305
Self-citationSelf-supervised learning: A succinct review
Rani, V., Nabi, S. T., Kumar, M., Mittal, A., & Kumar, K. (2023) · Archives of Computational Methods in Engineering · doi:10.1007/s11831-023-09884-2 · Cited by 295
A systematic survey of prompt engineering on vision-language foundation models
Gu, J., Han, Z., Chen, S., Beirami, A., He, B., Zhang, G., Liao, R., Qin, Y., Tresp, V., & Torr, P. (2023) · arXiv · doi:10.48550/arxiv.2307.12980 · Cited by 280
MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographs
Johnson, A. E. W., Pollard, T. J., Greenbaum, N. R., Lungren, M. P., Deng, C.-Y., Peng, Y., Lu, Z., Mark, R. G., Berkowitz, S. J., & Horng, S. (2019) · arXiv · doi:10.48550/arxiv.1901.07042 · Cited by 269
A Survey of Large Language Models in Medicine: Progress, Application, and Challenge
Zhou, H., Liu, F., Gu, B., Zou, X., Huang, J., Wu, J., Li, Y., Chen, S. S., Zhou, P., Liu, J., et al. (2023) · arXiv · doi:10.48550/arxiv.2311.05112 · Cited by 240
Learning to Exploit Temporal Structure for Biomedical Vision-Language Processing
Bannur, S., Hyland, S., Liu, Q., Pérez-García, F., Ilse, M., Castro, D. C., Boecking, B., Sharma, H., Bouzid, K., Thieme, A., et al. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01442 · Cited by 227
Evaluating progress in automatic chest X-ray radiology report generation
Yu, F., Endo, M., Krishnan, R., Pan, I., Tsai, A., Reis, E. P., Fonseca, E. K. U. N., Lee, H. M. H., Abad, Z. S. H., Ng, A. Y., et al. (2023) · Patterns · doi:10.1016/j.patter.2023.100802 · Cited by 200
What Makes Multi-modal Learning Better than Single (Provably)
Huang, Y., Du, C., Xue, Z., Chen, X., Zhao, H., & Huang, L. (2021) · arXiv · doi:10.48550/arxiv.2106.04538 · Cited by 197
Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond
Bai, J., Bai, S., Yang, S., Wang, S., Tan, S., Wang, P., Lin, J., Zhou, C., & Zhou, J. (2023) · arXiv · doi:10.48550/arxiv.2308.12966 · Cited by 193
NegBio: a high-performance tool for negation and uncertainty detection in radiology reports
Peng, Y., Wang, X., Lu, L., Bagheri, M., Summers, R., & Lu, Z. (2017) · arXiv · doi:10.48550/arxiv.1712.05898 · Cited by 175
Multimodal data integration for oncology in the era of deep neural networks: a review
Waqas, A., Tripathi, A., Ramachandran, R. P., Stewart, P. A., & Rasool, G. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1408843 · Cited by 165
Self-citationImproving Factual Completeness and Consistency of Image-to-Text Radiology Report Generation
Miura, Y., Zhang, Y., Tsai, E., Langlotz, C., & Jurafsky, D. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.416 · Cited by 149
Neural Machine Translation with Byte-Level Subwords
Wang, C., Cho, K., & Gu, J. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6451 · Cited by 147
Self-Instruct: Aligning Language Models with Self-Generated Instructions
Wang, Y., Kordi, Y., Mishra, S., Liu, A., Smith, N. A., Khashabi, D., & Hajishirzi, H. (2022) · arXiv · doi:10.48550/arxiv.2212.10560 · Cited by 118
Smit, A., Jain, S., Rajpurkar, P., Pareek, A., Ng, A. Y., & Lungren, M. P. (2020) · arXiv · doi:10.48550/arxiv.2004.09167 · Cited by 103
Application of large language models in medicine
Liu, F., Zhou, H., Gu, B., Zou, X., Huang, J., Wu, J., Li, Y., Chen, S. S., Hua, Y., Zhou, P., et al. (2025) · Nature Reviews Bioengineering · doi:10.1038/s44222-025-00279-5 · Cited by 94
PathVQA: 30000+ Questions for Medical Visual Question Answering
He, X., Zhang, Y., Mou, L., Xing, E., & Xie, P. (2020) · arXiv · doi:10.48550/arxiv.2003.10286 · Cited by 79
A Survey on Hallucination in Large Vision-Language Models
Liu, H., Xue, W., Chen, Y., Chen, D., Zhao, X., Wang, K., Hou, L., Li, R., & Peng, W. (2024) · arXiv · doi:10.48550/arxiv.2402.00253 · Cited by 37
He, K., Mao, R., Lin, Q., Ruan, Y., Lan, X., Feng, M., & Cambria, E. (2023) · arXiv · doi:10.48550/arxiv.2310.05694 · Cited by 27
Investigating the Catastrophic Forgetting in Multimodal Large Language Models
Zhai, Y., Tong, S., Li, X., Cai, M., Qu, Q., Lee, Y. J., & Ma, Y. (2023) · arXiv · doi:10.48550/arxiv.2309.10313 · Cited by 24
Explaining chest X-ray pathologies in natural language
Kayser, M., Emde, C., Camburu, O.-M., Parsons, G., Papiez, B., & Lukasiewicz, T. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-16443-9_67 · Cited by 14
Vision-Language Generative Model for View-Specific Chest X-ray Generation
Lee, H., Lee, D. Y., Kim, W., Kim, J.-H., Kim, T., Kim, J., Sunwoo, L., & Choi, E. (2023) · arXiv · doi:10.48550/arxiv.2302.12172 · Cited by 5
Research works printed but not linked: 110
- Abacha, 2020, “Overview of the VQA-Med task at ImageCLEF 2020: visual question answering and generation in the medical domain,”
- Abacha, 2019, “VQA-Med: overview of the medical visual question answering task at imageclef 2019,”
- Allan, 2020, 2018 robotic scene segmentation challenge, arXiv Preprint arXiv:2001.11190, 10.48550/arXiv.2001.11190 · DOI
- Allan, 2019, 2017 robotic instrument segmentation challenge, arXiv Preprint arXiv:1902.06426, 10.48550/arXiv.1902.06426 · DOI
- Bai, “CAT-ViL: co-attention gated vision-language embedding for visual question localized-answering in robotic surgery,”, 397
- Bai, 2018, A survey on automatic image caption generation, Neurocomputing, 311, 80, 10.1016/j.neucom.2018.05.080 · DOI · Paper page
- Baldi, 2021, Deep Learning in Science, 10.1017/9781108955652 · DOI
- Banerjee, 2005, “METEOR: an automatic metric for MT evaluation with improved correlation with human judgments,”, ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, 65
- Barhoumi, 2023, Efficient scopeformer: toward scalable and rich feature extraction for intracranial hemorrhage detection, IEEE Access, 11, 81656, 10.48550/arXiv.2302.00220 · arXiv · doi:10.48550/arxiv.2302.00220
- Bazi, 2023, Vision—language model for visual question answering in medical imagery, Bioengineering, 10, 380, 10.3390/bioengineering10030380 · DOI · Paper page
- Beam, 2020, Clinical concept embeddings learned from massive sources of multimodal medical data, Pacif. Symp. Biocomput, 25, 295, 10.1142/9789811215636_0027 · DOI · Paper page
- Ben Abacha, 2019, “Overview of the MEDIQA 2019 shared task on textual inference, question entailment and question answering,”, BioNLP Workshop and Shared Task, 370, 10.18653/v1/W19-5039 · DOI · Paper page
- Bigolin Lanfredi, 2022, Reflacx, a dataset of reports and eye-tracking data for localization of abnormalities in chest x-rays, Sci. Data, 9, 1441, 10.1038/s41597-022-01441-z · DOI
- Boecking, 2022, Making the most of text semantics to improve biomedical vision–language processing, Comput. Vis, 5, 1, 10.1007/978-3-031-20059-5 · doi:10.1007/978-3-031-20059-5
- Cai, 2024, Dynamic transformer architecture for continual learning of multimodal tasks, arXiv Preprint arXiv:2401.15275, 10.48550/arXiv.2401.15275 · DOI
- Chen, 2023, VLP: a survey on vision-language pre-training, Machine Intell. Res, 20, 38, 10.1007/s11633-022-1369-5 · DOI · Paper page
- Chen, A simple framework for contrastive learning of visual representations, arXiv Preprint arXiv:2002.05709, 10.48550/arXiv.2002.05709 · DOI · Paper page
- Cho, 2021, “Unifying vision-and-language tasks via text generation,”, 1931
- Clunie, 2024, Summary of the National Cancer Institute 2023 virtual workshop on medical image de-identification—part 2: pathology whole slide image de-identification, de-facing, the role of AI in image de-identification, and the NCI MIDI datasets and pipeline, J. Imag. Informat. Med, 24, 1183, 10.1007/s10278-024-01183-x · DOI
- Dai, 2023, InstructBLIP: towards general-purpose vision-language models with instruction tuning, arXiv Preprint arXiv:2305.06500, 10.48550/arXiv.2305.06500 · arXiv · doi:10.48550/arxiv.2305.06500 · Paper page
- Dosovitskiy, 2021, “An image is worth 16x16 words: transformers for image recognition at scale,”
- Dou, 2022, “An empirical study of training end-to-end vision-and-language transformers,”, 18145
- Dwork, 2006, “Differential privacy,”, 1, 10.1007/11787006_1 · DOI · Paper page
- Eslami, 2023, PubmedCLIP: how much does clip benefit visual question answering in the medical domain?, Find. Assoc. Comput. Linguist, 88, 1181, 10.18653/v1/2023.findings-eacl.88 · DOI · Paper page
- Gan, 2022, Vision-language pre-training: basics, recent advances, and future trends, arXiv Preprint arXiv:2210.09263, 10.48550/arXiv.2210.09263 · DOI
- Goodfellow, 2016, Deep Learning
- Han, 2023, MedAlpaca—an open-source collection of medical conversational AI models and training data, arXiv Preprint arXiv:2304.08247, 10.48550/arXiv.2304.08247 · arXiv · doi:10.48550/arxiv.2304.08247 · Paper page
- Hao, 2020, Probabilistic predictions of people perusing: evaluating metrics of language model performance for psycholinguistic modeling, arXiv Preprint arXiv:2009.03954, 10.18653/v1/2020.cmcl-1.10 · DOI
- He, SA-Attack: improving adversarial transferability of vision-language pre-training models via self-augmentation, arXiv Preprint arXiv:2312.04913, 10.48550/arXiv.2312.04913 · DOI
- Hu, 2022, “LoRA: low-rank adaptation of large language models,”
- Huang, 2022, Convolutional networks with dense connectivity, IEEE Trans. Pat. Anal. Machine Intell, 44, 8704, 10.1109/TPAMI.2019.2918284 · DOI · Paper page
- Huang, 2021, “What makes multimodal learning better than single (provably),”
- Ionescu, 2021, Overview of the imageclef 2021: multimedia retrieval in medical, nature, internet and social media applications, Exp. IR Meets Multilingual. Multimodal. Interact, 23, 345, 10.1007/978-3-030-85251-1_23 · DOI
- Irvin, 2019, CheXpert: a large chest radiograph dataset with uncertainty labels and expert comparison, AAAI Conf. Artif. Intell, 33, 590, 10.1609/aaai.v33i01.3301590 · DOI · Paper page
- Jeong, 2023, Multimodal image-text matching improves retrieval-based chest x-ray report generation, arXiv Preprint arXiv:2303.17579, 10.48550/arXiv.2303.17579 · DOI
- Ji, 2020, 5—computer vision applications, Comput. Vis. Pat. Recogn, 10, 191, 10.1016/B978-0-12-803467-5.00010-1 · DOI
- Khan, 2023, “The importance of robust features in mitigating catastrophic forgetting,”, 752, 10.1109/ISCC58397.2023.10218203 · DOI
- Khan, 2024, Brain-inspired continual learning: robust feature distillation and re-consolidation for class incremental learning, IEEE Access, 2024, 14588, 10.48550/arXiv.2404.14588 · arXiv · doi:10.48550/arxiv.2404.14588
- Kim, 2018, Bilinear attention networks, Adv. Neural Inform. Process. Syst, 31, 1564, 10.48550/arXiv.1805.07932 · DOI · Paper page
- Kingma, 2014, “ADAM: a method for stochastic optimization,”
- Koutsoubis, Future-Proofing Medical Imaging with Privacy-Preserving Federated Learning and Uncertainty Quantification: A Review
- Koutsoubis, Privacy Preserving Federated Learning in Medical Imaging with Uncertainty Estimation
- Kwon, 2023, Masked vision and language modeling for multi-modal representation learning, arXiv Preprint arXiv:2208.02131, 10.48550/arXiv.2208.02131 · DOI
- Lau, 2018, A dataset of clinically generated visual questions and answers about radiology images, Sci. Data, 5, 180251, 10.1038/sdata.2018.251 · DOI · Paper page
- Li, 2022, “Cross-modal clinical graph transformer for ophthalmic report generation,”, 20624
- Li, “Masked vision and language pre-training with unimodal and multimodal contrastive losses for medical visual question answering,”, Medical Image Computing and Computer Assisted Intervention (MICCAI), 374
- Lin, 2004, “ROUGE: a package for automatic evaluation of summaries,”, 74
- Lin, PMC-CLIP: contrastive language-image pre-training using biomedical documents, arXiv Preprint arXiv:2303.07240, 10.48550/arXiv.2303.07240 · DOI
- Lin, Medical visual question answering: a survey, Artif. Intell. Med, 143, 102611, 10.1016/j.artmed.2023.102611 · DOI · Paper page
- Liu, SLAKE: a semantically-labeled knowledge-enhanced dataset for medical visual question answering, IEEE 18th International Symposium on Biomedical Imaging (ISBI), 1650
- Liu, Medical visual question answering via conditional reasoning and contrastive learning, IEEE Trans. Med. Imag, 42, 1532, 10.1109/TMI.2022.3232411 · DOI
- Liu, A systematic review of deep learning-based research on radiology report generation, arXiv Preprint arXiv:2311.14199, 10.48550/arXiv.2311.14199 · DOI
- Liu, 2022, DePlot: one-shot visual language reasoning by plot-to-table translation, arXiv Preprint arXiv:2212.10505, 10.48550/arXiv.2212.10505 · DOI
- Liu, “Swin transformer: hierarchical vision transformer using shifted windows,”, 9992
- Mabotuwana, 2020, Framework for extracting critical findings in radiology reports, J. Digit. Imag, 33, 988, 10.1007/s10278-020-00349-7 · doi:10.1007/s10278-020-00349-7
- Manzari, 2023, MedViT: a robust vision transformer for generalized medical image classification, Comput. Biol. Med, 157, 106791, 10.1016/j.compbiomed.2023.106791 · DOI · Paper page
- Masci, 2011, “Stacked convolutional auto-encoders for hierarchical feature extraction,”, 52
- Mishra, 2021, “VT-ADL: a vision transformer network for image anomaly detection and localization,”, 01
- Mohsan, 2023, Vision transformer and language model based radiology report generation, IEEE Access, 11, 1814, 10.1109/ACCESS.2022.3232719 · DOI
- Monshi, 2020, Deep learning in generating radiology reports: a survey, Artif. Intell. Med, 106, 101878, 10.1016/j.artmed.2020.101878 · DOI · Paper page
- Moon, 2022, Multi-modal understanding and generation for medical images and text via vision-language pre-training, IEEE J. Biomed. Health Informat, 26, 6070, 10.1109/JBHI.2022.3207502 · DOI · Paper page
- Moor, 2023, Med-Flamingo: a multimodal medical few-shot learner, arXiv Preprint arXiv:2307.15189, 10.48550/arXiv.2307.15189 · arXiv · doi:10.48550/arxiv.2307.15189 · Paper page
- Nadkarni, 2011, Natural language processing: an introduction, J. Am. Med. Informat. Assoc, 18, 544, 10.1136/amiajnl-2011-000464 · DOI · Paper page
- Norgeot, 2020, Protected health information filter (philter): accurately and securely de-identifying free-text clinical notes, NPJ Digit. Med, 3, 258, 10.1038/s41746-020-0258-y · DOI · Paper page
- Papineni, 2002, “BLEU: a method for automatic evaluation of machine translation,”, Annual Meeting of the Association for Computational Linguistics, 311
- Pelka, 2018, Radiology objects in context (ROCO): a multimodal image dataset, Intravasc. Imag. Comput. Assist. Stent. Large-Scale Annot. Biomed. Data Expert Label Synth, 11043, 180, 10.1007/978-3-030-01364-6_20 · DOI · Paper page
- Pellegrini, 2023, Radialog: a large vision-language model for radiology report generation and conversational assistance, arXiv Preprint arXiv:2311.18681, 10.48550/arXiv.2311.18681 · DOI
- Pennington, 2014, Glove: global vectors for word representation, Empir. Methods Natur. Lang. Process, 14, 1532, 10.3115/v1/D14-1162 · DOI · Paper page
- Rai, 2021, Study of various methods for tokenization, Appl. Internet Things, 18, 193, 10.1007/978-981-15-6198-6_18 · DOI
- Ramesh, 2022, Improving radiology report generation systems by removing hallucinated references to non-existent priors, Machine Learn. Res, 193, 456, 10.48550/arXiv.2210.06340 · DOI · Paper page
- Ranftl, 2021, “Vision transformers for dense prediction,”, 12159
- Ranjit, 2023, Retrieval augmented chest X-ray report generation using openAI GPT models, arXiv Preprint arXiv:2305.03660, 10.48550/arXiv.2305.03660 · DOI
- Ren, 2024, Learning or self-aligning? rethinking instruction fine-tuning, arXiv Preprint arXiv:2402.18243, 10.48550/arXiv.2402.18243 · DOI
- Romanov, 2018, “Lessons from natural language inference in the clinical domain,”, 1586
- Rückert, 2022, “Overview of imageclefmedical 2022—caption prediction and concept detection,”, CEUR Workshop Proceedings, Vol. 3180, 1294
- Schmidt, 2019, Recurrent neural networks (RNNS): a gentle introduction and overview, arXiv Preprint arXiv:1912.05911, 10.48550/arXiv.1912.05911 · DOI
- Seenivasan, 2022, “Surgical-VQA: visual question answering in surgical scenes using transformer,”, Medical Image Computing and Computer Assisted Intervention—MICCAI, 33
- Sengupta, 2023, Automatic report generation for histopathology images using pre-trained vision transformers and BERT, arXiv Preprint arXiv:2312.01435, 10.48550/arXiv.2312.01435 · arXiv · doi:10.48550/arxiv.2312.01435
- Sharma, 2023, Evolution of visual data captioning methods, datasets, and evaluation metrics: a comprehensive survey, Expert Syst. Appl, 221, 119773, 10.1016/j.eswa.2023.119773 · DOI
- Shrestha, 2023, Medical vision language pretraining: a survey, arXiv Preprint arXiv:2312.06224, 10.48550/arXiv.2312.06224 · DOI
- Soviany, 2021, Curriculum learning: a survey, Int. J. Comput. Vis, 130, 1526, 10.1007/s11263-022-01611-x · DOI · Paper page
- Stripelis, 2021, Secure neuroimaging analysis using federated learning with homomorphic encryption, SPIE Med. Imag, 22, 1611, 10.48550/arXiv.2108.03437 · arXiv · doi:10.48550/arxiv.2108.03437
- Subramanian, 2020, “MediCAT: a dataset of medical images, captions, and textual references,”, 2112, 10.18653/v1/2020.findings-emnlp.191 · DOI · Paper page
- Sun, 2023, Aligning large multimodal models with factually augmented RLHF, arXiv Preprint arXiv:2309.14525, 10.48550/arXiv.2309.14525 · DOI
- Tanwani, 2022, “RepsNet: combining vision with language for automated medical reports,”, Medical Image Computing and Computer Assisted Intervention (MICCAI), 714
- Thawkar, 2023, XrayGPT: chest radiographs summarization using medical vision-language models, arXiv Preprint arXiv:2306.07971, 10.48550/arXiv.2306.07971 · arXiv · doi:10.48550/arxiv.2306.07971 · Paper page
- Ting, 2023, A survey on automatic generation of medical imaging reports based on deep learning, BioMed. Eng. OnL, 22, 1113, 10.1186/s12938-023-01113-y · DOI
- Tripathi, Building flexible, scalable, and machine learning-ready multimodal oncology datasets, Sensors, 24, 51634, 10.3390/s24051634 · DOI
- Tripathi, HoneyBee: A Scalable Modular Framework for Creating Multimodal Oncology Datasets with Foundational Embedding Models
- Tyagi, 2021, “Detecting pneumonia using vision transformer and comparing with other techniques,”, International Conference on Electronics, Communication and Aerospace Technology (ICECA), 12
- Vaswani, 2017, Attention is all you need, Adv. Neural Inform. Process. Syst, 30, 5998, 10.48550/arXiv.1706.03762 · DOI · Paper page
- Verspoor, 2013, Encyclopedia of Systems Biology, Chapter Natural Language Processing, 1495, 10.1007/978-1-4419-9863-7_158 · doi:10.1007/978-1-4419-9863-7_158
- Wang, GIT: a generative image-to-text transformer for vision and language, arXiv Preprint arXiv:2205.14100, 10.48550/arXiv.2205.14100 · arXiv · doi:10.48550/arxiv.2205.14100 · Paper page
- Wang, MedCLIP: contrastive learning from unpaired medical images and text, arXiv Preprint arXiv:2210.10163, 10.48550/arXiv.2210.10163 · DOI
- Wang, “SimVLM: simple visual language model pretraining with weak supervision,”, International Conference on Learning Representations (ICLR)
- Waqas, 2023, Revolutionizing digital pathology with the power of generative artificial intelligence and foundation models, Lab. Invest, 103, 100255, 10.1016/j.labinv.2023.100255 · DOI · Paper page
- Waqas, Digital pathology and multimodal learning on oncology data, Artif. Intell, 1, 1, 10.1093/bjrai/ubae014 · DOI
- Waqas, Embedding-based Multimodal Learning on Pan-Squamous Cell Carcinomas for Improved Survival Outcomes
- Xie, 2016, “Aggregated residual transformations for deep neural networks,”, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 5987
- Xie, 2022, “SimMIM: a simple framework for masked image modeling,”, IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 9643
- Xin, 2022, An improved transformer network for skin cancer classification, Comput. Biol. Med, 149, 105939, 10.1016/j.compbiomed.2022.105939 · DOI · Paper page
- Xu, 2021, “Learning domain adaptation with model calibration for surgical report generation in robotic surgery,”, 2021 IEEE International Conference on Robotics and Automation (ICRA), 12350, 10.1109/ICRA48506.2021.9561569 · DOI
- Yuan, 2023, RAMM: retrieval-augmented biomedical visual question answering with multi-modal pre-training, arXiv Preprint arXiv:2303.00534, 10.48550/arXiv.2303.00534 · arXiv · doi:10.48550/arxiv.2303.00534
- Zeng, 2020, “MedDialog: large-scale medical dialogue datasets,”, Conference on Empirical Methods in Natural Language Processing (EMNLP), 9241, 10.18653/v1/2020.emnlp-main.743 · DOI · Paper page
- Zhan, 2020, “Medical visual question answering via conditional reasoning,”, The 28th ACM International Conference on Multimedia, 2345
- Zhang, 2018, “Grounding referring expressions in images by variational context,”, IEEE/CVF Conference on Computer Vision and Pattern Recognition, 4158
- Zhang, Large-scale domain-specific pretraining for biomedical vision-language processing, arXiv Preprint arXiv:2303.00915, 10.48550/arXiv.2303.00915 · DOI · Paper page
- Zhang, Adapter learning in pretrained feature extractor for continual learning of diseases, arXiv Preprint arXiv:2304.09042, 10.48550/arXiv.2304.09042 · arXiv · doi:10.48550/arxiv.2304.09042
- Zhao, 2023, Retrieving multimodal information for augmented generation: a survey, arXiv Preprint arXiv:2303.10868, 10.48550/arXiv.2303.10868 · DOI
- Zhou, Learning without forgetting for vision-language models, arXiv Preprint arXiv:2305.19270, 10.48550/arXiv.2305.19270 · arXiv · doi:10.48550/arxiv.2305.19270
Cited by
Multimodal data integration for oncology in the era of deep neural networks: a review
Waqas, A., Tripathi, A., Ramachandran, R. P., Stewart, P. A., & Rasool, G. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1408843 · Cited by 165
Self-citationVision-language foundation model for 3D medical imaging
Wu, J., Wang, Y., Zhong, Z., Liao, W., Trayanova, N., Jiao, Z., & Bai, H. X. (2025) · npj Artificial Intelligence · doi:10.1038/s44387-025-00015-9 · Cited by 41
Sankar, B. S., Gilliland, D., Rincon, J., Hermjakob, H., Yan, Y., Adam, I., Lemaster, G., Wang, D., Watson, K., Bui, A., et al. (2024) · Bioengineering · doi:10.3390/bioengineering11100984 · Cited by 26
PathVLM-Eval: Evaluation of open vision language models in histopathology
Gilal, N. U., Zegour, R., Al-Thelaya, K., Özer, E., Agus, M., Schneider, J., & Boughorbel, S. (2025) · Journal of Pathology Informatics · doi:10.1016/j.jpi.2025.100455 · Cited by 7
Li, R., Li, J., Jian, B., Yuan, K., & Zhu, Y. (2025) · arXiv · doi:10.48550/arxiv.2510.00280 · Cited by 1
Metadata
- Authors
- Hartsock, I., & Rasool, G.
- Venue
- Frontiers in Artificial Intelligence
- Publisher
- Frontiers Media SA
- Published
- 2024-11-19
- Volume
- 7
- Pages
- 1430984-1430984
- ISSN
- 2624-8212 (ISSN-L)
- OpenAlex
- W4404509535
- Cited by
- 229
- References
- 93
- FWCI
- 32.35
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.