The DOI matches its DataCite record; 241 of the 241 references it cites were found in a registry or an index; none of the 170 sources its claims cite is checked for support yet.
A Systematic Survey of Prompt Engineering on Vision-Language Foundation Models
Open accessAbstract
Prompt engineering is a technique that involves augmenting a large pre-trained model with task-specific hints, known as prompts, to adapt the model to new tasks. Prompts can be created manually as natural language instructions or generated automatically as either natural language instructions or vector representations. Prompt engineering enables the ability to perform predictions based solely on prompts without updating model parameters, and the easier application of large pre-trained models in real-world tasks. In past years, Prompt engineering has been well-studied in natural language processing. Recently, it has also been intensively studied in vision-language modeling. However, there is currently a lack of a systematic overview of prompt engineering on pre-trained vision-language models. This paper aims to provide a comprehensive survey of cutting-edge research in prompt engineering on three types of vision-language models: multimodal-to-text generation models (e.g. Flamingo), image-text matching models (e.g. CLIP), and text-to-image generation models (e.g. Stable Diffusion). For each type of model, a brief model summary, prompting methods, prompting-based applications, and the corresponding responsibility and integrity issues are summarized and discussed. Furthermore, the commonalities and differences between prompting on vision-language models, language models, and vision models are also discussed. The challenges, future directions, and research opportunities are summarized to foster future research on this topic.
Overview
CoChat Summary
TL;DR
This survey systematically reviews prompt engineering for pre-trained vision-language foundation models across three families: multimodal-to-text generation, image-text matching, and text-to-image generation. It organizes prompting methods into hard and soft prompts, surveys applications and responsible-AI issues, and compares prompting on VLMs with prompting on language and vision models.
Key findings
- The paper proposes a taxonomy of prompting methods: hard prompts (task instruction, in-context learning, retrieval-based prompting, chain-of-thought prompting) and soft prompts (prompt tuning, prefix token tuning), while excluding architecture-modifying methods such as P-tuning and LoRA.
- It surveys prompting across three VLM types: multimodal-to-text generation (e.g., Flamingo), image-text matching (e.g., CLIP), and text-to-image generation (e.g., Stable Diffusion), covering applications including VQA, captioning, image classification, object detection, semantic segmentation, and text-to-video/3D/motion generation.
- The survey reports that in-context learning transfers to VLMs but with limits: Frozen performs well in a two-way setting with two new words but fails to transfer to a five-way setting with five new words; increasing in-context samples helps until saturation, and repeated content can hurt performance.
- Responsible-AI concerns are summarized across model types, including adversarial robustness, backdoor attacks, fairness/bias, and privacy; cited examples include a five-character text perturbation shifting Stable Diffusion output and Stable Diffusion generating only white male-appearing firefighters.
Methods
- Design: a comprehensive literature survey that builds a taxonomy and reviews cutting-edge research on prompt engineering for pre-trained VLMs; no original experiments are reported.
- Scope: three VLM categories—multimodal-to-text generation, image-text matching, and text-to-image generation—with prompting methods that maintain the model architecture; methods like P-tuning and LoRA are not the primary scope.
- Analysis: organizes prompting by model type, encoder-side vs decoder-side prompting, hard vs soft prompts, applications, responsible-AI considerations, comparisons with prompting in NLP and pure vision models, and challenges/future directions.
Limitations
- Paper-stated: the rapidly evolving field and wide range of existing prompt engineering approaches make an exhaustive overview challenging; the survey focused primarily on pre-trained VLMs from a prompting perspective and may not cover all recent advancements in other related areas.
- Evident: despite the title “systematic survey,” the text reports no systematic search protocol, databases, inclusion/exclusion criteria, or quantitative synthesis; coverage is narrative and selective.
- Evident: the survey explicitly excludes methods that modify model architecture, such as P-tuning and LoRA, so its scope is narrower than all prompt-engineering work on VLMs.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 Valid · 8 Not valid · 116 Not checked · 42 Own results and interpretations
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“One key finding is the versatility and applicability of prompt engineering across different types of vision-language models, including multimodal-to-text generation models, image-textmatching models, and text-to-image generation models.”
§ A Systematic Survey of Prompt Engineering on Vision-Language Foundation Models > 8 CONCLUSION
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Conversely, soft prompts based on image-text matching models, such as CLIP [2], have received extensive investigation.”
§ A Systematic Survey of Prompt Engineering on Vision-Language Foundation Models > 7 CHALLENGES AND OPPORTUNITIES
What the sources say
- Not matchedcites [2]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“One of the significant challenges in the field of prompting text-to-other generation models, particularly in the case of Text-to-Video (T2V) and Textto-3D (T2-3D) models, is their dependency on Text-to-Image (T2I) models.”
§ A Systematic Survey of Prompt Engineering on Vision-Language Foundation Models > 7 CHALLENGES AND OPPORTUNITIES
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim could be matched to a paper in Crossref, so there is nothing to check it against.
In the paper
“Incontext prompts show surprising results in few-shot [1,16,17,15] and zero-shot scenarios [29,30,1,15].”
§ A Systematic Survey of Prompt Engineering on Vision-Language Foundation Models > Visual Question Answering.
What the sources say
- Not matchedcites [1]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites [16]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites [17]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites [15]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites [29]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites [30]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This makes it possible to apply large-scale pre-trained models for realworld applications.”
§ A Systematic Survey of Prompt Engineering on Vision-Language Foundation Models > 1 INTRODUCTION
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
55 more claims
Create a free account to see all 166 claims and every source verdict.
- Probably not a substantial claim. 82
- It may state two separate claims as one. 49
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ A Systematic Survey of Prompt Engineering on Vision-Language Foundation Models > 4.1 Preliminary of Image-Text Matching Models
“Prominent models in this field, such as CLIP [2], ALIGN [58], ALBEF [58] and Multi-Event CLIP [59]”
§ A Systematic Survey of Prompt Engineering on Vision-Language Foundation Models > References
“58. J. Li et al. Align before fuse: Vision and language representa- tion learning with momentum distillation. Advances in neural information processing systems, 34:9694-9705, 2021.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ A Systematic Survey of Prompt Engineering on Vision-Language Foundation Models > 8 CONCLUSION
“Fig. 2: Prompting methods in multimodal-to-text generation.”
§ A Systematic Survey of Prompt Engineering on Vision-Language Foundation Models > 8 CONCLUSION
“Figure 2 illustrates a typical text-image generation framework, highlighting its key components and functionalities”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ A Systematic Survey of Prompt Engineering on Vision-Language Foundation Models > Prompting Models with Decoder-based Fusion Module.
“Frozen [44] and BLIP-2 [45] exemplify models that employ image conditional prefix tuning.”
§ A Systematic Survey of Prompt Engineering on Vision-Language Foundation Models > References
“44. D. Noever and S. E. M. Noever. The multimodal and modular ai chef: Complex recipe generation from imagery. arXiv preprint arXiv:2304.02016, 2023.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ A Systematic Survey of Prompt Engineering on Vision-Language Foundation Models > 4.1 Preliminary of Image-Text Matching Models
“By prompting, we substitute model input by following learnable prompts: for textual prompts: f l v ([{v i } M i=1 , {z i } M i=1 ] ) and for the general visual prompt: f l t ([{c i } M i=1 , {t i } M i=1 ]”
§ A Systematic Survey of Prompt Engineering on Vision-Language Foundation Models > 8 CONCLUSION
“| {z i } M i=1 | visual prompt tokens | | {t i } M i=1 | textual prompt tokens |”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ A Systematic Survey of Prompt Engineering on Vision-Language Foundation Models > 3.4 Understanding Prompting
“both OFA [49] and KOSMOS-1 [15] add language-only tasks during training to prevent the loss of language ability.”
§ A Systematic Survey of Prompt Engineering on Vision-Language Foundation Models > References
“49. Y. Zang et al. Unified Vision and Language Prompt Learning. arXiv:2210.07225, 2022.”
Citations
Cites
Items 201–241 of 241
Backdoor Defense via Suppressing Model Shortcuts
Yang, S., Li, Y., Jiang, Y., & Xia, S. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10097220 · Cited by 17
Towards Robust Prompts on Vision-Language ModelsPreprint
Gu, J., Beirami, A., Wang, X., Beutel, A., Torr, P. H. S., & Qin, Y. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2304.08479 · Cited by 11
Self-citationDomain Adaptation via Prompt LearningPreprint
Ge, C., Huang, R., Xie, M., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2202.06687 · Cited by 9
Explaining Visual Biases as Words by Generating Captions
Kim, Y., Mo, S., Kim, M., Lee, K., Lee, J., & Shin, J.-W. (2023) · arXiv.org · doi:10.48550/arxiv.2301.11104 · Cited by 8
PromptAttack: Probing Dialogue State Trackers with Adversarial Prompts
Dong, X.-J., He, Y., Zhu, Z., & Caverlee, J. (2023) · Annual Meeting of the Association for Computational Linguistics · doi:10.48550/arxiv.2306.04535 · Cited by 7
PromptAttack: Probing Dialogue State Trackers with Adversarial Prompts
Dong, X., He, Y., Zhu, Z., & Caverlee, J. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.677 · Cited by 5
LAMBADA: Backward Chaining for Automated Reasoning in Natural LanguagePreprint
Kazemi, M., Kim, N., Bhatia, D. S., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2212.13894 · Cited by 5
Scaling Robot Learning with Semantically Imagined Experience
Yu, T., Xiao, T., Stone, A., Tompson, J., Brohan, A., Wang, S., Singh, J., Tan, C., M, D., Peralta, J., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.11550 · Cited by 4
A Prompting-based Approach for Adversarial Example Generation and Robustness EnhancementPreprint
Yang, Y., Huang, P., Cao, J., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2203.10714 · Cited by 4
Optimizing Continuous Prompts for Visual Relationship Detection by Affix-Tuning
Xiao, S., & Fu, W. (2022) · IEEE Access · doi:10.1109/access.2022.3187263 · Cited by 2
Exploring the Universal Vulnerability of Prompt-based Learning ParadigmPreprint
Xu, L., Chen, Y., Cui, G., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2204.05239 · Cited by 1
OPT: Open Pre-trained Transformer Language Models
Zhang, S., Roller, S., Goyal, N., Artetxe, M., Chen, M., Chen, S., Dewan, C., Diab, M., Li, X., Lin, X. V., et al. (2022) · arXiv · doi:10.48550/arxiv.2205.01068
Self-citationOpenAI, Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.08774
Self-citationOn Evaluating Adversarial Robustness of Large Vision-Language Models
Zhao, Y., Pang, T., Du, C., Yang, X., Li, C., Cheung, N.-M., & Lin, M. (2023) · arXiv · doi:10.48550/arxiv.2305.16934
FateZero: Fusing Attentions for Zero-shot Text-based Video Editing
Qi, C., Cun, X., Zhang, Y., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.09535
Zhu, J., Yang, H., He, H., et al. (2023) · arXiv · doi:10.48550/arxiv.2306.07257
Avrahami, O., Fried, O., & Lischinski, D. (2022) · arXiv · doi:10.48550/arxiv.2206.02779
Multi-Concept Customization of Text-to-Image Diffusion
Kumari, N., Zhang, B., Zhang, R., Shechtman, E., & Zhu, J.-Y. (2022) · arXiv · doi:10.48550/arxiv.2212.04488
Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Models
Shu, M., Nie, W., Huang, D.-A., Yu, Z., Goldstein, T., Anandkumar, A., & Xiao, C. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1038
Multitask Vision-Language Prompt Tuning
Shen, S., Yang, S., Zhang, T., Zhai, B., Gonzalez, J. E., Keutzer, K., & Darrell, T. (2022) · arXiv · doi:10.48550/arxiv.2211.11720
Benchmarking Robustness of Adaptation Methods on Pre-trained Vision-Language Models
Chen, S., Gu, J., Han, Z., Ma, Y., Torr, P., & Tresp, V. (2023) · arXiv · doi:10.48550/arxiv.2306.02080
Self-citationWhat does CLIP know about a red circle? Visual prompt engineering for VLMs
Shtedritski, A., Rupprecht, C., & Vedaldi, A. (2023) · arXiv · doi:10.48550/arxiv.2304.06712
Effective Robustness against Natural Distribution Shifts for Models with Different Training Data
Shi, Z., Carlini, N., Balashankar, A., Schmidt, L., Hsieh, C.-J., Beutel, A., & Qin, Y. (2023) · arXiv · doi:10.48550/arxiv.2302.01381
Self-citationCleanCLIP: Mitigating Data Poisoning Attacks in Multimodal Contrastive Learning
Bansal, H., Singhi, N., Yang, Y., Yin, F., Grover, A., & Chang, K.-W. (2023) · arXiv · doi:10.48550/arxiv.2303.03323
Mitigating Test-Time Bias for Fair Image Retrieval
Kong, F., Yuan, S., Hao, W., & Henao, R. (2023) · arXiv · doi:10.48550/arxiv.2305.19329
Drag Your GAN: Interactive Point-based Manipulation on the Generative Image Manifold
Pan, X., Tewari, A., Leimkühler, T., Liu, L., Meka, A., & Theobalt, C. (2023) · arXiv · doi:10.48550/arxiv.2305.10973
Wu, W., Zhao, Y., Shou, M. Z., Zhou, H., & Shen, C. (2023) · arXiv · doi:10.48550/arxiv.2303.11681
Diffusion Self-Guidance for Controllable Image Generation
Epstein, D., Jabri, A., Poole, B., Efros, A. A., & Holynski, A. (2023) · arXiv · doi:10.48550/arxiv.2306.00986
Dream3D: Zero-Shot Text-to-3D Synthesis Using 3D Shape Prior and Text-to-Image Diffusion Models
Xu, J., Wang, X., Cheng, W., Cao, Y.-P., Shan, Y., Qie, X., & Gao, S. (2022) · arXiv · doi:10.48550/arxiv.2212.14704
CLIP-Mesh: Generating textured meshes from text using pretrained image-text models
Khalid, N. M., Xie, T., Belilovsky, E., & Popa, T. (2022) · arXiv · doi:10.48550/arxiv.2203.13333
FLAME: Free-form Language-based Motion Synthesis & Editing
Kim, J., Kim, J., & Choi, S. (2022) · arXiv · doi:10.48550/arxiv.2209.00349
Diffusion Models for Imperceptible and Transferable Adversarial Attack
Chen, J., Chen, H., Chen, K., Zhang, Y., Zou, Z., & Shi, Z. (2023) · arXiv · doi:10.48550/arxiv.2305.08192
A Pilot Study of Query-Free Adversarial Attack against Stable Diffusion
Zhuang, H., Zhang, Y., & Liu, S. (2023) · arXiv · doi:10.48550/arxiv.2303.16378
Rickrolling the Artist: Injecting Backdoors into Text Encoders for Text-to-Image Synthesis
Struppek, L., Hintersdorf, D., & Kersting, K. (2022) · arXiv · doi:10.48550/arxiv.2211.02408
Text-to-Image Diffusion Models can be Easily Backdoored through Multimodal Data Poisoning
Zhai, S., Dong, Y., Shen, Q., Pu, S., Fang, Y., & Su, H. (2023) · arXiv · doi:10.48550/arxiv.2305.04175
Personalization as a Shortcut for Few-Shot Backdoor Attack against Text-to-Image Diffusion Models
Huang, Y., Juefei-Xu, F., Guo, Q., Zhang, J., Wu, Y., Hu, M., Li, T., Pu, G., & Liu, Y. (2023) · arXiv · doi:10.48550/arxiv.2305.10701
Social Biases through the Text-to-Image Generation Lens
Naik, R., & Nushi, B. (2023) · arXiv · doi:10.48550/arxiv.2304.06034
Prompt Stealing Attacks Against Text-to-Image Generation Models
Shen, X., Qu, Y., Backes, M., & Zhang, Y. (2023) · arXiv · doi:10.48550/arxiv.2302.09923
Tu, C.-H., Mai, Z., & Chao, W.-L. (2022) · arXiv · doi:10.48550/arxiv.2212.03220
Zhang, S., Khan, S., Shen, Z., Naseer, M., Chen, G., & Khan, F. (2022) · arXiv · doi:10.48550/arxiv.2212.05590
Exploring the Benefits of Visual Prompting in Differential Privacy
Li, Y., Tsai, Y.-L., Ren, X., Yu, C.-M., & Chen, P.-Y. (2023) · arXiv · doi:10.48550/arxiv.2303.12247
Research works printed but not linked: 22
- Z. Zhang et al. Automatic chain of thought prompting in large language models. In The Eleventh International Conference on Learning Representations (ICLR 2023), 2023.
- X. Chen et al. PaLI: A jointly-scaled multilingual language- image model. In The Eleventh International Conference on Learning Representations, 2023.
- A. Radford et al. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019.
- J. Li et al. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. arXiv preprint arXiv:2301.12597, 2023. · arXiv
- J. B. Nici. AP Art History: 5 Practice Tests + Comprehensive Review + Online Practice. Barron's Educational Series, 2020.
- J. Qiu et al. Benchmarking robustness under distribution shift of multimodal image-text models. In NeurIPS 2022 Workshop on Distribution Shifts: Connecting Methods and Applications, 2022.
- J. Li et al. Align before fuse: Vision and language representa- tion learning with momentum distillation. Advances in neural information processing systems, 34:9694-9705, 2021.
- T. Gao et al. Making Pre-trained Language Models Better Few- shot Learners. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th In- ternational Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 3816-3830, Online, August 2021. Association for Computational Linguistics.
- Z. Guo et al. Texts as images in prompt tuning for multi- label image recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2808-2817, 2023.
- J. Wen et al. Visual Prompt Tuning for Few-Shot Text Classification. In Proceedings of the 29th International Confer- ence on Computational Linguistics, pp. 5560-5570, Gyeongju, Republic of Korea, October 2022. International Committee on Computational Linguistics.
- Y. Du et al. Learning to prompt for open-vocabulary object detection with vision-language model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14084-14093, 2022.
- D. Jeulin. Dead leaves models: from space tesselation to random functions proc. of the symposium on the advances in the theory and applications of random sets (fontainebleau, 9-11 october 1996) ed d jeulin, 1997.
- M. Ni et al. Imaginarynet: Learning object detectors without real images and annotations. In ICLR, 2023.
- R. He et al. Is synthetic data from generative models ready for image recognition? In ICLR, 2023.
- R. Gal et al. An image is worth one word: Personalizing text- to-image generation using textual inversion. In The Eleventh International Conference on Learning Representations, 2023.
- J. Canny. A computational approach to edge detection. IEEE Transactions on pattern analysis and machine intelligence, pp. 679-698, 1986.
- J. Ho et al. Cascaded diffusion models for high fidelity image generation. J. Mach. Learn. Res., 23(47):1-33, 2022.
- P. Lewis et al. Retrieval-augmented generation for knowledge- intensive nlp tasks. Advances in Neural Information Processing Systems, 33:9459-9474, 2020.
- S. Borgeaud et al. Improving language models by retrieving from trillions of tokens. In International conference on machine learning, pp. 2206-2240. PMLR, 2022.
- H. Salman et al. Unadversarial examples: Designing objects for robust vision. Advances in Neural Information Processing Systems, 34:15270-15284, 2021.
- A. Madry et al. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations (ICLR), 2017.
- J. Gu et al. Effective and efficient vote attack on capsule networks. In International Conference on Learning Represen- tations (ICLR), 2021.
Cited by
Unleashing the potential of prompt engineering for large language models
Chen, B., Zhang, Z., Langrené, N., & Zhu, S. (2025) · Patterns · doi:10.1016/j.patter.2025.101260 · Cited by 490
Generative AI Literacy: Twelve Defining Competencies
Annapureddy, R., Fornaroli, A., & Gatica-Perez, D. (2025) · Digital Government: Research and Practice · doi:10.1145/3685680 · Cited by 333
“Example : The most prominent example of AI legislation to date is the European AI Act [32], and European users should become familiar with it and its implications.”
Vision-language models for medical report generation and visual question answering: a review
Hartsock, I., & Rasool, G. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1430984 · Cited by 229
CLIP in medical imaging: A survey
Zhao, Z., Liu, Y., Wu, H., Wang, M., Li, Y., Wang, S., Teng, L., Liu, D., Cui, Z., Wang, Q., et al. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103551 · Cited by 281
Vision Language Models in Autonomous Driving: A Survey and Outlook
Zhou, X., Liu, M., Yurtsever, E., Žagar, B. L., Zimmer, W. M. X., Cao, H., & Knoll, A. (2024) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2024.3402136 · Cited by 401
Metadata
- Authors
- Gu, J., Han, Z., Chen, S., Beirami, A., He, B., Zhang, G., Liao, R., Qin, Y., Tresp, V., & Torr, P.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2023
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2307.12980
- OpenAlex
- W4385262478
- Cited by
- 280
- References
- 107
- Citations by year
- 2026: 60 · 2025: 112 · 2024: 91 · 2023: 17
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- University of Oxford
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Keywords
- prompt engineering, vision-language foundation models, image-text matching, text-to-image generation, CLIP, flamingos, Stable Diffusion, prompting
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-11-14, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-11-14 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.