The DOI matches its Crossref record; 154 of the 154 references it cites were found in a registry or an index.
Flamingo: A Visual Language Model for Few-Shot Learning
Abstract
Building models that can be rapidly adapted to novel tasks using only a handful of annotated examples is an open challenge for multimodal machine learning research. We introduce Flamingo, a family of Visual Language Models (VLM) with this ability. We propose key architectural innovations to: (i) bridge powerful pretrained vision-only and language-only models, (ii) handle sequences of arbitrarily interleaved visual and textual data, and (iii) seamlessly ingest images or videos as inputs. Thanks to their flexibility, Flamingo models can be trained on large-scale multimodal web corpora containing arbitrarily interleaved text and images, which is key to endow them with in-context few-shot learning capabilities. We perform a thorough evaluation of our models, exploring and measuring their ability to rapidly adapt to a variety of image and video tasks. These include open-ended tasks such as visual question-answering, where the model is prompted with a question which it has to answer, captioning tasks, which evaluate the ability to describe a scene or an event, and close-ended tasks such as multiple-choice visual question-answering. For tasks lying anywhere on this spectrum, a single Flamingo model can achieve a new state of the art with few-shot learning, simply by prompting the model with task-specific examples. On numerous benchmarks, Flamingo outperforms models fine-tuned on thousands of times more task-specific data.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“vision-language models (VLMs) such as VilBERT 19 and Flamingo 2 combine a language model with an image encoder and are trained on a multi-modal corpus of text-image pairs.This enables them to predict how a given sequence of words will continue in the context of a given image.”
From Talking about Large Language Models (2024) · cites this paper as [2]
In the citing paper
“the latest generation of language models are multimodal, allowing interleaved image and text input (OpenAI, 2023a; Alayrac et al., 2022; Team et al., 2023; Li et al., 2023b)”
From Cognitive Architectures for Language Agents (2023) · cites this paper as Alayrac et al., 2022
In the citing paper
“The Switch Transformer [201] was one of the first MoE-based Transformers, activating experts selectively in each layer, reducing computational overhead while outperforming dense Transformers in NLP tasks.”
From Multimodal fusion and vision–language models: A survey for robot vision (2026) · cites this paper as [201]
In the citing paper
“The dominant architectural paradigm connects specialized, external multimodal encoders-such as CLIP for vision or CLAP for audio-to the LLM backbone via alignment modules like Q-Former or simple MLPs [19, 86, 615, 1139], a modular design that allows for independent encoder updates without retraining the entire model [624].”
From A Survey of Context Engineering for Large Language Models (2025) · cites this paper as [19]
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 3×
“Language modelling has recently made substantial progress following the introduction of Transformers [115].”
BackgroundMethodInfluentialHochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 2×
“To ensure that at initialization, the conditioned model yields the same results as the original language model, we use a tanh-gating mechanism [41].”
MethodBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 4×
“In particular, BERT [23] inspired a large body of vision-language work [16, 28, 29, 38, 59, 61, 66, 101, 106, 107, 109, 118, 121, 142, 143, 151].”
BackgroundMethodInfluentialAn Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · cited 2×
“Im ag e ImageNet-1k [94] 3 Object classification Val Top-1 acc.”
BackgroundSwin Transformer: Hierarchical Vision Transformer using Shifted Windows
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 12×
“We evaluate the ability of our models to rapidly adapt to new tasks using in-context learning , analogously to GPT-3 [11], by interleaving support example pairs in the form of ( 𝑖𝑚𝑎𝑔𝑒, 𝑡𝑒𝑥𝑡 ) or ( 𝑣𝑖𝑑𝑒𝑜, 𝑡𝑒𝑥𝑡 ) , followed by the query visual input, to build a prompt (details in Method FT Shot OKVQA ( I )”
BackgroundMethodResultInfluentialEnd-to-End Object Detection with Transformers
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58452-8_13 · Cited by 18,285 · cited 2×
“[13] Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko.”
BackgroundMethodLearning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293 · cited 12×
“[85] we use “prompt-ensembling” in which we embed multiple texts using templates such as “A photo of a {class_name}” and average the resulting embedding.”
BackgroundMethodInfluentialExploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · cited 3×
“To measure the importance of text pretraining, we compare the performance of using a frozen decoder-only Transformer either pretrained on MassiveText (our main model) or pretrained on the C4 dataset [87] (row (iv) of Table 10).”
BackgroundMethod
Cited by
Sorted by citations
Foundation models for generalist medical artificial intelligence
Moor, M., Banerjee, O., Abad, Z. S. H., Krumholz, H. M., Leskovec, J., Topol, E. J., & Rajpurkar, P. (2023) · Nature · doi:10.1038/s41586-023-05881-4 · Cited by 2,028
Scientific discovery in the age of artificial intelligence.
Wang, H., Fu, T., Du, Y., Gao, W., Huang, K., Liu, Z., Chandak, P., Liu, S., Van Katwyk, P., Deac, A., et al. (2023) · Nature · doi:10.1038/s41586-023-06221-2 · Cited by 1,823
A Survey on In-context Learning
Dong, Q., Li, L., Dai, D., Zheng, C., Ma, J., Li, R., Xia, H., Xu, J., Wu, Z., Chang, B., et al. (2024) · Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2024.emnlp-main.64 · Cited by 649
MMMU: A Massive Multi-Discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI
Yue, X., Ni, Y., Zheng, T., Zhang, K., Liu, R., Zhang, G., Stevens, S., Jiang, D., Ren, W., Sun, Y., et al. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.00913 · Cited by 429
Talking about Large Language Models
Shanahan, M. (2024) · Communications of the ACM · doi:10.1145/3624724 · Cited by 381
Inner Monologue: Embodied Reasoning through Planning with Language Models
Huang, W., Xia, F., Xiao, T., Chan, H., Liang, J., Florence, P., Zeng, A., Tompson, J., Mordatch, I., Chebotar, Y., et al. (2022) · arXiv · doi:10.48550/arxiv.2207.05608 · Cited by 287
“In this case, the LLM can directly ask a question about the scene, and this question can be answered either by a person, or by another pretrained model, such as a Visual Question Answering (VQA) model [87, 88, 89, 90].”
RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control
Brohan, A., Brown, N., Carbajal, J., Chebotar, Y., Chen, X., Choromanski, K., Ding, T., Driess, D., Dubey, A., Finn, C., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.15818 · Cited by 276
“…[1, 2, 3] but emergent problem-solving [4, 5, 6] and creative generation of prose [7, 2] and code [8], while vision-language models enable open-vocabulary visual recognition [9, 10, 11] and can even make complex inferences about object-agent interactions in images [12, 13, 14, 15, 16, 17, 18].”
BackgroundInfluentialA Cookbook of Self-Supervised Learning
Balestriero, R., Ibrahim, M., Sobal, V., Morcos, A., Shekhar, S., Goldstein, T., Bordes, F., Bardes, A., Mialon, G., Tian, Y., et al. (2023) · arXiv · doi:10.48550/arxiv.2304.12210 · Cited by 182
“These modern SSL models are capable of representing both vision and text, and can be used in a number of applications that are multimodal, from visual-question answering to multimodal generation [Alayrac et al., 2022, Li et al., 2022c, Nichol et al., 2022, Rao et al., 2022].”
BackgroundMovieChat: From Dense Token to Sparse Memory for Long Video Understanding
Song, E., Chai, W., Wang, G., Zhang, Y., Zhou, H., Wu, F., Chi, H., Guo, X., Ye, T., Zhang, Y., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.16449 · Cited by 176
OCRBench: on the hidden mystery of OCR in large multimodal models
Liu, Y., Li, Z., Huang, M., Yang, B., Yu, W., Li, C., Yin, X.-C., Liu, C.-L., Jin, L., & Bai, X. (2024) · Science China Information Sciences · doi:10.1007/s11432-024-4235-6 · Cited by 167
“Flamingo [13] and OpenFlamingo [19] enhance a frozen pretrained LLM by incorporating novel gated cross-attention-dense layers, enabling conditioning on visual inputs.”
Language Is Not All You Need: Aligning Perception with Language Models
Huang, S., Dong, L., Wang, W., Hao, Y., Singhal, S., Ma, S., Lv, T., Cui, L., Mohammed, O. K., Patra, B., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.14045 · Cited by 235
Vision-Language Pre-Training: Basics, Recent Advances, and Future Trends
Gan, Z., Li, L., Li, C., Wang, L., Liu, Z., & Gao, J. (2022) · Foundations and Trends® in Computer Graphics and Vision · doi:10.1561/0600000105 · Cited by 155
Eight Things to Know about Large Language Models
Bowman, S. R. (2023) · arXiv · doi:10.48550/arxiv.2304.00612 · Cited by 137
Multimodal Foundation Models: From Specialists to General-Purpose Assistants
Li, C., Gan, Z., Yang, Z., Yang, J., Li, L., Wang, L., & Gao, J. (2024) · Foundations and Trends® in Computer Graphics and Vision · doi:10.1561/0600000110 · Cited by 331
Improving Factuality and Reasoning in Language Models through Multiagent Debate
Du, Y., Li, S., Torralba, A., Tenenbaum, J. B., & Mordatch, I. (2023) · arXiv · doi:10.48550/arxiv.2305.14325 · Cited by 1,202
“Our work is also related to existing works that focus on text generation by combining different models (Du et al., 2020; Liu et al., 2022b; Zeng et al., 2022; Alayrac et al., 2022; Du et al., 2023).”
Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces
Yang, J., Yang, S., Gupta, A. W., Han, R., Fei-Fei, L., & Xie, S. (2025) · 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52734.2025.00994 · Cited by 120
The Revolution of Multimodal Large Language Models: A Survey
Caffagni, D., Cocchi, F., Barsellotti, L., Moratelli, N., Sarto, S., Baraldi, L., Baraldi, L., Cornia, M., & Cucchiara, R. (2024) · Findings of the Association for Computational Linguistics ACL 2024 · doi:10.18653/v1/2024.findings-acl.807 · Cited by 98
“This approach has been proposed in Flamingo (Alayrac et al., 2022) with the integration of dense cross-attention blocks among the existing pre-trained layers of the LLM.”
InfluentialCognitive Architectures for Language Agents
Sumers, T. R., Yao, S., Narasimhan, K., & Griffiths, T. L. (2023) · arXiv · doi:10.48550/arxiv.2309.02427 · Cited by 72
“Finally, recent advances in vision-language models (VLMs; Alayrac et al., 2022) can simplify grounding by providing a straightforward translation of perceptual data into text (Zeng et al.”
BackgroundMethodArtificial intelligence for science in quantum, atomistic, and continuum systems
Zhang, X., Wang, L., Helwig, J., Luo, Y., Fu, C., Xie, Y., Liu, M., Lin, Y., Zhao, X., Yan, K., et al. (2025) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000115 · Cited by 214
Navgpt-2: Unleashing navigational reasoning capability for large vision-language models
Zhou, G., Hong, Y., Wang, Z., Wang, X. E., & Wu, Q. (2025) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-72667-5_15 · Cited by 63
VideoAgent: A Memory-augmented Multimodal Agent for Video Understanding
Fan, Y., Ma, X., Wu, R., Du, Y., Li, J., Gao, Z., & Li, Q. (2024) · arXiv · doi:10.48550/arxiv.2403.11481 · Cited by 60
PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answering
Zhang, X., Wu, C., Zhao, Z., Lin, W., Zhang, Y., Wang, Y., & Xie, W. (2023) · arXiv · doi:10.48550/arxiv.2305.10415 · Cited by 66
“Modality Source Images QA pairs VQA-RAD [18] Radiology MedPix ® database 0.3k 3.5k PathVQA [12] Pathology PEIR Digital Library [14] 5k 32.8k SLAKE [23] Radiology MSD [3], ChestX-ray8 [36], CHAOS [15] 0.7k 14k VQA-Med-2021 [5] Radiology MedPix ® database 5k 5k PMC-VQA Mixture * PubMed Central ® 149k 227k Figure 1: The…”
BackgroundVisualWebArena: evaluating multimodal agents on realistic visual web tasks
Koh, J. Y., Lo, R., Jang, L., Duvvur, V., Lim, M., Huang, P.-Y., Neubig, G., Zhou, S., Salakhutdinov, R., & Fried, D. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.50 · Cited by 50
Zambrano Chaves, J. M., Huang, S.-C., Xu, Y., Xu, H., Usuyama, N., Zhang, S., Wang, F., Xie, Y., Khademi, M., Yang, Z., et al. (2025) · Nature Communications · doi:10.1038/s41467-025-58344-x · Cited by 45
Multimodal fusion and vision–language models: A survey for robot vision
Han, X., Chen, S., Fu, Z., Feng, Z., Fan, L., An, D., Wang, C., Guo, L., Meng, W., Zhang, X., et al. (2026) · Information Fusion · doi:10.1016/j.inffus.2025.103652 · Cited by 105
“As a result, there is a pressing need for a universal and cross-platform computational framework that can address these challenges.”
Video-Bench: A Comprehensive Benchmark and Toolkit for Evaluating Video-based Large Language Models
Ning, M., Zhu, B., Xie, Y., Lin, B., Cui, J., Yuan, L., Chen, D., & Yuan, L. (2023) · arXiv · doi:10.48550/arxiv.2311.16103 · Cited by 36
JARVIS -1: Open-World Multi-Task Agents With Memory-Augmented Multimodal Language Models
Wang, Z., Cai, S., Liu, A., Jin, Y., Hou, J., Zhang, B., Lin, H., He, Z., Zheng, Z., Yang, Y., et al. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3511593 · Cited by 36
LLaVA-3D: A Simple Yet Effective Pathway to Empowering LMMs with 3D Capabilities
Zhu, C., Wang, T., Zhang, W., Pang, J., & Liu, X. (2025) · 2025 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00409 · Cited by 42
“Recent advancements in Large Multimodal Models (LMMs) [2, 5, 27, 44] have significantly enhanced their ability to understand and reason over visual and language inputs, leading to remarkable performance in 2D visual tasks.”
Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Models
Chu, Y., Xu, J., Zhou, X., Yang, Q., Zhang, S., Yan, Z., Zhou, C., & Zhou, J. (2023) · arXiv · doi:10.48550/arxiv.2311.07919 · Cited by 61
“For visual modality, GPT4 (OpenAI, 2023), Flamingo (Alayrac et al., 2022), Kosmos (Peng et al., 2023), BLIP (Li et al., 2022), Shikra (Chen et al., 2023), Emu (Sun et al., 2023) and Qwen-VL (Bai et al., 2023) have proposed different integration method to enable image understanding or generation…”
MethodData-Centric Foundation Models in Computational Healthcare: A Survey
Zhang, Y., Gao, J. G., Tan, Z., Zhou, L., Ding, K., Zhou, M., Zhang, S., & Wang, D. (2026) · ACM Computing Surveys · doi:10.1145/3800677 · Cited by 65
SALMONN: Towards Generic Hearing Abilities for Large Language Models
Tang, C., Yu, W., Sun, G., Chen, X., Tan, T., Li, W., Lu, L., Ma, Z., & Zhang, C. (2023) · arXiv · doi:10.48550/arxiv.2310.13289 · Cited by 67
“Recent studies focus on connecting LLMs with either the encoder of one additional type of input, such as image (Li et al., 2023a; Alayrac et al., 2022; Dai et al., 2023), silent video (Maaz et al., 2023; Chen et al., 2023b; Zhao et al., 2022), audio events (Gong et al., 2023b; Lyu et al., 2023) or…”
BackgroundGraphLLM: Boosting Graph Reasoning Ability of Large Language Model
Chai, Z., Zhang, T., Wu, L., Han, K., Hu, X., Huang, X., & Yang, Y. (2025) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2025.3627488 · Cited by 170
Gu, Z., Li, J., Shen, W., Yu, W., Xie, Z., McCrory, S., Cheng, X., Shamsah, A., Griffin, R. J., Liu, C. K., et al. (2026) · IEEE/ASME Transactions on Mechatronics · doi:10.1109/tmech.2025.3579247 · Cited by 130
“Potential approaches include leveraging vision-language models to extract information from videos [263], [264], [265], employing video generation models [266] or physics-based simulation [180], [192] to generate synthetic data, and adopting hybrid strategies that effectively combine these synthetic…”
Fu, H., Zhang, D. H., Zhao, Z., Cui, J., Liang, D., Zhang, C., Zhang, D., Xie, H., Wang, B., & Bai, X. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.02302 · Cited by 123
“To alleviate this bur-2 den, QwenVL [4] and Flamingo [2] reduce token redundancy using cross-attention, while Qwen2VL [57] enhances efficiency with dynamic resolution and multimodal rotary position embedding (M-RoPE) for simultaneously processing diverse modalities.”
Konwer, A., Yang, Z., Bas, E., Xiao, C., Prasanna, P., Bhatia, P., & Kass-Hout, T. (2025) · 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52734.2025.01955 · Cited by 17
TextMonkey: An OCR-Free Large Multimodal Model for Understanding Document
Liu, Y., Yang, B., Liu, Q., Li, Z., Ma, Z., Zhang, S., & Bai, X. (2026) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2026.3653415 · Cited by 103
“Unlike Flamingo’s [33] Resampler, our Token Resampler employs a similarity-based filtering mechanism to retain the most representative tokens and suppress redundant ones.”
Semantic Gaussians: Open-Vocabulary Scene Understanding With 3D Gaussian Splatting
Guo, J., Ma, X., Fan, Y., Liu, H., & Li, Q. (2026) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2026.3675320 · Cited by 40
“…are the most straightforward representation of 3D scenes, but allowing open-vocabulary understanding usually involves 2D vision-language models [6], which could struggle with consistency across different views, likely due to a lack of visual geometric knowledge; Point clouds are popular and…”
Learning visual language models for video understanding
Yang, A. (2023) · theses.fr (ABES) · doi:10.70675/802872a7z8092z4556za1fdz2c7af45f0e72 · Cited by 26
A Survey of Context Engineering for Large Language Models
Mei, L., Yao, J., Ge, Y., Wang, Y., Bi, B., Cai, Y., Liu, J., Li, M., Li, Z.-Z., Zhang, D., et al. (2025) · arXiv · doi:10.48550/arxiv.2507.13334 · Cited by 281
Watermarks in the Sand: Impossibility of Strong Watermarking for Generative Models
Zhang, H., Edelman, B. L., Francati, D., Venturi, D., Ateniese, G., & Barak, B. (2023) · arXiv · doi:10.48550/arxiv.2311.04378 · Cited by 12
Sparks of Large Audio Models: A Survey and Outlook
Latif, S., Shoukat, M., Shamshad, F., Usama, M., Ren, Y., Cuayáhuitl, H., Wang, W., Zhang, X., Togneri, R., Cambria, E., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.12792 · Cited by 17
“Unlike earlier learning-based models that were tailored for specific tasks, these large models boast versatility in addressing diverse tasks [21, 22].”
Background$π_0$: A Vision-Language-Action Flow Model for General Robot Control
Black, K., Brown, N., Driess, D., Esmail, A., Equi, M., Finn, C., Fusai, N., Groom, L., Hausman, K., Ichter, B., et al. (2024) · arXiv · doi:10.48550/arxiv.2410.24164 · Cited by 13
“Following the standard late fusion VLM recipe [3, 11, 30], image encoders embed the robot’s image observations into the same embedding space as language tokens.”
SGLang: Efficient Execution of Structured Language Model Programs
Zheng, L., Yin, L., Xie, Z., Sun, C., Huang, J., Yu, C. H., Cao, S., Kozyrakis, C., Stoica, I., Gonzalez, J. E., et al. (2023) · arXiv · doi:10.48550/arxiv.2312.07104 · Cited by 145
“This is accomplished through tool usage [41, 38], multiple input modalities [47, 2], and a wide range of prompting techniques [30], like few-shot learning [5], self-consistency [53], skeleton-of-thought [33], and tree-of-thought [56].”
AgroBench: Vision-Language Model Benchmark in Agriculture
Shinoda, R., Inoue, N., Kataoka, H., Onishi, M., & Ushiku, Y. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00716 · Cited by 14
“CLIP played a key role in introducing more sophisticated visual representation into language explanations e.g., Flamingo [2], BLIP [14, 15], Qwen [5], PaLI [8], LLaVA [17, 18], CogVLM [50], and Emu [44, 45, 51].”
MM-Spatial: Exploring 3D Spatial Understanding in Multimodal LLMs
Daxberger, E., Wenzel, N., Griffiths, D., Gang, H., Lazarow, J., Kohavi, G., Kang, K., Eichner, M. L., Yang, Y., Dehghan, A., et al. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00694 · Cited by 47
“MLLMs [49, 50, 64, 86, 103] have attracted significant research focus, tracing back to Frozen [106] and Flamingo [6, 8], with more recent works such as LLaVA [80] and MiniGPT-4 [140] introducing visual instruction tuning.”
RoboCache: A Distributed Key–Value Store for Petabyte–Scale Multimodal Robot Learning Datasets
Patel, T., Shivam, S., Padhy, A. K., et al. (2025) · International Conference on Computer and Applications (ICCA) · doi:10.1109/icca66035.2025.11430805 · Cited by 7
“To maximize client-side GPU utilization, the API layer includes a pipelined reader that prefetches and batches encoded tensors following the pattern of high-throughput data loaders in vision-language systems [9], [10].”
Cui, C., Ma, Y., Park, S., Yang, Z., Zhou, Y., Liu, P., Lu, J., Peng, J., Zhang, J., Zhang, R., et al. (2026) · Proceedings of the IEEE · doi:10.1109/jproc.2026.3678829 · Cited by 37
“In addition, as these technologies move toward real-world deployment, it will be essential to establish rigorous safety and performance standards to ensure their reliable and responsible implementation.”
Prismatic VLMs: Investigating the Design Space of Visually-Conditioned Language Models
Karamcheti, S., Nair, S., Balakrishna, A., Liang, P., Kollar, T., & Sadigh, D. (2024) · arXiv · doi:10.48550/arxiv.2402.07865 · Cited by 7
“Of primary concern is the generality of our model architecture; while the three component architecture we define in §2 is reflective of the majority of existing VLMs, there are other architecture innovations and optimization procedures that our study does not currently capture; as a notable example, we do not study ar…”
MethodAre large pre-trained vision language models effective construction safety inspectors
Chen, X., & Zou, Z. (2026) · Data-Centric Engineering · doi:10.1017/dce.2026.10044 · Cited by 19
“Therefore, we input five captions without corresponding images for in-context learning, inspired by Alayrac et al. (2022).”
Park, S., Cui, C., Ma, Y., Moradipari, A., Gupta, R., Han, K., & Wang, Z. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00756 · Cited by 6
Rai, N., Choi, D. D., Boyd, N. S., & Schumann, A. W. (2026) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2026.111859 · Cited by 10
Agentic AI Needs a Systems Theory
Miehling, E., Ramamurthy, K. N., Varshney, K. R., Riemer, M., Bouneffouf, D., Richards, J. T., Dhurandhar, A., Daly, E. M., Hind, M., Sattigeri, P., et al. (2025) · arXiv · doi:10.48550/arxiv.2503.00237 · Cited by 6
Mdp 3 : a Training-Free Approach for List-Wise Frame Selection in Video-Llms
Sun, H., Lu, S., Wang, H., Chen, Q.-G., Xu, Z., Luo, W., Zhang, K., & Li, M. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.02233 · Cited by 9
“With the success of VLMs [1, 44, 71], frames and queries can be mapped into a unified latent embedding space.”
Ibrahim, A., Alsukkar, M., Xu, Y., Hermaputi, R. L., Yin, S., & Xiao, Y. (2026) · Energy Sustainable Development/Energy for sustainable development · doi:10.1016/j.esd.2026.102104 · Cited by 11
MG-LLaVA: Toward Multi-Granularity Visual Instruction Tuning
Zhao, X., Li, X., Duan, H., Huang, H., Li, Y., Chen, K., & Yang, H. (2025) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2025.3643469 · Cited by 28
“We substitute the gated-fusion model with a resampler in [79].”
MethodGR00T N1: An Open Foundation Model for Generalist Humanoid Robots
NVIDIA, :, Bjorck, J., Castañeda, F., Cherniadev, N., Da, X., Ding, R., Fan, L. "., Fang, Y., Fox, D., et al. (2025) · arXiv · doi:10.48550/arxiv.2503.14734 · Cited by 10
Pseudo-Label Refinement for Multimodal Unsupervised Domain Adaptation
Shi, K., Lu, J., Fang, Z., et al. (2026) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2026.3660284 · Cited by 4
“In recent years, VLMs like CLIP [18] and Flamingo [17] have demonstrated their ability to learn joint representations of images and text through extensive pretraining on large-scale image–text datasets.”
ManuRAG: multi-modal retrieval augmented generation for manufacturing question answering
Li, Y., Dong, Z., Ameri, F., & Zhang, J. (2026) · Journal of Intelligent Manufacturing · doi:10.1007/s10845-026-02800-y · Cited by 5
VLLM-LAD: Visual Large Language Model for Zero-Shot Logical Anomaly Detection
Peng, Y., Lin, X., Ma, N., Liu, C., & Chen, Q. (2026) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2026.3675873 · Cited by 4
“Models such as Flamingo [18], LLaVA [19], and BLIP-2 [20] have demonstrated the ability to process both visual and textual inputs, enabling tasks like image captioning, visual question answering, and multimodal reasoning.”
Guo, W., Jin, X., Jiang, H., Xia, Y., Zhu, J., Qu, B., Zhong, W., Jiang, L., & Lin, X. (2026) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2026.116928 · Cited by 5
Li, X., Zhu, Z., Feng, L., et al. (2025) · doi:10.1109/aibdf67964.2025.11440744 · Cited by 4
“Reference [12] proposed a multi-stage visual token filtering mechanism that enables sparse cross-modal attention computation.”
Improving Noise Robust Audio-Visual Speech Recognition via Router-Gated Cross-Modal Feature Fusion
Lim, D., Kim, Y., Kim, D.-H., Yang, D.-H., & Chang, J.-H. (2025) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru65441.2025.11434748 · Cited by 4
“1(b), inspired by the Flamingo [24].”
Multimodal large language models for food safety detection within deep learning frameworks: a review
Ding, H., Chen, C., Song, X., Dong, G., Chen, H., Hou, H., Cui, X., Yu, W., & Wilsone, D. I. (2026) · Food Chemistry · doi:10.1016/j.foodchem.2026.149208 · Cited by 9
Kibria, M. R., Lafond, S., & Arslan, J. (2026) · Information Fusion · doi:10.1016/j.inffus.2026.104405 · Cited by 12
ImageGen-CoT: Enhancing Text-to-Image in-context Learning with Chain-of-Thought Reasoning
Liao, J., Yang, Z., Li, L., et al. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.01599 · Cited by 5
“This ability allows LLMs to adapt to new tasks by observing a few illustrative examples provided as context, without requiring parameter updates.”
MISSRAG: Addressing the Missing Modality Challenge in Multimodal Large Language Models
Pipoli, V., Saporita, A., Bolelli, F., Cornia, M., Baraldi, L., Grana, C., Cucchiara, R., & Ficarra, E. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00308 · Cited by 8
“By leveraging these complementary modalities, multimodal models have achieved remarkable success in tasks like image captioning [2, 5, 16, 29], audio-video question answering [11, 17, 28], and cross-modal retrieval [1, 9, 35, 40].”
FE-CLIP: Frequency Enhanced CLIP Model for Zero-Shot Anomaly Detection and Segmentation
Gong, T., Chu, Q., Liu, B., Zhou, W., & Yu, N. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.01971 · Cited by 12
“Among recent successes of large pre-trained vision-language models (VLM) [2, 20, 33], CLIP [33] is the first to perform pre-training on web-scale image-text data, showing unprecedented generality: e.g., its language-driven zero-shot inference, improved both effective robustness [40] and perceptual…”
Yan, Y., Zou, B., Liao, Y., Xu, G., Yao, R., Fan, H., Sun, J., Wang, X., Sprinkle, J. M., An, Z., et al. (2026) · Information Fusion · doi:10.1016/j.inffus.2026.104546 · Cited by 33
Ni, T., Yuan, X., Li, S., & Ni, W. (2026) · IEEE Internet of Things Magazine · doi:10.1109/miot.2026.3650753 · Cited by 3
“Compared with caption-centric LLM pipelines and non-private MLLM fi netuning, our focus is a privacy-fi rst integration suitable for IoT edge settings [4], [8].”
InfluentialAquaticCLIP: A Vision-Language Foundation Model and Dataset for Underwater Scene Analysis
Alawode, B., Ganapathi, I. I., Javed, S., Bennamoun, M., & Mahmood, A. (2026) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2026.3657138 · Cited by 18
Mixture of experts for radiology report generation
Song, X., Liu, Z., Hou, X., Li, X., & Zhang, Y. (2026) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2026.114437 · Cited by 3
Short-Video Propagation Influence Rating: A New Real-World Dataset and a New Large Graph Model
Xue, D., Qian, S., Hu, C., & Xu, C. (2026) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2026.3674132 · Cited by 6
“The extension of LLMs to multimodal domains has also gained traction, with models such as GPT-4 [29], Flamingo [37], and Qwen2-VL [17] supporting both textual and visual inputs.”
Vision–language foundation model driven agentic AI systems for healthcare
Chen, L., Xu, X., Qin, Y., Li, H., & Jiang, N. (2026) · The Visual Computer · doi:10.1007/s00371-026-04437-7 · Cited by 3
Huang, J., Tan, Z., Gong, S., Zeng, F., Zhou, J. T., Miao, C., Tan, H., Yao, W., & Li, J. (2026) · International World Wide Web Conference (WWW) · doi:10.1145/3774904.3792793 · Cited by 3
Low-light image enhancement: a multi-stage hybrid approach via Retinex and vision transformers
Sen, S. Y., Türkan, M., & Alankuş, G. (2026) · The Visual Computer · doi:10.1007/s00371-026-04485-z · Cited by 3
“…processing and deep learning with the advent of vision language models (VLMs) and MLLMs. VLMs (e.g., CLIP [63], ALIGN [64]) intend to align the visual and textual features for retrieval tasks, while MLLMs (e.g., Flamingo [65], Kosmos-1 [66]) fortify advanced multimodal generation and reasoning.”
Spatial Preference Rewarding for MLLMs Spatial Understanding
Qiu, H., Gao, P., Lu, L., Zhang, X., Shao, L., & Lu, S. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00075 · Cited by 4
“Recently, the success of large language models (LLMs) [14, 49, 51] has been extended into the multimodal domain, resulting in models that demonstrate impressive performance in integrating vision and language [1, 15, 20, 32, 70].”
EgoM2P: Egocentric Multimodal Multitask Pretraining
Li, G. K., Chen, Y., Wu, Y., Zhao, K., Pollefeys, M., & Tang, S. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.01008 · Cited by 3
“CLIP [81] recognition [42, 100, 128], hand motion and object interaction estimation [9, 24, 118], egocentric video prediction and generation [55, 57, 107, 114], egocentric camera localization [85, 93], among others.”
Token Activation Map to Visually Explain Multimodal LLMs
Li, Y., Wang, H., Ding, X., Wang, H., & Li, X. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00012 · Cited by 8
“Although extensive efforts have been devoted to enhancing the performance of MLLMs [4, 16, 35, 47, 51], research on MLLM explainabil-ity remains less explored.”
Kestrel: 3D Multimodal LLM for Part-Aware Grounded Description
Ahmed, M., Fei, J., Ding, J., Bakr, E. M., & Elhoseiny, M. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00839 · Cited by 13
“Key developments in this area include Flamingo [1], BLIP [29, 31], MiniGPT-4 [68], InstructBLIP [16], and LLaVA [35, 36], which advance AI’s ability to interpret and interact using vision and language.”
Delta-LLaVA: Base-then-Specialize Alignment for Token-Efficient Vision-Language Models
Zamini, M., & Shukla, D. (2026) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61042.2026.00356 · Cited by 3
“These systems typically rely on a projector to map visual features into the language embedding space, enabling joint reasoning over text and images [2, 14, 21].”
Zhuge, Y., Gong, S., Zhang, L., Xu, Q., Zhao, W., Zhan, J., & Lu, H. (2026) · IEEE Transactions on Image Processing · doi:10.1109/tip.2026.3689427 · Cited by 5
Health system learning enables generalist neuroimaging models
Kondepudi, A., Rao, A. T., Zhao, C., Lyu, Y., Harake, E. S., Banerjee, S., Ogle, J. D., Joshi, R. S., Meißner, A.-K., Hou, X., et al. (2026) · Nature Medicine · doi:10.1038/s41591-026-04497-1 · Cited by 17
“To address this, our connector module first employs a Perceiver-style architecture that operates sequencewise (for example, on T1, T2 and FLAIR independently) 45,46 .”
Toward Top-Down Reasoning: An Explainable Multi-Agent Approach for Visual Question Answering
Wang, Z., Wan, W., Lao, Q., Chen, R., Lang, M., Wang, X., Gao, F., Wang, K., & Lin, L. (2026) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2026.3651096 · Cited by 21
Wang, Y., Liu, Q., Fan, J., Hong, J., Chu, H., Tian, M., Gao, B., & Chen, H. (2026) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2026.3673470 · Cited by 13
“Such models have been effectively applied to critical areas, including autonomous driving, where precise scenario comprehension, especially comer case comprehension, is paramount for ensuring safety and functionality in complex environments [3], [4], [5], [6], [7], [8], [9].”
Pi-GPS: Enhancing Geometry Problem Solving by Unleashing the Power of Diagrammatic Information
Zhao, J., Zhang, T., Sun, J., Tian, M., & Hua, H. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00150 · Cited by 4
“Subsequent works [2, 7, 20, 34] expanded MLLMs by incorporating novel visual perception modules and hybrid vision encoders, enhancing their ability to address increasingly complex tasks.”
LeapVAD: A Leap in Autonomous Driving via Cognitive Perception and Dual-Process Thinking
Ma, Y., Wei, T., Zhong, N., Mei, J., Hu, T., Wen, L., Yang, X., Shi, B., & Liu, Y. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2025.3626711 · Cited by 25
“Built on large language models (LLMs) such as LLaMA [11] and Vicuna [12], a wide range of VLMs [13], [14], [15], [16], [17], [18] has emerged, extending their capabilities to multimodal understanding.”
Xia, S.-J., Zhang, H., Jiang, Y., Chen, X., Chen, Y., Li, Z., Wan, Z., & Sun, A. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5731427 · Cited by 3
SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Robotics
Shukor, M., Aubakirova, D., Capuano, F., Kooijmans, P., Palma, S., Zouitine, A., Aractingi, M., Pascal, C., Russi, M., Marafioti, A., et al. (2025) · arXiv · doi:10.48550/arxiv.2506.01844 · Cited by 695
“The success of text-based models has thus been extended to other modalities, sparking interest towards multi-modal vision-language (VLMs) (Alayrac et al., 2022; Chen et al., 2023; Huang et al., 2023; Liu et al., 2023b; Chen et al., 2024; Shukor et al., 2023b) and audio-language models (ALMs)…”
Chain-of-Detection: Enhancing Cross-Granularity Robotic Perception for Object Manipulation
Xu, T., Gao, H., Chen, C., et al. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2025.3626567 · Cited by 2
“While vision–language models (VLMs) [10], [11], [12], [13], [14] have introduced a new paradigm in robotic detection and reasoning, and these methods predominantly focus on coarse-grained object detection, which is often insu ffi cient for applications demanding precise and detailed understanding of…”
Lu, J., Jiang, M., Kong, J., Zhuang, D., & Lu, M. (2025) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2025.3645641 · Cited by 2
Wu, K., Li, X., Li, X., Zuo, K., & Lv, Z. (2026) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2026.3662386 · Cited by 2
“With the emergence of powerful multimodal large language models (MLLMs) such as Flamingo [62], GPT-4V [63], and LLaVA [64], a natural question is whether our continual learning framework can be extended to work with such models.”
Tran, N., Le, T., Ta, P., Nguyen, H., & Nguyen, H. D. (2026) · Neurocomputing · doi:10.1016/j.neucom.2026.133237 · Cited by 2
OpenUrban3D: Label-Free Open-Vocabulary Semantic Segmentation of Large-Scale Urban Point Clouds
Wang, C., Jing, K., ZHU, J., & Wang, D. (2026) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2026.3670962 · Cited by 2
Fine-Tuning Vision-Language Models for Multimodal Polymer Property Prediction
Vuong, A., Van, M.-H., Verma, P., Zhao, C., & Wu, X. (2025) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata66926.2025.11402116 · Cited by 2
“2-Vision family [33] and the Flamingo/OpenFlamingo family [11], [12].”
Yin, B., Wang, S., Zhou, X.-L., & Wang, H. (2026) · Advanced Engineering Informatics · doi:10.1016/j.aei.2026.104594 · Cited by 3
Generative AI for multimodal content: a survey with empirical and experimental evaluations
Taha, K. (2026) · Artificial Intelligence Review · doi:10.1007/s10462-026-11525-6 · Cited by 4
Generative IoT: advancing IoT with generative AI and large language models
Firouzi, F., Ray, A., Farahani, B., Daneshmand, M., Song, J., Wu, S., & Chakrabarty, K. (2026) · Digital Communications and Networks · doi:10.1016/j.dcan.2026.03.010 · Cited by 3
Attention re-alignment in multimodal large language models via intermediate-layer guidance
Chen, Y., Wang, P., Qin, G., Wu, W., Chen, M., & Hao, Y. (2026) · Scientific Reports · doi:10.1038/s41598-026-44935-1 · Cited by 2
Li, Y., Lin, Z., Zhang, Q., Ding, X., Yang, H., Pi, L., Yuan, W., Wen, Y., Guo, L., Zhang, Q., et al. (2026) · Medical Image Analysis · doi:10.1016/j.media.2026.104060 · Cited by 2
Metadata
- Authors
- Alayrac, J.-B., Donahue, J., Luc, P., Miech, A., Barr, I., Hasson, Y., Lenc, K., Mensch, A., Millican, K., Reynolds, M., Ring, R., Rutherford, E., Cabi, S., Han, T., Gong, Z., Samangooei, S., Monteiro, M., Menick, J. L., Borgeaud, S., Brock, A., Nematzadeh, A., Sharifzadeh, S., Binkowski, M., Barreira, R., Vinyals, O., Zisserman, A., & Simonyan, K.
- Venue
- Advances in Neural Information Processing Systems 35
- Published
- 2022
- Pages
- 23716-23736
- ISSN
- 1049-5258 (ISSN-L)
- OpenAlex
- W7133196460
- Cited by
- 966
- References
- 154
- Citations by year
- 2026: 847 · 2025: 107 · 2024: 3 · 2023: 7 · 2022: 2
- FWCI
- 15.48
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- neural information processing systems (publishedVersion)
- Oxford University Research Archive (ORA) (University of Oxford) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- vision-language models, few-shot learning, image-text models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
