The DOI matches its Crossref record; 246 of the 246 references it cites were found in a registry or an index.
The Revolution of Multimodal Large Language Models: A Survey
Open accessAbstract
Davide Caffagni, Federico Cocchi, Luca Barsellotti, Nicholas Moratelli, Sara Sarto, Lorenzo Baraldi, Lorenzo Baraldi, Marcella Cornia, Rita Cucchiara. Findings of the Association for Computational Linguistics: ACL 2024. 2024.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“users should be aware that limitations such as introducing hallucination71 and omitting necessary signals73 are inherent to LLMs and may still impact classification performance, though MoMA is fine-tuned using groundtruth clinical labels.”
From MoMA: a mixture-of-multimodal-agents architecture for enhancing clinical prediction modelling (2025) · cites this paper as [73]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 1×
“The introduction of the attention operation and the Transformer architecture (Vaswani et al., 2017) has enabled the creation of models capable of handling various modalities on an increasingly large scale.”
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404 · cited 5×
“While GILL refrains from fine-tuning both the LLM and the diffusion U-Net, alternative methodologies fine-tune the language model to expand its multimodal generation capabilities.”
InfluentialBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 1×
“Initially, this breakthrough was lever-aged for language-specific models (Devlin et al., 2018; Brown et al., 2020) but quickly extended to support diverse modalities (Li et al., 2019; Lu et al., 2019) and facilitate their integration within unified embedding spaces (Radford et al., 2021).”
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460 · cited 8×
“COCO (Lin et al., 2014) contains more than 120k images, each associated with five human-generated captions.”
InfluentialLanguage Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 2×
“Initially, this breakthrough was lever-aged for language-specific models (Devlin et al., 2018; Brown et al., 2020) but quickly extended to support diverse modalities (Li et al., 2019; Lu et al., 2019) and facilitate their integration within unified embedding spaces (Radford et al., 2021).”
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293 · cited 1×
“Initially, this breakthrough was lever-aged for language-specific models (Devlin et al., 2018; Brown et al., 2020) but quickly extended to support diverse modalities (Li et al., 2019; Lu et al., 2019) and facilitate their integration within unified embedding spaces (Radford et al., 2021).”
LoRA: Low-Rank Adaptation of Large Language Models
Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., & Chen, W. (2021) · arXiv · doi:10.48550/arxiv.2106.09685 · Cited by 13,453 · cited 2×
“This technique is orthogonal to quantization methods such as QLoRA (Dettmers et al., 2024), which further decreases the memory footprint of the LLM compared to the usual half-precision weights.”
Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A. C., Lo, W.-Y., et al. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00371 · Cited by 11,953 · cited 3×
“For output regions, LISA (Lai et al., 2023), GLaMM, GSVA (Xia et al., 2023b), NeXt-Chat (Zhang et al., 2023a), and LISA++ (Yang et al., 2023b) send the embedding corresponding to special tokens to the mask decoder of SAM, LLaVA-G to OpenSeeD (Zhang et al., 2023c), Lenna (Wei et al., 2023) to Grounding-DINO (Liu et al.…”
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · cited 2×
“…used are WebLI (Chen et al., 2023j), composed of 10B images and image-text pairs, and MMC4 (Zhu et al., 2023d), an extension of the text-only C4 (Raffel et al., 2020) dataset composed of 365M documents and 156B tokens relatives to different concepts, and OBELICS (Laurençon et al., 2024), an…”
Research works printed but not linked: 21
- Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Mar- garet Mitchell, Dhruv Batra, C Lawrence Zitnick, and Devi Parikh. 2015. VQA: Visual Question An- swering. In ICCV.
- Satanjeev Banerjee and Alon Lavie. 2005. METEOR: An automatic metric for MT evaluation with im- proved correlation with human judgments. In ACL Workshops.
- Davide Caffagni, Federico Cocchi, Nicholas Moratelli, Sara Sarto, Marcella Cornia, Lorenzo Baraldi, and Rita Cucchiara. 2024. Wiki-LLaVA: Hierarchical Retrieval-Augmented Generation for Multimodal LLMs. In CVPR Workshops.
- Yuxin Fang, Wen Wang, Binhui Xie, Quan Sun, Ledell Wu, Xinggang Wang, Tiejun Huang, Xinlong Wang, and Yue Cao. 2023. Eva: Exploring the limits of masked visual representation learning at scale. In CVPR.
- Tsu-Jui Fu, Wenze Hu, Xianzhi Du, William Yang Wang, Yinfei Yang, and Zhe Gan. 2024. Guid- ing Instruction-based Image Editing via Multimodal Large Language Models. In ICLR.
- Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. 2022. Masked Au- toencoders Are Scalable Vision Learners. In CVPR.
- Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. 2017. GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium. In NeurIPS.
- Yining Hong, Haoyu Zhen, Peihao Chen, Shuhong Zheng, Yilun Du, Zhenfang Chen, and Chuang Gan. 2023. 3D-LLM: Injecting the 3D World into Large Language Models. In NeurIPS.
- Jing Yu Koh, Ruslan Salakhutdinov, and Daniel Fried. 2023b. Grounding Language Models to Images for Multimodal Inputs and Outputs. In ICML.
- Alina Kuznetsova, Hassan Rom, Neil Alldrin, Jasper Ui- jlings, Ivan Krasin, Jordi Pont-Tuset, Shahab Kamali, Stefan Popov, Matteo Malloci, Alexander Kolesnikov, et al. 2020. The Open Images Dataset V4: Unified Image Classification, Object Detection, and Visual Relationship Detection at Scale. IJCV, 128:1956- 1981.
- Fangyu Liu, Guy Emerson, and Nigel Collier. 2023a. Visual Spatial Reasoning. TACL, 11:635-651.
- Fuxiao Liu, Kevin Lin, Linjie Li, Jianfeng Wang, Yaser Yacoob, and Lijuan Wang. 2023c. Mitigating Hallu- cination in Large Multi-Modal Models via Robust In- struction Tuning. arXiv preprint arXiv:2306.14565. · arXiv
- Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023e. Visual Instruction Tuning. In NeurIPS.
- Fan Ma, Xiaojie Jin, Heng Wang, Yuchen Xian, Jiashi Feng, and Yi Yang. 2023a. Vista-LLaMA: Reliable Video Narrator via Equal Distance to Visual Tokens. arXiv preprint arXiv:2312.08870. · arXiv
- Hongyu Wang, Shuming Ma, Shaohan Huang, Li Dong, Wenhui Wang, Zhiliang Peng, Yu Wu, Payal Bajaj, Saksham Singhal, Alon Benhaim, et al. 2023b. Mag- neto: A Foundation Transformer. In ICML.
- Jingkang Yang, Yi Zhe Ang, Zujin Guo, Kaiyang Zhou, Wayne Zhang, and Ziwei Liu. 2022. Panoptic scene graph generation. In ECCV.
- Jiabo Ye, Anwen Hu, Haiyang Xu, Qinghao Ye, Ming Yan, Guohai Xu, Chenliang Li, Junfeng Tian, Qi Qian, Ji Zhang, et al. 2023b. UReader: Universal OCR-free Visually-situated Language Understand- ing with Multimodal Large Language Model. In EMNLP.
- Shukang Yin, Chaoyou Fu, Sirui Zhao, Ke Li, Xing Sun, Tong Xu, and Enhong Chen. 2023a. A Survey on Multimodal Large Language Models. arXiv preprint arXiv:2306.13549. · arXiv
- Shukang Yin, Chaoyou Fu, Sirui Zhao, Tong Xu, Hao Wang, Dianbo Sui, Yunhang Shen, Ke Li, Xing Sun, and Enhong Chen. 2023b. Woodpecker: Hallucina- tion correction for multimodal large language models. arXiv preprint arXiv:2310.16045. · arXiv
- Licheng Yu, Patrick Poirson, Shan Yang, Alexander C Berg, and Tamara L Berg. 2016. Modeling Context in Referring Expressions. In ECCV.
- Yiwu Zhong, Jianwei Yang, Pengchuan Zhang, Chun- yuan Li, Noel Codella, Liunian Harold Li, Luowei Zhou, Xiyang Dai, Lu Yuan, Yin Li, et al. 2022. Re- gionCLIP: Region-based Language-Image Pretrain- ing. In CVPR.
Cited by
Sorted by citations
A Survey on the Memory Mechanism of Large Language Model-based Agents
Zhang, Z., Dai, Q., Bo, X., Ma, C., Li, R., Chen, X., Zhu, J., Dong, Z., & Wen, J.-R. (2025) · ACM Transactions on Information Systems · doi:10.1145/3748302 · Cited by 592
Industrial Foundation Models (IFMs) for intelligent manufacturing: A systematic review
Zhao, S., Liu, S., Jiang, Y., Zhao, B. M., Lv, Y., Zhang, J., Wang, L., & Zhong, R. Y. (2025) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2025.06.011 · Cited by 50
Wang, B., Zheng, Y., Han, X., Kong, L., Xiao, G., Xiao, Z., & Chen, S. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01715-x · Cited by 40
“Employing Multi-modal Large Language Models 163 , next-generation smart glasses not only enhance the quality of real-time decision-making 164 but also deliver personalized suggestions 165 derived from aggregated life and dietary data.”
Towards Retrieval-Augmented Architectures for Image Captioning
Sarto, S., Cornia, M., Baraldi, L., Nicolosi, A., & Cucchiara, R. (2024) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3663667 · Cited by 21
Self-citationSafe-CLIP: Removing NSFW Concepts from Vision-and-Language Models
Poppi, S., Poppi, T., Cocchi, F., Cornia, M., Baraldi, L., & Cucchiara, R. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73668-1_20 · Cited by 35
Self-citationWei, S., Wang, Z., Li, M., Liu, X., & Wu, B. (2025) · Information Processing & Management · doi:10.1016/j.ipm.2025.104089 · Cited by 20
Deep Learning in Palmprint Recognition: A Comprehensive Survey
Gao, C., Yang, Z., Jia, W., Leng, L., Zhang, B., & Teoh, A. B. J. (2026) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2025.3649416 · Cited by 17
“An emerging direction is the adaptation of large language models (LLMs) through multimodal fusion and synthetic palmprint generation [167].”
A Multimodal Framework Embedding Retrieval-Augmented Generation with MLLMs for Eurobarometer Data
Papageorgiou, G., Sarlis, V., Μaragoudakis, M., & Tjortjis, C. (2025) · AI · doi:10.3390/ai6030050 · Cited by 13
“Those approaches advance the models’ contextual understanding and efficiency, making them well suited for applications involving dynamic content generation and interactive AI [22,23].”
Deep Learning in Palmprint Recognition-A Comprehensive SurveyPreprint
Gao, C., Yang, Z., Jia, W., Leng, L., Zhang, B., & Teoh, A. B. J. (2025) · Qeios · doi:10.32388/yz5tfy · Cited by 13
Multimodal language models in agriculture: A tutorial and survey
Haghighat, M., Saleh, A., & Azghadi, M. R. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.104042 · Cited by 18
Large language models for detecting colorectal polyps in endoscopic images
Carlini, L., Massimi, D., Mori, Y., Antonelli, G., Rizkala, T., Spadaccini, M., Lena, C., Parasa, S., Bisschops, R., Renteln, D. A. V., et al. (2025) · Gut · doi:10.1136/gutjnl-2025-335091 · Cited by 12
A Generative AI-Empowered Digital Tutor for Higher Education Courses
Reicher, H., Frenkel, Y., Lavi, M., Nasser, R., Ran-Milo, Y., Sheinin, R., Shtaif, M., & Milo, T. (2025) · Information · doi:10.3390/info16040264 · Cited by 14
“This approach is known as prompt engineering and it is particularly useful with closed source commercial LLMs such as OpenAI’s GPT series, Anthropic’s Claude, Google’s Gemini, and others [27].”
Artificial intelligence performance in pediatric asthma
Yiğit, E. Ş., Taşkırdı, İ., Hacı, İ. A., & Tunçel, T. (2025) · Journal of Asthma · doi:10.1080/02770903.2025.2531500 · Cited by 9
“It answers questions with key-words, images or plugins (7,8).”
Leveraging multimodal large language model for multimodal sequential recommendation
Wang, Z., Liu, B., Huang, W., Hao, T., Zhou, H., & Guo, Y. (2025) · Scientific Reports · doi:10.1038/s41598-025-14251-1 · Cited by 10
“…general knowledge of MLLM, models such as CLIPViT, GPT4V, and LLaMA can effectively integrate and process multimodal data, including text, images, and audio, through pre-training and fine-tuning strategies, demonstrating significant zero-shot recommendation capabilities across various domains 28 .”
Zhu, F., Chen, J., Wen, J., Yang, Y., Yi, C., Tie, Y., Zhang, P., Cai, J., Niyato, D. T., & Guizani, M. (2026) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2026.3665395 · Cited by 26
“It necessitates the cross-modal fusion of subtle cues, such as language, gestures, and tone, to decipher a user’s implicit intent, which is not fully contained within any single modality [106], [107].”
Computer Audition: From Task-Specific Machine Learning to Foundation Models
Triantafyllopoulos, A., Tsangko, I., Gebhard, A., Mesaros, A., Virtanen, T. I., & Schuller, B. W. (2025) · Proceedings of the IEEE · doi:10.1109/jproc.2025.3593952 · Cited by 16
“Recent surveys have documented this trend [7], [8], [9], [10], [11], [12].”
Multimodal AI for UAV: Vision–Language Models in Human– Machine Collaboration
Krupáš, M., Urblík, Ľ., & Zolotová, I. (2025) · Electronics · doi:10.3390/electronics14173548 · Cited by 12
Can LLMs Generate Behaviors for Embodied Virtual Agents Based on Personality Traits?
Han, B., Kwon, D., Lin, S., Shrestha, K., & Gratch, J. S. (2025) · ACM International Conference on Intelligent Virtual Agents (IVA) · doi:10.1145/3717511.3747060 · Cited by 6
Hao, C., Wang, Z., Mao, X., He, S., Li, B., Liu, H., Peng, F., & Li, W. (2025) · Computers in Industry · doi:10.1016/j.compind.2025.104302 · Cited by 7
BRIDGE: Bridging Gaps in Image Captioning Evaluation with Stronger Visual Cues
Sarto, S., Cornia, M., Baraldi, L., & Cucchiara, R. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73229-4_5 · Cited by 7
Self-citationHyperbolic Deep Learning for Foundation Models: A Survey
He, N., Madhu, H., Bui, N., Yang, M., & Ying, R. (2025) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3711896.3736564 · Cited by 13
Liu, Y., Lerch, L., Palmieri, L., et al. (2025) · IEEE International Conference on Robot and Human Interactive Communication (RO-MAN) · doi:10.1109/ro-man63969.2025.11217803 · Cited by 4
“MLLMs typically consist of three main components [30]: a language model backbone, a vision encoder, and a vision-to-language adapter (see Fig.”
Boosting Text-To-Image Generation via Multilingual Prompting in Large Multimodal Models
Mu, Y., Li, H., Wang, J., Zhou, X., Wang, C., Luo, Y., He, Q., Xiao, T., Chen, G., & Zhu, J. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10889541 · Cited by 3
Multiple Large AI Models’ Consensus for Object Detection—A Survey
Iwanowski, M., & Gahbler, M. (2025) · Applied Sciences · doi:10.3390/app152412961 · Cited by 7
Song, W., Nabahirwa, E., Zhang, M., Fang, Y., & Ni, Z. (2026) · Discover Artificial Intelligence · doi:10.1007/s44163-026-00895-z · Cited by 12
Sun, H., Wang, T., Yuan, L., Zhao, J., & Lv, J. (2026) · Information Fusion · doi:10.1016/j.inffus.2026.104285 · Cited by 4
“Caffagni et al. [20] review earlier stages of MLLM development and analyze their architectural choices, multimodal alignment strategies, and training techniques.”
A Survey on Fine-Grained Multimodal Large Language Models
Peng, Y., Wang, Z., Li, G., Zheng, X., Yin, S., & He, H. (2026) · Chinese Journal of Electronics · doi:10.23919/cje.2025.00.336 · Cited by 10
Fluent and Accurate Image Captioning with a Self-trained Reward Model
Moratelli, N., Cornia, M., Baraldi, L., & Cucchiara, R. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-78456-9_14 · Cited by 4
Self-citationUnlearning Vision Transformers Without Retaining Data via Low-Rank Decompositions
Poppi, S., Sarto, S., Cornia, M., Baraldi, L., & Cucchiara, R. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-78122-3_10 · Cited by 7
“For its efficiency, LoRA has been used across different research fields, like the fine-tuning of large language models to adapt them for various multimodal tasks [5,6] and foundation models fine-tuning for improving their safety [38].”
Self-citationPixels of Faith: Exploiting Visual Saliency to Detect Religious Image Manipulation
Cartella, G., Cuculo, V., Cornia, M., Papasidero, M., Ruozzi, F., & Cucchiara, R. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-91572-7_14 · Cited by 6
Self-citationSheng, Z., Huang, Z., Qu, Y., Chen, J., Luo, Y., Chen, Y., Leng, Y., & Chen, S. (2026) · CHAIN · doi:10.23919/chain.2026.000005 · Cited by 9
“Recent advances in multimodal large language models (MLLMs) have demonstrated remarkable capabilities in understanding and reasoning over visual and linguistic information, enabling a wide range of applications from vi-* sual question answering (QA) to video analysis [2, 24, 44].”
Personalizing Multimodal Large Language Models for Image Captioning: An Experimental Analysis
Bucciarelli, D., Moratelli, N., Cornia, M., Baraldi, L., & Cucchiara, R. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-92089-9_22 · Cited by 17
Self-citationMoMA: a mixture-of-multimodal-agents architecture for enhancing clinical prediction modelling
Gao, J., Rahman, M., Caskey, J., Oguss, M., O’Rourke, A., Brown, R., Stey, A., Mayampurath, A., Churpek, M. M., Chen, G., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-02219-4 · Cited by 2
“…summary overwrote textual information (e.g., “ The specialist reports right basal atelectasis, small right pleural effusion, and cardiomegaly, whereas the Aggregator — guided by the original clinical and radiology notes — states no effusion or atelectasis and does not mention cardiomegaly ” ).”
Adapt to Scarcity: Few-Shot Deepfake Detection via Low-Rank Adaptation
Cappelletti, S., Baraldi, L., Cocchi, F., Cornia, M., Baraldi, L., & Cucchiara, R. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-78305-0_8 · Cited by 2
“Low-Rank Adaptation (LoRA) [27], which originates for parameter efficient fine-tuning of large language models [16,28], has demonstrated its effectiveness in various tasks [7,8,39,48].”
Self-citationWang, Z., Fang, M., Tang, L., et al. (2025) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2025.3558775 · Cited by 2
“Fine-tuning these models with carefully curated datasets has set new performance benchmarks in multiple tasks [8].”
InfluentialA Semantic Compression Framework for Video Surveillance Applications
Karaa, M., Bahria, R., Ghazzai, H., et al. (2024) · IEEE Canadian Conference on Electrical and Computer Engineering (CCECE) · doi:10.1109/ccece59415.2024.10667302 · Cited by 1
“We leverage the advancements of MLLM in video-related tasks such as understanding and captioning [4] to generate captions for surveillance videos.”
MethodLu, B., Wang, X., Ming, W., & Wang, X. (2025) · Journal Of Big Data · doi:10.1186/s40537-025-01290-8 · Cited by 1
Reasoning LLMs for User-Aware Multimodal Conversational Agents
Rahimi, H., Cattoni, J., Beghili, M., et al. (2025) · IEEE International Conference on Robot and Human Interactive Communication (RO-MAN) · doi:10.1109/ro-man63969.2025.11217707 · Cited by 1
“Recent advancements in Multimodal Large Language Models (MLLMs) [6] have significantly enhanced the capacity of robots to interpret textual and visual cues, enabling more intuitive and adaptive interactions [7], [8].”
Omnidirectional Spatial Modeling from Correlated Panoramas
Zhang, X., Fu, T., & Xu, Z. (2025) · ACM International Conference on Multimedia in Asia (MMAsia) · doi:10.1145/3743093.3770977 · Cited by 1
Igualde-Sáez, A., Rhomrasi, L., Ahsini, Y., Vinuesa, R., Hoyas, S., Garcia-Sabater, J. P., Fullana-i-Alfonso, M. J., & Conejero, J. A. (2026) · The Journal of Supercomputing · doi:10.1007/s11227-026-08291-1 · Cited by 1
“…deeper evaluations indicate that many models still default to superficial pattern matching rather than genuinely reasoning about visual content and math tasks [27], such as diagrams in geometry, charts in algebraic contexts, and multilingual notations require robust cross-modal alignment [28].”
Guainazzo, N., Delzanno, G., Ancona, D., & D’Agostino, D. (2026) · Sensors · doi:10.3390/s26051590 · Cited by 1
Deus in Machina: A Conversational AI for Spiritual Experience
Haslbauer, P., Reichherzer, C., Schmid, M., & Smolić, A. (2025) · International Conference on Digital and Interactive Arts (ARTECH) · doi:10.1145/3773699.3773922 · Cited by 2
Eck, S. O., & Page, K. (2026) · International Conference on Digital Libraries for Musicology (DLfM) · doi:10.1145/3815723.3815726 · Cited by 1
From text to multimodality: A comprehensive survey of LLMs and MLLMs across diverse domains
Anbukkarasi, S., Hemalatha, S., Arunkumar, B., Shanmugavalli, V., Kokila, S., & A, Y. K. (2026) · ICT Express · doi:10.1016/j.icte.2026.07.023 · Cited by 1
Burghardt, M. (2025) · Elsevier eBooks · doi:10.1016/b978-0-443-26629-4.00016-2 · Cited by 2
SoleCoach: Sole Pressure and IMU-based MLLMs for Skill Coaching
Hirano, T., Yoshihara, H., Peng, Y., Liao, C.-C., Wu, E., & Koike, H. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791181
“To overcome these limitations, large language models (LLMs) have been introduced into sports coaching.”
Ye, Z., Wang, X. Q., Zhang, C., Zhang, R., Sun, M., Li, Q., Yu, C., & Shi, Y. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790922 · Cited by 2
“Moreover, the models are trained on large-scale coarse-grained caption data crawled from the Internet [13].”
PERFORMANCE COMPARISON OF VISION-LANGUAGE MODELS IN IMAGE CLASSIFICATION
Özeren, D., Yüksel, M. E., & Yüksel, A. S. (2025) · International Journal of 3D Printing Technologies and Digital Industry · doi:10.46519/ij3dptdi.1693015
Enhancing Chinese Character Writing Learning: The Role of MLLM-based Intelligent Tutoring Systems
Su, B., Chen, Q., Peng, J., Tan, W., & Wang, L. (2025) · doi:10.1109/icaie64856.2025.11158428 · Cited by 1
“A. FunctionalArchitectureofChineseCharacterWritingITS The internal operating mechanism of MLLM is mainly divided into three components, including the input layer, the analysis layer, and the output layer [23].”
Optimizing Multimodal Language Models Through Attention-Based Interpretability
Sergeev, A., & Kotelnikov, E. V. (2025) · Communications in computer and information science · doi:10.1007/978-3-032-07175-0_25
Detection and Measurement of Hailstones with Multimodal Large Language Models
Alker, M., Schedl, D. C., & Stöckl, A. (2025) · doi:10.1109/icecer65523.2025.11400929
“Caffagni et al. [11] provide a comprehensive survey of recent multimodal LLMs, detailing how these models integrate visual and textual modalities with dialogue interfaces.”
Adversarial Iterative Pre-enactment Framework for Air Combat Based on Mental Simulation Theory
Han, S., Zhang, X., Hu, T., & Ma, H. (2025) · Lecture notes in computer science · doi:10.1007/978-981-95-3398-5_18
It's All Connected: A Survey for Multimodal Arabic AIPreprint
Aldabbas, F., Elsafty, H., & Sifa, R. (2025) · Research Square · doi:10.21203/rs.3.rs-8007923/v1
Jailbreaking MLLMs via Attention Redirection and Entropy Regularization
Du, J., Dong, F., & Zhang, F. (2026) · Electronics · doi:10.3390/electronics15010237
Evaluating MLLM Security for the Arabic and French Languages
Abdallah, R., Sanan, M., & Nassar, M. (2025) · doi:10.1109/etecom66111.2025.11319129
“…is composed of visual encoders that are responsible for extracting features from image inputs, an LLM serving as the backbone for handling textual data and integrating multi-modal inputs effectively, and adapters that align non-text modalities to the same embedding space as textual inputs [11].”
Multimodal Image Caption Generation with Semantic and Temporal Context
Rafi, S., Das, R., Rao, P. A., et al. (2025) · International Conference on Soft Computing & Machine Intelligence (ISCMI) · doi:10.1109/iscmi67495.2025.11358624
“The Transformer model [7] has since been investigated in various newer methods for vision-language problems suggested using the Transformer architecture in lieu of traditional RNNs to achieve new state-of-the-art performance by greatly enhancing the quality of the sentences produced.”
Fine-Grained Image-Text Multi-Modal Learning for Substation Defect Detection
Wu, G., Li, X., Cheng, L., Zhang, G., Zipei, D. O. N. G., & Dai, L. (2025) · International Conference on Computer, Vision and Intelligent Technology (ICCVIT) · doi:10.1109/iccvit67848.2025.11391367
GIICS: A dataset for gaze-direction-based intention inference in complex visual scenes
Yu, Q., Ling, C., Feng, S., Hu, J., Lu, Y., & Ding, N. (2025) · International Conference on Artificial Intelligence, Human-Computer Interaction and Robotics (AIHCIR) · doi:10.1109/aihcir67580.2025.11404921
“(3) We benchmarked numerous VLLMs (e.g., Llama 3 [6], LLaVA-1.”
OwnerHunter: Multilingual Website Owner Identification Powered by Large Language Model
Tu, C. S., Dong, E., Zhang, Y., Wang, S., Zhang, Z., Zhang, M., Li, Y., Xue, P., & Yang, J. (2026) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2026.3666852
“MLLMs can seamlessly integrate visual and textual modalities while providing a dialogue-based interface and instruction-following capabilities [17].”
Shao, X., Ma, Q., Wang, Y., et al. (2026) · Discover Applied Sciences · doi:10.1007/s42452-026-08539-8
Taranowski, A. O., & Samoylov, V. B. (2026) · Èlektronnoe modelirovanie · doi:10.15407/emodel.48.01.051
A comprehensive survey on table question answering: Datasets, methods and future directions
Xu, W., Liu, Y., Lu, L., Li, H., & Sun, G. (2026) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2026.114459
Литвиненко, М. І., Осієвський, С., Ленець, В. Г., Гармаш, Н. В., Орєхов, С. В., & Дрозд, К. А. (2026) · Системи обробки інформації · doi:10.30748/soi.2026.184.06
Developing a Sinhala Speech Recognition System Using Whisper
Thameera, W. S., Ruwanthilaka, S. D. S. S., Shanaz, A. L. F., et al. (2026) · International Conference on Advanced Research in Computing (ICARC) · doi:10.1109/icarc68737.2026.11454008
“Models such as OpenAI’s GPT-4 and Google’s Gemini integrate text, audio, and visual modalities, enabling tasks that range from natural language generation to multimodal reasoning [4].”
RADI: A Retrieval-Augmented Dynamic In-Context Learning Framework for AIGC Image Detection
Bi, T., Ma, R., Huang, Y., Wang, Y., Liu, J., & Zhang, X. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11462412
Token-Anchored Semantic Drift: Untargeted Transferable Attacks Against Commercial MLLMs
Shen, T. (2026) · International Conference on Generative Artificial Intelligence and Information Security (GAIIS) · doi:10.1109/gaiis69281.2026.11519348
“Commercial MLLMs [1]–[3]are becoming the default interface for image understanding and multimodal assistance.”
Taşkırdı, İ., Yiğit, E. Ş., Akarcan, S. E., & Tunçel, T. (2026) · Journal of Clinical Medicine · doi:10.3390/jcm15114025
FSMVision: Semantic Extraction of Finite State Machines from Diagrams via Multimodal AI
Bakchy, S. C., Islam, M. A., Rahman, S., & Rahman, M. T. (2026) · International Symposium on Quality Electronic Design (ISQED) · doi:10.1109/isqed69900.2026.11534790
“The MLLMs have demonstrated significant capabilities to analyze different media like images, videos, and audio [8].”
Harrouchi, L. E., Moussaoui, H., Karmoudi, M., et al. (2026) · Springer proceedings in mathematics & statistics · doi:10.1007/978-3-032-18042-1_16
SuperVoxelGPT: Adaptive and Ordered 3D Tokenization for Autoregressive Shape Generation
Li, Y., Zhang, C., Gao, X., & Guo, X. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37577-3_22
FLIC: A Real-World Dataset for Visual Estimation of Food Leftovers in Canteens
Piccoli, F., Callegaro, D., Marelli, D., et al. (2026) · Applied Sciences · doi:10.3390/app16115465
Hu, C., Chen, Y., Liang, T., Wei, Y., Li, A., Shi, H., Bian, L., & Shi, X. (2026) · International Conference on Multimedia Retrieval · doi:10.1145/3805622.3810757
Wang, H., & Wan, D. (2026) · International Conference on Multimedia Retrieval · doi:10.1145/3805622.3810610
Unified Multimodal AI Engine for Real-Time Summarization of Text, Images, and Documents
Rakshitha, G., Rajamani, T., Bhardwaj, M., et al. (2026) · doi:10.1109/icici68867.2026.11564932
“A study on architectural improvements in multimodal LLMs has shown that there are still problems with hallucinatory output and coherence when visually, audibly, and textually processed together [2].”
Match me if you can: aligning historical posters and newspaper articles via multimodal deep learning
Xu, F., Düring, M., & Leiva, L. A. (2026) · Digital Scholarship in the Humanities · doi:10.1093/llc/fqag088
García-López, I. M., Espinosa, J. M. M., & Soledad, M. (2026) · IEEE Revista Iberoamericana de Tecnologias del Aprendizaje · doi:10.1109/rita.2026.3701046
Wang, X., & Yang, Y. (2026) · International Journal of Pattern Recognition and Artificial Intelligence · doi:10.1142/s0218001426400215
Same, Same But Different? A Cross-Domain Analysis of Chatbot User Interfaces
Rockstroh, J., Spillner, L., Wenig, N., Mildner, T., Zargham, N., & Malaka, R. (2026) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3816046.3816229
“Interaction features such as Extended Input or Input Adjustment , as well as richer message formats like Image Chat Message [6] or Sticker Chat Message [37], rely on subsymbolic information processing enabled by state-of-the-art Vision Language Models [21, 22].”
Survey on Multi-Modal Large Language Models: Bridging Text, Vision, and Beyond
Bhashkar, K., & Kumar, S. (2025) · IEEE International Conference on Innovate for Humanitarian: Tech Solutions for Global Challenge (ICIH) · doi:10.1109/icih67754.2025.11608965
“The interface between the various modalities is Multimodal fusion module i.e., the language model is able to take and answer problems on mixed multimodal input [6].”
RaTA-Tool: Retrieval-Based Tool Selection with Multimodal Large Language Models
Mattioli, G., Turri, E., Sarto, S., Baraldi, L., Cornia, M., Baraldi, L., & Cucchiara, R. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-31397-3_21 · Cited by 2
Self-citationShi, S., Yin, R., Xu, N., Chen, J., Zhou, C., Du, T., Fu, Z. F., Wang, J., Wang, Z., & Ji, S. (2026) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3770855.3817756
Yu, Z., Xing, J., Zhao, L., Lei, D., Chen, L., Han, Y., & Ren, W. (2026) · Research Square · doi:10.21203/rs.3.rs-10211545/v1
Huang, X., Wang, X., Wang, B., et al. (2026) · 2026 5th International Conference on Energy and Electrical Power Systems (ICEEPS) · doi:10.1109/iceeps70377.2026.11662894
“Recent surveys and technical reports on multimodal large models further demonstrate this trend [2], [3].”
Generative AI Technologies, Techniques and Tensions: A Primer
Behrens, J. T. (2026) · Springer eBooks · doi:10.1007/978-3-032-24819-0_2 · Cited by 2
Zhao, B. (2026) · ITM Web of Conferences · doi:10.1051/itmconf/20269103014
“Interestingly, large-scale multimodal corpora heavily depend on machine-assisted data curation; however, human feedback continues to play a significant role in supervising annotation precision and monitoring overall data quality [7, 8, 13, 14].”
Metadata
- Authors
- Caffagni, D., Cocchi, F., Barsellotti, L., Moratelli, N., Sarto, S., Baraldi, L., Baraldi, L., Cornia, M., & Cucchiara, R.
- Affiliations
- University of Modena and Reggio Emilia (IT)
- University of Pisa (IT)
- ORCID
- Venue
- Findings of the Association for Computational Linguistics ACL 2024
- Published
- 2024
- Pages
- 13590-13618
- OpenAlex
- W4402670262
- Cited by
- 98
- References
- 171
- Citations by year
- 2026: 44 · 2025: 46 · 2024: 8
- FWCI
- 20.26
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Findings of the Association for Computational Linguistics: ACL (Findings of ACL) (publishedVersion, cc-by)
- Funders
- European Commission (PNRR-M4C2, PE00000013)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- multimodal large language models, multimodal language models, vision-language models, large language models, multimodal representation learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
