Visual Chain-of-Thought Prompting for Knowledge-Based Visual Reasoning
Open accessAbstract
Knowledge-based visual reasoning remains a daunting task since it not only requires machines to interpret the concepts and relationships from visual scenes but also associate them with external world knowledge to conduct a chain of reasoning on open-world questions. Previous works, however, treat visual perception and language-based reasoning as two independent modules, failing to attend to both modules throughout all stages of reasoning. To this end, we propose Visual Chain-of-thought Prompting (VCTP) for knowledge-based reasoning, which involves the interaction between visual content and natural language in an iterative step-by-step reasoning manner. VCTP contains three stages, see, think, and confirm. The see stage scans the image and grounds the visual concept candidates with a visual perception model. The think stage adopts a pre-trained large language model (LLM) to attend to key visual concepts from natural language questions adaptively. It then transforms key visual context into text context for prompting with a visual captioning model, and adopts the LLM to generate the answer. The confirm stage further uses the LLM to generate the supporting rationale to the answer, which is then passed through a cross-modality classifier to verify that it’s consistent with the visual context. We iterate through the think-confirm stages to ensure the verified rationale is consistent with the answer. We conduct experiments on a range of knowledge-based visual reasoning datasets. We found our VCTP enjoys several benefits, 1). it achieves better performance than the previous few-shot learning baselines; 2). it enjoys the total transparency and trustworthiness of the whole reasoning process by providing rationales for each reasoning step; 3). it is computation-efficient compared with other fine-tuning baselines. Our code is available at https://github.com/UMass-Foundation-Model/VisualCoT.git
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Visual CoT methods operate on a fixed image and remain reasoning-centric.
“Recent works such as Visual CoT prompting (Chen et al., 2024), ARGUS (Man et al., 2025), ReFocus (Fu et al., 2025), Chain-of-Spot (Liu et al., 2024d), GRIT (Fan et al., 2025), and spatial-reasoning studies (Zhong et al., 2026b; Zhu et al., 2026) improve multimodal reasoning by generating grounded reasoning chains or editing visual evidence, yet they operate on a fixed image and remain reasoning-”
From ACTIVE-o3: Empowering MLLMs with Active Perception via Pure Reinforcement Learning (2025) · cites this paper as Chen et al., 2024
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 56,035 · see citing works
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · see citing works
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293 · see citing works
Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations
Krishna, R., Zhu, Y., Groth, O., Johnson, J., Hata, K., Kravitz, J., Chen, S., Kalantidis, Y., Li, L.-J., Shamma, D. A., et al. (2017) · International Journal of Computer Vision · doi:10.1007/s11263-016-0981-7 · Cited by 5,107 · cited 1× · see citing works
“We choose the faster R-CNN model (Ren et al. 2015) released by (Han et al. 2021) to detect visual concepts in images, which was trained on Visual Genome (Krishna et al. 2017).”
MethodChain-Of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1800 · Cited by 4,505 · see citing works
VQA: Visual Question Answering
Antol, S., Agrawal, A., Lu, J., Mitchell, M., Batra, D., Zitnick, C. L., & Parikh, D. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.279 · Cited by 4,487 · see citing works
GQA: A New Dataset for Real-World Visual Reasoning and Compositional Question Answering
Hudson, D. A., & Manning, C. D. (2019) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2019.00686 · Cited by 3,703 · see citing works
Learning to Prompt for Vision-Language Models
Zhou, K., Yang, J., Loy, C. C., & Liu, Z. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-022-01653-1 · Cited by 3,304 · see citing works
Llama 2: Open Foundation and Fine-Tuned Chat Models
Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.09288 · Cited by 2,604 · see citing works
PaLM: Scaling Language Modeling with Pathways
Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.02311 · Cited by 2,590 · see citing works
LXMERT: Learning Cross-Modality Encoder Representations from Transformers
Tan, H., & Bansal, M. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1514 · Cited by 2,319 · see citing works
Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering
Goyal, Y., Khot, T., Summers-Stay, D., Batra, D., & Parikh, D. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.670 · Cited by 2,287 · see citing works
ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks
Lu, J., Batra, D., Parikh, D., & Lee, S. (2019) · arXiv · doi:10.48550/arxiv.1908.02265 · Cited by 2,109 · see citing works
Jia, M., Tang, L., Chen, B.-C., Cardie, C., Belongie, S., Hariharan, B., & Lim, S.-N. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-19827-4_41 · Cited by 1,965 · see citing works
Conditional Prompt Learning for Vision-Language Models
Zhou, K., Yang, J., Loy, C. C., & Liu, Z. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01631 · Cited by 1,918 · see citing works
CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning
Johnson, J., Hariharan, B., van der Maaten, L., Fei-Fei, L., Zitnick, C. L., & Girshick, R. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.215 · Cited by 1,781 · see citing works
Li, J., Li, D., Xiong, C., & Hoi, S. (2022) · arXiv · doi:10.48550/arxiv.2201.12086 · Cited by 1,741 · see citing works
Flamingo: a Visual Language Model for Few-Shot Learning
Alayrac, J.-B., Donahue, J., Luc, P., Miech, A., Barr, I., Hasson, Y., Lenc, K., Mensch, A., Millican, K., Reynolds, M., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.14198 · Cited by 1,555 · see citing works
Evaluating Large Language Models Trained on Code
Chen, M., Tworek, J., Jun, H., Yuan, Q., Pinto, H. P. D. O., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., et al. (2021) · arXiv · doi:10.48550/arxiv.2107.03374 · Cited by 1,436 · see citing works
Multimodal Chain-of-Thought Reasoning in Language ModelsPreprint
Zhang, Z., Zhang, A., Li, M., Zhao, H., Karypis, G., & Smola, A. J. (2023) · Trans. Mach. Learn. Res. · doi:10.48550/arxiv.2302.00923 · Cited by 994 · see citing works
Andreas, J., Rohrbach, M., Darrell, T., & Klein, D. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.12 · Cited by 933 · see citing works
Li, J., Li, D., Savarese, S., & Hoi, S. (2023) · arXiv · doi:10.48550/arxiv.2301.12597 · Cited by 908 · see citing works
OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledge
Marino, K., Rastegari, M., Farhadi, A., & Mottaghi, R. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00331 · Cited by 748 · see citing works
Bayesian Rationality: The Probabilistic Approach to Human Reasoning
Oaksford, M., & Chater, N. (2007) · Cited by 655 · see citing works
Do As I Can, Not As I Say: Grounding Language in Robotic Affordances
Ahn, M., Brohan, A., Brown, N., Chebotar, Y., Cortes, O., David, B., Finn, C., Fu, C., Gopalakrishnan, K., Hausman, K., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.01691 · Cited by 624 · see citing works
RegionCLIP: Region-based Language-Image Pretraining
Zhong, Y., Yang, J., Zhang, P., Li, C., Codella, N., Li, L. H., Zhou, L., Dai, X., Yuan, L., Li, Y., et al. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01629 · Cited by 623 · see citing works
VQA: Visual Question Answering: www.visualqa.org
Agrawal, A., Lu, J., Antol, S., Mitchell, M., Zitnick, C. L., Parikh, D., & Batra, D. (2017) · International Journal of Computer Vision · doi:10.1007/s11263-016-0966-6 · Cited by 570 · see citing works
FVQA: Fact-Based Visual Question Answering
Wang, P., Wu, Q., Shen, C., Dick, A., & van den Hengel, A. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2754246 · Cited by 490 · see citing works
Human Reasoning and Cognitive Science
Stenning, K., & van Lambalgen, M. (2008) · The MIT Press eBooks · doi:10.7551/mitpress/7964.001.0001 · Cited by 439 · see citing works
LXMERT: Learning Cross-Modality Encoder Representations from TransformersPreprint
Tan, H. X., & Bansal, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.07490 · Cited by 433 · see citing works
Prompting Visual-Language Models for Efficient Video Understanding
Ju, C., Han, T., Zheng, K., Zhang, Y., & Xie, W. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19833-5_7 · Cited by 423 · see citing works
Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language UnderstandingPreprint
Yi, K., Wu, J., Gan, C., Torralba, A. B., Kohli, P., & Tenenbaum, J. B. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.02338 · Cited by 388 · see citing works
Self-citationLeast-to-Most Prompting Enables Complex Reasoning in Large Language Models
Zhou, D., Schärli, N., Hou, L., Wei, J., Scales, N., Wang, X., Schuurmans, D., Cui, C., Bousquet, O., Le, Q., et al. (2022) · arXiv · doi:10.48550/arxiv.2205.10625 · Cited by 380 · see citing works
Multimodal Few-Shot Learning with Frozen Language Models
Tsimpoukelli, M., Menick, J., Cabi, S., Eslami, S. M. A., Vinyals, O., & Hill, F. (2021) · arXiv · doi:10.48550/arxiv.2106.13884 · Cited by 355 · see citing works
An Empirical Study of GPT-3 for Few-Shot Knowledge-Based VQA
Yang, Z., Gan, Z., Wang, J., Hu, X., Lu, Y., Liu, Z., & Wang, L. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i3.20215 · Cited by 334 · see citing works
A-OKVQA: A Benchmark for Visual Question Answering Using World Knowledge
Schwenk, D., Khandelwal, A., Clark, C., Marino, K., & Mottaghi, R. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20074-8_9 · Cited by 295 · see citing works
CLEVRER: CoLlision Events for Video REpresentation and ReasoningPreprint
Yi, K., Gan, C., Li, Y., Kohli, P., Wu, J., Torralba, A. B., & Tenenbaum, J. B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.01442 · Cited by 283 · see citing works
Self-citationExplicit Knowledge-based Reasoning for Visual Question Answering
Wang, P., Wu, Q., Shen, C., Dick, A., & van den Hengel, A. (2017) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/179 · Cited by 240 · see citing works
PaLI: A Jointly-Scaled Multilingual Language-Image ModelPreprint
Chen, X., Wang, X., Changpinyo, S., Piergiovanni, A., Padlewski, P., Salz, D., Goodman, S., Grycner, A., Mustafa, B., Beyer, L., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2209.06794 · Cited by 235 · see citing works
Domain Adaptation via Prompt Learning
Ge, C., Huang, R., Xie, M., Lai, Z., Song, S., Li, S., & Huang, G. M. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3327962 · Cited by 223 · see citing works
Socratic Models: Composing Zero-Shot Multimodal Reasoning with Language
Zeng, A., Attarian, M., Ichter, B., Choromanski, K., Wong, A., Welker, S., Tombari, F., Purohit, A., Ryoo, M., Sindhwani, V., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.00598 · Cited by 205 · see citing works
KRISP: Integrating Implicit and Symbolic Knowledge for Open-Domain Knowledge-Based VQA
Marino, K., Chen, X., Parikh, D., Gupta, A., & Rohrbach, M. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01389 · Cited by 204 · see citing works
MERLOT RESERVE: Neural Script Knowledge through Vision and Language and Sound
Zellers, R., Lu, J., Lu, X., Yu, Y., Zhao, Y., Salehi, M., Kusupati, A., Hessel, J., Farhadi, A., & Choi, Y. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01589 · Cited by 163 · see citing works
ConceptBert: Concept-Aware Representation for Visual Question Answering
Gardères, F., Ziaeefard, M., Abeloos, B., & Lécué, F. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.44 · Cited by 151 · see citing works
Pythia v0.1: the Winning Entry to the VQA Challenge 2018Preprint
Jiang, Y. L., Natarajan, V., Chen, X., Rohrbach, M., Batra, D., & Parikh, D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.09956 · Cited by 150 · see citing works
Mucko: Multi-Layer Cross-Modal Knowledge Reasoning for Fact-based Visual Question Answering
Zhu, Z., Yu, J., Wang, Y., Sun, Y., Hu, Y., & Wu, Q. (2020) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/153 · Cited by 148 · see citing works
Unified-IO: A Unified Model for Vision, Language, and Multi-Modal Tasks
Lu, J., Clark, C., Zellers, R., Mottaghi, R., & Kembhavi, A. (2022) · arXiv · doi:10.48550/arxiv.2206.08916 · Cited by 145 · see citing works
Jin, W., Cheng, Y., Shen, Y., Chen, W., & Ren, X. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.197 · Cited by 144 · see citing works
Self-citationMulti-Modal Answer Validation for Knowledge-Based VQA
Wu, J., Lu, J., Sabharwal, A., & Mottaghi, R. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i3.20174 · Cited by 141 · see citing works
Ding, Y., Yu, J., Liu, B., Hu, Y., Cui, M., & Wu, Q. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00503 · Cited by 137 · see citing works
MuRAG: Multimodal Retrieval-Augmented Generator for Open Question Answering over Images and Text
Chen, W., Hu, H., Chen, X., Verga, P., & Cohen, W. W. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.375 · Cited by 134 · see citing works
KAT: A Knowledge Augmented Transformer for Vision-and-Language
Gui, L., Wang, B., Huang, Q., Hauptmann, A., Bisk, Y., & Gao, J. (2022) · Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2022.naacl-main.70 · Cited by 130 · see citing works
Selection-Inference: Exploiting Large Language Models for Interpretable Logical ReasoningPreprint
Creswell, A., Shanahan, M., & Higgins, I. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2205.09712 · Cited by 122 · see citing works
Transform-Retrieve-Generate: Natural Language-Centric Outside-Knowledge Visual Question Answering
Gao, F., Ping, Q., Thattai, G., Reganti, A., Wu, Y. N., & Natarajan, P. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00501 · Cited by 105 · see citing works
Retrieval Augmented Visual Question Answering with Outside Knowledge
Lin, W., & Byrne, B. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.772 · Cited by 85 · see citing works
Language Models with Image Descriptors Are Strong Few-Shot Video-Language Learners
Wang, Z., Li, M., Xu, R., Zhou, L., Lei, J., Lin, X., Wang, S., Yang, Z., Zhu, C., Hoiem, D., et al. (2022) · neural information processing systems · doi:10.52202/068431-0617 · Cited by 76 · see citing works
Few-Shot Self-Rationalization with Natural Language Prompts
Marasović, A., Beltagy, I., Downey, D., & Peters, M. E. (2022) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2022.findings-naacl.31 · Cited by 64 · see citing works
Grounding Physical Concepts of Objects and Events Through Dynamic Visual ReasoningPreprint
Chen, Z., Mao, J., Wu, J., Wong, K.-Y. K., Tenenbaum, J. B., & Gan, C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.16564 · Cited by 59 · see citing works
Self-citationA Unified End-to-End Retriever-Reader Framework for Knowledge-based VQA
Guo, Y., Nie, L., Wong, Y., Liu, Y., Cheng, Z., & Kankanhalli, M. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3547870 · Cited by 44 · see citing works
Dynamic Visual Reasoning by Learning Differentiable Physics Models from Video and LanguagePreprint
Ding, M., Chen, Z., Du, T., Luo, P., Tenenbaum, J. B., & Gan, C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.15358 · Cited by 38 · see citing works
Self-citationWebly Supervised Concept Expansion for General Purpose Vision Models
Kamath, A., Clark, C., Gupta, T., Kolve, E., Hoiem, D., & Kembhavi, A. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20059-5_38 · Cited by 35 · see citing works
Image Scene Graph Generation (SGG) BenchmarkPreprint
Han, X., Yang, J., Hu, H., Zhang, L., Gao, J., & Zhang, P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.12604 · Cited by 28 · see citing works
Compositional Physical Reasoning of Objects and Events From Videos
Chen, Z., Dong, S., Yi, K., Li, Y., Ding, M., Torralba, A. B., Tenenbaum, J. B., & Gan, C. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3574322 · Cited by 27 · see citing works
Self-citationAn Embodied Dialogue System with Personality and Emotions
Konstantopoulos, S. (2010) · Cited by 12 · see citing works
Ding, M., Xu, Y., Chen, Z., Cox, D., Luo, P., Tenenbaum, J. B., & Gan, C. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2304.03767 · Cited by 11 · see citing works
Self-citationTextPSG: Panoptic Scene Graph Generation from Textual Descriptions
Zhao, C., Shen, Y., Chen, Z., Ding, M., & Gan, C. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00266 · Cited by 8 · see citing works
Self-citationOPT: Open Pre-trained Transformer Language Models
Zhang, S., Roller, S., Goyal, N., Artetxe, M., Chen, M., Chen, S., Dewan, C., Diab, M., Li, X., Lin, X. V., et al. (2022) · arXiv · doi:10.48550/arxiv.2205.01068
Flagged first, then by citations
3 of the 70 references couldn’t be matched to a known paper.
Research works printed but not linked: 2
- Antol, S.; Agrawal, A.; Lu, J.; Mitchell, M.; Batra, D.; Zit- nick, C. L.; and Parikh, D. 2015. Vqa: Visual question an- swering. In ICCV.
- Dou, Z.-Y.; Kamath, A.; Gan, Z.; Zhang, P.; Wang, J.; Li, L.; Liu, Z.; Liu, C.; LeCun, Y.; Peng, N.; et al. 2022. Coarse-to- fine vision-language pre-training with fusion in the backbone. arXiv.
Cited by
Sorted by citations
Hong, R., Lang, J., Xu, J., Cheng, Z., Zhong, T., & Zhou, F. (2025) · International World Wide Web Conference (WWW) · doi:10.1145/3696410.3714559 · Cited by 18 · see citing works
Spatial transcriptomics AI agent charts hPSC-pancreas maturation in vivoPreprint
Lin, Z.-W., Wang, W., Marin-Llobet, A., Li, Q., Pollock, S. D., Sui, X., Aljović, A., Lee, J., Baek, J., Liang, N., et al. (2025) · bioRxiv · doi:10.1101/2025.04.01.646731 · Cited by 22 · see citing works
APPLICATIONS OF MULTIMODAL GENERATIVE ARTIFICIAL INTELLIGENCE IN A REAL-WORLD RETINA CLINIC SETTING
Ghalibafan, S., Gonzalez, D. J. T., CAI, L. Z., Chou, B. G., Panneerselvam, S., Barrett, S. C., Djulbegovic, M. B., & Yannuzzi, N. A. (2024) · Retina · doi:10.1097/iae.0000000000004204 · Cited by 14 · see citing works
Ishaq, A., Lahoud, J., More, K., Thawakar, O., Thawkar, R., Dissanayake, D., Ahsan, N., Li, Y., Khan, F. S., Cholakkal, H., et al. (2025) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros60139.2025.11247360 · Cited by 14 · see citing works
Scenario-Aware Multimodal Chain-of-Thought Prompting for Rationales of VideoSocial Relations
Qin, P., Xu, T., Zhang, C., Wang, H., Hu, Y., & Chen, E. (2025) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2025.3571422 · Cited by 5 · see citing works
Comprehensive Survey on Prompts Generating via Knowledge-Guided Chain-of-Thought
Jia, Z., Geng, S., Zhao, Y., & Zhang, H. (2025) · International Journal of Crowd Science · doi:10.26599/ijcs.2024.9100038 · Cited by 5 · see citing works
Chain-of-Situation Aware Progressive Inference Learning
Liu, Y., Liu, F., Jiao, L., Bao, Q., Li, S., Li, L., Liu, X., Chen, P., & Ma, W. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2025.3576486 · Cited by 5 · see citing works
Harnessing Vision-Language Models for Improved Detection and Analysis of Harmful Algal Blooms (HAB)
Ansary, J., & Gupta, A. R. (2025) · ACS ES&T Water · doi:10.1021/acsestwater.5c00207 · Cited by 3 · see citing works
Wang, J., Zhuang, W., Wu, X., Chen, C., Bao, J., & Li, X. (2026) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2026.02.001 · Cited by 3 · see citing works
Yang, T., Feng, S., Jiang, Z., Zhang, Z., & Li, F. (2026) · Advanced Engineering Informatics · doi:10.1016/j.aei.2026.104628 · Cited by 3 · see citing works
Multi-Chain of Thought Prompt Learning for Aspect-Based Sentiment Analysis
He, Y., He, Z., Gu, T., Gu, B., Wan, Y., & Li, M. (2025) · Applied Sciences · doi:10.3390/app152212225 · Cited by 3 · see citing works
Autonomous Multimodal Reasoning via Implicit Chain-of-Vision
Huang, Y., Qi, H., Chen, Z., Zhang, H., Yu, H., & Zhao, Z. (2025) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw67362.2025.00279 · Cited by 2 · see citing works
Self-citationData-Driven Vision-Language Models for Remote Sensing: A survey
Zhou, Y., Zhao, S., Yang, X., Li, R., Zhang, T., Lan, M., Chen, C., Ma, L., He, H., & Li, K. (2026) · IEEE Geoscience and Remote Sensing Magazine · doi:10.1109/mgrs.2026.3696441 · Cited by 2 · see citing works
An agentic vision-action framework for generative 3D architectural modeling from sketches
Zhong, X., Meng, X., Li, Y., Fricker, P., Liang, J., & Koh, I. (2025) · International Journal of Architectural Computing · doi:10.1177/14780771251352950 · Cited by 2 · see citing works
AFSPrompt: An Axiomatic Fuzzy Set Prompt Pipeline for Knowledge-Based VQA
Wang, Z., Lang, Q., Liu, X., & Zhang, H. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2025.3573267 · Cited by 1 · see citing works
Ryu, H. S., Ju, U., & Wallraven, C. (2025) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2025.3616811 · Cited by 1 · see citing works
Neuro-Symbolic Reasoning with Multiple Large Language Models Combined by First-Order Logic
Kim, J., & Cho, S. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-08462-0_18 · Cited by 1 · see citing works
Ye, G., Zhao, H., Shen, Y., Li, J., Li, F., Jiang, Z., & Li, K. (2026) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2026.3676204 · Cited by 1 · see citing works
Self-citationMedical Cross-Modal Prompt Hashing with Robust Noisy Correspondence Learning
Liu, Y., Wu, Z., Chen, B., Zhang, Z., & Lu, G. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72384-1_24 · Cited by 1 · see citing works
Discriminative Representation Learning for Remote Sensing Visual Question Answering
Lyu, Y., Han, Y., Liu, Y., & Chen, H. (2025) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3777473 · Cited by 1 · see citing works
ACTIVE-o3: Empowering MLLMs with Active Perception via Pure Reinforcement Learning
Zhu, M., Zhong, H., Zhao, C., Du, Z., Liu, M., Huang, Z., Li, A., Chen, H., Zou, C., Chen, J., et al. (2025) · arXiv · doi:10.48550/arxiv.2505.21457
Scene-guided Attention Network for Spatial Understanding in 3D Scenes
Jiang, Y.-Q., Zhang, J.-W., Lin, C., Yu, Y., & Yang, Z. (2025) · International Conference on Multimedia Retrieval · doi:10.1145/3731715.3733426 · Cited by 1 · see citing works
Xie, J., & Yang, J. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-9866-0_22
Prompting Large Models for Knowledge and Reasoning Augmentation in KB-VQA
Liu, Q., Ying, M., Xiao, P., Li, G., & Yuan, X. (2025) · Lecture notes in computer science · doi:10.1007/978-981-95-0014-7_27
Chat2Edit: A Prompt-Based Image Editor with Live Feedback and Parameter Recommendation
Le, T. M., Dinh, P.-D. D., Che, Q.-H., Nguyen, D.-V., Nguyen, V.-T., Nguyen, T., Le, T.-N., & Tran, M. (2025) · Communications in computer and information science · doi:10.1007/978-3-032-10202-7_14
STARS: Semantics-Aware Text-guided Aerial Image Refinement and Synthesis
Townsell, D., Chen, L., Xie, M., Pan, C., & Zhang, W. (2025) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2025.104561
Visual Chain-of-Thought Prompting for Knowledge-Based Visual Reasoning
Association for Artificial Intelligence 2024, Chen, Z., Gan, C., Gutfreund, D., Hong, Y., Shen, Y., Sun, Z., & Zhou, Q. (2024) · Underline Science Inc. · doi:10.48448/g1y1-re64
Self-citationRationale-Based Video Caption Generation for Risky Factors in Driving Scenes
Zhang, H., Sasano, R., & TAKEDA, K. (2025) · IEICE Transactions on Information and Systems · doi:10.1587/transinf.2025edp7067
Ulhaq, A., Ali, K., Hassan, J. A., Javed, S., & Hardman, L. (2025) · International Conference on Digital Image Computing: Techniques and Applications (DICTA) · doi:10.1109/dicta68720.2025.11302432
MMToT: Multi-Modal Token-of-Thought Reasoning for Large Models
Xu, N., Lu, Z., Tian, H., Zheng, B., Cao, J., & Liu, A.-A. (2026) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2026.3654463
Huang, J., Wu, D., Zhu, H., Zhu, L., Xing, J., Wang, X., & Chen, B. (2026) · Lecture notes in computer science · doi:10.1007/978-981-95-5682-3_30
Multi-Modal Refined Prompting for Advancing Knowledge-Based Visual Question Answering
Zhu, L., Ying, M., Zhang, C., Liu, D., Wu, L. Y., Zhang, S., & Li, X. (2026) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2026.3660119
Ro, J., Kim, N., & Yoon, Y. (2025) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw69036.2025.00677
CoT-Pose: Chain-of-Thought Reasoning for 3D Pose Generation from Abstract Prompts
Cha, J., & Kim, J. (2025) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw69036.2025.00452
OPeMer: One-Shot LLM Prompting and Execution-Driven Multimodal Explainable Reasoning
Zhou, Z., Xue, D., & Qian, S. (2025) · IEEE International Conference on Knowledge Graph (ICKG) · doi:10.1109/ickg66886.2025.00059
LearnPhy: Progressive Hints to Incentivize Model Learning for Olympiad Physics
Li, H., Zou, Y., Lin, H., Qian, S., & Zhang, J. (2025) · doi:10.1145/3788149.3788207
Wang, W., Zhou, Z., Ma, Z., Chen, Y., Lin, Z., Sheng, H., Liu, P., Shao, W., Zhang, Q. E., & Qiao, Y. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37592-6_24
Zhao, H., Zhou, Z., & Ye, G. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11464318
Automated Knowledge Graph Construction From Unstructured Medical Data
Safi, F., Kacem, F., Jallouli, W., & Fathallah, H. (2026) · IEEE Middle East and North Africa Communications Conference (MENACOMM) · doi:10.1109/menacomm69507.2026.11532834
Can ChatGPT graduate as a civil engineer? Exploring the role of prompt design in model behaviour
Genç, O., Durgun, S., & Un, B. (2026) · Engineering Construction & Architectural Management · doi:10.1108/ecam-01-2026-0023
ReShift: Aha-Moment-Driven Reasoning-Level Backdoor Attacks on Vision–Language Models
Dou, Z., Zhao, Q., Gao, Z., & Biswas, S. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37627-5_9
Hu, Z., Yang, P., Yang, D.-L., & Liu, F. (2026) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3833389
Dense Vehicle Detection using Multi-Stage Prompting and Grid Overlay with Vision-Language Models
Choi, K. (2026) · The Journal of Korean Institute of Information Technology · doi:10.14801/jkiit.2026.24.7.149
Deng, J., Jiang, Y., Huang, W., Li, X., Sun, Q., & Zhuo, C. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-36846-1_22
Li, Y., Yang, Y., Yu, F., Hu, J., Fang, Z., & Huang, L. (2026) · International Conference on Big Data and Information Analytics (BigDIA) · doi:10.1109/bigdia71496.2026.11684568
Direct Preference Optimization for Perceptual Alignment via Vision-Language Consistency
Lee, S., & Lee, D.-G. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37520-9_18
CoRT-Predictor: Chain of Risk Thought Autoregressive Trajectory Predictor for Autonomous Driving
Jiang, Y., Liu, Y., & Liu, M. (2026) · doi:10.1109/cvprf71909.2026.00112
Yong, C., Li, L., Xu, C., Huang, J., & Shen, F. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7590072
Pistofidis, P., Petrou, A., & Botsaris, P. N. (2026) · Technologies · doi:10.3390/technologies14100666
- Authors
- Chen, Z., Zhou, Q., Shen, Y., Hong, Y., Sun, Z., Gutfreund, D., & Gan, C.
- Affiliations
- IBM (United States) (US)
- University of Massachusetts Amherst (US)
- University of California, Los Angeles (US)
- Carnegie Mellon University (US)
- Venue
- Proceedings of the AAAI Conference on Artificial Intelligence
- Published
- 2024-03-24
- Volume
- 38
- Issue
- 2
- Pages
- 1254-1262
- ISSN
- 2159-5399 (ISSN-L)
- 2374-3468
- OpenAlex
- W4393149666
- Cited by
- 48
- References
- 70
- Citations by year
- 2026: 20 · 2025: 25 · 2024: 3
- FWCI
- 9.88
- Type
- conference paper
- Language
- en
- Open access
- diamond
- doi.org
- Locations
- Proceedings of the AAAI Conference on Artificial Intelligence (publishedVersion)
- AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) (submittedVersion)
- Funders
- Cisco Systems
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Logic, Reasoning, and Knowledge
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- chain-of-thought prompting, large language models, visual captioning, rationale generation, visual grounding, few-shot learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
