KAT: A Knowledge Augmented Transformer for Vision-and-Language

Deep Residual Learning for Image Recognition

He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474

Attention Is All You Need

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404

Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks

Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472

Language Models are Few-Shot Learners

Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372

Learning Transferable Visual Models From Natural Language Supervision

Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293

Decoupled Weight Decay Regularization

Loshchilov, I., & Hutter, F. (2017) · arXiv · doi:10.48550/arxiv.1711.05101 · Cited by 12,522

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929

BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension

Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.703 · Cited by 8,821

Billion-scale similarity search with GPUs

Johnson, J., Douze, M., & Jegou, H. (2021) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2019.2921572 · Cited by 4,228

Dense passage retrieval for open-domain question answering

Karpukhin, V., Oguz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., & Yih, W.-T. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.550 · Cited by 3,575

Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-T., Rocktäschel, T., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.11401 · Cited by 3,101

DBpedia: A Nucleus for a Web of Open Data

Auer, S., Bizer, C., Kobilarov, G., Lehmann, J., Cyganiak, R., & Ives, Z. (2007) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-76298-0_52 · Cited by 2,454

LXMERT: Learning Cross-Modality Encoder Representations from Transformers

Tan, H., & Bansal, M. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1514 · Cited by 2,319

ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks

Lu, J., Batra, D., Parikh, D., & Lee, S. (2019) · arXiv · doi:10.48550/arxiv.1908.02265 · Cited by 2,109

Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision

Jia, C., Yang, Y., Xia, Y., Chen, Y.-T., Parekh, Z., Pham, H., Le, Q. V., Sung, Y., Li, Z., & Duerig, T. (2021) · arXiv · doi:10.48550/arxiv.2102.05918 · Cited by 1,977

Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks

Li, X., Yin, X. P., Li, C., Zhang, P., Hu, X., Zhang, L., Wang, L., Hu, H., Li, D., Wei, F., et al. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58577-8_8 · Cited by 1,791

ConceptNet — A Practical Commonsense Reasoning Tool-Kit

Liu, H., & Singh, P. (2004) · BT Technology Journal · doi:10.1023/b:bttj.0000047600.45421.6d · Cited by 1,570

Reading Wikipedia to answer open-domain questions

Chen, D., Fisch, A., Weston, J., & Bordes, A. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1171 · Cited by 1,546

VisualBERT: A Simple and Performant Baseline for Vision and Language

Li, L. H., Yatskar, M., Yin, D., Hsieh, C.-J., & Chang, K.-W. (2019) · arXiv · doi:10.48550/arxiv.1908.03557 · Cited by 1,198

Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering

Izacard, G., & Grave, E. (2021) · Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume · doi:10.18653/v1/2021.eacl-main.74 · Cited by 1,109

VL-BERT: Pre-training of Generic Visual-Linguistic Representations

Su, W., Zhu, X., Cao, Y., Li, B., Lu, L., Wei, F., & Dai, J. (2019) · arXiv · doi:10.48550/arxiv.1908.08530 · Cited by 950

OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledge

Marino, K., Rastegari, M., Farhadi, A., & Mottaghi, R. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00331 · Cited by 845

Unicoder-VL: A Universal Encoder for Vision and Language by Cross-Modal Pre-Training

Li, G., Duan, N., Fang, Y., Gong, M., & Jiang, D. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6795 · Cited by 795

Latent retrieval for weakly supervised open domain question answering

Lee, K., Chang, M.-W., & Toutanova, K. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1612 · Cited by 771

VQA: Visual Question Answering: www.visualqa.org

Agrawal, A., Lu, J., Antol, S., Mitchell, M., Zitnick, C. L., Parikh, D., & Batra, D. (2017) · International Journal of Computer Vision · doi:10.1007/s11263-016-0966-6 · Cited by 570

FVQA: Fact-Based Visual Question Answering

Wang, P., Wu, Q., Shen, C., Dick, A., & van den Hengel, A. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2754246 · Cited by 507

An Empirical Study of GPT-3 for Few-Shot Knowledge-Based VQA

Yang, Z., Gan, Z., Wang, J., Hu, X., Lu, Y., Liu, Z., & Wang, L. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i3.20215 · Cited by 375

Explicit Knowledge-based Reasoning for Visual Question Answering

Wang, P., Wu, Q., Shen, C., Dick, A., & van den Hengel, A. (2017) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/179 · Cited by 296

KRISP: Integrating Implicit and Symbolic Knowledge for Open-Domain Knowledge-Based VQA

Marino, K., Chen, X., Parikh, D., Gupta, A., & Rohrbach, M. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01389 · Cited by 211

Symbolic Knowledge Distillation: from General Language Models to Commonsense Models

West, P., Bhagavatula, C., Hessel, J., Hwang, J. D., Jiang, L., Bras, R. J. L., Lu, X., Welleck, S., & Choi, Y. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.341 · Cited by 209

Out of the Box: Reasoning with Graph Convolution Nets for Factual Visual Question AnsweringPreprint

Narasimhan, M., Lazebnik, S., & Schwing, A. G. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.00538 · Cited by 190

ConceptBert: Concept-Aware Representation for Visual Question Answering

Gardères, F., Ziaeefard, M., Abeloos, B., & Lécué, F. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.44 · Cited by 156

Multi-Modal Answer Validation for Knowledge-Based VQA

Wu, J., Lu, J., Sabharwal, A., & Mottaghi, R. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i3.20174 · Cited by 153

Boosting Visual Question Answering with Context-aware Knowledge Aggregation

Li, G., Wang, X. Y., & Zhu, W. (2020) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3394171.3413943 · Cited by 89

Weakly-Supervised Visual-Retriever-Reader for Knowledge-based Question Answering

Luo, M., Zeng, Y., Banerjee, P., & Baral, C. R. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.517 · Cited by 74

KB-VLP: Knowledge Based Vision and Language Pretraining

Chen, K., Huang, Q., Bisk, Y., McDuff, D., & Gao, J. (2021) · International Conference on Machine Learning · Cited by 6

NICE: Neural Image Commenting with Empathy

Chen, K., Huang, Q., McDuff, D., Gao, X., Palangi, H., Wang, J., Forbus, K. D., & Gao, J. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.380 · Cited by 5

Learning Inference Rules with Neural TP-Reasoner

Chen, K., Huang, Q., Smolensky, P., Forbus, K. D., & Gao, J. (2020) · Cited by 1

Flamingo: a visual language model for few-shot learning

Alayrac, J.-B., Donahue, J., Luc, P., Miech, A., Barr, I., Hasson, Y., Lenc, K., Mensch, A., Millican, K., Reynolds, M., et al. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1723 · Cited by 966

A Survey of GPT-3 Family Large Language Models Including ChatGPT and GPT-4

Kalyan, K. S. (2024) · Natural Language Processing Journal · doi:10.1016/j.nlp.2023.100048 · Cited by 335

ViperGPT: Visual Inference via Python Execution for Reasoning

Surís, D., Menon, S., & Vondrick, C. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01092 · Cited by 273

Prompting Large Language Models with Answer Heuristics for Knowledge-Based Visual Question Answering

Shao, Z., Zhou, Y., Wang, M., & Yu, J. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01438 · Cited by 240

Vision-Language Pre-Training: Basics, Recent Advances, and Future Trends

Gan, Z., Li, L., Li, C., Wang, L., Liu, Z., & Gao, J. (2022) · Foundations and Trends® in Computer Graphics and Vision · doi:10.1561/0600000105 · Cited by 155

Imagination-Augmented Natural Language Understanding

Lu, Y., Zhu, W., Wang, X. E., Eckstein, M. P., & Wang, W. Y. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.326 · Cited by 21

KAFA: Rethinking Image Ad Understanding with Knowledge-Augmented Feature Adaptation of Vision-Language Models

Jia, Z., Narayana, P., Akula, A., Pruthi, G., Su, H., Basu, S., & Jampani, V. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-industry.74 · Cited by 8

ReAugKD: Retrieval-Augmented Knowledge Distillation For Pre-trained Language Models

Zhang, J., Muhamed, A., Anantharaman, A., Wang, G., Chen, C., Zhong, K., Cui, Q., Xu, Y., Zeng, B., Chilimbi, T. M., et al. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-short.97 · Cited by 17

QAVA: Query-Agnostic Visual Attack to Large Vision-Language Models

Zhang, Y., Xie, R., Chen, J., Sun, X., Kang, Z., & Wang, Y. (2025) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2025.naacl-long.512 · Cited by 1

Advancement in Graph Understanding: A Multimodal Benchmark and Fine-Tuning of Vision-Language Models

Ai, Q., Li, J., Dai, J., Zhou, J., Liu, L., Jiang, H., & Shi, S. (2024) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2024.acl-long.404 · Cited by 2