The DOI matches its Crossref record; 118 of the 123 references it cites were found in a registry or an index.
Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations
Open accessAbstract
Despite progress in perceptual tasks such as image classification, computers still perform poorly on cognitive tasks such as image description and question answering. Cognition is core to tasks that involve not just recognizing, but reasoning about our visual world. However, models used to tackle the rich content in images for cognitive tasks are still being trained using the same datasets designed for perceptual tasks. To achieve success at cognitive tasks, models need to understand the interactions and relationships between objects in an image. When asked “What vehicle is the person riding?”, computers will need to identify the objects in an image as well as the relationships riding(man, carriage) and pulling(horse, carriage) to answer correctly that “the person is riding a horse-drawn carriage.” In this paper, we present the Visual Genome dataset to enable the modeling of such relationships. We collect dense annotations of objects, attributes, and relationships within each image to learn these models. Specifically, our dataset contains over 108K images where each image has an average of $$35$$ objects, $$26$$ attributes, and $$21$$ pairwise relationships between objects. We canonicalize the objects, attributes, relationships, and noun phrases in region descriptions and questions answer pairs to WordNet synsets. Together, these annotations represent the densest and largest dataset of image descriptions, objects, attributes, relationships, and question answer pairs.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“For a few of these applications, some datasets were used like SQuAD, MS-MARCO and the like. Some works curated their own dataset owing to the distinctive application involved [122, 124].”
From Automatic question generation: a review of methodologies, datasets, evaluation metrics, and applications (2023) · cites this paper as [122]
In the citing paper
“Most datasets that are currently available for training question generation systems are crowd-sourced [42, 47, 84, 92, 114, 115, 119, 120, 122, 123], which largely impacts the quality of generated questions.”
From Automatic question generation: a review of methodologies, datasets, evaluation metrics, and applications (2023) · cites this paper as [122]
In the citing paper
“Also, most works used factoid-based questions but Yes/No [133], MCQ type [124] and reasoning type questions [122] were also generated for some applications.”
From Automatic question generation: a review of methodologies, datasets, evaluation metrics, and applications (2023) · cites this paper as [122]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 1×
“To achieve success at cognitive tasks, models need to understand the interactions and relationships between objects in Ranjay Krishna Stanford University, Stanford, CA, USA E-mail: ranjaykrishna@cs.stanford.edu Yuke Zhu Stanford University, Stanford, CA, USA Oliver Groth Dresden University of…”
MethodImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 1×
“To achieve success at cognitive tasks, models need to understand the interactions and relationships between objects in Ranjay Krishna Stanford University, Stanford, CA, USA E-mail: ranjaykrishna@cs.stanford.edu Yuke Zhu Stanford University, Stanford, CA, USA Oliver Groth Dresden University of…”
MethodHochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 1×
“We then use the NeuralTalk (Karpathy and Fei-Fei, 2014) model to train a long short-term memory (LSTM) network (Hochreiter and Schmidhuber, 1997) to generate descriptions of regions.”
MethodFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Self-citationGoing deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Self-citationRich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
Research works printed but not linked: 1
- Hayes, P. J. (1985). The second naive physics manifesto. Theories of the commonsense world (pp. 1–36).
Cited by
Sorted by citations
Items 701–800 of 1,032
Image captioning for effective use of language models in knowledge-based visual question answering
Salaberria, A., Azkune, G., de Lacalle, O. L., Soroa, A., & Agirre, E. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.118669 · Cited by 78
Object-aware Video-language Pre-training for Retrieval
Wang, A. J., Ge, Y., Cai, G., Yan, R., Lin, X., Shan, Y., Qie, X., & Shou, M. Z. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00331 · Cited by 69
In Defense of Scene Graphs for Image Captioning
Nguyen, K., Tripathi, S., Du, B., Guha, T., & Nguyen, T. Q. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00144
Pseudo-Q: Generating Pseudo Language Queries for Visual Grounding
Jiang, H., Lin, Y., Han, D., Song, S., & Huang, G. M. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01507 · Cited by 72
Frido: Feature Pyramid Diffusion for Complex Scene Image Synthesis
Fan, W.-C., Chen, Y., Chen, D., Yu, C., Yuan, L., & Wang, Y.-C. F. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i1.25133 · Cited by 80
Li, Y., Wang, L., Wang, T., Yang, X., Luo, J., Wang, Q., Deng, Y., Wang, W., Sun, X., Li, H., et al. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3508072 · Cited by 69
“Visual Genome (VG) [33] is a large-scale scene graph dataset, which consists of 108,077 images with tens of thousands of object and relationship categories.”
BackgroundTensorize, Factorize and Regularize: Robust Visual Relationship Learning
Hwang, S. J., Kim, H. J., Ravi, S. N., Collins, M. D., Tao, Z., & Singh, V. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00112 · Cited by 68
Shuffle-Then-Assemble: Learning Object-Agnostic Visual Relationship Features
Yang, X., Zhang, H., & Cai, J. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01258-8_3 · Cited by 69
Multimodal feature fusion by relational reasoning and attention for visual question answering
Zhang, W., Yu, J., Hu, H., Hu, H., & Qin, Z. (2019) · Information Fusion · doi:10.1016/j.inffus.2019.08.009 · Cited by 70
ACMM: Aligned Cross-Modal Memory for Few-Shot Image and Sentence Matching
Huang, Y., & Wang, L. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00587 · Cited by 71
Length-Controllable Image Captioning
Deng, C., Ding, N., Tan, M., & Wu, Q. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58601-0_42 · Cited by 70
Learning Audio-Video Modalities from Image Captions
Nagrani, A., Seo, P. H., Seybold, B., Hauth, A., Manén, S., Sun, C., & Schmid, C. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19781-9_24 · Cited by 82
Multimodality Representation Learning: A Survey on Evolution, Pretraining and Its Applications
Manzoor, M. A., Albarri, S., Xian, Z., Meng, Z., Nakov, P., & Liang, S. (2023) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3617833 · Cited by 86
Knowledge Graphs for Multi-modal Learning: Survey and Perspective
Chen, Z., Zhang, Y., Fang, Y., Geng, Y., Guo, L., Chen, J., Liu, X., Pan, J. Z., Zhang, N., Chen, H., et al. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103124 · Cited by 79
Transferable Interactiveness Knowledge for Human-Object Interaction Detection
Li, Y., Liu, X., Wu, X., Huang, X., Xu, L., & Lu, C. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3054048 · Cited by 74
FuseCap: Leveraging Large Language Models for Enriched Fused Image Captions
Rotstein, N., Bensaïd, D., Brody, S., Ganz, R., & Kimmel, R. (2024) · 2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv57701.2024.00559 · Cited by 66
“This model is initially pre-trained on several detection datasets and then fine-tuned on the Visual Genome (VG) dataset [33].”
MethodJoint Visual Semantic Reasoning: Multi-Stage Decoder for Text Recognition
Bhunia, A. K., Sain, A., Kumar, A., Ghose, S., Chowdhury, P. N., & Song, Y.-Z. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01467 · Cited by 67
Enabling Multimodal Generation on CLIP via Vision-Language Knowledge Distillation
Dai, W., Hou, L., Shang, L., Jiang, X., Liu, Q., & Fung, P. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.187 · Cited by 107
Local self-attention in transformer for visual question answering
Shen, X., Han, D., Guo, Z., Chen, C., Hua, J., & Luo, G. (2022) · Applied Intelligence · doi:10.1007/s10489-022-04355-w · Cited by 66
Textual Context-Aware Dense Captioning With Diverse Words
Shao, Z., Han, J., Debattista, K., & Pang, Y. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3241517 · Cited by 69
Reale-Nosei, G., Amador-Domínguez, E., & Serrano, E. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103264 · Cited by 72
Osprey: Pixel Understanding with Visual Instruction Tuning
Yuan, Y., Li, W., Liu, J., Tang, D., Luo, X., Qin, C., Zhang, L., & Zhu, J. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02664 · Cited by 109
“Besides, Visual Genome (VG) [20] and Visual Commonsense Reasoning (VCR) [52] datasets are employed to add more multiple region understanding data.”
MethodInterpretable Visual Question Answering by Visual Grounding From Attention Supervision Mining
Zhang, Y., Niebles, J. C., & Soto, Á. (2019) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2019.00043 · Cited by 70
Diverse and Coherent Paragraph Generation from Images
Chatterjee, M., & Schwing, A. G. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01216-8_45 · Cited by 67
Making History Matter: History-Advantage Sequence Training for Visual Dialog
Yang, T., Zha, Z.-J., & Zhang, H. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00265 · Cited by 73
Semantics-Preserving Graph Propagation for Zero-Shot Object Detection
Yan, C., Zheng, Q., Chang, X., Luo, M., Yeh, C., & Hauptman, A. G. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.3011807 · Cited by 69
Auto-encoding and Distilling Scene Graphs for Image Captioning
Yang, X., Zhang, H., & Cai, J. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3042192 · Cited by 68
Multimodal feature-wise co-attention method for visual question answering
Zhang, S., Chen, M., Chen, J., Zou, F., Li, Y.-F., & Lü, P. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.02.022 · Cited by 68
Unshuffling Data for Improved Generalization in Visual Question Answering
Teney, D., Abbasnejad, E., & van den Hengel, A. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00145 · Cited by 122
SOLVER: Scene-Object Interrelated Visual Emotion Reasoning Network
Yang, J., Gao, X., Li, L., Xiu, W., & Ding, J. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3118983 · Cited by 72
Self-citationAI-Based Detection, Classification and Prediction/Prognosis in Medical Imaging
Yousefirizi, F., Decazes, P., Amyar, A., Ruan, S., Saboury, B., & Rahmim, A. (2021) · PET Clinics · doi:10.1016/j.cpet.2021.09.010 · Cited by 76
Survey on Deep Learning-Based Marine Object Detection
Zhang, R., Li, S., Ji, G., Zhao, X., Li, J., & Pan, M. (2021) · Journal of Advanced Transportation · doi:10.1155/2021/5808206 · Cited by 71
Towards Lightweight Transformer Via Group-Wise Transformation for Vision-and-Language Tasks
Luo, G., Zhou, Y., Sun, X., Wang, Y., Cao, L., Wu, Y., Huang, F., & Ji, R. (2022) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3139234 · Cited by 67
Scene Graph Refinement Network for Visual Question Answering
Qian, T., Chen, J., Chen, S., Wu, B., & Jiang, Y. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3169065 · Cited by 69
Self-citationA Broad Study of Pre-training for Domain Generalization and Adaptation
Kim, D., Wang, K., Sclaroff, S., & Saenko, K. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19827-4_36 · Cited by 77
Interacting-Enhancing Feature Transformer for Cross-Modal Remote-Sensing Image and Text Retrieval
Tang, X., Wang, Y., Ma, J., Zhang, X., Liu, F., & Jiao, L. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3280546 · Cited by 71
Language-only Efficient Training of Zero-shot Composed Image Retrieval
Gu, G., Chun, S., Kim, W., Kang, Y., & Yun, S. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01256 · Cited by 85
“The attribute tasks are built upon VisualGenome [23] with in-the-wild visual attributes from VAW [34].”
BackgroundAutomatic Arabic Image Captioning using RNN-LSTM-Based Language Model and CNN
Almuzaini, H. A., Tasniem, N., & Benhidour, H. (2018) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2018.090610 · Cited by 69
Aspect-Aware Multimodal Summarization for Chinese E-Commerce Products
Li, H., Yuan, P., Xu, S. H., Wu, Y., He, X., & Zhou, B. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6332 · Cited by 65
Integrating Part of Speech Guidance for Image Captioning
Zhang, J., Mei, K., Zheng, Y., & Jianping Fan, B. (2020) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2020.2976552 · Cited by 65
Exploiting Edge-Oriented Reasoning for 3D Point-based Scene Graph Analysis
Zhang, C., Yu, J., Song, Y., & Cai, W. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00958 · Cited by 74
Screen Parsing: Towards Reverse Engineering of UI Models from Screenshots
Wu, J., Zhang, X., Nichols, J., & Bigham, J. P. (2021) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3472749.3474763 · Cited by 80
Double-Stream Position Learning Transformer Network for Image Captioning
Jiang, W., Zhou, W., & Hu, H. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3181490 · Cited by 66
Multi-modal Alignment using Representation Codebook
Duan, J., Chen, L., Tran, S., Yang, J., Xu, Y., Zeng, B., & Chilimbi, T. M. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01520 · Cited by 71
Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Survey
Kuang, J., Shen, Y., Xie, J., Luo, H., Xu, Z., Li, R., Li, Y., Cheng, X., Lin, X., & Han, Y. (2025) · ACM Computing Surveys · doi:10.1145/3711680 · Cited by 84
Cartwright, M., Seals, A., Salamon, J., Williams, A. C., Mikloska, S., MacConnell, D., Law, E., Bello, J. P., & Nov, O. (2017) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3134664 · Cited by 69
Unifying neural learning and symbolic reasoning for spinal medical report generation
Han, Z., Wei, B., Xi, X., Chen, B., Yin, Y., & Li, S. (2020) · Medical Image Analysis · doi:10.1016/j.media.2020.101872 · Cited by 69
Graph-to-3D: End-to-End Generation and Manipulation of 3D Scenes Using Scene Graphs
Dhamo, H., Manhardt, F., Navab, N., & Tombari, F. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01604 · Cited by 64
Weakly-Supervised Visual-Retriever-Reader for Knowledge-based Question Answering
Luo, M., Zeng, Y., Banerjee, P., & Baral, C. R. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.517 · Cited by 74
Chandrakala, S., Deepak, K., & Revathy, G. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-022-10258-6 · Cited by 64
VindLU: A Recipe for Effective Video-and-Language Pretraining
Cheng, F., Wang, X., Lei, J., Crandall, D., Bansal, M., & Bertasius, G. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01034 · Cited by 76
Computer Vision and Natural Language Processing
Wiriyathammabhum, P., Summers-Stay, D. A., Fermuller, C. M., & Aloimonos, Y. (2016) · ACM Computing Surveys · doi:10.1145/3009906 · Cited by 69
A hierarchical and regional deep learning architecture for image description generation
Kinghorn, P., Zhang, L., & Shao, L. (2017) · Pattern Recognition Letters · doi:10.1016/j.patrec.2017.09.013 · Cited by 65
Natural Language Guided Visual Relationship Detection
Liao, W., Rosenhahn, B., Ling, S., & Yang, M. Y. (2019) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2019.00058 · Cited by 70
Learning Visual Commonsense for Robust Scene Graph Generation
Zareian, A., Wang, Z., You, H., & Chang, S. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58592-1_38 · Cited by 63
Socially situated artificial intelligence enables learning from human interaction
Krishna, R., Lee, D., Fei-Fei, L., & Bernstein, M. S. (2022) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2115730119 · Cited by 73
Self-citationDFMKE: A dual fusion multi-modal knowledge graph embedding framework for entity alignment
Zhu, J., Huang, C., & Meo, P. D. (2022) · Information Fusion · doi:10.1016/j.inffus.2022.09.012 · Cited by 67
Rethinking Data Augmentation for Robust Visual Question Answering
Chen, L., Zheng, Y., & Xiao, J. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20059-5_6 · Cited by 65
ProposalCLIP: Unsupervised Open-Category Object Proposal Generation via Exploiting CLIP Cues
Shi, H., Hayat, M., Wu, Y., & Cai, J. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00939 · Cited by 63
Scene Graph Prediction with Limited Labels
Chen, V. S., Varma, P., Krishna, R., Bernstein, M. S., Ré, C., & Fei-Fei, L. (2019) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2019.00220 · Cited by 64
Self-citationShow, Recall, and Tell: Image Captioning with Recall Mechanism
Wang, L., Bai, Z., Zhang, Y., & Lu, H. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6898 · Cited by 61
Long-Term Video Question Answering via Multimodal Hierarchical Memory Attentive Networks
Yu, T., Yu, J., Zhou, Y., Huang, Q., & Tian, Q. C. (2020) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2020.2995959 · Cited by 63
Violin: A Large-Scale Dataset for Video-and-Language Inference
Liu, J., Chen, W., Cheng, Y., Gan, Z., Yu, L., Yang, Y., & Liu, J. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01091 · Cited by 69
Ravichandran, Z., Peng, L., Hughes, N., Griffith, J., & Carlone, L. (2022) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra46639.2022.9812179 · Cited by 63
Li, J., Niu, L., & Zhang, L. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.02059 · Cited by 73
Pragmatically Informative Image Captioning with Character-Level Inference
Cohn-Gordon, R., Goodman, N. D., & Potts, C. (2018) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n18-2070 · Cited by 62
Unsupervised Cross-Media Retrieval Using Domain Adaptation With Scene Graph
Peng, Y.-X., & Chi, J. (2020) · IEEE transactions on circuits and systems for video technology (Print) · doi:10.1109/tcsvt.2019.2953692 · Cited by 55
Context-based information generation for managing UAV-acquired data using image captioning
Bang, S., & Kim, H. (2020) · Automation in Construction · doi:10.1016/j.autcon.2020.103116 · Cited by 62
Rank2Tell: A Multimodal Driving Dataset for Joint Importance Ranking and Reasoning
Sachdeva, E., Agarwal, N., Chundi, S., Roelofs, S., Li, J., Kochenderfer, M., Choi, C., & Dariush, B. (2024) · 2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv57701.2024.00734 · Cited by 63
“Some dense visual captioning methods aim to generate a caption for multiple regions in the same image [16,42,44] using the Visual Genome dataset [20].”
MethodA multimodal fusion approach for image captioning
Zhao, D., Chang, Z., & Guo, S. (2019) · Neurocomputing · doi:10.1016/j.neucom.2018.11.004 · Cited by 61
VQA-GNN: Reasoning with Multimodal Knowledge via Graph Neural Networks for Visual Question Answering
Wang, Y., Yasunaga, M., Ren, H., Wada, S., & Leskovec, J. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01973 · Cited by 68
Dual Graph Convolutional Networks with Transformer and Curriculum Learning for Image Captioning
Dong, X., Long, C., Xu, W., & Xiao, C. (2021) · Proceedings of the 29th ACM International Conference on Multimedia · doi:10.1145/3474085.3475439 · Cited by 74
Transformer Reasoning Network for Image- Text Matching and Retrieval
Messina, N., Falchi, F., Esuli, A., & Amato, G. M. (2021) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr48806.2021.9413172 · Cited by 73
Attend What You Need: Motion-Appearance Synergistic Networks for Video Question Answering
Seo, A., Kang, G.-C., Park, J., & Zhang, B. (2021) · doi:10.18653/v1/2021.acl-long.481 · Cited by 67
LAVIS: A One-stop Library for Language-Vision Intelligence
Li, D., Li, J., Le, H., Wang, G., Savarese, S., & Hoi, S. C. H. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-demo.3 · Cited by 68
KERM: Knowledge Enhanced Reasoning for Vision-and-Language Navigation
Li, X., Wang, Z., Yang, J., Wang, Y., & Jiang, S. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00254 · Cited by 70
“To obtain knowledge for view images, facts ( i.e., knowledge described by language descriptions) are retrieved from the knowledge base constructed on the Visual Genome dataset [16].”
MethodPasteGAN: A Semi-Parametric Method to Generate Image from Scene GraphPreprint
Li, Y., Ma, T., Bai, Y., Duan, N., Wei, S., & Wang, X. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.01608 · Cited by 61
Recent Advances and Trends in Multimodal Deep Learning: A ReviewPreprint
Jabeen, S., Li, X., Amin, M. S., Li, S., & Abdul, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.11087 · Cited by 116
Cops-Ref: A New Dataset and Task on Compositional Referring Expression Comprehension
Chen, Z., Wang, P., Ma, L., Wong, K.-Y. K., & Wu, Q. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01010 · Cited by 64
Polysemy Deciphering Network for Robust Human–Object Interaction Detection
Zhong, X., Ding, C., Qu, X., & Tao, D. (2021) · International Journal of Computer Vision · doi:10.1007/s11263-021-01458-8 · Cited by 59
Zero-Shot Instance Segmentation
Zheng, Y., Wu, J., Qin, Y., Zhang, F., & Cui, L. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00262 · Cited by 64
Reinforced Causal Explainer for Graph Neural Networks
Wang, X., Wu, Y., Zhang, A., Feng, F., He, X., & Chua, T. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3170302 · Cited by 67
Obj2Text: Generating Visually Descriptive Language from Object Layouts
Yin, X., & Ordóñez, V. (2017) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d17-1017 · Cited by 60
Intention Oriented Image Captions With Guiding Objects
Zheng, Y., Li, Y., & Wang, S. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00859 · Cited by 61
A survey of transformer-based multimodal pre-trained modals
Han, X., Wang, Y.-T., Feng, J.-L., Deng, C., Chen, Z.-H., Huang, Y.-A., Su, H., Hu, L., & Hu, P.-W. (2023) · Neurocomputing · doi:10.1016/j.neucom.2022.09.136 · Cited by 59
Deep Generative Probabilistic Graph Neural Networks for Scene Graph Generation
Khademi, M., & Schulte, O. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6783 · Cited by 59
StructFormer: Learning Spatial Structure for Language-Guided Semantic Rearrangement of Novel Objects
Liu, W., Paxton, C., Hermans, T., & Fox, D. (2022) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra46639.2022.9811931 · Cited by 59
Fei, H., Liu, Q., Zhang, M., Zhang, M., & Chua, T. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.329 · Cited by 62
MeaCap: Memory-Augmented Zero-shot Image Captioning
Zeng, Z., Xie, Y., Zhang, H., Chen, C., Chen, B., & Wang, Z. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01337 · Cited by 75
“Using well-annotated image-text pairs, supervised models [7, 17, 23, 34, 39, 41, 48, 49, 56] have achieved promising results on typical IC benchmarks [1, 25, 32, 63].”
MethodViTs as backbones: Leveraging vision transformers for feature extraction
Elharrouss, O., Himeur, Y., Mahmood, Y., Alrabaee, S., Ouamane, A., Bensaali, F., Bechqito, Y., & Chouchane, A. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.102951 · Cited by 69
Training for Diversity in Image Paragraph Captioning
Melas-Kyriazi, L., Rush, A. M., & Han, G. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1084 · Cited by 59
Explainable Video Action Reasoning via Prior Knowledge and State Transitions
Zhuo, T., Cheng, Z., Zhang, P., Wong, Y., & Kankanhalli, M. (2019) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3343031.3351040 · Cited by 59
M3P: Learning Universal Representations via Multitask Multilingual Multimodal Pre-training
Ni, M., Huang, H., Su, L., Cui, E., Bharti, T., Wang, L., Zhang, D., & Duan, N. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00397 · Cited by 57
Lee, J., Herskovitz, J., Peng, Y.-H., & Guo, A. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501966 · Cited by 61
Interpretable Visual Question Answering by Reasoning on Dependency Trees
Cao, Q., Liang, X., Li, B., & Lin, L. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2943456 · Cited by 58
Wu, J., Hu, W., Wen, Y., Tu, W., & Liu, X. (2020) · Sensors · doi:10.3390/s20247080 · Cited by 58
UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-training
Zhou, M., Zhou, L., Wang, S., Cheng, Y., Li, L., Zhou, Y., & Liu, J. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00414 · Cited by 74
Self-citationCoSSL: Co-Learning of Representation and Classifier for Imbalanced Semi-Supervised Learning
Fan, Y., Dai, D., Kukleva, A., & Schiele, B. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01417 · Cited by 63
NLX-GPT: A Model for Natural Language Explanations in Vision and Vision-Language Tasks
Sammani, F., Mukherjee, T., & Deligiannis, N. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00814 · Cited by 66
Contrastive Video Question Answering via Video Graph Transformer
Xiao, J., Zhou, P., Yao, A., Li, Y., Hong, R., Yan, S., & Chua, T. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3292266 · Cited by 73
Metadata
- Authors
- Krishna, R., Zhu, Y., Groth, O., Johnson, J., Hata, K., Kravitz, J., Chen, S., Kalantidis, Y., Li, L.-J., Shamma, D. A., Bernstein, M. S., & Fei-Fei, L.
- Affiliations
- Stanford University (US)
- Technische Universität Dresden (DE)
- Yahoo (United States) (US)
- Snap (United States) (US)
- Centrum Wiskunde & Informatica (NL)
- Corresponding author
- Krishna, Ranjay
- ORCID
- Venue
- International Journal of Computer Vision
- Published
- 2017-02-06
- Volume
- 123
- Issue
- 1
- Pages
- 32-73
- ISSN
- 0920-5691 (ISSN-L)
- 1573-1405
- OpenAlex
- W2277195237
- Cited by
- 5,107
- References
- 123
- Citations by year
- 2026: 313 · 2025: 508 · 2024: 690 · 2023: 761 · 2022: 671 · 2021: 738 · 2020: 586 · 2019: 411 · 2018: 230 · 2017: 149
- FWCI
- 156.12
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- International Journal of Computer Vision (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
- Centrum Wiskunde & Informatica (CWI), the national research institute for mathematics and computer science in the Netherlands (submittedVersion)
- arXiv (Cornell University)
- Funders
- Brown Institute for Media Innovation
- Yahoo Inc.
- Stanford University Computer Science Department
- Office of Naval Research
- Toyota USA
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Keywords
- object relations, picture description, question answering, object properties, WordNet synsets, visual cognition, crowdsourced annotation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
